These aren't tense suffixes, just infinitive ones.
French has four of these, straightforwardly derived from the four/five Latin conjugation classes:
-āre evolved into -er: amāre - aimer; lavāre - laver; parabolāre - parlare - parler
-ēre evolved into -oir: bibēre - boivre - boire; habēre - avoir; movēre - mouvoir
-ĕre evolved into -re: vincere - vaincre; prehendere - prendere - prendre; esse - essere - esre - estre - être. Because the short vowel disappeared, the /r/ was brought in contact with the preceding consonant. This caused some sound changes in the infinitive, future and conditional that aren't found in the other tenses. For example, *nascere has a stem ending in /s/ in the imperfect and the present subjunctive (elle naissait, qu'elle naisse) and in /k/ in the perfect and the past subjunctive (elle naquit, elle naquît), but in the infinitive a /t/ appeared between the /s/ and the /r/, giving naître. So you'll sometimes see this suffix split between the different results of this sound change: -aître (from -Vscere); -indre (from -Vngere); -oudre (from -olere), etc.
īre evolved into -ir: tenīre - tenir, *sufferīre - souffrir.
So they arose naturally, from expected sound changes from Latin (although the /R/ from the -ir ending disappeared at the same time as the /R/ from the -er ending, and was reintroduced later).
Of course, there were some verbs that changed class, either way back in Vulgar Latin (the classical verb for souffrir was suffĕre, which should have given souffre) or later (Latin cŭrrĕre evolved into Old French courre -still preserved in the expression chasse à courre- and was changed into courir later).
Modern loanwords from Latin always take a -er suffix, without regard for the Latin suffix. Exigĕre was borrowed as exiger for example.
There's also one verb that doesn't have an infinitive suffix in : fiche, an obsolete verb mostly used nowadays as a politer version of foutre.
As a final note, while these infinitive suffixes can be classified as deriving from the Latin conjugation classes, they aren't really linked to the French conjugation classes. French verbs are split into three groups: the regular -er verbs (from Latin -āre), the regular -ir verbs (from Latin -iscīre) and the irregular verbs (from everything else).