À is a preposition, it is used to introduce the prepositional phrase that serve to complement the sentence. And the rule says that after a preposition the verb must be in the infinitive, whether à or another preposition.
The choice of the preposition depends on what precedes the preposition and not on what comes after.
Prêt à frire.
If you use a verb after (être) prêt then this verb is preceded by à, this à is not part of the infinitive as such, it is part of the verb phrase.
We have a different preposition after different expressions. For example:
(J'ai) envie de partir.
(Je suis) heureux de rester.
Here it's (avoir) envie de, être heureux de that is followed by the infinitive, we don't have à but de.
Here is a related but not similar question: When would one use “à” before a verb?When would one use “à” before a verb?