For linguistic studies that try to answer this question, please see my other longer answer.
Here I summarise only the conclusion of one of the papers, which should significantly assist all French L2 learners. I do not use blockquotes to conserve readability.
Source: p 189, French prepositions à and de
in infinitival complements,
A pragma-semantic analysis (2008) by Lidia Fraczak, as part of Adpositions: Pragmatic, Semantic and Syntactic perspectives (2008) edited by D Kurzon, S Adler
We hope to have shown that the “abstract” prepositions should not be excluded
from the analysis of linguistic signs as they convey differential pragma-semantic
values. The use of the preposition à in expressions containing an infinitival
complement is related to “ambivalent vision” of the fact described by the complement,
which implies simultaneous consideration of two contrary versions of the
fact, with a “transfer” from one to the other. The preposition de is compatible, in
such expressions, with “monovalent vision” of a fact. This pragma-semantic description,
apart from having a purely linguistic interest, may be of use for foreign
students learning French, by helping them to improve their skills concerning the
correlations between the preposition à or de and different lexical elements.
Our study does not end here, as it is necessary to include not only verbal
expressions but also those whose main element is an adjective (e.g. prêt à faire, impatient de faire – “ready to do”, “impatient to do”) or a noun (e.g. capacité à
faire, capacité de faire – “capacity of doing”). Our first analysis of such expressions
allows us to consider that the opposition in terms of “ambivalent vision” / “monovalent
vision” is probably valid there as well.