Tout à fait directly translated would be 'all to do', how does this get inferred as 'absolutely'?
je m’en fous directly translated would be 'I am in craziness', how does this get inferred as 'I don't care'?
'Fais gaffe' directly translated would be 'Make/do a blunder', how does this get inferred as 'Watch out'?
« Tout à fait »
From http://micmap.org/dicfro/search/dictionnaire-godefroy/fait, "a fait" already meant "absolutely" in ancient French.
« Je m'en fous »
It's not "fous" as in "crazy" but rather the verb "foutre" (vulgar). It's vulgar for "Je m'en moque" (directly translated as "I'm laughing about it")
« Fais gaffe »
From https://originedesmots.blogspot.fr/2014/02/gaffe.html: "faire gaffre" means "keep guard" in ancient french