Due to my first language being an Asian one, I had lots of difficulties with the prononciation of French words (which is the cause of much joking) such as "but", "pute", "cadeau", "gateau", etc. How can you guys tell the difference in relation to how the speech organs are used as well as the difference as made out by hearing?
The distinction between those sets of consonants isn't really in the configuration of the speech organs, but in the timing of the vocal folds' (lack of) vibration.
Consonants such as /p, b, t, d, k and g/ are called stop consonants, because they're produced by completely stopping the airflow for a fraction of second, increasing the pressure in your vocal tract, before releasing the pressurised air in a burst.
While you're pronouncing them, you can keep your vocal folds in a few configurations, but I'll ignore all but two here: they can be spread open (and not vibrate) or somewhat closed (and vibrating).
In French (and most other Romance languages), the /p, t, k/ series is produced with the vocal folds open during the closure, then closing just after the air burst (typically 20 to 40 milliseconds afterwards). The /b, d, g/ series is produced with the vocal folds closed throughout the closure, well before the burst. If you hold your hand to your throat while saying "abat" or "dégât", you should feel an uninterrupted vibration throughout, while this vibration should briefly stop while saying "appât" or "typique". The presence or absence of vibration during the closure of the stop is what French natives unconsciously listen to when they distinguish courte from gourde.
Languages like Mandarin or Cantonese, however, use a slightly different timing to distinguish their pairs of stops. The sounds represented by p, t and k in pinyin (/pʰ, tʰ, kʰ/) are produced with the vocal folds open for much longer after the burst of air, typically 80 to 90 ms. This causes a puff of air to flow out of your throat between the consonant and the following vowel. Meanwhile, the sounds represented as b, d and g in pinyin (/p, t and k/) are produced much like p, t and k are in French, with the closing of the vocal chords typically 10 to 30 ms after the burst. The clue a speaker of those languages will pay attention to to distinguish 炭 and 旦 is the presence or absence of this puff of air after the release burst of the stop.
The obvious consequence of this is that a French speaker is likely to mishear 炭 and 旦 as both starting with /t/ while a Mandarin speaker will have a similar issue with temps and dent, because they're listening for the wrong signal.
(Of course, many Asian languages do not have this exact issue, for example Thai, Korean and Hokkien have a three way distinction, while Japanese is intermediate between French and Mandarin, but hopefully I guessed your problem right).
Check out this question in the Linguistics SE for more information on aspiration and voicing.
In these pairs of sounds, the difference results from using or not your vocal chords to add a vibration to the air flowing through your throat. Technically, linguists will call these voiced vs. nonvoiced (or voiceless) sounds.
/p/, /t/ and /k/ are not produced with a vibration, whereas /b/, /d/ and /g/ are produced with this vibration.
You can do this exercise: place your index and your thumb on your throat, on the level of your Adam's apple bump. then try to pronounce some pairs like bain/pain. You should feel the difference in vibration under your fingers.