Your apprehension of this matter is correct.
It is the subject of a subsection in LBU 14th edition (§ 672 c)). It is possible to download a pdf of this source here: https://www.pdfdrive.com/le-bon-usage-grammaire-fran%C3%A7aise-14e-%C3%A9d-d157189845.html.
Here is what can be read there. (Capital letter divisions, bold type and italics due to user LPH;
Body parts and possessives
From A Comprehensive French Grammar, 228-229,
use the indirect object pronoun to refer to the person affected when the action applies to someone else's body. The indirect object pronouns (complément objet indirect) are me, te, lui, nous, vous, leur.
Il m'a tordu le bras. He twisted my arm.
Elle lui lave les cheveux. ...