The phenomenon you've stumbled upon, sometimes called the Person Case Constraint in the linguistic literature, reflects a general tendency of languages with weak pronouns or polypersonal verbal agreement: some combinations of pronouns are prohibited depending on person and case (the grammatical role they have in the sentence, i.e. subject, direct or indirect object).
The way the constraint works in French is that some combinations of direct and indirect pronouns are prohibited (some other languages block subject and object combos too):
Any combination of first and second person object pronouns is prohibited: "Je me te prête" is never grammatical, whether it means "I loan myself to you" or "I loan you to myself". Likewise "Je me me prête" and "Je te te prête" are impossible.
All combinations of third person pronouns are allowed: Je la lui prête, Je le leur prête, Je leur en prête, Je les leur prête, etc. Note however than in casual speech, any combination of la/le/les and lui/leur is reduced to just the indirect object pronoun (Je leur prêtee can stand for "je la leur prête", "je le leur prête" or "je les leur prête")
When those two groups are mixed, only some combinations are allowed: combos of first or second person direct objects and third person indirect objects are prohibited; combos of first or second person indirect objects and third person direct objects are allowed. In other words, "Je vous leur prête" (I'm loaning you to them) is ungrammatical while "Je vous les prête" (I'm loaning them to you) is grammatical.
Your sentence (Ils nous te prêtent) violate the first rule: it has a first person indirect object pronoun and a second person direct object pronoun, a prohibited combination.
The usual way to avoid violating the person case constraint in French is to use a strong indirect object pronoun instead of a weak one. Since the constraint only affects weak elements dependant on the verb (clitics or affixes), but not independants words like strong pronouns or nouns phrases, you can rephrase your sentences to "ils te prêtent à nous".
The root cause of the constraint probably comes from a desire to avoid ambiguity by prohibiting the rarer combinations: the first and second persons almost always refer to human beings involved in the action and as such tend to be subjects and indirect objects (recipients or experiencers of the actions) and more rarely direct objects, and all the combos that French prohibits are those where a first or second person direct object would appear together with an indirect object, in contradiction of this tendency.