There is nothing illogical in où je suis né.
Où can be used with both space and time references and this was already the case with the latin ubi from which où comes from.
On the other hand, unlike Spanish (El día cuando nací) or Italian (Il giorno cuando sono nato), quand can only be used as a conjonction de coordination, e.g.:
Ma sœur avait trois ans quand je suis né.
When a relative adverb introduces a time or a location, où is used, thus:
Le jour où je suis né.