I think the key is that "tournée" is a participle acting as an adjective (it would be the same with an adjective), while "vite" is an adverb. Because of mieux and plus they're both adverbial phrases but they're fundamentally different.
I can't think of an example with an adjective but no agreement, or with an adverb and an agreement.
It makes sense because an adverb qualifies a verb, which has no inherent gender. Whereas an adjective or past participle will always qualify the subject.
And as LPH pointed out the only there can also be locutions, which could have an exception (I say could because I haven't found an example yet but I can't rule it out) but should always be invariable.