For whatever reason, I increasingly use ne ... que to express "only", as opposed to perhaps juste or seulement.
But there are some contexts where I'm not sure if it's possible or stylistically desirable, and if so, how to do it. One of those contexts is in fragments. For example:
— Tu regardes des films d'horreur ?
— Only rarely.
Each of these sounds like an unlikely or strictly ungrammatical rendering of "only rarely":
✗ Juste rarement.
✗ Seulement rarement.
✗ Rien que rarement.
I note that rien que is flexible in the sense that you can use it in some contexts without any verb to negate, but here it doesn't seem to suit.
Are my premises right? What's the solution? Must one paraphase?