Yes and no. Let's see how.
When the action is finished in the past one uses the passé composé form of the verb (together with the auxillary). Since “regarder” is transitive, the auxillary is “avoir” and hence the corresponding form for the first person plural.
When the action is described in sense of “was being undertaken/used to be” with it's completion being left vague, then the imparfait is used “nous regardions” (without the auxillary avoir) and directly conjugating the main verb.
(In my experience) In normal conversations though the French speakers seem quite intelligent and can make out what you want to convey by the context, tone and sense.