Using the degree sign is often done because it is directly available on a French (AZERTY) keyboard. That gives n° and N°. It looks like the famous perfume named N° 5 uses a degree sign:

Doing it is however sometimes considered incorrect because the degree sign is technically a small zero while the upper o is the vowel ending the word numéro. Using the degree sign also makes the plural unaesthetic to say the least: n°ˢ or n°s.
To get a consistent display, you then need to use a superscript "o", possibly followed by a superscript "s" if plural:
Numéro: HTML: no or No, UTF-8: nᵒ or Nᵒ
Numéros: HTML: nos or Nos and UTF-8: nᵒˢ or Nᵒˢ.
Sometimes, the masculine ordinal glyph is used as in nº and Nº. It is not recommended because depending on the font used, the o might be underlined which is not a French thing, and there is no matching "s".
Similarly, the "Numero glyph" (№) is unused in French because the "o" is underlined and there are neither lowercase nor plural forms.
Capitalization follows the generic rule, if numéro is starting a sentence, the initial N should be in uppercase, otherwise, a lowercase n is used.
Note also that a space is required after no, e.g.:
No 4