Indeed, "ne" can always be omitted in spoken French ("ne ... que", "ne ... jamais", ...).
If you are trying to speak or write in a formal way however, I would recommend never to omit it in an official document1.
When spoken, it is also usual to shorten it, just pronouncing the N (a sort of a contraction). When speaking formally, "Je n'suis pas d'accord." will be better than "Je suis pas d'accord."
Additional note: in literary French, "pas" can be omitted in "ne ... pas". This is very rarely heard in spoken French however, and is specific to "ne ... pas".
1. My first instinct would be to compare it to the inflected negative auxiliary verbs (e.g. the "don't" contracted form) in English, which I would avoid in formal documents. Unlike the French omission though, the contraction is not considered unformal according to American recommendations (which can be summed up as "write as you talk") and English onesEnglish ones (thanks to mikeyreillymikeyreilly for pointing that out).