For reference, the usual diacritics are as follows.
Accent aigu: é
Pronunciation: Uniformly causes the vowel to be pronounced [e] (as in English "may").
There are some rare exceptions where it's pronounced [ɛ] (as in English "beg") instead, such as événement (recommended spelling changed to évènement in 1990) and in inversions like aimé-je.
Usage: Most notably on past participles of -er verbs, such as parlé. Many nouns end in it as well, such as musée. Besides that, it's simply to mark the "ay" sound in many words, such as présenter. One word that shows both cases is the surprising créé (past tense of créer).
Notes: An e without an accent sometimes disappears from pronunciation (the e muet). However, if it has any accent, this is not possible. An e with an accent other than é is generally pronounced [ɛ].
Pronunciation: Causes a hard c [k] to be pronounced as a soft c [s] instead.
Usage and notes: See this answer.
Tréma: ä ë ï ö ü ÿ
Pronunciation: Pronounce this and the preceding vowel as two distinct sounds.
Usage: Purely used in fixed forms, not triggered by any grammatical shift. Some examples are aïeux, ambiguë, and borrowed words from languages that use this more often, such as Hawaï.
Notes: ï is by far the most common, followed by ë.
Accent grave: à è ù
Pronunciation: This accent on à and ù does not change their pronunciation. However, è is pronounced [ɛ].
Usage: à and ù are only in fixed forms, such as là "there" vs. la "the" and où "where" vs. ou "or". However, è can appear not only in fixed forms, such as the common ending -ère (père, prière, bière), but also grammatically.
Specifically, it appears when an e that is normally caduc becomes obligatoire as a result of conjugation. For example, in the verb lever, the first vowel is reduced or even dropped. But in the present-tense conjugation lève, which has only one syllable, the stress has moved back to the first e and it is no longer optional. This is marked with the accent grave. (This is not 100% regular.)
Accent circonflexe: â, ê, î, ô, û
Pronunciation: The vowel usually becomes the variant it would have in an open/"long" syllable. The difference is negligible for î and û. For ê, the same note as above applies: if it has an accent, this letter can't be silent. For â, it becomes a back vowel [ɑ]; compare sache and lâche.
Usage: A few different purposes. One is to distinguish words, e.g. du "of the" vs. dû "had to" or jeune "young" vs. jeûne "a fast from food". Another is to signal historical letters that have disappeared, often s, as in fête, honnête, forêt (compare feast, honest, forest). It also regularly appears in certain conjugations, such as the passé simple and the past subjunctive.
Capital letters are often written without their accents. This leads to some amusing situations, like this sign I saw at the Eiffel Tower advertising some seemingly unappealing snacks:
(salé "salted" vs. sale "dirty")
Edit: As jlliagre and Frank point out, although it is common practice, omitting accents on capital letters is not considered correct French by the Académie.
P.S. Here are cross-platform instructions on typing all of them. Edit: Here is a script written in AutoHotkey that allows you to type French and Spanish accents on a PC as you would on a Mac.