The book that I'm studying French from always translates "The door was shut" as "La porte était fermée" instead of "La porte a fermée". Why so? Also, what would be the translation of using "était fermait"?
This is a correct translation, and on that matter , French is rather similar to English. It is a passive form of the verb fermer, which is a transitive verb.
John shut the door - the door was shut
Jean a fermé la porte - la porte était fermée
Out of context, la porte a fermé alone does not make sense: you then mean that the door itself did an action of shutting down something - but you don't tell what ! OR you can use the verb fermer as an intransitive verb with porte as subject, but you have to add an adverb, and then it indicates how the door is shut.
La porte ferme hermétiquement. La porte ferme mal.
Note for the sake of completeness that in some contexts, fermer can be used as an intransitive verb but has then the sense of "to close at the end of the day" or "to shut down a business":
Le restaurant ferme à minuit.
Cette usine a fermé en 2002.
In that sense, you can also use fermer intranstively with the word "portes" as subject, but in the plural, as a metonymy for the whole buidling or the whole business.
Ne sois pas en retard: les portes ferment à 20h.
In all cases the translation has to be "La porte était fermée." It can never be "La porte a fermé(e)", as that is never said (nor written).
You probably mean "La porte a été fermée.", which is the passive voice (The door was shut (by somebody)). It must be that in that part of the book you are reading the word "shut" is always considered to be the adjective "shut" and never the past participle of "to shut".
"La porte a fermée." is never written nor spoken even if the spelling is "La porte a fermé.". What you can say, if for example a strong gust of wind pushes the door closed is this;
- La porte s'est fermée (sous la force du vent).
- Le vent a fermé la porte.
In « La porte était fermée », fermée can be seen as an adjective, explaining the state of the door. « La porte était fermée » translates « The door was shut », and its grammar analysis is exactly as you could read « La porte était blanche » or « La porte était cassée ». If something like « La porte était fermée [par Robert] » is implied, then it is passive form, as explained by Greg. Anyway, there is no real difference between these two forms.
If using the avoir auxiliary, you will not have a final « e »: it will be « La porte a fermé », and it will most probably be incorrect. fermer being transitive, « La porte a fermé [quelque chose] » (The door closed [something]) is obviously wrong.
However, rarely, it might mean also "At that precise moment, the door became shut". For instance, « La porte a fermé brusquement, poussée par le vent ». But 1) it would rather correspond to the English sentence « The door shut [in his face] », and 2) even when meaning this, you will more often encounter « La porte s'est fermée ».