Tolkien's essay On Beowulf is a deeply interesting piece of work, both as a commentary on Beowulf and as a window into the way Tolkien himself thought, a window on his views about literature and myth. In this essay, Tolkien's analysis of _Beowulf_ places great significance on the fantastic elements of the poem, and in particular on the monsters. At the same time the essay is a forthright defense of the use of the fantastic in serious literature and an outline of the roles these elements can play. Thinking really seriously about Tolkien's comments on _Beowulf_ gives us an excellent window into his own work.
Tolkien places the fantastic or mythical elements of Beowulf in a specific historical context. In his view, the Beowulf poet was a Christian writing in a time not far removed from pagan day, who took seriously pagan myths even if he did not believe them as such. The significance of the monsters is derived from the view of their author. Tolkien argues that the importance of the monsters comes from the fact that they are real monsters, not merely analogical constructs; in the story, they are real, mortal, flesh-and-blood beings (linked, famously, to Cain, the son of Adam). The dragon is not a representation of greed; rather, in a shocking turn, it is a dragon. It can be slain and can slay; it is not slain by the light of God, or by Goodness, but by a sword. These are first and foremost flesh-and-blood creatures. They are not precisely human, but neither are they precisely beasts.
This view of monsters allows them to play the role which Tolkien believes that they play. For Tolkien, the monsters do not represent in a clear, one to one way some abstract concept, but they do have some kind of significance - they represent essential characteristics of the world, and essential characteristics of human life. In Tolkien's analysis, they represent something about the mortality inherent to human affairs, the doom which attends all life, the tears of mortal things. The dragon is not greed, but it does stand in for the greediness of things, and their animal brutality, and the terrible danger which human beings face in creating anything. Grendel is always lurking outside Heorot. The monsters are a part of, or an indication of, the world-view of the poet, his thought and his attitude on human life.
It is an important part of this role for Tolkien that this kind of meaning is never explicit - probably not even in the poet's mind. Its greatness comes from the fact that it is mythic-poetic. The monsters are not slated into a schematic moral scheme; the poem is not didactic. Rather, the meaning comes from the poet's treatment of the subject. We should not look for precise and pre-arranged symbolism, in a medieval style. Tolkien says, in fact, that if there is a flaw in the poem it is that it is a little too allegorical - that the dragon represents greed too much, instead of simply being a dragon. Looking for precise meanings in each of the monsters in Beowulf is ultimately futile and in fact detracts from the greatness of the poem, which lies in its ambiguity.
Tolkien's defense of the fantastic, therefore, is centered on ambiguity (it bears obvious similarities to his arguments in On Fairy Tales). The fantastic can represent things without being analogical. The fantastic and the mythical can be meaningful, as meaningful as any realistic mode. But to be meaningful, the monsters and the fantastic elements must be treated as actually existing, and there must be a certain ambiguity in the presentation of these fantastic elements. Tolkien's target, in his analysis of Beowulf, seems to be the mythic and poetic as against the allegorical and simple.
This seems to be a decent way to look at Tolkien's use of fantastic elements, and especially his use of monsters, in Lord of the Rings itself (and throughout his work). The use of the fantastic harks back to the pre-Christian world of fairy-tale and fable and mythology, in the same way that the Beowulf poet made use of the pagan Norse matter in a Christian context. Moreover, Tolkien's views in On Beowulf seem to indicate that we should think of monsters in Tolkien first, and above all, as monsters, as creatures which exist and which threaten protagonists. They are neither precisely human nor precisely animal; they exist in some vaguely-defined region in between. These monsters do represent things, but they do not do it in a simple way; their role as representations is secondary to their function as monsters. And this representation is inevitably going to be ambiguous, multi-faceted, not explicit, not clear. We shouldn't think too hard about the precise nature of monsters because it is unlikely that their precise nature can be classified and put into neat boxes. Monsters do not play simple representational roles. They are evil, yes, but they are not put into Lord of the Rings as examples of the way evil operates; they are put into the text as examples of evil.
Shelob, for instance, is not first and foremost a symbol of hunger and vicious greed; Shelob is first and foremost a giant spider. Shelob represents something about evil, and the nature of evil, and the way that it threatens the good, but that role cannot be stated in precise Catholic theological terms. And her role as a symbol is and must be secondary to her role as a spider in the story.
I guess what I'm trying to say here is that it seems to me that we're focusing, perhaps, too much on the theological elements of the text and especially on the symbolism of the text and the nature of evil. I think that's valid, but it's important to keep in mind that Tolkien's work isn't allegorical. I don't mean to accuse us on ignorning the merits of the story qua story because I think we're giving plenty of consideration to the story. But I think Tolkien's view was that it's necessary to remember the purely narrative elements of a story. It's pointless to get too hung up on the subtleties of the status of monsters or of elves in Tolkien's work because (in my view) their status is not precisely enumerated and determinable. Tolkien absolutely has something to say about the nature of evil and of good, and that thing is certainly tied to his Catholic theology, but Lord of the Rings is not a textbook about that theology, and we should not express the views of good and evil in Lord of the Rings to be precise. Rather, things in the book are broad, mythical, imprecise, magical. That's at the heart of Tolkien's method and his thought about the relation between myth and literature.