In 1539 Olaus Magnus published, in Venice, the Carta Marina, the first geographically accurate map of Scandinavia ever known to continental Europe. Both the northern seas and the less populated northern land masses of the map are filled with monsters like a sharp-toothed sea serpent—called a grampus—with razors running down its back and the ability to sink ships by firing powerful jets of water through its snout; a square-headed black beast forty feet long with giant red eyes and long feathers hanging from its chin; the leviathan, a three hundred foot long creature with a bifurcated tale and a penchant for ramming ships (also known to be the sworn enemy of the grampus); just to name a few.
In 1752 Eric Pontoppidan named for the first time the kraken, a monster so large as to be mistaken for an island and which, with its many and massive tentacles was known to be able to take down even the largest man-of-war, in a paper which immediately became a standard and widely praised text on natural history.
European aristocrats throughout the Early Modern era believed that unicorns were real and used their horns as both religious relics and symbols of royal power.
These were not silly farmers living in the dark ages telling stories to one another. These were scientists and rulers in ages of discovery and invention, the time in our history when science was booming and critical thought and inquiry were in vogue. To these societies, to these men, these monsters were real.
And in a sense, they were. None of these creatures come from nowhere. There is always a kernel, some seed of origin, one of history, recognizable through the pages; the evidence for unicorns came from the tusks of narwhals, the descriptions and actions of both grampus and leviathan are distortions of orca and humpback whales visage and behavior, and the kraken had strong ties to observations of giant octopi (dead ones of which were taken to be kraken young). There are connections to reality in every aspect of monstrosity.
Aside: Now some readers will look at this and say that I am doing exactly what Tolkien protested in his “Monsters and Critics;” that I am pushing over the tower looking for the stones from which it was made. And they would be right. Well they would be right in that I choose to value documents from history as more than simply maps or stories or poems or whatnot, but choose to use them to see deeper into the societies from whence they came. They would be wrong in accusing me of pushing over the tower. Viewing a poem as poetry and as an historical document are not mutually exclusive. It certainly would be a shame if they were because then we’d have to give up on the study of Antiquity almost entirely to preserve the artistic integrity of Homer, Herodotus, Aeschylus, and Thucydides (basically our only sources of any kind for any information about one of the most influential societies in all of human history prior to 400BCE).
The idea of the monster, whether sea- or otherwise, is one so natural to how humans conceptualize the unknown (what lies off the edge of the map) that in every age there is some form of that same archetype. And for anyone who thinks we in our modern, enlightened state are beyond such wild and unfounded speculation simply take a gander at the many, many alien invasion movies made in the past fifty years. It may seem easy to declaim ‘monsters’ as figments of an author’s imagination, as necessary opposition in a binary secondary reality, or as allegorical figures in some great morality play, but what they really are are natural extrapolations of our reality into the void in the form of comprehensible, concrete creatures. There is a reason Shelob resides in dark caves where Frodo and Sam lose even their sense of touch. It's the same reason the we understand so little are the ones we call barbaric and at the same time fascinate us so much. It is the same reason children spend so much time fearing what's under the bed or in the closet. People fear what they cannot see, what they do not know, and when there is a true abyss, they fill it with their worst nightmares (a deceptive dragon who will force us into committing mortal sin, a demon so villainous it devours the bravest of soldiers for breakfast, a curse so powerful it can turn true love into murder-suicide, etc).
This is not so say that the monsters of legend are allegorical or in any way metaphorical. Rather they are simply created from that most basic of human instincts, fear of the dark.
On a more Tolkien-specific note, the monsters of his work actually, for me at least, help establish not necessarily the reality of the story but the reality of the work as a piece of historical literature. Tolkien’s objective (one of many to be sure, but certainly one of his most important ones) was to create an history/mythology for England and given the human predisposition for including the fantastic in our works of literature Tolkien would have been remiss in his duty had he eliminated such a prolific and intriguing element from his books.
Indeed, in our eagerness to examine the nature of monstrosity in pre-Modern mythology, we neglected to really discuss the way Tolkien used his dragons and spiders and various other villainous creatures as connections to the extant mythology of the primary reality. We can see in Smaug a potential origin for Beowulf’s dragon, in Bilbo’s duel with the spiders the same story frame as Sigurd with his dragon, and in the one ring the same curse as Andvari’s ring or Beowulf’s helm.
Tolkien also used them as connections to the primary reality in another way, one not dissimilar to his creation of the hobbits. These creatures are very much defined by their environments. The hobbits are creatures which are so very English in nature because of their strong connection to the rolling hills of the English countryside. Just so we are painted a picture of Smaug perfectly suited to his adopted underground abode, the trolls to their rocky and wooded foothills, the spiders to their dense, ancient forests, and Gollum to the jagged and moist caves. The reader can imagine a long-dead story teller looking up at a singular mountain from a town on a lake telling the tale of the great and greedy dragon which once resided within or how a group of travelers might sing the tales of giant spiders who once lurked in the shadows of a mysterious wood. And that is the function of the monsters. They are not there to foil or be foiled. They are not there to embody evil. They are there because when humans look at the sea we see sea-serpents and when a ship sinks we tell tales of the great tentacle beast which dragged it to its watery grave and when we find a giant pointy horn-looking thing we imagine it came from a unicorn. The monsters themselves are mythological extensions of the world they inhabit.