Something missed by casual readers of The Lord of the Rings (or ignored by 'sophisticated' ones!) is the 'editorial' apparatus that presents the book as based-upon an ancient manuscript; in other words, the claim that the book is real history.
Why did Tolkien do this - especially considering that he had not done so in The Hobbit?
Well, in the first place, it was not unusual for the early novelists to claim explicitly and non-ironically that their books were either real records of actual events, or based upon such accounts (e.g. Robinson Crusoe by Daniel Defoe - if considered as a novel). Indeed, this was so common as to be almost normal up to the twentieth century.
However, the practice had stopped before Tolkien began work on his Silmarillion legendarium with the unpublished (in his lifetime) Lost Tales commenced late in the 1914-18 War. These stories included a very elaborate feigned-historical framing and explanation of their provenance - that is, the links of how it was that these stories came into our world, into the hands of the modern reader.
The practice had become very rare by the time Tolkien published Lord of the Rings - except in a self-conscious manner that was intended to be taken ironically or satirically (as with the tongue-in-cheek 'editorial' apparatus of Farmer Giles of Ham).
Tolkien, by contrast, was not-at-all ironic, but indeed very serious and 'literal' in his within-text claim that LotR came from 'The Red Book of Westmarch', a strategy repeatedly pursued in the Prologue and Appendices that bracket the story; and this was later buttressed by the editorial introduction he included with The Adventures of Tom Bombadil.
Furthermore, it is a significant part of the failure of The Silmarillion of 1977, that any attempt at framing was abandoned by Christopher Tolkien - a decision CRT soon regretted, and endeavored to 'undo' indirectly, by his History of Middle Earth.
Note: Verlyn Flieger has written an excellent account of Tolkien's lifelong wrestling with the matter of framing his stories - Interrupted Music, 2005.)
So why was it so important for Tolkien to frame his stories by serious attempts to explain how they came down to modern readers - against the trend of 20th century fiction?
I think the answer is simply that he wanted to create an imaginative bridge that explained why these stories were not-just-entertainment, not just 'escapism; but were intended to be 'relevant': of serious concern to modern readers.
This was especially important to Tolkien because he eschewed the usual means of making imagined world relevant - which is allegory.
Tolkien's world was not meant to be allegorical, and he reacted quite aggressively against those who said it was; but real in-it-own-terms. Yet without any framing and linking between the stories and ourselves this detailed, autonomous, not-allegorical world-building might make the stories feel simply irrelevant...
By providing a feigned history to bridge between the stories and ourselves, Tolkien created a single imaginative conception of the stories as forming part of our living world - hence obviously of relevance and serious concern to modern readers.
It seems pretty clear to me that Tolkien did not want to admit that he was doing this! - and indeed (e.g. in the preface to the second edition of LotR) he sometimes denied that his stories had any 'purpose' except to entertain people who happened to share his taste for such things.
But this is to ignore the great efforts he made to frame and link LotR. These went far beyond parody, satire or the merely fictively-sophisticated.
Indeed, setting aside defensiveness; the truth of the matter was apparent in Tolkien's own - passionate, and non-ironic - practice of referencing his own work in commenting on everyday life; e.g. labelling the attacks on trees (e.g. by chain-saws) as orcish, or modern bureaucracy as the work of Saruman.
And even developing (post LotR) Galadriel as more and more an echo of the Blessed Virgin Mary to whom he was religiously devoted. This linkage would be merely blasphemous unless it were underpinned by a very serious and real imaginative link between his sub-created world and God's creation as known by modern people.
In other words, Tolkien wrote and lived-by a belief in the potential reality of the imagined: and the actual reality of his own imagined world.
Tolkien did not, however, theorize the reality of imagination.
So we can see that for Tolkien it was vitally important the seriously imagined worlds were regarded as really-real - despite that, at another and theological level, he denied this very assertion.
According to available biographical data, and confirmed by the brilliant analysis by RJ Reilly in his Romantic Religion; Tolkien was apparently unaware of his friend Owen Barfield's extensive and rigorous philosophical work that coherently theorized the reality and truth of imagination.
Of course; Barfield was working from a metaphysical basis that Tolkien's orthodox, traditional Catholicism did not share.
But we, looking back, can now perceive that Barfield explained the reality of Tolkien's profound intuitions regarding his own world; and why it is that an imagined provenance for The Lord of the Rings was so important to Tolkien - and so effective for some of those most serious of his readers who regard the Prologue and Appendices as a vital part of the effect made by the whole book.