You are right, Amadeus, the Bible is very clear about the Millennium being just that -- a millennium, or a thousand years. It is the last thousand years of this creation.
(An aside here, apart from the topic of the post. I have sometimes wondered that if Adam were originally designed to live a thousand years, and Christ is the 'second Adam' -- see Romans -- then it is fitting His reign be for a thousand years...)
After the Millennium (Rev. 20:7) there will be one last battle which will be won not by might or strength, but by the power of God (Rev. 20:9)
Then go to 2 Peter 3:10-13 for the destruction of this creation. The very elements themselves will be destroyed. After that is the new creation of Revelation 21.
Now, are those who have died here on earth in heaven or hell now? Paul stated clearly that absent from the body is present with the Lord (2Cor. 5:8 ). Therefore although Jesus was the first into heaven from earth, He has called home many to be with Him since them.
Now, if you go to Psalm 1, you will find something interesting at the very end of it: "Therefore the wicked will not stand in the judgment, nor sinners in the assembly of the righteous."
This is totally different from whether or not your name is in the Book of Life. Those are the names of those who are saved. The others are condemned eternally for their unbelief (not for their sins, which have all been atoned for). The Judgment is for the righteous only. It is where some will receive rewards, hearing "Well done, good and faithful servant!" These rewards, or 'crowns' can be both won and lost according to our lives here as Christians. But that has nothing to do with unbelievers who will not be in that assembly at all.
About hell, if the condemned are not there now, they are in the 'anteroom to hell', as referred to by Jesus in his story of Lazarus and the rich man in Luke 16, starting with verse 19.
There is one additional, interesting thought that comes from C.S. Lewis from, I think, his book "The Great Divorce" -- that those who find themselves in hell will look back and say "but I was always in hell," and those who find themselves in heaven will look back and say, "but I was always in heaven." This has to do with the way we see our past, and not with predestination, by the way. Each man is free to choose to believe or not, free to follow the truth or not.