G. K. Beale and Sean McDonough have some helpful notes on Revelation 21.1-4, specifically verse 4. After discussing some clear parallels between Rev. 21 and Isaiah, they focus on some of the imagery of the new heaven and new earth. In Isaiah (25.8, 35.10, and 51.11) the prophet predicts “that ‘pain and grief and groaning’ will have ‘fled away.’”
Very helpfully, they also note how both Isaiah and John write about the sea that is gone in the new creation. Isaiah uses language about Yahweh “drying up” the Red Sea (51.10); in the very next verse (51.11), Isaiah writes about the redeemed obtaining everlasting joy and living without sorrow and sighing. John writes similarly, “the sea was no more” (Rev 21.1); there will be no more sorrow nor crying, etc. (Rev 21.4).
In other words, as Beale and McDonough note, the Exodus “event” and the New Creation “event” have huge parallels. It is significant that Yahweh overcame/destroyed the chaotic sea and his Egyptian enemies which resulted in joy and peace for his people, Israel. In the New Creation, the same can be said: there will be no more chaotic sea, there will be no more Pharaohs or Serpents. All that is left is joy and peace, life and rest.
The above references and quotes are found in G. K. Beale and Sean McDonough, “Revelation” in Commentary on the New Testament Use of the Old Testament ed. G.K. Beale and D. A. Carson [Grand Rapids: Baker Academic, 2007], 1151.
Of course, there are quite a few more allusions and references to be addressed, but this is a great start. Thanks, Beale and McDonough!