Had Fraggle Rock gone to a "6th season" (my understanding is that HBO split the real 4th season into the 4th & 5th seasons that we know of), there likely wouldn't have been the series ending episode arc that we were given. Television series need a sufficient amount of time to ramp up production and keep personnel so there would have been some indication from the studio offices that the series would be continuing.
There are always exceptions, especially in the competitive world of network television where television shows end quite suddenly for a variety of reasons (Isn't that right, Rosanne?), but Fraggle Rock was on HBO and so it wouldn't have been exposed to those sorts of pressures.
Even removing the "But they wrapped up the series so how would they continue?" concern, though, there would be the additional concern of original content. Writers are a creative group and I'm sure that they would have come up with any number of scenarios to keep the series running.
* Perhaps Gobo takes over for Uncle Travelling Matt and explores a new tunnel each week (a la "Star Trek: Original Series" model) and meeting new exciting Fraggles?
* Continue with the stand-alone episodes for which I'm sure the writers could've pumped out by the baker's dozens.
As much as I would have personally enjoyed a 6th season of Fraggle Rock, there is only so much material that any premise can give to you before you have to expand or significantly alter that premise. As others have written, "Jumping the Shark" is a very real concern with any creative endeavor and I enjoy the fact that Fraggle Rock, at least from my perspective, didn't. I haven't enjoyed many of the long-in-tooth reboots that have occurred lately (Blade Runner, The Thing, Conan the Barbarian, Tron, Star Trek, Alien, Star Wars, etc.).
Everyone gets old eventually; It sucks that good things come to an end but the likely alternative is that good things eventually turn into bad things if they continue to exist for too long. Fraggle Rock existed for 4 real seasons (stretched into 5). This is one case where Quality triumphed over Quantity.