I guess those probably still exist. You’re referring to the music and effects track of the episode. A TV episode's soundtrack is usually composed of two individual tracks: dialogue and music & effects. You've probably noticed the M&E track playing on the menus of the Fraggle DVDs.
The songs, however, are a different case. Muppet Show and Fraggle Rock music was pre-recorded. For FR, those songs were recorded not only for use in the episodes, but to be usable when producing albums. For TMS, those songs were recorded simply to be inserted into the episode.
Unfortunately, after that they were likely discarded or recorded over.