Recently there's been some talk about these videos on the Tough Pigs forum. There's been some discussion on whether Rowlf's Rhapsodies was all imrpovised, as there wasn't a writer credited (aside from Jerry Juhl getting a "creative consultant" credit and a list of writers from The Muppet Show), especially considering it has a lot of oddball dialogue. But it seems like the scenes with Frank Oz's characters are most organized, and Oz is known to be a master ad-libber (makes me wonder if he just did voices in that one).
After reading that on the Tough Pigs forum, I looked at the credits for all the videos I could, and all of them have Jerry Juhl listed as "creative consultant", with the list of writers listed under The Muppet Show part of the credits. I wondered if the list of writers for TMS also included whoever wrote the linking footage (Jim's Red Book says that Juhl and Henson selected the clips to use). The credits start out with credits specifically for the linking footage, then there's credits for The Muppet Show (in a different color from the first set of credits as well as the TMS title appearing), but the earlier part of the credits does list people who aren't involved in the linking footage, such as the guest stars and the main Muppet performers, whether they performed in new footage or not (sure, we don't know for sure that Nelson, Whitmire, and Mullen didn't perform in any of them, but I'm sure Jerry Nelson would have performed Camilla in Muppet Weird Stuff if he was involved). Also, if the dialogue was mostly improvised, it seems like the linking footage for all of the others was a lot more organized and not as crazy as in Rowlf's Rhapsodies.
Somebody also noted that The Muppet Revue, Muppet Treasures, and Muppet Moments are all the weakest because they have the same basic plot (while also noting The Muppet Revue as feeling more like a clip show than the other two, which I agree with - and it is one of my favorites in the collection). I wonder if this would be the case if different characters hosted those, like if Muppet Treasures was hosted by Kermit and Scooter, or if Muppet Moments was hosted by Fozzie, Gonzo, and Rowlf. Or if they had different reasons for being up in the attic other than to clean (well, in Muppet Moments, they do go up to find things to get rid of). Interestingly, back when I was 8 years old, I put both The Muppet Revue (which I had seen) and Muppet Treasures (which I hadn't) on my Christmas wishlist, and for some reason, those were the only two I put on my list (needless to say, I didn't get either, them being out of print and all). But in the past few years, out of the few I have not seen, Muppet Treasures is the one I least want to see (though I do still want to see it).
Looking at the guides at Muppet Wiki, it seems like Muppet Treasures and Muppet Moments both have times when Kermit and Fozzie just stop what they're doing to watch some guest star clips. Seems like they don't come across any props or anything that remind them of moments with the guests, they just decide they want to watch Peter Sellers, Zero Mostel, Bernadette Peters, and Liza Minelli. Of course, The Muppet Revue kind of has that with Linda Ronstadt and Paul Williams, but those also seem different for some reason.
I often think about how none of the videos use sets from the actual show. For a long time, I assumed they were taped in New York, but I think somebody said those were actually taped in England (Louise Gold does perform a chicken in Country Music, and somebody said that one of the England-based performers who wasn't on the show operated Rowlf's right hand in Rowlf's Rhapsodies). I've also wondered if the sets had been destroyed by then. But I also wonder if the videos work better by not taking place anywhere in the theater. Would it have seemed weird if the videos took place backstage before a backstage clip was shown? Of course the attic seems better for when Kermit and Fozzie are cleaning up, but Fozzie's Muppet Scrapbook and Children's Songs and Stories could have easily been done backstage, Rowlf's Rhapsodies could have took place in the orchestra or even up on the stage (though we don't really know where it is - is it in Rowlf's home, a studio of some sort, or is it actually the Muppet Theater stage?), perhaps even with the plain brick wall background.
At Muppet Wiki, there's been some discussion on whether the recording studio from Rock Music with the Muppets was meant to be the same one from Helen Reddy's "Blue" number, only at a different angle. I'm not convinced. Aside from it being a different color scheme, when that clip does get introduced in the video, it's presented as a clip (with a transition from it appearing on a TV screen) as opposed to being edited into the plot. If Dr. Teeth had pointed into the direction we were watching and told Helen Reddy she's on, as if she was there live in the studio, that'd be one thing, but he talks to her on the phone before introducing the clip.