You called?
#3 Joel Grey
Singing in opening "Comedy Tonight" with the monsters, villains etc.
#7 Florence Henderson
Song: "So Happy Together" with Florence and the monsters (performed by Jerry Nelson, referred to as "Lizard")
#28 Rich Little
Classic Henson "Inchworm" sketch (Lenny eats worm that turns out to be the nose of Gorgon Heap). Performed by Jim Henson.
#32 Steve Martin
Auditioned for emcee, but Kermit didn't let him. Performed by Richard Hunt ("well we've got a great show for you tonight, starring the amazingly beautiful and talented me, plus other good things...")
#65 Spike Milligan
Appears in various scenes in the background as a Native American with feathers on his head.
#96 Diana Ross
Singing in "I Go To Rio" as a lifeguard.
I think that's them?