Well, I've seen the first episode many, many times, and I can assure you that no musician APPEARED on that particular episode. Not a FAMOUS one at least.
The ONLY one I can think of is this:
Actress and sometime-singer Holly Robinson-Peete played 6 year old Sally, who was featured as Gordon's friend in the premiere episode. That's the most logical answer for you, since Ms. Robinson is a singer (sang the 21 Jump Street theme) and is often a trivia answer in regards to Sesame's 1st episode.
Of course, Grace Slick of Jefferson Airplane sang the counting song with the Spies and Racecars ("2"), but she was just heard, not seen.
Also, there's a "Milk" segment that features a folksy guitar soloist (probably not famous) on the soundtrack.
That's it.