I have in my collection the following:
*My Name is Roosevelt Franklin (1974)
*Sesame Disco (1979)
*Sesame Street Fever (1978)
*Let Your Feelings Show (1977)
*C is for Cookie (no year listed, but let's assume 1975 or so)
*Just the Two of Us (1982)
*The Sesame Street Book and Record (1970)
*Sesame Country (1981)
*The Sesame Street Fairy Tale Album (1977)
*Concert On Stage Live (1973)
*Merry Christams Sesame Street (1975)
*Let a Frown Be Your Umbrella (1974)
*Sesame Street 25 Greatest Hits (1975)
*The Muppet Alphabet Album (1976)
Also included are 45's:
*Somebody Come and Play/A Very Silly Dance
*Boy From Indiana/Walk Down the Street (Bob sings these)
*Sir Duke/Muskrat Love (Bob sings these)
*Sing/What's the Name of That Song?
*Cookie's Rhyming Song/Up and Down
*This Land is Your Land/Michael Row Your Boat Ashore (Pete Seeger and Brother Kirk Featuring Big Bird)
*My Sister, My Father, My Mother, My Grandma, My Grandpa, My Dog and Me~Seven (sung by girl Anything Muppet)/Six (Ernie and Bert)
You can see pictures of these in my collection in the Muppet Merchandise Section, under the thread "Finally!"
And of course, thanks, MikeBennidict for the link.