Hello, yes what you have is definitely collectible. They were made in the 1970s. Though I would have to say the Grover puppet is probably the most rare one you have because the Fisher Price Muppets I've seen on eBay a lot. Though personally I don't think it's much about the value in terms of how much they are worth, but the value knowing that they were made when The Muppet Show and Sesame Street were really popular back in the 1970s and 1980s. For more information on Muppet collecting. I would suggest Muppet Wiki and there's also a page on Muppet Central under Collectibles. Again if your looking for information on how much they are worth, I'm not sure myself but there might be someone on here that might know. But me personally, in terms of collectibles I like the merchandise that came from the 70s and 80s like what you have. I notice that you said in terms of allowing your kids to play with them, I guess all kids are different. But in terms of them being collectibles, I wouldn't be rough with them. When growing up, I played with my collectible Muppet Puppets, that's kind of how I learned about the artform of puppetry. I still have my first Kermit puppet like the one you mentioned, the Fisher Price puppet. He's currently on my desk. Again, I think the years basicly say it all. As a person who admires Muppet merchandise I feel anything from Jim Henson's time and merchandise that looks close to the actual characters like the Palasades figures are the most valuable. But that's just my opinion.
http://muppet.wikia.com/wiki/Fisher_Price
One thing I want to mention about the Grover puppet and probably one of the reasons why I consider him rare is because there was several toy companies that made the Sesame puppets and a couple of them used similar molds but they slightly differed for example the rubber Ernie and Bert puppets. One company made a Bert puppet where Bert's unibrow was sculpted but the head (at least the one I had) was made of hard plastic and not Rubber so his mouth didn't work. And there was another one where Bert had a fabricated unibrow. There was an Ernie that's mouth wasn't as widely opened as another. Stuff like that, plus there was even a company from Italy I think that made them for the international Sesame productions. But yeah I think Grover, Roosevelt Franklin, Harry Monster and especially The Count being the rarest out of all of them, especially if he still has his eyeglass and cape. Another thing to mention is that most of these puppets came with a plastic rod (most collectors would call it a wand) which would easily attach to the puppets wrist so they can move very similar to how most Muppets work.
So your Grover is probably from 1973 (older then your Kermit and Fozzie which was probably made between 1976-1978) either by
http://muppet.wikia.com/wiki/Sesame_Street_puppets_(Questor)
or
http://muppet.wikia.com/wiki/Sesame_Street_puppets_(Harbert) which is quite rarer and I'm not sure how to tell the difference between them.
I have that puppet myself but it's a little warn out, I found him in a yard sale once. I love it, it looks quite close to the real Grover. Though it's in storage right now.
Anyway, I hope this information helped. And like I mentioned in terms of allowing your kids to play with them it really depends. If their gentle and careful, looking and holding it, knowing it's a collectible I don't see why not. But for any kind of playful, rough playing...I would suggest something more current. But that's just my opinion.