I'm kind of on the fence about it as well. I find myself wondering a lot more about "How did they do that?" when I see Kermit with his rods digitally removed, which I never do when I watch the Muppet Show. I think it's a lot easier to enjoy the performance without everything being "perfect."
I think at the end of the day, no one truly believes that Kermit is a living breathing frog. But they do believe in him and his personality. If the performer sells it, the audience will buy it. No matter how perfect the creation looks, if the performance is bad, no one will give it a chance. Think about Jar Jar Binks -- he was a great special effect (for the time), but who wants to see him?
One other thing you might think about is The Muppet Movie, which was made in an era before computer rod removal. I've watched that film probably a million times, but have never seen a rod in it. I'm not saying that they're not there -- but the manipulation and editing was so tight, that I've never noticed them.
As I said, I'm still on the fence whether to remove rods on the film I'm working on now. I'm focusing more on the performance.