Okay, now I'm really interested in reading about that, because that's always been a question lingering in my mind... my two assumptions were either Jim felt Caroll being Big Bird and Oscar (essentially the original two Muppet leads of SST) may have jeopardized his credibility of performing other characters, or perhaps being Big Bird and Oscar were demanding enough parts that didn't leave Caroll time to perform other characters.
Either way, we have to remember too, that first year, it was just Jim, Frank, and Caroll: Jim and Frank couldn't have done everything for SST themselves.