That's true, y'know?
In all honesty, I don't believe I ever detected any radical differences in a character's voice one day, then another as a really young child, but I did understand as I got older (about the same age that Super Scooter pointed out) that if there's a difference in a character's voice, it meant someone else was playing him... and I believe the biggest example of this for me as a kid would be hearing Jerry Nelson's Crazy Harry in TMM, and hearing John Lovelady's Crazy "Donald" in The Muppets' Valentine Special.