It could be possible that the "other times" Statler sounded off may have been times Jerry Nelson was voicing him (he was the original performer to continue the character after Richard Hunt's death from the Muppet Christmas Carol to around It's a Very Merry Muppet Christmas) Steve Whitmire started doing Statler soon after (and in fact one line in VMX was accidentally left unlooped by Nelson and could be considered Steve's "debut"). To me Whitmire sounds closer to Hunt's version than Nelson though Jerry actually was the very first person to play Statler in the Muppet Show pilot (Richard did the character in the show proper onwards).
I actually think Dave sounds quite like Jim's Waldorf (granted some times more than others) and think he was totally the right choice. There's always been some voices Dave does that sound very Jim-like some of them more gruff ones (like the lion from Slyvester Stallone's gladiator number) and his pig from the mountain climbing number in John Denver episode, and some of the higher pitched ones like Waldorf (i bet he would have done a good Newsman too) Traveling Matt also sounded very Jim-like and back in 1982 when i was around 10 and not quite as adept as voicespotting as i am now, i thought it WAS Jim playing him at first.