Yeah, but that doesn't prevent the dialogue from being either pre-recorded for the puppeteer to perform to, or else looped afterwards. Whoever actually performed Rowlf and Kermit in each shot, Jim provided both voices. He didn't have to be physically operating the puppet on-set at the time.
Obviously the vocals were pre-recorded (although I don't think it's the same recording that appears on the album - anyone know this for sure?), but it's pretty common to re-record dialogue that was performed live on-set for a variety of reasons. If someone else was indeed performing Rowlf for those shots where he speaks, they could have voiced him in order to give Jim something to react to, but Jim would then have re-recorded the dialogue later.