Heralde is correct that the musical numbers (not just The Mayhem) were pre-recorded and then the Muppeteers would perform the characters to the track being played. I assume this technique was used in the films as well. As for Outlands - someone, and I honestly do not remember who, but someone who would know (Matt Vogel maybe?) commented they were singing live. They had multiple puppeteers so one (I assume the main performer did the mouth and others did arms and legs. It was beautiful in that they made no attempt to hide the fact that these were puppets performing. They were all in black, so as not to be too distracting, but aside from the beauty of their songs there is beauty in the puppeteering, and I am happy people could see that.