I don't build Muppet replicas so I don't know if this will work for a Kermit, but I do think this is one of the best methods for making removable arm rods...
I like to use two different sized square brass tubes. I've often used a 5/32" tube and one that was the next size down. The second tube must be smaller than the first so that the two tubes can be "telescoped" with one fitting inside the other. The smaller tube needs to be large enough to fit inside the larger tube snugly. To test the fit to see if it is snug enough I usually telescope the tubes and hold them in the air vertically. If the inner (smaller) tube drops out immediately then I know the fit is not tight enough.
The tubes can be cut in to short pieces about 3/4"-1" long, usually the larger one needs to be a bit longer than the smaller one. You can epoxy the larger brass tube to a plate inside the puppet's hand; the smaller tube needs to be soldered or epoxied on to the end of the arm rod.
You'll also need to leave a small hole in the puppet's hand so that the rod can be inserted.
The arm rod with the smaller tube on the end of it should fit inside the larger one snugly enough to give you firm control of the puppet's hand, but can still be pulled out relatively easy. This method is a little more work than some of the others that have been discussed here on the forum, but imho it's a more professional, elegant and functional solution than using double sided tape, gator clips, etc.
Some people use a variation on this method, but use plastic tubing inside the hand and insert a metal rod with a tip that has been bent in to a loop, which requires a little less work to make. Be sure to experiment and try different techniques to discover what works best for you and the puppet you're building.
Sorry I don't have any pictures or diagrams to illustrate this right now. I hope that's helpful!