Hand position inside the puppet (ie: comfort and control when operating the mouth) is very important, too.
I will second Blink's suggestion to practice with just a bare hand in front of a mirror or video camera before worrying too much about the puppet. This will clearly show how moving just the thumb has a stabilizing effect on the hand's position (this is important when conveying emotion and critical to controlling eye focus). A great little addition to the bare hand practice sessions would be a
Peepers Puppet. This will give you a set of eyes that can be "focused" while practicing lip synch and movements.
Once you get good with just your bare hand and a Peeper, then you'll feel much more comfortable when your hand is buried inside a bunch of foam. You'll also know how your hand is supposed to feel when moving up and down outside of the puppet, and quickly realize the trouble a poorly designed puppet mouth can cause when it does not support the hand properly. A poorly designed mouth can and will hinder your abilities to effectively lip synch and maintain good head positions. In the worst cases, bad mouth design will lead to tremendous fatigue that will zap the fun right outta performing and learning.
So, start simple with just your hand in the mirror. Hold your hand at the angle needed at first --- then above your head as if you were performing with an actual puppet once you get the lip synch stuff down. Then add a set of eyes to your hand and practice in that arm up (puppet up!) position a lot more.
THEN, check out some of the links already provided in this thread for a good stock puppet with a well thought-out mouth grip design. Don't be afraid to email the companies and ask how their grip is set up (if it is good for small-med-or large hands, etc). Think of it like a runner buying a properly fitted and functioning pair of sneakers. You can't wrap a pair of heavy construction boots around the feet of a marathon runner and expect them to perform very well for very long. Similarly, the puppet's mouth design, and overall weight, are the two most important considerations that will dictate its performability.
But keep it simple and don't worry about getting a good puppet on your hand until you're really comfortable with the basics of lip synch, head position (focus) and the emotional acting that little things like simple changes to your arm's angle can do.
Best of luck!