There's stigmas associated with it - a new one now is that homeschooling apparently leads to mass shootings, because evidently with the string of mass shootings we've had, it turns out many of the shooters just happened to be homeschooled.
Then, there's also the fact that most homeschool is faith-based, so that probably doesn't sit well with others since God and religion are not to be taught in public school.
And, of course, there are those who assume that homeschooling students goof off all day long and don't have to do any real school work at all.