I didn't really mind Brian being the character they used to rant about stuff. I remember seeing the American Dad episode about Bush with commentary... THAT was really ranty and you can tell Seth was really ticked off the entire commentary. Brian did it way too much. He's far better in episodes where him and Stewie go on adventures. They gave him a crapload of depth in the bank vault episode. But I really hated when they turned Peter into a child. it was cute the first couple times, but they do it every episode.
But the thing that was really missing was the tone of the show. The Simpsons was basically supposed to be part based off of Matt Groening's life part parody of 1950's sitcoms... Family Guy was basically a parody of late 70's early 80's type sitcoms, and everyone learned a lesson with tongue firmly in cheek. Even the cutaway gags made more sense (and were much quicker), and the 1980's cartoon parodies were usually confined to something Peter was watching on TV. The show got renewed and seemed like where it was when it left off, then the gamers and AS viewers took over the show, they dumped the 80's cartoon references for dumb stuff... The show has recently gotten somewhat better, but I really miss those first few seasons.