I must have overlooked it in the guides, but I had not realized that David was only refusing to give the broom back until she gave him some respect. All this time, I thought he was just afraid of what she would do with the broom (and he does seem that way when she finally gets it back), I thought him making her, in her old lady form, say "please" was more to see if that was actually her (thinking she would not say the word).
I also thought that the characters recognized her from the movie, but there's no such fourth wall breaking there.
In the last few days I had been thinking about how back then, The Wizard of Oz was not available on home video (which was not much of a thing back then) and only broadcast once a year. Maybe there was some Super-8 releases, but really, now the film is a lot more easily accessible to the public than it was in 1975, so maybe back then kids were a lot less used to the witch (though it was prominent enough to be the focus on an episode of Mister Rogers Neighborhood to show how it was done).
Kind of weird at the end, the broom falls back on Sesame Street, and David is crying because he knows he'll have to go through it all again, even though he did get her to say "please" and saw that if he had just given her the broom she would have left. I mean, when she comes back, just give her the broom immediately! Kind of hilarious (or harsher) in hindsight when the witch says she'll never come back to Sesame Street, considering they chose not to rerun it and it's only now available for fans to watch.