Think about it, JK Rowling said that there were about 1000 students at Hogwarts. You figure two parents for every child, plus both parent's side of the family would know. (Unless of course, you're a Dursley and refuse to tell anyone, which wouldn't matter anyway, because he has like one family member.) Now, I know 1000 + family and whatnot doesn't seem a whole lot for a world so big, but Hogwarts is only for the United Kingdom it seems. In book four, we meet the Beauxbaton school, which is obviously from France. They have to have at least 1000 students also. And there's also Durmstrang(I'm not quite sure where they're from), and we know that they have a lot of students also. Plus, all these kids have familys too. So think about, that's only 3 schools within a fairly close proximity of each other, and they have that many kids. Think about how many other wizarding schools there must be.
There's very many adults in the wizarding world also, as told in SS, when Vernon sees a ton of people wearing strange cloaks and celebrating.
Now, also, most of the families nowadays are half-blood wizards, meaning at least one parent was a muggle. What are the chances that they wouldn't tell their own family that they married a witch or wizard. It just doesn't seem plausible to me that the whole world wouldn't know by now, or have at least some idea of a world of which they counter-live with.
One other thing is, how did Hermione find Diagon Ally and such? Both her parents were muggles, and the letter they receive when they turn 11 only tells them what they need to buy and such, how did she know where to go?
Excuse any poor grammar, it's 5:30am and I can't sleep.
jerrica . ravenclaw