So it goes like this:

1) Parents get a voucher they can use for private schools.
2) Lower/middle class parents look at the cost of private schools and realize they still can't afford it.
3) Rich parents who were afraid of the "diversity" of inner city schools and were ALREADY willing to pay for private schools make use of the voucher thus reducing the personal cost to them of avoiding public schools.

The taxes I pay to my local school are not for the benefit of my child, they are to help educate the populace. My neighborhood is safer because of public education.