If the HS director wants their program to grow, they need to visit and recruit. Continuously.
People don't quit jobs, they quit bosses. People don't quit band, they quit band directors. If they love music and band, they'll stay in no matter what. If they have no connection with their director, without that love of music to fall back on, they'll quit.
And if they love the band director, they'll stay in band for four years no matter WHAT is going on at their HS campus. But I don't know a single kid that's in athletics/band/choir/orchestra in high school who is there solely because they liked their middle school teacher/coach - at least, not for longer than a year.
It's up to a HS director to make that connection.