I'd say it depends on the situation. If they're coming to performances and being supportive, I'd say that's a great thing. If they're being negative in any way, it might be something to address.

Maybe consider inviting them to come and speak to your students sometimes. Make them feel they are a part of the band culture they helped establish. We did that in a previous situation, and the community and the students seemed to like it.