1.) What in your history makes players NOT want to play for you. It's seldom as simple as "I don't like the guy". Usually it's because you don't pick good players, play buddy ball, or don't pay for things you said you'd pay for.
2.) What level are you trying to play? If you're trying to compete at the Conference level and have a league team budget ... of course they aren't going to play for you.
3.) What type of personality do you have? Are you a leader? Have you had success with building a team before?
Or...sometimes its as simple as someone in the area already has the good players and they've been together a long time and don't want to split up...