I went to my last interview in slacks and a nice polo. The days of the stuffy suit and tie are gone. Teaching is physical now. We are expected to be up and moving around. I got the job and no one said anything. I have done hiring and I have never commented on someones attire unless they were haggard and I have never seen that.

Maybe in some places with Pseudo-professional driven image you may be judged. However, teaching has changed.