I agree with what everyone's been telling you. I'd also add that even though it hurts to have a smear campaign put against you, the right people will listen to your assistant and see what you're actually doing and put the pieces together.
Again, some will believe him and it'll hurt, but more people will end up seeing someone that trash talks their head and that will go a long ways come interviews.