What about each team appointing a panel of fans to complete an assessment of the referee after each match? The results should then form a 'league table' that is published and that should allow all referees to see where they are in the table.
If there isn't a formal way of referees being judged for their poor practice then they will not have the incentive to improve.