Another solution is to place tolerance in the context of social contract theory: to wit, tolerance should not be considered a virtue or moral principle, but rather an unspoken agreement within society to tolerate one another’s differences as long as no harm to others arises from same. In this formulation, one being intolerant is violating the contract, and therefore is no longer protected by it against the rest of society.
https://en.wikipedia.org/wiki/Paradox_of_tolerance