Yea, First Past the Post voting systems (with a bonus electoral college) have this problem: parties become big overlapping tents with the ever present crazy people. This actually makes the system less stable (whereas one would expect a two-party system to be more stable), not just because it creates polarization and lacks escape valves of smaller parties, but mainly because it gives immense leverage to candidates: to win nationally you need 30% to 50% of the votes, then to lead the party you just need maybe 25% of the votes and to be the head of the crazy faction you maybe only need 20%, so the winner can potentially take it all with 1.5% to 2.5% of the vote, that’s horrific leverage.