Christopher Plummer has made a great bad guy in several films as an older actor, e.g., Inside Man. Gene Hackman was great as the bad-guy sheriff in Unforgiven. The 007 villain Le Chiffre showcases three great bad guys: Peter Lorre (1954), Orson Welles (1967) and Mads Mikkelsen (2006).
Alan Alda was like Andy Grifith, He occasionally did a 180 from his good guy image by playing heavies.
Even though he is known for his likeable roles, Morgan Freeman played one scary mother..... in STREET SMART.
So far you are the only who mentioned ladies. Angela Landsbury as the Machiavellian mother in THE MANCHURIAN CANIDATE.
Ned Beatty - absolutely chilling (and believable!) as Sheriff J.C. Connors in White Lightning (1973).
Jack Elam made a career out of being the bad guy in westerns....then he turns up as this guy and my appreciation for him grew even more.
I don't know if Kathy Bates role in Misery counts...it could be more psychotic than bad guy...but she certainly left an impression.
Oh yeah, for sure. A true monster. She's also not exactly sweet in Gaslight. Barbara Stanwyck, if only for Double Indemnity and The Strange Loves of Martha Ivers.
Anthony Zerbe. Not only did he give Charlton Heston a hard time in The Omega Man but he also gave The Red Hand Gang some trouble a few years later. PS Already mentioned but Richard Lynch was a good baddie. Check out The Ninth Configuration for a meditation on faith and bikers wearing mascara. (Actually, he wouldn't have looked out of place alongside Al Pacino in Cruising..)