Doing that shows just how far some who call themselves "progressive" when they're nothing of the sort, shows Soviet/Rightwing success. Tearing down monuments to traitors to the U.S., i.e. of Civil War "confederates", renaming schools and streets is a wise thing; why should traitors be honored on public property?
Jefferson was a slave owner, as were most land-owning white men of the time. Still, he helped write the Constitution that was used to ban slavery so why tear down statues of him? A Plaque could easily be added to mention he owned slaves, but that slavery isn't something allowed today.
There are fringe groups on both sides but those on the "left" are less numerous than on the "right"; just note the widespread use of the battle flag of the Northen Virginia Army that people associate with the confederacy. Especially in the South but nationwide now. Those who take pride in it tend to be hardcore racists, Neo-Nazis, and outright White Nationalists, groups who invariably vote Republican.