Unfortunately, this critique is warranted.  An excerpt:

Many over the years have accused the ACLU of making a virtue out of practicing a kind of reductio ad absurdum in reverse, by not only defending the rights of the most marginal and often despised one percent among us but also insisting that those rights be honored by the rest of us, the 99 percent whose opinions on that one percent typically range from indifference to hostility. But the ACLU has done so on the principle that it is vital to defend the freedoms and rights of even the most repugnant among us, lest the very repugnance of such people confuse us into committing self-sabotage on our own freedoms and rights, which are inescapably the same freedoms and rights. Were we to make them situational, contingent upon popularity, those freedoms and rights (our own and everyone’s) would start to erode—gradually, then suddenly (to quote Hemingway)….