Some women said, “I never saw Louis CK playing with himself in front of me”- one (Sarah Silverman maybe?) even said “I did but it didn’t bother me”. All were, to one degree or another, chastised.
So far no one that I’m aware of has come forward and said, “Joss Whedon never tried that kind of behavior with me”– and if they did, inevitably people would respond, “So it wasn’t enough that he did with Charisma Carpenter? And Ray Fisher? And Gal Gadot?” etc.
And to continue my earlier example, several have at first said, “I never saw Woody Allen molest a kindergarten girl”, only to be challenged with “So you don’t think Dylan Farrow is telling the truth and therefore don’t believe women?”
If we’re not separating one accuser from dozens- which many aren’t- then saying “I never saw Scott Rudin throw a stapler, therefore I’m fine working with him” isn’t all that different than saying “I never saw Cosby roofie any of multiple women, so sign me up for the reboot of his sitcom”’. All that will happen is Jackman would be accused (rightly or wrongly) of being part of the problem– the same kind of complicit behavior, as it happens, that Karen Olivo is calling out today.