“AI is nowhere near to being ready to replace you at your job. It is, however, ready enough to convince your boss that it’s ready to replace you at your job.”
I remember reading an article or blog post years ago that persuasively argued that the danger of AI is not going to be that it ends up doing things better than humans, but that it causes a lot of harm when entrusted with tasks it actually isn’t good at. I think that thesis seems much more plausible now, watching people respond to clearly flawed AI systems.
That reminds me of a fairly recent article about research around visualisation systems to aid with interpretable or explainable AI systems (XAI). The idea was that if we can make AI systems that explain their reasonings, then they can be a useful tool, especially in the hands of domain experts.
Turns out that actually, the fancy visualisations that made it easier to understand how the model had come to a conclusion actually made subject matter experts less accurate in catching errors. This surprised researchers and when they later tried to make sense of it, they realised that they had inadvertently dialled up people’s likelihood to trust the model because it looked legit.
One of my favourite aphorisms is “all models are wrong, some are useful.” Seems that the tricky part is figuring out how wrong and how useful.
This is nothing new though. For decades, managers have fallen for “solution in a box” sales pitches even though front line workers know it’s doomed to fail as soon as they set eyes on it. This time the solution just happens to be “AI.”
It’s worse now than ever though, many managers have been steeped in tech optimism their whole working careers. The failures of “revolutionary new systems” have been forgotten about while the success of other things are lauded.
They’ve been primed to jump on any new “innovation” and at the same time B2B marketing has started adopting some of the most manipulative practices that used to be only used on consumers. They’ve crafted a narrative that shapes discourse so the main objections that appear are irrelevant to the actual issues managers might run in to.
Stuff like “but what if it is TOO good?!” and “what if the wrong people get their hands on this AMAZINGLY POWERFUL new tech?!”
Instead of “but does this actually understand anything or is it just giving output that looks correct?” or “ Wait, so, how was this training data obtained? Will there be legal issues from deliverables made with this?”
The average manager has been primed by the zeitgeist to ask the sales rep the kinds of questions they want to answer.
Seems to me that a lot of the world’s problems start with “well, the managers think…” They all seem extremely bad at the whole managing thing, good thing we don’t overpay them or anything like that.
“AI is nowhere near to being ready to replace you at your job. It is, however, ready enough to convince your boss that it’s ready to replace you at your job.”
I remember reading an article or blog post years ago that persuasively argued that the danger of AI is not going to be that it ends up doing things better than humans, but that it causes a lot of harm when entrusted with tasks it actually isn’t good at. I think that thesis seems much more plausible now, watching people respond to clearly flawed AI systems.
Never attribute to malevolence that which can be explained by incompetence.
Including the end of humanity at the hands of the robots apparently
That reminds me of a fairly recent article about research around visualisation systems to aid with interpretable or explainable AI systems (XAI). The idea was that if we can make AI systems that explain their reasonings, then they can be a useful tool, especially in the hands of domain experts.
Turns out that actually, the fancy visualisations that made it easier to understand how the model had come to a conclusion actually made subject matter experts less accurate in catching errors. This surprised researchers and when they later tried to make sense of it, they realised that they had inadvertently dialled up people’s likelihood to trust the model because it looked legit.
One of my favourite aphorisms is “all models are wrong, some are useful.” Seems that the tricky part is figuring out how wrong and how useful.
This is nothing new though. For decades, managers have fallen for “solution in a box” sales pitches even though front line workers know it’s doomed to fail as soon as they set eyes on it. This time the solution just happens to be “AI.”
It’s worse now than ever though, many managers have been steeped in tech optimism their whole working careers. The failures of “revolutionary new systems” have been forgotten about while the success of other things are lauded.
They’ve been primed to jump on any new “innovation” and at the same time B2B marketing has started adopting some of the most manipulative practices that used to be only used on consumers. They’ve crafted a narrative that shapes discourse so the main objections that appear are irrelevant to the actual issues managers might run in to.
Stuff like “but what if it is TOO good?!” and “what if the wrong people get their hands on this AMAZINGLY POWERFUL new tech?!”
Instead of “but does this actually understand anything or is it just giving output that looks correct?” or “ Wait, so, how was this training data obtained? Will there be legal issues from deliverables made with this?”
The average manager has been primed by the zeitgeist to ask the sales rep the kinds of questions they want to answer.
Seems to me that a lot of the world’s problems start with “well, the managers think…” They all seem extremely bad at the whole managing thing, good thing we don’t overpay them or anything like that.
Probably bosses are trying to convince AI that AI is ready.
That’s great though. Then said boss can rehire the people they fired for a noicely risk-adjusted premium.
Stupidity traditionally hurts (the wallet)