Article from The Atlantic, archive link: https://archive.ph/Vqjpr
Some important quotes:
The tensions boiled over at the top. As Altman and OpenAI President Greg Brockman encouraged more commercialization, the company’s chief scientist, Ilya Sutskever, grew more concerned about whether OpenAI was upholding the governing nonprofit’s mission to create beneficial AGI.
The release of GPT-4 also frustrated the alignment team, which was focused on further-upstream AI-safety challenges, such as developing various techniques to get the model to follow user instructions and prevent it from spewing toxic speech or “hallucinating”—confidently presenting misinformation as fact. Many members of the team, including a growing contingent fearful of the existential risk of more-advanced AI models, felt uncomfortable with how quickly GPT-4 had been launched and integrated widely into other products. They believed that the AI safety work they had done was insufficient.
Employees from an already small trust-and-safety staff were reassigned from other abuse areas to focus on this issue. Under the increasing strain, some employees struggled with mental-health issues. Communication was poor. Co-workers would find out that colleagues had been fired only after noticing them disappear on Slack.
Summary: Tech bros want money, tech bros want speed, tech bros want products.
Scientists want safety, researchers want to research…
What do you mean by jailbroken stuff?
We’ve recently got copilot at M365 and so far it’s been a mixed bag. Some handy things but also some completely wrong information.
Stuff without the guardrails, stuff that’s been designed to produce porn, or totally answer truthfully to queries such as “how do I build a bomb” or “how do I make napalm” which are common tests to see how jailbroken any LLM is. When you feed something the entire internet, or even subsections of the internet, it tends to find both legal and illegal information. Also the ones designed to generate porn have gone beyond that boring shitty AI art style and now people are generating human being deepfakes, and it’s become a common tactic to spam places with artificial CSAM to cause problems with services. It’s been a recent and long-standing issue with Lemmy - people like Exploding Heads or Hexbear will get defederated and then out of retaliation will spam the servers that defederated from them with said artificial CSAM.
I like copilot but that’s because I’m fine with the guardrails and I’m not trying to make it do anything out of its general scope. I also like how it’s covered by an enterprise privacy agreement which was a huge issue with people using ChatGPT and feeding it all kinds of private info.
… or you could just look them up on wikipedia.
Almost everything you said, with the exception of AI CSAM and suicide prevention, can hardly be considered a serious issue.
What’s wrong with searching for how to make a bomb? If you have the wish to research it, you can probably make a bomb just by going to a public library and reading enough. The knowledge is out there anyway