@solalnathan There is a very dangerous trend in so-called #AISafety right now, which is the effective altruism theology driving this agenda, funding it, and convincing college freshmen that they need to reorient their entire careers to prevent the AGI apocalypse. They have hijacked the conversation and are steering into very dangerous directions, and they started the large language model race in the name of creating "human friendly AGI."

@timnitGebru @solalnathan

I saw a lot of job postings related to "AI Alignment". Is that related to this trend?

I didn't understood 100% what it meant but some of the descriptions sounded like Effective Altruism stuff.

@rcalsaverini @timnitGebru @solalnathan Yes. "AI Alignment" comes from the LW/'Rationalist' movement, and is the idea that AI is best understood as a magical genie, and that everyone who is concerned about the dangers of AI should focus on praying for a 'good genie' instead of an 'evil genie', which is dangerous nonsense in large part because it enshrines mystification about what AI is and how it works and what the effects on the world are.

@flaviusb

I tend to agree that not all funding should go towards "AI Alignement" and that LW/AE are overconfident in the existential risks linked to AGI (or even the emerge of AGI). I think "classical AI Safety" (for lack a better term) needs more funding and recognition.

Yet, I still think you strawman the general idea behind alignment.

What do you mean by "mystification"? Many people in alignment are literate in ML and push for concrete alignment of existing AI.