TikTok has announced that several hundred jobs in its trust and safety team are at risk, as the platform moves towards greater reliance on AI for content moderation. The UK cuts follow a broader global pattern: in September the company axed 300 content moderators in the Netherlands, as well as another 500 in Malaysia the following month.

In classic mythology, the Hydra was a snake-like creature that grew two heads for every one severed. Tackling content moderation seems to be our modern-day Hydra: every “solution” raises new problems, every “answer” generates new questions, and every new restriction creates more platforms with looser moderation (and morals).

A degree of automation is necessary: outsourced human moderators are typically poorly paid, inadequately trained, traumatised by the worst the internet has to offer, and overwhelmed by the task at hand. Meta employs just 15,000 content moderators for Instagram and Facebook, roughly one per 300,000 active users, meaning they have less than 30 seconds on average to assess the problems with a post.

Use AI instead, though, and the problems multiply. AI systems are prone to error — in 2020, Facebook’s automated systems blocked ads from struggling businesses, while last year its algorithm mistakenly flagged posts from the Auschwitz Museum. They also struggle with more nuanced areas such as sarcasm, satire, coded language, or geographical variations; for example, Holocaust denial is criminalised in France and Germany, but not in the US.

The Online Safety Bill hasn’t killed the Hydra either: it has only hastened its messy growth. How exactly does AI moderation deal with the concept of “harm”? When the BBC tried to use an AI tool to measure the abuse of politicians online, it ended up accidentally flagging plain descriptive words including “Tory” and “hypocrite”. If AI defines “toxic” as “rude, disrespectful or unreasonable”, as the BBC’s bots did, then half of X would be flagged.

There are other issues. Clause 9 of the Online Safety Bill also creates an obligation for platforms to stop users from “encountering dangerous content”, and therefore filter material before it is published. Many see this as a breach of freedom of expression, particularly as users are not always notified if or why a post has been taken down. The opacity of the system — and its internal biases — are concerning. Platforms such as X are moving towards “community-driven” moderation through initiatives like Community Notes, but AI algorithms are still ranking, flagging, promoting and demoting content according to secret internal standards.

By design, social media is built not for security but addiction. Big Tech may be excited by the “revolutionary impact” (read: cost savings) of AI, but we need human oversight to ensure moderation decisions are fair, accurate, and contextually appropriate. We cannot rely on AI, which is still very much in its infancy, to make moral judgements, or appreciate irony, or stop users from evading its filters, which is currently all too easy. Platforms like TikTok can try to absolve themselves of responsibility, but this Hydra is not dying down anytime soon.

---

Kristina Murkett is a freelance writer and English teacher.

kristinamurkett

---