Fuck AI

They have multiple people annotating the same video. They look for discrepancies in how data is annotated between people. Probably most often it's annotator error. It also allows them to spot actual oddities that maybe only one person noticed. There's a lot of redundancies. I think some might have a conception of AI-bros recklessly feed raw data into LLMs and sell people the garbage-out machine. It's not that at all. The source content and the written records are validated. It goes through refinement again and again to check. Who knows how many times. I've gotten by own annotations back at much later stages for further verification. That comes after even more people have looked at it and done whatever their assignment was with the data.

This inherently has nothing to do with AI. It's mass data harvesting and refinement. It's bigger than LLMs. Whatever technologies supersede LLMs will be fed these datasets. If they do it right once, they'll never need you or any professional ever again regardless of whether LLMs ultimately flop. They won't need you for the next iteration of AI technology. In theory of course.

The business here isn't AI. It's what big tech has always been after. Data.

Even if you get fired you wasted their time and made some money. Just have another job ready.

Introducing high level errors costs lots of money. The later in the pipeline stuff gets noticed, the worse it gets.

Don't listen to someone telling you to not listen to other people. I'm a level 69 LLM wizard and clearly outrank the guy annotating stuff. I never faced his dilemma because I'm so far into the ass of AI companies that they ask me to polish their plumbobs.

Jokes aside.

Poison the data. Get money.

You won't significantly poison the model, but you'll make it more expensive to train.

If it needs to be flagged it doesn't enter the dataset. LLMs can't be fed with errors, they're regurgitation machines, not thinking machines.

The things that could be made better by you are adversarial networks, which can punish the LLMs in reinforcement learning but there's an inevitable glass ceiling of model collapse in any case. These slop machines will always make errors, the broader you want them to get – that's their nature. Nobody is doing AI right nowadays, in 95% of cases they're just bumbling idiots in a gold rush.

Some errors will always slip through. The more insidious your errors are, the harder to spot they are, the higher the chance they won't notice.

Do your worst.