Data Whisperers: Teaching AI What's Right and Wrong

Interested in being a guest? Email us at [email protected]

Ever wondered who teaches AI what's right and wrong? That's exactly what we explore in this eye-opening conversation with Olga Megorskaya
Chief Executive Officer from Toloka, the company providing the human infrastructure behind today's most advanced AI systems.

Toloka stands at the fascinating intersection of human expertise and artificial intelligence, serving major players like Anthropic, Amazon, Microsoft, and Shopify. As Olga explains, while the industry has long focused on algorithms and hardware, data quality remains the overlooked pillar of AI development. There's a surprising "glass ceiling" that even synthetic data can't break through – at some point, you need human experts to push AI capabilities forward.

The work is surprisingly complex. Toloka manages thousands of specialized professionals who create benchmarks, evaluate model outputs, and even test for safety issues. One particularly fascinating project involves testing computer use agents that can operate across your entire system – reading emails, managing files, and interacting with applications. These expanded capabilities introduce new risks that must be carefully evaluated by human experts who can identify potential harmful scenarios.

What makes this conversation truly thought-provoking is Olga's vision for the future. The technologies being developed to manage human expertise at scale may ultimately transform how work itself is organized. And while tech giants make billion-dollar investments in the space, independent players with specialized expertise continue to thrive. Toloka is also taking tangible steps to improve diversity in AI, ensuring their benchmark datasets reflect diversity and even creating virtual training environments where female leadership is normalized.

Ready to understand the invisible human infrastructure behind the AI revolution? Listen now and discover why the most "unsexy" part of AI might actually be where its most important future technologies are being developed.

Support the show

More at https://linktr.ee/EvanKirstel