E55 - Why Traditional Testing Fails for AI Systems - Dušanka Lečić

From prompt failures to hallucinations: what breaks in AI testing

This time I talk with Dušanka Lečić about why testing chatbots breaks everything we know about traditional QA. She explains how chatbot bugs are invisible – they hide in prompts, retrieval logic, and chunks, not in code – and why the same input can produce dozens of valid outputs. Dušanka shares her framework for testing context retention, hallucination control, and accuracy, and reveals why stress testing a chatbot means checking for typos and user frustration, not system load.

"For the same input we have a lot of diffe...