Testing for Ghosts in the Machine: Assuring 'Good Enough' Software Quality in AI-based systems
Thursday, May 23
15:25 - 15:55
Join my talk on trying to master software quality in AI’s unpredictable world. I’ll share both challenges and solutions for developing reliable tests, with a focus on chatbots. Get ready to implement robust and automated testing strategies. Or at least get a bit closer to achieving it!
The presentation is the culmination of my experiences in two distinct projects - I have been professionally responsible for a quality of a chatbot and, despite the experiences, I’ve also created a hobby open-source project which utilizes the OpenAI API.
In this talk, I will explain the why - the state of current automated tests in the AI realm and why it’s important. I plan to dissect various methodologies, illustrating their strengths and weaknesses through real-life examples, predominantly focusing on GPT-based chatbots.. The presentation will also cover dead ends in hope to spare the suffering for those embarking on their own ghostbusting adventures in AI. Finally, I’ll share a few proved solutions which should greatly improve the quality and stability of an AI-based system on an example of a chat bot.
This presentation is designed to be accessible and engaging, requiring just basic knowledge about gen AI and software development cycle.
Artur Patoka
Hi there! I’m a Python enthusiast with a decade of hands-on experience in all kinds of automation development. Recently, I’ve been having the role of Senior Software Engineer in Test, diving deep into Python and Pytest. My journey has taken me through both corporations and startups and across countries, enriching me with cool international insights. I’m a huge advocate for quality assurance, not just as gatekeepers of product quality but as champions for customer happiness. In this role, I strived daily to make a difference. When I’m not in my professional zone, I love travelling, surfing and sharing my Python know-how through tutoring. I have enjoyed lighting up several PyCon conferences with my workshops. Let’s talk Python and quality assurance, and maybe share a few global adventures!