Interested in being a guest? Email us at [email protected]

Your most powerful product might also be your biggest liability: an AI agent making decisions you can’t see and answers you can’t predict. We sat down with Andre Scott of Coralogix to unpack how to make black-box systems measurable, accountable, and—most importantly—improvable over time.
We trace the journey from monoliths to microservices to LLMs and explain why old-school “index everything, analyze later” monitoring breaks under today’s data explosion. Andre introduces an analytics-first approach that processes telemetry in-stream and then stores what matters in your own object storage. That shift delivers cost control and true data ownership, turning observability from an insurance policy into a growth engine. We dig into open tooling like an LLM trace kit built on OpenTelemetry that captures prompts, responses, and metadata, so you can evaluate correctness, flag prompt injection, and enforce guardrails at runtime.
Bias and hallucinations don’t announce themselves; they creep in through context loss, retrieval misses, and model updates. The fix is continuous evaluation with small, purpose-trained models that run outside your app to score tone, safety, factuality, and leakage risks. Think of agents like employees: give them performance reviews, train them with real data, and escalate when risk spikes. We also explore Olly, CoraLogix’ agentic SRE that reads your telemetry, answers business-grade questions, and recommends alerts and remediations—especially handy when cloud outages ripple through your stack.
Regulation is coming fast, and accountability rests with the teams who ship AI into production. If you deploy it, you own the risk. The practical playbook is clear: embrace analytics-first observability, capture LLM telemetry, make evaluators your crown jewels, and keep the data that teaches your models to improve. Subscribe, share this with your engineering and product teams, and leave a review with the one place you’d add guardrails first.

Support the show

More at https://linktr.ee/EvanKirstel