Anomaly tracking in stochastic systems?

Discussions
, , ,
1

Hey guys I’ve been working at a company that builds a tool that is centered around a chatbot with an LLM agent. We use a logging tool to look-back at previous conversations in our test environment and look for potential ways we could improve the experience. The traffic has been getting too big for manual review and we’ve been yet unable to isolate anomalies and potentially misleading responses. How do you think about anomaly tracking with LLMs?

3 Likes