How Do You Evaluate the Quality of LLM Responses?

[ivoqa (Ivo T.)]

Oooo this is something I have been waiting for so long to grab my hands on! Tnx for the link, it interests me immensely how are they testing AI

Exactly on that topic I have a short funny anecdote that I wrote here:

QA training changes due to generative AI

That is a great question, to which even people working in the field often don’t know the answer. Just few weeks ago I was attending a developers conference and there was this one presenter who was working for some company that is using Microsoft’s AI platform (https://oai.azure.com/) to create new products around it. He seemed genuinely an expert in AI and what he was doing so after the talked I approached him and I asked him the normal question: how do they test such systems?! He said they ha…

TLDR: most people using AI commercially have no clue how to test it