What modifications are necessary for QA training procedures for new joiners in light of generative AI, such as ChatGPT, and other AI technologies? Does anyone have insights into the adjustments required for QA Training procedures
That is a great question, to which even people working in the field often don’t know the answer.
Just few weeks ago I was attending a developers conference and there was this one presenter who was working for some company that is using Microsoft’s AI platform (https://oai.azure.com/) to create new products around it. He seemed genuinely an expert in AI and what he was doing so after the talked I approached him and I asked him the normal question: how do they test such systems?!
He said they have no idea! I was dumbfounded. He said they have no clear guidance, their goal is to put the product to market ASAP. I would have a plethora of further questions had he had time 
I got a feeling the correctness of genAI is not so critical in their (current) line of work as thousands of such companies are speeding to put some AI products out and grab some money.
I can imagine, however, how tough must it be. For example, pretend you have 1000 PDFs that you want your new AI bot to scan and then you ask a question “make a summary in 10 sentences”. How the heck can you be sure what to expect? How to test the validity of such response?
I imagine future QA training procedures for AI field will probably involve some fuzzy logic and more of a list of certainty checkpoints rather than hard data responses like we’re used to in today’s automation. For example, you must design a series of questions and expect answers to fall into a fuzzy array of possible data. The training is there then to distinguish what can be labelled as correct and what not.
2 Likes
Yes, this is an AI-based product, but I have concerns that it may impact the regular QA training for those who work with systems that do not integrate this AI. I believe the training plan, which was created for regular QA, also needs to be revisited.