🤖 Day 2: Read an introductory article on AI in testing and share it

[ansha_batra (Ansha B)]

How can we ensure a balanced approach that leverages both AI capabilities and human expertise in testing processes?

[ansha_batra (Ansha B)]

Your open-minded approach to considering the advantages offered by AI suggests a willingness to adapt and innovate within your team.

Just curious to know- What strategies do you think are most effective for integrating AI into quality assurance processes while ensuring a balanced approach that leverages both AI automation and human expertise?

[billmatthews]

I like that you used ChatGPT to recommend the articles

[damelis (Damelis Querales)]

Hello, everyone

Here the article:
Link: https://testsigma.com/blog/is-ai-really-important-in-software-test-automation/

This article takes a tour of the tasks of a QA before and after AI, drawing an overview of the possibilities and challenges presented by this technology.

The essential concepts:

Efficiency: AI is revolutionising software testing, making it faster, more accurate and more efficient.

Capacity: AI-powered tools can analyse tons of data and automatically create test cases, freeing up time for other tasks.

Industry transformation: AI is transforming software testing from a labour-intensive and error-prone process to a faster and more reliable one

As a manual QA, personally, I believe that adopting AI is very similar to automation, since the goal is to reduce repetitive work. This means that the QA testers should keep doing what they are already doing but also have knowledge of AI-based test automation tools and use them to improve their and the QA team’s efficiency.

After all, it’s about doing more in less time and with higher quality

[tester_ida (Ida Tester)]

https://testsigma.com/blog/scope-and-impact-of-ai-in-automation-testing/

My notes:

• AI makes testing software easier and better.
• It helps find problems in software more accurately and quickly.
• AI can handle tricky situations in software.
• Using AI in testing saves time and makes software better.
• It’s important for testers to learn how to use AI.
• AI testing works for different types of software like websites and mobile apps.
• It helps companies release software faster and cheaper, and customers are happier.

[rogerrobar]

AI with Software Testing (Browserstack)

Summary: Kind of loose aggregation of the author’s perceived benefits to using AI in software testing. Topics: Accelerating Timeline, Better Automation, Clarity on Client Requirements, and Mastering Test Automation with ChatGPT. Very high level, but perhaps some useful links provided.

I definitely see potential use in my own projects through all of the above topics, so I will use it to delve more deeply. Challenges are overcoming organisational resistance and opportunities are to speed up higher quality deliveries.

[melissafisher]

Hello
My article is a blog I found on medium by Mayank Gupta

How Artificial Intelligence in Testing works?

A few notes I wrote:

• Ai in testing is not meant to replace human testers but to augment their capabilities. Basically to compliment/enhance/make better.
• It makes me think about drawing out the testing process and exploring options of how AI can help enhance what we do. Some examples in the article are below:

• test idea generation
• test selection and prioritisation
• defect prediction and analysis

Oftent I have used other people to bounce ideas around on these types of things, however, now I could use AI instead. Might be really useful if you’re the solo tester on a project / or others are off work.

[rosie (Rosie Sherry)]

Hey Simon,

I’m choosing A Curious Journalists Guide to Artificial Intelligence.

It’s an introductory guide for journalists, which isn’t testing specifically, but I often feel that our role is an investigative one. Where we need to write, communicate tell stories to many people. The idea of automating tedious tasks and working with datasets is also intriguing, as is the navigating the hype and ethics of it all.

Here’s the ChatGPT summary of it:

1. AI Basics and Importance for Journalists: The guide introduces journalists to the fundamentals of artificial intelligence (AI) and machine learning (ML), explaining how these technologies work and why they are relevant in today’s media landscape.
2. Applications of AI in Journalism: It explores practical ways AI can be utilized in newsrooms, such as automating tedious tasks (e.g., transcription), analyzing large datasets to uncover trends, and enhancing reporting accuracy, while emphasizing the importance of human oversight in AI’s application.
3. Navigating AI Hype and Myths: The guide addresses the hype surrounding AI, debunking common myths and urging journalists to approach AI with a critical lens, recognizing its limitations and the potential biases inherent in AI systems.
4. Ethical and Legal Considerations: Highlighting ethical reporting on AI, the guide stresses the need for transparency about AI’s human creators and the datasets it uses, while also discussing copyright issues and the ethical implications of AI-generated content.
5. Educational Resources for Further Learning: Each chapter includes resources and “homework” for journalists to deepen their understanding of AI, from technical guides on machine learning to philosophical and ethical discussions on AI’s role in society.

[hiroksarker]

To ensure a balanced approach that leverages both AI capabilities and human expertise in testing processes, consider the following strategies:

Balancing Automation and Human Expertise in AI-Driven Testing

1. Early Collaboration
2. Test Strategy Design
3. Test Case Selection
4. Test Result Analysis
5. Continuous Learning

AI In Software Testing: Strategies For Enhanced QA & Debugging

6. Balancing AI and Human Elements in Testing
7. Evaluating the Use of AI and Adjusting Your Methods
8. Skillset and Training for AI Testing

By implementing these strategies, organizations can strike a balance between AI-based automation and human insights, ensuring optimal software quality and a competitive advantage in the market.

[maaret (Maaret Pyhäjärvi)]

A talk on AI I listened earlier this week made a claim about miniature stop signs - a form of adversial examples testing - to fool models trained to recognise traffic signs. Since we have so many stories of someone doing this type of testing, I used the basic article around reading back on this one.

Popularised by version of the story of trying four kinds of stickers: Researchers hack a self-driving car by putting stickers on street signs - Autoblog and the academic original: [1707.08945] Robust Physical-World Attacks on Deep Learning Models

Highlights I learned from the academic original:

• It’s a special brand of testing / hacking to seek inputs that fool AI generated models
• Very relevant when models are used in safety-critical systems and in scenarios where one cannot rely on human decision making (speed of deciding, even paying attention)
• They recommend adversial examples testing both in lab and in field
• They tested an adversial example of black and white stickers misclassifying 100% times in lab and 84,8% from screens from captured video

Here’s the thing: the story I went hunting for is one of miniature traffic signs. I did not find that story. I found stories of other ways of tampering with traffic signs, but not one where size of it would somehow have been the issue. There is a good change that “can fool a machine” has turned into “some people actually fooled a machine live” before the story made its way to a stage this week.

[gruja (Grujica Stanisic)]

Hello all!
I will not share any article , seems that we have huge amount and I can only repeat on of them.
Here are the my highlights for AI in testing:

1. AI can be used as a great assistant. But it can not serve well if tester do not have knowledge. Seems that correction has to be done in lot of cases, especially in coding.
2. AI automation tools can automatically correct test scripts when they encounter failures. Reducing time for fixing the tests.
3. AI tools can analyze vast amounts of data to forecast potential defects or performance issues.
   There also other interesting and useful thing that can be done with AI tool, but those are my favorites ( most useful) .

[akanksha.suneri]

Hello Everyone,

Here are my answers to tasks of Day 2 of 30 Days of AI in Testing.

1. This is the article I read - https://www.xenonstack.com/blog/ai-in-software-testing

2. Main takeaways from the article -

• AI is becoming an integral part of Software testing,
• It helps both developers and Testers
• Help Manual Testers as well as Automation developers.
• Helps in Defect Analysis and Defect Prediction.
• Saved Time + Money = Faster time to Market

1. Potentials: Test Cases generation, Test Data, Test Coverage, Test Maintenance, Test Report Analysis. Defect Analysis.
2. Challenges: While AI offers many benefits, there are challenges such as the need for skilled resources, data quality issues, and the complexity of integrating AI into existing testing processes.

Overall, the article suggests that AI has the potential to significantly improve software testing by making it more efficient, accurate, and integrated into the development process.

Below are some AI based testing platforms that i have come across and used.

1. Applitools
2. Copado
3. Functionize

Thanks,
Akanksha

[mrukavina (Marijana Rukavina)]

I am a fan of the community in general and specifically the testing community.
I chose to listen to the Test Guild podcast by Joe Colantonio, featuring Tariq King talking about “No B.S Guide to AI in Test Automation.”

Tariq King has vast experience in software testing and artificial intelligence (AI), long before the hype around the current generation of AI. He started in computer science and then moved into testing. He believes that while building software is challenging, testing it to ensure it works as intended is even tougher and more intriguing. I totally agree, as I’ve also moved from development to testing myself.
I find his thoughts and talks very interesting, down to the ground and realistic.

From his talk, I took away several important points:

• Maintaining automated tests takes a lot of effort, especially as systems change. The biggest problem is flaky tests - the tests that work sometimes and sometimes not, without any changes in the system or the test. AI could be a game-changer in making tests more stable, addressing a big challenge teams face.

• Before the latest AI technologies, like ChatGPT, people were skeptical about AI. Now, it’s easier for everyone to see how it works, which makes them more likely to use it. This is especially true for testers, who like to explore and understand new tools firsthand.

• AI will change some jobs but won’t replace people. Instead, it will change how we do things currently, hopefully for the better.

• The concept of using AI to test AI is fascinating, but we need to proceed with caution, especially in sensitive fields like medicine. Testing is all about understanding and communicating risks so that informed decisions can be made. While AI can assist, human judgment is crucial because AI can hallucinate a lot and be very convincing about it. So we cannot trust it in this form at the moment and as such we cannot test with it without keeping the human in the loop. We still need humans to ensure the AI is doing what it should, balancing verification (checking if the product meets specifications) with validation (ensuring it meets user needs without bias, unwanted behavior, or errors).

• Overall, it’s important to stay realistic about what AI can and can’t do. We should try it out and see for ourselves what it’s capable of.

Fun fact I used chatGPT to help me formulate my thoughts around it.
The process went like this:

1. Listen to podcast and take notes
2. Write the content in my own words
3. Give the first prompt to chatGPT: “Rewrite using simple language, more natural English syntagma and make it simple, up to the point and causal”
4. Give the second prompt to chatGPT: “you changed it a lot please keep all of my points in place, I feel like you ditched a lot of content that I provided” Naughty boy!
5. Regenerate again
6. Combine my words with the multiple responses I got from it.

So it was also more like conversation and mixture of human intervention and AI as Tariq is describing

[lisacrispin]

I don’t have an article (looking forward to suggestions here). However, I can highly recommend the book How To Test a Time Machine by Noemi Ferrera. It has an excellent chapter with a comprehensive introduction to using AI/LLM tools for testing. AND it has another excellent chapter on how to test AI and LLMs themselves. Both sides. I don’t have the book to hand right now so I can’t summarize it. I can say that it’s practical and down-to-earth. https://www.packtpub.com/product/how-to-test-a-time-machine/9781801817028

From my own experience, tools like ChatGPT and IDE AI assistants can give you ideas to brainstorm test scenarios, they can explain code (usually fairly reliable), they can generate Gherkin scenarios and the step code to run them - but you better know what you are doing, because those may or may not be correct. I did a workshop on using AI for testing at Agile Testing Days last year with Rachel Kibler. One of the things her team does with ChatGPT is to generate FAQs from their testing notes, and standardize bug reports so that all the information needed is in each one using a standard template.

[sanj_kichu (Sanjana Preetham)]

https://testsigma.com/blog/is-ai-really-important-in-software-test-automation/

I read this article & found it to be a good starting point

[raja19 (Raja S)]

How to gradually incorporate AI in software testing | TechTarget

I found this article very useful and highlights to me are,

• how to incorporate AI in testing gradually
• recommends taking smaller bets approach to evaluate and also build confidence in usage.
• Good summary on how to incorporate into defect analysis and continuous testing.
• It also provides good summary on benefits and how to choose appropriate tool.

Interesting read. Pls check it out.

[neil_thompson]

1. I am quoting here a blog post: might turn out a little controversial, as several people are quoting articles from vendors or related firms, which tend to be very positive about AI in testing. So, here with some trepidation is a contrary view, in the interests of balance: A Super-Quick Guide to Evaluating “AI” Claims – DevelopSense (author is Michael Bolton).

2. My main takeaways are that Michael starts with a logical semantic argument, proceeds to highlight risks (concept & methodology) then asks about experiment, experience & exploration (parts of a critical thinking methodology). He is not questioning a specific tool or type of tool.

        Then the “probe deeper” questions give more detailed food for thought.
   

3. I am an independent consultant and not currently in a contract (well, not a software testing one using AI-enabled tools) so this debate doesn’t currently relate to my day-to-day context. I am currently more interested in the testing of AI systems than using AI-enhanced tools as part of testing in general (but I am interested in both).

4. My personal reflections are that Michael is not the only author publishing this kind of analysis, and I think some relevant discussions may be found on social media eg Linkedin. But I haven’t fully investigated yet.

[gunesh (Gunesh Patil)]

So good to see everyone back here with interesting articles.

I actually stumbled on a reddit thread last week: Which open source projects will blow up in 2024? and I found loads of content on AI.

Interestingly, I found a repo from this thread, called GPT4All which hosts Open-source large language models that run locally on your CPU and nearly any GPU.

So, I thought why not give it a shot!

I went ahead and downloaded an instruction based model trained by Mistral AI - mistral-7b-instruct-v0.1.Q4_0.gguf available on GPT4All

and as I had no fear of OpenAI knowing my confidential stuff, I went ahead and fed one of the functional specification document for one of upcoming feature at work and got an awesome test plan in about 1hr (blame my computer)

That’s how I used AI in testing.

Looking forward to interesting ones from you!

[gunesh (Gunesh Patil)]

Love the second prompt

[arnaudruch (Arnaud)]

Hi everyone,

Went through QA Revolution: Accelerate Testing with ChatGPT – QA Revolution.

One of the ideas I liked in it is how AI can be used to refine and improve the quality of the different artifacts we are creating.

Based on this, I fed chat GPT with a snippet of my test automation code and asked it whether my code was properly written, as I’m not an expert automation developer.

I found the results insightful, although there was not so much to improve. My simple code might have not been written too badly

Maybe such a Chat GPT driven code review might be a good practice to implement on a regular basis.