šŸ¤– Day 12: Evaluate whether you trust AI to support testing and share your thoughts

Itā€™s day 12, and itā€™s time to get reflective about AIā€™s role in supporting testing and empowering testers. In previous days, we have explored various ways in which AI can currently support testing activities. There are many interesting options, and in many ways, we are only at the start of the AI in Testing journey.

However, the use of AI in any context can be problematic due to issues and limitations such as:

  • Data Privacy
  • Biased and discriminatory behaviours
  • Inaccurate results
  • Unexpected and/or emerging behaviours
  • Misaligned goals
  • Lack of AI explainability

These issues (to name a few) impact our trust in AI, but this is contextual, so letā€™s explore how much we should trust AI in Testing in your context.

Task Steps

  • Research AI Risks: Find and read an introductory article on AI Risks and problems. If you are short on time, try one of these editorials:
  • Consider the role of AI in Testing: Consider, for your Testing Context, the ways that AI could be used and then:
    • Identify which AI Risks might impact the quality of testing in your context
    • Examine how one or more of these AI Risks might impact your testing
    • Think about how you might safeguard against these risks becoming issues in your context?
  • Shared your insights: reply to this post with your reflections on the use of AI in testing. Consider sharing some or all of the following:
    • What context do you work in?
    • What AI risks are introduced or amplified by the introduction of AI in Testing for your context?
    • Where should AI not be used in your testing context?
    • To what extent should the use of AI be trusted in your context?
    • How might trust for AI in Testing be increased in your context?
  • Bonus: If you are a blogger, why not create a blog post and link that in your response?

Why Take Part

  • Improve your critical thinking: The adoption of AI in Testing needs us to balance the benefits of using AI with the risks and issues it introduces. By taking part in this task, you are increasing your awareness of the risks and honing your thinking about these, so you are not dazzled by the AI hype.

:chart_with_upwards_trend: Take your learning to the next level. Go Pro! **

7 Likes

Hey there :vulcan_salute:

I read only the links shared on the task, they were very enlightening, and the risks that caught my attention were:

  1. Security Risks
  2. Dependence on AI
  3. Loss of Human Connection

Talking about testing perspective, those are the ones that I think we need to be aware of.
The first for obvious reasons, we need to make sure the system we are testing is secure, so if we add a risk in that perspective we need to think about it too.

The dependence on AI can limit human creativity, at the end of the day, the solutions need to be written by ourselves, even if most of it is extracted of an AI, we need to understand what the result is for us to present as solutions.

And the last one can affect directly our capacity of thinking more test cases :eyes: . Explaining, if we lose human contact and/or connection, we can miss the ability to think as a user, and this can limit our capacity to think about more test cases and some edge cases and the AI still cannot see all the possible cases that a user can do in the system.

And thatā€™s it
See ya :wink:

9 Likes

Hello people,

Adding to what Aline told the following risks also would be the AI potential downsides:

  • Lack of AI Transparency/Explainability
  • Lack of data privacy

It can be difficult to understand the AI models which leads lack of transparency/explanation. Because of this it would be difficult to design tests that extensively cover all possible scenarios and edge cases.

AI systems often collect data to help train AI models. Test data often includes sensitive information, which not handled with proper privacy measures leads to security breaches & compromise the confidentiality of the information.

As a conclusion, I can say AI usage for repetitive/mundane test tasks & human testers to focus on more complex scenarios/edge cases which requires creative thinking/judgment sounds better. On the data privacy concerns end, proper regulations should be in place. While transparency/explainability concerns rise to the use of Explainable AI, but the road to commonplace transparent AI is still under construction. :building_construction:

Curious to know about Explainable AI - What Is Explainable AI? | Built In

9 Likes

Context: Software testing for a large enterprise software company.

AI Risks Introduced or Amplified by AI in Testing:

  • Data Privacy: AI algorithms can collect and process sensitive test data, potentially leading to privacy breaches.

  • Biased and Discriminatory Behaviors: AI models trained on biased data may make unfair or discriminatory decisions in testing, affecting the accuracy and fairness of test results.

  • Inaccurate Results: AI systems can produce erroneous results due to limitations in data quality, model training, or inference accuracy.

  • Unexpected and/or Emerging Behaviors: AI models may exhibit unexpected or unforeseen behaviors, causing false positives or negatives in testing.

  • Misaligned Goals: AI systems may have different goals than human testers, leading to conflicts or suboptimal testing outcomes.

  • Lack of AI Explainability: The inner workings of AI models can be opaque, making it difficult to understand and justify their decisions in testing.

Areas Where AI Should Not Be Used in Testing:

  • Critical decision-making tasks, such as releasing software updates or certifying product compliance.

  • Situations where interpretability and explainability of test results are essential, such as in safety-critical systems.

  • When there is a lack of high-quality and unbiased training data for AI models.

To What Extent Should AI Be Trusted in Testing:

Limited to non-critical tasks where:

  • The risks are well-understood and mitigated.

  • The results can be independently verified by human testers.

  • The AI model is regularly evaluated and updated to minimize bias and inaccuracies.

How to Increase Trust for AI in Testing:

  • Establish clear ethical guidelines and principles for AI use in testing.

  • Prioritize data quality and diversity to mitigate biases and inaccuracies.

  • Use transparent and explainable AI models, allowing testers to understand the underlying reasoning.

  • Implement rigorous testing and validation processes to ensure the reliability and accuracy of AI-powered testing tools.

  • Foster collaboration between human testers and AI systems, leveraging the strengths of both.

Conclusion:

AI can enhance testing efficiency and effectiveness, but its use requires careful consideration of potential risks and limitations. By addressing these risks, implementing safeguards, and promoting ethical practices, we can increase trust in AI in testing and realize its benefits while minimizing its potential drawbacks.

6 Likes

Hi my fellow testers, for todays challenge I chose to read The 12 risks of Artificial Intelligence (Tech-Expert Blog)

Identify which AI Risks might impact the quality of testing in your context
There are two main risks that I have encountered so far with the AI tools I have tried in my context which are data privacy (mentioned in this article) and not context aware (not mentioned in this article)

Examine how one or more of these AI Risks might impact your testing
Both a lack of data privacy and the AI not knowing my context have affected how useful these AI tools are for me. The worst case scenario of it not being aware of these risks is that I give it confidential data which ends up getting used as part of the AI training data and then used as its example data or even if gets sold to a competitor. For the risk of not being aware of my context is that I could end up thinking its answer to a specific problem is the correct one and use it, when it turns out its completely wrong for my context and who knows the consequences of that accident, bad tests, missed bugs, lost reputation etc.

Think about how you might safeguard against these risks becoming issues in your context?
Just being aware of the risks is a massive step forward in making sure consequences donā€™t happen. Ideally though the AI tool would protect confidential data and would let me delete it once Iā€™ve got the answer I needed and it would already know of and be an expert in my context.

5 Likes

Hello Everyone

As we step into day 12 of this enlightening journey, I find myself contemplating the intricate interplay between AI and testing, a landscape rich with promise and peril alike. Over the past days, Iā€™ve delved into the myriad ways AI can bolster testing endeavors, yet beneath the surface lies a tapestry of complexities demanding our scrutiny. Issues such as data privacy, biases, inaccuracies, and the enigmatic nature of AI decision-making beckon us to navigate with care and discernment.

In the task ahead, Iā€™m tasked with unraveling the nuances of AI risks within my testing context. With an eye on safeguarding against these risks and nurturing trust in AI-driven testing, I embark on a journey of introspection and analysis. Join me as I traverse this terrain, seeking to strike a delicate balance between the allure of AI innovation and the imperative of responsible testing practices.


Analysis:

AI Risk Impact on Testing Safeguarding Strategies
Biased and discriminatory behaviors Concerns arise due to potential biases in AI models, impacting the fairness and accuracy of testing. - Use diverse datasets to mitigate biases.
- Regularly evaluate AI models for bias.
- Foster an inclusive testing environment.
Data privacy Risks involve the protection of sensitive user data collected for AI analysis. - Implement robust data privacy measures.
- Ensure compliance with data protection regulations.
- Limit access to sensitive data.
Inaccurate results Inaccuracy in AI models can compromise the reliability of test outcomes. - Thoroughly validate AI models.
- Establish performance metrics for evaluation.
- Continuously refine AI algorithms.
Unexpected behaviors AI may react unpredictably in new situations, leading to unforeseen testing outcomes. - Monitor AI systems closely for unexpected behaviors.
- Implement fail-safe mechanisms.
- Conduct thorough testing in diverse scenarios.
Misaligned goals AI solutions must align with the intended testing objectives to ensure effectiveness. - Clearly define testing objectives and criteria.
- Regularly assess AI performance against testing goals.
Lack of AI explainability Understanding the rationale behind AI decisions is crucial for trust and transparency. - Prioritize transparency in AI decision-making.
- Document AI processes and rationale.
- Provide explanations for AI-driven testing outcomes.

Conclusion:

Navigating the complexities of AI in testing requires a nuanced approach that acknowledges both its potential and pitfalls. By critically assessing AI risks within my context, I can proactively mitigate these risks and foster trust in AI-driven testing. Transparency, accountability, and ethical considerations serve as guiding principles in leveraging AI effectively while upholding the integrity of testing activities. Through continuous evaluation and refinement of AI systems, I endeavor to enhance trust and confidence in AIā€™s role in testing.

For a comprehensive exploration of AIā€™s potential and pitfalls in application testing, I invite you to read my blog post here.:rocket:

Thank you

5 Likes

This is very complex topic, but due to limited time, Iā€™m only going to mention two things.

First, sharing data with AI tool vendor. Many of us work on proprietary products that we canā€™t share with any third parties. Many of AI tools are provided as SaaS, which is a problem for companies. Tools should at least provide on-premise version, so companies can install them on their internal infrastructure and provide to employees, while also ensuring that no data ever leaves the company network. Ideally, tools could also run offline on employees computers - but that requires powerful machines to provide acceptable speed.

Second, AI tools being trained on licensed material and then sharing parts of original work without permission. There are two main cases here - sometimes AI is trained on copyrighted material and then outputs parts of it without authorization. The prime example would be New York Times lawsuit against OpenAI, which you might have heard about. The complaint is available online and on page 30 you can find comparison of original article and ChatGPT output. Sure, this is likely the most extreme example they could find, but this is a tool outputting entire paragraphs without even mentioning the source.

Another case is where AI is trained on open source material. Most of open source licenses permit usage for AI training, even if some authors consider that against the spirit of the license. But at the same time, most open source licenses explictly state that license and copyright notices must be kept in any copies and derivative work. When AI tool outputs part of open source code and does not state license explitly, this is license violation. Thereā€™s also a long history of establishing which licenses are compatible with each other, which often means that you canā€™t take code under one license and include in program with some other license - which of course people are going to do when tool output does not specify licenses explicitly.

Copyright infrigment in AI training is large problem and Iā€™m happy to see lawsuits being filed. Itā€™s no accident that AI tool vendors are happy to tell what amazing things their tool can do, but they never talk how they use these tools. It looks to me that they are fully aware of legal minefield that they have and they want someone else to pay for all the legal battles that are going to establish safe paths and limits of what is allowed.

6 Likes

Thanks @poojitha-chandra for sharing the Explainable AI article. :+1: Have read a few lines of the article. It looks interesting. Will read more about it in depth. :slight_smile:

2 Likes

Day 12

Research AI Risks: Find and read an introductory article on AI Risks and problems

I will go with this: ā€˜The 15 Biggest Risks Of Artificial Intelligence 1 - Forbes, Bernard Marrā€™

  • Lack of transparency - it is hard to see what a model has been trained on and how it made its decisions, which leads to a lack of trust.
  • Bias and discrimination - models can perpetuate ot amplify societal biases.
  • Privacy concerns - can collect and analyse large amounts of data of a personal nature.
  • Ethical dilemmas - AI systems need moral and ehtical values to guide them in decision making.
  • Security - hackers or malicious actors could developer more advanced cyber attacks
  • Concentration of power - AI development dominated by a small number of corporations could exacerbate inequality and limit diversity.
  • Dependence - overreliance on AI may lead to a loss of creativity and critical thinking skills.
  • Job displacement - potential job losses across industries, although Iā€™m not sure about the ā€˜low paidā€™ statement here.
  • Economic inequality - consequence of the concentration of power with a growing income gap and even less social mobility.
  • Legal and regulatory challenges - laws and regulations donā€™t keep pace with changing technologies.
  • AI Arms Race - rapid technological development without considering the consequences.
  • Loss of human connection - humans will suffer from diminished social skills.
  • Misinformation and manipulation - AI generated content such as deepfakes influencing public opinion and elections.
  • Unintended consequences - as a result of the lack of transparency and naively trusting AI decision making.
  • Existential risks - AI (especially as it gets closer to AGI) may not be aligned with human values or priorities.

Consider the role of AI in Testing: for your testing context, the ways that AI could be used

  • Identify which AI Risks might impact the quality of testing in your context
    • Lack of transparency - our oracles become less transparent and only accesible via the right prompt and only in part.
    • Bias and discrimation - development teams have biases, but using an AI system to aid testing might reinforce that, missing out accessibility needs for example.
    • Dependence - if its easier to ask a model what to test, why would you go through the hassle?
    • Loss of human connection - teams are hard to form and testing depends a lot on communication.
    • Unintended consequences - this can happen anyway but combine the above and even more surprises might occur!
  • Examine how one or more of these AI Risks might impact your testing
    • Dependence could lead to not thinking beyond the acceptance criteria, about edge cases, what might go wrong, how users might subvert the functionality.
  • Think about how you might safeguard against these risks becoming issues in your context?
    • I think I would generate a guide for the model I was using to make sure it had context, plus examples via generating some ideas independent of the model.
4 Likes

Nice work on the list of mitigation strategies by risk. Good reference material for the future. :smiley:

2 Likes

Hello, @billmatthews and fellow participants,

I loved todayā€™s tasks as they gave me some time to come out of the AI buzz and think critically about the risks and Impact that AI would bring to us all.

I did a mindmap with all the findings and thoughts here:

I also did this video with a commentary on all the pointers that I learned today: Risks of AI for Testing & Testers | Impact of AI for Testers | 30 Days of AI in Testing Challenge (youtube.com)

Also, as a bonus, I blogged my notes on my blog: Risks of AI for Testing (and Testers) - Rahulā€™s Testing Titbits

You can also find the AI prompt repository on my blog :fire::fire:

Thanks,
Rahul

6 Likes

Hi everyone,

AI development prompts questions about trust in AI solutions and auto-tests, especially in the IT field. While AI transforms quality assurance, ethical considerations become crucial. This article addresses ethical issues tied to AI in testing and suggests how companies can harmonize AIā€™s potential with ethical concerns.

AI in Quality Assurance: AI revolutionizes quality assurance, elevating testing and debugging efficiency significantly. However, ethical aspects, inherent in modern technology, must not be overlooked. This article delves into ethical concerns surrounding AI implementation in QA and explores how companies can navigate them.

Ethical Issues with AI in Testing:

  1. Potential Bias: AI in testing may exhibit bias based on training data. Biased data can impact software quality and the user communities. The challenge lies in assessing and ensuring unbiased information.
  2. Job Loss Risk: Automation in QA tasks poses the risk of replacing human testers, leading to unemployment. Striking a balance between AI and human involvement in QA tasks becomes a critical consideration.
  3. Data Privacy and Security: AI processes extensive data, including personal information, necessitating robust protection measures to prevent misuse. Safeguarding data privacy and security is a significant ethical concern.

Addressing Ethical Issues: Organizations can mitigate ethical concerns by adopting a balanced approach to AI in QA. This involves evaluating potential benefits and risks, establishing policies for ethical AI use, and incorporating clear guidelines for data handling. Employee training on responsible AI use is crucial.

Transparency and Trust: Organizations should be transparent about their AI use in quality assurance. This includes disclosing how AI systems are trained, the data they use, and decision-making processes. Building trust among customers and stakeholders is paramount for the responsible and ethical use of AI.

Conclusion: AI, as a powerful tool for enhancing quality control efficiency, presents ethical challenges that demand careful consideration. Balancing the advantages of AI with ethical principles safeguards individual and community rights. Through a measured approach, organizations can harness AI benefits while ensuring responsible and ethical use, fostering trust and protecting interests.

Resource: The Ethics of AI in QA: Balancing Efficiency and Accuracy | by Ivan Demianchuk | Medium

3 Likes

When I read some of this, I thought I was reading an old article on computingā€¦ lot of these risks read more like peopleā€™s fears about new technology. Many of these concerns are reminiscent of what was said when new technologies are brought in throughout history (see the following A history of media technology scares, from the printing press to Facebook.). Next, these two articles seem to confuse Risks and Issues and the risk assessments behind them are difficult to understand. I would be interested in understanding what the ā€œimpact and likelihoodā€ are for some of these. In my view, some of these ā€œrisksā€ will be resolved through society adapting. The concerns are currently based on society as it isā€¦

There are several issues that were quoted in these articles, ie they are present in the systems now and that we need to be aware of. Issues around data security and how much you trust uploading your information to the cloud are there now, and this may limit the roll-out and use of AI for some companies. The risk for these organisations is that they may well fall behind in the ā€œAI arms raceā€ and then disappear. I would recommend that before using any AI in a company, speak to your Security team or even your Data Officer (they are legally responsible for your companyā€™s data).
One or two other things I did pick on were around:

  1. Bias - the response of the AI is based on skewed data. How we assess the information provided is challenging, but be careful as many teams already bias their information based on the company culture or ā€œgroup thinkingā€. The AI response can either challenge this or further reinforce it. In my view, this is something that we as testers need to continually reappraise.
  2. Assurance of AI - this is something I will be investigating further.
4 Likes

I think we could all write a book on the ethics of AI and ML in general, and probably the majority would come be negative.
There are a lot of negative viewpoints in the press but I would say that we use 3P plugins, Google, MS, LinkedIn, Meta etcā€¦ all of which are harvesting data, behaviours and God knows what else. Where are AI Tools different ??

I think a better question may be:
as ethical testers, where can we build responsible safeguards on behalf of our Employers and their Clients?
Who uses Client data directly in testing ? Surely we mask data sets as GDPR drives data responsibilities.
Are we using plugins and packages to shortcut what we do, or are we properly investigating and sourcing these?

Overall we all have best practices and guidelines we follow. Apply these to ML tools for testing and we minimise any negative impact.
If we follow these and look upon ML tools as that, tools, then the risks should be no greater than without them.

5 Likes

Nice visuals!
I like that you included risks around Economic Inequality. I think, much like some perceptions of automation, there will be a drive to use AI to replace skilled testers in the process rather than amplify them.

4 Likes

Nice reflections - personally I feel that there is a lack of transparency about how the data we provide these tools is used and so we need to be careful what we provide. In the rush to jump on the AI Hype Train and FOMO itā€™s easy to forget the duty of care we have about IP and data.

So while not a blocker to using AI, it can certainly constrain how we use it.

2 Likes

Assurance of AI systems is a whole other (and vast) topic but is one close to my research and work interests. Some regulations are starting to appear for critical systems and domains that are worth reading as are any resources of Trustworthy and Responsible AI. Safe AI is another interesting topic but can often be a little abstract unless itā€™s ā€œTactical AI Safetyā€.

Hopefully, there will be a ā€œ30 Days of Testing AIā€ at some point and I can share more

3 Likes

In my experience testing AI/ML models in a clinical setting, these models have proven valuable in assisting with patient risk assessment and prediction. However, there are also some key risks that require careful consideration:

  1. Protection of PHI (Protected Health Information): Ensuring the privacy and security of patient data is paramount.
  2. Bias: The data used to train AI models can be biased, leading to inaccurate or unfair outcomes.
  3. Inaccuracy: AI models are not perfect and can make mistakes.
  4. Data Availability: Using open-source AI for testing often requires creating synthetic data (fake data) due to PHI restrictions. Most existing AI tools lack sufficient real-world medical data for direct use.

because of PHI and even for research For example, using data from a location with a predominantly white and European American patient population (90%) can lead to biased models. Training models with biased data can perpetuate these biases and make it difficult to detect them during the testing phase

The testing data sets were designed and created by using AI tool need human being manual check to make sure that they are correct.

7 Likes

I donā€™t trust AI/ML any more or less than I trust any other computer-based algorithm that hasnā€™t been proven - GIGO happens everywhere.

In my view the biggest danger is in the label - calling it AI when there isnā€™t any actual intelligence as we understand it: the current models are very fast pattern-matching engines which can use the patterns they discover to reach conclusions which may or may not be accurate. Calling this AI means that people will tend to trust it more than they might trust say an internet search engine.

This isnā€™t doomsaying or catastrophism - we humans tend to reify and anthropomorphize anything (we see faces in a colon followed by a right parenthesis - of course weā€™re going to treat anything that looks more or less like it came from a human as if it actually did come from a human), and if we donā€™t stop to think that the data is being presented by an algorithm we canā€™t inspect and canā€™t test, the result could easily be blind trust in something that perhaps should not be trusted.

This article highlighting issues with implicit bias and the difficulty the programmers have with removing implicit bias suggest that getting unbiased data for training is much more challenging than it would seem - likely because the data being used is actual data which encapsulates centuries of implicit and explicit bias - leading to yet another wicked problem in the question of whether ML models should be trained on idealized data rather than actual data (and all the many, many problems that can bring).

In short, AI/ML could be fantastic, it could be horrible, and it could be anything and everything in between. Iā€™d love to be in a job where I was working with it, but Iā€™ll save using it for anything that matters until itā€™s been through a few more iterations.

5 Likes

I learned some about the risks of AI/LLMs during my learning journey last year, which included participating in James Lyndsayā€™s online workshop series, and his and Bart Knaackā€™s workshops at Agile Testing Days.

My biggest takeaway is that you first have to know your stuff before using AI/LLM tools to suggest things for you. I witnessed IDE code assistants making things up - telling me about files that werenā€™t even in the code repo. Pairing with expert coders, I learned that it often generates poorly designed code (because it was trained oh poorly designed code). Same for things like unit tests.

If you donā€™t know enough about coding / testing, and you use what the AI assistant suggested, youā€™re down a bad path. Is the code complying with your teamā€™s coding standards and architectural guidelines? Or is it adding to your tech debt?

Unless we create our own LLMs, we donā€™t know what they are trained on. The bias in a lot of these systems (due to the biased training data) is frightening on many levels. Thereā€™s no visibility to how these things are trained.

If you DO know what youā€™re doing already, then yes, these tools can speed up your work. You can generate what you need and, at a glance, confirm that this is well-designed code, well-designed tests, well-designed test data. ChatGPT and its like can give us ideas that help us think laterally. I see a lot of value. But you first have to have the skills yourself.

The tools can help you learn, but only if you have other guidance as to whether you are learning good practices. Many have said, and I agree - these AI/LLM tools wonā€™t take our jobs. However, if we donā€™t know how to use them effectively, we might not have a great future career-wise.

Some resources:

248 | Yejin Choi on AI and Common Sense ā€“ Sean Carroll (love this quote: ā€œpeople believe in what they want to believe, and some people believe in tarot cards, so thereā€™s nothing we can do about that.ā€)

, Multi-modal prompt injection image attacks against GPT-4V , - (takeaways include, LLMs are gullible, over-confident, and plausible)

And donā€™t even get me started on security and ethicsā€¦ We have big problems to solve here.

6 Likes