Does anyone have any experience with testing in regulated industries?
(e.g. aviation, health, machinery, agriculture, etc)
Iām trying to learn as much as possible as fast as possible (thereās a LOT, lol). And while get lots of stuff about record keeping, standards and compliance, detailed processes and workflows, risk-based testingā¦ but Iād really like to understand this from a more day-to-day practical perspective.
Can you share some activities, stories, concerns that are intrinsic with working in such environment?
P.S. Iāll likely get a QA lead with 15+ years experience to run a webinar/round table about it, and would like to make sure weāre diving into real concerns, not just repeating what everyone in the industry knows.
I work in such an environment for 12 years now (medical devices) but canāt say that there is a standard way of dealing with things. Even in the same company we have different quality management systems in different departments while adhering to the same standards and regulations.
What all have in common is that they are audited on a regular basis to get the certificates to be allowed to sell stuff to the market and that the auditor checks if you fulfill the requirements of the standards and laws and that you fulfill your own setup process.
That means that the day to day work is build on top of that process. If the process says that you need to make screenshot of every dialog that is shown, you do that.
The standard differ a lot between the mentioned industries, though. Automotive is stricter in some areas than building medical devices and more lenient in others. Focusing on one of the mentioned industries might help for your round table.
Yes, the last 10years Iāve worked for large life science companies, Pharma producers specifically. They each follow the regulations in their own peculiar way and yet another form of qa. A qa focussing on more on evidence, traceability and approvals.
Bring an open mind and a quest for critical thinking
I work in a medical device context (so FDA regulation).
While there are guidelines, I think a lot of it isnāt so much prescriptive as holding you to whatever process you said you were going to follow. In fact, in our organization (and others Iāve seen), āQA Engineerā doesnāt mean a software testerāitās someone responsible for making sure weāre dotting our iās and crossing our tās on the paperwork side and doing what we said we were going to do.
And this talk by Griffin Jones, āWhat is Good Evidenceā, which is likely applicable to regulated environments more generally, not just medical devices: https://www.youtube.com/watch?v=i8he7Rejn5s
I LOVED this @jesper - Can you expand on it please?
Seems so contradictory. āopen mind and critical thinkingā while under so demanding regulations.
I appreciate the takes @tazthegulk
So how do you tackle these?
What is under your control or influence?
Can you perhaps draw innovation and agility from somewhere else?
What I am pulling from this is that the regulations are not prescriptive but work as an accountability tool. Am I getting this right?
Seems a contrarian opinion to the rest of the thread, particularly to what @raymund and @tazthegulk said.
@c32hedge Can you help me understand a bit more about the % split between what regulations say you must do vs your own criteria/standards?
Actually, what c32hedge says supports what I wrote: The regulations mostly tell you do this or that but not in a way that tells you how you need to do that. Thatās where the variation at my company comes from.
When youāre audited you get hints what you should do differently which might just be a favorable way to do things the auditor likes. If that is added to the QMS you suddenly do things differently than others.
In terms of requirements you can sometimes reach out to the regulator and ask for assistance. If no luck there work with the business as early as possible to clarify requirements before theyāre finished.
Funding is harder but look for cost savings on things like licensing and learn the funding model.
Time lines it becomes the old adage of being pragmatic and seeing what testing you might need to drop.
Have an open mind that while the world of gxp regulations seem familiar, they are a world of its own. The terms might be similar to software testing terms, but have nuances. Usually to the more formal and documented side.
The most recent fda guidance ācomputer software assuranceā aka CSA finally puts a focus on designing the approach based on critical thinking and a more risk based approach.
A quality management system is a core concept. This is what describes your way of working, this is what you are audited towards. Also it is in itself under change control. It details HOW you follow the regulations. The regulations arenāt (contrary to many beliefs) that specific.
This leads to differences, even within a company, as @raymund mentions. Everyone think they have the truth but really each have their own.
I spent a good chunk of my previous career working in chemistry testing laboratories covering environmental, food and agricultural testing.
Most of the testing involves QA and QC scenarios off the top of my head.
QA
documentation surrounding proficiency testing. This involves taking notes of what was being done, types of instruments used, periodic instrument performance reports, out of spec reports
documentation surrounding method validation and verification on laboratory analytical methods. This is where precision and accuracy studies come in. Weirdly enough there is a certain degree of traceability in raw chemicals and certified reference standards come into play here.
Yes Iāve worked on a few projects in regulated industries.
Key takeaways:
Make sure to document and show some sort of traceability between your tests and the requirements/laws.
People automatically assume this means they need to do test cases and then trace them back to requirements in a test case maangement tool, but there are actually other ways to go about this
Some requirements/laws are open to interpretation, so it can be trickier to test some mor than others
I think you are spot on @diament. This is probably one of the biggest and most important learnings/differences Iāve observed from those coming from areas not so heavily regulated.
I started my testing journey as a Manual tester at a Health Insurance company after having worked there in various other roles for many years. Health Insurance meant Financial regualtions as wel as the Health industry ones.
I then moved to working for a SAMD (Software as a Medical Device) using AI for diagnostic purposes. Which involved regulations from FDA, TGA and other countries, not to mention all the various Privacy regulations for all the different jurisdictions.
I now work for a Goverment Lottery which has again a different set of regulations due to being a Govermemt Agency as well as specific regulations for Gambling etc
In all these different businesses the one thing people tended to get caught out by was being able retrospectivly provide traceability for the coverage of requirements/rules or regulations. Trying to get a clear answer on which ones actually apply and which ones dont etc.
I found a lot of people approach it more from the standard ui/front end development approach where they focus on the behaviour or look and feel of the ui. Once a thing has been tested it often isnt documented for reference. Which is a problem when auditors come knocking.
Or even worse when a year or two down the track a issue is discovered in prod and some higher up demands to know āhow that got through testingā. But no one knew it was a rule or that it needed to be tested.
You cant prove that, if you dont have any form of traceability of where requirements/rules came from that are covered by testers. Also theres often no record/process of who is validating that what is being tested is sufficient and not missing requirements/rules that the testers might be unaware of. Which means the testers are often expected to be the SMEās of the system and business by default. This is more a problem for newly established agile teams when the rest of the business is used to working waterfall.
