In a true agile world I believe the end user or their representative (product owner) is involved throughout the delivery which is the whole point really! However does that mean UAT is consumed in QA or is it still distinct in which case what is it called and who manages it and who executes it in your experience? Especially since everyone seems to “do agile” differently!
2 Likes
That’s fine as long as the product owner really is in touch with end users. That’s going to be very dependent on what the product is, who its end users are, and what the structure of the development organisation is.
One product owner I worked with was very much the contact point between user groups and the company. However, the product was very mature and many of the people who came to user groups were product geeks who liked nothing better than to put the bonnet up and rummage around in the code. But they in turn were the client organisations’ IT people, and we had no evidence that the actual day-to-day users, admin staff or even lay end users accessing the app through mobile devices were being accurately represented.
Still, this was better than other roles I’ve been in where a distributed user base had no direct feedback mechanism and the product owner had no contact with them at all, or at best spoke with representatives of large corporates who went to big trade shows and so had no contact at all with many users who were one-employee companies with a white van who had to use the app as a condition of doing business with High Street clients at local level.
2 Likes
How project I’ve worked on are conducted is really dependent on how available the client is. I have never been on a project which is truly greenfield, i.e. a project where everything is new and everyone is 100% available.
In most cases the real end-user isn’t available for testing. In some cases a subject matter expert (SME) is used but even then they aren’t available 100% of the time.
In traditional testing, everything is tested at the end (inverting the test pyramid). The solution is to push testing down. Get more unit testing, contract testing, etc…
I believe traditional UAT is testing too late too. We want to see if we can be less reactive and more proactive. My company has User eXperience (UX) experts. They get involved during story writing. If possible, they will conduct tests and surveys with the end-user. Based on what they learn about the end-user (or someone representing them) gets baked into the story. When the story is kicked off we like to have a Business Analyst (BA), UX, Quality Advocate (QA) and Developer (Dev) make sure we are all on the same page. If possible we’d like to see the product owner (PO) there as well. The BA and UX will clarify information in the story. A QA is there is make sure we thought about everything (usually negative cases as well as the happy path). We all work to make sure it can be implemented (the Dev helps here) or some sort of compromise is agreed upon.
After developing the story we do a desk check. Every role involved in the kick off (not necessarily the same UX, BA, QA) should also be available for the desk check. This is to make sure we all agree everyone stayed on track. Sometimes a Dev might tweak the design because the technology pushes them in that direction. Sometimes they get caught up in the details and miss a few acceptance criteria. Sometimes non-functional issues arise that we didn’t think about because the implementation details hadn’t been formed. Again, we might have to make a few compromises.
Personally, I also like to see our build pipeline such that there is an environment we push to for showcases. A showcase is when we demonstrate what we implemented in the given timeframe (sprint, iteration, etc.). We’ll have a dev environment which might be occasionally unstable. We’ll have a QA automation environment were we run some automated tests. Ideally, we’ll have an Exploratory environment for QA to do exploratory testing (this will not get deployed to automatically; we want QA to approve a deploy whenever they need a new build to test something). Once a build has passed the QA environment, we would be doing the desk check. We then want an approval stage before we push to the Showcase environment. In this environment, we set up the data to demonstrate the new features.
We could use the Showcase environment for UAT testing. Really at this point end-users shouldn’t find a lot (hopefully they find nothing).
Now all this is ideal. I usually set up a subset of this. Usually because the system we are programming has to link to a legacy system. If the legacy system has a dev, QA and production then we often have to have our new environments share a back-end legacy system. Even worse, the data is usually outside of our control and we have to share the data with other departments.
Darrell
2 Likes
A “true agile” doesn’t exist. Agile and End-user testing fit together in so many ways.
- On one project we work agile internally as a development org (ie IT company that does Application Management) and the customer does it’s own end-user testing afterwards.
- Another project has bi-weekly demo/sprint test with customer representatives in-house
- A project to build upon MS Dynamics 365 as a standard system is only tested via UAT and some internal functional testing
… and everything in between. From no UAT to a large waterfall UAT in the end. The later usually for large enterprise applications or public services, the former for usually for business-to-consumer apps, websites etc.
In the examples (and very real projects) above, the test manager from the IT shop manages it and the participants are public administrators, nurses and medical secretaries, health authority specialists - and the customer sides subject matter experts etc.
User Acceptance Testing is one of the testing type which is performed by client in order to ensure that end product meets the business requirements. If we compare the UAT found in waterfall projects with Agile one’s then the UAT found in Agile projects are more strict and well timed.
While working on Agile projects, software testing companies figure out number of opportunities to inject the UAT activities. Lets have each of them one by one:
- Agile UAT begins with user stories which includes both story and acceptance criteria. Mentioning the acceptance criteria before starting on a story creates a focus that helps the team to stay focused on actual requirement and minimizes their potential for rework.
- Later on, UAT in Agile projects is found in the end of sprint demonstration. Here, the stakeholders can provide their feedback by increasing the acceptance level.
- In last, UAT involves a dedicated sprint to perform overall user acceptance test and fixing discovered defects. It should be done by Product Owner along with the subject matter experts.
Here, we can easily figure out that UAT is a necessity of every project in order to increase the product’s market value.
Hope this information is helpful for you.
2 Likes
Basically UAT is a process that confirms that the output of a project meets the business needs and requirements. UAT in an Agile project generally is more rigorous and timely than the classic end of project UAT Testing done in waterfall projects. To know more about UAT process you may follow. https://bit.ly/3iBsSvx
The classic definitions of the user acceptance test is process that confirms that the output of a project meets the business needs and requirements. UAT in an agile project generally is more rigorous and timely than classic end of project UAT found in waterfall projects.