Hi @eman, shifting left or right means going beyond the traditional “testing phase” in the work cycle to contribute as tester. Imagine a waterfall diagram:
analysis — design — development — testing — deployment into production — operations/support
On the left, it can include pairing with developers and participating in events like three amigos, where the PO/analyst, the developer and the tester discuss a user story or requirement to catch questions and problems early and get a common understanding of what needs to be done.
On the right, it can include working together with support personnel or using monitoring of the production deployment to guide risk analysis and testing decisions.
Especially in a continuous delivery context but also in more traditional settings it’s helpful to look beyond the edge of the own plate to look for a solid foundation for these decisions - and it doesn’t get much more solid than looking what actually happens in live usage of your software.
I hope this helps
30 Days of Automation in Testing Day 4: What Types of Testing Can Automation Support You With?
Below are some types of testing I applied automation during 5 years working on automation testing:
Regression Testing: For re-testing preexisting application functions that
are being carried forward to new versions (usually the majority, unless
app is brand new)
• Smoke Testing: For getting a quick high-level assessment on the
quality of a build and making go / no-go decision on deeper testing
• Static & Repetitive Tests: For automating testing tasks that are
repetitive and relatively unchanging from one test cycle to the next
• Data Driven Testing: For testing application functions where the same
functions needs to be validated with lots of different inputs & large data
sets (i.e. login, search)
• Load & Performance Testing: No viable manual alternative exists
Day 4 - #30daytestingchallenge
Challenge - What types of testing can automation support you with? Share an example
Automation is very useful in below areas:
User Interface / GUI or Front-End: Automation tests written to mimic the different flows , functions and interactions on a Web /Client application. This can be done by using appropriate tools like Selenium/Protractor / QTP. The core approach is to first identify , locate and store the Elements on these pages/screens and then perform operations on them using programming. Validations are implemented using Native tool assertions or by using custom programming depending on the tool support for these. Frameworks like TestNG , Junit , Jasmine make it easy to manage large number of tests and plan batch execution of tests to achieve automated regression testing. Being able to run these tests on a nightly basis using CI/CD tools like Jenkins/Octopus allows us to achieve high level of reliability and quality in the application or system under test.
Database: In projects that involve lot of back end validations i.e. projects that use ETL / SSIS / SSRS / SSAS etc. Automated tools can be used to run OLTP validation queries on the database and validate the results using the tool assertions or custom assertions. Tools like Selenium used in conjunction with programming languages like Java , C# can leverage ODBC/JDBC connectivity to achieve this.
API or Web Services: Tools like SOAP UI , Rest Assured enable automating the REST / JSON service calls using programming languages like Groovy / Java. Being able to automate API testing means significant reduction in effort and improvement in quality since each API end point can have multiple methods and each method can have multiple possible tests. It’s humanly not possible to reliably test all these manually every time a change is made in the system. If we rely on manual testing for API validation then we will be forced to take a risk based approach which can cause regression defect leakage.
Mobile The leading mobile platforms i.e. iOS and Android support three types of applications i.e. Native apps , Hybrid Apps and Mobile Web based apps. In addition to this AR/VR features , AI features are recently added to the mix. Mobile automation using tools like Appium , Robotium etc. enable us to execute automated tests on virtual mobile devices or device emulators or on a farm of real devices. In addition to mobile devices automation us widely used for Smart TV software based devise like Roku etc.
Performance : Performance testing scripts using tools like Load Runner , J-meter can generate virtual user load on Information Systems to mimic actual user loads to measure and optimize performance. These can be added to a CI/CD pipeline to run more frequently and identify the performance issues early rather than at the end of a project lifecycle.
Security : I’m not sure about writing automated security tests , but off the shelf tools like HP Web Inspect etc. run attacks and report the vulnerabilities which are then later assessed for possible resolutions.
Day 4: What type of Testing can automation support you with?
As a manual tester, there are certain tests that cannot be performed by simply executing a combination of steps and requires sophisticated tools to simulate load from the number of users accessing the same applications simultaneously and system vulnerability to attacks/intrusion (XSS scripting).
Non-functional testing involves testing the application against the non-functional requirements, which typically involve measuring/testing the application against defined technical qualities (also known as the ‘-ilities’ because they all end in ‘-ility), For example, Vulnerability, Scalability, Usability.
Performance, Load, Stress Testing- measuring how a system behaves under an increasing load (both numbers of users and data volumes), load testing is verifying that the system can operate at the required response times when subjected to its expected load, and stress testing is finding the failure point(s) in the system when the tested load exceeds that which it can support.
Security, Vulnerability Testing (Penetration tests) -Security testing tests the software for confidentiality, integrity, authentication, availability, and non-repudiation. Individual tests are conducted to prevent any unauthorized access to the software code.
Usability Testing-Usability testing looks at five aspects of testing, - learnability, efficiency, satisfaction, memorability, and errors.
Compatibility Testing- tests that the product or application is compatible with all the specified operating systems, hardware platforms, web browsers, mobile devices, and other designed third-party programs (e.g. browser plugins).
Hi @maos, thanks for the information.
Challenge - What types of testing can automation support you with? We can automate many types of testing. For example, regression testing will save us from routine. Security testing, Load, Stress testing are very difficult to perform manually. However, it is necessary to monitor the results of the autotests. I believe that this is one of the important parts - a clear presentation of the autotests results, it will help us to react quickly to changes and make the right decisions.
I believe this varies depending on the context. But generally, automation can possibly support heaps in below areas.
Repetitive Tests: Test cases that have a scope to get repeated over a period of time are a good candidate for test automation. This is one of the main reason for teams to usually automate regression/ smoke/ sanity tests. They usually save heaps of time, cost and man power in the long run.
Unit Tests: Automation could also help run those system level unit tests that a developer would want to verify before deploying to staging or test or prod environment.
API Tests: In a situation where the application interacts with multiple services, it makes sense to test those interactions in an automated fashion.
Non Functional Tests: While non-functional sounds too broad, automation is usually in testing the performance aspect more than any other non-functional aspects (UX, UI, Usability) of the application.
You can find the references here too: https://qakumar.wordpress.com/2018/07/05/day-4-what-types-of-testing-can-automation-support-you-with/
Based on this thread and some conversations we’ve been having in slack, it looks like we share a lot of the same experience.
On one hand there seems to be a very conventional idea of “Test Automation.” That is, I have an application, perhaps a web-based one, and I use some software, perhaps Selenium to run tests on it.
On the other hand there is “task automation to assist in testing” and it looks like a lot of us are doing this as well.
I recently learned Selenium, and haven’t been able to use it yet (though I have been practicing!). I may not even have the opportunity to do conventional, automated testing where I work, but I can see a LOT of use for it to speed things up, reflecting what @heather_reid said above.
Similarly I’ve started using a program called AutoIT to greatly speed up some repetitive tasks in a GUI. It might be possible to develop some sort of test framework with AutoIT but I haven’t explored this, and I do not test GUIs at my job.
And then there’s Python to automate…everything. (You know, the boring stuff.) I have written programs to speed up checking of hard to read data files, to pull logs from servers and validate data, and to automate generally boring tasks in the office.
There are a ton of tools out there and I look forward to learning Cucumber, POSTman, and a few others next.
#Day 4 : What type of Testing can Automation support with ?
Data Driven Testing
Load \Performance Testing
Black Box Testing
Web Service\API Testing
I work on automation for automating the regression tests- of a web app.
Day4: Ok, here’s where I am sad to be late on my challenge since I’m a little late to the discussion.
So, for this one of the things I wanted to say is Regression. Because it’s obvious right? Repetitive tests, big test suits you have to run every 2 months or LESS which sometimes take your entire QA team to be testing for over 2 or 3 days (sometimes more). Count how much money that is. If you are going to be repeating this process every release automating regression sounds like a good idea.
However I was reading this piece which suggest automating Regression tests is bad advise. What the author says
“unfortunately according to many testers and devs, regression testing means run tests over and over – automatically or repeating testing procedures, even worse – repeating them blindly without considering context. Problem is a regression bug is not necessarily a bug that we already know about and have a test about, regression bugs are not bugs who “rose from their graves”. Therefore, repeating tests to find regression bugs is like throwing bunches in the dark to hit a mosquito.”
I’m very interested in your opinions on this.
I read this here:
Automate all the things
… that can be automated.
Especially in the context of automation as tool supported testing/ confirmation. So everything from small scripts, to coding frameworks to robot automation.
I have implemented Automated Security Testing using OWASP-ZAP where everything is automated…Enabling the zap and then running the scan through API’s. Also, automated functional testing was integrated and run as part of it to have better coverage.
Comparing two documents e.g. pdf’s. This was bit tricky as there are tools to compare web page url’s like launching the url and then taking screenshots. But comparing documents was bit tricky which I have implemnted by converting pdf’s to image and then comparing image pixel-by-pixel. This was achieved by using 2 tools and through 2 commands. Also, section of pdf’s can be ignored from comparing.
If you visit the site:
It list of all of type of testing and what types of testing automation could support you with
Below are types of testing that we could apply automation.
Unit tests : Very low level, close to the source of your application.
Integration tests: Verify that different modules or services used by your application work well together, These types of tests are more expensive to run as they require multiple parts of the application to be up and running.
Functional tests : Focus on the business requirements of an application.
Performance testing: Check the behaviors of the system when it is under significant load.
Smoke testing: Basic tests that check basic functionality of the application.
Exploratory testing: Automation is key to make this possible and writing tests sooner or later will become part of your development workflow.
@rk_manne - this is just about the same thing I was going to write
- The only thing is Security would be handled by other tools not within the scope of this conversation (or section).
Vulnerability Checks: can be handled by an automated tool using ZAP and BurpeSuite to scan the entire directory of a project’s code base and look for any flaws.
Dictionary / Brute Force Hacking: Again, another means of testing that can be automated but not in scope of this club
Below is my experience:
1.Unit test: Very low level, close to the source of your application
2. Smoke test: I applied it in the support ticket project, because ticket was create by the client and depended on the need of client so we can not use automation for ticket.
3. Regression test: build framework to cover the regression test cases
4. API testing: use to parse JSON, compare with database automatically…
5. Data-Driven: use to generate data from API and database
6. Mobile testing: robotium to test application…
7. Load test and performance test with Jmeter
8. Non functional testing
As Peter mentioned on Twitter, automation can help all kinds of testing. In general, some ways (“how”) it can help are:
- Consistent repetition of actions and checks (computer are dumb, but they do not get tired);
- Inspection of hard to reach environments (e.g: unit testing, API);
- Precision in analysis (e.g.: performance testing).
I have read all of those on the topic, It’s very useful for me and below in my experience:
- Front End: Selenium and Protractor for automation test case to run a regression test
- API or Web Services testing: Postman to check manually after that using Selenium for automation test case to run a regression test
- Performance testing: Use J-meter
- Mobile testing: Use Appium, Robotium