Ooh, I wasn’t prepared for this! New challenge! There’s gonna be a speaker’s panel. We’ve got Rick, Lena, Nancy, Cassandra, Jack and Richard. Just listening to them present themselves, I’ve gotta tell you – we have a lot of very experienced people here!
Let’s see what questions we have…
If you had to recommend a resource aside from Ministry of Testing for testing newbies, what would it be?
We’ve got some silence and giggles on this one! Lena says that the owasp site is great for learning about security. Richard mentions automationintesting.com and test automation university. In terms of books – Explore It by Elisabeth Hendrickson and the books by Lisa Crispin and Janet Gregory. Twitter and conferences like Nordic Testing Days and Romanian Testing Conference are also great places to get information. Jack says if you’re in a big company, to use your network there too. Cassandra follows individuals within the community instead of aggregated content – and she has a twitter list of them.
Ooh, Richard is telling us that they’re planning on creating pathways that go cross-site, e.g. from MoT to other places. That sounds cool and like not reinventing the wheel!
Are there any channels or media where we are not present that we could tap into?
Rick says we could be more present at tech conferences. I highly agree with this! When we see the new stuff being invented we can get into conversations and maybe blog about the tech conferences from the tester perspective. Nancy adds project management conferences. They are, in her experience, hard to break into. They are still stuck in the “testing is scripts” mindset. Richard is suggesting going to meetups in your city (also a great strategy for getting free food). His specific example is the product owner space – especially since a lot of testers (this one included) have gone into that space.
Lena gets a whole new paragraph because she says “sales”. We need to educate ourselves more on the business side! And Jack reminds us that there are places we’re not visible in our own workplaces – and not to leave that out.
Have you ever seen stable functional automated tests for web applications?
Richard reminds us to start off that people like to blame Selenium – but Selenium does what you tell it to. If you build checks on a shifting basis, you will have flaky checks. Then people start to blame the framework. Make your basis deterministic and stable. Also, check your checks : run them multiple times, change their assertions. And make sure you design them in the place they will run. It doesn’t help to design them on your supermachine if your pipeline isn’t that powerful.
Rick also adds that if you don’t know what you’re doing manually, then you really shouldn’t start to automate. Lena advocates to find the tests you can throw away – not all tests need to live for ever.
If you have an idea for a great product, how and when do I hire testers for it?
Rick says that we need to know what we want and then find the people to fill it. If you’re hiring QA – what does that mean for you? Cassandra says that in situations like this, you alone are the first person at the start. And then the questions become more “where do I need help” “what do I want to keep” “what do I need” “what can I give away”.
Lena says she would always hire testers before developers. She believes that it’s easier to teach a great tester to code than a great coder to test. Richard says it’s about finding your problems. If you notice a problem – solve that problem. His example is that he is currently rewriting some of the front end of the website. It’s probably not the best use of his time, but it’s the best solution at the moment. In another area, they’ve decided to outsource some work. And if it’s testing specific and you notice patterns again and again – then you might want to hire someone specifically for that role. On the flip side, if you’re having bad quality and you hire a tester – the number of issues will increase at first. Because testers will find them. That’s what happens if you bring the specialists in!
After a few years of testing, testers move into other spaces (and rarely the other way around). Why do you think that is? Is it good or bad?
Cassandra has a theory! Can’t wait to hear it She was at one point asked to focus on one role (product ownership). But she said no because her focus and passion lies in testing. Her theory is that people are attracted to product ownership roles is that we see that role as having the power (because product owners make decisions). And some people see testers as not having the power (even though that’s not necessarily the case).
Nancy says there are three reasons:
- Developers are paid more than testers
- Testers are seen as less important
- It’s hard to get out of a testing role once you’ve been pigeonholed
She thinks all of that is crap (I agree!), but it’s the reality that she sees.
Rick adds that he also has a programming background. And sometimes he finds something that he can fix himself – and that can “spiral” into development! From a psychological perspective, he mentions cynicism. If you spend a lot of time looking for problems, you become cynical. And people don’T want to talk to you. So we need to get away from that – we can also talk about positive things, ideas, chances. We need to own our role as informers and then we’ll stay in the role.
Richard says he thinks it has to do with testing being an introductory role, and people “falling” into testing. People grow up saying they want to be a developer. There are university degrees and bootcamps for it. If we fix the avenues and the entry points, then we will get more people who will stay there. He doesn’t see a problem with testers moving to other roles – they will take all the good stuff they’ve learned into those roles?
Vern is answering! That’s not in the rules!!! Vern thinks it’s about quality. And that people can impact the quality more by writing the code or making decisions (I can empathise with this – a lot of the time, testers are asked into a project to fix the symptoms of quality problems at other places!).
Lena used to be a senior developer. She works with universities to bring courses about testing and she needs to work with other hiring managers. She wants to encourage them to hire people with a passion for testers, not people who want to be developers. Hiring managers need to learn more about testing and that it’s high status and high complexity tasks. Conferences that bring in developers as speakers and idolise code are not helping us. (Applause for that).
Do you have any failure that you want to share with us?
Maik has asked this question and gave an example of how he misconfigured an email that spammed lots of email addresses every 15 minutes for 24 hours.
Jack was testing a website with real data and the user’s account was charged.
Rick deleted an entire acceptance test environment at a bank. Data, builds, batches…. Everything. When he was showing how he did it, he accidentally deleted production as well.
Lena was teaching testing to a group of developers, testers and architects. They were exploring and accidentally broke the entire test environment: where the release was currently being tested.
Nancy has a hiring story. She interviewed a candidate over skype who she thought was absolutely amazing. He worked for a week, and he wasn’t working how she’d expected based on his interview answers. She finally realised it wasn’t the guy she interviewed. She met with him and spoke to him. After a few rounds, she found out that he’d got a friend to do the interview for him. (omg!)
Richard’s previous company had dashboards, and they’d just released. A customer came in and mentioned that the graph is wrong. Richard was called in, and the graphs were indeed showing skyrocketing 404 errors. They’d taken down production! And an external party had to notice it…. Moral of the story – if you have monitoring, look at it!
Why do you test?
Richard is an information fiend. He always wants to know what happens and how things work. He has a thirst for knowledge and tests everything.
Jack says it just fits with him.
Cassandra says that she actively pursued testing. She learned about languages (clojure was the first), she found a mentor. And through this, she discovered testing. And that was much more exciting than development. She thinks it’s amazing that we get paid to learn. Testing challenges her and is exciting. There’S always something to do.
Nancy is an eternal pessimist. She always thinks of what’s going to go wrong. When she was developing, she would be overwhelmed because she could think of all the ways it wouldn’t work. She ended up giving away all her developing activities to focus on the testing.
Lena was gonna be funny and say she doesn’t know how to stop. And then she realised it’s the only thing that it’s an intellectual challenge.
Rick says it’s the essence of who he is as a person. He hates repetition, squandering of potential. There is always something more to explore or see. The ability to do that and be valued for that is the essence of humanity and evolution. (Wow, I’m getting a bit teary here!).
What do you think about testing tests? Mutation testing for example.
Richard says he actively encourages it if you’ve coded your own logic in your tests. He wouldn’t necessarily run mutation tests on his code, but he will test the tests to check if they’re brittle (change assertions, rerun etc).
Jack says if your timeframe allows it, then testing your tests can be a good idea. But there probably isn’t enough time!
Cassandra is relatively new to automation, but she tests any tests by asking herself “what am I actually trying to test” and “will this test or experiment give me the information I need”.
Jack also mentions maintaining regression suites to check that your suites are still relevant and serving their purpose.
Lena says that you can have someone else look over your work to test it.
If you have 200 deployments per week, and 100s of microservices – how does exploratory testing fit in?
Nancy says “easier than scripted testing at any rate!”. She suggests business critical things are put into checklists. Use most of your day hands-on-keyboard testing so you can gain the most information. She also recommends knowing and ranking technical complexity – where are the risky places? It’s also worth looking at whether something has been changed or not. High business risk, high technical complexity and high change – that’s a top priority. Talk more about what you do test and what you don’t test. If people are worried about what you’re not testing, then renegotiate. She also suggests educating the people that control the time and money – and making them responsible for the decisions too.
Richard reminds us that exploratory testing isn’t a phase. You can be learning and applying the whole day. Exploratory testing belongs at every stage (even in production). You need to rely on the speed that the humans can learn and hope that the robots can be fixed in time.
Rick says do exploratory testing in production. He says it’s an unpopular opinion, but I’m used to this idea now!