TestBash Spring 2023 - Failing for the Right Reason - a Fresh Look on TDD - Selena Small & Michael Milewski

During TestBash Spring, @selenasmall and @saramic gave a talk Failing for the Right Reason - a Fresh Look on TDD.

We’ll use this Club thread to keep the conversation going, share resources and answer any questions we don’t get to during the live session.

Questions Answered Live

@the_qa_guy - You stopped the testing at the model layer. Why didn’t you go down to the DB as well, especially on the username validation? That could be checked there as well

Anonymous - Do you also have QAs participating in your process?

Anonymous - Do you pair with testers to help them get more familiar with the code? Any tips and tricks welcome

@bischoffdev - How do you keep track of the different test coverages if these are done by disconnected teams across the company? For a small app it is quite easy.

@testerfromleic - How would you like “us testers” to use the information you’ve shared in this talk?

@fullsnacktester - Any tips for following a layered approach when different people, or even different teams at doing frontend, backend and db layers?

Questions Not Answered Live

@jfox94 - Without dedicated QA in your team, how do you ensure that testing is unbiased?

@mkutz - How much mocking do you use? And how do you make sure it doesn’t make your test suite rely strongly on internal details?

@anjanavijay - Are there any major challenges faced in the past without having a dedicated QA team aligned with development?

@soumyap - is it recommended that a single QA writes all the tests at the different layers or is it a collective responsibility?

@lee.marshall - What would you suggest when working with devs who are not wanting to pair or mob, only ever work alone?

Anonymous - Are you able to share the 2nd from last slide with the different testing diagram

Anonymous - To what level it’s really writing tests BEFORE implementation? I feel there was a need to have at least the webpage locators, endpoints, …

2 Likes

@jfox94 Can you give more context as to what kind of bias you’re referring to?

1 Like

to continue on @selenasmall question to @jfox94 - certainly, developers are biased - as demonstrated in the video - some won’t even test that their code works by using it in the browser. 1 solution is to pair program, the pair will pick apart the solution of the other developer and will be more likely to “try to break it” - yes it takes a special developer who can write some code and then effectively try to break it by testing it’s limitations.

2 Likes

Hi Selena.
Im referring to the bias that developers have towards their own projects. A dedicated tester can look subjectively at a development item whereas a developer may be somewhat blind to their own mistakes.

3 Likes

@mkutz we only use mocking in low-level unit tests, eg a single model. At a higher level, we may also mock the integration to a 3rd party service such as a Xero API. In the case of the 3rd party service, the API is very static so relying on it not changing is okay. In the case of unit tests, rather than testing what every method does and how it is implemented we try to test the behavior of that unit.

1 Like

@anjanavijay the major challenge without having QA is that developers generally DO NOT KNOW how the app works. They do not use it, they do not know all the business flows, and they are not across how infuriated some customers are. As shown in the presentation I was happy to commit the code WITHOUT testing it in the browser first. That is the value of QA. If your developers are “too specialised” or too overworked to understand the app then you need a QA role to fill that. Developers are expected to understand the product, write the tests, write the code and support it in production -that is a lot. Also having dedicated QA means someone can look at testing more wholistically as a function across teams (just like DevOps might work across a number of teams for infrastructure) - taking this burden off developers is important - just understand that developers still should be writing most of the mundane automated tests to develop the product but how that is tested in production or integrated between teams is where QA can shine

1 Like

@soumyap In our presentation, we are suggesting the layers we have written should be written by developers as part of the development cycle. This is development 101. However, developers can only test the things they think of - that’s where QAs come in. They’re finding scenarios in production across the actual app that developers would never have thought of. How often have you used a website or mobile app and been annoyed that some part of the experience doesn’t seem to work? Developers writing TDD code would not likely pick that up, but a QA might

1 Like

Every developers code will need to be reviewed at some stage and be shipped to production. There is a lot of ideas out there that doing this in small increments is overall cheaper than making huge sweeping waterfall releases. Assuming we agree on that, pairing, working with others is the fastest form of feedback and a reduced feedback cycle speeds up delivery of the team. Note this is NOT the perceived speed of a developer coding by themselves with headphones on but the actual delivery of bug free features to the customer.

this is terrifying for many

create a super safe environment and give it a go. Often people will actually be ok with the social aspect of it, can pickup little tips and tricks from each other and finally have their ideas verified early as they write the code

hey anonymous - not sure which slide? do you mean

but here is the whole deck as PDF failing_for_the_right_reason.pdf

1 Like

Spikes are imperative to TDD - if you’re not sure what you’re going to build or perhaps how to build it, spike code could be written to prove its technical feasibility or try out a UI. The key with spike code is that once you have figured out what you’re going to build, the spike code is thrown away. At which stage, your new feature or design can be test-driven. Chances are the implementation code will be slightly different because your tests will drive out a better-quality design.

Test infrastructure evolves over time, which will make it easier to write these tests quickly using webpage locators that already exist. The design system of your site should mean that pages are very similar, perhaps built with reusable components which means tests that make assertions in the browser may end up looking fairly similar or using webpage locators that were already written for a different test.

For production code, tests are ALWAYS written before implementation.

1 Like