Do you proactively read PRs?

🙋 Questions
,
1

I’m curious if any testers out there are proactively reading new pull requests that pop in to the code based/apps that they test?

Have you ever spotted potential danger before the PR is even merged?

Do you feel empowered to do so?

If your team doesn’t use PRs, do you have another way of watching for changes? Maybe it’s a scheduled deploy/release instead of continuous

8 Likes
2

As well as reading PRs, I encourage my team go use the PR stage to pull code locally and test it. And I sometimes do the same, or deployment the branch to a test environment.

I have found many problems by actively reviewing PRs, I’ve also learned things and given praise.

6 Likes
3

That makes me very happy to read!

3 Likes
4

It honestly surprises me this isn’t the norm…I know it isn’t, but I find it surprising.

I should probably talk about this more often.

2 Likes
5

We use a little different approach. We add the concerned developer and all QA’s on the reviewer list.

  1. Developer and QA’s to take a look at the coding patterns ( Need approvals from the dev and one other QA)
  2. QA’s will pull the branch on their local and run the tests
  3. We have created an automation pipeline to run against local branches, that can be used to mimic the pipeline execution.

Only when all three checks pass, we merge the code.

3 Likes
6

Yes, reading PRs is the second thing I start doing when testing.

  1. Check the description of the story (requirements) + comments
  2. Check the PR
  3. Check how it really works
2 Likes
7

In my experience, I went through 3 approaches.

  1. I did not review the changes in PRs, I was only interested in the end result of black-box testing.
  2. Testing on the local environment of a given PRs and sometimes fixing trivial defects.
  3. The third approach, which I still use today and consider the most reasonable.
    Testing changes on a feature branch, on which the whole application is put on a temporary development environment (correspondingly smaller scale). In this case, I very often do a test review first, e.g. Are all the paths covered that I am going to test manually, both happy paths and edge cases. If not then I return PR with a request to add such tests.
4 Likes
8

It really isn’t the norm either. I gave a talk about this to our local tester user group a few years back and even the suggestion of reading PRs was met with confused or even disengaged reactions. OR the testers wanted to but weren’t empowered to do so.

3 Likes
9

I love a good pull request. I’ve used them to add to my list of oracles to inspire test ideas. And I then ask questions like this:

  • Well, what’s changed?
  • Why so?
  • What risks might that introduce?
  • What don’t I understand?
  • What questions could I ask and explore to answer about those risks?
5 Likes
10

Ouch! Clearly worth talking about much more then.

2 Likes
11

I have always checked out the branch, pull and test the tracked changes… is that really not the norm?

3 Likes
12

I’ve received resistance when trying to make it the normal, I was previously surprised it wasn’t the normal also.

I’ve also had support, and some are already doing it and encouraging it. So I’d say it’s mixed.

I also fully admit, some PR’s I’m reviewing by reading the diff in GitHub.

13

I’ve received resistance contributing unit tests to a project in a tester role. I was told any tests I write need to be browser automation. You know who told me that? Not the devs, they loved the added tests. It was my test manager. That wasn’t even that long ago.

You know what’s funny is the unit tests shifted detecting the regression left so browser tests weren’t needed. I’ve never been just a warm body to pump out test automation. :sweat_smile:

1 Like
14

Sounds like you are truly adding value, even if your manager can’t see it.

I’ve had a lot of support from my recent managers, to get involved in any aspects of Engineering, from writing automated tests, to contributing production code and pairing. Really enjoyed pairing with my team today, I got a turn driving, it was loads of fun.

1 Like
15

I’m about to give a talk at my quality community of practice to get more QAs into PRs. I know I’ll also get the confused looks as well. So it’s going to be a chat of this is why it’s important and here’s some strategies to start stepping your toes into it.

I really want to get more Quality peeps into the Unit Testing/Integration testing part of development, not actually doing it (unless they want to), but to help guide on what behaviors we should cover and validate we’re covering the business requirements and the most important test cases.

4 Likes
16

Yes in my current organization, we actively read PRs before writing scripts or conducting functional testing.

Whenever the code is merged, we try to read it most of the time.
It helps in many ways :

  1. It lets us know what developer has changed to fix the code or implement the features
  2. it saves time, e.g. if we don’t see the PR and directly write the scripts and then scripts fail because of not meeting the expected result then we have to raise a bug and then it gets fixed which usually doesn’t happen in the case of PR when we see something wrong in the code we report it directly and the developers fixed it
  3. it is different from typical testing we do as we have to understand the developer’s perspective that what they may be thinking while writing the code to fix the issue or implement new features.
18

Yes, I do feel empowered to look into PRs. I’ve heard of companies where testers do not have permission to see the source code of the product (usually enforced by the system), and honestly, I wouldn’t want to work in place like that.

In my current team the ideal situation is where testers approve the PRs before they merge. That approval may be based on actual experience working with a CI-provided build that has a PR applied. Obviously, it’s not always a case - some changes are trivial/small/obvious enough that just looking at the code is enough; sometimes testers in the team are busy with other things and don’t have time to take a look; sometimes another developer will do the testing. You know how it is.

Were I able to find an issue during a review phase? Yes, I was. Did it happen that I missed an issue I could have found during a review phase? Yes, it did :slight_smile: .

However, I am a bit confused by the first question, one about pull requests popping out. Unless you are working on a real open source app with actual community, that might be a sign of miscommunication between developers and testers. I mean, testers shouldn’t first learn about changes from the PR - these things should have been refined, possibly picked into a sprint, communicated during standup. I know that not everyone follows the same agile-like process, but the point is that there are many opportunities for testers to get involved way before PR is submitted.

1 Like
19

PR’s are gold mines for Learning and finding bugs, plus just improving code. Great source for planning what to test. Looking for code that doesn’t make sense.

I’ve been carefully using Chatgpt for reviews as well. Ask it to look for problems with the code , suggest changes/improvements after sanitising the code. It has been pretty useful

3 Likes
20

I have always made a point to teach my teams git and to pull code to test branches themselves. I would get push back from Dev leads or some software managers. Until it saved them from pushing out extremely buggy code and had to admit the devs appreciated the wingman. It always helped the QA/QE evolve and the developer appreciate and respect our department much more for just getting into the process. In bigger companies it can be a hassle due to folks marking their territory but it should be the norm.

3 Likes