What are the advantages and potential drawbacks of using open-source testing frameworks?

🙋 Questions
, ,
1

During TestBash Autumn we invited three fantastic community members and industry experts to take part in the discussion: The “Whens” and “Whys” of Automation. And they were: @mahatheed , Diego Molina and @lborodajko !

One of the questions we didn’t get to was:

What are the advantages and potential drawbacks of using open-source testing frameworks?

And you? What do you think?

4 Likes
2

The biggest drawback (with open source in general) is the lack of dedicated support, you can raise issues on GitHub and ask on forums, but that usually takes a lot more time.

Some of the advantages would be greater control, flexibility, customization, and no vendor lock-in.

7 Likes
3

Second the point about issues taking longer to resolve than commercial alternatives. There’s also a danger that open source projects just lose momentum and die
 they may still be useful but nobody is maintaining the project and community dries up. Arguably this is a bigger problem on commercial/closed source apps though, because if those get pulled (looking at you Google) then nobody can continue using it because the owners have complete control.

5 Likes
4

Jonathan Blow had a good talk on his stream about his gripes with open source software, you can find the clip here:

He had four main concerns -
Lack of Creativity:

  • Open source software rarely exhibits creativity. He suggests that instead of innovating, open source projects often copy existing solutions in a subpar manner.

Pull Requests and Class Structure:

  • Finds the concept of pull requests offensive. He argues that there is an inherent class structure in open source, where those who submit pull requests are considered lower class.

Quality and Friction:

  • He suggests an alternative approach where only trusted individuals are allowed to contribute, leading to higher quality and less friction. He believes that for complex projects like LLVM, contributions from those who barely know the code may not be worth the effort.

Technical Judgment:

  • He emphasizes the importance of maintaining a culture of technical judgment in open source. He argues that it matters whether code contributions are good, mediocre, or bad, and losing this culture would lead to the decline of software quality.

He had some other arguments against more-so the culture of open-source projects, but they’re more difficult to concisely state.

2 Likes
5

Ooof, I must say I totally disagree with the person in the video!

I must admit I was able to watch just 2 minutes but it was enough for me. I’d want to refute every single of his statements but in the end that’s just my personal opinion (and his, but he posted it in a video for all the word to see :smiley: ).

Just two things:

Open source software rarely exhibits creativity. He suggests that instead of innovating, open source projects often copy existing solutions in a subpar manner.

I think that’s simply not true. He doesn’t offer any proof of this, it’s hearsay.

He suggests an alternative approach where only trusted individuals are allowed to contribute

Not just that, he claims that “if you submit a PR, you’re kind of lower class”
 OMG what kind of bull is that?! I don’t even know how to refute this, it insults my intelligence.

3 Likes
6

Haha, he’s a very controversial figure, he has a lot of other insanely hot takes.

I don’t really agree either, but it’s cool to see someone who is so counter-culture to current standards to give you another perspective and challenge your beliefs, especially from someone who is so skilled.

2 Likes
7

I don’t think I can generalize enough to push my opinion in a certain direction.
Anything can happen with both paid and free tools used in testing.
I recommend learning when tools can be helpful and what they can do for you, and constantly be on the lookout and trying things.
Adapt quickly!

1 Like
8

Kate Paul (@katepaulk) once shared:

Open-source is your friend.

You can often use free or open-source applications instead of their paid equivalents. If you have the permissions to manage your machine, you can install freeware or open-source software yourself.

In the context of being the first tester on a team/in a company:

Another common technical challenge is a scarcity of resources. As the first tester, you might be working in a smaller organization, which usually means a more limited budget than you might find in larger companies with full testing teams. In addition, it may not have occurred to anyone in the organization that there would be a need for a separate testing budget.

Read the full article: Challenges With Software.

1 Like