Crowdville’s recent post to The Club got me thinking: Does crowd testing add real value, or is it just a fad?
You see, signing up for a few crowd testing platforms to keep my testing teeth sharp on unsuspecting apps in exchange for beer money really appeals, but only if it’s going to lead to better products.
I remember at my old job we had a category (a label I think it was) of defects, in Jira, which were reported by a crowd-testing platform called Applause and to be honest 95% of the reported bugs were trivial things which got discarded.
Crowd testing might be nice for someone just starting out to get some hands-on experience, or maybe even for someone who’s been in testing longer to break the monotony of their regular job. From what I know, nobody gets rich from doing it, it’s like you mentioned beer money.
I signed up for Crowdville’s campaign but I later noticed it’s for people from UK only, and I’m an Adidas-wearing squatting-Slav from Bosnia. 
As someone trying break in to testing I gave this a go. I tried a couple of companies. One had a good training program which was really helpful, but took such a long time. Others just let you crack on - if you get assigned to any projects. I didn’t really get many opportunities come my way in the few months I was actively trying. One I did get to do had unclear and out of date test cases, I found the support quite vague and I found no bugs. I’ve no idea if there were none or I just missed them. This company paid irrespective of finding bugs but others don’t.
So from my perspective it wasn’t really worth it. Maybe for people with experience it would make good beer money but for me it wasn’t sufficient reward for the time invested so I focussed my energies elsewhere.
The more I think about it, the more I think “cold” testing like this could actually be useful.
I’m thinking in terms of the persona of a completely new user who has just downloaded an app and doesn’t know or care about user stories or test charters or any of that. They just want to do a bunch of things, which aren’t necessarily covered by the path the dev team thought people would take. Call it not-reading-the-instructions testing. It’s how I break things by mistake all the time 
It may highlight apps that have been designed and built in a bubble rather than how someone flicking through the App Store will try to use it. If the user tries to do B then A instead of A then B, what happens? Does the app save them from themselves and help them achieve their goals regardless, or does it perhaps fall into an inconsistent state from which it cannot recover?
Dunno really. I think there’s something in it. I just don’t know what it is