I am now part of a site with multiple online games. all browser-based. Here are some test heuristics we have so far and still thinking of other areas to look into. We use these as our acceptance criterias too:
Play games on the site
Balance is charged correctly after each bet according to outcome
Bets can be viewed once completed
Bets appear in the history
Games can be searched via search bar
No in-game abnormal errors/behaviour
usability checks like buttons and links are not overlapping with the game itself
Testers should learn or develop a generic approach to testing. When you have done that, you should be able to test anything, although there will be times that domain knowledge is important.
I used to teach a four-day training course on the topic, so I can only mention a few things in this post. I strongly recommend Bach and Bolton’s Rapid Software Testing course. There’s not much other good training out there, and of course you don’t want to go anywhere near ISTQB.
Good testing is an investigation
You can’t know all the questions when you start. As you learn more about the application you will be able to think of more tests you can do. My top-level considerations are:
Context – time, resources, risks, constraints, objectives etc. This should drive everything you do.
Inventory – what is there to test? This includes functional areas, but might also include things like network traffic and local storage.
Oracles – how do we know if it’s right? The functional specification is just one of many sources including “reasonable user expectation”.
What should we test, and when – develop a lightweight and flexible risk-based test plan.
How to test – exploratory testing, automation (subject to ROI), scripted testing (where necessary, perhaps for regulatory reasons)? Would any custom-built tools be useful?
Reporting – minimal but sufficient.
Time and resource management – ad-hoc, formal or session-based?
What can you do to it?
For the testing itself, I think in terms of what can I do to it, what is it doing (at a deep level), how might it not do that, what won’t it let me do, how might I bypass those constraints etc. This approach allows you to verify all the documented requirements, and also to find bugs that a requirements-based approach cannot find.
My experience of testing casino games
I had a lot of fun testing Java (not JavaScript) and Flash-based casino games in the past and was almost always able to break the bank by bypassing the game’s constraints. This involved numerous techniques including network traffic inspection, decompilation of the embedded binary files, finding brief “windows of opportunity” to perform certain actions, running the games a slow as possible to make investigation and circumvention easier, recording games to inspect closely later, and using keyboard navigation to bypass constraints that restricted mouse operation. I found a way to make my balance go negative (so the casino was funding my losses) and even reported rendering errors such as shadows pointing in the wrong direction.
My favourite was a roulette game where I was able to determine the result as soon as the game started (most game results are predetermined) and there was a one-second window of opportunity when I could cancel my bet, so I would never lose. Then I used keyboard navigation to bypass the supposedly modal dialog that enforced a maximum stake of ÂŁ10. Together, this meant I could bet all my money every time with no risk of losing.
So about casino games, according to the country/state they are hosted in the casino can tinker the succesrate of for example slot machines to a certain amount.
Example: If you put in 1 euro every time you try the slot machine. On the 100th of time you’ll get 80 euros back.
Meaning the casino made 20 euro profit. This is a measurement you can also test, probably one of the harder ones and preferred to test with automated tests.
This is why you see people in casino’s lurking at other people playing the slot machines and when somebody leaves with over 60-80 attempts of no luck, they’ll hop in. Because it’s about to happen that you’ll win.
Not sure if that counts towards online casino’s but I’m guessing it’s the same effect on how to make profit. Very important test case for the company
If there are location restrictions (i.e. specific states only), you could check that those are implemented correctly.
If there are time limits, you could check that things like timezone hopping or overriding the local date/time doesn’t allow you to bi-pass the limits.
Like @kristof suggested, make sure the statistical probabilities are correct (big win rate, small win rate, etc) - and you’ll probably need to do a lot of reps via automation and use some standard deviations to measure that.
You could also check for pot splitting (if that’s an option). If 3 people split $1, everyone get’s .33, but who get’s the extra 0.01 (and if it’s implemented with floats, you might magically get an extra cent after doing 3 splits because of the rounding/display code)
Check for keyboard shortcuts left over from the devs that could jump people into specific scenarios.
idk, that’s just off the top of my head for now.
I would actually be looking more at the business/product goal every so often, things like
“cannot play won’t pay”. Ability to login and recover your account, and user experience when it comes to topping up your account would be far more important to me as tester than whether the game actually works as a game or not. That’s the thing that pays my salary, so that’s what I would be testing in as many scenarios, browsers, networks, account differences like currencies and all the various payment methods as possible around. Payment methods would be the biggie.
Testing that the games are actually playable or randomly allow people to win every so often, in order to engage audience, is a side hustle that is the stats people’s job to cover. And all just a distraction that the stakeholders do not really care about, and neither should the QA. @mtest summarises it really well :
A few things I want to add in response to the main questions
Player account management
payment processing
each game is a test area - the main one is whether your RNG is truly random (or close enough)
the game platform on which the game is hosted - how do you account for concurrent games, ensure no multiple login, etc
compliance testing - if you want the site to be hosted in a legal jurisdiction, you need to get compliance tested. this would include things like responsible gambling
Yeah, I was thinking you could go a level deeper on the win rate. I know games often have a sort of sine-wave type of hype built in to try and keep you hooked. You could use some stats to measure that as well.
