It feels like I’m breaking every service out there

Testing Thoughts
#1

Since the past weeks I needed to reach out to major online services for issues I’m experiencing using their platform. If it was one service, I could say it was bad luck on my part, two seems a coincidence but after the third one I started to suspect something else is going on.

Side note: all these services have changed their platform in the past 3 months.

Even though my testing hat is always on, I used their services in their “normal” way, but in doing so I ended up with a broken experience where I needed to reach out to tech support to help me out. Because of my technological background I provided as much detail as I could offer: system specifications, proxy logs, cookie values, etc.

Tech support acknowledge that I hit an edge case and that the engineering team needed to investigate further. So here I am doubting myself if I’m a natural born talent for breaking systems or that these major services have lowered their quality work?

Have you had a similar experience or are you also noticing a decline in quality with services you use?

1 Like
#2

I certainly challenge poor systems design or obvious bugs when I see them - and I see them fairly regularly. When the issues become such that it renders the service a fail for me, then I write - sometimes directly to the CEO if the company is big enough.

Quite often a consumer end user will not have access to tech support, just to “customer services”, who are not the best people to talk to in these circumstance - hence the e-mail to the CEO (a little research is needed to get the correct contact details!).

I point out that I have a professional interest, I then provide a bug report-styled narrative of my experience, and sometimes I even end by saying that if I were submitting my invoice for that report, it would have cost them £x! The worst example I saw was a customer self-service touchscreen terminal in a supermarket with a counter-intuitive workflow (including a flashing cursor in one of two text input fields, clearly marked, but where to move from one field to the next, you actually had to use a large green “Next” arrow at the bottom right of the screen which by its size and placement implied that it was only there to move to the next page…)

In this case, it was clear that the code had been tested via an automated process, but at no point had an actual human user ever been let near the application. I would like to think that the CEO had a conversation with their Head of IT systems development over test polices and strategies…

1 Like
#3

Robert, that seems like a good plan. But I’m talking about major social media companies (the really big ones) and I guess their CEO would not be impressed by my e-mail. Even though I would not go this route for my current situation, I would definitely take this lesson to heart when dealing with more normal sized companies.

Yes, customer kiosks are often developed and tested from a business point of view but barely tested with real humans, preferably the clients you have build the kiosk for in the first place. I have the example where you could click on a product photo to have it full screen displayed… but there was no way to actually go back to the smaller images (only touch screen display, no mouse, keyboard or swiping possible). So each time a user touched a product to look it into more detail, tech support had to reset the application to display all products again. It does keep people working though…

#4

You may be onto something when you note that all the services changed their platforms recently. Regardless of whether the problems were caused by the platform changes directly, it’s unlikely the releases around those times were “business as usual.” Perhaps there was not enough planning and testing for unknowns that could have been anticipated. Or maybe there was just more work, and quality suffered.

1 Like
#5

I thought about the same thing @hattori, but since these are the example businesses where quality and security are always advocated to be “part of their DNA” it would be really bad if they have lost that DNA. Is this a worrying sign for things to come?

#6

Quick update on the escalated issues I reported:

One of these services has come back to me asking me to log out of my account in the desktop browser and remove all my session cookies.

I also need to remove their mobile app from my phone and do a full re-install.

If I’m hitting an issue that’s impacting not just me, but many more users I can see this becoming a support nightmare, especially for users that are less technical savvy.

1 Like
#7

I hate that. Uninstall / reinstall app to make the problem go away. Its irritating enough to get that response with free apps, but with paid services I think its because often there is not enough investment in test. Not on the same level as yours I think, but I keep running into bugs with a heavily advertised language learning app.

1 Like
#8

I feel your frustration Simon and I think this is where @robertday was hinting on where he sends a professional test review with a (fictive) invoice to indicate that testing has a cost factor, but is a necessary spending to prevent much more loss because of reputation damage or customer distrust in your service. Especially for paid services I would definitely go with the approach of Robert.

Your statement “there is not enough investment in test” is a battle I’ve been fighting since 2004 and I have to say it has gotten way better since then but we’re still far from our destination. The more we as an industry reveal these untested issues and emphasise on having proper testing in place we can achieve our goals. But that’s a big maybe.

2 Likes
#9

This reminds me of a leading manufacturer of commercial jetliners with a tagline “Build Something Better.” This well-regarded company fit ten years of work into about seven, to catch up with their competitor. News reports say the new model is highly dependent on flight control software, and their competitor has more experience there. A company’s culture depends more on how it actually does business than on its size, or the reputation it tries to project.

1 Like