Okay, I missed the questions again. Still intending to improve…
Ravneet is starting off telling a story about developers who didn’t want to do unit testing. When she talked to them, she found out that the developers were working on a huge legacy base and were only writing unit tests based on the code alone as an oracle. They didn’t see the purpose – the tests were only showing the behaviour as was.
We do need purpose. Ravneet asks why we develop software at all. Answers are “to solve a problem” “to make people’s life easier” (my aside: sometimes to take a giant excel sheet and make something beautiful and wonderful… that produces excel … I’ve seen too much!!!)
She sums up: we’re trying to change the world for the better. And yet, a lot of the time we end up developing products and features that no one wants. She uses the well-known figure of 65% of features not being used. And she’s been told by people at conferences that that figure is wrong: it’s way too low!
For example, Google Glass. Got developed on the campus and stayed there. Juicero – another silicon valley invention (I’d never even heard of this one… I’m just not a cool kid). She’s showing a picture of how people completely bypassed the juicing machine, and just squeezed the juice bags.
Next product: Spectacles – glasses that film everything. It wasn’t needed.
So how do we avoid making products and services that no one wants? We need a culture of experimentation. We have success criteria and can evaluate the outcome of the experiment against them. If the object we’re testing doesn’t meet the success criteria, we don’t continue with the test. And we need this experimentation because software development is inherently uncertain. We make plans, have long meetings. And these plans and meetings are inconclusive. We should be out there experimenting and testing.
How can we create a culture of experimentation? There are three questions:
- Should we build it? Does it matter?
- Is it usable?
- Does it break?
Starting with should we build it? Does it matter? Ravneet is telling us about a company building machines that needed to scale. So they decided to go digital. Their plan was to automate the documentation for the engineers. The first thing they did was to go and find the technicians and ask if they’d need or use that documentation. They asked questions about what was hard, where they needed help. It turned out that the problem wasn’t the documentation. They already just googled the information. So the idea was killed. On the other hand, through the conversations, they found out that getting a replacement part was a difficult process. And another idea was born.
It’s a good idea to have a hypothesis when you go out to talk to people. A suggestion. Then people can take a stance on it, and will be able to tell you alternatives or you can look for another option completely.
Ravneet is now telling the story of a yellow Walkman. Sony wanted to launch it and see if users liked it or not. They got a focus group. They asked people on the streets what they thought. People were pretty excited. Based on that feedback, they let people take either a black or a yellow Walkman. Everyone took the black one, even though they’d praised the yellow one.
Why is this? People don’t like to make you feel bad. They’re unlikely to give you bad news, and once there is a tendency towards one opinion, other people will follow that (especially if it’s socially “preferred”). Also, (and I’m gonna add my own caps on this, cos it’s so damn true): HUMANS ARE TERRIBLE AT SELF REPORTING ON THEIR BEHAVIOUR.
Ahem. Anyway. We’re onto the topic of minimal viable product. Yes, it’s a point that gets talked about a lot. Ravneet reminds us that MVP isn’t version 1.0 of your product. It doesn’t have to contain a single line of code. It could be paper. It could be an idea.
There have been some very small MVPs. Facebooks first website was very small. Another guy who wanted to build an online shoe shop didn’t have any shoes. So he took pictures of shoes, put them on the website. When someone ordered the shoes, he went and bought that pair and sent them.
Dropbox’s MVP was a video. Based on that, people sent emails saying they wanted it. I wondered for a while how this is different to people saying they like a yellow Walkman, but I think it’s because the email as an action is less like the saying you like it and more like choosing the yellow Walkman. That’s just my theory though.
We’re looking at the iPhone now. The first model had very few features that we now think of as absolutely necessary. It had what it needed to prove the product in the market.
An MVP lets us answer the hypothesis: if we do <something> for <a certain group of people> they will <respond in a certain way>.
On to the next question. We’ve discovered that there is a need for the product or feature. How can we make it usable? Ravneet mentions options like parallel prototyping, A/B testing and multivariate testing. Electronic Arts (who make Sim City) wanted to launch a new version to create more revenue. They had one version with a banner offering a discount, but didn’t find any sales increase. Another version of the web page, which didn’t have the banner, actually increased sales. That sounds contradictory – after all, giving a discount should give you more customers. However, in this case, the clients were more interested in buying the games without a discount.
The next question is “does it break”. I have started writing the software and need to make sure it always works. So we need a continuous delivery pipeline. System testing is an important part of this – and if we’ve done the work on “does it matter” and “is it usable”, then our system tests will be better.
Ravneet is introducing the lean startup model: build – learn – measure (it’s a circle, and maybe we should learn first).
To help us with the experimentation culture, Ravneet is showing us an experiment template:
- What assumption will this test and how?
- What are the pivotal metrics?
- … and so on (we’re running out of time, so the slide didn’t stay up so long)
To finish up, we’re talking about why it’s difficult to establish a culture of experimentation. Fear can be an aspect. Ravneet says that there are two other aspects: we don’t like killing our ideas. Also, we need to understand that knowledge is a valid output to an experiment. Even if the experiment itself disproves our hypothesis: that is valuable!
Factors that support culture of experimentation are:
- Cross functional teams
- Focus
- Not having long plans
- Understanding from management
Agile helps us with this, as does actually getting out of the office to talk to people (users), and also adding analytics to the product so we have a way of knowing what is going on.
We know that we have a culture of experimentation when:
- We talk more about outcomes than about features
- We observe more instead of having long meetings
- We automate repetitive tasks
As a final slide, Ravneet talks about experimentation in large corporations. Obviously, there are places we can’t fail (like, heart transplants for example). But sometimes there can be a way around some of the problems. Sony brought out a watch as a kickstarter project that wasn’t associated with Sony at all. That makes a safe space without bias to experiment.