Estimating effort on a story


(Heather) #1

A discussion has just come up today on the testers.chat slack channel about using planning poker to estimate for stories.

It started with “In planning poker, do we estimate our own work or everybody else’s work?”

I’ve written about this with @melthetester in our games in testing article series. I personally feel that there should be a shared understanding of the task to be undertaken so that there is an idea across the board of the amount of dev & test effort that will go in to a story.

I think varying estimates open the floor to discussion about the task to be performed and offer the opportunity for everyone to have an understanding of each others work.

What do you think? How do you estimate effort on stories? Is it purely dev effort? Do you include deployment of the feature to the live environment in the estimate? Do you include testing?

Update: why do you estimate? As highlighted by @wojciech, we may all estimate for different reasons.


How do you use story points?
Providing Value to Agile Ceremonies as a Tester - Masterclass Further Discussion
(Josh) #2

To begin the case for estimating the whole team effort, I’d say that whilst it has a steeper curve at the start of a project, or when a team is founded, it offers numerous benefits, e.g:

  • Absence of some team members can be filled in by other team members
  • Strengths and weaknesses can be observed in individuals, encouraging knowledge sharing and personal development
  • Estimates will be made given consideration of the perspective of all team members, not just one. If the test effort thinks a task is relatively large because of how tedious the work involved is, it may be that they don’t realise they actually complete the work considerably quicker than it feels like they will.
  • If one persons estimate is considerably higher or lower, there is a great opportunity for that person to get involved and learn more about it, if they do see opportunities to make the process more efficient they may well benefit the whole team and again, provide knowledge sharing sessions to get everyone up to scratch.

The only real benefit I could see to estimating only your own effort on a task is that the initial estimate is liable to be more accurate. In isolation like this, though, wrong estimates are liable to be considerably less accurate. Where the test effort thinks that a task is small, they may not realise the full implications of the change.


(Wojciech) #3

Why do you estimate?

I think it is hard to answer estimation questions without more context. There are many reasons teams do estimations.

When we know what are the problems you are addressing with estimations, it should be easier to suggest solutions to improve your process.


(Heather) #4

As I wasn’t the person who originally asked the question, all I can provide are my own reasons for estimation.

When we estimated, we were trying to see what we thought we might reasonably achieve in a sprint based on the people we had and the time they had available. At the end of a sprint we would retrospect on our estimations to see what we had over or underestimated, why we thought we might have done that, what we might do better the next time.

We never used estimates as a hard delivery date. I have heard stories of many teams being beaten with the estimates stick.

Perhaps we should say that others responding could add their own reasons for estimation. I will update the original post to add this.


(Paul) #5

What I like to do if I get the chance is to take some stories of different complexities from the set, execute and time tests on them, multiply by say 1.3 to include some sort of buffer and then extrapolate this to the whole test set - noting that if I find a significant number of defects then the time will increase. It may not be the best or most elegant approach but in terms of deriving a reasonable time estimate for functional testing it is a good place to start from.


(Wojciech) #6

That makes sense. But why did you need to see what can be achieved in a sprint? For example, what would happen if you did not know that and just worked on the most important tasks instead?

I am still trying to understand why you estimate. What problem are you solving with estimations.


(Heather) #7

When it gets to the stage of regular releases you need to be able to estimate what you can realistically achieve in a sprint.

It helps also to communicate roughly to management (when they feel features should be added) how long a task may take to achieve. This can change the priorities of tasks as the realisation that task X could not be reasonably achieved within a sprint may change the perceived priority. I have seen this happen.

I take it that you don’t use estimations yourself?


(Wojciech) #8

Why? (We need 3 more why’s to get to 5, so stay with me!)

There are many good reasons for it, I just want to understand yours before I can suggest any improvement to the process.

Cool, so we have the first reason for estimating, I understand your reason number one for estimating stories is to be able to facilitate better communication with management in order to change priority based on task size.

How often does that happen?

Oh, I do, but that depends on the team and project. As I said, there are many problems that can be effectively solved by estimates. Let me share two stories along with problems we solved.

Problem: What is the rough cost of a project?
Story: An example from a client recently, we spent 1 week estimating a complex project, and afterwards it got canceled because the high level estimate of 6 months was too much. The estimation process was a great tool to inform the project management team of the rough costs.

Problem: Reduce the risk of not delivering on time, resulting in millions of pounds of losses in fines.
Problem: Reduce risk of integration issues across many teams delivering a huge project.
Story: Another example from a different project. We had a hard deadline because of shareholder agreements (in 9 months time). We used estimations half way through the process to make the decision we need to share the work with another team because otherwise we would not be able to deliver on time. We estimated from the very beginning, but only half way through the project we realized we are far too behind the schedule we had to take an action. So, in essence, we used the estimation process to inform resource allocation. (P.S. We also used estimations to set milestones to create touch base points with 6 other teams to do integration testing for risk reduction) .The project was delivered on time eventually.

I have also been on many projects where there were no estimations. We did not have any problem there that could be solved by estimations.

I hope this clarifies a bit why I ask so many questions and why I need to know more about your environment before we suggest improvements :slight_smile:


(Heather) #9

But, as I’ve mentioned in the original question, I’m not the one who originally asked it.

How often did management in my company before my last one change priorities based on estimates of effort?

A: Every single sprint.

I’m no longer in a position where I need to provide estimates so it’s not me who needs improvements to my environment :smile:


(Melissa) #10

This is a great thought exercise. It can be good for a team to do estimates if only to be able to communicate about what the story is, what it needs, and what other things might be involved, such as services, Ops, additional testing (whether that is functional or non-functional). The “why” of estimates varies from team to team and business to business. Improvements come when the culture of the team changes, or the culture of the business changes, or even at the whim of the Dev Manager. Being aware of the benefits and pitfalls of any tool is always a good idea. Having ways to keep everyone fulling engaged on the task (even if it doesn’t seem useful to some) is also a good idea. That’s all @heather_reid and I were suggesting when we added it to our articles.


(Wojciech) #11

I feel like I have caused frustration on your side, not sure why. I am sorry.

Agreed. Also worth keeping in mind, there are many other more effective ways of solving that problem though, like pair programming, pair testing, story kickoffs and signoffs, etc. Depends on the severity of this problem.

Agreed. That also goes back to why you estimate, though. Why do you need this estimates to be so fine-grained? Also, if you wanna facilitate better communication inside team, why are you using estimates for that? That might be OK, but as I said there are other ways of doing it as well. So that is why we wanna separate why you estimate from everything else.

Fair enough! Well in that case probably fine grained estimated are justified.

I guess what I am trying to say is that I see to many estimations done for the wrong reasons, “cargo cult style estimations”, focusing on the practice instead of principle/value delivered, and since you have asked these questions I took the liberty to drive the conversation a bit towards that subject as well. Sorry about taking over the topic!

P.S.
I am posting here to share knowledge, but also I am bit selfish, I would like o learn as well! So please disagree with me, that is how I learn very often.


(Heather) #12

I’ve had to defend estimates and the actual estimates we provided a lot in my past so it’s always a touchy subject with me :smile:

It’s a fair point. In the original questioners question we never actually asked why they needed to estimate. We should have. We asked more about why there was discomfort estimating the total work involved. They mentioned that they wouldn’t know other team members skill sets and how long it would take someone else to do the work.

We tried heaps of ways back when I was on an estimating team to facilitate better communication. Some worked, some didn’t and we touched on this a bit in the article. Estimates were one thing that worked really well in that situation so we kept to them. I don’t want to say too much about what didn’t work for many reasons on a public forum so happy to discuss in Slack or a private message.

Estimates don’t need to be hard deadlines. I stress this all the time, when I was back there I would really emphasise that estimates were estimates and don’t hold people to delivering on those. I would get a bit snippy with senior management when they used estimates as hard deadlines, particularly with changing priorities so often.

Agreed, many estimate for the sake of ritual or to get a great big stick to crack over the whole teams back. I would always err on the side of caution with them.

Having said all of this, estimates can be a really clear, quick and easy way for senior management or (hate using this term) less technical people who have an interest in the product to get an idea of how long things will take. It’s not ideal but that is one of the common uses and I think the advantages of that use really depend on the situation and what has already been tried, as you mentioned in your questions.

To get back on track a bit, did you ever have to estimate the entire effort of a story or was it purely the developer effort and then test could take the time they needed?


(Wojciech) #13

Sorry, I can be a bit too direct sometimes. I blame it on my eastern European roots :stuck_out_tongue:

Sometimes only dev, and sometimes also both. Depending on the project. On occasions, we have also included DBAs, SAs, members of other dev teams, etc, whoever was impacted by the work and could change the estimate.

On one project I enjoyed working on a lot, we did dev+test estimations not only because both parties had valuable input, but also because we used the estimation meetings as a team building exercise. We wanted to make sure dev and test feel part of the same team in whatever they do. Everybody was expected to be equally responsible for quality of the product, so everybody was invited everywhere.

Been there as well, but was too young and inexperienced to know what to do about it back then. I am glad more and more people talk about estimations, I would hope it will help avoid those frustrations.


(Nicolas) #14

To add my 2 cents to the conversation. I have worked with teams that do not estimate and teams that do estimate. As you have mentioned in your conversations, estimations are much more than a simple letter or number. Estimations when done correctly bring a lot of good things to the team. It improves the communications, it can show the gaps in knowledge, raises awareness of the other members knowledge and work, reduces the silos in the teams, and helps bring a better sense of accomplishments to the teams. How is that? I think one of the best things about estimations is that the team can learn about themselves in terms of how much they can deliver in a sprint and in return this means that they will be (at some point) completing the sprints with all the stories / tasks in the “Done” column. This alone is very good for the team’s health.

Now, regarding what to estimate, that will depend on how to team works. There are teams that break the tasks and split the testing part of it for example or the monitoring work for after it is deployed and running in production. In any case, when estimating, you have to consider this and estimate accordingly. If there is no other story or work in the sprint that includes testing, monitoring, etc, then estimate it inside the one story you have or raise awareness that you are not considering this things. Fro example, some times “trivial” changes in the code may need complex regression testing. This can lead to a very wrong estimation. So again, another point where estimations are more useful for the conversation that it brings, that for the numbers it provides.

And as a final thought, I think it is important to understand that even if you choose to estimate or not, you have to think about communication of the team, and having the correct goal, alignment of what we are trying to achieve in the sprint, together with success metric or criteria. How does a successful sprint look like? How do we measure it? If you can measure that, and you are regularly doing it, then that is all you need! (sort of)


(Heather) #15

Yes! I think you are wording what I meant much better than I did myself :slight_smile:


(conrad) #16

As a “Agile” tester, I find the estimates are done on the dev work, then afterwards estimates on the test work - both are inaccurate, and the points (planning poker style) often underestimate. But the management side point of estimating is to let them know how long in terms of months it will take to deliver a design. But also to let us know how well we are doing in a sprint retrospective. And for all the ways points can be misused, points can be taken too seriously, because a 2 in a junior engineer is equivalent to a 0.5 in a senior, and so on kind of arguments are not useful ; so we never try to compare point values to other teams, nor use it to beat the juniors. Sprint success is about delivering as much as we thought we could or more, repeatably. And not getting blocked.

I like the way we can use it to push ourselves to perform and optimise - because the only way to know if you are continuously improving the process (a key to agile) is if you see your speed going up and up, and points are just one metric of many. I am not convinced it’s the right one to use to measure efficiency of the team, but it’s a basic way to measure your ability to navigate obstacles better each development cycle. It’s called Kaizen http://www.gettingagile.com/2008/11/22/a-kaizen-mindset/