Project Management in Practice

Beyond the alphabet soup of PRINCE2, MSP, MoP, PMBOK, ITIL, Agile

Category Archives: Estimating

Building PMO metrics


The Christmas and New Year is always a good time to take some time off the hurly burly of daily grind and reflect on how things are going. Towards the end of the year I did some work on what metrics would help us run our PMO more efficiently. Metrics are always difficult to establish, especially as they only tell a story once you have a baseline to measure against. This is probably a heavy topic for the first post of the year. Apologies for that.

Building-PMO-Metrics

While I see a pressing need for making decisions on evidence, I am also cautious against spurious interpretations of metrics, which can easily happen if taken out of context. You only have to look at statistics driven sports such as baseball or cricket where fans and officials will take diametrically opposing views of players or tactics using different statistics. Numbers are just that. What you interpret from them is what gives them meaning.

The first task was to explore what type of metrics would be useful for our business. I work for a IT professional services firm. It has unique challenges from other types of businesses. I did some research on what other similar organisations are doing. I found this compilation from OpenAir and excellent resource. There are three articles in this and the first one by Thomas Loh is by far the best. This was an excellent start. The key is not to go chasing every metric under the sun, but the ones that you need to measure. That is even more crucial when your PMO is lean and you are in the process of building its maturity. Capturing metrics and analysing them takes effort and time. You cannot afford to be spending either frivolously.

The standard metrics of utilisation, profitability, billing rate etc are quite easy to measure after the effect. We were looking at getting at least one forward looking metric that can help validate our decision making. We decided to invest in our effort in an area that is most challenging for a services business like ours – that is the pull between resource and demand.

Resource-vs-DemandIn services business you either have too much work or too many people. It is crucial to have a good handle of this to maximise profitability. The cycle of winning new business always takes time. If you have left your efforts to bring in new work too late, you will inevitably have periods of low revenue. Unlike products which you can sell at a later time and recoup some revenue, if not all, lost consulting time cannot be archived and sold. That is effectively lost.

To ensure an optimum work pipeline, we can use the charge rate to either stick to our margins, because work is plentiful or use discounting effectively to be more competitive than usual in tough market times. We want to be making a decision on them at the correct times (i.e., not stick to higher margins when market is tough or give away margins when not necessary). We are looking at using Backlog (total value of contracts yet to be executed) as a forward measurement for that.

The aim is to look at recording the backlog value three months out and updating the actuals at the end of the month. As we currently do not have a baseline, I do not expect us to be able to use this effectively in the next year. However, once we have built a picture, we should be able to predict with some confidence what it means to be at a certain point in our backlog and what that is likely to mean in terms of likely actual income.

Because we are looking at it three months out, we’re likely to have enough time to win new business to fill up the pipeline if it is looking less than promising. If pipeline is strong, we know we do not need to compromise on margins. There is likely a follow up on this topic this time next year on how this measurement plays out. Hopefully my challenge is not unique to me and the process is helpful for others to reflect on.

I am keen to understand what predictive measurements you have successfully implemented.

Image credit: communicationstuff.com

Enhanced by Zemanta

Is ‘No Estimates’ for real?


No Estimates is a concept that I had not come across until very recently. The basic premise of No Estimates is there is significant inaccuracy in software development estimates. Even when you spend a lot of time estimating, it is still not necessarily accurate enough to give you an exact picture. Therefore, the time spent estimating does not give you a reward for that effort. Proponents of No Estimates argue you may as well spend that time and effort into understanding what the most important feature is to the user and concentrate on getting that delivered well. Once delivered, you concentrate on the next most import feature and so on.

Is No Estimation for real

When I first read about it, my initial reaction was it was like throwing the baby out with the bath water. Just because we are not as accurate at estimating simply giving it up seemed like lunacy. On reflection however, I can see it being useful for in-house teams that are supporting a business application. There the organisation has already seen value in continuous improvement and has accepted the software developers as part of the package to deliver the business outcome. You can concentrate on small measurable improvements to the application as directed by business. The users in this case are well versed with the application and are able to explain the desired improvement. Interestingly enough, in my experience this type of development is comparatively easier to estimate than others.

I also had a thought that product development teams may be able to use this in some manner. The rationale was they have quite a bit of control in terms of what they can cut out of scope if something turns out to be more difficult than initially perceived. However, the tradeoffs they make are not necessarily linear. There is usually not comparison that you can unilaterally say is feature A versus feature B. Feature A may very well be more important, only if it is likely to  be no more than 20% more in cost. No Estimates as a concept is not well geared to help in this scenario.

Even in scenarios where No Estimate is a reasonable course, I see significant risk of what I call ‘design blinkers’ – only taking into account the immediate requirement(s), rather than an overall view to limit rework. Design blinkers happen in all form of software development. This is because you can only design for your known scope. By limiting your thinking horizon to the next most important feature, you are in effect likely to encourage less holistic design.

From the reading I’ve done so far, it appears there is a level of misunderstanding in what estimates are used for in projects. There are no perfect estimates – projects are by nature risky. You can only ever be expected to deliver estimates with complete accuracy early on. They should get progressively accurate as you go through the various phases of the project. Initial estimates to judge whether the client has the cost appetiate to even embark on the journey, then further refining to understand whether your organisation can execute the project safely with a level of profit. These estimates will be less accurate than your feature level estimates, that you will be building up when the project goes into execution. In most cases you use multiple methods of estimating than simply a combination of all the features – previous track record, comparative scale with other similar projects and some level of feature based estimation.

In my view there will be very few situations where you can get away with doing no estimates and be able to make the decisions you or your customer needs to make. The answer is not to avoid estimating, but estimate at the level of accuracy that is required to make the decisions. Never estimate $X or hours Y. Always clearly state estimates with their level of accuracy i.e., $A to B or hours Y to Z.

Image Credit: Dilbert (Scott Adams)

How do I construct the perfect project estimate?


I went to the New Zealand PRINCE2 User Group Wellington Chapter meeting on Tuesday night. As usual the calibre of speakers were excellent. Conrad McDonnell and Barry Calvert spoke on various aspects of estimation. It was quite interesting perspective, one from an IT/vendor angle and another a construction angle.

Importance of a shared vision is not something that I had previously connected to a good estimate. Conrad was quite persuasive on his argument on the need to understand the vision as a pre-requisite in this respect. It is amazing how many projects I have worked on that has started with not all stakeholders having clarity on what what success looks like. Here I do not mean success delivering to a set of agreed deliverables, but something that would advance a measurable business outcome. I can see his point of view. He gave the example of the NASA programme to land on the moon and the vision that John F. Kennedy outlined in his speech.

Perfect Project Estimate

He talked about the vision in the context of a progressive estimate with increasing level of certainty. The key to any good vision is to not assume it is understood by everyone. It must be communicated with a feedback loop. Before the programme or project begins, high level planning must occur. The key to estimation at this stage is to understand your levels of uncertainty and recording your risks. It is key to remember there will always be different levels of certainty at all stages in your project or programme. In an IT context, you may have near certainty about cost of hardware, but little clarity on required design effort. As you execute, you must continually plan, execute and re-plan. Expect to take a few detours on your journey. This is where clarity of vision will help.

Barry talked from a construction point of view and had an analogy of a pancake. It only has four ingredients – flour, eggs, baking powder and milk. Flour is the one that determines the size – what is required, by whom, when – the basic facts. That sets your base criteria for estimate. Eggs are what sets your environmental conditions and is complementary to your flour – what environment is it being delivered, the intended use, slope of the land, other influencers. Baking powder is what gives it the fluff – the nice to haves – types of finishing, flooring, fittings etc. Milk is what gives it the consistency – this is your repeatable methodology and constant refinement of it. Even though he gave the example from a different industry, I like the principle of it. I have a feeling I will re-use it.

One thing that was clear from both speakers was the folly of trying to go back to the customer with a number too early in the piece. Whatever you do, customers forever remember the first number or date.

There is nothing called a perfect estimate. You can only ever get reasonably good at it.

How do I estimate based on risk?


Risk Management - Image from bigthink

Risk Management – Image from bigthink

I have previously contemplated the challenges of keeping to scope and budget and keeping customers happy. If a customer adds new requirements on to a project, it is an easy discussion to have around additional time and cost, or a reduction in scope elsewhere. Grey areas in scope aside, you can show a definite impact on the project because of an omission that needs to be rectified. Where I see challenges are in trying to prevent slow bleeds.

One approach is to state assumptions that all dependencies will be met and manage exceptions by change requests. From experience I find that is a hard way to manage. How long does it take to do a task? The answer is always depends. You may be doing a task very similar to that you have done elsewhere. However, you cannot assume it will take the same amount of effort or elapsed time. The environment where you are deploying, number of other suppliers involved and in-house expertise of your customer all play significant roles. Experience tells us that we must take into account these factors as risks when we estimate project cost and duration.

My area of expertise is in delivery of IT projects. However, I believe the principle is the same in all fields of work. In an IT context, even in simplest projects you will have a customer taking on the task of solving a business problem through a project. The supplier will be engaged to deliver the project to a scope. Usually interactions are required with subject matter experts in the business, the IT service provider for the customer (could be in-house or another supplier), possibly procurement … to note a few. There are risks that we must associate with each of the players and estimate the project accordingly. While it is difficult to create a good risk profile for a new customer, we should be able to build a reasonable picture for customers we deal with regularly.

Think of a scenario where one of the other suppliers delay delivering a component that is a pre-requisite for your delivery. It may only have been delivered a day or two late. However, you have created an optimum resourcing based on something being delivered on the agreed date. In reality, you cannot re-assign those resources unless you have significant notice of the delay. Even if you know well ahead of time, in many occasions it is impractical to re-allocate those resources to another project, due to the short available window. By the time they are up to speed with the other project, it will be time for them to start working on the original assignment. Effectively the supplier through no fault of their own ends up absorbing the cost.

Sometimes the lost time is less than days, maybe a few hours every week due to lax turnarounds to required information or pre-requisite delivery. However, at the end of a project it does end up being a significant impact. You cannot really go back with a change request every time something is delayed by an hour. This will give a new meaning to death by change request. And you can be guaranteed a breakdown in communication with your customer. How so do we then ensure projects do not suffer this fate?

PRINCE2 Risk Probability Impact Grid

PRINCE2 Risk Probability Impact Grid

The only way around this dilemma is to expect some level of these risks materialising. PRINCE2 suggests using a probability impact grid to evaluate risks. While PRINCE2 is predominantly useful for the end customer to run the project, there is no reason the suppliers cannot use similar techniques to analyse risk. It is worth analysing what your particular scale for impact or probability is to get a good feel for impact. When you are constructing your estimates and plans, have these built in.

The key message here is to initiate a change request only when the risk that you have tolerance for in your plan is exceeded. Many IT projects fail to take this into account and bleed. One can legitimately ask why doing a PERT estimate should not take these risks into account for the pessimistic scenario. The key to remember on that is when you get your technical staff to estimate the pessimistic scenario, they are taking into account pessimism from a technical point of view, whether that be an unknown pattern of software development, familiarising with new API etc. They are not taking into account human or communication risks.

In my observation, construction projects are better at taking risk budget into account than IT ones. This is possibly because a half finished construction project is an obvious reminder of a failed project. Failure of an IT project is usually a less visible eye sore. That has probably made those in IT projects lazier than we should be.

How much do you think it’ll cost?


How many times have you heard that? “Oh, I’m not going to hold you to this, but just a ball park figure?” Sounds even more familiar? And now the clincher, “You know our systems, surely you have an idea.” Trying to answer this sort of question from the client too soon causes enormous headaches in projects.

Cost is something that is never a black and white answer. Cost of the same project will differ markedly depending on what technology and process choices you make, what functional and non-functional standards you sign up to, how much of the subject matter expertise your clients are willing to provide to the project team and even some mundane things like what time of year you want the project to be implemented. A cost estimate therefore must accompany the scope, assumptions and conditions that it is based upon.

Many times you will have technical staff working with clients that will be put on the spot by these kinds of questions. They will have the best interest of the client and their own organisations by trying to answer the question. Unfortunately, this sets the expectation of cost from the client’s point of view. Some of the technical staff may be very skilled in their own responsibilities, but may lack the full project life cycle knowledge. Therefore the risk of underestimating or completely missing required activities is high.

In all likelihood the client has not had the time to frame what it is that they want. If they have a figure to go by, they will form the scope in due course and wrap that around the figure. This happens inevitably, even if there is significant gap between any cost figures you may have provided and when they had the time to form scope. There may not be any malice involved at all. This is how we as human beings operate – piece together bits of information to build the picture.

If you try to reset the expectations of the client at this point, it is tricky. There is always a chance to give the impression that now that they are interested, you are trying to squeeze them. Chance to set the correct expectation has long gone. By trying to be helpful, you have a situation on your hands.

You may come up with the same situation with your sales staff as well. They may be the ones that are responsible for communicating costs to your customers. Your challenges will be equally as pronounced if they miss the caveats. There is a fine balance between the cost appetite a customer has and the benefits they wish to gain from a project. In the quest of landing a sale you should not set a project up for failure. If the project is a keenly desired one for which your organisation is happy to make a loss on, then your management must be in a position to knowingly take the risk on. This should not come as a surprise during the project.

How can you go about ensuring the correct expectations at the outset? The first thing I make a policy on is to never provide estimates on the spot. I have only seen bad outcomes in trying to do that. Have standard estimating methodologies that you follow. The methodology may be different depending on the type of project you are managing – i.e. software development, construction, policy development etc. But there must be a methodology. Document the risks, assumptions and constraints that accompany the particular estimate. This way once the customer has had the time to think of the whole solution and the cost is different than the first one you provided, you can have an open and honest discussion on why that has transpired. This must extend to the whole team. Train them to tell the client that they will take the information back and follow up with an estimate.

First impressions do last long with customers – especially when costs and timelines are involved.

%d bloggers like this: