What makes a good benefits analyst?

Some years ago we published the results of a survey on what were the key skills and competencies of a portfolio business analyst.  Even then we struggled to find a suitable and commonly used job title for those analysts involved in supporting the development of business cases and benefits management plans.  We have come across so many names – business case analyst, benefits analyst…

Five years later and benefits management remains an aspiration rather than a reality in most organisations.   Heather van Wyk presented on her experiences in benefits management at the EMEA PMI Congress in Berlin.  She relates how, when she quizzed the audience, only a handful of participants felt that businesses were successful in implementing effective benefits processes.

I was part of a ‘conversation’ with Heather, a specialist in benefits analysis, and with the PMO manager and a senior business analyst from a major South African Retail group.  The topic of this conversation was benefits realisation, and in particular:

  • What the barriers are to implementing benefits realisation
  • The stakeholders that matter when considering the delivery of outcomes
  • The characteristics of a good benefits analyst

Here are just some of the insights.

On the challenges of using benefits as an input to project justification

There was enthusiastic agreement with the suggestion made by Heather that a good benefits categorisation scheme is necessary when comparing projects for inclusion in a project portfolio. There needs to be some way to rank projects that claim to deliver to such widely different outcomes as making a company easier to do business with or shaving two minutes off every internal ordering transaction.

There was also a sharing of experiences about the game-playing and over-claiming of benefits that seem to be the natural result of using benefit schemes to justify a project.  One conversationalist wondered out loud whether the use of benefits was a hindrance rather an aid to selecting a project.

Why, another wondered, was it necessary or even useful to use the same way of valuing a project when considering justifying a project and when choosing between projects?

This proved a good trigger. Heather described her experiences of using a mapping technique called BIP (this maps benefits to impacts to products).

BIP model example

The focus of interest by the business sponsors of the project is really in the impacts – or as she calls them – improvement areas. She sees her main role as facilitating the elicitation of these improvement areas and aiding the business come up with performance indicators to measure the outcomes. The immediacy of the outcomes and that they are directly understandable in terms of operational changes to the managers made them much more useful in helping decide whether the project was valuable – to them.

So the conversation turned to measurement and how to value differences.

On the importance of measurement systems and KPIs

Performance indicators for changes in performance are universally agreed to be a critical part of both change and the benefits management process.  But developing good measures was often much trickier than it appeared.  Some of the group related how unanticipated and unwanted effects arose from what turned out to be inappropriate measures.

The development of ‘measurement systems’ was seen as a very important part of the benefits analyst role.  Indeed when discussing which training experiences were most relevant to the role it was Six Sigma that was regarded as more relevant than analysis courses or indeed the commonly available benefits management courses!

Six Sigma training was regarded as more relevant than analysis courses

The identification of good measures which relate to business outcomes and are cheap and easy to collect is a significant challenge.  We discussed the distinction between signpost KPIs and business KPIs (see breakout).  Signpost KPIs are easier to collect.

They include such measures as usage of the new system and how many individuals access the new functionality.  As the name implies, they don’t show that the business outcome has been achieved, but they are a sign the business is changing practices in the right direction.

Some signpost KPIs can be generally applied to portfolios of similar projects, which is always a good thing.  But, as Heather pointed out, signpost KPIs must be supported by KPIs that measure the delivery of the required outcomes.  If you don’t, you can end up an ‘A’ for effort, and an ‘E’ for achievement.

It turns out that very often there are two ‘bits’ of management needed. The first is to bring out the change in the improvement area. The second is to ensure that the improvement results, where it is claimed it should, into the benefit. We’ve all heard the one about the operation being a complete success, but the patient died!

On the importance of using the language that suits the business

It’s more important that the project’s justification is presented in business relatable terms – the changes in people, processes, and practices –than that it is read back in benefit categorisation language. For very necessary reasons benefits tend to be couched in abstract generic terms that fail to engage.

All the participants in the conversation emphasised the importance of using business language.  The business must be able to hear their voice in the benefits case.  As one jokingly commented, a good benefits analyst will always ensure the business hear their words being read back to them – even when they aren’t!

In fact, Heather told of how in her organisation the business was weary and wary of ‘change management’ vocabulary, and she has taken to using terms from business leadership. Her project sponsors found this much more acceptable and understandable.  This led to a short digression on the use of Agile and its impact on the project-sponsor link.

On links to Agile

The group wanted to share their experiences of how Agile changed the dynamic around the delivery of value – an expressed aim of the Agile framework.  The common experience was that this generally appeared to focus on the value of features and functionality, not so much on business outcomes.

The questions came in a flurry. Who carries the ‘business case memory’? Who ensures the bridge between the strategically-valuable outcomes and the products delivered was maintained? Was such a concern even relevant in an Agile environment?  Should the product owner attempt to adopt the role of the sponsor?  How could the crucial interventions by programme manager and benefits analyst be incorporated into an Agile environment?

In projects of any significance (and many Agile initiatives do not fall into this category), the business case or the project charter defines the desired project outcomes – Why we are doing this, and who wants it?  What return on investment is targeted?  These are the absolutes.  Change these, you change the project.

How to deliver the products or outcomes? How to ensure adoption?  These are much more volatile.  This is where product development life cycles such as Agile can add so much. As one of the group observed, perhaps one major intervention the sponsor (or project manager) could have is to make sure that the Agile-inspired look-back sessions focus on seeing whether the effort expended was actually connected to the business case.

Agile inspired look-back sessions should cover whether the effort expended remains connected back to the business case. But do they?

The fear the conversationalists had was this was not happening very often, if at all.

On skills and competencies

The PMO  manager believed that while her department has a good robust benefits categorisation process, its use was not driving appropriate behaviours in projects.  She is determined that this should be one of their early improvement areas. She believes that the introduction of a benefits analyst capability would be instrumental in making this happen.  Such a person would have these characteristics:

  • Individual style and gravitas. The test should be “Could I put this person in front of my CEO?” A business analyst needs presence and business empathy as well as good commercial awareness.  Such a person is likely to have had first-hand knowledge of business leadership.
  • Breadth of experience. All the individuals who had successfully filled the role had worked in multiple and varied operational areas.  They were not IT and were not perceived as such.
  • Fluent in business language. The ability to work using business vocabularies and more interested in re-validating understanding than applying deep analysis.
  • Able to create measurement systems. They need to be at least reasonably numerate, with perhaps a level of understanding of statistics and error rates needed to complete a Six Sigma course.
  • Design thinking. Though being able to dwell in the problem space is important, design and developing solutions are possibly more important. Part of this is the competence often described as critical appraisal, which has proved to be a difficult skill to pin down.
  • Resilience. The experience of failure and learning from it to build resilience and consolidate understanding of what works and when is really valuable. The role is not an easy one!
  • Curiosity. Being naturally curious with a real desire to understand the business position is essential and hard to fake.

The idea of this conversation came from the PMO manager at the retail group.  It is a great format and worked for these reasons:

  • All the participants had a desire and need to improve organisational practices in this area
  • All had experience in attempting approaches and were prepared to share
  • Everyone’s contribution was equal. There was no leader, no expert. It was an informal non-facilitated session
  • The group was small enough to ensure a genuine conversation round the table
  • Two uninterrupted hours were set aside – this is necessary for a conversation to take place

My thanks to all of the participants for taking part and for allowing me to share our insights.

If you have any ideas for future conversations contact Louise Worsley.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s