Shaun and the EFQM Model

I first encountered the EFQM Model in 1994. The organisation I worked for had adopted it as its quality framework of choice (albeit after being more or less made to by the Cabinet Office). But we approached this new opportunity to learn with alacrity. We trained up a lot of people, mainly from 1st and 2nd tier management, and willingly set aside an inordinate amount of time and resource to the annual self assessment process (which we all agreed was quite a lot of fun). Evidence was prepared, assessed and scored. Finally the output was produced. A long list of over a hundred strengths and 200 areas for improvement. Then momentum, strangely, began to erode …

“Come on everyone! Let’s identify some key priorities  from the long list!”

“But we already know what our priorities are”

“Oh. Erm, in that case let’s look at this valuable output and see if we can identify some potential areas for process improvement”

“Do we have to?”

“Well it’d be a shame for all that time and effort to go to waste”

“OK …. well …  if you ask me it’s communications. There. Do something with communications”

“What sort of communications?”

“Just communications in general, really. It just seemed that when we did the self assessment that communications cropped up quite a lot – so let’s do something around that”

“We’re already doing lots around communications, though”

“There you are then. Now if it’s OK with you I’ve got more important things to do. I did enjoy the FEMQ wotsit though, and I’m more than happy to participate again next year”

A Familiar Story?

Is this typical?  In my experience I’d have to say that it is. The process is fraught with perils and these perils are not particularly well publicised, so new practitioners repeat the same mistakes over and over. The good news is that it can work well provided the pitfalls and limitations are anticipated. What are they? Well, here are the things that immediately come to  mind …

The process must satisfy a reasonable cost/benefit analysis

The more resources you pour into the front end of the process, the harder it is to cover that expense with an equal or greater amount of benefit. The process works on a “consensus” approach, which is good, but don’t overdo it. Self assessment generally won’t tell you a lot that you don’t already think you know. Don’t be surprised if the key AFIs turn out to be the things that bite you on the bum on a daily basis.

Skillful management of the self assessment process

Speaking of bums, “leadership” is a pain in the bum to assess. Loads of people get defensive and there’s a huge temptation to bottle out of an argument on the key points. You can spend a lot of your assessment time on this topic and get absolutely nowhere. Take my advice and assess it last when people are more focussed on wanting to get home.

Start with leadership first thing in the morning when everyone is pumped up on coffee and muffins is just asking for trouble.

It’s the economy, stupid

The model does not force you to assess the economic viability of the organisation very well. Not near as well as it asks you to look at process control. This has led to a number of high scoring award winning companies folding. Understandably this has impacted the credibility of the model.

Process control has to be assessed against a backdrop of economic sustainability, otherwise you may be assessing the paradox of “non-viable excellence” *The principle of cause and effect is critical, so processes and process results should be assessed simultaneously, otherwise links can be lost, forgotten or disguised There are a few more, too, but the message is that it takes a bit of work. The Model has been around now for about 15 years. It’s still here so there must be something in it, but it hasn’t conquered the world, so it obviously isn’t the “silver bullet” that we all thought it was going to be

Unfortunately for a lot of companies it gets played with once then gets put back in the toy box. Maybe it’s time the EFQM started to ask why?

Post from: Capable People Blog

The EFQM Enigma

Last week I published a review of ISO 9004:2009. In it I noted just how “EFQM” it was. That sparked a short exchange between Mark Harbor and I on Twitter about the merits of the EFQM self-assessment approach and the limitations of the typical ISO 9001 audit-driven approach. Something from that debate concerned me and it was a while before I could put my finger on what it was. Now I think I have

It is my firm belief that when we compare EFQM and ISO 9001 the strength of one framework is the weakness of the other and vice versa. In other words, what one framework does well, the other does badly, and the match is almost a perfect negative

In this post I’m going to try to explain exactly what I mean by that

My history with EFQM and ISO

My involvement with each model goes well beyond academia. Those of you who know me from Capable People will be aware that I’ve been training ISO 9001 lead auditors for about ten years, however prior to that, in a past life (in the 1990s) I worked extensively with the EFQM Model. I assessed on numerous occasions for the UK Excellence Award and the North East Excellence Award, trained assessors for the North East Excellence Award on a couple of occasions, and also got involved in upwards of 50 internal EFQM self-assessments for various organisations. It is from these direct experiences that I draw my conclusions

The reason I found it necessary to describe my battle scars, particularly with regard to the EFQM Model, is simply because it works so well on paper. If you’d never been through the pain of self-assessment, and suffered the frustration of post-assessment inertia, you’d never guess it had a single fault … but it does

The strengths of the EFQM approach

Frankly the EFQM approach has a few faults, but let’s start with the strengths, because the glass might just be half-full. I’ll try and list them;

• Its criteria covers strategic processes in far more detail than ISO 9001
• It does the “systems approach” better too
• Its criteria are “weighted” and identify that some processes are more critical than others (which they are)
• It does “leadership” in a more detailed and academically sound way
• It makes a more concerted effort to direct assessors to identify cause and effect relationships (sometimes in vain of course, but it tries, nonetheless)
• It includes financial/business results and some financial processes within its criteria (not simply “quality”)
• It directs assessors to examine the integrity and breadth of “results” in a better way, including an appreciation of direct and indirect measures, and the benefit of a balanced range of metrics
• It actually has criteria that support the “Involvement of People” quality principle
• EFQM self assessment is surprisingly good fun, if you like that sort of thing

The weaknesses of the EFQM approach

Although it has strengths it does have its significant weaknesses or, in EFQM language, Areas For Improvements (AFIs). These are what I consider have always been the most significant ones;

• The use of documented evidence or the requirement to provide “proof” (as opposed to testimony) within the self-assessment process is usually limited
• Although the criteria, in theory, covers strategic issues, financial measures and results, the output from assessment will only ever be as good as the inputs allow. In my experience of going through numerous assessments, there is an almost universal reluctance from the senior team to allow unfettered access to this sensitive information “warts and all”. Therefore the principle of “Garbage In – Garbage Out” (GIGO) usually applies
• Although the criteria includes financial performance, it does not do it in sufficient enough detail to allow a realistic assessment of the sustainability of the business. Assessors may well look at how budgets are allocated and managed, which is a good thing in itself, but sustainability is the $10,000 question. Consequently there have been numerous examples of award winners getting into commercial difficulties a very short time after receiving an EFQM based award. It could therefore be argued that the model awards a deceptively high score for companies that are going out of business albeit in an “excellent” way. This feature may well partially explain why it seems to have retained its popularity a little longer within the public sector in the UK. In this sector financial management more or less is management of budgets, and the issue of commercial sustainability is not really a factor in the mix
• The assessment does not identify any clear “rights” and “wrongs” – just a set of “coulds” and “could do betters”. Fair enough, you might think, but in my experience that almost always leads to strangulation of the process by inertia once the assessment is complete. Typically the assessment will yield upwards of 150 strengths and 150 AFIs, with no direction on priorities (that is for the company to decide). The problem is that this wealth of data  usually completely overwhelms the organisation and brings the process of improvement via self assessment to a sudden stop. You can have too much information
• The process, done properly, is incredibly hungry on resources and often struggles to satisfy even the briefest of cost versus benefit analysis

I must confess that between the years 1994-1999 there was no bigger disciple of EFQM than I. However, after a few years, Groundhog Day well and truly kicked in. I looked back over the fifty or so assessments that I’d been involved in and struggled to identify even a small hand full that had delivered real improvements. That is, improvements that I felt the organisation could not have identified anyway, simply by intuition. The fact was that most companies already knew fine well what their biggest problems were before the process began, and I could see in the faces of many a senior manager during the assessor feedback an expression that suggested “this is an expensive way of telling us what we already knew”. I’ve heard senior teams criticised on numerous occasions for a lack of “buy-in” or “commitment”, but sometimes you need to see things from their perspective. After a while I found myself asking, hand on heart, “is this an effective use of so much resource?”

My biggest criticism, however, is that these weaknesses have existed within the EFQM framework for almost 20 years. They are actionable, but the guardians of the model have done little to resolve them. Is that continuous improvement?

EFQM and ISO 9001

Each framework having more or less the exact opposite strengths and weaknesses actually carries a thick irony – the solutions are staring us in the face. To be fair, there has been some movement on the ISO 9001 side to incorporate some of the EFQM strengths. This was seen most obviously when ISO 9000:2000 was published. The under-pinning “8 principles of quality management” were introduced, as were some new EFQM-influenced criteria, most notably Customer Satisfaction and Continual Improvement. However, to my eyes, this was done in a very superficial and even a clumsy way. The clauses were brief and ill defined, leading to a large degree of elasticity in the way the are applied. Now we also have ISO 9004:2009, which moves even further in the EFQM direction. However, in Mark’s words, “does it ever deliver truly strategic information?” Probably not

And ISO 9001 does have its strengths

There is clear potential for a meeting of minds between the frameworks. For all its weaknesses, ISO 9001 has the inarguable strength that it requires auditability and proof. An ISO 9001 audit may not be strategic but, done properly, it should at least be factual, reliable and performed in a reasonably cost-effective way. ISO 9001 systems also usually benefit from two levels of independent scrutiny and regulation (again maybe not perfect but its there). Plus ISO 9001 certification is worldwide and widespread and it has found a way (by fair means or foul) to role out a commercially viable model and system of assessment

The conclusion? Put both frameworks in a blender and turn it on. We might just end up with a half-decent smoothie

3rd December 2009: Update to this article

Matt Fisher posted a very useful comment to this post yesterday and told us that the most recent EFQM revision has taken some of these issues on board

Here’s a link to a press release on the subject, from EFQM

The criteria has in fact been expanded with regard to sustainability. On a first review it does appear to relate to environmental as opposed to economic sustainability (profitability in other words), which was the weakness to which I was referring in my post

Post from: Capable People Blog

EFQM and ISO 9001 – A comparison of approaches

I have just returned from the Ninth Annual IRCA Conference, held in Yokohama, Japan. It was a great privilege being invited to speak at such a prestigious and high profile event, and to be among such interesting and distinguished company

The theme of the event was “Auditing for Effectiveness” and I guess that I was invited because I have been quite vocal on the subject over the past year or so. I still find it hugely flattering whenever I realise that someone is listening

Auditing for Effectiveness: The CPD Workshop

Anyway, prior to the event I ran a 2 day CPD workshop for a number of individuals who wanted to explore the concepts in more depth. I had put together a 2 day program earlier in the year, and I must pass on my gratitude to the people of Gloucestershire College for allowing my to test out the new materials on them in May. The feedback I received from that trial run was extremely valuable and helped me refine elements of the workshop to the extent that it received excellent feedback

The workshop explored ways in which the EFQM RADAR assessment mechanism could be incorporated into more general QMS audits to add depth and rigour to the process. Here’s a group shot of the workshop delegates

The 9th Annual Conference

The annual conference was something of a daunting affair. It was held in the very grand surroundings of the Pacifico Yokohama Conference Centre, with over 300 delegates attending. I had a one hour slot mid-afternoon, the other speakers included:

• Simon Feary, Director of the IRCA and Chief Executive of the CQI
• Masakuzu Ikoma, JAB (Japan Accreditation Bureau)
• Noriyuki Hishino, Technical Advisor of LRQA in Japan
• Ashihiko Aoki, Cannon
• Paul Palmes, Business Standards Architects Inc
• Ian Nield, Investors in Excellence Ltd

All in all the delegates all received good value for money, and it appeared that the general consensus was that third party certification could not carry on drifting in its current direction, and decisive action to protect the integrity of the system was required. This would clearly involve improvements in both technique and governance

Free stuff for my blog readers

If anyone is interested in receiving copies of the powerpoint slides I used during my 1 hour presentation on the topic of “Auditing for Effectiveness” you can download them from the Members Area of this blog. You’ll need a password which you can request via the contact form on our main site – MAKE SURE YOU TYPE YOUR EMAIL ADDRESS IN PROPERLY – otherwise I can’t send it

We love japan

Finally I need to thank everyone we met while we were in Japan. When I say “we” I mean me and Mrs Shaun, who tagged along for the experience. The people and the county are quite lovely, and we have never encountered such kindness in all our lives. Particular thanks go to the IRCA team in Japan, Masayo, Miho and Yuki, who looked after us so well. I’d also like to nod in the direction of Ian Nield from Investors in Excellence. His operation may well be worthwhile checking out – they have some interesting ideas

Post from: Capable People Blog

9th Annual IRCA Conference: Yokohama 2010

I think most people would agree that the quality of internal auditing (of any discipline) isn’t always what it should be. Typically reports are run off in a rush, with a poor level of detail, calling into question how rigorous the audit process actually was. And as for overall value …

There are a few techniques that people can use to increase the chances that the audit process will have the rigour and depth of scrutiny necessary to generate decent management information, turtle diagrams, for instance. Lately, though, I’ve been looking at how we can borrow part of the EFQM assessment approach to improve our internal audits. In an earlier post I highlighted a few EFQM weaknesses, but I think there is an opportunity to take one of its most simple and easy to apply strengths, and apply it to internal auditing – and it is equally applicable to quality, environmental and health & safety audits

For those of you who don’t know, the EFQM assessment process uses a numerical scoring system. Its not an exact science, of course, but it does help assessors rank and rate comparative data between companies and processes, and maintain a consistent approach. At the moment that scoring system is based on the “RADAR” Scorecard (see below)

The EFQM RADAR assessment approach uses a type of balanced scorecard to encourage the assessor (or auditor for that matter) to ask a series of key questions, namely;

R. To what extent are results used to set targets for process performance?

A. To what extent is a clear approach (procedures for example) defined and understood?

D. To what extent is the approach deployed (i.e. does everyone follow the approach or is deployment patchy)?

A. To what extent is the process assessed (i.e. measurement that asks the question “is it working the way it should”)?

R. To what extent is the process reviewed? (i.e. a review of whether the overall approach is still relevant and suitable)?

My view is that many internal audit systems could benefit from using this type of framework. It would improve the depth and consistency of process scrutiny no end. Internal audit report forms and checklists could be constructed to force auditors to follow the discipline. A similar scoring system could even be used to try to generate comparative data, audit to audit

Its a different approach, right enough, but worth thinking about maybe? Drop me a line if you’re interested in working with me to integrate this approach

Post from: Capable People Blog

Using EFQM RADAR to improve the quality of auditing