(With thanks to Chris Baker, Technical Director, Institute of Internal Auditors)

Changing risk profile

The head of internal audit needs to know whether anything has changed in the risk profile of the organisation to create the desire for the audit. Since management is responsible for managing risks, the head of internal audit will discuss with the management responsible for the information security risks their assessment of the effect of the recent events on the organisation’s risk profile

The considerations to take into account relate to the evaluation of the risks and of the responses that management believes are in place to address these risks. They include:

1. The size of a risk depends on the impact on the organisation if the risk event crystallises and the likelihood that it will crystallise. The evaluation of the size may therefore have changed because:

• The projected impact of losing personal data may now be thought to be higher than before because the damage to reputation could be greater given the publicity and public interest in the issue, and the potential sanction demanded by the data protection regulators may be higher than before
• It is possible that the likelihood of this happening might be changed – perhaps there will be increased interest of external parties in trying to force an incident or perhaps managers have decided immediately to follow the actions of HM Customs and Revenue, to remove the drive bays and connections ports and thus to terminate the possibility of moving any data onto digital media
• The actual effectiveness of existing responses to the risk may change – e.g staff may be more sensitive to the risk as a result of the publicity.
• The perceived effectiveness of responses to the risk may also change – managers may have been relying on technical access controls to protect access to confidential data, not taking into account the vulnerabilities related to transferring data outside the organisation

All of these may change the relative priority of data security issues and the appropriate treatment in the internal audit plan Source of assurance and skills available to internal audit. The internal audit plan will take into account not only the risk analysis but also those areas on which those responsible for governance want the independent and objective assurance that internal audit can offer as well as the skills available to internal audit to provide that assurance

Given the greater focus on data security issues, senior management and the board may feel a need for more independent and objective assurance. This would be a reason for including a new project in the internal audit plan

However, the head of internal audit may be able to minimise the work to be done by internal audit by reviewing the work being done by other assurance sources. Internal audit can assist the organisation by helping senior management and the audit committee to understand all the monitoring and assurance activities that the organisation undertakes and by providing a bridge between the data security specialists and the audit committee, if one is needed

Although in an ideal world, all internal audit activities will have the skills necessary to address data security issues, it may still be the case that some organisations do not have those skills available. The head of internal audit will have reported this to those responsible for governance and obtained their approval of the implications – that certain assurances could not be provided. Given the changes in the perceptions of data security issues, this may no longer be acceptable. In that case, the head of internal audit will be required to identify and source skilled resources from elsewhere

Post from: Capable People Blog

A risk-based approach to internal audit planning

Can you audit a process if it is not supported by documented procedures?

This is a question I ask at an early stage in all my IRCA auditor courses. It always gets the same response. A few shrugs. One or two firm “no’s” and an occasional “Erm … yes?” The question I ask proves only one thing. It is a concept that few are totally comfortable with. The session moves on. I suggest that it is possible to audit in the absence of procedures, at which point the guys who had previously given me the firm “No!” generally retort along the lines of “You’re a mad man. It cannot be done!” Or something. They invariably fall right into my trap

“Why do you think you can’t audit in the absence of procedures?” I ask.

“Well, without procedures you can’t tell what they are supposed to be doing” they often respond
“Why can’t you ask them?” I respond
“Because they could tell you anything” they generally reply
“So could their procedure …”

At this point I can almost smell victory. But the argument usually has one last hurrah along the lines of …

“How can the auditor decide whether they are doing the right things though?”
“Well, there are standards, specifications, orders, contracts … the auditor could check those”

At this point we start to see that the procedure is not the be all and end all that we conveniently pretend that it is. Often they are little more than a cover, a smoke screen or a diversion. The critical thing for the auditor is to check the required standards are being met (contract, order, standard etc) and verify that they are through the records and testimonies that are available. Procedures may help along the way, let us not suggest that they are all worthless, but they are not always that significant in the mix

If I was to be granted one wish with regard to QMS auditing and how it may improve, it would be for the discipline to be blessed with a greater degree of intelligence and consideration. Auditing by numbers just does not work, and an auditor that needs such a comfort, really needs to try another profession. It is said that we humans utilise no more than 10% of our brains at a time. For many QMS auditors I have met, the figure is considerably less

Post from: Capable People Blog

Auditing Non-Documented Procedures

Planning anyone?

A significant part of my lead auditor courses are devoted to audit planning, and everyone immediately sees why “planning” is so important. But how often do you get anything resembling an audit plan from your third party auditor? If you’re lucky you may get an unexplained list of clauses that he intends to cover, but does he give you a timetable? I have seen several examples where an auditor has turned up to perform a multi-site audit without giving an indication even which site he intends to visit on a given day. Why is this important? BECAUSE THE WORLD DOES NOT REVOLVE AROUND, OR STOP FOR, THE AUDIT!

Clients are busy people. They have specific working patterns, customer visits, operational pressures THEY NEED TO KNOW. The reason auditors don’t like drafting audit plans, of course, is that planning is NON-CHARGEABLE. It is a pre-audit activity that most third party auditors (as non-salaried sub-contractors) don’t get paid for. Is that an acceptable excuse? I don’t think it is

Additionally, on more than one occasion, I’ve heard an auditor insist that he must cover “all clauses on each site”. What is rationale behind that? Different sites usually do different things, some things (design for instance) may be limited to one site. Top management functions are often housed at HQ. This approach is a nonsense and only confirms that this guy probably does not understand the concepts of process, system or effectiveness

pet subjects

All audits should be performed to an agreed audit scope. The scope being defined as the limits and boundaries of the audit. The application of a scope naturally assumes that some things are going to be outside the scope of an audit. But what happens if that turns out to be your favourite thing in the whole world? On a couple of occasions I have seen “health & safety” listed as a topic to be reviewed during a QMS audit to ISO 9001. Well, important as this may be (and with the possible exception of “noise” as referred to in clause 6.4), it is sadly outside the scope of ISO 9001. Other auditors, I find, have an inordinate fondness for auditing cosy subjects like training records, with a disproportionate allocation of time between operational and back office functions. Why would this be? Well, I could suggest that back office functions usually involve being nice and warm, sitting down and being supplied with regular tea and biscuits, whereas operational aspects can be cold, noisy, wet and involve too much standing up. And none of us likes that!

The Blabbermouth

When the auditor is talking he is gathering no useful information, and some of them do A LOT of talking. Oh, the places he’s been, the companies he has saved from oblivion. What a hero! Even if these stories are ever true (and I do have my doubts about most of them) they are wholly irrelevant to the execution of the day job. A critical quality of a competent auditor is knowing how to shut up and listen

Mr “disaster waiting to happen”

Auditors need to understand that risk is a function of likelihood and severity, and that proportionality of controls relies on an accurate assessment of these factors. Not all auditors have allowed this penny to drop, unfortunately. Many see it as their job simply to identify a risk, and then demand some sort of action, irrespective of the magnitude. In the English speaking world, we call them nit-pickers. A term with a questionable linguistic origin, but we all know what it means, and it fits all too many practitioners

Fag Packet Audit Reports

Q. What is an audit report?

A. Management information

That’s what an audit report is, pure and simple. Or at least that is what it can be. Trouble is, our auditor from hell likes his short cuts. This can involve the use of pre-populated audit reports, consisting of a range of stock cut and paste conformity statements. Highly useful management information, I don’t think. The very idea that you can audit a College on one day and a waste contractor on another and write pretty much the same report and consider that to be OK is ludicrous

Who is to blame?

I’ve heard many people blame the IRCA and trainers for this dearth of competence but, while I do accept there are some rotten trainers around, I think on balance that is unfair. You might say I would say that. In general terms the responsibility for ensuring that your staff are competent lies with you, the employer. Trainers may or may not help, but if the training has been ineffective YOU DEAL WITH IT. Even if the training has been excellent, how long can you expect the effects to last? I know better than anyone else what a transient influence I am on my customers. I may spend 5 days of my life with them, then I am gone. What happens when we go our separate ways? I rarely find out. Blaming the trainer is like blaming the driving school that taught a driver that caused a fatal accident. Does that ever happen? No, and with good reason, it would be nonsense. For me the responsibility lies with the certification body and it would be nice if they were prepared to live by the requirements of clause 6.2.2, rather than just expect that of their clients

Post from: Capable People Blog

The Auditor From Hell!

It’s here. A year after the publication of ISO 9001:2008, the companion document ISO 9004 has been updated. And whilst the most recent changes to ISO 9001 have been minimal and to all intents and purposes largely cosmetic, the changes to ISO 9004 have not. The changes are big. In fact the revised standard is barely recognisable from its predecessor. It’s different

So what are the changes?

… Where do we start?

OK, gone is the old title “Guidelines for Performance Improvement”. The new title for ISO 9004:2009 is “Managing for the sustained success of an organisation – a quality management approach”. Gone is the old familiar format that mirrored ISO 9001. The ISO 9001 requirements as “boxed text” accompanied by some general hints and tips outside the boxed text, there to help us understand and apply the various requirements of ISO 9001. That is gone

In fact, ISO 9004 no longer follows the structure and requirements of ISO 9001 in any real way. It no longer goes through the ISO 9001 requirements and offers specific clause by clause advice. It actually does more or less what the title implies, it offers guidance on a more general “quality management approach”

This calls into question what the intended application of ISO 9004 actually is. It can no longer really function as an implementation guide to ISO 9001, firstly because it no longer tries to, but secondly because the scope of its content is now fundamentally different. It contains, for example, guidance on such matters as:

• Strategy and policy formulation
• Strategy and policy deployment
• Financial resources
• Knowledge, information and technology
• Natural resources
• Innovation & learning

Wow. That’s different. Good topics though these might be for any management system, they are, arguably, out-with the current scope of ISO 9001:2008. What is more, it appears that ISO 9004 is starting to use some established terms in a different way to ISO 9001. “Policy” for instance. If we look at the way ISO 9001 uses the term “Policy” (with reference to clause 5.3) it deals very much with the one page “statement of intent” that we all know and (maybe) love. ISO 9004 appears to be using the term “Policy” in a broader sense, something more detailed, meaningful and less neutral. And strategy? Well, ISO 9001 currently does not even go there

The most obvious “hit you in the face” feature of ISO 9004:2009, however, is that it borrows very heavily from the EFQM Excellence Model

All of those new topics I listed above feature heavily in the excellence model, and have done for a couple of decades. We saw a small movement to an “excellence model approach” in 2000 when the “8 Principles” were introduced. These principles were lifted, more or less, from the principles that underpinned the EFQM excellence model at the time. Some of them (Continual Improvement, Customer Focus) even generated some significant new requirements within ISO 9001:2000. Many people expected ISO 9001:2008 to move a little further in that general “excellence” direction. It did not, of course. Some of us were pleased, some of us were disappointed. Maybe ISO 9004:2009 is a kind of half-way house? Maybe it has been developed this way as a means of placating those of us that maintain ISO 9001 standards are old fashioned or not challenging enough? Maybe ISO is saying, “OK you want something more challenging? There you are. Next time be careful what you wish for!”

Either way, as a general observation, I have to say that I am detecting some initial confusion. Not that the contents are in any way badly written or irrelevant, just that practitioners are simply confused as to what the intent of ISO 9004:2009 actually is. How are we to use it? Will certification bodies develop a certification scheme for it? (there’s a thought), how are ISO 9001 auditors meant to use it? All these questions remain for the moment, so far as I can see, unanswered

So, yes, it appears to be a “quality” document, but will it be used in a “quality” way? Only time will tell

KWVBPN9H9MGS

Post from: Capable People Blog

ISO 9004:2009 – A review

I’m re-working and re-publishing this article as it has once more become topical. Right now on the CapablePeople QHSE Community Forum on LinkedIn*, we’re having a discussion about the ills of third party management system certification. We quickly appear to be reaching consensus that there are problems, and Certification Bodies are attracting a degree of criticism. In my opinion they deserve it and have brought it all on themselves

I have long been of the opinion that Certification Bodies should not be allowed to have a high influence on the review process where management system standards are concerned. Personally I don’t think they should be allowed a voice at all. It is literally none of their business. The simple fact is that Certification Bodies have a clear commercial conflict of interest which, I would strongly suggest, will heavily influence their views on the standard, what it should contain and, more to the point, how difficult it should be to achieve. The simple truth is that it is in the CBs commercial interest to achieve as many registrations as possible. That is not opinion, that is plain fact, and the easier a standard is to achieve, the more registrations are achievable. In the past I have likened this to allowing a traffic warden to paint his own yellow lines. This, I would argue, is a very large problem, and until CBs are put in their place, then a steady degradation in the credibility of standards will be the result, as they gradually become easier and easier

However, there is obviously nothing inherently wrong with certification schemes as such. After all they seem to work quite well in, say, the restaurant and hospitality sector. But schemes for rating restaurants and hotels are administered differently. There are fewer commercial conflicts of interest and they are more clearly run for the primary benefit of the consumer, not to suit the interests of the establishments. Notably, hotel and restaurant “star rated” schemes tend to incorporate an element of “mystery shopper”. These things are not so true of management system certification schemes, and in this post I suggest that this might be the real underlying reason for the credibility crisis, and not the ISO standard itself

To illustrate the point, and for a bit of fun, let’s just consider what would happen if the Michelin Star Restaurant award were run along the same lines as third party QMS certification

We’ll start by eaves-dropping on the strategic management team meeting at Michelin Star HQ …

Michelin Star Award Scheme – Annual General Meeting

“Thank you all for coming to this special meeting today, gentlemen. To start, I think it would be appropriate if I summarise things so we can all agree on where we stand. (Ahem) Since the inception of the Michelin Star restaurant recognition scheme in the early nineties, the scheme has grown to the point that it now recognises over 170,000 establishments in the UK alone, reaching a peak of 195,000 in 2001.  Over the past 7 or 8 years we have seen a slight year on year decline in the number of recognitions, and the time has come to act on the causes of that decline, to protect the future of the scheme. There is a fair amount of circumstantial evidence that suggests that the scheme may be suffering a credibility crisis as a result of some negative press. Negative press not from our customers, you understand (the recognised establishments), but from the food consuming public”

“The peasants are revolting!”

“Ha ha, very good Reg. But seriously we must find a way to shut them up because they are starting to become bad for business”

“What do they know anyway? They need to respect our assessment expertise. We’ve assessed more hot dinners than they’ve … had … hot … dinners … oh, that metaphor doesn’t really …”

“A valid point nonetheless Reg. But the fact remains that they are noisy and irritating and we need to do something to put a stop to their continual moaning”

“What sort of things are they saying anyway?”

“Oh, I’m surprised you have to ask. The usual rubbish about it being too easy to achieve a Michelin Star, the old accusations that we want to issue as many stars as possible, stars are meaningless, that sort of thing. As an example, the other day I got a complaint asking me how we could award a star to Café Joe on Dewsbury High Street, as it was, in their words, ‘a shit-hole’ – pardon my French”

“How did you respond to that?”

“I simply explained the facts. That the Michelin Star is awarded based on a sample generated at the time of the assessment and that we can only make our recommendation based on what we witness at the time of the assessment”

“Did you also explain that we give them two months notice, turn up only by appointment and only speak to members of staff that we have agreed well in advance?”

“Er, no – I thought it would only confuse matters. Oh, by the way Reg, how did your assessment of Il Polio go yesterday?”

“Oh very well, very well. Infrastructure was sound, doors, tables, floor, that type of thing. Work equipment was in order, plates, forks etc. Food tasted like a tramp’s underpants, but the process for putting it on the table was sound, so the system met our criteria”

“So you’ll be recommending recognition?”

“Absolutely. After all, it’s about system and process - that’s what is important. When are the great unwashed going to wake up to that fact?”

“Probably never, but back to the job at hand. What we need to do is to inject some sort of credibility into the scheme”

“You’re a mad man – it cannot be done!”

“Quiet Ron! – I believe it can. What we need to do is to conduct a high profile review of standards, to prove we’ve got them right. We do need to demonstrate that the stars actually mean something”

“Agreed”

“So as a starter, what I’d like you to do, Reg, is a customer survey – find out what the customer thinks – we are, after all, a customer focused organisation”

“Ask the general public?”

“Oh Christ no – we already know what that rabble thinks, thank you very much. No, our customers – those great establishments who pay their fees and dues to the scheme. We need to find out what they think”

“What sort of things do you want me to ask them?”

“Well Reg, basically just ask them if they think the scheme is fair, or whether they would like us to make it more difficult for them”

“Oh … right. How many do you want me to ask? All of them?”

“What??? Have you been drinking? No! You’re an auditor for god’s sake … a random sample of, say, two hundred will be just fine”

“When you say random sample, do you mean random sample or ‘random sample’ …?”

“Well ‘random sample’ of course. We certainly don’t need to go looking for trouble. Especially from the up-market west end brigade”

“You’re right there. I did an assessment in the west end last week, you know, one of these posh places. Assessment went well enough, a few observations …”

“Like what?”

“Just the usual. Portion sizes too small, prices too high, no childrens’ menu, ambience could be improved by the addition of generic piped muzak, not everyone likes rabbit – consider the addition of some family favourites such as lasagne or gammon & pineapple … – just the type of findings you’d expect from one of these places. Anyway, I’m sitting in the closing meeting with the owner of the place, going through these observations and he starts changing colour, then steam starts coming out of his ears – literally”

“Medical condition?”

“That’s what I thought at first, then he starts turning the air blue with all manner of abuse. You know the type of stuff, you don’t know what you’re talking about, you’ve never ran a restaurant …”

“Ah yes … if I had a pound for every time I’ve heard those old chestnuts …”

“Exactly. So I say to him that actually I do know what I’m talking about because I’ve personally issued over 500 Michelin Stars – 180 in Rotherham alone – and I gave him a few practical suggestions”

“Quite right, after all, the assessor should try to add value …”

“Exactly. I gave him some examples of good practice that I’d seen at Il Polio. The massive portions at low low prices … the pet’s menu … the karaoke corner … cheesey chips …”

“I bet that shut him up”

“Huh, you’d think, but no. Made him worse if anything. Told me what I could do with my star. Have you ever had Jamie Oliver telling you to ‘thtick your thtar up your arthe?’ Good job I was wearing my anorak”

“Hah! He’ll be back”

“Yeah, no doubt, but I had the last laugh on him. Because of his attitude towards my best practice suggestions I gave him a non-conformance –‘ failure to consider benchmark data for continual improvement’ …”

“Ha ha you got him there Reg!”

“Bang to rights!”

“People like him just haven’t embraced the continual improvement philosophy. In fact he probably can’t even spell it”

“That’s right. He’s dyslexic”

“Er, yeah right, so back to the job at hand. Don’t waste your time with that load of pretentious so and so’s. They’re in the minority thankfully, and we need to focus on where our bread’s buttered. So can you conduct this vox pop and get back to me with the results in, say … a year?”

“Or thereabouts?”

“Yeah, yeah, a year or thereabouts”

Two years later …

“I’ve got the results of that customer survey you asked for, sir”

“Oh, that’s great Reg. What are your findings?”

“Do you want me to read out all 200 responses or just the summary?”

“Erm, just the summary if you don’t mind Reg …”

“Right you are. Well, the vast majority of the respondents overwhelmingly indicated that they didn’t want the standard to be raised at all”

“That’s fine, anything else?”

“Yes, a little. Some establishments, mostly the Michelin Star takeaways and burger vans, commented that if the standard was raised, they would be worried whether they would be able to meet the higher criteria and that they might have to drop out of the scheme”

“Oh no no, that would never do. There are bloody loads of them – we’d be ruined”

“Exactly. So do you want to hear my recommendation?”

“Yes, go on”

“We leave it alone”

“But what about all this noise from the great unwashed? How are we going to address that little problem of perception?”

“Well, we could publicise the fact we’ve conducted a review …”

“ That won’t be enough – they really want us to do something about our standards, but go on …”

“We could change a few words and re-date the scheme criteria? Recommend a more regular review at unspecified intervals – that might kick it into the long grass for a few years …”

“Brilliant!”

“I’ve also got an idea for extending the scope of the Michelin Star scheme to embrace the not-for-profit sector”

“Not for profit? What does that mean?”

“Well, works canteens, army messes, prison kitchens, homeless shelters … there are LOADS of them!”

“Brilliant – great job! Let’s do it! What are you up to tomorrow?”

“Just a routine surveillance on Il Polio – but don’t worry – I’m taking a packed lunch”

——————————————————-

What if all schemes were run like this ….?

*if you want to join the CapablePeople Community Forum on LinkedIn, simply send a request to join and, provided I like the cut of your jib, I’ll be happy to welcome you in so you can join the debate and get all the links and things that get posted. Shaun

Post from: Capable People Blog

What a way to run a certification scheme

Last week I published a review of ISO 9004:2009. In it I noted just how “EFQM” it was. That sparked a short exchange between Mark Harbor and I on Twitter about the merits of the EFQM self-assessment approach and the limitations of the typical ISO 9001 audit-driven approach. Something from that debate concerned me and it was a while before I could put my finger on what it was. Now I think I have

It is my firm belief that when we compare EFQM and ISO 9001 the strength of one framework is the weakness of the other and vice versa. In other words, what one framework does well, the other does badly, and the match is almost a perfect negative

In this post I’m going to try to explain exactly what I mean by that

My history with EFQM and ISO

My involvement with each model goes well beyond academia. Those of you who know me from Capable People will be aware that I’ve been training ISO 9001 lead auditors for about ten years, however prior to that, in a past life (in the 1990s) I worked extensively with the EFQM Model. I assessed on numerous occasions for the UK Excellence Award and the North East Excellence Award, trained assessors for the North East Excellence Award on a couple of occasions, and also got involved in upwards of 50 internal EFQM self-assessments for various organisations. It is from these direct experiences that I draw my conclusions

The reason I found it necessary to describe my battle scars, particularly with regard to the EFQM Model, is simply because it works so well on paper. If you’d never been through the pain of self-assessment, and suffered the frustration of post-assessment inertia, you’d never guess it had a single fault … but it does

The strengths of the EFQM approach

Frankly the EFQM approach has a few faults, but let’s start with the strengths, because the glass might just be half-full. I’ll try and list them;

• Its criteria covers strategic processes in far more detail than ISO 9001
• It does the “systems approach” better too
• Its criteria are “weighted” and identify that some processes are more critical than others (which they are)
• It does “leadership” in a more detailed and academically sound way
• It makes a more concerted effort to direct assessors to identify cause and effect relationships (sometimes in vain of course, but it tries, nonetheless)
• It includes financial/business results and some financial processes within its criteria (not simply “quality”)
• It directs assessors to examine the integrity and breadth of “results” in a better way, including an appreciation of direct and indirect measures, and the benefit of a balanced range of metrics
• It actually has criteria that support the “Involvement of People” quality principle
• EFQM self assessment is surprisingly good fun, if you like that sort of thing

The weaknesses of the EFQM approach

Although it has strengths it does have its significant weaknesses or, in EFQM language, Areas For Improvements (AFIs). These are what I consider have always been the most significant ones;

• The use of documented evidence or the requirement to provide “proof” (as opposed to testimony) within the self-assessment process is usually limited
• Although the criteria, in theory, covers strategic issues, financial measures and results, the output from assessment will only ever be as good as the inputs allow. In my experience of going through numerous assessments, there is an almost universal reluctance from the senior team to allow unfettered access to this sensitive information “warts and all”. Therefore the principle of “Garbage In – Garbage Out” (GIGO) usually applies
• Although the criteria includes financial performance, it does not do it in sufficient enough detail to allow a realistic assessment of the sustainability of the business. Assessors may well look at how budgets are allocated and managed, which is a good thing in itself, but sustainability is the $10,000 question. Consequently there have been numerous examples of award winners getting into commercial difficulties a very short time after receiving an EFQM based award. It could therefore be argued that the model awards a deceptively high score for companies that are going out of business albeit in an “excellent” way. This feature may well partially explain why it seems to have retained its popularity a little longer within the public sector in the UK. In this sector financial management more or less is management of budgets, and the issue of commercial sustainability is not really a factor in the mix
• The assessment does not identify any clear “rights” and “wrongs” – just a set of “coulds” and “could do betters”. Fair enough, you might think, but in my experience that almost always leads to strangulation of the process by inertia once the assessment is complete. Typically the assessment will yield upwards of 150 strengths and 150 AFIs, with no direction on priorities (that is for the company to decide). The problem is that this wealth of data  usually completely overwhelms the organisation and brings the process of improvement via self assessment to a sudden stop. You can have too much information
• The process, done properly, is incredibly hungry on resources and often struggles to satisfy even the briefest of cost versus benefit analysis

I must confess that between the years 1994-1999 there was no bigger disciple of EFQM than I. However, after a few years, Groundhog Day well and truly kicked in. I looked back over the fifty or so assessments that I’d been involved in and struggled to identify even a small hand full that had delivered real improvements. That is, improvements that I felt the organisation could not have identified anyway, simply by intuition. The fact was that most companies already knew fine well what their biggest problems were before the process began, and I could see in the faces of many a senior manager during the assessor feedback an expression that suggested “this is an expensive way of telling us what we already knew”. I’ve heard senior teams criticised on numerous occasions for a lack of “buy-in” or “commitment”, but sometimes you need to see things from their perspective. After a while I found myself asking, hand on heart, “is this an effective use of so much resource?”

My biggest criticism, however, is that these weaknesses have existed within the EFQM framework for almost 20 years. They are actionable, but the guardians of the model have done little to resolve them. Is that continuous improvement?

EFQM and ISO 9001

Each framework having more or less the exact opposite strengths and weaknesses actually carries a thick irony – the solutions are staring us in the face. To be fair, there has been some movement on the ISO 9001 side to incorporate some of the EFQM strengths. This was seen most obviously when ISO 9000:2000 was published. The under-pinning “8 principles of quality management” were introduced, as were some new EFQM-influenced criteria, most notably Customer Satisfaction and Continual Improvement. However, to my eyes, this was done in a very superficial and even a clumsy way. The clauses were brief and ill defined, leading to a large degree of elasticity in the way the are applied. Now we also have ISO 9004:2009, which moves even further in the EFQM direction. However, in Mark’s words, “does it ever deliver truly strategic information?” Probably not

And ISO 9001 does have its strengths

There is clear potential for a meeting of minds between the frameworks. For all its weaknesses, ISO 9001 has the inarguable strength that it requires auditability and proof. An ISO 9001 audit may not be strategic but, done properly, it should at least be factual, reliable and performed in a reasonably cost-effective way. ISO 9001 systems also usually benefit from two levels of independent scrutiny and regulation (again maybe not perfect but its there). Plus ISO 9001 certification is worldwide and widespread and it has found a way (by fair means or foul) to role out a commercially viable model and system of assessment

The conclusion? Put both frameworks in a blender and turn it on. We might just end up with a half-decent smoothie

3rd December 2009: Update to this article

Matt Fisher posted a very useful comment to this post yesterday and told us that the most recent EFQM revision has taken some of these issues on board

Here’s a link to a press release on the subject, from EFQM

The criteria has in fact been expanded with regard to sustainability. On a first review it does appear to relate to environmental as opposed to economic sustainability (profitability in other words), which was the weakness to which I was referring in my post

Post from: Capable People Blog

EFQM and ISO 9001 – A comparison of approaches

How many people really understand “root cause”? How many people really make a serious attempt to apply clause 8.5.2c of ISO 9001 in the correct spirit? How many auditors (presumably for a quiet life) are quite prepared to be complicit in this charade?

These questions and more are considered by Mike Mickelwright in this funny little video clip. We ALL know what he is talking about, right?

Click here to view the embedded video.

Post from: Capable People Blog

Top Ten Unacceptable Corrective Actions

I have just returned from the Ninth Annual IRCA Conference, held in Yokohama, Japan. It was a great privilege being invited to speak at such a prestigious and high profile event, and to be among such interesting and distinguished company

The theme of the event was “Auditing for Effectiveness” and I guess that I was invited because I have been quite vocal on the subject over the past year or so. I still find it hugely flattering whenever I realise that someone is listening

Auditing for Effectiveness: The CPD Workshop

Anyway, prior to the event I ran a 2 day CPD workshop for a number of individuals who wanted to explore the concepts in more depth. I had put together a 2 day program earlier in the year, and I must pass on my gratitude to the people of Gloucestershire College for allowing my to test out the new materials on them in May. The feedback I received from that trial run was extremely valuable and helped me refine elements of the workshop to the extent that it received excellent feedback

The workshop explored ways in which the EFQM RADAR assessment mechanism could be incorporated into more general QMS audits to add depth and rigour to the process. Here’s a group shot of the workshop delegates

The 9th Annual Conference

The annual conference was something of a daunting affair. It was held in the very grand surroundings of the Pacifico Yokohama Conference Centre, with over 300 delegates attending. I had a one hour slot mid-afternoon, the other speakers included:

• Simon Feary, Director of the IRCA and Chief Executive of the CQI
• Masakuzu Ikoma, JAB (Japan Accreditation Bureau)
• Noriyuki Hishino, Technical Advisor of LRQA in Japan
• Ashihiko Aoki, Cannon
• Paul Palmes, Business Standards Architects Inc
• Ian Nield, Investors in Excellence Ltd

All in all the delegates all received good value for money, and it appeared that the general consensus was that third party certification could not carry on drifting in its current direction, and decisive action to protect the integrity of the system was required. This would clearly involve improvements in both technique and governance

Free stuff for my blog readers

If anyone is interested in receiving copies of the powerpoint slides I used during my 1 hour presentation on the topic of “Auditing for Effectiveness” you can download them from the Members Area of this blog. You’ll need a password which you can request via the contact form on our main site – MAKE SURE YOU TYPE YOUR EMAIL ADDRESS IN PROPERLY – otherwise I can’t send it

We love japan

Finally I need to thank everyone we met while we were in Japan. When I say “we” I mean me and Mrs Shaun, who tagged along for the experience. The people and the county are quite lovely, and we have never encountered such kindness in all our lives. Particular thanks go to the IRCA team in Japan, Masayo, Miho and Yuki, who looked after us so well. I’d also like to nod in the direction of Ian Nield from Investors in Excellence. His operation may well be worthwhile checking out – they have some interesting ideas

Post from: Capable People Blog

9th Annual IRCA Conference: Yokohama 2010

This topic has taken on somewhat of a scary feel for me lately. In June I will be presenting to several hundred representatives from the third party certification industry in Japan, at the IRCA 2010 Conference in Yokohama. This is my topic

I think I know why I was asked. I’ve had a lot to say about weaknesses in third party audit systems on this blog and a few other places, and I have expressed strong views on the need for modernisation of traditional and (dare I say it) out-dated conformity focussed approaches. It may be that the industry is starting to agree with some of the things I have been saying. I hope …

Anyway, I really need to start constructing something. An argument. Something that is challenging enough to make people think differently, but engaging enough not to turn people off and create a defensive response. A dilemma and no mistake

Fortunately I have been able to review a JIBDEC/JAB action document, so I know a little about where they are coming from. That document talks about the need for a balanced effectiveness/conformity assessment. Not so much “does it work to its own procedures?” more like “does it do what the customer wants it to?” The JIBDEC/JAB document also talks about how the audit needs to add value. It is this second aspect that has got me thinking. What do we mean when we say we want the third party audit to “add value?”

Third party audits – adding value for whom?

When we talk about “value adding” third party audits are we really suggesting that a third party auditor, without performing any meaningful research or necessarily having any great expertise in the sector, should be able to go into an established company for a couple of days and tell them much that they did not already know? Come on, let’s take a trip to planet earth for a second, is that realistic?  I think anyone who thinks that a third party audit is going to yield a treasure trove of hitherto unknown and valuable management information is living in a dream world. It is not going to happen. The good news, though, is that it doesn’t matter. It doesn’t matter because third party certification schemes were never designed or intended to do that in the first place. Its just that somehow over the years we’ve lost sight of what the primary purpose actually is, and for whose benefit they exist. Third party certification schemes exist primarily to provide information not to the audited organisation, but to customer organisations. It is these organisations that we really should be thinking about. What do CUSTOMER organisations want from third party certifications? How can we improve third party certifications to increase value for CUSTOMER organisations?

It’s the customer, stupid

How could we be so blind? The reason the vast majority of companies seek registration is not because they want it for some intrinsic reason (although I accept one or two may do), nor is it because they believe the CB will send along a kind of Tom Peters to sort out all their problems and tell them how to conquer the world. It isn’t these companies we should be focussing on. We must think about the external value of certification and what that is supposed to mean in the public domain. When we think about adding value we should be asking questions like

• What is it supposed to say about a company that has certification?
• How high are the standards?
• Dare we trust the certification?
• Is it consistently applied?

What if every hotel was rated 5 star?

Maybe we need to start thinking about it in the same way that we think about star rating systems on hotels, because the same principle applies. When we choose a hotel for a family holiday we look at the hotel’s star rating and this adds value to us as customers. The ratings are extremely useful so long as we feel we can trust them. They give us information and help us to reduce the risk of choosing a hotel that might not suit our needs

But what would happen if we lost faith in the integrity of the star rating system? Well, for a start, we’d pay no attention to it. We would not use it as a selection tool, hotels would stop going through the rating process and the scheme would soon fall into disuse. Pretty soon there would be no more pointless star ratings. That is what would happen if the integrity was lost

I think it is this parallel we need to consider when we talk about breathing new life and credibility into third party management system certification. Above all else the system has to be consistent, trusted and enforced to a level that means something to prospective CUSTOMER organisations. We need to be clear about what the badge means and go back to basics to a certain extent. Maybe there needs to be more fails too. Think about it. Should it be our aim to get everyone certified? Great for CBs in the short-term, lots of fees, but in fact it is the last thing we should ever want if we think about it properly. All differentiation would effectively be lost

The day everyone achieves certification is the day that nobody needs it any more

Post from: Capable People Blog

Value adding third party audits

ISO 9001, that part of the ISO 9000 series that defines the quality management system requirements, clearly means many things to many people. There are certainly people with a vested interest in promoting its virtues and (sometimes) overstating its merits in the process. There are also some very vociferous opponents, including some career critics who have made a good living from slagging it off. But let’s try to be objective, (whilst simultaneously declaring an obvious interest in the its continued existence)

Why do people seek registration?
Check out this thread for some contributions. The simple inarguable truth is that most registered organisations seek registration to meet the requirements of an important customer, or an important prospect. It basically allows the company to bid for work they would otherwise be excluded from, as the customer has defined it as a condition to supply. Is this right? Well, we’ve talked about this in the past. The customer is king, if he says “jump!” we should all say “how high?”

Interestingly, when the benefits of ISO 9001 registration are debated, most people launch into often unquantified justifications revolving around control, conformity and efficiency themes, almost as if doing it mainly because the customer wants it is somehow something to be ashamed of. The fact that it opens doors economically is often overlooked. This obvious economic advantage of offering access to more contracts has to be a major benefit, and “quality guys” should not be afraid of recognising it

Is it a mark of excellence?
Some people might claim that it is but they are just plain wrong. There may well be a decent set of good business practices woven into that clumsy and badly written document, but the requirements set out in ISO 9001 are no more than a base line. Successful certification indicates that the bones of a documented QMS is in place, it is auditable and it is supported by a basic set of management processes and, if we’re lucky, a PDCA theme running through it. The company may be a million miles from world class and may even be going steadily out of business at the same time. So I’m saying, in quality terms, it’s no more than our starter for ten

Why the confusion then?
The ten thousand dollar question, but I can offer a personal opinion. From my own personal experience I find that quality guys don’t speak the language of business, and business guys don’t speak the jargon of quality. That means strategic, commercial and quality people often have parallel agendas and, because the communication and transparency are poor within the processes (the principle that we should all be seeking the same end result, a successful business), is lost in the ensuing conflict and battle for resources. How often do we hear the QA department moaning about poor top management commitment to quality? Time for a common language and a clear focus on what is important to the business and the customer, whichever way we want to badge it up

A business approach to managing quality
Another favourite theme, but let’s expand on it a little this time by exploring the cost of quality in economic terms. Quality, generally, is defined as “the degree to which a set of inherent characteristics fulfills requirements” or “conformity to requirements” or a variation on that theme. Often “quality” is stated only in product conformity terms, as if that was the only thing that mattered to the customer and, as a consequence, the lower the level of functional defects, the better things are. However there comes a point at which investing in conformity related improvements does not yield any economic benefits. Eventually we reach a point at which (at the current price) we offer the customer a tolerable level of defect, a lower rate of defect may come at a price that the customer is not prepared to pay. So “better quality” (in this situation) increases the chances the customer will seek a cheaper provider, possibly one with an inferior defect rate. So examining the likelihood of repeat business only from a conformity and defect rate perspective is a dangerous mistake to make. Customers care about lots of things. Once the level of defects is tolerable they start to care more about other things. After sales service, response times, ease of dealing with, user-friendliness of contact systems, flexibility, image even. If these are important to the customer, then they become the new conformity

Post from: Capable People Blog

The real benefits of ISO 9001 certification