Conformance, compliance, effectiveness and process auditing
What is an audit?
People audit for lots of different reasons, so there are lots of different types of audit. They vary in size (sometimes the audit takes in virtually all the company’s activities, sometimes just a single procedure); they vary in complexity (sometimes there are lots of standards, specifications, customer requirements and legislation to check on, sometimes not so many); and they vary in focus (why we want to do the audit in the first place).
Each of these parameters has its own audit terminology. In plain English, the main parameters are:
*What the audit will look at (the “SCOPE”)
* What the audit will check against (the “CRITERIA”)
* Why we do the audit (our audit “OBJECTIVES”)
It is critical that the auditor never loses sight of these parameters, and also that the parameters are clearly understood by the auditee, more about communication requirements later. If nothing else, an audit is a process designed to promote clarity and transparency. If it is shrouded in secrecy and delivered as something of a black art, the auditor is doing the reverse. The auditor’s role is actually quite narrow and limited. The auditor compares the evidence against the audit criteria, and then the auditor records the result. The result is then passed to the audit client (the party requesting the audit) and the client will do with that report whatever they choose…
Conformance audits versus effectiveness audits
Results are always important, and it should be a major objective of any auditor to clarify whether the important results are being achieved. Methods, on the other hand, may or may not be critical. An auditor will adopt a conformance approach when methods, as well as results, are important. Generally this means that the activity will be supported by detailed procedures and those procedures must be followed. This is common in heavily automated processes, many medical processes and legal processes for instance. Irrespective of whether results are good, there must be evidence that the results were achieved in the right way, otherwise future problems are inevitable
Sometimes methods are not critical. There may be a team of people working together, performing similar job roles, but they may each have their own different ways of doing things, and it may not matter. Many support functions such as Sales, Marketing, Customer Services and Training may actually require flexibility. This is generally because, unlike an automated manufacturing process, we can’t control the consistency of the input to the process, so it may well need to be flexible to allow for that. A good auditor must be able to apply common sense to the evidence and understand that in some instances, differences in methods do not present significant risks and problems. In fact they are a good thing as, in the examples previously identified; if the process is not flexible there WILL be problems
The term “compliance” is reserved for audits where the criteria contains mandatory requirements, most commonly legal requirements. In compliance audits there is understandably less of a requirement for flexibility, but there is an increased requirement that the auditor very clearly understands the requirements and interprets them correctly, as the potential impact can have a significant implication for the auditee
Sometimes the primary focus of the audit is to establish whether situations are improving. This is a common focus of a “follow up” audit. That is, some time in the past a conformance, effectiveness or a compliance audit identified problems and a follow up audit is scheduled some time in the future to establish that problems have been resolved and things are getting better. In order for the auditor to focus accurately on the audit objectives it is important that in planning the audit, the auditor does some background research to establish the levels of past performance in order to clearly and accurately report the “before” and “after” situation. This generally involves examining previous audit results and process performance indicators in advance of performing the audit
Process audits versus procedural audits
This contrast is primarily one of scope rather than focus. A procedural audit will generally be quite narrow in scope and will look in detail at the execution of a particular operation. A process audit has a broad scope and examines the broader context of the operation from inputs through to outputs and results. A process audit generally involves drawing evidence from a broader range of sources and people and requires more forethought and planning. A well written procedure more or less tells the auditor what sort of questions should be asked during the audit step by step, however when an auditor considers a process audit, there may be no single document that leads the auditor through the process in that step-by-step way. That means the auditor must establish the methodology through effective planning. Despite being generally a “bigger job” and more complex and time consuming, process audits generate information on the general efficiency and appropriateness of working methods. They can identify duplication, bottle-necks, delays, weaknesses in communication and confusion that procedural audits cannot pick up. A useful approach in auditing a process is to adopt the “grave to cradle” approach. That means to start the audit by looking at the results, then to use the findings from that stage to focus the audit trails for other parts of the process, tracking back to process inputs. This link will take you to a diagram that defines the “turtle diagram” approach to process auditing. Taking a central theme for the audit (in this case management responsibility &objective setting) the diagram offers a template to help the auditor explore generic themes associated with process control
This approach can be adopted for any process, as there will always be generic themes of objectives, measures, equipment, training, responsibilities, communications, methods and so on