Introduction

We are all in the Conformity Assessment industry - in the business of determining whether something (a thing, a service, an idea, an outcome) is 'good enough'. While the language may vary - we might grade, or verify, survey or inspect - any system of acceptance of product, or of grouping products into 'types', is an act of conformity assessment.

Sometimes the acceptance criteria are clear, public documents, such as regulations or standards. Sometimes they are complex technical specifications. Sometimes the stated requirements are incomplete, or there are implied requirements, while other times they are more nebulous - just ask three colleagues whether your clothing is appropriately formal, or whether dinner was 'good'.

Conformity assessment has been used for a long time, probably since humans started any form of trade or exchange, but as our technology has become more complex and our trading partners more remote, conformity assessment has played an increasingly important role. As this complexity developed, it has become apparent there are a number of types of conformity assessment, with varying levels of validity and varying costs and benefits. To assist in domestic and international trade, these types have been captured and described by the International Standards Organisation, and various committees have determined the actions and requirements to be met to provide a common level of understanding when speaking about conformity assessment.

This is critical in a global environment. Comparability and competition on equal technical terms demand that the bodies performing conformity assessment meet common, minimum levels. Customers, purchasers and regulators need to know that conformity assessment bodies are competent to do their job, both within a given economy and between economies. It is a stated objective of the World Trade Organisation that (within equivalence criteria) products should be able to move between economies without the need for extensive re-testing, re-inspection and re-certification. Once subjected to conformity assessment, a product should not need further work to satisfy equivalent requirements, anywhere.

In the global sphere there is a cluster of issues that mirror those associated with domestic conformity assessment. These are as valid in choosing a food critic as they are in choosing a weapons inspector, and they boil down to the following:

• Integrity - is there potential for conflict of interest?
• Definitions - assessment demands a common baseline - in measurement and vocabulary
• Capability - can the assessment discriminate between good enough and not good enough?
• Effectiveness - does the assessment give its buyer the risk minimisation they want?
• Efficiency - does the assessment minimise the time, money, resources needed?

Types and Definitions of Conformity Assessment

The most basic type of conformity assessment is the supplier's declaration of conformity and it is likely that the responsibility for assuring us of the conformity of the product is based upon a supplier's declaration. For most basic items, and for many highly standardised but high technology items, we accept the supplier's declaration of conformity eg. that the battery is, indeed, AA sized. Even in complex equipment, the supplier's declaration may be used, but usually in a context where the cost of product recall, means that suppliers have an interest in providing compliant product. To support the generation of the declaration there is an ISO Guide on the content of such a declaration " ISO/IEC Guide 22:1996 General criteria for supplier's declaration of conformity.

Without further support, the supplier's declaration of conformity has limitations. These may include questions about the competence of the supplier to make the claims, the basis or justification the supplier has in making the claims, or simply the credibility of the supplier in making claims regarding equipment about which they have a financial interest. Often the supplier's declaration of conformity is considered inadequate without further corroboration.

The next four types of conformity assessment offer more specialised tools that can be selectively used to underpin or substantiate the supplier's declaration. Because of the more specialised focus of these types their selection must be judicious, and take into account the diverse strengths and weaknesses of the different types of conformity assessment.

Thus we may require that personnel performing some activities be approved or certified in such a way that their competence can be readily determined. Such systems of personnel certification are widespread, but are not always operated by transparent organisations. Examples could include licensing of drivers, tradespeople and professionals, certification of skilled or competent personnel, and approval systems operated by larger manufacturers with specialist products eg. the installation of certain membrane systems. Personnel certification attempts to limit the performance of defined tasks to those who have demonstrated a competence (being the combination of education, aptitude and experience). It can be stringent or lax, and it may be expensive to operate its governing body. Again the ISO committee has developed a standard for personnel certification, ISO/IEC 17024:2002 General requirements for bodies operating certification schemes for persons.

Unfortunately, there are many situations where the skills of the person have a limited impact upon the outcomes. This may be because of the size or complexity of the project, or the level of mechanisation involved. Between the 1900s and the 1960s people began to think methodically about the systems within which personnel operated. From these early beginnings was developed a common way of considering the management and systems within organisations. Certain recurring themes, standardised procedures, control, error trapping and resolution, suggest systems common across a wide range of industries. A series of documents were developed that standardised the approach to these management systems. It is possible to certify that a management system is applied within a company, offering interested parties added assurance that the systems promote the realisation of uniform product. Management systems are a two-step process, with ISO/IEC 9001:2000 Quality management systems - requirements being applied to the producer organisation, while the certifier body is supposed to operate to ISO Guide 61:1996 General requirements for assessment and accreditation of certification/registration bodies. There are also "special interest" management systems available, with a focus upon, for example, environmental or safety management systems.

So far we have not considered any conformity assessment that characterises the product, so it is appropriate that we turn to the next two types of conformity assessment.

Testing is a means of determining specific characteristics of a product. A test involves following a defined procedure and reporting the outcomes. Testing brings a "scientific" and objective approach, with technology providing often astonishing powers of characterisation. Nonetheless, testing can be expensive, it cannot be exhaustive, and may not reflect the characteristics of interest, the performance in the field or even, in extreme conditions, the characteristics of the population that the tested sample was drawn from. Testing demands tight control over the testing operation so as to minimise outside influences and to assure repeatability of results. If we consider the range of influences that may cause variation in testing results, we find that almost all aspects of the operation of a testing laboratory can have an effect, and that to be credible a testing laboratory must have:

• qualified and experienced staff;
• properly calibrated equipment;
• an adequate quality assurance system;
• proper sampling procedures;
• sound testing procedures;
• valid test methods;
• traceability of measurements to national and international standards;
• accurate recording and reporting procedures; and
• suitable testing facilities.

Clearly the previous systems of conformity assessment don't go far enough, and a separate standard has been developed for the operation of testing laboratories: ISO/IEC 17025:1999 General requirements for the competence of testing and calibration laboratories.

Inspection is a form of conformity assessment that relies upon sensory evaluation and professional judgement, accompanied where necessary by measurement and indicative testing. Inspection offers a different approach to assurance. Where testing characterises a product through a number of precisely defined features, inspection can bring a comprehensive approach that addresses more general issues. Inspection is usually not destructive of products, and inspection can confirm that processes are followed, dimensions and specifications are met and that the intent of codes or regulations is fulfilled. Because of the human element, inspection can be expensive, unreliable and incomplete. Almost every inspector has to judge 'good enough' on the basis of incomplete data. Nevertheless, inspection is adaptable to different situations, conditions and requirements and, by virtue of its human element, can serve to bring an industry together.

If we consider the range of influences that may cause variation in inspection results, we find a list almost identical to that for laboratories. To be credible an inspection body must have:

• qualified and experienced staff;
• appropriate equipment;
• an adequate quality assurance system;
• proper sampling procedures;
• sound inspection procedures;
• traceability of measurements to national and international standards;
• accurate recording and reporting procedures; and
• systems to safeguard inspector integrity and probity.

Again, the previously considered systems of conformity assessment don't go far enough, and a separate standard has been developed for the operation of inspection bodies: ISO/IEC 17020:1998 General criteria for the operation of various types of bodies performing inspection.

Finally, for high integrity products, there may be demand for additional types of product certification. Product certification is an assertion from an outside provider that serially produced products are compliant. In order to make this assertion the product certifier usually calls upon a range of other conformity assessment techniques, including testing, inspection and management systems certification. Systems of product certification can include design appraisal, type testing, batch testing, 100% testing, testing of items from stock or market. Assurance can be enhanced through additional systems audits and product (and production) inspections.

While no guidance is offered on the selection between various product certification schedules, clearly the level of confidence possible will be related to the type, frequency and nature of the interaction. Product certification bodies are expected to operate in accordance with ISO Guide 65:1996 General requirements for bodies operating product certification systems and develop rules under which the product certification system operates. The certification applies these rules, normative product standards and possibly management systems standards to the product manufacturer.

Overlaps and Similarities Between Types of Conformity Assessment

It would be nice to believe that these types of conformity assessment were self-explanatory and were so clear-cut in their differences that there could be no confusion. Regrettably, the real world doesn't work like this and there are tremendous overlaps in the types of conformity assessment necessary. Where, for instance, would one place a workshop that overhauls and inspects, and then certifies the compliance of the overhauled product? Which conformity assessment type is 'best'? Even when we move away from the ISO structures and documents, the basic principles of the division of conformity assessment may still be identified.

It is also often found that a range of conformity assessment activities are called upon for a given product or service. As an example, for a shipment of metal there may be chemical tests and physical tests performed, a supplier's declaration regarding the processing and heat treatment of the metal and a pre-shipment inspection of the goods to establish compliant form, quantity, finish and packing. Perhaps the inspectors can also halt shipment if they have doubts regarding the testing results. The conformity assessment activities are normally selected on the basis of suitability, economy and risk.

It is fair to argue that for those activities where the output is information about a product, rather than a transformation of a product or about an underlying /contributing system, some additional attention is necessary in order to underpin the reliability of the result. For these sorts of conformity assessment bodies the influence of systems, personnel and physical resources are such that an integrated approach is essential.

Choosing the Conformity Assessment That is Most Suitable

Each of the various types of conformity assessment have differing strengths and weaknesses. Purchasers of products and services must consider their particular needs and the respective strengths of the differing conformity assessment systems when specifying, to ensure that they receive what they want. In writing specifications the levels and types of conformity assessment must be established. It may be poor practice and sometimes impossible, to revise the conformity assessment requirements once a project is under way.