ISO 9001 Registration for Small and Medium-Sized Software Enterprises 9780773573970

Citation preview

IS0 9001 Registration for Small and Medium-Sized Software Enterprises

IS0 900 1 Registration for Small and Medium-Sized Sofhvare Enterprises

Antonio J. Bailetti and

Chris FitzGibbon

CARLETON UNIVERSITY PRESS

Copyright O 1995 Government of Canada ISBN 0 88629 255 7 Published by Carleton University Press in collaboration with and support from Carleton University Development Corporation. This publication was prepared by the Research Unit in Telecommunications Technology Management, Carleton University, under contract to Industry Canada. Printed and bound in Canada by The Coach House Printing Company. Published by: Carleton University Press 160 Paterson Hall, 1125 Colonel By Drive Ottawa KlS 5B6 Canada

Distributed by: Oxford University Press 70 Wynford Drive Toronto M3C 1J9 Canada (416) 441 294 1

(613) 788 3740

The authors acknowledge the generous support of Industry Canada and the Carleton University Development Corporation in the development of this book.

CATALOGUING IN PUBLICATION DATA Bailetti, Antonio J. (Antonio Jose), 1948IS0 9001 registration for small and medium-sized software enterprises Includes bibliographical references and index. ISBN 0-88629-255-7 1. Software engineering - Management. 2. Computer software - Quality control. 3. Computer software - Standards. I. FitzGibbon, Chris, 1970- 11. Title.

Contents

I Preface

9

Purpose Benefits of This Book How to Use This Book Highlights of IS0 9000 Principles Underlying IS0 9000 Software Developers and IS0 900 1 Registration Benefits of IS0 9001 Registration How This Book was Developed I1 Phase-by-Phase "How To" Approach to IS0 900 1 Registration

16

Phase 1: The Start Phase 2: Selection of a Registrar Phase 3: Preparation for the Audit Phase 4: Pre-Audit Assessment Phase 5: The Audit Phase 6: Post-Audit Compliance Phase 7: IS0 9001 Registration Phase 8: Annual Surveillance Visits I11 Answers to Frequently Asked Questions What are the critical factors for successful IS0 9001 registration? How much does registration to IS0 9001 cost? How long does it take to register to IS0 9001? What are the criteria for selecting a registrar? What documentation will the audit team require? How can software quality be measured? How long does registration last? Can the firm be delisted? Is a consultant needed?

33 33 34 36 39 40 43 45 45 45

What are the differences among the Software Engineering Institute's Capability Maturity Model, IS0 900 1, and the Malcolm Baldrige National Quality Award? 46 What courses are available? 48 What is a bilateral MOW? 49 What is TickIT certification? 49 What is ITQS? 50 Should IS0 9001 affect the company's other QMS? 51 How does a company build evidence of a disciplined software development process? 51 What types of non-compliances notices are there? 52 Must IS0 9001 registered companies comply with revisions to the standard? 52 What are some complaints about IS0 9000? 52 IV Guidelines for the Application of IS0 9001 to Software Development

Quality Management System - Framework Quality Management System - Life Cycle Activities Quality Management System - Supporting Activities V Sources of Information

Accredited Registrars To Obtain IS0 Standards or a List of Accredited Registrars Publications Other Sources of Information VI Appendices Appendix A: Nations That Have Adopted IS0 9000 Appendix B: International Standards on Quality Appendix C: The 20 Requirements of IS0 900 1 ( 1994) Appendix D: IS0 9001/9000-3 Coverage of Software Development Processes Appendix E: Implementation Schedule (Amita Management & Information Technology Adaptors Corp.)

62 62 63 64 66 68 68

70 71 72 73

Appendix F: Typical Components of a Software Company's QMS Documentation Appendix G: QMS Documentation Table of Contents (Software Kinetics Ltd.) Appendix H: The Documentation Hierarchy of a Typical QMS Appendix I: Metrics Template Appendix J: The 5 Levels of the SEI Capability Maturity Model Appendix K: The 7 Elements of the Malcolm Baldrige National Quality Award

VII Glossary VIII Bibliography

IX Index

74

75 77 78 79 80

I Preface

Purpose This book is for executives, quality managers, and project leaders of small and medium-sized enterprises who design, develop, maintain, and evolve software. It provides a comprehensive approach to IS0 9001 registration using IS0 9000-3 guidelines. The phase-by-phase approach incorporates lessons learned by managers of Canadian software companies registered to IS0 9001. For each phase of the registration process, the book provides directives to executives, quality managers, and IS0 9001 project leaders, highlighting what they should focus on. Quotes from quality management system (QMS) auditors and consultants with IS0 9000 registration experience serve as reminders of the lessons learned by others. The book provides answers to questions frequently asked by managers of Canadian companies that have not yet registered to IS0 9001. It also includes guidelines for the application of IS0 9001 to software development; sources of information about registrars and publications; a list of nations that have adopted IS0 9000; a list of international standards on quality; the 20 requirements of IS0 9001; IS0 9001 and IS0 9000-3 coverage of software development processes; an example of an implementation schedule; the table of contents of a sample QMS; the documentation hierarchy of a typical QMS; a metrics template; the five levels of the SEI Capability Maturity Model; and the seven elements of the Malcolm Baldrige National Quality Award. The book includes the revisions made to the standard in 1994.

Benefits of This Book This book assists users to: reduce the time required to register; identify and prioritize the activities required to plan, execute, and complete the registration process; assess the costs of registration;

make interactions with consultants and registrars more productive; and increase the likelihood of success the first time the firm attempts IS0 900 1 registration.

How to Use This Book To use this book: establish an IS0 Steering Committee with representatives of all the functions affected by IS0 9001; examine Section II: Phase-by-Phase, "How to" Approach to IS0 9001 Registration; and examine Section 111:Answers to Frequently Asked Questions. The IS0 Steering Committee should then assign the necessary fact-finding tasks, get the individuals who design and develop software to become involved in the process, and meet frequently to review progress. For the registration process to be successful, the IS0 Steering Committee must be led by a senior executive who is committed to the goal. The Committee must appoint a "management representative" as required by the standard. The management representative, referred to in this book as the IS0 Coordinator, will lead and monitor the preparation and registration of the QMS to the IS0 9001 standard.

Highlights of I S 0 9000 IS0 9000 is a series of international quality management standards adopted by the 71 countries listed in Appendix A. Many large corporations require their suppliers to be registered to an IS0 9000 standard. Registration to an IS0 9000 standard is becoming a prerequisite for doing business for second- and third-tier suppliers. IS0 9000 standards provide guidelines on quality management and assurance. The focus is on the QMS of a supplier's engineering and production processes. This focus was selected in the belief that a highquality process results in the production of high-quality goods and services. Registration to an IS0 9000 standard provides third-party assurance that

PREFACE

11

a supplier has a documented QMS that satisfies the standard's requirements, and that the IS0 program has been implemented at the company's facility. IS0 9000 quality standards are not specific to any industry; rather, they are general to any manufacturing process. Their application to any particular business is subject to interpretation. IS0 9000 quality standards apply to companies of all sizes. IS0 9000 quality standards emphasize achieving customer satisfaction through prevention of non-conformance rather than through testing. IS0 9000 quality standards apply to the QMS of the engineering and production processes; they do not certify the specifications or performance of products. The implementation of IS0 9000 standards is driven by a commitment to quality management, continuous improvement, compliance requirements, reliability needs, and procurement demands. The IS0 9000 series contains a total of five documents, three of which describe quality standards in different domains: I S 0 9001: Quality Systems - Model for Quality Assurance in Design, Development, Production, Installation, and Servicing; IS0 9002: Quality Systems - Model for Quality Assurance in Production, Installation, and Servicing; and IS0 9003: Quality Systems - Model for Quality Assurance in Final Inspection and Test. The other IS0 9000 documents, IS0 9000 and IS0 9004, are guidelines for the selection of the appropriate standard and its use. Both IS0 9000 and IS0 9004 contain several parts that are advisory in nature and contain no additional requirements. Appendix B lists international standards on quality.

Principles Underlying I S 0 9000 The principles underlying IS0 9000 quality standards are:

Say what you do Do what you say

Document each step in your company's business process Ensure that all processes adhere to written procedures

Show what you have done

Verify

Document evidence that your QMS meets IS0 requirements and that the quality standard is being implemented effectively Conduct periodic internal audits to ensure continued suitability, compliance, and effectiveness of the QMS

SofhYare Developers and I S 0 9001 Registration Software companies that wish to register their QMS to an IS0 9000 standard must implement the IS0 9001 standard requirements outlined in Appendix C. The IS0 9001 standard is the most comprehensive quality standard in the IS0 9000 series. It applies to companies that wish to register their design, development, and production processes. Registration to IS0 9001 certifies that a company's QMS and the processes used to design, develop, produce, install, and service a product or service meet the 20 requirements specified in Appendix C. "High-quality software is not produced by accident; it has to be planned from the start of the project, and the characteristics of quality must be built into the product. It is no good producing a system, discovering major errors at the testing stage and then trying to correct them to produce a quality product; quality cannot be added as an extra ingredient at the end of a project." Darrel Ince, Introduction to S o w a r e Project Management and Quality Assurance. London, UK:McGraw-Hill, 1993, p. 169. IS0 9000 quality standards are created to provide generic quality management standards that could apply to any company in any industry. Software development, however, requires special consideration. The production process in software is a relatively insignificant part of the total development effort. Definition of requirements, as well as design, implementation, maintenance, verification,and validation, account for a larger share of development activities. Such needs led to the introduction of IS0 9000-3. The objective of IS0 9000-3 is to facilitate the implementation of IS0 9001 requirements to the software engineering process. IS0 9000-3 does not add any further requirements to those in IS0 9001. IS0 9000-3 is a

document created to assist companies in interpreting IS0 900 1 requirements within the context of software. IS0 9000-3 focuses on such items specific to software development as joint reviews, acceptance testing, and configuration management. Appendix D shows how the various elements of IS0 9001 and IS0 9000-3 relate to one other. The issuance of a revised IS0 9001 standard in August, 1994 made the 1991 edition of IS0 9000-3 obsolete. It is not clear when a replacement to IS0 9000-3 will be issued.

Benefits of I S 0 9001 Registration The benefits that managers of software firms associate with IS0 9001 registration include: increased efficiency of the company's internal operations; clearer definition of responsibility and accountability; better traceability of quality problems to their root causes; "At DY 4, IS0 9001 has actually decreased the number of procedures such as needless signatures and approvals ... Therefore, registration improved our efficiency significantly." Dave Pedley, DY 4 Systems Inc. less time required to fix an error; fewer recurring errors; "Following IS0 9000 registration, IBM was able to cut two months off some projects' cycle time because of a faster start-up through reuse and easy retrieval." David Shier, Software Quality Assessment Group, IBM Canada Ltd. lower costs and increased productivity; improved reuse practices and easier retrieval of software components; fewer procedures - elimination of unnecessary approvals and redundant work practices; increased designer self-discipline; increase in the number of activities that are performed properly on the first try; reduced number, cost, and scope of customer audits; greater access to foreign markets that require IS0 9001;

14

IS0

9001 REGISTRATION

FOR SMALL AND MEDIUM-SIZED SOFTWARE ENTERPRISES

"It has come to the point in the UK that software producers without IS0 9000 are regarded with some suspicion." Michael Jaques, Prior Data Sciences Lid. greater likelihood of winning government contracts; improved ability to compete with larger, more established, software firms; greater customer confidence in the company's products; broader exposure to new clients; and competitive advantage over those firms not registered to IS0 9001. Many of the claims of benefits from IS0 9001 registration are unsubstantiated because of the small number of software companies registered to date.

How This Book Was Developed This book features a practical, easy-to-use approach. "How To" Approach The book incorporates the lessons learned by Canadian software companies registered to IS0 9001 quality standards. While it describes a sound, practical process, there may be other, equally valid approaches to IS0 9001 registration. Contributors Executives who championed the IS0 9001 registration effort, together with software quality consultants and IS0 9000 auditors, helped develop Section II of this book. Executives from several small and medium-sized companies that had not yet registered to IS0 9001, but were considering doing so, generated the list of frequently asked questions included in Section III. IS0 9000 auditors, consultants, government officials, and quality managers reviewed drafts of the book.

This book has benefited from comments provided by Bob Rand of Industry Canada, whose assistance is gratefully acknowledged.As well, valuable insights and suggestions were provided by: Bob Armstrong, DMR Group Inc. John Birke, IBM Canada Ltd. Archie Bowen, CompEngServ Inc. Gary Dean, Simware Inc. Micheline Gray, Amita Management & Information Technology Adaptors Corp. Michael Jaques, Prior Data Sciences Ltd. Brian Kohn, certified Quality Systems Lead Auditor Sonny Lundahl, Amita Management & Information Technology Adaptors Corp. Mark Miller, Lead TickIT Auditor Peter Papakostantinu, Papa & Associates Inc. Dave Pedley, DY 4 Systems Inc. Malcolm Phipps, Quality Management Institute Barry Purvis, CompEngServ Inc. Lane Smith, Software Kinetics Ltd. Jean White, Qualidoc Software Canada Ltd. Aulis Viik, Gallium Software Inc. Research The academic and professional literature, publications issued by the registrars, and publications provided by national and international standards bodies were reviewed. Key directives from these studies were identified and included in the book.

I1 Phase-by-Phase, "How To"Approach to I S 0 9001 Registration

Companies that develop software are interested in the assessment and registration of their QMS and engineering processes to IS0 9001 using IS0 9000-3 guidelines for software. There are several accredited organizations, called registrars, that assess and register companies' QMS and engineering processes. An accredited registrar is any organization formally recognized by a national accreditation body as competent to register QMS to IS0 9000 standards. The registrar employs auditors to perform this task. Of the accredited Canadian registrars, Canadian General Standards Board (CGSB), Litton Systems Canada, Quality Management Institute (QMI), SGS International Certification Services Canada (SGS), Groupement Quebecois de Certification de la Qualite, and Warnock Hersey Professional Services actively seek to assess the QMS of software companies. To register companies to IS0 9001 for software, all accredited registrars follow a similar process. From the company's perspective, it is important that the auditors who work for the registrar be knowledgeable in software development and IS0 900 1 auditing. This section provides a comprehensive, phase-by-phase, "how to" approach to achieve IS0 9001 registration using IS0 9000-3 guidelines. The approach incorporates the lessons learned by several Canadian software companies that have successfully registered to IS0 9001 quality standards. It also benefits from comments made by quality systems auditors and consultants with expertise in IS0 9000. Directives for executives, quality managers, and project leaders highlight the focus at each phase of the registration process.

Phase I : The Start The registration process starts when a senior executive recognizes the need to register the company and makes the decision to pursue IS0 9001 registration. Shortly afterward, an IS0 Steering Committee is established, an IS0 Coordinator is appointed by a senior executive of the company, and the IS0 900 1 Registration project begins.

PHASE-BY-PHASE "HOW TO" APPROACH TO I S 0 9001 REGISTRATION

17

I S 0 Steering Committee In small companies, the IS0 Steering Committee comprises 1) the IS0 Coordinator; 2) those who design and develop software; and 3) individuals in functions that may be affected by IS0 9001 registration. When the size of the company makes the inclusion of all staff impractical, the IS0 Steering Committee should represent only the stakeholders. The IS0 Steering Committee assigns the responsibility for IS0 9000 to a champion within each function of the company.

I S 0 Coordinator A senior manager is best suited for the role of IS0 Coordinator. This person becomes the management representative, and thus must have the authority and responsibility to oversee the effective implementation and maintenance of the QMS. The IS0 Coordinator will mainly be responsible for 1) the translation of the IS0 900 1 standard into the company's documentation and procedures; 2) the preparation of a schedule and plan for the IS0 9001 Registration project; 3) obtaining executive approval for the required resources; 4) the buy-in of the software developers; and 5) negotiations with consultants and registrars. IS0 9001 requires the supplier's quality management executives to appoint a member of their own team as the management representative. If possible, the IS0 Coordinator is assigned full time to the IS0 Registration project. In some cases, the IS0 Coordinator may also have other management responsibilities. These should not include operating or production responsibilities that would conflict with the autonomous authority required by the IS0 9001 standard. I S 0 9001 Registration Project The company's effort to register should be managed as a project. IS0 9000 registration is best approached through a project plan, timetables, a budget, milestones, and progress reports. Directives for Executives, Managers, and Project Leaders 1. Always place the company's customers first! The achievement of quality is the primary objective of the IS0 9001 standard.

18

I S 0 9001 REGISTRATION FOR SMALL A N D MEDIUM-SIZED SOFTWARE ENTERPRISES

"You must have a champion or I S 0 registration won't happen." MichaelJaques, Prior Data Sciences Ltd. 2. Senior management's understanding and commitment are prerequisites for successful I S 0 9001 registration. A lack of management commitment is the most commonly identified obstacle to registration. 3. Top management must lead the registration effort. Procedures will change, and the authority to overcome barriers to change will be needed.

"The whole process o f seeking and obtaining IS0 recognition for an organization i s doomed without solid senior management support. You would be wasting your time without it." Lane Smith, SoftwareKinetics Ltd. 4. Secure support for the I S 0 9001 registration project from all levels of management. The company must encourage all its employees to think positively if it hopes to register successfully.

5. Have a dual focus: process improvement and registration. Build momentum for process improvement before attempting registration. "Grass-roots efforts are nice, but the kind o f work involved in I S 0 9001 compliance often requires the clout that only top management possesses.'' Brian Charles Kohn, certijied Quality Systems LeadAuditor 6. Reinforce the belief that disciplined software development, process improvement, and a concern for quality are good for the company and that registration benefits all employees. Sell I S 0 9001 for its benefits and avoid malicious compliance.

7. Ensure that all members of the I S 0 Steering Committee become familiar with IS0 9000-3:The Guidelines on the Application of IS0 9007 to the Development, Supply and Maintenance of Software. Training may be

PHASE-BY-PHASE "HOW

TO" APPROACH TO I S 0

9001 REGISTRATION

19

required to increase the Committee's understanding of the standard and how it applies to the company. 8. I S 0 9001 registration is a means to an end, not an end in itself.

Phase 2: Selection of a Registrar During Phase 2, the IS0 Steering Committee determines the scope of business activity to be registered, identifies the criteria for selection of possible registrars, gathers information, and selects a registrar while the IS0 Coordinator reviews and upgrades the QMS. Scope of Business Activity to Be Registered The statement of scope is a generic description of which company processes are covered by the IS0 9000 registration. It requires no specific format or wording. For example, Software Kinetics Ltd. identifies its scope of business activity registered under IS0 9001 as "software development, systems engineering and systems integration." For DY 4 Systems Inc., the scope of business activity registered is "design and manufacture of industry-standard, board-level products and related s o h a r e to defence and aerospace system integrators." Upon registration, a company's scope of activity will appear on the registration certificate and often in directories of other registered companies. Criteria for the Selection of a Registrar Selecting a registrar is no different from choosing a sub-contractor: select three or four from a list of suppliers, and ask them to tender according to the requirements. The following criteria have proven helpful in selecting a registrar: 1) customer credibility; 2) qualifications of the auditors and familiarity with software; 3) quality of service; 4) cost; and 5) location. Further criteria are listed on page 39. Gathering Information Once the IS0 Steering Committee determines the scope of the business activity to be registered and the criteria for selecting the registrar, the IS0 Coordinator then 1) issues requests for proposals (RFPs) to three or four

20

IS0

9001 REGISTRATION

FOR SMALL AND MEDIUM-SIZED SOFTWARE ENTERPRISES

registrars; 2) requests information on the qualifications of the registrars' audit teams and processes; and 3) provides information on the company to the registrars. The RFPs issued to the registrars should describe the company's products or services and request quotes for all stages of the registration cycle, including the submission of the application, the documentation review, the registration audit, any corrective follow-ups, annual surveillance visits, and all directly related expenses. Registrars supply companies with the following information: 1) a detailed description of the assessment and registration procedures; 2) documents that contain the requirements for registration; and 3) documents that describe the rights and duties of registered suppliers, including fees to be paid by applicants and registered suppliers. Registrar Selection The Standards Council of Canada is the national accreditation body that recognizes an organization as competent to register QMS to IS0 9000 standards. To be formally recognized in Canada, a registrar must comply with the requirements set out in CAN-P-10, National Standards System Criteria and Procedures for Accreditation of Organizations Registering Quality Systems. The names and addresses of accredited registrars are included in the section of this book entitled Sources of Information, beginning on page 62. Each registrar, after a review of the RFP and the scope of business activity to be registered, presents the company with a proposal for services and a price for these services. Normally, the IS0 Steering Committee meets with one or more of the registrars to clarify issues. The IS0 Coordinator should find out who in the registrar's staff would be the company's contact person and then meet with that person before making a decision. Based on all the information, the IS0 Steering Committee selects the registrar, then arranges for an official application form to be sent to the registrar's office, along with all applicable fees and deposits. QMS Upgrade Good people do good work, but they perform even better within an organized framework. The process of developing a QMS requires documentation to be effective: the system must be one that the engineers respect and

that allows them to see the improvement in work methods. If your company does not presently have a QMS, then resources must be assigned to develop one. The book by David Smith and Kenneth Wood, Engineering Quality Software (New York: Routledge Chapman Hall, 1989) is a good reference that describes how to develop a QMS. Directives for Executives, Managers, and Project Leaders 1. Software development differs considerably from manufacturing. Keep this in mind when searching for consultants, registrars, and information on I S 0 9001.

"Our criteria for registrar evaluation were: 10% Location 20% Cost 70% Familiarity with the industry." Dave Pedley, DY 4 Systems Inc. 2. Determine the scope of the business activity to be registered. 3. Determine the criteria to select a registrar, and start the process of selecting one well in advance of registration. 4. Obtain information on I S 0 9000 registration services from several registrars, and meet with representatives of at least two. 5. Select the registrar that best meets the needs of the company and its

customers. 6. Focus on developing an effective QMS that serves the company first and serves I S 0 9001 and its auditors second.

Phase 3: Preparation for the Audit During Phase 3, the IS0 9001 Registration project is detailed, employees are trained, internal auditors are appointed, work procedures are documented, and the quality manual and the master list of the QMS documentation are developed and assessed by the registrar.

22

IS0

9001 REGISTRATION

FOR SMALL AND

MEDIUM-SIZED SOFTWARE ENTERPRISES

Detailing the I S 0 Project Following the selection of a registrar, the company's IS0 Steering Committee and the registrar agree on the details of the IS0 9001 Registration project. The activities to be undertaken are identified, as well as the individuals responsible for performing them, and a schedule is established. Sufficient resources should be allocated to undertake the following: 1) reorganization of the quality manual; 2) inclusion of non-software development activities in the QMS; 3) addition of detail to operating plans; 4) clarification of quality management roles and responsibilities; 5) introduction of changes to operating procedures; and 6) process improvement. The IS0 Steering Committee should hold regular status meetings where minutes are taken. The minutes should be distributed throughout the company. Internal Auditors The IS0 Steering Committee appoints the internal auditors and ensures management's support for their work. Internal auditors are company employees who are qualified or have been trained to conduct internal QMS audits. It is a requirement of IS0 9001 registration that internal auditors conduct periodic, planned audits to determine the effectiveness of the company's QMS, to ensure that activities follow prescribed procedures, and that QMS procedures accurately represent activities. Internal auditors evaluate work practices for completeness, effectiveness, and quality. Internal audits follow a schedule and are documented. Responsibility for corrective action is assigned by the IS0 Steering Committee. Corrective action, as it is called in the standard, identifies what went wrong and how a current process could be improved to prevent future undesirable outcomes. Follow-up audits must verify the effectiveness of the corrective action taken. A process to record and correct self-identified problems is established by the IS0 Coordinator. I S 0 9001 Training The IS0 Coordinator should prepare a training and implementation schedule for all employees involved in the engineering process used to develop software. Training usually starts with a short seminar on the benefits of registration and how it will affect employees (e.g., jobs, future contracts).

IS0 9001 training should be provided to everyone so that expectations are clear. This training should continue until the company is registered to IS0 900 1, and as needed thereafter. The IS0 Coordinator should strive to learn from those who have already registered to IS0 9001. If possible, company personnel should visit registered sites so that they can benefit from the experiences of others. Documentation and Procedures The IS0 Coordinator establishes templates, standards, and tools so that all documentation and procedures follow a common format. A sentence-bysentence traceability matrix should be prepared that relates company documentation and procedures to IS0 9001 requirements. The traceability matrix should be used to identify 1) the IS0 9001 requirements that are fully covered by the existing documents and procedures of the company; 2) the requirements that are partially covered; and 3) those that are not covered. A company may also wish to prepare a traceability to IS0 9000-3, but this is not mandatory. The work procedures should be documented by the people who do the work. It is the IS0 Coordinator's responsibility to ensure that all procedures are readily available to the people doing the work, and that the procedures are accurate. Quality Manual Have staff write the quality manual, or, if a consultant is given this responsibility, ensure that staff 1) play an active role in its creation, and 2) verify the written procedures for accuracy. Staff must be involved in the development and evolution of the procedures to ensure a sense of ownership and acceptance of the manual. Consultants may also assist in the preparation of a plan of action, the review of the management system, and the supervision of the production of the QMS procedures. "No one knows your process better than your own people. Why pay consultants money to learn the process your people already know?" Lane Smith,Software Kinetics Ltd. Do not write too much. The procedures should be written to the employees' level of capability. There is a tendency to complicate documentation. The objective of the QMS is to help staff do their jobs, not to be an imposition.

Documentation should be divided by department or function into easily usable local manuals. Buy, borrow, or re-use existing materials, documents, and processes whenever possible. The IS0 Coordinator issues the IS0 9001-compliant Quality and Procedures Manual, then tracks and updates the number of copies to ensure that outdated copies are not kept in circulation. Amendment and revision records must be maintained. If an electronic medium is used, establish a policy to control printed copies. New work practices such as IS0 900 1 record keeping, vendorlsubcontractor assessments, and internal audits may have to be introduced to facilitate IS0 900 1 registration. Assessment of Documentation The IS0 Coordinator sends the company's quality manual and the master list of the QMS documentation to the registrar. The registrar reviews the information and assesses its compliance with IS0 9001 requirements. The registrar may require revisions to the quality manual and procedures. These are made by the IS0 Steering Committee and department managers. Directives for Executives, Managers, and Project Leaders 1. Understand the details of the I S 0 9001 Registration project and monitor its progress. Identify major areas of weakness and address the most difficult problems first. Use internal audits to provide specific feedback.

"It took about three years to move from: "We have to do this as well as our jobs?" to "IS0 9000 is integrated and helps us with our business." Geof Kelland, Northern Telecom Canada Ltd. 2. Assure employees of management support. With such support, many

corrective actions can be solved by the employees themselves. "If there is a conflict between a program and quality, drop the program. I would rather have fewer programs of high quality than many with no quality." Terry Curtis, Bell-Northern Research

3. Do not allow audits to delay process improvement. Involve customers and developers in process improvement.

"[QMS]training is very important. No matter how good the QMS, if staff are not appropriately trained in its use, there will be problems or even total failure." TickIT Guide, Section 4.2.5 4. Provide I S 0 9001 training so that all staff understand their roles and responsibilities. Training is required for the I S 0 Steering Committee and all personnel who must change their work processes in order to comply with I S 0 9001. 5. Large volumes of documentation have proven to be an obstacle to eficiency. Each document must have a purpose and add value to the QMS. --

--

- -

"Corrections to documents, unless the corrections are very minor, should be submitted to the registrar for review prior to the registration audit." Peter Papa kostan tinu, Papa & Associates Inc. 6. The audit should measure how well the company complies with its own documentation and how well that documentation reflects the activities of the company. Beware of idealizing your business processes or presentingthem the way you think the auditor would like to see them. 7. Do not throw out the existing QMS. Do not attempt to build the

company's QMS around the standard. 8. All staff should be kept informed of the development of the QMS and trained in its use from the start. -

"DY4 thoroughly reviewed documentation with the registrar prior to the audit. It would be too late to start discussing documentation issues during the audit!" Dave Pedley, DY 4 Systems Inc.

26

I S 0 9001 REGISTRATION FOR SMALL AND MEDIUM-SIZED SOFTWARE ENTERPRISES

9. Ensure that the company's documentation and procedures comply with IS0 9001 requirements and that all new work practices necessary to meet IS0 9001 requirements have been introduced. 10. Pay attention to quality management activities not related to the software. For example, include in the process development labs that calibrate test equipment, anti-static measures, etc.

Phase 4: Pre-Audit Assessment The company may choose to conduct a pre-audit assessment in preparation for the official audit. This phase may be omitted by companies that feel they are ready to be audited by the registrar. In a pre-audit assessment, a consultant or an audit team visits the company and identifies areas of noncompliance with the standard. Registrar or Consultant? A pre-audit assessment may be performed by a registrar or a consultant. Each offers a considerably different service. Registrars can identify areas of non-compliance, but cannot make suggestions for bringing them into line with the standard. Consultants can identify areas of non-compliance, as well as advise how to incorporate the I S 0 9001 standard into the company's QMS and engineering processes. "The consultant's job is to get to know the company processes and to define and document the current quality system. The next task will be to consult company principals and others whose responsibilities and authorities are defined in the Quality Policy manual and set in place the procedures under each component of the standard as it applies to that company. I believe that a consultant is shirking their responsibility if this is not done." Jean White, Qualidoc Ltd. Walk-through or Trial Audit? A pre-audit assessment can range from a one-day walk-through with the auditor or the consultant, to a trial audit. Walk-throughs provide immediate verbal feedback. Trial audits give a sense of what the official audit will be

PHASE-BY-PHASE "HOW

TO" APPROACH TO I S 0

9001 REGISTRATION

27

like, as well as a better understanding of how the auditors apply the standard in the company's context. A trial audit is most beneficial when a QMS has been formally documented. Documentation Review The IS0 Coordinator ensures that the company's QMS documentation is accurate. The auditor reviews the documentation, office layout, organizational chart, quality manual, and information on the company's products and services. On rare occasions, the IS0 Coordinator and the auditors may agree that the document review should be performed in the company's premises. In this case, the auditor can request and immediately obtain further documentation. On-site document reviews give the company the opportunity to learn more about how the auditor applies the standard. The auditors review the company's documentation, determine to what extent it complies with the IS0 9001 standard, and inform the company of any areas where it does not comply. The documents and procedures can then be corrected and made ready for another review or re-assessment at the beginning of the registration audit. If the auditors find that the company's documentation is poor, they may request that the quality manual and operating procedures be re-written before the registration process continues. Registrars usually will not conduct an audit until both the documentation system and the company staff are ready.

"We found it useful to have a pre-assessment done before the registrar performed their audit." Dave Pedley, DY 4 Systems Inc. Directives for Executives, Managers, and Project Leaders 1. Use consultants who are experienced in both software development and I S 0 9000 standards. The best consultants are former auditors or individuals who have championed the registration for their employer. 2. Ensure that the company's QMS documentation is accurate and that company staff are ready for the pre-audit assessment.

28

IS0

9001 REGISTRATION FOR SMALL AND MEDIUM-SIZED

SOFTWARE ENTERPRISES

"The company is responsible for having proved their quality system for at least three months before final assessment so that the auditors can find the evidence they are looking for.'' Jean White, Qualidoc Ltd. "A trial audit would be most beneficial when the QMS has been documented and there is a consensus in the company that what has been written in the procedures is what actually happens in day-to-day activity." Jean White, Qualidoc Ltd. 3. Ensure that the managers of the areas found not to comply with IS0 9001 requirements are held responsible for implementing a plan to remedy the situation.

"A thorough pre-audit containing an element-by-element assessment of the way your company does business forms the basis for a detailed implementation plan." John Birke, IBM Canada Ltd. 4. Ensure that all employees learn from the pre-audit assessment.

Phase 5: The Audit During Phase 5, the auditors perform a detailed, on-site audit of the company's QMS to ensure compliance with IS0 9001 and the documented system. The objective of the audit is to determine the degree to which the company adheres to its documented procedures. Audit Date The company's IS0 9001 Coordinator and the registrar's representative agree on the date for the audit. The audit date should be set only when the company is ready to be audited. If necessary, the audit can be postponed until the company has completed its pre-audit preparations. However, rescheduling may create considerable delays for those companies whose registrars have heavy workloads.

Audit Duration The duration of the audit depends on the size and complexity of the organization that seeks registration. A two- to three-day audit is typical for singlesite IS0 9001 audits. However, the audit can last up to a week.

The Audit Before the audit begins, the auditors meet with all key personnel. Each auditor is then assigned an escort who is knowledgeable about the company's organization, processes, and QMS. Before leaving the premises, the auditors meet with the company's management and the IS0 Steering Committee to provide an oral report describing the extent of the company's compliance with the IS0 9001 requirements. The auditors may also present the written report before they leave, or may send this a week or two after the audit. The audit report includes the results of the assessment and details of any non-compliance. Directives for Executives, Managers, and Project Leaders 1. Provide all employees with "audit awareness" training just before the audit.

2. Have a pre-audit meeting with all key personnel and auditors. This reassures

staff and gives all concerned a sense of confidence at the start of the audit. "The more questions the auditor is forced to ask to get basic information, the more frustrating the whole exercise becomes for all concerned. There is an optimum level of cooperation. No auditor expects the auditee to confess all the problems the company has and to implicate their colleagues.'' Mark Miller, Lead TickIT Auditor 3. Be honest and open during the audit, but do not offer information that has

not been requested by the auditors. If you don't know the answer to an auditor's question, say so. 4. Provide each auditor with an escort who can take notes during the entire

audit.

30

I S 0 9001 REGISTRATION

FOR SMALL AND MEDIUM-SIZED SOFTWARE ENTERPRISES

5. When audits are scheduled, make sure the right people are available and not absent for holidays or away on company business.

Phase 6: Post-Audit Compliance During Phase 6, the company undertakes the necessary corrective action to satisfy the auditors. If the audit report identifies areas of non-compliance, the company must describe the actions it will take to remedy these. Time Limits The registrar and the IS0 Coordinator set time limits for responses to areas of non-compliance. The company must take the necessary corrective action to satisfy the auditors before registration can proceed. A re-audit may be required, the size and scope of which depend on the severity of the noncompliance. Corrective Action The company responds to the notices of non-compliance by taking corrective actions, including an investigation of the cause of the problem, the implementation of preventive actions and controls, and the documentation of the changes. Directives for Executives, Managers, and Project Leaders 1. Don't fall into the trap of blaming the auditor. Work with the auditors, not against them. 2. It is not enough for auditors to say that they do not like a process. They must demonstrate how a firm's process does not comply with the I S 0 9001 standard. 3. Investigate the root cause of each notice of non-compliance, and implement corrective action. 4. Clearly describe to the auditors what action will be taken to remedy non-

compliance.

PHASE-BY-PHASE "HOW

TO" APPROACH TO I S 0

9001 REGISTRATION

31

Phase 7:I S 0 9001 Registration When the IS0 9001 requirements have been satisfied, the auditors notify the company and begin to prepare the registration documents. The registrar sends a congratulatory letter to the company and prepares to present the official registration certificate. Directives for Executives, Managers, and Project Leaders 1. Present the registration plaque at a meeting with all employees who were involved in the registration process.

"... Seeking IS0 9001 registration just to put a certificate on the wall doesn't get your project team through the tough times." Brian Charles Kohn, certijied Quality Systems Lead Auditor 2. Emphasize that I S 0 9001 is a continuous process of improvement, and

that the job is not over.

Phase 8: Annual Surveillance Visits Accredited registrars are required by most national standards bodies to perform periodic surveillance audits to ensure that the registered company continues to comply with the standard. In Canada, this requirement is set out in CAN-P- 10. The number of surveillance audits required is established by each registrar; some registrars hold one surveillance audit each year, while others require bi-annual surveillance audits. As a result of such audits, auditors can issue notices of non-compliance and deadlines for compliance. The 1994 revision to IS0 9001 has added the requirement to report on the performance of the QMS to company management for "review and as a basis for improvement of the quality system." Further, it clearly indicates that management is expected to seek improvement rather than accept stagnant quality systems. Normally, a complete re-audit will take place within three years of registration. A re-audit usually requires 70% of the work of the original audit. The company may seek RFPs from other registrars, and may select another registrar to perform the registration, if it wishes.

32

IS0

9001 REGISTRATION FOR SMALL AND MEDIUM-SIZED

SOFTWARE ENTERPRISES

Periodic surveillance audits and re-audits continue for as long as the company wishes to hold an IS0 9001 certificate. Directives for Executives, Managers, and Project Leaders 1. Continue to improve the company's QMS. 2. Gather data and facts to improve the software development process in a systematic fashion.

3. Encourage company-wide learning of quality improvement practices.

I11 Answers to Frequently Asked Questions

This section provides answers to the following questions: What are the critical factors for successful IS0 9001 registration? How much does registration to IS0 9001 cost? How long does it take to register to IS0 9001? What are the criteria for selecting a registrar? What documentation will the audit team require? How can software quality be measured? How long does registration last? Can the firm be delisted? Is a consultant needed? What are the differences among the Software Engineering Institute's Capability Maturity Model, IS0 900 1, and the Malcolm Baldrige National Quality Award? What courses are available? What is a bilateral MOU? What is TickIT certification? What is ITQS? Should IS0 9001 affect the company's other QMS? How does a company build evidence of a disciplined sofhvare development process? What types of non-compliance notices are there? Must IS0 9001 registered companies comply with revisions to the standard? What are some complaints about IS0 9000? What are the critical factors for successful I S 0 9001 registration? The critical factors for successful IS0 900 1 registration are: Senior management support. Senior management must be directly involved in the registration process, obtain a commitment to IS0 9001 registration at all levels of the organization, and assist the IS0 Coordinator and IS0 Steering Committee to overcome obstacles to IS0 900 1 registration.

Project management. The IS0 Coordinator must develop departmentlevel templates, issue document tracking and amendment procedures, commit dates and resources, and continuously monitor the registration project's progress at all company levels. Continuous self-assessment. Management and staff must understand that IS0 9001 is a continuous process of improvement that benefits the company and its customers. QMS infrastructure. Site-wide processes must be defined early. Process and document templates should be standardized, and the format re-used throughout the organization. Training. The IS0 Steering Committee and the internal audit team must consist of well-trained, competent individuals. Consultants and auditors must be well qualified and must previously have completed a registration process. Staff training should communicate the benefits of IS0 9001 registration. Education should consist of 1) overview and awareness training; 2) explanations of IS0 9001 standard and IS0 9000-3 guidelines; 3) assessor and lead assessor training; 4) implementation training; and 5) preaudit training. People network. The IS0 9001 project must be championed by a senior manager. Other important components of the people network are the IS0 Steering Committee; the functional coordinators; the department representatives; experienced consultants; and the software developers. It is important to minimize turnover and to nurture the people network during the registration process. How much does registration to I S 0 9001 cost? "We were spending 30% of our revenues on correcting screw-ups when we first measured it three years ago. We are now spending only 8% with a target of 5-696 for next year.'' A QA managerfrom a software company that registered to I S 0 9000 a year ago The cost of registering a company's QMS to IS0 9001 comprises: consulting expenses (if required); internal preparation costs;

ANSWERS TO FREQUENTLY ASKED QUESTIONS

35

registration fees; and annual surveillance charges. Costs will vary with the size and complexity of the organization that seeks registration. Consulting charges often run to tens of thousands of dollars, and internal preparation costs exceed $100,000 in most cases. Consultants do not usually include travel and living expenses in their quoted fees, yet these must also be paid by the company. The registration fees are paid to the registrar. They may consist of: an application fee; pre-audit assessment fees and expenses (optional); documentation review charges; the audit, the audit report, and IS0 9001 registration, as well as the auditor's travel and living expenses; and corrective action reviews and associated expenses. A registrar's fees can range from $7,000 for a small software company to as much as $20,000 for a large company. In addition, the auditors' expenses must also be paid by the audited company. Although price is a factor in the selection of a registrar, it should be neither the deciding nor the most important factor. Registration fees are a small part of the entire registration cost. Typical I S 0 9001 registration costs Consulting Charges Internal Effort Registrar's Fees Annual Surveillance Auditors' travel and living expenses

$25,000 $100,000 $ 13,000 $5,000 Vary

Annual surveillance expenditures, paid directly to the registrar, average between 30% and 50% of the original registration charge. Costs of $3,000 to $6,000 are common regardless of whether the registrar provides annual or bi-annual audits. Organizations that demonstrate exemplary compliance may have their surveillance charges reduced.

36

IS0

9001 REGISTRATION FOR SMALL AND MEDIUM-SIZED

SOFTWARE ENTERPRISES

How long does it take to register to I S 0 9001? Small and medium-sized software companies with a reasonably mature QMS take approximately 9 to 18 months to register to IS0 900 1. Firms that comply with IS0 9001 standards can complete the registration in three months, whereas firms that lack a QMS may take as long as two years to become registered. Most of this time is spent in the development of the company's documentation and ensuring that the system is properly implemented. The greatest determinant of time is management commitment. Another very strong factor is the length of a typical development cycle and the number of concurrent development projects. For example, if an organization executes a final test once every three years, it may take three years to gather enough evidence of compliance. If the organization has a six-month development cycle, however, and has three simultaneous projects, then every part of the process could be executed within a two-month period, thus providing the necessary evidence. "We will not even begin the registration process unless there is full management commitment." IS0 9000 auditor Additional factors affecting the time needed to comply with and register to I S 0 9001 include the maturity of the QMS and the availability of resources, especially time and training, dedicated to the registration process. Because of high demand, many registrars report significant delays in beginning the registration process. This is a problem particularly when qualified software auditors are desired. "Consider how to phase in the QMS incrementally; this will be easier to tackle and easier for other people to absorb. Some things need to be done early (for example, project management, documentation andlor reviewing standards) but others (for example, coding standards) could probably be left until later." TickIT Guide, Section 4.2.3 No single IS0 9001 implementation schedule is suitable for all IS0 9001 registrations. This section provides a schedule over a one-year period for the "how to" approach described earlier. This should be customized to suit

i

ANSWERS TO FREQUENTLY ASKED QUESTIONS

37

your organization. Many companies have registered their QMS in far less time while others have taken longer.

A TYPICAL I S 0 9001 IMPLEMENTATION SCHEDULE Phase 1: The Start (months 0 to 1) decision made to pursue IS0 9001 registration IS0 Steering Committee established IS0 Coordinator appointed IS0 Steering Committee members become familiar with IS0 9001 and IS0 9000-3 a schedule and plan are established, reviewed, and approved Phase 2: Selection of a Registrar (months 1 to 3) scope of registration determined criteria for selecting a registrar are established information from potential registrars is gathered RFPs are issued and a registrar is selected IS0 Coordinator reviews and upgrades the QMS Phase 3: Preparation for the Audit (months 1 to 6) registration project is detailed: activities to be undertaken, the individuals responsible for performing these activities, and a schedule are identified company-wide IS0 9001 training begins IS0 Coordinator establishes templates, standards, and tools; QMS documentation is developed IS0 Coordinator issues the IS0 900 1-compliant Quality and Procedures Manual internal auditors are selected and trained; internal audits are scheduled and performed; corrective action is conducted as needed QMS documentation is sent to registrar audit date negotiated Phase 4: Pre-Audit Assessment (months 6 to 9) pre-audit assessment is conducted (optional) areas of non-compliance are identified QMS is modified as needed

38

IS0

9001 REGISTRATION

FOR SMALL A N D MEDIUM-SIZED SOFTWARE ENTERPRISES

Phase 5: The Audit (month 9) pre-audit training pre-audit meeting if necessary, notices of non-compliance are issued by the registrar audit report is issued Phase 6: Post-Audit Compliance (months 9 to 12) plans and time limits are set for responses to areas of non-compliance corrective action is taken a re-audit is conducted, if necessary the registrar begins preparation of the registration documents Phase 7: IS0 9001 Registration (month 12) official registration presentation of IS0 900 1 registration certificate Phase 8: Annual Surveillance Visits (continue indefinitely) maintenance and improvement plan are established registrar conducts periodic surveillance audits internal audits are scheduled and conducted a complete re-audit is conducted within three years of registration "From start to finish, IS0 registration for one branch within the company took just under a year. Although we had met many of the requirements before we even started, we were very thorough and methodical in the approach we took. This allowed us to ensure that all the required steps were properly completed. Another company branch was registered in less time, approximately 4 months, by applying what we had learned from our previous experience." Lane Smith, SofMare Kinetics Ltd. In preparation for registration, many activities can be performed concurrently. For example, the registrar can be selected while documentation is being prepared. IS0 900 1 registration may require additions or amendments to the QMS; these changes require a phasing-in strategy, such as phasing in one new QMS component to every department simultaneously,or several components to one department at a time. As components of the QMS are released, staff feedback should be encouraged and used to improve the QMS.

)

ANSWERS TO FREQUENTLY ASKED QUESTIONS

39

Appendix E displays the IS0 9001 Implementation Schedule of Amita Management & Information Technology Adaptors Corp. as an example of a schedule used by a small firm to obtain registration to IS0 9001. What are the criteria for selecting a registrar? Selecting a registrar is no different from selecting a sub-contractor. Select three or four from a suppliers' list, and ask them to tender based on the requirements. The following criteria have proven helpful in selecting a registrar:

CUSTOMER CREDIBILITY Does the registrar have credibility with your current and prospective customers? Is a specific registrar required or preferred by your customers? Will your customers recognize the registrar's accreditation? Is the registrar accredited in your customers' countries? Will the registrar provide a complete list of companies they have registered so that references may be obtained? How long has the registrar been in business?

"[Auditors] must exhibit a wide and detailed information technology knowledge overview coupled with a broad understanding of computing and associated hardware and the relationship between hardware and software at all levels." Requirements ofthe National Registration Scheme for TicklTAuditors, Appendix I , Section C2 QUALIFICATIONS A N D FAMILIARITY

WITH SOFTWARE

Does the registrar specialize in software? Does the lead auditor or members of the audit team possess sufficient software expertise? Do the auditors have experience assessing software organizations? What other software companies have they registered? Is the registrar accredited by their national accreditation body and/or an international accreditation body? QUALITY OF SERVICE

What proportion of the auditing is subcontracted? How eager is the registrar to serve your business?

40

I S 0 9001 REGISTRATION FOR SMALL AND

MEDIUM-SIZED SOFTWARE ENTERPRISES

"Remember, you are the customer." Michael Jaques, Prior Data Sciences Ltd. Does the registrar return your calls? Are the auditors approachable? What is the registrar's approach to registration, surveillance,and re-registration? Under what conditions can registration be suspended, withdrawn, or cancelled?

COST Is the registrar's cost (including annual maintenance fees) competitive for the service they provide? Is there an application fee? If so, how much? What is the daily billing rate? What is the cost to review the QMS documentation? What is the cost of the initial audit? Is there a charge for follow-up visits? What is the annual charge for surveillance visits and continued registration? How much is re-registration afier the initial registration expires? What is the penalty if the IS0 9000 project has to be dropped or delayed mid-stream? LOCATION Where is the registrar located, and what is the required travelling distance? For companies with the same processes and documentation at other locations: 1) must the entire registration process be repeated, or, 2) can additional sites be registered with only an audit (i.e., without a separate application, documentation review, etc.)? What documentation will the audit team require? Appendix F lists some of the documents that the auditors may wish to examine. Documentation control is time consuming. The Quality Manager must ensure that only the latest copy of all procedures is available. Normally,

procedures are kept in binders and someone manually replaces outdated protocols. However, it is possible for all documentation to be kept in an online retrieval system (such as Lotus Notes). The on-line documentation should be modified only by the Quality Manager, and passwords and controlled access should replace signatures. It is best to ensure that an online system is acceptable with your registrar before implementing it. IS0 9001 requires that the company:

... establish, document and maintain a quality system as a means of ensuring that product conforms to specified requirements. The supplier shall a) prepare documented procedures consistent with the requirements of this International Standard and the supplier's stated quality policy, and b) effectively implement the quality system and its documented procedures.

'

With the release of the 1994 edition, IS0 9001 now requires a registered organization to have a quality manual. The manual should cover: pre-system development activities such as the bidding process, the format and content of the project plan; contractual matters such as the content of a contract and the style and purpose of contract reviews; and the format and content of documents used during d e ~ e l o ~ m e n t . ~

"Keep it simple! Don't commit documentation overkill. Over-documenting your system or over-defining communications can truly have a negative impact, especially for a small firm.'' I S 0 9000 auditor It is not necessary to document all activities; rather, only the QMS elements necessary to ensure adequate control require documentation.

1 IS0 900 1, Section 4.2. 2 Darrel Ince et al., Introduction to Software Project Management and Quality Assurance (New York: McGraw-Hill, 1993), p. 171.

Therefore, not all IS0 9001 requirements may apply to all companies; for example, procedures on calibration are not needed for companies with no equipment. Appendix G shows the QMS Documentation Table of Contents from Software Kinetics Ltd. as an example of the documentation contained in an IS0 9001 (1987) compliant quality manual. Although not contained in any of the IS0 9000 publications, the QMS documentation hierarchy (or pyramid) model is a common and widely accepted format for a QMS. Appendix H gives one version of a QMS documentation hierarchy. The highest level of the QMS and the first section of the quality manual is the Quality Policy and Manual, which describes a company's commitment to quality and the structure and methods for maintaining its QMS. It includes the company's policy, its current objectives, its organizational structure and job descriptions, its primary functions, and a procedures index. As the Quality Policy applies to the entire organization, management must ensure that it is understood, implemented, and maintained at all levels of the company. The 1994 revision requires that the policy be relevant to the supplier's organizational goals and the expectations and needs of the company's customers. Every employee should be familiar with the Quality Policy, and should be aware that the auditors may ask such questions as: 1) Who is the Quality Manager? 2) What are his or her responsibilities? 3) Where is the QMS Procedures Manual stored? 4) Do you know where instructions and local procedures are stored? 5) How do you know that this is the latest issue? Also contained in the first level of the quality manual is a general description of the company's management policies, its objectives, quality plans, methods for reviewing the QMS, and the responsibilities for the various clauses of the standard. This level of the quality manual is maintained by the IS0 Coordinator or equivalent (e.g., the Quality Manager). The second level of the quality manual is the QMS Procedures. These usually describe various functional activities. For example, software development may include everything from training and sub-contracting procedures to contract review, and verification and validation procedures. Procedures specify who does what, when it is done and in what sequence, and what documentation verifies that the activity was performed as required. Typical contents of the QMS Procedures section of the quality

ANSWERS TO FREQUENTLY ASKED QUESTIONS

43

manual include planning and estimation, project life-cycle control, development and testing procedures, and documentation control. "It really makes a lot of extra work for each department to write their own QMS Procedures, since 1) they are usually not used to writing that sort of document, and 2) there is no consistency of style. Work instructions can certainly be written by the people conducting the process." Jean White, Qualidoc Ltd. The third level of the quality manual contains work instructions and local procedures for the day-to-day operation of the QMS, including contracts, projects, or tasks. The manuals are best created and maintained by the staff that use them. Contents include operating procedures, test methods, project plans, and task procedures. The final level of the QMS documentation hierarchy contains records and forms. These provide repeated and objective evidence that the required quality is achieved and that the QMS is operating properly. How can software quality be ~neasured?~ IS0 9000-3 states that there are no universally accepted measures of software quality. It does, however, provide some guidance: the quality of software products should be measured; the engineering process used to produce the software should be measured. There are two reasons for measuring quality: to establish objectively, consistently, and reliably the presence or absence of quality in the software systems being produced; and to address problems associated with resistance to the introduction of, and adherence to, a QMS.

3 Extracted in part from D.C.Ince et al., "IS0 9000-3 and Software Measurement," BT TechnologyJournal 12, no. 1 (January1994).

44

IS0

9001 REGISTRATION FOR SMALL AND MEDIUM-SIZED

SOFTWARE ENTERPRISES

Some key measures of software quality are obtained by answering the following questions about any software-based system: How good a match is it to stated and implied needs? Is it delivered on time and at the predicted cost? Does it provide sufficient usability for its intended user population? Is it designed to cope flexibly with future requirements? Is it compatible with other systems with which it will have to communicate? Is it reliable and predictable? Will it operate safely? All these characteristics can, and should, be quantified and measured. The TickIT Scheme Brochure, p. 3 Without measurement of critical success factors, it is difficult to establish whether a QMS is improving. For product measurement, collection of post-delivery defect metrics is considered the absolute minimum in order to: estimate current quality levels in terms of user-perceived defects; identify trends; and to specify targets. Metrics should be reported and used to manage the development and delivery processes, i.e., metrics should provide feedback to software developerd Process measurement in IS0 9000-3 has two requirements: it should provide a quantitative view of process quality (i.e., defect density may be an indicator of quality); and it should have a defect-tracking system to capture the likelihood of an individual activity's entering the software system and removing defects from it. Before adopting metrics, it must first be clear what is to be measured and why. A metrics framework must be simple and flexible, yet it must make explicit the purpose and method of use. Appendix I provides a metrics template.

4 IS0 9000-3, Section 6.4.

ANSWERS TO FREQUENTLY ASKED QUESTIONS

45

How long does registration last? IS0 9001 does not require that re-registration occur after a certain period of time. It is common, however, for registration to last for three years. Postponement of a re-audit for a few months is possible with most registrars. It should be noted that a registrar can require a complete re-audit in the event of major changes affecting the supplier's activities and operations, such as a change in ownership or changes in personnel or equipment. A reaudit may also be ordered if analysis of a complaint or any other information indicates that the supplier no longer complies with the requirements of the registrar.5 Assuming the QMS continues to meet IS0 9001 requirements, a company can continue to re-register for as long as it wishes. Can the firm be delisted? In rare cases, a company can be delisted. Suspension or withdrawal of registration may occur if a company 1) fails to demonstrate its ability or willingness to comply with the program requirements; 2) fails to pay its fees; or 3) requests delisting. A company is usually given warning and ample time to correct a deficiency. An increase in the frequency of audits usually precedes the withdrawal of registration. Once withdrawal has occurred, companies may re-apply for IS0 9001 registration or appeal the withdrawal. Is a consultant needed? "Former auditors seem to make good consultants." Dave Pedley, DY 4 Systems Inc. Executives of software companies are cautious about relying on consultants for assistance in their IS0 9001 registration for three reasons: Very few consultants are familiar with software development processes. There is the fear that a consultant may provide incorrect advice and cause unnecessary changes in work practices. 5 ISO/CANCO 227 (Rev.) Draft - General Requirementsfor Bodies Operating Assessment and Cert$cation/Registration of Quality Systems, March 1994, Section 4.5.4., p. 8.

Consultants may follow the standard to the letter because they are not certain how an individual auditor will interpret it. Some consultants sell a packaged solution and apply it whether it fits or not. "It is not the consultant's job to tell the company how they should conduct their processes, but how they should control them to meet the IS0 requirements." Jean White, Qualidoc Ltd. "Ideally, it would be best if staff could develop the organization's procedures and documentation; however, my experience has shown that most small software companies cannot afford to have their software developers writing documents. A consultant can provide this service." Peter Papa kostan tin u, Papa e+ Associates Inc. Good consultants can add experience and expertise to the registration process and identifi potential problems that are less visible from inside the company. Many companies find that pre-assessments provided by consultants are worth the expense. Often, software companies do not have the resources to dedicate to IS0 9000. A good consultant may prove to be very valuable in this situation, especially if the consultant has software experience. To determine which consultants are good, many software executives refer to the experiences of others in the industry. Many IS0 9001 registered software companies found a lack of qualified software consultants and responded by providing their own consultancy. Do not hesitate to ask for a consultant's software qualifications. If consultants are qualified, they will be happy to display their credentials. What are the differences among the Software Engineering Institute's Capability Maturity Model, I S 0 9001, and the Malcolm Baldrige National Quality Award? All three are approaches to process improvement. The Capability Maturity Model (CMM) was introduced by the Software Engineering Institute (SEI) at Carnegie Mellon University in 1987. The CMM was designed solely to improve the software development process; the other two approaches were not. The CMM consists of a rating scheme designed to correlate the answers to 101 questions with a five-level framework for the engineering practices in a successful software development organization. Appendix / gives a brief

ANSWERS TO FREQUENTLY ASKED QUESTIONS

47

description of the framework. The SEI rating scheme distinguishes between critical and non-critical questions. To reach a given level, a software engineering organization must be able to answer yes to 90% of the critical questions and 80% of the non-critical questions for that level. Appendix C lists the 20 elements of IS0 900 1. To receive the Malcolm Baldrige Award, a company must establish a QMS that meets the evaluation criteria listed in Appendix K.This award is only for companies in the United States. Although the three models overlap, compliance with one does not ensure compliance with another. For example, reaching an advanced maturity level in the CMM indicates that certain key engineering practices are implemented, whereas IS0 9001 requires business practices that are not addressed in the SEI questionnaire. Likewise, when compared with IS0 900 1, the CMM's focus on software engineering offers significant additional guidance, particularly in the areas of technology, process definition, and metrics. The Malcolm Baldrige Award places heavy emphasis on continuous improvement. Although IS0 9001 and SEI CMM both require continuous improvement, IS0 9001 requires periodic surveillance visits, while SEI CMM specifies periodic reassessments every 12 to 18 month.

THETHREE

ELEMENTS MOST EMPHASIZED I N EACH PROCESS IMPROVEMENT

METHODOLOGY

CMM management of process quality information and analysis strategic quality planning

I S 0 9001 management of process quality leadership information and analysis

Malcolm Baldrige Award customer focus and satisfaction management of process quality human resource development and management

What courses are available? It is a requirement of IS0 9000 registration that internal auditors are trained and qualified to conduct internal QMS audits. Courses in internal auditing satisfy this requirement. Courses are also available on more advanced procedures, documentation and implementation procedures, general IS0 9000 awareness, and TickIT registration. Such courses are offered by several registrars, consultants, universities, and colleges. Unfortunately, with the exception of the TickIT courses, few are specific to the software industry. Companies may consider sending representatives to lectures, discussion groups, workshops, and question-and-answer sessions, especially if speakers or others attending are from the software industry. Videos, conference papers, manuals, and guides may share the valuable experiences of other firms. The following are examples of typical IS0 9000 courses:

DURATION

*

COURSE SUBJECT

(DAYS)

PARTICIPANTS

IS0 9000 Awareness

1 or less

All Employees

IS0 9000 Executive Overview

1 or less

IS0 Steering Committee

IS0 9000 Documentation

1 to2

IS0 Coordinators

IS0 9000 Implementation

1 to 4

IS0 Coordinators

Internal Quality Management System Auditing

2 to 4

Internal Auditors

5

Quality Managers

Auditor Training Introduction to TickIT

1 or less

TickIT Auditor Training

5

IS0 Steering Committee Quality Managers

ANSWERS TO FREQUENTLY ASKED QUESTIONS

49

What is a bilateral MOU?

A Memorandum of Understanding (MOU) is an agreement that permits foreign recognition of a company's registration. A MOU is an agreement between two registrars from different countries that acknowledges cooperative and reciprocal recognition of their registrations. MOUs reflect a mutual level of confidence and acceptance of the policies, procedures, and personnel between two registrars. "For international recognition of IS0 9000 registration, the MOU is the weakest approach; sister company registration and registrar accreditation in other countries are stronger." Bob Armstrong, DMR Group Inc. "Many Memorandums of Agreement are set up by individual registration bodies; they are often financially motivated to define an exclusive working territory." Geoff Kelland, Northern Telecom Canada Ltd. There are three distinct stages to these agreements: Base MOUs signify only an agreement between registrars to cooperate in the future. Developing MOUs acknowledge that two organizations agree to work together or even audit each other to build the confidence necessary to offer reciprocity. Full MOUs or bilateral agreements describe situations where registrars accept each other's certificate as equivalent to their own. For recognition in a foreign country, registration by a registrar accredited in that country is a better solution than a MOU. The best registration is the one that is most likely to be accepted by your customers. What is TickIT certification? With the release of IS0 9000-3, the IS0 recognized that software development and maintenance are different from most other industrial production processes. To some, IS0 9000-3 appeared inadequate, and a more sectoral approach seemed desirable for the software industry. The UK Department of Trade and Industry (DTI), with the assistance of the British Computer

Society (BCS), created the TickIT scheme. Using IS0 9000-3 as guidance, TickIT adds to IS0 9001 audits specific software industry-related requirements for TickIT auditors, as well as a separate accreditation program for TickIT registrars. "[Before TickIT] the UK software ... industry faced the situation where untrained assessors, working for unaccredited certification bodies, made inappropriate judgements about quality systems they did not understand." CSA Position Paper, TicklT Provides Proven Quality Benefits.. ., p. 2 The UK's creation of TickIT has not been without controversy, even though the expertise of TickIT auditors in the software industry has been generally well received by firms internationally. The scheme's industryspecific requirements and the associated costs have made the scheme controversial with registrars and national accreditation bodies, and in the information technology industry outside the UK. Two Canadian software development organizations, Prior Data Sciences Ltd. and the IBM Toronto Lab, have been TickIT registered. Several countries, including Australia, Japan, Sweden, and Singapore, are reportedly considering the TickIT approach. However, the United States has decided not to adopt a TickIT approach to software registration at this time. TickIT Products TickIT Guide TickIT Video TickIT Information Brochure TickIT Case Studies TickIT Auditor's Training Course What is ITQS? ITQS is an abbreviation for the Agreement Group for Assessment and Certification of Quality Systems in the Information Technology and Telecommunication (IT&T) sector. ITQS is a registrar-based European organization. Its objective is the mutual recognition among the ITQS participants of assessment reports and certificates for quality systems in the IT&T sector through harmonized assessment and registration services. The

scope of ITQS includes IT&T hardware, software, and services. ITQS is an international scheme, recognized by the European Organization for Testing and Certification (EOTC), and is a member of ECITC, the European Committee for IT&T Testing and Certification. Non-European registrars may also participate in ITQS. Should I S 0 9001 affect the company's other QMS? "The manual doesn't have to take the IS0 9001 format; in fact, I recommend working with what makes sense for your business." Michael Jaques,Prior Data Sciences Ltd. The creation of separate QMS to satisfy other quality programs is not recommended. To avoid confusion and prevent redundant paperwork, all the company's existing QMS must be reviewed and harmonized in a single QMS and manual. It is not mandatory for the manual to adopt the IS0 9001 format. A common alternative approach creates a matrix that shows each requirement of the standard, citing the corresponding manual reference. How does a company build evidence of a disciplined software development process? IS0 9001 does not specify what evidence demonstrates a disciplined software development process. Usually, a company begins by defining its process in a quality manual or similar document that describes the life cycle, the activities performed at each phase of the cycle, procedures for design review, storage of documents, etc. Such a manual offers only partial evidence that an organization's software development process is a disciplined one. Further evidence must show that the company has for some time adhered to the procedures outlined in its manual. For example, a signature page might follow the software from its initial, top-level design through to acceptance testing. Other evidence might include attendance lists from QMS-related meetings, copies of project plans, notes on design reviews, or other appropriate documents. However, evidence and quality records are not the same. Evidence is tangible proof that a company follows its procedures; quality records merely document the achievement of the required quality and the effective operation of the QMS.

What types of non-compliance notices are there? If the company's documented QMS does not comply with the requirements of IS0 9001, the auditors may issue notices of non-compliance. Once a non-compliance is raised, it cannot be withdrawn until the company establishes compliance with the standard. Many registrars differ in their approach to non-compliance. One approach divides notices of non-compliance into two types: major and minor. A notice of major non-compliance would indicate total non-compliance with one of the 20 requirements of the standard. Minor non-compliance indicates a minor breach, e.g., the company's written procedure differs from its actual practice, or a procedure was not followed in certain circumstances. Typically, companies are given six weeks to correct a minor non-compliance and approximately 13 weeks for a major non-compliance. If a major non-compliance is discovered, the auditor will usually insist on returning to the site to ensure that the correction is acceptable before a registration is issued. The correction of minor non-compliances can usually be verified during the next surveillance audit, unless the combined effect on the QMS is deemed by the auditor to be equivalent to a major non-compliance. Must I S 0 9001 registered companies comply with revisions to the standard? Yes, a company has a 12-month transition period in which to adjust to revisions to the standard. It is a good idea to have the registrar audit the company to the new standard at least three months prior to the end of the 12-month period to avoid any last-minute notice of non-compliance. Registrars are required to notify clients of any standard revisions well in advance of changes to their auditing procedures. All IS0 standards are reviewed in principle every five years. What are some complaints about I S 0 9000? "From the viewpoint of many SC7 contributors, as it exists today, IS0 9000-3 is not satisfactory." Letter to the Chair ISO/IEC/JTCI and Chair ISO/TC176 on IS0 9000-3,

P-3 The following are typical complaints from executives of software companies about IS0 9001 and IS0 9000-3:

ANSWERS TO FREQUENTLY ASKED QUESTIONS

53

IS0 9001 is too subjective; it can be interpreted in different ways. IS0 900 1 is based on the traditional manufacturing approach to quality control. It needs interpretation before if can be applied to the development and production of systems containing software. "Organizations are not hindered from having a unified QMS by using IS0 900 1 and IS0 9000-3. For example, the Design Control section of a manual could contain separate items for Design Review - Hardware, Design Review - Software, Design Review - Publications, etc." Jeatz White, Qualidoc Ltd. "The notion that IS0 9000 impedes Total Quality Management is not true! Good companies use IS0 9000 as a base to TQM." Bob Armstrong, DMR Group Inc. "Although IS0 9000 registration is designed to reduce the number of external audits, [some companies] have found no decrease in the number of customer audits." Bob Armstrong, DMR Group Inc. IS0 9000-3 interprets the application of IS0 9001 to software; however, its interpretation of the design and production phases varies from those described in IS0 900 1. As a result, organizations developing systems that contain both software and hardware need to set up two different QMS: one for hardware engineering based on IS0 9001, and one for software engineering based on IS0 9000-3. This situation hinders organizations from setting a total unified systems-wide and consistent Q M S . ~ IS0 9000-3 specifies the classical model of the software life cycle. This model does not incorporate such modern sofnvare development practices as prototyying, re-use, etc. There has been a lack of confidence in the ability of IS0 9000 auditors to understand the software industry. Unlike other accreditation bodies, the Standards Council of Canada does not accredit registrars to any specific scope. Once accredited, registrars may operate in any industry provided they have the expertise to perform the necessary registrationlaudit work.

6

Franqois Coallier and Jim Roberts, Letter to Chair ISO/IEC/JTCl and Chair ISO/TC176 orz IS0 9000-3. Point Position Paper, September 1993,p. 4.

54

IS0

9001 REGISTRATION

FOR SMALL AND

MEDIUM-SIZED SOFTWARE ENTERPRISES

Auditors may require procedures that could be detrimental to the company's success. However, this situation can be avoided if the selection process for the registrar is effective. I S 0 9000-3 implies a centralized organization where quality assurance is based on independent audit systems. However, many organizations are shifting away from independent audits toward total quality management (TQM) techniques such as self-assessments and continuous improvement. These perspectives are not addressed in I S 0 9000-3. The direct and indirect costs of I S 0 9000 registration are high, especially for small companies.

IV Guidelines for the Application of I S 0 9001 to Software Development

This section summarizes the document IS0 9000-3: Guidelinesfor the Application of I S 0 9001 to the Development, Supply, and Maintenance of Software. The guidelines for the QMS are organized into 1) Framework; 2) Life-Cycle Activities; and 3) Supporting ~ctivities.' The issuance of a revised IS0 9001 standard in August 1994 made the 1991 edition of IS0 9000-3 obsolete. It is not clear when a replacement to IS0 9000-3 will be issued.

Quality Management System - Framework Management Responsibility Establish a Quality Policy, including objectives for quality and the company's commitment to quality. Ensure the Quality Policy is understood, implemented, and maintained at all levels of the organization. Establish a quality infrastructure with appropriate responsibilities and authority. Identify the requisite verification resources and personnel. Appoint a management representative with executive responsibility to: ensure that an IS0 9001-compliant QMS is established, implemented, and maintained; and report to the company's management on the performance of the QMS. Establish a partnership with the customer to: achieve continuous feedback; ensure compliance of the software to the customer's agreed requirements specification; and verify and accept test results. Conduct and document periodic reviews of the QMS.

7 This interpretation of IS0 9000-3 has been extracted with modifications from Subhash Puri, I S 0 9000 Certification and Total Quality Management (Ottawa: Standards-QualityManagement Group), 1992.

Quality Management System Establish, document, and maintain an effective and integrated QMS that spans the entire software production life cycle and ensures total quality. Document and implement the QMS via a quality manual, procedure manuals, work instructions, etc. Define and document a quality plan. "You should first look at what the company is doing and ask whether you are following good software engineering practices. Michael Jaques, Prior Data Sciences Ltd. Internal Quality Management System Audits Establish an effective internal quality audit system to verify the QMS's effectiveness and compliance with the standard. Take appropriate corrective action on the deficiencies identified by the audit. VeriFy and record the implementation and effectiveness of the corrective action. Corrective and Preventive Action Establish and maintain documented procedures for implementing corrective and preventive action. Procedures include: the handling of customer complaints; the investigation of non-conformities; the sources of information used to detect, analyze, and eliminate potential non-conformities; and the determination of action needed to deal with problems requiring corrective action. Analyze quality records and information to detect and eliminate deficiencies. Apply controls to ensure appropriate, effective, and timely corrective and preventive actions.

Quality Management System - Life-Cycle Activities Contract Reviews Establish and maintain documented contract review procedures. The company should review each contract to ensure that: the requirements are properly defined and documented;

GUIDELINES FOR THE APPLICATION OF I S 0

9001 TO SOFTWARE DEVELOPMENT 57

possible risks are identified; any discrepancies are resolved; the company has the capability to meet the requirements; and the company has the capability to meet its contractual obligations. Review the contract to ensure that it adequately addresses all the requisite criteria for the acceptance and handling of changes and problems, standards and procedures to be used, and the customer's responsibilities with regard to the specification of requirements, installation, facilities, tools, and software to be provided. Specification of Customer Requirements The company should ensure that all functional requirements for software development have been obtained from the customer, such as performance, safety, reliability, security, and privacy. The customer's specifications must be properly documented. The document must clearly establish and identify the appropriate responsibilities of both the company and its customer, the methods of approval for the requirements and any ensuing changes, the review procedures, the interfaces between the software product and other software or hardware products, etc. Development Planning Establish a software development plan to include: a project definition and objectives; project organization details such as resources, teams, responsibilities, sub-contractors, human resources, etc.; project phases such as development, required input and output for each phase, verification procedures, potential problem analysis strategy and procedures, etc.; a project schedule; identification of requisite plans such as a quality plan, a configuration management plan, an integration plan, and a test plan; identification of how the project will be managed; development methods and tools; progress control procedures; documentation of inputs and outputs from each development phase; and verification procedures for each phase.

58

ISO goo1 REGISTRATION FOR SMALL AND MEDIUM-SIZED SOFTWARE ENTERPRISES

Quality Planning "Prior didn't change its software development process at all! We did, however, add documentation to cover procedures not previously dealt with such as: maintenance, contract review, and pre-contract activities covered by I S 0 900 1." Michael Jaques, Prior Data Sciences Ltd. Define and document how the requirements for quality will be met. The quality plan should identify: the quality objective; the defined input and output criteria for each phase of development; the details of the test verification and validation; and the specific quality responsibilities, such as reviews and tests, configuration management and change control, defect control and corrective action, etc. Design and Implementation Establish and maintain a well-structured and disciplined procedure for software design and implementation. The design activities should include: proper design rules and internal interface; systematic design methodology appropriate to the type of software being developed; past design experiences. Design implementation should accord with the established rules, such as the programming rules, programming languages, consistent naming and coding conventions, etc. Implementation methods should be appropriate to customer requirements. Establish procedures for formal, documented design reviews. Design verification and validation are required. Testing and Validation Establish and maintain documented procedures for testing, validation, and field testing of the software products at each stage of their development.

GUIDELINES FOR THE APPLICATION OF I S 0

9001 TO SOFTWARE DEVELOPMENT 59

Acceptance The company and its customer must establish a well-defined procedure for acceptance of the final product. The acceptance procedure should take into consideration the schedule, the procedures for evaluation, the software and hardware environments and resources, and the acceptance criteria. Replication, Delivery, and Installation The company must establish criteria for replication prior to delivery. This includes: the number of copies of each software item to be delivered; the copyright and licensing concerns; the custody of the master and back-up copies; and the period of obligation for the company to supply copies. Establish appropriate procedures for verifying the correctness and completeness of all copies of the delivered software product. The company and its customer must establish clear procedures for installation. Maintenance The company should establish and maintain documented procedures for the maintenance of the s o b a r e if required by the contract. Maintenance activities include problem resolution, interface modification, and continuous performance improvement. Items to be maintained may include programs, data and their structures, specifications, documents for purchase and/or use, and documents for the company's use. For effective maintenance, the company should establish an appropriate plan that specifies the scope of maintenance, the initial status of the product, the support organizations, the maintenance activities, and the maintenance records and reports. Maintenance records and reports must be kept according to the plan. The company and its customer must establish and document procedures for incorporating any necessary changes into the software product.

Quality Management System - Supporting Activities Configuration Management The company should develop a configuration management plan and establish an appropriate system for identifying, controlling, and tracking the previous versions of each software item. Configuration management may include configuration identification and traceability, change control procedures, and procedures for recording the status of the software items. Document and Data Control The company should establish and maintain procedures to control QMSrelated documents and data. Document control applies to such items as procedural, planning, and product documents. Procedures should be established for the approval and issuance of documents, as well as for changes to them. Control of Quality Records The company should establish and maintain documented procedures for the identification, collection, indexing, access, filing, storage, maintenance, and disposition of quality records. It is acceptable for records to stored electronically. Measurement As far as possible, the company should develop quantitative means for measuring the quality of software products and the quality of the development and delivery processes. Rules, Practices, and Conventions The company should clearly identify the rules, practices, and conventions employed in the established QMS. Tools and Techniques The company should identify and use appropriate tools and techniques for effective management of the QMS.

GUIDELINES FOR THE APPLICATION OF I S 0

9OOl TO SOFTWARE DEVELOPMENT

61

Purchasing The company should establish and maintain documented procedures to ensure that the purchased products or services conform to the specified requirements. Thus, sub-contractors must be assessed, and the purchased products verified and validated. Records of acceptable sub-contractors must be established and maintained. Control of Customer-Supplied Software Product Where a company is required to include or use software products supplied by the customer or by a third party, the company should establish and maintain documented procedures for the control of validation, storage, protection, and maintenance of such products. Training The company must identify training needs and provide appropriate training facilities and opportunities. Records of training activities should be maintained.

V Sources of Information

This section provides sources of information on IS0 9001 registration. It includes a list of accredited registrars, several sources of IS0 standards and lists of accredited registrars, a list of relevant publications, and other useful sources.

Accredited Registrars These registrars are accredited by the Standards Council of Canada and have expressed an interest in registering software companies to IS0 9001. Canadian General Standards Board Conformity Assessment Branch 222 Queen Street, Suite 1402 Ottawa, Ontario KIA 1G6 Tel.: (613) 94 1-8709 Fax: (613) 941-8706 Groupement QutbCcois de Certification de la QualitC Bureau de normalisation du QuCbec 70 Dalhousie Street, Suite 220 Quebec City, Quebec GlK 4B2 Tel.: (418) 643-58 13 Fax: (418) 646-33 15 Litton Systems Canada Limited Quality Systems Registrars 25 City View Drive Etobicoke, Ontario M9W 5A7 Tel.: (4 16) 249- 1231 Fax: (416) 246-2049

SOURCES OF INFORMATION

63

Quality Management Institute 2 Robert Speck Parkway, Suite 800 Mississauga, Ontario L4Z 1H8 Tel.: (905) 272-3920 (800) 465-3717 Fax: (905) 272-3942 SGS International Certification Services Canada Inc. 90 Gough Road, Unit 4 Markham, Ontario L3R 5V5 Tel.: (905) 479- 1160 (800) 636-0847 Fax: (905) 479-9452 Warnock Hersey Professional Services Ltd. 8810 Elmslie Street LaSalle, Quebec H8R 1V8 Tel.: (514) 366-3100 (800) 561-5051 Fax: (514) 366-5350

To Obtain I S 0 Standards or a List of Accredited Registrars The IS0 9000 series of quality standards, as well as an updated list of accredited registrars, are available from the standards Council of Canada (SCC), the American Society for Quality Control (ASQC), the American National Standards Institute (ANSI),or the British National Accreditation Council for Certification Bodies (NACCB).Addresses and phone numbers are given below. In addition, I S 0 9000 standards may be purchased directly from the ISO. Standards Council of Canada (SCC) 45 O'Connor Street, Suite 1200 Ottawa, Ontario KIP 6N7 Tel.: (613) 238-3222 (800) 267-8220 Fax: (613) 995-4564

64

IS0

9001 REGISTRATION FOR SMALL AND MEDIUM-SIZED

SOFTWARE ENTERPRISES

American Society for Quality Control (ASQC) 61 1 East Wisconsin Avenue P.O. Box 3005 Milwaukee, Wisconsin 5320 1-3005 Tel.: (414) 272-8575 (800) 248- 1946 Fax: (414) 272- 1734 American National Standards Institute (ANSI) 11 West 42nd Street, Floor 13 New York, New York 10036 Tel.: (2 12) 642-4900 Fax: (212) 398-0023 National Accreditation Council for Certification Bodies (NACCB) Audley House 13 Palace Street London, England SWlE 5HS Tel.: (07 1) 233-7 1 1 1 Fax: (071) 233-51 15 International Organization for Standardization 1, rue de Varembe, P.O. Box 56, CH-1211 Geneva 20, Switzerland Tel.: +41 22 749 01 1 1 Fax: +41 22 733 34 30

Publications "The American Society of Quality Control's monthly publication [Quality Progress] provided a lot of industry information.'' Dave Pedley, DY 4 Systems Inc. Most of the articles and books dealing with I S 0 9000 standards apply to the manufacturing sector. The following periodicals are good sources of IS0 9000 knowledge.

IS0 9000 News International Organization for Standardization 1, rue de Varembe, P.O. Box 56, CH- 1211 Geneva 20, Switzerland Tel.: +4122 749 01 11 Fax: +4122 733 34 30 Quality Progress magazine American Society for Quality Control (ASQC) 6 11 East Wisconsin Avenue P.O. Box 3005 Milwaukee, Wisconsin 5320 1-3005 Tel.: (414) 272-8575 (800) 248- 1946 Fax: (414) 272- 1734 Quality Systems Update Centre for Energy and Environmental Management (CEEM) 10521 Braddock Road Fairfax Station, Virginia 22032 Tel.: (703) 250-5900 (800) 745-5565 Fax: (703) 250-5313

"I found the TickIT Guide very useful.. . ." Michael Jaques, Prior Data Sciences Ltd.

TickIT News, TickIT Guide, TickIT Case Studies, and the TickIT Video DISC TickIT Office 2 Park Street London, England W 1A 2BS Tel.: (071) 602-8536 Fax: (071) 602-8912

ITQS Regulations, European Information Technology Quality System Auditor Guide, and ITQS News Agreement Group for Assessment and Certification of Quality Systems in Information Technology and Telecommunications (ITQS) ITWS Secretariat C/OAIB-Vin~otteInter Avenue du Roi, 157 B - 1060 Brussels Belgium Tel.: +32 2 536 82 1 1 Fax: +32 3 536 85 85 Quality Digest QCI International 1350 Vista Way, P.O. Box 882 Red Bluff, California 96080 Tel.: (916) 527-8875 (800) 527-8875

Fax: (916) 527-6983 Software Process, Quality e+ IS0 9000 Systems and Software Ltd. 142 Highway 34, Suite 350 Holmdel, New Jersey 07733 Tel.: (908) 946-0005 Fax: (908) 946-4 149 A recent book published by the ASQC, IS0 9000for Sofhare Developers, by Charles Schmauch and A Guide to Soware Quality System Registration under IS0 9001, by the Software Quality Systems Registration Committee, may also be of interest. The ASQC's address is given above.

A Guide to Software Quality System Registration under IS0 9001 Information Technology Association of America (ITAA) 1616 North Ft. Myer Drive, Suite 1300 Arlington, Virginia 22209-3 106 Te.1: (703) 522-5055 Fax: (703) 525-2279

SOURCES OF INFORMATION

67

Other Sources of Information Besides the TicklT News, the DISC TickIT Office also sells the TicklT Guide, case studies, and a video. ITQS publishes a guide for auditors identical to that contained in the TicklT Guide; it is called the European Information Technology Quality System Auditor Guide. "If you read and hear enough viewpoints from enough experts, the odds are overwhelming that you will become totally confused." Michael LaBoeuf Many consultants have published reports on the ins and outs of IS0 9000 registration. A few are good. Members of the software community are often glad to share their IS0 9000 experiences. Often, a contact with another organization's IS0 Coordinator proves to be very valuable. For those with e-mail access to the Internet, the IS0 9000 Discussion Group has over 1500 subscribers who exchange information and advice. Many subscribers belong to the software community. For a free subscription, send e-mail to: [email protected] citing the message: SUB IS09000 (your name). For greater involvement, consider membership on a standards committee, such as the Technical Advisory Committee, ISO/TC 176. Membership provides an opportunity to offer detailed comments on draft amendments and to vote on whether Canada should accept such drafts. Technical Advisory Committee members can influence final ~ o n t e n t . ~

8 Patricia Billingsley, "ErgonomicStandards Go Beyond Hardware",IEEE Software (March 1994):84.

VI Appendices

Appendix A Nations That Have Adopted I S 0 9000 (November 1994) The following countries have adopted IS0 9000 standards as their national standard: Algeria Argentina Australia Austria Barbados Belgium Brazil Brunei Darussalam Bulgaria Canada Chile China Colombia Costa Rica Croatia Cuba Cyprus Czech Republic Denmark E ~ptY Finland France Germany Greece Hungary Iceland India Indonesia

Ireland Israel Italy Jamaica Japan Malawi Malaysia Mexico Mongolia Nepal Netherlands New Zealand Norway Pakistan Papua New Guinea Peru Philippines Poland Portugal Republic of Korea Romania Russian Federation Singapore Slovakia Slovenia South Africa Spain Sri Lanka

Sweden Switzerland Syria Tanzania Thailand Trinidad and Tobago Tunisia Turkey

United Kingdom United States Uruguay Venezuela Vietnam Yugoslavia Zimbabwe

70

IS0

9001 REGISTRATION FOR SMALL AND MEDIUM-SIZED

SOFTWARE ENTERPRISES

Appendix l3 International Standards on Quality (August 1994) TITLE

DATE

DESCRIPTION

IS0 8402 IS0 9000- 1 IS0 9000-2 IS0 9000-3

1994 1994 1993 1991

IS0 9000-4 IS0 9001 IS0 9002 IS0 9003 IS0 9004- 1 IS0 9004-2 IS0 9004-3 IS0 9004-4 DIS 9004-5 CD 9004-6 DIS 9004-7 NP 9004-8 IS0 10011-1 IS0 10011-2 IS0 10011-3 IS0 10012- 1 CD 10012-2 DIS 10013 CD 10014

1993 1994 1994 1994 1994 1993 1993 1993

Vocabulary Guidelines for Selection and Use (Revision to IS0 9000: 1987) Guidance on Implementing IS0 900 1,9002,9003 (1987 Version) Guidelines for the Application of IS0 900 1 ( 1987 Version) to the Development, Supply and Maintenance of Software Application for Dependability Management Design, Development, Production, Installation, and Servicing Production, Installation, and Servicing Final Inspection and Test Quality Management and Quality System Elements Guidelines for Services Processed Materials Quality Improvement Quality Assurance Plans Project Management Configuration Management (publication expected in late- 1994) Quality Principles (developing outline and specification) Auditing Auditor Qualification Audit Program Management Management of Measurement Equipment Control of Measurement Processes Quality Manuals (publication expected in late- 1994) Economic Effects of the Management of Quality (topical paper, not standard) Continuing Education and Training Inspection and Test Records

NP 10015 NP 10016

1993 1993 1993 1992

ISO: International Organization for Standardization Standard DIS: Draft International Standard CD: Committee Draft NP: New Project Source: Columbia Audit Resources, 116 North Fifth, Pasco, Washington 9930 1-5512; Tel.: (509) 547- 1243, Fax: (509) 547-6307

APPENDICES

71

Appendix C The 20 Requirements of I S 0 9001 (1994) Management Responsibility Quality System Contract Review Design Control Document and Data Control Purchasing Control of Customer-Supplied Product Product Identification and Traceability Process Control Inspection and Testing Control of Inspection, Measuring, and Test Equipment Inspection and Test Status Control of Non-conforming Product Corrective and Preventive Action Handling, Storage, Packaging, Preservation, and Delivery Control of Quality Records Internal Quality Audits Training Servicing Statistical Techniques Source: IS0 9001: Quality Systems, Model for Quality Assurance in Design, Development, Production, Installation and Servicing, 2nd ed.(Geneva: International Organization for Standardization, 1994).

72

ISO goo1 REGISTRATION FOR SMALL AND MEDIUM-SIZED SOFTWARE ENTERPRISES

Appendix D I S 0 900 119000-3 Coverage of Software Development Processes ISO

PROCESS

9001 (1987)

IS0

9000-3 (1991)

Not covered

General guidelines on measurement of product and process

Requirements management

High-level (two paragraphs)*

Limited to contractual approach

Project management

High-level (one paragraph)

Basic elements

Sub-contractor management

Product-purchase view

Basic guidelines for softwaredevelopment sub-contracts

Quality Management System

Heavy coverage, but in terms of conformance to specifications, requirements, and documented processes

Guidelines for software quality planning

Design

Not covered

Recommendation that adequate methods and tools be used

Implementation

Not covered

Recommendation that adequate methods and tools be used

Verification and validation

Basic design reviews

Basic guidelines on testing and acceptance testing

Configuration management

Basic document control

Fundamentals of source-code configuration management

Maintenance and support

Not covered

Basic guidelines on corrective and adaptive maintenance

Measurement

.

* The 1994 release of IS0 9001 has increased the management involvement required in the I S 0 9001 registration process. Source: Fran~oisCoallier, "How IS0 9001 Fits into the Software World:' IEEE Software (January, 1994): 99.

APPENDICES

73

Appendix E Implementation Schedule (Amita Management & Information Technology Adaptors Corp.)

IS09000 MASTER PLAN

Framework Develop Implementation Plan Implementation Plan Review

--

Implementation Pian Approved ~ s s c s cumnt s proccsscs Determine tools used Develop a model process Approve model process P

--

Review Quality Manual Rcvisc Quality Manual Approve Quality manual Conformnnce Prepamtion -Document all Processes --

I

I

4th Quarter

Ycrtr 2

1st Quarter

I 2nd Quarter I 3rd Quarter I 4th Quarter

+

I

I

+

-

Document Processts - Develop LI Rocesses Develop Quality Manual

- -

1st Quarter

Name I S 0 9000 Implcmcntation

Year 1 2nd Quartcr I 3rd Quarter

I

-

I

I

+

A

All processes -- documented Sclcct Auditor Auditor Selected Steady State Operations Validation -lnternnl Audit C o m t i v c Action -

Send Documents to Auditors --Pre Audit Briefing Audit Registration

I

+ I

I

+

1st Quarter

74

IS0

9001 REGISTRATION

FOR SMALL AND MEDIUM-SIZED SOFTWARE ENTERPRISES

Appendix F Typical Components of a Software Company's QMS Documentation The following list of possible QMS documentation appears in the European IT Quality System Auditor Guide. Auditors may look for these subjects or their equivalent during an audit. This list is not exhaustive. Design documentation, such as: design input requirements (all activities) design descriptions (all activities) test (verification and validation) plans acceptance criteria user documentation (including operation, installation andlor training manuals, as appropriate) support and maintenance documentation Planning documentation, such as: design and development activity plans resource plans quality plans Procedural documentation, such as: quality system procedures design and development control procedures methodology procedures configuration management procedures replication procedures distribution control procedures archiving control procedures Reference documentation, such as: manufacturer's reference manuals CADICAM guides or user manuals component reference lists International, European or national standards technical guides Source: European Information Technology Quality System Auditor Guide (Brussels: ITQS, January 1992).

APPENDICES

75

Appendix G QMS Documentation Table of Contents (Software Kinetics Ltd.) Following is the Table of Contents for QMS documentation from an IS0 900 1 ( 1987) registered software company. Table of Contents 1 POLICY STATEMENT

1

2 INTRODUCTION 2.1 Purpose 2.2 Quality System Objectives 2.3 Intended Use 2.4 References/Applicable Documents 2.4.1 Quality System Documentation 2.4.2 Corporate and Operation Management Documentation 2.4.3 Supporting Documentation 2.5 Procedures for Quality System Review, Amendment and Re-Issue 2.5.1 Quality System Review 2.5.2 Quality System Review Responsibilities 2.5.3 Manual Amendment and Re-Issue 2.5.4 Configuration Index Document 2.5.5 Management Review

2 2 3 3 4 4 6 6 6 6 7

8 8 9

3 ORGANIZATION 3.1 Corporate Organization Chart 3.2 Quality Organization Chart 3.3 Facility Identification 4 QUALITY RESPONSIBILITIESAND LEVEL OF AUTHORITY 4.1 President 4.2 Quality Manager 4.3 General Manager 4.4 Project Manager 4.5 Project Engineer 4.6 Software Specialists

15 15 15 16 16 17 17

5 QUALITY SYSTEM POLICIES 5.1 Management Responsibility 5.2 Quality System 5.3 Contract Review 5.3.1 Decision to Respond 5.3.2 Response Development 5.3.3 Response Submission 5.3.4 Contract Award 5.3.5 Planning 5.4 Design Control 5.5 Document Control 5.6 Purchasing 5.6.1 Sub-contractor Control 5.7 Purchaser Supplied Product 5.8 Product Identification and Traceability 5.9 Process Control 5.10 Inspection and Testing 5.1 1 Inspection, Measuring and Test Equipment 5.1 1.1 Software Development Environment 5.12 Inspection and Test Status 5.13 Control of Nonconforming Product 5.14 Corrective Action 5.15 Handling, Storage, Packaging and Delivery 5.16 Quality Records 5.17 Internal Quality Audits 5.18 Training 5.19 Servicing 5.20 Statistical Techniques

6 NOTES 6.1 Acronyms 6.2 Definitions

APPENDICES Appendix A - Quality System - IS0 9001 Requirements Traceability Matrix A1

APPENDICES

77

Appendix H The Documentation Hierarchy of a Typical QMS

. .

sonmlns the qualw policy, O " J " J " 6 Jh~ O U objestira M I of the OMS, rraponslbilltiea and authortty, summaryot sitewide processes crested and mainmined by the IS0 Cmrdinmor organi&ion.wide sppiicstion

lalRRi,- .. . . . . ..' p "1

1

t A;.

+

Quality Procedures

-

a.

.

& ,

>*

%

i 7 7 s ~ + > = e .+ - .. . -. .q .

i'

conmins the nsponsibiinlrs of each department with regad to theqvalilq assurance process, davment control, configuretion management, sornstiveastbn, and intemai audii p m e d u r r j developed and maintainedby tlw ISOCoOrdlnalor department-wideappllcalion

.. .11.:n-~?~r--\:\

>MS-RelatedWork Instructions, Contrai Specifications, and Quality Plans

~ l n 1 8 L 68181186 s mehod. and g u i m l n a , rg., developmmt, quelity, and teat plans h n e n end mslntaln86 by ~ M r sngineers e applicable to specific conlracls, projectl, or tasks

. \

i ,

.

.:'.-r-y++F~qp .. q2

Records ,'

I.

contains evidence of quality and control mesrunmentr mechani8ms. and quality e.g., records wrlnen and maintained by s h a r e engineers

78

ISO

goo1 REGISTRATION FOR

SMALL AND MEDIUM-SIZED SOFTWARE ENTERPRISES

Appendix I Metrics Template The following quality measurement template has been suggested for use in an IS0 900 1-compliant QMS: Quality Attribute Object Environment Perspective Purpose

Scale Collected by Now Target by MonthfYear Minimum Failure Response

What is being measured (e.g., reliability, correctness) The delivered software system Allows for specialization (e.g., large or small number of users) Viewpoint in the quality attribute (e.g., defects that do not affect the customer can be disregarded) Focuses on why data are being collected and how they will be used (e.g., to improve reliability levels for network products) How it will be measured (e.g., confirmed customer incident reports per month) Responsibility allocated (e.g., Quality Manager's Office) Present position, if known (e.g., a current quality measure) Desired position The lowest quality level acceptable Action taken upon failure (e.g., full quality audit of the sub-system)

Source: D.C. Ince et al., "IS0 9000-3 and Software Measurement:' BT Technology Journal (January 1994): 112.

APPENDICES

Appendix J The 5 Levels of the SEI Capability Maturity Model

LEVEL

KEY PROCESS AREAS

CHARACTERISTICS OF

FOCUS

MATURITY L E V E L

1 Initial level

The software process is characterized as ad hoc, and occasionally even chaotic. Few processes are defined, and success depends on individual effort.

2

Basic project management processes are established to track cost, schedule, and functionality. The necessary process discipline is in place to repeat earlier successes on projects with similar applications.

Repeatable level

3

Defined level

4

Managed level

5

Optimized level

heroes '

software project planning software project tracking software sub-contract management software configuration management • requirements management

project management and commitment process

The software process for both management and engineering activities is documented, standardized, and integrated into an organizationwide software process. All projects use a documented and approved version of the organization's process for developing and maintaining software.

organization process focus organization process definition peer reviews training program inter-group coordination software product engineering integrated software management

defined engineering process

Detailed measures of the software process and product quality are collected. Both the software process and products are quantitatively understood and controlled using detailed measures.

quantitative process management software quality management

product and process quality

Continuous process improvement is enabled by quantitative feedback from the process and from testing innovative ideas and technologies.

defect prevention technological change management process change management

continuous process improvement

79

Appendix K The 7 Elements of the Malcolm Baldrige National Quality Award EVALUATION C R I T E R I A E L E M E N T S

1.0 Leadership 1.1 Senior executive leadership 1.2 Quality values 1.3 Management for quality 1.4 Public responsibility 2.0 Information and Analysis 2.1 Scope and management of quality data and information 2.2 Competitive comparisons and benchmarks 2.3 Analysis of quality data and information 3.0 Strategic Quality Planning 3.1 Strategic quality planning process 3.2 Quality goals and plans 4.0 Human Resource Development and Management 4.1 Human resource management 4.2 Employee involvement 4.3 Quality education and training 4.4 Employee recognition and performance measurement 4.5 Employee well-being and morale 5.0 Management of Process Quality 5.1 Design and introduction of quality products and services 5.2 Process quality control 5.3 Continuous improvement of processes 5.4 Quality assessment 5.5 Documentation 5.6 Business process and support service quality

5.7 Supplier quality

APPENDICES

6.0 Quality and Operational Results 6.1 Product and service quality results 6.2 Business process, operational, and support service quality results 6.3 Supplier quality results 7.0 Customer Focus and Satisfaction 7.1 Determining customer requirements and expectations 7.2 Customer relationship management 7.3 Customer service standards 7.4 Commitment to customers 7.5 Complaint resolution for quality improvement 7.6 Determining customer satisfaction 7.7 Customer satisfaction results 7.8 Customer satisfaction comparison

81

VII Glossary

For additional definitions, refer to IS0 8402 ( 1994). Accreditation: The formal recognition by a national standards body of a registrar's ability to certify QMS to IS0 9000 standards consistently, impartially, and competently. A company may be registered to an IS0 9000 standard by a non-accredited registrar; however, the registration is of little value if the market lacks confidence in the registrar's ability to register against defined criteria. Any registrar willing to demonstrate compliance with the criteria and requirements for accreditation may apply to a national standards body (such as the Standards Council of Canada) for accreditation. ANSI: American National Standards Institute. Applicant: An organization applying for an assessment of its QMS by a registrar. ASQC: American Society for Quality Control. Assessment: Same as audit. Assessor: Same as auditor. Audit: Also known as quality audit, which is defined in IS0 8402 as a systematic and independent examination to determine whether quality activities and related results comply with planned arrangements, and whether these arrangements are implemented effectively and are suitable to achieve objectives. Auditor: A person who is qualified and authorized to perform IS0 9000 QMS audits. Auditors are restricted to determining the degree of a QMS's compliance to the standard and listing areas of non-compliance. Auditors cannot give advice or tell companies what they should be doing. Certification: Same as registration. Certificate of Registration: A certificate issued by a registrar recognizing a QMS to be in accordance with IS0 9000 standards; in the case of sofiware companies, certificates show registration to the IS0 9001 Quality Standard using IS0 9000-3 Guidelines for Software. Contract review: By IS0 8402 definition, systematic activities carried out by the supplier before signing the contract to ensure that requirements for quality are adequately defined, free from ambiguity, documented, and can be realized by the supplier.

GLOSSARY

83

Corrective action: As defined in IS0 8402, the action taken to eliminate the causes of an existing non-conformity, defect, or other undesirable situation in order to prevent recurrence. Design review: By IS0 8402 definition, a documented, comprehensive, and systematic examination of a design to evaluate its capability to fulfil the requirements for quality, identify problems, if any, and to propose solutions. Document: Written information containing (for example) instructions or guidance for personnel on what they should do or how they should do it? First-party audit: An internal audit. IS0 9000 requires internal assessment to be completely independent from operational activities. Internal audit: See first-party audit. Internal auditor: A competent staff person authorized and trained to manage and perform internal audits of the organization's QMS. ISO: The International Organization for Standardization, founded in 1946 to develop and promote common standards for the purpose of reducing costs, improving efficiency, and increasing productivity. The objective was to foster the international exchange of goods and services. The Geneva-based organization comprises representatives from more than 90 countries, and includes industry representatives who attend 180 technical committees that draft and revise IS0 quality standards, such as the IS0 9000 series. The IS0 does not actually register QMS or accredit registrars. IS0 is responsible for the development, maintenance, and evolution of international guidelines and standards. ITQS: Agreement Group for Assessment and Certification of Quality Systems in Information Technology and Telecommunications. Management review: Defined in IS0 8402 as a formal evaluation by top management of the status and adequacy of the quality system in relation to quality policy and objectives. Memorandum of Understanding (MOU): An agreement that permits foreign recognition of a company's registration. MOU: See Memorandum of Understanding. NACCB: National Accreditation Council for Certification Bodies (United Kingdom).

9 British Standards Institution, European Information Technology Quality System Auditor Guide (January1992), p. 6.

Objective evidence: By IS0 8402 definition, information that can be proved true, based on facts obtained through observation, measurement, testing, or other means. Pre-assessment: Also known as pre-audit. A preliminary examination of the QMS by a registrar or consultant. In Europe, pre-assessments always include consultancy, and therefore should not be performed by registrars. Preventive action: Action taken to eliminate the causes of potential nonconformity, defect, or other undesirable situation. Procedure: Defined in IS0 8402 as a specified way to perform an activity. QMS: Quality Management System. Quality: IS0 8402 defines quality as the totality of characteristics of an entity that bear on its ability to satisfy stated or implied needs. In the book Engineering Quality Sofrware, quality is referred to as the whole concept of specifying, designing, and implementing software and hardware that meets the requirements of the user. It involves all stages in the life cycle, and thus addresses the various methods and tools that can be used to achieve it. Quality improvement: By IS0 8402 definition, the actions taken throughout the organization to increase the effectiveness and efficiency of activities and processes in order to provide further benefits to both the organization and its customers. Quality management: Defined in IS0 8402 as the overall management functions that determine an organization's quality policy objectives and responsibilities, and implement them by such means as quality planning, quality control, quality assurance, and quality improvement within the quality system. Quality Management System (QMS): Used interchangeably in this book with quality system. It is defined in IS0 8402 as the organizational structure, responsibilities, procedures, processes, and resources for implementing quality management. Elements of the QMS can be tailored to suit particular projects and will be documented in a quality manual. Quality Manager: The individual responsible for the QMS. Correspondence between the registrar and the registering organization will normally travel through this person. IS0 9000 does not require a firm to have a Quality Manager. Quality manual: By IS0 8402 definition, a document stating the quality policy and describing the quality system of an organization.

Quality plan: A document setting out the specific quality practices, resources, and sequence of activities relevant to a particular product, service, contract, or project. The quality plan should clearly define the quality objectives; the types of test verification and validation activities to be carried out; detailed planning of testing, verification, and validation activities; and specific responsibilities for quality activities.lo Quality policy: The overall intentions and direction of an organization with regard to quality, as formally expressed by top management. The quality policy must be understood, implemented, and maintained at all levels in the organization. Quality record: A quality record documents the achievement (or otherwise) of an organization's required quality, and the effective operation (or otherwise) of its QMS.~ Quality system: Used synonymously in this book with Quality Management System. RAB: Registrar Accreditation Board (United States). Registrar: Also known as registration body or accredited organization.A registrar is an accredited, independent organization that conducts audits of QMS and issues IS0 9001, IS0 9002, and IS0 9003 certificates. Registrars are prohibited from providing any kind of consultancy, such as telling an organization how to comply with a requirement of the standard. Registration: A procedure by which an accredited registrar officially declares the QMS of an applicant to be compliant with IS0 9000 quality standards. CAN-P- 10 defines registration as a procedure by which a registration organization indicates the relevant particulars of a supplier's assessed quality system in an appropriate, publicly available list. May also be referred to as certification. Registry mark: The official logo of an accredited registrar that may be used by a registered company in literature and advertising material to publicize the fact that it has been found to be in compliance with IS0 9000 standards. Registrants are usually prohibited from displaying the registry mark on any product. Report: The document recording methods and results of the registrar's audit of an organization's QMS.

'

10 Ibid., p. 9. 1 1 Ibid., p. 6 .

SCC: Standards Council of Canada. Second-party audit: An audit of a supplier undertaken by or on behalf of a purchasing organization. Software: IS0 defines software as a program, or set of programs and associated data, procedures, rules, documentation, and materials concerned with the use, operation, and maintenance of an automated information or message processing system or computer system. Software quality assurance: A planned and systematic pattern of all actions necessary to provide adequate confidence that software components conform to the established requirements and specifications.12 Software quality assurance program: The planned, systematic,and integrated series of performance, verification, validation, audit, and review activities, including related management activities, that are implemented to ensure that computer software performs in a satisfactory manner.13 Software quality assurance program procedure: A document that specifies, as applicable, the purpose and scope of an activity; what shall be done and by whom; when, where, and how it shall be done; what materials, equipment, and documentation shall be used; and how the program procedure shall be controlled. l 4 Third-party audit: An audit of an organization undertaken by an independent registration body (in this case, an IS0 9000 registrar). TickIT: An IS0 9000 registration scheme for information technology (IT) developed by the UK Department of Trade and Industry (DTI) with the assistance of the British Computer Society (BCS). Under the TickIT scheme, auditors are required to pass a rigid set of criteria to become TickIT accredited. Total Quality Management (TQM): IS0 8402 defines TQM as the management approach of an organization, centred on quality, based on the participation of all its members and aiming at long-term success through customer satisfaction as well as benefits to the organization and to society-

12 Canadian Standards Association, Quality Assurance Program for Previously

Developed Software Used in Critical Applications. CANICSA-Q396.1.2-89,p. 9. 13 Ibid., p. 10. 14 Ibid., p. 10.

VIII Bibliography

Bamford, R., and Deibler, W. 11, "Comparing, Contrasting IS0 9001 and the SEI Capability Maturity Model.'' Computer (October 1993): 68-70. Billingsley P., "Ergonomic Standards Go Beyond Hardware." IEEE Sofrware (March 1994): 82-84. Canadian Standards Association. Quality Assurance Program for Previously Developed Software Used in Critical Applications. CANICSA-Q396.1.2-89, 1989. CAN-P- 10: Criteria and Procedures for Accreditation of Organizations Registering Quality Systems. Standards Council of Canada, December 1991. Coallier, F., "How IS0 9001 Fits into the Software World." IEEE Sopware (January 1994): 98-100. Coallier, F. and Roberts, J. Letter to Chair ISO/IEC/JTCl and Chair ISO/TC176 on I S 0 9000-3. Point Position Paper, September 1993. Computing Services Association. TickIT Provides Proven Quality Benefits to Both Customers and Suppliers of Sofhyare Systems. Position Paper. London, 1994. Draft - General Requirements for Bodies Operating Assessment and Certification/Registrationof Quality Systems. ISO/CANCO 227 (Rev.), March 1994. European Information Technology Quality System Auditor Guide. Brussels: ITQS, January 1992. Hutchens, S. Jr., "Facing the IS0 9000 Challenge." Compliance Engineering (Fall 1991): 19-25. Hutchins, G. I S 0 9000: A Comprehensive Guide to Registration, Audit Guidelines, and Successful Certijication. Vermont: Oliver Wight Publications, 1993. Ince, D.C. et al., "IS0 9000-3 and Software Measurement.'' BT Technology Journal (January 1994): 109-1 17. Ince, D.C. et al. Introduction to Sofhyare Project Management and Quality Assurance. London, UK: McGraw-Hill, 1993. IS0 9001: Quality Systems - Model for Quality Assurance in Design/Development, Production, Installation and Servicing. 1st ed. Geneva: International Organization for Standardization, 1987.

88

IS0 9001 REGISTRATION FOR SMALL AND MEDIUM-SIZED SOFTWARE ENTERPRISES

IS0 9001: Quality Systems - Model for Quality Assurance in Design, Development, Production, Installation and Servicing. 2nd ed. Geneva: International Organization for Standardization, 1994. IS0 9000-3: Guidelines on the Application of IS0 9001 to the Development, Supply and Maintenance of Software. 1st ed. Geneva: International Organization for Standardization, 1991. IS0 8402: Quality Vocabulary. 2nd ed. Geneva: International Organization for Standardization, 1994. Myers, W., "Debating the Many Ways to Achieve Quality." IEEE Sofhyare (March 1993): 102-103. Puri, S.C. IS0 9000 Certijication and Total Quality Management. Ottawa: Standards-Quality Management Group, 1992. Registration Board for Assessors. Requirements of the National Registration Schemefor TickIT Auditors, January 1994. Schmauch, C.H. IS0 9000for Software Developers. Milwaukee: ASQC Quality Press, 1994. Smith, D. and Wood, K. Engineering Quality Sofrware. New York: Elsevier, 1987. TickIT Guide: Guide to Software Quality Management System Construction using EN29001. British Department of Trade and Industry and the British Computer Society, 1992.

IX Index

Acceptance of final product 59 Acceptance testing 13,51 Accreditation 39,53 alternatives to: MOUs 49 lists of registrars 63 to TickIT 50 Accredited registrars defined 16 list of 62,63 American National Standards Institute (ANSI) 63,64 American Society for Quality Control (ASQC) 63,64 Amita Management & Information Technology Adaptors Corp. 73 Annual surveillance visits 3 1 costs 35 Application fee 35,40 ASQC Quality Progress (periodical) 65 Assessment of documentation by registrar 24 Audit report 30,35 Auditing procedures 48 Auditors 16 escorts for 29 qualifications of 16 Audits 28,29 cost 35 duration 29 preparation 2 1 British Computer Society (BCS) 49 CAN-P- 10 20,3 1

Canadian General Standards Board (CGSB) 16,62 CEEM 65 Configuration management 13,57, 58,60,72 Consultants 23,27,45,67 cost 34 fees 35 Continuous improvement 47 Contract reviews 56 Conventions 60 Corrective action 22,30,56 cost 35 Costs 34 consultants 34 corrective action 35 documentation review 35 internal preparation 34 registrars 40 Courses 48 Criteria for selecting a registrar 19 Delisting 45 Delivery 59 Department of Trade and Industry (DTI) 49 Design review procedures 5 1 Development planning 57 Development procedures 43 DISC TickIT Office 65 Disciplined software development building evidence of 5 1 Document control 60 Documentation 23,25,27,40,56,74

on-line retrieval system 4 1 procedures 43,48 quality policy 42 records and forms 43 work instructions 43 Documentation control 40 Documentation hierarchy 42 Documentation review 27,35 by registrar 24 cost 35 ECITC (European Committee for IT&T Testing and Certification) 51 Engineering Quality Sofhyare (book) 21 Escorts for auditors 29

European Information Technology Quality System Auditor Guide (publication) 67

European IT Quality System Auditor Guide 74 Evidence 51 Groupement QuCbCcois de Certification de la QualitC 16,62 Guide to Software Quality System RegistrationlISO 900 1 (book) 66 IBM 50 Implementation procedures 48 Implementation schedule 36,37 Included software product 61 Information Technology Association of America (ITAA) 66 Installation 59 Internal auditors 22 Internal audits 12,22,56 International accreditation bodies 39

International Organization for Standardization (ISO) address 65 Internet discussion group 67 IS0 8402 70 IS0 9000 highlights 10 IS0 9000 series benefits 13 complaints 52 highlights 10 publications 64 revisions to standard 52 standards I1 I S 0 9000for Software Developers (book) 66 IS0 9000 News (newsletter) 65 IS0 9000- 1 70 IS0 9000-2 70 IS0 9000-3 18,43,44,49,70 comparison with IS0 9001 72 complaints 52,53 IS0 9000-4 70 IS09001 11,12,70 comparison with IS0 9000-3 72 comparison with other quality programs 46 complaints 52 documentation 41 documentation requirements 41 requirements 4 1 IS0 9002 11,70 IS0 9003 11,70 IS0 9004- 1 70 IS0 9004-2 70 IS0 9004-3 70 IS0 9004-4 70 IS0 10011-1 70

INDEX

IS0 10011-2 70 IS0 10011-3 70 IS0 10012-1 70 IS0 Coordinator 16,17,20,22-24, 27,37,42 improves QMS 19 IS0 Steering Committee 16,17,19, 20,22,24,25,37 ISOITC 176 67 ITQS defined 50 publications of 66 Litton Systems Canada Limited 16, 62 Lotus Notes 41 Maintenance 59 Malcolm Baldrige National Quality Award 46 Management commitment 18 Management representative 17 Management responsibility 55 to lead IS0 registration effort 18 Management support 24 Measurement 60 Memorandum of Understanding (MOU) 49 Metrics 43,78 National accreditation bodies 39,50 National Accreditation Council for Certification Bodies (NACCB) 63,64 Non-compliance 30,52 responses to notices of 30 Notices of non-compliance also see non-compliance 52 Operating procedures 43 Periodic surveillance 35,40,52 also see annual surveillance visits

31

Post-audit compliance 30 Pre-audit assessment 26,27 cost 35 Pre-audit meeting 29 Prior Data Sciences Ltd. 50 Procedures 12,13,23,42,43 audit 48 development 43 documentation 48 implementation 48 operating 43 sub-contracting 43 task 43 testing 43 violation of 52 work instructions 43 Process improvement 25 Process measurement 44 Purchasing 6 1 Pyramid model 42 Quality Management Institute (QMI) 16,63 Quality Management System (QMS) 20,25,56,83 framework 55 improvement 3 1 life-cycle activities 56 supporting activities 60 Quality manual 23,24,41,42,5 1, 56 amendments to quality manual 24 controlling printed copies 24 Quality planning 58 Quality policy 42,55 Quality Progress (periodical) 65 Quality records 60

91

Quality Systems Update (periodical) 65 Re-audits 31,43 Records and forms 43 Registrar accreditation 39 Registrars delays 36 lists of accredited registrars 62, 63 process used by 16 selection of 39 Registration 3 1 critical success factors for 33 life span of 45 of multiple sites 40 removal of 45 suspension of 45 time required for 36 Registration schedule 36 Replication 59 Requests for proposals (RFPs) by registrars 19 Requirement specification 57 Rules 60 Scope of business activity 19 SEI Capability Maturity Model 46 SGS International Certification Services Canada Inc. 16,63 Software Engineering Institute also see SEI 46 Software Kinetics Ltd. 19,42,75 Sofiare Process, Quality e5 IS0 9000 (periodical) 66 Standards Council of Canada (SCC) 20,53,62,63 address 63 Sub-contracting procedures 43 Systems and Software Ltd. 66

Task procedures 43 Technical committees 67 Testing 11,13,43,5 1,55,57,58 TickIT 48,49 certification 49 courses 48 defined 49 TicklT Case Studies 65,67 TicklT Guide 65,67 TicklT News 65 TicklT Video 65,67 Tools and techniques 60 Traceability matrix documentation to IS0 9000-3 23 documentation to IS0 9001 23 Training 18,22,25,36,43,48,61 audit awareness 29 IS0 9000 awareness 22 Trial audit also see pre-audit assessment 26, 27 Warnock Hersey Professional Services Ltd. 16,63 Work instructions 43

Questionnaire The terms and conditions of the contract with Industry Canada require an evaluation of this publication be performed. It would be appreciated if you could return this form to the address below. 1 How do you rate the content of this publication?

Good

Poor 1

2

3

4

Excellent 5

2 Would you recommend it to a colleague? No Yes 3 Will you apply what you learned in your business? No Somewhat Extensively 1 2 3 4 5 4 Will you pursue IS0 9001 registration for your software development process? Maybe Yes No -

5 Are you interested in further IS0 9001 education? No Yes

6 If you are interested in further IS0 9000 education, what aspects interest you the most?

Thank you. Please return this survey to: Carleton University Development Corporation Carleton University Ottawa ON K1S 5B6 Canada Fax: (613) 788-3980