Security Articles
Certification Exams - The Nuts & Bolts of Development
By Dennis Smith, CPP June 2005
Shortly after I passed the CPP [Certified Protection Professional] certification Exam, I remember wondering about the exam process; not just taking the exam itself, but asking myself questions like - how were the exam questions written? Who decided how to weight the various subject areas? What process is in place to assure that the exam questions accurately reflect the current industry standards? Are the rumors really true about 12 people sitting in a room thinking up "trick" questions to torture people who are taking the exam? Over the past 2 years I've received the answers to those questions and more.
In 2004 I was asked by the Professional Certification Board of ASIS International to participate, with a group of approximately 30 CPPs, in the ASIS Item Development Group [IDG] that met for 2 days in Minneapolis, MN. The purpose for the group was to submit questions for inclusion in a newly designed, updated CPP Certification Exam. The resulting exam update that will be implemented in June of this year is the product of that exercise.
I was contacted again in 2005 by the ASIS Professional Certification Board and asked to be a member of the IDG charged with the responsibility of submitting questions for updating the PSP [Physical Security Professional] Certification Exam. I met with other members of the IDG in May, 2005, in Orlando, FL . I know what you're thinking, an NO, I didn't have time for any of the local attractions.
Sounds easy so far, but the work begins several weeks before the IDG actually joins efforts at the joint meeting location. The first step after being selected as a member of the IDG is to become familiar with A Guide to Preparing Multiple Choice Items, which lays out the guidelines that must be followed in creating questions for submission. Next on the to-do list is reviewing one of the reference materials [specific to the certification exam that your particular group is working on - CPP, PSP or PCI] furnished by ASIS International.
Each IDG Group member is charged with writing a pre-selected number of exam questions [called items], while being sure to follow the rules contained in the guide. The basis for all questions drafted for the exam is a Role Deliniation Study conducted by ASIS International, to assure that current information regarding security management roles reflect the current environment of the security industry. These questions are then sent to a reviewer from an professional testing company. The questions are returned to the writer with suggestions/recommendations for changes or improvements. After applying the auditor's recommendations in re-writing the questions, several hard copies of each question are made and taken to the joint meeting of the IDG. The Rubric of each question submitted must be identified by noting the correct Domain, Task and Knowledge breakdown. For example, the PSP Exam is based on three Domains. Domain I: Physical Security Assessment; Domain II: Selection of Integrated Security Measures; and Domain III: Implementation of Physical Security Measures. Each domain is broken down into applicable Tasks. Domain I contains four Tasks, Task 01: Identify Assets to Determine Their value and Criticality; Task 02: Assess the Nature of the Threats so That the Scope of the Problem Can be Determined; Task 03: Conduct a Physical Security Survey in Order to Identify Vulnerabilities of the Organization; and Task 04: Perform a Risk Analysis so That Appropriate Countermeasures Can be Developed. Each Task contains several Knowledge areas, each with its own numbered identification. A question about the physical security assessment and involves how risk management is used during the process of identifying the value and criticality of assets would carry the Rubric ID 01/01/04. Every question submitted must come from one of the recommended study materials. Each question must accurately reflect the information contained in the study reference.
Once the IDG Group members meet, copies of each question are distributed throughout the group and repeatedly reviewed by other group members. The reviewers decide whether the question and answer accurately reflect information from the noted text, if the question was written within the guidelines of the Guide to Preparing Multiple Choice Items, if the question is assigned to the proper Rubric, etc. At this point, the question can be kept as a valid item, re-written, or rejected.
The process continues as Exam questions that have made it through these initial stages are sent forward to the proper Professional Certification Board, Test Management Committee. The submitted questions are again reviewed, refined or possibly rejected by this group. If the questions make it through this process, they are added to the database and selected to be pre-tested for validity prior to being included on the appropriate certification exam.
Since experiencing this process twice, I have a much greater appreciation for the hard work that goes into development of a high quality professional certification exam. I challenge those of you who have not acquired certification - investigate the value you will receive as a certified professional in your field of specialty. ASIS International offers board certification for Certified Protection Professional, Physical Security Professional and Professional Certified Investigator. Achieving board certification will demonstrate your competency and make you stand out in your field, and rightly so!
If you would like further information on ASIS Professional Certifications, you can find it on the ASIS International web-site at http://www.asisonline.org/certification/index.xml
Dennis Smith, CPP
Chairman, KC ASIS
Security Specialist - American Century Investments