|
This Guest paper was submitted
for publication January 2010. It is copyright to Dr. Paul D. Giammalvo under
the Creative Commons License 3.0 BY, NC, ND
|
|
Limitations to this Research and Opportunities for Future Research
A common theme throughout this paper has been to assert that it is a very preliminary and experimental research approach and that the model created is obviously not without its weaknesses. Every attempt has been made to identify these weaknesses with the hope and expectation that it will serve as a catalyst in igniting interest in further study on this topic.
To summarize the limitations of this model and set the stage for further research, here are the key topics:
- Relative difficulty of the exams: It is impossible to compare the relative difficulty of the different exams without being able to access at least a representative sample of questions from each. Based on first-hand experience, having taken both the PMP and CCE exams, we can say with some certainty that the AACE exams are much more difficult than the PMP exam. But unless one has taken all the certifications, it becomes impossible to make a fair comparison. There needs to be some way of accessing, comparing and rating the relative difficulties of the various exams is needed.
- Passing scores: On the one hand we have the PRINCE2 credentials, which require only a 50% to 55% grade to pass, while the AACE exams require a 70% score on the problem solving sections and a pass/fail basis for the narrative portion. Somewhere in between is the PMP exam that, at least until a year or so ago, required only 106/175 questions correct or 60.6% to pass. Moreover, there is no data available to see if the passing grade is consistent with the difficulty of the exam. There needs to be some consistent, independent approach to match the passing grade with the difficulty of the exam questions, both internally and between one organization and another.
- Different types of questions: Both the AACE and asapm exams consist of multiple choice and narrative style questions, while the PMP exam consists only of multiple-choice questions. As written communications is an essential part of being a project or program manager, the use of only multiple-choice questions would appear to be unrepresentative. There needs to be a sound psychometric test that mixes multiple choice, essay, matching and fill-in-the-blank questions that will consistently measure differing levels of knowledge.
- Total number of questions and time limits: Another cause for concern is the total number of questions and the average time allowed for answering each. The PRINCE2 Foundation consists of 75 questions with a 1-hour time limit, i.e. 0.8 minutes per question, while AACE's Certified Cost Engineer (CCE) credential consists of 84 questions spread over 7 hours, i.e. 5 minutes per question. This compares to the PMP exam that allows 4 hours to complete 200 questions, i.e. 1.2 minutes per question. There needs to be some independent research done to establish a basis for comparison between the various professional organizations.
- Calculating the value of a bachelor's degree: There is evidently a big discrepancy in how the value of a degree is calculated in terms of level of effort. There needs to be a more realistic way to calculate the value of a degree vis a vis work experience.
- Validating work experience: The practice of the different professional organizations varies widely. PMI does not validate whether the work experience claimed was on successful projects. They do audit 10% of the applicants, but only verify that the hours claimed were correct and not whether the project was a success or failure or whether the work done by the applicant was satisfactory. With the exception of AACE's C3PM, which requires that a logbook of work experience be submitted, AACE, like PMI, does not actually verify the work experience. There needs to be a way of encouraging all organizations to address this in a more robust manner.
- Determining the ideal or appropriate ratio of Testing and Assessment vs. Education and Experience: This too is an area that needs more rigorous research and assessment. To raise the professional image of the practice of project and program management, a closer look at the ratio of experience and education compared to doctors, lawyers, commercial airline pilots and accountants is needed.
|
| |
|
Anomalies or Observed Discrepancies in the Model
