IT Certifications Revisited:
A Closer Look at the Real Meaning of IT Certifications

Abstract

Individuals and enterprises spend literally billions of dollars each year on training and testing that is primarily for the purpose of acquiring professional certifications. We ask, what's actually in it for the individuals and the enterprises?

---

WHAT IS CERTIFICATION?

The meaning and value of certifications are widely misunderstood. One reseller of certification tests and practice quizzes boldly states, "Professional certification is a process by which a person develops the knowledge, experience, and skills to perform a specific job."[1] This is utterly wrong.

Certification is a procedure by which an agency assesses an individual's knowledge in a certain area, usually by requiring that the individual pass an online multiple-choice test, and avers to their knowledge based on the results.[2] Rarely -- for example, with some CompTIA, Cisco and RedHat certifications -- the procedure also includes some observed performance of tasks. Even more rarely, one must have some amount of "verified experience" working in the field to gain a certification. But certification is not usually a process that "develops the knowledge, experience, and skills to perform a specific job," but is rather a process that develops the knowledge, experience, and skills to pass a certain test.

Certifications, by the way, should not be confused with certificates. Some think of a certificate as a lightweight quasi-degree from an institution of higher education or recognized provider of training and/or education, but certificates may be offered by any enterprise that cares to put ink to paper. To further confuse matters, even top notch universities are now issuing certificates for which there are no institutional or programmatic accreditation.[3] Certificates merely state that an individual has completed some course of study, perhaps a single, brief course. In any case, certificates are not the same thing as certifications.

There are literally hundreds of certifications available,[4] and if one counts "certificate programs," the number reaches into the thousands.[5] We'll focus on certifications. The concept of certification seems sound enough. That one can provide correct answers at an established rate from an established bank of questions in a manner that is statistically consistent would seem to suggest that one knows the same things as others who pass the same test.

So that is exactly what the industry claims: that common sense dictates that those with certifications know things. And that is the extent of their assertion.

A typical industry white paper from an enterprise which is selling training intended to prepare the student to pass certification tests tells us, "Obtaining a professional certification is an indication that you're an expert. It's proof to employers, peers and even yourself that you speak and think in a specific language. It comes with the expectation that co-workers and decision-makers will call on you for advice and insight. Your experience and expertise will be key to a business' growth in the face of internal and external changes."[6] How careful is the language! The certification is "an indication" of expertise. The certification is "proof" only that one can "speak... a specific language." The certification may cause colleagues to "call on you for advice," but offers no assurances as to whether you'll be able to provide such advice. The certification means that you "will be key" to the business -- an unmeasurable standard, and a meaningless statement.

Pearson VUE, one of the two monsters of the certification testing industry, pushes certification not because they can prove it means something, but because it might mean something. They say, "Certifications can have a direct effect on hiring decisions, customer satisfaction, sales opportunities, salaries and regulation of ethical practices." They argue in favor of certification because certifications can affect the behavior of decision-makers.[7]

Perhaps the loudest and most important claim of the certification industry is that those with certifications earn more than those without. However, we do not know whether or not certifications mean more pay. We know that people with more knowledge and more years of experience get paid more than those with less knowledge and experience. We know that people with more knowledge and more years of experience tend to have more certifications than those with less knowledge and experience. We know that if you do a study to determine whether or not people with more knowledge and more years of experience and more certifications get paid more than those with less knowledge and less experience and fewer certifications, you will find they do. What we do not know is whether, all other things being equal, those with certifications get paid more. No objective, double-blind studies have been conducted. The notion that certifications lead to increased pay is credible but unproven.

Yet as certifications have become more commonplace, they have, ironically, come to mean less and less. Again, those who are selling training that is intended to prepare the student to pass certification tests tell us, "In terms of standing out, just one certification may not be enough. Globally, IT professionals hold an average of about three certifications apiece." This suggests that certifications mean so little that you have to have many before anybody accepts that they mean anything at all.[9]

WHAT IS COMPETENCE?

Competence is another thing entirely. "Competency... refers to the skill and knowledge needed to successfully complete a task. Those who have competencies are qualified to perform their work safely and often with little or no supervision."[10]

We know how people become competent: a combination of experience and training. In the IT arena, such training necessarily includes hands on work completing comprehensive, task-oriented exercises under the watchful eye of a personal facilitator. This, by the way, rules out the many e-learning solutions that lack any sort of facilitation. While stand-alone e-learning may help sharp, motivated learners to become competent, it does not assure it, while task-oriented, in-depth, hands on, facilitated training does.

It is of course possible, although non-trivial, to measure competence. MTCS, a British company that offers "training and competence solutions," explains that while they are willing to certify the competence of professionals in certain fields, the process of doing so does not involve mainstream certification testing at all. It involves (this list is taken verbatim from their website):

• Observation records, which is someone observing you carrying out an activity (your skill level is also being checked).
• Answering questions, which is an assessment of your knowledge
• Making or fixing something
• Supporting written evidence - a report perhaps
• Witness statements completed by a reliable witness
• Appraisals completed by Supervisors and Managers
• Log book pages and Survival and Medical certificates
• Feedback from your Assessor
• Photographs
• Videos [11]

The conspicuous absence of conventional certification testing from the process is telling. Despite the numerous certifications that exist in their markets, they don't even make passing reference to them in describing how they determine competence.

We do not argue that certification has no place in the process of assessing competence. We do argue that if one seeks to assess competence, or to identify the competent, certifications are at best one of many indicators that must be carefully examined in order to reach a conclusion.

WHAT DO ENTERPRISES NEED?

Enterprises need competent employees. Hiring managers and human resource departments strive to identify prospective employees who will perform well on the job, and they usually wish to find rather than develop such employees. That is, when, for example, the Ferd's Sporting Goods chain wishes to hire an IT professional to manage all or part of their IT infrastructure, they almost always seek to hire someone who can do the job, rather than someone who has to learn to do the job. Enterprises seek to hire employees who will perform well on the job. This obvious truth is an important one: enterprises do not care about certifications, they care about on-the-job performance. If certifications provide compelling evidence that a candidate is more likely to perform well on the job -- and only if the certifications do so -- then they have value to enterprises.

WHAT DO INDIVIDUALS NEED?

Job seekers need knowledge, skills, and evidence of competence. Employees benefit from any activity that increases their competence, such as training and on-the-job experience. They all benefit if they have credentials that either present compelling evidence of competence or that increase their value in the job market. If certifications provide compelling evidence that a candidate is more likely to perform well on the job, or if those with certifications consistently are paid more than those without, then they have value to individuals.

WHAT DOES CERTIFICATION PROVE OR SUGGEST REGARDING COMPETENCE?

Astonishingly little. As we discussed above, those who offer certifications go to great lengths to avoid making any claims regarding the probable on-the-job performance of those with certifications.

This is worth noting: we at SST have no profit motive with regard to the question of whether or not there is a correlation between certifications and competence. Our training assures that attendees will become competent and thus also will be able to prepare for and pass related certification tests. And the evidence suggests that competent folks and certified folks are more valuable and get paid more. Nonetheless, we'd like to know if there is actual evidence that those with certifications perform better on the job than those without.

It is not as if it would be impossible to prove the case one way or the other. Any institution or enterprise that cared to do so could fund a large double-blind study in which the on-the-job performance of professionals was independently assessed, then look at the certifications of the individuals, and determine if and how certifications were correlated with competence.

We looked for such a study. Starting at the reference desk at a nearby university, we searched the WWW, publishers' web sites, and numerous academic sources for work related to our question. We looked at hundreds of papers that touched on the issue of professional certifications from publishers such as:

• JSTOR (Journal Storage) (https://www.jstor.org/)
• Elsevier (https://www.elsevier.com/)
• TandFonline (Taylor and Francis) (https://www.tandfonline.com/)
• Emerald (http://www.emeraldgrouppublishing.com/)
• De Gruyter (https://www.degruyter.com/)
• NEJM (https://www.nejm.org/)
• NIH (https://www.nejm.org/)
• Google Scholar (https://scholar.google.com/)

We cannot claim to have read every extant paper. Elsevier alone responds to a query for "studies on IT certifications" with a list of 51,852 books, journals or webpages. But we looked long and hard.

While it is impossible to prove a negative, we feel quite confident when we say this: no studies on the correlation of certifications and job performance or competence have been conducted.

We also made inquiries to the two largest companies that provide certification testing, asking the question, "Are there any studies that indicate that in areas such as IT, employees who have passed certification tests are more competent on the job than those who have not?"[12] Prometric did not respond. Pearson VUE responded by sending statistics that summarized results of a survey[13] of people who took Pearson VUE certification tests. The survey did not address the question we asked. The survey responses strongly support the notion that training is a good way to improve job performance, advance professionally, and increase income, and suggests that a combination of training and certifications is superior to merely training, but is conspicuously silent on the question of whether certified professionals outperform non-certified professionals.

Beyond that, what we did find was interesting, but often off-point and largely unconvincing.

We found that there were papers on how such studies could be performed, and on how studies that were related to the subject could be refined so as to reveal useful information.[14]

We found that there is evidence that there is a correlation between more rigorous teacher certification and job performance.[15]

We also found that not everyone agrees that there is a correlation between more rigorous teacher certification and job performance. For example, one study suggested that new teachers who have certifications are no more or less effective than new teachers who do not have certifications.[16]

We found that certification is indeed not the same thing as competence: "Certification means qualification. But, having a qualification, by education or training, does not necessarily indicate that the person is competent. Competence means successful demonstration of all competencies in the qualification topic(s)."[17]

We found papers that suggested that there may be a correlation between "expert ratings and certification," but which subsequently stated that such conclusions were "susceptible to alternative interpretations due to design constraints." (In other words, they were flawed studies.)[18]

We found papers that suggested while certifications "have conceptual and practical appeal, evidence of their efficacy is thin and quality assurance is weak."[19]

We also found that, as a practical matter, very few certifications matter much in the marketplace. An analysis from the research firm Burning Glass Technologies demonstrated that in unlicensed fields like IT, barely 9% of job listings asked for even one certification, and of those few that did, 66% of the requests are for one of the 50 most popular certifications, and 75% were for one of the top 100.)[20]

We found a lot of ambiguity. For example, a study about nursing certification stated, "Supporting literature suggests that certification reflects competence in a specified area of practice. Contrary to this belief, competency with certification has not been supported by empirical evidence, thus rendering the ideas only assumptions." The study further suggests that "credentials are indicators of what an individual should be able to do, not what that individual can or will do."[21]

The closest thing we could find to a claim that certifications of any type had any relationship to actual competence on the job came from a 2009 article in the American Journal of Critical Care.[22] The authors used hierarchical linear modeling in a secondary data analysis of 48 samples and came to the lukewarm conclusion that "Specialty certification and competence of registered nurses are related to patients' safety. Further research on this relationship is needed." This is hardly convincing.

Perhaps most important, we found that while there is essentially no evidence that having a certification means that one is competent, competent people can always get certified. The most important and accurate way to measure competence is by direct observation from experts who observe individuals as they successfully complete tasks, which is not required by purveyors of most of the popular certifications.

CONCLUSIONS

There is no compelling or even mildly persuasive evidence that individuals with IT certifications perform better on the job than those without such certifications.

Certifications may be a good investment for an individual, especially one fortunate enough to have an employer who is willing to provide the time, pay for the courses, and then pay for the tests required to acquire certifications. Certifications may help the job seeker get an interview, and may qualify them for higher pay. Those with certifications seem to get paid more than those without, although there is scant evidence of causation.

Employers who want to assure that they hire employees who are competent are much better served by doing a reasonable job of screening candidates, and carefully checking references, than by demanding that candidates have certifications. And employers who wish to improve the performance of employees are much better served by focusing on training and their own assessments of competence than by sending off unsuspecting workers in search of meaningless certifications.

Employers are best served by training for competence, because competence assures both value as a contributor in the work environment and the ability to become certified. But the converse is not true: certification does not assure competence. That is because rather than undertake comprehensive, competence-building training, individuals can undertake so-called "brain dumps" which are little more than rote test preparation, and can get certified even with rather large gaps in knowledge and experience. There is simply no evidence to the contrary.

Phineas Longstaff
Chief Education Officer
Software Skills Training, Inc.

---

Phineas Longstaff

Phineas Longstaff, the CEdO (Chief Education Officer) at Software Skills Training.com, has worked in the field of technical education for almost 40 years as a technical trainer, programmer, course developer, media developer, manager, executive, entrepreneur, editor, and author. He welcomes your comments, criticism and input. Please contact him at Phineas.Longstaff@software-skills-training.com.

---

BIBLIOGRAPHY

The sources below were consulted in the preparation of this paper. Those sources which were directly quoted are listed only in the endnotes, which are below the bibliography.

Note: for WWW-based sources, we have included the dates on which we most recently accessed links so that should posts be deleted or links become inaccurate, readers can use the Internet Archive (https://archive.org/web) to find sources.

Gilbert, Nestor. (January 12, 2017). History of Learning Management System [sic]. FinancesOnline. [Online]. Available: https://learning-management.financesonline.com/history-learning-management-system/, accessed July, 2019.

Gorski, Tom. (Feb 14, 2018). How many learning management systems are there? Quora. [Online]. Available: https://www.quora.com/How-many-learning-management-systems-are-there, accessed July, 2019.

Green, Alison. (May 19, 2012). Why are employers so bad at training people? AskAManager.org. [Online]. Available: https://www.askamanager.org/2012/05/why-are-employers-so-bad-at-training-people.html, accessed July, 2019.

Kamath, Akshatha. (May 29, 2019). How To Measure The Effectiveness Of Corporate Training. Simplilearn. [Online]. Available: https://www.simplilearn.com/how-to-measure-effectiveness-corporate-training-article, accessed July, 2019.

Kim , Joshua. (September 19, 2017). How Big is the LMS Market? Inside Higher Ed. [Online]. Available: https://www.insidehighered.com/blogs/technology-and-learning/how-big-lms-market, accessed July, 2019.

Markow, Will, Dan Restuccia and Bledi Taska, and Scott Bittle. (2017.) The narrow ladder: The value of industry certifications in the job market. Boston, MA, Burning Glass Technologies. [Online]. Available: https://www.burning-glass.com/wp-content/uploads/BurningGlass_certifications_2017.pdf, accessed August, 2019.

McNamara, Carter, Ph.D. Evaluating Training and Results (ROI of Training). Free Management Library. [Online]. Available: https://managementhelp.org/training/systematic/ROI-evaluating-training.htm, accessed July, 2019.

Patten, Brigg. (May 23, 2016). The Good, Bad and Ugly of Employee Training. Training Industry. [Online]. Available: https://trainingindustry.com/articles/strategy-alignment-and-planning/the-good-bad-and-ugly-of-employee-training/, accessed July, 2019.

Why Employee Training Fails And How To Prevent It In Your Organization. (July 24, 2019). Panopto. [Online]. Available: https://www.panopto.com/blog/why-employee-training-fails/, accessed July, 2019.

---

ENDNOTES

1. Kyle, Mary. (May 5, 2015.) Choose A) Performance-Based Test, or B) Multiple Choice Test. GoCertify. [Online]. Available: http://www.gocertify.com/articles/choose-a-performance-based-test-or-b-multiple-choice-test.htm, accessed August, 2019.
2. Definitions of the term certification, as it applies to the industry that offers testing as a means of providing independent confirmation of the knowledge of professionals, can be found in a great many places, such as the online Business Dictionary (http://www.businessdictionary.com/definition/certification.html) and the Oxford English Dictionary (https://www.lexico.com/en/definition/certification). In general, individuals are not required to do more than take a test or tests to acquire certifications, and our definition reflects this.
3. Database of Accredited Postsecondary Institutions and Programs; Massachusetts Institute of Technology. (2019.) US Department of Education, Recognition and Accreditation. [Online]. Available: https://ope.ed.gov/dapip/#/institution-profile/121415, accessed September, 2019.
4. The Wikipedia page titled List of professional designations in the United States lists about 650 certifications. While Wikipedia cannot be considered to be a completely reliable source, the number is credible. See: https://en.wikipedia.org/wiki/List_of_professional_designations_in_the_United_States, accessed August, 2019.
   Further research reveals that Prometric, a major provider of certification services, offers 421 certification tests: https://www.prometric.com/en-us/for-test-takers/pages/find-my-test.aspx.
   Corporate Minority, a free career resource platform that focuses on minority groups, lists 206 certifications: https://corporateminority.com/list-professional-certifications-career/
   Clearly, there are a great many professional certifications.
5. The Guide to Online Schools lists 7,452 certificate programs in the US. [Online]. Available: https://www.guidetoonlineschools.com/degrees, accessed August, 2019.
6. Day, Ryan. (November 2, 2018). 18 IT and Business Certifications Worth Having. Global Knowledge Resource Library. [Online]. Available: https://www.globalknowledge.com/us-en/resources/resource-library/articles/18-it-and-business-certifications-worth-having/, accessed July, 2019.
7. The Value of Certification. (2015). Pearson VUE. [Online]. Available: https://www.pearsonvue.co.uk/Documents/Grow-your-programme/VoC.aspx, accessed July, 2019.
8. Day, Ryan. (November 2, 2018). 18 IT and Business Certifications Worth Having. Global Knowledge Resource Library. [Online]. Available: https://www.globalknowledge.com/us-en/resources/resource-library/articles/18-it-and-business-certifications-worth-having/, accessed July, 2019.
9. Certification Versus Competency: What's the Difference? (November 2, 2017.) DCM. [Online]. Available: https://www.dcmgroup.ca/en/news/certification-versus-competency-whats-the-difference, accessed August, 2019.
10. Warburton, Richard, MD. (2016.) What do we mean by 'evidence' required to support competence certification? MTCS (company specializing in training and competence solutions). [Online]. Available: https://www.mtcsuk.com/mean-evidence-required-support-competence-certification, accessed August, 2019.
11. In August, 2019, we submitted the following question to Prometric.com (via https://www.prometric.com/en-us/contact-us/pages/our-solutions.aspx) and Pearson VUE (via https://home.pearsonvue.com/Test-Owner/Contact-business-development/Email-form.aspx): "Are there any studies that indicate that in areas such as IT, employees who have passed certification tests are more competent on the job than those who have not?" Pearson VUE responded by sending a white paper. Prometric did not respond to email requests or to telephone messages.
12. 2018 Value of IT Certification. (2019.) Pearson VUE. [Online]. Available: https://home.pearsonvue.com/voc, accessed September, 2019.
13. McKillip, Jack and Caryl Cox. (1998.) Strengthening the criterion-related validity of professional certifications. Elsevier. [Online]. Available: https://doi.org/10.1016/S0149-7189(98)00010-X, accessed August, 2019.
14. Hawk, Parmalee P., Charles R. Coble, Melvin Swanson. (May 1, 1985.) Certification: It Does Matter? Journal of Teacher Education. [Online]. Available: https://journals.sagepub.com/doi/abs/10.1177/002248718503600303, accessed August, 2019.
15. Kanea, Thomas J. Jonah E. Rockoffb, Douglas O. Staigerc. (2008). What does certification tell us about teacher effectiveness? Economic of Education Review. [Online]. Available: https://www0.gsb.columbia.edu/faculty/jrockoff/certification-final.pdf, accessed July, 2019.
16. Certification vs. Competency. (2019). Centranum Talent Management Systems. [Online]. Available: https://www.centranum.com/resources/competency-management/certification-versus-competency/, accessed July, 2019.
17. McKillip, Jack and Caryl Cox. (1998.) Strengthening the criterion-related validity of professional certifications. Elsevier. [Online]. Available: https://doi.org/10.1016/S0149-7189(98)00010-X, accessed August, 2019.
18. Brown, Jessie and Martin Kurzweil. (2017.) The Complex Universe of Alternative Postsecondary Credentials and Pathways. Cambridge, MA. American Academy of Arts & Sciences, pp. 4. [Online]. Available: https://www.amacad.org/sites/default/files/academy/multimedia/pdfs/publications/researchpapersmonographs/CFUE_Alternative-Pathways/CFUE_Alternative-Pathways.pdf, accessed June, 2019.
19. Markow, Will, Dan Restuccia and Bledi Taska, and Scott Bittle. (2017.) The narrow ladder: The value of industry certifications in the job market. Boston, MA, Burning Glass Technologies. [Online]. Available: https://www.burning-glass.com/wp-content/uploads/BurningGlass_certifications_2017.pdf, accessed August, 2019.
20. Redd, Nary Lou and Judith W. Alexander. (February 1997). Does Certification Mean Better Performance? Nursing Management. [Online]. Available: http://www.innovationlabs.com/research/pre/reading_materials/article%20Redd.pdf, accessed July, 2019.
21. Kendall-Gallagher, Deborah and Mary A. (2009.) Competence and Certification of Registered Nurses and Safety of Patients in Intensive Care Units. American Journal of Critical Care. Online]. Available: http://ajcc.aacnjournals.org/content/18/2/106.full.pdf, accessed August, 2019.