Sigh….it happened again. No matter how many explanations provided. Regardless of the questions posed. Ignorant to the wasted funds. It happened again.
You know, I am disappointed with the frequency with which companies throw away large sums of money. Talk about waste and abuse! Hundreds of thousands of dollars per company are tossed annually in the name of perpetuating organizational culture audits, employee engagement surveys, and 360-degree leadership assessments. The real shame? The follow-up questions are usually posed. Engagement is going up, but why isn’t productivity? Our leaders are supposedly world-class, yet why can’t they affect the culture or retain good talent?
Okay, I’ll level with you….the real shame isn’t that these questions are posed by companies that somehow made to onto the Fortune 500 list. It’s that the leaders either don’t care about the answers, or just aren’t bright enough to know what to do with the answers. And you know what the answer almost always is? Lack of validity in the instruments they are using to measure engagement, organizational culture, and leadership, along with flawed administration of these same tools.
See, the best laid plans are worthless if the wrong mechanism is used in pursuit of a goal. Just as the Boston Red Sox desperately need good pitching, if their front office goes out and signs another slugger, their losing ways will continue. Organizational development tools, even if marketed in the most flashy, impressive fashion, are still only as good as their underlying validity and reliability estimates. So, why can’t executives understand that?
It’s simple. Most executives simply don’t know any better. Their backgrounds are not typically heavy on statistical analysis and evaluation, and more than they usually have knowledge of the intricacies of operational tasks on the production line. In fact, I’d argue that in the vast majority of cases, it would be an inappropriate use of their time to spend it researching instrumentation metrics. No, they leave that to those further down in the organization. The secondary problem? Most further down also lack the requisite knowledge to adequately determine the validity and reliability of data instruments.
The solution, then? Executive coaches need to impress upon senior executives in all organizations the vital nature of either hiring specified statistical experts or bring in that talent to help educate decision makers (and if need be to facilitate) on the selection of appropriate instruments.
Engagement surveys that fail to produce ROI are likely well-packaged and marketed scams. Leadership development programs without demonstrable assessment and results-oriented measurement are fiscal vacuums. And organization culture vendors that fail to produce financial outcomes are snake oil salesmen.
We all know the definition of insanity – continuing to do the same thing, while expecting different results. Take a look at the statistical reliability and validity of your chosen tools and surveys. Don’t be an insane leader…..
I/O Musings of a Skeptical Positivist 
Very good point Trevor, I think I shall have a talk with my boss. Falling into this trap happens very easy, you think you are doing something different to change a problem when, in fact, you are still creating the outcome, just in a different way. 🙂
I can tell that you simply comprehend this topic, nicely written and nice job. I am thinking this is the greatest post I have read in your blog yet. I will check back typically, I am glad I came across your internet site. <Harvard Business Review