{"id":1115,"date":"2009-10-28T01:00:39","date_gmt":"2009-10-28T09:00:39","guid":{"rendered":"https:\/\/systematichr.com\/?p=1115"},"modified":"2009-10-22T20:10:06","modified_gmt":"2009-10-23T04:10:06","slug":"normative-data-for-employee-surveys-%e2%80%93-worth-the-spend","status":"publish","type":"post","link":"https:\/\/systematichr.com\/?p=1115","title":{"rendered":"Normative Data for Employee Surveys \u2013 Worth the Spend?"},"content":{"rendered":"

Guest Author:\u00a0 Stephen B. Jeong, Ph.D.<\/em><\/p>\n

As a child, whenever I would screw up, my mother always said, \u201cWhy can\u2019t you be more like Billy?\u201d\u00a0 Billy was a straight-A student who excelled in every sport with which he was involved \u2013 an all-around \u201cwunderkind\u201d who could do no wrong.\u00a0 Needless to say, I didn\u2019t like being compared to Billy all that much.<\/p>\n

Those of you who have had experience with employee surveys \u2013 satisfaction or engagement \u2013 may be familiar with the concept of \u201cbenchmarking.\u201d\u00a0 Benchmarking involves comparing a company\u2019s survey scores \u2013 on a range of topics such as communication, supervision, engagement, and efficiency \u2013 with scores from a group of companies on comparable survey questions.\u00a0 Most commonly utilized benchmarks are scores from companies that fall into the same or similar industry sector, or scores from companies deemed \u201chigh performers.\u201d\u00a0 This idea of gauging our company\u2019s performance in reference to other companies can be tremendously appealing \u2013 it\u2019s simple, intuitive, and sexy.<\/p>\n

Notwithstanding the above, I find it hard to justify \u2013 from a scientific point of view \u2013 the current level of enthusiasm for the use of these benchmarks.\u00a0 There is no doubt, when carefully selected, normative benchmarks can provide useful insights into an organization\u2019s standing.\u00a0 Before pulling out your pen and checkbook, however, I\u2019d like to point out a few things for your consideration.<\/p>\n

    \n
  1. Differences in company strategy \u2013 While most companies have in common the goal of increasing revenue, there are key strategic differences among companies that work to diminish the actual (as opposed to perceived) value that benchmarks bring to interpretation of survey data.\u00a0 Depending on their strategic goal, one company might emphasize \u201cinnovation,\u201d while another, \u201cefficiency.\u201d\u00a0 One might focus on \u201ctraining,\u201d while another, on \u201cR&D.\u201d\u00a0 In other words, companies vary on the extent to which they place more or less value on one or more aspects of their operations or culture.\u00a0 This is true even those within the same industry. Take the \u201cmicrochip\u201d industry; some are now focused on producing cheaper solar panels while others are continuing to pour money into improving wafer machines.\u00a0 These varying strategies can and do have an impact on survey scores.\u00a0\u00a0 So, what am I saying?\u00a0 The point here is that an overall survey score of 89% (satisfied employees) on \u201cinnovation\u201d may be fabulous for one company, but unacceptable for another.\u00a0 So, drawing conclusions from the difference observed between one company\u2019s score on \u201cinnovation\u201d or \u201ccustomer service\u201d against a group of other companies (even those in the same industry) downplays the meaningful differences that exist among these companies.<\/li>\n
  2. Timing of data collection (historical effects) \u2013 Employee survey is a collection of attitudes.\u00a0 Attitudes, in turn, are susceptible to constant fluctuations in one\u2019s emotional state.\u00a0 Imagine that your company just announced the second round of layoffs and reported that revenues were less than expected for the past quarter.\u00a0 We all know that the conditions \u2013 both internal and external \u2013 can impact our responses to survey questions.\u00a0 This is what statisticians call \u201cerror.\u201d\u00a0 This means that, any temporary condition \u2013 like layoffs \u2013 that can either inflate or deflate survey scores can contribute to increasing the size of this error.\u00a0 Imagine now, the timing of the surveys from different companies that make up a given benchmark.\u00a0 It is highly unlikely that the data were collected within the same month or even the same year.\u00a0 We\u2019ve gone through a fairly significant roller-coaster ride in the past 12 months.\u00a0 Can you really draw firm conclusions from your Q2 of 2009 surveys scores when compared to data collected between Q2 of 2007 through Q2 of 2009?<\/li>\n
  3. Importance of past performance (historical trending) \u2013 Benjamin Franklin emphasized the importance of gauging current performance using past data.\u00a0 For example, in order to stop cursing, he carried around a notepad to keep track of the number of times he cursed each day.\u00a0 After several weeks, he would draw a simple chart to check his progress.\u00a0 Similarly, one of the most important diagnostic tool available to organizations is historical survey data. Historical trend data provide information that, I would argue, is substantially more important than comparison to external benchmarks.\u00a0 This is primarily because one company\u2019s culture \u2013 like one\u2019s personality \u2013 tends to remain fairly stable over time.\u00a0 This means that, any significant shift in upward or downward direction (as measured by standard or average deviation) tells a lot about what is happening to different aspects of that company\u2019s culture.\u00a0 Moreover, because you are likely to be aware of the changes that your company has undergone in the past 12 months, you are able to more reliably factor this into your interpretation of the results.\u00a0 From this perspective, historical shifts deserve much more attention than any discrepancy found between your survey results and some external benchmark data.<\/li>\n<\/ol>\n

    To summarize, there\u2019s quite a bit of \u201chype\u201d tied to the use of benchmarking data; more so than can be justified.\u00a0 While they can provide useful information when selected and used appropriately, differences in company strategy, cultures, and historical effects all work to make external benchmark data, in general, less useful than they appear on the surface.\u00a0 In worst cases, benchmarks can lead to grossly misleading conclusions and what I would call here the \u201cWhy can\u2019t you be more like Billy\u201d syndrome.\u00a0 Well-functioning companies are like Olympic athletes, you don\u2019t need to be good at everything to win the gold, just your event.\u00a0 By the way, Billy is now a history teacher and although I watch the History Channel from now and then, I would never think for a moment about trading professions.<\/p>\n

    Stephen B. Jeong, is currently the Managing Director of Waypoint People Solutions – www.waypointps.com<\/a>, a human capital consulting firm that focuses on high precision employee diagnostic surveys using cutting-edge measurement technology and methodologies. He holds Ph.D. in Industrial-Organizational psychology from the Ohio State University and has been advising private, public, and government organizations since 2000.\u00a0 He can be reached at stephen.jeong@waypointps.com.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

    Guest Author:\u00a0 Stephen B. Jeong, Ph.D. As a child, whenever I would screw up, my mother always said, \u201cWhy can\u2019t you be more like Billy?\u201d\u00a0 Billy was a straight-A student who excelled in every sport with which he was involved…<\/p>\n","protected":false},"author":15,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","footnotes":""},"categories":[37,27,10,33,8,22,44],"tags":[],"class_list":["post-1115","post","type-post","status-publish","format-standard","hentry","category-change-management","category-data-metrics","category-engagement","category-finance","category-strategies","category-industry-news","category-innovation"],"_links":{"self":[{"href":"https:\/\/systematichr.com\/index.php?rest_route=\/wp\/v2\/posts\/1115","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/systematichr.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/systematichr.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/systematichr.com\/index.php?rest_route=\/wp\/v2\/users\/15"}],"replies":[{"embeddable":true,"href":"https:\/\/systematichr.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=1115"}],"version-history":[{"count":4,"href":"https:\/\/systematichr.com\/index.php?rest_route=\/wp\/v2\/posts\/1115\/revisions"}],"predecessor-version":[{"id":1122,"href":"https:\/\/systematichr.com\/index.php?rest_route=\/wp\/v2\/posts\/1115\/revisions\/1122"}],"wp:attachment":[{"href":"https:\/\/systematichr.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=1115"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/systematichr.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=1115"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/systematichr.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=1115"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}