Wednesday, December 16, 2009

Change Evaluation

It is mind boggling to hear the rationale used by change managers, internal or external, about this topic. Experience shows that the great majority of change efforts fail -- only a few succeed.

Many feel that change programs cannot be truly evaluated because there are too many variables to contend with, that we do not have the luxury or opportunity to set up control groups, etc. Others, to justify their lack of success, will tell you that things would have been much worse without their intervention. Still others will insinuate that not enough time has passed to truly evaluate a change initiative, that we must wait. To me these and other excuses prevent us from advancing the state of our knowledge about organizational diagnosis and intervention.

It has been said that success has many fathers, and failures none. It is easy to appropriate credit when things go well (even though we might not have had much to do with it), but very difficult to be accountable should the results be less than we anticipated.

I have been around long enough to realize that success is highly dependent on favorable market conditions. Market pull makes geniuses of all of us. We come to believe our own PR. The truth of the matter is that the best is required from us when the markets get tough and the likelihood of success much more problematic. I have seen the rise and fall of many giant companies -- only a few have risen from the ashes.

Experience has shown that failures result from three sources: (1) we used the wrong methods, (2) we used the correct methods but improperly, or (3) both. Evaluation is essential to our learning. The feedback we gain helps us adjust our thinking and improve our skill set. Without an explicit change strategy, change managers take the ultimate risk of "flying blind." When you fly blind, you take the chance of running into unexpected consequences, of doing harm to yourself and others, and more hurtfully to be labeled a charlatan.

In my view, it is not a sin to admit that something did not work. As human beings we are not infallible, we do not know all there is to know, and we are prone to make incorrect assumptions. By admitting our humanity, we stand a better chance of becoming masters in our chosen profession. It has been said that most people learn more from their mistakes than they do from their successes. "You can fool some of the people sometime, but you cannot fool all the people all the time." Our clients are intelligent human beings, they are not easily convinced by our contrite rationale.

At a recent meeting I attended in Oxford, one of the well respected participants asked me why had a particular culture change program (now in its 6th year) failed to make the changes stick. I have been reflecting on his question since.

There are two general ways to evaluate change programs, I was taught a few years back -- maybe there are more now but I am not aware of any. One is called the summative approach and the other the formative. In the former case, you compare longitudinally a number of metrics. The metrics are selected prior to the launch of the effort. Success can be established by comparing the gap between "what was" with "what is now". This approach has also been called the "goal attainment method."
Any unfinished business can be incorporated in future changes. This is the simplest form of evaluation but not necessarily the most instructive.

In the case of formative evaluation, you evaluate progress along the way, at critical phases and milestones, and at the end. This approach is much more demanding but can yield important "lessons learnt" in both end results accomplishment as well as to the efficacy of the change methods used.

I strongly urge practitioners to build into the change plan an evaluation component. The type and epth depends on the complexity of the change program. You do not need to use a 10 pounds sledge hammer to drive down a penny nail. All in moderation! I urge a vigorous debate about change efficacy.

In later blogs, I will discuss three specific change evaluation methods, ranging from the easy to the more complex. Choosing the appropriate method is important from a cost point of view. A simple change might not warrant large expenditures for evaluation, and vice versa.

Enjoy the journey along the learning curve.