Evidencing Improvement

In my view, the United Kingdom has led the world in the advancement of Lean management in higher education (click here to learn more) – but with two important qualifications:

  • Application has been mainly in administrative processes, not in teaching and other academic processes.
  • An emphasis on measuring the results of improvement “projects.”

I’d like to focus here on the second point, measuring the results of improvement, what is called “evidencing improvement.”

evidencing_benefits

Click on image to view report.

The article, “The Importance of Evidencing Improvement” (Efficiency Exchange web site, 17 November 2016) makes the case that creating formal reports that provide evidence of results is necessary to drive continuous improvement forward. In addition, such reports are necessary to satisfy the new “Value for Money” reporting requirement from the U.K. Higher Education Funding Council. Note that it contains an onerous 13 pages of guidelines for reporting “Value for the Money,” while the report on “Value for the Money” reporting is 83 pages. My brief research into the matter reveals that evidencing improvement reports seems to spawn ever-more reports – including reports on reports – and hence ever-more cost to the public in the name of reducing costs to the public.

The publication shown in the image at right is a (no doubt helpful) guide for evidencing improvement intended for use by other institutions of higher education in the U.K. This 50-page guide from the good people of the University of Strathclyde is a well-intentioned effort to help others who must toil under such self-imposed and government-imposed regulatory requirements, with the hope that it helps drive business process improvement (BPI) forward while yielding sought-after benefits to society.

As you likely imagine, I strongly oppose such reporting of improvement results. Why? Broadly, because they slow down and limit the magnitude and scope of improvement. Let’s have a look at the detailed reasons:

  1. Requiring formal reports on improvement results is an excellent example of the application of traditional thinking to non-traditional (progressive) process improvement work. It is a fundamental misunderstanding of the mindset behind Toyota-style improvement (i.e. Lean). This often happens in organizations where they seek to apply what they know from experience to that which they do not know. Rather than abandoning current knowledge in order to gain new knowledge, they apply current knowledge (reports, “projects”) in the quest for new knowledge, thereby greatly limiting the new knowledge that can be obtained – and also limiting the results that are desired.
  2. It reflects improvement within the framework of existing boundaries that define the status quo. Therefore, few or no breakthroughs are possible. These boundaries are what Lean seeks to break free of.
  3. Written reports nourish the bureaucracy and help it thrive, which Lean teams should never do. Lean teams should not conform to the bureaucracy. Instead, the bureaucracy should learn from Lean teams to reduce and simplify stultifying bureaucratic work.
  4. It takes significant resources – people, time, money – to write internal reports and reports to government agencies. These resources are diverted from actually improving processes, which is where they are needed most.
  5. Requirement for evidencing will influence the type of improvements people pursue, and certainly narrow the focus to what is organizationally acceptable and that which can be easily defended when presented to institutional and government leaders.
  6. Requirement for evidencing will reduce the number of improvements, as more improvement “projects” naturally results in more reporting.
  7. Requirement for evidencing will limit the speed of improvement. People will work on fewer large projects than on many small ones. Many small improvements should be the focus, rather than a few large improvements. The savings in resources from small improvements will almost certainly be small, so the motivation for doing them will be small as well, given that reports are usually written with an eye towards impressing other people.
  8. Requirement for evidencing will limit and likely reduce learning. The former chief financial officer of The Wiremold Company says: “Don’t bean count Lean.” He says this because many resources and outcomes are not easily measured and it limits people’s thinking of what improvement is or what it can be.
  9. Evidencing leads to prioritization. What is the basis for prioritization? Saving money, saving time, saving people, student learning, student satisfaction, pleasing the boss, etc.? Prioritization often leads people to focus on what can be done, not on higher aspirations of what should be done.
  10. The need for evidencing improvement is closely associated with the commonly accepted view that “what gets measured gets managed” – which is a false promise. Don’t accept that; question your assumptions!
  11. The skills that people develop are relate to report-writing and therefore not towards continuous improvement. In this, and other ways, such formal reporting requirements disrespect people – where “people” are the various stakeholders of higher education.
  12. There is a natural tendency to write long reports because the bigger the report, the more improvement that has been achieved. Isn’t that right? No, that’s wrong.
  13. Quantifying benefits also has a long history of being demanded by leaders, yet is also unconvincing as justification for continued investment or as a rationale for others to embark on on process improvement. People in leadership positions often do not believe what they read in reports.
  14. More labor hours and money will likely go into writing reports than into actual improvement work, thus assuring that the destiny of U.K. higher education is likely to be unchanged. It will end up where politicians want it to end up.

I am certain that I have missed some other important reasons. Can you think of any?

If you have time to write reports, you are improving too slowly. You should be improving at a rate faster than it can be evidenced in a formal report, thus rendering such reports irrelevant.

The focus of improvement in Lean is the elimination of waste, unevenness, and unreasonableness. Evidence of that should always be visual – either the genba itself or a few one-page charts at the genba. Never written reports.

At a recent Kaizen Conference, someone asked:

“We have been doing kaizen since 2009. What is the method to evaluate our kaizen achievements?”

The answer from sensei Chihiro Nakao was:

“Go to the genba to see how you have changed it.”

In order for improvement to be assessed this way, institution leaders, government leaders, auditors, and other stakeholders, must learn how to see that improvements have been made through training. Needless to say, that training should be gained through their personal participation in process improvement activities – hopefully, kaizen.

3 thoughts on “Evidencing Improvement

  1. Rachel McAssey

    Thanks Bob – good post. One of my favourite way of evidencing benefits in a recent project was a visual approach. Prior to the project were two over-flowing in-trays with work in progress. Post improvement there is one very small tray with a visual marker that shows when other staff need to be called in to help and it also encourages the team to take further root cause analysis about why it is happening. It won’t necessarily look good in a VFM report, but really helps people to focus on the problem

    Reply
  2. Stuart Morris

    Whilst agreeing that we should not report for the sake of reporting we still need to “show” improvement to increase buy in. One of my events was used as a case study in the Strathclyde document and by evidencing how staff felt about the process before and after (3 months and 12 months) we demonstrated to the team how they had improved. Human nature results in people only remembering the extremes in their daily tasks but by asking them 10 specific questions then comparing previous answers we could show them how THEY had improved and now we have been asked to help them look at another area of their department. Whilst not wanting to disagree with you Bob, you are by far more eminent in this field then I am, we must still help organisations new to LEAN “see” the benefits of the approach to free them to improve more.

    Reply
    1. Bob Emiliani Post author

      Hi Stuart – It’s OK to disagree with me. And, eminence is a relative thing… We’re all just trying to do a few simple things: learn and improve. I do agree that the benefits need to be seen and shown (and also celebrated). Mr. Nakao has it exactly right: “Go to the genba to see how you have changed it.”

      Reply

Leave a Reply