Phase 6: Evaluate — A Look Back to Move Forward

In The End, or as Close to the End as You Can Get, the machine that was developed using your project’s chosen new product development (NPD) method was installed and started up.

Formal processes were developed during this phase (i.e. work instructions, production schedules, maintenance schedules, etc.).

Now that the machine is installed and running; it’s judgement day.

Everything is compared against the original objectives ─ machine output, user and material input, and long term projections of financial success (or failure).

This doesn’t mean that everything will be perfect.

This is a new machine. Basically a prototype.

This is where planning and action come into play.

Sure, there were issues with the initial startup.

Users had to become accustomed to the machine, and the new processes and procedures surrounding it. The machine had to be “tweaked” to optimize its performance. But, after that initial startup, the machine and processes should be run in order to monitor the performance.

Follow-up meetings with the entire group of stakeholders should happen on a regular basis. This is where the continuous improvement cycle of Plan-Do-Check-Act comes into play.

After operating for a while someone sees an opportunity for improvement. As we learned earlier, recognizing that there is a problem is a huge undertaking. A short term plan is laid out to implement a change. The change is made (Do). The change is allowed to run for a short period, and then it must be checked (Check). If it passes the short period test, implement it permanently (Act). Be prepared to start this process over again.

You may choose to do this Plan-Do-Check-Act process ad hoc. However, this loose approach often backfires because people will perceive it to be not of the utmost importance.

This is where regularly scheduled meetings with the stakeholders needs to take place. They aren’t held forever, but until everyone feels comfortable that the changes implemented were successful.

Once this point has been reached, there is one last meeting where a look back at the entire process is done ─ the post mortem.

Back in Phase 1 (Idea discovery) and Phase 2 (Business case building) the group of stakeholders developed a list of objectives that this new machine had to meet. The machine was designed and built to meet those objectives. It was installed, started up, and has been running for awhile with adjustments made to keep working at the highest possible efficiency.

The post-mortem is the time to go through that list of objectives one last time.

The group should consist of the original stakeholders involved in Phase 1 and 2. Each objective should now have sufficient data to determine if the machine is a success, a near miss, or a failure.

The near misses should be looked at as opportunities to improve upon using the Plan-Do-Check-Act process.

The failures should be evaluated with respect to the actual objective and the process used to attempt to achieve it.

No matter the success rating; the group should document why it was so.

Why did one succeed and another fail? This information can be used for future projects.

The post-mortem for any project should not just be about this particular project. This is a time that it can be used as springboard to improve current and future projects.

As part of this review a list of questions should be developed that cover that project in general, and for each phase of the project.

The following examples illustrate the types of questions to ask.

Describe any early warning signs of problems that occurred later in the project.

How should we have reacted to these signs?

How can we be sure to notice these early warning signs next time?

Were all team/stakeholder roles and responsibilities clearly delineated and communicated? If not, how could we have improved these?

Did those who reviewed the deliverables provide timely and meaningful input? If not, how could we have improved their involvement and the quality of their contributions?

Did the test facilities, equipment, materials, and support people help to make the test an accurate representation of how the deliverables will be used in the “real world?” If not, how could we have improved on these items?

These may seem like a lot of daunting questions. To help with this process, you should employ a facilitator. They should have professional experience at guiding the group through the process, and can act as a referee when issues arise.

Depending on the complexity of the project and the types of questions asked during this phase, it may not be accomplished in one day. However long it takes, it is well worth the effort.

It will make all of your future projects function better ─ and provide better results. That is if you decide to implement any of the suggestions that come out the process.

From these idea-to-launch articles, you should have learned that there are hidden sources of waste within your processes.

If looked at in the right light, these wastes can be reduced, or removed, through utilizing one of the many new product development processes discussed. They go beyond a standard project management process, and can make some lasting improvements.

There are going to be pitfalls along the way. However, if you can recognize them early and often, they can be mitigated.

I wish you the best in removing your Muda in the Genba. Remember, it is not always about the destination, but the journey along the way.