Looking for the right pilot as proof-of-concept


Any change, even one that promises big benefits, includes concerns that something might go wrong. Too many times there is no practical way to prove that the concerns are either easily solvable, or they are too small relative to the benefits. There are also concerns from the unknown, what we cannot even imagine.  Even the value of the benefits is often hard to translate to actual impact on the goal.

Proof-of-concept is a general expression for providing a logical proof that the concept works. Theoretically a good Future-Reality-Tree (FRT) can provide such a proof, but it might not be good enough to eliminate all concerns, especially not the concerns from the unknown.

A more robust way to prove a concept is through simulation. However, it requires utmost care that the underlining assumptions are valid in the reality where the concept is considered.  It is not trivial to check the assumptions behind computerized simulation that has been developed by other people.  One category of assumptions that need to be carefully checked is the behavior of uncertainty. Another important aspect is identifying real effects that have not been included in the simulation.  For instance, most simulations do not consider human behavioral aspects like Parkinson Law.

The most robust way to prove a concept is running a pilot. The pilot should give a better idea of the value and reveal the negative branches and their impact, providing the opportunity to trim or reduce the negative impact.

A pilot generates considerable hassle.   Management attention is given beyond what is usually required in such a project.  All pilots are aimed at proving a concept.  This actually points to the very first task of planning a pilot:  defining the concept to be proven and its resulting benefits, including assessing the reduced uncertainty due to the pilot.

For instance, consider the case of choosing a project as a pilot for proving the concept of CCPM. The concept is aimed at completing the project on time, preferably earlier than the realistically expectations.  The CCPM concept includes planning the critical chain, cutting the task times and inserting buffers, mainly the project buffer.  In the execution phase the use of buffer management as the only priority mechanism is a key insight.  The choice of the particular project should handle the concerns that meeting the due-date for that project might be due to either special attention given to that project or that the good result is merely arbitrary.

It is important to realize that pilots should be done only when there is strong conviction that the concept offers considerable value, but might also cause damage.

The main factors in designing the right pilot are:

  • Being able to significantly reduce the potential damage from full implementing of the concept.
  • Providing good information on the value from full implementation.
  • Limited consumption of special management attention.
  • Limited investment in the pilot.
  • Limited hassle throughout the organization. The impact of the pilot on the daily management and performance of the organization as a whole should be relatively low.

A suggestion for planning the pilot:

Define a-priori the performance measurements and the decision rules to determine whether to implement the concept after the pilot’s completion.

The concern is that if the measurements and rules for such a decision are not defined before the pilot is implemented, then there is high probability that no decision would be made, and the situation that led to the pilot, the conflict between believing in the value and being concerned by negatives, will continue indefinitely.

A key case for considering a pilot for a TOC implementation is for a distribution chain. The size of the full implementation, which could cover wide geographical area, many regional warehouses and huge number of retailers, makes the decision to abandon the current rules and move to the dynamic TOC solution very tough.  Let’s just state some of the reasonable concerns of senior managers:

  • The improved availability would not lead to significantly improved sales.
    • For instance, because clients have always reasonable alternatives.
    • Or, because the short items are not high runners.
  • The resulting inventory levels, and their impact on cash-flow, might be still high, maybe even higher than now.
  • Transport costs would go up.
  • New difficulties would emerge in loading trucks with very small batches of many SKUs.
  • Getting used to new software modules and implementing them in many sites would take long time, causing problems in daily management and by that harming the performance.

These concerns, while still relying on the big promise of achieving a decisive-competitive-edge of vastly improved availability and lower inventory levels, lead to going for a limited pilot as a proof-of-concept.

How should such a pilot be defined?

There are several options to consider:

  1. Start by implementing the solution at the central warehouse, but waiting with the regional warehouses.
  2. Focus on 3-4 regional warehouses.
  3. Choosing one family of products, including both fast and slow runners, and covering the way from the central warehouse to several, or even all, regions.

The issue of defining the characteristics of a good pilot is one of the most important open issues of TOC implementations. I highly recommend that TOCICO would arrange for several known TOC experts to publically discuss the issues during the next TOCICO annual conference.

I like to express my own view on the above options for a pilot on distribution.

The effectiveness of the replenishment solution depends on maintaining stable and flexible flow, according to a clear set of priorities, throughout the distribution chain. The relationships with suppliers have to be carefully re-thought in order to maintain as fast and frequent replenishment as possible.  The number of different suppliers poses a difficulty that is not lower than the difficulty to deal with huge number of small retailers.  It is part of the overall implementation plan to grow gradually through suppliers and though retailers.

What a pilot cannot afford is to cause disappointment from the results. Implementing a pilot only at the central warehouse exposes the chain to negatives that are not part of the final process. The regions would still order relatively large quantities, based on their local view of the supply chain, forcing the central warehouse to hold high inventory levels.  This could easily cause disappointment from the results and shutting down the whole implementation.

Focusing on several regions causes two different negative branches. One is that as long as the central warehouse cannot ensure fast and reliable response to the regions, the availability at the regions becomes questionable, leading again to disappointment.  The other negative branch is that the regions in the pilot might demand, and get, special treatment.  This could lead to good results in the pilot, but causing huge resistance in all other parts of the organization because they have to deal with tougher conditions.

So, my preference is to run a pilot on one family of products, including the central warehouse and all the regions for that family of products. I don’t see an urgent need to go to the retailers as part of the pilot, especially when they are managed by another organization. The outcome of the pilot is gaining somewhat reduced overall inventory in the central warehouse plus the regions while ensuring excellent availability.  The retailers might still order in batches, but the number of retails would reduce the negative impact of the  batching. The experience would lead to better understanding the actual impact on transportation cost and on loading the trucks, even when most trucks would carry also items that not part of the pilot.

I hope that this post would raise the issues behind planning the executing TOC pilots in variety of TOC applications.


Published by

Eli Schragenheim

My love for challenges makes my life interesting. I'm concerned when I see organizations ignore uncertainty and I cannot understand people blindly following their leader.

3 thoughts on “Looking for the right pilot as proof-of-concept”

  1. Eli –
    You say “All pilots are aimed at proving a concept.” This is wrong. It leaves out a crucial human dimension to improvement initiatives. Often, a pilot is an essential means of building and adapting both understanding and relationships among the stakeholders, including consultants. Improvement initiatives require both trust and understanding, and if you don’t have enough of those, whatever new thing you’re trying won’t work. Whether the concept is valid or not.


  2. Rob, I probably interpret the term “a pilot” in a different way than you. Certainly any big project requires building trust and establishing the right communications. Thus, the project plan has to take these requirements into consideration and plan the appropriate initial activities to establish that. Calling an initiative “a pilot” means there is no decision to go ahead with the project as it depends on the resulting pilot. This means the pilot is designed to answer the question: “Would this concept work and produce good results HERE?” The concerns for the pilot might include whether there is enough trust between the key players to achieve the results. This is what I understand “proof-of-concept” means. Not in the generic scientific way, but in the specific pragmatic way – “is it good for us at this time?”


    1. Eli – I was disagreeing with the statement, but you’re right, I do interpret “pilot” differently. I believe that the purpose of a pilot should be to gather information and build relationships so that useful decisions can be made about whether and how to proceed. That may include finding the extent to which we believe a given technology will work in a given environment at a given time. I’m not sure it’s worth diving into the idea of “proof of concept” – I guess a pilot could “prove” that a technology MIGHT work, but for me that’s a pretty narrow vision.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s