The failure of a grand technological idea – part 2

The inquiry team continued its quest to learn the right lessons from the gap between expectations and outcomes

The formal structure of the gap being inquired:

The task of raising hypotheses is to keep an open mind to all possibilities, based on the very few facts known. Every hypothesis would then be used to direct the team to look for information that would either invalidate or support the hypothesis.

The team came up with the following hypotheses:

Hypotheses 1: The expectations were not realistic to start with – it is impossible to build a perfect system that will always warn when it should and never when it should not.

Hypothesis 2: The project people developed what they were capable of developing. What seemed to be too difficult was not developed.

Hypothesis 3: The project people focused on preventing false alarms even on the expense of failing to raise the alarm when real approach is done.

Hypothesis 4: There were no clear and detailed specifications, approved by top management, of what the Wise-Cameras should do.

Hypothesis 5: There was not enough involvement of highly professional security people in the development of the system.

Hypothesis 6: The project team did not have all the skills required for such a mission, and they tried to conceal this from top management by announcing success.

The hypotheses were verbalized in regular daily language by the members of the team. There are obvious causal connections between some of the hypotheses. So, several hypotheses could be valid. At this stage the team simply checks each hypothesis whether explains the gap.

Verbalizing a potential explanation:

Note that additional effects that are required to cause the gap are actually new hypos that need to be validated as well – checking for known facts that would tell whether the effect existed in the specific case.

Validating the facts until the operational cause(s) are clear

Going through verbalizing several possible explanations, drawn from the basic hypotheses, the team had no difficulty to come up by the following straight-forward facts:

• Top management had verbalized only high level of specifications with the following main requests:
  • The system should identify intruders before they arrive to a door or a window of a protected building.
  • The number of false alarms would be minimal – not more than 5% of the alarms should be false.
  • The system should have clear advantages on any other camera-base protection systems.
  • The management did not specify in writing that the system should eliminate the need for human guards to watch the screens.  However, this request had been raised in several informal talks.
• The project leader, Raphael Turina, told top management he’d achieve all the requests that appear in writing, and will strive to get to the point where watching the screens would not be necessary.
• Sam Fuller, the CEO, said that Raphael promised to let top management know whether there is a need to watch the screens.
• Raphael said that he notified Sam that the written requests were all answered.
• The idea of people rolling towards the building was not raised by the team and thus was not considered.
• The main technical challenge was to distinguish between a person and an animal moving.  The team assumed animals have four legs and based the ultra sophisticated image recognition on this idea.
• All the project professionals were found to have the highest professional skills.
• Both Raphael and Alex have very wide experience in security.
• None of the project professionals and internal management were involved in planning the test for the event.

Facts that were validated indirectly:

• The project team were reluctant to inform top management of failures or specifications that were not achieved
  • Raphael, Alex and other members of the team deny this tendency
  • However, checking mails and reports to top management showed detailed reporting of successful internal tests and no reporting of difficulties
  • One clear failure of identifying movement in extreme weather conditions was not reported
• Top management were certain that the system did not need human intervention during the identification stage
  • Gilbert, the manager of the external testing team and the head of the inquiry team, testified that he understood no human watching the screens was necessary
  • Nobody was supposed to watch the screens during the external test

A summary of the revealed facts:

The project lacked a clear definition of the required characteristics. It strived to cover everything, but some problems were raised. Two problems were clearly identified:

1. In very bad weather the Cameras are unable to identify human movements.
2. The project team had a conflict between the need to identify suspicious attempt to come close to the building and activating false alarms.   This conflict led to the decision to identify human movement by moving with two legs. Possible exceptions and bypasses were not discussed.

The problems were not reported to Top Management, which were under the impression that the system is perfect and eliminate the need for watching the screens. The impression of the project people were that all written requests have been achieved. The project team thought there is no need to mention the state of issues that were not raised in writing. Management was under the impression that all the requests, not just those in writing, have found the appropriate answers.

Questions:

1. We know now what happened – is there anything else we need to know?
2. What do we do now? Is the inquiry complete?

To be continued!