By Fred Nickols and Rob Foshay
I have been a knowledge worker doing knowledge work since I was a teenager. I joined the U.S. Navy in 1955 when I was 17, and I spent 20 years there, retiring in 1974 as a chief petty officer. I recently posted some thoughts about my “technician’s perspective” to an ISPI-related discussion list.
Rob Foshay, who is well-known in the performance improvement field, responded; and I think my post and his response combine to make a good column on a particular kind of knowledge work: troubleshooting. Let’s begin with my post about the technician’s perspective.
My rating in the Navy was that of fire control technician (FT). That rating is responsible for operating, maintaining, and repairing complex, shipboard weapons systems (gunnery systems and missile systems, including radars and computers). My perspective is that of a technician, a technical troubleshooter. The list below is some of what I needed to know about the systems on which I worked:
- What the system does and how it does it
- Why the system works the way it does
- How the system is supposed to work
- How the system is actually working
- How to tell when the system is not working properly
- How to figure out why it is not working properly
- How to jury-rig it when the required parts are not available
- How to make the system work better
- How the system is wired
- The components that make up the system and how they are connected
- How to test system components and how to test the system
- How to trace various paths through the system
- What the system is supposed to accomplish
Later, I was trained as a classroom instructor, a developer of instructional systems, a writer of programmed instructional materials, and, last, as an internal organization development consultant. Upon retiring from the Navy, I took up a career as a management consultant. There, I found myself working on issues associated with the performance of people, processes, modern computer systems, and organizations.
Guess what? I needed to know the same kinds of things. It is all about systems.
In response to my post, Rob Foshay wrote:
Great list, Fred!
To it, the research on troubleshooting done at Carnegie-Mellon and Xerox PARC added these components of domain knowledge to your list:
- Failure modes of each component or subsystem (how it works when it does not work)
- Probability of each component failure and failure mode
- Cost or time, or both. of test procedures, repair, and substitution procedures
- Repair history of the equipment (what has been recently repaired)
They further found that, when troubleshooting, experts use this kind of reasoning: First, they formulate hypotheses about possible causes, then they construct mental models of what the abnormal behavior symptoms would be if a particular component or subsystem fails in a particular way. Each of these possible causes becomes a hypothesis, and they are ranked according to probability of the failure. Then, selecting the test procedure that has the highest information value and least cost or time to test, they use a split-half strategy to rule out hypotheses until one remains as the most likely.
A number of other observations about expert behavior have been observed across troubleshooting in a wide range of domains, including electronics, medical diagnosis, mechanical systems maintenance, software debugging, and so forth:
Experts typically generate no more than about seven hypotheses; if the actual cause is not among those candidates, there is a good chance it will never be found.
- Experts do not engage in a free-form inductive search, unless all other options have been exhausted. Instead, they are most likely to simply recall the procedure they used last time they saw this set of symptoms, and apply it. This leads to a kind of error that only experts can make: confusing two faults with very similar symptoms, and attempting whatever fix they use most often without adequate testing. Real experts have learned to discipline their tendency to do this.
- A common error novices make is to fail to generate a range of hypotheses. Instead, they fixate on one symptom, ignore (or fail to test for) disconfirming evidence, and use a repair procedure for the hypothesis they have. Then they are astonished when it does not work.
- Novices typically gather far more information, from far more tests and repair procedures, than experts do. This is precisely because they follow procedures rather than engaging in the reasoning process above.
- Expert troubleshooters were observed to place replaced components back in the new component’s box, then leave the box on top of the equipment. This communicated to the next technician what components were recently replaced, and, thus, unlikely to be the cause of further problems.
- One of the favorite data-gathering techniques of analysts is to hang out in the local bar with a table of expert troubleshooters. They trade war stories about problems they fixed. But what makes each story valuable and interesting to other experts is that it illuminates some little-known aspect of the components of domain knowledge the experts have.
The significance of this research is that, contrary to Mager’s book on troubleshooting, if you represent troubleshooting as a set of procedures to be learned, you will actually cause damage by inhibiting the ability to learn the domain knowledge and reasoning processes of an expert. Troubleshooting thus should not be represented as a completely defined procedural task. It is actually an example of one kind of complex, ill-structured problem solving involving a number of types of domain knowledge, and a range of modeling and reasoning skills.
Incidentally, this is discussed in a chapter on troubleshooting training in the 2003 textbook I did with Ken Silber and Mike Stelnicki. Van Merrienboer and Kirschner also discuss it in their work on the 4C/ID model.
To conclude this month’s column, I have often thought that the smartest decision I ever made was to stay in the Navy for 20 years. After more than 40 years spent in the private sector since retiring from the Navy, I see no reason to change my mind. My technician’s perspective has served me well and continues to do so.
Foshay, W.R., K. Silber, M. Stelnicki. (2003). Writing training materials that work: How to train anyone to do anything. New York, NY: Pfeiffer.
Mager, R.F. (1982). Troubleshooting the troubleshooting course or debug d’bugs. Belmont, CA: Lakewood Publishing Company.
van Merrienboer, J.J.G., and P. Kirschner. (2007). Ten steps to complex learning: A systematic approach to four-component instructional design. London: Routledge.
About the Authors
Fred Nickols is a longtime member of ISPI and a regular contributor to its various publications. He is the managing partner of Distance Consulting LLC and maintains a website that contains more than 200 free articles, papers, and book chapters. They can be accessed at www.skullworks.com.
Rob Foshay is a life member of ISPI who publishes and presents regularly to performance improvement and instructional design specialists. He may be reached at email@example.com or through www.TIfPI.org or www.foshay.org. He is affiliated with the learning technology PhD programs of Walden University and University of North Texas at Denton.