|
Computer Program - Scenario Analyzer
SARBC Search and Rescue Society of British Columbia
Computer Program - Scenario Analyzer
The Background
SCENARIO ANALYSIS
IN
LAND SEARCH PLANNING
By
Kenneth Hill
Waverley Ground Search and Rescue
Nova Scotia, Canada
In this paper, I shall propose a solution for a problem that can be illustrated by
describing a Nova Scotia search for a lost deer hunter in the fall of 1986. On his way
home from work one cold afternoon, a 29-year-old man parked his car next to a large
forested region just outside the city limits of Halifax, apparently intending to hunt in the
vicinity of the roadside during the few remaining minutes before dusk. He had neither
compass nor flashlight, nor was he dressed for the weather. When he did not return
home that evening, his wife contacted the Royal Canadian Mounted Police, who, in
turn, called the Waverley Ground Search and Rescue team. It was a member of the
team who found the man's car the next morning, providing an identifiable last known
location from which to start a search.
The first problem facing the search coordinators was to estimate the point at
which the man had entered the woods. This, at first glance, appeared to be simple.
The man's car was found on the east side of a road that ran roughly north and south.
There were many other possible parking sites on both sides of the road, so it
appeared that the man had selected this particular site for convenience. In fact,
although the man's footprints could not be discerned from many others found in the
area (it was a popular hunting ground), there was a path leading directly from the
vicinity of the man's car and heading, in a southeasterly direction, toward the deep
woods. Moreover, local hunters informed the coordinators that there were many deer
in this area east of the road, and that there were no deer toward the west. Finally, the
conclusion that the man had travelled east rather than west was supported by the
well-known rule, taught in mandatory hunter safety courses, that you should always
hunt on the same side of the road on which you park your car.
After two days of intensive searching, in which approximately 90% of search
resources had been deployed to the area east of the road, the man's clothes were
found next to a lake located to everyone's surprise, west of the road. As expected,
the man's remains would be found in the middle of the lake the following spring.
Apparently suffering from hypothermia, he had irrationally disrobed and decided to
swim the icy water toward what he somehow believed was safety.
With perfect hindsight, of course, the search coordinators realized that they
should have paid more attention to the area west of the road. After all, the man was
found less than a kilometre from his car, in a segment that would have received first
priority if there had been any reason to suppose he had gone west rather than east.
Indeed, his clothes were found by the first team tasked to search the woods east of
the road. Unfortunately, the search plan was overly influenced by the preferred
scenario that the hunter had travelled east; and the allocation of resources was made
according to this assumption. While the alternative scenario--that the hunter had
crossed the road and entered the woods on the west--was not dismissed entirely, it
was not seriously considered until well into the second day after all of the high and
medium priority segments had been thoroughly searched. This, then, brings us to the
search planner's dilemma. Specifically, at what point in a prolonged search do you
abandon a preferred scenario and begin searching segments having a higher priority
according to some alternative scenario?
Analyzing Scenarios
Scenario analysis, as I shall describe one solution to this dilemma, is inspired
by procedures used by the U.S. Coast Guard in searching for missing vessels at sea
(see Stone, l983, for an excellent discussion). Frequently, when a boat is reported
missing, there may be multiple scenarios accounting for its disappearance, each of
which may determine its location. One possibility is that the boat's engines may have
failed, in which case, its location will be determined largely by prevailing winds and
currents. The search planner can then compute a "probability distribution" of possible
locations, based on formulas accounting for wind and ocean velocities. This
probability distribution, of course, is specific to the "failed-engine" scenario.
Alternatively, there may have been a failure in the boat's navigation system, forcing
the captain to rely on dead reckoning clues, such as landmarks and buoys. In the
"failed navigation" scenario, analysis of wind and currents may be of little use, but
rather the scenario requires consideration of the captain's navigation skills and
experience, as well as the nature of available cues (much like the land search for the
lost person). In both cases, the resulting location distributions are weighed by the
probabilities of the respective scenarios.
The application of scenario analysis to the task of finding a lost person
represents an extension of this method. In this case, the "location distributions" are
the set of POA's, or probabilities of area, assigned to each of the segments in the
search locale (including the ROW or "rest of the world" segment). Presently, when the
land search planner solicits POA judgments from members of the overhead team, the
scenario problem is likely handled in one of two ways. On the one hand, a particular
scenario may be specified, such as the "he went east" scenario described above, and
all planners are implicity expected to assume the same scenario. Alternatively, and
probably more common, there may be some initial discussion about what the lost
person may have done, followed by a request that search planners proceed with
assigning their POA to the segments. In this case, planners are free to assume
whatever scenario they prefer. Indeed, five individuals may allocate their POA
according to five different scenarios, that is, make entirely different assumptions about
the lost person's behaviour, resulting in averaged or "Mattson" POA's that are
something less than optimal for planning purposes.
Scenario analysis involves having the Plans Chief present a number of explicit
scenarios to those individuals who will be assigning POA's to the segments and
asking them to estimate the probability of each scenario (expressed as decimal
percentage points). Planners would then proceed to assign POA to all of the
segments for each of the scenarios. So, for example, if there are two scenarios,
planners would estimate POA for each segment twice. (With recent methods for the
rapid assignment of POA, discussed below, and the use of computer software,
planners should be able to analyze as many as five scenarios without undue
disruption of their management tasks, although two or three scenarios should probably
suffice for most searches). Once a planner has assigned POA to every segment for
every scenario, each POA is weighted by multiplying it by the probability of the
scenarios to which it applies. The weighted POA for each segment are then summed
across scenarios, yielding one set of POA for a given planner. These planning POA
can then be averaged with the POA from other planners in the usual manner, yielding
a set of consensus or Mattson POA with which to plan the search.
An Example of Scenario Analysis
Returning to our lost hunter, let's assume, for illustration purposes, that the
Plans Chief has divided the search locale into six segments (normally, of course, there
may be many more segments than six). Segments 1-3 are east of the road, while
segments 4-6 are to the west. Let's further assume, consistent with the facts of the
incident, that the subject is actually located in the area west of the road, in segment 4
(this, of course, is not known to the searchers). After discussion with other search
managers, and taking into account all available evidence, the Chief decides that there
are only two scenarios worth considering: the "East" and "West" scenarios as
described earlier. He then asks each of the planners to estimate the relative
probabilities of these scenarios on a percentage basis. For example, a particular
planner may decide that the "East" scenario has a probability of .70 while the
probability for the "East" scenario is .30. In other words, this particular person
believes that the odds are 7 out of 10 that the subject went east, and only 3 out of 10
that he went west.
Next, the Plans Chief ask his colleagues to estimate POA for each of the six
segments, as well as the ROW segment, while assuming only one scenario at a time
(it's a good idea to vary the order in which scenarios are analyzed, for example, some
planners would analyze the West scenario first, while others would start with the East
scenario). After POA has been assigned to all of the segments for the first scenario,
planners would repeat the process, but now while assuming the second scenario.
Table 1 presents hypothetical POA for one planner. This person has assigned
a probability of .7 to the East scenario, and .3 to the West scenario. His unweighted
POA assignments appear in the second and third columns from the left. In the
second column, for example, these POA estimates are made under the assumption
that the East scenario is valid, while POA appearing in the third column pertain to the
West scenario. As can be seen from this table, weighted POA are obtained by
multiplying the POA for each segment by the appropriate scenario probability, and
doing this for each scenario. Finally, the resulting or planning POA are computed by
adding the weighted POA per segment across scenarios. The planning POA are the
figures that will be used for computing the consensus POA, which will be shifted as
the search proceeds.
Segment Initial POA Weighted POA Planning POA
East West East(p=.70) West(p=.30)
1 .30 .10 .70 X .30 = .210 .30 X .10 =.030 .210 + .030 = .240
2 .25 .05 .70 X .25 = .175 .30 X .05 =.015 .175 + .015 = .190
3 .20 .05 .70 X .20 = .140 .30 X .05 =.015 .140 + .015 = .155
4 .10 .30 .70 X .10 = .070 .30 X .30 =.090 .070 + .090 = .160
5 .05 .25 .70 X .05 = .035 .30 X .25 =.075 .035 + .075 = .110
6 .05 .20 .70 X .05 = .035 .30 X .20 =.060 .035 + .060 = .095
ROW .05 .05 .70 X .05 = .035 .30 X .05 =.015 .035 + .015 = .050
TABLE 1. Analysis of East and West Scenario
for the Lost Hunter Example.
Note: p = probability that scenario is valid:
ROW = "Rest of the World"
The value of scenario analysis becomes most apparent when we actually begin
shifting the POA. Recall that segment 4 contains the lost person. If search resources
are assigned to the segment having the largest POA after each shift (an assumption
that is not always true, but this doesn't affect our example), and assuming a probability
of detection of .50, segment 4 (containing the subject) would be covered by the third
tasking. If, however, search planners had only considered their preferred scenario,
that is, the East scenario, then resources would not have been deployed to segment 4
until the sixth or seventh tasking. That is, only later in the search would the updated
POA for segment 4 become higher than that of the remaining segments.
Some readers may recognize scenario analysis as the application of
conditioned probabilities to the prediction of the lost person's location. As such, it
represents an extension of the same reasoning involved in the use of Bayesian
mathematics for shifting probabilities of area, as prescribed in NASAR's "Managing the
Search Function" course. Generally, conditional probability provides a means for
estimating the likelihood of events when there is some degree of uncertainty in the
assumptions underlying those events. For example, weather forecasters frequently
compute conditional probabilities in making their predictions, expressed, for example,
in such statements as "There's a 50% chance of rain tomorrow." This prediction may
actually be a briefer--and more understandable--way of saying, If the weather front
reaches us (.60 probability), there's an 80% chance of rain; otherwise there is only a
10% chance of rain." In this case, the .50 figure (actually "rounded off" from .52)
represents the total probability of rain, taking into account both weather-front
"scenarios" at once.
Psychological research indicates that, without actually doing the simple
arithmetic described above, people have difficulty understanding predictions expressed
as conditional probabilities, either overestimating or (more frequently) underestimating
such predictions. So, for example, in the case of looking for a lost person, the
tendency would normally be to underestimate the POA of a segment that would
receive a relatively high priority under some less probable scenario. However,
scenarios--like high POA segments--can "lose" probability as the search proceeds and
no evidence is found to support them. At some point in the incident, after a preferred
scenario has not "panned out," the search planner has to begin taking alternative
scenarios more seriously. The problem, of course, is when. Scenario analysis, as
proposed here, is intended to provide a simple and systematic solution to this
problem.
When Should Scenarios be Analyzed?
There are typically over 100 lost person incidents per year in the province of
Nova Scotia, involving approximately 200 subjects. Occasionally, despite our best
efforts, someone dies before that can be found, usually of hypothermia. In retrospect,
nearly all of these incidents involved the absence or unreliability of critical planning
information, such as a place last seen (PLS), or a direction of travel. In most cases,
as with the lost hunter described earlier, there were only two, competing scenarios,
either of which would have led to markedly different POA distributions. Unfortunately,
the search planners had either stayed with their preferred scenario too long, or had
wasted resources on "long shot" scenarios. Two examples should be instructive. In
one incident, a 9 year old boy left a rural home and disappeared in the woods. An
eyewitness swore he saw the child wade across a river, which flowed near the front
of the boy's house, and travel north. However, there was also some indication that
the boy may have rather left from the vicinity of his backyard and travelled east. The
search was conducted under the assumption that the first scenario was valid;
unfortunately, the second proved to be the correct one. In a second example, an 81-
year-old man had a habit of walking up and down an 8 mile stretch of highway and
entering the woods at various points along the road. One night he did not return
home. One eyewitness reported that he was positive he had seen the man walking
north on the highway just before nightfall. Another witness was "fairly sure" she had
seen him going south about the same time. Although the planners believed the first
report to be more reliable (as in fact it turned out to be), search resources were
distributed somewhat evenly between scenarios, resulting in what amounted to two
separate searches. Scenario analysis would have allowed the planners to integrate
the competing scenarios into a single plan.
Which Scenarios Should be Analyzed?
A scenario can be defined as a hypothetical sequence of events resulting from
the missing subject's behaviour or mishap. Scenarios worth considering are those
which would affect the assignment of POA to the segments of the search locale. I
have mentioned alternative directions of travel, last known locations, or place last seen
as examples of important scenarios, because these will normally have a critical impact
on the search plan. Any scenario which has a non-trivial possibility of being valid, as
estimated by the search planner, should be included in the analysis. By "non-trivial I
would (somewhat arbitrarily) propose a 10% or 1 in ten criterion. That is, any scenario
that has at least a 10% chance of being valid, according to existing evidence, should
be included in the analysis, as described in this paper.
One final point warrants discussion. I have been describing scenario
probabilities as if they necessarily add up to 1.0. In practice, they do, but theoretically
there is always the possibility that some unknown scenario, that is, one unanticipated
by the search planners, is in fact the correct one. Indeed, the logic of including a
ROW segment accounts for the possibility that none of the segments in the search
locale contains the lost person. Would it not be possible as well to have an "all other"
scenario in case existing scenarios are incorrect. The answer, unfortunately, is no. A
little thought should reveal that it would be impractical if not nonsensical to ask
planners to assign POA to the segments according to some unspecified "mystery
scenario." The results, for planning purposes, would be meaningless. Rather, the
search planners should try to anticipate such mystery scenarios by considering all
available evidence and discussing it thoroughly before deciding on the appropriate
scenarios. The result should be superior to the current practice, in which planners
implicitly assign 100% probability to a single scenario.
Scenario Analyzer
Listing 1 contains a short MicroSoft BASIC program for implementing scenario
analysis, called, appropriately enough, Scenario Analyzer. The program asks the user
to enter (1) the number of segments, (2) the number of scenarios, (3) the probabilities
of each scenario, and then (4) the initial POA of each segment as well as the ROW,
considering one scenario at a time. It then weights the initial POA by the probability of
each scenario, and computes a distribution of planning POS for one planner. Note
that the probabilities of scenarios, as well as the POA, are assigned by entering
ratings rather than decimal numbers or percentages. That is, the user is asked to rate
the possibility that a particular segment contains the subject by assigning it a number
between one and nine, where 1 = very low possibility, 5 = average possibility or "even
chance," and 9 = very high possibility. This rating system is a variation of the so-
called "O'Connor" method, named after its originator (see Bownds, Ebersole, Lovelock
and O'Connor, 1991). The O'Connor method translates entries into decimal POA,
which can then be used for computing the consensus or Mattson POA. The user will
discover this to be a rapid method of assigning POA, much simpler -- and less subject
to arithmetic errors -- than the method of attempting to spread 100 percentage points
among any number of segments. These advantages make it ideal of scenario
analysis, when the same segments need to be assessed two or more times.
Users should enter the program exactly as it appears (lines following
apostrophes may be omitted). Programmers may want to add extras, such as printing
options. Once the program is entered, you can "debug" it by comparing its output to
the sample run in Listing 2.
For users of Scenario Analyzer, some caveats are in order. It has not yet been
"field tested" and consequently must be considered experimental. It should only be
applied by experienced land search planners who are able to assess the apparent
reliability of the method and to compare it to the traditional method of assigning POA.
Consequently, I cannot accept responsibility for the manner in which the program
Scenario Analyzer, or indeed the ideas expressed in this paper, are applied to real
searches.
References
Bownds, J., Ebersole, M., Lovelock, D. and O'Connor, D. (1991). Reexamining the
Search Management Function (Part 1). Response, 10, 12-15: 28.
Stone, L.D. (1983), The Process of Search Planning: Current Approaches and
Continuing Problems. Operations Research, 31, 207-233.
Users should be warned that the method by Scenario Analyzer to convert
rating to percentages may lead to slightly different POA estimates in some cases than
the method suggested by Bownds et al. Normally, the two methods will not differ by
more than one or two percentage points for some segments and will not differ at all
when planners use the full range of rating. The relative order of segments, in terms of
converted POA, will always be identical.
An executable version of Scenario Analyzer, not requiring the MicroSoft BASIC
interpreter, is available from the author or download here.
The author's address: 15 Cascade Drive, Halifax, Nova Scotia, Canada B3M 1Z4.
Listing 1. MicroSoft BASIC Scenario Analyzer
'Scenario Analyzer
'Copyright 1991 by Kenneth Hill, Waverley Ground Search and Rescue, Nova Scotia
'Assign probabilities to scenarios, segments and ROW by rating between 1 and 9,
'with higher numbers indicating higher estimated probability.
10 CLS : INPUT "How many segments, not including ROW (1 - 99)"; numSegs
20 IF numSegs <1 or numsegs > 99 THEN BEEP: GOTO 10
30 DIM rateSeg(numSegs +1), planPoa(numSegs +1)
40 INPUT "How many scenarios (1 - 5)"; numScens
50 IF numScens < 1 or numscens > 5 THEN BEEP: GOTO 40
60 DIM probScen(numScens)
70 IF numScens = 1 THEN probScen(1) = 1: PRINT: GOTO 195
80 PRINT : countBits = 0
90 FOR n = 1 TO numScens
100 PRINT "Rating for scenario"; n; "(1-9)";
120 INPUT probScen(n)
130 IF probScen(n) < 1 or probscen(n) > 9 THEN BEEP: GOTO 100
150 countBits = countBits + probScen(n)
160 NEXT n
170 FOR m = 1 TO numScens
180 probScen(m) = probScen(m) / countBits
190 NEXT m : PRINT
195 FOR i = 1 TO numScens : countBits = 0
200 FOR t = 1 TO numSegs
210 PRINT "Rating for segment"; t; "in scenario"; i; "(1 - 9)";
220 INPUT rateSeg(t)
230 IF rateSeg(t) < 1 or rateseg(t) > 9 THEN BEEP: GOTO 210
240 countBits = countBits + rateSeg(t)
250 NEXT t
260 PRINT "Rating for ROW in scenario"; i; "(1 - 9)";
270 INPUT rateSeg(numSegs + 1)
280 IF rateSeg(numSegs + 1) < 1 or rateseg(numsegs + 1) > 9 THEN BEEP:
GOTO 260
290 countBits = countBits + rateSeg(numSegs + 1): PRINT
300 FOR k = 1 TO numSegs + 1
310 rateSeg(k) = (rateSeg(k) / countBits) * probScen(i)
320 planPoa(k) = (planPoa(k) + rateSeg(k)
330 NEXT k
340 NEXT i : CLS
350 FOR i = 1 TO numSegs
360 PRINT "Planning POA for segment"; i; "=";
365 PRINT USING "**.**"; planPoa(i) * 100; : PRINT "%"
370 IF i=20 OR i=40 OR i=60 OR i=80 THEN PRINT:INPUT "Hit Return Key"; B$:CLS
380 NEXT i
390 PRINT "ROW = "; :PRINT USING "**.**"; planPoa(numSegs + 1) * 100;
:PRINT"%"
400 INPUT "Run Again (y/n)"; A$
410 IF A$ = "Y" OR A$ = "y" THEN RUN
Listing 2. Sample Run of Scenario Analyzer
How many segments, not including ROW (1 - 99)? 6
How many scenarios (1-5)? 2
Rating for Scenario 1 (1 - 9)? 6
Rating for Scenario 2 (1 - 9)? 4
Rating for segment 1 in scenario 1 (1 - 9)? 2
Rating for segment 2 in scenario 1 (1 - 9)? 6
Rating for segment 3 in scenario 1 (1 - 9)? 5
Rating for segment 4 in scenario 1 (1 - 9)? 3
Rating for segment 5 in scenario 1 (1 - 9)? 1
Rating for segment 6 in scenario 1 (1 - 9)? 4
Rating for ROW in scenario 1 (1 - 9)? 1
Rating for segment 1 in scenario 2 (1 - 9)? 9
Rating for segment 2 in scenario 2 (1 - 9)? 5
Rating for segment 3 in scenario 2 (1 - 9)? 1
Rating for segment 4 in scenario 2 (1 - 9)? 2
Rating for segment 5 in scenario 2 (1 - 9)? 7
Rating for segment 6 in scenario 2 (1 - 9)? 1
Rating for ROW in scenario 2 (1 - 9)? 2
Planning POA for segment 1 = 18.79%
Planning POA for segment 2 = 23.77%
Planning POA for segment 3 = 15.12%
Planning POA for segment 4 = 11.14%
Planning POA for segment 5 = 13.10%
Planning POA for segment 6 = 12.39%
ROW = 5.69%
Run Again (y/n)?
Home Page | Back to Computer Programs
Last Updated: Thursday, 25-Dec-2003 13:01:41 PST
by MCDPRI
|
