110 How to choose the best techniqueor combination of techniquesto help solve your particular forecasting dilemma Manager^s guide to forecasting David M. Georgoff and Robert G. Murdick One thing may be more certain for managers today than anything else: they have almost too much to think about and keep in mind in trying to assess problems realistically and solve them. Some respond by developing prejudices against any new idea because they don't have enough time to learn the new concepts properly. Others throw up their hands and admit they can't judge the idea with everything else they are handling. Looking at forecasting at a time when they may need good forecasts more than ever, many managers are downplaying their importance. One reason may be that-like many other things-when forecasts are right, you don't hear about them. But when they're wrong.... In response to this problem, the authors have compiled a chart that profiles the 20 most common forecasting techniques and rates their attributes against 16 important evaluative dimensions. The result is a practical guide that will help executives sort out their priorities when choosing a technique and enable them to combine methods to achieve the best possible results. Mr Georgoff is professor of marketing at Florida Atlantic University and chairman of the Department of Management, Marketing, and International Business. He has published articles and worked as a consultant to large corporations in the areas of new product marketing, marketing planning, market research, and forecasting. Mr. Murdick is professor of management at Florida Atlantic University. Previously he worked at the General Electric Company for 14 years. Well known in the field of management information systems, he is the author or coauthor of 18 books on management and marketing, the most recent of which is MIS: Concepts and Design (Prentice-Hall, second edition, 1986). Early in 1984, the Houston-based COMPAQ Computer Corporation, manufacturer of IBM-compatible microcomputers, faced a decision that would profoundly affect its future. Recognizing that IBM would soon introduce its version of the portable computer and threaten COMPAQ'S dominance in this profitable market, the company had two options. It could elect to specialize in this product line and continue to market its highly regarded portables aggressively, or it could expand market offerings to include desktop microcomputers. The latter move would force the year-old company to confront IBM on its home ground. Moreover, COMPAQ would have to make a substantial investment in product development and working capital and expand its organization and manufacturing capacity. COMPAQ'S management faced several important unknowns, including the potential market's size, structure, and competitive intensity. Management recognized that the company's vitality might seriously erode if it did not expand its product line. If the expansion were successful, COMPAQ might enjoy economies of scale that could help ensure its survival in a dynamic and very competitive industry. If COMPAQ'S market assumptions were incorrect, however, its future might be bleak. Many of today's managers face similar new market realities and uncertainties. Continually confronted with issues critical to their companies' competitive future, they must deal with novel and rapidly changing environments. In short, they must judge a broad range of dissimilar influences. For more than a decade, new forecasting techniques have theoretically helped managers evaluate these varied factors. Much of the promise of Authors' note: We thank Steven C. Wheelwright for his valuable assistance in the preparation of this article.Guide to forecasting 111 these techniques has been unrealized, however, even as a quickening succession of related advances have been overwhelming decision makers with new alternatives. As the number of techniques proliferates, management also realizes that some of its crucial assumptions and projections about the economy have become quite tenuous. Equipped only with a little history, meager and questionable data, and frail and changing theoretical tools, the forecaster must nevertheless make critical decisions about altered futures. As an example, COMPAQ Computer's quandary was further complicated because new technologies, competitors, and products were already transforming a market that had been only recently established. COMPAQ'S forecast of the size, direction, and price trends of the 1984 microcomputer market was confounded by uncertainties about the market's response to several vital factors: The entry of IBM's new portable computer. IBM's 23% price cut in June 1984 and its potential erosion of margins. The entry of lap portables introduced by Hewlett-Packard and Data Ceneral. The launch of IBM's new PC AT, complicated by unexpected delivery delays and compatibility problems. The introduction of desktop computers by Sperry, NCR, ITT, and AT&T. Eventually, COMPAQ entered the desktop segment of the market, even though 1984 was unforgiving and rampageous. Several large competitors restricted their programs,- many smaller companies went into-or to the edge of-receivership. Financially and competitively, COMPAQ succeeded. During 1984, sales rose from $111 million to $329 million and earnings increased from $4.7 million to $12.8 million. The market's dynamics, however, make such results increasingly difficult to achieve,- positive and negative events-both expected and unforeseenhave a decisive effect. Even when managers anticipate outcomes, grave uncertainties about timing, form, and impact persist. Despite the difficulty, the vice president of marketing and the CEO-the two executives most directly involved with the decision-demonstrated what can be done. They used an extended series of consumer and dealer surveys coupled with periodic evaluations of the technology to assess the future market and to guide the development of products and programs to accommodate the industry's fluid and rapidly evolving needs. Managers can use forecasting techniques to help them reach important decisions. A large and fastgrowing body of research deals with the development, refinement, and evaluation of forecast techniques. Managers also have greater access to both internal and external data and can benefit from a multitude of computer software programs on the market, as well as easier access to computer capabilities for analyzing these data. Forecaster's chart while each technique has strengths and weaknesses, every forecasting situation is limited by constraints like time, funds, competencies, or data. Balancing the advantages and disadvantages of techniques with regard to a situation's limitations and requirements is a formidable but important management task. We have developed a chart to help executives decide which technique will be appropriate to a particular situation; tbe chart groups and profiles a diverse list of 20 common forecasting approaches and arrays them against 16 important evaluative dimensions. We list techniques in columns and dimensions of evaluation in rows. Individual row-column intersections (cells) reflect our view of a technique's characteristics as they apply to each dimension. Brief descriptions of the forecasting methods are given on the chart. We have used different colors to show which dimensions represent a strength for a particular technique and which represent its weaknesses. The strengths are highlighted in color; weaknesses are indicated by a gray cell. Naive extrapolation, for example, is strong in internal consistency in that it easily reflects changes in management decisions. It is weak, however, in forecast form. It is important to keep these distinctions in mind when you are using the chart. The chart is useful in two ways. The first is in deciding which technique will suit your particular needs as a forecaster. The second is in deciding how to combine techniques to further improve the result. In this section, we discuss the simpler approach; we talk more about combining methods later. To use the chart, look at the 16 questions listed in the first column after the dimensions. They are the most common questions a manager will ask when deciding to use a certain forecast. The first question sets out the various time spans a forecast would have to cover. Everyone who uses the chart will have to answer question 1. But each of the following questions can be answered with a yes or no. If you an-112 Harvard Business Review January-February 1986 swer no to a question, you don't have to look across that row. In responding to question 1, make note of those techniques whose time span matches your needs. We have found it easiest for forecasters to write down the technique's column letter. The row number of each dimension and the column letter of each technique are written along the horizontal and vertical axes. With regard to question 1, for example, if your forecast horizon is short-term, you can write down the cell letters for naive extrapolation (A), sales-force composite (B), jury of executive opinion (C), and so forth. But you would ignore the letters for scenario methods (D), Delphi technique (E), historical analogy (F), and so on. The columns you have now listed represent techniques that are qualified for further consideration. Next read down the column of each of these techniques and note any gray cells. If these gray cells are associated with questions to which you have answered yes, then the dimension either precludes use of the technique or the technique can be used but it has difficulty accommodating that dimension. Such precautions will help you determine whether you mustor wish to-eliminate certain techniques from further consideration. An arrow in a cell indicates that its evaluation is the same as the cell to its left. After you have answered all the questions and have a list of surviving techniques, note the cells that are highlighted in color. Those cells represent specific strengths of a technique and can guide you in making a final selection. In the course of the exercise, you may have eliminated a technique that you like, have heard about, or routinely use. You can go back to that one and compare its strengths and weaknesses with those of the methods that the chart has indicated would be best for you. You can then decide whether you would rather proceed with the technique that the chart indicates corresponds most closely to your specific requirements or whether you can accommodate the eliminating factors in order to use the technique that you initially favored. Important considerations When considering each question, you should remember some "tricks of the trade" con-, ceming: Time horizon. Most managers will want the forecast results to extend as far into the future as possible. Too long a period, however, may make the technique selection process even more confusing because of the varying abilities of the techniques to accommodate different time spans. In choosing an extended time horizon, the forecaster increases the complexity, cost, and time required to develop the final product. You can break down the time needed to produce a forecast into development (Dev) and execution (Ex) time. Development time includes the gathering and entry of data, the modification of programs to the company's specific requirements, and the start-up of the system. Execution time is the time it takes to produce a forecast with a particular technique. Initially, of course, development time is a significant concern for the forecaster; once the forecast technique is firmly established, however, execution time is a more appropriate concern. Ibchnical sophistication. Experience shows that computer and mathematical sophistication is integral to many techniques. Although many executives have improved their skills in this area, not all have sharpened their quantitative skills enough to be comfortable with some of the forecast results a computer will spill out. Cost. The cost of any technique is generally more important at the beginning when it is being developed and installed; after that, any technique's potential value to a decision maker usually exceeds the expense of generating an updated forecast. Data availability. Before choosing a technique, the forecaster must consider the extensiveness, currency, accuracy, and representativeness of the available data. More data tend to improve accuracy, and detailed data are more valuable than those presented in the aggregate. Because a technique's ability to handle fluctuations is important to a forecast's success, the manager must match the sensitivity and stability of a technique to the random and systematic variability components of a data series. Variability and consistency of data. Beyond changes that might occur in the company's structure or its environment, the manager must look at the kind of stable relationships assumed among a model's independent variables (represented by the "external stability" dimension). For example, while most historically oriented quantitative forecasts might use expected levels of automobile production as a basis for deterrnining demand for steel, the forecast model may not reflect changes over time in the average amount of steel used in automobiles. These relationships sometimes do change, but any variation is usually so gradual that it will not aiffect a short-term forecast. When the forecasts are long-term, however, or when the company expects a substantial change in a vital relationship, the forecaster should either apply judgment in a quantitative technique or use a qualitative method.Manager's guide to forecastingBrief descriptions of methods Judgment methods Counting methods Time series methods Naive extrapolation: the application of a simple assumption about the economic outcome of the next time period, or a simple, if subjective, extension of the results of current events. Sales-force composite: a compilation of estimates by salespeople (or dealers) of expected sales in their territories, adjusted for presumed biases and expected changes. Jury of executive opinion: the consensus of a group of "experts," often from a variety of functional areas within a company Scenario methods: smoothly unfolding narratives that describe an assumed future expressed through a sequence of time frames or snapshots. Delphi technique: a successive series of estimates independently developed by a group of "experts" each member of which, at each step in the process, uses a summary of the group's previous results to formulate new estimates. Historical analogy: predictions based on elements of past events that are analogous to the pre.sent situation. Market testing: representative buyers' responses to new offerings, tested and extrapolated to estimate the products' future prospects. Consumer market survey: attitudinal and purchase intentions data gathered from representative buyers. Industrial market survey: data similar to consumer surveys but fewer, more knowledgeable subjects sampled, resulting in more informed evaluations. Moving averages: recent values of the forecast variables averaged to predict future outcomes. Exponential smoothing: an estimate for the coming period based on a constantly weighted combination of the forecast estimate for the previous period and the most recent outcome. Adaptive filtering: a derivation of a weighted combination of actual and estimated outcomes, systematically altered to reflect data pattern changes. Time series extrapolation: a prediction of outcomes derived from the future extension of a least squares function fitted to a data series that uses time as an independent variable.Association or causal methods Time series decomposition: a prediction of expected outcomes from trend, seasonal, cyclical, and random components, which are isolated from a data series. Box-Jenkins: a complex, computer-based iterative procedure that produces an autoregressive, integrated moving average model, adjusts for seasonal and trend factors, estimates appropriate weighting parameters, tests the model, and repeats the cycle as appropriate. Correlation methods: predictions of vaiues based on historic patterns of covariation between variables Regression models: estimates produced from a predictive equation derived by minimizing the residual variance of one or more predictor (independent) variable. Leading indicators: forecasts generated from one or more preceding variable that is systematically related to the variable to be predicted. Econometric models: outcomes forecast from an integrated system of simultaneous equations that represent relationships among elements of the national economy derived from combining history and economic theory. Input-output models: a matrix model that indicates how demand changes in one industry can directly and cumulatively affect other industries. Dev = Ex = Indicates strength Development time Execution time Indicates weaknessDimensi Time Resource requirements Input Output Dns Span Urgency Frequency Mathematical sophistication Computer Financial Antecedent Variability Internal consistency External consistency External stability Detail Accuracy Capability for reflecting direction changes Capability for detecting direction changes Form Questions Is the forecast period a: Present need, or Short-, Medium-, or Long-term projection? Is the forecast needed immediately? Are frequent forecast updates needed? Are quantitative skills limited? Are computer capabilities limited? Are only limited financial resources available? Are only limited past data available? Does the primary series fluctuate substantially? Are significant changes in management decisions expected? Are significant environmental changes expected? Are significant shifts expected among variable relationships? Are component forecasts required? s a high level of accuracy critical? Should turning )oints be reflected )romptly? Should turning )oints be idenified early? s an interval or )robabilistrc orecast critical? Judgment methods Naive extrapolation Present need to Medium Rapid results are a strong advantage of this technique.* Dev Short Ex Short Can easily accommodate frequent updates. Minimal quantitative capabilities are required. Computer capabilities are not essential. Very inexpensive to implement and maintain. Some past data are required, but extended history is not essential. Has difficulty adequately handling wide fluctuations. Can reflect changes. Can reflect chanqes. but quality can also vary substanlialiy. Often insensitive to shifts. Focus can be readily restricted. Often provides a limited practical level of accuracy. Can be very responsive to shifts. Apt to miss urning points. Provides point orecast with crude estimated ange. Sales-force composite Short or Medium Forecast can be assembled, combined, and adjusted relatively quickly. Dev Short Ex Moderate Forecast can be quickly compiled, but data collection restricts rapidity. Nominal processing does not require a computer. Inexpensive to implement and maintain. Past data are helpful but not always essential. Significant changes are frequently not transmitted and; or realistically reflected. Generally has difficulty realistically reflecting changes. Can often )rovide useful xeakdowns. Can be very accurafeorsubecttosubstaniai bias. Can only )rovide crude, subjectively determined irobabilistic orecast. Jury of executive opinion Short or Medium In-house group forecasts are quicker than outside experts'. Dev Short Ex Short to Moderate Can accomplish quickly. Financial requirements are nominal for executive groups; they may be higher for outside experts. Does not handle fluctuations well but can accommodate them if the panel meets frequently. If changes come from an internal corporate group, technique can readly reflect them. Reflects changes well; ;echnique combines a range of expertise. Usually aware of shifts and can reflect them in the forecast. Can reflect component orecasts, but is generally concerned with aggregate orecasts. ^ay be most accurate under dynamic conditions. Early turning )oint identificaion can be a strength under dynamic conditions. Only subjecively determined approximate ange or frequency distribuion is possible. Qualitative Scenario methods Medium or Long Urgency seriously compromises quality Dev Moderate to Long Ex Moderate Frequency need is moderate; updates are generally provided as need arises. Usually expensive for thorough efforts. ^ Technique's extended view dampens impact of shortrun influences and random variability. Can readily reflect internal changes. Reflects changes well. Adapts well to shifts. Generally confined to aggregate forecasts. Mot particularly accurate, but usually most accurate when lorizons are extended and conditions are dynamic. Can readily adjust if recognized, but long ime horizon often precludes he need. Can detect cyclical turning points early under dynamic conditions, but ong time horizon often precludes the need. Delphi technique Medium or Long Urgency seriously compromises quality Dev Moderate Ex Moderate to Long Usually used for one-time forecasts, but they can be revised as new informafion becomes available. Expense depends on makeup and affiliation ot participants. Can accommodate changes, but ease of retlecting them depends on group's background. —Pw Technique is subjective, but distributions are an inherent part of technique. Historical analogy Medium or Long Forecast can be computed quickly if data are available; data gathering may cause delay. Dev Moderate Ex Moderate Sophistication level is variable, but some quantitative skill is desirable. A computer may be helpful. If data are readily available, out-of-pocket costs are minimal. Extended history is essential. Can crudely reflect changes at best. Can handle changes, but forecast quality can vary substantially. Can accommodate shifts crudely. nferential relationships are often tenuous; predictions are suspect. Can only predict Toncyclical points very crudely. n limited situaions, only an approximate range can be urnished. A B DCounting methods Market testing Medium Substantial lag Is involved. Dev Moderate Ex Long to Extended Extended, basically used for one-time forecasts. Technical competencies are generally needed. A computer is generally needed for data analysis. Generally very expensive. Past data are useful but not essential. Substantial fluctuations limit the accuracy of projections. Can reflect changes well if they are incorporated into original research design. Seriously weak in handling changes. Seriously weak in accommodating shifts. Handles detail but scope can be limited. Provides highest accuracy in new product and limited data conditions. Responsive, but this is not one of its purposes. Early turning point identification is not a purpose or capability of this technique. Can provide interval estimates. G Market survey Consumer market survey Medium Method of gathering data may cause a substantial time lag. Dev Moderate Ex Long to Extended Depending on methodology, frequent updates are possible, but updates are generally provided at extended intervais. Generally expensive for good controls. Handles fluctuations poorly, but tracking improves performance. Generally cannot validly reflect changes. Ease of handling changes depends on consumers' awareness and interpretation. Seldom reflects significant shifts. Has limited predictability with durables, somewhat better with nondurables. Often highly responsive to demand shifts. Can be responsive to turning points but usually cannot anticipate them. With probability sampling, accommodates any desired fornn. H Industrial market survey Medium or Long Moderately expensive, depending on controis. Past data very helpful but not essential. Wide fluctuations are frequently a significant concern. If changes are recognized, adjustments can be made. Reflects changes indirectly; it is frequently very sensitive to them. If carefully controlled, can handle shifts weii. Can be most accurate approach in special cases. Can be very sensitive to turning points. 1 Time series methods Moving averages Short. Medium, or Long Rapid resuits are a strong advantage of this technique. Dev Short Ex Short Forecast can be systematicaily updated easily. Minimal quantitative capabilities are required. A computer is helpful for repetitive updating. If data are readily available, out-ofpocket costs are minimal. Past history is essential. Can accommodate fluctuations with appropriate averaging period. Cannot validly reflect changes. Cannot validly reflect changes. Cannot validly reflect shifts. Focus can be readily restricted. Accurate under stable condition.s. Variable lags always exist. Cannot anticipate turning points. Confidence limits can be easily derived based on variability of data series J Exponential smoothing Present need to Short or Medium W Only recent forecasts and current data are required once alpha is determined. Can accommodate fluctuations with suitable alpha. Can only moderately reflect changes with prior trend. Can only moderately reflect shifts with prior trend. Generally rates high in accuracy for short-term forecasts. Depending on alpha value, can be very responsive. Generally only provides point forecast. K Adaptive filtering Short or Medium Forecast can be produced quickly once programmed and past data are available. Dev Moderate Ex Siiort A fundamental competency level is required. A computer is essential. Forecast is moderately expensive to develop. Past history is essential although detail and extent vary. Absorbs random fiuctuations and adjusts to systematic shifts. Deals very well with systematic shifts in variables. L Time series extrapolation Short, Medium, or Long Computation is quick if data are available; data gathering can cause delays. Dev Short to Moderate Ex Short W A computer is helpful for repetitive updating. If data are readily available, out-of-pocket costs are minimal. Wide fluctuations result in decreased confidence in projected outcomes. Cannot validly reflect changes. Cannot validly reflect shifts. Normally accurate for trends and stafionary series. Very unresponsive. Probability range is easily constructed. M Time series decomposition Short or Medium Program setup and data gathering may cause delays, but once programmed, computation is quick. Dev Moderate Ex Short Moderately expensive to acquire, develop, and modify Past history is essential with some detail required. Can isolate and determine the level of component effects. Can only moderately reflect changes with prior trend. Can only moderately reflect shifts with prior trend. Effectively isolates idenfifiable components. Generally responds slowly Generally cannot predict turning points unless series lags. N Box-Jenkins Short. Medium, or Long Operationalizing program can take time, but forecast can be produced quickly. Dev Long Ex Moderate A high level of understanding is required. A computer is essential. Acquisition and modification costs are expensive. Past history is essential with detail required. Handles variabiiity effectively. Frequently the most accurate for short-tomedium-range forecasts. When points are identified, adjusts quickly. A weak predictive ability IS possible 0Association or Causai methods Correlation methods Short, Medium, or Long Data evaluation may cause delays, but forecast computation is quick. Dev Moderate Ex Short to Moderate A fundamental competency level is required. A computer is desirable. If data are on hand, development costs are moderate. w Technique is good if covariation is high; otherwise it is poor. Insensitive to significant changes unless they are correlated with predictor variables. Insensitive unless they are related to predictor variabies. Predictive accuracy is weakened if shifts occur. Predictive accuracy can vary widely. Can adapt quickly to turning points. Can predict turning points only if a iagged relationship exists. Regression models Short, Medium, or Long Model formulation takes time, but forecast computation is gulck. Dev Moderate to Long Ex Shonto Moderate A computer is essential for most oases. May handle large fluctuations well with appropriate independent variables. Insensitive to changes, but they can be reflected among predictor variables. Handles changes well if they are appropriately reflecfed in predictor variabies. A restricted focus might substantially compromise technique's predictive accuracy. Can be accurate if variable relationships are stabie and the proportion of explained variance is high. Sensitive to changes once they are identified. if relationships are stabie, can effectively predict turning points. Confidence limits are provided. Leading indicators Short, Medium. or Long Data evaluation may cause delays, but forecast computation is quick. Dev Moderate Ex Short to Moderate Extended history is helpful in initial development. Can readily adjust to systematic and random patterns. Insensitive to changes unless they are reflected in the indicators. Sensitive to changes if they are retiected in appropriate indicators. Focus can be readily restricted, depending on indicators used. Oniy moderately accurate under most conditions. Especially effective in forecasting cyclical changes. Probability range is easily constructed. Econometric models Short, Medium, or Long Model building is lengthy, but producing forecast is quick. Dev Long to Extended Ex Short to Moderate Forecast can be updated quickly if data are available. A high level of understanding is required. A computer is essential for all cases. Development costs are substantial; operating costs are moderate. Highly sensitive to relevant changes. Genarally confined to aggregate forecasts. Give spotty performances in dynamic env.ronments. Confidence iimits are provided. Input-output models Medium or Long Original model may require up to a year to develop, Dev Extended Ex Short to Moderate Extended detailed history is required. Time lag further reduces accuracy. Insensitive to changes. Can be modified to reflect changes. Cannot validly reflect shifts without updafed coefficients. Effectively reflects demand by SIC groups. With stable reiationships, predictive accuracy can be very good. Cannot anticipate turning points but can effectively predict outcomes. Confidence limits can be developed. 7 8 10 11 12 13 14 15 16 Q R TGuide to forecasting 119 Amount of detail necessary. While aggregate forecasts are easy to prepare, the manager will need specific information (including individual product classes, time periods, geographic areas, or productmarket groupings, for example) to determine quotas or allocate resources. Since forecasts vary widely in their ahility to handle such detail, the manager may want a technique that can accurately predict individual components and then comhine the results into an overall picture. Otherwise, the forecaster can use one technique to provide an overall picture and then use past patterns or market factors to determine the component forecasts.' Accuracy. While accuracy is a forecaster's holy grail, the maximum accuracy one can expect from a technique must fall within a range hounded hy the average percentage error of the random component of a data series. Also, hecause of self-defeating and selffulfilling prophecies, accuracy must he judged in light of the control the company has over the predicted outcome and within the time and resource constraints imposed on the forecaster. Rememher also that accuracy alone is not the most important criterion. The forecaster may wish to forgo some accuracy in favor of, for example, a technique that signals turning points or provides good supplemental information. Ibming points. Because turning points represent periods of exceptional opportunity or caution, the manager will want to analyze whether a technique anticipates fundamental shifts. Some techniques give false turn signals, so the forecaster must keep in mind not only a technique's ahility to anticipate changes hut also its propensity to give erroneous information. Form. Final form varies greatly; it is always advisahle to use a technique that provides some kind of mean or central value and a range of possihle 1 For additional discussion, see G. David Hughes, "Sales Forecasting Requirements," in The Handbook of Forecasting: A Manager's Guide, ed. Spyros Makridakis and Steven C. Wheelwright (New York: John Wiley & Sons, 1982|, p.l3. 2 For a discussion of examples, see Spyros Makridakis et al., "The Accuracy of Extrapolation ITime Series) Methods," Journal of Forecasting, April-June 1982,p. inland ' Steven P Schnaars, "Situational Factors Affecting Forecast Accuracy," fournal of Marketing Research, August 1984, p. 290. 3 See Essam Mahmoud, "Accuracy in Forecasting; A Survey," journal of Forecasting, April-June 1984, p. 139: Spyros Makridakis and Robert L. Winkler, "Averages of Forecasts: Some Empirical Results," Management Science, September 1983, p. 987, and Victor Zamowitz, "The Accuracy of Individual and Group Forecasts from Business Outlook Surveys," Journal of Forecasting, January-March 1984, p. 10. outcomes. If even remotely accurate, such information helps the manager determine more explicitly risk exposure, expected outcomes, and likelihood distributions. Improving the forecast Because no dramatic breakthroughs in technique development have occurred during the past several years, efforts to improve forecasts have shifted to searching for a better approach to technique selection. In part, these attempts have explored the strengths and performance characteristics of various techniques.^ Our chart extends this approach by helping the forecaster match different tecliniques' strengths and characteristics to the needs and constraints of the required forecast. Managers can improve their projection in the following ways: Comhining forecasts. Simulating a range of input assumptions. Selectively applying judgment. Combining forecasts The research on combining forecasts to achieve improvements (particularly in accuracy) is extensive, persuasive, and consistent.-^ The results of combined forecasts greatly surpass most individual projections, techniques, and analyses by experts. Because top-rated experts and the most popular techniques cannot consistently outperform an approach that combines results, and because the manager cannot predetermine which experts or techniques will be superior in any situation, combining forecasts-particularly with techniques that are dissimilar-offers the manager an assured way of improving quality. The forecasting chart can help the manager select the best combination of techniques. As the chart shows, each method has strengths and weaknesses. By carefully matching two or more complementary techniques, the forecaster can offset any technique's limitations with the advantages of another, all the while retaining the strengths of the first. Simply compare an approach's highlighted cells against those of other qualified methods. Various techniques incorporate very different underlying notions. Not knowing which of these will ultimately prove to be most accurate in a particular economic environment, forecasters120 Harvard Business Review January-February 1986 can add to their awareness of possible outcomes by evaluating the range and the distribution of the projections produced hy the various methods." part article on scenario forecasts by Pierre Wack in the September-October 1985 and November-December 1985 issues of HBR provides a good example of this. ^ Simulating various outcomes The manager can also establish a range of probable outcomes hy varying the combination and the levels of inputs of a particular technique. Such sensitivity analysis can underscore the most critical variables, the range and distribution of expected outcomes, and the probable outcomes from different assumptions. Using judgment while many quantitative forecasts incorporate some subjectivity, forecasters should rely more heavily on the output of a quantitative forecast than on their own judgment. Forecasting research has concluded that even simple quantitative techniques outperform the unstructured intuitive assessments of experts and that using judgment to adjust the values of a quantitatively derived forecast will reduce its accuracy^ This is so because intuitive predictions are susceptible to bias and managers are limited in their ability to process information and maintain consistent relationships among variables.' The forecaster should incorporate subjective judgments in dynamic situations when the quantitative models do not reflect significant intemal and extemal changes. Even in these cases, the forecaster should incorporate the subjective adjustments as inputs in the model rather than adjusting the model's final outcome. When confronted with extended horizons or with novel situations that have limited data and no historical precedent, judgment or counting methods should he used. Applying judgment in such situations, however, should be done on a structured basis. The forecaster should also employ judgment to stimulate thought and explore new relationships but, where possible, quantitative techniques should be incorporated to test and support assumptions. The twoForecasting strategies 4 See Hillel J. Einhom and Robin M. Hogarth, "Prediction, Diagnosis, and Causal Thinking," lournal of Forecasting, January-March 1982, p. 23. 5 For survey articles that address this issue, see Mahmoud, p. 139; and Robin M. Hogarth and spyros Makridakis, "Forecasting and Planning: An Evaluation," Management Science, February 1981, p. 115. 6 Lennart Sjoberg, "Aided and Unaided Decision Making: Improved Intuitive Judgment," Journal of Forecasting, October-December 1982, p. 349. [There are] three basic strategies of forecasting.... The deterministic strategy assumes that the present has a close causal relation to the future. This is the strategy that would be used by a cardsharp, who had stacked the deck of cards, to predict the deal. In economic forecasting, the strategy would be used to predict construction expenditures by a knowledge of construction contract awards already made. The symptomatic strategy assumes that present signs show how the future is developing; such signs do not "determine" the future but reveal the process of change that is already taking place. Thus, a falling barometer may reveal a coming storm, or a rising body thermometer an incipient illness. In economic forecasting, this strategy calls for the spotting of "leading indicators"-time series whose movements foreshadow rises or declines in general business activity. The systematic strategy assumes that, though changes in the real world may seem accidental or chaotic, careful analysis can reveal certain underlying regularities (sometimes called principles, theories, or laws). The way to find these regularities is to black out much of reality and hold only to the abstractions that make up a system, such as a solar system, or a nuclear system, or an economic system. Though the theories that result from this process of abstraction are "unreal," they may nevertheless possess the power to affect the real world-provided, of course, that the theories are sound. The test of the soundness of a theory is how it measures up when applied to reality: An atomic explosion confirms Einstein's E = me'. Similarly, a price cut that leads to increased sales confirms the hypothetical demand curve that no man has ever seen outside an economics textbook. To be sure, economic "laws" do not have the consistency of those in the physical sciences. Nevertheless, economic relations or theories, derived from a study of the past, may be useful tools for prediction, within some acceptable range of probable error. From BusinessForecaslIng: With a Guide to Sources ot Business Data by Leonards. Silk and M. Louise Curtey (New York: Random House, 1970), p. 3. Copyright © 1970 by Random House, inc. Reprinted with the permission of the publisher.Harvard Business Review Notice of Use Restrictions, May 2009 Harvard Business Review and Harvard Business Publishing Newsletter content on EBSCOhost is licensed for the private individual use of authorized EBSCOhost users. It is not intended for use as assigned course material in academic institutions nor as corporate learning or training materials in businesses. Academic licensees may not use this content in electronic reserves, electronic course packs, persistent linking from syllabi or by any other means of incorporating the content into course resources. Business licensees may not host this content on learning management systems or use persistent linking or other means to incorporate the content into learning management systems. Harvard Business Publishing will be pleased to grant permission to make this content available through such means. For rates and permission, contact [email protected].