No open-access location is known for this record yet.
No open-access location is known for this record yet.
Empirically grounding analytics (EGA) is an area of research that emerges at the intersection of empirical and analytical research. By "empirically grounding," we mean both the empirical justification of model assumptions and parameters and the empirical assessment of model results and insights. EGA is a critical but largely missing aspect of operations management (OM) research. Spearman and Hopp (2021, p. 805) stated that "since empirical testing and refutation of operations models is not an accepted practice in the IE/OM research community, we are unlikely to leverage these to their full potential." They named several "examples of overly simplistic building blocks leading to questionable representations of complex systems" (p. 805) and suggested that research using analytical tools like closed queuing network models and the Poisson model of demand processes could incorporate empirical experiments to improve understanding of where they do and do not fit reality, highlighting "the importance of making empirical tests of modeling assumptions, both to ensure the validity of the model for its proposed purpose and to identify opportunities for improving or extending our modeling capabilities. The fact that very few IE/OM papers make such empirical tests is an obstacle to progress in our field" (p. 808). They concluded that "Editors should push authors to compare mathematical models with empirical data. Showing that a result holds in one case but not another adds nuance and practicality to research results. It also provides stimulus for research progress" (p. 814). These arguments remind of Little's (1970) observation that many potentially useful analytical models are not widely adopted in practice. Thus, EGA research can help to close two major gaps between (1) the empirical and analytical subdivisions in the OM field and (2) scholarly output and practical relevance. As a journal focused on empirical research, the Journal of Operations Management (JOM) seeks to encourage EGA submissions and publications, but doing so requires our community of authors, reviewers, and editors to share an understanding of the expectations. While such contributions have been encouraged for some time in the verbiage on the JOM website, a more formal effort to draw out examples of EGA research was driven by an editorial call (Browning & de Treville, 2018), and we have since had many discussions, panels, webinars, and workshops to continue to develop and communicate the expectations. This editorial represents another step in that development. In a general sense, an EGA paper combines mathematical, stochastic, and/or economic modeling insights with empirical data. Modeling captures non-linearities and elements of distributions and allows these parameters to be incorporated into decision making, whereas empirical research transforms observations into knowledge. Analytical models are evaluated in terms of their results and insights, which might prompt further extensions to or modifications of the model, including new or different inputs and recalibrations. Most modeling papers stop there because the primary contribution is the analytical model. Although some realism is required, it falls short of empirical grounding, and a gap is often left between the model's insights and what implementation in practice will entail. Filling this gap by empirically grounding an analytic model creates knowledge by linking analytical insights to what has been observed using empirical methods (such as case studies, action research, field experiments, interviews, analysis of secondary data, etc.) to establish a theoretically and empirically relevant research question. Moreover, since analytical models tend to make many simplifying assumptions, EGA research can help tease out where these assumptions are valid and where they excessively bias results. Figure 1 situates two kinds of EGA research with traditional analytical models. Typically, publications with analytical models focus on the center of the figure: the model details and the insights derived from it. The left side of the figure refers to the empirical grounding of the model, that is, whether there is empirical evidence to justify the model's assumptions, parameters, and specific calibrations. The right side of the figure refers to empirical evidence of the impact of the model, that is, whether the model fits the problem situation, can be used in real time, and provides useful output. The concerns expressed above by Spearman and Hopp stem from the expectation that a single paper will present both the model and the empirical testing. This expectation leads to the situation in which empirical testing serves only to demonstrate the model in action, rather than preparing the way for the insights encapsulated in the model to be deployed in practice. Given the lack of openness (among some) to publishing further empirical testing, the model may be accepted by the research community based on its analytical strength—but the first question anyone from practice will ask is, "Who else has used this, and what were the results?" JOM is interested in papers that address questions related to both empirical sides of the development and use of analytical models—their grounding and their impact—that is, either side of Figure 1. Are data available for model parameters? How well do the results work in a variety of real situations? Are the results practically implementable? Are they useful to practitioners? Will managers actually use them? Figure 1 thus highlights important but often undervalued elements encountered in empirically grounding insights from analytical models. Both sides of Figure 1 require a significant amount of empirical research—and it is empirical work on either side of Figure 1 that is the primary contribution of an EGA paper in JOM. It is usually expecting too much of single paper for it to address both sides of Figure 1 sufficiently. On the left side of the figure, analytical models are linked to data and observations of reality: Their assumptions, parameters, and calibration should bear resemblance to a real situation. Here, an empirical contribution focuses on the empirical discovery of a new regularity (new assumption) that leads to the development or revision of analytical models to exploit that new-found regularity. Contributions on the left side of Figure 1 represent the "heavy lifting" of empirically grounding models, transforming mathematical insights into a form that permits measurement and application, and making existing mathematical and modeling insights available to address an observed problem. Finding, collecting, preparing, and analyzing data requires a substantial amount of work—especially when it is impossible to obtain data from the company or situation on which a model was developed. Key parameter values may be unobservable and require estimation from available data. Also, the assumptions that made the model tractable may not hold in key contexts: Empirical research needs to address this tradeoff between parsimony and accuracy. At JOM we want the value of such research to be recognized. Contributions on the right side of Figure 1 assess an existing model's performance in real contexts and address emerging issues. Experiments, field tests, and intervention-based research methods are likely candidates for this type of EGA research. These contributions typically build on the empirical insights from the left side of Figure 1 and the insights/results of prior analytical models, but they add the new knowledge created when the effect on decision making of the nonlinearities captured by analytical models is observed empirically. We classify these contributions as EGA as well, although one could also consider them as "analytically grounded empirics." Engaging in either side of Figure 1 will trigger an improvement process where the model is revised based on new assumptions or the availability of new data, and/or its effectiveness (usefulness) and efficiency are increased in the real-world context. This will require a toggling back and forth between inductive reasoning to capture the new empirical evidence, deductive reasoning through the analytical model, and abductive reasoning (Josephson & Josephson, 1994) to reconcile the emerging insights and empirical regularities. The surprising and unexpected results that trigger the abduction logic indicate that both the model and its empirical grounding matter to creating actionable knowledge. Creating space for abduction is one of the reasons why successful EGA contributions are more likely to come from the sides than the center of Figure 1. Again, JOM encourages papers that tackle either side of Figure 1 and empirically motivate a significant revision to existing models (see examples below). The above-described empirical grounding is often replaced in the modeling literature (where the focus is the model formulation and insights) by either stylized assumptions (explicit simplifications that still capture key elements of the problem situation) or artificial (simulated) data to assess the model performance. Table 1 identifies four types of modeling efforts, depending on the source of assumptions and data for assessing model performance (cf. the left and right sides of Figure 1), together with the key contribution of each type of study (the italicized terms in each cell). The upper-left quadrant (a stylized model tested with artificial data) is common where an analytical insight is a paper's primary contribution. Empirical grounding can take the form of either moving to empirical data applied in an actual situation (lower-left quadrant) or observing areas in practice where the model requires extension (upper-right quadrant). Analytical Insight Parsimonious Causal Insights Explore implications Test of Value Quality of approximation Solutions/Designs Performance improvement An effective EGA process will encourage moving across the quadrants in Table 1: Progress made in any quadrant can open new doors in adjacent quadrants. As we gain fluency in managing the EGA research process, it will become easier to take analytical insights into the field, transforming them into effective interventions through a multi-stage process that links analytical and empirical publication outlets. All else being equal, JOM is more interested in empirical studies that assess the effectiveness and usefulness of a model (despite its simplifying assumptions), that is, the right side of Figure 1. However, it should be noted that items in the lower half of the table—referring to analytical models that are fit to empirical data but provide no insight into how implementation of the model increased knowledge, understanding, or improvements to the model—typically do not qualify as EGA even though the research is carried out in a real context. The contribution from EGA papers (in JOM) must be foremost empirical—even if some of the insights arise from the analytical model—but the use of the data must translate into model improvements that further improve the results derived from the model. This strategy, however, should not be confused with intervention-based research (IBR), where the outcome of the intervention is to improve existing theories or develop new theoretical insights as a result of the engagement with the problem situation (Chandrasekaran et al., 2020; Oliva, 2019). Tables 2 and 3 summarize aspects of several EGA papers that we will discuss further in this section. We begin with some example papers (in Table 2) that fit best on the left side of Figure 1, followed by papers (in Table 3) that fit best on the right side. Most of these papers have been published in JOM and exemplify the new space that we are seeking to develop, in which empirical work is done to improve the usability of a model. Serrano et al. (2018) began from the observation that financial risk propagates upstream in supply chains, the so-called "contagion effect," similar to what has been observed with upstream propagation of order variability. They combined models that incorporate policies and constraints that drive agent behavior to reproduce the observed propagation behavior in terms of values and dynamics of variance of payments, and grounded the model on findings from the empirical finance literature to identify factors that drive such propagation and explore what payment variability propagation would look like when encountering rare but important states of nature. Chuang et al. (2022) began from the recognition that retail managers allocate auditing effort (i.e., inspections) for reducing inventory record inaccuracy (IRI) to groups of SKUs rather than to individual SKUs, as most models in the literature assume. This paper proposed the use of survival analysis to transform available data into an estimate of the data degradation rate in the group of SKUs and developed a simple model to allocate auditing effort optimally for groups of SKUs. The paper further grounds the model by empirically estimating the unobservable cost of IRI and testing the model with data from the research site, where it outperformed current managerial practices and the prevalent industry method for allocating auditing effort. Abbey et al. (2015) began from the observation that customers differ in their preference for new or remanufactured products, and wondered whether this observed heterogeneity might affect pricing decisions. They demonstrated the existence of these two customer groups empirically and then extended a pricing-demand model to incorporate them. When customers are assumed to be homogenous in their preference for new or remanufactured products, pricing-demand models indicate that the price for new products should decrease when remanufactured products are added to the mix. The extended model, however, indicates that the price for new products should increase when remanufactured products are added, with this price increase reducing cannibalization and increasing profit. Gray et al. (2017) began from the observation that companies offshored then reshored production. They examined nine offshoring cases, six of which were subsequently reshored. The reshoring cases were motivated by emergent problems such as intellectual-property protection, quality, and/or logistics. The authors observed that the offshoring decision was made on the basis of per-unit landed cost, with other factors not considered even though information was available. The reshoring decision, however, was based on much richer data. They modeled the decision process using system dynamics, with loops grounded in observations from the cases. The model reproduced the observed behavior and also suggested a risk of oscillation: Although the reshoring decision brought richer information into the decision process, there was no evidence that offshoring decisions were updated to consider more than per-unit landed cost. Craig et al. (2021) began from the observation that the majority of chargebacks used by retailers to penalize supplier fulfillment errors arose from execution errors rather than product-related factors. Mechanisms in place tended to prioritize simplicity over accuracy and might not take into consideration the type of error encountered. Also, errors were experienced differently by retailers and suppliers, and the authors argued that such differences should be considered in designing the chargeback system. An error that is inexpensive for a supplier to fix and costly for the retailer should perhaps be treated differently than one that is difficult and expensive for a supplier to fix and relatively inexpensive for the retailer, for example. The researchers extracted data concerning fulfillment errors, chargebacks, and penalties from supplier compliance manuals. They used the efficient order quantity (EOQ) model to exemplify a traditional OM tradeoff model to provide insight into how the optimal decision is affected by how the error rate changes with the order size. The researchers used numerical experiments to demonstrate first the potential magnitude of the cost-of-execution errors and second the potential for the chargeback mechanisms in common use to substantially under- or overcharge for execution-quality cost. Finally, the authors presented a game showing the importance of considering supplier and retailer power and margin in designing chargeback penalties and coordinating efforts to improve execution quality. de Treville, Petty, et al. (2014) began from the observation that 10% of the deals assessed as worth evaluating at a venture capital (VC) firm were not evaluated, with the reason logged as a lack of capacity: Had more capacity been available, the VC firm would have evaluated all deals that passed their initial assessment. How would the cost of the additional capacity needed compare to the expected benefit from evaluating more deals? Exploring this question began with development of an analytical model that estimates how the expected value of the highest deal value grows with increased evaluations for a given tail index of the distribution of deal values. The insight was that the highest deal value grows little from increased evaluations if deal values are described by a distribution that has a tail that decays exponentially (tail index of 0, covering distributions up to the exponential distribution). If, however, the tail grows polynomially (tail index > 0), then the value of the best deal is expected to increase significantly with an increase in evaluations. It was not possible to observe the tail weight of deal values considered by the VC firm, so the authors used the tail weight of VC-backed initial public offerings—substantially above 0—to get a rough idea of whether positive tail weights should be considered. The value of the increase in the best deal under various tail weights was then compared to a rough estimate of the cost of increasing VC capacity. de Treville, Schürhoff, et al. (2014) presented a tool that used real-options theory to calculate the value of postponing the decision about what to produce under demand risk. This work began from the realization—captured in a Harvard Business Review article (de Treville & Trigeorgis, 2010)—that the value of reducing the time between production commitment and observation of demand could be conceptualized as a real option, with this insight qualitatively explored in a company. Model development consisted of combining models from quantitative finance with the newsvendor model. The analytical model proposed—known as the Cost-Differential Frontier—was applied to two cases to illustrate how incorporation of these real options would affect the decisions companies made about investment in lead-time reduction. Although this article is at the edge of EGA, at the time it was generally agreed that the contribution was more the model than it was the empirical insights, and it was not considered to fit well at JOM. Gaur et al. (2007) began from the realization in Fisher and Raman (1996) that developing the ability to transform the dispersion of forecaster estimates into an estimate of the standard deviation of demand would be useful in the absence of historical demand data. Fisher and Raman (1996) simultaneously developed an analytic model to better understand supply-demand mismatches and sought to better match production to demand in a company. The model, while elegant, was too computationally demanding and also not sufficiently flexible to answer the needs of the company project. The authors then combined analytical tools from OM into a model to allocate production between suppliers who in This an estimate of the standard deviation of demand for the but no historical demand data was available. The authors thus created a to transform dispersion of forecaster estimates into a rough estimate of the standard deviation of demand that was to them to out the at the company and obtain initial that the would supply-dema
Loading…
Stored only when both papers are in the corpus — 28 citations exist upstream.
Papers that cite the same references — the relational fallback while embeddings are still backfilling.
Imported from OpenAlex (CC0) — nothing here is generated.