## Effects of Extreme Values on Price Indexes: The Case of the Air Travel Price Index

## Effects of Extreme Values on Price Indexes: The Case of the Air Travel Price Index

**JANICE LENT***

### ABSTRACT

This paper examines the effects of extreme price values on the Fisher and Trnqvist index formulas. Using a simple model, we first consider the impact of outliers on the unweighted arithmetic, harmonic, and geometric means of a collection of values. Then, under the same model, we investigate the effect of a single extremely high or low price on the price index formulas (weighted means). Further investigation using Taylor series approximations leads to some general conclusions regarding the relative robustness of the Fisher and Trnqvist indexes. These are illustrated with empirical results based on airfare data from the U.S. Department of Transportation's Origin and Destination Survey.

KEYWORDS: Price index, extreme value, Taylor series. JEL Categories: C43, C13, E31.

### INTRODUCTION

Many economists have come to favor the "superlative" Fisher and Trnqvist price indexes over the more traditional Laspeyres formula (see, e.g., Diewert 1976; Aizcorbe and Jackman 1993). The U.S. Bureau of Labor Statistics recently began publishing a new price index series targeting the Trnqvist formula. The choice between the Fisher and Trnqvist formulas may be based on a variety of factors, including other price index formulas currently in use by the organization producing the index and the relative sensitivity of the two formulas to extreme values. In this study, we compare the Fisher and Trnqvist formulas with respect to the latter criterion-sensitivity to extreme values.

Extreme-valued price ratios often occur as a result of deep discounts or "free" promotional goods or services. Such extreme-valued ratios can be either large or small, depending on whether the discounted price appears in the numerator or denominator of the price ratio. Less often, extremely high prices appear with converse effects. The Laspeyres formula is sometimes criticized as sensitive to extreme values, because it is based on an arithmetic mean of the price ratios. We will see, however, that such sensitivity depends on the direction of the outlying value (high or low), as well as on the weights used in the selected mean.

In the next section, we consider the effect of an extreme value on the unweighted arithmetic, harmonic, and geometric means. The third section contains a discussion of the corresponding effects on the Fisher and Trnqvist index formulas under differing assumptions regarding the correlation between the expenditure-share weights and the prices. This correlation is related to the elasticity of substitution (i.e., the extent to which consumers shift their purchases toward lower priced items when relative prices change).

The fourth section presents an empirical example: the case of air travel index estimates computed using data from the Passenger Origin and Destination Survey collected by the Bureau of Transportation Statistics. The extreme-valued price ratios in this application resulted from a change in data-collection procedures and are in this sense artificial. They do, however, provide an opportunity to compare the performances of the different index formulas under the conditions represented by the application. We summarize our conclusions in the final section.

### EFFECTS OF EXTREME VALUES ON UNWEIGHTED MEANS

The following simple model shows the effects of an extreme value
on three types of unweighted means. Let *x*_{1}, ...,
*x*_{n} be a collection of nonnegative values,
where *x*_{i} = for *i* = 1, ...,
*n* - 1, while *x*_{n} = *y * for
some factor *y* > 0; that is, *x*_{n} is
an outlier in the collection. We define the unweighted arithmetic,
harmonic, and geometric means, respectively, as follows:

,

,

and

.

For *M* ∈ {*A, H, G*}, let

.

Then

, , and

.

We first consider the rate at which the various means approach
as *n* approaches infinity. For fixed *y*, we
have

as *n* → ∞. For the harmonic mean also,

as *n* → ∞, and, similarly,

as *n* → ∞. Thus, as *n* becomes large, all three of
the means approach at approximately the same rate. Their
behavior in the presence of an outlier differs, however, under
various assumptions about the outlier itself. If we suppose that
*n* is fixed, we may follow derivations similar to those above
to arrive at the results, which are summarized in table
1.

The results shown in table 1 for
*f*_{G}(*y*) may lead us to conclude that
price index formulas based on the geometric mean are, overall, the
most robust formulas available; at the very least, they represent a
sensible choice when both high and low outliers are expected to
occur. By contrast, while *A* is robust to low outliers, it is
sensitive to high outliers; similarly, *H* is robust to high
outliers but sensitive to low ones. In most applications, however,
price indexes are not computed as unweighted means. In the next
section, we examine the effect of expenditure-share weights on the
Laspeyres, Paasche, Fisher, and Trnqvist indexes, with special
emphasis on the latter two.

### EFFECTS OF EXTREME VALUES ON PRICE INDEXES

#### Price Index Formulas

We begin by presenting several population index formulas. The Laspeyres index measuring price change between time periods 1 and 2 is defined as

,

where *p*_{jt} denotes the price of item
*j* at time *t* ∈ {1,2}, *q*_{jt}
denotes the quantity of item *j* purchased at time
*t*,

,

and *N* denotes the number of items in the target
population. The weight *w*_{jt} is the
*expenditure share* for item *j* in period *t*; the
price ratios
*p*_{j}_{2}/*p*_{j}_{1}
are often called *price relatives*. Clearly *L* is the
arithmetic mean of the price relatives with weights representing
first period expenditure shares. The Paasche index is a harmonic
mean of the price ratios, with second period expenditure-share
weights:

.

The Fisher index is simply defined as , while the Trnqvist is a geometric mean of the price ratios with weights representing the averages of the period 1 and period 2 expenditure shares, shown as

,

where *w* _{j,}_{1,2} = (*w*
_{j}_{1} + *w*
_{j}_{2} ) /2.

#### Extreme Values and the Elasticity of Substitution

To examine the effects of an outlier on the indexes described
above, suppose we have a collection of *n* items priced in time
periods 1 and 2. Suppose further that for *j* = 1, ...,
*n* we have *p*_{j}_{1} =
*q*_{j}_{1} = 1 and that for *j* =
1, ..., *n* - 1 we also have
*p*_{j}_{}_{2} = 1, while
*p*_{n}_{2} = *y* (i.e., we assume
for simplicity that the above is 1.) For *t* ∈ {1,2} ,
let *x*_{jt} =
*p*_{jt}*q*_{jt}, the
expenditure level for item *j* in period *t*. We wish to
allow the quantity of an item purchased to vary in response to price
change and an assumed elasticity level. When
*p*_{j}_{2} =
*p*_{j}_{1}, we assume that
*q*_{j}_{2} =
q_{j}

where ≤ *τ* ≤ 1, and *τ* is assumed constant. Then

.

We define the elasticity *τ* in this way, because *τ*
provides a convenient means of examining the effects of extreme
values under conditions of high and low elasticity, defined
relatively. Note that higher values of *τ* indicate less impact
of price change (represented by the price ratios) on second period
item-level expenditure levels.

For *j* = 1, ..., *n* - 1, we have
*q*_{j2} = *q*_{j1} = 1;
and

*q*_{n2}. = *y*^{-τ}

The resulting first and second period expenditure-share weights are as follows:

;

;

and

.

The "average weights" used in the Trnqvist index are

,

and

.

Note that when *τ* is small (low or zero elasticity) and
*y* is large,

*w*_{n 1} < *w*_{n, 1,
2} , (1)

so the Laspeyres index gives less weight to high outliers than
does the Trnqvist index. Similarly, when *τ* and *y* are
both small,

*w*_{n 2} < *w*_{n, 1,
2} , ,(2)

indicating that the Paasche index gives less weight to low outliers than the Trnqvist. Under conditions of low elasticity, we therefore observe the following phenomena: although the Laspeyres index, based on the arithmetic mean, is sensitive to high outliers, it assigns them weights that are low relative to the Trnqvist weights, while the Paasche index, a harmonic mean, assigns lower weights to low outliers. The weights in the Laspeyres and Paasche indexes can therefore be expected to compensate, at least partially, for the sensitivity of the arithmetic and harmonic means to high and low outliers, respectively.

Under this simple model, the values of the Laspeyres, Paasche, Fisher, and Trnqvist indexes are as follows:

;

;

;

and

.

Both the Fisher and Trnqvist indexes are known as superlative
indexes, because economic theory suggests that they approximate a
true cost of living index under relatively weak assumptions
regarding economic conditions (Diewert 1987). (In the application
considered in the next section, these indexes should be viewed as
cost of flying indexes rather than as cost of living indexes.) We,
therefore, focus on the relative robustness of *F* (*n*,
*y*, *τ*) and *T* (*n*, *y*, *τ*)
under the assumptions *τ* = 1 and *τ* = 0. The value
*τ* = 1 indicates that consumers shift their purchases toward
items (or item categories) whose relative prices have decreased
between periods 1 and 2, while *τ* close to zero represents the
case of little or no change in buying behavior in response to price
change.

First consider the case *τ* = 1, where a value of *τ*
represents the assumption that consumers alter the quantities of the
items they purchase so as to maintain the same level of expenditure
on each item-a situation corresponding to a fairly high level of
elasticity. In this case, we have, for fixed *n* and large
*y*,

, (3)

while

. (4)

So, for reasonably large *n*, *T* is more robust than
*F* in the presence of high outliers. For the case of low
outliers, we have

(5)

and

(6)

for fixed *n* as *y* approaches 0. Under the simple
model, we may therefore conclude that, with regard to robustness,
conditions of high elasticity favor the Trnqvist index over the
Fisher.

With *τ* = 0, we have

,

and

.

Note that *F* (*n*, *y*, 0) = *L* (*n*,
*y*, 0) = *P* (*n*, *y*, 0). For fixed *n
*and large *y*,

, (7)

while

. (8)

As a rough rule of thumb, the above approximations suggest that
*T* is likely to outperform *F* whenever outliers are as
large as *n*^{2}. The relative robustness of *T*
and *F* thus depends on the relative values of *y* and
*n*, which may, in turn, depend on the aggregation level being
considered. Equations (4) and (8) also indicate that, for large
values of *n*, *T* is much more robust to high outliers
under high elasticity than it is under low elasticity. For low
outliers, however, the elasticity assumption has less impact on
*T*. With *n* fixed and *y* small, we have

, (9)

and

, (10)

revealing that, under conditions of low elasticity, *T* is
more sensitive to low outliers than *F*. Equations (5), (6),
(9), and (10) suggest that *T* is somewhat more robust to low
outliers for *τ* = 0 than for *τ* = 1, while *F* is
much more robust.

The above results lead us to conclude that, under conditions of
low elasticity, the Fisher index may often be more robust to
outliers than the Trnqvist: the Fisher is more robust to low
outliers and, when *n* is sufficiently large relative to any
price ratios in the dataset, the Fisher is also more robust to high
outliers. Conditions of higher elasticity (*τ* close to 1)
render both indexes more robust to extremely high values. Under
conditions of high elasticity, the Trnqvist is preferable to the
Fisher, because it is less sensitive to both high and low
outliers.

The numerical examples shown in appendix A illustrate these
conclusions. Tables
A1 and A2
give values of the Fisher and Trnqvist indexes under the single
outlier scenario described above. (Note that these are not random
values produced by a Monte Carlo simulation but simply the values of
the functions *F* (*n*, *y*, *τ*) and *T*
(*n*, *y*, *τ*) for the given parameters.) Table A1
gives index values under the assumption that *τ* = 1 (high
elasticity). Under this assumption, the Trnqvist is clearly more
robust than the Fisher to both high and low outliers.

Table A2 shows values of the indexes under the assumption that
*τ* = 0. The bold numbers in this table highlight points at
which *y* becomes large enough, relative to *n*, to render
the Trnqvist index better than the Fisher for approximating the
mean = 1 in the presence of a high outlier. As expected,
the turning points occur as *y* approaches
*n*^{2}. The examples in table A2 also illustrate that,
under low elasticity, both indexes are more sensitive to high
outliers and less sensitive to low outliers than they are under high
elasticity.

#### Taylor Series Results

The single outlier model employed in the previous subsections
does not, of course, account for the data complexity often
encountered in practical applications. Here, we look beyond the
single outlier model to examine Taylor series expansions that shed
further light on the relative robustness of the Fisher and Trnqvist
indexes under the general assumption of low elasticity. Following
and expanding on the development of Lent and Dorfman (2004a), we
assume that the price indexes are computed from a collection of
expenditure share weights and *sub-indexes
I*_{g}, which here take the place of the price
ratios
*p*_{j}_{2}/*p*_{j}_{1}
in the previous subsection. Each * _{g}* is an
aggregate of the ratios

*p*

_{j}

_{2}/

*p*

_{j}

_{1}for all items

*j*in a particular item category

*g*. In practice, the standard formulas are often applied in this two-stp fashion. The categories into which we divide the items may be defined according to item characteristics, geographic area of purchase, or both.

We begin by defining some notation. For *t* ∈ {1,2}and for
each item category *g*, let

,

and let

, ,

and

.

Next, with *w*_{g} defined as the Trnqvist
weights,

,

let

, ,

and

.

We expand each of the superlative indexes about the point at
which all of the sub-indexes *I*_{g} equal the
mean . The relevant partial derivatives are given in
appendix B. From the general form of the third-order approximation
given by Lent and Dorfman (2004a) for a geometric mean, we have the
following approximation of the Trnqvist index:

. (11)

The second-order approximation for the Fisher index is

(12)

Thus, to the second order, we have

. (13)

Consider the relative values of
*w*_{g}_{1} and
*w*_{g}_{2} in the presence of high
outliers among the *I*_{g} and high correlation
between the *I*_{g} and the
*w*_{g}_{2} (the case of low
elasticity). Under these conditions, we are likely to have
*w*_{g}_{2} >
*w*_{g}_{1} for large values of
(*I*_{g} - )^{2} and thus
*T*_{I} - *F*_{I} > 0.
Similarly, in the presence of low outliers, we are likely to have
*w*_{g}_{2} <
*w*_{g}_{1} for large values of
(*I*_{g} - )^{2}, resulting in
negative values of *T*_{I} -
*F*_{I}. Thus, the approximation shown in
equation (13) indicates that, under conditions of low elasticity,
the Fisher index may be more robust than the Trnqvist to both high
and low outliers.

### AN EMPIRICAL EXAMPLE

For the air travel price index series, the apparent elasticity of
substitution is low-in some cases, even negative. The series,
therefore, exemplify only the behavior of the different indexes
under conditions of low elasticity (*τ* close to 0). Note that
the elasticity reflected in the data, rather than the actual
elasticity, is the quantity that affects the performance of the
indexes; Dorfman et al. (1999) showed that the elasticity reflected
in sample survey data need not always equal the true population
elasticity.

The air travel price index series shown in figures 1, 2, 3, 4, 5, and 6 are based on data from the Bureau of Transportation Statistics' quarterly Origin and Destination (O&D) Survey. The sample for the O&D Survey comprises about 10% of all passenger itineraries having some U.S. component (i.e., itineraries that include at least one flight arriving at or departing from a U.S. airport) and includes about 6 to 7 million itineraries per quarter. Data items collected include trip route, class of service (e.g., coach, first class), and transaction fare including taxes. Note that the scales differ across figures, so comparisons across figures are distorted in some cases.

When goods and services are sampled for the purpose of estimating a price index, the sample items generally remain in the sample over an extended time period (e.g., two years) unless they are taken off the market by the retailer. The stable sample allows comparison of prices across time for identical items. Ratios of prices in different time periods for individual items are the building blocks of the traditional price index estimators.

In the O&D Survey, however, the sampling is performed
independently for each reference quarter. Since the itineraries
selected in a given quarter may not match those selected for a
previous or subsequent quarter, we developed and tested a two-stage
process for matching categories of itineraries across quarters and
comparing average prices within categories across time. The ratio of
average prices for different time periods is called a *unit value
index*. These sub-indexes are then aggregated by the Fisher,
Trnqvist, and other index formulas. The index series are based only
on data from sample itineraries flown on domestic carriers and are
chained quarterly.^{1}^{}
Lent and Dorfman (2004b) provide a more detailed description of the
index estimation methodology.

The figures show the Laspeyres, Paasche, Fisher, and Trnqvist index series for various classes of service and for all classes combined. Note that, in all of the figures, the Paasche series runs either slightly below the Laspeyres series or even (for business class service) above the Laspeyres, indicating low or negative elasticity of substitution. Lent and Dorfman (2004a) describe a method of estimating the elasticity of substitution; elasticity estimates computed by their method, measuring elasticity of substitution between unit value categories as described above, run close to 0 for these data. Although air travel passengers readily substitute one carrier for another in response to fare changes, little substitution across trip origin/destination pairs occurs. Since origin/destination pairs far outnumber carriers, this substitution behavior leads to low overall estimates of elasticity of substitution between the unit value categories.

In examining figures 1 through 6, it is important to note that the Class of Service variable in the O&D Survey was redefined and standardized in 1997-98. (Formerly, air carriers had used a variety of service classifications in reporting this information, so the variable values had to be recoded by the Bureau of Transportation Statistics.) We therefore expect some unusual data values to affect the index series during this period; indeed, many of the series display a visible break between the fourth quarter of 1997 and the first quarter of 1998. These breaks may be exacerbated, because a lower percentage of the O&D Survey observations were "matched" across time during 1997-98 (see Lent and Dorfman 2004b for a description of the across-time matching method), resulting in lower than usual effective sample sizes.

Figures 1 and 2 show the series for all classes of service combined and for restricted coach class (by far the largest class), respectively. The series in figure 2 behave in typical fashion: the Laspeyres series runs just above the others, displaying a slight upward drift, while the Paasche shows a similar downward drift, and the two superlative series run between them, closely tracking each other. This type of behavior results from the large number of observations and because the 1997-98 break has relatively little impact on these series. Figure 1 is similar to figure 2, except for the noticeably larger effect of the 1997-98 change, which lifts the Trnqvist series slightly above the others. Recall that, under conditions of low elasticity, the Trnqvist index is often more sensitive to outliers than the Fisher.

Index series for other classes of service (categories comprising fewer observations) are shown in figures 3, 4, 5, and 6. For the unrestricted first and restricted first class indexes (figures 3 and 4), the Laspeyres series runs very slightly above the Paasche, indicating low but positive elasticity. For the unrestricted first class series, the 1997-98 break sends the Trnqvist above the other series, while the Trnqvist for restricted first class is "bumped down" and runs well below the others for 1998 and subsequent years. In both cases, the Trnqvist continues to roughly parallel the Fisher after the break, indicating that unusual data values generated the level shifts. Note also that the Trnqvist's upward shift for unrestricted first class is noticeably less severe than its downward shift for restricted first class, perhaps due to its greater robustness to high outliers than to low ones.

The business class index series (figures 5 and 6) display the
relatively rare phenomenon of negative elasticity. The Paasche
series runs above the Laspeyres, indicating that consumers are
shifting their purchases toward *higher* priced services as
relative prices change. It is important to note that sample survey
data may not always reflect true population elasticity; in this
case, the class-of-service categories are coarsely defined, and many
different types of restrictions may apply to tickets in the
restricted categories. (Restrictions may include, for example, a
requirement of advance ticket purchase or, in the case of roundtrip
itineraries, a Friday or Saturday night stay at the destination.)
Elasticity estimates based on these data reflect substitution
*between* these categories but not *within* them (for the
same route and carrier) and may therefore suffer a downward bias.

On the other hand, since business class service is typically paid for by a third party (i.e., the passenger's employer), very low elasticity is expected. Some business class passengers may even choose higher priced tickets assuming that "you get what you pay for," and such behavior could also explain the negative elasticity indicated. Under negative elasticity, quantities purchased are positively correlated with price change, and this correlation may cause expenditure shares to increase dramatically when prices increase. The Trnqvist index, whose weights are average expenditure shares, therefore assigns large weights to some high price ratios. Apart from the negative elasticity, the movements of the business class series appear similar to that of the first class series, that is, the Trnqvist index is shifted up or down during the 1997-98 period, while the other series are less affected by the unusual values. Table A3 in appendix A shows unweighted percentiles of the distributions of the unit value indexes for the first class and business class categories over the crucial period. The outliers are clearly sufficient in number and severity to impact the tails of the sample distributions.

### CONCLUSIONS

Practitioners may often consider robustness to outliers an important criterion in selecting a price index formula, especially for item categories such as airfares, in which extreme prices may regularly result from frequent flyer awards and other price discriminatory discounts. Although price index formulas based on different types of means inherit the relative robustness of these means, the weights applied in price index calculation also play a crucial role. This paper shows that, under conditions of low elasticity of substitution, the high correlation between the weights and the price ratios may offset the sensitivity of the Laspeyres and Paasche indexes, making the Fisher a more attractive option than the Trnqvist. The choice between index formulas is therefore more complex than the mere selection of an arithmetic, harmonic, or geometric mean. It requires information on the elasticity of substitution reflected in the data as well as an estimate of the magnitude of outliers (high or low) that can be expected.

### ACKNOWLEDGMENT

Comments from Alan Dorfman of the Bureau of Labor Statistics resulted in significant improvements to this paper.

### REFERENCES

Aizcorbe, A. and P. Jackman. 1993. The Commodity
Substitution Effect in CPI Data, 1982-1991: Anatomy of a Price
Change. *Monthly Labor Review* , December 1993:25-33.
Washington, DC: U.S. Government Printing Office.

Diewert, W.E. 1976. Exact and Superlative Index
Numbers. *Journal of Econometrics* 4:115-145.

_____. W.E. 1987. Index Numbers. *The New Palgrave:
A Dictionary of Economics*. Edited by J. Eatwell, M. Milate, and
P. Newman. London, England: MacMillan.

Diewert, W.E., R. Feenstra, and W. Alterman. 1999.
*International Trade Price Indexes and Seasonal Commodities.*
Washington, DC: U.S. Department of Labor, Bureau of Labor
Statistics.

Dorfman, A., S. Leaver, and J. Lent. 1999. Some
Observations on Price Index Estimators. *Proceedings of the
Federal Conference on Survey Methodology,* Arlington, VA,
November 1999.

Lent, J. 2003. Chain Drift in Experimental Price Index
Series for Air Travel, *Proceedings of the 2003 Joint Statistical
Meetings, Section on Survey Research Methods,* CD. Alexandria,
VA: American Statistical Association.

Lent, J. and A. Dorfman. 2004a. Using a Weighted Average of the Jevons and Laspeyres Indexes to Approximate a Superlative Index. Working paper.

_____. 2004b. A Transaction Price Index for Air Travel. Working paper.

### APPENDIX B

#### Partial Derivatives

To derive equations (11) and (12), the function
F_{I} is expanded around the point ** I =
μ** = (

*μ*, .,

*μ*). The general formula for the third-order Taylor expansion is

.

For a derivation of the third-order expansion of
*T*_{I} (equation (11)), see Lent and Dorfman
(2004a). The first- and second-order partial derivatives of
*F*_{I} evaluated at ** I =
μ** (used in the derivation of equation (12)) are as
follows:

### END NOTES

^{1}Price index chaining is done
by estimating long-term price changes as products of shorter term
changes (links). Quarterly chaining can lead to "chain drift," as
seen in the Laspeyres and Paasche series in the figures in this
section. For more information on chain drift in the airfare indexes,
see Lent 2003.

### ADDRESS FOR CORRESPONDENCE

*J. Lent, Bureau of Transportation Statistics, Research and Innovative Technology Administration, U.S. Department of Transportation, 400 7th Street SW, Washington, DC 20590. Email: janice.lent@dot.gov