Data Envelopment Analysis in the Service Sector ||

Harzer wirtschaftswissenschaftliche Schriften

Georg Westermann Editor

Data Envelopment Analysis in the Service Sector

Westermann


GABLER EDITION WISSENSCHAFT

Harzer wi rtschaftswi ssenschaftl iche Schriften Herausgegeben vom Fachbereich Wirtschaftswissenschaften der FH Harz

In den ,Herzer wirtschaftswissenschaftlichen Schriften" werden Beitrage zu aktuellen okonomischen Fragestellungen veroffentlicht. Die FH Harz in Wernigerode, an der ehemaligen Nahtstelle zwischen Ost und West gelegen, leistet mit dieser Reihe des Fachbereichs Wirtschaftswissenschaften einen Beitrag zur Erfullung der Bruckenfunktion zwischen Theorie und Praxis, zwischen Wirtschaft, Technik und Kultur.

Georg Westermann (Ed.)


Springer Fachmedien Wiesbaden GmbH

Die Deutsche Bibliothek - CIP-Einheitsaufnahme

Westermann, Georg: Data envelopment analysis in the service sector/ Georg Westermann (ed.). - Wiesbaden: Dt. Univ.-Verl.; Wiesbaden: Gabler, 1999 (Gabler Edition Wissenschaft : Herzer wirtschaftswissenschaftliche Schriften)

Aile Rechte vorbeholten

© Springer Fachmedien Wiesbaden 1999 Ursprünglich erschienen bei Deutscher Universitäts-Verlag 1999

Lektorot: Ute Wrosmonn

Dos Werk einschliel31ich oiler seiner Teile ist urheberrechtlich geschutzt. Jede Verwertung aul3erhalb der engen Grenzen des Urheberrechtsgesetzes ist ohne Zustimmung des Verlag~~ unzuli:issig und strafbar. Dos gilt insbesondere fur Vervielfi:iltigungen, Ubersetzungen, Mikroverfilmungen und die Einspeicherung und Verarbeitung in elektronischen Systemen.

hHp:/ /www.gabler.de hHp:/ /www.duv.de

H&hste inhaldiche und technische Oualiti:it unserer Produkte ist unser Ziel. Bei der Produktion und Verbreitung unserer Bucher wollen wir die Umwelt schonen. Dieses Buch ist deshalb auf si:iurefreiem und chlorfrei gebleichtem PaP.ier gedruckt. Die Einschweii3Folie besteht aus Polyi:ithylen und demit aus organischen Grundstolten, die wader bei der Herstellung noch bei der Verbrennung Schadstoffe freisetzen .

Die Wiedergabe von Gebrauchsnamen, Handelsnamen, Warenbezeichnungen usw. in diesem Werk berechtigt ouch ohne besondere Kennzeichnung nicht zu der Annahme, dal3 solche Nomen im Sinne der Worenzeichen- und Markenschutz-Gesetzgebung als frei zu betrachten wi:iren und daher von jedermann benutzt warden durften.

ISBN 978-3-8244-7012-9 ISBN 978-3-663-08343-6 (eBook) DOI 10.1007/978-3-663-08343-6

Data Envelopment Analysis in the Public and Private Service Sector v

Preface to "Efficiency in the Private and Public Service Sector"

The Aims of this Book

The October 1998 Symposium m Wemigerode, Germany, was the successive European Conference on Data Envelopment Analysis (DEA) to the 1997 meeting in Marseilles. The label, "European", has to be seen with respect to two features:

First, and very obviously, the conference site, The Harz University of Applied Studies and Research, is located in the middle of the European continent. Secondly, an explicitly formulated goal of the symposium was the formation of a European Network of researchers who are concerned with the development and application of the DEA methodology. Looking at the list of participants, this aim has been fulfilled. Over and above that, contacts with colleagues from North and South America as well as Australia and Asia were invigorated and enlarged.

Although DEA has often and effectively been applied to measure efficiency in North American companies and public institutions, DEA is largely unknown amongst European practitioners and academics. Especially the public and private service sector shows some specificity that classical measurement and benchmarking instruments normally fail to serve. Missing prices for (non-marketed) public goods or distinct firmspecific solutions to the same problem - and ,thus, different production techniques -are only two of the frequently arising problems.

Efficiency in the Private and Public Service Sector - Recent Developments in the Application and Methodology of Data Envelopment Analysis was chosen as the title of the conference volume in order to emphasize the adjustment and development of the DEA methodology to the requirements of the service sector and to highlight the widespread application of the instrument - especially to problems the orthodox methods fail to solve or to emerging fields within the service sector.

A striking observation with respect to the DEA literature is that empirical applications in most cases are lagging far behind the theoretical methodological developments. There is often a clear cut distinction between contributions from mathematicians and/or programmers and papers by DEA users. This distinction might not be confined only to the DEA methodology. But the fact should nevertheless be noticed and the theory-to-application-lag should not be allowed to become too large.

VI Data Envelopment Analysis in the Public and Private Service Sector

Context of the Contributions

The number of presentations during the Wemigerode Symposium clearly exceeds the scope of this volume. We decided to include papers that fit into the service sector context either from the methodological or from the empirical point of view. Thus, papers not published in this book are not necessarily of inferior quality.

Lawrence M. Seiford summarizes the development of the DEA methodology over the past two decades. In addition, he provides an agenda for the future in this field of research. The need for a "stochastic" DEA is emphasized and a comprehensive literature survey is given. Thus, this first article may serve as an introduction.

The contributions of Rajiv D. Banker et al. and of Rob Ball and Elizabeth Roberts are splendid examples for successfully applying recent methodological developments in order to solve actual empirical problems in the public accounting industry or in the health sector. Other papers concentrate more or less on theoretical or methodological issues and can be summarized as follows:

Matthias Staat argues in his paper that including productivity relevant but individually uncontrollable variables " ... may lead to comparisons of qualitatively different DMUs." He contrasts the effects of different model specifications for empirical analyses.

Laurens Cherye and Tom van Puyenbroeck point to the fact that for empirical investigations the use of the radial efficiency measure in combination with the

existence of zero data and/or slacks might " ... result in wrong management conclusions." They develop a modified additive model to overcome these difficulties.

Holger Scheel investigates whether the BCC model shows the property of continuity. Continuity ensures that small data errors cause only small perturbations in the efficiency measure. Again the case of zero data is shown to be of crucial importance.

Wenbin Liu and John Sharp explain the empirical problems arising from a mixture of positive and negative responding outputs to changes in inputs. They derive new DEA models from a goal programming approach, which might be applied in cases showing the above mentioned properties.

Data Envelopment Analysis in the Public and Private Service Sector VII

Dieter Gstach compares the performance of the stochastic DEA+ model to the Ray

Frontier Approach by Lothgren (1997). This contribution concentrates on simulations

in order to demonstrate how both models behave when bounded or unbounded noise is

assumed.

Rolf Fare et al. demonstrate how DEA models might be used for computing shadow

prices for firm outputs. This seems especially useful in cases when " ... a market price is

unavailable, administered or distorted." The public service sector is an example for

these non-marketed goods.

Five mainly application oriented contributions build the final part of the book:

Georg Westermann and Gerhard Johnson apply DEA efficiency scores from

different input-output combinations to construct strategic management portfolios for

social service units. The transformation of inputs into capacity, production and final

effectiveness is analyzed.

Katrin Allen summarizes the state-of-the-art in applying DEA to studies in the

ecological context. This paper corresponds very well to the methodological aspects by

Liu and Sharp within this volume.

M. C. Sampaio de Sousa and Francisco S. Ramos measure the efficiency of public

spending in Brazilian Municipalities. The huge database applied, the input-output

combinations and the results are of interest for all researchers engaged in accounting

for public administration performance.

Finn R. Forsund and Kjell 0. Kalhagen are concerned with the evaluation of

Norwegian colleges and trace the respective productivity developments for three years.

This contribution convinces by its input-output model and the database on the

department level..

P.-Y. Badillo compares the DEA scores for technical efficiency to the financial

performance of telecommunication operators in Europe and the USA over the period

from 1986 to 1997. The effects of monopoly and deregulation on both indicators are

discussed.

VIII Data Envelopment Analysis in the Public and Private Service Sector

Acknowledgements

This book would not have been completed without the help of many people to whom I am most grateful.

The ministry of education of Sachsen-Anhalt funded the travel expenses of the invited speakers of the conference. The Hochschule Harz made its facilities available to the symposium and provided financial support for the printing of this volume.

The help of Caren Labohm, Ricarda Otto and Stefan Schneider in organizing and running the conference in Wemigerode is also acknowledged with many thanks.

As editor I am indebted to Stefan Schneider who was responsible for collecting the papers from the authors and completing the manuscript.

While I am giving away thanks for advice and help, I am still responsible for remaining mistakes.

Georg Westermann

Data Envelopment Analysis in the Public and Private Service Sector

Contents

Lawrence. M. Seiford:

Data Envelopment Analysis: Twenty Years Out

Matthias Staat:

Treating non-discretionary variables one way or the other:

implications for efficiency scores and their interpretation

Laurens Cherchye, Tom van Puyenbroeck:

Non-Radial Efficiency as Semi-Radial Efficiency

Holger Scheel:

Continuity of the BCC Efficiency Measure

Wenbin Liu, John Sharp:

DEA Models via Goal Programming

Dieter Gstach:

Bounded vs. Unbounded Noise in Efficiency Estimation:

Performance of Alternative Estimators

Rolf Fare, Shawna Grosskopf, Pontus Roos:

Price Indexes for Nonmarketed Goods

Rob Ball, Elizabeth Roberts, Richard Thomas:

Lessons Learned for DEA Practice from Health Care

Applications in the UK

Rajiv D. Banker, Hsihui Chang, Reba Cunningham, Ram Natarajan:

Recent Advances in Data Envelopment Analysis:

An Illustrative Application to the U.S. Public Accounting

Industry

IX

1

23

51

65

79

103

121

133

159

X Data Envelopment Analysis in the Public and Private Service Sector

Georg Westermann, Gerhard Johnson:

Combining DEA and "Transformation Stages":

Management Strategies for the Disability Service

Units of the St. Georg Association

Katrin Allen:

DEA in the ecological context- An overview

Maria da Concei-;;ao Sampaio de Sousa, Francisco S. Ramos:

Measuring Public Spending Efficiency in Brazilian

Municipalities: A Nonparametric Approach

Finn R. F0rsund, Kjell Ove Kalhagen:

Efficiency and Productivity of Norwegian Colleges

Patrick-Y. Badillo:

Efficiency and financial performances in telecommunications

183

203

237

269

309

Data Envelopment Analysis: Twenty Years Out

Lawrence M. Seiford'

Abstract

This paper briefly traces the evolution of DEA from the initial publication by Charnes, Cooper and Rhodes (1978) to the current state-of-the-art (SOA). The state of development of DEA is characterized at four points in time to provide a perspective in both directions-past and future. An evolution map is provided which illustrates DEA growth during the twenty year period, the timing of the major events, and the interconnections and influences between topics. An extensive DEA bibliography is provided.

(Portions of this paper are based on Seiford, Lawrence M. ,Data Envelopment Analysis: The Evolution of the State-of-the-Art (1978--1995)." Journal of Productivity Analysis 7, no. 213 (1996):pp. 99-138.)

1 Mechanical and Industrial Engineering, University of Massachusetts, Amherst, MA 01003 USA

[email protected]

2 Data Envelopment Analysis: Twenty Years Out

Structure

Introduction

2 In the beginning

3 Evolution ofDEA

References

Data Envelopment Analysis: Twenty Years Out 3

1 Introduction

This paper briefly traces the evolution of DEA from the initial publication by Chames, Cooper and Rhodes (1978) to the current state-of-the-art (SOA). In characterizing the development of DEA over the past 20 years, I describe the then-current SOA at four points in time. These "snapshots" of the development provide a perspective in both directions, i.e., in terms of what was known as well as what remained unknown. The particular milestones (1980, 1985, 1990 & 1995) were chosen to allow a broad-brush description of incremental growth, enhancement and improvement of the methodology. For accuracy and ease of verification, the development and timeline is with respect to published articles (instead of working papers or oral tradition).

In the discussion that follows the reader is referred to the Evolution Map (Figure 1) as a pictorial guide.

2 In the beginning

Before DEA could flourish, the necessary foundations to support its growth and development had to be in place. Antecedents include works of Afriat (1972), Aigner and Chu (1968), Shephard (1970), Debreu (1951), and Farrell (1957)2, the conceptual definitions of Koopmans (1951) and Pareto (1927}, and the Linear Fractional transformation ofChames and Cooper (1962).

With these pieces in place, DEA actually started with Rhodes' dissertation topic-how to evaluate Program Follow-Through in U.S. education. The first published article describing the methodology and labeling the approach as Data Envelopment Analysis was Chames, Cooper, and Rhodes (1978). (Some researchers still recall the TIMS XXIV International Meeting in Hawaii in June 1979 where Chames and Cooper gave the first presentation on DEA.)

For several years I heard rumors of early DEA-type work from the mid-60s. Through the assistance of Knox Lovell, I finally obtained copies of four papers, Bressler ( 1966), Boles ( 1966), Seitz ( 1966), and Sitorus ( 1966), which appeared in the Proceedings of the 39th Annual Meeting of the Western Farm Economics Association. These four papers were presented in a single session at the meeting. The purpose of the session was to recall Farrell's approach to the attention of economists and statisticians. (It was

2 Contrary to numerous statements in the literature, Farrell did not employ LP in his 1957 paper. In fact, it was A. J. Hoffman, one of the discussants for the paper, who pointed out to Farrell (after the fact) that the problem he had described could be formulated and solved as an LP. In a later paper, Farrell and Fieldhouse (1962) provide the LP formulation for the single output case which Hoffman suggested.


well received in 1957 but there had been virtually no further application of the method.) The four papers: (i) summarize Farrell's ideas; (ii) provide LP formulations and efficient computational procedures for a variety of problems in technical efficiency including the multiple output case; and give illustrative applications to (iii) steamelectric generating plants and (iv) aggregate census data, respectively. Although it appears that DEA was anticipated over a decade earlier, the technique again lay dormant until the ratio interpretation and introduction of the methodology to the ORIMS community by Charnes, Cooper, and Rhodes (1978). As we shall see, whether due to the timing, the packaging, or the proponents, this time the methodology flourished as evidenced by the rapid growth and widespread diffusion across disciplines.

3 Evolution of DEA

In our attempt to characterize the development of DEA over the past 20years we will examine the state-of-the-art (SOA) at several milestones in the evolution of the methodology For each we will contrast what was then known, what remained unknown, the state of DEA computation, range of application areas, and what was considered to be then-current SOA.

3.1 State-of-the-art circa 1980

In 1980, the SOA ofDEA was much simpler than it is today. Model choice was limited to the single constant-returns-to-scale model of Charnes, Cooper, and Rhodes (1978) which measured only technical efficiency. The few published applications were publicsector, not-for-profit and primarily in education (Bessent and Bessent, 1980; Banker, 1980; Charnes and Cooper, 1980; Charnes, Cooper and Rhodes, 1980; Schinnar, 1980). The focus was on relative efficiency of organizational units; other uses for the methodology were not yet discerned although a game-theoretic interpretation had been proposed by Banker (1980).

DEA computation in 1980 was extremely primitive. SOA DEA codes were based on a

naive implementation of epsilon as 10-6. (As shown in Ali and Seiford (1993), this can produce unreliable results.) The most significant break-through in advancing SOA applications at this time was the Program Follow-Through/ Non-Follow-Through evaluation that formed the basis of Ed Rhodes' dissertation and was published in Management Science in 1981.



By 1985, DEA theory was considerably more advanced. Model selection had expanded to encompass a wide range of models. The constant-returns-to-scale model of Charnes, Cooper, and Rhodes (1978) was joined by the variable-returns-to-scale model of Banker, Charnes, and Cooper (1984) for measuring scale efficiency, Multiplicative models for piecewise log-linear frontiers (Charnes et al., 1982, 1983), and the nonoriented Additive model (Charnes et al., 1985). A firm link to production theory was established through the theoretical characterizations of the inherent structure and capabilities of Pareto-Koopmans (empirical) frontier production functions given in Charnes et al. (1985).

The primary focus remained on relative efficiency but application areas now included hospitals (Bedard, 1985; Nunamaker, 1983; Sherman, 1981, 1984), post offices (Deprins et al., 1984), electric utilities (F\"are et al., 1983, 1985; Thomas, 1985), banking (Gold, 1982; Joseph et al., 1983; Sherman and Gold, 1985), mass transit (Kusbiantoro, 1985), courts (Lewin et al., 1982), agriculture (F\"are et a!., 1985), maintenance (Bowlin, 1984), mining (Byrnes et a!., 1984), pharmacies (Capettini, 1985), and USAF fighter wings (Charnes et al., 1985). Applications in education were now numerous due to the efforts of the Bessants' Educational Productivity Council at UT Austin (Bessent eta!., 1981, 1983, 1984, 1985; Blair, 1983; Garrett, 1985; Katims, 1985; Reaves, 1983; Splitek, 1981; Stone, 1984; Thorogood, 1983).

The perspective on DEA was widening; for example, issues of ownership versus efficiency were being examined (Byrnes, 1985; F\"are eta!., 1985) and DEA was also making inroads into marketing (Charnes et a!., 1985; Eechambadi, 1985). Links between DEA and basic production theory were established in Byrnes eta!. (1984) and F\"are et a!. (1985). The first of several comparisons of DEA with regression (Bowlin et a!., 1985) had appeared and the controversy or misunderstanding over the NonArchimedean (epsilon) had arisen (Boyd and F\"are, 1984; Charnes and Cooper, 1984)'. Researchers were beginning to look at stochastic issues (Sengupta, 1982).

However, DEA advances in this period were, for the most part, limited to models and theoretical enhancements. DEA computation had not yet progressed beyond the early stages. Examples of then-current SOA include most productive scale size (MPSS) (Banker, 1984), the Additive model and associated Pareto-Koopmans foundation for DEA (Charnes eta!., 1985), and window analysis (Charnes eta!., 1985).

3 The role of the NonArchimedean epsilon in detecting non-proportional inefficiencies (slacks) is much better understood today. See Ali and Seiford (1993) and Ali, Lerme, and Seiford (1995).



By 1990, DEA was becoming fully developed. Significant advances had been made on all fronts: models, extensions, computation, and practice. The UNC conference (1988) and the Austin DEA conference ( 1989) had contributed greatly to this development. Dialog and collaboration between researchers from Economics and Operations Research/Management Science had been initiated. The Journal of Econometrics Special Issue resulting from the UNC conference appeared with lead articles by Bauer (1990) and Seiford and Thrall (1990) providing dual perspectives.

Theoretical refinements and advances were numerous. Studies comparing the various DEA models (Ahn et al., 1988; Charnes et al., 1990; Epstein et al., 1989; F\"are et al., 1988; Seiford et al., 1990) provided a framework for understanding implicit assumptions and requirements. A Malmquist index (F\"are et al., 1989) had been developed to examine components of productivity growth, and technical, scale, and allocative efficiency (Banker and Maindiretta, 1988; Morey et al., 1990; RetzlaffRoberts, 1990) had been compared and contrasted. Non-convex models had been introduced by Petersen (1990) which enlarged the perspective on basic assumptions of DEA. A number of significant model extensions had been developed including: the capability to handle nondiscretionary variables and/or categorical variables (Banker and Morey, 1986); the ability to incorporate judgement (restricting multipliers (Dyson and Thanassoulis, 1988; Wong and Beasley, 1990), the Cone Ratio model (Charnes et al., 1989, 1990), and Assurance Regions (Thompson et al., 1986, 1990)); and model ordinal relationships (Golany, 1988). Connections were being established with the field of decision analysis via DEA-inspired consensus ranking approaches (Cook et al., 1990) and game theoretic interpretations (Banker et al., 1989, Charnes et al., 1989, 1990; Clarke, 1988). Finally, sensitivity and stability studies (Charnes et al., 1989, 1990) and translation invariance (Ali and Seiford, 1990) round out the theoretical contributions.

Computational issues had surfaced and been addressed (Ali, 1990). The nonArchimedean models had been correctly implemented in a DEA code (e.g., IDEAS, 1989) as a two-stage preemptive procedure. Other computational requirements, specific to DEA, had been recognized and production-quality DEA codes were available that employed specialized pricing rules, anticycling techniques to address degeneracy, and eschewed sparse matrix techniques.

Published applications from this period addressed more complex issues involving property tax valuation (Adolphson et al., 1987, 1989), software development (Banker et al., 1987, 1989), institutions of higher learning (Ahn, 1987; Ahn et al., 1987, 1988, 1989), university departments (Beasley, 1990; Tomkins, 1988), energy use (Baxter et al., 1986), DRG reimbursement (Borden, 1986, 1988), site selection (Bowen, 1990; Desai et al., 1990), spatial efficiency (Desai and Storbeck, 1990), farming (Byrnes et


a!., 1987), unions (Byrnes et a!., 1988), sports (Camm, 1988), electric cooperatives (Chames et a!., 1989), Chinese cities (Chames et a!., 1989), individual physicians (Chilingerian, 1989, 1990), highway maintenance (Cook eta!., 1988, 1990), regulatory environments (F\"are et a!., 1986, 1989), organizational slack (Golden, 1989), airlines (Johnston, 1990), logistics systems (Kleinsorge eta!., 1989), parks (Rhodes, 1986), pubs (Sant, 1989), construction (Shash, 1988), telecommunications (Majumdar, 1990), and US Army recruiting (Thomas, 1990).

Examples of SOA include relaxed non-convex assumptions (Petersen, 1990), various approaches to placing restriction on the possible range of multipliers for incorporating judgement or managerial preference (see earlier references), a more balanced perspective on DEA ( Epstein et a!., 1989; Stolp, 1990), and connections with Econometrics (Varian, 1990).


In the recent years (1990-1995) there have been significant theoretical advances but it has been DEA practice that has evolved the most extensively. DEA is now recognized as a versatile and effective tool for data analysis and is often used as an exploratory technique (E-DEA) for "visualizing" the data. Applications are frequently large-scale requiring significant computational power, and a flexible user interface for data management and model management has become as important as a robust and accurate optimizer. Parallel processing environments and new solution approach can dramatically reduce solution times as reported in Barr and Durcholz. (1997).

The applications continue to become more sophisticated and recent studies have focused on revenue transfers (local aid) (Ali et a!., 1993), TQM (Bailey, 1993), benchmarking and identification of best-practice (Collier and Storbeck, 1993; Chilingerian, 1995; Golany and Thore, 1997), forecasting bank failures (Barr et a!., 1993, 1994; Siems, 1991 ), strategy (Day et a!., 1994, 1995; Ali and Lerme, 1997), pollution (Haynes et a!., 1994), improved performance indexes in sports (Anderson, 1997), and X-efficiency (Bohnet and Beck, 1990; Frantz, 1992; Leibenstein and Maital, 1992). At the same time DEA has moved from being an esoteric research methodology to a more mainstream analytical tool as evidenced by (i) its inclusion in an introductory MBA OR textbook (Anderson, Sweeney, and Williams, 1991), and (ii) its being featured in Fortune magazine (Norton, 1994).

Significant theoretical contributions from this period would include the free disposal hull (FDH) (non-convex) model of Tulkens (1993) as well as work on the statistical foundations of DEA (Banker, 1993; Simar, 1992) and the chance-constrained framework introduced in Land eta!. (1993). Examples of current SOA would be the graphical backend for visualization of DEA results developed by Paradi et a!. at the


University of Toronto, large-scale benchmarking studies for the Federal Reserve Bank and USAF medical treatment facilities by Barr, Seiford and others, cost containment in HMOs (Chilingerian and Sherman, 1997), and cone analysis to reveal preferred practice regions (Ward eta!., 1997).

3.5 Current view of State-of-the-art in DEA

The previous sections have described the development and migration of DEA. We tum now to the question: what is the current state-of-the-art? SOA DEA models would include CCR, BCC, Additive, and FDH. The interrelationships between these models are best explained within the framework given in Ali, Lerme, & Seiford (1995) and the interested reader is referred there.

Any of the convex models can be combined with various theoretical extensions (e.g., nondiscretionary variables, categorical variables, ordinal relationships, etc.) and multiplier restrictions can be introduced to incorporate judgement. For cross sectional/time series/panel data one could employ window analysis or a Malmquist index to examine changes across time periods.

However, some modeling issues are still not easily handled. Negative inputs or outputs cause difficulty; ranking units can be problematic. With large scale application becoming more frequent, computational issues become even more important. Stochastic issues are being addressed and statistical tools are being developed but much work remains to be done.

3.6 Future issues in DEA

It's usually the case that new methodologies and generalizations give rise to new possibilities and new questions and DEA is certainly no exception. DEA has become an important and widespread analytical tool. With increasing use in real-world largescale complex applications, the need for validation of these studies becomes more critical.

Software has become increasingly important for the large-scale and complex DEA studies now being conducted. This dependence on software raises additional issues. How can one insure the availability of robust accurate DEA software? How should one validate DEA codes as producing accurate results? If validation should be on a suite of test problems, what types and how wide a range of conditions are sufficient? This validation issue remains important whether one is using specialized DEA software or a standard LP package, e.g., SAS, GAMS, LINDO, etc., to perform the analysis.


Along with the issue of accurate results, there is a need to develop more effective means for presentation of the results. What visualization/presentation formats should be implemented for viewing solution results? These graphical back-ends would be used both for E-DEA (exploratory DEA) and for management presentations. The visualization capability becomes critical for large-scale problems particularly if one is searching for patterns or explanations of inefficiency across units and subunits.

The final topic, Stochastic DEA, should come as no surprise. It appears on almost everyone's list of Future Research Areas for DEA and presents a formidable challenge. The essential problem is noise (e.g., measurement error) in the underlying data. Promising approaches and partial solutions are given in Banker (1993), Simar (1992), Korostelev eta!. (forthcoming), Land et a!. (1993), and Olesen and Petersen (1995). However, the development of Stochastic DEA which can incorporate measurement error and other sources of noise that inevitably contaminate the data used in an analysis is far from complete. Lovell (1994) eloquently states the importance of this problem. "Until a stochastic DEA is developed, statisticians and econometricians will remain skeptical of the managerial and policy implications drawn from DEA."

In my opinion, it is this last topic, Stochastic DEA, which is the most critical and the most difficult future issue in DEA. However, I'm confident that when we reach the next progress milestone, the year 2000, it will be listed among the SOA accomplishments of the prior five years.

References

Adolphson, Donald L., Gary C. Cornia, and Lawrence C. Walters (1989): Railroad Property

Valuation Using Data Envelopment Analysis, Interfaces 19, no. 3, p. 18-26.

Adolphson, Donald L., Gary C. Cornia, and Lawrence C. Walters (1987): The Relative Efficiency of

Railroads and Obsolescence, Proceedings of the Seventeenth Annual Program on the Appraisal

of Utilities and Railroad Property for Ad Valorem Taxation, 97-130. Wichita, KS: Wichita

State University.

Afriat, S. N. (1972): Efficiency Estimation of Production Functions, International Economic review

13, no. 3, p. 568-598.

Ahn, Tae Sik. (1987): Efficiency and Related Issues in Higher Education: A Data Envelopment

Analysis Approach, Ph. D. dissertation, Graduate School of Business, University of Texas.

Ahn, Tae Sik, V. Arnold, A. Charnes, and W. W. Cooper (1989): DEA and Ratio Efficiency Analyses

for Public Institutions of Higher Learning in Texas, in: Research in Governmental and

Nonprofit Accounting, 165-185. Editors James L. Chan, and James M. Patton. Greenwich, CT:

JAI Press.


Ahn, Tae Sik, A. Charnes, and W. W. Cooper. (1988): Efficiency Characterizations in Different DEA Models, Socio-Economic Planning Sciences 22, no. 6, p. 253-257.

Ahn, Tae Sik, and Lawrence M. Seiford (1993): Sensitivity ofDEA to Models and Variable Sets in a Hypothesis Test Setting: The Efficiency of University Operations, in: Creative and Innovative Approaches to the Science of Management, 191-208. editor Yuji Ijiri. New York: Quorum Books.

Aigner, D. J., and S. -F Chu {1968): On Estimating the Industry Production Function, American Economic Review 58, no. 4, p. 826-839.

Ali, Agha Iqbal {1990): Data envelopment analysis: computational issues, Computers, Environment and Urban Systems 14, no. 2, p. 157-165.

Ali, Agha Iqbal, and Catherine S. Lerme {1997): Comparative Advantage and Disadvantage in DEA, Annals of Operations Research (73 ).

Ali, Agha Iqbal, Catherine S. Lerme, and Robert A. Nakosteen {1993): Assessment of Intergovernmental Revenue Transfers, Socio-Economic Planning Sciences 27, no. 2, p. 109-118.

Ali, Agha Iqbal, Catherine S. Lerme, and Lawrence M. Seiford (1995): Components of Efficiency Evaluation in Data Envelopment Analysis, European Journal of Operational Research 80, no. 3, p. 462-473.

Ali, Agha Iqbal, and Lawrence M. Seiford (November 1993): Computational Accuracy and Infinitesimals in Data Envelopment Analysis, INFOR 31, no. 4, p. 290-297.

Ali, Agha Iqbal, and Lawrence M. Seiford (November 1990): Translation Invariance in Data Envelopment Analysis, Operations Research Letters 9, no. 6, p. 403-405.

Anderson, David R., Dennis J. Sweeney, and Thomas A. Williams (1991): Linear Programming Applications: Data Envelopment Analysis, in: An Introduction to Management Science: Quantitative Approaches to Decision Making, p. 147-152. Sixth ed. St Paul, MN: West Publishing Company.

Anderson, Timothy R., and Gunter P. Sharp {1997): A New Measure of Baseball Batters Using DEA, Annals of Operations Research (73).

Bailey, Marshall Hamilton, III. (1993): Public Administration Efficiency Through Total Quality Management, Ph. D. dissertation, George Mason University.

Banker, Rajiv D. {1980): A Game Theoretic Approach to Measuring Efficiency, European Journal of Operational Research 5, p. 262-266.

Banker, Rajiv D. (July 1984): Estimating Most Productive Scale Size Using Data Envelopment Analysis, European Journal of Operational Research 17, no. I, p. 35-44.

Banker, Rajiv D. (October 1993): Maximum Likelihood, Consistency and Data Envelopment Analysis: A Statistical Foundation, Management Science 39, no. 10, p. 1265-1273.


Banker, Rajiv D. (1980): Studies in Cost Allocation and Efficiency Evaluation, D. B. A. dissertation, Graduate School of Business Administration, Harvard University.

Banker, Rajiv D., A. Chames, Richard L. Clarke, and W. W. Cooper (1989): Erratum: Constrained game formulations and interpretations for data envelopment analysis, European Journal of Operational Research 42.

Banker, Rajiv D., A. Chames, and W. W. Cooper (1984): Some Models for Estimating Technical and Scale Inefficiencies in Data Envelopment Analysis, Management Science 30, no. 9, p. 1078-1092.

Banker, Rajiv D., A. Chames, W. W. Cooper, and Richard L. Clarke (June 1989): Constrained Game Formulations and Interpretations for Data Envelopment Analysis, European Journal of Operational Research 40, no. 3, p. 299-308.

Banker, Rajiv D., A. Chames, W. W. Cooper, and Ajay Maindiratta (1988): A Comparison of DEA and Translog Estimates of Production Frontiers Using Simulated Observations From a Known Technology, in: Applications of Modem Production Theory: Efficiency and Productivity, editors Ali Dogramaci, and RolfHre. Boston: Kluwer Academic Publishers.

Banker, Rajiv D., A. Chames, W. W. Cooper, and A. P. Schinnar (1981): A Bi-Extremal Principle for Frontier Estimation and Efficiency Evaluations, Management Science 27, no. 12, p. 1370-1382.

Banker, Rajiv D., R. F. Conrad, and R. P. Strauss (January 1986): A Comparative Application of Data Envelopment Analysis and Translog Methods: An Illustrative Study of Hospital Production, Management Science 32, no. I, p. 30-44.

Banker, Rajiv D., and Holly Hanson Johnston (1994): Evaluating the Impacts of Operating Strategies on Efficiency in the U.S. Airline Industry, in: Data Envelopment Analysis: Theory, Methodology, and Applications, editors A. Chames, W. W. Cooper, Arie Y. Lewin, and Lawrence M. Seiford. Boston: Kluwer Academic Publishers.

Banker, Rajiv D., and Chris F. Kemerer (October 1989): Scale Economies in New Software Development, IEEE Transactions on Software Engineering 15, no. 10, p. 1199-1205.

Banker, Rajiv D., and Ajay Maindiratta (1988): Nonparametic Analysis of Technical and Alloctive Efficiencies in Production, Econometrica 56, no. 6, p. 1315-1332.

Banker, Rajiv D., and Ajay Maindiratta (1986): Piecewise Loglinear Estimation of Efficient Production Surfaces, Management Science 32, no. I, p. 126-135.

Banker, Rajiv D., and Richard C. Morey (July 1986): Efficiency Analysis for Exogenously Fixed Inputs and Outputs, Operations Research 34, no. 4, p. 513-521.

Banker, Rajiv D., and Richard C. Morey (December 1986): The Use of Categorical Variables in Data Envelopment Analysis, Management Science 32, no. 12, p. 1613-1627.

Banker, Rajiv D., and Robert M. Thrall (9 October 1992): Estimation of Returns to Scale Using Data Envelopment Analysis, European Journal of Operational Research 62, no. I.


Banker, Rajiv D., Srikant M. Datar, and Chris F. Kemerer (December 1987): Factors Affecting Software Maintenance Productivity: An Exploratory Study, in: Proceedings of the 8th. International Conference on Information Systems, p. 160-175. Pittsburgh.

Barr, Richard S., and Matthew L. Durcholz (1997): Parallel and Hierarchical Decomposition Approaches for Solving Large-Scale Data Envelopment Analysis Models, Annals of Operations Research (73).

Barr, Richard S., Lawrence M. Seiford, and Thomas F. Siems (1993): An Envelopment-Analysis Approach to Measuring the Managerial Efficiency of Banks, Annals of Operations Research 45,p.l-19.

Barr, Richard S., Lawrence M. Seiford, and Thomas F. Siems (1994): Forcasting Bank Failure: A Non-Parametric Frontier Estimation Approach, Recherches Economiques de Louvain 60, no. 4, p. 417-429.

Bauer, Paul W. (1990): Recent developments in the econometric estimation of frontiers, Journal of Econometrics 46, no. y,, p. 39-56.

Baxter, Lester W., Stephen L. Feldman, Arie P. Schinnar, and Robert M. Wirtshafter (April 1986): An Efficiency Analysis of Household Energy Use, Energy Economics, p. 62-73.

Beasley, J. E. (1990): Comparing University Departments, Omega 18, no. 2.

Bedard, Jean Catherine (1985): Use of Data Envelopment Analysis in Accounting Applications: Evaluation and Illustration by Prospective Hospital Reimbursement, Ph. D. dissertation, Graduate School of Business, University of Wisconsin.

Bessent, Authella M., and E. Wailand Bessent (December 1981): A Fractional Programming Model for Determining the Efficiency of Decision Making Units, ERIC Clearinghouse on Educational Management, University of Oregon, Eugene, Oregon.

Bessent, Authella M., and E. Wailand Bessent. ,Determining the Comparative Efficiency of Schools Through Data Envelopment Analysis." Educational Administration Quarterly 16, no. 2 (March 1980): 57-75.

Bessent, Authella M., E. Wailand Bessent, A. Charnes, W. W. Cooper, and N. Thorogood (March 1983): Evaluation of Educational Program Proposals by Means of Data Envelopment Analysis, Educational Administration Quarterly 19, no. 2, p. 82-107.

Bessent, Authella M., E. Wailand Bessent, Joyce Elam, and D. Long (1984): Educational Productivity Council Employs Management Science Methods to Improve Educational Quality, Interfaces 14, no. 6, p. 1-8.

Blair, Larry Delwood (1983): A Comparative Analysis of the Financial Practices of School Districts Selected by Data Envelopment Analysis Efficiency Indices, Ph. D. dissertation, College of Education, University of Texas.

Bohnet, A., and M. Beck (1990): The Impact of the Income Tax on Work Effort and X-inefficiency in Enterprises, in: Studies in Economic Rationality: X-Efficiency Examined and Extolled. Essays written in the tradition of and to honor Harvey Leibenstein, 227-251. editors K. Weiermair, and M. Perlman. Ann Arbor: 1990.


Boles, James N. (1966): Efficiency Squared - Efficient Computation of Efficiency Indexes, Proceedings of the Thirty Ninth Annual Meeting of the Western Farm Economics Association, p. 137-142.

Borden, James Patrick (1986): An Assessment of the Impact of Diagnosis Related Group (DRG)based Reimbursement on the Technical Efficiency of New Jersey Hospitals, Ph. D. dissertation, Drexel University.

Borden, James Patrick (June 1988): An Assessment of the Impact of Diagnosis-Related Group (DRG)-Based Reimbursement on the Technical Efficiency of New Jersey Hospitals Using Data Envelopment Analysis, Journal of Accounting and Public Policy 7, no. 2, p. 77-96.

Bowen, William M. (1990): The Nuclear Waste Site Selection Decision--a Comparison of Two Decision-Aiding Models, Ph. D. dissertation, Indiana University.

Bowlin, William Frank (1984): A Data Envelopment Analysis Approach to Performance Evaluation in Not-for-profit Entities with an Illustrative Application to the U.S. Air Force, Ph. D. dissertation, Graduate School of Business, University of Texas.

Bowlin, William F., A. Charnes, W. W. Cooper, and H. David Sherman (1985): Data Envelopment Analysis and Regression Approaches to Efficiency Estimation and Evaluation, Annals of Operations Research 2, no. I, p. 113-138.

Boyd, G., and Rolf Fare (1984): Measuring the Efficiency of Decision Making Units: A Comment, European Journal of Operational Research 15, p. 331-332.

Bressler, R. G. (1966): The Measurement of Productivity Efficiency, Proceedings of the Thirty Ninth Annual Meeting of the Western Farm Economics Association, p. 129-136.

Byrnes, P. (1985): Ownership and Efficiency in the Water Supply Industry: An Application of the Nonparametric Programming Approach to Efficiency Measurement, Ph. D. dissertation, Southern Illinois University.

Byrnes, P., Rolf Fare, and S. Grosskopf ( 1984): Measuring Productive Efficiency: An Application to Illinois Strip Mines, Management Science 30, no. 6, p. 671-681.

Byrnes, P., Rolf Fare, S. Grosskopf, and C. A. Knox Lovell (1988): The Effect of Unions on Productivity: U.S. Surface Mining of Coal, Management Science 34, no. 9, p. 1037-1053.

Byrnes, P., Rolf Fare, S. Grosskopf, and S. Kraft (1987): Technical Efficiency and Size: The Case of 1980 Illinois Grain Farms, European Review of Agricultural Economics 14, no. 4, p. 367-381.

Camm, J.D., and T. J. Grogan (November 1988): An Application of Frontier Analysis: Handicapping Running Races, Interfaces 18, no. 6, p. 52-60.

Capettini, Robert, David A. Dittman, and Richard C. Morey (June 1985): Reimbursement Rate Setting for Medicaid Prescription Drugs Based on Relative Efficiencies, Journal of Accounting and Public Policy 4, no. 2, p. 83-110.

Charnes, A., Charles T. Clark, W. W. Cooper, and Boaz Golany (1985): A Developmental Study of Data Envelopment Analysis in Measuring the Efficiency of Maintenance Units in the U.S. Air


Forces, in: Annals of Operation Research, 95-112. editors Russell G. Thompson, and Robert M.

Thrall.

Chames, A., Richard L. Clarke, and W. W. Cooper (1989): An Approach to Testing for Organizational Slack with R. Banker's Game Theoretic Formulation of DEA, in: Research in Governmental and Nonprofit Accounting, 211-230. editors James L. Chan, and James M.

Patton. Greenwich, CT: JAI Press.

Chames, A., and W. W. Cooper (1980): Auditing and Accounting for Program Efficiency and Management Efficiency in Not-For-Profit Entities, Accounting, Organizations and Society 5,

no. I, p. 87-107.

Chames, A., and W. W. Cooper (1962): Programming With Linear Fractional Functionals, Naval

Research Logistics Quarterly 9, p. 181-186.

Chames, A., and W. W. Cooper (1984): The Non-Archimedean CCR Ratio for Efficiency Analysis: A Rejoinder to Boyd and Fiire, European Journal of Operational Research 15, no. 3, p. 333-334.

Chames, A., W. W. Cooper, D. Divine, T. W. Ruefli, and D. Thomas (1989): Comparisons of DEA and Existing Ratio and Regression Systems for Effecting Efficiency Evaluations of Regulated

Electric Cooperatives in Texas, in: Research in Governmental and Nonprofit Accounting, p. 187-210. editors James L. Chan, and James M. Patton. Greenwich, CT: JAI Press.

Chames, A., W. W. Cooper, Boaz Golany, Lawrence M. Seiford, and J. Stutz (October 1985):

Foundations of Data Envelopment Analysis for Pareto-Koopmans Efficient Empirical Production Functions, Journal of Econometrics 30, no. 1/2, p. 91-107.

Chames, A., W. W. Cooper, David B. Leamer, and Fred Y. Phillips (March 1985): Management Science and Marketing Management, Journal of Marketing 49, no. 3, p. 93-105.

Chames, A., W. W. Cooper, Arie Y. Lewin, Richard C. Morey, and John J. Rousseau (1985): Sensitivity and Stability Analysis in DEA, Annals of Operations Research 2, p. 139-156.

Chames, A., W. W. Cooper, Arie Y. Lewin, and Lawrence M. Seiford (1994): Data Envelopment

Analysis: Theory, Methodology, and Applications, Boston: Kluwer Academic Publishers.

Chames, A., W. W. Cooper, and S. Li (1989): Using DEA to Evaluate Relative Efficiencies in the Economic Performance of Chinese Cities, Socio-Economic Planning Sciences 23, no. 6, p. 325-344.

Chames, A., W. W. Cooper, and Edwardo L. Rhodes (1980): An Efficiency Opening for Managerial Accounting in Not-For-Profit Entities, in: Management Accounting 1980: Proceedings of the

University of Illinois Management Accounting Symposium, editor H. P. Holzer, 21-4 7. University of Illinois, Urbana, Illinois.

Chames, A., W. W. Cooper, and Edwardo L. Rhodes (June 1981): Evaluating Program and Managerial Efficiency: An Application of Data Envelopment Analysis to Program Follow Through, Management Science 27, no. 6, p. 668-697.

Chames, A., W. W. Cooper, and Edwardo L. Rhodes (1978): Measuring the Efficiency of Decision

Making Units, European Journal of Operational Research 2, no. 6, p. 429-444.


Charnes, A., W. W. Cooper, Lawrence M. Seiford, and J. Stutz (1982): A Multiplicative Model for Efficiency Analysis, Socio-Economic Planning Sciences 16, no. 5, p. 223-224.

Charnes, A., W. W. Cooper, Lawrence M. Seiford, and J. Stutz (1983): Invariant Multiplicative Efficiency and Piecewise Cobb-Douglas Envelopments, Operations Research Letters 2, no. 3, p. 101-103.

Charnes, A., W. W. Cooper, Quan Ling Wei, and Z. M. Huang (1989): Cone Ratio Data Envelopment Analysis and Multi-objective Programming, International Journal of Systems Science 20, no. 7, p. 1099-1118.

Charnes, A., W. W. Cooper, Quan Ling Wei, and Z. M. Huang (1990): Fundamental Theorems of Nondominated Solutions Assocaited with Cones in Normed Linear Spaces, Journal of Mathematical Analysis and Applications 150, no. I, p. 54-78.

Charnes, A., Stephen Haag, Patrick V. Jaska, and John Semple (1992): Sensitivity of efficiency classifications in the additive model of data envelopment analysis, International Journal of Systems Science 23, no. 5, p. 789-798.

Charnes, A., Z. M. Huang, John J. Rousseau, and Quan Ling Wei (1990): Cone Extremal Solutions of Multi-Payoff Games with Cross-Constrained Strategy Sets, Optimization 21, no. I, p. 51-69.

Chames, A., Z. M. Huang, John Semple, T. Song, and D. Thomas (1990): Origins and Research in Data Envelopment Analysis, The Arabian Journal for Science and Engineering 15, no. 4B, p. 617-625.

Chames, A., and L. Neralic (1989): Sensitivity Analysis in Data Envelopment Analysis-Part I,

Glasnik Matemativcki. Serija III 24 (44), no. I, p. 211-226.

Charnes, A., and L. Neralic ( 1989): Sensitivity Analysis in Data Envelopment Analysis-Part 2,

Glasnik Mathemativcki. Serija III 24 (44), no. 2/3, p. 449-463.

Charnes, A., and L. Neralic (1990): Sensitivity Analysis of the Additive Model in Data Envelopment

Analysis, European Journal of Operational Research 48, no. 3, p. 332-341.

Chilingerian, Jon A. (1995): Evaluating Physician Efficiency in Hospitals: A Multivariate Analysis of Best Practice, European Journal of Operational Research 80, no. 3, p. 548-574.

Chilingerian, Jon A. (1989): Investigating Non-Medical Factors Associated with the Technical Efficiency of Physicians in the Provision of Hospital Services: A Pilot Study, Annual Best Paper Proceedings of the Academy of Management, p. 85-89.

Chilingerian, Jon A., and David H. Sherman (1997): DEA and Physician Report Cards: Using Assurance Regions to Benchmark Clinical Best Practices in an HMO, Annals of Operations Research (73).

Chilingerian, Jon A., and H. David Sherman (1990): Managing Physician Efficiency and Effectiveness in Providing Hospital Services, Health Serv Manage Res 3, no. I, p. 3-15.


Clarke, Richard Lee (1988): Effects of Repeated Applications of Data Envelopment Analysis on Efficiency of Air Force Vehicle Maintenance Units in the Tactical Air Command and a Test for the Presence of Organizational Slack Using Rajiv Banker's Game Theory Formulations, Ph. D. dissertation, Graduate School of Business, University of Texas.

Collier, David, and James Storbeck (1993): Monitoring of Continuous Improvement Performance Using Data Envelopment Analysis, Proceedings of Decision Sciences Institute, p. 1925-1927.

Cook, Wade D., and Moshe Kress (1990): A Data Envelopment Model for Aggregating Preference Rankings, Management Science 36, no. II, p. 1302-1310.

Cook, Wade D., and Moshe Kress (1990): A m-th Generation Model for Weak Ranking of Players in a Tournament, Journal of the Operational Research Society 41, no. 12, p. 1111-1119.

Cook, Wade D., Moshe Kress, and Lawrence M. Seiford (1993): On the Use of Ordinal Data in Data Envelopment Analysis, Journal of the Operational Research Society 44, no. 2, p. 133-140.

Cook, Wade D., Yaakov Roll, and Alex Kazakov (1990): A DEA Model for Measuring the Relative Efficiency of Highway Maintenance Patrols, INFOR 28, no. 2, p. 113-124.

Cook, Wade D., Yaakov Roll, and Alex Kazakov (October 1988): Measurement of the Relative Efficiency of Highway Maintenance Patrols in Ontario, Ministry of Transportation, Toronto, Ontario, Canada.

Day, D. L., A. Y. Lewin, and H. Li (1995): Strategic Leaders or Strategic Groups: A Longitudinal Data Envelopment Analysis of the U. S. Brewing Industry, European Journal of Operational Research 80, no. 3, p. 619-638.

Day, D. L., Arie Y. Lewin, R. J. Salazar, and Hongyu Li (1994): Strategic Leaders in the U.S. Brewing Industry: A Longitudinal Analysis of Outliers, in: Data Envelopment Analysis: Theory, Methodology, and Applications, editors A. Charnes, W. W. Cooper, Arie Y. Lewin, and Lawrence M. Seiford. Boston: Kluwer Academic Publishers.

Debreu, G. (1951): The Coefficient of Resource Utilization, Econometrica 19, no. 3, p. 273-292.

Deprins, D., L. Simar, and H. Tulkens (1984): Measuring Labor-Efficiency in Post Offices, in: The Performance of Public Enterprises: Concepts and Measurement, p. 243-267. editors M. Marchand, P. Pestieau, and H. Tulkens. North-Holland: Elsevier Science Publishers B.V ..

Desai, Anand, and James E. Storbeck (1990): A Data Envelopment Analysis for Spatial Efficiency, Computers, Environment and Urban Systems 14, no. 2, p. 145-156.

Desai, Anand, James E. Storbeck, Kingsley E. Haynes, Homee F. E. Shroff, and Yan Xiao (1990): Extending Multiple Objective Programming for Siting Decision Sensitivity, Modeling and Simulation 22, p. 153-158.

Dyson, R. G., and E. Thanassoulis (1988): Reducing Weight Flexibility in Data Envelopment Analysis, Journal of the Operational Research Society 39, no. 6, p. 563-576.

Eechambadi, Narasimhan Varadarajan (1985): Efficiency Analysis of Market Response and the Marketing Mix: Extending Data Envelopment Analysis to a Competitive Environment, Ph. D. dissertation, Graduate School of Business, University of Texas.


Epstein, Michael K., and John C. Henderson (1989): Data Envelopment Analysis for Managerial Control and Diagnosis, Decision Sciences 20, no. I, p. 90-119.

Fare, Rolf, R. Grabowski, and S. Grosskopf (1985): Technical Efficiency of Philippine Agriculture, Applied Economics 17, p. 205-214.

Fare, Rolf, S. Grosskopf, B. Lindgren, and P. Roos (1994): Productivity Developments in Swedish Hospitals: A Malmquist Output Index Approach, in: Data Envelopment Analysis: Theory, Methodology, and Applications, editors A. Charnes, W. W. Cooper, Arie Y. Lewin, and Lawrence M. Seiford. Boston: Kluwer Academic Publishers.

Fare, Rolf, S. Grosskopf, and James Logan (1983): The Relative Efficiency of Illinois Electric Utilities, Resources and Energy 5, no. 4, p. 349-367.

Fare, Rolf, S. Grosskopf, and James Logan (1985): The Relative Performance of Publically-Owned and Privately-Owned Electric Utilities, Journal of Public Economics 26, p. 89-106.

Fare, Rolf, S. Grosskopf, and C. A. K. Lovell (1994): Production Frontiers, London: Cambridge University Press.

Fare, Rolf, S. Grosskopf, and C. A. Knox Lovell (1985): The Measurement of Efficiency of Production, Boston: Kluwer-NijhoffPublishing, Kluwer Academic Publishers.

Fare, Rolf, S. Grosskopf, and D. Njinkeu (1988): On Piecewise Reference Technologies, Management Science 34, no. 12, p. 1507-1511.

Farrell, M. J. (1957): The Measurement of Productive Efficiency, Journal of the Royal Statistical Society, Series A 120, no. Part 3, p. 253-290.

Farrell, M. J., and M. Fieldhouse (1962): Estimating Efficient Production Frontiers Under Increasing Returns to Scale, Journal of the Royal Statistical Society, Series A Part II, p. 252-267.

Frantz, Roger (May 1992): X-Efficiency and Allocative Efficiency: What Have We Learned? American Economic Review 82, no. 2, p. 434-438.

Fried, H., C. A. Knox Lovell, and S. Schmidt (editors), (1993): The Measurement of Productive Efficiency: Techniques and Applications, London: Oxford University Press.

Garrett, Allan Warren (1985): Constrained Facet Analysis and Related Linear Programming Models: Tools for the Evaluation of the Efficiency, Productivity and Effectiveness of School Classrooms, Ph. D. dissertation, College of Education, University of Texas.

Golany, Boaz (1988): A Note on Including Ordinal Relations Among Multipliers in Data Envelopment Analysis, Management Science 34, no. 8, p. 1029-1033.

Golany, Boaz, and Sten Thore (1997): Restricted Best Practice Selection in DEA: An Overview with a Case Study Evaluating the Socio-Economic Performance of Nations, Annals of Operations Research (73).

Gold, Franklin Harold (1982): Data Envelopment Analysis: An Application to a Savings and Loan Association, M.S. thesis, Alfred P. Sloan School of Management, M.I.T..


Golden, Peggy A. (1989): Measuring Organizational Slack and an Application of the Slack Construct to the Prediction of Merger And Acquisition, D. B. A. dissertation, University of Kentucky.

Haynes, Kingsley E., Samuel Ratick, and James Cummings-Saxton (1994): Toward a Pollution Abatement Monitoring Policy: Measurements, Model Mechanics, and Data Requirements, The Environmental Professional 16, p. 292-303.

Johnston, Holly Hanson (1990): Empirical Studies in Management Accounting: Three Essays on the U.S. Airline Industry, 1981-1985, Ph. D. dissertation, Carnegie-Mellon University.

Joseph, D. A., R. P. Cerveny, and Edwardo L. Rhodes (September 1983): Application Architecture and Technologic Efficiency : A Comparative Study of Computerized Bank Transaction Systems, Proceedings of the AIDS National Meeting.

Katims, Michael Allen (1985): Using Efficiency Analysis to Evaluate Program Effects of Educational Intervention, Ph. D. dissertation, College of Education, University of Texas.

Kleinsorge, Ilene K., Phillip B. Schary, and Ray D. Tanner (1989): Evaluating Logistics Decisions, International Journal of Physical Distribution and Materials Management 19, no. 12.

Koopmans, T. C. (1951): Analysis of Production as an Efficient Combination of Activities, in: Activity Analysis of Production and Allocation, editor T. C. Koopmans. New York: Wiley.

Korostelev, A. P., L. Simar, and A. B. Tsybakov (1995): On Estimation of Monotone and Convex Boundaries, Pub. Inst. Stat. Univ. Paris 34, no. I, p. 3-18.

Kusbiantoro (1985): A Study of Urban Mass Transit Performance: Concept, Measurement, and Explanation, Ph. D. dissertation, University of Pennsylvania.

Land, Kenneth C., C. A. Knox Lovell, and Sten Thore (1993): Chance-constrained Data Envelopment Analysis, Managerial and Decision Economics 14, no. 6, p. 541-554.

Leibenstein, Harvey, and Shlomo Maital (May 1992): Empirical Estimation and Partitioning of XInefficiency: A Data-Envelopment Approach, American Economic Review 82, no. 2, p. 428-434.

Lewin, Arie Y., Richard C. Morey, and T. J. Cook (1982): Evaluating the Administrative Efficiency of Courts, Omega 10, no. 4, p. 401-411.

Lovell, C. A. K. (1994): Linear Programming Approaches to the Measurement and Analysis of Productive Efficiency, TOP 2, no. 2, p. 175-248.

Lovell, C. A. Knox, Lawrence C. Walters, and L. L. Wood (1994): Stratified Models of Education Production Using Modified DEA and Regression Analysis, in: Data Envelopment Analysis: Theory, Methodology, and Applications, editors A. Charnes, W. W. Cooper, Arie Y. Lewin, and Lawrence M. Seiford. Boston: Kluwer Academic Publishers.

Majumdar, Sumit Kumar (1990): The Impact of a Competitive Environment on Corporate Performance in U.S. Telecommunications, Ph. D. dissertation, University of Minnesota.

Morey, Richard C., D. J. Fine, and S. W. Loree (1990): Comparing the Allocative Efficiencies of Hospitals, Omega 18, no. I, p. 71-83.


Norman, Michael, and Barry Stoker (1991): Data Envelopment Analysis: The Assessment of performance. Chichester, England: John Wiley.

Norton, Rob (October 1994): Economics for Managers: Which Offices or Stores Really Perform Best? A New Tool Tells, Fortune 38.

Nunamaker, Thomas R. (June 1983): Measuring Routine Nursing Service Efficiency: A Comparison of Cost Per Patient Day and Data Envelopment Analysis Models, Health Services Research 18, no. 2, part I, p. 183-205.

Olesen, 0., and N. C. Petersen (March 1995): Chance-Constrained Efficiency Evaluation, Management Science 41, no. 3, p. 442-457.

Pareto, V. (1927): Manuel d'economie politique, deuxieme edition. Paris: Marcerl Giard.

Petersen, Niels Christian ( 1990): Data Envelopment Analysis on a Relaxed Set of Assumptions, Management Science 36, no. 3, p. 305-314.

Reaves, Linda Jean (1983): Using Data Envelopment Analysis to Operationalize the Concept of Equal Education Opportunity, Ph. D. dissertation, College of Education, University of Texas.

Retzlaff-Roberts, Donna Lynn (1990): Incorporating Uncertainty Into Allocative Data Envelopment Analysis, Ph. D. dissertation, University of Cincinnati.

Rhodes, Edwardo L. (1986): An Exploratory Analysis of Variations in Performance Among U.S. National Parks, in: Measuring Efficiency: An Assessment of Data Envelopment Analysis, p. 47-71, Editor Richard H. Silkman. New Directions for Program Evaluation, No. 32, San Francisco, Jossey Bass, Inc.: American Evaluation Association,.

Rhodes, Edwardo Lao (1978): Data Envelopment Analysis and Approaches for Measuring the Efficiency of Decision-making Units with an Application to Program Follow-Through in U.S. Education, Ph. D. dissertation, School of Urban and Public Affairs, Carnegie-Mellon University.

Sant, R. (1989): Measuring the Efficiency of Pubs for Allied Breweries Ltd Using DEA, Unpublished M. Sc. thesis, University of Warwick.

Schinnar, A. P. (1980): Frameworks for Social Accounting and Monitoring of Invariance, Efficiency and Heterogeneity, in: Models for Alternative Development Strategies, Hague, The Netherlands: Institute of Social Studies.

Seiford, Lawrence M. ( 1990): Models, Extensions, and Applications of Data Envelopment Analysis: A Selected Reference Set, Computers, Environment and Urban Systems 14, no. 2.

Seiford, Lawrence M., and Robert M. Thrall (October 1990): Recent Developments in DEA: The Mathematical Programming Approach to Frontier Analysis, Journal of Econometrics 46, no. 1-2, p. 7-38.

Seitz, Wesley D. (1966): Efficiency Measures for Steam-Electric Generating Plants, Proceedings of the Thirty Ninth Annual Meeting of the Western Farm Economics Association, p. 143-151.


Sengupta, Jati K. (1982): Efficiency Measurement in Stochastic Input-Output Systems, International Journal of Systems Science 13, p. 273-287.

Sengupta, Jati K. (1989): Efficiency Analysis by Production Frontiers: The Nonparametric Approach, Kluwer Academic Publishers Group, Dordrecht.

Shash, Ali H. (1988): A Probabilistic Model for U.S. Nuclear Power Construction Times, Ph. D. dissertation, Department of Civil Engineering, University of Texas.

Shephard, R. W. (1970): Theory of Cost and Production Functions. Princeton, NJ: Princeton University Press.

Sherman, H. David, and Franklin Gold (June 1985): Bank Branch Operating Efficiency: Evaluation with Data Envelopment Analysis, Journal of Banking and Finance 9, no. 2, p. 297-315.

Sherman, H. David (October 1984): Hospital Efficiency Measurement and Evaluation: Empirical Test of a New Technique, Med Care 22, no. 10, p. 922-38.

Sherman, H. David (1981): Measurement of Hospital Technical Efficiency: A Comparative Evaluation of Data Envelopment Analysis and Other Efficiency Measurement Techniques for Measuring and Locating Inefficiency in Health Care Organizations, D. B. A. dissertation, Graduate School of Business Administration, Harvard University.

Siems, Thomas F. (1991): An Envelopment Analysis Approach to Measuring Management Quality and Predicting Bank Failure, Ph. D. dissertation, Southern Methodist University.

Silkman, Richard H. (Editor), ( 1986): Measuring Efficiency: An Assessment of Data Envelopment Analysis, New Directions for Program Evaluation, No. 32, San Francisco, Jossey Bass, Inc.: American Evaluation Association.

Simar, Leopold (1992): Estimating Efficiencies from Frontier Models with Panel Data: A Comparison of Parametric, Non-Parametric and Semi-Parametric Methods with Bootstrapping, The Journal of Productivity Analysis 3, no. 1/2, p. 171-203.

Sitorus, Bistok L. ( 1966): Productive Efficiency and Redundant Factors of Production in Traditional Agriculture of Underdeveloped Countries: A Note on Measurement, Proceedings of the Thirty Ninth Annual Meeting of the Western Farm Economics Association, p. 153-158.

Splitek, David Franklin (I 981 ): A Study of the Production Efficiency of Texas Public Elementary Schools, Ph. D. dissertation, College of Education, University of Texas.

Stolp, Chandler (1990): Strengths and Weaknesses of Data Envelopment Analysis. An Urban and Regional Perspective, Computers, Environment and Urban Systems 14, no. 2, p. 103-116.

Stone, Martha Jean (1984): A Comparative Analysis of the Personnel Practices of School Districts Selected by Data Envelopment Analysis Efficiency Indices, Ph. D. dissertation, College of Education, University of Texas.

Thomas, David Alan (1990): Data Envelopment Analysis Methods in the Management of Personnel Recruitment Under Competition in the Context of U.S. Army Recruiting, Ph. D. dissertation, Graduate School of Business, University of Texas.


Thomas, Dennis Lee (1985): Auditing the Efficiency of Regulated Companies Through the Use of Data Envelopment Analysis: An Application to Electric Cooperatives, Ph. D. dissertation, Graduate School of Business, University of Texas.

Thompson, Russell G., P. S. Dharmapala, and Robert M. Thrall (1994): Sensitivity Analysis of Efficiency Measures With Applications to Kansas Farming and Illinois Coal Mining, in: Data Envelopment Analysis: Theory, Methodology, and Applications, editors A. Charnes, W. W. Cooper, Arie Y. Lewin, and Lawrence M. Seiford. Boston: Kluwer Academic Publishers.

Thompson, Russell G., Larry N. Langemeier, Chih-Tah Lee, Euntaik Lee, and Robert M. Thrall (1990): The Role of Multiplier Bounds in Efficiency Analysis with Application to Kansas Farming, Journal of Econometrics 46, no. 1,2, p. 93-108.

Thompson, Russell G., F. D. , Jr. Singleton, Robert M. Thrall, and Barton A. Smith (November 1986): Comparative Site Evaluation for Locating a High-Energy Physics Lab in Texas, Interfaces 16, no. 6, p. 35-49.

Thorogood, Nellie Jean Carr (1983): The Application and Utilization of Data Envelopment Analysis for Decision Support in the Administration of Instructional Programs for an Urban Community College, Ph. D. dissertation, College of Education, University of Texas.

Tomkins, Cyril, and R. H. Green (June 1988): An Experiment in the Use of Data Envelopment Analysis for Evaluating the Efficiency of UK University Departments of Accounting, Financial Accountability and Management 4, no. 2, p. 147-164.

Tulkens, Henry (1993): On FDH Efficiency Analysis: Some Methodological Issues and Application to Retail Banking, Courts, and Urban Transit, The Journal of Productivity Analysis 4, no. 1/2, p. 183-210.

Varian, Hal R. (1990): Goodness-of-Fit in Optimizing Models, Journal of Econometrics 46, no. 1/2, p. 125-140.

Ward, Peter, James E. Storbeck, Stephen L. Mangum, and Patricia E. Byrnes (1997): An Analysis of Staffing Efficiency in U.S. Manufacturing: 1983 and 1989, Annals of Operations Research (73).

Wong, Y. -H B., and J. E. Beasley (September 1990): Restricting Weight Flexibility in Data Envelopment Analysis, Journal of the Operational Research Society 41, no. 9, p. 829-835.

f f

; t ~ }

C'CI ~ Q) .

> I ~ · t:: ~ Q) ll ~ ~j .. Jl. < I W 1 c h

Treating non-discretionary variables one way or the other:

implications for efficiency scores and their interpretation

Matthias Staat'

Abstract

This paper explains the main DEA-techniques to model continuous and categorical non-dis

cretionary variables as well as a related two-stage approach. The implications of using either

alternative are demonstrated in practice using the pharmacy data from the original study by

Banker and Morey (1986b) on categorical non-discretionary variables.

It is argued that the model appropriate for continuous non-discretionary variables rests on

rather restrictive assumptions about the production technology. The model for categorical

non-discretionary variables does not result in higher efficiency scores, i. e. a more robust

assessment of the inefficiency of production units, as Banker and Morey claim. In addition, its

efficiency scores can not be compared across observations with different values for the cate

gorical variable as each category is evaluated by a differently sized data set. The bias

resulting from this practice is discussed in Zhang and Bartels (1998).

The two-stage approach to modelling non-discretionary variables does not rest on more

restrictive theoretical assumptions than the model for categorical non-discretionary

variables. It does, however, use the full data set for the evaluation of each unit and is

therefore recommended for the empirical analysis when non-discretionary variables are a

relevant factor.

1 Mannheim University, Lehrstuhl fur VWL, ins b. Mikrookonomie, D-68131 Mannheim, Germany

[email protected]

24 Treating non-discretionary variables one way or the other ...

Structure

Data Envelopment Analysis, non-discretionary variables, efficiency ranking

2 Introduction

3 Model formulation

4 Comparison of results

5 Conclusion

References

Treating non-discretionary variables one way or the other ...

1 Data Envelopment Analysis, non-discretionary variables,

efficiency ranking.

25

Despite potentially important practical implications for the efficiency evaluation of

decision making units (DMUs) non-discretionary variables -variables that that have a

productivity relevant influence on the production process but can not be controlled by

the individual DMU- have remained a field of primarily theoretical interest. Ignoring

non-discretionary variables in practical applications either by leaving them out of the

data or by treating them as controllable variables may lead to comparisons of

qualitatively different DMUs. The efficiency scores obtained will then have little

meaning.

The reason why techniques handling non-discretionary variables are rarely applied

may be the confusion over how to specify models for non-discretionary variables and

doubts about what is implied by the various specifications. For instance, Banker and

Morey (1986a) introduced a model for continuous non-discretionary variables which

leads to lower efficiency scores compared to the standard model. Ruggiero (1996)

shows that their approach is not fully consistent with production theory and that the

efficiency scores generated by it may in fact be too low. Banker and Morey (1986b)

demonstrate how to model the effects of categorical non-discretionary variables. They

suggest that this model restricts the set of peers more stringently than the approach for

the continuous case and therefore should tend to result in higher efficiency scores. An

alternative two-stage approach DEA-regression procedure can be used to analyse the

effects of non discretionary variables.

This study tries to clarify these points showing the implications of several

specifications on the original Banker and Morey (1986b) data. Exceptions to the

assertion that the categorical approach leads to higher efficiency scores than the

continuous approach are demonstrated. Also, if categories indicating a small value of a

non-discretionary input contain only a few observations there may in fact be too few

peers to generate meaningful efficiency scores. In addition, each of the categories is

evaluated by a different number of potential peers. This may distort the results in

general and allows only a very limited interpretation of the efficiency scores. Several

alternative ways of treating non-discretionary variables are contrasted using the

Banker and Morey ( 1986b) data.


2 Introduction

Non-discretionary variables are productivity relevant inputs or outputs which are not

(fully) under the control of the decision making unit (DMU) in charge of the

production process. The term fixed or contextual variables is used as well. In addition,

there may be factors which are neither inputs nor outputs to the production process but

nevertheless influence performance. Lovell (1994) labels them environmental factors.'

If a non-discretionary variable is the cause of a DMU being rated inefficient the DMU

would be unable to improve its efficiency to the maximum value of 1 because it can

not or only partially influence the level of that variable. Since the level of such a fixed

variable can not be influenced by the DMU its efficiency score should not depend on

it.

Using DEA, a DMU is evaluated via comparison with a production frontier made up

of actual observations. Therefore it is critical that all observations being part of the

frontier are comparable to DMU0. Regression methods can -theoretically- fit more

than one hypothetical production frontier on a sample of heterogeneous DMUs,

allowing e. g. for different slope parameters for different subsamples. With DEA the

same effect has to be achieved although it is only possible to identify one production

frontier per sample.

There are two ways out of this problem: One is to come up with additional criteria for

the comparability of DMUs in a given sample. These result in restrictions for the

reference technologies. The other way is to alter the sample according to ones beliefs

about which DMUs are comparable with DMU0. Only the first strategy is associated

with non-discretionary variables by Lovell (1994, section 8.2); the second with what

he calls environmental factors (section 8.3).

Non-discretionary variables are a potentially important factor in almost any DEA

analysis. However, the topic is hardly ever touched by practitioners. The usual set of

parameters for the evaluation of, for example, hospital productivity -a standard

application of DEA models- includes type of ownership, size of the hospital and case

mix. These factors are productivity relevant in different ways. For instance, it is often

found that non-profit hospitals are run less efficiently than for-profit hospitals. The

type of ownership can usually not be influenced by the hospital management but on

2 Other authors use the non-discretionary and environmental as synonyms.

Treating non-discretionary variables one way or the other ... 27

the other hand it has no direct influence on the functions performed in the hospital. No

special treatment for this variable is necessary. 3

The variable case mix has a different character: if some hospitals treat a mix of cases

which is more demanding in terms of therapy than others this should be reflected by

the fact that none of the hospitals with standard case mixes should be part of the set of

peers of the disadvantaged hospitals. Put differently, if an analysis disregarding case

mixes reveals that hospitals with difficult case mixes are less efficient this can hardly

be interpreted as managerial slack.

Finally, the size of a hospital can not be changed in the short run by hospital managers.

Assuming full capacity use, one could consider larger hospitals to have an advantage

over smaller ones since they can usually afford higher investment in equipment that

allows for more efficient treatment.• This should be a reason to consider non

discretionary techniques.

The classical examples for the application of techniques treating non-discretionary

variables from the Banker and Morey (1986a, 1986b) papers are a restaurant chain

where advertising budgets are set by the central marketing division rather than

individual restaurant managers and pharmacies located in communities of different

sizes. The size of the community and the size of its marketing budget cannot be

controlled by the DMUs' managers; therefore these variables are labelled non-discre

tionary. Not all restaurants of the chain can be compared to each other as, for instance,

the restaurants with a temporary promotion may have an easier time selling certain

products than restaurants which sell them for the regular price. Pharmacies in smaller

communities may have a more limited sales potential than pharmacies in larger

communities. Therefore, comparisons between pharmacies from communities of

different sizes may not make sense.

It is, however, desirable to include as many as possible DMUs in the analysis. For

instance, all the available information on restaurant performance should be used to

evaluate the performance of a long established restaurant. Its performance can be

3 This is only true if public hospitals do not have functions in the health sector different from private hospitals.

4 Of course, one could also imagine situations in which smaller hospitals have an advantage over larger ones. Lovell ( 1994, p. 213) describes procedures which can be applied the direction of the effect is not known ex ante.


compared to all restaurants which have been established around the time it went into

business or later.

It is important to recognise non-discretionary variables as such and to treat them

accordingly because the credibility of DEA results depends critically on whether the

set of potential peers comprises only DMUs which can be compared to the DMU being

evaluated. Comparing DMUs operating in different environments without controlling

for environmental factors will lead to differences in the environment being wrongly in

terpreted as differences in efficiency.

The following section will present the basic5 approaches to handle different types of

non-discretionary variables. These basic principles apply to all direct extensions and

refinements as well as to a number of related approaches. Some examples illustrate

how the models discussed work in practice and what is implied by that. A pragmatic

aside asserts in a non-technical way which model may be best used in a given

situation. The implications of handling the problem in different ways are demonstrated

on the original Banker and Morey ( 1986b) data in the next part of the paper. A final

section will take stock of the results.

3 Model formulation

3.1 Standard Model

Banker and Morey ( 1986a) begin their discussion of possible alternatives to the

standard model by noting that one element of the input vector xij, i = 1,2, may not be

under the control of DMU0 to be evaluated by comparison with the DMUs

j=l, ... ,N.•

The standard model is displayed here in its input-oriented, variable-returns-to-scale

(VRS) form as formula (1). The value of the efficiency parameter /%, 0< 80 $1,

describes what fraction of the actual inputs of DMU0 would suffice to produce its

5 Refinements and variants of these basic models will only be mentioned in passing as the purpose of this paper is to structure the problem of treating non-discretionary variables instead of the enumeration all of its aspects.

6 To make the exposition as transparent as possible the two-input-one-output example from Banker and Morey (1986a) is used. The generalisation is a matter of notation.


output were it to employ an efficient technology. There may be additional positive

slacks for some of the inputs and outputs, s1- ,s•, respectively. The efficient reference

technology is formed by a A.-weighted average of input-output combinations of other

efficient DMUs.

min z0 = B0 - es•- es; s,;.,,+ ,,-

N

s. t. LY;A.1 -s• = Y, j

N

:~::XuA. 1 - s; = B0x10

I N

LAJ =I j

o •. A. j,s+,s,- ~ 0

3.2 Continuous non-discretionary variables

(1)

Banker and Morey (1986a) now point out that if any of the inputs in the above formu

lation can not be controlled by the DMUs then DMU0 has no possibility of reducing

that input to a fraction ~ of its original level. They therefore suggest the following

reformulation of model (1), where the superscript ''f' in B& indicates that the case of

continuous non-discretionary or fixed variables is treated. 7

The input vector again consists of two inputs, one discretionary (d), the other one fixed

(f). Only the discretionary input directly enters the determination of the maximum

input reduction possible for inefficient units. The parameter ~ has vanished from the

constraint for Xfi so has the slack of that variable from the objective function.

The role now played by the non-discretionary input is reduced to ensuring that the

reference technology has on average no advantage over DMU0 with respect to the

variable x1. This is how it assures comparability of DMU0 and its reference technology;

it corresponds to the first strategy of solving the problem that was mentioned in the

introduction. It is tantamount to relaxing the original set of constraints as the

A.-weighted average of the fixed inputs now only has to meet the condition "5. x 10 "

7 The superscript ''!' will be used to indicate a continuous fixed variable whereas the superscript "c" will be reserved for the categorical variables discussed in section 3.3.


instead of the stronger condition "$ B0x10". Banker and Morey (1986a, p. 515) point to

the fact that softening the original condition "enriches the comparison set". As a con

sequence, Banker and Morey's proposition 1 states that 80 :2: B[. 8

N

s. t. LYy11 -s+ =Yo J

N

:~.::X.q-1 1 - sd- = B~ X do j

N

LXDAJ -sf- =xfo

(2)

The fact that the reference technology on average has to have the same value for the

non-discretionary indicator as DMU0 implies that, for example, a restaurant with one

week of promotions can be sensibly evaluated by a reference technology made up of

two restaurants each with A. = 0.5 where one has no, the other two weeks of

promotions in a given period. This point will be taken up again in section 4.

Lovell (1994, section 8.3) points out that one could also ensure comparability of the

DMUs by excluding all DMUs for which x0 > x10 holds from the reference technology

of DMU0 as higher levels of the non-discretionary input indicate an advantageous

position. The character of the restriction changes: Conditions on the reference technol

ogy are replaced by eligibility conditions on individual DMUs for inclusion in the

sample (and therefore in the reference technology as well). This leads to

8 An extension of this specification to the treatment of simultaneous continuous non-discretionary inputs and outputs is mentioned in section 2.4.

Treating non-discretionary variables one way or the other ...

min Z0 = 9~ -ES+ -esd-e,.>..,.r* .sd-

N

s. t. L>?·1 -s• = Y, J

N

~>djA j - Sd- = 8~Xdo 1

31

(3)

and is identical with the model Ruggiero (1996) proposes. Also, this corresponds to

the second strategy mentioned in the introduction: restricting the sample to DMUs

comparable with DMU0.

Ruggiero ( 1996) concludes that e,f may be somewhat too low due to the inclusion of

DMUs in the reference technology which do not belong there because they operate in

an advantageous environment compared to DMU0 and therefore e~ > e~ (see

proposition 1, p. 559). Whether ~ ("/" indicating the Lovell/Ruggiero specification)

will be lower or higher than 80 can not be determined a priori as the original

restriction is replaced by a different type of restriction.

As pointed out by Lovell (1994, section 8.3), one consequence of this type of

restriction is that unlike in the Banker and Morey (1986a) method discussed so far, the

sample size changes from evaluation of one DMU to the next. For DMUs with small

values of x1 the number of potential peers may be only a fraction of the number

available for the evaluation of DMUs with large values for the fixed parameter. This

point will also be discussed more in-depth in section 4.

Table 1 gives some examples for a DMU0 with both the discretionary and the fixed

input and the output equal to 1 being evaluated by different peer units. The first data

column in Table 1 contains values for DMU0. The following columns list data for

efficient peers. To keep the examples simple, a reference technology is made up of just

as many efficient peers as necessary to demonstrate a certain effect. The first row of

data contains the values for the output, followed by the discretionary and the fixed


input in row 2 and 3, respectively. Row 4 contains the weight for the efficient peer and

rows 5 to 7 contain the efficiency scores calculated by the different models.

Table 1: Efficiency scores for various methods w. r. t non-discretionary variables

DMU0 Peers (a) (b) (c) y 1 1 118 4.5 1

Xd 1 0.5 118 2 0.5

Xf 0.5 118 2 1

A. 1 0.8 0.2

Bo 0.5 .5 1 ()f 0.5 .5 0.5

0

B' 0 0.5 1 (?) 0.5

Example (a) demonstrates how all different models lead to the same efficiency score.

Suppose 00 is .5 since there existed a DMU with the same output as DMU0 using only

half of all inputs to produce that output. Assuming there are no other dominant

(combinations of) peers e~ would also be .5 as xd of the peer unit is half of xod and the

fixed parameter naturally meets the softer condition of the non-discretionary approach

(see section 3.2). In this case, €fo would also be .5 since the only member of the

reference technology has no advantage over DMU0 with respect to Xp

The case that is relevant for the Levell/Ruggiero specification is that of a "large" peer

DMU with 4.5 times the output and twice the input ofDMU0 and another "small" peer

with values that are just 118 of DMU0 's. This situation is considered in example (b).

According to Levell/Ruggiero DMU0 should not be evaluated by a reference

technology containing the large DMU. Therefore, DMU0 is rated as efficient as long as

no other efficient peer (combination) dominates it. The other two approaches would

permit the large DMU in the reference technology and thus give each an efficiency

score of0.5.

Finally, there are cases when B~ = B~ '* 00 • This happens when the efficiency of DMU0

critically depends on x1 when evaluated by means of the standard model as in example

(c). Here, only x1 is used in the same way by the technologies of DMU0 and the

technology of the efficient peer but the latter is more efficient in using xd. Due to

x,d = x0d, 00 =I holds. Softening the constraint for x1 the performance with respect to


xd now becomes critical and B~ drops from 1 to 0.5. Since there is no advantage of the

peer over DMU0 with respect to the fixed parameter B'r, is also .5; i.e. lower than the

original value ~ = 1. There can be no general a priori ranking for the values of 90 and

B~.

3.3 Categorical non-discretionary variables

Banker and Morey (1986b) consider the case when non-discretionary variables are not

continuous. Only the category into which the value for the non-discretionary variable

falls is known." The example they use to illustrate their model are pharmacies located

in communities of different sizes. Pharmacies in smaller communities naturally have a

lower sales potential and should therefore not be evaluated through peers located in

communities. Banker and Morey ( 1986b) classify the communities into eleven

categories which were also used by the U.S. Census of Population and Housing,l980. 10

This classification is supposed to reflect differences in market size defined by

population figures.

Banker and Morey ( 1986b) generate dummy variables which are set to 1 if a com

munity is of equal or smaller size than indicated by the dummy. If a community

belongs to, say, population size category 3, the dummies for population sizes 1 to 3 are

set to one and the dummies for category 4 and above are set to 0. Banker and Morey

(1986b) then treat the dummies like the non-discretionary variables in specification (2)

above. It is obvious that the reference technology for certain pharmacies can only

consist of pharmacies located in communities of the same or of smaller size. This is so

because x01, x = 1 , . . . , 11, assumes the value 0 for all classes indicating larger

communities (markets) than the one of DMU0 and any pharmacy from a larger

community will violate the restrictions for the reference technology with respect to x1

The same could be accomplished by using a variable with values I to 11 and applying

the Lovell/Ruggiero specification (3) above. In fact, Ruggiero notes that his model for

continuous non-discretionary variables can be interpreted as a variant of Banker and

Morey's (1986b) approach for categorical non-discretionary variables. However, it

would be more in the spirit of the method developed by Ruggiero ( 1996) to base the

decision about whether to accept a DMU in the set of peers on actual instead of

9 The same study contains a model for discretionary categorical variables.

10 The upper limits of the II classes are 199,499,999, 1499, 1999,9999, 19999,24999,49999,99999,249999.


categorised population sizes. Again, this point will be discussed in more detail in

section 4.

Banker and Morey (1986b, p. 1614) motivate their approach by the following

example:

However, suppose we are attempting to estimate the resources (such as

labour and capital) that a branch of a bank needs to obtain a given level of

deposits, given a population base of say 100.000, with a specific income,

age, and other demographic characteristics. Then in DEA the branch in

question might well be compared to a composite branch built from a branch

with a population of 80,000 and another with a population of 120,000, both weighted equally. While this may seem like a very reasonable

approximation, it is clear that the branches employed for this comparison

would be less controversial if we were to insure that the peer group

consisted only of branches with a population of 100,000 or less.

What is desired for the above situations is a method for insuring that the composite reference members be constructed from members which are in

the same or possibly from those in a category which is deemed to be operating in an even more difficult or unfavourable situation.

The first paragraph clearly calls for the approach suggested by Ruggiero ( 1996). In the

second paragraph, the conditions for being a member in the reference group are

somewhat softened because conditioning on "the same ... situation" can lead to

comparisons of rather different DMUs. For instance both the community of I 00.000

and the community of 120.000 in Banker and Morey's example belong to the same

market size category (11) which ranges from 100.000 to 249.999 (see footnote 10).

Thus, the categorical approach in this case makes exactly the kind of comparisons

possible it was designed to prevent.

Maybe the above comparison is allowed for good reason and the classification puts

communities that constitute markets of the same size into the same categories. But

maybe this categorisation was - as categorisations often are - designed such that there

are sufficient members in each cell. This consideration - legitimate as it may be - is

Treatmg non-discretionary variables one way or the other ... 35

certainly not related to market size and the categories above could be irrelevant or

misleading in the given context."

Also, using this classification of DMUs, reference groups now become possible that

would not be possible applying any other approach. For instance, a DMU from a community of 100000 could now have a reference technology consisting exclusively

of peers from communities with a population between 200000 and 220000.

Of course, one could "solve" the problem by choosing only peers which operate under

more difficult circumstances for comparison. This would leave some inefficiencies

undetected that can only by demonstrated by comparison with DMUs which operate

under like circumstances. Efficiency scores generated in this way could be considered

upper bounds to the "true" scores.

4 Comparison of results

4.1 General aspects

At this point it seems fitting to collect some of the basic results and discuss some implications of using one of the two (three) methods designed to handle non

discretionary variables.

A slight modification of the standard model is necessary to handle continuous non-dis

cretionary variables according to Banker and Morey (1986a, see formula (2)). This

leads to lower efficiency scores since it results in "enriching the comparison set". It

was also mentioned that this implies that the reference technology and DMU0 be on

average the same with respect to the non-discretionary variable. The model is

11 Often, no ready made categorisation will be available and one has to make up a classification ad

hoc. Consider international comparisons which are listed among the most important novel applications

in Seiford ( 1996). Suppose country size matters: There is no obvious answer to the question into how

many categories a sample of countries should be divided by size. Even if there is an answer, say three

categories, what is a small, a medium size and a large country w. r. t. the matter of interest may not

always be simple to decide. This reintroduces a certain arbitrariness into the analysis.


applicable for a technology like the one discussed in Ray (1988, p. 171, equation (11))

for which multiplicative separability in the discretionary and the fixed factors holds. 12

The modifications to the standard model called for by the models described in Banker

and Morey ( 1986b) as well as Ruggiero ( 1996) are more significant than the ones just

described (see formula (3) above). However, these approaches are easy to implement.

The model can be applied by selecting all admissible peer-DMUs for each DMU and

then using the standard model without any further modifications.

In order to apply the Lovell/Ruggiero model one would have to be convinced that the

non-discretionary parameter affects productivity in a direct way. For instance, in

Banker and Morey's (1986b) study (see also section 4.2.1 of this paper) population size

was used as a surrogate for sales potential. They mentioned, however, that sales

potential depends also on unknown demographic characteristics of the population such

as age and income distribution, education etc. (see the quote on p. 34). In a case like

this, it would not make much sense to apply this model using actual population size

since all that is known is that communities of the same size have about the same sales

potential. 13

What is called for in this case is the approach suggested by Banker and Morey (1986b)

with broad categories of population sizes. This model, too, is not without peculiar

aspects. Most notably, it would allow for reference technologies that consist of peers

which all have a higher sales potential than DMU0• 14 Of course, any empirical method

would suffer from imprecise information. But unlike regression which transforms

imprecise information - interpreting it as a measurement error - into imprecise results,

DEA will translate it into biased results about the efficiency ofDMUs.

The following table collects some characteristics of the models discussed.

12 Ray {1988) points out that his formulation amounts to modelling the non-discretionary factor like a parameter for Hicks-neutral technical progress.

13 The model would be more appropriate for a technical parameter of a production schedule like investment where higher investment means better facilities and in tum better capabilities of production.

14 In the pharmacy data set, actual population sizes are known such that an additional restriction preventing that could be implemented. In cases when only the categories are known, however, this will not be possible.


Table 2: Models, characteristics, implications

Model Relative Apply if I problems Ease of use

(formula) efficiency

BM 1986a f)~ ()f DMU0 and reference technology on average the same Code modification

(2) w.r.t. indicator. of standard model necessary

BM 1986b Claim: Only categorical indicator available. Standard model with

(3) ()'"' ~ ()f Continuous indicator with loose connection to varying data sets

but see productivity relevant factor.

section May lead to reference technologies with peers that are

4.2. all advantaged w.r.t. Xf

Ruggiero ()/ ~ ()f Continuous indicator. Standard model with

(3) Close correspondence between indicator and en-varying data sets

vironment!technology.

4.2 Banker and Morey (1986b) data revisited: Part I

4.2.1 Data

The above models are now applied to the original data set used in Banker and Morey

(1986b, Appendix A, p. 1624 ff. ). The purpose of their study was to demonstrate the

difference between modelling a non-discretionary variable as a continuous indicator

vs. modelling it as a discrete indicator, i.e. specification (2) vs. a variant of

specification (3) above. Their results will be reproduced below. 15 In addition, the

results for the standard specification (1) will be reported. They will serve as a baseline

for specification (2). According to Banker and Morey (1986a), specification (2) will

yield lower efficiency scores than specification ( 1) whereas specification (3) will yield

a higher score than specification (2).

The data consist of 69 pharmacies located in communities of different sizes ranging

from only 500 to more than 200.000 inhabitants. The population count is the so-called

non-discretionary parameter as it can not be controlled by the pharmacy owner once

the pharmacy has been established. It is modelled as an input since Banker and Morey

15 The results are slightly sensitive to the scaling of the data. As Banker and Morey do not indicate how they scaled the data their results are not to the third digit past the dot the same as the ones reported here. However, this does not produce spurious effects for the comparisons.


(1986b) assume that the larger the community, the easier it is to achieve an efficient

input/output combination.

The other three (discretionary) inputs are labour as well as other operating costs and

the average value of the inventory.

The two outputs are number of prescriptions and their value.

4.2.2 Results

Table 3 lists the results for selected DMUs. The data columns contain the efficiency

score and in order to economise on space only the population classes of the peers of

the inefficient DMUs are listed. The different rows refer to the different methods of

handling non-discretionary variables just discussed.

Pharmacies #15 and #52 were chosen by Banker and Morey in their study to dem

onstrate the effect of treating population size as a categorical non-discretionary

variable vs. treating it as a continuous non-discretionary indicator. Pharmacy #15 is

located in a community which has a population of 2500 and therefore belongs into

population category 6.

When treating population size as a continuous variable one of #15 peers belongs to

category 5, i.e. it serves an even smaller population whereas the second peer belongs

to category nine with a population of over 30000. When population is treated as a

categorical indicator the set of peers does contain again two observations, the one from

category 5 also contained in the set of efficient DMUs when population is treated as a

continuous indicator and another peer from population category 6. The efficiency

score of #15 rises slightly from .56 to .57 using the categorical model. This is in line

with the authors' expectations.

The peer-DMU from population size category 6 that was contained in the second of

the above reference technologies is located in a community with a larger population

(over 4000) than the community of pharmacy #15. Therefore, the Lovell/Ruggiero

approach which - when applied rigorously - only allows peers that have the same or

smaller population again produces a new set of peers from categories 4 and 5 and a

still higher efficiency score of .58.


Table 3: Banker and Morey (1986b) results compared

Pharmacy (Category) #15 (6) #52 (8) #26 (6)

Model specifications

Peer categories 4, 5, 6 4,6,9,9

non-discretionary, continuous Efficiency score 0.56 0.65

Peer category 5,9 4, 6, 9, 9, II

non-discretionary, categorical (II} Efficiency score 0.57 0.86 0.95

Peer category 5,6 5,5,6 6,6,6,6

non-discretionary, Ruggiero Efficiency score 0.58 0.86

Peer category 4,5 5, 5, 6

non-discretionary, categorical (2) Efficiency score 0.57 0.65 0.77

Peer category 5,6 3, 9, II 4,6, 6, 7

Standard model (baseline) Efficiency score 0.62 0.66 I

Peer category 3, 5, 6 4, 6, 9, 9, II

The highest efficiency score is generated by the baseline specification treating

population size as a continuous discretionary variable. This is in line with proposition

1 in Banker and Morey (1986a). While the restrictions for the non-discretionary cases

allow all reference technologies which have a (weighted) population - measured in

actual numbers or in categories - of less than the community of pharmacy #15 the

baseline specification only allows reference technologies with population less than or

equal to 80*(population of#15). The latter turns out to be much more restrictive in this

case (see section 3.2).

While one can clearly identify the effect of using different specifications the actual

efficiency scores are very similar when looking at the results for #15. Banker and

Morey (1986b) also give an example for a major change of the efficiency score due to

a change in specification. Pharmacy #52 serves a population of 23.166 and therefore

belongs to category 8. The reference set for the continuous non-discretionary approach

consists of five pharmacies, two from smaller and three from larger communities than

its own. Except for the pharmacy in the community of category 6 none of the original

set of peers remains in the reference technology when the categorical approach is used.

Thus, the efficiency score rises from 0.65 to 0.86. Disregarding the non-discretionary

character of the variable population size altogether leads to virtually the same results

as treating it as a continuous non-discretionary variable (see baseline).


For these two pharmacies the propositions in the papers cited hold. The continuous

non-discretionary approach leads to lower efficiency scores than the discretionary

approach, the Lovell/Ruggiero specification leads to higher scores than the continuous

approach as does the categorical non-discretionary specification. Note, however, that

for the latter to happen in all cases, the categorical approach would have to result in a

more stringent restriction than the continuous one. As discussed in section 3.3 of the

paper, this is not the case. With the categorical approach it would theoretically be

possible for #15 (or #52) to have a set of peers consisting of pharmacies from

communities larger than its own (but not larger than 99999). This would not be

possible under the continuos non-discretionary approach.

A look at the results for #26 confirms that this does indeed happen when using the

categorical specification. This pharmacy is located in a community of 2217. This

community is slightly smaller than the one #15 is located in but belongs to the same

category, 6. The continuous non-discretionary specification rates #26 as efficient. The

categorical case gives an efficiency score of only 0.95 for #26. All other specifications

again lead to an efficiency score of I. The peers forming the reference technology in

the categorical specification belong to the same category as #26 does but all are lo

cated in larger communities. The actual populations range from 2718 to 5607. There

fore it is not necessarily true that, as Banker and Morey (1986b, p. 1619) claim: "the

number of DMUs identified as technically inefficient will not increase under the

categorical treatment, ... ".

This can be demonstrated by changing the number of categories used. One could

naively assume that it only matters whether a pharmacy is located in a small (less than

25000 inhabitants) or a large community (25000 and over). The results for this

specification are also reported in Table 3. Banker and Morey (1986b) report 41

inefficient DMUs for the continuous and 36 inefficient for categorical (11 categories)

case. However, reducing the number of categories to 2 results in 42 inefficient DMUs

which shows that this does not hold. For #26 the efficiency score drops to 0.77 using

the variant with two categories.

Finally, the fact that for the latter two methods the sample size changes from

optimisation to optimisation may raise concern about whether there are sufficient


observations in each of the categories'• and whether the efficiency rankings can be

compared across categories. Since efficiency measures vary with the ratio of

parameters" to observations only the B.; for the DMUs of one and the same category

can be compared. In a model of type (3) with non-discretionary inputs there will be

more potential peers for DMUs with large values of that input and less for DMUs with

small values.

Before some further empirical illustrations of the models are presented some

extensions of the DEA model to incorporate non-discretionary variables are briefly

mentioned.

4.3 Further Extensions

4.3.1 Simultaneous non-discretionary inputs and outputs

The techniques described up to now were developed to handle either non-discretionary

inputs or non-discretionary outputs. Golany and Roll (1993) develop a model that

handles non-discretionary inputs and outputs simultaneously. They note that the same

linearised model can be derived from different ratio forms. By choosing a convenient

ratio form (see Golany and Roll, 1993, p. 423f) they are able to extend the basic model

to simultaneous non-discretionary inputs and outputs.

Formula (4) describes a problem with two inputs and two outputs, one discretionary

and one fixed each. This model also differs from the above specifications in that

constant returns to scale are assumed. For a variable-returns-to-scale specification

L 1 A 1 =I must hold and in an input-oriented model there would be no difference in the

treatment of discretionary and fixed outputs. This results in:

16 This concern is expressed in Lovell (1994, section 8.3). It is obvious that the DMU with the smallest value for any non-discretionary input can not have a reference technology matching that value in a VRS specification.

17 See Zhang and Bartels ( 1998) for Monte Carlo evidence on this point .


N

s. t. LYJ1A.1 -sd • = ~0 j

N N

LYpA; -s/ = LA;lfo

N

LXdJAJ- sd- =~X do j

N N

LXpA j- s,- = LAJXfO

(4)

Golany and Roll (1993) also contains an extension of the basic model to partially

controllable variables. The method can be applied by using model (2) above after the

variables have been transformed accordingly.

4.3.2 DEA-regression two-stage procedures

Ray (1988, 1991)" treats non-discretionary variables combining DEA and regression.

He performs standard DEA on the discretionary subset of the variables in the first

stage and regresses the efficiency scores derived, h, on the non-discretionary factors in

the second stage.

In Ray (1991), the second stage regression is used to predict maximum efficiency, h*, given the set of non-discretionary variables. This adjusted maximum efficiency may be

well below 1. The difference h *-h is then interpreted as managerial inefficiency. The

maximum efficiency measure given x1 is derived by adding the highest positive value

of the error term to the intercept in the regression. The residuals derived using this

adjusted intercept will all be non-positive. Thus, the predicted value for the efficiency

measure will at least be equal to the observed value derived in the first stage DEA and

18 Chames et al. (1981) mention some earlier studies using two-stage procedures.


consequently the inefficiency measure is lower than the first stage results would

suggest. 19

While Ray (1988) justifies his empirical procedure on theoretical grounds Fried et al.

(1993) propose a similar two-stage procedure for reasons of empirical practicability.

They analyse a sample of American credit unions. The two stages of their approach

comprise a free-disposal hull (FDH) model on the first and a logit regression on the

second stage. The logit model uses a binary indicator on whether a DMU was found to

be inefficient or efficient in the first stage analysis as the dependent variable. The first

stage relies solely on discretionary indicators while the second stage tests whether the

first stage findings can be explained by the non-discretionary indicators disregarded in

the first stage analysis. Fried et al. (1993) also apply a SURE system estimator to the

slacks of each variable used in the first round as the dependent variable and regress

them on the set of non-discretionary indicators. While the logit (single equation)

approach results in some plausible parameters explaining the variation of efficiency

through differences in the environment, the SURE approach yields no clear cut results.

As pointed out by Lovell (1994, section 8.3) both methods have one advantage over

the Lovell/Ruggiero (1996) and Banker and Morey (1986b) type of models: there is no

reductions in sample size for particular DMUs. Both models are also reminiscent of

partial regression (see Greene, 1997, Ch. 6) in that the variation not explained in a first

round analysis is analysed in a second stage.20

To further analyse what factors drive the results originally obtained by Banker and

Morey (1986b) a two stage procedure will be applied in the next section.

19 It should be noted that the proper estimation procedure in case of a censored dependent variable -the efficiency score is censored at 1- would be a !obit specification. Therefore, Ray's estimates give biased results (see Greene, 1997, Ch. 20).

20 Using partial regression, however, the effects of the non-discretionary factors would be "netted out" during the first stage of the procedure and the "pure" efficiency effects would be analysed during the second stage. This may be an alternative to the two approaches described.


4.4 Banker and Morey (1986b) revisited: Part II

4.4.1 The effects of changing sample size

The fact that for the model relating to categorical variables the sample size changes

from optimisation to optimisation may for one raise concern about whether there are

sufficient observations in each of the categories" and also whether the efficiency

rankings can be compared across categories. Since Zhang and Bartels ( 1998)

demonstrate that efficiency measures vary with the ratio of parameters to observations

-8 rises c. p. if more parameters are added to the model while the number of

observations remains constant just as the R2 of a standard regression model rises under

the same circumstances- only the ~ for the DMUs of one and the same category can

be compared.

To demonstrate the effect of changing the ratio of parameters and sample size the

following experiment is conducted. The sample is first sorted by population size, then

it is split into two halves in two different ways. First, two samples are created, one

consisting of all odd numbered pharmacies, the other of all even numbered ones. Next,

two samples, one containing pharmacies 1 to 37, the other pharmacies 38 - 69 are

created." The reason for this exercise is to find out how structural efficiency is

influenced by changing the number of DMUs in this sample by deleting observations

in a random versus deleting them in a systematic way.

Deleting randomly for instance the odd or the even numbered pharmacies, corresponds

to the results obtained by Zhang and Bartels (1998) about the relationship between

sample size, number of parameter and efficiency scores. Deleting systematically, i. e.

deleting the small or the large pharmacies from the sample, corresponds to restricting

the sample to comparable DMUs. The Banker and Morey (1986b) as well as the

Lovell!Ruggiero method do therefore suffer from the same type of bias for the

efficiency parameters that can be detected by deleting observations systematically.

This bias may in fact be smaller than the one create by randomly deleting observations

as one deletes systematically DMUs which are less likely to be an efficient peer for the

21 This concern is expressed in Lovell (1994, section 8.3). It is obvious that the DMU with the smallest value for any non-discretionary input can not have a reference technology matching that value in a VRS specification.

22 The small pharmacy sample contains 37 pharmacies as there is a jump in population size between pharmacies 3 7 and 3 8.


DMUs left in the sample. From the point of view of the pharmacies in the small

sample it may not matter all that much whether the largest five pharmacies are taken

out of the sample or not.

Table 4: Sample size effects

model specifications Average score using

Standard variables no variable for size

all observations .88 .82

Even .90 .88

Odd .92 .84

Small .92 .90

Large .91 .84

Table 4 above shows the results for the two experiments. As was to be expected in

view of the results obtained by Zhang and Bartels (1998) the efficiency scores derived

with the split samples suggest lower structural inefficiency than those for the

resprective full samples. However, there does not seem to be a difference for this

particular data set between deleting systematically of randomly as the efficiency scores

of the respective subsamples are similar. Hence, the sample size effect seems to carry

through.

4.4.2 The determinants of efficiency

Table 5 presents the original Banker and Morey (1986b) results by population size

category. There were no observations in the first two categories of the classification

they used such that nine categories remain. As discussed in section 3.3 the pharmacies

in smaller communities can only be evaluated by other pharmacies in about as small or

even smaller communities.

Therefore, the 3 pharmacies in category 3 will only be compared with each other. The

7 observations in category 4 -see the column "nr. of obs."- can be compared with each

other and with the 3 smaller peers in category 3. This results in 10 potential peers

altogether- see the column entitled "nr. of peers". There are 14 potential peers for the

pharmacies in category 5.


Table 5: Banker and Morey (1986b) data revisited

POPCAT nr. ofobs. nr. of peers Theta_8

3 <1000 3

4 <1500 7 10 .98 .99

5 <2000 4 14

6 <10000 23 37 .88

7 <20000 2 39 .94 .88

8 <25000 40 .86

9 <50000 10 50 .79

10 <100000 9 59 .83 .81

II <250000 10 69 .81

Total .88

As a consequence, the efficiency scores are in effect derived from three different

DEAs, one based on 3, another on 10 and the third on 14 observations. Not

surprisingly, given the fact that six variables are used in the analysis, only one of the

fourteen smallest peers is inefficient and the average efficiency of this group is .99.

For the middle three categories evaluated by 37 to 40 peers the average or structural

inefficiency is .88 whereas it is .81 for the largest categories which is based on 50 to

69 peers. The overall average being again .88. Of course, smaller pharmacies might be

more efficient than larger ones but in this case one can not draw this type of

conclusion as long as the effects of sample size may still play a role.

To demonstrate the properties of their new model Banker and Morey ( 1986b) compare

their results to findings generated by a model for continuous non-discretionary

variables. To generate these results they replace the population categories by a variable

on actual population size and employ model (1) above. They point out the fact that the

categorical approach generate less (36) inefficient DMUs than the model bases on the

continuous variable and note that this is in line with their theoretical assertion that the

categorical approach will result in lower inefficiency but that the inefficiency detected

will have a higher credibility than the inefficiency derived by the continuous type

model. Interestingly three out of the five newly efficient DMUs are among the first 14

DMUs where efficiency scores are derived on the basis of extremely small sample

sizes.


To gain some insight into whether the results are driven by sample size or population

size, a DEA on the full sample without using the variable for population size is carried

out. Next, the efficiency scores given by that specification will be regressed on an

indicator for the population category." The results of the second stage tobit-regression

are shown in Table 6. It is clear that by not using an indicator for population size

during the DEA-stage, all the efficiency relevant information relating to size should

still be contained in the efficiency scores. However, the tobit model does not detect a

significant relationship between size and efficiency. Probably even more surprising, if

the effect found was a significant one, its direction would be just the opposite of what

Banker and Morey ( 1986b) assumed: Here, pharmacies in larger communities seem to

be in a disadvantaged position whereas Banker and Morey ( 1986b) considered a larger

community to be advantaged with respect to sales potential.

Table 6: "Determinants" of efficiency: Tobit model I

Dep. V ar. theta _p

Coefficient

!-value

Population categories

-0.02

-1.55

Constant

1.02

9.96

s. e.

0.25

The next table shows the results of a similar analysis. This time the efficiency scores

presented in Table 5 are regressed on the number of peers also listed in that table. This

gives the -in view of the Zhang and Bartels (1998) results not at all surprising- finding

that sample size matters when taking account of population size in the way Banker and

Morey (1986b) do.

23 This corresponds to the strategy applied by different researchers to model the effect of nondiscretionary variables: a two-stage procedure consisting of a DEA model using only discretionary variables on the first stage and some regression procedure applied to the scores of the first round analysis and the non-discretionary factors on the second (see section 4.3.2).


Table 7: "Determinants" of efficiency: Tobit model II

Dep. Var. Theta_8 m. of peers _cons _se

coef. -0.01 1.24 0.21

t-value -3.95 14.56

This suggests that all results based on model specifications that apply restrictions on

individuals peers such that each DMU0 will have a different number of potential peers

will generate biased results.

5 Conclusion

The paper discussed several aspects related to the treatment of non-discretionary

variables. To assure compatibility of the empirical models with the theory of

production functions one has to assume multiplicative separability between

discretionary and non-discretionary factors. One way of doing so is interpreting non

discretionary factors as Hicks-neutral technical progress (see Ray, 1988). A more

general way of modelling the influence of non-discretionary factors seems desirable.

On the empirical side, some models possess the undesirable property that the

efficiency scores are derived using subsamples of different sizes. In view of the results

obtained by Zhang and Bartels (1988), these results have to be considered as biased.

Especially if it seems inappropriate to evaluate DMUs by reference technologies

which are only on average comparable to DMU= but which contain individual peers

that are advantaged with respect to a non-discretionary variable, the only alternative

are two-stage approaches. This is a field where future research efforts should be

directed.

References

Banker, R. D. and R. C. Morey (1986a): Efficiency analysis for exogenously fixed inputs and outputs,

Operations Research 34 (4): 513-21.


Banker. R. D. and R. C. Morey (1986b): The use of categorical variables in data envelopment analysis,

Management Science 32: 1613-27.

Charnes, A., W. Cooper, A. Y. Lewin and L. M. Seiford (1997): Data Envelopment Analysis: Theory,

Methodology and Applications", 3 ed.

Fried, H. 0., C. A. K. Lovell and P. Vanden Eeckhaut (1993): Evaluating the performance of US

credit unions, Journal of Banking and Finance 17: 251-265.

Golany, B. and J. Roll (1993): Some Extensions of Techniques to Handle Non-Discretionary Factors

in Data Envelopment Analysis, Journal of Productivity Analysis 4: 419-432.

Golany, B. and J. Roll (1997): Restricted best practice selection in DEA: An overview with a case

study evaluating the socio-economic performance of nations, Annals of operations research 73:

117-140.

Greene, W. H. (1997): Econometric Analysis, New York.

Lovell, K. (1995): Linear programming approaches to the measurement of productive efficiency, Top

2: 174-248.

Ray, S. (1988): Data Envelopment Analysis, Nondiscretionary Inputs and Efficiency: An Alternative

Interpretation, Socio-Economic Planning Sciences 22: 167- 176.

Ray, S. (1991): Resource-Use Efficiency in Public Schools: A Study of Connecticut Data,

Management Science 37: 1620-28.

Ruggiero, J. (1996): On the measurement of technical efficiency in the public sector, European Journal

of Operations Research 90: 553- 565.

Seiford, L. M. (1996): Data Envelopment Analysis: The Evolution of the State of the Art (1978 -

1995), Journal of Productivity Analysis 7: 89-138.

Zhang, Y. and R. Bartels (1998): The Effect of Sample Size on Mean Efficiency in DEA with an

Application to Electricity Distribution in Australia, Sweden and New Zealand, Journal of

Productivity Analysis 9: 187- 204.

Non-Radial Efficiency as Semi-Radial Efficiency

Laurens Cherchye and Tom Van Puyenbroeck'

Abstract

Existing non-radial efficiency measures have focused on solving the slack problem associated

with the Debreu-Farrell notion of efficiency, sometimes at the expense of the underlying

economic intuition as regards their eventual projection. In this paper we take another

perspective. Building on the idea that any reference point can be obtained using a semi-radial

method, we start by imposing some (desirable) properties on the reference. Specifically, we

require the projection to be as close as possible to the radial one while still belonging to the

efficient subset of the production technology. In a second step efficiency scores are then

computed by reference to the obtained projection. These scores can be decomposed in a

Debreu-Farrell component and a factor that captures the divergence between reference and

evaluated input-output mixes. This second component can also be interpreted as measuring

the extent to which the radial projection deviates from the closest Pareto-Koopmans efficient

feasible point. This new way of measuring non-radial efficiency allows to maintain the

attractive interpretation of radial measures while avoiding the slack problem.

1 Centre for Economic Studies, Catholic University of Leuven, Naamsestraat 69, B-3000 Leuven, Belgium

[email protected]

[email protected]

52 Non-Radial Efficiency as Semi-Radial Efficiency

Structure

Introduction

2 Decomposing non-radial efficiency

3 Towards some desirable properties of the projection direction

4 Conclusion

References

Non-Radial Efficiency as Semi-Radial Efficiency 53

1 Introduction

The purpose of the two benchmark DEA models of Chames, Cooper and Rhodes

[CCR (1978, 1979)] and Banker, Chames and Cooper [BCC (1984)] was to put into

practice the notion of efficiency as expressed in the earlier work of Debreu ( 1951) and

Farrell ( 1957). As the inverse of Shephard's distance function the associated "Debreu

Farrell" measure is closely linked to the microeconomic theory of production.

Moreover, this measure has a straightforward cost interpretation as it can be written as

the ratio of reference to actual costs (input orientation) or actual to reference revenues

(output interpretation), independently of the price vector that is used [see Russell

(1985)].

These convenient properties follow from the radial projection of inefficient decision

making units (DMUs) on the reference frontier. It is well known that the

equiproportionate nature of comparisons has a drawback however since it implies that

Debreu-Farrell efficiency does not necessarily coincide with the more general Pareto

Koopmans efficiency concept [as introduced by Koopmans (1951)]. This is the so

called slack problem: "efficient" projections can sometimes increase their technical

efficiency by a further non-radial change of some input or output dimensions. This

problem is likely to increase when the number of dimensions becomes big relative to

the number of observations. It is even more pronounced when the proportionality

(CCR) or convexity (BCC) assumption is dropped, so that only the deterministic and

free disposal postulates are maintained like in the increasingly popular Free Disposal

Hull (FDH) model [see Tulkens (1993)].

In some instances the slack problem can influence the efficiency scores considerably,

which in tum could induce wrong management conclusions. In such cases it seems

more appropriate to call for measures that ensure inefficient DMUs are compared to

Pareto-Koopmans undominated references by projecting in a non-radial way. As their

name suggests, these non-radial measures essentially differ from their radial

counterparts in that the input (output) mix of the input (output) efficient projection may

deviate from the mix of the inefficient observation. However, this feature is as such not

addressed in well-known non-radial variants such as the Russell measure, introduced

by Hire and Lovell ( 1978), and the additive measure of Chames et a!. (1985) and many


of their successors. An explicit analysis of such mix deviations relative to the radial

projection may be rewarding due to the latter's attractive interpretation.

In this paper we present a measure which aims at preserving the nice economic

intuition of the radial efficiency estimates while still computing efficiency with respect

to a Pareto-Koopmans efficient projection. In doing so, we exploit the characteristic

that any non-radial measure can be re-interpreted as a semi-radial efficiency measure.

The associated efficiency scores can thus be decomposed in an appealing way.

Specifically, they are the product of a classical Debreu-Farrell measure and a factor

that captures differences in input-output mixes of an evaluated DMU and its reference.

These latter projection points are determined starting from some axiomatic properties.

Essentially, we require that the reference should be "as close as possible" to the radial

projection in terms of mix deviation. In a second step then, we calculate efficiency

scores using the procedure outlined in an earlier paper [Cherchye and Van

Puyenbroeck (1998)].

The plan of the paper is as follows. In section 2 we shortly repeat the procedure to

compute the efficiency scores. In section 3 we propose a set of desirable properties for

an efficient projection, and show how such a projection can be obtained in a DEA

framework. Concluding remarks follow in section 4.

2 Decomposing non-radial efficiency

In Cherchye and Van Puyenbroeck (1998) we introduced an (input and output

oriented) decomposable measure which can be computed as soon as one disposes of a

reference point for the DMU under study. For simplicity, we will mainly concentrate

on its input orientation, but the intuition straightforwardly extends to the output

orientation. Throughout the paper we consider a general situation where N

observations use an m-dimensional semi-positive input vector x = (xh ... , xm) to

produce an s-dimensional semi-positive output vector y = (y1, ... , y,).

Assume for the moment that an input vector x is projected on an "arbitrary" reference

xR. We propose to compute the input efficiency score for x as follows:


(1)

In Cherchye and Van Puyenbroeck (1998) it was shown how the input measure (1) can

be decomposed in an intuitive way by exploiting its semi-radial nature, as will now be

recaptured briefly. In figure 1 we present an input vector x. (=(7,4)) with xb (=(2,3)) as

the reference vectors. The vectors differ in two dimensions. First, there is the obvious

fact that xb Pareto-Koopmans dominates x •. Second, the input mixes of x. and xb differ.

Indeed, the proportion x/x1 for x. is 4/7 whereas it amounts to 3/2 for xb. The same can

also be expressed by means of the vectors v. = (1, 4/7) and vb = (1, 3/2), where the first

input is taken as the numeraire.

Applying (1) gives an efficiency score of 44.72% for x •. This score can now be

decomposed in two factors. To see this we first construct the lines perpendicular on the

radials through x. and xb. These lines take the form fi = v/ X; (i=a,b). Both

perpendiculars are depicted in figure 1, together with a line through x. which is parallel

to the perpendicular through xb. This allows us to identify xc, which is the closest

projection of x. on the radial through xb. In fact xc corrects for the deviation between v.

and vb. In order to get from x. to xb one first needs to adjust the input-mix (x. ~ xc), so

as to apply an equiproportionate reduction of all inputs (xc ~ xb) afterwards. The

inefficiency of x. thus results from (i) a deviation from the reference input mix and (ii)

a cost level which is too high, even after adjustment for the mix deviation.

Figure 1: decomposing non-radial efficiency

8

6

2

· ,~26/2)~x, + (312)x,

',

''-...,X

',,, :8 .. · '',(13/2)~x, +(3/2)x,

0 J_~~--·_· ---+------+----'"'",-'-'>..--------;

0 4 Input x1

6 8


The cosine of the angle between xb (or xc) and X a constitutes a measure for (i). The ratio

of the cost level of xb over that of xc can be used to estimate (ii). Summarising, measure

(1) is equivalent to:

cost level x b . = x cosO, Wtth 8 the angle between X a and Xb (or Xc)

cost level x c

T T = vb xb x Xa xb

v/ X a llxa II x llxb II = 0.5 X 0.8944

= 44.72%

The decomposition of measure (1) thus yields two factors. The first one captures

radial inefficiency due to a higher cost level after adjustment for differences in input

proportions (Debreu-Farrell component (DF)). The second factor estimates the

inefficiency due to the deviation between the actual and the reference input mix (mix

correspondence component (MC)). Its value will equal one if both the evaluated and

reference input vectors lie on the same radial, and will be smaller than one otherwise.

In the general case an input vector x will be projected on a reference xR. Denoting the

vectors which define the perpendiculars on the radials through x and xR as respectively

v and vR, we get:

· . llxRII v/ xR xr xR mput effictency score= Txif = v R r x x llxll x llx R II (2)

An analogous formulation holds for the slack based output efficiency measure:2

T T . UR Y Y YR

output effictency score= u R r y R x IIYII x IIY R II (3)

2 One cannot take ~~~;111 as a measure for output efficiency as this ratio can be decomposed in the

following way: ~~;1( ::::, x[li( 11J:t~.IIJl Deviations from the reference output proportions would

thus be rewarded, which is clearly undesirable.


where y and YR denote the actual and reference output vector, respectively. The vectors

u and uR define the perpendiculars to the radials. The interpretation of the output

measure (3) follows immediately.

Two additional comments are in order here. First, note that both the input and output

measures allow for zero entries in the data set. Second, the above decomposition was

conducted for an "arbitrary" reference input- or outputvector. By this we mean that the

reference could have been obtained in various ways, e.g. as an exogenously given

"most prefered target", after applying an additive DEA-model, following a Fare-Lovell

procedure, etc. It can also be computed for radial projections, which would however

be a rather trivial exercise given that the MC component in such a setting is evidently

equal to 1. In all cases, it is fundamentally the semi-radial character of the projection

that allows to resurrect (mix-corrected) Debreu-Farrell efficiency measures and their

associated economic interpretation.

3 Towards some desirable properties of the projection direction

Using this last basic concept of semi-radiality, one can actually proceed in the opposite

way. In particular, by imposing restrictions on the nature of the mix correspondences

one may narrow down the set of allowable non-radial projections. In this section we

provide a simple application of such an approach that eventually leads to a novel non

radial measure. It must be noted at the outset that this way of proceeding contrasts

with the usual approach in the literature, where the focus has mainly been on axiomatic

properties of non-radial measures instead of the reference points themselves. This of

course resulted in nicely behaving measures, but the economic intuition associated

with the efficient projections is not always obvious. Here we take the other perspective

and start from some desirable characteristics of the reference projection.

One such a characteristic is obviously that it should belong to the Pareto-Koopmans

(input or output) efficient subset of the reference technology (C.l ). A second property

we propose is related to the natural theoretical and empirical benchmark status of

Debreu-Farrell efficiency measures. Specifically, it is considered desirable that the

direction of projecting is the closest possible to the radial one (with the difference

measured by the cosine of the angle between actual and reference vectors), so as to


maintain the attractive interpretation of radial measures as much as possible (C.2).

Intuititively, we thus seek for a reference point that is as near as possible to the

Debreu-Farrel reference while in any case being freed of the slack problem.

For ease of exposition we again concentrate on the input orientation, but extensions to

the output oriented projection are straightforward. Consider an input-output

combination (X0 J10 ). Let L(y.) denote the input correspondence that assigns the subset

of all input vectors x which allow to produce the output vector y/ Further define Eff

L(y.) as the efficient subset of L(y.): 4

The following conditions should then be met by a reference vector xR for X 0 :

C.l XR E Ef!L(y.)

In fact, C.l becomes redundant in view of C.2. We choose to state both properties

separately however to illustrate the difference between the projection introduced here

and other non-radial projections in the literature which, as far as the reference point is

concerned, exclusively focus on C.l. The reference projection satisfying C.2 can be

identified in two steps: one first locates the efficient subset of L(y) and then looks for

its cosine minimising element in a second step. Once the reference is identified, one

can of course compute an efficiency score using (2).

In a DEA context specific assumptions are made about L(y.). For the constant returns

to scale (CRS) model with free disposability of both inputs and outputs (FD) it is

defined in the following way (fork= l, ... ,s; I= l, ... ,m and}= l, ... ,N):

3 See e.g. Fiire, Grosskopf and Lovell (1985) for a discussion of L(yj in a general setting. 4 We use the symbol<=(>=) to indicate "smaller than or equal to", whiles(<::) means<=(>=) and*·


When the assumption of constant returns to scale is relaxed to variable returns to scale

(VRS), we get:

In some instances the convexity assumption is hardly defensible. Then it seems

advisable to use an FDH reference technology, which differs from the previous model

in that the intensity vector e can only contain zeros or ones [see Tulkens (1993)]:

To obtain a projection under EJJ L(yJFDH is relatively easy. One only needs to identify

those observations that are not Pareto-Koopmans dominated by another observed

input-output combination (applying e.g. the additive FDH model [see Bardhan et a!.

(1996)]). The reference that satisfies C.2 is then the one corresponding to the smallest

angle with x •. 5

The algorithm to be applied for L(yJFD-cRs and L(yJFD-YRS is analogous. The main

difference is that efficient subsets do no longer consist of individual points but instead

of efficient facets constituting the corresponding convex polyhedrons. Algorithms and

software codes to identify these facets have been developed in the domain of multiple

objective linear programming (one can e.g. use the ADBASE code [Steuer (1995)] or

one of its variants [e.g. Armand and Malivert (1991) or Strijbosch eta!. (1991)]).6 Each

facet F can be represented as the set of all convex combinations of P efficient extreme

points {(xEI> Y£1), ... ,(xEP• YEP)} defining it. The associated set IF of input values can

thus be reconstructed as follows (for I =1, ... , m and}= 1, ... , P):

' Note that for the FDH reference technology the efficient subset is normally discontinuous. In certain cases this may lead to a reference observation identified under C.2 that does not Pareto-Koopmans dominate (x.,. y.). If one wishes that the cosine minimising reference is not only Pareto-Koopmans efficient but also guaranteed to be Pareto-Koopmans dominant, this is accomplished by adding to C.2 the following (FDH-specific) condition: xR <= x •.

'See Steuer (1986, 1994) for an overview of algorithms and software codes.


For each F one has to identify that element of IF which maximises the cosine with X0 •

This boils down to the following mathematical programming problem:

(4) subject to: N z>j =1 j=l

aj >= 0

It can easily be checked that the objective of (4) is concave with respect to the

elements of a, implying that a maximum will always be obtained. Problem (4) can thus

be solved by means of simple Lagrangian techniques.

Summarising, for L(y,)FD-cRs and L(yJFD-vRs the projection satisfying C.2 can be found

following a three-step procedure: (i) identify the facets that together constitute the

efficient subset, (ii) solve problem (4) for each of these facets, (iii) look for the

minimum of the corresponding objective function values. Of course, these three steps

need only be executed when slacks are associated with radial projections. Moreover,

when locating the efficient facets it suffices to consider only those observations that

are found Pareto-Koopmans efficient by the constant or variable returns to scale

additive DEA models ofChames et al. (1985).7

To illustrate the new measure, consider the example presented in figure 2. For

simplicity it is assumed that all six DMUs produce the same output level. Table 1

shows the scores (and their MC and DF components) for the inefficient observations d,

7 Sometimes it is computationally more efficient to proceed by identifying all efficient facets of the production possibility set following e.g. the algorithm of Yu et al. (1996). This procedure is straightforwardly analogous to the above. Of course, for an inefficient observation it then suffices to consider only those efficient facets that are constructed from at least one vertex that ParetoKoopmans dominates it.


e andf We first consider the FD-CRS and FD-VRS technologies in which cases the

reference frontier is represented by the full line. Obviously there is a slack problem

associated with the radial projection for observation f Applying the above procedure

yields c as the appropriate reference under C.2. As f is not "naturally enveloped" the

MC score is below one.8 Note that the DF score off coincides with that of e. In fact, e

is the closest projection off on the radial through c. When an FDH technology is

assumed, the slack problem is usually more pronounced. This is also illustrated here

(see the dotted line). Now d also has to adjust its input mix if it is to become a member

of the Pareto-Koopmans efficient subset. Correspondingly, its MC score is below 1.

The needed input mix adjustment, as captured by the cosine of the angle between a and

d, is even greater than for f

Figure 2: A hypothetical example

....

121 10 t

8

;; 6 (1.7

! 4 ~

Table 1: Efficiency results

DMU CCRIBCC score MC

D 0.586 1.000

E 0.625 1.000

F 0.623 0.997

-· d (3,/0)

2 4

DF 0.586

0.625

0.625

-c-(6,3)

6 8

Input I

FDH

score

0.677

0.625

0.623

_ e e (9.6. 4.8) -• f(8,6) ----

10 12

MC DF 0.989 0.685

1.000 0.625

0.997 0.625

8 The term "not naturally enveloped" is taken from Bessent et al. (1988) which use it to indicate inefficient units that have a mix of inputs or outputs different from any point belonging to the efficient subset.


4 Conclusion

While most non-radial proposals allow to solve the slack problem associated with

radial efficiency measurement, they do not possess an immediate interpretation like the

prevalent Debreu-Farrell measures. The literature has mainly concentrated on the

axiomatic properties of the non-radial measure itself rather than on any economic

intuition associated with the eventual projection.

In this paper we built on the semi-radial re-interpretation of non-radial measures to

change the perspective: an axiomatisation of the (non-radial) direction of projection is

proposed which ensures it to be as close as possible to the radial one while always

selecting a Pareto-Koopmans undominated reference. Consequently, the ensuing

projection will equal the equiproportionate one as long as the latter belongs to the

efficient subset. In the other case it will minimise adjustments in the input-output

structure. Once the "best" reference is obtained, the decomposable measure we

introduced in an earlier paper can be used to obtain nicely interpretable efficiency

scores. Specifically, it enables to split up total inefficiency in a Debreu-Farrell and a

mix correspondence component.

The obtained efficiency results capture useful additional information, especially when

radial efficiency measurement is associated with slacks. The mix correspondence score

then gives a measure for the least extent to which the input (output) proportions need

adjustment in order for the evaluated observation to become "naturally enveloped".

One could also consider it as a composite relative measure for the slacks associated

with radial projection. The Debreu-Farrell component, on the other hand, estimates the

residual inefficiency after correcting for the input (output) mix deviations.

The analysis presented above can be extended in some ways. For example, other

desirable properties for reference points or mix-deviation factors can be imposed to

provide a characterisation of other (existing as well as novel) efficiency measures. In

any case, it seems worthwile to recognize what we have subsumed here under the

general header of semi-radial efficiency measurement. As DEA problems are mostly

formulated in terms of multiple inputs and outputs, the possible divergence between

reference and actual input or output proportions at least potentially becomes a relevant

dimension of the problem setting. Inefficiency does then no longer only depend on a

ratio between minimum (maximum) to actual input (output), but also on the proportion


between the several inputs (outputs). In this sense, the particular projection procedure

and associated measure introduced here may constitute a valuable alternative when

slacks indeed appear systematically if efficiency is estimated in a radial way.

References

Armand, P. and C. Malivert (1991): Determination of the Efficient Set in Multiple Objective Linear

programming, Journal of Optimization Theory and Applications, 70 (3), p. 467-490.

Banker, R.D., Charnes, A. and W.W. Cooper {1984): Some Models for Estimating Technical and

Scale Inefficiencies in Data Envelopment Analysis, Management Science 30 (9), p. I 078-1092.

Bessent, A., Bessent, W., Clark, T. and J. Elam (1988): Efficiency Frontier Estimation by Constrained

Facet Analysis, Operations Research 36, p. 785-796.

Bardhan, 1., Bowlin, W.F., Cooper, W.W. and T. Sueyoshi {1996): Models and Measures for

Efficiency Dominance in DEA: Part I: Additive Models and MED Measures, Journal of

Operations Research Society of Japan 39, p. 322-332.

Charnes, A., Cooper, W.W. and E. Rhodes {1978): Measuring the Efficiency of Decision Making

Units, European Journal of Operational Research 2, p. 429-444.

Charnes, A., Cooper, W.W. and E. Rhodes {1979): Measuring the Efficiency of Decision Making

Units, European Journal of Operational Research 3, p. 239.

Charnes. A., Cooper, W.W., Golany, B., Seiford, L. and J. Stutz (1985): Foundations of Data

Envelopment Analysis for Pareto-Koopmans Efficient Empirical Production Functions, Journal

of Econometrics 30, p. 91-107.

Cherchye, L. and T. van Puyenbroeck {1998): Learning from Input-Output Mixes in DEA: A

Proportional Measure for Slack-Based Efficient Projections, Managerial and Decision

Economics, forthcoming.

Debreu, G. {1951): The Coefficient of Resource Utilisation, Econometrica 19(3), p. 273-292.

Fare, R., Grosskopf, S. and C.A.K. Lovell {I 985): The Measurement of Efficiency of Production,

Kluwer-NijhoffPublishing, Boston.

Fare, R. and C.A.K. Lovell (1978): Measuring the Technical Efficiency of Production, Journal of

Economic Theory 19 (I), p.J50-162.

Farrell, M.J. (1957): The Measurement of Productive Efficiency, Journal of the Royal Statistical

Society Series A 120 (3), p 253-281.


Koopmans, T. (1951): Analysis of Production as an Efficient Combination of Activities, in:

Koopmans, T. (ed.), Activity Analysis of Production and Allocation: Proceedings of a

Conference, Yale University Press, New Haven, p. 33-97.

Russell, R. (1985): Measures ofTechnical Efficiency, Journal of Economic Theory 35(1), p. 109-126.

Steuer, R.E. (1986): Multiple Criteria Optimization: Theory, Computation and Application, John

Wiley, New York.

Steuer, R.E. (1994): Random Problem Generation and the Computation of Efficient Extreme Points,

in: Multiple Objective Linear Programming, Computational Optimization and Applications 3,

p. 333-347.

Steuer, R.E. (1995): Manual for the ADBASE Multiple Objective Linear Programming Package,

Faculty of Management Science, University of Georgia, Athens.

Strijbosch, L.W.G., van Doorne, A.G.M. and W.J. Selen (1991): A Simplified MOLP Algorithm: The

MOLP-S Procedure, Computers and Operations research 18 (8), 709-716.

Tulkens, H. (1993): On FDH Efficiency Analysis: Some Methodological Issues and Applications to

Retail Banking, Courts and Urban Transit, Journal of Productivity Analysis 4, p. 183-210.

Yu, G., Wei, Q., Brockett, P. and L. Zhou (1996): Construction of All DEA Efficient Surfaces of the

Production Possibility Set under the Generalised Data Envelopment Analysis Model, European

Journal of Operational Research 95, p. 491-510.

Continuity of the BCC Efficiency Measure

Holger Scheel 1

Abstract

Continuity is a desirable property of an efficiency measure. It ensures that small data errors

cause only small errors in the efficiency measure. In this paper continuity properties of the

BCC efficiency measure are studied. Examples are given where this measure 'jumps" under

arbitrary small data perturbations. However, it is shown that under weak assumptions it

depends continuously on the input output data. Implications to the stability of efficiency

classifications are discussed, and the results are illustrated by an empirical example.

1 Operations Research und Wirtschaftsinforrnatik, Universitlit Dortmund, D-44221 Dortmund, Germany

[email protected]

66 Continuity of the BCC Efficiency Measure

Structure

Introduction

2 Discontinuities of the BCC measure

3 Stability of the BCC measure

4 An illustrative example

Appendix

References

Continuity of the BCC Efficiency Measure 67

1 Introduction

Data Envelopment Analysis (DEA) has become a standard approach to efficiency

measurement. It is a family of methods for the evaluation of relative efficiency of

"decision making units" (DMUs). We refer the interested reader to Seiford (1996) for a

recent survey. The classical DEA models developed by Chames et a!. (1978) and

Banker eta!. (1984) classify each DMU to be evaluated as "efficient" or "inefficient".

Moreover, for each inefficient DMU a measure is computed which indicates the

proportional input reduction which is necessary, ceteris paribus, to change its

classification from inefficient to (radially) efficient.

Let each DMU k in the set of n DMUs be characterized by its input output data

collected in the row vector (X",r*), where we suppose all entries to be nonnegative and

at least one input and one output to be positive. Let (X,Y) denote the matrix of input

output data, where each row represents one DMU and each column represents one

input or output. We assume that each column contains at least one positive element.

Following Banker eta!. (1984) we assume that the production possibility set exhibits

variable returns to scale and is thus given by

T(X,Y)= {(x,y) I ..17X :s; x, ..17Y ;e: y, ..1.Te =1, A ;e: 0},

where e = ( 1 , ... , 1 )r in appropriate dimension. Recall that DMU k or, synonymously,

the input output vector (X,Y') is called (radially) input-efficient in T if it is impossible

to reduce inputs proportionally without reducing any output, i. e. if there is no B < 1

such that (BX",Y') E T. The BCC efficiency measure hk(X,Y) ofDMU k is defined as

the optimal value of the linear program

minimize e

s.t. AT X :::; ex' ATY "= Y' ATe 1

(1)

A "= 0,

cf. Banker et a!. (1984). Notice that by defining the BCC measure hk in this way we

focus on radial efficiency measurement, i. e. we ignore "mix inefficiency" which is

often included in DEA measures by a non-Archimedian component.


In economic theory an efficiency measure is supposed to satisfy certain conditions.

Weak versions of the axioms introduced2 by Fare and Lovell ( 1978) can be formulated

as follows:

(I) Indication of efficiency:

hiX. Y) = 1 if and only ifDMU k is (radially) input-efficient.

(H) Homogeneity:

hix",Y) = ~ hk(X,Y) for every e with ( eX',Y") E T(X,Y). [x" denotes the matrix X

where row X' is replaced by eX'.]

(M) Monotony:

If X'::;; x and Y" ~ y then hiX,Y) ;?: hk( X, Y ). [(X, Y) denotes the matrix (X,Y)

where row (X,Y") is replaced by (x, y ).]

Notice that the homogeneity property (H) provides a key for interpreting efficiency

measures. In fact, ifDMU k is assigned an efficiency measure of hiX,Y) = e< 1, then

a proportional reduction of inputs to ex suffices to become efficient, i.e. hix",Y) = 1.

Hence an efficiency measure satisfying (H) may be viewed as a stability measure

which indicates the feasible input change preserving the efficiency status. It is easily

verified that the BCC measure satisfies (I), (H) and (M).

Russell ( 1990) introduced "continuity" into the above axiom system.

(C) Continuity:

hiX,Y) depends continuously on the data matrix (X,Y).

This property, defined formally below, is desirable in particular from a practitioner's

perspective: It ensures that small perturbations of data cause only small changes in the

efficiency measure. Russell (1990) formulated conditions for general production

possibility sets that ensure continuity of radial efficiency measures. Recently Kuntz

2 Fiire and Lovell ( 1978) proposed four axioms. However, Russell ( 1985) showed that their property "E.2" is implied by the other three conditions.


and Scholtes (1996) and Scheel and Scholtes (1998) studied continuity of radial

measures assuming a constant returns to scale technology. Following the route taken in

the latter paper we study in the sequel the continuity property (C) of the BCC measure

which is based on the variable returns to scale technology T.

Motivating our results we present in the next section two examples where the BCC

measure violates (C). Section 3 contains the main continuity results. Moreover, some

implications to the stability of efficiency classifications are discussed. In the final

section the results are summarized by means of an illustrative empirical example.

2 Discontinuities of the BCC measure

Whenever in applications an input or output is measured on a continuous scale then

inaccuracies in the data matrix (X,Y) cannot be avoided since a measurement tolerance

must be chosen a priori. It is possible to choose an arbitrary small measurement

tolerance, but it is impossible to choose a zero tolerance. Since higher precision usually

induces higher measurement cost, it is desirable that improved data quality yields to a

better estimation of the "true" efficiency measure, i.e. if a sequence of data matrices

(X,,Y1) converges by increasing measurement precision to the "true" data (X,Y), then the

corresponding sequence of BCC measures hk(X,,Y,) should converge to the "true"

measure hiX,Y). This continuity property, however, may not be satisfied by the BCC

measure.

Notice first that zero data may cause discontinuities of hk as can be illustrated by the

following example adapted from Scheel and Scholtes ( 1998). Consider a data matrix

for two inputs and one output given by

The BCC measure of DMU 1 is h1(X,,Y1) = 1 for every t e (0, 1] and h1(X,,Y,) = 0.75

for t = 0. Assuming that t = 0 is the "true" value then even choosing an arbitrary high

measurement precision cannot ensure that the true efficiency measure can be

computed. Moreover, DMU 1 may be classified falsely as efficient.


This example may suggest that - like in the constant returns to scale case studied in

Scheel and Scholtes (1998) - discontinuities of the efficiency measure are always

caused by zeroes in the data matrix. However, the BCC measure h* can jump although

the data matrix is strictly positive.

y

40

2 3 30

20

10

10 20 40 50 X

Figure 1: Discontinuity with positive data matrix

Consider the production possibility set in Figure 1, where the underlying data matrix is

strictly positive. The efficiency measure of DMU 3 is h3 = 0.4 but jumps to unity after

an arbitrary small increase of output Y 3•

Notice that continuity is not only desirable in the presence of measurement errors but

also in view of the reliability of measures computed by linear programming software.

3 Stability of the BCC measure

Continuity means that small data perturbations cause only small changes in efficiency

measures. In fact we shall require that these changes are bounded by the size of

perturbations. Following Scheel and Scholtes (1998), h* is called continuous at (X,Y)

with respect to certain admissible perturbations if there exists a positive scalar y such


that lhiX.Y) - hk (X',Y11 ~ r II(X,Y)-(X',Y)II for every data matrix (X',Y1 in a

neighborhood of(X,Y) which is an admissible perturbation.

As indicated by the example above, zeroes in the data matrix may play an important

role for the continuity of efficiency measures. It can be shown that the BCC measure

may jump if an inefficient DMU has a zero input. In fact, if hk(X, Y) < 1 and X contains

a zero entry X/ then by adding for every DMU I* k an arbitrary small positive number

to X,' we obtain A-1 = 0 for I * k and A.k = 1 for every feasible A. in the linear program (I),

whence hk jumps to 1.

Remark 1 The BCC measure of an inefficient DMU with a zero input is not continuous

if perturbations in the corresponding input column of the data matrix are defined as

admissible.

Hence, if a DMU with zero inputs appears inefficient, one has to look carefully at the

data quality before using this classification. However, in most applications zeroes in

the input data occur only if these inputs are not used by this DMU at all. This means

that the zero values do not suffer from a positive measurement tolerance, i.e. one can

expect that a zero value is exactly zero. For a useful notion of stability it is thus not

necessary to consider arbitrary data perturbations but only perturbations of the

nonvanishing data. Following Scheel and Scholtes (1998), we call an efficiency

measure stable if it is continuous with respect to perturbations of non vanishing entries

of the data matrix.

In order to derive conditions which ensure stability of the BCC measure recall that a

DMU is called radially output-efficient if it is impossible to increase all outputs

proportionally, i.e. the output oriented BCC measure which is defined as the optimal

value of the linear program

maximize¢

s.t. ATX ~ x* ATY ~ ¢Y* Are I

(2)

A ~ 0

is unity. Notice that in Figure 1 DMU 3 is radially output-efficient but not radially

input-efficient. The following theorem shows that indeed this is a necessary and


sufficient condition for instability of the BCC measure. The proof can be found in the

appendix.

Theorem 1 hk is stable at (X,Y) if and only if output-efficiency of DMU k implies input

efficiency.

A violation of the assumption of the theorem can be detected by computing both the

input oriented and output oriented BCC measure. Notice that if the assumption of

Theorem 1 is violated then there exists a solution for program (2) where all input

slacks are positive. However, since the condition "output-efficiency implies input

efficiency" is satisfied for almost every data matrix3, one can expect in practical cases

the BCC measure to be stable.

Efficiency measures which satisfy the indication property (I) measure inefficiency

rather than efficiency since they assign unity to every efficient DMU. Replacing (I) by

the requirement that an efficiency measure greater than or equal to unity indicates

efficiency, Andersen and Petersen (1993) introduced an extension to DEA measures

which allows the ranking of efficient DMUs. We will later use this extension to obtain

stability results for the classification of DMUs as efficient.

The extended BCC measure, also called "superefficiency measure", can be defined4

by

Hk(X. Y) =sup{ e I (eX', Y*) is input-efficient in T(X,Y)},

where X denotes the input matrix X with row X replaced by eX'. The measure Hk

yields for every data matrix (X,Y) an efficiency measure in the interval (O,oo], where

infinity means that efficiency is preserved under arbitrary proportional increase of

inputs.

' This means that the condition is satisfied for all matrices in an open and dense subset of data matrices. It can be shown that {(X,Y) I H,(X,Y) * I, G,(X,Y) * I} is an appropriate subset with H, given by (3) and the corresponding output oriented extended BCC measure G, given by (2) where the k-th row of (X,Y) is removed.

4 In contrast to Andersen and Petersen (1993) we include this definition explicitly, since some authors argued that their measure may not exist in some cases where the corresponding linear program (3) is infeasible, cf. e.g. Wilson (1995).

Contmuity of the BCC Efficiency Measure 73

Note that for inefficient DMUs H* coincides with the BCC measure hk> whereas for

efficient DMUs H* '2 hk = 1. Indeed, if the efficiency measure is Hk >1 then DMU k

turns inefficient if it increases its inputs more than to H* X'. Clearly Hk coincides with

the optimal value of the linear program

minimize B

s.t. x-• A s BX* y-• A '2 y* Are

A '2 0,

(3)

where (X- *, Y- k) is obtained by deleting row k of (X, Y) and the optimal value is set to

infinity if the feasible set is empty.

A sufficient condition for stability of the extended BCC measure has to be stronger

than for the standard BCC measure. This can be seen from Figure 1 where DMU 2 is

output-efficient and input-efficient, hence the assumption of Theorem 1 holds.

However, when decreasing output y3 of DMU3 the efficiency measure H2 jumps from

2.5 to infinity. Note that such jumps can only occur if Hk '2 1, since for inefficient

DMUs the values coincide with the BCC measure which is continuous provided the

assumption of Theorem 1 holds.

We shall study now the "direction" of possible jumps of BCC measures. In the

examples given before, discontinuities never appear as downward steps. The following

theorem shows that such steps are indeed impossible.

Theorem 2 Hkcanjump only upwards.5

The theorem means that efficient DMUs either remain efficient under small data

perturbations or tum inefficient with a continuous decrease of their efficiency measure.

Moreover it follows that if the assumption of Theorem 1 does not hold then jumps of

the standard BCC measure hk are always upwards. These results have useful

implications for the stability of the DMUs' efficiency status which we shall discuss

now.

' A formal statement of the theorem can be found in the Appendix.


Given an input output data matrix, the set

E(X,Y) := {k I DMU k is input-efficient in T(X,Y)}

is called efficiency classification. Since the simple efficiency status of a DMU attracts

much interest in practical efficiency valuations it is desirable that this classification is

stable, i.e. if (X',Y1 is a data matrix in a neighborhood of (X,Y) where only

non vanishing entries are perturbed then the equation

E(X,Y) = E(X',Y')

holds. A stable efficiency classification ensures that efficient DMUs remain efficient

and inefficient DMUs remain inefficient under small perturbations of nonvanishing

data.

In the spirit of DEA as a "fair" methodology for efficiency measurement, cf. Epstein

and Henderson (1989), a DMU should be classified as inefficient only if a positive

minimal distance to the efficient frontier can be quantified. This means that every

inefficient DMU should remain inefficient under small data perturbations, i.e. E(X,Y)

;;;;< E(X', n for every perturbed matrix (X', Y1. In view of the theorems stated above this

inclusion holds if and only if none of the inefficient DMUs is output-efficient.

To ensure that the efficiency classification E is stable, the reverse implication E(X,Y) ~

E(X',Y1 must hold for every (X', Y1 near (X,Y) as well. This inclusion6 means that

efficient DMUs remain efficient under small data perturbations. It can easily be shown

that a DMU k remains efficient under data perturbations if and only if HiX. Y) > I,

where the "if' follows from Theorem 2. To see the "only if', assume HiX,Y) = I.

Then any proportional increase of the input data X' will decrease () in the first

constraint of program (3), and Hk which coincides with the optimal value of (3) will

decrease as well. Hence, DMU k turns inefficient.

Remark 2 The BCC efficiency classification E(X,Y) is stable if and only if HiX,Y) ;t. 1

for each DMU k and there is no output-efficient DMU which is input-inefficient.

' If this inclusion does not hold for the whole set E, there may be a subset cr(X, Y) of E which satisfies a(X,Y) c E(X',Y1 for every (X',Y1 near (X,Y), i.e. cr denotes the set of stable efficient DMUs. This classification is closely connected to the classification introduced by Charnes eta!. (1986). Following their notation, we have cr = E and E = E u E' u F.


4 An illustrative example

In order to illustrate the results of the previous section let us consider an empirical

application where the efficiency of 63 agencies of an insurance company with respect

to launching a new contract was evaluated. Table 1 contains the inputs and outputs

used for the evaluation.

Inputs No. of clients Type A No. of clients Type B

No. of clients Type C Potential new premiums (DM)

Outputs No. of new contracts Sum of new premiums (DM)

Table 1: Inputs and outputs of insurance agencies

The aim of the agencies is to sell as many contracts with as many premiums as

possible which is indicated by the two outputs.

The clients served by the agencies and possibly buy the new contract are classified into

several types which reflect the current insurance coverage, e.g. the total policy value

etc. The potential new premiums depend on the clients' current coverage, hence it is

included as an additional input. The data set contains some zeroes in the input "No. of

clients Type C". This means that there are no such clients, i.e. there is no measurement

tolerance for this input. As expected, each output-efficient DMU is input-efficient,

hence the BCC measure is stable. Statistics of the results7 can be found in Table 2.

No. ofDMUs MinH,

MaxH,

Mean H,

Std. deviation H,

Table 2: Results

Efficient II

1.029

Inefficient 52

0.292

0.976

0.651 0.209

7 The original data are available from the author on request.


The results show that all efficient DMUs achieve optimal values greater than unity,

whence each computed efficiency measure is stable. Moreover, we can conclude from

Remark 2 that the efficiency classification is stable as well.

To sum up the paper it can be stated that the BCC measure and the corresponding

efficiency classification are stable in most practical situations. The case where

discontinuities may occur can be detected by computing both the input oriented and

output oriented BCC measure. Additional information about the stability of the

efficiency classification can be obtained by the extended BCC measure.

Appendix:

Proofs of the theorems

Theorem 1 hk is stable at (X, Y) if and only if output-efficiency of DMU k implies input

efficiency.

Proof If there is an input i with X/= 0 then it follows from the first constraint that A-1 =

0 for every DMU l with X/> 0. We conclude that the optimal value of (1) is left

unaltered under small perturbations of nonvanishing data if we delete all these rows l

and input column i from the data matrix. Moreover, we can delete every output column

j with Y/ = 0 since the corresponding constraints in program ( 1) are redundant. Hence

we can assume without loss of generality that (.x*,Y") is strictly positive.

If DMU k is output-efficient but not input-efficient then we have max { ¢ I (X,¢ Y") E

T(X,Y)} = 1 and (X,Y") E T(X-k,Y-k) whence max{¢ I (X,¢ Y") E T(X-k,Y-k)} =I. For

every arbitrary small e> 0 it follows (X,(1+e)Y") it' T(x-k,y-k), thus for every feasible

solution of the corresponding program (1) with e > 0 we have Ak = I, i.e. the BCC

efficiency measure jumps to unity. This proves the "only if'.

To prove the "if'-part we consider two cases. First, suppose that there exists a

nonnegative vector A with Are = 1 and Y -k.A_ > Y". Note first that the solution set of

Contmuity of the BCC Efficiency Measure 77

program ( 1) is nonempty and bounded since (} is unique and A. is bounded by the

convexity constraint. In view of Robinson ( 1977), it remains to show Othat the

constraints of (1) are regular, i.e. there is a A.> 0 with A_Te = 1 such that all constraints

are strictly satisfied. By assumption there exists a nonnegative vector A. with A.re = 1

and y-k A.> Y". This inequality remains valid if we replace A. by (A.+ee)/(1 +ne) withe>

0 sufficiently small, therefore we assume A. > 0. Choosing (}sufficiently large the first

inequality is strictly satisfied as well.

Now we tum to the second case, i.e. for every nonnegative vector A. with A.re = 1 there

is an output} such that Yf"?.lj-kA.. It follows that max{¢ I (X,¢ Y") e T(X-k,Y-")} = 1,

i.e. DMU k is output-efficient and thus input-efficient by assumption. Now let (X,,Y,)

be a sequence of data matrices tending to (X. Y) and (A., f),) be a corresponding sequence

of solutions. Since the sequence is bounded we may consider convergent

subsequences. Assume that (A-,,8,) converges to some vector (A.,{}). Recall that the

solution set of ( 1) is nonempty and bounded whence we conclude applying Lemma 2

of Robinson (1977) that (A.,(}) is feasible for (1). Since DMU k is input-efficient the

optimal value of ( 1) is unity and thus (} = 1. This is true for every subsequence whence

for the complete sequence, i.e. hk is continuous.

Theorem 2 "Hk can jump only upwards". I.e., Hk is lower semicontinuous:

liminf HiX,,Y,) "?. Hk(X,Y). (X,.Y,)-+(X,Y)

Proof Starting from the dual program of ( 1) which is

maximize Y"q + m

s.t. X'p

Yq +w-Xp :s; 0

p,q ~ 0

the assertion follows analogously to Lemma 1 of Scheel and Scholtes ( 1998), setting m

=0.


References

Andersen, P. and N.C. Petersen (1993): A procedure for ranking efficient units in Data Envelopment

Analysis, Management Science 39, 1261-1264.

Banker, R. D., A. Charnes, and W. W. Cooper (1984): Some models for estimating technical and

scale inefficiencies in Data Envelopment Analysis, Management Science 30, 1078-1092.

Charnes, A., W. W. Cooper, and E. Rhodes (1978): Measuring the efficiency of decision making

units, European Journal of Operational Research 2, 429-444.

Charnes, A., W. W. Cooper, and R. M. Thrall (1986): Classifying and characterizing efficiencies and

inefficiencies in Data Envelopment Analysis, Operations Research Letters 5, 105-110.

Epstein, M. K. and J. C. Henderson (1989): Data Envelopment Analysis for managerial control and

diagnosis, Decision Sciences 20,90-119.

Fare, R. and C. A. K. Lovell (1978): Measuring the technical efficiency of production, Journal of

Economic Theory 19, 150-162.

Kuntz, L. and S. Scholtes (1996): Sensitivity of efficient technologies in Data Envelopment Analysis,

Technical report, University of Cambridge, Cambridge CB2 lPZ, England.

Robinson, S. M. (1977): A characterization of stability in linear programming, Operations Research

25, 435-447.

Russell, R. R. (1985): Measures of technical efficiency, Journal of Economic Theory 35, 109-126.

Russell, R. R. (1990): Continuity of measures of technical efficiency, Journal of Economic Theory

51,255-267.

Scheel, H. and S. Scholtes (1998): Stability ofDEA efficiency scores, Judge Institute of Management

Studies Working Paper Series 36/1998, University of Cambridge, Cambridge CB2 lPZ,

England.

Seiford, L. M. ( 1996): Data Envelopment Analysis: The evolution of the state of the art (1978-1995),

Journal of Productivity Analysis 7, 99-137.

Wilson, P. W. (1995): Detecting influential observations in Data Envelopment Analysis, The Journal

of Productivity Analysis 6, 27-45.


Wenbin Liu and John Sharp'

Abstract

In this paper, we investigate the relationship between Data Envelopment Analysis (DEA) and

Multiple Criteria Decision Making Theory. We re-examine DEA models from a goal

programming perspective. It has been shown in this work that many known DEA models and

new ones, can be derived via this approach.

As an illustrative application of the approach, the effectiveness of some antidepressant

pharmacotherapies is examined using one of the DEA models derived in this work.

1 Canterbury Business School, University of Kent, Canterbury, CT2, 7PE, UK

[email protected]

80 DEA Models via Goal Programming

Structure

Introduction

2 Goal Programming And Input-Output Systems

3 Merit Functions - Measures of Performance

4 DEA Models with General Goals

5 An Application

References

DEA Models via Goal Programming 81

1 Introduction

Data Envelopment Analysis (DEA) has become a standard non-parametric approach to

productivity analysis, especially to relative efficiency analysis of Decision Making

Units (DMUs). Since the introduction of the first DEA model CCR in 1978, it has been

widely used in efficiency analysis of many business and industry evaluation

procedures. Excellent literature survey can be found in, for instance [7] and [ 15].

Many DEA models exist that are designed to cope with various situations, such as the

CCR model [4], the BCC model [3], Additive model [5], and Cone Ratio model [6]

among the most well-known DEA models. Most of the fundamental DEA models are

derived from economic efficiency theory including Debreu-Farrell efficiency, Pareto

Koopmans efficiency, and more general technical efficiency axiomatic approaches (see

[8], and [13]). These models have then been modified to handled more complicated

applications in various ways.

In this paper we intend to demonstrate that many DEA models, known or new, can be

derived directly from goal programming. We have no intention of exhausting every

possibility, rather to show the essential ideas of the approach, through a small

illustrative application. There seems to be no similar work in the literature though

some relations between the DEA and multiple criteria decision making have been

noticed (see, for instance [ 17]).

2 Goal Programming And Input-Output Systems

In this section we very briefly introduce some basic concepts in multiple criteria

decision making theory and goal programming. We then examine input-output

systems from a point of view of goal programming, since they are the essential

elements of a DEA model.

Goal programming is a different way of seeking "good" solutions in multiple criteria

decision making processes. In this approach, instead of optimising multiple objective

functions, we set up a group of goals to be achieved. It may be impossible to achieve

all these goals simultaneously. Goal programming (GP) provides a mathematical tool

to investigate whether these goals can be simultaneously achieved, if not, to find some


compromise solutions. The details of conventional goal programming theory may be

found in, e.g., [10] and [II]. Here we are mainly concerned on whether these goals

have been set properly or can be further improved.

In the followings we very briefly recall some basic ideas in (GP). We first examine

goal setting.

To this end, it is very useful to recall an important concept in multiple criteria decision

making theory: preference. It can be viewed as an order relation, and is closely related

with goal setting.

Definition 2.1 Let Y be a set and let y,,y2 E Y. A preference or an order> on Y is a

subset of YxY denoted by {>}such that y, > y 2 iff (y,,y2 ) E {>} Similarly one can

define a preference or an order ::? , < and .5:

Normally we require that the preference is transitive, etc, see [10]- [11] for the details.

The most frequently used order m DEA IS Pareto preference. Let

X=(x,,x.),Y=(y.,,y.) e R•. Then m Pareto preference

X> Yiff X; ;::y; {i = 1,2,,n)and X"" Y. X <Yiff- X> - Y. In this paper, we assume the

Pareto preference is selected if not explicitly stated otherwise. However we have to

emphasise that other orders such as K-cone order and lexicographic order are also very

useful in DEA model building. The former can, for instance, lead to the well-known

Cone Ratio model ([6]) and the latter may let us build DEA models which are able to

express the preferences of the evaluators.

It is reasonable to assume that one has a preference selected before setting up goals.

Setting up goals for a particular application is by no means trivial, and has been

extensively studied in the literature, see, for instance, [10] and [11]. Here we only

examine some cases relevant to our later discussions. Let A = {..t,, ... , ..tJ' be the n

dimensional decision making variable and S be a constraint set of the variable. Assume

that we wish to maximise the quantity

where x; (i = l, .... ,n) are known constants. Instead of solving (a linear programming

problem)


n

max"' X·A.. .<eS f I I

we may set the goal to find ..1. = (i!,, ... , ..tJ such that

i,x;A; ~g 0 , AES, I

83

where g 0 is referred to as goal level. This type of goal will be referred to as positive

response type (PRT), as a higher level of the goal is desirable. Similarly one can set up

the following goal:

±x;A; ~g0 , A ES. I

This type of goal will be referred to as negative response type (NRT).

Similarly we can set multiple goals. In this work we only examine the following two

basic types of goals (and their mixtures):

n n

(L x:A; ~ g? , ... , l:x;"A; ~ g~), A E S (PRT) I I

or

<±xi A;~ g? , ... ,i,x;" A;~ g~), A E S. (NRT) I I

It is clear that one has to select a preference before setting any goals. Again we

emphasise that there are other types of goals very useful in DEA theory.

2.1 Input-Output Systems

In the following we shall apply the basic ideas introduced above to examine input

output systems which are fundamental to DEA theory. In the discussion to follow, it is

assumed that there are n decision-making units to be evaluated. Each DMU consumes

varying amounts of m-different inputs to produce s-different outputs. Let the m-


dimensional vector X 1 = (x11 ,x21 , ... ,x.,)' denote the inputs of DMUi, and the s

dimensional vector Y1 = (y11 ,y21 ,. .. ,y,)' denote the outputs of the DMUi. Let

X=(X,X2 , ... ,XJ be the input matrix and Y=(J';,Y2 ... ,Y,) be the output matrix. In

DEA, each of the DMUs is viewed as an input-output system with a goal pre-set by

evaluators. In a sense, DEA evaluates the efficiency of the DMU by finding whether

not this goal level (or its performance) can be further improved. This point will be

further expanded in Section 4. In the following we try to examine the DMUs

according to their desired input-output response.

Positive Response Systems (PRS)

In many applications, a DMU is expected to yield a higher level of outputs when its

input level is increased, if the unit is being operated efficiently. The efficiency of this

DMU is judged by how much more output can be produced by the unit. The higher the

extra output the unit yields, the more efficient it is considered to be. Such a DMU will

be referred to as a Positive Response System (PRS).

To be more precise we have to specify a preference. If the Pareto preference ts

selected, then a frequently used goal type associated with a (PRS) is

Unless explicitly stated otherwise, we in this paper shall always use this goal type with

a (PRS). Of course one may select other preferences according to the needs of an

individual application, and set up different types of goals for (PRS).

Negative Response Systems (NRS)

In some situations, an efficient DMU may be expected to yield a lower level of its

outputs when its input level is increased. An example of such outputs is the pollution


level of a factory, taking investment as the input. Such a DMU is referred to as a

Negative Response System (NRS).

A frequently used (Pareto) goal type associated with a (NRS) is

Unless explicitly stated otherwise, we in this paper shall always use this goal type with

a (NRS). Again one may select other preferences according to the needs of an

individual application.

It is clear that the type of an input-output-system does not depend on the sign of the

input or output data (i.e., whether they are positive or negative).

Mixed Response Systems (MRS)

In practical problems it is most likely that the desired input-output relations of the

DMUs show a mixture of both the above responses. For instance, it may be that the

outputs of the DMUs can be divided into two groups: outputl and output2, and the

desired input-outputl response is positive and the desired input-output2 response is

negative. Such a DMU is referred as to a Mixed Response System (MRS). For a power

station, it is more practical to consider total amount of electricity produced, total

profits, and overall pollution level as the outputs, instead of considering only the

pollution level. If one takes its staff numbers and investment as the inputs, then this

system is a mixed response system.

It is sometimes possible to transfer a NRS to, say, a PRS, by redefining the outputs

(e.g., by setting new-output =-old-output or new-output= 1/old-output, etc). However

such a transformation may completely change the nature of the original input-output

system (e.g., CRS, etc), so that the classic DEA models may not be suitable for the

new input-output systems - as no existing DEA models are fully translation invariant.

Not only the efficiency scores (see [I]) but also the classification as efficient or

inefficient (see [2]), if a non-convex technology is used, may be changed by such


translations. Hence one should use the original data whenever possible (see [12]).

Furthermore, it may be very difficult or impossible to transfer between other types of

goals and preferences.

Currently most of the existing DEA models only deal with positive response systems.

It is possible to modify them to suit other types of applications, see, e.g., [2] and [14].

It is the purpose of this work to address this issue in a more systematic way. We

approach this problem by re-examining DEA models from a goal programming

perspective. To this end we have to introduce more concepts from multiple criteria

decision making theory.

3 Merit Functions - Measures of Performance

It is natural and important to ask whether it is possible to express our preference over

the outcomes in terms of numbers so that the larger the number the better the

performance, and more importantly to quantify any extra achievement or performance

that is beyond the initial goals.

Suppose that we select a preference> on a set Y, A merit function m(.)is a function

from Y toR+ such that m(y1)>m(yJ if y 1>y2; that is, it is a (strictly) monotone

function on Y. We here only examine some merit functions closely related to DEA

models. It is therefore plausible to require them to satisfy some of the economic

efficiency axioms. We shall however not discuss this complex issue here (see, e.g. [9]).

Let us first look at the single objective case to illustrate the ideas. Let A= (Aw·· A,)'

be the n-dimensional vector of decision variables and S be a constraint set of these

variables.

We wish to find A= (A" ... ,A. )' to achieve the following (PRT) goal:

Ix;A; ~ g 0 ,A. E S, I

where g0 is referred to as the goal level. The goal may not be achievable. We shall

only examine the case where the goal can be achieved. When the goal is indeed

achievable, one wishes to know how much extra performance beyond the initial goal is


achievable and how to quantify this extra performance by using, for example, a

suitable merit function. One may wish to know what is the best goal level achievable,

as the current goal level may be set too loosely. It is clear that these questions are

closely related to DEA models.

In what follows we only consider those A, e S which are able to achieve the initial goal

level. For the particular goal set above, let s• = L~ x1A.1 - g 0 ~ 0 and let

where w > 0. Then m is clearly a merit function and the extra performance of a strategy

A. can be measured by m(A.) for this PRT goal. Therefore for the single objective PRT

goal, the highest achievable goal level can be found by solving

subject to

"'" 1 - + - 0 + > 0 1 s L..t X;A; S - g ,S _ , A E .

If the optimal solution is zero, then the current goal level is the best and no extra

performance can be possibly achieved with the current constraint. Otherwise, it is

possible to improve on the current goal. The highest achievable goal level is, g0 + s+,

and s+ is the extra performance beyond the initial goal when the highest goal is

achieved. This merit function is referred to as additive type, since

When the goal level g0 is positive, one can define a radial type merit function by


m(2) = w(},

where L~X;A; - s+ = ~0 ,(} ~ 0. or define an "almost" radial type merit function by

where we[O,l] is a small positive weight and L~X;A; -s+ =~0 ,s+ ~0,(}~0. We

define the almost radial merit function because the pure radial one is not strictly

monotone in the multiple objective case in the Pareto preference, but the almost one is.

We can easily define such a merit function for the case g 0 < 0 in a similar way, though

we shall not get into the details here.

Possible extra performance can then be found by solving

max (}

subject to

or

subject to

where E is a very small positive number. There are many more useful merit functions

though we shall not introduce them as here we only intend to illustrate the ideas. One

can similarly introduce merit functions for the NRT goals.


3.1 Multiple Objective Cases

The above discussions can be easily generated to multiple goals. Let us examine the

following two basic types:

n n

(Lx;A.; ~g~, ... ,L:xf"A.; ~g~), A.eS, (PRS) I I

or

n n

(Lx;A.; ~g~·····Lx;mA.; ~g~, A.eS. (NRS) I I

Possible extra performance beyond the goal level for a PRS goal may be found by

solving (using the additive merit function)

subject to

"'" 1 , _ + _ 0 "'" m ' _ + _ 0 + > 0 · - 1 L..IX1 A 1 S1 -gi•···•L.,IX;A; Sm-gm,Sj_ ,)- , ... ,m,

or if g,0 > 0 (using the almost radial merit function)

max ()+e(st + ... +s~),

subject to

for a NRS goal, by solving (using the additive merit function)


subject to

"'n I 1 - _ 0 "'n m 1 - _ 0 - > 0 · _ 1 L.,1X;A; +s1 - g 1 , ••• ,L.,1x; A; +sm- gm,sJ _ ,J- , ... ,m,

or if g~ > 0 and X; ~ 0 (using the almost radial merit function)

max 1-B+e(s) + ... s;;;),

subject to

where Eisa very small positive number. We mention again it is very easy to introduce

almost radial type of merit functions for the case g~ < 0 (i =I ,2, ... m). For instance, for

a NRT goal with g~ < 0, one can find the extra performance by using the following

problem (using the almost merit function):

maxB+&(s) + ... +s;;;),

subject to

There are many more possible merit functions can be introduced to suit a particular

application. For instance, one can easily introduce more non-radial types of merit

functions using the Fare-Lovell's (or Russell's) measure and Zieschang's measure

(see, [8] and [18]). We now examine the mixture of the above two types.


Mixed Goal Types

In general let

and

Then a possible mixture goal setting reads:

It follows from the merit functions introduced above that using the additive merit

function, the possible extra performance can be found by solving

max( S PR WPR + S NR WNR)

subject to

where s PR and s NR are nonnegative vectors and ( wPR, wNR) are weight vectors. One

can also use the almost radial merit function if g ~R, g ~R > 0 and finds the possible

extra performance by solving

subject to


where s PR and s NR are nonnegative vectors and 1.1 is the [I norm.

In the next section we shall derive some possible DEA models using the above ideas.

4 DEA Models with General Goals

We assume that there are n decision-making units to be evaluated. Each DMU uses m

different inputs to produce s-different outputs. Let the m-dimensional vector

X 1 = (x11 ,x,1, ... ,xmi)' denote the inputs of DMU1, and the s-dimensional vector

Y1 =(y11 ,y,1 , ... ,y,)'denote the outputs ofDMU1. Let X=(X,,X,, ... ,X.) be the input

matrix and Y = (Y,, Y, ... , r.) be the output matrix. It is reasonable to assume that the

goal type of this input-output system has been pre-set. If, for instance, one wishes to

minimise the input and maximise the output, then it is a positive response system. We

extend this input-output system to the following virtual input-output system:

n n

X(A.) =_LA.; X;, Y(A.) =_LA.; Y,, I I

where A.=(A,, ... ,A..)are the mixture vectors in a technology set Sto be specified in

applications. The set S should at least contain the unit vectors e" ... ,e •. These input

output units are referred to as virtual DMUs, or simply virtual units. Now assume that

we wish to evaluate efficiency of DMUi' One of the essential assumptions in DEA is

that if the initial goal level (X1,Y)cannot be further improved, or no extra

performance can be achieved, then DMU1 is efficient. Otherwise it is inefficient.

Assume that the input-output system is a PRS, for instance. Then the initial goal will

be


If this goal level is found to be the best in the sense that it cannot be improved further

in a merit function, then this DMU is considered to be efficient. Of course, this

procedure will depend on the preferences and merit functions used, and thus leads to

different DEA models, depending on how these are chosen.

As an illustration of this general approach, we shall derive a few DEA models from the

above procedure. We do not emphasise however the importance of the actual DEA

models to be derived, but rather the systematic way to derive them.

4.1 Additive Types

In this subsection we use additive merit functions. We make no assumption as to

whether the input or output date are positive or negative. We first examine the PRS

case. In this case the initial goal can be stated as to find 'A such that

Of course we know that this goal is indeed achievable by letting -1-1 = 1,-1-1 = 0 (i i' )).

Then our next step is to find whether the system is able to produce superior

performance, or improve the goal level.

Indeed if we can find a virtual unit which can produce superior performance, then this

DMU is considered to be inefficient by DEA. Otherwise it is considered to be efficient.

From the discussion about finding extra performance in Section 3, for a PRS we have

the following linear programming problem to find whether the j-th unit is efficient or

not: (ADD-PRSI)

max WJP Sfp + WPR SpR

subject to


where YA and XA. are the virtual inputs and outputs with A being in a selected

technology set S and w'P, wPR are nonnegative weight vectors. The values

'L:S;P and 'L:S~R identify the amounts of extra performance that the evaluated system

should be able to produce if ran efficiently. If they are zero, the j-th unit is classified to

be efficient by this DEA model. It follows that if a unit is efficient for a particular set

of non-zero weights (i.e. none of the weights is zero), then the unit is efficient for any

set of non-zero weights. Therefore we only consider non-zero weight sets here. If we

treat all the inputs and outputs equally then all the weights should be equal.

Equivalently we can take all weights to be one. Then the above is the well known

Additive Model of DEA if one assumes a convex technology set. If one makes the

weight wRP(w'R)very small, then one has the input (output) oriented Additive DEA

model.

Let us now examine the NRS case. Following the same argument as above, it is clear

that one can solve the following linear programming problem to find whether the j-th

unit is efficient or not.

max WIP S JP + WNR S NR

subject to

where again w1P and W'R are positive weight vectors. Then we have derived a DEA

model for the NRS case. It should be noted that here we make no assumption about

sign of the input or output data.

It is interesting to note that if one redefines the outputs as -Y that is, using negative

outputs if the original outputs are positive, then this DEA model become identical

with the PRS model obtained above. Therefore one can apply the Additive Model to

NRS provided one uses "negative" outputs.

Again one can have input or output oriented models as well. It is more complicated to

deal with the MRS case. Here we only examine the case where the output of j-th unit

can be decomposed into


with {PR},{NR} being fixed index sets independent of j, such that the input-output

system Xi ~ Yf" is a PRS, and Xi ~ rtR is a NRS. In such a case, let

(yPR J (yPR J

yj = ~NR 'y = y NR

Then the extra performance of the virtual units can be found by solving the following

linear programming problem: (ADD-MRS I)

max WIP SIP + WPR S PR + WNR S NR

subject to

where w 1P, wPR, wNR are again nonnegative weight vectors. The j-th unit is efficient if

and only if the maximum is zero. One again can have input or output oriented model.

4.2 Almost Radial Types or Combinations

In this section we use the almost radial type of merit functions or the combinations of

the additive and the radial. There are many possible combinations even for the case

with only two different kinds of merit functions. We shall not examine all of these

possibilities.


Again we first examine the PRS case. From the discussions in the last section,

assuming that we do not discriminate between input and output efficiency, we can find

possible extra performance by solving: (RAD-PRSI)

subject to

where 1.1 is the /1 norm, w1 (=) w2 are two positive weights and E is a small positive

number. In this model we have assumed that all the inputs and outputs are positive. If

one makes w1, w2 different, then one has oriented models. We can also use the

combinations of the additive and almost radial types of merit functions. For instance,

we can use an almost radial merit function for the outputs and an additive one for the

inputs. This is particularly useful for the oriented models. We only examine one

example here- an output oriented DEA model:

subject to

This is of course the output oriented CCR model. If we take S = R: and it is the output

oriented BCC model if we assume a convex technology set. We here only assume that

the outputs are positive. One can similarly derive the input oriented CCR and BCC

model.

Let us now examine the NRS case. Similarly we have the following two DEA models,

using the almost radial merit function for both inputs and outputs: (RAD-NRS 1)


max w, (1 - (}in) + Wz (1 -(}out) + e(ls IP I, + Is NR I,) subject to

or using the additive merit function for the outputs and the almost radial for the inputs:

(RAD-ADD-NRS 1)

subject to

where 1.1 is the /1 norm, w1 (=) w2 are two positive weights and E is a small positive

number. In the first model we have assumed that all the inputs and outputs are positive.

We only assume that the inputs are positive in the second. It is not difficult to write

down DEA models using the almost radial merit functions for the case where the

inputs or the outputs are negative (see e.g. [14]).

Finally let us examine the MRS case. Again we only examine the case where the

output of j-th unit can be decomposed into

with {PR}, {NR} being fixed index sets independent of j, such that the input-output

system X 1 ~ Yf" is a PRS, and X 1 ~ r;R is a NRS. For the sake of simplicity, we

only introduce one input oriented DEA model for this case where we use the almost


radial merit for the inputs and an additive one for the outputs. Then the extra

performance of the virtual units can be found by solving the following linear

programming problem: (RAD-ADD-MRSl)

subject to

Here we assume that the inputs are positive. This model will be used in the next

section.

5 An Application

As an illustrative application of the new DEA model (RAD-ADD-MRS1), we now

examine the effectiveness of antidepressant pharmacotherapies using amitriptyline,

imipramine, sertraline, and paroxetine respectively, from a cost saving perspective.

This has been studied via a decision tree method in [16], where the clinical statistics

have been summarised.

Using the DEA model (RAD-ADD-MRS1), we offer an alternative for evaluation of

the therapies. In this model, we use the (selected) costs of the therapies as the inputs.

The selected outputs are the probabilities of dropout, efficacy, and relapse, see [ 16] for

the details. These three outputs are used to measure the degree of success of the

therapies in [16]. The data can be found in Tables 3-4 in [16] and are summarised in

the following table.


Table 1: Summary of inputs (in Pounds) and outputs (in probability) of the DEA

model

Drug Costs Efficacy Dropout Relapse

Amitriptyline 47.16 0.55 0.27 0.22

Imipramine 59.16 0.65 0.27 0.22

Sertraline 138.72 0.70 0.19 0.25

Paroxetine 120.72 0.708 0.19 0.25

For an effective therapy, one would wish to minimise the probabilities of dropout and

relapse and to maximise that of efficacy. Although all the data are positive, this input

output system is actually a MRS. If we choose the efficacy as the first output, and the

dropout and relapse as the second and third output respectively, the sub input-output

system: input ---+output! is a PRS and the sub system: input ---+ (output2, output3) is a

NRS.

We have

and

X 1 = (47.16),X2 = (59,76),X3 = (139.72),X4 = (120.72).

Y1= (0.55, 0.27, 0.22)', Y2= (0.65, 0.27, 0.22)', Y3= (0.7, 0.19, 0.25)', Y4

= (0.7 0.19, 0.25)1

We now apply the model (RAD-ADD-MRSI). It is clear that we have

and

yPR = (0.55,0.65,0.7,0.708).


It is then possible to solve the following problem to find whether the j-th drug 1s

effective or not:

subject to

where we have taken S = {A: A; ~ 0 (i = 1,2,3,4); I~ A; = 1}; that is, the convex

technology set, or S ={A: A; ~ 0 (i = 1,2,3,4)}. The same efficiency results have

been found with these two technology sets.

The above programming problems are solved by a linear programming solver. It

appears that the therapies using amitriptyline, or inipramine, or paroxetine are efficient

and the therapy using sertraline has an 87% efficiency rate. This finding seems to

agree with the conclusion from a decision tree analysis in [16]: There seems no clear

cost argument demonstrated for switching between the therapies.

References

Ali, A. and L.M. Seiford ( 1990): Translation in variance in data envelopment analysis, Operations

Research Letters 10,403-405.

Allen, K. (1998): DEA in the ecological context - An overview, working paper in European

Symposium on Data Envelopment Analysis, Fachhochschule Harz, Wemigerode.

Banker, R.D. (1984): Chames, A. and W.W. Cooper, Some models for estimating technical and scales

inefficiencies in data envelopment analysis, Management Science 30, I 078-1092.

Chames, A., Cooper, W.W. and E. Rhodes (1978): Measuring the efficiency of decision making units,

European Journal of Operational Research 2, 429-222.


Charnes, A., Cooper, W.W., Golany, B., Seiford, L. and J. Stutz (1985): Foundations of data

envelopment analysis for Pareto-Koopmans efficient empirical production functions, Journal of

Econometrics 30, 91-107.

Charnes, A., Cooper, W.W., Wei, Q.L. and Z.M. Huhng (1989): Cone ratio data envelopment analysis

and multi-objective programming, Int. J. Systems. Sci. 20, I 099-1118.

Charnes, A., Cooper, W.W., Lewin, A.Y. and L.M. Seiford (1994): Data envelopment analysis,

Kluwer Academic Publishers, Dordrecht.

Fare, R. and C.A. Lovell (1978): Measuring the technical efficiency of production, Journal of


Ferrier, G.D., Kerstens, K and P.V. Vorden Eeckaut (1994): Radial and nonradial technical efficiency

measures on a data reference technology, Recherches Economiques de Louvain 60, 449-479.

Liu, P.L. (1985): Multiple Criteria Decision Making, Plenum, New York.

Nemhauser, G.L., Khan, A.R. and M.J. Todd (1989): Handbooks in O.R. and Management Sciences,

Vol. I, Chapter 10, North Holland.

Pastor, J.T. (1996): Translation invariance in data envelopment analysis, Annals of Operations

Research, Vol. 66,93-102.

Russell, R. (1988): On the axiomatic approach to the measurement of technical efficiency, in:

Eichhorn, W. (ed), Measurement in Economics, Heidelberg, Physica-Verlag, 207-217.

Scheel, H. (1998): Negative data and undesirable outputs in DEA, working paper in EURO Summer

Institute.

Seiford, L.M. ( 1996): Data envelopment analysis: evolution of the state-of-the-art (1978-1995),

Journal of Productivity Analysis 7, 99-137.

Stewart, A. (1994): Antidepressant pharmacotherapy: cost comparison of SSR!s and TCAs, British

Journal of Medical Economics 4, 67-79.

Stewart, T. (1996): Relationships between data envelopment analysis and multi-criteria decision

analysis, Journal of the Operational Research Society 47, 654-665.

Zieschang, K. (1984): An extended Farrell efficiency measure, Journal of Economic Theory 33, 387-

396.

Bounded vs. Unbounded Noise in Efficiency Estimation:

Performance of Alternative Estimators

Dieter Gstach1

Abstract

Whether we assume that noise, which might disturb the productio~ process, is unbounded or

bounded opens methodologically different possibilities of estimating a production frontier and

thus efficiency. The present paper provides simulation evidence about statistical properties of

technical efficiency estimators for multi-input, multi-output technologies under these two

possible assumptions: The Ray Frontier Approach (RFA) from Lothgren (1997) designed for

unbounded noise and DEA+ proposed in Gstach (1998), developed for bounded noise. RFA,

unlike earlier approaches in the realm of stochastic frontier analysis, is capable of efficiency

estimation in the case of multiple outputs as well and lends itself for comparison with DEA+.

Several settings with varying sample sizes, types of distributions and noise to signal ratios are

investigated.

1 Vienna University of Economics, Dept. VW 6, Augasse 2-6, 1090 Vienna, Austria

[email protected].

104 Bounded vs. Unbounded Noise in Efficiency Estimation ...

Structure

Introduction

2 Production Model and Data Generating Process

3 Simulation Settings

4 Results

5 Conclusions

References

Bounded vs. Unbounded Noise in Efficiency Estimation ... 105

1 Introduction

In this paper evidence from Monte-Carlo simulations is provided concerning some

statistical properties of two different approaches to technical efficiency estimation for

multiple-output production under noisy conditions. These approaches are the Ray

Frontier Approach (RFA) from Lothgren (1997) and DEA+ proposed in Gstach

( 1998). An alternative to these techniques is the distance function approach as

proposed in Grosskopf and Hayes (1993), Hetemiiki (1994) and Coelli and Perelman

(1996), but comparison here is confined to DEA+ and RFA.

Earlier work by Gstach (1997) yielded encouraging evidence about the small sample

properties of DEA+ estimators for the single output case and motivated extension of

research to the multiple output setting. Corresponding properties of RF A, which is a

multiple-output extension of the Aigner, Lovell and Schmidt (1977) and Meeusen and

Van den Broeck (1977) stochastic frontier analysis (SF A) approach, have to my

knowledge not yet been investigated. Because of this close link between SFA and

RF A, results from Monte Carlo studies about SFA performance (starting with Olson,

Schmidt and Waldman (1980)) are also relevant in the RFA context.

Both estimation techniques share two common properties: First they are based on the

assumption of proportional impact of noise and efficiency on all outputs. Secondly

they require parametrization of the error structure to distinguish between the noise and

the inefficiency component, unlike for example the panel data approach presented in

Kneip and Simar (1996).

The distinguishing features between RFA and DEA+ concern their respective domain

of noise and the way of moulding production. In case of RF A it is assumed that noise

is unbounded like in all stochastic frontier models. Furthermore RFA requires

parametrization of the production relationship. DEA+ on the other hand assumes noise

to be bounded, while making no assumption about production parameters, both points

reflecting the descent from DEA.

In some sense thus the two approaches compared here appear as competitors: This

would be the case if RFA were based on a flexible functional form (for example

translog) and if a normal distribution were accepted as proxy for a symmetrical,

unimodal and thin tailed and possibly bounded distribution. If, on the other hand, one

of these conditions is violated the two approaches should rather be considered as

complements, each with its own shortcomings and strengths.


The remainder of this paper is organized as follows: In Section 2 the production

structure and the data generating process will be defined. Section 3 describes the

various settings and steps of the Monte-Carlo-Simulations in detail. Exposition and

discussion of the results is contained in Section 4 and Section 5 concludes.

2 Production Model and Data Generating Process

Estimation of efficiency is based on observed outputs, denoted y, which must be

distinguished from often unobserved quasi deterministic outputs, denoted y . But only

the latter should be used to appreciate the performance of some decision making unit

(DMU). Based on this distinction output (production possibility) sets of a multiple

output technology can be correctly specified as

(I) Y(x) = {.Y e 9{~1 xcan produce y} with S > 1.

It is assumed that these sets satisfy strict convexity and monotonicity. The efficient

frontier in Farrells sense corresponding to input mix x is then defined as set

and the corresponding Farrell output-efficiency e of a point (x,y) as

(3) e(x,y) =max{.~. I A.ji e yF (x))}

Determining this efficiency thus amounts to find an output vector in Y(x) with output

proportions equal to those of y , which has maximum norm. But fixed output

proportions can also be guaranteed by keeping the angles 0, used in a polar-coordinate


representation of ji = I.YI m(B) fixed. The maximum norm one is looking for may thus be

defined as a function

(4) h(x,B) = max{r I r m(B) e Y(x)

of inputs and polar-coordinate angles of the output vector. This representation is used

in Lothgrens RF A approach, which gives rise to an alternative definition of the Farrell

efficient frontier, analogous to (2):

(5) yF (x) = {y e Y(x) I h(x,B) = I.YI}

and corresponding efficiency measure

( 6) c(x, ji) = I .Jill h(x, B)

With either (3) or (6) the efficient output level corresponding to production (x,ji) may

be defined as

(7) yF (x) = ji I c(x,ji)

Finding the efficient frontier is thus the crucial step in either approach to efficiency

measurement. In that respect DEA+ and RFA follow completely different routes. The

data generating process (DGP) motivating the use of DEA+ is assumed to consist of

density functions for noise v and inefficiency u and a function relating frontier outputs

to observed outputs as follows


The crucial assumptions of this DGP are thus a bounded noise term and proportional

errors. Because of this boundedness of the noise term equation reference to:

production has a natural deterministic equivalent:

A BCC type of DEA applied to {x" y1 };.1 will then yield estimates .V (x,) and

furthermore i$, of pseudo-frontier and pseudo error terms. The essential step ofDEA+

is then to estimate the parameters determining g,(.) (most importantly vmax and gu(.)

from the convoluted density of the { i$, t terms (with known domain [o, oo) !) via

maximization of the corresponding likelihood function. So the DEA+ estimate of the

true frontier is finally defined as

To motivate the use of RFA on the other hand, the DGP described above must be

modified with regard to noise. More precisely (8) must be replaced by


But with errors applying to all outputs equi-proportionally as defined in (10), frontier

estimation might be conducted via regression of the norms of observed output vectors

on values of x and () . This can be seen by combining the definitions of

m(h),h(x,B~/(x) andy from above, leading to the formulation IY11 = h(x1,B;)e'·-··.

Using a flexible translog function in the explaining parameters x1 and B1 as proxy for

the true functional form h(.) and ln~y1 1)-values as outputs, MLE is applied to this

regression equation to estimate the functional and the distributional parameters. This

leads to frontier point estimates

This procedure thus is an extension to the techniques investigated in Aigner, Lovell

and Schmidt (1977) and Meeusen and Van den Broeck (1977) as mentioned in the

introduction.

3 Simulation Settings

As common basis for the simulations conducted a technology with two inputs, two

outputs and decreasing returns to scale was chosen. Furthermore an input/output

separable form of this technology was assumed for ease of generating data (see for

example Kumbhakar (1996), where such functions are analyzed in an SFA context).

The output side exhibits constant elasticity of transformation while the input side is

modeled as a constant elasticity of substitution function. The specific relationship

between inputs, inefficiency, noise and outputs for DMU i then reads as follows:

( 15) ~0.5y 2 + 0.5y 2 = x0·2 x05 e'•-•• 1,1 1,2 1,1 1,2

All simulations are carried out on the basis of a fixed design of the input side,

characterized by xu values, where i = l... .. n and j = 1,2. This should be kept in mind,

when interpreting the results, as it reduces variability of the results compared to a


random design setting. The advantage of the fixed design is better comparability of the

results between the RFA and the DEA+ approach.

The xij values are fixed as a grid of order statistics. To construct the latter a normally

distributed random variable r with E(r) = 10 and STD(r) = 3 is defined first. The

sample size n was chosen to be the square of some integer. This then gives rise to

order statistics rk for k = I ... Fn with corresponding expected values denoted r,. Now

all possible variations of elements r, of order two with replacement yield an [n x 2]matrix, which I use as input-matrix. The first vector of this input-matrix is accordingly

given through k., x1,2 ) = (~, ~ ), while the last vector is (x,,., x • .J = ~~. ,r,J Thus I

have

With these fixed inputs and some random values for noise term v, and inefficiency

term u, the norm of the output of DMU is fully specified as

~ y~, + y~2 = J2 x~i2 x~i e'·-·· . This norm is then split by introducing a random

technology parameter a, to yield output values defined by

- ~ 0.2 o.s ,,-u; -~1 a 0.2 05 ,,-u, y 1 - a. x., x. 2 e , y. 2 - - . x. 1 x. 2 e . I, I /, I, I, I I, I,

These technology parameters are drawn independently from a [0,1]-truncated-normal

with

(17) E(a,) = 0.5, STD (a,)= 0.2

All stochastic models used are parametrized via mean inefficiency f.J. = E(u) and

noise-to-signal ratio 1J = VAR(v)/[VAR(v)+ VAR(u)]. The latter will simply be referred to

as noise in the sequel. In case of the bounded noise term underlying the DEA+ model


an extra parameter is needed to fully specify the two distributions involved (see

below).

For the inefficiency term u two different distributions are considered:

u- iid exponential with parameter B, u e [O,+oo ). The value of B is derived from given

f.J, via f.J. = 1/B, so VAR(u) = 1/82 = f.J:.

u- iid halfnormal with parameter a •. ue[O,+oo). The value of a, is derived from the

relationship f.J, = a • ..J2fi, while VAR(u) = (1 - 2/ 7r )a: = (1r /2- 1) f.J: .

The distribution of the bounded noise term v used in evaluating DEA+ performance is

v-iid Symmetrical Beta with Parameters a and vma,,ve[-vma,,+vmaJ. The first

coefficient is set to a = 2. This together with values for f.J, and 1J and the definition

VAR(v) = v!,)(2a + 1) then determines vmax = J1J(2a + 1)VAR(u)/(1-1J).

The distribution of the unbounded noise term $v$ used in evaluating RFA performance

IS

v- iid Normal with Parameter a,. The specific value of a, is derived from the

specifications of f.J, and 1J via a,= J VAR(u)/(1-1]).

I performed simulations for a fixed mean inefficiency of f.J, =0.2, sample sizes of

n=JOO, 225, 400, noise-to-signal-ratios of IJ= 0.1, 0.2, 0.4 and exponential and

halfnormal inefficiency distributions. Simulations for each RFA setting were

replicated 300 times and 200 times for each DEA+ setting. The extremely CPU time

consuming bootstrapping step used to bias-correct the naive DEA estimates ruled out

greater number of replications so far.

The performance of both approaches was measured with the following three statistics:


( 18) Bias of Mean inefficiency estimator: A, - f..l,

(19) Bias ofNoise estimator: r,, -1],

(20) Mean absolute Deviation (MAD) of frontier estimates: Elln(V (xJ)-ln(/ (x, ))I

For applied work the performance of the mean inefficiency estimator possibly is of

most interest. The other two statistics were added primarily for diagnostic purposes.

The results for these statistics should be compared in magnitude to the fixed mean 0.2

of inefficiency and the standard deviations of 0.2 resp 0.15 (for exponential resp.

halfnormal inefficiency, all on log-scale), as this sheds light on the identification task

of the estimators. Interpreting the results keep in mind, that due do the parametrization

chosen, overall variability V AR(w) changes across inefficiency specifications: The

composed error w = v-u in the halfnormal specification has only about 80% of the

variability compared to the exponential specification.

4 Results

Let's start with the case of exponential inefficiency and look at the results for Mean

Inefficiency Bias of the two approaches under consideration as given in Table 1.

Except for the most unfavorable n=100, noise=40% case, the permanently negative

bias ofRFA has magnitude less than 5% (-.01) in terms of the target figure of0.2 with

standard deviations across MC-trials of at most 15% (0.03) again in terms of target

value. I carried out simulations for settings with different population inefficiencies (0.1

and 0.4) as well with similar results, so I just documented the intermediate case of 0.2

true mean inefficiency.

In the case of bounded noise and consequent application of DEA+ matters deteriorate

slightly. Dropping like before the worst case ofn=100 with 40% noise the maximum

bias is roughly 10% with maximum standard deviations ranging up to 25% in the 40%

noise scenarios. For less noisy settings (up to 20%) DEA+ has maximum standard

deviations of around 15% like RF A. Convergence is rather poor, due to the boundary

estimation step involved in DEA+ , an observation in line with the findings m

Korostelev, Simar and Tsybakov (1995) or Park, Simar and Weiner (1997).


Looking more closely at these Monte-Carlo distributions and running Kolmogorov

tests with H 0 :jJ ""normally distributed I found that this hypothesis in only 3 of 18

cases considered in Table 1 could be rejected at significance levels of at least 10%.

Therefore the tabulated standard deviations may be used to construct approximate

confidence bands to test for difference in mean inefficiencies of two samples.

Noise estimation ofRFA for the smallest samples is negatively biased in all except one

(n=100, 77 = 0.1) case, thus generally attributing too much weight to the noise

component in total variability and too little weight to inefficiency (see Table 2). In a

much more pronounced fashion amounting to complete failure the same holds true for

DEA+ , which is mainly due to a highly overestimated shape-parameter of the Beta

distribution. Thus through the DEA+ lens noise appears as extremely peaked

distribution with long, thin tails as opposed to the true inverse U-shape as given by the

true shape-parameter value f3 = 2.

Standard deviations of the RF A noise estimators on the other hand are very large for

n=lOO, 77 < 0.2 compared to DEA+. Thus using mean squared errors as goodness-of-fit

criterion most of the advantage ofRFA with moderate noise and small samples is lost.

Only when noise may appropriately be modeled as normal and samples are at least of

size n=400, RFA may be used with more confidence. For illustration: The range

between the 10% and the 90% quantiles of the distribution of the RF A noise estimator

~, when actually 77 = 0.4, shrinks from [0.16,0.64] for n=1 00 to [0.30,0.50] for n=400.

In a paper by Coelli ( 1994) about the finite sample properties of stochastic production

functions a simpler model with a constant term but no regressors and normal noise,

halfnormal inefficiency distribution is investigated along the same lines. Coelli's

model is parametrized with r = 0'; /(0'; + 0';) instead of the noise parameter 77 used

here. Because of MLE's invariance property the 77 -estimates may be transformed via

r (1 -77 )/(!- 27] ") to yield parameter estimates comparable with Coelli's results. Table

7 contains the corresponding figures, all of which are expressed in percentage terms of

the true parameter values for r .

The standard deviations of the estimators are indeed of similar size, although Coelli's

results are more accurate owing to more replications (1000 vs. 300). The fact, that

standard deviations of the r -estimator are smaller for RF A than in Coelli's setting

when n=400, despite a noticeable loss in degrees of freedom ( 12 parameters here vs. 3

in Coelli) has to do with different 0' 2 = 0'; + 0'; values employed. The 10% noise case


here corresponds to o- 2 =0.0882, 20% to o- 2= 0.1199 and 40% to o- 2 = 0.2150. Coelli,

in the light of the invariance results of Olson et a!. (1980) keeps o- 2 constant at 0.25

and uses mean inefficiency values of around 0.4, depending on the y value chosen. So

my 40% case comes closest to his choice of a and this is nicely reflected in the

tabulated standard deviations.

Comparing Table 5 with Table 7 you will also note, that percentage bias of y for the

RF A model in terms of target values differs from the corresponding figures for TJ

roughly by factors between 0.2 (TJ=0.4, n=100) and 30 (TJ=O.l, n=400). This finding

of course is easily explained by different definitions of parameters, but it tells, that

information about statistical properties of y are of little if any use in trying to assess

the relative contributions of noise and inefficiency to total signal variability in applied

work.

Turning now to some overall measure of fit of the estimated frontier points, the mean

absolute deviations of a samples estimated frontier points from their true values is

informative (see Table 3). As the deviations are measured in log-scale, they may be

compared directly with the bias figures for the inefficiency estimator. Not surprisingly,

these mean absolute deviations are typically two to three times larger than the bias of

the inefficiency estimators, as the former focuses on point estimates. Again the

assumption of unbounded noise and thus application of RF A leads to better estimators

in terms of mean squared errors as compared to the bounded noise case.

The case of halfnormal inefficiency appears quite similar in terms of statistical

properties as a glance at the figures in Tables 4 - 6 reveals. Thus the general

observations made above for exponential inefficiency still hold. As far as mean

inefficiency is concerned, DEA+ and RFA again exhibit moderate differences

between. Mean squared errors (MSE) of the DEA+ estimator are smaller than the MSE

of RF A in three instances (n= 100, TJ 5, 0.2 and n=225, TJ = 0.1) and vice versa for the

other settings. Permanent underestimation of inefficiency via RF A is in line with

Coelli's findings.

Bias estimates for noise given in Table 5 on the other hand show, that Coelli's findings

(with equal error structure) of permanent underestimation ofthe y-parameter, which is

inversely related to the TJ -parameter used here, does not generalize to this setting with

multiple outputs. Increasing sample size in my case is accompanied by switching bias

from positive to negative, although at decreasing magnitude. This means that the noise


contribution to total variability is underestimated on average for larger sample sizes,

while it is constantly overestimated in Coelli's setting (see also Table 7 for the

corresponding r representation).

5 Conclusions

This paper investigated the finite sample properties of two basic models for technical

efficiency estimation with multiple outputs. These models are Lothgren's ray frontier

approach (RFA) and Gstach's adaptation of DEA to noisy settings (DEA+}, both

requiring a parametrization of the error structure to identify inefficiency contained in

noisy performance signals. But the approaches differ basically in their respective idea

of noise: RF A assumes unbounded (log-normal) noise, as has become standard in

stochastic frontier estimation. DEA+ on the other hand is developed under the

assumption of bounded (log-symmetrical-beta) noise, to exploit the non-parametric

structure of DEA.

For benchmarking I used estimators for mean sample inefficiency, for noise-to-signal

ratio and for mean absolute deviations of frontier point estimates from their true values

(MAD). The performance of these estimators was analyzed for a known technology

with two inputs and two outputs and known error structure in a Monte-Carlo

experiment with fixed mean inefficiency of 20%. Settings with different noise to

signal ratios (10%, 20% and 40%}, different sample sizes (100, 225, 400) and different

inefficiency distributions (exponential, halfnormal) were considered.

Mean inefficiency estimation with DEA+ for the smallest sample size (n=lOO) and

moderate noise ( ~ 0.2) has slight advantages in terms of mean squared errors

compared to RF A. But better convergence of RF A makes it a superior choice for

larger samples and more noise, given that the unbounded noise assumption is valid.

With n=400 the RFA mean inefficiency estimator has maximum bias (across noise

specifications and inefficiency distributions) of ~ 1% and maximum standard

deviation ~ 14% in terms of the single fixed target value of 0.2. Bias of DEA+ is

predominantly positive, while RF A on average underestimates inefficiency in all

settings.

Noise estimation with a sample size of n=!OO is a hopeless task in a statistical sense

for practical purposes with bias of up to 98% (DEA+) and standard deviations of up to


150% (RFA) in terms of (varying) target values. For sample size n=400 at least for

RFA matters improve somewhat, while DEA+ simply fails as far as noise estimation is

concerned. But even RF A noise estimators with n=400 have bias of up to 11% and

standard deviations of up to 36%.

Overall accuracy of DEA+ frontier estimates assessed via mean absolute deviations

from the frontier (log-scaled) is about as good as the corresponding RF A estimates for

n=100 and noise :5 20%. But RFA converges much faster leading to roughly twice the

accuracy of DEA+ for n=400. For noise=20% and n=100 RFA on average filters

around 80% of the error and DEA+ around 70%.

Further research will have to investigate the sensitivity of the two models to

inappropriate noise specifications, i.e. employing RF A when noise in fact is bounded

resp DEA+ when noise is unbounded. Analyzing the first possibility would indicate

the quality of the Normal approximation to other kinds of noise, while analyzing the

second would shed some light on the issue of DEA+ as sort of a multidimensional

order statistics estimator.

Appendix

DEA+ RFA

n-100 N-225 n=400 n-100 n-225 n-400

Noise 10%, Bias -.007 .010 .011 -.009 -.000 -.000

STD .022 .019 .015 .029 .018 .014

20%, Bias .019 .025 .013 -.003 -.000 -.000

STD .030 .029 .021 .035 .021 .017

40%, Bias .032 .021 .005 -.005 -.003 -.001

STD .064 .049 .025 .048 .031 .020

Table 1: Monte-Carlo Distribution of Mean Inefficiency Estimator for Log

Exponential Inefficiency with Mean 0.2.


DEA+ RFA

N-100 N -225 n-400 n-100 N -225 n-400

Noise 10%, Bias -.094 -.085 -.074 .020 -.015 -.009

STD .012 .017 .019 .114 .039 .031

20%, Bias -.178 -.144 -.097 -.018 -.014 -.007

SID .047 .061 .050 .131 .068 .056

40%, Bias -.226 -.136 -.073 -.027 -.006 -.004

STD .228 .153 .097 .186 .120 .080

Table 2: Monte-Carlo Distribution of Noise Estimator for Log-Exponential

Inefficiency with Mean 0.2.

DEA+ RFA

N-100 n-225 n=400 n-100 n-225 n=400

Noise 10%, MAD .053 .045 .038 .043 .023 .016 STD

.008 .006 .007 .015 .006 .005

20%, .069 .059 .047 .047 .029 .021

MAD STD .011 .014 .Oll .016 .008 .006

40%, .105 .076 .054 .062 .041 .029

MAD SID .027 .024 .009 .022 .012 .009

Table 3: Monte-Carlo Distribution of Frontier Estimators assessed via Mean

Absolute Deviations from true frontier points for Log-Exponential Inefficiency

with Mean 0.2.

DEA+ RFA

N-100 N-225 n-400 n-100 n-225 N-400

Noise 10%, Bias -.020 -.001 .005 -.019 -.002 -.000

STD .014 .Oll .009 .027 .014 .Oll

20%, Bias -.000 .018 .021 -.011 -.001 -.001

STD .016 .012 .014 .031 .019 .013

40%, Bias .027 .040 .035 -.016 -.003 -.000

STD .056 .036 .031 .059 .032 .022

Table 4: Monte-Carlo Distribution of Mean Inefficiency Estimator for Log

Halfnormal Inefficiency with Mean 0.2.


DEA+ RFA

N-100 n -225 n-400 n -100 n-225 n-400

Noise 10%, Bias -.098 -.096 -.090 .068 -.013 -.011

STD .003 .006 .012 .156 .061 .036

20%, Bias -.195 -.189 -.166 .014 -.016 -.010

STD .012 .022 .036 .161 .095 .059

40%, Bias -.311 -.301 -.246 -.005 -.014 -.019

STD .249 .164 .144 .245 .150 .105

Table 5: Monte-Carlo Distribution of Noise Estimator for Log-Halfnormal

Inefficiency with Mean 0.2.

DEA+ RFA

N-100 n-225 n-400 n-100 n-225 n-400

Noise 10%, MAD .054 .038 .034 .042 .022 .016 STD

.008 .005 .005 .017 .007 .005

20%, .057 .052 .047 .043 .026 .019

MAD STD .008 .007 .009 .016 .008 .006

40%, .091 .079 .067 .054 .037 .027

MAD sm .036 .019 .018 .024 .017 .011

Table 6: Monte-Carlo Distribution of Frontier Estimators assessed via Mean

Absolute Deviations from true frontier points for Log-Halfnormal Inefficiency

with Mean 0.2.

RFA Coelli (1994)

n-100 n-400 N-100 n-400

Noise -4.1 0.4 -0.7 -0.2

10%, BIAS 9.1 1.5 4.8 1.8

STD

20%, BIAS -2.0 0.4 -1.9 -0.6

STD 11.6 3.1 10.2 3.3

40%, BIAS -5.4 0.7 -8.9 -2.6

STD 29.1 9.5 28.8 10.2

Table 7: Monte-Carlo Distribution of Gamma Estimator y =a~ /(u~ + u;) for

Log-Halfnormal inefficiency with Mean 0.2. All figures in percentage terms of

true gamma values (=0.9612 for 10% noise, 0.9167 for 20% and 0.8050 for 40%).


References

Aigner, D.J., Lovell, C.A.K and P. Sclunidt (1977): Formulation and estimation of frontier production

function models, Journal of Econometrics, 6: 21-37.

Coelli, T. J. (1994): A monte carlo analysis of the stochastic frontier production function, Working

paper, Econometrics Department, University of New England.

Coelli, T. J. and S. Perelman (1996): Efficiency measurement, multiple-output technologies and

distance functions: With application to european railways, CREPP Discussion Paper 96/05,

University of Liege, Belgium.

Grosskopf, Shawna and K. J. Hayes (1993): Local public sector bureaucrats and their input choices,

Journal of Urban Economics, 33: 151-166.

Gstach, D. (1997): Using {DEA+ to estimate multiple output technologies: First Monte Carlo

evidence, Journal of the International Atlantic Economic Society, 3(4), Research Note.

Gstach, D. (1998): Another approach to data envelopment analysis in noisy environments: {DEA+,

Journal of Productivity Analysis, 9: 161-176.

Hetemaki, L. (1994): The impact of pollution control on firm production technology: A stochastic

distance function approach, in: Brlinnlund, R., Kristriim, B., Lofgren, K. G., and L. Mattson,

Environmental Economics, Swedish University of Agricultural Sciences, Department of Forest

Economics, Report No. I 06, Umea.

Kneip, A. and L. Simar (1996): A general framework for frontier estimation with panel data, Journal

of Productivity Analysis, 7(2/3): 187-212.

Korostelev, A., Simar, L. and A. Tsybakov (1995): Efficient estimation of monotone boundaries, The

Annals of Statistics, 23: 476-489.

Kumbhakar, S. C. (1996): Efficiency measurement with multiple outputs and multiple inputs, Journal

of Productivity Analysis, 7(2/3): 225-255,.

Liithgren, M. (1997): A multiple output stochastic ray frontier production model, Working Paper

Series in Economics and Finance 158, Stockholm School of Economics.

Meeusen, W. and J. van den Broeck (1977): Efficiency estimation from Cobb-Douglas production

functions with composed error, International Economic Review, 18: 435-444.

Olson, J.A., Sclunidt, P. and D.M. Waldman (1980): A Monte Carlo study of the stochastic frontier

production function, Journal of Econometrics, 13: 67-82.

Park, B.U., Simar, L. and C. Weiner (1997): FDH efficiency scores from a stochastic point of view,

Discussion Paper 9715, Universite Catholique de Louvain, Institut de Statistique.


Rolf Fiire', Shawna Grosskopf and Pontus Roos3

Abstract

The purpose of this paper is to outline theoretical and empirical guidelines for specifying and

estimating a price index for goods or services for which there is no market price available (or where those prices are administered or distorted or distorted). The case we are thinking of

include many public services such as publicly provided education and health services.

Obviously, one of the difficulties in constructing price indexes for nonmarketed goods is

finding a proxy for output prices to use in the price index. This we address by turning to shadow prices. We outline several theoretical alternatives, as well as providing some

practical approaches to estimation of shadow prices ofnonmarketed goods.

We also provide linear programming, or DEA, method of estimating the price index for

individual observations.

1 Department of Economics and Department of Agricultural and Resource Economics, Oregon State University, Corvallis, OR 97331-3612, USA

2 Department of Economics, Oregon State University, Corvallis, OR 97331-3612, USA

3 Institute for Health Economics (IHE), Box 2127,220 02 Lund, Sweden

122 Price Indexes for Nonmarketed Goods

Structure

Introduction

2 Output Price Indexes

3 Shadow Prices

4 Summary

References

Price Indexes for Nonmarketed Goods 123

1 Introduction

The purpose of this paper is to outline theoretical and empirical guidelines for

specifying and estimating a price index for goods or services for which a market price

is unavailable, administered, or distorted. This includes for instance, such services as

publicly provided education and health care. One may also think of nonmarketed

goods such as enviromental quality, though we do not address directly some of the

more complicated issues which arise in that context.4

The principal difficulty in constructing price indexes for nonmarketed goods is finding

a suitable proxy for output prices. This we address by turning to shadow prices. In this

paper we outline several theoretical approaches as well as providing some practical

empirical approaches to estimation of shadow prices of nonmarketed goods.

2 Output Price Indexes

We define our output price indexes in terms of production. In particular, we define the

indexes as ratios of revenue functions, which is in the spirit of Koniis (1924) 'true'

index of the cost of living. We also provide a linear programming or DEA5 method of

computing the index for individual observations.

More specifically, suppose we have k = I, ... ,K units of observations such as

government offices, schools or stores that supply a vector of services over which we

wish to compute price indexes. We show how these observations may be used in

calculating the revenue functions that enter into the price indexes. Since we are

especially interested in sectors in which outputs are not marketed, this also involves

identification of shadow prices for those outputs. Finally, using output shares, we

illustrate how the K individual price indexes can be aggregated into one index.

We begin with specification of the technology, which at time t, consists of the set of all

input vectors x' = (x; , .... x~ )e ~:and output vectors y' = (y; , .... y~ )e ~7 such that x' can produce y'. We denote this set by T', and we assume that it satisfies standard

4 For a discussion of some of these, see Fare and Grosskopf(l998).

5 The term DEA (Data Envelopment Analysis) was coined by Chames, Cooper and Rhodes (1978).


regularity conditions, such as those in, e.g., Fare and Primont (1995). Given output

prices6 p 1 = (p; , .... p'.. )e 9\~ we can define the revenue functions as

(1) R1 (x 1 ,p1 )= max{p 1y 1 : ~~ ,y1 )e T 1 } y'

This function is the maximal revenue that can be achieved at prices p 1 with the given

input vector x 1 • If prices change, say to p 1• 1 , another maximal revenue can be

calculated relative to technology T 1 with the same input vector x 1 , which we write as

R1 (x 1 , p 1• 1 ). The ratio of the two revenue functions is a Fisher and Shell (1992) output

price index.

Here, period t technology T 1 , and the input vector X1 are fixed while prices change

from p 1 to p 1• 1 • This output price index is the analog to the Kontis cost of living

index; both are defined as ratios of support functions.

The revenue function is homogeneous of degree + 1 in output prices, thus by Euler's

theorem we may write

(3) Rl(xl PI)= ~8R 1 (x 1 ,p 1 L 1

' L...J a I Pm' msl 'Pm

and

(4) M 8RI( I 1+1)

Rl(xl pi•')=" x ,px 1+'. , L..... a 1+1 Pm

m•l 'Pm

By Shephard's lemma (Shephard , 1970) the partial derivatives are the optimal

supplies of outputs, i.e.,

6 We address the issue of how to obtain these prices in Section 3 of the paper.


(5)

and

(6)

aR 1k.P 1L I (I 1\ ---'---'-'---'-Ym X ,p ;.m-l, ... ,M, ap~

aR I( I 1+1) X 'p - I ( I I+ I\ - 1 M --'--:-1+7-1--'- Ym X ,p ;.m- , ... , .

apm

125

The outputs y~ (x 1 , p 1 ) can be interpreted as the observed outputs at period t, but the

mixed period outputs y~ (x 1 , p,.1 ) in (6) are not observed. However, if the two vectors

of outputs are approximately the same or if we take y~(x',p')=y~(x',p,.1 ), then the

price index (2) is a Laspeyres output price index (Laspeyres, 1871)

(7) ""M t t+l

P'( I t t+l)- L.Jm•lYmPm X 'p 'p - "M I I

L..m•IYmPm

The price index (2) can be computed using linear programming or DEA methods. For

this purpose we assume that there are k = l, ... ,K observations of inputs

x'·k = (x~ 1 , •••• x~N) and output vectors y1·' = {y~ 1 , •••• y~). These data may be used to form

a reference technology, i.e.,

T' ={k.y'):

(8)

K

y~ ~2:z;y~,m=l, ... ,M, k•i

K

x: ~ 2:z~x~, n = l, ... ,N, k•l

z; ~ O,k = l, .... ,K}

The technology in (8) satisfies constant returns to scale, i.e., .lT1 = T', .l> 0 with strong

disposability of both inputs and outputs, i.e., for X1 ~ x' ,(x1 ,y') E T 1 then (x 1 ,y' )e T'

and .Y' ~ y' ,(x', y 1 ) e T' then (x', y') e T', respectively.


Now suppose that output prices are known for each k = 1, ... , K and t, • l,k ( I I ) 1.e.,p = PkJ•···oPw is a known vector of prices.7 Then for each k', the revenue

functions that form the price index in (2) may be estimated as the solutions to the

linear programming problems below.

K

s.t. y~ ;<;; :~:z;y~, m = l, ... M, k•l

K

x; ~ L:z;x~, n = I, ... N, k•l

z: ~ 0, k = l, .. .K.

R 1 (k' ,x*'.l ,p*'.l+l )= M

m.a~ LP;~~y~ Y ,z m=l

K

s.t. y~ :<> Z:z;y~, m = l, ... M, k·l

K

x; ~ Z:z;x~, n = I, ... N, k•l

z; ~ 0, k = l, ... K.

The results from (9) and (10) are used to obtain the output price index

P1(k,xk·1 ,pk·1 ,p*·,.') for each observation k=l, ... ,K. These individual price indexes

may be aggregated into a total index using revenue shares, where the share for k' is

defined by

(11) R 1(k' xk'.l p*-.1 )

s(k') = ' ' z::., R 1 (k, x*·1 , p*·1 )'

7 We shall relax this assumption in Section 3.

Pnce Indexes for Nonmarketed Goods 127

The overall index is then given by

K

(12) p 1 = :Ls(k')P'(k' ,xk',t ,pk',t ,pk',t+l ). k'•l

After some manipulation this index can be shown to take the simple form of

( 13) "K R' (k xk,t k,t+l)

P' = L.k., • ,p "K R' (k k,t k,t) ~k-=1 ,X ,p

Thus the overall output price index is the ratio of sums of revenue functions, one for

the period t+ 1 prices and one for the period t prices.

We note also that parallel to our discussion of the Laspeyres form of the output price

index, one may also specify a Paasche index, and therefore a Fisher type index.

3 Shadow Prices

In this section we summarize work done on identifying shadow prices for nonmarketed

outputs. This section borrows heavily from Fare and Grosskopf ( 1998).

Fare and Grosskopf (1990) developed a method for computing prices for firm inputs

based on Shephard's (1953) input distance function. Here, following Fare, Grosskopf,

Lovell and Yaisawamg (1993) and Fare and Primont (1995) we show how their

method can be used to compute shadow prices for firm outputs.

Denote inputs by x' = (x: , .... x~ )E 9l~ and outputs y' = (y: , .... y;, )E 9l~ .8 The output

distance function is defined by

(14) D.(x,y)=inf{B:(x,y/B)ET}

Where T is the technology given by

8 We suppress the time index for simplicity in this section.


(15) T = {(x,y): x can produce y}.

Fare and Primont (1995, p.22) show that T can be recovered from the output distance

function, i.e.,

(16) T = {(x,y):D.(x,y)s!},

if and only if outputs are weakly disposable in the sense of

(17) (x,y)e T and 0:58:51 => (x,~)e T.

Let p e 9i~ ,p * 0 be a vector of output prices and consider the revenue maximization

problem

R(x, p) =max py s.t. D. (x, y) :51. y

The Lagrangian problem associated with (18) is

(19) r(y,B) = py + B(l- D0 (x,y )~

and the first order conditions are

It can be shown, see e.g., Fare and Primont (1995, p. 54) that the optimal Lagrangian

equals maximum revenue, i.e.,

(21) B = R(x,p}

Combining expressions (19) and (20) proves that the output gradient of the output

distance function equals revenue- deflated output shadow prices, i.e.,

Equivalently, output shadow prices equal the output gradient of the output distance

function times maximal revenue, i.e.,


p = R(x,p)'\1 YD.(x,y}

For two different outputs m and m' it follows that their relative prices equals the

corresponding ratio of distance function derivatives,

(24) Pm· _ 8D.(x,y)/8ym' Pm - 8D.(x,y)/8ym

Thus, one may compute shadow prices of outputs using only data on input and output

quantities. These shadow prices are normalized by maximal revenue. 'Absolute'

output shadow prices may be computed in relative terms as above, or if at least one

observed output price is known and one is willing to assume that it equals its shadow

price, absolute prices may be retrieved following Fare, Grosskopf, Lovell and

Yaisawarng (1993). These shadow prices are derived based solely on production

technology (in conjunction with price information in the aforementioned case).

Since the production technology is often characterized in dual terms, typically the cost

function, one may ask whether shadow prices also be computed based on the cost

function. Let we 91~ denote input prices, then the cost function is defined as

(25) C(y, w)= m}n{wx: (x,y)e T}.

As is well- known from duality theory, the technology may be recovered from the cost

function.

From the profit maximization problem

max{py- C(y, w}} y

it follows that

p = '\1 yc{y, w),

i.e., output shadow prices are equal to their associated marginal costs. In contrast to the

distance function approach to shadow pricing, input prices rather than input quantities,

are the required data.

In a similar vein, one may also derive output shadow prices based on a cost indirect

output distance function

13 0 Price Indexes for Nonmarketed Goods

(28) ID.(w/c,y)=inf{B:0'eT,wx:>c},

where c represents the budget or target costs. Based on the duality between the cost

indirect output distance function and the cost indirect revenue function (see Fare and

Grosskopf, 1994),

(29) IR(wjc,p)= sup{py: ID.(w/c,y):> 1},

and specifying the Lagrangian and following the logic used with the output distance

function above we have

(30) p = IR(w/ c,p )'V YID. (w/c,y)

Thus, duality theory provides several alternative models which can be employed to

derive shadow prices of outputs. Implementation of these generally requires

specification of a (differentiable) functional form of technology, estimation and

derivation of the shadow prices based on the derivative properties derived above. This

is a approach taken in Fare, Grosskopf, Lovell and Yaisawarng (1993 ), among others.

Shephard (1970) provides an alternative approach that is closely related to the

approach proposed for the computing of our price indexes, namely a linear

programming approach. The general idea is to solve for maximal revenue, but instead

of solving for optimal output vectors given output prices, he solves for optimal output

prices given outputs and inputs. Specifically, for each observation k'= I, .. .K, solve

( k' ) k' H y ,w*. =maxpy p,w

M

(31) s.t. LPmYtm- L w,xkn :> 0, k = I, ... ,K,

w, ~ wk',n' n = l, ... N,

p ~ 0, w~ 0.

The solution vector gives the revenue maximizing shadow price of outputs given

inputs and outputs, p(x,y).


4 Summary

In summary, we have specified a producer based output price index as ratios of

revenue functions, which we propose for use in computing price indexes for

nonmarketed goods such as public services. We suggest computing these as solutions

to simple linear programming problems. The difficulty, of course, is identifying output

prices when the outputs or services are not marketed. We outline several ways in

which shadow prices of these services may be derived using duality theory. We also

discuss a model proposed by Shephard ( 1970) which solves for output shadow prices

in a linear programming framework. All of the approaches outlined here are applicable

to the multiple service/output case which characterizes, for example, most public

service providers.

References

Aigner, D. and Chu, S.F. (1968): On estimating the industry production function, American Economic

Review 58, 226-239.

Althin, R. (1995): Three essays on efficiency measurement, PhD dissertation, Lund, Sweden.

Charnes, A., Cooper, W.W., and Rhodes, E. (1978): Measuring the efficiency of decision making

units, European Journal of Operational Research, 2:6, November, 429-444.

Clement, J., Grosskopf, S. and Valdmanis, V. (1996): A comparison of shadow prices and

reimbursement rates of hospital services, Annals of Operations Research, 67, 163-182.

Fare, R. and Grosskopf, S. (1994): Cost and Revenue Constrained Production, Bilkent University

Lecture Series, New York: Springer- Verlag.

Fare, R. and Grosskopf, S. (1990): A distance function approach to price efficiency, Journal of Public

Economics, 43, 123-126.

Fare, R. and Grosskopf, S.( 1998): Shadow pricing of good and bad comodoities, American Journal of

Agricultural Economics, 43, forthcoming.

Fare, R. and Grosskopf, S. Lovell, C.A.K., and Yaisawarng, S. (1993): Derivation of shadow prices

for undesirable outputs: a distance function approach, The Review of Economics and Statistics,

374-380.


Fare, R. and Primont, D. ( 1995): Multi-Output production and duality: Theory and applications,

Boston: Kluwer Academic Publishers.

Fisher, F. and Shell, K. (1972): The pure theory of the national output deflator, in: F.M. Fisher and K.

Shell, The Economic Theory of Price Indices, New York: Academic Press, 49-113.

Grosskopf, S. and Hayes, K. ( 1993): Local public sector bureaucrats and their input choices, Journal of

Urban Economics, 33, 151-166.

Grosskopf, S., Hayes, K., Taylor, L., and Weber, W. (1997): Budget- Constrained Frontier Measures

of Fiscal Equality and Efficiency in Schooling, Review of Economics and Statistics, 79: I, 116-

124.

Konils, A.A. (1924): The problem of the True Index of the Cost of Living, Translated Econometrica,

7 (1939), 10-29.

Laspeyres, E. (1871): Die Berechnung einer mittleren Warenpreissteigerung, Jahrbilcher fUr

Nationalokonomie und Statistik, 16,296-314.

Paasche, H. (1874): Ober die Preisentwicklung der letzen Jahre nach den Hamburger

Borsennotirungen, Jarbilcher fur Nationalokonomie und Statistik, 23, 168-178.

Shephard, R. W. (1953): Cost and Production Functions, Princeton University Press.

Shephard, R. W. ( 1970): Theory of Cost and Production Functions, Princeton University Press.

Lessons Learned for DEA Practice from Health Care

Applications in the UK

Rob Ball (Head of Department), Elizabeth Roberts (Doctoral Research Student) and

Richard Thomas 1

Abstract

The literature on DEA shows limited practical application to public sector services in the UK.

We have applied DEA to compare the efficiency and performance of a number of hospitals in

Scotland. This has given us insight on a number of key issues related to the application of

DEA, which have been further extended through discussions with a variety of managers in the

field. This paper discusses these issues, including the introduction of weight limitations,

issues of comparability of DMUs, the robustness of the models and the sensitivity of the

results, illustrating them using real data from Scotland.

1 Department of Management and Organization, University of Stirling, Stirling, FK9 4LA, UK

Robert Ball: [email protected]

Elizabeth Roberts:

Richard Thomas:

[email protected]

[email protected]

134 Lessons Learned for DEA Practice from Health Care Applications in the UK

Structure

Introduction

2 DEA Modelling in a Health Service Context

3 Development of the Sample ofDMUs

4 Choosing the DEA Model

5 Identification of Inputs and Outputs

6 Results from the Basic Model

7 Introduction of Weight Restrictions

8 Evaluation of the Approach

Appendix 1

Appendix 2

References

Lessons Learned for DEA Practice from Health Care Applications in the UK 135

1 Introduction

Technical and theoretical developments to the DEA methodology have been widely

discussed over recent years, following its instigation by Charnes, Cooper and Rhodes

(1978). However, it has been observed that, despite the fact that DEA appears to be

ideally suited to the measurement of efficiency in health care organizations, there are

few examples to be found showing DEA being used as a practical tool for efficiency

assessment by managers. There are only a handful of papers relating its application to

the provision of health services in the UK, although there are a greater number using

data from the United States. In many cases, the DEA literature relating to health care

organizations appears to use the health data to illustrate facets of the DEA technique,

rather than using the DEA technique to investigate health care data, such as the

situation commented on by McMillan (1987).

Coincidentally, the search for adequate and appropriate methods for the measurement

of efficiency and performance has become a 'central preoccupation for public sector

organizations' (Flynn, 1986). The potential reasons for this development are

numerous:

1. There are now severe restrictions on the level of public finance

available and constraints have been placed on all areas of spending by

government;

2. Public sector services have been perceived to be inherently inefficient

and in need of new measures to target these inefficiencies (Mooney

and Ludbrook, 1984);

3. The role of management within the NHS has changed, giving more

emphasis to the aim of improving the efficiency with which resources

are used (Levitt and Joyce, 1987);

4. Government agencies, such as the Audit Commission, have adopted

the pursuit of efficiency as one of their primary responsibilities.

Consultation with health care managers, and the presentation of a discussion paper at

the Young OR Conference in the UK, has enabled the authors to gain an insight into

the suitability of the DEA technique for addressing the performance assessment


requirements of the Health Services in the UK. This paper explores these issues using

data from the NHS in Scotland.

2 DEA Modelling in a Health Service Context

If DEA is to be used to successfully examine the efficiency of a group of hospitals, or

other health care organizations, there are several key stages to the modelling process.

The following diagram, adapted from Golany and Roll ( 1989), has been proposed as a

guide as to what these different stages should be:

1 Define Population ofDMUs J ~

I Set Goals for Analysis I ~

.I Select DMUs to be Compared ~~;::.:::-------1 ·1 ~ I

~ ! ' ; i

List Relevant Factors I '

l j i

~ I I I Set Factors' Examine Factors I

I ... J

Measurement Scales (Judgement)

I

l l y Define Production Examine Factors II I Relationships (Correlations) I

+ • I

Formalise Initial

I I Examine Factors III

1-Model (Trial Runs) --- --· ------------·

I l Formalise Final Model I

l Computer Environment I Present Initial Results I

I ~ ~ + I Analyse by Factor I I General Conclusions I

and Special Analyses I Analyse by I

Individual DMU

Figure 1: An Application of Data Envelopment Analysis to Health Care Data


Whilst the above diagram is useful as an indicator of the complexity of the DEA

modelling process, a more simplified structure will be used here, with the key stages as

follows:

1. Definition of the Sample ofDMUs;

2. Formulation of DEA Model Type and Orientation;

3. Selection of Variables to be used as Input and Output Factors;

4. Results of the Preliminary Analysis;

5. Revisions to the Model - Sample Size, Variables Included, Addition

of Weight Restrictions;

6. Results from the Revised Model;

7. Evaluation of Results- Sensitivity and Robustness.

3 Development of the Sample of DMUs

As was referred to above, the investigation presented is based on the analysis of

data from the NHS in Scotland. However, this is an extremely complex

organization, as can be seen in the diagram below from the Scottish Office

(1998), depicting its organizational structure:


Health Boards X 15

Figure 2: The NHS in Scotland (1998)

- Policy

- Performance Management

- Local Strategy

- Monitoring of Implementation

There are many levels of decision-making in the Scottish NHS and analyses using

DEA could be carried out at a variety of levels, such as, investigating the relative

efficiency of the health boards, of which there are fifteen. However, given the previous

experience of the authors and the availability of the relevant data, the decision was

taken to focus on the analysis of individual hospitals. This is the main point at which

services are delivered. This also reflects the current attention given to this by the UK

Government and the Department of Health.


The majority of hospital-based activity in Scotland is managed through acute hospital

trusts, as referred to in figure 2. (Some trusts concentrate on specific areas of health care

such as community, mental health and dental services, or provide only one type of

service such as outpatient facilities, and are not included here.) The majority of the acute

trusts provide a cross section of services at the hospital sites under their control, with the

size and scope of the trusts varying quite considerably.

According to the viewpoints of the health care managers interviewed, there are many

different types of trust, providing services at a number of hospital sites. Therefore, to use

all the trusts in a single DEA sample would not be appropriate or useful.

Therefore, the level of service provision to be used for comparison will be the hospital

level, one level down from the trusts in the organizational structure. However, it is still

necessary to reduce the sample still further, as there are over 200 hospitals in Scotland,

which also vary markedly in size and the services they offer. The approaches in the

literature show how the sample of all hospitals can be reduced according to some

specified characteristics. For example, Byrnes and Valdmanis (1995) focused on

community hospitals, Sherman (1984) used a sample of teaching hospitals and Grosskopf

and Valdmanis (1987) used large, urban hospitals.

These types of study, however, developed their sample from very large groupings of

hospitals. With Scottish data, refining the sample according to very strict characteristics,

such as location, size and type, would result in a large number of very small samples.

There are only a handful of hospitals in each area, which are often of different types, such

as community, acute, teaching, children's, maternity and geriatric. The hospitals in

Scotland can actually be divided into some 49 categories, called functional

classifications, and defined by the ISD (1996). It is necessary to make some

generalisations in order to obtain an appropriate sample with an adequate number of

DMUs.

Therefore, following the approach of Parkin and Hollingsworth (1996) and Ozcan et a!.

( 1992}, the sample chosen for the evaluation consisted of those hospitals that are

classified as 'acute', of which there are 74 in Scotland. In terms of sample size, the figure

of 74 DMUs is consistent with the sample of California hospitals used by Grosskopf and

Valdmanis (1987). These 'acute' hospitals cover 15 functional classifications and the

hospitals in these classes can be divided into four broad categories:


1. General Hospitals - may have a teaching element and a maternity

department, will cover the majority of surgical functions but not

necessarily all of them and may vary in size quite considerably;

2. GP Practitioner Cottage Hospitals - will have a limited surgical

function, may have a maternity department but no teaching element;

3. Mixed Specialist Hospital - may have a maternity department, will

cover the basic surgical functions but not highly specialised treatment;

4. Specialist Children's Hospitals - may have teaching element, will

cover complete range of paediatric services.

The initial sample of 74 DMUs, the acute hospitals in Scotland, on which the primary

stages of the analysis were carried out (and referred to in Roberts and Ball, 1997) was

found to have several limitations. These could seriously affect the usefulness of the

results obtained and their acceptability in a health care environment. The widely held

belief within many hospitals, which was stressed by two of the health care managers

interviewed, was that each hospital is 'unique' in its operation, facilities and

environment. This is to such an extent that they believe it cannot be sensibly compared

with any other hospital, unless they can be convinced that the other hospital operates

under the same or very similar constraints.

Analysis of the actual data relating to this sample also identified a further problem in

terms of the large variations in the level of services offered by each hospital. Many of

the hospitals did not over the complete range of services included as output factors.

Therefore, the output data in particular was found to contain many zero values, which

could potentially have a distorting effect on the calculation of efficiency scores using

DEA.

In particular, several DMUs were identified with unusual and even unique output

patterns, which reflected the concerns of the health service managers. To counteract

this, the sample was reduced to 47 DMUs, which were, therefore, a more acceptable

set of DMUs in terms of their homogeneity, based on analysis of their functional

classifications. Further revisions to the sample of DMUs, and some alternative

approaches to the measurement of hospital-based activity, have also been considered


and will be debated following the identification of appropriate input and output factors

and the development of an initial DEA model.

4 Choosing the DEA Model

In the majority of investigations into health service organizations, the original DEA

model developed by Charnes, Cooper and Rhodes (1978) and denoted by CCR has

been used for the analysis of efficiency. (See Hollingsworth and Parkin (1995),

Sherman (1984) and Ehreth (1994).) Thus, the majority of models have been

developed with the assumption of constant returns to scale (CRS) and the free

allocation of weights. Parkin and Hollingsworth ( 1996) introduced the idea of

examining the efficiency of hospitals, whilst taking account of varying returns to scale

(VRS). However, the analysis here, will employ the CCR model for two important

reasons:

1. As mentioned above, most models used by previous authors to

measure hospital efficiency have used the CCR model, probably

because it is the simplest form of the DEA methodology, being easier

to understand and interpret.

2. The data sample employed here, which consists of 'acute' hospitals,

can be said to be homogeneous, meaning the constant returns to scale

assumption should still be valid (Smith and Mayston, 1987).

In addition to the choice of model, it is also necessary to determine its orientation, be it

input minimisation or output maximisation. In this case, the input minimisation model

will be utilised, since most of the thinking in the health care sector has reflected a wish

to minimise costs without reducing services. Figure 3 represents the DEA model to be

solved in the following analysis, expressed in ratio form.


m

MINIMISE: ea = I V d i a , i= 1

m S

SUBJECT TO: I V d i} - I UrOrJ '2:. 0, j = l.. .. a .... n i= I

s

AND: I U rOra = 1 r=l

U, and V; > E FOR ALL rAND i (E being a very small positive number)

s = number of outputs

m = number of inputs

Figure 3: Input Minimisation Orientation of the CCR Model

5 Identification of Inputs and Outputs

The selection of the variables is an extremely important element of the modelling

process - if inappropriate variables are chosen, the results of the analysis will yield no

benefit. In Scottish Acute Hospitals, the cases can be divided into six main categories:

inpatients, consultant outpatients, PAM (Professions Allied to Medicine) outpatients,

day cases, day patients and Accident and Emergency attendances.

The first step in the selection of variables is the identification of all possible factors

and those for which for the data is not available must then be eliminated from the

model. In this analysis, a potential input factor would be the 'total number of trained,

learning and other nurses', as used by Hoiiingsworth and Parkin (1995).

Unfortunately, such data was not available. Nevertheless, there are a large number of

inputs and output factors for which the data was available that could therefore be

employed, as shown in Table 1.

Lessons Learned for DEA Practice from Health Care Applications in the UK

Inputs 'Average No. of Staffed Beds' 'Capital Charge' 'Total Costs' 'Total Direct

Costs', 'Total Allocated Costs' 'Total Inpatient Expenditure' 'Total Outpatient

(8) Expenditure' 'Total Daycase Expenditure'

Outputs 'Total Inpatient Discharges' 'Inpatient Discharges, Surgical' 'Inpatient

(30) Discharges, ITU' 'Inpatient Discharges, Medical' 'Inpatient Discharges,

Obstetrics and Gynaecology' 'Inpatient Discharges, Other' 'Total Inpatient

Days' 'Inpatient Days, Surgical', 'Inpatient Days, Medical' 'Inpatient Days,

Obs and Gynae' 'Inpatient Days, Other' 'Inpatient Days, ITU' 'Consultant

Outpatients Attendances, Other including ITU' 'Consultant Outpatients

Attendances, Surgical' 'Consultant Outpatients Attendances, Medical' 'Total

Consultant Outpatient Attendances' 'Total New Consultant Outpatient

Attendances' 'Total A&E Attendances' 'Total New A&E Attendances' 'Total

Daycase Attendances' 'Daycase Attendances, Medical' 'Daycase Attendances,

Surgical' 'Daycase Attendances, Other' 'Total Daypatient Attendances'

'Daypatient Attendances, Medical' 'Daypatient Attendances, Other including

Mental Health' 'Total PAM Outpatient Attendances' 'Total New PAM

Outpatient Attendances' 'Total Outpatient Attendances, A&E, Consultant and

PAM'

Table 1: Potential Variables

143

A case could be made for the inclusion of all of the above, based on their previous use

in the DEA literature, the fact that they intuitively describe some element of hospital

performance or they are acceptable to health care experts having been used in official

internal investigations. Nevertheless, to employ all 38 factors would be inappropriate

and unnecessary. It is therefore essential to reduce the list to more manageable

proportions and several approaches have been identified to narrow down the list of

potential variables. These are expert advice, previous use in the literature, evaluation

and manipulation of the data using statistical methods and, finally, heuristics. Each of

these is discussed below.

(i) Opinion of Experts: It is our belief that this should be an essential part of the

model building process. A member of the NHS Executive with whom the issues

surrounding DEA application were discussed suggested that 'inpatient discharges'

should always be used as an output in place of 'inpatient days', which is not seen as a

reflector of efficiency. Support was also expressed for disaggregating inpatient activity

factors, as 'inpatients' may be seen as too broad a category of hospital activity. The


importance of including some measure of day case activity was also stressed. In terms

of inputs, apart from favouring financial measures, the possible redundancy of the

'average staffed beds' was highlighted as this is covered to a certain degree by 'capital

charge'.

(ii) Previous Use in the Literature: The use of 'inpatient discharges' rather than

'inpatient days', as proposed by the experts, is supported by the literature (See Ehreth,

1994). Different levels of disaggregation of the inpatient factor have been identified,

such as, the three categories (acute, ICU and surgical) used by Grosskopf and

Valdmanis (1987) and the four categories (surgical, medical, obstetrics and others)

used by Parkin and Hollingsworth (1995). Outpatient discharges and A&E attendances

have also frequently been used as output factors in the literature. Turning to the input

factors, evidence from the literature suggests that these should reflect three main

characteristics, to be measured in a number of ways:

1. Hospital Size - number of staffed beds, net plant assets, number of

admissions, fixed assets, bed days available.

2. Staffing Levels - number of full time employees, number of staff in

each category, payroll expenditure, direct salary costs.

3. Supply Expenses - operational costs, total value of supplies, cost of

drug supplies.

(iii) Evaluation of the Data: Disaggregation of the output factor 'inpatient discharges'

was not practical, as many of the DMUs do not have data entries for all categories.

There were several zero or missing values for some of the output categories,

particularly 'day patients' and 'PAM outpatients', as these services were not provided

at all of the hospitals. Statistical analysis concluded that there were several pairs of

highly correlated output factors. The policy of removing one of each pair of highly

correlated factors is not however as generally accepted in DEA as it is in regression (as

discussed by Nunamaker, 1985).

A further statistical technique is the use of regression to establish that the factors to be

included are related technical efficiency, rather than being arbitrary measures of input

and output. Golany and Roll (1989) advocated regression as a means of 'eliminating


redundancies and reducing the list of factors', to be used not as 'reliable rules but

merely as indicators for a need to examine some of the factors more closely.' Using a

multiple regression procedure, the key output factors were identified as the measures

of inpatient, day case and outpatient activity.

(iv) Heuristics: The final approach is to run several alternative models and then analyse

the results to determine which groupings of inputs and outputs best describe the health

care situation being investigated. This is an approach frequently found in the literature,

notably Ehreth (1994).

Taking on board the important factors identified above, and trying several DEA models,

the mix of variables chosen for this DEA application were determined to be:

INPUTS: 'total direct costs' (TDC), 'total allocated costs' (T AC) and

'capital charge' (CAP).

OUTPUTS: 'total inpatient discharges' (TID), 'total consultant

outpatient attendances' (COA), 'total accident and emergency

attendances' (AEA) and 'total day case attendances' (DCA).

6 Results from the Basic Model

Following the selection of the variables and the DEA model type, the results from the

DEA model can be obtained; in this case specially written DEA software package

developed at the University of Stirling was utilised.

The model selected, as we have seen from the previous section, had four outputs ('total

inpatient discharges', 'total consultant outpatient attendances', 'total accident and

emergency attendances' and 'total day case attendances') and three inputs ('capital

charge', 'total direct costs' and 'total allocated costs'). The simplest form of the DEA

model, the CCR model with an input minimisation orientation, was used and the

results from this model were investigated without employing weight restrictions.

Of the 47 DMUs in the sample, 20 were rated as 100% efficient, with the mean

efficiency score being 87.98%, with a standard deviation of 15.08. A full breakdown

of the efficiency analysis is given in appendix 1. Further to this, the results from the

model can be analysed in numerous ways, such as, focusing on the least efficient


DMUs, in order to attempt to categorise the characteristics of inefficiency. Table 2

summarises the DEA results for the five least efficient units and it can be seen that

each of them is dominated by the contribution of the 'total inpatient discharges' factor.

DMU Efficiency Non-zero Dominant Dominant RatioofTDC Score Output Input Factor Output Factor toTAC

Categories (Virtual) (Virtual)

#40 66.0 COA TDC TID 3:2

#34 53.8 COA TDC TID 3:2

#20 52.0 COA,AEA TDC TID 7:4

#41 51.4 COA,AEA TDC TID 5:4

#25 45.2 COA,AEA, TAC TID 4:1 DCA

Table 2: Analysis of the Inefficient DMUs

Similarly, the efficient DMUs can also be analysed. 12 of the 20 efficient DMUs have

more than 300 staffed beds but the DMU ranked as number 1 in the overall list of

efficiency scores is one of the smaller hospitals, with 102 'average staffed beds'. (The

ranking of efficient DMUs is based on the number of occasions they appear in the

reference groups of the inefficient units). Also, 4 of the DMUs are included in the

reference set for 10 or more of the inefficient units (#30, #12, #13 and #14). These all

have 'total direct costs' as the dominant input factor and either 'total inpatient

discharges' or 'total day case attendances' as the dominant output factor (based on

virtual input and output values). The efficient DMUs can be further analysed by

refinements to the DEA model, such as super-efficiency and the development of cross

efficiency matrices.

Inefficient DMUs can be analysed in tum to determine areas for potential

improvement, which is useful for health care managers. For example, DMU #1 has an

efficiency rating of 95.23% and could achieve a 100% efficiency score by reducing

'total direct costs' from £59, 679k to £56, 833k, that is, a decrease of 3 million pounds.

Alternatively, the results from the model can be analysed in more general terms,

primarily in terms of the allocation of the factor weights. Table 3 summarises the

contributions made by each of the factors, according to the virtual weights calculated

for each of them, in order to identify the most influential of the inputs and outputs.

Values have been calculated by excluding the zero values and hence the means do not

sum to 100 for the inputs and outputs, as might be expected. It shows that the 'total


direct costs' factor is found to be the dominant input, contributing most to the

efficiency score for 43 of the 47 hospitals.

This is confirmed by the mean value for the TDC virtual input being 89.7%. For the

outputs, it is 'total inpatient discharges' that appears to be the most dominant factor, as

it contributes most to the efficiency scores for over half of the DMUs.

CAP TDC TAC COA AEA TID DCA

mean 9.88333 89.7093 35.6 46.86 28.8611 59.6436 21.9742

stdev 6.85766 11.4837 37.135 24.8859 32.9552 33.282 23.5668

min 1.1 57.7 2.4 12.7 0.3 0.2 0.7

max 32.5 100 100 99.3 92.1 100 100

count 24 43 17 25 18 39 31

dom 0 43 4 11 5 25 6

KEY: Mean/StDev/Max/Min: calculated by excluding zero values for virtual weights. Count: Number of times input/output factor is used in efficiency calculations. Dom: Count of the number of times that factors contributes most to the efficiency score calculation. Factors codes as defined above.

Table 3: Analysis of the Virtual Weights

Table 4 illustrates the distribution of the numbers of factors involved in the efficiency

calculations, showing that none of the efficiency scores have been calculated using all

seven factors - for every DMU, at least one factor has been given a virtual weighting

of zero.

I No. of Factors 2 3 4 5 6 7

Count 3 8 17 15 4 0

Table 4: Distribution of Factors Used to Calculate Efficiency Scores.

Taking tables 3 and 4 in combination illustrates one of the major issues for debate in

the application of the DEA methodology, that is, the free allocation of weights. The

information they contain can be used in the development of weight restrictions, in

conjunction with the views of the health care managers, to be examined next.


7 Introduction of Weight Restrictions

As was seen in table 3, the efficiency scores for each DMU have been calculated based

on contributions from some but not all of the input and output factors. Table 5 to

follow, which presents a selection of both efficient and inefficient DMUs, shows the

widely varying patterns of factor weights associated with the basic DEA methodology.

Virtual Factor Weights(%)

DMU CAP TDC TAC COA AEA TID DCA

#1 7.8 92.2 0 31.3 0 68.7 0

#7 7.7 0 92.3 13.4 85.0 0 1.6

#12 0 100 0 12.7 0 43.1 44.3

#16 0 64.9 35.1 69.5 0 12.0 18.5

#26 0 100 0 33.8 0 37.7 28.5

#42 11.3 88.7 0 20.9 1.9 73.0 4.2

Table 5: Virtual Factor Weights for Selected DMUs

According to Wilkinson ( 1991 ), these widely varying weighting patterns, and the fact

that some of the selected factors are completely excluded from the efficiency

calculations, are likely to produce results of limited value. Thus, the applicability of

weight restrictions has become a contentious issue in DEA debate. A variety of

methods have been proposed to actually impose weight restrictions, ranging from the

introduction of factor inequalities to the development of closed cones, within which

the factor weights may vary to a proscribed degree. Some of the key papers on this

issue are Dyson and Thanassoulis (1987), Thompson et al (1990), Wong and Beasley

(1990), Charnes eta!. (1989), Thanassoulis eta!. (1995), Roll and Golany (1993) and

Roll eta!. (1991). The reasons for the introduction of weight restrictions in a health

service context have been summarised as follows by Ball et a!. ( 1997):

• A DMU that has specialised in a particular area to the neglect of

others currently has more chance of being classified as efficient than the

good all-rounder;

• The lack of discrimination, given a reasonable number of inputs and

outputs, is unsatisfactory, as most DMUs will be 100% efficient.

Lessons Learned for DEA Practice from Health Care Applications in the UK

Eliminating factors is conceptually unsound and a very crude form of

weight limitation - a variable gets a weight of either zero or one;

• In many problems, not all inputs contribute to the production of every

output. This raises the possibility of reaching 100% efficiency on the

basis of a meaningless ratio;

• Allowing some inputs and outputs to be more highly weighted than

others may be appropriate, where specialist knowledge or policy

suggests this to be sensible.

149

In this investigation, weight restrictions have been introduced by attaching constraints

to the virtual inputs and outputs, following the approach presented in Ball and Roberts

( 1998), in which several different scenarios are developed to determine the impact a

range of weight restriction options.

However, the dilemma in applying weight restrictions is in finding the right balance. If

the restrictions are too loose then their adoption has no effect. On the other hand, very

tight restrictions leave no scope for flexibility and the resulting model could be

infeasible.

In this case, five additional scenarios have been developed to incorporate weight

restrictions, based on the information obtained in the initial analysis and the viewpoint

of the health care managers. Table 6 summarises the five scenarios, with the results

from the weighted models shown in summary form in table 7 and in greater detail in

appendix 2. For those factors where the maximum weight specified is 100%, the

weight restriction applied was simply a minimum constraint. It will obviously not be

possible for that factor to achieve a virtual weight of 100%, due to the minimum

constraints applied to the remaining factors.

Weighting on Virtual Weight for Each Factor (Min- Max%)

Scenario CAP TDC TAC COA AEA TID DCA

I 5- 25 50- 100 5-25 20-35 20-35 20-35 20-35

2 10- !00 10- 100 10- 100 5- 100 5- 100 25- 100 25-100

3 10- 50 10-50 10-50 5-50 5- 50 25-50 25-50

4 I I I 5- 50 5- 50 25-50 25-50

5 I I I I I 25-50 25-50

Table 6: Weight Restriction Scenarios


In relation to the outputs, the following scenarios are explored: each output should

contribute equally to the efficiency score, each output should contribute something to

the efficiency score and finally, some of the outputs should contribute more than

others to the efficiency score for each DMU. These represent different approaches to

health care provision in hospitals, in terms of the priorities of patient care. The input

weighting strategies reflect the importance attached to the direct costs, although

allocated costs should not be ignored, as they represent the implied cost of

'bureaucracy'.

Model or Weighting Scenario

Basic 1 2 3 4 5

Mean Score 87.98 67.74 70.49 65.07 72.70 74.53

St. Dev 15.08 22.61 21.97 22.48 21.84 22.11

No. of Eff. DMUs 20 4 4 4 6 7

Minimum Score 45.2 9.7 7.7 7.4 8.0 8.0

Table 7: Impact of Weight Restrictions

Just four DMUs are efficient in all scenarios (#2, #3, #13 and #43), suggesting that

their rating as efficient is robust. However, the DMU ranked number 1 with no

restrictions to the weights (#30) does not achieve an efficiency score under any of the

alternate weighting scenarios, dropping to 29 in the ranking under scenario 5. The

most significant impact of introducing weight restrictions is on DMU # 35, which is

efficient under the basic model and scores less than 10% under all the different

scenarios and is ranked 47th. This is due to the fact that it can no longer base its

efficiency score on a dominant contribution (99.7% in the basic model) from just one

output, 'total consultant outpatient attendances'.

Detailed analysis of the results from the scenarios can be used to illustrate aspects of

performance for each of the hospitals, such as, identifying the good all-rounder or

those particularly efficient in one aspect of service provision. Additional scenarios

could also be developed to investigate this further, in order to find the most robustly

efficient DMUs who remain unaffected under all scenarios. In addition, this type of

analysis may raise such as, whether all of the DMUs in the sample are appropriate for

comparison- a point stressed firmly by many of the health care managers interviewed.

Further analysis of the data may also be appropriate at this stage, as strong weight

restrictions will significantly affect some of the DMUs with more unusual data


patterns or a strong emphasis in one particular area. For example, table 8 shows how

efficiency scores are related to functional class for the basic model and for the weight

restricted model denoted by scenario 5. The hospitals in class 2 appear much more

resilient to the inclusion of weight restrictions, with just a small reduction in the mean

efficiency score, when compared with the hospitals in classes 5 and 7+.

Functional No. of Mean Score No. of Mean Score No. of Efficient Codes DMUs

(Basic Model) Efficient (Scenario 5) DMUs (Seen. 5)

DMUs (Basic)

1 7 97 4 90 2

2 10 94 6 90 2

5 10 78 2 56 0

7+ 8 89 5 55 0

11 6 77 0 72 0

12 6 95 3 90 3

Overall 47 88 20 75 7

Table 8: Efficiency Scores and Functional Class

The result of such analysis may be to redefine the samples being used for evaluation,

by, for example, assessing efficiency for the hospitals in each functional class in turn,

if appropriate sample sizes can be obtained.

8 Evaluation of the Approach

Interpreting the above analysis, in terms of evaluating the choice of model, sample and

variables and the results obtained, is clearly an important aspect of the DEA

application procedure. From the perspective of the DEA technology, the sample

provided an appropriate number of DMUs, in relation to the number of variables

included, and the results could be interpreted in a variety of ways. However, by taking

the perspective of the health care manager and their views on the methodology, those

aspects of the DEA technique which are most significant to its acceptance in the health

services have been identified:

1. The selection of the variables - ensuring that all those included are

relevant and contribute to the efficiency calculations for each of the


DMUs, leading to the introduction of weight restrictions, devised

through consultation rather than applied arbitrarily.

2. The definition of appropriate samples - developing carefully

selected samples of DMUs, chosen for their inherent similarity rather

than according to externally given definitions. These may represent

one aspect of hospital activity, rather than comparing across large

groups of hospitals as a whole, or be much smaller samples.

3. The presentation of the results - moving away from generalising

about average efficiency scores and focusing on the performance of

each individual DMU, with particular reference to target setting for

improving efficiency and the importance of identifying peer groups.

Appendix 1:

DEA Results for the Basic Model

Efficiency Reference Reference

DMU Score Ranking Set Count Group

I 95.2 23 0 2 II 14

2 100 7(1) 6 2

3 100 10(1) 3 3

4 85 32 0 2 3 14 16

5 95.7 21 0 2 3 16

6 100 16(1) 2 6

7 100 17(1) I 7

8 77.5 38 0 2 6 13 16

9 79 37 0 2141643

10 100 20(1) I 10

II 100 6(1) 7 II

12 100 2(1) 13 12

13 100 3(1) 12 13

14 100 4(1) 10 14


15 91.1 26 0 II 14 30

16 100 5(1) 7 16

17 93.8 24 0 13163547

18 83.9 33 0 12 30

19 77.2 39 0 12 30

20 52 45 0 12 30

21 86.2 30 0 14 30 33

22 71.5 41 0 12 13 27 30

23 100 18(1) 23

24 71 42 0 27 31 35 45

25 45.2 47 0 13

26 90.1 28 0 35 45 47

27 100 9(1) 4 27

28 73.8 40 0 II 13 30

29 83.3 34 0 II 14 30

30 100 I (I) 18 30

31 100 14(1) 2 31

32 100 19(1) 32

33 100 15(1) 2 33

34 53.8 44 0 12 27 30

35 100 8(1) 6 35

36 92 25 0 13

37 85.6 31 0 12 13 30 35

38 80.1 36 0 13 14 16 30 35

39 86.9 29 0 12 30

40 66 43 0 12 13 14 30

41 51.4 46 0 12 30

42 90.3 27 0 1112143043

43 100 II (I) 3 43

44 95.2 22 0 II 12 13 30

45 100 12(1) 3 45

46 82.2 35 0 12 13 30

47 100 13(1) 3 47


Appendix 2:

Efficiency Results Using Alternative Weighting Scenarios

Basic Scenario 1 Scenario 2 Scenario 3 Scenario 4 ScenarioS

DMU Score Rank Score Rank Score Rank Score Rank Score Rank Score Rank

1 95.2 23 84.1 12 82.9 15 79.9 13 84.7 16 84.9 17

2 100 7(1) 100 3(1) 100 2(1) 100 2(1) 100 5(1) 100 5(1)

3 100 10(1) 100 4(1) 100 4(1) 100 4(1) 100 6(1) 100 7(1)

4 85 32 72.2 23 70.8 27 70.7 20 70.8 29 71.2 30

s 95.7 21 21.2 46 53 38 51.4 37 53.9 38 86.3 16

6 100 16(1) 62.3 30 95.7 9 81 12 89.4 II 96.8 II

7 100 17(1) 85.7 10 85.5 12 85.4 9 86.1 14 87.4 15

8 77.5 38 72.6 22 75.4 23 72.6 19 76.6 23 77.3 26

9 79 37 74.9 21 75.7 22 73.7 17 76.4 24 77.1 27

10 100 20(1) 76.7 17 86.2 II 81.3 II 86 15 88.2 14

11 100 6(1) 96.5 6 97.1 7 96.6 6 97.6 8 98.1 10

12 100 2(1) 91.7 8 97.6 6 92.6 7 100 4(1) 100 3(1)

13 100 3(1) 100 I (I) 100 1(1) 100 I (I) 100 1(1) 100 1(1)

14 100 4(1) 97.4 5 97.9 5 97.9 5 97.9 7 98.6 8

IS 91.1 26 76.2 18 72.7 24 69 22 76.3 25 76.4 28

16 100 5(1) 88.9 9 93.4 10 91.5 8 95.4 9 98.2 9

17 93.8 24 85 II 84 13 76.6 15 88.5 13 89.7 13

18 83.9 33 50.6 38 48 39 37.5 40 49.7 39 49.9 39

19 77.2 39 57.6 33 64.2 33 49.9 38 69.4 30 69.4 33

20 52 45 37.6 42 41 41 34.3 42 45.4 41 46.1 41

21 86.2 30 46.1 39 40.7 42 35.5 41 42.2 43 42.8 43

22 71.5 41 67.5 28 64.8 31 59.3 29 67.5 33 67.9 34

23 100 18(1) 31.3 44 26 46 25.5 46 26.1 46 26.2 46

24 71 42 68.2 26 64.5 32 56.2 32 71 28 71 31

25 45.2 47 23.7 45 33.1 45 28.7 45 35.4 45 36.5 45

26 90.1 28 71.6 25 78.4 18 65.5 25 89.5 10 90.1 12

27 100 9(1) 67 29 60.8 35 56 33 61.2 36 61.9 37

28 73.8 40 57.3 34 56.6 37 52.6 36 57.5 37 57.7 38

29 83.3 34 67.6 27 67.6 29 67.3 23 69.2 31 70.5 32

30 100 1(1) 80 15 70.9 26 61.9 27 73.7 27 73.7 29

31 100 14(1) 60.8 32 61.4 34 53.2 35 63.9 35 63.9 36


32 100 19(1) 82.4 13 70.5 28 60.4 28 78.5 21 78.5 23

33 100 15(1) 53.7 37 46.4 40 44.2 39 46.9 40 47.2 40

34 53.8 44 37.8 41 35.1 44 30.3 44 36.9 44 37.3 44

35 100 8(1) 9.7 47 7.7 47 7.4 47 8 47 8 47

36 92 25 75.5 20 81.2 16 79 14 82.2 19 83.3 19

37 85.6 31 75.5 19 77.1 20 66.2 24 83.1 17 84.6 18

38 80.1 36 77.7 16 76.4 21 73.4 18 78.1 22 78.2 24

39 86.9 29 54.3 36 71.7 25 58.4 30 75.2 26 78.7 22

40 66 43 62.3 31 60.6 36 54.8 34 64.5 34 64.5 35

41 51.4 46 34 43 39.4 43 32.1 43 43.5 42 45.8 42

42 90.3 27 80.5 14 78.6 17 74.1 16 82.3 18 82.4 20

43 100 II (I) 100 2(1) 100 3(1) 100 3(1) 100 3(1) 100 6(1)

44 95.2 22 72 24 77.5 19 64.1 26 78.7 20 78.9 21

45 100 12(1) 96 7 96 8 82.8 10 100 2(1) 100 4(1)

46 82.2 35 43.2 40 65 30 57.1 31 68.7 32 77.9 25

47 100 13(1) 57.1 35 83.8 14 70.2 21 88.9 12 100 2(1)

References

MacMillan, W. D. (1987): The Measurement of Efficiency in Multiunit Public Services, Environment

and Planning, Vol. 19, pp. 1511-1524.

Flynn, N. (1986): Performance Measures in Public Sector Services, Policy and Politics, Vol. 14, No.3,

pp. 389-404.

Mooney, G. H. and Ludbrook, A. (1984): The NHS: Efficiency need not be a Dirty Word, British

Medical Journal, Vol. 288, No. 6433, pp. 1817-1818.

Levitt, M.S. and Joyce, M.A. S. (1987): The Growth and Efficiency of Public Spending, Cambridge

University Press.

Golany, B. and Roll, Y. (1989): An Application Procedure for DEA, Omega, Vol. 17, No. 3, pp. 237-

250.

Byrnes, P. and Valdmanis, V. (1995): Analyzing Technical and Allocative Efficiency of Hospitals, in:

Charnes, A., Cooper, W. W., Lewin, A. Y. and Seiford, L. M (eds.): Data Envelopment

Analysis: Theory, Methodology and Applications, Kluwer.

Sherman, H. D. (1984): Hospital Efficiency Measurement and Evaluation, Medical Care, Vol. 22, pp.

927-938.

!56 Lessons Learned for DEA Practice from Health Care Applications in the UK

Grosskopf, S. and Valdmanis, V. (1987): Measuring Hospital Performance: A Non-parametric

Approach, Journal of Health Economics, Vol. 6, No.2, pp. 89-107.

Information and Statistics Division (1996): Scottish Health Service Costs 1995-96, lSD, NHS in

Scotland.

Parkin, D. and Hollingsworth, B. (1996): Measuring Production Efficiency of Acute Hospitals in

Scotland, (1991-1994): Validity Issues in: Data Envelopment Analysis, Working Paper,

Department ofEpiderniology and Public Health, University of Newcastle.

Ozcan, Y. A., Luke, R. and Haksever, C. (1992): Ownership and Organizational Performance: a

Comparison of Technical Efficiency Across Hospital Types, Medical Care, Vol. 30, pp. 781-

794.

Roberts, E. and Ball, R. (1997): Efficiency and Performance Assessment in the Health Service- Using

Ideas of Policy to Develop Practical DEA Models, Paper Presented at the First DEA

Symposium in France, Marseille, June 26th - 28th, 1997.

Charnes, A., Cooper, W. W. and Rhodes, E. (1978): Measuring the Efficiency of Decision- Making

Units, European Journal of Operational Research, Vol. 2, No.6, pp. 429-444.

Hollingsworth, B. and Parkin, D. (1995): The Efficiency of Scottish Acute Hospitals: An Application

of Data Envelopment Analysis, IMA Journal of Mathematics Applied in Medicine and Biology,

Vol. 12, pp. 161-173.

Ehreth, J. L. (1994): The Development and Evaluation of Hospital Performance Measures for Policy

Analysis, Medical Care, Vol. 32, No.6, pp. 568-587.

Smith, P. and Mayston, D. (1987): Measuring Efficiency in the Public Sector, Omega, Vol. 15, No.3,

pp. 181-189.

Nunamaker, T. (1985): Using Data Envelopment Analysis to Measure the Efficiency of Non-profit

Organizations: a Critical Evaluation, Managerial and Business Economics, Vol. 6, No. I, pp.

50-58.

Dyson, R. G. and Thanassoulis, E. (1988): Reducing Weight Flexibility in DEA, Journal of the

Operational Research Society, Vol. 39, No.6, pp. 563-576.

Thompson, R. G., Langemeier, L. N., Lee, C. T., Lee, E. and Thrall, R. M. (1990): The Role of

Multiplier Bounds in Efficiency Analysis with Application to Kansas Farming, Journal of

Econometrica, Vol. 46, pp. 93-108.

Wong, Y-H. B. and Beasley, J. (1990): Restricting Weight Flexibility in DEA, Journal of the

Operational Research Society, Vol. 41, No.9, pp. 829-835.

Lessons Learned for DEA Practice from Health Care Applications in the UK !57

Charnes, A., Cooper, W. W., Wei, Q. L. and Huang, Z. M. (1989): Cone-ratio Data Envelopment

Analysis and Multi-objective Programming, International Journal of Systems Science, Vol. 20,

pp. 1099-1118.

Thanassoulis, E., Boussofiane, A. and Dyson, R. G. (1995): Exploring Output Quality Targets in the

Provision of Perinatal Care in England Using DEA, European Journal of Operational Research,

Volume 60, pp. 588-608.

Roll, Y. and Golany, B. (1993): Alternate Methods for Treating Factor Weights in DEA, Omega, Vol.

21, pp. 99-109.

Roll, Y., Cook, W., and Golany, B. (1991): Controlling Factor Weights in DEA, liE Transactions, No.

23, pp. 2-9.

Ball, R., Monaghan, C., Thomas, R. E. and Wagner, R. (1997): Data Envelopment Analysis: A

Practical Tool for Policy Makers?, University of Stirling, Departmental Working Paper.

Ball, R. and Roberts, E. (1998): The Relevance ofDEA in the Public Sector- A Discussion Paper for

Young OR 10, Young OR 10 Keynote Papers, Operational Research Society.

Recent Advances in Data Envelopment Analysis:

An Illustrative Application to the U.S. Public Accounting

Industry

Rajiv D. Banker, Hsihui Chang, Reba Cunningham and Ram Natarajan1

Abstract

In this paper we present some recent methodological innovations in Data Envelopment

Analysis and empirical results from the application of these innovations to the U.S. public

accounting industry. This paper draws on three different working papers: Banker, Chang

and Cunningham (1999), Banker, Chang and Natarajan (1999) and Banker and Natarajan

(1999). We describe how a consistent estimator of aggregate technical and a/locative

inefficiency can be obtained using DEA models and how it can be used to derive firm-specific

estimates of a/locative inefficiency. We also provide a statistical foundation for the various

two-stage methods used in the prior DEA literature to estimate the impact of contextual

variables on productivity. Finally, we document the presence of significant technical and

a/locative inefficiencies in the U.S. public accounting industry and explain the variation in

productivity across firms through a set of contextual variables.

1 School of Management, The University of Texas at Dallas, Richardson, TX 75083-0688, U.S.A.

[email protected]

[email protected]

160 Recent Advances in Data Envelopment Analysis ...

Structure

Introduction

2 Valuating Allocative Inefficiency Using DEA Models

3 Evaluating Contextual Variables Affecting Productivity Using DEA

4 Evaluating the Productivity of Public Accounting Firms

5 Conclusion

Appendix

References

Recent Advances in Data Envelopment Analysis ... 161

1 Introduction

In less than 20 years smce its inception, Data Envelopment Analysis (DEA) has

become an important and widespread analytical tool for evaluating factors affecting

efficiency. Seiford (1996) surveys the evolution of DEA from the publication of the

Chames, Cooper and Rhodes ( 1978) study to the current state of art. He identifies

statistical tests for model specification and stochastic DEA as two important areas for

future research in DEA. While the original DEA models specify the production set

relating inputs to outputs only in terms of properties such as convexity and

monotonicity and do not impose any explicit parametric structure on the production set

or the distribution of efficiency of individual observations, statistical properties can be

derived for the DEA estimator and a variety of statistical tests can be devised if

additional structure is specified (Banker 1993, 1996).

A model specification issue in DEA that has received inadequate attention is the

evaluation of allocative inefficiency in multiple output - multiple input production

models. A recent paper (Banker, Chang and Natarajan 1999, BCN hereafter) shows

that the DEA technical inefficiency measure using a single aggregate output variable,

constructed from multiple outputs weighted by their prices, reflects the aggregate

technical and allocative inefficiency. BCN employ this result to construct statistical

tests of the null hypothesis of no allocative inefficiency analogous to those of the null

hypothesis of constant returns to scale described in Banker (1996). BCN apply the

above methodology to an analysis of the productivity of firms in the public accounting

industry in the U.S. and document the presence of significant technical, scale and

allocative inefficiencies, but do not find any significant changes in these efficiency

measures over time.

Many studies have used a two-stage procedure to evaluate a set of contextual variables

believed to explain the variation in DEA inefficiency scores (Grosskopf 1996). A

recent study (Banker and Natarajan 1999, BN hereafter) investigates assumptions

about the underlying stochastic processes that generate the data about the inputs,

outputs and contextual factors to theoretically justify procedures such as OLS

(Ordinary Least Squares), COLS (Corrected Ordinary Least Squares), TOBIT

(Tobin's, 1958, Censored Regression Model) and MLE (Maximum Likelihood

Estimation) used in the second stage analysis ofDEA inefficiency scores. In addition,

BN use the DEA+ framework developed by Gsatch (1998) and conditions for the

consistency of ML estimators identified by Greene ( 1980) to propose a methodology


for a statistically consistent two-stage estimation of the impact of contextual variables

on inefficiency. Banker, Chang and Cunningham (1999, BCC hereafter) identify and

analyze contextual variables affecting the productivity of the U.S. public accounting

industry.

In this paper, we collate selected text and results from BCN, BN and BCC to describe

recent advances in DEA methodology and their application to the U.S. public

accounting industry. The remainder of the paper has the following structure. Section

2 describes the methodological advances in the BCN paper. Section 3 summarizes

salient methodological aspects of the BN paper. Section 4 describes the empirical

findings in BCN and BCC based on the application of these advances in DEA to the

U.S. public accounting industry. Concluding remarks are provided in section 5.

2 Evaluating Allocative Inefficiency Using DEA Models

Schmidt and Lovell ( 1979) describe two ways in which an observation about a

production process can exhibit inefficiency. It can be technically inefficient in the

sense that it fails to produce the maximum level of outputs from a given level of

inputs, or it can be allocatively inefficient in the sense that the marginal revenue

product of an input is not equal to the marginal cost of that input. Using a stochastic

production frontier estimation (SFE) approach, Schmidt and Lovell ( 1979) and

Kumbhakar (1987) extend the analysis of Aigner, Lovell and Schmidt (1977) and

Meussen and van den Broeck (1977) to describe the calculation of aggregate

inefficiency and its technical and allocative components. Kumbhakar ( 1996) discusses

the modeling of technical and allocative inefficiencies in both cost-minimizing and

profit-maximizing frameworks with special emphasis on multiple inputs and multiple

outputs. Kumbhakar's paper also uses a stochastic frontier framework.

Unlike the SFE-based models, which provide both estimation methods and statistical

tests for allocative efficiency, prior research dealing with allocative inefficiency in

Data Envelopment Analysis has focused only on its measurement. Banker and

Maindiratta ( 1988) describe the calculation of aggregate, technical and allocative

inefficiency through DEA-based linear programs for situations when the set of

observed output, input and price data is not consistent with profit-maximization (or

cost-minimization) for at least one firm in the sample. They define allocative

inefficiency as aggregate inefficiency divided by technical inefficiency.


Implementing Banker and Maindiratta's (1988) programs involves complete

knowledge of both input and output prices as well as input and output quantities.

While the researcher may have input and output quantity data, he may not have

information about individual output or input prices, except for the knowledge that the

firms operate in the same competitive market place. Very often, the available data

consists only of aggregate revenues and costs, and quantities of multiple inputs and

multiple outputs. In such a situation, BCN show how an aggregate technical and

allocative inefficiency measure, equivalent to the one described by Banker and

Maindiratta (1988), can be calculated. They then measure allocative inefficiency as

aggregate inefficiency divided by technical inefficiency as in Banker and Maindiratta

(1988). More importantly, the BCN study describes statistical tests for determining

the presence of allocative inefficiency in the observed sample ofDMUs.

Let Yi = (Yii•· ··Yri····YR) ~ 0 and Xi = (xii····Xij··· .x1j) ~ 0, j=l, .... N be the observed

output and input vectors generated from an underlying production possibility set T=

{(X,Y)J outputs Y can be produced from inputs X} for a sample ofN firms in the same

industry.2 Each output r is sold by all firms in the same competitive market at a price

Pr- Let P=(p~. ..... , PR) be the vector of output prices. The revenue from output r for

firm j is then n~ = PrY~· Denote the aggregate revenues as ni = L p,y,1 . The technical

inefficiency e; ~ 1 of an observation (Xj,Yj) E T, measured radially by the reciprocal of

Shephard's (1970) distance function, is given by BJ =t{Xi,Yi) = sup{OJ(Xi, BYj)ET}.

Assume that the production set T is monotonically increasing and convex and the 1+8

probability density function f(8) is such that f(8)=0 if 0< 1 and fJ(O)dB>O for 8>0. I

Then, a consistent estimator of BJ is obtained as BJ by solving the following Banker,

Chames and Cooper (1984) and Banker (1993) model:

BJ= Max 0

s.t. LA., n,, ~ e n,j Vr = I, ... R

' L A.,x,, :o; xu Vi= 1, .. .1

'

2 At least one output y~ and one input x;i are assumed to be strictly positive.

(1.0)

(1.1)

(1.2)


(1.3)

(1.4)

Aggregate technical and allocative inefficiency is measured in Banker and Maindiratta

( 1988) as follows:

(2.0)

Vr = i, ... R (2.1)

Vi= 1, .. .1 (2.2)

(2.3)

B,A.k :?:0 (2.4)

BCN consider the linear program in (1) after replacing the R constraints in (1.1) by a

single constraint, I A.k nk :?: Bn 1 for the aggregate revenue, and denote the resulting k

DEA technical inefficiency measure as BJ. They show that BJ is a consistent

estimator of the technical inefficiency for a derived production set with a single

aggregate output. More importantly, they also prove that BJ = BJ. The allocative

inefficiency, e; , is then calculated as BJ I e: . The equivalence of BJ and BJ is useful

for establishing the statistical consistency of the aggregate technical and allocative

inefficiency estimator.

Based on Banker ( 1993) and analogous to the tests of constant returns to scale described

in Banker (1996), BCN develop a variety of procedures to test the null hypothesis of no

allocative inefficiency against the alternative of the presence of such inefficiency based

on assumed structure for the distribution of t(9), where t(.) is an appropriate

transformation function. The following illustrate the test procedures for the case of

t(.)=ln(.):


(i) If In( BJ) is distributed as exponential over [0, oo ), then under the null hypothesis of

N

z)n<B)) no allocative inefficiency, the test statistic is calculated as ~1~:;-'-- and is evaluated

z)n<BJ) j.:l

relative to the critical value of the F-distribution with (2N, 2N) degrees of freedom.

(ii) If In( BJ) is distributed as half-normal over the range [0, oo ), then under the null

N

L(1n(B)))' hypothesis of no allocative inefficiency, the test statistic is calculated as ";1;:;:.'--

L(ln(BJ))' j=l

and is evaluated relative to the critical value of the F-distribution with (N, N) degrees of

freedom.

(iii) If no such assumptions are maintained about the probability distribution of

inefficiency, then a nonparametric Smimov's test statistic given by max {FA(ln( e:)) -F8(ln( e;)) I j = 1, ... N}, where FA(.) and F8 (.) refer to the empirical distributions of

In( B) ) and In( BJ ), is used.

However, as Banker (1993, 1996) points out, while ln(Bj) and ln(BJ} are

asymptotically independent of each other they need not be independently distributed

for finite samples. For finite samples, they need not also follow the true distribution of

In( e;) and In( Bf ). An important caveat, therefore, is that the above tests are designed

for large samples, their small sample performance need to be evaluated with Monte

Carlo experimentation.

3 Evaluating Contextual Variables Affecting Productivity Using DEA

Many studies have assessed the impact of contextual factors on DEA efficiency scores

(Grosskopf 1996). In these studies, the relative efficiency of each organization is

evaluated in the first stage based on data about their input consumption and output

production. The efficiency score is then regressed on the potential contextual factors in

the second stage to identify the factors whose impact on productivity is statistically


significant. Alternative second stage methods have included the use of logarithmic

transform of the relative efficiency score as dependent variable in an OLS regression,

and the use of a TOBIT procedure to reflect the fact that the logarithm of the DEA

efficiency score is bounded above by zero.

A question that has not been addressed in earlier studies is whether such a two-stage

approach is statistically valid to assess the significance of individual contextual

variables. BN address this issue by specifying the basic data generating process to

derive the appropriate statistical estimation models. They also make a contribution to

the stochastic DEA literature by presenting estimators of the inefficiency of individual

DMUs conditional on the estimated value of the composite error comprising the one

sided inefficiency component and a two-sided random noise component.

Consider observations on j = l, .... N decision making units (DMUs) containing a

vector of outputs Yi = (Yti• ... YRi), a vector of inputs Xi = (xtj, ... X 1j) and a vector of

contextual variables Zi = (ztj, ... Zsj) that may influence the overall productivity in

transforming the inputs into the outputs. Thus, for instance, in Farrell's (1957) original

setting for efficiency evaluation, the outputs Y may represent a farm's production

measured in tons of grain, the inputs X may be labor, capital and materials and the

contextual variables Z that influence productivity may be factors such as ownership of

the farm and management methods.'

The basic model in BN is described for the case of a single output, y, to maintain

direct contact with parametric stochastic frontier models. The extension to the multiple

outputs case is straightforward.4 The data generating process (DGP) described in BN

specifies the "true" production function g(X) and the generation of random variables

representing the inputs X and productivity p :<> 1. The production function g(.) is

monotone increasing and concave, and relates the input vector X to a single output y as

specified by the equation

y = g(X) * p (3)

3 Contextual variables should not be confused with non-discretionary or exogeneously fixed inputs. 4 Our extension to the multiple output case involves an additional vector of random variables specifying the proportion of each output. The data generating process then determines the output vector Yi as in the single output case on the ray defined by the vector of random variables specifying the output mix.


The random variable representing productivity p is itself generated by the process

In p =- h(Z)- u + v (4)

where Z is the vector of contextual variables hypothesized to affect productivity, h(.)

;::: 0 is a monotone increasing and convex function, u represents technical inefficiency

and has a one-sided distribution, and v represents random noise and has a two-sided

distribution. The following structure is imposed on the probability density functions

generating these variables:

fz,( z,) = 0 for all z, < 0 (Sa)

fu(u) = 0 for all u < 0 (5b)

fv(v) = 0 for all v > yM (5c)

(5d)

Substituting equation (4) into (3), we obtain

In y = In g(X) - h(Z) + E (6)

where E = v - u is a composed error term whose p.d.f. is given by

v" fE(E) = Jr,(v)fu(v-E)dv (7)

If, in addition, a parametric structure is imposed for the two functions g(.) and h(.)

such that In g(X; a) is linear in its parameter a, and h(Z; ~) is linear in its parameter ~.

then

In y =In g(X; a)- h(Z; ~) + E (8)

It is evident from an inspection of (8) that it is of the form specified in parametric

stochastic frontier estimation (PSFE). Therefore, ordinary least squares estimation of

(8) will yield unbiased estimators of all a and ~ except the intercept term (Schmidt

1976). The impact of individual contextual variables z, can be assessed by evaluating

the significance of the corresponding ~' if, for instance, h(Z; ~) = ~T Z. The OLS

estimator of the intercept term in such a parametric production function, however, is

biased. This is addressed in PSFE by employing corrected ordinary least squares

(COLS), where the estimator for the intercept is obtained by solving the moment


equations (Olson, Schmidt and Waldman 1980). Alternatively, maximum likelihood

estimators (MLE) can be obtained for all the parameters in (8) as in Aigner, Lovell and

Schmidt ( 1977) or Meeusen and van den Broeck ( 1977) by specifying a parametric

form for the p.d.f. ofu and v consistent with (5). These estimators, however, may not

be consistent and asymptotically normally distributed because of the upper truncation

for the p.d.f. ofv, unless Greene's (1980) conditions are satisfied.

To develop a DEA based estimation procedure that is consistent with the DGP

described in (3)- (5), BN adapt the DEA +method introduced by Gstach (1998). This

involves defining:

In g(X) = In g(X) + yM and (9a)

u = u + (VM-v) +h(Z) ~0 (9b)

Since g(X) is derived from g(X) by multiplication with a constant factor, g(X) is also

monotone increasing and concave. Substituting (9a) and (9b) into (6) yields

In y =In g(x) - u (10)

Therefore, the logarithm of the DEA inefficiency estimator, lnB, obtained by

performing DEA on the inputs-output observation (Xi, Yi) , j = 1, ... N, IS a

consistent estimator of u . As observed earlier, these concepts extend directly to the

multi-output case. See, also, Banker, Janakiraman and Natarajan (1999) for

statistically consistent estimation of general monotone and concave or convex

functional relationships.

To estimate the impact of individual contextual variables z,, BN rely on the following

relation motivated by (9b):

In e = h(Z) +E' (11)

where e = u + (VM - v) ~ 0. Since h(.) is a monotonically increasing and convex

function, a second stage DEA estimation on the pseudo "input-outputs" observations

(In 01 , Z 1 )yields the inefficiency estimator fi/j and a consistent estimator of £1 can be

(rji j - 1 )in Bj calculated as . To evaluate the statistical significance of individual z,, a

rjlj


third stage DEA estimation is first performed on the pseudo observations (1nB1, Zj')

where z-s is the original Z vector without the Zs variable. Let the resulting inefficiency

estimator be lj/1-'. Under the null hypothesis that the marginal impact of Zs (i.e.

8h(Z)/8z5) is zero, the asymptotic distributions of lji and V' are identical (Banker

1996). If the asymptotic distribution of c1 is assumed to be exponential or half

normal, the null hypothesis of no impact of Zs is tested by comparing the ratios

N

I (v; j - 1 )In ej I VI j against critical values obtained

;=I

;=I

from F-

distributions with (2N,2N) or (N,N) degrees of freedom, respectively.

However, in general, such a distributional assumption for c1 may not be justified.

Therefore, BN propose an alternative approach of regressing lji-' on lji:

(12)

Since, lj/-' and lji are asymptotically identically distributed if the null hypothesis is

true, an asymptotic test of the null hypothesis is obtained by evaluating the hypothesis

~I= J.

To relate this analysis back to the initial objective of rationalizing the two stage

approach used in several prior empirical studies that regress the logarithm of the

inefficiency on the contextual variables, some additional structure is imposed by

assuming h(Z) = ~ T Z, so that the evaluation of the impact of a variable z5 is reduced to

the test of significance of the hypothesis ~s = 0. It is evident from (11) that estimators

of~ may be obtained from the relation:

- T M In B = ~ Z + V - (v- u) (13)

An inspection of the relation in ( 13) indicates that it is analogous to the estimation of

stochastic production frontiers, although the dependent and independent variables are

not inputs or outputs. Therefore, standard results for parametric stochastic frontier


estimation apply in this case. Thus, OLS estimation of (13) yields unbiased estimators

/3 (Schmidt 1976), but the OLS estimator Po consistently estimates yM- E(v-u).

It may appear that TOBIT analysis could be justified as MLE if the distribution of E

were truncated Normal. In general, however, MLE yields consistent estimators if

Greene's ( 1980) conditions are satisfied. But, if the distribution of E is truncated

Normal then Greene's conditions are not satisfied, and therefore consistency property

cannot be established for the TOBIT estimator using this approach.

Greene ( 1980) identifies the following two sufficient conditions to ensure that the ML

estimator is consistent and its asymptotic distribution is Normal:

v" /,(c)= fJ,(v)f.(v-c)dv=O atE=VM

The first condition is always satisfied at E = VM. The second condition implies that

fv(VM) = 0 [e.g. Beta] or fu(O) = 0 [e.g. Gamma or Lognormal].

Suppose u is distributed Gamma (2,A.) and v as N(O,cr/) truncated above at vM. Let

( c a J ( VM a J ~ • . c,= -+-2. , c,= -+-2. , and r (.)and F (.)be the standard normal density and a, A. a, A.

distribution funtions, respectively. BN derive the p.d.f. of E = v- u as

.aJ + &

2..tl A.

f(E)= a,e( MJ[{f(c,}-j'(c,}}+c,{F'(c,}-F'(c,}}] A.' F' V

a,

(14)

Since (13) can also be expressed as E = ~Tz + yM -In B, the log-likelihood function

can be formed as 2)nf(,BrZ+VM-lnB) using (14) and this function can then be

maximized with respect to ~. A., crv and yM to consistently estimate the unknown

parameters.

BN also derive the conditional distribution of the inefficiency u given E and show that

it is truncated between 0 and yM- E. The conditional mean E(uiE) = E(t) + Var(t)/E(t),


where t is distributed as N(- (e +~),a; J truncated between 0 and yM- E, is a

consistent estimator of u given E. Also, the conditional mode M(ule) is a MLE as in

Jondrow et al. (1982). The mode is given by the following:

M I Ife ~ V -~ then M(ule) = yM- E (15)

+ cr; A.

(16)

4 Evaluating the Productivity of Public Accounting Firms

In this section, we discuss the main empirical results from BCN and BCC. These

studies employ the analytical models described in sections 2 and 3 to estimate the

relationship between input, output and contextual variables for the U.S. public

accounting industry. The public accounting industry has undergone many changes in

recent years. The economic slowdown of the early 1990s, advances in information

technology and considerable growth in the demand for consulting services are believed

to have Jed to an increase in competition among the firms and significantly impacted

the economics of the profession. While there is intense debate among academics,

practitioners and regulators about the effect of competition on public accounting firms,

very little empirical evidence exists on whether they efficiently utilize their resources,

whether they allocate these resources to the different types of services to generate

maximum revenue and what firm-specific factors affect their productivity.

The data used in the two studies were obtained from Accounting Today's annual

surveys of the top 100 accounting firms for the three years 1995, 1996 and 1997.

Accounting Today constructed the database from firms' responses and its own research

and estimates. This annual survey of the profession's largest practices has now

become one of the most often cited benchmarks in the business. All data reported in

the annual surveys are for domestic U.S. operations and exclude foreign holdings.

Non-CPA firms such as American Express Inc., Padgett Business Services and H&R


Block are eliminated from the database. There are a total of 93, 92, and 93

observations for the years 1995, 1996 and 1997, respectively.

BCN focus on the production correspondence between service revenues generated and

human resources employed by public accounting firms. There are three service

outputs: Accounting and Auditing (A&A), Tax Services (TAX), and Management

Advisory Services (MAS), each measured in millions of dollars of revenues. A&A

includes compilations, special reports, and reviews in addition to engagements

involving the attest function. TAX includes tax research, planning and preparation

work. MAS is defined as consulting, systems development, integrating and reselling

computer equipment and software, and any other management assistance. The three

human resource input variables considered are the number of partners (PARTNERS),

the number of other professionals (PROFESSIONALS) and the number of other

employees (OTHERS). The designation PARTNERS includes all owners and

shareholders. PROFESSIONALS includes those trained to perform the accounting

and other services offered by the firm. Generally, these include staff accountants,

senior accountants, and managers. OTHERS are clerical and support personnel;

usually involved in administration, printing of reports, record keeping and the like.

Personnel costs constitute a significant fraction of total costs for the public accounting

firms. A recent national survey indicates that employee costs and partner

compensation account for 74.5% of the revenues, while capital costs are less than 7%,

for accounting practices with revenues in excess of one million dollars (Texas Society

of Certified Public Accountants 1997).

Recall the programs described in section 2 for the calculation of the technical

inefficiency BJ and the aggregate technical and allocative inefficiency e: . BCN use

the revenues from the three services as output variables and the number of partners,

professionals and other employees as input variables in the linear program described in

( 1) to calculate BJ . In contrast, the calculation of e: involves the use of the aggregate

revenue from all three services as the single output variable. The estimation of the

inefficiency measures is carried out on a year by year basis. The allocative

inefficiency, e;, is then calculated as e: I BJ.

The cross-sectional distribution of the technical, aggregate and allocative inefficiencies

is shown in table 1. The mean (median) values of technical inefficiency BJ are 1.2213

(1.1496), 1.1825 (1.1043) and 1.1941 (1.1520) for 1995, 1996 and 1997, respectively.


The corresponding values for allocative inefficiency, e;, are 1.1671 (1.1459), 1.2266

(1.1907) and 1.2072 (1.1787), for 1995, 1996 and 1997, respectively. Both

distributions are skewed to the left and more than 25% of the sample firms are found

to be technically efficient. While the distributions are quite informative, additional

assumptions on the inefficiency variables are necessary to make inferences about the

level of inefficiency in the industry.

Panel A of table 2 provides approximate 95% confidence intervals for the logarithm of

the technical and allocative inefficiency measures, InBJ and !no;, respectively. Four

different types of confidence intervals are provided. The first three are for the mean of

the log-transformed inefficiency measure whereas the fourth one is calculated for the

median. CI -1 is calculated as mean(In B)± 1.96 sy};ln B) where SD(ln B) is the sample

standard deviation of lnB. Cl-2 is calculated as mean(InB)± 1.96 m:Ji(inB) assuming

lnB is distributed as exponential over [0, oo ). CI-3 is calculated as mean(InB)

±1.96mean(inB)~"- 2 assuming a half-normal distribution for lnB. CI-4 is 7rN

calculated by first sorting the observations in ascending order and then choosing the

observations given by 0.5(N+ 1) ± 0.98JN as the bounds for the interval (Lehmann

1975, p.184 ). For all three years of data, none of these intervals is found to include

zero implying that the public accounting industry operated under significant technical

and allocative inefficiency.

The null hypothesis of no allocative inefficiency is tested against the alternative of

existence of inefficiency by using F-statistics similar to those described in Banker

( 1996). Panel B of table 2 provides two tests to evaluate the null hypothesis. Test I is

based on the assumption of identical exponential distributions for In BJ and In e: under the null hypothesis while test II assumes identical half-normal disributions. All

the F-statistics presented in panel B reject the null hypothesis of no allocative

inefficiency at the 5% level for all three years of data suggesting that the public

accounting industry exhibited significant allocative inefficiency.

To test for efficiency improvements over time, BCN first pool the observations from

all three years after converting all revenue data to constant 1995 dollars and then re-

estimate e:' the aggregate inefficiency, and e;' the technical inefficiency, using a


single program for all three years of data. Statistical tests of differences in In( B/ ) (or

In( Bf} as the case may be) are performed for each pair of years. For example, BCN

evaluate whether the parameter describing the distribution of firm inefficiency in 1995

equals that which describes the distribution in 1996, against the alternative that the two

parameters are unequal. The test statistics are similar to those described in Banker

(1993) to evaluate the null hypothesis of no difference in the inefficiency distributions

of two sub-samples. The tests are based on assumed exponential or half-normal

distributions for the inefficiency variable. The tests do not reject the null hypothesis of

no change in efficiency for all three years and for both distributional assumptions at

the 5% significance level. That is, there is no evidence of efficiency improvement

over the three-year period. The results are robust for the restricted sample of 79 firms

for which data are available for all three years.

BCC examine whether the variation in firm productivity can be explained by

contextual variables such as the percentage of revenues from management advisory

service business (MAS%), percentage of revenues from tax service business (TAX%),

service diversity (HERFINDEX), the number of offices (OFFICES) and whether the

firm is a Big Six accounting firm (BIG6). Herfindahl index for a CPA firm's services

is calculated as [(A&A%/100)2 + (TAX%/100)2 + (MAS%1100)2]. The higher the

HERFINDEX, the lower is the service diversity.

To evaluate the impact of different contextual variables identified by BCC, the

following version of the linear model specified in (13) is used:

(17)

lnB = ~o +~~TAX%+ ~2MAS% + ~3 HERFINDEX + ~4 /nOFFICES + ~s BIG6 + E

This model is estimated using two different methods. The first method uses OLS

estimation and the second allows the error term E to consist of two components where

E = u - v + yM where u is distributed as Gamma (2,/...) and v as N(O,cr/) truncated

above at yM and uses MLE. As described in BN and explained in section 3, while the

OLS yields consistent estimators of the coefficients of the contextual variables, the

MLE method provides consistent estimates of all the parameters for the data

generating process specified in (3) to ( 5).


Tables 3 and 4 describe the results from these two estimation methods. The following

salient points emerge from the analysis of the results. Both methods produce very

similar results. This is because the MLE method estimates low values for the

parameters of the one-sided disturbance variable, u, and it estimates values of the

upper truncation point, VM, for the truncated Normal random variable, v, at 2.65, 3.07

and 3.91 standard deviations away from zero for the three years of data. It appears that

except for the identified systematic variation in inefficiency across different public

accounting firms attributable to the contextual variables, the variation in their

performance is due only to the two-sided random noise term.

Of the five contextual variables, the percentage of revenues derived from taxes

(TAX%) and the logarithm of the number of offices (lnOFFICES) have significantly

negative impact on productivity, while the Big Six status variable (BIG6) has a

significantly positive impact, under both models at the 5% significance level. MAS%

is not significantly associated with productivity. Given that the correlation between

TAX% and MAS% is of the order of -0.6 for all the three years of data, it is possible

that MAS% is not able to explain any additional variation in productivity over and

above that explained by TAX%. Accounting firms with more offices do not appear to

be more productive. In fact, there is strong evidence to the contrary. Productivity is

not found to be affected by service diversity, measured by the HERFINDEX variable,

during all the three years.

5 Conclusion

The main objective of this paper is to present some recent methodological innovations

in Data Envelopment Analysis and empirical results from the application of these

innovations to the U.S. public accounting industry. This paper draws on three

different working papers: Banker, Chang and Cunningham (1999), Banker, Chang and

Natarajan (1999) and Banker and Natarajan (1999). It documents the presence of

significant technical and allocative inefficiencies in the public accounting industry and

explains the variation in productivity across the firms through a set of contextual

variables.

We describe in this paper how a consistent estimator of aggregate technical and

allocative inefficiency can be obtained and how it can be used to derive firm-specific

estimates of allocative inefficiency. We also provide formal statistical tests to evaluate


the null hypothesis of no allocative inefficency against the alternative of the presence

of such allocative inefficiency. While such tests have been available for parametric

stochastic frontier estimation methods, the BCN paper is perhaps the first in the DEA

literature to propose appropriate statistical tests for allocative inefficiency. The

theoretical advance in BN also contributes to the evolving new field of stochastic

DEA. We introduce contextual variables in Gsatch's (1998) DEA+ framework to

provide a statistical foundation for the two-stage methods used in the prior DEA

literature to analyze the impact of contextual variables on productivity. We also show

that a simple OLS regression of the logarithm of the DEA inefficiency score on the

contextual variables can provide consistent estimators of the productivity impact of

these variables.

The empirical results in BCC and BCN have important implications for the public

accounting industry. Our analysis of the data for the top I 00 U.S. public accounting

firms for the period 1995 to 1997 indicates that, contrary to common beliefs in the

prior literature, the public accounting industry operated under significant technical,

scale and allocative inefficiencies. This finding implies that public accounting firms

have not fully reaped scale economies from mergers and productivity gains from

investments in information technology and that they can generate significant cost

savings by better utilizing their human resources.

Appendix

Table 1: Cross-Sectional Distribution of the Various Inefficiency Estimators

Panel A: Technical inefficiency, Bf

YEAR Mean Std. Dev. 25% Median 75%

1995 (N-93) 1.2213 0.2449 1.0000 1.1496 1.3632

1996 (N-92) 1.1825 0.2175 1.0000 1.1043 1.3088

1997 (N-93) 1.1941 0.2189 1.0000 1.1520 1.2816


Panel B: Aggregate technical and allocative inefficiency, B/ YEAR Mean Std. Dev. 25% Median 75%

1995 (N-93) 1.4239 0.2913 1.1894 1.3888 1.6140

1996 (N-92) 1.4439 0.3001 1.2047 1.4656 1.6248

1997 (N-93) 1.4367 0.2995 1.2038 1.4074 1.6831

Panel C: Allocative inefficiency, B; = B/ I B;.

YEAR Mean Std. Dev. 25% Median 75%

1995 (N-93) 1.1671 0.1457 1.0751 1.1459 1.2188

1996 (N-92) 1.2266 0.1939 1.0764 1.1907 1.3480

1997 (N-93) 1.2072 0.1838 1.0817 1.1787 1.3104

Table 2: Tests of the Existence of Various Inefficiencies in the Public Accounting

Industry

Panel A: 95% confidence intervals for lnB; and lnB;. CI-1 is calculated as

mean(lnB )± 1.96SD(1nB)/ JN where SD(lnB) is the sample standard deviation of lnB,

CI-2 as mean (In B)± 1.96 mean(in B)/ JN, CI-3 as mean(lnB) ± 1.96mean(ln B)~ n- 2 nN

and CI-4 is calculated by first sorting the observations in ascending order and then

choosing the observations given by 0.5(N+ 1) ± 0.98Jl.i as the bounds for the interval.

YEAR Interval Type 'B lnB1

'v lnB1

1995 (N-93) CI-1 (0.144,0.220) (0.125,0.171)

CI-2 (0.145,0.219) (0.118,0.178)

CI-3 (0.154,0.21 0) (0.125,0.171)

CI-4 (0.037,0.080) (0.048,0.067)

1996 (N-92) CI-1 (0.118,0.187) (0.163,0.223)

CI-2 (0.121,0.184) (0.154,0.233)

CI-3 (0.129,0.176) (0.163,0.223)

CI-4 (0.009,0.080) (0.058,0.088)

1997 (N-93) CI-1 (0.128,0.197) (0.151 ,0.206)

CI-2 (0.130,0.196) (0.142,0.215)

CI-3 (0.138,0.188) (0.151 ,0.206)

CI-4 (0.028,0.087) (0.057 ,0.089)


Panel B: F-statistics for testing the null hypothesis of no allocative inefficiency against

the presence of such inefficiency. Test I computes the F-statistic as

N IN ~ 1n(BJ} ~ ln(Bf) and this is evaluated relative to the critical value of the F-

distribution with (2N, 2N) degrees of freedom. Test II computes the F-statistic as

N IN ~ (ln(B: )) 2 f.; (ln(B: )) 2 and this is evaluated relative to the critical value of the F-

distribution with (N, N) degrees of freedom.

YEAR Test I Test II

1995 (N-93) 1.830 (0.00 I) 2.246 (0.00 I)

1996 (N-92) 2.265 (0.001) 3.142 (0.001)

1997 (N-93) 2.098 (0.00 1) 2.898 (0.00 1)

Table 3

Impact of Contextual Variables Affecting Productivity in Public Accounting

Firms -- Ordinary Least Squares Estimation

The model estimated is: In e: = Po + PI TAX% + Pz MAS% + P3 HERFINDEX + p4

/nOFFICES + Ps BIG6 + E, where In e: is the logarithm of the aggregate technical

and allocative efficiency estimator for firm j, TAX% is revenues from tax services

expressed as a percentage of total revenues, MAS% is revenues from management

advisory services expressed as a percentage of total revenues, HERFINDEX is

[(A&A%/100)2 + (TAX%/100)2 + (MAS%/100)2], lnOFFICES is the natural logarithm

of the number of offices and BIG6=1 if the firm is one of the Big Six firms, and 0

otherwise. The values given below the coefficient estimates are p-values for two-sided

tests. * indicates significance at the 5% level.

Variables Coefficient 1995 1996 1997

Estimates (N=93) (N=92) (N=93)

Intercept [3. 0.3622 0.1872 0.1924

(0.003) (0.095) (0.079)

TAX% [3. 0.0027 0.0056' 0.0058'

(0.175) (0.009) (0.005)


MAS% 13> -0.0018 0.0003 -0.0012

(0.144) (0.818) (0.353)

HERFINDEX 13> -0.3473 -0.1960 -0.0720

(0.098) (0.285) (0.692)

LnOFFICES ~ 0.0725' 0.0655' 0.0406'

(0.001) (0.001) (0.022)

BIG6 13. -0.5001' -0.5151' -0.3621'

(0.001) (0.001) (0.001)

F-value 11.837 11.48 9.922

Adj. R2 0.371 0.366 0.327

Table 4

Impact of Contextual Variables Affecting Productivity in Public Accounting

Firms -- Maximum Likelihood Estimation

The model estimated is: lnBf = ~0 + ~ 1 TAX%+ ~2 MAS% + ~3 HERFINDEX + ~4

/nOFFICES + ~5 BIG6 + E, where In e: is the logarithm of the aggregate technical

and allocative inefficiency estimator for firm j, TAX% is revenues from tax services

expressed as a percentage of total revenues, MAS% is revenues from management

advisory services expressed as a percentage of total revenues, HERFINDEX is

[(A&A%/100)2 + (TAX%/100)2 + (MAS%/100h lnOFFICES is the natural logarithm

of the number of offices and BIG6=1 if the firm is one of the Big Six firms, and 0

otherwise. The error term is defined as E = u-v+VM where u is distributed as Gamma

(2,A.) and vas N(O,o}) truncated above at VM. The values given below the coefficient

estimates are p-values for two-sided tests. * indicates significance at the 5% level.

Variables Coefficient 1995 1996 1997

Estimates (N=93) (N=92) (N=93)

Intercept 13. -0.2163' -0.3296' -0.4467

(0.044) (0.001) (0.143)

TAX% 0.0041' 0.0056' 0.0063'

(0.031) (0.001) (0.008)

MAS% -0.0012 0.0006 -0.0013

(0.247) (0.560) (0.381)


HERFINDEX (3. -0.1107 -0.1848 -0.0539

(0.425) (0.196) (0.741)

LnOFFICES ll 0.0695' 0.0617' 0.0415'

(0.001) (0.001) (0.054)

BIG6 (3. -0.4686' -0.4535' -0.3631'

(0.001) (0.001) (0.001)

a, 0.1633' 0.1682' 0.1614'

(0.001) (0.001) (0.001)

I. o.oo5o' 0.005!' 0.0050'

(0.001) (0.001) (0.001)

yM 0.4322' 0.5164' 0.6303

(0.001) (0.001) (0.118)

References

Aigner, D. J., Lovell, C. A. K. and Sclunidt, P. (1977): Formulation of Estimation of Stochastic

Frontier Production Function Models, Journal of Econometrics, 6, 21-37.

Banker, R. D. (1993): Maximum Likelihood, Consistency and Data Envelopment Analysis: A

Statistical Foundation, Management Science, October, 1265-1273.

Banker, R. D. (1996): Hypothesis Tests Using Data Envelopment Analysis, Journal of Productivity

Analysis, 7, 139-159.

Banker, R. D., Chang, H. and Cunningham, R. (1999): The Public Accounting Industry Production

Function, Working Paper, The University of Texas at Dallas.

Banker, R. D., Chang, H. and Natarajan, R. (1999): Efficiency of Public Accounting Firms, Working

Paper, The University of Texas at Dallas.

Banker, R. D., Chames, A. and Cooper, W. W. (1984): Models for the Estimation of Technical and

Scale Inefficiencies in Data Envelopment Analysis, Management Science, 30, 1078-1092.

Banker, R. D., Janakiraman, S. and Natarajan, R. (1999): Estimation of Monotone and Concave or

Convex Functions, Working Paper, The University of Texas at Dallas.


Banker, R. D. and Maindiratta, A. (1988): Nonparametric Analysis of Technical and Allocative

Efficiencies in Production Econometrica, November, 1315-1332.

Banker, R. D. and Natarajan, R. (I 999): Evaluating Contextual Variables Affecting Productivity Using

Data Envelopment Analysis, Working Paper, The University of Texas at Dallas.

Charnes, A., Cooper, W. W., and Rhodes, E. (1978): Measuring the Efficiency of Decision Making

Units, European Journal of Operational Research, 429-444.

Craswell A., Francis J. R., and Taylor, L. (1995): Auditor Brand Name Reputations and Industry

Specializations, Journal of Accounting and Economics, 20, 297-322.

Farrell, M. J. (1957): The Measurement of Productive Efficiency, Journal of the Royal Statistical

Society (A,general) 120, pt. 3, 253-290.

Greene, W. H. (1980): Maximum Likelihood Estimation of Econometric Frontier Production

Functions, Journal of Econometrics, 13,27-56.

Grosskopf, S. (1996): Statistical Inference and Nonparametric Efficiency: A Selective Survey, Journal

of Productivity Analysis, 7, 161-176.

Gstach, D. (1998): Another Approach to Data Envelopment Analysis in Noisy Environments: DEA+,

Journal of Productivity Analysis, 9,161-176.

Jondrow, J., Lovell, C. A. K., Materov, I. S. and Schmidt, P. (1982): On The Estimation of Technical

Inefficiency in the Stochastic Frontier Production Function Model, Journal of Econometrics, 19,

233-238.

Kumbhakar, S. ( 1987): The Specification of Technical and Allocative Inefficiency in Stochastic

Production and Profit Frontiers, Journal of Econometrics, 34, 335-348.

Kumbhakar, S. ( 1996): Efficiency Measurement with Multiple Outputs and Multiple Inputs, Journal of

Productivity Analysis, 7, 225-255.

Lehmann, E. L. (1975): Nonparametrics - Statistical Methods Based on Ranks, Holden-Day and

Mcgraw-Hill.

Meeusen, W. and van den Broeck, J. (1977): Efficiency Estimation from Cobb-Douglas Production

Functions with Composed Error, International Economic Review, June, 435-444.

Olson, J. A., Schmidt, P. and Waldman, D. A. (1980): A Monte Carlo Study of Estimators of

Stochastic Frontier Production Functions, Journal of Econometrics, 13,67-82.

Schmidt, P. (1976): On the Statistical Estimation of Parametric Frontier Production Functions, Review

of Economics and Statistics, May, 238-239.


Schmidt, P. and Lovell, C. A. K. (1979): Estimating Technical and Allocative Inefficiency Relative to

Stochastic Production and Cost Frontiers, Journal of Econometrics, 9, 343-366.

Seiford, L. M. (1996): Data Envelopment Analysis: The Evolution of the State of the Art, Journal of

Productivity Analysis, 7, 99-13 7.

Shephard, R. W. (1970): Theory of Cost and Production Functions, Princeton, N. J., Princeton

University Press.

Texas Society of Certified Public Accountants (1997): Management of an Accounting Practice

Survey, Dallas.

Tobin, J. (1958): Estimation of Relationships for Limited Dependent Variables, Econometrica,

January, 24-36.

Combining DEA and "Transformation-Stages":

Management Strategies for the Disability Service Units of

the St. Georg Association

Georg Westermann and Gerhard Johnson'

Abstract

In this paper we analyze the efficiency of social service units in a way very similar to the

design of hospital efficiency studies. The houses of the St. Georg Association care for

mentally disabled persons. The explicitly formulated goal of the association is to help the

patients to reach a higher quality of daily-life. Our approach shows the possibility to include

qualitative measures into health sector productivity analysis. We design our analysis in such

a way as to provide management information for controlling the service units. This is

accomplished with the help of portfolio techniques and norm strategies.

1 Gerhard Johnson is Professor for Human Resource Management at the Hochschule Harz, University of Applied Studies and Research.

Georg Westermann is Professor for Public Sector Management at the Hochschule Harz, University of Applied Studies and Research.

184 Combining DEA and "Transformation-Stages" ...

Structure

Introduction

2 Questions to be answered

3 St. Georg Association

4 Design of the Investigation

5 Results of the Investigation

6 Conclusion

References

Combining DEA and "Transformation-Stages" ... 185

1 Introduction

The health sector in almost all industrialized countries in the world is consuming a

steadily growing share of those nations' GDP.' It is therefore not surprising that more

and more scientific efforts are being made to develop suitable management tools.

Governmental health authorities responsible for granting budgets to the different

institutions within the sector are often especially interested in discovering inefficient

units.

Developing adequate instruments for measuring the efficiency of different institutions

within the health sector is still an ongoing task. GrosskopfNaldmanis (1987)

convincingly state that "... empirical analysis of productive performance ... is

complicated by the nature of the 'productive' process behind health care delivery.

Clearly, the conceptual output- improved health status - is difficult to measure as an

output."

During the last 15 years there has been a fast growing literature suggesting Data

Envelopment Analysis DEA as an appropriate methodology to measure the

performance of health providing institutions. 3 There seem to be two main advantages

of DEA that convince more and more researchers to apply this linear programming

method:

(I) DEA is able to process with multiple inputs and outputs that are expressed in technical

terms rather than in (often unavailable) cost terms.

(2) DEA allows for different hospitals to employ different production techniques.

The following table (1) provides a roughly structured picture of the DEA efficiency

studies accomplished in the health sector'. It becomes evident that hospitals and

nursing homes are of special interest. This paper is to be seen within this strand of

2 For an empirical illustration see Ferlie E. et al. (1996).

3 A discussion and an overview can be found in Banker/Das/Datar (1989), Brooks (1995) or Breyer/Zweifel (1996).

4 This is, of course, not a complete overview.


literature when it examines the performance of social service units m a way very

similar to the hospital productivity studies.5

Table (1) DEA Efficiency Studies in the Health Sector

\ "I h· II ' ' '·" I ) \II lnpuh ()utpuh I llh.l l' lll..~ ( Ul l lqll

Banker (1984) Nursing hours Patients under 14 years Labor/Capital

Hospitals General service hours Patients between 14 and 65 efficiency

Ancillary service hours Patients older than 65 Quantity oriented

Beds

Meyer/Wohlmann- Cost per case Patient judgement Cost efficiency stener ( 1985) Investment per patient % cases without Quantity/Quality Hypothetical hospitals Patients under 60 years

complications oriented

No emergency patients Differentiated inputs

Banker/Conrad/Straus Nursing services Patients under 14 years Labor/Capital s (1986)

Other services Patients between 14 and 65 efficiency

Hospitals Capital Patients older than 65 Quantity oriented

Grollkopf/Valdmanis Physicians Acute care Labor/Capital ( 1987)

Non-Physicians Intensive care efficiency

Hospitals Admission Surgeries Quantity oriented

Plant assets Ambulatory&emergency care

Borden ( 1988) Beds Cases in 9 DRG categories "Total" efficiency

Hospitals Nonpayroll expenses Quantity oriented

Staff

Nurses

Nyman/Bricker Nursing hours SNF patients Labor efficiency (1989) Social service worker ICF patients Quantity oriented Nursing homes hours Limited care patients Differentiated outputs

Therapist hours Personal care patients Other worker hours

Residential care patients

Banker/Das/Datar 6 total departmental 8 inpatient services Cost efficiency

5 This seems to be appropriate because according to Weisbrod (1991 ) the main characteristics of health care are the strong effects on the quality of life and the complexity of the production processes. Thts also holds for social service units.


(1989) cost categories 2 outpatient services Quantity oriented

Hospitals

Thanassoulis ( 1993) Total cost Teaching units Cost Efficiency

Hypothetical hospitals Regular patients Quantity oriented

Severe patients Differentiated outputs

Donni (1994) Nurses Infants Labor efficiency

Day-Care Centers Cleaning/cooking staff Toddlers Quantity oriented

Qualified staff

ByrnesNaldmanis Registered Nurses Medical-surgical acute Labor/Capital (1994)

Practical Nurses discharges efficiency

Hospitals Management Staff

Medical-surgical intensive Quantity oriented care

Technical Staff Maternity discharges

Aides & orderlies

Beds

Chilingerian (1994) Average length of stay Low-severity cases Cost efficiency

Hospital Physicians Cost ancillary services discharged in a healthier Quality oriented state

High-severity cases discharged in a healthier state

Fare/Grosskopf/ (Estimated) real labor Inpatient discharges Labor/material Lindgren/Roos (1994)

(Estimated) real other Long-term patient bed efficiency

Hospitals input (exc. capital) days Quantity oriented

Ambulatory doctor visits

Norman/Stoker Headquarters Non-psychiatric in-patients Cost efficiency (1991) administration cost discharges&deaths Quantity oriented District Health 7 hospital services cost Psychiatric in-patients bed-Authorities 2 community health days

services cost Attendance day-patients

Day cases

Attendance out-patients

Roos (1997) Total cost Patients Cost Efficiency

Hypothetical eye Change in daily life Strongly quality surgery departments activities oriented

The layout of the paper is as follows. In section 2 we will pose and explain the main

questions within our investigation, while section 3 describes St. Georg Association as

the research object. In section 4 the implementation and design of the public sector

"transformation stages concept" within DEA is of special interest, while section 5


introduces the results of our calculations. Section 6 concludes with some general

remarks.

2 Questions to be answered

The questions we asked when we started our research project were m principle

twofold. On the one hand we wanted to know whether it was possible to trace

efficiency even in a field as difficult as the care for disabled people. The last column

of table (1) above soundly demonstrates that most of the recent studies neglect the

quality of the output produced in the health sector. 6 Thus, our study aims at explicitly

defining and measuring quantitative and qualitative efficiency concepts for the social

service units. On the other hand we were seeking to explore the possibilities of

obtaining results that could be used to provide a management tool for controlling the

system of different service units within the St. Georg Association. Here we wanted to

be as close as possible to the methodologies known by business managers.

3 St. Georg Association

St. Georg is a non-profit organization that cares for mentally disabled persons of

different ages with a variety handicaps. Most of the disabled people live together in

groups that are structured similarly to families. The groups live together in units that

are called ,houses". And there is always a caregiving person to whom the group

members relate most closely.

In addition to the life in the ,families" the disabled persons go to work - when possible

- or attend treatment by psychologists, psychiatrists or other therapists. The ultimate

goal is to enable the group members to lead a life as independently as possible.

The different ,houses" are grouped together within three regions and are controlled by

regional managers who have to report to the central management. In 1997 the ,total

sales" of these three units amounted to 124 Mio. German Marks. Within the next few

6 Moreover, the few studies considering qualitative efficiency concepts are based on the analysis of hypothetical hospitals.


years the organization will be restructured, which should lead to a group structure with

the regional organizations as fully owned subsidiaries.

The aim of the central management for starting this investigation was to develop

instruments to control the system of different houses with respect to the efficient

production of the services provided. In other words, management was looking for a

system for benchmarking the houses. The DEA methodology seems to be an adequate

instrument within that context.

4 Design of the Investigation

Measuring the efficiency in the public sector is in itself a difficult task. Measuring the

efficiency of organizations that provide social services is even more difficult because

of the measurement problems -especially with respect to the output-side. Most of the

outputs lack reliable market prices.

In order to grasp the structure of the production processes in this sector we rely on a

theoretical approach going back to Bradford, Malt and Oates (1969). They divide the

process of the production of public social services into several, so-called, ,stages of

transformation". The word transformation is chosen with respect to the transformation

of inputs into outputs. The distinction between the stages is especially based on the

consideration of different outputs.

For our analysis we concentrate on the following three different ,transformation

stages" given in table (2).

Table (2): Transformation Stages

TS 1: Readiness To Produce Services

TS II: Production of Services

TS III: Effect of Services on Customers

Stage I describes the ability of the different units to efficiently transform the respective

factors of production into the desired capacity. In our context, Stage I is the readiness

to care for a certain and fixed number of disabled patients. Stage II describes the

efficiency of transforming inputs into ,sold" output. Below we will define what ,sold"


output means within this context. Stage III is designed to compare the final outcome of

care on the disabled persons with respect to the required inputs.

Following this approach means to be more precise in the definition of what is the goal

of production in the health sector. All studies in the above table (1) that use "quantity

oriented" efficiency concepts aim at measuring productivity in stages I or II.

GrosskopfNaldmanis (1987) are well aware of this constraint when they label their

approach as a model of " ... the production of the intermediate good- health services."'

Heads of health administrations, politicians and consultants often forget about the

existence of stage III when they aim mainly at input saving activities.

As an analytical instrument to measure the efficiency of the service units, we apply the

Data Envelopment Analysis. We chose DEA especially for two well-known reasons:

(a) DEA offers the possibility to include simultaneously variables that are measured in

different units, such as hours of care, number of beds, staff costs etc. There is no need

to transform or weigh those variables in order to accomplish an evaluation. This is

quite advantageous especially with respect to the output of social service units.

(b) DEA identifies structural differences between the evaluated units as well as efficiency

differentials. The effect is that non-efficient units are compared only against efficient

units or combinations of efficient units that apply the most similar production

technique.

For our computations we apply the following well known CCR8 (or constant returns to

scale) model ofDEA:

mine1-&er s7 -&er si s.t.:

S/

+ s, ;.,,,s7.si~O

0

7 Grosskopfi'Valdmanis (1987), p. 90.

8 It is based on the seminal work of Charnes/Cooper/Rhodes ( 1978).

(I)

Combming DEA and "Transformation-Stages" ... 191

Y1 and X1 are the r- and s-vectors of outputs and inputs respectively of firm 1; Y and X

are the matrices of outputs and inputs of all firms within the sample. The parameter 81

to be minimized accounts for efficiency, the n-vector A.1 provides information about

reference sets, s+ and s· are the excess inputs and output slacks respectively, vector eT

contains only elements 1, and E is the positive so-called Non-Archimedean constane.

We calculate input oriented models (TS I and TS II) as well as an output-oriented

model (TS III). In order to clearly rank the houses we additionally include the possible

input and output slacks into the efficiency measure. Adding the slacks to the necessary

proportional reduction is accomplished in the way suggested by Ali/Lerme ( 1990) or

Chang/Guh ( 1991).

Before the presentation of some of the results we have to define the input/output

models that we apply to describe the production processes on the three transformation

stages. We decided to calculate separate models for each transformation stage because

they shed light on totally different managerial problems.10 The following table (3)

provides the details.

Table 3: 1/0 models with respect to transformation stages

;\I odd Inputs Outputs

TS I (capacity) Sta!T(FTE) Maximum capacity (beds•days)

Other costs (DM)

TS II (production) Staff(FTE) Days charged to customers

Ot.hcr costs (DM)

TS Ill (effect) Staff(FTE) Hours that ,parent persons" care for group

Other costs (DM) members

Days charged to customers

9 See Chames/Cooper (1984).

10 Additionally, the output side seems to provide only constrained possibilities for the management of the DMUs to substitute between outcomes.


We are aware of the fact that especially the variable on the output side of our model

TS III is a very rough proxy for the effect of care on the disabled persons.'' The

validity of that variable is based on the assumption that the more hours of care by a

parent person a disabled person experiences the stronger is the positive effect on the

personal development.

The management of the St. Georg Association is about to introduce a system of

assigning an outside tutelary to each of the disabled patients. Management plans to

direct a questionnaire to those neutral guardians in order to collect information about

the development of the patients. After receiving those questionnaires, the TS III model

can be re-calculated with more precise data.

5 Results of the Investigation

In this section we are going to present the results of our investigation. We proceed by

providing the efficiency scores of the DEA runs for TS I and TS II first. Then those

results are combined into a strategic management portfolio. This portfolio serves as a

basis to develop management norm strategies. The findings of TS III are then added

and interpreted with respect to the outcomes of TS I and TS II.

Table (4) shows how efficiently the different houses produce their respective

maximum capacity. This represents the efficiency concept connected with stage I. In

column 1 we plotted the position of a certain house within this ranking. Column 2

contains the identification number of the houses, and column 3 the input oriented DEA

efficiency score. Columns 4 and 5 are concerned with the number of staff members

and the amount of other costs that would have to be saved to become efficient. In the

last column (Peer House) we display the houses that should serve as yardsticks for the

inefficient ones. Those were identified by using the weights of the efficient units

within the "A vectors of the inefficient houses.

We do not want to extend the interpretations of those numbers any further, especially

because this first run is very conventional and the possibilities to save inputs are

evident.

11 For more details and especially the variable "care-output-unit" see Johnson et al. (1999).


With respect to the heterogeneous production techniques, the two efficient houses

represent production that is both more capital intensive (DMU #11) and more staff (or

labor) intensive (DMU #6). This differentiation was confirmed by the intuitive

judgements of the members ofthe central management.

Table 4: Results TS I (capacity)

I'OSI I I< J:\ 1101 Sl·. 1· 1·1·1< W\<' Sl \H 0111.1( I'•. I( s \\I'<; ('OS IS ll<ll ......

S\\ I\(,

l 6 1,00000 0,0 0,0 6

I II 1,00000 0,0 0,0 II

3 10 0,98288 0,2 4703,0 II

4 31 0,94887 0,6 15 177,7 II

5 28 0,94455 1,1 15091,3 6

6 17 0,93614 0,9 19912,1 II

7 4 0,93340 2,7 35878,2 6

8 27 0,80799 4,0 61580,7 6

9 21 0,78799 8,2 154260,6 II

10 5 0,75016 4,7 117123,5 II

11 20 0,74672 4,6 118707,8 II

12 22 0,74234 10,8 289018,0 II

13 I 0,73662 11,9 262986,0 II

14 9 0,73023 6,8 146458,7 11

15 25 0,73019 4,9 62984,7 6

16 23 0,72676 10,0 203501,8 II

17 12 0,71258 9,9 167522,2 11

18 2 0,70986 9,8 249881,3 11

19 24 0,69967 7,2 96810,2 6

20 19 0,67702 11,6 200210,1 II

2 1 26 0,67297 2,9 184511,7 II

22 3 0,67141 4,4 98097,6 II

23 14 0,66830 5,3 204715,0 II

24 8 0,66827 6,2 100646,2 6


25 16 0,66425 7,1 152950,6 II

26 29 0,66077 8,9 128310,0 6

27 18 0,65257 14,2 277628,9 II

28 13 0,65161 8,7 135980,8 6

29 30 0,62609 9,0 146981,8 6

30 15 0,59932 14,4 393121 ,6 II

31 7 0,59261 12,0 246703,6 II

In the next table (5) we present the results of the TS II run. Here we want to analyze

how efficiently the houses adapted the consumption of inputs to the amount of care

that could really be sold to customers. For the managers of St. Georg Association it

contains information about re-scaling possibilities for the inefficient DMUs. The

structure of this table is exactly the same as in table (4) above.

The most important result of this table is that there seem to be differences in the ability

of the managers of the houses to adapt to the demand for care. Some houses produce

the readiness to care quite efficiently (house #6) but fail in adjusting to the demanded

, bed-days" by customers. Others seem to offer their capacity with too high amounts of

inputs but afterwards produce the sold ,care days" efficiently.

It should become clear that identifying those different patterns of strength and

weaknesses leads to developing management strategies for the respective houses. We

have applied the portfolio technique to assess the positions of the different houses with

respect to their performances.

Table 5: Results TS II (production)

I'OSIIIO' 1101 ..... : Fl FIC IF '< \ S I \I+ 0 IIIH( 1'. FR S\\ I'C; ( "OSIS ll<ll SF

S\\I'C ; I 3 1,00000 0,0 0,0 3

I 11 1,00000 0,0 0,0 II

I 28 1,00000 0,0 0,0 28

4 4 0,99178 0,3 4428,2 28

5 17 0,96659 0,5 10417,5 3

6 10 0,90337 1,1 26545,2 3

7 31 0,89758 1,2 30402,9 3


8 9 0,85700 3,6 77635,0 3

9 27 0,85122 3,1 47716,1 28

10 25 0,82618 3,1 40576,7 28

II 6 0,81664 6,2 74978,5 28

12 21 0,79868 7,8 146482,4 3

13 22 0,79663 8,5 228120,7 II

14 I 0,78070 9,9 218971,9 3

15 20 0,76722 4,2 109099,8 3

16 5 0,76304 4,5 111085,4 3

17 24 0,75913 5,8 77643,5 28

18 23 0,75419 9,0 183072,7 3

19 12 0,74754 8,7 147145,8 28

20 30 0,72423 6,6 108403,5 28

21 19 0,71078 10,4 179282,9 28

22 2 0,70677 9,9 252542,6 3

23 29 0,70338 7,8 112193,3 28

24 16 0,69599 6,4 138491,5 3

25 13 0,69104 7,7 120590,8 28

26 18 0,68704 12,8 250084,1 3

27 8 0,66919 6,2 100367,1 28

28 14 0,66504 5,4 185319,6 II

29 26 0,63890 3,2 132714,7 II

30 15 0,63662 13,1 356525,2 II

31 7 0,58951 12,1 248580,8 3

The next figure (1) is a first attempt to plot the strategic positions of the DMUs.

Drawing lines with respect to the average performance in the two calculations under

consideration, one can assign each house to one of four strategic fields. 12 One is now

able to discuss with the management of the St. Georg Association the development of

norm strategies for these fields. Norm strategies are going to serve as an instrument for

the strategic controlling of the different houses. Moves of DMUs from one field into

another field indicate changes in their respective efficiency performances and deserve

modified strategies.

12 For an application of this approach to bank branches see Westermann (1999).


Strategies for the fields:

• Houses assigned to field I supply their capacity with above average efficiency. But the

production of the sold output shows weaknesses. Here it seems to be necessary either to

intensify the efforts to gain new customers or to downscale the house.

• Houses in field II should try a clear strategy of contraction in inputs because they are too

input consuming in both respects: readiness to care and production of care.

• Houses situated in field III should be able to expand their capacities without a

proportionate input increase.

• The units in the fourth field are above average within both respects. They should expand

their businesses if that is possible. They also should serve as benchmarks for inefficient

houses.

Thus, the analysis accomplished is still quite orthodox with respect to the efficiency

concept pursued. In a next step we try to integrate the results (see table 6) from the TS

III DEA run. This computation is output oriented and aims at displaying the

possibilities of the houses to dedicate further time to the disabled persons. The best

DMUs efficiently transform the three inputs Staff (FTE), Other Costs (DM) and Days

Charged to Customers into the single output Hours , Parent Persons" Care for Group

Members. 13

Figure (1): Strategy Portfolio

Strategy Portfolio

Ill >. 1,1 IV Ill u _., Ql c 0,9 c G)

:s u Ill ca !f 0,7 • 2! G)

0,5 0,5 0,7 0,9 1,1

production efficiency

13 For more details and especially the variable "care-output-unit" see Johnson et al. (1999).


Table 6: Results TS III (outcome)

I'OSIIII 1"\ 1101 sr I· HU I I-"'<\ I'< HI" \SI-" 1'1· 1"1{ .. 1'\IH' I " 1101 Sl 1'1· HSO' II

I 6 1,00000 0,0 6

I 7 1,00000 0,0 7

I 9 1,00000 0,0 9

4 29 1,00000 0,0 29

5 4 1,07474 2876,0 29

6 27 1,08416 1887,6 29

7 28 1,13794 2528,6 29

8 25 1,16942 2582,5 6

9 18 1,26890 10957,8 7

10 19 1,28238 9585,0 29

11 12 1,32613 10566,7 29

12 5 1,35236 7829,4 9

13 17 1,43528 7050,2 9

14 10 1,45590 6359,5 9

15 30 1,48588 9382,5 29

16 21 1,49442 17828,5 9

17 8 1,51976 7143,3 29

18 1 1,54622 24066,4 9

19 16 1,54752 10123,9 7

20 2 1,55580 18348,5 9

21 15 1,66954 21109,4 9

22 23 1,67482 20568,7 9

23 13 1,70357 11569,7 29

24 31 1,72456 9212,7 9

25 20 1,76058 12763,7 9

26 22 1,93421 39759,3 9

27 24 1,96320 12027,8 29

28 3 2,12601 11784,5 9

29 11 2,13517 11447,8 9

30 26 3,02627 17708,1 9

31 14 3,37857 28205,3 9


What can social service managers learn from this third strand of computations? It

worth to notice is for example that house #6 that was efficient in producing capacity to

care is also efficient in caring for the patients. House #7 in contrast performs very

badly in capacity production but is efficient in dedicating time to the group members.

For reasons of a comprehensive insight in the efficiency structures of St. Georg

Association it seems to be appropriate to construct a strategic portfolio similar to

figure ( 1 ). In order to avoid a three-dimensional plot in table (7) we apply a three-digit

approach to indicate high/low efficiency in the three DEA runs TS I, TS II and TS III.

Column 5 in table (7) additionally provides the allocation of houses across the

respective fields.

Table 7: Strategic Fields in the Three Dimensional Case

·1 Sl rttkil•nc~ 'I Sll Eltkil·nc~ I Sill l·: lficil·m·~ l· idcl 1-'idcll'upulatinn

High Low Low I -High High Low 2 2

High High High 3 7

High Low High 4 -Low Low Low 5 2

Low High Low 6 2

Low High High 7 2

Low Low High 8 II

It is obvious that a remarkable percentage (27%) of houses is situated in field 3 and

thus can be labeled as being above average in their efficiency with respect to all three

efficiency concepts. Another global feature seems to be the trade-off between TSIITSII

and TSIII. Exactly 50% (field 2 and 8) of the DMUs trade capacity and production

efficiency against outcome efficiency.

With respect to the strategies that should be pursued for the houses in the fields 1-8 we

will concentrate only on one illustrating example. The distinguishing feature of the

most populated field 8 is the low efficiency in the transformation of staff and other

cost into capacity and sold beds. In those houses the ,parent persons" can avoid

management and auxiliary tasks because other staff are responsible for that.


Management should explore the possibilities of especially reducing auxiliary staff

without decreasing the extraordinary high level of care. 14

6 Conclusion

In this paper we analyze the efficiency of social service units in a way very similar to

the design of hospital efficiency studies. The houses of the St. Georg Association care

for mentally disabled person. The explicitly formulated goal of the association is to

help the patients to reach a higher quality of daily-life. Thus, it is not reasonable to

tackle the efficiency problem with bare quantitative concepts.

Our approach shows the possibility of including qualitative measures into health sector

productivity analysis. Moreover, we design our analysis such that it will provide

management with information for controlling the service units. This is accomplished

with the help of portfolio techniques and norm strategies.

Our variable for the effects of care on the disabled persons is a very rough proxy. The

planned questionnaire containing a neutral judgement on the condition and

development of the disabled persons is going to provide a more adequate database. A

second weakness of our present analysis is the absence of data with respect to the

structure of handicaps and the differences between the houses. This problem is going

to be solved by assigning the group members to different handicap or case groups. The

number of persons within those groups can then be used as output variables in further

DEA calculations.

We are aware of the fact that the present analysis is not at all perfect but we hope to be

on a way that it nevertheless can help the management of the St. Georg Association to

care more efficiently for their customers. But caring more efficiently must not neglect

all those non-economic variables inherent in human relations.

14 In addition, it would be worth analyzing the situation in those houses with a more decomposed staff variable.


References

Ali A.l. and C.S. Lerme (1990): Data Envelopment Analysis Models: A Framework, School of

Management, University of Massachusetts at Amherst, Working Paper.

Banker, Rajiv D. (1984): Estimating Most Productive Scale Size Using Data Envelopment Analysis,

European Journal of Operational Research, Vol. 17, pp. 35-44.

Banker, Rajiv D., Conrad, Robert F. and Strauss, Robert P. (1986): A Comparative Application of

Data Envelopment Analysis and Translog Methods: An Illustrative Study of Hospital

Production, in: Management Science, Vol. 32, No. I, pp. 30-44.

Banker R.D., D. Somnath and S.M. Datar (1989): Analysis of Cost Variances for Management Control

in Hospitals, Research in Governmental and Nonprofit Accounting, vol.5, 1989,pp.269-291.

Borden, J. P. (1988): An Assessment of the Impact of Diagnosis-Related Group (DRG)-Based

Reimbursement on the Technical Efficiency of New Jersey Hospitals Using Data Envelopment

Analysis, Journal of Accounting and Public Policy, Vol.7, No.I, pp.77-96.

Bradford, D.F., R.A. Malt and W.E. Oates (1969): The Rising Cost of Social Public Services: Some

Evidence and Reflections, National Tax Journal, 22 (1969), pp. 185-213.

Breyer, F. and Zweifel P. (1996): Health Economics, Oxford University Press, Oxford.

Brooks, R. ( 1995): Health Status Measurement: A Perspective on Change, MacMillan Press.

Byrnes P. and V. Valdmanis (1994): Analyzing Technical and Allocative Efficiency of Hospitals, in:

Charnes A., W.W. Cooper, A.Y. Lewin and L.M. Seiford (1994): Data Envelopment Analysis:

Theory, Methodology and Application, Kluwer Academic Publishers, Boston, pp. 129-44.

Chang, K.-P. and Guh, Y.-Y. (1991): Linear Production Functions and the Data Envelopment

Analysis, European Journal of Operational Research, Vol. 52, pp. 215-223.

Charnes, A. and Cooper, W.W. (1984): The non-Archimedean CCR Ratio for Efficiency Analysis: A

Rejoinder to Boyd and Flire, European Journal of Operational Research, Vol. 15, pp.333-334.

Charnes, A., Cooper, W.W. and Rhodes, E. (1978): Measuring the Efficiency of Decision-Making

Units, European Journal of Operational Research, Vol. 2, pp. 429-444.

Chilingerian, Jon A. (1994): Exploring Why Some Physicians' Hospital Practices Are More Efficient:

Taking DEA Inside the Hospital, in: Charnes A., W.W. Cooper, A.Y. Lewin and L.M. Seiford

(1994): Data Envelopment Analysis: Theory, Methodology and Application, Kluwer Academic

Publishers, Boston, pp. 1167-193.

Donni, Olivier (1994): Efficiency of Day-Care Centers in Belgium, Paper presented at the European

Economic Association, Ninth Annual Congress, Mastricht.


Fare R., S. GroBkopf, B. Lindgren and P. Roos (1994): Productivity Developments in Swedish

Hospitals: A Malmquist Output Index Approach, in: Charnes A., W.W. Cooper, A.Y. Lewin

and L.M. Seiford (1994): Data Envelopment Analysis: Theory, Methodology and Application,

Kluwer Academic Publishers, Boston, pp.253-72.

Ferlie E., A. Pettigrew, L. Ashburner and Louise Fitzgerald (1996): The New Public Management in

Action, Oxford University Press, Oxford, p. 2.

Grosskopf, S. and Valdmanis, V. (1987): Measuring Hospital Performance: A Non-Parametric

Approach, Journal of Health Economics, Vol. 6, pp. 89-107.

Johnson G., H. Johnson and R. Klaus (1999), Personalcontrolling im sozialen Bereich -

Qualitatssicherung bei der Betreuung behinderter Menschen, in: PERSONAL, 51. Jg. 1999,

Volume 4.

Meyer, Manfred and Wohlmannstetter, Viktor (1985): Effizienzmessung in Krankenh!iusern, in: ZfB,

Vol. 55, No. 3, pp. 262-280.

Norman, Michael and Stoker, Barry (1991): Data Envelopment Analysis - The Assessment of

Performance, Wiley, Chichester.

Nyman, J. and Bricker, D. (1989): Profit Incentives and Technical Efficiency in the Production of

Nursing Home Care, Review of Economics and Statistics, pp. 586-594.

Roos, Pontus ( 1997): Measurement of Output and Productivity of Hospital Services: A Discussion of

the Malmquist Index Approach, The Swedish Institute for Health Economics Working Paper,

1997:7.

Thanassoulis, E. (1993): A Comparison of Regression Analysis and Data Envelopment Analysis as

Alternative Methods for Performance Assessments, Journal of the Operational Research Soc.,

Vol. 44, No. II, pp. 1129-1144.

Westermann G. (1999): De Ia comptabilite d'efficience a le developpement de Ia strategie -

Application de Ia methode DEA aux succursales des banques, forthcoming in: Badillo P. -Y.

and J.C. Paradi (eds.).

DEA in the ecological context- An overview•

Katrin Allen'

Abstract

Data envelopment analysis has a high application potential for environmental manage

ment/ecological controlling, especially for eco-benchmarking projects. The aim of this paper

is to provide some basics for DEA applications within this scope. DEA assumes that inputs

and outputs are 'goods ', but from an ecological perspective also 'bads ' have to be consid

ered. The potential and state-of-the-art of DEA in the ecological context are outlined. Model

ling alternatives, axiomatic modifications, and application driven extensions are described,

followed by a concept for DEA applications in environmental performance measurement as

well as a framework for future research.

1 Financial support by the Deutsche Forschungsgemeinschaft (DFG) is gratefully acknowledged.

2 Technical University of Aachen, Templergraben 64, D-52064 Aachen, Germany

[email protected]

204 DEA in the ecological context- An overview

Structure

Introduction

2 Environmental Management and Ecological Controlling

3 DEA in the ecological context

4 Ecological extension ofDEA

5 A concept for applications ofDEA in EPM

6 Summary and outlook

References

DEA in the ecological context- An overview 205

1 Introduction

Data envelopment analysis (DEA) is a collection of models measuring the relative effi

ciency of decision making units (DMUs) with linear programming techniques.' DEA

shows a high potential for applications within the field of environmental manage

ment/ecological controlling (EM/EC). Private and non-profit organisations look for

practical, flexible, and objective tools supporting EM/EC, due to the ecological and

economic benefits to be gained by environmental protection activities. The aim of this

paper is to provide some basics for applications ofDEA in EM/EC. The main problem

is that DEA assumes inputs and outputs are 'goods'. This assumption no longer holds,

when - from an ecological perspective - consideration of outputs to be minimised (e.g.

S02-emissions) and inputs to be maximised (e.g. recycling of wastepaper) is required.

The paper is organised as follows: Section 2 starts with an overview of EM/EC, its

problems and instruments. Section 3 points out the potential and limitations ofDEA in

the ecological context and provides a literature review. Selected aspects of an ecologi

cal extension of DEA are examined in section 4, grouped into modelling alternatives,

axiomatic modifications, and application driven extensions. A concept for applications

in environmental performance measurement is suggested in section 5 considering the

potential and limitations outlined so far. The final section provides a summary and a

framework for future research.

2 Environmental Management and Ecological Controlling

"Organizations of all kinds are increasingly concerned to achieve and demonstrate

sound environmental performance by controlling the impact of their activities, prod

ucts or services on the environment [ ... ]".' Environmental protection has become a

'trendy' topic not just because companies want to develop and achieve the idea of

3 For an introduction into DEA see e.g. Seiford {1996) and Chames et al. {1994), chapters I to 3.

4 Introduction to the ISO 14000 series of standards and guidelines in the field of environment developed from the International Organization for Standardization. For details see http://www.iso.ch.


'sustainable development''. It has become an important competitive factor as there are

increasingly more economic benefits to be gained:

• General growth from the stakeholders about environmental matters

and a sustainable development.

• Implementation of increasingly stringent environmental legislation

and economic tools, e.g. taxes, tradable permits.

• Identifying and taking advantage of ecological factors of perform

ance, e.g. improvement of the corporate image, cost savings.

• Identifying and taking countermeasures against potential ecological

risks, e.g. fines, bad publicity.

Environmental management (EM) is the part of the overall management that has to

pursue environmental objectives in the planning, implementation, and control of inter

actions in all parts of a company. For this the EM has to implement a corresponding

organisational structure and incentives for the ecological motivation of the employees.•

The increasing importance of EM is underlined by the efforts of the European Union.

The objective of EMAS' is to promote continuous improvements in the environmental

performance of industrial activities by committing sites to evaluate and improve their

environmental performance and provide relevant information to the public. Addition

ally, the norm ISO 14001' "[ ... ] specifies the requirements for an environmental man

agement system, to enable an organization to formulate a policy and objectives taking

into account legislative requirements and information about significant environmental

impacts."

5 "Sustainable development is development that meets the needs of the present without compromising the ability of future generations to meet their own needs." World Commission on Environment and Development (1987), p. 43.

6 For an introduction into EM see e.g. Dyckhoff(l998b).

7 Council regulation EEC No 1836/93 of 29 June 1993 allowing voluntary participation by companies in the industrial sector in a Community ~;co-management and audit .scheme. The full text is provided on http://www.emas.lu.

8 ISO 14001 'Environmental management systems- Specification with guidance for use', see fn. 4.


A specific ecological controlling can support the extensive information and co-ordi

nation needs of environmental management. Ecological controlling (EC) can be de

fined as a subsystem of the overall management and the overall controlling system. Its

task is to co-ordinate the development and coupling of the ecologically-oriented plan

ning, control, and information search as subsystems of the EM. By doing this, it sup

ports the ability for co-ordination, reactions, and adaptability of the strategic and op

erational EM and therefore guarantees an improvement in the efficiency and effective

ness of the ecologically relevant decisions and activities of the overall management.•

To support the EM, the EC has to develop various powerful instruments10• Strategic

instruments are e.g. ecological early warning systems, strength-weakness-profiles, risk

management, the estimation of potential impacts and acceptability of new technologies

or products, ecological value chain analysis, and ecological portfolio analysis. Typical

operational instruments are ecological checklists and life cycle assessment. Addition

ally several concepts for environmental accounting and ecological budgeting can be

mentioned. Eco-benchmarking is usually based on environmental performance indica

tors, which also represent an original EC instrument. Both can show a strategic or op

erational orientation. With respect to the focus of this paper three instruments are de

tailed below:

Life cycle assessment (LCA)" is a process to analyse and assess the environmental

impacts of a product, process or activity over its whole life cycle. A LCA study usually

contains four steps:

I. Goal definition and scoping

2. Inventory analysis

All material and energy flows that cross the system boundary are

identified and quantified. The result is the so-called material and en

ergy balance (inventory table).

9 Ecological modification of Horvath (1994), p. 144, suggested by Rudiger (1998), p. 283.

1° For extensive overviews see e.g. Gunther (1994) and Rudiger (1998). The classification strategic/operational is not always clear.

11 For an introduction into LCA see e.g. Miettinen!HiimiiHiinen ( 1997).


3. Impact assessment

The material and energy balance is translated into potential environ

mental impacts. Usually weights are used to aggregate some repre

sentative inputs and outputs into one/several indicators.

4. Improvement assessment

Options for reducing the environmental impacts of the regarded pro

duct, process or activity are identified and evaluated.

Environmental performance indicators (EPis) 12 can be defined as numbers, which

capture a quantifiable aspect of ecological relevance in a concentrated manner. EPis

appear as absolute or relative numbers based on monetary, non-monetary or partially

monetary size (e.g. cost for waste disposal, amount of waste in tons, cost for waste dis

posal per ton of waste). EPis are integrated in several instruments of EC, but they also

represent an original instrument, supporting information, planning, control, and

checking purposes on a strategic as well as on an operational level. They can be used

for either target-performance comparisons, comparisons over time, or benchmarking

projects. ISO 14031 differs between environmental condition indicators and environ

mental performance indicators, the latter including management performance indica

tors and operational performance indicators. 13 Environmental operational performance

indicators, which are based on the material and energy balance, are most commonly

used. Isolated EPis can not reflect complex systems and interrelations. Therefore it is

recommended to choose some meaningful, logically-related EPis to develop an envi

ronmental performance indicator system (EPIS). 1' EPis are meaningless without com

parative values, which can be determined in benchmarking projects.

Eco-benchmarking can be defined as the continuous process of ecological improve

ments of products, processes, methods, and functions by comparison with the 'best of

the class' and realisation of the 'best ecological practice' in their own company. 1' Eco

benchmarking projects can focus on comparisons over time, comparisons within a

company, and comparisons among companies operating in similar or different

12 For details see Seidel eta!. (1998).

13 ISO 14031 'Guidelines on environmental performance evaluation', see fn. 4.

14 See e.g. the EPIS in Loew/Kottmann (1996), p. 12.

1' See Goldmann!Schellens (1995).


branches of industry. In analogy to the ISO 14031 indicators, objects of an ecological

benchmarking project can be:

1. Environmental impacts

Benchmarking of adverse of beneficial changes to the environment

resulting from the organisations' activities, products, or services.

2. Environmental practice

Benchmarking of normative/strategic/operative elements of the EM

system.

3. Environmental performance

Benchmarking of measurable results of the EM system, related to the

organisation's control of its environmental aspects.

There are numerous problems with which EM/EC are generally associated, especially

in regard to the instruments LCA, EPis, and eco-benchmarking: The database needed

for the instruments is extensive, novel, and complex. Especially as there are multiple

inputs and outputs which are measured on different scales. For a full description of a

product, sometimes thousands of inputs and outputs are required. The functional rela

tionships between the inputs and outputs are often unknown. Usually there are no mar

ket prices or objective preferences for the consumption of natural resources and the

emission of harmful chemicals, to allow for the aggregation of the inputs and outputs.

A lot of approaches in the literature build on weight coefficients, equivalence numbers,

or monetary conversion. Although, the assessment of such values turns out to be very

difficult due to subjective and political influences. Further problems stem from the da

tabase. The standardisation of the inventory analysis step in LCA and of EPis is an

important presupposition for comparisons, especially for eco-benchmarking. Though

so far, standardisation is at a low level and countermeasures like ISO 14000 are still in

their infancy. However, even if companies have reliable and well-developed environ

mental data, it might not be available to the public.

For these reasons there is an increasing need for practical, flexible, and objective tools

to support the EM/EC. In particular, one looks for methods which allow for considera

tion of multiple inputs and outputs without requiring difficult value judgements for

ecologically-oriented comparisons of plants, processes, products, management systems

or functions.


3 DEA in the ecological context

3.1 Problem framework

DEA and activity analysis approaches in general assume that inputs and outputs are

'goods', i.e. objects with a positive value, where a positive value does not necessarily

mean a monetary advantage. In the environmental context, we also have to consider

'bads', i.e. objects with a certain negative value. Furthermore, 'neutrals' as objects

without any kind of value with respect to the decision problem at hand, may also arise.

Combining these three classes with the criteria 'position as input or output in the pro

duction process' results in six categories of different ecological desirability, which are

shown injigure 3.1.

neutral bad

Input reduct

output product

D D desirable indifferent undesirable

Figure 3.1 Categorisation of ecologically relevant objects according to the

'standard case'

The 'standard case' specification represents a weak preference order for a certain cate

gorisation of ecologically relevant objects. It will be illustrated in the following by

means of the example of a waste-burning power plant as shown in figure 3.2. 16

16 Regarding the 'standard case' and the waste burning power plant example see Dyckhoff (1994) p. 65 and Dyckhoff (1998a) p. 97, respectively.


/ / 470 kWh

970kWh

1000 kg .. 6000m'

waste 7001 8001 .. heating

60ka power

330kg 6000m ' ., plant 890 kWh

/

Figure 3.2 Categorisation for a waste heating power plant

Factors and products as goods on the input and output side respectively, represent the

classical DEA case. For a waste-burning power plant, the use of water should be kept

to a minimum. The products, electric power and heat, which are the aims of the trans

formation process, should be maximised. A bad on the output side is undesirable, and

therefore its quantity should be minimised, in this example pollutants such as waste

gas, waste water, scrap, and cinders. The expression 'reduct' is used for the symmetric

case of bads on the input side. Waste to be burned at the power plant represents such

an undesirable object whose input should be maximised. If one is completely indiffer

ent towards air on the inputs side, and residual heat on the output side, they represent a

by-factor and by-product respectively.

The 'standard case' specification of ecologically relevant objects is chosen here as the

problem framework for an ecological extension of DEA. Though it should be noted

that the categorisation can be subjective and arbitrary. Furthermore, the categorisation

can change due to time, place, information, quantity or quality. Imagine for example,

that the output residual heat could be sold to a nearby swimming centre or that there is

a penalty for emitting more than a given limit.

3.2 Potential and limitations

Miettinen/Hamalainen (1997) "[ ... ]see DEA as a promising tool for LCA." Within the

scope of air pollution monitoring, Cooper et a!. (1996) point out that comparitive

evaluations of performance like DEA "[ ... ]could lead to improvements from learning


how the very best performers deal with satisfying constraints." The idea to apply DEA

within the scope ofEM/EC builds on two trains of thought:

Ignoring the ecological perspective for the moment, DEA has in general proved its

'controlling' potential. Following Wemer/Brokemper (1996) we are able to differen

tiate applications for information, planning, and control systems. Within the area of

intelligence activity, DEA is predominantly used for weak point analyses and for the

preparation and analysis steps in benchmarking projects." DEA can also support the

planning of future activities, e.g. the assessment of minimal cost input combinations,

the development of investment strategies or the choice of location." The efficiency

score represents a comprised feedback, which can be used for control purposes, espe

cially for control over time in DEA window analysis." Furthermore, DEA may support

the generation of control models.'"

From a different perspective, there are some arguments underlining the 'technical'

potential of DEA in the ecological context: Activity analysis based approaches like

DEA show special qualities for EM/EC. Especially as their structure is comparable to

that ofthe inventory table in LCA, as an important database ofEM/EC." DEA explic

itly allows for the consideration of multiple inputs and outputs measured on different

scales. DEA does not require information on functional relationships or a priori

weights for the aggregation of different impacts of production processes on the envi

ronment. The general questions addressed by DEA are important for ecological com

parisons. For example, which inputs and outputs cause the inefficiency of a DMU, and

to what extent? Finally, DEA is flexible regarding modelling needs and a lot of exten

sions to the basic DEA models are relevant for applications to the context at hand.

Apart from this confident judgement, conceptual limitations and pitfalls ofDEA have

to be considered, e.g. its sensitivity to outliers or model choice, and unreasonable re

sults due to the flexibility of the weights. 22 The outlined potential is further endangered

17 Schefczyk (1993) and Cook et al. (1992).

18 Ray/Kim (1995) and Athanassopoulosffhanassoulis (1995).

19 Day et al. (1995).

20 Epstein/Henderson (1989).

21 See for detailed arguments Fl!re et al. (1996), p. 162, and Souren!Rildiger (1998), p. 306-308.

22 See e.g. Stolp ( 1990).


by the practical problems occurring within the field of EM/EC mentioned in section 2.

Applications in the ecological context will have to cope with a high number of inputs

and outputs in combination with a small number of comparative objects at a low level

of standardisation respectively with incomplete data. This relationship may weaken the

differentiation power of DEA. Though first, solutions for modelling 'bad' objects in

contrast to the classical DEA focus on 'goods' have to be found. The following litera

ture review reveals different approaches as to how to address this problem.

3.3 Literature review

The DEA literature already includes a number of articles dealing with either applica

tions or theoretical investigations within the ecological context. To the best of my

knowledge and using the 'published' data available (in English), table 3.1 gives an

overview. The articles are sorted by 'year of publication' as well as 'connection in

contents'. The columns 'type' and 'essentials' do not summarise the complete articles,

but the facts relevant for the ecological extension of DEA. Some of the listed articles

do not explicitly mention DEA, though there are many more related papers (e.g. fo

cusing on the use of distance function and multi-criteria approaches in the ecological

context), which could not be included in this review."

Only five articles exclusively focus on the theoretical background of undesirable out

puts in DEA. A fact that is all the more surprising when considering that undesirable

outputs do not only appear in the ecological context, but also, for example, within

health care (the complications of medical operations) or within business (tax pay

ments). Furthermore, the articles show very different and sometimes arbitrary ap

proaches regarding the treatment of undesirable outputs. In section 4, selected aspects

mentioned in table 3.1 are explained and discussed in detail.

23 See e.g. Fare et al. (1993) or Cooper et al. (1996).

I Aut

hor(

s)

I Typ

e I Es

sent

ials

I A

ppli

cati

on

I Rem

ark

s I

Fir

e e

t al.

(19

89)

T+

a I

stro

ng/w

eak

disp

osab

ilit

y o

f und

esir

able

out

puts

U

.S. p

aper

mil

ls

o se

vera

l non

·rad

ial h

yper

boli

c ef

fici

ency

mea

sure

s tr

eati

ng i

nput

s, d

esir

able

out

puts

and

un

desi

rabl

e ou

tput

s as

ymm

etri

call

y o

conv

ersi

on in

to L

P ap

prox

imat

ions

B

alle

t al.

(19

94)

T+

a o

4 m

odel

s w

ith

line

ar o

r hy

perb

olic

eff

icie

ncy

mea

sure

on

prod

ucts

or

prod

ucts

and

U

.S. a

gric

ultu

re

poll

utan

ts,

assu

min

g w

eak

disp

osab

ilit

y o

f und

esir

able

out

puts

o

calc

ulat

ed s

hado

w v

alue

s us

ed to

adj

ust a

con

vent

iona

l tot

al f

acto

r pr

oduc

tivi

ty g

row

th

mea

sure

for

the

gene

rati

on o

f und

esir

able

out

put

Kao

/Y an

g (1

991)

T

+a

desi

rabl

e ou

tput

'for

est s

tock

ing

in m

'lha

' cho

sen

as a

mea

sure

for

soi

l con

vers

atio

n fo

rest

dis

tric

ts i

n T

aiw

an

repl

acin

g th

e un

desi

rabl

e ou

tput

'soi

l ero

sion

'

Hay

nes

et a

l. (

1993

) t

max

imis

atio

n o

f rec

ipro

cal o

fCC

R-m

odel

'nor

mal

isat

ion'

wit

h ch

emic

al r

esid

uals

take

n as

(th

e on

ly)

inpu

ts

Hay

nes

et a

l. (

1994

) T

+a

max

imis

atio

n o

f rec

ipro

cal

ofC

CR

-mod

el 'n

orm

alis

atio

n' w

ith

chem

ical

res

idua

ls ta

ken

poll

utio

n pr

even

tion

in

as (

the

only

) in

puts

N

.J. c

hem

ical

pla

nts

Gol

any

et a

l. (

1994

) a

II S

02-

emis

sion

s tr

eate

d as

a c

ateg

oric

al v

aria

ble

on th

ree

leve

ls s

uch

that

a D

MU

can

po

wer

pla

nts

in Is

rael

on

ly b

e co

mpa

red

to D

MU

's in

its

or a

bet

ter c

ateg

ory

o in

corp

orat

ion

of s

tand

ards

Lov

ell e

t al.

(19

95)

a II

Car

bon

and

nitr

ogen

em

issi

ons

are

conv

erte

d by

taki

ng th

eir

reci

proc

al

mac

roec

onom

ic p

erfo

r-m

ance

OE

CD

cou

ntri

es

Fir

eJG

ross

ko

pf (

1995

) t

II st

rong

/wea

k di

spos

abil

ity

of u

ndes

irab

le o

utpu

ts

o in

corp

orat

ion

of j

oint

ness

bet

wee

n de

sira

ble

and

unde

sira

ble

outp

uts

I o

disc

ussi

on o

f Hay

nes

et a

l. (1

993)

I

o no

n-ra

dial

mod

el a

ccou

ntin

g fo

r de

sira

ble

outp

uts

to b

e in

crea

sed

and

at th

e sa

me

tim

e

I fo

r un

desi

rabl

e ou

tput

s an

d in

puts

to b

e de

crea

sed

at d

iffe

rent

rat

es

Tyt

eca

(199

6)

t o

CC

R-m

odel

s 'n

et p

rodu

ctio

n',

'pol

luta

nt=

fact

or',

and

'nor

mal

isat

ion'

as

EP

is

lite

ratu

re r

evie

w

/ o

disc

ussi

on o

fFD

H a

nd 'i

deal

' fro

ntie

r in

com

pari

son

to th

e D

EA

mod

els

on

EP

M

Fir

e e

t al

. (19

96)

T+

a II

wea

k di

spos

abil

ity

for

unde

sira

ble

outp

uts

U.S

. fo

ssil

fue

l-fi

red

com

pare

d to

o

deri

ve a

n E

PI

from

dec

ompo

sing

ove

rall

eff

icie

ncy

into

an

inpu

t pro

duct

ive

effi

cien

cy

elec

tric

uti

liti

es

IF-i

ndex

an

d an

env

iron

men

tal

inde

x T

ytec

a (1

997)

T

+a

CC

R-m

odel

s 'n

et p

rodu

ctio

n',

'pol

luta

nt=

fact

or',

and

'nor

mal

isat

ion'

as

EP

is

U.S

. fo

ssil

fuel

-fir

ed

com

pare

d to

el

ectr

ic u

tili

ties

IF

-ind

ex

Tyt

eca

(199

8)

T+

a 7

sust

aina

ble

deve

lopm

ent i

ndic

ator

s st

ress

ing

econ

omic

, soc

ial,

and/

or e

nvir

onm

enta

l U

.S.

foss

il fu

el-f

ired

co

mpa

red

to

aspe

cts

elec

tric

uti

liti

es

IF-i

ndic

ator

s

Cal

lens

ffyt

eca

(199

8)

t C

CR

-mod

el f

or a

n un

ique

agg

rega

te s

usta

inab

le d

evel

opm

ent

indi

cato

r th

at s

houl

d be

de

vide

d in

to p

artia

l in

dica

tors

str

essi

ng e

cono

mic

, so

cial

, an

d/or

env

iron

men

tal a

spec

ts

Gal

lez/

Tyt

eca

(199

8)

T+a

C

CR

-mod

els

'net

pro

duct

ion'

and

'pol

luta

nt=

fact

or' a

s E

P!s

Van

den

Eec

kaut

et

a!.

(199

7)

T+

a 14

EP!

s de

rive

d by

com

bini

ng I

. E

colo

gica

l vs

. ra

dial

/non

-rad

ial

econ

omic

-eco

logi

cal

impr

ovem

ents

, 2.

DE

A v

s. FD

H,

3. s

tron

g vs

. w

eak

disp

osab

ility

of u

ndes

irab

le o

utpu

ts,

4. C

RS

vs.

VR

S

Cou

rcel

le e

t a!

. (1

998)

T

+a

• st

art

from

uni

que

aggr

egat

e su

stai

nabl

e de

velo

pmen

t in

dica

tor

• FD

H m

odel

'pol

luta

nt=

fact

or' w

ith u

ndes

irab

le o

utpu

t res

idue

rat

e is

com

pare

d to

C

CR

-mod

el w

ithou

t und

esir

able

out

put

Pio

t-L

epet

it e

ta!.

(19

97)

T+

a •

focu

s on

red

ucin

g fe

rtili

sers

/pes

ticid

es f

or c

ausi

ng e

xter

nal e

nvir

onm

enta

l eff

ects

•

use

subv

ecto

r and

non

-rad

ial t

echn

ical

eff

icie

ncy

mea

sure

s du

e to

qua

si f

ixed

fac

tors

Pio

t-L

epet

it!V

erm

ersc

h (1

998)

T

+a

• B

CC

-mod

el a

nd s

lack

adj

uste

d m

easu

re 'p

ollu

tant

=fa

ctor

' with

und

esir

able

out

put

'org

anic

nit

roge

n'

• st

rong

/wea

k di

spos

abil

ity

of u

ndes

irab

le o

utpu

t •

deri

vati

on o

f a s

hado

w p

rice

of t

he u

ndes

irab

le o

utpu

t

Ath

anas

sopo

ulos

et

al. (

1998

) T

+a

• ch

oose

'pol

luta

nt=

fact

or' m

odel

ling

for

the

und

esir

able

out

puts

con

trol

labl

e em

issi

ons

and

num

ber

of a

ccid

ents

as

a m

easu

re f

or s

afet

y •

DE

A u

sed

to g

ener

ate

targ

et s

etti

ng s

cena

rios

e.g

. fo

r po

llut

ion

emis

sion

s

All

en (

1998

b)

t •

EM

IEC

app

lica

tion

pot

enti

al o

fDE

A a

nd li

tera

ture

rev

iew

•

com

pari

son

of a

n ec

olog

ical

ly e

xten

ded

Add

VR

S m

odel

and

mod

elli

ng a

lter

nati

ves

• st

ruct

ure

for

ecol

ogic

ally

rel

evan

t ext

ensi

ons

and

axio

mat

ic m

odif

icat

ions

ofD

EA

•

a co

ncep

t for

EPM

app

licat

ions

Dyc

khof

f/ A

llen

(19

98)

t de

riva

tion

of a

gen

eral

ised

Add

VR

S m

odel

whi

ch is

eco

logi

call

y sp

ecif

ied

acco

rdin

g to

th

e 's

tand

ard

case

'

I Schee

l (19

98)

I t

I effi

cien

cy m

easu

re c

onsi

ders

tha

t any

cha

nge

of o

utpu

ts i

nvol

ves

both

des

irab

le a

nd

unde

sira

ble

outp

uts

(ass

umpt

ions

: st

rong

dis

posa

bilit

y, r

ay r

egul

arit

y)

Tab

le 3

.1

Ove

rvie

w o

fDE

A li

tera

ture

in t

he

JF: J

aggi

/Fre

edm

an (

1992

) t:

theo

ry

ecol

ogic

al c

onte

xt

a:

appl

icat

ion

U.S

. fo

ssil

fuel

-fir

ed

com

pare

d to

el

ectr

ic u

tiliti

es (

verb

al)

JF-i

ndex

U

.S.

foss

il fu

el-f

ired

el

ectr

ic u

tiliti

es

EU

mun

icip

al s

olid

w

aste

col

lect

ion

and

sort

ing

prog

ram

mes

Fren

ch c

erea

l far

ms

Fre

nch

pig

farm

s

UK

ele

ctri

city

gen

erat

ing

plan

ts

Eur

opea

n ec

onom

ies

'net

pro

du

ctio

n'

'pol

luta

nt=

fact

or'

} se

e se

ctio

n 4.

1 'n

orm

alis

atio

n'


4 Ecological extension of DEA

This section discusses selected aspects of an ecological extension of DEA. Section 4.1

shows possible extensions of basic DEA models whereas section 4.2 also considers

axiomatic modifications. Section 4.3 outlines the relevance of some well-known DEA

developments for applications in the ecological context. These reflections do not claim

completeness or profundity but represent a brainstorming to highlight the potential and

flexibility ofDEA for applications in EM/EC.

4.1 Modelling alternatives

At first sight, the integration of 'bads' in DEA does not seem to be a problem. Without

going into mathematical details, the main characteristics, limitations, and possible im

provements for the modelling alternatives are discussed in this section."

4.1.1 Data transformation

An intuitive approach to integrate 'bads' into DEA is shown for example in Lovell et

a!. (1995). Carbon and nitrogen emissions are converted by taking their reciprocal and

regarded as usual outputs afterwards. Conversely, a reduct may be treated as a normal

product after this kind of data transformation. The advantages of this approach are that

the selected DEA model can be applied without any modification and the ordinal

ranking of DMUs according to the inverted quantity stays the same. Though the scale

and intervals of the original data get lost and the reciprocal of zero values is not feasi

ble. Furthermore, it can be shown that the efficiency classifications and rankings when

choosing the reciprocal can differ from those of the alternatives 'translation' and 'pol

lutant=factor'. 25

Similar arguments appear when changing the data by translation, i.e. by subtracting

the pollutant (reduct) quantity from a positive constant and regarding it as an usual

product (factor). The constant is usually the maximal quantity of all DMUs regarding

24 For more details and an empirical illustration see Allen (1998b).

25 An example was given by Prof. Robert Dyson in the presentation on "Pitfalls and protocols in DEA'' at the European Summer Institute XVI "DEA- 20 years on", University of Warwick, England, 16--26 August 1998.


that pollutant (reduct) plus 1, in order to avoid zero values. As previously pointed out,

the DEA models are suitable without any modification. Although, on the other hand,

translation results in moving the zero to a different position and can yield results devi

ating from those of the alternatives.'•

Even if reciprocal or translation are applied, their use is dependent on the chosen effi

ciency measure. For example, if carbon emissions are modelled by taking the recipro

cal and an input-oriented model is chosen, ecological inefficiencies remain disre

garded. Such problems are important arguments in the following discussion of model

transformations.

4.1.2 Model transformation

Oriented models

Table 4.1 shows the structure of the ratio forms for the oriented models that have been

suggested for DEA applications in the environmental context." None of these models

accounts for 'reducts'. It is therefore assumed that the inputs exclusively consist of the

factors (F), the outputs comprise the products (P) and the pollutants (A). The transfor

mation of non-linear ratio forms into DEA LPs is demonstrated in Chames et a!.

(1978).

Model Input orientation output orientation

'pollutant=factor' p F+A max-- (I) min--

F+A p

'normalisation' A min A'=-p

'net production' P-F A (and 'profitability') max A minP-F

'joint output' P-A F max-F- min--

P-A

Table 4.1 Oriented models with pollutants

26 Please note the difference to the 'translation invariance' problem addressed e.g. in Pastor (1996), where negative variables are focused on without changing the minimax direction.

27 Some of these models are described in detail in the unpublished work of Prof. Mikulas Luptacik. I am grateful for his support.


As an intuitive approach to integrate pollutants into DEA, one can treat them as usual

factors. From an activity analysis perspective, the 'pollutant=factor' approach has the

advantage of a 'natural' min/max direction for each object category. Model ( 1) shows a

BCC model for input orientation according to the ratio shown in table 4.1.

(1)

min 8,A,s

s. t.

6-& L sk kEFvAvP

7t

L I.P xf + S; = ex? (i E F) p=l

7t

LI.Pyj +sj = eyJ (j E A) p=l

7t

LI.Pyj -s1 = yJ (j E P) p=l

I.P ~0 (p= 1, ... ,7t) ;s; ,s1 ~0 (i eF,j e Pu A) ;6 free

An inefficient DMU becomes efficient when it radially reduces its factors and pollut

ants according to e. Ecologically relevant impacts caused within the factors (consump

tion of non-renewable materials/'resource nature') and pollutants are accounted for,

but the interpretation of their proportional decrease may be difficult. The model is ap

plied e.g. in Piot-Lepetit/Vermersch (1998).

The corresponding output-oriented model aims at increasing the products. From an

ecological perspective it is not relevant. The model can be reduced to the minimisation

of a weighted sum of normalised pollutants as suggested by Tyteca ( 1996) and simi

larly by Haynes eta!. (1994). Factors and products are no longer explicitly considered.

The pollutant quantities are divided by a quantity measuring the DMU's activity level,

usually a production quantity. This simplified ecological efficiency measure is suitable

when technical efficiency is almost given and low ecological impacts are caused within

the factors.

DEA m the ecological context- An overview 219

In the input-oriented model 'net production' the pollutants are regarded as peculiar

outputs which should be minimised. An inefficient DMU becomes efficient when it

radially reduces its pollutants according to e. Possible improvements in the products

and factors which form a kind of 'net production' are neglected. Therefore, the model

is suitable for DMUs operating near to technical efficiency but on different pollution

levels. Though the model shows limitations, when ecologically relevant impacts are

caused within the factors. Changing to the output-oriented model results in a neglect of

inefficiencies within the pollutants. LP formulations of the 'net production' model are

presented as a CCR model e.g. in Tyteca (1996). An interpretation of 'net production'

(a weighted sum of factors subtracted from a weighted sum of products) might be con

troversial unless prices for factors and products are known. In this case, the efficiency

score can be regarded as a kind of ecological profit.

The main idea of the model 'joint output' is to consider the fact that undesirable out

puts m outputs of the production process, which is preferable from a material/energy

balance perspective. The input-oriented model considers inefficiencies of factors what

could be reasonable for production processes characterised by an intensive consume of

the 'resource nature'. The pollutants are indirectly considered by subtraction from the

product quantities. A change to the output-oriented model causes difficulty with inter

pretation. In this case, the objective function value yields the necessary radial increase

in a weighted sum of pollutants subtracted from a weighted sum of products, whereas

factor inefficiencies are neglected. The idea of a 'joint output' is underlying the models

in e.g. Fare eta!. ( 1989).

The neglect of ecological inefficiencies and the interpretation problems of the oriented

models would even deteriorate if reducts had to be integrated. For an improvement,

efficiency measures are needed, where individual, i.e. asymmetric, non-proportional

minimax directions for~ category respectively object can be defined. As a first step

target setting could be used. Thanassoulis/Dyson (1992) present models where a) one

input or output is given pre-emptive priority to improve, b) a general preference struc

ture can be specified that attaches different degrees of importance to (non

proportional) input and/or output changes, or c) a DMU can specify target levels it

would ideally wish to adopt. Inputs and outputs which are not included in the target set

are considered in the objective function only by means of a non-archimedean value.


An additive model

Being ecologically-motivated and taking a production theory, particularly an activity

analysis perspective, in Dyckhoff/ Allen ( 1998) a weighted additive model with vari

able returns to scale is derived. The integration of a multi-dimensional function allows

for the extension of this generalised AddVRS model for more complex preference

structures. Model (2) shows the resulting LP formulation, when specifying the model

according to the 'standard case'. Reducts are neglected, in order to simplify a compari

son with the oriented models below. By also neglecting the pollutants and having the

special case of all weights gk fixed on a value of one, this model (2) would then be

come an AddVRS model as presented by Charnes et a!. ( 1985).

max Po L gksk keFuAuP

7t

s. t. LIJ'xP + s I I = x? (i e F) p=l

7t

L"P p o YJ +s1 =y1 (j e A) (2) p=l

7t

Lt.Pyj -s1 = yJ (j e P) p=l

7t

LAP =1 p=l

t.P ~ o (p = 1, ... ,7t); s1 ,s1 ~ 0 (i eF,j ePu A)

The 'ECO-AddVRS' model (2) considers any kind of technical and ecological

(in)efficiencies. The underlying L1-measure aims at a maximisation of product, pollut

ant, as well as factor slacks. The ecological extension of the additive model is theoreti

cally founded and includes some modelling alternatives as special cases. However, the

main disadvantage is that the efficiency score is dependent on the input/output scales

and its interpretation is therefore not meaningful. Though the influence of scales can be


compensated by the suitable choice of weights gk, or by using efficiency measures

holding for units in variance."

The alternative approaches based on data and model transformations indicate that the

modelling of pollutants causes additional complexity and is obviously not a trivial task.

Apart from the aforementioned indications for improvements, further theoretical in

vestigation is advisable: What are the impacts of the alternative approaches? Which

relationships exist between the alternative approaches? And under which presupposi

tions should a certain approach be selected?

4.2 Axiomatic modifications

4.2.1 Relaxing the convexity assumption (FDH)

One fundamental assumption of DEA is that convex combinations of the 1t DMUs can

be technically realised." This assumption may be relaxed by using the Free Disposal

Hull (FDH) technology as described in Deprins eta!. (1984) and Tulkens (1993). In

this case, the DMUs are compared between themselves and classified 'efficient' as

soon as there do not exist dominating real DMUs. In contrast to classical DEA, ineffi

cient DMUs can not be opposed to virtual, i.e. non-existing convex combinations of

observed DMUs. In technical terms, DEA models with variable returns to scale (VRS)

like (1) and (2) become FDH models by appending constraints incorporating integer

variables as follows:

(3) f...PE{0,1} (p=l, ... ,7t)

Figure 4.1 contrasts the resulting staircase-like frontier (with more efficient DMUs) to

a DEAvRs frontier.

28 E.g. Lovell/Pastor (1995).

29 Postulate I in Banker et al. (1984).


product

.........

•

factor or •-------L...-..J pollutant

DEAvRs

FDH

'ideal' frontier

Figure 4.1 DEAvRS• FDH, and 'ideal' frontier (adapted from Tyteca 1996)

Tulkens ( 1993) describes the advantages of FD H over D EA as I) reference can not be

made to virtual DMUs, 2) FDH is less sensitive to outliers than DEA, 3) FDH is suit

able for situations in which reference has to be made to 'good practice', 4) efficiency

measures and necessary improvements of inefficient DMUs can be more explicitly

quantified, 5) the final choice of a benchmark for an inefficient DMUs stays with the

decision maker, and 6) computation is easier. Though Tyteca (1996) finds an interest

ing argument for preferring classical DEA over FDH in the ecological context "namely

that [ ... ] we do not worry about the fact that we compare existing points to an abstract,

'artificial' frontier since we would instead refer to a frontier that is eventually even

located outside the convex envelope of the existing points." He continues that the defi

nition of such an 'ideal' frontier- e.g. the dotted line in figure 4.1 - could be based on

either the best available technology ("technological definition") or a quasi zero-waste

state ("thermodynamic definition"). Neither a best practice nor a best available tech

nology concept, but just a dynamically adapted zero-waste state frontier fully corre

sponds to the idea of sustainability.30

4.2.2 Strong versus weak disposability

Table 3.1 showed several articles modifying the classical DEA assumption of strong

disposability for the pollutants, e.g. Fiire et al. (1996). Using the present paper's spe-

30 Tyteca (1996), pp. 295-298, and Callensfryteca (1998), last section.

DEA tn the ecological context- An overview 223

cific terms in their definitions, a factor is strongly disposable, if the same level of out

puts can be produced at no cost with higher quantities of factors. Products are assumed

strongly disposable, if it is possible to reduce their quantities at no cost using the same

inputs. For pollutants, the assumptions of weak disposability is introduced. Pollutants

hold for weak disposability, if their production can be reduced only at the expense of a

joint reduction in some other products, or a joint increase in the use of some factors,

where the joint movements are considered as proportional. Figure 4.2 opposes the dif

ferent technologies resulting from 1) regarding the pollutant as a strongly disposable

factor for the DEAvRS case or 2) assuming weak disposability for the pollutant."

product product (strong disp.) (strong disp.)

•

pollutant •-------'-----' (strong disp.)

1)

Figure 4.2 Strong versus weak disposability

• "--------'----1~ pollutant

(weak disp.)

2)

Vanden Eeckaut et a!. ( 1997) give the following estimation which approach should be

chosen: "The answer to this question relies on the nature of the problem analyzed.

When there is a strict relation between pollution and the inputs (due to regulation or

technological characteristics) then the first model has to be preferred [weak dispos

ability] .... However if it is possible to substitute pollution with inputs then pollution

has to be considered as an input [strong disposability]. An example of this substitution

31 Regarding the disposability assumptions and for further illustrations see e.g. Fare et al. (1996), Fare et al. (1989), Vanden Eeckaut et al. (1997), and Scheel (1998).


is the increase of labor which improves the quality of the control inside the plant and

by consequence the emission of pollution."

4.2.3 Further axiomatic modifications

The most important characteristic of production processes including ecologically rele

vant objects are strong relationships between factors and pollutants as well as between

products and pollutants. The influence of such relationships on the DEA results could

possibly be detected by correlation analyses before selecting inputs and outputs for the

DEA model. Also, axiomatic modifications could cope with such relationships.

Fiire/Grosskopf (1995) implement the notion of nulljointness- i.e. to produce 'good'

outputs one will ~ to produce some 'bad' outputs - into models with weakly dis

posable pollutants. Further interesting results can be expected when relaxing the DEA

assumption of fixed input/output relationships. In this case, a DMU can achieve effi

ciency not just by reducing undesirable and increasing desirable objects, but also by

'changing the mix' of its inputs and outputs.

4.3 Application driven extensions

In this section some well-known DEA developments are presented, which show a high

potential for DEA applications in EM/EC."

4.3.1 Categorical data

The convexity assumption of DEA causes problems in cases where input or output

variables appear as categorical variables. In such cases the reference units should be

constructed from DMUs which are in the same category or possibly from those in an

even more unfavourable category. Models for an incorporation of categorical variables

were presented by Banker/Morey (1986b) and Kamakura (1988).

In the ecological context, categorical variables could be useful e.g. to compare compa

nies on different levels of environmental certification, to describe pressure levels of

stakeholders or the level of competition surrounding a company, or to consider differ

ent technologies of e.g. waste burning plants. Golany et a!. (1994) treat S02-emissions

32 For the LP formulations, an empirical illustration, and further ideas see Allen (1998b).


as a categorical output. It should be noted that some of the variables mentioned here

may be exogenously fixed (see below).

4.3.2 Ordinal data

DEA assumes that all inputs and outputs can be measured on a cardinal scale. Though

it could happen that one or several factors may only be measurable on an ordinal scale

or that the DMUs can only be ranked relatively to such ordinal data. The incorporation

of ordinal data in DEA is addressed in Cook et al. (1993) and (1996).

Ordinal data may appear in EM/EC for example when waste coming out of a process

can be used in a downstream recycling process only if it reaches a pre-defined quality

level. For a pollutant coming out of a production process it may be useful to model its

toxicity class. This ordinal information can also express a rough estimation of the cor

responding disposal costs. Regarding data availability and standardisation problems in

the LCA context, experts could suitably substitute imprecise or non-existing quantita

tive or qualitative information on emissions by ordinal approximations.

4.3.3 Exogenously fixed inputs or outputs

In practical applications managers must often deal with situations where some inputs

or outputs are beyond their discretionary control. Banker/Morey (1986a) and Go

lany/Roll (1993) developed DEA models in order to estimate the extent to which the

controllable inputs/outputs can be improved while keeping the exogenously fixed in

puts/outputs constant.

In ecologically-oriented applications of DEA exogenously fixed inputs or outputs may

appear when describing the surroundings of a company, for example regarding the

strictness and costs of local environmental regulations, the availability of alternative

energy sources, the existence of purification plants or facilities for disposal of toxic

wastes, the population number in the neighbourhood, their and other stakeholders' at

titudes, as well as characteristics of soil, topology and weather.

4.3.4 Incorporation of weights restrictions and value judgements

The incorporation of weight restrictions and value judgements covers a considerable

part of the DEA literature, an overview is given in R. Allen et al. (1997). The flexibili

ty of the weights in DEA sometimes causes problems, for example when the DEA re-


suits crucially differ from prior views of the decision maker, when the discrimination

power of DEA is weak, or when the efficiency classification of a DMU results from

just one extreme input or output. To minimise these problems, even partial information

about the preferences of each kind should be incorporated into the analysis. In section

2, it was pointed out that the assessment of weights on ecologically relevant objects is

complex and difficult. Therefore, it is likely that in the ecological context 'simple'

more than 'sophisticated' approaches are used for an incorporation of weights restric

tions and value judgements. For example, decision makers might be able to express at

least ordinal preferences, assurance regions (i.e. relative relationships between

weights), or some absolute weight restrictions on pollutants, when information or an

agreement on their toxicity is not available.

4.3.5 Incorporating standards

Apart from the empirically observed DMUs one may sometimes want to integrate ac

cepted standards or benchmarks of performance into DEA because"[ ... ] it may happen

that DMUs forming the 'efficient frontier' are themselves lagging in efficiency behind

some known 'standard'." The integration of standard DMUs results in a 'standard en

velope' that can be composed of observed as well as standard DMUs and is usually

located outside the former 'self envelope'. The scores of inefficient DMUs will be

lower and previously efficient DMUs might now be evaluated as inefficient. It is rec

ommended to set several multi-dimensional standards instead of one single standard

for each input and output dimension."

From an ecological perspective, standards can be used for example to model environ

mental regulations. In this case, the standard DMUs consist of the input/output vector

of the observed DMUs, where the pollutant under regulation is replaced by the regula

tion's limiting value. Furthermore, in analogy to the discussion of an 'ideal' frontier in

section 4.2, one can define standards as best available or zero-waste technologies. In

both cases the incorporation of standards changes the DEA perspective from past ob

servations (static tool) towards projected values (prospective tool). A distance measure

representing the relative position of such standard envelopes and the best practice

frontier can provide useful information on the ecological position of the observed

DMUs. In their application on power plants in Israel Golany eta!. (1994) incorporated

33 For details see Golany/Roll ( 1994).

DEA m the ecological context- An overview 227

one standard DMU for each observed site. Management knowledge, government

regulations and technical data were taken as sources for the definition of the standard

DMUs.

4.3.6 Ecological efficiency over time

Information on developments over time are important for the EM/EC and could be

provided in the DEA framework for example by using window analysis or Malmquist

indices. Though it should be noted that regarding the impacts of pollutants on nature

absolute quantitative reductions are preferable compared to relative improvements,

where the production level and therefore the level of negative impacts of the pollutants

may be increased. This fact could be considered by selecting a model just focused on

pollutant minimisation or by including an ordinal variable which is high (low) for

DMUs that reduced (increased) the absolute pollutant quantities over time.

Although it is sometimes recommended to use window analysis to overcome the dis

advantage of a low number of DMUs due to data availability problems, this argument

should be regarded critically from an ecological perspective. A DMU could estimate

its ecological position as positive although being compared to old-fashioned technolo

gies or despite not accounting actively for environmental activities in the past.

5 A concept for applications of DEA in EPM

In section 2, environmental performance measurement (EPM), especially at the opera

tional level and by means of indicators and benchmarking, was described as an impor

tant task of EM/EC. The assessment of weights in this context is an extremely difficult

task. DEA does not need such a priori weights, but the state-of-the-art approaches de

scribed in section 4.1 still contain some technical or interpretation problems. Unless

more theoretical insight, improved models, and axiomatic modifications are available,

the potential of DEA for applications in the ecological context can be exploited by re

ducing the complexity caused by the integration of ecologically relevant objects. Fig

ure 5.1 illustrates a concept for applications of DEA in EPM.34

34 See Allen (1998a).


In step 1 the DMUs' inventory tables are collected to serve as the database. In the case

of numerous inputs and outputs, a selection of the most relevant inputs/outputs as well

as an aggregation of inputs/outputs should be undertaken as far as necessary and possi

ble. In step 2 the inventory tables are transformed into a categorised inventory table

according to the 'standard case'. The main idea of the concept (steps 3 to 5) is to de

velop a system of indicators, and then to apply DEA to each indicator. In terms of the

underlying ratio form, each indicator is constructed as a weighted sum of objects from

one desirable category and a weighted sum of objects from one undesirable category.

This structure is typical for EPis in practice. Either the indicators just consist of for

example one pollutant over one product quantity, or weights have to be assessed to

aggregate for example several factor over several product quantities. As an advantage

over common EPis, in this concept no weights are required to achieve an aggregated

efficiency score for multiple objects. Though if value judgement on inputs/outputs ex

ist they can and should be integrated in step 4. As an advantage over the unique indi

cators coming out of the models in section 4.1, these indicators allow for concrete in

terpretations. Especially, the choice of a core indicator set allows for consideration of

ecological and technical efficiency measures.

The concept can not solve general problems of EPM (e.g. data availability and stan

dardisation, see section 2) and is not unproblematic within itself. Neither the selection

and categorisation of inputs/outputs nor the development of the indicator set are trivial

tasks. Though experts are more likely to agree on these aspects than on weights for

inputs/outputs. The concept represents a simplified possibility to apply DEA for EPM

if one wants to avoid intensive theoretical investigations or if first insights are quickly

required to find out where detailed analyses should be focused. For applications of

DEA in EPM, for example for an ecological product benchmarking, a certain degree of

simplification and aggregation will always be necessary.

The line of thought underlying this concept is confirmed by proceedings and argu

ments in some of the first DEA applications in the EM/EC context, e.g. Callens/Tyteca

(1998): "[ ... ] it is important not to base decisions on one unique, aggregate

sustainability indicator; instead, it is suggested to develop two or three partial indica

tors that stress different aspects of the problem.""

" Theoretical arguments are given in Dyckhoff/Rildiger ( 1997).

DEA m the ecological context- An overview

3 Inventorv table OMU1 2

Inputs Outputs t!!!'l!.•~l ___ _J~rj!liw;ti_ __

lJ!I~t---- Jio.li.d.l!'UJL _ ~~r ____ wut;_w_ile_L-

r.~e..-v --- ~~:~~.--

'core' env. lndicaiOr system material: raw m:ue:nal~ r _____ ~~CIQ\!!Oiill~ __

solid \Y3Ste: wastes tor rccjfhmz ______ ~~e_9.u~tt.!!_~ __

waste water: water consumgqon _ _ _ _ _ l!!_odUC:t_gl!!!l.!!!!,C!.. - -

waste air: <» f"SP\qm!?Jons ______ e!.odJ!C!,9'!!,0~1~ __

energy: 'Pi~it0:~~m:n

r~ft;re!!.C'< . ..U!!!'L _____ _ '!_l~a_!!c~ o.f:. iJ2p'!,l,si~u!P~ __

~r~a,!!o.!! '![ ~j.:_c~~~~~i!!_e~ funher non·DEA analyses

Step l Inventory analysis

tep 2

categorisation of inputs/outputs

('standard case')

tep 3 choice of 'core' EPis (desirable over undesirable object)

tep 4 DEA on each 'core' indicator

Step 5

interpretation/conclusions

Figure 5.1 A concept for applications ofDEA in EPM

6 Summary and outlook

229

Starting from an overview on environmental management/environmental controlling

and their main problems, the potential of DEA applications in this scope was pointed


out. A literature review underlined this potential, but also revealed different and some

times arbitrary approaches regarding the integration of ecologically relevant objects in

DEA. Each of the 'state-of-the-art' approaches shows conceptual limitations or inter

pretation problems. It is therefore vital to go back into DEA's axioms and foundations

in the theory of production and particularly activity analysis background for a system

atic and consistent derivation of ecologically extended models. For example, in

Dyckhoff/ Allen ( 1998) a generalised AddVRS model is derived and ecologically

specified according to the 'standard case'. As several examples proved in section 4, it

is unproblematic to extend such theoretically-founded models according to the real

EM/EC application context.

To my mind, new flexible efficiency measures, axiomatic modifications as well as the

exploitation of synergies with multi-criteria and distance function approaches show an

extraordinary potential for DEA applications in the ecological context. Figure 6.1 in

cludes these keywords in a framework on three levels of different theoretical and prac

tical contents.

DEA for EM/EC

ecological extension of basic DEA models integration of e.g. weights, standards, targets, specific data types handle incomplete and unstandardised data, low number of OM Us specific pitfalls, aggregation levels, and needs ofEM/ EC

Puu·lt .llwn in tht.nr~

analysis of alternative approaches flexible efficiency measures axiomatic modifications, e.g. disposability, changing the mix absolute/relative development over time

parametric approaches stochastic approaches

,.......----~ --MCDM J ratio analysis

distance function approaches

applicatioo

Figure 6.1 Framework for future research


References

Allen, K. (1998a): Moglichkeiten und Grenzen der Data Envelopment Analysis im Rahmen des Oko

Controlling; in: Dyckhoff, H./Ahn, H. (Hrsg.): Produktentstehung, Controlling und Um

weltschutz, Heidelberg, 327-352.

Allen, K. ( 1998b ): Basics of DEA for applications in environmental management/ecological control

ling: Framework, foundations, ecological extensions, and prospects; lnstitut flir

Wirtschaftswissenschaften, RWTH Aachen, working paper 98105.

Allen, R. I Athanassopoulos, A. D. I Dyson, R. G. I Thanassoulis, E. (1997): Weights restrictions and

value judgements in data envelopment analysis - Evolution, development and future directions;

in: Annals of Operational Research 73, 13-34.

Athanassopoulos, A. D. I Thanassoulis, E. ( 1995): Separating market efficiency from profitability and

its implications for planning; in: Journal of the Operational Research Society 46, 20-34.

Athanassopoulos, A. D. I Lambroukos, N. I Seiford, L. (1998): Data envelopment scenario analysis

for setting targets to electricity generating plants, submitted working paper.

Ball, V. E. I Lovell, C. A. K. I Nehring, R. F. I Somwaru, A. (1994): Incorporating undesirable out

puts into models of production: An application to US agriculture; in: Cahiers d'economie et so

ciologie rurales 31, 60-74.

Banker, R. D. I Charnes, A. I Cooper, W. W. (1984): Some models for estimating technical and scale

inefficiencies in data envelopment analysis; in: Management Science 30, I 078-1092.

Banker, R. D. I Morey, R. C. (1986a): Efficiency analysis for exogenously fixed inputs and outputs;

in: Operations Research 34, 513-521.

Banker, R. D. I Morey, R. C. (1986b): The use of categorical variables in data envelopment analysis;

in: Management Science 32, 1613-1627.

Callens, I. I Tyteca, D. (1998): Towards indicators of sustainable development for firms - a produc

tive efficiency perspective; in: Ecological Economics.

Charnes, A. /Cooper, W. W. I Golany, B. I Seiford, L.M. I Stutz, J. (1985): Foundations of Data En

velopment Analysis for Pareto-Koopmans efficient empirical production functions; in: Journal

of Econometrics 30, 91-107.

Charnes, A. I Cooper, W. W. I Lewin, A. Y. I Seiford, L. M. (1994): DEA- Theory, methodology and

applications, Boston/Dordrecht/London.


Charnes, A. I Cooper, W. W. I Rhodes, E. (1978): Measuring the efficiency of decision making

units; in: European Journal of Operational Research 2, 429-444.

Cook, W. D. I Johnston, D. A. I McCutcheon, D. (1992): Implementation of robotics: Identifying

efficient implementors; in: Omega 20, 227-239.

Cook, W. D. I Kress, M. I Seiford, L. M. (1993): On the use of ordinal data in data envelopment

analysis; in: Journal of the Operational Research Society 44, 133-140.

Cook, W. D. I Kress, M. I Seiford, L. M. (1996): Data envelopment analysis in the presence of both

quantitative and qualitative factors; in: Journal of the Operational Research Society 47,

945-953.

Cooper, W. W. I Hemphill, H. I Huang, Z. I Li, S. I Lelas, V. I Sullivan, D. W. (1996): Survey of

mathematical programming models in air pollution management; in: European Journal of Op

erational Research 96, 1-35.

Courcelle, C. I Kestemont, M.P. I Tyteca, D. I Installe, M. (1998): Assessing the economic and envi

ronmental performance of municipal solid waste collection and sorting programmes; in: Waste

Management & Research 16(3), 253-263.

Day, D. L. I Lewin, A. Y. I Li, H. (1995): Strategic leaders of strategic groups: A longitudinal Data

Envelopment Analysis of the U.S. brewing industry; in: European Journal of Operational Re

search 80, 619-638.

Deprins, D. I Simar, L. I Tulkens, H. (1984): Measuring labor-efficiency in post offices; in: Mar

chand, M. I Pestiau, P. I Tulkens, H. (Eds.): The performance of public enterprises: Concepts

and measurement, North Holland, 243-267.

Dyckhoff, H. (1994): Betriebliche Produktion: Theoretische Grundlagen einer umweltorientierten

Produktionswirtschaft, 2"• ed., Berlin.

Dyckhoff, H. (1998a): GrundzUge der Produktionswirtschaft, 2"• ed., Berlin.

Dyckhoff, H. (1998b): Umweltrnanagement; in: Berndt, R. I Altobelli, C. F. I Schuster, P. (Eds.):

Springers Handbuch der Betriebswirtschaftslehre 2, Berlin, 389-431.

Dyckhoff, H. I Allen, K. (1998): An ecologically motivated generalisation of data envelopment analy

sis (DEA); submitted working paper.

Dyckhoff, H. I RUdiger, C. (1997): Ein M6glichkeitstheorem fur Oko-Indizes; in: Zeitschrift fur

Angewandte Umweltforschung (ZAU) 10, 58-65.

Epstein, M. K. I Henderson, J. C. (1989): Data Envelopment Analysis for managerial control and

diagnosis; in: Decision Sciences 20,90-119.


Flire, R. I Grosskopf, S. (1995): Environmental decision models with joint outputs; submitted working

paper.

Fare, R. I Grosskopf, S. I Lovell, C. A. K I Pasurka, C. (1989): Multilateral productivity comparisons

when some outputs are undesirable: A non-parametric approach; in: The Review of Economics

and Statistics, 90-98.

Fare, R. I Grosskopf, S. I Lovell, C. A. K. I Yaisawamg, S. (1993): Derivation of shadow prices for

undesirable outputs: A distance function approach; in: The Review of Economics and Statistics

75(2), 374-380.

Fare, R. I Grosskopf, S. I Tyteca, D. (1996): An activity analysis model of the environmental per

formance of firms - application to fossil-fuel-fired electric utilities; in: Ecological Economics

18,161-175.

Gallez, C. I Tyteca, D. (1998): Explaining the environmental performance of firms with indicators; in:

Us6, J. L. I Brebbia, C. A. I Power, H. (Eds.): Ecosystems and sustainable development- Ad

vances in ecological sciences, Vol. I, 317-330.

Golany, B. /Roll, Y. (1993): Some extensions of techniques to handle non-discretionary factors in

data envelopment analysis; in: The Journal of Productivity Analysis 4 ( 4), 419-432.

Golany, B. I Roll, Y. (1994): Incorporating standards via DEA; in: Chames, A. I Cooper, W. W.

Lewin, A. Y. I Seiford, L. M. (Eds.): DEA - Theory, methodology and applications, Bos

ton/Dordrecht/London, 313-328.

Golany, B. I Roll, Y. I Rybak, D. (1994): Measuring efficiency of power plants in Israel by data en

velopment analysis; in: IEEE Transactions on Engineering Management 41(3), 291-301.

Goldmann, B. I Schellens, J. (1995): Betriebliche Umweltkennzahlen und llkologisches Benchmark

ing; Schriftenreihe Wirtschaft und Umwelt, Bd. 6, Kllln.

Giinther, E. (1994): 6kologieorientiertes Controlling- Konzeption eines Systems zur l:lkologieorien

tierten Steuerung und empirische Validierung, Miinchen.

Haynes, K. E. I Ratick, S. I Bowen, W. M. I Cummings-Saxton, J. (1993): Environmental decision

models: U.S. experience and a new approach to pollution management; in: Environmental In

temationa119, 261-275.

Haynes, K. E. I Ratick, S. I Cummings-Saxton, J. (1994): Toward a pollution abatement monitoring

policy: Measurements, model mechanics, and data requirements; in: The Environmental Pro

fessional 16, 292-303.

Horvath, P. (1994): Controlling, 5" ed., Miinchen.


Jaggi, B. I Freedman, M. (1992): An examination of the impact of pollution performance on eco

nomic and market performance: Pulp and paper firms; in: Journal of Business Finance and Ac

counting 19,697-713.

Kamakura, W. A. (1988): A note on "the use of categorical variables in data envelopment analysis";

in: Management Science 34, 1273-1276.

Kao, C. I Yang, Y. C. (1991): Measuring the efficiency of forest management; in: Forest Science

37(5), 1239-1252.

Loew, T. I Kottmann, H. (1996): Kennzahlen im Umweltmanagement; in: Okologisches Wirtschaften

2, 10-12.

Lovell, C. A. K. I Pastor, J. T. (1995): Units invariant and translation invariant DEA models; in: Op

erations Research Letters 18, 14 7-151.

Lovell, C. A. K. I Pastor, J. T. I Turner, J. A. (1995): Measuring macroeconomic performance in the

OECD: A comparison of European and non-European countries; in: European Journal of Op

erational Research 87, 507-518.

Miettinen, P. I Hlimalainen, R. P. (1997): How to benefit from decision analysis in environmental life

cycle assessment (LCA); in: European Journal of Operational Research I 02, 279-294.

Pastor, J. T. (1996): Translation invariance in data envelopment analysis: A generalization; in: Annals

of Operations Research 66, 93-102.

Piot-Lepetit, I. I Vermersch, D. I Weaver, R. D. (1997): Agriculture's environmental externalities:

DEA evidence for French agriculture; in: Applied Economics 29, 331-338.

Piot-Lepetit, I. I Vermersch, D. (1998): Pricing organic nitrogen under weak disposability assump

tion: An application to the French pig sector; in: Journal of Agricultural Economics 49(1 ),

85-99.

Ray, S. C. I Kim, H. J. (1995): Cost efficiency in the US steel industry: A nonparametric analysis

using Data Envelopment Analysis; in: European Journal of Operational Research 80, 654-671.

Rudiger, C. (1998): Controlling und Umweltschutz: Grundziige eines koordinationsorientierten Oko

Controlling; in: Dyckhoff, H. I Ahn, H. (Hrsg.): Produktentstehung, Controlling und Um

weltschutz, Heidelberg, 271-298.

Scheel, H. (1998): Undesirable outputs in efficiency valuations; submitted working paper.

Schefczyk, M. (1993): Operational performance of airlines: An extension of traditional measurement

paradigms; in: Strategic Management Journal 14, 301-307.

Seidel, E. I Clausen, J. I Seifert, E. K. (1998): Umweltkennzahlen, Miinchen.


Sci ford, L. M. (1996): Data Envelopment Analysis: the evolution of the state-of-the-art (1978-1995);

in: The Journal of Productivity Analysis 7, 99-137.

Souren, R. I Rudiger, C. (1998): Produktionstheoretische Grundlagen der Stoff- und Energiebilan

zierung: Eine Analyse aus Sicht des 6ko-Controlling; in: Dyckhoff, H. I Ahn, H. (Hrsg.): Pro

duktentstehung, Controlling und Umweltschutz, Heidelberg, 299-326.

Stolp, C. (1990): Strengths and weaknesses of data envelopment analysis. An urban and regional per

spective; in: Computers, Environment and Urban Systems 14(2), 103-1 f6.

Thanassoulis, E. I Dyson, R. G. (1992): Estimating preferred target input-output levels using data

envelopment analysis; in: European Journal of Operational Research 56, 80-97.

Tulkens, H. (1993): On FDH efficiency analysis: Some methodological issues and applications to

retail banking, courts, and urban transit; in: The Journal of Productivity Analysis 4, 183-210.

Tyteca, D. ( 1996): On the measurement of the environmental performance of firms - A literature

review and a productive efficiency perspective; in: Journal of Environmental Management 46,

281-308.

Tyteca, D. (1997): Linear programming models for the measurement of environmental performance of

firms- Concepts and empirical results; in: Journal of Productivity Analysis 8, 183-197.

Tyteca, D. (1998): On sustainability indicators at the firm level: Pollution and resource efficiency as a

necessary condition towards sustainability; in: Journal oflndustrial Ecology 2(4).

Vanden Eeckaut, P. I Tyteca, D. I Courcelle, C. (1997): A simple business environmental perform

ance indicator based on productive efficiency, submitted working paper.

Werner, T. I Brokemper, A. (1996): Leistungsmessung mit System- Data Envelopment Analysis als

Instrument des Controlling; in: Controlling, Heft 3, 164-170.

World Commission on Environment and Development (1987): Our common future, Oxford.

Measuring Public Spending Efficiency in Brazilian

Municipalities: A Nonparametric Approach

Maria da Conceiyao Sampaio de Sousa and Francisco S. Ramos 1

Abstract

The importance of public expenditure in such social services as education and public health

represent a decisive contribution to a nation 's progress. A major problem is how to allocate

government spending in such a way as to provide public services efficiently. In a political

federation as the Brazil, the critical issue becomes the choice of the degree of decentralization

in the provision of public services. Decentralization is advantageous in the production and

distribution of public services due to: i) the proximity of users facilitates the ranking of the

priorities; ii) it is easier to control the use of resources; iii) the requirements of managerial

capacity are lower.

In this paper we assess the performance of Brazilian municipalities regarding the utilization

of public revenue. The paper inquires whether, for a given availability of services, local

governments minimize the expenditure needed to finance those services. To answer these

questions, a cost-efficiency frontier will be determined by using various techniques of

efficiency analysis: two DEA variants- DEA-F and DEA-V- and the FDH approach.

Our results suggest that the Brazilian recent municipal decentralization policy does not lead

to an efficient use of public resources. The outcome of this policy was a proliferation of small

municipalities that, due to their size, do not benefit from the "economies of scale inherent to

the production of certain public services. They tend to operate with higher average costs thus

bringing about a considerable waste of resources, which can be inferred by estimating the

excessive public spending that characterizes those cities.

1 Universidade de Brasilia and Universidade Federal de Pernambuco. We are indebted to CNPq (Conselho Nacional de Desenvolvimento Cientifico e Tecnol6gico) and IPEA (Instituto de Pesquisas Econ6micas e Sociais) for support during the elaboration of this paper. We would also like to thank Maria da Conceiyao Silva for providing the data set

238 Measuring Public Spending Efficiency in Brazilian Municipalities ...

Structure

Introduction

2 Methodology

3 Data

4 Efficiency Results

5 Efficiency and Returns to Scale: Does Decentralization Benefits Brazil?

6 Concluding Remarks

References

Measuring Public Spending Efficiency in Brazilian Municipalities ... 239

1 Introduction

The importance of public expenditure policies for economic development is by now

widely recognized.' In particular, investments in such social services as education and

public health represent a decisive contribution to a nation's progress. Regarding

income distribution, recent research shows that policies of public spending have played

an important role in creating and preserving a more equitable form of economic

growth. 3 Indeed, when the objective is reducing poverty and income disparities, those

policies seem to be more efficient than redistributive tax policies.' The latter point is

particularly relevant for Brazil, a country characterized by such extreme inequalities,

and correspondingly enormous social debt, that seriously jeopardize the long run

growth prospects of the economy. An adequate public provision of certain key goods

and services may contribute to reduce significantly those disparities.

In those circumstances, a major problem is how to allocate government spending in

such a way as to provide public services efficiently and attenuate income disparities. In

a political federation as is the case of Brazil, the critical issue becomes the choice of

the degree of decentralization in the provision of public services. There is now a broad

consensus that such decentralization enjoys a presumption of being substantially

advantageous in the production and distribution of public services. Various reasons are

given for local governments having a superior advantage in providing such services.

First, the proximity of users facilitates the identification of the priorities and thus

allows an easily ordering of the services to be supplied. Second, it is easier to control

the use of resources if they do not need to run through long bureaucratic channels

before reaching their destination. Therefore, decentralizing resources reduces the

opportunities for corruption and inefficiencies so commonly associated with the public

sector. Third, the requirements of managerial capacity are lower, which is essential for

the good performance of projects in small and medium sized municipalities.

2 See, for instance, Van de Walle and Nead (1995), Harberger (1977), Goode (1984).

3 See Selowsky (1979), Meerrnan (1979), Anand and Ravaillon (1993), Lipton and Ravaillon (1995).

4 Several studies have shown that the tax system is inefficient to correct income disparities generated by successive interventionist policies. See Andie (1977), Bird and de Wullf(1978).


In short, the need to make local governments responsible for the provision of a

significant share of public goods is, by now, largely recognized. However, the mere

decentralization of public spending does not guarantee, per se, a satisfactory provision

of the corresponding services. It is also necessary to ensure that public funds are used

in the best possible way by the municipalities. Yet, this condition is not easy to fulfill.

For instance, in Brazil, it is by no means clear that the additional resources granted by

the 1988 Constitution to the municipalities are being used to maximize the

community's welfare as measured by the array of public services made available to the

population5. So, it is necessary to establish efficiency criteria that could be used to

evaluate how well or poorly are local governments spending public resources.

The objective of this paper is to assess the performance of Brazilian municipalities

regarding the utilization of public revenue. The paper inquires whether local

governments allocate resources so as to maximize the supply of public goods;

alternatively, whether, for a given availability of services, they minimize the

expenditure needed to finance those services. To answer these and other related

questions, this paper analyzes the relationship between aggregate municipal spending

and the corresponding provision of public services. In particular, a cost-efficiency

frontier will be determined by using various techniques of efficiency analysis.

This paper is organized as follows. Section 2 presents and discusses the methodology

used to compute the efficiency levels of Brazilian municipalities. Section 3 presents

the database and discusses the physical indicators chosen as proxies for the supply of

public services. Section 4 provides nonparametric efficiency measures consisting of

FDH and DEA calculations. Section 5 covers the relationship between efficiency and

returns to scale. Finally, Section 6 draws some lessons and conclusions from the

experience of the Brazilian municipalities.

2 Methodology

Public expenditure patterns may be analyzed from a number of perspectives. Among

them we find studies which emphasize the role played by demand on the provision of

5In 1991, municipal revenues represented 5,8% of GDP compared with only 2,9% for the period 1980-87, before the implementation of the 1988 Constitution (Afonso (1994)).


public goods and those who dwell in aspects of the technical efficiency associated with

the production of those goods. The first approach is based upon the pioneer work of

Bergstrom and Goodman (1973), who see the effective supply ofloca1 public services

as a response of local governments to the demand for such services. Therefore, they

associate municipal expenditures with typical demand variables such as income,

relative prices, and other socioeconomic variables. This type of analysis is particularly

appropriate to explain the levels and distribution of the various local services. An

alternative approach highlights the efficiency aspects of local public spending as stated

in the work of Vanden Eeckaut, Jamar and Tulkens (1991). Here, the main concern is

to identify the minimum cost at which a given quantity of goods and services could be

provided. Therefore, it becomes a classical problem of cost minimization. Under this

approach, the emphasis is placed on the determination of a cost-efficiency frontier

associated with the production of public goods that would permit to estimate the

efficiency levels of the various municipalities. That approach was used in this study.

2.1 Measuring efficiency levels: parametric and nonparametric approaches

Efficiency levels may be measured by using different methodologies. The distinction

among those methods lies on the type of technique applied to depict the frontier. For

instance, parametric and nonparametric approaches could be used to determine the

boundaries of the production set. In the first case - the parametric approach - the

frontier is supposed to be described as a function characterized by constant parameters.

Firstly used by Aigner and Chu (1968), this method specifies a production function and

then determines its parameters so that the estimated function envelops the data from

above and minimizes the distance between the observations and the graph of the

function.' The obvious shortcoming of this method comes from the fact that the

computed efficiency levels depend, crucially, on the particular specification attached a

priori to the production function. The very definition of efficiency is intrinsically

linked to the functional form chosen.

' For a detailed discussion concerning the estimation of parametric production frontiers see Forsund, Lovell and Schmidt (1980) and Bauer (1990).


Nonparametric approaches have a more flexible character as they impose no functional

forms a priori on the underlying technology. They require only that the production set

fulfill such properties as free disposal, convexity, and piece-wise linearity of the

technology. Under this approach, the central problem is to determine, according to pre

established hypotheses, which observations could be considered elements of the

frontier. Those elements are found by solving, for each observation, a system of linear

equations. In the vast literature relative to this topic, the method of determining the

efficient points is known as Data Envelopment Analysis (DEA).7

Farrell (1957) was the precursor in the use of this type of methodology. In his seminal

paper, this author assumed proportionality between inputs and outputs (constant

returns to scale). Along the same lines, Charnes, Cooper and Rhodes (1978)

established a DEA method to evaluate cost-efficiency frontiers. The production set

generated by this approach - known as the CCR model, after their authors - is a convex

hull of the data set. For each observation, the input-efficiency is measured as the

horizontal distance between the observed point and the appropriate facet of the

polyhedron. In this procedure, the efficiency of an observed decision making unit with

respect to best observed practice in the sample of n observations is computed by

solving a nonlinear problem'.

This first DEA version - referred to as DEA-C from now on - implies strong

restrictions concerning the form of the productive set. In particular, it imposes constant

returns to scale and convexity. To tackle such problems, this approach was

subsequently extended to incorporate non-constant returns of scale. Two variants of

this method were proposed: one characterized by non-increasing returns of scale

(DEA-CD) and another allowing for the existence of variable returns (DEA-V).

Non-increasing returns to scale, as found in Fare, Grosskopf and Lovell (1985, 1994),

is modeled by adding an additional constraint implying that the production set is

convex, includes the origin and satisfies the condition of free disposal: it allows for

constant and decreasing returns to scale. If the restriction is modified to exclude the

origin, we have the DEA-V measurement of efficiency: it allows for variable returns to

7 See Chames, Cooper and Rhodes (1978, 1981 ), Fare, Grosskpof and Lovell (1990), Seiford and Thrall (1990).

'See Chames, Cooper and Rhodes (1978).


scale, with increasing returns for lower levels of output and decreasing returns for

higher levels of output. This model is known in the literature as the BCC model, after

Banker, Charnes and Cooper (1984)9 •

Although representing a clear advance relative to parametric techniques, these

approaches still rely on hypotheses very restrictive on the structure of the production

set. Deprins, Simar and Tulkens (1984) have proposed weaker assumptions. They

postulate that the frontier of the production set is simply the boundary of the free

disposal hull (FDH) of the data set. Strong disposability of inputs and outputs is

maintained as well as variable returns to scale but no convexity hypothesis is required.

In this method - henceforth referred to as FDH - the frontier is obtained by comparing

inputs and outputs so as to establish the dominant points. An observation is declared

inefficient if it is dominated by at least another observation, domination, here,

meaning the ability to produce more output with less input. Consequently, if an

observation is not dominated by any other it is declared FDH efficient. In terms of the

DEA linear programming approach, this implies adding to problem the n+ 1

restrictions L~., A.j ~ 1 and~ e {0,1 }, j =1, ... , n. 10

Figure 2.1 illustrate well this methodology for the one-input one-output case and allow

its comparison with the DEA techniques (DEA-C, DEA-CD and DEA-V). It shows the

FDH frontier as well as the DEA-C, DEA-CD, and DEA-V frontiers. The implications

of the convexity assumption are clearly perceived. The proportionality hypothesis in

Farrell's frontier presupposes constant returns to scale. The segment OF gives the

frontier of the polyhedral production set. For the DEA-C method, characterized by

constant and decreasing returns of scale, the frontier takes the piecewise linear form

represented by line OABC. Finally, excluding the origin, the segment VABC

corresponding to the variant DEA-V, which presents variable returns to scale, gives the

frontier of the production set.

We can evaluate quantitatively the efficiency level associated with the different

productive activities by using the concept of distance. Considering observation M on

figure 2.1, it is easy to compute the efficiency measures implied by the different

9 See Fare, Grosskopf and Lovell (1985, 1994)

10 For more technical details see Deprins, Simar and Tulkens (1984) and Tulkens (1990).


methodologies as the distance between this point and the various calculated frontiers.

Hence, MF, MY and MH correspond, respectively to the efficiency measures

computed by using Farrell (DEA-C), DEA-Y and FDH methods.

Figure 2.1: Alternative forms oftbe production frontier

output

_...er------e

MF->DEA-C

MY-> DEA-CD and DEA-Y

MH->FDH

Fonte: Tulkens (1990)

2.2 Computational Aspects

To assess the efficiency levels of the Brazilian municipalities we consider one input,

the aggregate cost of the various public services supplied by the community, and

several outputs here represented by the indicators of the different public services.

Those variables are described in detail in Section 3.


(2.1)

subject to

j = l, ... n

where ck and cj represent aggregated total expenditures of municipality k and j,

respectively; Ykr and y1, are the quantities of the rth output indicator of these

municipalities. The variables 'YJ represent the weights attached to the municipalities

against which the municipality k is being evaluated. The value of the objective function

gives the efficiency degree in terms of inputs. DEA variants - DEA-C and DEA-V -

calculations were obtained by adding to program (2.7) the appropriate constraints. To

compute the several variants of the linear programs embedded in DEA methods, we

used the MINOS solver incorporated in the GAMS (General Algebraic Modeling

Systems) language."

As for the FDH, in spite of the fact that this methodology was expressed in terms of an

integer programming approach, its implementation is very easy and does not require

solving any linear program. Exhaustive vector comparisons allow the computation of

all relevant variables. FDH efficiency measurements were then obtained by using a

special program written in the GAMS programming language. This program evaluates

the efficiency degree of the municipalities as follows.

An observation is said to be inefficient if one or more municipalities dominate it.

Recall that domination in this context means that (a) there are other municipalities that

spend less than the dominated and (b) all the indicators of public services of those

11 See Brooke and Meerhaus (1988).


others municipalities are equal or greater than those associated with the dominated

municipality. Conversely, a municipality is said to be efficient if it is undominated.

Using these definitions, the efficiency degree and the excess spending for each

municipality can be computed." Thus, if a municipality is inefficient and dominated by

more than one other municipality, the most dominant is the one with the lowest

expenditure. The degree of cost-efficiency for an inefficient municipality is computed

as the ratio between the expenses of its most-dominating municipality and its own

expenses. By construction, this ratio is smaller than one and greater than zero. The

excess spending, for each cost-inefficient municipality, is calculated multiplying the

complement to unity of its cost-efficiency degree (1-degree of efficiency) by the its

total expenses. These excess spending estimates society's loss "due to the inadequate

use of municipal resources.

3 Data

The implementation of the methodologies outlined above requires information about

aggregate total costs (inputs) and the quantity of public services available to the

population for the municipalities (outputs). In view of the large number of

municipalities and the interregional income disparities typical of Brazil, as well as to

obtain meaningful comparisons, the municipalities were aggregated by state of the

federation. In some cases, when the number of observations (municipalities) was small

thus restricting the possibilities of comparison, data were aggregated by region".

Initially, information for the 4984 Brazilians municipalities was collected. After a

detailed scrutiny of the data, the sample was purged by deleting the data for those

municipalities for which some key information was invalid or missing. This way, 620

communes were dropped since 493 of them had a recorded population of zero and 127

others had no data on current expenditure. 14 Subsequently, cities having more than

12 See Deprins, Simar and Tulkens (1984) and Vanden Ecckaut, Tulkens and Jamar ( 1991 ).

13 See subsection 3.2, for details.

14 The reason why this happens lies on the flourishing creation of new municipalities in Brazil. Indeed, some municipalities, although legally created, have not yet being dismembered from the mother-commune. As a result, they do not report output indicators. In addition, null values for


100,000 inhabitants were dropped to increase data homogeneity. Those large

municipalities will be analyzed late in a separate paper to determine their relevant

characteristics. The final data set was composed of 3756 municipalities aggregated into

15 states and 1 region (the Northern Region)".

3.1 Input and output indicators for the Brazilian municipalities

Aggregate total costs were computed as the value of municipal current spending

(input). 16 As for output measurements, due to the impossibility of quantifying directly

the supply of public services, they were approximated by a set of selected indicators,

which are observable factors taken as proxies for the services supplied. After a careful

choice, six output indicators were retained. A list of inputs and outputs are provided in

Table 3.1.

Table 3.1 - Input and Output Indicators and the Corresponding Municipal

Services

Indicators Code Source Municipal Services of which indicators serve as proxies

Input Indicator

I. Current Spending I.DSP92_C STN I. Aggregate total costs

Output Indicators

!.Total resident population I. POP91_T IBGE' I. Administrative services

2. Domiciles with access to safe 2. DOM91_A IBGE 2. Public health services water

3. Domiciles served by garbage 3. DOM9l_L IBGE 3. Public health services collection

current expenses may be also explained by the fact that the data were not yet been reported by the STN (National Treasure Secretary ).

1' For more details on the data set see Sampaio de Sousa, and Silva (1997).

16 We have dismissed investment expenditures as they are more erratic and would certainly jeopardize the comparison among municipalities. Such expenses will be considered when the study will be repeated for several years.


4. Illiterate population 4.ANALF91 IBGE 4. Educational services

5. Enrollment in primary and 5. EDUC MEC' 5. Educational services secondary municipal schools

Sources: 1: STN - Secretaria do Tesouro Nacional (National Treasure Secretariat); 2:

IBGE - Instituto Brasileiro de Estatistica, Censo Demognifico de 1991; 3: MEC -

Ministerio da Educa91io e Desportos.

Data for current spending were obtained from the Secretaria do Tesouro Nacional

(STN). The data for output indicators Ns. 1 to 4, on Table 3.1, were obtained from the

1991 Population Census. Data on enrollment in primary and secondary municipal

schools (indicator 5) refer to 1991 and were provided by the Coordenadoria Geral do

SEEC/INEP of the Ministerio da Educa91io e dos Desportos (MEC). Below, those

indicators will be briefly discussed.

Current spending (DSP92 _C) corresponds to the total amount of municipal current

expenditure for 1992 as defined by Law 4320/64. It is expressed in cruzeiros, at prices

of 1992." Total population (POP91_T) refers to the population residing between

August 31 and November 1, 1991. The variable Domiciles with safe water supply

(DOM91_A) includes the number of those served by Rede Geral (potable water

network). Domiciles served by garbage collection (DOM91_L) includes those where

the garbage is collected directly or indirectly by public or private services. In the first

case, the garbage is collected at the domicile whereas in the second case the garbage is

dropped in garbage dumps and subsequently removed. Data on illiteracy (ANAL91)

include people who are five years old or older and are unable to read and write a

simple letter in their current idiom. Finally, student enrollment (EDUC) is the sum of

students enrolled in the preprimary, primary, and secondary municipal schools.

17 This variable represented, in 1992, 75% of the total municipal expenditures.


4 Efficiency Results

The methodologies presented in Section 2 were used to measure technical efficiency

for the Brazilian municipalities. Due to obvious space constraints, we will discuss, step

by step, only results for the states of Bahia and Minas Gerais. The analysis can be

applied mutatis mutandis to the other states".

4.1 Measuring efficiency levels for the municipalities of Bahia and Minas

Gerais

In the analysis of the state of Minas Gerais, the sample consisted of only the

municipalities having less than 100,000 inhabitants. Hence, from the 723 communes

included in the database, we considered only 701 municipalities. In the case of the state

of Bahia, we exclude municipalities with population greater than 40,000 inhabitants.

That is, from a total of 411 municipalities, only 402 were included in the sample.

Tables 4.1 to 4.6 summarize the results obtained. Tables 4.1 and 4.2 categorize 701

municipalities of Minas Gerais and 402 municipalities of Bahia into efficient and

inefficient ones, according to three distinct methodologies: Farrell's method (DEA-C),

DEA-V, and FDH. Results for the variant DEA-C, which allows for constant and

decreasing returns, are not reported as they are highly correlated with those produced

by the DEA-V methodology.

Table 4.1 - Rating of the Municipalities of Minas Gerais by Methodology

Population Number of Efficient Municipalities Inefficient Municipalities

Classes Observa-

tions

DEA-C DEA-V FDH DEA-C DEA-V FDH

# T% # 1% # T% # 1% # 1% # 1%

18 Detailed results for all the Brazilian municipalities are shown in Sampaio de Sousa, Ramos and Silva ( 1997).


0-4999 198 I 0,51 2 1,01 100 50,5 196 99,0 197 99,5 98 49,5

5000-9999 191 0 0,00 3 1,57 110 57,6 188 98,4 191 100 81 42,4

I 0000·14999 99 I 1,01 2 2,02 62 62,6 97 98,0 98 99,0 37 37,4

15000-19999 74 I 1,35 I 1,35 58 78,4 73 98,6 73 98,6 16 21,6

20000-24999 35 2 5,71 2 5,71 26 74,3 33 94,3 33 94,3 9 25,7

25000-29999 22 I 4,55 2 9,09 15 68,2 20 90,9 21 95,4 7 31,8

30000-34999 18 I 5,56 2 11,1 14 77,8 16 88,9 17 94,4 4 22,2

>35000 64 3 4,69 18 28,1 53 82,8 46 71,9 61 95,3 II 17,2

TOTAL 701 10 1,43 32 4,56 438 62,5 669 95,4 691 98,6 263 37,5

Table 4.2- Rating of the Municipalities of Bahia by Methodology

Population Number of Efficient Municipalities Inefficient Municipalities

Classes Observa-

lions

DEA-C DEA-V FDH DEA-C DEA-V FDH

# % # % # % # % # % # %

0-4999 6 I 16,7 2 33,3 6 100 5 83,3 4 66,7 0 0,0

5000-9999 67 0 0,0 0 0,0 41 61,2 67 100 67 100 26 38,8

I 0000-14999 113 0 0,0 0 0,0 65 57,5 113 100 113 100 48 42,3

15000-19999 72 I 1,39 I 1,39 45 62,5 71 98,6 71 98,6 27 37,5

20000-24999 58 0 0,0 I 1,72 36 62,1 58 100 57 98,3 22 37,9

25000·29999 22 0 0,0 0 0,0 16 72,7 22 100 22 100 6 27,3

30000·34999 19 I 5.26 I 5,26 II 57,9 18 94,7 18 94,7 8 42,1

>35000 45 3 6.67 10 22,2 40 88,9 42 93,3 35 77,8 5 11,1

TOTAL 402 66 1,49 15 3,73 260 64,7 396 98,5 387 96,3 142 35,3

It is apparent from Tables 4.1 and 4.2 that the FDH results strongly diverge from those

obtained by using the DEA-C and DEA-V methods. Whereas the FDH methodology

shows a large number of efficient municipalities, on the other extreme Farrell's method

finds only 1.4% and 1.5% efficient communes, in the states of Minas Gerais and Bahia

respectively. This proportion slightly increases when the DEA-V method is used.

Several reasons could be invoked to explain such divergence.


Firstly, the frontier of cost efficiency computed by using FDH is based on a stricter

concept of domination than the frontiers calculated by the DEA methods. A

municipality is FDH dominated if and only if all his output indicators are inferior to

the ones of an efficient municipality - effectively observed- with which it is compared,

and its current spending is equal or superior to the expenses of this dominant

municipality. On the other hand, a municipality is DEA-dominated by a fictitious

observation defined as being a linear combination (convex if we use DEA-V) of a

group of efficient municipalities. Therefore, FDH frontiers depend on the possibilities

of factual comparison whereas DEA frontiers "fabricate" their own possibilities of

comparison. When, by lack of information, comparison is impossible, the FDH method

declares the observation efficient by default leading to an increase in the number of

municipalities ranked as efficient.

Furthermore, the convexity hypothesis incorporated in DEA frontiers puts an

unnecessary restriction on the underlying technology for producing public services. In

particular, Farrell's method (DEA-C), that adjusts a cost-efficiency frontier

characterized by constant returns, ignores local non-convexities, hence underestimates

systematically the efficiency degree of the municipalities. So, it may not constitute an

adequate adjustment of the production frontier. Considering that, the data's suggestion

that in most states there are scale economies for small municipalities, is a shortcoming

that may significantly reduce the credibility efficiency measurements based on DEA

C.1' This point will be further discussed (see Section 5 below).

Finally, recall from Section 2 that the method FDH envelops the data tightly while the

frontiers engendered by FDH, DEA-V, and DEA-C are "nested" in one another, with

the FDH frontier staying more close to the data and the DEA-C frontier being the

farthest away." Such a close envelopment results in having more municipalities being

ranked as efficient.

19 Vanden Eeckaut, Tulkens, and Jamar (1991) already pointed out the poor adjustment of Farrell's frontier for the Belgian municipalities they have examined.

20 See section 2.


4.2 Characteristics of the FDH methodology: efficiency by default and

outliers

More detailed FDH results are presented in tables 4.3 and 4.4. At this point, two

aspects of the FDH methodology deserve special attention, viz. efficiency by default

and outliers.

Efficiency by default: Recall that in the absence of a sufficient number of similar

municipalities ("pairs") with which one given municipality can be compared, this

municipality, instead of creating a relationship of the type dominant/dominated, is

declared efficient by default. This ranking of efficiency does not result from any

effective superiority but is due to the lack of information that would allow pertinent

comparisons. In addition, by construction, the FDH concept of efficiency by default

applies both to the municipality that presents the lowest level of spending and to those

with the highest values for at least one output indicator. Notice that the occurrence of

efficiency by default is higher for large municipalities. This is explained by the fact

that the number of large communes is relatively limited whereas there are plenty of

small municipalities. As the FDH method rests on the possibility of comparing

observed municipalities, some large municipalities are declared efficient merely due to

the impossibility of finding similar communes with which they could be compared.

This does not mean that those municipalities are inefficient; it only means that if there

were more municipalities in the relevant range, results could be different.

This extreme form of the sparsity bias that characterizes the FDH technique certainly

leads to a significant overestimation of the number of efficient municipalities and thus

constitutes a serious shortcoming of the FDH approach. Such a limitation is

particularly constraining when there are only a few observations and/or the data are

highly heterogeneous as is the case of the Brazilian municipalities. To partly correct

this problem, we suggested to use the results obtained with the use of other

methodologies. Thus, municipalities declared FDH efficient by default, and also found

to be efficient when the method DEA have been applied, were considered as

effectively efficient.

On the other hand, the FDH methodology is particularly suited to detect the most

obvious cases of inefficiency as this technique is very assertive regarding the

measurement of inefficiency. To each municipality declared FDH-inefficient, it is


possible to find at least one municipality in the sample that presents a superior

performance relative to the first (dominated) municipality.

Table 4.3- FDH Characterization of Efficient Municipalities: Minas Gerais

Classes of Population Number Efficient Efficient and Municipalities ofMuni- Munici- Dominating Efficient by Default cipalities palities Municipalities

Abs % (3/2) Abs % (5/2)

(I) (2) (3) (4) (5) (6)

0-4999 198 100 60 60,0 40 40,0

5000-9999 191 110 76 69,1 34 30,9

10000-14999 99 62 31 50,0 31 50,0

15000-19999 74 58 32 55,2 26 44,8

20000-24999 35 26 14 53,8 12 46,2

25000-29999 22 IS 9 60,0 6 40,0

30000-34999 18 14 5 35,7 9 64,3

35000 + 64 53 15 28,3 38 71,7

TOTAL 701 438 242 55,2 196 44,8

Outliers: By definition, nonparametric frontiers are defined by the extreme values of

the dimensional space of inputs and outputs. Thus, the appearance of outliers, atypical

observations that differ significantly from the rest of the data, may considerably

influence efficiency computations. It is thus necessary to verify whether the divergence

does not result from evaluation errors. However, once one is convinced of the

reliability of the data set, this kind of information may provide valuable information.

Table 4.4- FDH Characterization of Efficient Municipalities: Bahia

Classes of Population Number Efficient Efficient and Municipalities ofMuni- Munici- Dominating Efficient by Default cipalities palities Municipalities

Abs % (3/2) Abs % (5/2)

(1) (2) (3) (4) (5) (6)

0-4999 6 6 0 0,0 6 100,0


5000-9999 67 41 19 46,3 22 53,7

10000-14999 113 65 39 60,0 26 40,0

15000-19999 72 45 27 60,0 18 40,0

20000-24999 58 36 20 55,6 16 44,4

25000-29999 22 16 7 43,8 9 56,2

30000-34999 19 II I 9,1 10 90,9

35000 + 45 40 7 17,5 33 82,5

Total 402 260 120 46,2 140 53,8

For instance, one can identify among the efficient municipalities some that not only

dominate several others but also appear, systematically, as the most dominant

municipality (the one with the lowest expenditure ) and, hence, have a decisive

influence on the FDH measurement of the efficiency levels. These municipalities,

somehow, compared with their pairs, possess the "best technology" thus defining the

technological frontier. Removing one of those municipalities reduces the requirements

to belong to the frontier so those municipalities previously declared inefficient may

tum out to be ranked as efficient.

This point is illustrate in Table 4.5, where some selected municipalities of the state of

Minas Gerais are outliers in the sense discussed above, e.g. they dominate several other

cities thus having a unambiguous influence upon efficiency computations. Consider

first the case of Capiro Branco, which dominates 46 other cities. This municipality,

with a lower spending, has all its output indicators exceeding those of the 46 cities. In

addition, Capiro Branco was also ranked as most dominatini in 18 cases.

Besides Capiro Branco, the municipalities of Alto Jequitiba and Brasopolis also

contribute to build up the boundaries of the technological frontier. Indeed, for 17 out of

the 20 municipalities dominated by Alto Jequitiba, this commune appears as the IllQS!

dominating, thus determining the efficiency rank of the concerned municipalities. As

for Brasopolis, this proportion is still higher as it functions as the most dominating in

10 out of 12 cases. Indisputably, those cities contribute to define the best-practice

fmntkr for the production of public services. Removing them from the sample will

substantially increase the number of efficient municipalities. This is due to the fact that

the concept of efficiency is a relative one and depends not solely on the performance of

the municipality analyzed but also on the achievements of the other cities with which

they are compared. Notice that Itapeva and Jacui, in spite of the fact that each one of


them dominates 9 municipalities, do not influence the assessment of the efficiency

levels as there is no case where they function as most-dominating municipality.

Table 4.5 - Selected Dominating Municipalities: Minas Gerais

Dominating Municipalities Population # of Dominated # of cases where

( 1000 inhabitants) Municipalities the municipalities is ID.QS1: dominatinll

Capim Branco 6334 46 18

Concei9i!o do Rio do Ouro 7695 31 15

Alto Jequitiba 7435 20 17

Lumimirias 5193 23 12

Natercia 4361 20 5

Prados 7371 20 I

Resende 9706 20 8

Florestal 5053 17 I

Guimarania 5739 17 2

Lagoa Dourada 10118 14 2

Ribeirao Verde 3614 14 9

Borda da Mala 15410 13 6

Santa Rita de Caldas 9258 13 I

Brasopolis 13711 12 10

Bandeira do Sui 4100 II 3

Perdigao 4546 II 4

Crucilandia 4579 10 7

Muzambinho 17887 10 6

Sao Tiago 9642 10 I

ltaguara 10671 9 I

ltapeva 5529 9 0

Jacui 6616 9 0

It is worth to point out that many of those outliers are small cities with less than I 0,000

inhabitants. This may be explained by the fact that the possibilities of comparison are

higher for those municipalities but also suggest that small cities are typically

inefficient. This point will be further discussed in Section 5.


Another type of outlier applies to inefficient municipalities. Among those

municipalities, the outliers are the ones, which show extremely low efficiency levels

and/or are dominated by several other municipalities. They represent the most obvious

cases of inefficiency. Table 4.6lists some of those municipalities for the state of Minas

Gerais.21 A typical example is the city of Dores de Guanha. This municipality is

dominated by 55 other communes. Furthermore, to finance the public services supplied

to its population, this municipality, if efficient, needed to spend only 52,8% of its

present expenses. This amount corresponds to the expenses of the municipality of

Cipotiinea - its most dominating municipality - that presents all output indicators

superior to those of Dores de Guanha. From an administrative point of view, this

information has a great relevance. Indeed, this result indicates that Dores de Guanha

could make better use of its resources as several other municipalities - and Cipotiinea

in particular - do. This analysis applies also to the other municipalities listed in Table

4.6.

Table 4.6 -Method FDH: Selected Dominated Municipalities: Minas Gerais

Municipaliti Current Population Efficiency Excess # of dominant Most-es spending degree spending municipalities dominating

municipality

Cr$ 10• mil

Habitantes

Dores de 3159 5403 0,5277 1492 55 Cipotll.nea Guanha

Bel oriente 2413 16718 0,2118 17666 41 Montesia

Joanesia 2886 6941 0,6268 1077 30 Alto Jequitiba

Monjolos 2375 2941 0,5562 1054 28 Acaiaca

Sao 4130 8322 0,5562 1833 25 Resende Gons:alodo Rio Preto

Cajuri 2639 3721 0,6559 908 24 Fortalez

Alpercata 3485 6752 0,5191 1676 22 Alto Jequitiba

Nacip 2259 4003 0,761 540 20 CapelaNova Raydan

Joaquim 2651 4441 0,6722 869 19 CruciHindia Felicio

2' The complete list of the dominated cities is presented on appendix I


Marlieria 2410 3540 0,7154 686 IS Carranca

Iturama 33856 45699 0,3472 22100 16 Vi yo sa

Sao Miguel 2667 6315 0,6783 858 14 Alto Jequitiba

Nova Uniao 2520 4865 0,7726 573 13 Lumimirias

Santa Rita 2273 4064 0,784 491 12 Crucilfmdia do Ibitipoca

Serranos 2562 2036 0,6077 1005 II Rio Doce

Delfinop6lis 3836 6698 0,585 1592 10 Conceiyao do Ouro

Juramenta 3141 6389 0,5759 1332 9 Alto Jequitiba

Conceicao 3839 3839 0,4993 1922 8 Perdigao do Para

Felix 6478 11926 0,4731 3413 6 Brasopolis

Sao 5863 4941 0,3321 3916 5 Luminarias Francisco de Sales

Santa 26510 25931 0,407 15720 3 Saloure Barbara

Pratapolis 3402 9395 0,3894 8183 2 ltanhandu

5 Efficiency and Returns to Scale: Does Decentralization Benefits

Brazil?

The efficiency results for Brazilian municipalities pose a question that deserves to be

carefully examined: the relationship between the size of the municipality and its

efficiency. Indeed, the different methodologies utilized in this research seem to

indicate that smaller cities tend to be less efficient than larger ones. Both under the

FDH method and the DEA variants, the quality of the frontier adjustment improves

significantly as the size of the municipality increases. Figure 5.1 shows that the

proportion of inefficient municipalities is reduced as we consider municipality classes

with larger populations.


Figure 5.1 - Inefficient municipalities grouped according population in alternative

methodologies

~

I ~ ; -0

"1-

BAHIA

100

90 80 70

60 50 40

30 20 10

0-4999 5000-9999 10000- 15000- 20000- 25000- 30000- 34500+ 14999 19999 24999 29999 34999

Cla11 of populaUon

MINAS GERAIS

0-4999 5000-9999 IOOIJO.. 15000- 200IJO.. 25000- 30000· >35000 14999 19999 24999 29999 34999

Class of pepul>lion

I _ OEA.C 1-DEA-V

-FDH c____

Such a result is not uniquely explained by the lack of possibilities of comparison as

stated in the "efficiency by default problem" since it pertains also to the DEA methods,

particularly the DEA-C version, for which the phenomenon of"efficiency by default",

while existent, is much less important. Its justification lies rather on the existence of

local increasing returns to scale prevalent among small municipalities. For those

communes, a given proportional increase in all output indicators could be achieved

with a proportionally inferior augmentation of current expenditures. This implies that it

would be possible to increase the size of the typical Brazilian municipality and yet


provide the required public services to these expanded communities without incurring

in an equivalent increase in public expenditure.

Local non-convexities arise because cities too small are unable to exploit the

economies of scale that characterizes the production of certain public services, hence

do not use the available resources optimally. In the case of educational services, there

is ample evidence that operating costs decrease with enrollment due to existence of

high fixed costs. 22 Consequently, larger schools tend to be more cost-efficient because

the fixed costs are diluted among a higher number of students. This fact, clearly,

discriminates against small municipalities as their schools have, very probably, only a

few students on average and thus tend to present excessively high average costs. Were

those cities larger, they would be able to enroll a greater number of students and

reduce the cost per student without significant loss of educational quality.

A similar explanation applies to other local public services. For instance, the existence

of important fixed costs involved in the production of administrative services (e.g. the

creation and maintenance of a physical and human administrative infrastructure)

explains why the per capita cost of those services is probably higher for smaller

municipalities. A recent paper by Maia Gomes and MacDowell (1997) corroborate this

point. Examining the Brazilian case, they show that municipal per capita expenses with

personnel are substantially higher in small cities. For municipalities with 50,000 or less

inhabitants, those authors show that the average cost curve for personnel expenses is

clearly decreasing. Hence, the dismembering of municipalities creates unnecessary

administrative costs and pushes those communes to work on the decreasing portion of

their average cost curves23 .

Local increasing returns to scale are also responsible for the precarious adjustment of

Farrell's frontier. Indeed, only five municipalities and 3 among those with less than

30,000 thousands inhabitants, in Minas Gerais and Bahia respectively, are at the cost

efficiency frontier (Tables 4.1 and 4.2); the fitting does not significantly improve with

increases in the size of the municipality . This phenomen can be better observed by

22 See Tan e Mingat (1992)) and Sampaio de Sousa (1996).

23 For a good account of the dismembering of Brazilian municipalities, see Maia Gomes and MacDowell ( 1997).


carefully examining Farrell's multipliers.24 Within the DEA-C approach, the presence

of local non-convexities may be computed by the value of the sum of the weights as

stated in the linear program described in Section 2, LYi, evaluated at the optimal

solution. Thus. LYi < 1 implies that locally (for the municipality considered) returns to

scale are increasing; Iyi > 1 points out to decreasing returns; and when LYi = 1, returns

are constant25. Figures 5.2 and 5.3 illustrate this point. They show the logarithm of LYJ

for the municipalities of Minas Gerais and Bahia as a function of the population. We

observe that, in both states, for the majority of small municipalities, the dependent

variable (logarithm of LYi) is negative, indicating the existence of economies of scale.

Those results are maintained for the other states of the Federation as well. A brieflook

at the Farrell's multipliers for the other states indicates the existence of increasing

returns in municipalities with population under 50,000. Thus Farrell's approach, by

imposing proportionality between inputs and outputs, captures as inefficiency what

actually is local increasing returns.

Notice that there is no contradiction between these results and the prevalent decreasing

returns found in the aggregate analysis conducted in Section 3. Indeed, the smooth

frontier computed from aggregate data fails to acknowledge the considerable local

nonconvexities that characterize a large number of Brazilian municipalities.

Fig. 5.2- Minas Gerais - Returns to scale

1.5 .-----------------------,

.::.. 0.5

~ 0 r .0.5

j -1:: -2

••• •

• 80000 100000

• ·2.5 ...._ _____________________ __.

24 The weights Y; defined on section 2.

" See Banker, Chames and Cooper (1984).

Population

Measuring Public Spending Efficiency in Brazilian Municipalities .. .

Fig. 5.3- Bahia -Returns to scale

2,----------------------------------,

·;; e e .. .. e ~

1,5

... 0 .s

•

• • • • • ••

• •• • • 63000

Populalion

' •

•

83000

261

Summing up, the proliferation of small communes, resulting from the intense

dismemberment of municipalities since 1985, reduces the average efficiency levels of

the Brazilian cities and results in a considerable waste of resources. Tables 5.2 and 5.3

present estimates for those losses. The results show that the inefficiency losses are

substantial for municipalities under 10,000 inhabitants. Depending on the methodology

used, the waste varies from 11% to 50% of the available resources.

Table 5.2 - Excess Spending Under Different Methodologies: Minas Gerais

Classes of Number of Spending Excess of Spending Popu-lation Municipa-

lities

(1000 CR$) FDH DEA-V DEA-C

Abs % Abs % Abs % Abs %

0-4999 198 28,24 440996 47041 10,67 183241 41,55 223457 50,67

5000-9999 191 27,24 526056 58220 11,07 208046 39,55 258637 49,17

I 0000-14999 99 14,12 404209 32765 8,11 167093 41,34 184998 45,77

I 5000- 19999 74 10,56 452040 43289 9,58 206083 45,59 214979 47,56

20000-24999 35 5,00 246718 12976 5,26 103406 41,91 107158 43,43

25000-29999 22 3,14 226392 32381 14,30 104028 45,95 105984 46,81

30000-34999 18 2,57 159682 3249 2,04 52371 32,80 54180 33,93

>35000 64 9,12 1385493 135371 9,77 430306 31,06 552989 39,91

TOTAL 701 100,00 3841586 365296 9,5 1 1454576 37,86 1702385 44,31


Table 5.3 - Excess of Spending Associated with Different Methodologies: Bahia

Classes of #Munici- Spending Excess of Spending Popu-lation palities

{1000 CR$) FDH DEA-V DEA-C

Abs % Abs % Abs % Abs %

0-4999 6 1,49 12786 0 0,00 7245 56,66 7945 62,14

5000-9999 67 16,67 186591 64398 34,51 101085 54,17 88744 47,56

I 0000-14999 113 28,11 377071 129631 34,38 174535 46,29 154960 41,10

15000-19999 72 17,91 307973 107675 34,96 132526 43,03 123297 40,04

20000-24999 58 14,43 395502 110195 27,86 163176 41,26 160091 40,48

25000-29999 22 5,47 131466 34541 26,27 70537 53,65 67380 51,25

30000-34999 19 4,73 151509 51288 33,85 75118 49,58 70149 46,30

>35000 45 11,19 570848 43797 7,67 245270 42,97 295397 51,75

TOTAL 402 100,00 2133746 541527 25,38 969495 45,44 967967 45,36

These results indicate that the smaller municipalities are seriously handicapped

regarding the efficient provision of public goods and services. Although further

research is needed on this matter, the prevailing concentration of the Brazilian cities

into the population bracket of under 10,000 inhabitants represents a significant extra

cost for the country as a whole. Very probably, the typical size of those cities is far

below the optimal size required to minimize the cost of the production of public

services'•. Hence, a more rational utilization of public funds should consider not

dismembering but regrouping municipalities. Unfortunately, opportunistic political

considerations may well prevent any serious initiative in this direction. It would take

changing the present array of political and economic incentives to stop and possibly to

revert the dismemberment process.

6 Concluding Remarks

In this paper we have attempted to appraise, quantitatively, the efficiency levels of the

Brazilian municipalities. The objective was to evaluate the performance of the local

26 The paper by Maia Gomes and MacDowell also questions the economic and fiscal viability of the smaller communes.


governments regarding the utilization of public resources. For that purpose, the paper

analyzed the relationship between aggregate municipal current spending and various

indicators of the production of local public services by constructing nonparametric

cost-efficiency frontiers. Different techniques of efficiency analysis were used to

determine this frontier: two DEA variants - DEA-C and DEA-V - and the FDH

approach.

Results obtained by the different methodologies were compared and its main

advantages and shortcomings, discussed. Emphasis special was given to the FDH

procedure. Compared to the DEA techniques, this method is less restrictive as it is

based on weaker hypothesis. Furthermore, instead of calculating an abstract frontier by

referring to a fictitious combination of municipalities, as DEA methods do, this

procedure build up its cost-efficiency frontier by contrasting actually observed

municipalities. That gives to the efficiency scores, mainly those applying to inefficient

municipalities, a credibility that the DEA methods lack. Yet, a main concern with FDH

lies on the fact that, by lack of comparability, this methodology tends to declare a great

number of municipalities efficient by default thus providing limited discriminatory

power. This problem is particularly important when the pattern of observations is

relatively heterogeneous as is the case with most of the Brazilian municipalities. When

this heterogeneity was combined with a relatively small number of municipalities, the

FDH method nearly collapsed by declaring most of the communes efficient by default.

In such cases, the DEA variants proved to be more effective as they handle better this

kind of problem. In short, each method has its advantages and disadvantages. The

appropriateness of their use depends on the particular question being examined and

should not be determined a priori.

As for economies of scale, our results suggest that the Brazilian recent municipal

decentralization policy does not lead to an efficient use of public resources. The

outcome of this policy was a proliferation of small municipalities. Due to their size,

these communes do not benefit from the economies of scale inherent to the production

of certain publics services. They tend to operate with higher average costs thus

bringing about a considerable waste of resources, which can be inferred by estimating

the excessive public spending that characterizes those cities. Therefore, to prevent

further losses in the overall efficiency of local public spending, this excessive

dismembering of communes should be avoided.


It is important to stress the exploratory nature of this study. Efficiency scores should be

used carefully as more detailed analysis is required to determine if the measured scores

reflect genuine technical inefficiencies or if they are explained by the action of others

factors. For instance, in some cases, dominated municipalities may well be intrinsically

different from the dominating ones, and what is regarded as inefficiency could

correspond simply to the effects of such municipality-specific characteristics. In

particular, no attempt was made to include variables reflecting the quality of public

services. Therefore, the low efficiency scores found for some municipalities could well

result from the higher quality of the services provided. Furthermore, indications of

inefficiency arise not only out of administrative incompetence or the lack of

appropriate incentives. They may also be due to the fragility of the data set. Indeed,

due to the deterministic nature of nonparametric models, the computed efficiency

levels crucially depend on the quality of the information used. Missing variables,

measurement errors, and other statistical discrepancies may significantly reduce the

credibility of the estimated scores. Hence, a high priority should be conferred to

improving the quality of the data set by using any suitable information available.

Finally, a close examination of the data set seems to indicate that grouping

municipalities by state of the federation does not apprehend all the complexity of the

economic and social relationship that characterize the Brazilian municipalities. This

problem is particularly relevant when estimating cost-efficiency frontiers that are based

on peer comparisons. Significant differences among municipalities of the same state

substantially increase the degree of heterogeneity of the information, restrict the range

of comparability and, thus, distort the relative basis on which those models are

established. Hence, it is essential to redefine the aggregation base by using criteria

other than the ones implied by the traditional geopolitical division.

References

Aigner, D. J. and S. F. Chu (1968): On Estimating the Industry Production Function, American

Economic Review 58: 826-839.

Anand, Sudhir and Ravaillon, M. (1993): Human Development in Poor Countries: On the Role of

Private Incomes and Public Services, Journal of Economic Perspectives 7: 133-50.


Bauer, P. W. (1990): A Survey of Recent Econometric Development in Frontier Estimation, Journal

of Econometrics 46:39-56.

Bergstrom, T. C. and Goodman, R. P. (1973): Private Demand for Public Goods, American Economic

Review 63: 280-296.

Bird, R. and De Wulff, L. H. (1978): Taxation and Income Distribution in Latin America: A Critical

Review of Empirical Studies, IMF Staff Papers : 639-682.

Broke, A, Kendrick, D. and Meeraus, A (1988): Gams: A User Guide, The Scientific Press, USA.

Charnes, A., Cooper, W. W. and Rhodes, E. (1978): Measuring Efficiency of Decision Making Units,

European Journal of Operational Research I: 429-44.

Charnes, A., Cooper, W. W. and Rhodes, E. (1981): Evaluating Program and Managerial Efficiency:

An Application of Data Envelopment Analysis to Program Follow Through, Management

Science 27: 668-97.

CIDE - Funda9ao Centro de Informayoes e Dados do Rio de Janeiro (1996): Anmirio Estatistico do

Estado do Rio de Janeiro 93-94, Rio de Janeiro.

Debreu, G. (1951): The Coefficient of Resource Utilization, Econometrica 19: 273-92. New York:

Wiley and Sons.

Debreu, G. (1959): Theory of Value, New York: Wiley and Sons.

Deprins, D., Simar, L. and Tulkens, H. (1984): Measuring Labor Efficiency in Post Offices, m:

Marchand, M., Pestiau, P. and Tulkens, H. eds.: The Performance of Public Enterprises :

Concepts and Measurements, Amsterdam: North Holland.

Flire, R. Grosskpof, S. and Lovell, C. K. (1985): The Measurement of Efficiency of Production,

Boston-Dordrech: Kluwer-Nijhoff Publishing,.

Fare, R. Grosskpof, S. and Lovell, C. K.(l994): Production Frontiers, Cambridge: Cambridge

University Press.

Forsund, F., Lovell, C. A. K. and Schmidt, P. (1980): A Survey of Frontier Production Functions and

of their Relationship to Efficiency Measurement, Journal of Econometrics 13: 5-25.

Goode, R. (1984): Government Finance in Developing Countries, Washington: The Brookings

Institution.

Harberger, A. (1977): Fiscal Policy and Income Distribution, in Franck, C. R. and Webb, R.C. eds.:

Income Distribution and Growth in Less Developed Countries, Washington: The Brookings

Institution.


IBGE, (1994): Censo Demogn\fico de 1991, Numero I, Brasil, Rio de Janeiro.

IBGE/DPE/DECNA (1996): Regionaliza9ilo das Transa9oes do Setor Publico: Resultados das

Administra9oes Publicas e da Atividade Empresarial do Govemo, 1992, Rio de Janeiro.

Koopmans, T. C. (1957): Three Essays on the State of Economic Science, New York: McGraw-Hill.

Lovell, C. A. K. (1993): Productions Frontiers and Productive Efficiency, in: Fried, H. 0, Lovell, C.

A. K., and Schmidt, S. S. (1993): The Measurement of Productive Efficiency, Oxford

University Press.

Lipton, M. and Ravaillon, M. (1995): Poverty and Policy, in: Behrman, J. R. and Srinivasan, T.N.,

eds.: Handbook of Development Economics, vol. 3, Amsterdam: North Holland.

MacFadden, D. (1978): Cost Revenue and Profit Functions, in Fuss and MacFadden, eds.: Production

Economics: A Dual Approach to Theory and Applications Amsterdam: North Holland.

Maia Gomes, G. e MacDowell, C. (1997): Os Elos Fn\geis da Descentraliza9ilo: Observa9oes sobre as

Finan9as dos Municipios Brasileiros, 1995, Anais do XXV Encontro Nacional de Economia,

Recife, PE, pp. 645-660.

MEC/SEDIAE/INEP (1996): Estatisticas da Educa91io no Brasil, Brasilia, DF.

Meerman, J. (1979): Public Expenditure in Malaysia: Who Benefits and Why, New York: Oxford

University Press.

MF/STN (1996): Finan9as do Brasil: Receita e Despesa dos Municipios, Anode 1995, Volume XLI,

Brasilia, DF.

Perelman, S. (1986): Frontieres d'Efficacite et Performance Technique des Chemins de Fer, Annals of

Public and Cooperative Economics 4: 449-59.

Pestieau, P. and Tulkens, H. (1990): Assessing the Performance of Public Sector Activities: Some

Recent Evidence From the Productive Efficiency Viewpoint.

Sampaio de Sousa, M. C. (1997): Efficiency and Equity Aspects of Social Spending in Selected

Countries of Latin America and East Asia: A Comparative Approach, Anais do XXV Encontro

Nacional de Economia, Recife, PE, pp. 1328-1347.

Sampaio de Sousa, M. C and Silva, M. C. ( 1997): lndicadores de Servicos Publicos para o Brasil: .

Uma Analise em Nivel de Municipios, IPEA, Brasilia.

Schmidt, P.: On the Statistical Estimation of Parametric Frontier Production Functions, The Review

of Economics and Statistics 58: 238-289.


Seiford, L. M. and Thrall, R. M. (1990): Recent Developments in DEA: The Math Programming

Approach to Frontier Analysis, Journal of Econometrics 46:7-38

Selowsky, M. (1979): Who Benefits from Government Expenditures: A Case Study of Colombia,

New York: Oxford University Press.

Shepard, R. W. (1970): Theory of Cost and Production Functions, Princeton: Princeton University

Press.

Simar, L. (1992): Estimating Efficiencies from Frontiers Models with Panel Data: A Comparison of

Parametric, Non-Parametric and Semi-Parametric Methods with Bootstrapping, The Journal of

Productivity Analysis 3: 171-203.

Tulkens, H. (1990): The Measurement of Productive Efficiency by FDH Frontiers, Document de

Travail, CORE, Universite Catholique de Louvain, Louvain-la-Neuve.

Vanden Eeckaut, Tulkens, H. and Jamar, M.A. (1991): A Study of Cost-Efficiency and Returns of

Scale for 235 Municipalities in Belgium, Core Discussion Paper n• 9158, CORE, Universite

Catholique de Louvain.

Van de Walle, D. and Nead, K. (1995): Public Spending and the Poor: Theory and Evidence, The

World Bank.


Finn R. F0rsund1 and Kjell Ove Kalhagen2

Abstract:

Regional colleges in Norway were reorganised in 1994 with the purpose of promoting

efficiency and productivity. This is the first effort of checking what actually has happened

afterwards with efficiency and productivity. DEA and Malmquist index approaches are used.

Data for three years, 1994, 1995 and 1996 at a department level for about 100 units where

collected by questionnaire and direct contacts. The three outputs where final exams

distributed on two types; short- and long studies, and research publications. inputs where

number of academic and non-academic staff in full time equivalents, current expenses other

than salaries, and building size in square metres. Typical cross section efficiency results show

a large share of efficient departments, with a disproportionate number of efficient

departments giving theoretical general education, and a large variation within the group of

inefficient units. The difference between professional and arts and science departments may

be explained by the nature of the teaching production function, but calculations for a sub

sample of professional departments (e.g. nurses, engineers, teachers) show almost the same

variation within this group. The productivity change each year was mainly positive, with most

departments experiencing a positive productivity effect from frontier shift, but a greater

variation from positive to negative as regards the contribution from catching up.

1 Department of Economics University of Oslo and The Frisch Centre

2 The Frisch Centre

270 Efficiency and Productivity of Norwegian Colleges

Structure

Background

2 Measures of outputs and inputs

3 The method

4 Data

5 Efficiency results

6 The productivity development

7 Further research

References

Efficiency and Productivity of Norwegian Colleges 271

1 Background

Pressure on public sector expenditures has generated interest in performance indicators

the last decades. Higher education in Norway is almost exclusively state run. The

sector consists of colleges and universities. Recent interest in overhauling the

performance of the public sector of Norway resulted in the creation of a Parliamentary

Commission looking into cost efficiency. Performance of the college sector was paid

special attention, because with effect from October 1994, 98 colleges were merged

into 26 new ones. One purpose of the reform was to obtain a more efficient use of the

resources according to educational- and research policy objectives. The task of the

Commission in 1997 was to find out if this potential has been realised.

The new state run colleges consist of totally 109 departments, varying from 1 to 8 with

an average of 4,5 departments. The colleges offers a lot of studies; professional studies

(health and social studies, teacher training, engineering, media, and degrees of

Bachelor of Commerce and graduate engineer), university subjects (minor and major

subjects), or arts and science in general. The colleges are fully financed by the

Ministry of Education, Research and Church Affairs.

In contrast to universities, colleges are relatively more teaching intensive. Another

difference is that the colleges, although required to carry out research, do not have a

national responsibility for performing basic research.

As a part of the work of the Commission the Frisch Centre has undertaken to

investigate the efficiency and productivity of colleges for the relevant time period.

The department level turned out to be the most disaggregated level suitable for data

collection. In our analysis we will regard each department in the colleges as

comparable production units producing education and research. A more ideal level

would have been each study organised under departments.

The initial plan was to collect data for a suitable number of years before the reform

and up to the latest available year, 1996. But it turned out to be impossible to get data

for the pre-reform period for enough departments, leaving us with data for the years

1994, 1995 and 1996. With such a limited number of years our intention with the

productivity part of the study is more to explore the possible methods and result

presentations rather than offer conclusive insights. In defence of the exercise it may be

underlined that this is the first time such an exercise is performed with the applied


methodology, and it may serve as a catalyst for improving the data production in the

sector, or as Rhodes and Southwick (1993, p.146) expressed it: " .. our intention in this

exploratory exercise is to identify areas for more thorough investigation and to bring

some light, however dim, on a question of relative performance that has received little

previous exposure".

When studying inefficiency there are two methodological problems that should be

separated: i) establishing a frame of reference for efficient operations, ii) defining the

efficiency measures. As to the former we will use the non-parametric approach of

DEA, as introduced by Chames et al. (1978) based on an idea of Farrell (1957),

assuming a piecewise linear frontier production structure, and as to the latter we will

use the Farrell (1957) efficiency measures. The motivation for imposing a minimal

structure on the production possibilities is that the technology for college production is

rather unknown, and typically multi-output. Furthermore, there are no prices on

outputs; they are not traded in markets.

Among previous studies using DEA for analysing efficiency in higher education

related to our study we would like to mention Tomkins and Green (1988), Ahn et al.

(1989), Beasley (1990), Rhodes and Southwick (1993), Johnes and Johnes (1993) and

(1995), Doyle and Arthurs (1995), and Sarafoglou and Haynes (1996). Typically, all

studies have used proxies for the ideal output variables (Flemming, 1991 ), such as

number of students at different levels, exam points, number of research publications of

various categories, and research grants. Inputs used have been number of employees

of different categories, especially faculty- and administrative staff, wage bill, building

and equipment investments, expenditure general and maintenance, equipment, support

functions, and research grants. Only Ahn et al. (1989) use data for several years, but

do not calculate productivity changes, but focus on changes in efficiency scores by use

of "windows analysis". We will explicitly calculate productivity changes. The studies

all show a significant dispersion of efficiency scores, and deal with sensitivity analyses

in different ways to illustrate the impact of choice of model specifications. We may

note that quality issues seldom have been dealt with, probably due to Jack of data, but

Rhodes and Southwick (1993) do a two-stage analysis with quality-related variables in

the second stage of correlating efficiency scores with explanatory variables.


Conceptual issues in defining outputs and inputs are dealt with in Section 2. The DEA

method and Malmquist index are presented in Section 3, and the data structure is

shown in Section 4. The efficiency distributions are given in Section 5, and

productivity results and a more detailed analysis of productivity determinants

performed in Section 6. Some remarks on further research are offered in Section 7.

2 Measures of outputs and inputs

When studying productivity and efficiency the key to success is, first of all, to base the

study on theoretically satisfactorily definitions of outputs and inputs, and then to

operationalise these definitions without compromising too much. A fruitful approach

to understand what the institutions in question are producing, is to inspect the

objectives of the activities. In general terms a college produces educational services,

research, and dissemination of knowledge in society at large. Ideal measures of outputs

may be measures of the human capital added for students taking degrees as to

education, addition to scientific knowledge as to research (person-specific knowledge

and general knowledge, according to Beasley, 1990), and increase in enlightenment of

society at large as to interactions college - society (and contribution to "national

culture" according to Higgins, 1989). Operational measures of the first category may

be number and type of exams. Research may be measures by number of research

publications of different types; from prestigious international journals to national

language local working papers (see e.g. Johnes and Johnes (1993) for a classification).

Interacting activities may be measures by newspaper articles, media appearances,

participation of scientific staff in public commissions, and consulting for public and

private sector. Ideal and most commonly used measures are presented in Table I.

The classification of inputs can in general be cast in the KLEM format, i.e. Capital3,

Labour, Energy and Materials. Ideal measures of inputs are hours of labour of different

types, such as scientific faculty, administration and support staff, building space,

various categories of equipment, and current inputs such as energy, cleaning,

maintenance, postage, telephones, stationary. It is usually possible to operationalise

Labour straightforwardly by hours worked by different categories. Areas of buildings

3 K is used instead of C due to tradition.


Table 1: Ideal output measures and operationalisations

Variables

Education

Research

Interaction society

Ideal measures

Addition to human capital

Addition to scientific knowledge

Increase in general knowledge, impact on decision-making

Operationalisations

Stock of students, Flow of exams, degrees

Research publications External research funds PhD's

Newspaper articles, media exposure, participation in public commissions, consultancies

may be supplemented with year of building to indicate functionality. Equipment

should include PCs, but these are difficult to operationalise because ideally we are

interested in the potential productivity of the PC, and actual purchase or replacement

value do not correspond well to the role of the equipment in research. May be capacity

in Bytes and speed in Herz could serve. Usually one has to use purchase figures, and

we have to cope with the distortions created.

The quality dimensions are of especial importance for college outputs. Number and

types of exams do not tell us the full story of the addition to human capital. One way

of capturing the quality dimension of exams would be to have a measure of the

success of the candidates after graduation. In a society where wages are strongly

influenced by productivity a measure of lifetime income would serve as a quality

measure of education. But such information is very difficult to come by, and the

egalitarian structure of Norwegian wages makes the quality signals very weak. A

more limited measure would be the time it takes for students to get jobs after

graduation, assuming that people from the most prestigious colleges get jobs first

(see e.g. Johnes et a!. ,1987). But such measures, which are possible to get from

special labour market surveys, depend heavily on the state of the relevant labour

market. With a low rates of unemployment, as in Norway in the relevant years, many

candidates experience such low waiting times that a correlation breaks down, e.g.


Table 2: Quality dimensions

Variables

Education

Research

Interactions society

Student material

Staff material

Quality measures

Time before getting first job Income level Reputation of college

Citations Peer recognition

Impact on decisions

Qualifications at entry Number of hours studying

Degrees Seniority Participation in networks International conferences

275

because a need for a holiday before entering the labour market may be more

influential than the expected quality of the education.

Quality of research could be captured by influence measures by citation indices (but

see e.g. Flemming (1991) and Higgins (1989) on problems using these). The extent

(national/international) and type of networks of faculty could represent quality, and

also international conference participation. Where relevant the diffusion of research

into practical adaptations in business could be a measure of quality.

It is very difficult to measure the quality of the interactions with society. Impacts

through citations of media exposure could be one way.

The role of students should be paid particular attention. Students are the "carriers" of

education outputs, but are also inputs. The personal qualities of the students determine

how much human capital is actually absorbed during the education. The number of

hours used by students studying will obviously also influence the build-up of human

capital.

Quality of staff may also be of importance. Measures used have been years of

experience, seniority, etc. (see e.g. Johnes and Johnes (1993) p. 343).


The use of proxies for the ideal variables, as portrayed in Table 1, makes necessary

explicit measures of quality. Some suggestions are provided in Table 2.

3 The method

3.1 The DEA Approach

The technology set, S, can in general be written:

S = { (y,x) ly can be produced by x} (1)

where y is the vector of M outputs and x a vector of R inputs. It is assumed that the set

is convex and exhibiting free disposability of outputs and inputs. Farrell (1957)

technical efficiency measures can be defined with respect to this set, and they are

identical to distance functions (introduced to economics in Shephard, 1953) or their

inverse. The input-oriented technical efficiency measure, E1j for unitj is:

(2)

i.e. we seek the maximal uniform proportional contraction of all observed inputs

allowed by the feasible technology set.

Introducing a set ofN observations the set, S, is estimated as a piecewise linear set by:

S={(y,x) I LAnYnm ~Ym (mEM),x, ~ LA.nxnr (rER),A.. ~O(nEN)} (3) neN neN

where An is the weight for observation n when defining the reference point on the

frontier, and N, M, R are also used as symbols for the index sets. It is assumed that the


envelopment of the data is done as "tight" as possible, i.e. minimum extrapolation and

inclusion of all observations are assumed. Further, constant returns to scale (CRS) is

specified. A special form of variable returns to scale (VRS) is obtained by restricting

the sum of the weights to be I:

(4)

A piecewise linear production set with (4) included was first formulated in Afriat

( 1972) as the relevant set for efficiency analysis.

The estimator for the input-saving efficiency measure for observationj is then:

£ 1•1 = (5)

Min{e I LAnYnm ~ Yjm (Vm EM), Bx1, ~ LAnxn,(Vr E R), LAn =I, An~ 0 (Vn EN}} .<,0 neN neN neN

This problem is a linear programming problem with M+R (CRS) (+1 if VRS)

constraints, and can be solved in a standard wa/. Following Charnes et al. (1978) this

is called the DEA model. The VRS case was reintroduced by Banker et a1.(1984),

without reference to Afriat ( 1972).

The Farrell technical efficiency measures are radial, and measure the relative distance

to the frontier from an observation. There are two natural directions: keeping output

fixed and input-orient the measure, and keeping input fixed and output-orient the

measure. The efficiency measures can be interpreted as total factor productivity

measures in the standard meaning of an index of outputs on an index of inputs. The

input-oriented (or input-saving) measure is the ratio of the productivity of the

observation and the corresponding reference point on the frontier, keeping outputs

constant, the output-oriented (or output-increasing) measure is the ratio of the

productivity of the observation and the corresponding reference point on the frontier,

keeping inputs constant. Since the numerators (denominators) of the productivity

4 We are using an in-house program of the Frisch Centre.


indices in the input-oriented (output-oriented) case are identical, we do not have to

worry about how the output (input) index is constructed. The efficiency score is based

on proportional change of all magnitudes. Assuming that the input (output) index is

homogenous of degree 1 in the inputs (outputs) the unknown input (output) index for

the observation cancels out, and we are left with the efficiency score (see F0rsund

(1997) for further explorations).

For a VRS frontier technology the basic efficiency measures are extended to cover

scale (see, F0rsund and Hjalmarsson 1974, 1979). A sort of a scale measure, termed

gross scale measure in F0rsund and Hjalmarsson (1979), but here renamed more

appropriately technical productivity measure, is defined as the ratio of the productivity

of the observation and the productivity at the corresponding ,(i.e. keeping observed

output ratios and input ratios) technically optimal scale point on the frontier. We know

(see Frisch (1965) or e.g. F0rsund, 1996a) that the latter productivity is maximal. The

pure scale measures defined in F0rsund and Hjalmarsson (1979), here simplified to

scale measures, may also be interpreted as productivity measures by forming ratios of

productivities with the input- and output corrected reference points respectively on the

frontier and optimal scale point. To realise that also in these cases we do not have to

know the productivity indices is a little more involved, and require the introduction of

the enclosure of the VRS production function by the smallest cone, i.e. a CRS

technology. We will return to this explanation after the graphical presentation of the

DEA frontier and the efficiency measures provided in Figure 1.

Two inefficient units, P1 and P2 are shown in Figure 1, and the concepts used in DEA

analysis are introduced. The efficiency measures for observations P1 are:

Input- saving efficiency: E1 = Xp /x 1

Output- increasing efficiency: E2 = Y1 /yG,

Scale efficiency, input orientation: E4 = E3 I E1 = (y 1 /xp) I (Ya lxa) ,

Scale efficiency, output orientation E5 = E3 I E2 = (yG/x1) I (y8 /x8 ).


Output CRS-frontier

Referencing units, peers ( E1)

y

YG

\ VRS-fronti"

YB

YI

Yz

Figure 1:

0

Self evaluator

!~ Frontier reference point (E1 )

•A!

L__ Output-slack (E 1) . r~~- .. , Input

DEA frontier, concepts and efficiency measures

The way these measures are defined they are all between zero and one. The

productivity- and scale measures can be expressed as ratios of productivity of the

observation, P1, and its two corresponding frontier points, F and G respectively, and

the maximal productivity at the frontier at B. These measures can also be expressed as

ratios of the slopes of the rays from the origin through these points and the slope of

the ray to the point of maximal productivity, B. Returning to the productivity

interpretation above for the E3, E4 and E5 measures in general, note that the

productivity measure is identical to the input- and output-oriented efficiency measures

with the CRS support technology as the frontier reference technology, as stated above

for Figure I. But this is a general result because with more dimensions we require that

observed output ratios and input ratios are kept fixed. Therefore, the last two relations

are also general. These can than be used to give E4 and E5 productivity interpretations.

The two main technologies, CRS and VRS are shown in the figure. We note the

special feature ofVRS in the DEA case: the technology does not include the origin. A


non-increasing returns to scale technology (NIRS) could also be specified, in Figure I

with OBCD as graph.

The terminology we will use is indicated in Figure 1. The efficient units when

calculating the efficiency score for an inefficient unit are termed referencing units, or

peers i.e. the efficient units with positive 8-weights in (5), and the point on the frontier

is the reference point. Calculating, in the VRS case, E1 for unit Ph units A and B are

referencing units (peers) and F is the reference point. Unit D is efficient, but is a self

evaluator calculating both input- and output- oriented measures.

We know slacks are an integral part of a LP problem. In Figure 1 we have an output

slack when calculating E1 for unit P2• With more dimensions we can also have input

(output)- slacks when calculating input (output)-oriented efficiency, and we have a

choice of presenting the radial efficiency measures, or non-radial ones including slacks

(see e.g. Torgersen et al. (1996) for an overview).

Finally, the LP programme also calculates the duals and gives us all the shadow prices,

which can be utilised to calculate marginal transformation rates and productivities.

The Farrell technical efficiency measure in the CRS case (E1 = E2 ) is the most used,

but also the extended Farrell measures have been used in the literature under various

names. However, the comprehensive scheme offered above, predating this literature,

based on F0rsund and Hjalmarsson (1974) and (1979), seems to have gone mainly

unobserved5. Since the student enrolments of colleges are determined by the

Government it is most relevant to calculate input-saving efficiency measures here.

3.2 The Malmquist productivity index

The productivity index is based on binary comparisons for a production unit between

two time points (or between two different units at the same point in time). The time

5 For instance, Banker et al. (1984) call E3 for "technical and scale efficiency", and E4 for "(input) scale efficiency", while Fare and Lovell (1978), Fare et al. (1985), Fl!re et al. (1994a) do not recognise E3 as a scale measure, but as a technical efficiency measure for CRS technology, probably due to E3 = E1 (CRS) = E2(CRS), and call E4 input scale efficiency measure and E5 output scale efficiency measure.


periods to be compared, are denoted 1 and 2 for short. Only quantities are involved,

and at least one technology has to be known. As a convention we will compare a unit

observed in period 2 with the same unit observed in period 1, i.e. expressions

involving period 2 observations will be in the numerator and expressions involving

period 1 observations will be in the denominator.

Introducing cross-section data sets for several years the technology set, S, has to be

dated, e.g. S\ t 0 T, where T is the set of years. Caves et al. (1982) introduced

productivity indices for discrete observations based on Malmquist (1953). The basic

idea is to utilise Farrell efficiency measures, or distance functions, for the two

observations against a common reference frontier. An efficiency measure can itself be

interpreted as a ratio of the observed productivity and the productivity at the

corresponding point on the reference technology. The Malmquist productivity index,

~1 1·2 , for comparison between two time periods 1 and 2 for a unit j with frontier

technology from period 1 as reference, based on input-oriented efficiency measures, is:

(6)

1,2ET,jEN

The index notation system is that observation years are shown as superscripts, and

technology year as subscript. We have picked out two years called 1 and 2 as

observation years, and used one of them, 1, as technology reference. In general, any

year in the set t can be used as technology reference. The numerator shows the

proportional adjustment, by the scalar 22, of the observed input vector of the period 2

observation required to be on the frontier function of the reference period 1 with

observed outputs. The denominator shows correspondingly the adjustment by i of

the observed input vector of period 1 for the observation to be on the same period 1

frontier function. Note that the measure with different time reference for year of

observation and reference technology now may be greater than one, if the observation

is not feasible within the technology in question. In fact, the measure itself may be

infeasible to calculate. If M1j 1•2 > (<) 1, then the observation in period 2 is more (less)

productive than the observation in period 1.

An output-oriented Malmquist index can be defined in a similar way. Under the CRS

assumption it would be equal in value to the input-oriented index, and the efficiency


measures will always be feasible in principle. If we want to interpret the index as a

total factor productivity index we must base the efficiency measures on comparing

the observations with the corresponding optimal scale points, i.e. we must use the

measure we have termed E3; the technical productivity measure. In practical

applications this is as if we use a CRS reference technology enveloping the actual VRS

technology (enveloping the VRS technology with the smallest cone) (see F0rsund

(1997) for further discussion).

In the presence of inefficient observations change in productivity is the combined

effect of change in efficiency and shift in the frontier production function6. Fare et al.

(1994b) 7 showed how the CCD index in the case of inefficient observations could be

decomposed when there are two time periods and one of them is used as reference

technology. The Malmquist productivity index, ~1 u, can be multiplicatively

decomposed into two parts showing the catching up, MCj/. 2 u, and the technology

shift, MFJ,f

£2 £2 £2 M 1,2 - ~- __.!l:_' __}_}_- MC1,2 . MF2

}1 - £1 - £1 £2 - }1,2 }1,2 }1 }1 }2

1,2 E T (7)

The catching-up effect, MC11,2 '· 2 , expresses the relative movement of the observed

unit to the frontier, a higher (lower) "contemporary" efficiency score for the second

period implying increased (decreased) efficiency. The frontier technology change is

expressed by the ratio of the efficiency scores for the same second period observation

relative to the two technologies. The numerator expresses the scaling of period 2

inputs in order to be on period I technology, while the denominator expresses the

scaling of the same input vector in order to be on period 2 technology, in both cases

subject to period 2 observed outputs. This then serves as a measure of technology shift,

and is greater than one if period 2 technology is more efficient relative to period I

technology for the input-output mix of the period 2 observation.

6 See e.g. Nishimizu and Page (1982) for such a decomposition in the parametric frontier case.

7 Originally circulated as a working paper in 1989.


If another year than the two observation years is chosen as basis for reference

technology, the expression for frontier shift is slightly complicated by imposing

chaining (see Berget a!., 1992):

£2 £2 MF I,2 = )I ;2

}I 2 I I , i,1,2 E T , j E N ' E E ji jl

(8)

The chained frontier technology change is a relative change between period i

technology and period 2 technology in the numerator, and period i technology and

period 1 technology in the denominator.

4 Data

We shall concern ourselves here with data for the 1994, 1995 and 1996 academic years

(the latest year for which data are available at the time of writing). In addition to

public data (NSD, 1997), the data used in the present paper was collected by the

Foundation for Research in Economics and Business Administration (SNF) (now the

Frisch Centre), at college department level. We sent out questionnaires to the 26

regional colleges comprising 109 departments and received data from 23 of them

comprising 99 departments. Unfortunately the project had a very tight time schedule,

so the quality of the data may be negatively influenced by this. In order to secure

quality we followed up the questionnaire by telephone contact and gave all colleges

the opportunity to see our first version of the data for themselves and communicate

any corrections. Although there was some problems with interpretations of our

variable definitions and the tight time schedule, in our opinion the data are of

sufficient quality to express reliable structural features and trends in the regional

college sector.

4.1 Output measures

As proxy-variables for research output (R&D) we asked for information according to

the following typology:


(i) Papers in professional journals

(ii) Papers in academic journals

(iii) Authored books

( iv) Contributions to edited books.

Due to lack of information, some departments could not split up their research

production into these four categories. We have therefore summed (no weighting) these

four categories into one category called R&D.

As output-measure for person-specific increased knowledge we have used total

number of exam credits8 (product of candidates and exam credits). We have split this

measure into two categories due to typical difference in resource usage between short

and long education:

(i) Short education: Studies that is stipulated from 6 months up to 2 years,

plus one year extension course.

(ii) Long education: Studies that are stipulated for 3 years or more.

4.2 Input measures

Four input-measures are used in the analysis:

(i) Faculty staff: Number of faculty staff man-labour year

(ii) Administrative staff: Number of administrative staff man-labour year

(iii) Net operating expenses: operating expenses minus wage costs

(iv) Building capital: Number of square meters.

Two measures of staff man-labour year are used in the analysis. One for the staff with

a solely research and teaching functions and one for staff with only administrative

functions.

8 By stipulation full time students will obtain 20 exam credits per year.


In the analysis we use net operating expenses (operating expenses subtracted by wage

costs) for other inputs, such as cleaning, heating, stationary, telephones and postage,

maintenance9.

We have included size of building in the analysis. It is not obvious that this is an

interesting input in our context. Of course, some minimum space is needed for the

educational process and research, but above that it is difficult to argue that more space

promotes the production of our outputs. Effects of space like it is more expensive to

clean rooms in large buildings, and higher costs associated with central heating the

more space, would be captured by operating expenses. Table 3 summarises the

variables used in the analysis.

Table 3: Variables and definitions in DEA model

(v) Variables

Output measures:

Shortedu

Longedu

R&D

Input measures:

F acuity staff

Admmistrative staff

Net operating expenses

Size of building:

Definitions

Studies that are stipulated from 6 months up to 2 years,

Plus one year extension courses

Studies that are stipulated for 3 years or more

Research publications

Number of faculty staff man-labour year

Number of administrative staff man-labour year

Operating expenses minus wage costs

Number of square meter building

9 Smce Buildings are represented by area rent should have been taken out of the expenditure figures. It could also be argued that maintenance should be taken out, since it could be used as a proxy for buildings, see Ahn et al. (1989).


4.3 Structure of data

The structure of data can graphically be illustrated by joining variables in pairs as

shown in Figure 2. Each histogram represents one department, and the width of the

histogram represents the relative size of the department measured by the number of

full time student equivalents. By ranking the departments in increasing order by the

ratios we obtain information about the total variation in the distribution, the shape of

the distribution, and the localisation of large and small units. The extent of the outlier

problem will be revealed, and data to be double-checked are pinpointed. With totally

seven variables, there are a lot of possible combinations to be shown. We have focused

on six. We have done the calculations for all three years, but will only show the

structure for 1996.

The three first distributions, Panels a, b and c, shows the ratio between the three

products short education, long education, R&D and the input faculty staff

. ..__ ...................... 0 U U U ~ ~ U U ~ ~ I

Panel a: Long education/faculty staff

Panel c: R&D/faculty staff

Panel b: Short education/faculty staff

11000

eooo 1000

i eooo ,,.., 'g <OOO

f:rm ;roo

·ooo ~"'""" ... :Qillli):ID(mmUIEWlJJJ

Panel d: Expendituresllong education


IIIXXXl

IIIXXXl

'"""" i,lOOX> I,"""" jiiXXXl ~SliD . """"

Panel e: Expenditure/shan

education

287

0 ~ ~ ~ ~ ~ ~ ~ ~ ~ I fWettv....O.II~rilt-..df.,._.

Panel f: Administrative stafl7faculry staff

Figure 2:

equivalents.

Salter diagrams 1996. Relative size measured by full time student

Exam credits for the product long education per faculty staff varies gradually from 16

to 314. A little tail of departments representing about 9% of the population of students,

has no long education at all. There is a tendency that medium-sized departments

dominate the most "productive" part of the distribution, but with exceptions.

Exam credits for the product short education per faculty staff (Panel b) varies from 4

to 538 with a median of 35. Also for short education we have departments

representing about 9% of the population of students, with zero output. The distribution

has a different shape with a large share of departments having modest productivity.

Middle-sized departments dominate the most productive part of the distribution, which

has a more marked "best practice" tail than for long education. Small and large

departments dominate the part of the distribution with lowest productivity.

The distribution for R&D per faculty staff (Panel c) is somewhat skewed like the one

for Short education. There are 12 departments that representing about 7% of the total

number of students, with no R&D production at all. On the other side of the

distribution a group of departments that represents 7% of the students has extreme

high R&D production. These units are smaller than the average measured by relative


student population. One of the two most extreme departments is very small and has

over two R&D contributions per faculty staff. Generally we observe that small

departments have R&D productions characterised by larger variance than larger

departments.

In Panel d we have the ratio between operating expenses and exam credits for the

product long education. The distribution shows large variation, from 99 to 8410 NOK

per exam credits. The median is about 522. One department is extreme within a tail

representing about 5% of the students.

In Panel e we have the ratio between operating expenses and exam credits for the

product short education. The distribution shows large variation, from 63 to 159 275.

The median is 1067.We recognise the same extreme department having almost 20

times as high ratio than the median. The distribution is visually dominated by this

observation. Double-checking revealed that the department had had extremely low

number of exams of both types that year.

In Panel f we look at the ratio between the inputs administrative staff and faculty staff

We find a smooth distribution with no extreme outliers, but the most extreme

department has a somewhat higher ratio than the next one. We would expect to see a

mix of economies of scale and professional departments needing more technical

laboratory or equipment staff classified as administrative (not teaching). There is a

relatively even mix of small and large departments in the distribution, but the lowest

ratios are dominated by small departments, indicating diseconomies of scale, while

around the median value medium-sized units dominate. Some large departments have

relatively high ratios. These are professional departments and the technical staff effect

could dominate. But it should be remembered that the distributions are all partial and

that the simultaneous approach below is needed for a proper look into issues like

economies of scale.

Effictency and Productivity of Norwegian Colleges 289

5 Efficiency results

5.1 Efficiency distributions

The technical efficiency of a college reflects the potential for increasing the college

output without increasing the use of resources (output efficiency) or the potential for

reducing the use of resources without reducing the school output (input efficiency).

The analysis makes use of the input efficiency definition. This is due to the fact that

student capacities are regarded as exogenous in the short run. We allow for variable

returns to scale, which means we believe size of college is of importance calculating

the efficiency scores.

The technical measure and the scale measure for 1996 are presented in Figure 3,

Panels a and b. Along the horizontal axis we have all the 99 departments. Each

histogram represents a department and the width is the ratio between student mass at a

department related to the total student population in the college sector. Efficiency is

measured along the vertical axis. The departments are ranked according to increasing

efficiency score.

The distribution for the input saving technical efficiency measure shows that 47

departments of99 are technically efficient (score equal to 1), and these best practice

0 Ot 0.2 Ol 0.4 O.S 06 07 0.8 Ot 1 Relative department size

Panel a: Input-saving efficiency

1996

•.. o,e 0,1

0,0

:a o.• o.• 0.3

0.2

0,1

. ~~~~~~~~~

o 0.1 0.2 o.3 o • o.s o.& 0.1 o._a o.e ' Relative department size

Panel b: Technical productivity

measure 1996


Eflkiency score

"/ ,..:r-. ,.,, ....

Rela1ive dcpartmenl size

Panel c: Efficiency distributions. Input-saving measure, 1994, 1995, 1996

Figure 3: Efficiency measure distributions. Relative size measured by

full time student equivalents 1996.

(BP) departments have a share of students at about 55% . Worst practice departments

(WP) have a share of students at about I 0% when WP is defined efficiency scores

lower than 60% (or 0,6). From the figure we see that the BP units mainly consists of

small and big departments, while WP mainly consists of medium-sized departments.

Panel b shows the distribution for the technical productivity measure. Of 99

departments 31 are scale efficient, and the optimal scale departments have a student

mass at about 33%. The scale efficient units consist mainly of small and medium-sized

departments. WP productivity departments have a student mass at about 15%

efficiency, when WP is defined as efficiency lower than 60% (or 0.6). WP mainly

consists of small and medium-sized departments, but the extreme worst tail consist of

small ones.

Panel c shows the shift of the distribution for input-saving efficiency over the years

1994, 1995 and 1996. The tops of the histogram distributions like in Panel a are

exhibited as step curves. We see that the shape and location of the distributions for

1994 and 1996 are quite similar (but note that movements of individual departments

cannot be seen), and that the distribution for 1995 shows somewhat higher

Effictency and Productivity of Norwegian Colleges 291

inefficiencies that year. The share of students at efficient departments is remarkably

stable10.

5.2 The Peer index

Panel c of Figure 3 shows us that the share of students at efficient units is relatively

high at about a level of 50% for all years for input-saving efficiency. These units are

the peers that inefficient units may study in order to improve their performance. The

efficient units cannot be further ranked as to efficiency score. This has been pointed

out as a problem in the literature, and ways of ranking them have been introduced (see

Andersen and Petersen, 1993). We will here prefer to show an alternative ranking

introduced in Torgersen eta!. (1996). For each efficient unit we have in Figure 4

calculated the share of total potential input saving as to faculty staff that is represented

by the inefficient units that have the efficient unit in question as a peer. We know that

in general there may be several peers for an inefficient unit (in Figure 1 units A and B

are peers for unit P1). The potential input saving is therefore weighted by the weight

of the peer in the calculation of the frontier reference point (the 8n in Eq. (5)). The

peer index is input (or output) specific. We are only showing the index for faculty staff

for the input-saving measure for the three years, identifying the ten most important

peers.

Panel a

10 Note that this approach is different from "window analysis" (Ahn et al. , 1989), where different cross section sets are created by dropping and adding years.


Panel b

....

Panel c ( 1996)

Figure 4: Peer index for faculty staff (input saving efficiency). Ten most important peers

5.3 Stability

A very important opportunity provided by times-series cross section data is to check

on the stability of best practice units. If the turnover is very high then the yearly

efficiency results are driven by time-specific conditions and it is difficult to learn from

the exercise as to policy implications. If the set of best practice departments is fairly

stable, then one has a much more reliable basis as to required policy actions in order

to improve efficiency.


The VRS model yields a fairly high proportion of best practice departments for all

years, 52% for 1994, 45% for 1995, and 43% for 1996. Such relations caution us to

look for self-evaluators. There are 11 in 1994, 9 in 1995 and 10 in 1996, or a little in

excess of 1/5 of the best practice departments each year. Of the best practice units in

1994, about 2/3 are also best practice ones in 1995, and of the remaining efficient

ones a little less than 2/3 remains efficient also in 1996. Of the efficient units i 1995

above 2/3 remain efficient in 1996. The set of units remaining efficient in all years

represents somewhat above Yz of the best practice units each year, or varying from 27

to 24% of the total number of departments. In this set no unit is a self-evaluator in all

years, and only two are for two years, while the percentage of self-evaluators varies

from 1/5 to Jess than 1/10 for each year.

Another way of looking at stability is to inspect the group of most influential best

practice departments. The Peer index for each year in Figure 4 shows us the most

influential peers. Choosing the faculty-oriented index, we have that of the 10 most

influential peers each year, 6, 7, and 4 of the units in the years 1994, 1995, and 1996

respectively remaining efficient all the years belong to the 10 most influential. Of

these, two units, no. I and 67, remain in the top-ten set all years, while five units are in

the top-ten set two of the years. Although not based on any formal test, we conclude

that there is enough stability in our results to claim that the study has revealed some

structural features worth while pursuing for policy purposes.

6 The productivity development

6.1 The Malmquist productivity index

The strength of our approach to calculate productivity growth is that we get the

development for each unit. As a background for a discussion of distributions of

productivity change it may be useful to inspect the average changes of the variables,

set out in Table 4.


Table 4: Percentage change in variables

Variable (95-94)/94% (96-95)/95% (96-94)/94%

Short edu 25.5 0.1 25.6

Long edu 3.4 15.9 23.0

R%D 16.0 10.8 28.6

Faculty 0.4 1.9 2.2

Adm. Staff -0.5 2.9 0.0

Expenditure -19.4 -26.2 -36.1

m2 -1.9 0.3 -1.6

Regarding the three outputs we see a strong average growth in short education in the

first period and a moderate increase in long education, while short education is at a

standstill in the second period while long education has strong growth. A strong

substitution is indicated. Research and development has a high growth in both periods.

As to the inputs all except expenditures (net of wages) are more or less at a standstill.

The expenditures decrease quite strongly. This average development points to

productivity increase on the average driven by output growth and expenditure

decrease. The individual variability was demonstrated in Section 4. The variability in

the outputs short education and research and development, and in the input

expenditures, is much stronger than in the other variables.

Figure 5 shows productivity distributions for pairs of years (1994-95, 1995-96 and

1994-96) in Panels a-c. The frontier for the starting year 1994 is used as reference

technology. Since we are assuming VRS-technology, the Malmquist-index is based on

the technical productivity measures. The productivity index is calibrated such that

productivity estimates lower than I are indicating decreased productivity, and larger

than one increased productivity. If a unit obtains 1.10 this shall be interpreted as a I 0%

productivity growth. The width of the histogram is still proportional to the relative size

measured by the number of full time student equivalents.


'" ! ~ ... E

~

3.5

2,5

1.5

o.5

3.5

if 3

~ 2.5

i 2

~ 1.5

~ 1

0,5

0.1 0.2 o.3 o.• o.5 o.e o.1 o e o.9

RtlaUvt department size

Panel a: Productivity growth 1994-95

0,1 0,2 0 ,3 0,4 0,5 0.6 o. 7 0.6 0.9

Rel•tlvt department aln

Panel b: Productivity growth 1995-96

0.1 0.2 0.3 0,4 0.5 0,6 0, 7 0.6 0.9

Relallvt department t in

Panel c: Productivity growth 1994-96

Figure 5: The Malmquist productivity index

295


Panel a shows the productivity growth in 1994-95. Departments with decreased

productivity represent about 35% of the student mass (in 1996), and departments with

increased productivity growth represent 40%. These variations are large taking into

account the short period, and as expected from the average changes set out in Table 4

and individual variability illustrated in Figure 2. There is a group of departments with

almost no productivity growth covering about 25% of the students. We have a mix of

medium-sized and small departments here. Small and medium sized departments also

dominates the WP group with decreased productivity, and then some large

departments, while medium-sized departments dominates the top group with

productivity growth.

Panel b shows productivity growth distribution for the period 1995-96. Departments

with positive productivity growth represent about 55% of the students (in 1996). In

contrast to Panel a, there is no longer a group of departments with constant

productivity. Large and small departments dominates the group with productivity

decline, while small and medium-sized departments dominates the group with

productivity growth, the latter again in the maximum growth group.

In Panel c we show the productivity growth for the whole period 1994-96. Since we

are applying an index that is chained, productivity growth is simply the multiplication

of the two corresponding numbers for a unit in Panels a and b. Therefore it is not

surprising that we observe different trends regarding which type of departments having

productivity growth. The share of departments with positive productivity growth

increases further, with over 70% of the students at departments with productivity

growth. For 1994-96 we observe no clear pattern indicating whether there are small or

big departments dominating the group with productivity growth, but the positive

productivity growth part of the distribution starts with large and medium-sized units

dominating, then a part with small units, and lastly some large and small units at the

top end. Note that the numbers are rather large for such a short period. The three large

units in the top group in Panel c have an growth in productivity of about 150 % , while

the small best practice outlier has a growth of almost 300%. But the significant

changes in average values revealed in Table 4, and the large individual variation

illustrated in Section 4 support the reliability of the results.


6.2 Decomposition of the Malmquist-index

In Figure 6 we have decomposed the productivity growth from 1995 until 1996 into a

part called "frontier shift" (Panel a) and a part called "catching up" (Panel b), in

accordance with Equations (7) and (8). From Panel a we can see that most of the

departments have gained from a positive shift in the frontier transformation function.

About 67% of the departments (relative size measured by full time student equivalents

in 1996) have benefited from the frontier function shift. The large units have the most

modest impact, while the top group consists of small departments. As to decline

through frontier shift all the groups are represented, with medium-sized departments

dominating the group with most modest impacts, and then large departments. There is

a little tail (about 4% of the students) with a marked contribution in decrease in

productivity from frontier shift.

The "catching up" effect (Panel b) shows large variations, especially at the upper end

of positive productivity growth contribution. The departments that are catching up the

best practice departments represents about 45 % of the students. Large and medium

sized departments, and some small ones, dominate the group with productivity growth,

with the latter group clearly dominating the top part. A share at about 20% shows now

change. This group consists of the departments that are on the frontier both in 1995

and 1996. The units with reduced productivity growth represent about 35% of the

students. Some large departments belong to this group with productivity decline

contributed by catching-up, except from the worst practice group where small

departments dominate (worst practice defined as productivity growth lower than 0.6,

i.e. productivity decline of 40%). Summing up, it seems that positive frontier shift is

most important for small units, negative shifts most important for large units, while

positive catching-up is most important for medium-sized units, and negative catching

up for both small and large units.

6.3 Characterisation of productivity change

It is interesting to examine to what extent changes in the variables from 1995 to 1996

effects the estimated Malmquist index. The classic hypothesis of Verdoom (see

F 0rsund, 1996b) is that there must be growth in output in order to realise productivity

growth. In the spirit of Verdoom we want to investigate the average relationship

between productivity and changes in all the variables. We have made a regression


analysis were the regressors represent the percentage change in the variables from

1995 to 1996. The dependent variable is the estimated Malmquist index from in the

relevant period.

1,8

1,6

;;; 1.4 "1 "' 1, 2 ;

1 = -- ------- ---- -=---...... - .......-n1'1'"f'Fl!!lT-JIII':nlllriiU ~Ul :;:

O.B w ~ ~ 0,6

£ 0,4

0,2

0 0 0,1 0,2 0.3 0.4 0,5 0,6 o. 7 0.6 0.9

Panel a: Malmquist decomposition, frontier shift

2.5

g. 1,5

!!' £ 1 ··-···------------- - -

""' nil O~LU~JJ~W-~~~

0 0,1 0,2 0.3 0.4 0,5 0,6 0.7 0,8 0,9

Pan el b : Malmquist decomposition, catching up

Figure 6: Decomposition of the Malmquist productivity index for the period 1995- 1996. Relative size measured by full time student equivalents.


Table 5: Drivers for the Malmquist productivity index 1995-96. % change in the DEA variables as explanatory variables

Variable: Estimate St.dev. t-value p-value

Shortedu 0.002 0.002 0.96 0.34

Longedu 0.012 0.001 8.54 0.00

R&D 0.349 0.363 0.96 0.34

Faculty staff -1.362 0.813 -1.67 0.10

Adm. staff 0.099 1.614 0.06 0.95

Net oper. exp. 0.000 0.000 -1.06 0.29

Buildmg (m2) 0.031 0.052 0.60 0.55

Total number of observations: 89, R-squared: 0,449, F-value: II ,247

The results from the estimation process are presented in Table 5. In general one would

expect positive signs for output growth, and negative for input growth. But we observe

that there are only two variables having a significant effect on the estimated

Malmquist index choosing a 10% rejection level. These are long education and

faculty staff As expected there is a positive correlation between growth in long

education and the Malmquist index and a negative correlation for faculty staff It is

surprising that changes in operating expenditures are not significant, but this illustrates

the great variance of this variable. The picture above is also relevant for the period

1994 - 1995, and also looking at the decomposition of the Malmquist index into

"catching up". For "frontier shift" it is interesting to note that there are no significant

correlations. We therefore conclude that especially changes in the long education

product and also the faculty staff input are the main drivers behind average

productivity growth.

6.4 Anatomy of productivity change

The development over time for each department that lies behind the average relations

analysed in Table 5, can be illustrated graphically following the classification in Table

6. In Quadrant I we have departments that obtain both positive productivity growth

and positive output growth. These departments have an efficient expansion because

output is growing faster than inputs. In Quadrant II we have departments that


Table 6: Characterization of change

II I

Positive adjustment capability Efficient expansion

Positive productivity growth Positive productivity growth Negative output growth Positive output growth

M > 1, output growth in % < 0 M > 1, output growth in % > 0 "Lean and Hungry" "Top Dog"

Negative adjustment capability Inefficient expansion

Negative productivity growth Negative productivity growth Negative output growth Positive output growth

M < 1, output growth in % < 0 M < 1, output growth in % > 0 "Fat Cat" "Puppy Dog"

III IV

combines positive productivity growth with negative output growth. This is only

possible if inputs are reduced more than outputs. These departments have positive

adjustment capability. In Quadrant III we have departments that obtains a decrease in

both productivity growth and output growth. These departments also have less

reductions in inputs than in outputs, i.e. negative adjustment capability, because the

reductions in inputs are not sufficient to obtain positive productivity growth. In

quadrant IV we have departments that combines negative productivity growth with

positive output growth. These have inefficient expansion because inputs are increasing

more than outputs.

In Figure 7, Panel a we have shown the distribution on the four quadrants when

productivity is linked with growth in R&D. Each square represents one department and

the size of the square is proportional to the number of full time student equivalents in

1996. We can see that the departments are distributed on all quadrants. We observe

units with both negative, zero and high R&D growth, the range is from 100% decline

to 400% increase. (Units going from zero to a positive number have been excluded,


2_4

o24 III

100 0

M(95-96)

2.4

II

0.24

0 IV

400 R&D% Panel a: Research

D E C0 0

0 0 0

0

IV

250 Longedu%

Panel b: Long education

301


2 ,4

0

IV 0 .2•

-100 0

Panel c: Short education

Figure 7: Productivity- and output growth

and units going from a positive number to zero have been given the figure 1 00). Some

units have remarkable high productivity growth and reduction in R&D growth. This

may indicate a substitution effect towards more teaching, meaning an increase in the

number of grade points. But we should have in mind the possibility of lag effects

between faculty input and R&D. One cannot expect a stable relationship year by year.

Panel b shows the distribution when we focus on the long education product. We

observe a longitudinal pattern; growth in grade points is the main driver behind

productivity growth. This is accordance to the average structure revealed in Table 5.

There are relative few units in quadrants II and IV. The majority of departments

experience an increase in long education, but there are also a number of departments

with negative adjustment capability.

In Panel c we are comparing the productivity growth with growth in the short

education product. We no longer find the longitudinal pattern as in Figure 6, in

accordance with the insignificant coefficient in Table 5. Departments are spread over

all quadrants. The average growth in short education is about zero, and it is noticeable

that many departments show positive adjustment capability. There may be a

substitution effect here: the departments with positive adjustment capability have


managed to increase long education sufficiently to achieve positive productivity

growth.

7 Conclusions and further research

In view of the variables we have had to use in the study and the ideal variables set out

in Section 2, it is obvious that the study is far from perfect. However, in order to

generate sufficient interest in engaging in the hard work at the institutional level of

collecting new types of data we believe the study has been worth while. The proxies

used for the three outputs where final exams distributed on two types; short- and long

studies, and research publications. The four inputs where number of academic and

non-academic staff in full time equivalents, current expenses other than salaries, and

building size in square metres.

Typical cross section efficiency results show a large share of efficient departments,

with a disproportionate number of efficient departments giving theoretical general

education, and a large variation within the group of inefficient units. The difference

between professional and arts and science departments may be explained by the nature

of the teaching production function, but calculations for a sub-sample of professional

departments (e.g. nurses, engineers, teachers) show almost the same variation within

this group. The productivity change was mainly positive, with most departments

experiencing a positive productivity effect from frontier shift, but a greater variation

from positive to negative as regards the contribution from catching up. Positive

frontier shift is most important for small units, negative shifts most important for large

units, while positive catching-up is most important for medium-sized units, and

negative catching-up for both small and large units.

Although some doubt has been voiced as to the legitimacy of the present study

representing "true" efficiency, at least the structural differences between departments

as to efficiency and productivity warrant further research.

There are several ways of improving upon the analysis:

304 Efficiency and Productivity of Nmwegian Colleges

7.1 Stage two- analysis

In order to address the question of why units differ in efficiency a second set of

explanatory variables may be introduced (see e.g. Rhodes and Southwick, 1993). The

stage two analysis tries to capture other variables that may affect the efficiency scores.

In order for the procedure to be statistically sound, the new set of explanatory

variables must be uncorrelated with the variables used in the first stage. It is usual to

focus on non-discretionary variables outside the control of the units. We have tested

the quality of staff by position, using as dependent variable the technical input-saving

efficiency score obtained by DEA. It had a (weakly) significant effect on efficiency

scores for two of the years. Number of individual studies offered by a department was

not significant any year, but here we have a covariation problem with inputs used in

the first stage. Other variables could be the location of the college (urban - rural, co

location with other institutions of higher learning), concentration or not of campus

(spread out on different locations or in one location).

7.2 Separating professional and arts and science departments

It may be legitimate to question the assumption of the same technology for all types of

departments. We have experimented with a subgroup of departments giving only

professional education, since the lion's share of efficient departments are arts and

science, and the underlying technology characterising professional education, like

small student groups, need for laboratories, practice outside the college, etc. may well

indicate different technologies. It turned out that the difference in efficiency scores and

the shape of the distribution was very much alike the one for the total sample for 1996.

Further investigations as to teaching technology is warranted. Are small teaching

groups necessary, or just tradition, etc.

7.3 Quality variables

There is an obvious need for variables capturing quality aspects, as discussed in

Section 2. There is also room for improvement of the variables used. The research

output can be more elaborately designed by weighting, and research for departments


like Music and Media must be introduced. Only written reports have been used in this

study.

References

Afriat, S. (1972): Efficiency estimation of production functions, International Economic Review

13(3), 568-598.

Ahn, T., V. Arnold, A. Chames and W. W. Cooper (1989): DEA and ratio efficiency analyses for

public institutions of higher learning in Texas, Research in Governmental and Nonprofit

Accounting, 5, 165-185.

Andersen, P. and N.C. Petersen (1993): A procedure for ranking efficient units in Data Envelopment

Analysis, Management Science, 39, 1261-1264.

Banker, R. D., Charnes, A. and W. W. Cooper (1984): Some models for estimating technical and scale

inefficiencies, Management Science 39, 1261-1264.

Beasley, J. E. (1990): Comparing university departments, OMEGA Int. Journal of Management Sci.

18(2), 171-183.

BergS. A., F. R. flll'sund and E. Jansen (1992): Malmquist indices of productivity growth during the

deregulation of Norwegian banking, 1980- 1989, Scandinavian Journal of Economics, 94,211-

228.

Busch, T., L. Fallan and A. Pettersen (1997): Diciplinary differences in job satisfaction, self-efficacy,

goal commitment, and organizational commitment among faculty employees in Norwegian

colleges: an empirical assessment of indicators of performance, Rapport i T0H-serien 1997:2,

Avdeling for 0konomisk-Administrativ utdanning, H0gskolen i Sill'-Tmndelag.

Caves, D. W., L. R. Christensen and W. E. Diewert (1982): The economic theory of index numbers

and the measurement of input, output and productivity, Econometrica, 50, 1393-1414.

Charnes, A., W. W. Cooper and E. Rhodes (1978): Measuring the efficiency of decision-making units,

European Journal of Operational Research 2, 429-444.


Doyle, J. R. and A. J. Arthurs (1995): Judging the quality of research in business schools: the UK case

study, Omega 23, 257-270.

Erlandsen, E., F. R. F01'sund og K. 0. Kalhagen (1998): Effektivitet og produktivitet i de statlige

heyskoler [Efficiency and productivity in the public colleges], SNF- rapport 14/98, Oslo.

Farrell, M. (1957): The measurement of productive efficiency, Journal of the Royal Statistical Society,

Series A (General), 120 (III), 253-281 (290)

Flemming, J. (1991): The use of assessments of British university teaching, and especially research,

for the allocation of resources- A personal view, European Economic Review 35,612-618.

Frisch, R. (1965): Theory of production, Dordrecht: D. Reidel.

Fare, R. and C. A. K. Lovell (1978): Measuring the technial efficiency of production, Journal of


Fare, R., S. Grosskopf and C. A. K. Lovell (1985): The measurement of efficiency of production,

Boston: Kluwer- Nijhoff.

Fare, R., S. Grosskopf and C. A. K. Lovell (1994a): Production frontiers, Cambridge: Cambridge

University Press.

Fare, R., S. Grosskopf, B. Lindgren and P. Roos (1994b): Productivity developments in Swedish

hospitals: a Malmquist output index approach, in Chames, A., W. W. Cooper, A.Y. Lewin and

L. M. Seiford (eds.), Data Envelopment Analysis: theory, methodology, and applications,

Boston!Dordrecht/London: Kluwer Academic Publishers, 253-272.

Fersund, F. R. (1996a): On the calculation of the scale elasticity in DEA models, Journal of

Productivity Analysis, 7(2/3), 283-302, 1996.

Fersund, F. R. (1996b): Productivity of Norwegian establishments: a Malmquist index approach, in

D. G. Mayes (ed.) : Sources of productivity growth, Cambridge: Cambridge University Press,

315-330.

Fersund, F. R. (1997): The Malmquist productivity index, TFP and scale, Memorandum no. 233, Dept.

of Economics, School of Economics and Commercial Law, Gllteborg University.


F0rsund, F. R. and L. Hjalmarsson (1974): On the measurement of productive efficiency, Swedish

Journal of Economics 76 (2), 141-154.

F0rsund, F. R. and L. Hjalmarsson (1979): Generalised Farrell measures of efficiency: an application

to milk processing in Swedish dairy plants, Economic Journal 89,294-315.

Higgins, J. C. (1989): Performance measurement in universities, European Journal of Operational

Research 38, 358-368.

Johnes, G. ( 1990): Measures of research output: university departments of economics in the UK,

1983-88, The Economic Journal, 100, 556-560.

Johnes, G. (1997): Costs and industrial structure in contemporary British higher education, The

Economic Journal, 107, 727-737.

Johnes, J. and G. Johnes (1993): Measuring the research performance of UK economics departments:

an application of Data Envelopment Analysis, Oxford Economic Papers, 45, 332-347.

Johnes, J. and G. Johnes (1995): Research funding and performance in U.K. university departments of

economics: a frontier analysis, Economics of Education Review, 14, 3, 301-314.

Johnes, J., J. Taylor and G. Ferguson (1993): The employability of new graduates: a study of

differences between UK universities, Applied Economics 19,695-710.

Johnes, J., J. Taylor and B. Francis (1993): The research performance of UK universities: a statistical

Aanalysis of the results of the 1989 Research Selectivity Exercise, J.R. Statistical Society, 156,

part 2, 271-286.

Kalhagen, K. 0. (1998): En analyse av teknisk effektivitet og produktivitet i h0gskolesektoren baser!

pa Data Envelopment Analysis [An analysis of technical efficiency and productivity based on

Data Envelopment Analysis], SNF-arbeidnotat nr. 38/98, Oslo.

MalmqUist, S. (1953): Index numbers and indifference surfaces, Trabajos de Estadistica, 4, 209-242.

Nishimtzu, M. and J. M. Page (1982): Total factor productivity growth, technological progress and

technical efficiency change: dimensions of productivity change in Yugoslavia 1965-78,

Economic Journal 92, 920-936.


Norsk Samfunnsvitenskaplige Datatjeneste (1997): Statistikk om h0gre utdanning 1997. 0konomi,

Studenter, Ansatte. NSD-publikasjon av desember 1997.

Rhodes, E. L. and L. Southwick (1993): Variations in public and private university efficiency,

Applications of Management Science, Public Policy Applications of Management Science 7,

145-170.

Sarafoglou, N and Haynes, K. E. (1996): University production in Sweden: a demonstration and

explanatory analysis for economics and business programs, The Annals of Regional Science,

30, 285-304.

Shephard, R. W. (1953): Cost and production functions, Princeton: Princeton University Press.

Sinuany-Stem, Z., A. Mehrez and A. Barboy (1994): Academic departments efficiency via DEA,

Computers Ops Res. 21, 543-556.

Tomkins, C. and Green, R. (1988): An experiment in the use of Data Envelopement Analysis for

evaluating the efficiency of UK university departments of accounting, Financial Accountability

& Management 4, 147-164.

Torgersen, A.M., F. R. Fmsund and S. A. C. Kittelsen (1996): Slack-adjusted efficiency measures and

ranking of efficient units, Journal of Productivity Analysis, 7, 379-398.

Efficiency and financial performances in

telecommunications

P.-Y. Badillo'

Abstract

Since the break-up of AT & Tin 1984, the telecommunications sector has been in a process of

quick and very important changes. In this paper the evolution of the performances in the

telecommunications sector are analysed by combining two approaches : the DEA (Data

Envelopment Analysis) method and the use of financial indicators. Various DEA analysis,

among which some « window analysis », are applied and throw light and shade on different

facets of the performances of the operators at the international level. Then a brief financial

analysis gives some indications about the financial positioning and dynamics of the

operators. The first main conclusion concerns the use of the network : the European

operators, especially the French operator, offer a limited access to the network with high

prices. The evolving situations of the operators are the second issue : in the United States, in

a competitive context the differences between operators have increased and some

restructurations have occurred; in Europe, the less deregulated operators, Deutsche Telecom

and France Telecom, are behind British Telecom and Telecom Italia.

' Professor at the University of Aix-Marseille; Director of the EJCM (Ecole de Journalisme et de Communication de Marseille), Universite de Ia Mediterranee, CEFI (Centre d'Economie et de Finances Internationales), CID (Recherche sur le Calcul economique, !'Innovation et Ia Decision), France

[email protected]

310 Efficiency and financial performances in telecommunications

Structure

Introduction

2 Monopoly, Deregulation and Efficiency

3 Performances of the Telcos: the DEA Method

4 The Financial Analysis

5 Conclusions: Key Factors for Performances

Annex

References

Efficiency and financial performances m telecommunications 311

I Introduction

For the last fifteen years the telecommunications sector has been in a process of quick

and very important changes. In this paper the evolution of the performances in the

telecommunications sector will be analysed combining two approaches : the DEA

(Data Envelopment Analysis) method and the use of financial indicators. The study

relies on a very complete and original database including financial and technical data

since 1986 on telecommunications operators. The database includes the main operators

in the world (United States of America and European operators, about 12 operators)

and, when the statistics are available, for some indicators less important operators.

So we shall combine two approaches on a quite long period (1986-1997) in order to

examine one of the most strategic sectors of the contemporary society. The first results

are presented hereafter and the research is developing. By observing the efficiency

scores and indicators of financial performances for different firms and countries, we

shall be able to compare the different trajectories and conclude on their relative

efficiency, trying to establish a link with the type of (de)regulation. With such

methodological grounds and with a very rich database we shall try to answer two very

important questions :

• is there a same movement from the point of view of efficiency in production activities and from the point of view of a financial analysis ?

• is there a parallel in the evolution of the different firms or on the contrary a

very contrasted scheme ?

In our paper we shall begin with a brief presentation of the stakes and difficulties

behind an analysis of the performances of the telecommunications operators : are

monopolies inefficient and what are the effects of deregulation, especially from the

point of view of efficiency '? (2). We shall quickly expose the main questions before

defining our methodology based on the DEA method and financial analysis. Then

different possible measures of performances in telecommunications will be proposed

and we shall compare the performances of different telecommunications operators

using the DEA method (3). In a fourth part, a brief financial analysis will be made,

which will give some indications about the financial positioning and dynamics of the

operators ( 4). Our results will emphasize the evolution of the performances of the

operators especially for the recent period.


2 Monopoly, Deregulation and Efficiency

Natural monopoly is a special case from the point of view of economic theory. The

market economy which is omnipresent today has admitted for a long time the existence

of natural monopolies. Until the sixties in highly capital intensive industries -like the

electric power industry, telecommunications, airlines- the regulated monopoly

appeared as the guarantee of economic efficiency -thanks to increasing returns-. Yet as

far back as 1962 two American economists attached their names to what is now called

the Averch-Johnson effect. They showed that the regulated monopoly AT&T suffered

a phenomenon of overinvestment. Being not in a competitive situation AT&T would

have invested beyond the optimum. So the community should have « overpaid » the

telecommunications services. The Averch-Johnson effect was the first line of attack

against the regulated monopoly. On the occasion of the AT&T break up and since then

many new arguments in favour of deregulation have appeared. It is nevertheless

possible to find a common denominator to all these analyses : the regulated monopoly

would be inefficient -overinvestment, bureaucratic management, plethoric manpower

and the inefficiency would appear obvious especially thanks to productivity indicators

or performance analyses. Since about forty years one of the main themes of economic

research on regulation has been the effects of regulatory policies, as noted by R.G.

Noll (1989, p. 1254). Nevertheless studies concerned by the effects of regulation on

productivity have been few. In their article on the effects of economic regulation

published in 1989, P.L. Joskow and N.L. Rose were distressed that« so little effort has

been devoted to measuring the effects of regulation on innovation and productivity

growth ». Since then, regarding telecommunications, the impact of regulation on

productivity or efficiency has been examined in the case of the United States, but less

has been done on other countries and on comparisons between countries.

Thus today productivity and performance analyses are a very important task for firms

as well as for public authorities. Productivity and performances mean efficiency and

competitive advantage, but are to a certain extent an unexplored domain, especially in

the field of an international comparison. Moreover and paradoxically productivity can

be analysed from many different point of views and can give rise to highly shaded or

even opposite measures. The telecommunications sector does not escape this paradox.

Yet on the basis of productivity ratios some economists and decision makers

sometimes recommend massive dismissals measures in order to « improve >>

Effic1ency and financial performances m telecommumcations 313

productivity. Besides, in a more deregulated and competitive context, management and

strategic decisions are often based on some indicators of financial performances.

2.1 The effects of regulation on performances

Previous studies which have covered a vast range of effects of regulation - on prices,

costs of production, investment, service quality, productivity, profits - do not show

undisputable results (for a review of literature see Badillo [ 1999]). In the case of

telecommunications, the break-up of AT&T, the changes in US regulatory rules since

then, and the changes in other countries such as the United Kingdom and France, have

questioned the effects of regulation on productivity and more generally on

performances ; the empirical findings on this issue are various and may differ a lot

from a methodological point of view and on the results. The long debates about natural

monopoly have proved how difficult it is to evaluate, even theoretically, the

advantages and losses associated with different types of market structures and

regulatory constraints (see the abundant theoretical literature; for example A.E. Kahn

(1971], W.J. Baumol, J.C. Panzar and R.D. Willig [1982], R.R. Braeutigam [1989],

J .J. Laffont and J. Tirole [ 1993], J .J. Lafont [ 1994]). The difficulties in estimating a

production function or a cost function have long been the origins of different empirical

findings. In the recent years similar difficulties appear in the discussion about the

comparison between price-cap regulation and rate-of-return regulation (see for

example R.R. Braeutigam and J.C. Panzar [1993]).

In order to evaluate empirically the effects of regulation on productivity and

performances, methodological choices are crucial. P.L. Joskow and N.L. Rose (1989)

point out four basic empirical methodologies for measuring the effects of regulations :

comparing regulated a~d unregulated firms or markets, using variation in the intensity

of regulation, using controlled environment experiments, and structural

estimation/simulation models of regulated firms and markets. The approach by using

controlled environment experiments is certainly promising but is very specific and our

studies have not relied upon it. Structural models of behaviour or performance,

combined with simulation techniques, may be also useful ; they need careful

implementation : for example, the results depend upon the accurate identification and

estimation of demand and cost functions. The first and second basic methodologies

comparing regulated and unregulated firms and using variation in the intensity of

314 Efficiency and fmancial performances in telecommumcations

regulation may be based on cross-sectional or on time-series analyses ; the main

question is the specification of the regulatory regimes. In the first approach the

dependent variable of interest, which may be prices, costs or the rate of technical

change, is defined as a function of exogenous economic characteristics which

influence performance independently of regulation and a variable - generally a dummy

variable - indicating the influence of regulation. In the second approach, variations in

the regulatory constraints over time and space must be estimated. In these two

approaches it is also important to take care of non-regulatory differences between

firms or markets and of the possible interactions between regulatory structures,

economic characteristics of firms or markets and performances. In fact, it is generally

difficult to determine causal relationships between different variables, and more

particularly for telecommunications industry it may be difficult to distinguish the

effects of regulation from the effects of competition. D. Kridel, D. Sappington and

D. Weisman (1996) call this difficulty the "Competition Effect Pitfall" in the

implementation and interpretation of empirical findings in telecommunications ; they

show that the results of econometric models which use proxies for competitive

pressure may be biased and specifically the true impact of a change in regulatory

regime may be over-estimated.

In this paper, our aim is to measure different facets of the performances of

telecommunications operators at the international level. This approach may give some

indications about the link between regulation and performances. In any case, this does

not mean a causality between regulation and performances ; many pitfalls effects could

explain performances of the firms independently of the regime of regulation.

2.2 Productivity, efficiency, performance and financial results

Productivity is an ambiguous concept and we need to define it as precisely as possible.

We can find an early definition in J.B. Say in the "Traite d'Economie Politique"

(1803): "To obtain more product with the same amount of work is the main purpose of

the industry". A common approach is to consider productivity, in the case of a

production unit, as the ratio of its output to his input : according to Z. Griliches ( 1987,

p. I 0 I 0), "productivity is a ratio of some measure of output to some index of input use.

The meaning and quality of such a measure depends on the definition and quality of its

ingredients and on the particular formula and the associated weights used to aggregate

EflicJency and financial performances m telecommunications 315

the various components into one output or input index". So, as far as we have one input

and one output productivity is very easy to evaluate; but generally a unit of production

uses several inputs and produces several outputs. In this case it is necessary to

aggregate inputs and outputs, and the problem arises. Moreover, as C.A. Knox Lovell

(1993, p. 3) points out, "Productivity varies due to differences in production

technology, differences in the efficiency of the production process, and differences in

the environment in which production occurs".

Since we are interested not only by productivity, but by performances, a large

definition of performances might include many aspects : "Economic performance is the

term used to measure how well industries accomplish their economic tasks in society's

interest" (W.K. Viscusi, J.M. Vernon and J.E. Harrington Jr [1995], p. 73). From a

theoretical point of view this refers essentially to efficiency, which may be analyzed in

a static or dynamic way (technical progress), with an approach centered on different

facets of efficiency such as allocative efficiency or/and X-efficiency (H. Leibenstein

[ 1966 ]). From an empirical point of view the emphasis will be put in this paper on

some measures of firms' performances in order to have an approach of productive

efficiency and also of financial results of the firms.

From these general definitions, we can precise our limited goals in the present paper :

in the second section we shall analyse the productive efficiency of the different firms

or countries (the level will change with the availability of the data; we shall called

Telcos the telecommunications companies ; the operators or the countries observed are

described in annex). It is usually recognized that productive efficiency has two

components : technical efficiency which "refers to the ability to avoid waste by

producing as much output as input usage allows, or by using as little input as output

production allows", and allocl!tive efficiency which "refers to the ability to combine

inputs and outputs in optimal proportions in light of prevailing prices" (C.A. Knox

Lovell, op. cit., pp. 9-1 0). Our approach will be based on efficiency through the DEA

method while a specific and complementary financial analysis will be proposed in the

third section. In this paper we do not present technical aspects ofDEA method (see for

example A. Chames, W. W. Cooper, A. Y. Lewin and L. M. Seiford, chapter 1 in P.-Y.

Badillo et J. C. Paradi (Eds.) [1999]) and we only briefly schematize our financial

analysis (see section 3).


3 Performances of the Telcos: the DEA Method

We shall first define the outputs and inputs that can be used for DEA analysis (§2.1),

before presenting our results. The performances of the Telcos will be studied from

different points of view : after measuring their ability to obtain a high turnover (§2.2),

especially on the recent period (§2.3 ), we shall perform another and more original

series of DEA analysis which will permit to estimate the performances of the operators

to develop the use of their network (§2.4), especially the use of new

telecommunications services (§2.5).

3.1 Definition of outputs and inputs for the DEA method

OUTPUTS

As far as OUTPUT (Telecommunications Services) is concerned, it is possible to take

TOTAL TURNOVER or evaluate it in PHYSICAL TERMS (e.g. number of

subscribers, minutes of toll use, optional services sold, etc.).

At first glance, Total Turnover does not seem to pose any particular problem from a

data gathering point of view. But, when the data are examined closely, it is remarkable

how different the methods of collecting the statistics is for each telecommunications

company (Telco). When comparing data obtained from the U.S. with data from Europe

for example, the Europeans publish data that include the total economic activities of

the Telco - as turnover also comprises revenues apart from its monopoly activities -

while the FCC (Federal Communications Commission) in the U.S. collects data strictly

from the Telcos' telecommunications activities.

An evaluation of the "physical" output is similarly troublesome for two reasons : first,

the enterprises do produce multiple forms of outputs, second, all the operators do not

furnish the same statistics on the same set of outputs. Hence, one can envisage at least

two indicators of physical output : the total number of calls or the total number of

minutes of communications on the network.

Eftlc1ency and financial performances in telecommumcations 317

The total number of calls completed on the network

The statistics relative to this measure have been utilized in other studies for calculating

productivity. We consider that productivity ratios constructed this way (using the

number of calls as the numerator) must be treated with the utmost prudence for the

following reasons :

a. First of all, the number of calls may well be evidence of data representing volume which is very sensitive to the evolution of the tariff structure. This means that as the price is lowered in the framework of "regulation", as it was in the United Kingdom, the number of calls increases. In this instance we cannot report the results as productivity improvement due to either a change in capital structure or labour deployment, but merely productivity changes resulting from output volume changes (calls completed).

b. The other major effect on the number of calls is the size of the network. In effect, a telephone network offers to its subscribers the potential for calls which is an exponential function with respect to network size (a function of the type: f{n(n-1)12}).

Thus, for a network four times larger, the number of potential calls is much greater. And effectively there is more domestic communications traffic in the U.S. than in Europe (for example, a U.S. company will have many subsidiaries and branch offices all over the United States and moreover the intra-country tariffs in Europe are much higher than inter-state tariffs are in the U.S.).

c. Furthermore, the number of calls also depends on the range of services offered on the network. The American Telcos have offered a whole series of services, introduced well ahead of the Europeans, that have generated a large number of calls (but not necessarily for longer duration).

The total number of minutes used on the network

This data is very interesting when characterizing output in physical terms. Quite

obviously, it also poses problems, particularly when it comes to obtaining comparable

figures from different countries.


INPUTS

In accordance with the input selection, it is possible to construct partial or total factor

productivity. The accuracy of the indicators LABOUR PRODUCTIVITY and

CAPITAL PRODUCTIVITY is affected as both labour and capital are evaluated

differently by each Telco.

The factor - Labour

When evaluating the LABOUR FACTOR we shall use two methods : the total number

of employees which represents a "physical" evaluation of labour and gives the

productivity figures on a per employee basis. The wage and salary values, which

provide an economic evaluation of productivity and permit us to calculate, on average,

the productivity per $1 spent on wages and salaries.

The factor- Capital

Dealing with the CAPITAL FACTOR we shall use, once agam, two types of

calculations : the number of lines installed which represents the "physical" capital and

permits us to evaluate the Telco's infrastructure; and the tangible assets represented by

the estimate of net plant and equipment, to provide an "economic" measure of the

stock and equipment used in production (taking into account depreciation of the

equipment and its functionality as relating to their age and technical obsolescence).

The possibility of choosing different inputs and outputs means that multiple DEA

analyses can be conducted, and depending on the indicators chosen, different results

can be obtained.

We shall now give and discuss the main results of our analyses based on the DEA

method. First we shall present a standard analysis which includes allocative efficiency

because it contents data evaluated with the weights of prices. In a second part we shall

propose a more robust analysis because we shall use only physical inputs and outputs.

The synthesis will enlighten the main results.

Efficiency and financial performances in telecommunications 319

3.2 A first approach : the performances of the firms to obtain a high

turnover

Our approach is input oriented with the variable returns to scale hypothesis. We

observe one output : the turnover for the year 1994 (In the following paragraph 2.3 a

complementary analysis will be proposed for the recent period 1992-1997). This

output is obtained through the two main inputs : capital and labour. According to the

data used for evaluating capital and labour, different DEA analysis can be performed.

The table hereafter indicates the three DEA analysis that aim to measure the

performances of the operators to obtain a high turnover in 1994.

Summary of the DEA analysis performed in the paragraph 3.2

Inputs Output

DEA 1 Wages and salaries* and net plant and Turnover* equipment*

DEA2 Wages and salaries* and number of Turnover* main lines

DEA3 Number of employees and number of Turnover* main lines

* in constant $

If we evaluate the inputs in money, the evaluation of labour is done by wages and

salaries, and capital corresponds to the net plant and equipment. The results are given

in figure I as DEA 1 :

European Telcos are in very good position : the efficient firms are ALL (Deutsche Telekom), OK, IRL, ITA, UK (British Telecom);

very near these efficient firms we find : FR (France Telecom) and some BOCs (SW and BA). ESP and AUT are behind.

These first results seem to indicate that a link between regulatory regime and

performance is not obvious : some BOCs as well as some European operators obtain a

good level of performance.


However we can ask ourselves about the input in capital : capital is a very specific and

difficult indicator to evaluate; and the framework of an international comparison

reinforces this measurement difficulty (with the problems linked to different account

systems, depreciation and so on).

So we have used a physical indicator of capital : the number of main lines which

constitutes the basis of the capital of an operator in telecommunications. Thus a new

DEA analysis is conducted (see DEA 2 in figure I) with still the same output

(turnover) and the following inputs : labour evaluated by wages and salaries and

capital through the number of main lines. The results are very near the first analysis,

and this physical evaluation of capital (number of lines), even if it is not a perfect one,

suffers less critical comments than an evaluation in money.

Output : Turnover If now we substitute N (number of employees) to wages and salaries

we obtain new results which show how the inputs labour and capital evaluated in

physical terms are used by the different operators to obtain their turnover : in this case

(DEA 3 in figure 1), the BOCs appear in a relative better situation and the following

operators obtain worse performances than in DEA I : BEL, DK, FR and UK.

Figure 1: Efficiency scores: DEA I to 4

Inputs: Capital and labour Output: Turnover

1.2 .---------------------------------------------------------,

0.8 -

0.6 -

0.4

0.2

Effic1ency and financial performances in telecommunications 321

3.3 The evolution for the recent period 1992-1997: a "window analysis"

In order to study the evolution of the Telcos performances, we first made a DEA

analysis for the year 1989 which permits the comparison with DEA 3. We estimate the

turnovers of the different operators for 1989, after converting current data into

statistics expressed in fixed value dollars (in $1994 and in price 1994). Every impact of

dynamics due to inflation or exchange rates has been eliminated. This DEA analysis

(DEA 4 in figure 1) does not show a very important evolution between 1989 and 1994.

The BOCs were already in a position of leader in 1989 and the other European

operators have improved their scores of efficiency during the period 1989-1994. This

is a very preliminary result about the dynamic analysis, which is coherent with the fact

that the period 1989-1994 corresponds to a first stage of deregulation.

It is more interesting to observe the recent evolution. We made a senes of DEA

analysis similar to the DEA 1 to 3 for the period 1992-1997. The number of operators

included in our analysis has been limited by the availability of data. So we have

compared the main European operators (France Telecom, Deutsche Telekom, Telecom

ltalia, British Telecom) and some American Telcos for which data have been available,

that is to say Ameritech, Bell Atlantic (before and after the merger with Nynex), US

West. Given the small number of operators, and in order to have enough data for an

analysis, we have performed a "window analysis" using a three-year window, on the

period l 992-1997. As a whole, the European operators keep a good score for DEA I,

while the American operators are still the leaders for DEA 3. In the recent period the

hierarchy is the following one for DEA 3 (from the weakest to the highest score) :

France Telecom, Deutsche Telekom, British Telecom, Telecom Italia, then the

American Telcos.

Our conclusion arises now : if the European operators obtain good performances,

when we evaluate their capacity to have an high turnover with inputs evaluated in

money, their situation is worse when labour and capital are not weighed by

prices. We can note that Switzerland (not shown in the above figure) has a good

performance in any case with high salaries : probably it is due to a very high turnover

explained by the specific position of this operator (a monopoly with a lot of

international communications).

With all these DEA analysis we don't show a discriminated evolution between the

main operators, which would be linked more specifically to different regulatory

322 Efficiency and financtal performances in telecommunications

schemes. These first results can be compared to those ofT. Sueshoyi [ 1994] who uses

OCDE data but only for the year 1987 ; Sueyoshi applies a stochastic frontier

production analysis and takes into account three inputs (the number of telephone main

lines, the number of employees and the total amount of capital investment) and one

output (the total amount of telecommunications service revenues). He shows that four

countries, i. e. Iceland, Norway, Switzerland and the United States (taken as a whole,

the firms are not studied), were efficient according to the method employed, while

Germany had a bad score (0,70) as well as Italy (0,71), and the United Kingdom and

France reached intermediate scores (around 0,85). The results of Sueyoshi based on the

same type of inputs and output are comparable to those we obtain with DEA 4,

especially concerning the position of the American operators and of Switzerland (the

case of Germany is specific since the German operator was included in Deutsche

Bundespost in 1987, which induces difficulties for some statistics). By studying the

more recent period (1992-1997) we bring to light the favorable dynamics of an

operator, Telecom Italia, while among the European operators the position of France

Telecom seems to become a little less good, Deutsche Telecom has regressed a little,

and some movements have caracterized the situation of British Telecom because of the

regulatory evolution (price-cap regulation) which has affected directly its turnover.

However there has been no great gap, nor very significant evolution.

We propose now a complementary study based on "physical" data which enlightens

technical efficiency.

3.4 Performances of the operators : their capacity to use the network

We shall now use only physical indicators and try to appreciate how the different

operators have developed the accessibility of their network. Two inputs will be

considered : labour measured by the number of employees and the number of main

lines as an estimation of the capital of each operator.

We have chosen two different outputs : the first often quoted is the number of calls.

The second is a statistics very difficult to obtain : the number of minutes of use of the

network; this statistics is the best approach to appreciate the real use of the network.

Effictency and financial performances in telecommunications 323

First, we shall provide a large analysis for 19 operators in 1993, for the output number

of calls. Then we will try to grasp the dynamics of performances : so a "window

analysis" will be performed for a limited number of operators with the number of

minutes of communications as the output. Finally, the same type of analysis will be

conducted but with the number of calls as the output.


Inputs Output

DEA 5 Number of employees and number of number of calls main lines

DEA 6 Number of employees and number of number of minutes of communications et 7 main lines

DEA 8 Number of employees and number of number of calls et 9 main lines

DEA 5 (see figure 2) corresponds to a large analysis for 19 operators in 1993 with the

number of calls as the output. The BOCs appear in the first position. Only Finland,

Denmark and Switzerland obtain a good score. Other European operators are very far

behind: with a score of only 25% Germany, France and the United Kingdom have to

do a lot to improve the use of their network.

Figure 2: Efficiency scores : DEA 5

Number of calls (output) and number of employees

and of lines (inputs)

1.2

0.8

0,6

0 ,4 - --• I

0.2

0

• I • I I I I

ALL OK ESP AM BA BS NY PA SW US BOC FIN FR ITA NTT NDL POL UK SUI


One may wonder if there has been an evolution during the recent years. In order to

give an answer we propose a "window analysis" for the periods 1988-1990 (DEA 6)

and 1995-1997 (DEA 7). The figure 3 gives the scores corresponding to DEA 6 and

DEA 7 with only one point by firm at the begining and at the end of the period. The

preceding result is confirmed : the American telcos are far behind the European

operators concerning the efficiency in giving access to the network. The score of the

European firms is about 40% at both the beginning and the end of the period ( 1988-

1990 or 1995-1997). The DEA 8 and 9 analysis give similar results and therefore are

not presented here.

It seems that the different regimes of regulation have some impact on productivity or

on performances of the different firms. We can verify that the first place belongs to the

BOCs and that the United Kingdom, which was the last in 1989, is before France

Telecom and Deutsche Telekom in 1995-1997. British Telecom, submitted to an

evolution of the regulation, has very much improved its relative position ; as a matter

of fact, this improvement has been correlated to an important slowdown in the number

of employees (246000 employees in 1989, 124 700 in 1998).

We shall be able to confirm (or infirm) these first conclusions by a new analysis which

takes into account the development of telecommunications industry from a general

point of view.

Figure 3: Efficiency scores

DEA 6 and 7 (1988-1990 and 1995-1997). The use of the network (number of minutes of communications as the output).


3.5 The development of the telecommunications services

We have to precise our analysis because it is a new approach of the problem. In the

previous paragraphs we estimated performances on a classical basis with outputs like

turnover or number of calls. Now we consider that for the last decades, even in the

USA or in the United Kingdom where deregulation or privatization have taken place,

national operators have been in a situation of quasi monopoly in their area (it is the

case for the BOCs, and in the United Kingdom BT had not still a strong competition

from Mercury until the recent years). In such a situation it is interesting to appreciate

how some new services have been developing in different countries. The outputs for

the following analyses will be the numbers of Fax, of pagers and of mobiles which

indicate the level of development of telecommunications in each country, compared to

the number of lines and the revenue from telecommunications services. In other words

we shall analyze in which proportion an operator with a large infrastructure in lines

and an important revenue contributes to the development of the new services of

telecommunications.


Inputs Outputs

DEA 10 and DEA Turnover and number of main Fax, Pagers and Mobiles II lines

Figure 4: Efficiency scores DEA 10 and DEA 11

The contribution to the development of new services (Fax, Pagers

and Mobiles, 1990-1996).

ALL AUT BEL ESP FIN FR ITA NDL PORT UK UE USA JAP


The results for 13 countries for 1990 and 1996 (DEA 10 and DEA 11 in figure 4) are

the following: the three leaders in 1990 as well as in 1996 are : USA, Japan and

Finland. We can observe that the three main European operators are very far from the

level of these leaders. Nevertheless France and Italy have progressed very quickly

during the period.

4 The Financial Analysis

4.1 A brief presentation of the indicators

Our goal is now to observe the performances of the Telcos from a financial point of

view. We need not a detailed financial analysis for each operator ; we have chosen

some indicators in order both to compare the firms at an international level and to have

an idea of the evolution of the main financial criteria. From the database which has

been constituted different ratios can be calculated; the three ratios which are reported

hereafter are very significant of the financial situations of the Telcos.

I. The first ratio is a classical measure of the profitability of a firm from the point of view of equity owners. The return on equity measures the ability of the firm to generate profits available to common shareholders. This ratio is the following one:

Rr= Net Income I shareholders' equity

2. Profitability can be analysed from another point of view, which insists on some economical aspects. Among the potential indicators, one has appeared very significant for comparing the operators from the point of view of their ability to generate earnings thanks to their operating activity. We shall report here a ratio with the sum of the operating income and depreciation and amortization in the numerator and the revenues from sales in the denominator. So the numerator is an estimation of earnings before interest, taxes, depreciation and amortization, which is noted EBITDA. This statistics is often considered as the most important indicator of their operating performances by US firms.


The EBITDA margin or ratio is the following one :

EBITDA I Turnover

3. It is also useful to compare the performances of the tel cos from the point of view of their financial independence, which means to have an idea of the influence of debt. The third ratio reported here takes into account the capacity of the firm to generate Cash Flow in order to cover its debt. More precisely, the numerator corresponds to the indicator called "Capacite d' Autofinancement" in the French conception which differs a little from the American "Cash Flows from Operating Activities" ; of course, the estimation of this statistics, noted CAF, is not easy on an international level because of the differences in the presentation of financial statements from country to country. In the denominator we use only the longterm debt noted D, which are the loans and other borrowings with amounts falling due after more than one year. It is clear that the ratio is decreasing when the long-term debt of a firm tends to increase while the company has difficulty to increase or even maintain its cash-flow.

So the third indicator will be called Cash Flow to long-term debt ratio and noted

CAF I D.

4.3 Main results of the financial analysis

It is necessary to insist again on the great precautions which have to be taken in order

to make international comparisons because of the difficulties in collecting comparable

data. Moreover, the financial time series are not always homogeneous; in many cases

we can observe some statistical breaks : either some important changes in accounting

principles were introduced in a country or for a firm and the series became not

comparable from one year to another or there happened a special event, such as for

Deutsche Telecom the reunification of East and West Deutschland and the

corresponding changes in the financial statements.

We can give an example of the difficulties to analyse financial performances in

dynamics. If we are studying France Telecom, we must remember that there are at

least four important discontinuities in the time series :


the introduction of the value added tax (TV A) at the end of 1987 leads to very large changes in data for 1988, especially the falls in the turnover and the net income, which have repercussions on ratios such as Rr

there is also a discontinuity in the capital time series because of a change in the accounts in 1991. This explains the decline of the ratio Rr between 1990 and 1991.

In 1994 for the first time the accounts were published for the group France Telecom. Moreover, there was a change in the fiscal system applied to France Telecom. For 1995 and 1996 the only collected data concern the group and no direct comparison is possible between the data for the company France Telecom (head office) until 1994 and the group from 1994.

Similarly the regulation of France Telecom changed at the end of 1996 and this affects almost all the data.

However, we have elaborated an original database on over ten years which permits to

measure the financial performances. In order to have a clear interpretation we present

and analyse herafter the three chosen indicators for some operators which constitute a

good representative sample of the main financial situations in the telecommunications

sector at an international level.

The study of the three financial ratios (see figure 5 hereafter which gives results for

1988 and 19972) leads to three main results :

I. The differences between operators are important. In particular, we can observe that the less deregulated operators (deregulation appears only at the end of the period of observation), France Telecom, Telecom Italia and Deutsche Telecom, obtain less good results in terms of return to equity and, at a weaker degree, in terms of the CAF I D ratio (except for the last year concerning CAF I D), but better results with the EBITDA ratio which reflects operating performances. We can propose the following explanation : deregulation needs good financial performances for equity holders and a capacity of the firm to resist to competition pressures, while a regulated monopoly is not very much concerned by return to equity, nor by the capacity to reimburse the debt but can obtain a high turnover and consequently high operating ratios without price constraint. Price regulation has been in effect only recently for France Telecom, Deutsche Telekom and Telecom Italia.

! For Nynex and Pacific Teles1s the data are for 1988 and 1996 smce the two compames merged respectively w1th Bell Atlantic and SBC m 1997.


However, differences inside the group of the Europeans and inside the group of the Americans have increased and are to-day important. So, while we have observed, thanks to DEA analysis, a great contrast between the European and the American operators concerning the use of their network, the financial indicators of performances show a weaker opposition, especially at the end of the period.

2. In fact, if we observe the dynamics of each operator, some significant changes appear in financial performances. One of the most obvious changes is the progression of the financial performances of BT and Telecom Italia among the Europen operators. BT has to-day a quite high level of return to equity, a good EBITDA ratio and above all a high cash flow compared to its long-term debt. The Rr and CAF I D ratios ofTelecom Italia have also increased.

In a very slight deregulated context, France Telecom seemed until recently to have good tinancial performances if we refered to the EBITDA ratio, thanks to a price policy which, nevertheless, limited the uses of telecommunications services; but, on the one hand, the CAF I D ratio has been at very lower levels than those of BT or Telecom Italia, and on the other hand, in dynamics the EBITDA ratio has been declining. The situation of Deutsche Telecom is the worst of all the European operators concerning the EBITDA ratio. So from a financial point of view DT and France Telecom have inferior performances by comparison to BT and Telecom Italia.

Figure 5: Three financial ratios 1988 and 1997: Rf, Ebitda ratio, CAJt'ffi

120%

100%

80%

80%

• 0%

20'>\

0% Nl. BA BS NY PA SoN US BT OT F'T ITA

•Rree . RIV7 [] El>itdl88

C E-97

. CAf/088

. CAf/097

__ j

3. If deregulation is an incentive for better performances, it has not the same effects on every firm. In a competitive context some firms can be in trouble. In the United States the increased competition and the evolution of the regulation rules


have been favorable to some BOCs, but others have had difficulty in adapting themselves. According to a normal competitive process mergers have taken place; in 1997 Bell Atlantic entered into a merger agreement with Nynex and SBC Communications Inc., formerly called Southwestern Bell, merged with Pacific Telesis. In 1998 SBC entered into another merger with Ameritech and a merger between Bell Atlantic and GTM was announced. If we consider the two mergers in 1997, the BOCs with very good financial performances were in a position of leaders in these mergers with the BOCs which had worse financial performances (for example, Nynex had a very bad CAF I D ratio in the three years preceding the merger with Bell Atlantic).

The analysis of the financial performances shows quite well the evolution of the telecommunications sector at the international level : in the United States, in a competitive context the differences between operators have increased and some restructurations have occurred; in Europe, Deutsche Telecom and France Telecom are behind British Telecom and Telecom Italia.

5 Conclusions: Key Factors for Performances

We can enlighten the progress and the interest of our study at two main levels :

- methodology,

- results concerning the behaviour of the Telcos, their performances, and the

impact of the regulatory regime on the development of telecommunications uses

(in a large acceptation).

1. DEA appears to be a very interesting method to evaluate performances, from the

productive efficiency point of view, of the observed firms by taking into account

many inputs and many outputs. Moreover, we propose an analysis on a very

recent period for some indicators.

A financial analysis gives some complementary indications about the relative

forces and difficulties of the Tel cos.

At this stage of our research we want to precise that the DEA analyses and the

financial ratios have to be interpreted with precaution : of course the results are

dependent of the panel of units we observe and of the data. A very difficult task

is to obtain and organize a reliable database for a long period. This is why we ask


the reader to consider some results as preliminary (the DEA scores may be

influenced, for example, by too narrow a panel, and so on).

2. Keeping this remark in mind, we can give some conclusions about the

performances of the telecommunications operators.

The performing Telco will be the one which :

- encourages the growth of the number of calls,

- favours an increased use of its network from the point of view of the number

of minutes of communications,

- reduces its tariffs,

- contributes to the development of new services.

This is an original approach of the performances by comparison with many works

which relied on a global indicator based on productivity or cost, as well as with

the study of Sueyoshi [ 1994] which does not take into account the use of the

network and the development of services. Not only we have shown that the use of

the network is very low in Europe comparatively to the USA and Japan if we

consider the number of calls as well as the number of minutes, but we have also

given an evaluation of the contribution of the Telcos to the development of

telecommunications uses in a very large acceptation. With this last consideration

it appears that the USA and Japan are far ahead the other Telcos; probably this is,

to a certain extent and with a lot of precaution, due to the deregulation regime of

these countries. The United Kingdom is a special case among Europeans, since

this country was very far behind the other European Telcos in the 80's and today

BT is at the same level as the other European Telcos.

To emphasize the differences in the uses of telecommunications we have

constructed hereafter a last figure : the horizontal axis indicates the use of the

network and the vertical axis the cost of the access to the network (with adapted

scales in the figure below). Clearly we have two groups of situations : a group

with a very important use and a very low cost of the access (it is the group of the

BOCs), and the group 2 with less uses and a cost which is very high and variable

(from 20 to 80 and even 120 -for Switzerland-). Note that we obtain the same


type of figure with the number of calls or the number of minutes of

communications on the horizontal axis. The figure above corresponds to the year

1993 for which many data have been available, but for the more recent years,

1996-1997, similar results can be noticed: France telecom has a very high

turnover by line and a very weak number of minutes by line (high prices and low

access) ; the other European Telcos have approximately the same number of

minutes by line, but for Deutsche Telekom and British Telecom the turnover by

line is 3 to 8 times lower (low access but lower prices, comparatively to France

Telecom); the American operators have both a low turnover by line and a large

number of minutes by line (low prices and best access).

Figure 6: The use of the network (horizontal axis : number of calls

per line) and an indicator of price (vertical axis: turnover per line) 140

I 120 •

I ·-

- f-. ·-

100

80

• •j Group I . ~

60

• 40 •---I I • Groupe • • ~lA ' • • 20

0 I 0 2 3 4

Undoubtedly deregulation will lead the group 2 in the direction of the group 1. Even if

the link between deregulation and performance is not obvious to establish, we think

that by many ways we have throw light on it.

Some complementary results are provided by the financial analysis. In a deregulated

context such as in the United States, the performing firms both in productive and

financial terms absorb less performing operators. The situation of European operators

is contrasted : DT is in a course of a restructuring processs with relatively bad

performances, while FT has relatively good productive performances but a weak

development of services and not very good financial performances. BT and Telecom


Italia are in a better situation. BT was very far from the point of view of physical

performances in the mid-1980s but has improved these performances, and has obtained

good financial performances with a net increase of the Cash Flow to long-term debt

ratio.

Two issues are emerging :

firstly in a deregulated framework telecommunications operators clearly have

good productive performances, especially in the use of the network, in the

diffusion of new services; but the weakest firms increase their debts and finally

have to enter into mergers; so, in the USA where deregulation is already far in

advance, the recent process of concentration raises the question of a new

deregulation ;

secondly in a country such as France where the deregulation is only at an early

stage, France Telecom has long obtained relatively good productive

performances with high prices but has not impulsed strongly the use of the

network and new services and has not favored a quick development of the

information society (the more obvious example is at the present time the case of

Internet) ; moreover the situation of France Telecom is evolving with financial

performances inferior to those of British Telecom or Telecom Italia.

There is some correlation (but we are not able to show a causality) between market

organization productive performances (DEA method), and financial performances : the

efficient BOCs and BT have a high return on equity, with a low CAF/D ratio, and a

good diffusion of services (only for the BOCs) while operators in an early stage of

deregulation (France Telecom or Deutsche Telekom) have less good performances.

Finally, we have to keep in mind that the evolution in the telecommunications sector is

continuous. For example, taking into account the burst of the number of mobiles in

France during the last twelve months would probably lead to new analysis.

Nevertheless it remains that the European operators, especially France Telecom, offer

consumers a limited access to the network with high prices. In the context of the

emerging global information society (see the Bangemann report), in which the

diffusion of information gives rise to knowledge and competitivity, this situation has to

change. It is probably one of the most important challenges that the European

regulation organizations have to take up.


Annex: database

The database includes the telecommunications operators of the following countries (the abbreviations used in the paper are indicated in brackets)

GERMANY (ALL) or Deutsche Telekom AUSTRIA (AUT) or OPT

BELGIUM (BEL) or BelgacomDENMARK (DK) or Tele Danemark

SPAIN (ESP) or Telefonica FINLAND (FIN)

FRANCE (FR) or France telecom NETHERLANDS (NDL) or PTI Nederland

IRELAND (IRL) or Telecom Eireann PORTUGAL (PORT)

ITALY (ITA) or Telecom ItaliaPOLAND (POL)

JAPAN (JAP) or NTI UNITED KINGDOM (UK) or British Telecom

SWITZERLAND (SUI) or PTI SUISSE

For the DEA analyses I to 9 the data are those of the main operator of each country (for example, British Telecom for UK); for the DEA analyses 10 and II the data are those of the countries.

For the United States data on each of the seven Bell Operating Companies as well as on the total of the BOCs were collected :

AMERITECH (AM) PACIFIC TELESIS (PA)

BELL ATLANTIC (BA) SOUTHWESTERN BELL (SW)

BELL SOUTH (BS) US WEST(US)

NYNEX(NY)

The group of the seven BOCs has been symbolized by BOC.

References

Badillo, P.-Y. (1994): Les productivites des operateurs de telecommunications, Communication at the

XLth International Conference of the Applied Econometrics Association, Osaka, Japan, 24 and

25 March.

Badillo, P.-Y. (1997): Has deregulation an impact on productivity and performances? -Some

preliminary results for telecommunications-, Communication at the EURO CPR '97, 23-25

March 1997, Venice, published in Communications et Strategies, 1997.

Badillo, P.-Y. (1999): L'efficience des operateurs de telecommunications : une comparaison

intemationale, in: Badillo, P.-Y. and Paradi J.-C. (Eds.) (1999): La methode DEA, analyse des

performances, Paris, Hermes.


Badillo, P.-Y. and Paradi J.-C. (Eds.) (1999): La methode DEA, analyse des performances, Paris,

Hermes.

Baumol, W. J., Panzar, J. C. and Willig, R. D. (1982): Contestable Markets and The Theory of

Industry Structure, New york, Harcourt Brace Jovanovich.

Braeutigam, R. R. (1989): Optimal Policies for Natural monopolies, in: Schmalensee, R. and Willig,

R. (Eds), Handbook of Industrial Organization, Amsterdam, North-Holland, Volume II, pp.

1289-1346.

Braeutigam, R. R. and Panzar, J. C. (1993): Effects of the Change from Rate-of-Return to Price-Cap

Regulation, American Economic Review, mai 1993,83 (2), pp. 191-198.

Charnes, A., Cooper, W. W., Lewin, A. Y. and Seiford, L. M. (Eds.) (1994): Data Envelopment

Analysis : Theory, Methodology and Applications, Boston, Kluwer Academic Publishers.

Gnltches, Z. (1987): Productivity : measurement problem, in: Eatwell, J., Milgate, M., Newman, P.

(Eds.), The New Palgrave Dictionary of Economics, London, Macmillan Press, pp. 1010-1013.

Joskow, P. L. and Rose, N. L. (1989): The Effects of Economic Regulation, in: Schmalensee, R. and

Willtg, R. D. (Eds.) (1989): Handbook of Industrial Organization, Volume II, Elsevier Science

Publishers, pp.1449-1506.

Kahn, A. E. (1971): The Economics of Regulation, Principles and Institutions, Second Printing 1989,

Cambridge, Mass, MIT Press.

Kndel, D., Sappington, D. and Weisman, D. (1996): The Effects of Incentive Regulation in the

Telecommunications Industry: A Survey, Journal of Regulatory Economics, 9 (3), pp. 269-306.

Lafont, J. J. and Tirole, J. (1993): The Theory of Incentives in Procurement and Regulation,

Cambndge, Mass, MIT Press.

Lafont, J. J. ( 1994): The New Economics of Regulation Ten Years After, Econometrica, 62 (3), May,

pp. 507-537.

Leibenstein, H. (1966): Allocative Efficiency vs. X-Inefficiency, American Economic Review, 56 (6),

June, pp.392-415.

Lovell, K. C. A. (1993): Production Frontiers and Productive Efficiency, in: Lovell et al. (Eds.)

(1993): The measurement of Productive Efficiency, New York Oxford, Oxford University

Press, pp.3-67.

Noll, R. G. (1989): Economic Perspectives on the Politics of Regulation, in: Schmalensee, R. and

Willig, R. D. (Eds.) (1989): Handbook of Industrial Organization, Volume II, Elsevier Science

Publishers, pp.l253-1287.


Schaffnit, C., Rosen, D. and Paradi, J. C. (1995): Best Practice Analysis of Bank Branches : An

Application ofDEA in a large Canadian Bank, Working Paper.

Seiford, L. M. (1996): Data Envelopment Analysis : The Evolution of the State of the Art (1978-

1995), Journal of Productivity Analysis, Vol. 7 n° 2/3, July 1996, pp. 99-137.

Sueyoshi, T. (1994): Stochastic frontier production analysis: Measuring performance of public

telecommunications in 24 OECD countries, European Journal of Operational Research, 74,

1994, pp. 466-478.

Viscusi, W. K., Vernon, J. M. and Harrington, J. E. Jr (1995): Economics of Regulation and Antitrust,

Cambridge, Mass., The MIT Press, Second Edition.

Date post:	23-Dec-2016
Category:	Documents
Upload:	georg
View:	234 times
Download:	6 times

Data Envelopment Analysis in the Service Sector ||

Documents