On January 26, 2010 the Grattan Institute released a report on measuring school performance. The main recommendation of the report is to replace measurement of average school performance with so-called value-added indices. The idea is very simple – to measure student progress as the primary outcome – and by employing an appropriate statistical model to extract that component of the improvement which can be attributed to the school.

The report was particularly well timed. The federal government launched the My school website on January 28 which publishes average student performance by school, reported within groups of “similar schools”. Moreover, results for the 2010 National Assessment Program – Literacy and Numeracy (Naplan) are to be published in May 2010 and will, for the first time in Australia, mean that value-added measures can be calculated.

There is much to like about the report and it is difficult to argue against it main conclusion – that measuring student outcomes at one time point and averaging over each school does not provide a valid measure of school performance. Obviously this depends on what one means by a school’s performance. If one means the ability of a school to attract and retain smart students while deterring less smart students then the average student outcome is probably the only meaningful measure. But if by school performance you have in mind the effect of the school on the student’s learning outcome, then school averages will be hopelessly biased. The reason is that students are not randomly allocated to schools. Rather, gifted students tend to concentrate in some schools while disadvantaged students concentrate in others.

MySchool tries to correct for this by measuring the level of disadvantage of the school. They use a single measure of disadvantage (ICSEA) largely based on the student’s ABS census district (not the post code as claimed in the Grattan report). This is then averaged over the school. So ICSEA really measures the disadvantage of the geographical catchment region of the school – not of the particular students who happen to attend the school. According to the government fact sheet:
The Index of Community Socio-Educational Advantage (ICSEA) is a special measure that enables meaningful and fair comparisons to be made across schools.
This “fact” is wrong in several ways. First, using a student’s community status as a proxy for the student’s status causes bias because the actual socio-economic differences between schools are diluted by the crudeness – a bit like regression to the mean. Second, even if the advantage of each student was correctly measured (rather than by census district), to use this data correctly it must be linked directly to the student’s score, not averaged over the school. Thirdly, the most pertinent and easily measured index of the student’s aptitude is surely their result on the previous Naplan test. This is now available and to not use it is both scientifically invalid and wasteful of the expensive data already collected.

There are some interesting features of the Australian educational landscape that are pertinent. Based on an international standard test (PISA) the variation in scientific literacy outcomes in Australia is larger than the OECD average (by 11%). But this higher inequality is not explained by the popular notions of disadvantaged schools failing to compete with elite schools In fact, 81% of the variation in outcome is within schools. This suggests that measuring individual student outcomes over time will be useful in targeting problem students as well as assessing intervention programs.

Naplan tests are administered at years 3, 5, 7 and 9 every second year. With publication of the 2010 results in May, there will no longer be a practical excuse to not consider more meaningful measures, assuming that Naplan can match the students in their 2008 and 2010 exercises. To not use the student level identifiers (the main value of which is to track student level changes) is statistically indefensible. No trained statistician would publish the school level averages when student identifiers were available.

An appropriate statistical model would try to deconstruct each student’s outcomes into different components. For instance one might say that the 2010 year 9 result depends primarily on (1) that student’s 2008 year 7 result, (b) the students ability to learn, (c) the student’s non-school circumstances and (4) the school environment. It is the last of these components that we think of as measuring school performance. The role of the statistical model is to measure this adjusting for the first three components which are out of the school’s control. The non-school circumstances might include things like parents’ education, occupation, marital and employment status, family size, parity, gender, ethnicity, migration status and language preference. In the Australian context, MySchool already measures socio-economic disadvantage of the student based on their postcode.

There are some misleading statements in the report but nothing that undermines their main contention. For instance, at various points it is claimed that naïve school averages have “consistently been shown to produce biased estimates of school performance compared to value added modeling.” As pointed out above, it is really a matter of how one defines school performance and the matter is not resolved by a mathematical or empirical study. Nevertheless, most reasonable people would agree with the author’s view that averages school performance is not a good measure.

At another point, the report notes that overseas experience shows that the volatility of value-added measures is greater for smaller schools. They then argue that small school results should not hold implications for those schools. This is surely incorrect since each school’s value-added measure will come with a measure of statistical significance which takes into account the fact that measures for smaller schools are less reliable. Even for the smallest school, if results were sufficiently bad then some action would be indicated.

The report is (probably deliberately) vague about exactly how the value-added measures are to be calculated. The measure will depend on a statistical model which will require some expert statistical modeling. They do mention however that academic research has shown that the final measures are quite insensitive to the exact details of the mode employed. Nevertheless, it would have been nice to have an indication of the kind of statistical model that one might employ, even if relegated to an appendix. For those of a more mathematical background, one might start with a fixed effects regression model such as

y_it = βx_it + α_ty_i,t-1+q_j 1_{stud I in school j} + ε_it

where y is the Naplan result, “i” is the index of the individual student, t is the time, j is the school and x are the individual level covariates of non-school environment. A better alternative would be a so-called “random effects” model the historical achievement term α_ty_i,t-1 is replaced by a more complicated but flexible term.

This report is timely and persuasive. While those opposed to school performance measurement will find reasons to question the validity even of value-added measures, the reality is that performance data will continue to be generally available and we would all be better served if the current school averages were replaced by something imperfect but better.

Download Report here: Measuring What Matters: Student Progress by Dr. Ben Jensen

News Item: Alternative proposed to schools league tables

In March, 2009, the Boston Consulting Group released a report titled “The Innovation Imperative in Manufacturing“. BCG co-wrote the report with the National Association of Manufacturers and the Manufacturing Institute of the US.)  The goal was to assess the level of innovation among US firms in the area of manufacturing, both across the different states of the US as well as in comparison to other countries. The strength of this report is its rich dataset, which combines a survey of over 1000 companies with hour-long interviews with 30 senior executives.

The portion of the report that compares across different states of the US is quite interesting. Unsurprisingly, it shows states like California, Connecticut, Massachusetts and New York leading in both innovation inputs and innovation performance, while other states like Alaska, Florida and Maine lag on both dimensions (see page 13 of the report). There are also states with high levels of innovation inputs but low innovation performance, and vice-versa. The most useful parts of the report are Section 4 (What Drives Innovation Success?) and Section 5 (The Role of Government). Both these sections contain interesting snippets from interviews with executives, presenting a clear description of what they view as the issues to be addressed by firms and governments.

In contrast to the within-US comparison, the international comparison raises a greater number of concerns. At the heart of the analysis is a newly devised ranking system which attempts to measure the “innovation friendliness” of 110 countries and the 50 US states. Based on the rankings, the authors conclude that the US is “falling short in its commitment to innovation and in its innovation performance”. The US appears at number eight in these rankings, ahead of Japan, Canada and the European countries, but behind Singapore (#1), South Korea (#2), Switzerland (#3), Iceland (#4), Ireland (#5), Hong Kong (#6), and Finland (#7).

There are two main issues with this ranking system, and they have to do with measurement. A good measure should have high reliability as well as construct validity (see Judd et al., Research Methods in Social Relations, Chapter 3. ISBN 978-0030311499). Reliability refers to the extent to which a measure is free from error, such as when it is measured again and independently produces the same result, or when multiple questions are asked about the same item being measured and yield highly correlated responses. Validity concerns whether the measure really captures what is claimed to be measured. For example, if a group of students were told to take a mathematics test in a foreign language they lacked fluency in, the test scores may not be valid measures of the students’ mathematical abilities.

Herein lies the problem with the BCG report: exactly how “innovation friendliness” was measured is not clearly described. As such it is difficult to ascertain whether the measure is reliable and/or valid, and it is upon this measure that their key conclusions are drawn. It would have been more reassuring to the reader had they included an Appendix describing in a little detail how “innovation friendliness” was measured. What were the questions asked in the surveys and interviews? How were the many different dimensions of innovation (Exhibit 1 on page 9 of the report) combined into a single-dimensional scale of “friendliness”? How were the results of these individual-level surveys aggregated to form a country-level index? Aggregation issues might explain why the top positions in the ranking are dominated by small countries, including Singapore, Switzerland, Iceland, Ireland and Hong Kong. Is it reasonable to compare a tiny city-state like Singapore to a country like the US? Perhaps it might be more useful to compare manufacturing innovation in Singapore to that within a particular metropolitan area (e.g., the San Francisco Bay Area), rather than to the whole of the United States.

The issues of reliability and validity are especially important because this report is based on interviews and survey data. It would be difficult for an independent third party to validate those measures, unlike in reports that utilize publicly available data. An example of an index based on public data is the IPRIA Innovation Index, which uses data on patents from the US Patent Office, R&D expenditure from the OECD, and trade data from the World Bank (http://www.ipria.org/publications/reports.html). We do not imply that the IPRIA report is better; every approach has tradeoffs, and while the IPRIA report may exhibit greater transparency, the BCG report offers analysis based on a richer set of data than would be available publicly.

Another issue concerning the BCG report is that it does not explore in sufficient depth the observed differences in “innovation friendliness” across country. Firms, especially multinational ones, choose to locate different manufacturing activities in different geographic areas due to a variety of reasons. Hence, one would expect a wide range responses within each country depending upon firm-level needs, the competitive environment a firm operates in, intellectual property protection, and the type of innovative activity it engages in. It is unclear from the report whether the differences across countries are of a significant magnitude. For example, how significant is the difference between 1.80 as scored by the US versus 1.88 scored by Ireland (data from pg 25 of the report)? Is that difference of 0.08 really enough to say that the US is much worse off than Ireland? Also, not every country strives to be “innovation friendly” to the same degree. They face different tradeoffs, being at different stages of economic development and with manufacturing playing different roles within their economies. Is greater “innovation friendliness” necessarily better? Countries that want to be more highly ranked would have to incur a cost of doing so (e.g., by increasing innovation-friendly tax credits). Not all countries may benefit in the same way (if at all) from a higher ranking. In the example above, if the US spends a hypothetical sum of $1 billion to improve its score from 1.80 to 1.88, it would match Ireland in the rankings, but would it recoup enough benefit to justify the added expenditure?

In conclusion, the BCG report contains a valuable description of what executives think, while raising many interesting questions about the appropriate level of “innovation friendliness” for each country.

 

Original Report: The Innovation Imperative in Manufacturing: How the United States Can Restore its Edge

This report was commissioned by the Menzies Research Centre. The goal of the report is as a broad review of the proposed Carbon Pollution Reduction Scheme (CPRS). It pays particular attention to the White Paper on the CPRS as well as the associated Treasury modelling. However, it does not conduct modelling of its own. The broad conclusion reached is that there is more value to providing additional modelling of the impact of the CPRS in particular in relation to adjustment and transitional costs. They argue that a mistake in terms of the assessment of the magnitude of these costs could derail, politically, the CPRS at some future point. The report argues for an independent regulatory impact review, say by the Productivity Commission. This would evaluate the form of the proposed CPRS as well as other factors.

On its face, the report brings together a number of criticisms of the government’s proposed scheme but does not evaluate their substance using new evidence. For the most part this is useful but there is a tendency to concentrate on the obvious and not explore issues in more detail:

“While R&D spending on new technologies is a very important component of any response to climate change, there are clearly bounds on how much it is sensible to spend on that R&D. All R&D spending has an opportunity cost — the same funds could have been spent elsewhere and the resources used in the R&D (talented researchers, for example) could have been deployed elsewhere. Without a carbon price signal, it is extremely difficult to make judgements about the appropriate amount of R&D spending.” (p.26)

This statement is not wrong but there is more that surely could have been said about this issue; especially what it means for the reports claims regarding the potential usefulness of delay. If this delays a carbon price, the implication here is that it will delay efficient decision making, by both private and public agents, on R&D issues.

Similarly, the report discusses the potential impact on “balance sheets” especially firm valuation. It claims that “[t]his reduction in balance sheet values is likely to make investment very difficult, particularly as funding can often revolve around balance sheet asset values.” (p.29) It wonders if this achieves any objective. What a strange thing to wonder as the whole point of establishing a carbon price is to signal its scarcity on balance sheets and change where we invest. This looseness diminishes the report’s usefulness.

This lack of exploration becomes important when considering one of the report’s main claims that there is insufficient modelling. This centres around adjustment costs at a macroeconomic level something the Treasury modelling did not consider. However, the G-Cubed model did consider this and provided an estimate of such costs. This is the CIE’s evidence that those costs are important but also their claim for more modelling. But if the G-Cubed model already has these estimates, why is more modelling required? That information is in the hands of the Government and, indeed, it is far from clear that it has not been taken into account — especially given the low emissions target set by the Government. This is something the CIE did not comment on at all but it would appear very relevant for the issue of adjustment costs and the CIE’s related claims about the importance of dynamic consistency.

In summary, as a review there is nothing new in this report. And as an instrument to claim for more modelling and a delay to the introduction of the CPRS, it does not make that case fully; especially, considering the need to conform to international timetables and the benefits that might come from providing investors with clearer price signals.

Original Report: Review of the proposed CPRS

News Item:  ETS a ‘black hole of uncertainty’

Report by KPMG Econtech for Universities Australia

In modern economic research, we have clever techniques and grand questions. Alas, the clever techniques typically don’t work on the grand questions. Consequently, our best journals are filled with precise answers to not-so-interesting questions, while lesser-ranked journals (and books) proffer rough-and-ready answers to the big questions.

So there is something bittersweet about reading a report titled that aims to ‘measure the net economic benefits over time of government policies aimed at increasing university funding’ (p1). The quest is bolder than most academic economists would dare attempt to answer. But given the gaps in our knowledge, the reader knows that there will be some canyon-like leaps of faith between the introduction and the conclusion.

The benefits of boosting university funding, we are told by this report, will fall into five categories: increased productivity, increased labour force participation, more cash from international student fees, public returns from university research, and a bigger population. Let’s take them in turn.

Increased productivity: The report’s estimate of the impact on productivity is the part of the report that I find hardest to critique. This is due to the authors’ cunning propensity to refer to ‘the Leigh wage premium’ at regular intervals (this actually gives me more credit than I deserve, since my numbers are estimated from a bog-standard Mincer model on HILDA data). My best guess is that a university degree raises annual earnings by 45-50%, which the report scales down to 40% to be conservative. Of course, it is possible that the marginal university student would have a higher or lower rate of return, but the quasi-experimental literature suggests that the Mincerian number is a useful starting point.

Increased labour force participation: At risk of sounding like a spurned lover, I wasn’t sure why the report used my estimates of the impact of education on productivity, but the ABS’s estimates of the impact on participation. My estimate is that those with a bachelor degree are 10 percentage points more likely to be working (ie. to have positive earnings) than those with just year 12. By contrast, the ABS numbers (used in the report) are 16 percentage points for men, and 23 percentage points for women (p40). The disparity is most likely due to the fact that the ABS report does not make any adjustment for age. Since older cohorts are less educated and less likely to be working, this approach probably over-estimates the causal impact of university attendance on workforce participation.

More cash from international student fees? Well, perhaps. But it is a little odd to read at one point that universities have turned to international student fees to make up for a shortfall in Commonwealth cash. If so, wouldn’t it be reasonable to think that a new injection of government funding will reduce the income that they garner from international students?

Public returns from university research: Reviewing 21 studies, the report concludes that a ‘conservative estimate’ of the rate of return to publicly funded research is 20%. But when your rate of return is nearly twice as high as Bernard Madoff’s, it’s worth pausing to check the numbers. It turns out that the studies on which the report relies are generally based on scientific research, such as on tomato harvesting or cardiovascular disease. The problem with extrapolating these studies to all university research is that they are not necessarily characteristic of what the typical university faculty does. Raising GDP may be a useful by-product of scientific research, but intellectual exploration in the humanities and social sciences is rarely aimed at boosting national income. For example, would we really expect an Australian Research Council on ‘Economic Inequality: Trends, Causes and Consequences’ to increase GDP? If anything, such a project might well produce proposals that reduce national income. I expect that the same goes for much of the research currently being undertaken in Australian universities.

A bigger population: The last channel through which more university funding can boost GDP is via its impact on increasing Australia’s population. Since some international students become permanent residents, an expansion of the government sector means a bigger population. But is that a gain in social welfare? If all that mattered was a nation’s GDP, Americans would be 18 times better off than Australians. Perhaps our nation would be a smidgin better off if we had more people, but as a first pass, per capita GDP is a better metric of wellbeing than total GDP.

Although the report probably overstates the public returns to an increase in university funding, I enjoyed reading it nonetheless. The analysis is intellectually curious, and peppered with interesting findings. For example, did you know that the completion rates for undergraduates and PhD students are just 65% and 54% respectively? Or that staff-student ratios in universities rose from 15.6 in 1996 to 20.5 in 2006? If this report were a piece of university research, I wouldn’t expect it to boost GDP, but I would appreciate its contribution to a complex debate.

Original Report: KPMG Report commissioned by Universities Australia
News Item: Boost funding to reap rewards: Universities Australia study

Related Commentary from Andrew Norton

In this report, Access Economics, AE, (commissioned by Telstra and quoted by it at the Senate inquiry on the National Broadband Network) have provided perhaps the first comprehensive attempt to value investments in a national high-speed broadband network in Australia. Previous estimates of the value of high-speed broadband infrastructure to Australia have argued that the contribution to the economy may be between $12 billion to $30 billion per annum. However, these were based on a scaled down version of US studies for basic broadband.  AE have explicitly considered the value of high-speed broadband (of download speeds beyond 12Mbps) above and beyond the now widespread availability of basic broadband services and, as a result, have downgraded the potential economic benefits.

AE find that a national ʻcarrier-gradeʼ network to 90 percent of the population based on a fibre to the node infrastructure and constructed between 2009 and 2016 would generate a net present value of economic benefits to Australia of $9.5 billion. They also explore other investment scenarios including a roll-out to less dense areas first and various delayed options. The report estimates, for example, that a two year delay of the network will cost the Australian economy $3.2 billion over the period 2009-2010. This is an important figure given Telstraʼs claims of what delayed investment in the National Broadband Network is costing Australia.

The estimates are generated through a computable general equilibrium model of the Australian economy. The primary input into that model is an estimate of the impact on multi-factor productivity from high-speed broadband deployment. The greater the coverage of that deployment the higher the productivity impact. AE provide a comprehensive review of studies elsewhere of the productivity impacts of broadband (both basic and high-speed) as well as the impact of ICT in general. They note that:

The literature surveyed above provides an imprecise sense of the scale of    the productivitybenefits that may flow from the rollout of HSBB. Unfortunately, at this point, there are insufficient data to estimate the precise impact and hence any estimate will be subject to a good deal of uncertainty. (p.20)

It is then assumed “that economy-wide multifactor productivity levels would be around 1.1 per cent higher in an Australian economy with HSBB available everywhere relative to an Australian economy without any HSBB after ten years.” (p.20) AE view this number (amounting to a boost to productivity growth of 0.1 of a percentage point per annum) as being “conservative” although the sectoral impacts are adjusted with health benefitting more as a
result of video conferencing while financial services benefits less as services are available at current speeds. The precise sectoral benefits are not stated. That said, video conferencing is available at current speeds (and between hospitals and businesses) and so it is not clear whether such adjustments are appropriate.

The entire analysis relies on these assumptions regarding the productivity impact of high-speed broadband. Given its importance, a thorough analysis would have tested the model for different assumptions including faster or slower productivity impacts, more or less homogeneous sectoral impacts and also assumptions that take into account the current availability of high-speed broadband to many businesses throughout the country — making the
likely impact more consumer-sourced than commercially driven.

Critical to the report are the results on the impact of a delayed investment in broadband. To compare net present values of alternative options requires an assumption of the social discount rate. Without reference or discussion, AE assume a discount rate of 7 percent (p.31). This is higher than rates usually assumed. For instance, the Garnaut Climate Change Review used a social discount rate of 4 percent. A high discount rate tends to favour investments that pay off sooner rather than later. This raises the estimated costs of delay and explains why those costs were so high in AEʼs report. Like the assumed productivity rate, the report should have tested alternative discount rate assumptions. That, combined with no allowance for any option value associated with delay that Telstra often argues is critical in telecommunication investment analysis, means that the costs of delay are likely to be significantly over-stated by this report.

Original Report: Impacts of a national high-speed broadband network

News Item: Telstra cagey over broadband research

Fujitsu Consulting (FC) recently released their Australian Bank Fee Survey 2009. The resultant publicity, typified by the Daily Telegraph, has tended to focus on the headlines of Australian households paying close to $1000 per year in bank fees, paying $200 more than they “should” and paying 22% on average more than British households and 11% more than American households. Unusually for the topic of bank fees this note will not comment on whether they are too high, too low or perhaps too opaque. Instead this note will focus on the lack of substantiation behind the FC report’s primary quantitative findings. As headlines the claims are alluring. The aim of this piece is to explain why the headlines should be treated with caution.

The FC findings are based on a benchmarking of bank fees for Australia, USA and UK. The results highlighted appear to be FC projections for 2009. The total fees calculated by FC assume that all households use the full range of products and services, with “typical” transaction patterns for their particular customer segment, as indicated by FC surveys. This means that every household in a customer segment is assumed to have the transactions and uses of those surveyed and to pay fees for a full range of banking services: mortgage, credit card, transaction account, personal loan and other banking services.

These may appear to be innocuous assumptions. They can be tested by examining the total bank fees actually paid divided by the total number of households. Here a quite different picture emerges for Australia. Looking at the latest full year statistics, the Reserve Bank of Australia reports Australians spending $4376M on bank fees in 2007.[1] The Australian Bureau of Statistics projects around 8.2M households in 2007.[2] This gives an average $534 paid per household in banking fees in 2007. This is significantly less than the “close to $1000” reported by FC, based on their assumption of households being segmented and using and paying for all bank services, at transaction levels and product mixes determined by surveys.

Now it is common to try to hold some factors constant in international benchmarking comparisons. While details are scant it appears that FC has tried to adjust US and UK fees to reflect what they would be if the US and UK had Australian transaction levels and product mixes for different customer segments. So we might not expect the FC reported fees to equal or even approximate the actual fees in the US and UK.

However if the Australian data has been used as the baseline for the benchmark, it is reasonable to ask, does the benchmark at least approximate the actual historical values for Australia in some way? Unfortunately it does not. The benchmark relied on for FC’s analysis is almost double the actual observed average fee levels. FC’s own reported graph on page 2 has fees estimated to rise less than 10% per year in 2008 and 2009, not the near doubling implied by their benchmarking process. What is going on here?

There are two obvious potential sources of error in the FC benchmarking approach. The most prominent potential source of error is the primary assumption made by FC – that all Australian households use a full range of products and services. If in fact a number of households only use a portion of available services then this would tend to overstate bank fees.

A second potential source of error is the use of customer surveys to determine typical transaction and product bundles. In fact one way to check the survey validity would be to see if an extrapolation of the results of the surveys to the nation as a whole would approximate the average total fee levels actually observed. As noted here they do not appear close.

Less obviously the treatment of conditional fees could explain the difference between the FC benchmark and the realised historical numbers. Observed historical fees inherently take into account the proportion of conditional fees that are ultimately paid. It is not clear what assumptions are made by FC in allowing for conditional fees.

It appears the FC benchmarking approach does not answer any question about the total level of fees actually paid by Australian households. Instead it appears directed at a different and more limited question – if everyone used all banking products and they had the transaction and product bundles equivalent to those obtained from some survey data then what would average fee levels be. This question has limited usefulness when it calculates an Australian fee level so starkly different from the observed average.

The significant difference between the FC results and observed results encapsulates the major criticism of this report – its use of a highly hypothetical fee to try to substantiate claims about the level of bank fees paid on average by households. Together with apparently nonsensical claims for UK regulation tying fees to costs leading to a drop in bank overdraft fees by 220% (such a drop would imply the banks now pay customers to take out the letters) this approach undermines the quantitative foundation of the arguments made – and means the headlines should be treated with extreme caution.

[[1]] Reserve Bank of Australia, table F06 Domestic banking fee income http://www.rba.gov.au/Statistics/AlphaListing/alpha_listing_b.html

[[2]] Australian Bureau of Statistics, 3236.0 Household and family projections, Australia, 2001 to 2026, p62, median series value used. http://www.ausstats.abs.gov.au/ausstats/subscriber.nsf/0/DF2989BFFA7392E1CA256EB6007D63F4/$File/32360_2001%20to%202026.pdf

Download report here (follow the links): Australian Bank Fee Survey 2009

News item regarding report: Australian Bank Fees’ 22% higher than UK

---

FOOTNOTES
1. ↑
2. ↑

Changes to Australia’s cartel laws, such as the introduction of criminal penalties for price fixing, are currently in the pipeline. As such, a report by the IPA into cartels and antitrust action against cartels is timely. Unfortunately, the report tends to be one-sided. It does not present a balanced survey of the economic literature. While it provides a potentially useful case-study of the Australian cardboard box market, the report fails to properly empirically test the source of price changes.
The body of the report begins with a brief, but confused, summary of the relevant economic literature. It does not consider the differences between legal, tacit collusion and illegal, explicit collusion. It also appears to confuse different cost concepts. However, the bottom-line of the summary is correct: cartels are likely to break down over time due to changes in market conditions “such as reductions in demand or the entry of other firms” (p.4).
This statement is relatively uncontroversial, but raises two key economic questions: how long do cartels last and why do they break down? The first of these questions is important for antitrust enforcement. If cartels are highly unstable and break down rapidly, then vigorous enforcement of cartel laws may have costs that outweigh the benefits.
To address this question, the report discusses six case studies. For three of these (diamonds, coffee and mercury) cartels appear to have lasted considerable lengths of time: 74 years, 27 years and 11 years respectively. So cartels do break down – in the long run!
A case study of liner shipping is used to show the difficulties of maintaining a cartel. The final two examples (South African cement and US electrical equipment) are used to suggest incorrect or harmful prosecution.
The report concludes this section by referring to academic meta-analyses of cartels. The most recent and comprehensive of these is Levenstein and Suslow (2002). However, the report only refers to the shortest cartels ranging in duration from one year to 3.7 years. The full Levenstein and Suslow results (updated in 2006) look at 72 cartels. They show that 13 cartels lasted three years or less, most lasted four to six years, but there was a ‘long tail’ with fifteen cartels lasting more than a decade. Levenstein and Suslow also compare their results with five previous studies, all of which give an average cartel duration of greater than five years.[1]
It is, of course, a subjective judgment as to how long is ‘too long’ for consumers to put up with a cartel. In my opinion, even three years sounds like a long time to suffer artificially raised prices. But how successful are cartels at raising prices?
The IPA report addresses this issue by referring to two U.S. examples where cartel prosecutions did not appear to lower prices. If cartels raise prices then prosecution that ends the cartel should lower the prices.
The report does not, however, refer to meta-analyses of the price effects of cartels. O’Connor and Lande, in a series of recent papers, both survey existing price studies of cartels (including 82 refereed journal articles) and carry out their own empirical research. They conclude that many cartels significantly raise prices – on average by about 25%. Further international cartels are more effective at raising prices than purely domestic cartels.[2]
Finally, the IPA report refers to two recent Australian cartel cases involving cardboard boxes and petrol.[3] As already noted, a full empirical analysis of these cases would be useful to see if consumers were overcharged and, if so, by how much. However, the report only presents a price graph for three paper products and attempts to infer a conclusion from this without proper empirical analysis.
The report also briefly refers to the public choice literature on antitrust authorities and concludes that the profit motive will always work to undermine cartels, and that this is better than having a “vast and intrusive bureaucracy” (p.16). Unfortunately the report does not carry out a cost-benefit analysis on any antitrust authority to try and support this claim.
Finally, the Executive Summary of the report makes a series of claims that are not supported by the analysis in the report itself. For example, the Executive Summary claims that “[g]overnment intervention against cartels is seldom effective …”. This claim is simply not supported by the analysis in the report. It also claims that “[c]ost savings are often a goal of cartel participants and to the extent that these are achieved they may result in lower prices and better service”. While the first part of this claim is trivial, the second is not supported by the analysis in the report and indeed flies in the face of the empirical studies on price rises under cartels referred to above. I suggest that the reader skip the Executive Summary unless they are after a controversial headline.

---

[1] M. Levenstein and V. Suslow (2006) “Determinants of International Cartel Duration and the Role of Cartel Organization” Ross School of Business Working Paper Series No. 1052, October, University of Michigan. The IPA report refers to an earlier 2002 University of Michigan working paper by these authors.
[2] See J. Connor and R. Lande (2007) “Cartel overcharges and optimal cartel fines”, working paper, http://ssrn.com/abstract=1029755. Also J. Connor and R. Lande (2005), “How high do cartels raise prices: implications for optimal cartel fines”, Tulane Law Review, 80, 513., and J. Connor and R. Lande (2006) “The size of cartel overcharges: Implications for U.S. and EU Fining policy”, Antitrust Bulletin, 51, 983.
[3] The author of this note was a Member of the Australian Competition and Consumer Commission during (at least part of) the prosecution of these cases.

Original report: The adverse effects of government actions against cartels

News item: Tougher Penalties for price fixers warranted

The McKinsey Global Institute (“MGI”) recently issued a thoughtful and provocative report on climate change and economic growth. Their stepping-off point is that if the world is to meet the twin goals of reducing the amount of carbon released into the atmosphere, and also maintaining economic growth, then the world must use carbon much more efficiently than it currently does. They call this imperative “the carbon revolution.” By their estimates, it requires a ten-fold increase in carbon productivity by 2050.

This is a great way to frame the challenge of addressing climate change and economic growth simultaneously. One cannot argue with the notion of carbon productivity because it is an identity: GDP per unit of carbon emissions. And the ten-fold increase the MGI argue is essential is both daunting and sobering. But what’s the right number? This depends on the assumptions which are made. One cannot question the logic of carbon productivity, but the required increase depends delicately on the assumptions. The main failing of this intriguing report is that too advocate-like and not scholarly enough. But done correctly, it frames the problem just right. The study does not just suggest how large the productivity increase needs to be — 5.6 percent per annum until 2050 — but addresses how to achieve this, and how much it will cost. And herein lie some of the issues with the analysis.

Carbon productivity is GDP divided by emissions. So any estimate of required carbon productivity naturally depends on both GDP and emissions. The amount of emission allowed is what is at issue, so we need to know worldwide GDP in 2050. MGI assume that the recent rate of economic growth worldwide — 3.1 percent per annum — will continue until 2050. That was a bold assumption before the recent financial crisis. But set aside recent events for the sake of argument.

As economies become richer they tend to grow less fast — a phenomenon known as “growth convergence” (due to Robert Solow and Trevor Swan). Will China and India really still be growing at 8-10 percent per annum in 2050? Maybe, but probably not. Perhaps other currently poor countries will grow faster and help compensate. But given the size of China and India there would need to be an absolute economic revolution in sub-Saharan Africa and other places as well. So the assumption which MGI make tends to overstate the challenge of carbon productivity growth. That doesnʼt really diminish how startling the challenge is–maybe it’s 4.5 percent per annum instead of 5.6. That’s still daunting.

How, then, can a 4+ percent annual carbon productivity growth be achieved. There is some good news here. According to the report, there are a number of opportunities which are actually profitable as of today. Things like: better home insulation, water heating, air conditioning and lighting. The report rightly points out that these might not happen because of various market failures (like homeowners being credit constrained and not being able to afford the upfront costs or agency problems in rented housing). These could be addressed with subsidies or loans, for example. MGI calculate that there are 7 billion gigatons of annual emissions like this. The report then focuses on the costly opportunities, like clean energy technologies, which make up the other 20 billion gigatons required to meet the 5.6 per annum carbon productivity increase.

The main issue here is that the costs of making these investments are emphasized, while the benefits from the already profitable opportunities are basically not factored in. This leaves the impression that the actual challenge is bigger than the data suggest. Even accounting for more realistic GDP growth in China and India leaves the challenge a 6.5-fold increase in carbon productivity by 2050. This means 17 billion gigatons of reductions — 7 of which are profitable. And those profits could be reinvested in chasing down the other 10 billion gigatons. That’s still a huge challenge, but not as large as the report makes out.

Original Report: The Case for Investing in Energy Productivity, McKinsey Global Institute, February 2008

News Item: Energy Saving Most Effective in CO2 Cutting – report

In September, 2008, the Australian Government released a report, Venturous Australia, containing a comprehensive review of the national innovation system (“The Review”). It was assembled by Dr Terry Cutler along with an 11-member expert panel, and also incorporates suggestions from over 600 public submissions. It calls for urgent action because over the past decade, Australia has fallen behind its peer countries to a dramatic degree in terms of its investments in education and R&D (Nelson, 1983). This is a source of concern because it is unclear how long the boom in natural resources will continue to fuel the economy; innovation is therefore seen as an important source of future growth.

The Review contains 72 recommendations covering a broad range of areas including: (a) reforming the educational sector, (b) encouraging the private sector to invest in innovation and to take a leading role in the commercialisation process, (c) increasing funding towards universities and research agencies, (d) incentives, fellowships and programs to increase the quality of research, (e) reforming the intellectual property system and improving the flow of ideas among universities, firms and other research agencies, (f) replacing the existing system of R&D tax concessions with tax credits, (g) R&D grants to improve linkages between research institutions and market-oriented firms, (h) attracting international knowledge flows and venture capital to Australia, (i) increasing the level of innovation within the Government, and (j) implementing new institutional structures to manage R&D nationally.

In short, the Review touches upon every aspect of Australia’s National Innovation System and proposes changes throughout. It also reports of a deep rift in perceptions towards innovation amongst the various players (government, academics and industry), although these are not surprising and also likely to exist in most other countries.

The concerns raised by The Review, as well as its recommendations, are in line with other similar efforts, such as by the Australian National Innovation Summit (1999) and the Annual Innovation Index (Gans and Stern, 2003; Gans and Hayes, 2007). Indeed, it is worrisome that Australia has not made much progress in building innovative capacity over the past decade, during which the earlier reports were publicly available.

Overall, The Review is extremely important in presenting an ambitious agenda for re-architecting Australia’s future. In this IdeaCheck, I analyze whether it is consistent with academic fundamentals, and whether it is a good blueprint for action.

Is the Review Consistent with Academic Fundamentals?

Academic researchers have reached a clear consensus that producing new knowledge is a highly uncertain process, and that commercializing new knowledge is risky and prone to market failures. Therefore a party investing in its early production may not reap the future returns. As such, individuals and private firms are likely to under-invest in innovative activities. Yet, numerous studies suggest a strong link between innovative activities and economic growth, as well as positive social and economic benefits of having a high level of education and innovation. This argument has been used as a motivating force (including in The Review) for an active role by government to invest in R&D as well as to help resolve market failures.

However, controversy surrounds the question of how much governments should invest in R&D (as opposed to other areas such as healthcare, welfare, etc.) and the optimal degree of public support for private R&D (Hall, 2002; Klette, Moen & Griliches 2000). These are important issues which I hope will continue to be the subject of energetic public debate in Australia. However, to the extent that Australians have elected a government that has chosen to invest in creating a robust national innovation system, the real question at hand is whether or not The Review is aligned with what has been learnt from academic research, and whether it offers to make good use of the money invested.

From this perspective, there is good news. The Review is in line with key insights from recent research on managing innovation. These include:

(1) National Innovation Systems are complex systems, and they need to be managed as coherent sets of activities involving multiple participants, not piecemeal. Partly because of this, complementary sets of activities are often agglomerated within geographic clusters, such as the electronics firms, venture capitalists and research universities in Silicon Valley.

(2) Well-developed institutions are important for nurturing a highly-skilled and innovative workforce.

(3) Highly skilled people are crucial in the process of knowledge creation, including “star” individuals who are highly productive and occupy central positions in the social networks that link them to other innovators.

(4) Firms are central to the commercialization process. They are increasingly relying upon relationships and alliances with other organizations for innovative activity, as well as being connected to a global network of scientists and innovators outside their firms for new ideas. Startup firms are an important part of the innovation ecosystem because they are often more willing than existing firms to explore new approaches.

(5) Innovation should not be viewed only to include scientific research, but more broadly to encompass new business practices, art, and other forms of human endeavor. Good ideas often come ‘knowledge brokering’ across different domains to create unexpected but valuable combinations (Hsu and Lim, 2006).

(6) Innovation is a cumulative process. Hence, a delicate balance must be managed between the intellectual property rights of inventors, customers and firms. Moreover, mechanisms have to be in place to allow for the effective flow of ideas among innovators, as well as for ideas to be transformed into commercial applications.

In my opinion, the Innovation Review does a great job of incorporating a strong understanding of these issues into its analyses and recommendations.

Is the Review a Good Blueprint for Action?

The Innovation Review is clearly written and provides a comprehensive set of recommendations. It addresses the major issues that are raised in similar documents produced by other countries. It also does an amazing job of integrating good ideas from the large number of public submissions (which in itself is quite an achievement). Moreover, The Review takes a broader perspective towards innovation than in many other countries, where the focus is exclusively on science and technology. So, it presents Australia with a compelling vision for creating a vibrant national innovation system.

However, as a plan for action, the Review does raise several concerns.

Firstly, there are too many recommendations (six dozen, in all!). The risk is that many of the recommendations will end up being left on the shelf. For example, the government may be tempted to focus more on tax credits, leaving aside some of the other recommendations that are more difficult to grapple with but equally important. Undoubtedly, the large number of recommendations is consistent with innovation being a complex issue, but The Review needed to have presented a clearer way to prioritize among these recommendations.

Secondly, The Review for the most part does not suggest tangible milestones, a timeframe for achieving goals, nor an appropriate person or organization to be responsible for implementing each recommendation. In fact the endpoints (which are specified on the final page of the overview section) are noticeably vague. While reading the Report, I felt it would have been much more helpful if the 5-7 most important recommendations were highlighted, along with intermediate milestones and a “champion” identified in each case to be responsible for its implementation.

Thirdly, while I have stated that The Review is ambitious, I wonder whether it is actually ambitious enough to achieve the stated goals. While Australia deliberates about how to implement this Review, the other countries are not standing still. Many of them have quite similar plans and a number of them are already aggressively implementing changes to their own innovation systems. Australia is trying to catch up to a moving target.

Finally, for the recommendations to have real impact, many important changes will have to occur at the micro level. These will require significant effort and time to implement. For example, in my teaching and research experience, I come into contact with numerous students, scientists and innovators. I have been surprised by the overall lack of a vibrant entrepreneurial culture. Many whom I’ve met are talented and inventive, but few have articulated to me an intense desire to form entrepreneurial firms to commercialize new ideas. Among those interested, many are ill-equipped to actually act upon their intentions because they lack the requisite skills, exposure to hands-on entrepreneurial activities, a support network of other entrepreneurs, and access to relevant sources of funding such as venture capitalists. This is a disquieting because The Review gives entrepreneurial firms and workplaces a central role in its vision of the future. Actually creating such entities will require big changes in mindsets and skill sets.

Conclusions

Overall, there is a lot to like in The Review. It sets an ambitious agenda for revamping Australia’s National Innovation System. It contains many interesting suggestions for improvement across a broad range of areas. However, a great deal of work is needed ahead to make Venturous Australia not just a dream, but a reality.

References

Gans, J.S. and R. Hayes (2008), Assessing Australia’s Innovative Capacity: 2007 Update. Gans, J.S. and S. Stern (2003), Assessing Australia’s Innovative Capacity in the 21st Century. Hall, B. (2002), “The Assessment: Technology Policy,” Oxford Review of Economic Policy, 18(1):1-8. Hsu, D. and K. Lim (2006), “The Antecedents and Innovation Consequences of Organizational Knowledge Brokering Capability,” Wharton and MBS/IPRIA Working Paper. Klette, T.J., J. Moen and Z. Griliches (2000), “Do subsidies to commercial R&D reduce market failures?”, Research Policy, Vol. 29: 471-495. Nelson, R.R. (editor), (1993) National Innovation Systems, Oxford University Press, 1993.

Original Report:Report on the Review of the National Innovation System, Venturous Australia – Building Strength in Innovation

News Item: Tax breaks may boost innovation: Review

Watson Wyatt’s report seeks to predict the most significant changes in the global pensions industry in the years between 2008 and 2020. A common approach to such prediction is to identify several important trends in recent years and then prophesise which trends will wax and which will wane in the ensuing years.

The authors take a more analytical approach of identifying the fundamental forces that are reshaping the industry and then predicting the changes that will flow from those forces.

I found the identified forces somewhat unconvincing. There are forces acting on the global pensions industry that are more fundamental than the forces identified by the authors. The connection between identified drivers of change and predictions of change in the report is weak because the fundamental drivers of change are not clearly identified and articulated.

The weak connection between forces and predictions is not helped by the methodology. The forces are identified by interviews, a survey of market participants and the authors’ understanding of the industry. But it is not clear how the interview, and survey data is related to the identification of forces. The report does not even state what the interview questions are.

Let me suggest four fundamental forces of pension industry change which encompass many of the forces identified in the report.

• Individualisation/de-corporatisation of pensions: Control of pensions is moving from corporations (DB plans) and Governments (Social Security) to individuals. Direct involvement of individuals in the control of their pension savings is supported by corporations seeking less cost and risk from pensions, households seeking pension mobility and empowerment, and governments wanting households to take responsibility for their retirement income.The consequences of this force go well beyond the rise of defined contribution over defined benefit plans. Reform of Social Security in the US and pension mobility in Australia are expressions of this force. Decorporatisation and individualisation changes the nature of distribution and advice provision of products and increases demand for customised products, especially individually managed accounts. Governance structures are fundamentally different in a system centered on individuals rather than corporate or government bodies.
• Aging of populations: Governments across the globe see the pension industry change through the lens of population aging. Government policy in changes to social security, tax law, regulation and governance seeks improvements in the demographic problems of nations.
• Globalisation: Ongoing integration of the global economy effects critical elements of the pension industry. Globalisation drives the war for talent, with talent following to locations where it is most highly compensated. It also changes the ownership structure of the industry as large investment management firms are integrated into global banks and diversified financial services firms.
• Increased change and volatility in financial markets: The three decades of great moderation in the markets and economies of OECD countries may have ended permanently. A return to the trends of history is a return to greater market volatility. Moderation in market conditions favours passive management. Volatility and structural change favour high alpha—high fee models of hedge funds and private equity.

Original Report: Defining moments – new research exploring the future investment landscape report by Watson Wyatt

News Item: Index fixation