Press Notice (A) of Session 2003-04, dated 11 March 2004
Memorandum by the Comptroller and Auditor General
ESTIMATING THE LEVEL OF SPIRITS FRAUD
Mr Edward Leigh MP, Chairman of the Committee of Public Accounts, said today that in the context of the debate about tax stamps on bottled spirits, the significant uncertainties underlying the respective estimates of spirits fraud produced by Customs and the Scotch Whisky Association show that further work must be done in this area to produce a solid estimate on which all parties can rely.
Mr Leigh was speaking as the Committee published a Memorandum presented to it by Sir John Bourn, Comptroller and Auditor General and head of the National Audit Office. At its hearing on 26 January 2004 the Committee considered the Comptroller and Auditor General's 2002-03 Standard Report on HM Customs and Excise. The Committee was concerned at the significant disparities between estimates of the level of revenue losses from spirits-related excise duty fraud (spirits fraud) produced by Customs of some £600 million for 2001-02 and by the Scotch Whisky Association (representing the UK alcohol trade through the Joint Alcohol and Tobacco Consultation Group) of some £100 million to £150 million for 2001-02. Customs' estimates of a growing level of spirits fraud form part of the basis for the Government's proposal in the Pre Budget Report 2003 to introduce "tax" stamps on bottled spirits from 2006-07. The Committee therefore asked the Comptroller and Auditor General to assess the extent to which there had been a reconciliation of the two different estimates to establish where the disagreements lay.
The National Audit Office commissioned statistical advice from the London School of Economics. This found that the methodologies used by Customs and the Scotch Whisky Association for estimating spirits fraud, by calculating a total level of spirits consumption and deducting from this the known level of legitimate duty-paid spirits ("gap analysis"), were similar. The differences between the fraud estimates arose mainly from the different survey data used by Customs and the Scotch Whisky Association to estimate spirits consumption and the adjustments made to reflect the under-reporting in such surveys.
There is uncertainty inherent in basing an estimate of spirits consumption on survey data. But it is not practical to ask the total population and some kind of survey must be used. Customs base their estimates upon surveys carried out on household expenditure and consumption by the Office for National Statistics and the Department for Environment, Food and Rural Affairs. The Scotch Whisky Association estimates are based upon a different survey, (the Omnibus Survey) carried out by the Office for National Statistics. For the purposes of this review, and with the agreement of the Scotch Whisky Association, the National Audit Office reperformed analysis work using data from the General Household Survey. This was undertaken because the Omnibus Survey only provided data for periods from 1997 onwards, while the General Household Survey has data from 1992-93 which means a more robust method can be used to adjust for the under-reporting of spirits consumption.
A substantial degree of uncertainty attaches to any estimate made of spirits fraud based on surveys and to assumptions made about underreporting. Hitherto, Customs have presented the estimate as a single figure, whereas statistically it falls within a range of uncertainty. The NAO calculated that the Customs' estimate of £600 million should more properly be presented as a range between £330 million and £1,080 million. Likewise the NAO calculated that the Scotch Whisky Association estimate of £100 million to £150 million lay between £10 million and £260 million. The high level of uncertainty implicit in the methodologies has not hitherto been made clear in the published estimates provided by Customs. Further uncertainty was raised on Monday, 8 March 2004, when the Office for National Statistics informed the National Audit Office that they had discovered an error in their analysis of the General Household Survey or 1992-96. As a result the National Audit Office is only able to say with certainty that the Scotch Whisky Association estimate of £100 million to £150 million based on data from the Office for National Statistics Omnibus Survey will itself have a wide confidence interval. The Office for National Statistics has said they will now undertake further work to correct the error in the analysis of the General Household Survey.
The examination by the NAO of the differences in the respective estimates of fraud by Customs and Scottish Whisky Association has shown that both methodologies use survey data (although gathered in different ways) and both make allowances for generally recognised under-reporting by respondents to surveys. The NAO's technical adviser and the Office for National Statistics consider both methods for collecting data to be reasonable to professional statisticians. But, according to the NAO, it is difficult to accept that both methods are reliable in this case when they result in such widely different estimates of consumption.
It is therefore clear that further work needs to be done by the Office for National Statistics, with Customs and the Scotch Whisky Association, to explain why there are such different estimates for consumption and therefore alcohol fraud. It is welcome that the Office for National Statistics are considering a longer term project to explain the contrasting results of the General Household and Expenditure and Food surveys, and further work to identify ranges for General Household Survey-based estimates. In the meantime, neither survey estimate can be accepted as unequivocally reliable and great care is needed in determining what reliance is to be placed on the results at present available.
Mr Leigh said today:
"Whilst I welcome the fact that Customs have produced fraud estimates to help them tackle the illicit alcohol trade, the National Audit Office has highlighted the extent to which current estimates of spirits fraud are riddled with uncertainty. This is as true of the estimates produced by Customs as of those produced by the Scotch Whisky Association. Clearly, it would be unwise for Customs to place too much reliance at present on their own figures. I urge them to develop a firmer estimate or at least one whose limitations are clearly recognised.
I am further alarmed that the ONS discovered errors in their analysis of the General Household Survey data only at the 11th hour, and call on them to rectify this as quickly as possible."
to view Memorandum