Rational approach vs messy reality: Do policy appraisals help decision-makers learn?


When policies are developed, a fairly standard procedure, called a “policy appraisal” is often followed. Research is commissioned, stakeholders consulted and policy impacts assessed, supposedly leading to “better” regulation.

This rational approach, which is designed to help decision-makers think and learn, often doesn’t reflect the reality of the ‘policy mess’.

A 2009 paper by Claire Dunlop of the University of Exeter explores whether policy appraisals for technically complex issues are actually reinforcing limited learning in government.

Dunlop uses the case study of UK’s biofuel policy, where after four years of policy development and extensive appraisal, the Renewable Transport Fuel Obligation (RTFO) came into force, requiring that biofuels make up 2.5% by volume of road transport fuel sales. After the policy had started to be implemented, there were questions about the sustainability of biofuels (particularly their negative impact on food security and deforestation) leading to calls for a review, and in some cases a moratorium, on all policies aimed at increasing the use of biofuels. The second review recommended a few adaptations to the existing policy.

Dunlop dissects the differences in decision making learning that take place in the first review process versus the second. She argues that there are several reasons that enhanced learning took place in the second review process:

  • — Prior to the second review, the government had a firm stance against slowdown in biofuels adoption. The government’s willingness to adapt the policies following the second review suggest that they would be open to policy change should more damning evidence against biofuels be presented in future.
  • — While the government was against the idea of a moratorium, the minister asked for it to be reviewed as a possible policy option. This recognition of public concern is an important step towards enhanced learning.
  • — The second review focused on indirect effects, crystallizing for decision-makers that some aspects of biofuels impacts were intangible, and could not be rationalised.

Dunlop concludes that the evidence given by appraisals can be too conjectural or unclear to force decision-makers to reconsider the premises on which policy is based and engage in deep forms of learning in the time available to them. In the case of UK’s biofuels policy appraisal process, evidence lagged too far behind policy development to trigger any fundamental re-thinking.

However, measures can be taken to enable decision-makers to learn, she says.

1. Deeper learning may result from ‘knowledge brokers’ located beyond the immediate circle of government conducting reviews of policy appraisal. The unique professional position of Chief Scientific Advisers, spanning the boundary between science and politics, may give them the right blend of credibility and political authority for their advice to be trusted.

2. Appraisals may lead to more enhanced learning if they are conducted in the public eye and beyond the immediate circle of government.

3. Second appraisals do not mean the first was ineffective.

The full paper can be read here.