Further observations on evidence and quack policy
I donāt think anyone can be against the idea that the evidence base used to influence policy should be high quality and robust. Poor quality research should not be given undue influence. The question then is how that assessment of quality is made and by whom.
All scientific knowledge is provisional. We may have plenty of corroborating evidence in favour of a particular position, enough to draw conclusions with a degree of confidence, but never certainty. I don't think there should be any disagreement about that. The disagreement comes over the degree of confidence that would justify action and the type of action it justifies.
I donāt disagree with what I think is your main point, which is that it is important to look at costs and benefits and to think about substitution effects when assessing the desirability of intervention. However, once weāve done that to the best of our ability we might well conclude intervention is justified. The question then is how we assess how well weāve accounted for everything. If, on the other hand, we were to apply a standard to the measurement of costs, benefits and substitution effects that was so stringent that it meant they were never acceptably measured, and therefore no intervention could ever be justified on the basis of āevidenceā, we might as well drop the pretence that we're trying.
You ask ādoes anyone think preferences are fixed?ā. My answer is pretty much the whole of post-Samuelsonian microeconomics, which dominated the field until relatively recently, and still dominates much economics education. This would include some of the most fundamental theoretical results in welfare economics. It would include most of conventional cost-benefit analysis, including some of the key valuation techniques for intangibles. You donāt have to say that you are assuming fixed preferences, but if you invoke arguments making use of those sorts of ideas/methods then the assumption is implicit. In the last decade or two microeconomics has become much more comfortable with ideas such as peer-group effects, references levels, multiple-equilibria, and the use of techniques like evolutionary game theory. These types of approaches can encompass preferences being endogenously determined and they can deliver very different results to those derived from conventional microeconomic analysis. They can also place a question mark over the use of standard techniques (eg. if well-being depends on relative rather than/as well as absolute consumption then standard cost-benefit analysis becomes problematic).
Much of the move to consider reference levels, rank effects, peer groups etc originates in the behavioural economics literature. The literature also points to phenomena such as time-inconsistent preferences, loss aversion, inconsistencies/āirrationalitiesā in discounting, differences between decision utility and realised utility. These are all relevant to the question of how best to assess utility/well-being.
I didnāt say that your arguments were superficial, but that your engagement with the various policy areas you discuss is relatively superficial. Your arguments may well be logically sound and fully justified in relation to the material you cover. But your discussion of each policy area is not heavily embedded in the various literatures ā or at least not explicitly; your publication typically focuses upon only one or two example references in each policy area. It may be that you are taking these as exemplars and that knocking those down means the whole edifice tumbles. Iām not sure that works. To draw robust conclusions about the current state of knowledge in a complex field would need much closer engagement. And to dismiss large bodies of literature on the basis of an a priori application of a very particular understanding of what constitutes āproperā science would be a controversial move.
Your counter-arguments in a number of cases rely on thought experiments rather than a weight of contrary empirical evidence. The unwary reader might take these experiments as carrying the day, but that would be imprudent. For example, you are clearly a strong believer in bargaining solutions to social problems. Coaseās point was that these can work at small-scale with no transaction costs and equality of arms. But in situations where those conditions donāt hold bargaining solutions become problematic/impossible. The reader may miss that if they donāt know that particular literature at all. It is an important piece of the jigsaw they need in order to appraise the plausibility of proposed solutions of whatever type.
Your critique of evidence-based policy making focuses on a selection of examples in a selection of policy areas. Whether or not that critique is valid it can hardly count as a critique of the reasoning behind āmuchā evidence-based policy, unless the case is made that the sample of areas considered is representative of 'policy' generally.
The thesis that your publication could be part of a move to question academic independence, which is advanced briefly in the original post but not argued through at any length, has little to do with its content or, to be honest, with authorial intent. Once it is in the public domain the publication - any publication - takes on a life of its own. It could be represented in all sorts of ways, including as an argument to government that they shouldnāt listen to academics because they are all āself-interestedā and an elite trying to foist their ideas on to government against the wishes of the public. Few will read the report itself to find out whether that is what it actually says. That process can be seen to some extent in the press release which accompanied the report: it simplifies the argument, cuts out all the nuance, and strikes a different, more negative, tone than the report itself.
Representations of the report could, I hypothesize, contribute to a broader discourse questioning the integrity of academic endeavour - whether that was the original intention or not. Time will tell if that hypothesis is corroborated. But to develop the argument more fully would take us into territory perilously close to post-structuralist literary criticism, something I know you hold in low regard.