Saturday 30 April 2016

Econometrics, the Bane of our Existence: Weekend Reading

Fun though it is to have a punt at predicting the price of oil two years hence, it is constantly amazing to me that apparently hard-headed people fall for economic forecasting all the time, without once giving thought to the fact that (a) empirically, it has a staggeringly awful track record - you'd never take medicine from a physician with a history of failure like that; and (b) intellectually, it is bankrupt.

I could write my own lengthy & vehement essay on the wretched pseudo-science (and very likely will): but you'll probably prefer this, by a proper professor.



Demetrius said...

Back in the 50's we had some of the early birds in the Econometrics and Game Theory trades in the Common Room at the LSE. The great question in my mind was why did they lose so much at the poker and bridge tables?

drsolly said...

Full disclosure - my PhD is in econometrics.

The problem is that so many economists are mathematically innumerate and statistically insignificant. Worst of all, there's a failure to understand that the accuracy of historic data is inadequate to support many of the analyses and conclusions that are drawn.

It is quite rare to be able to make any statement based on an econometric analysis other than "Things will be much the same as they have been".

Nick Drew said...

Demetrius - yes, and at the even higher-stakes tables of LTCM etc!

drsolly - welcome, and congrats on your courageous disclosure!
I'd add that a lot of people in my sector (energy, where forecasting is endemic and shockingly bad) are engineers, with a strong propensity to believe anything that comes confidently presented in a big array of numbers and some appropriate B-S mumbo-jumbo on 'methodology'

and they routinely bet vast sums of money on this crap, often completely naked exposures, mouthing all the time that they are 'risk-averse' - see the 'Risk' section in any AR you care to read

(before the floodgates open from the C@W engineer crew, let me agree that (a) not all engineers ... and (b) not only engineers)

andrew said...

In no particular order

String theory - worse than econometrics - there are a number of classes of string theory that cannot currently be shown to be wrong

Mathiness - as drsoly said, in many of the social sciences (not just economics), mathematical expressions are laid out that look coherent and logical, but on close inspection are not

The anti-Zionist/Anti-Semite error. Yes, lots of economics is not overly useful and indeed has been shown to be useless (econometrics - some say macroeconomics) but there is no reason to class ALL economics / ecomomists as useless.
micro-economists who look at small areas of competition, trade and get hard evidence / data do produce useful results.

I commend noah smith -

Nick Drew said...

no reason to class ALL economics / ecomomists as useless... micro-economists who look at small areas of competition, trade and get hard evidence / data do produce useful results

agreed, Andrew, that's why I led with econometrics, rather than the broader brush used by the Prof

Methuselah said...

Was part of a team, that put together an economic forecast for Unilever to make one of its largest ever investments in the UK at the time. Decades later the plant is still there and working.

Was it luck, good forecasting, sound economics - or the fact it was so long ago there were no desktop PCs at the time and your work had to be thought through?

Nick Drew said...

Methuselah - welcome, old chap! (sorry)

can't answer your question, though I might guess it was partly based on some demographic trends that are sometimes easier to *get right* than purely financial matters (was it a food plant, or detergents ..?)

not to say your success was luck; but there have of course been some very significant successful punts - the one I always cite was John Browne (BP) calling the oil-price bottom in 1998, it was $10, but the Economist was suggesting it might fall to $6

Browne called it, and BP bought Amoco, Arco and BurmahCastrol, thereby doubling the size of his company overnight: can't get much better than that in commodities

it was still a naked punt!

Sebastian Weetabix said...

Points about engineers are well made, ND. There is a clear tendency among thicker ones to believe in anything a computer spits out; I see this among my peers every week.

Everything in life is a naked punt. Having some mathy predictions allows people to kid themselves they are a making a fact based decision rather than making a choice based on their own judgement. Saves them having to take responsibility and actually earn their pay.

andrew said...


I suspect you are right :
"- the fact it was so long ago there were no desktop PCs at the time and your work had to be thought through?"

Not just thought through, but as you had to think it through rather than kick of another run of a spreadsheet model etc, two other things happen

1 - you tend to concentrate on the '20%' of the factors that are important and ignore the rest
2 - you are realistic about the limits of knowledge and so the limits of the model

These 2 things tend to improve the quality of decision making.

Electro-Kevin said...

From what I read here I gather econometrics puts the 'educated' in guess. It serves two purposes:

- sways decisions in favour of whoever pays for it (self fulfils prophecy)
- covers arses to some degree when things go wrong "well, the empirical facts suggested it was the right thing to do, boss."

If a camel is a horse designed by committee then what does econometrics produce ? A bull with tits on it ?

My guess is that neither Churchill nor Eisenhower would have had much time for this advice choosing what they felt in their guts instead.

In respect to the previous post "Economists come out in favour of Brexit" I'm glad they do, but I'd rather ditch all economists, especially BBC ones.

James Higham said...

When's yours coming out? I'll get a copy.

Steven_L said...

... (BP) calling the oil-price bottom in 1998 ... Browne called it, and BP bought Amoco, Arco and BurmahCastrol ... it was still a naked punt!

But what was the actual downside - to him? This guy was paid a salary (and a bonus?) to run a limited liability company. He made a punt with other peoples' money. I suspect a lot of high level business decisions are basically punts with other folks cash. Having pages of mathematical mumbo-jumbo to justify yourself is useful. It was handy for the bankers that loaded up with CDO's to blame their quants from what I remember.

Council managers are the same really. They do the most barking things because people in higher places have written a nonsensical text advising them to. If it goes wrong, its not their fault, they keep their cushy job, pension and chance of climbing another rung. If they do something they thought of - or God Forbid suggested by one of their underlings and it goes wrong - they risk all that.

So they all meet up with their peers in other councils and collude to write nonsensical texts. There's a market for them. Staying at your desk and actually doing your job is risky!

Nick Drew said...

I suspect a lot of high level business decisions are basically punts with other folks cash

yes, Stephen, and I wouldn't have a problem with it, if the punt was characterised properly to the shareholders - and the speculative business model correctly described in the prospectus and AR etc. The issue is that as mentioned above, the AR's of most industrials swear blind they are risk-averse, never speculate on commodity prices etc etc

a very longstanding problem

Steven_L said...

Next you'll be complaining about politicians being economical with the truth and making promises they have no intentions of honouring!

You know, when I worked in utilities (sales and call centre side of things) they always wanted you to believe the blatant lies they made you tell the customers and you had to try and keep it a secret if you were a 'cynic'.

I don't imagine the investor relations bit is all that much different, other than they are harder jobs to get and brighter people doing the lying.

Raedwald said...

My Master's had a significant micro economics chunk much of which was taught by an econometrist. He was also a betting man and liked to share his analysis of election polls in terms of probability. He never made much money. Most useful was learning about something called Student's T - a method of drawing complex probability findings when one has virtually no data.

Anyone having had to satisfy the Treasury tests for public infrastructure spending over £5m to include a Monte Carlo analysis of probable contractor tender levels will be grateful for such econometric tricks - pulling conclusions from thin air in a way that laymen not in the club cannot understand. How many billions of poorly-spent tax money has been justified by such tricks I cannot begin to estimate ...

Nick Drew said...

Next you'll be complaining about politicians being economical with the truth and making promises they have no intentions of honouring!

I am just a naive innocent: things were so much simpler in the army

hovis said...

Nick you did make me chortle inadvertently - most people do take medicines with poor track records - much is Pharma huff and puff - I say this as someone who used to work in the R&D arm of one of the biggest.

Agree about econometrics too - we love the sense of certainty - esocially when it sounds complicated.

hovis said...

That should have read - we love the sense of certainty when it is backed up by complicated sounding reasoning or methodology.

Especialy agree with S-W and EK.