The CEO’s Margin of Error — A Story

One thing I remember from the blurry haze that was my time “dancing the electric slide” at Bain in the early 2000s was a mini-presentation to the juniors by one of the partners who told us what it was like being a CEO.

Being a young whippersnapper, I lapped this stuff up. Unlike some of my well-connected colleagues, I didn’t meet many CEOs. I was fascinated by the idea of seeing one up close.

The partner in question had been CEO of a couple of mid-tier listed companies in Australia. “Australia?” you cry. “Get real.” Whatever, these were companies with valuations in the hundreds of millions of dollary-doos. They had hundreds (or thousands? I don’t know) of employees. That’s spicy enough for me!

Anyway, what he told us stuck with me, and applies strongly to what I’m doing these days (algo trading), so I’m writing it down for posterity.

A CEO’s Responsibilities to Predict the Future

The first thing he explained was that the CEO’s job was to be the central source of information, managing expectations vs reality between a number of sources of pressure. Let’s see if I can remember them all and create a diagram…

Wow, that’s a lot of stakeholders! I do hate the word “stakeholder”, but in this case, it’s easier to say than “affected parties”, which also sounds stuffy. Anyway, all these people hold stakes, and are trying to drive them through the CEO’s spleen.

What do all these stake-wielding jerks want? They want to know what’s going on. Shareholders want to know if the stock will go up. The media wants its pound of flesh. Employees just want to know if they can show up tomorrow.

So the job of the CEO is to know, within a reasonable margin of error, what’s going to happen in the near future, and hold everyone at bay.

That’d be nice, right? We all want to know the future. Well, it’s bloody hard, and that’s one reason why CEOs are paid well when they do a good job, I suppose (not that I disagree that they’re often overcompensated).

But the most interesting part is that he explained that when predicting the future, you have to be within the margin of error both ways, whether things are going well or badly.

In other words, it’s obviously bad if performance is worse than you predicted. But if it’s also significantly better than you predicted, that’s also bad, as it shows you don’t have a firm grasp on what’s going on in your world.

Applications to Trading

This hits home because in algorithmic trading (my current shindig), you always have a caveat of “past performance is not an indicator of future performance”.

In its simplest terms, this just means “just because a stock went up, you can’t assume it’ll keep going up”. Everyone knows that, and it’s generally true, other than looking at the past and over select periods. (E.g. the stock market has generally gone up consistently over the last 100 years, but there have been lots of scary periods in the middle.)

But a more nuanced view, applied to an algorithm, is “just because your algorithm worked on 10 years of past data, you can’t assume it’ll continue to work in the future”.

In other words, if I say “buy when RSI < 30, sell when it’s over 70”, if that happened to work over the last two years, it might not work next month. Or ever again.

It’s satisfying when it DOES work, though. Here are a couple of strategies that are behaving roughly in line with what I thought.

The first one is alarming a bit. A win rate of 100%! Of course, it has only done two trades, so we need more data.

But then, that second one,w ith its return of 0.18% vs an expected 0.19%… and a win rate of 60% vs an expected 56.3%… that is sweet.

How much longer will these be valid? I don’t know yet… once I have more data, I’ll have to add procedures to decide if an algorithm is still valid or if it needs to be rebuilt.

What’s the right margin of error?

This is something that all traders grapple. How many times does a strategy have to fail before it’s deemed non-functional? That’s a whole other question of statistical analysis.

You need to have two things

• A statistically significant sample considering the parameters of the prediction
• Defined parameters that mean your prediction is either out of bounds or within bounds

As a starting point, I’m putting “within 1/4 of a standard deviation” and requiring 50-100 trades for a 95% confidence interval. But I can vary that later…