Adios, “Middleware Jobs”: AI and the Future of Organisation Design

I used to consult on organisation design (albeit as a junior schlepper, but that means I know how the sausage was made… it was made of me) and have architected my share of teams in tech companies. So I have a few thoughts on how AI will affect company structures and organisation designs.

In everyday corridor conversations, people thinking about the impact of artificial intelligence on work are usually considering one of two questions:

1. How can I cut costs? and/or
2. What will happen to my job?

These are valid concerns, of course. Running a lean company is every manager’s ambition, and job security is the concern of anyone with (or without) a paycheck.

But efficiency and job security are short-term questions. And there’s an easy answer, anyway: If your job primarily involves the transmutation of information from one place to another without dealing with humans or physical sites — in other words, if you have a “middleware” job — then the entire concept of the work that you’re doing will cease to exist within 10 years from now — earlier, if you’re in a tech-forward industry.

How Will AI Affect Organisation Design?

Now that the bad news is out of the way — that if you’re asking, then yes, your middleware job and those of everyone around you will probably vanish — let’s look at how AI will inform organisation design.

Because the better question to be asking is: If you could remove every single annoying part of the head office of your company, what would be left? The answer is “nothing!” Who’d want to keep annoying bits?

Companies will be comprised of, at most, two things:

1. Business site operations, for operations that require physical sites — e.g. manufacturing (factories), mining (mine sites), construction (buildings), or logistics (warehouses) — anywhere where there is a building, equipment, and goods of some kind
2. Customer site operations, for operations that directly serve people — e.g. storefronts, public transport vehicles, and restaurants — anywhere where there’s a physical interface with a human

That’s it. Everything else will go. Human resources, software engineering, data analysts, operations managers, customer support, legal, accounting and finance, and design — it’ll all disappear. In fact, with physical automation (robots!), so too will vanish a lot of back-end infrastructure ops currently done by people.

And, in many cases — good riddance. This may sound harsh, especially in times of economic uncertainty, where houses in western countries are more expensive than ever as a multiple of incomes, and paychecks are no more certain than they ever were.

But none of those jobs were what people dreamed of doing as kids. No kid wants to be a data analyst, HR rep, or customer service agent. Kids want to be astronauts, race car drivers, scientists, athletes, and artists. The good news is, they still can.

So where are these “middleware jobs” going?

Employees as Groups of Functions

Here’s why you can abstract out human employees into AI-powered software for corporate office jobs, and a framework for thinking about it. In a nutshell, they’re groups of functions, or arrays of functions, in code speak.

The core of the thesis is that most back-office jobs are “middleware”. By “middleware” I mean that most corporate, back-office jobs involve taking information from one place and placing it in another.

For example, many of my daily activities back in my corporate times were things like

• Collect data from client databases, analyse, and create analyses and then presentations
• Summarise notes from meetings and send an email
• Collect opinions from various internal and external people and create a product spec document
• Use a draft template, some web research, and company-specific information to create a pitch deck
• Run consumer research survey, collect, analyse, and summarise results into a product change plan

The list goes on. The important thing to note in the above is that none of the above involved a human face-to-face interaction. There were other things that did, e.g. running training sessions or meeting clients, but I’m just talking about the back-office work.

Any work that doesn’t involve a face-to-face or face-to-machine interaction is just a machine-to-face-to-machine interaction. In the past, people used process definition and outsourcing to move jobs to low-cost offshore centres. In the present and future, the offshore centres will be made redundant — but so too will the onshore centres. In machine-face-machine interactions, the middleware “face” is the redundant part that we will abstract away into functions.

Let’s look at a typical job description for example. I’ll start with a common one: operations analyst. It’s a job I’ve had (with a different title, but this is what most people call it) and that many of my peers have had.

So, the above are major things that the senior operations analyst does. Never mind the descriptions of their ideal prior experience and qualifications (or hobbies!), because those aren’t necessary for this analysis.

Let’s define the major responsibilities of the operations analyst in terms of functions.

function analyseOperationalData(serviceLineData: any[], historicalMetrics: any[]): any {
  const extractedData = extractRawData(serviceLineData);
  const cleanedData = cleanExtractedData(extractedData);
  const analyzedData = performDataAnalysis(cleanedData, historicalMetrics);
  const visualizations = createDataVisualizations(analyzedData);
  return generateAnalyticalReport(analyzedData, visualizations);
}

function buildPredictiveModel(historicalData: any[], marketTrends: any[], seasonalFactors: any[]): any {
  const preparedData = prepareModelingData(historicalData, marketTrends, seasonalFactors);
  const model = developPredictiveModel(preparedData);
  const testedModel = testModelAccuracy(model, preparedData);
  deployModelToProduction(testedModel);
  return generateForecastsAndRecommendations(testedModel, preparedData);
}

function optimizeOperationalEfficiency(performanceMetrics: any[], processMaps: any[], customerFeedback: string[]): any {
  const identifiedProblems = identifyEfficiencyProblems(performanceMetrics, customerFeedback);
  const rootCauses = analyzeRootCauses(identifiedProblems, processMaps);
  const proposedSolutions = designEfficiencySolutions(rootCauses);
  return createImplementationPlan(proposedSolutions);
}

function facilitateCrossFunctionalCollaboration(projectBriefs: any[], stakeholderRequirements: any[]): any {
  const engagedStakeholders = engageProjectStakeholders(projectBriefs);
  const gatheredRequirements = gatherProjectRequirements(engagedStakeholders, stakeholderRequirements);
  const developedPlans = developProjectPlans(gatheredRequirements);
  return supportProjectExecution(developedPlans);
}

function managePeformanceDashboards(kpis: string[], systemData: any[]): any {
  const definedKPIs = defineRelevantKPIs(kpis);
  const integratedData = integrateDashboardData(systemData);
  const designedDashboard = designPerformanceDashboard(definedKPIs, integratedData);
  maintainDashboardFunctionality(designedDashboard);
  return updateDashboardWithLatestData(designedDashboard, integratedData);
}

function supportStrategicAnalysis(businessQuestions: string[], relevantDataSets: any[]): any {
  const framedProblems = frameBusinessProblems(businessQuestions);
  const collectedData = collectRelevantData(framedProblems, relevantDataSets);
  const performedAnalysis = performStrategicAnalysis(collectedData);
  const generatedInsights = generateStrategicInsights(performedAnalysis);
  return presentStrategicFindings(generatedInsights);
}

Or in summary format:

const businessIntelligenceFunctions = [
  function analyseOperationalData(serviceLineData: any[], historicalMetrics: any[]): Report,
  function buildPredictiveModel(historicalData: any[], marketTrends: any[], seasonalFactors: any[]): Forecasts,
  function optimizeOperationalEfficiency(performanceMetrics: any[], processMaps: any[], customerFeedback: string[]): ImplementationPlan,
  function facilitateCrossFunctionalCollaboration(projectBriefs: any[], stakeholderRequirements: any[]): ProjectExecution,
  function managePeformanceDashboards(kpis: string[], systemData: any[]): Dashboard,
  function supportStrategicAnalysis(businessQuestions: string[], relevantDataSets: any[]): StrategicFindings
];  

Of course, this is at a high level. Each of these functions has a bunch of subfunctions, dependencies, and data requirements.

For example, to run analyseOperationalData, you have to be able to hook into the organisation’s data stores (via packages and APIs), know how to clean the data, figure out the best visualisation, create the visualisation, and determine an insight (e.g. trends or outliers).

But you can already tell from that description that all of this is possible through software, and made even easier through LLMs that are super good at understanding abstract data sets.

Compounding the inevitable is the fact that a good operations analyst is rarely someone who’s a genius, but rather someone who’s just thorough, hard-working, available, and quick. The insights are about as sophisticated as “To meet demand, we need more supply at 7am on Saturdays”.

And, if you are lucky enough to find someone who’s thorough, hard-working, and quick, you have to make sure that they’re well-trained, happy enough to stay doing that work, available when you need them, and paid enough to not quit for something better.

Further Benefits of Redefining Work Output as Function Arrays

There are even more benefits of replacing human functions with AI ones, beyond not having to work about utilisation rates, HR problems, or the cost of humans in general.

Firstly, you can structure a company around information flow. Everything is an input, output, and workflow. This lets you think about things much more cleanly than worrying about who will do a task (let alone having to fill that vacancy).

Secondly, you can get rid of human data models. No more presentations, spreadsheets, or emails. You can just communicate in JSON or YAML or whatever structured markup you like. At times when you need a human interface, it’s super easy to publish something for review.

Thirdly, you can scale any function, no matter how trivial. Functions don’t care how many times they’re invoked. Ask a human to do 100 sales pitch presentations in a short period and they’ll burn out and/or get bored, no matter what you’re paying them. Call software to do it, and it’ll do it in a jiffy and say “What’s next?”

Fourthly, you can change the spec of any project as many times as you like, refactoring all the code and data schema downstream. You can optimise for resource efficiency, change inputs and outputs, and the code doesn’t care an iota.

Finally, just to reiterate, all the humans doing these functions will disappear.

What’s Left Beyond Middleware Jobs?

So, gone are all the “middleware” layers of management. The only thing that’s left is production and output. So what’s left?

The only things left will be where the rubber hits the road — physical production of goods people need — and direct human-to-human interfaces, where one part can’t be done by a machine.

This means, in our present world, the following human jobs will remain:

• R&D
• Manufacturing
• Mining
• Warehouses
• Data centres
• Logistics
• Small-scale production (e.g. restaurants)
• Consulting
• Medicine
• Sales
• Beauty

The scary thing, if you can’t tell, is that a lot of that is already being heavily assisted by machinery and artificial intelligence — if not replaced. At this stage, I’d prefer it if a human cut my hair. It seems tricky. But if in the future, a machine could do it for a fraction of the price — well, sign me up. (There are some prototypes out there already!)

Maybe the last bastion of work to not be overtaken by artificial intelligence will be the design and refinement of AI models themselves. But even their days are numbered.