How companies stagnate
I once had a manager who said something along the lines of 'The job of a good manager is to set up a process where a team doesn't depend on them'. The idea is that a manager should be able to go on holiday for two weeks, nothing would break, nobody will be blocked in their work and the manager would go undisturbed on their holiday.
Sadly, this isn't the norm but the exception in corporate culture. Have you ever worked with a micromanager? Have you ever worked with a manager who clearly doesn't know a thing about your work? Have you been denied a promotion because of reasons? Due to complex social and lack of system design, companies end up creating the cause for their own suffering - Middle Management (or at least one of the causes). It's a meme at this point and the incredible part is that very often these people are not even aware of the issues they're creating in a company. It's a whole class of people that are otherwise smart but when tasked with the title of 'manager' or 'head of X', they become a victim to their own lack of understanding of systems as well as their own psychology. What's especially problematic is that they try to reduce complexity in their organisation by creating structures and heuristics that are simple to understand. The result - a mess of problematic incentives and politics that drive an organisation.
People are self-serving
Most social behaviours come from the utility of transactions with other humans. I help you out, you help me back. Social hierarchies and the psychology around that evolved at a time where our survival was threatened if we don't work together. Working together with other humans was fundamental to securing food and shelter. Fast-forward to today, we've built structures (institutions, companies, governments) that ensure our most basic needs. However, deep-down we're still just monkey-adjacent bipedal mammals and our psychology to seek our own-survival hasn't changed at all - it's literally in our brain hardware to detect threats and try to remove them. Here you may say 'Hold on...
Why are we talking about monkeys and evolution?'
If you want to understand a behaviour in the world, you need to take into account our species' evolutionary psychology. The way this translates in the modern world is at the root of all problematic behaviours we see in societies today. People no-longer need to look for shelter or food and we've since created a society where you can rely on the structures around you to support your life. You don't need a whole tribe of people, your own family is enough, hence the move toward individualism in recent decades. Of course, individualism is normal and required for a functioning human but when taken to an extreme... it's extremely detrimental to the environments in which it's presented. Particularly in companies, this is easily incentivised through personal goals and performance-based bonuses. The only thing this achieves is behaviour which optimises for their own self-interest. Have you ever seen someone claim credit for something that they didn't do? Or prop-up their experience/contributions to make a case in front of higher-ups that they deserve a promotion? I'm sure you can come up with other examples too. One can even make an argument that it's not their fault - they're just placed in an environment where this is rewarded. Instead of collaboration, companies start rewarding individual contributions.
What is performance anyway?
The institutions humans have created have significantly grown in complexity. The hierarchies we've created are no longer about raw physical power and ability, but of competence in specialised areas.
We've created various domains like accounting, investing, software engineering and so on. Each depends not on physical ability like strength or stamina, but on some combination of knowledge and decision-making in certain situations. This is what most of knowledge work is - you've learned a bunch of things and you apply them in the context of a series of tasks that your employer needs. Companies nowadays often optimize for people who are specialists in certain areas.
One of the biggest problems in companies and especially large corporations is they try to reduce complexity and uncertainty by creating processes, rules and structures to manage that complexity. We try to neatly separate and structure people into groups by their skillset - software engineers, product, designers; departments they work in - payments, orders, risk and skill level - junior, mid-level, senior, manager. It's all an attempt to reduce the complexity of a business by segregating people among arbitrary criteria. What does this do? It creates silos. It's really baffling that whenever you start reading about company and team structure, most people seem aware that knowledge silos are bad and yet most companies work like that. On paper, it makes sense - this is my backyard, here's the fence and it will be my boundary of responsibility. However, in a complex system like a business (any business really), when people start working together, you quickly notice that these boundaries blur. When you have a piece of work that multiple people need to contribute to, you have an enormous overhead of communication and coordination between them. Just look at the rate of increase for communication paths as you add more people to a project:
This is why many companies correctly prefer smaller team sizes like Amazon's two-pizza teams. What gets overlooked is when a piece of work like a new initiative or a single feature spans multiple teams. If there are multiple teams, each responsible for their small part of the business or software product, you can easily fall into a trap where you need the input of 3 different teams to add a single input field and a button on a web page. This is extremely common in software products where teams are segregated through the use of so-called microservices architecture. Unfortunately, more often than not, microservices are an attempt to solve an organisational issue rather than a technical one.
So if you're tasked with a job that's impossible to do on your own both as an individual or as a team... why are you judged on individual performance?
It's yet another attempt at reducing the complexity of the environment. John didn't deliver a project on time? No bonus. It's a simple heuristic that Middle Managers use. What they don't see is that maybe John needed to wait for 3 weeks for some other department to do their bit on the project (and this is simplifying it as often you need assistance from multiple teams). They of course, have their own goals to chase so why should they bother helping you? Examining dependencies, mapping the multitude of relations between teams and problematic incentives is much more difficult. Middle Managers don't like difficult things - they like simple boundaries, metrics and heuristics.
When you need to do a job in any organisation with hundreds of people, you certainly need some sort of separation but my main thesis is that organisations often fail in getting the separation correctly. You can clearly see this by observing how many restructuring projects fail. A company tries to shuffle things around, laying off some amount of people, often rehiring them after a while. A couple of years later when nothing has changed in terms of revenue and costs, executives try it again and the cycle continues.
What's the reason for that? It's because fundamentally companies don't change the approach to how teams are structured. They merge and split departments but the core issue remains. You don't solve complex issues like productivity, stagnating revenue and growing costs by treating symptoms. You do so by fixing the root cause.
'What's the root cause?' you may ask
So far we've covered the following main points:
- People are self-serving - they won't do something for you unless they gain something from the interaction too
- Team topology is not a trivial task - businesses are complex and projects often span multiple departments
I don't like simplistic summaries because they often feel reductive and there's a lot more at play with these topics but I'll do my best. Most companies incentivise individualistic behaviour through their attempts to organise a nebulous thing like a business into neat categories that don't depend on each other. Middle Managers then attempt to then coordinate communication overhead by organising meetings. Talking to people and scheduling meetings becomes their job. They feel productive and present themselves as such to higher-ups through all the hard work they did because they 'led a team', 'facilitated a ton of meetings', 'had a full calendar' , thus making themselves irreplaceable. That's the new paradigm for an individual's survival needs - preserving their status in the company hierarchy. How would software developers know what to do if there's no manager to tell them their next task?? If they were to go away on holiday tomorrow, the whole thing would crack. This, of course, defines the cornerstone of corporate culture - multiple meetings for coordination, planning, budgeting, demoing, town halls, huddles and so on and so on. It creates a system of work that's based around an attempt to manage complexity rather than embracing it.
None of these things are new problems and what baffles me is that solutions exist but they are difficult to implement. Once a company reaches a certain scale or dominance in a market, it can survive for many years until a startup disrupts it. In his book, Beyond Command and Control, John Seddon describes the alternative - a company culture where the people closest to the problems are the ones with most knowledge of how to solve them. They allow and enable people to implement solutions without gatekeeping responsibilities for the end-to-end lifecycle of projects/products. They don't create 10 levels of hierarchy and managers that know nothing about the work being done. This is precisely why startups can easily disrupt giant companies with a team of 10 people - they lack the communication overhead and focus on customer-oriented metrics rather than their own self-serving incentives.