How corporations lead you to career stagnation
How do I progress in my career?
That's something I've asked this myself a few times over the years. First, let's try to define what progress is. It's not immediately obvious what progress is and I don't think there's a universal definition for everybody. In most cases, this will be tied to some vague path towards activities they enjoy doing, mastery over some set of skills and responsibility for teams, products, software architecture, coding practices and so on. Why someone may want to gain all of these is out of scope for this article but I'd like to explore it in the future.
This article is somewhat a continuation of the previous one. I'll cover some of the same concepts but from the angle of career development.
I'll focus mainly on the software engineering domain as that's where I've worked four about 10 years now but just like the previous article, a lot of this is relevant for many industries, domains and jobs because the underlying issues are how humans interact with one another and the environment that is established.
For the time-being let's take the above as the definition of progress - maximising skills, knowledge, wisdom and responsibility in the context of your job. By that definition, the high end of the spectrum would be approximately an excellent software engineer/CTO who can drive the technical side of a large-scale project (say, millions of users) or one with many technical complexities through in-depth understanding of both the tech and business. I use that as a rough guideline/definition because in my view it encapsulates a lot of other skills - understanding the product and customers, stakeholder management, ability to structure workflow processes, form productive teams and motivate them as well as writing good quality code. The three broad categories here are - tech, business and interpersonal skills. Essentially, a great tech leader would need to understand each of them and be proficient so that they can navigate how everything comes together into a functioning system that is a business.
I fully recognize that people have individual goals that may not be aligned with the above definitions and that's completely fine. I certainly don't aim to be the CTO in a big tech company like Google, a fancy unicorn startup or one of the trendy AI companies like OpenAI and Anthropic. Otherwise, I would be grinding toward that rather than writing this article...
How do we get there?
Now that there's a vague definition of progress, the next step is how to walk towards that. To clarify - you may have a different answer to what a good software engineer is. Perhaps, for you it means being great at one particular skill like software engineering in embedded systems and that's perfectly fine, the world needs such people. Some people thrive on knowing the ins and outs of low-level, high-latency computing and they will have a different definition likely tied to achieving high degree of mastery over the relevant concepts. The common goal is to learn as much as one can in the categories of skills that you find interesting. Others view their work as just a vehicle to do other things outside of their job and that's ok too but it's not what we're talking about here.
In my case it's the intersection of both tech, business and how do we structure a workflow that achieves certain goals. Probably not to the extent of leading bleeding edge tech like the ones mentioned above but still to a good-enough extent where I can solve interesting problems through tech. Awareness and self-reflection of your own goals and what you want from work/life helps a lot.
The simple and direct answer to the how question is to place yourself in environments that will allow you to learn the skills you want and need.
Here lies the crux of my main point - you need to take agency over your own career without depending on managers or companies to develop them. You need to carefully examine the environment you're in and whether there's alignment with your future goals. For example, in a previous job I spent about 5 months being on a team that dealt with urgent customer issues. I had to quickly debug, prioritize the most urgent tasks and communicate with stakeholders from other departments. I didn't learn much about good software design (or maybe I learned a bit about bad design, depending on how you view it), but I learned about communicating with stakeholders and customer-facing products. On the other hand, I once spent 3 months untangling a mess of package dependencies and improving unit tests alone. Arguably, this was a mess that shouldn't exist in the first place and didn't gain much from this experience. It felt like a tedious chore and cleanup after someone else's mess.
What you work on and what skills you'll gain is a direct consequence of the environment you're in. Particularly in software engineering, it's very difficult to define what good means but I broadly categorise it as great problem-solving and the ability to write modular, maintainable, testable code. This depends on the company you work for - does the company value these things and therefore allow you to learn them? Mediocre and companies that have sub-optimal or poor software engineering practices will not allow that. Great companies with effective ways of working and high engineering quality standard like Big Tech companies will allow you to do so - think Google, Meta, Netflix, Apple. I want to note that there are some asymmetries here - great software engineers can exist in bad environments and vice versa - not all engineers in Big Tech are great but they do set a bar for a lot of the industry.
Another asymmetry that can be observed is of course that hierarchy levels are not translatable between companies. A senior engineer at a company like AWS likely has worked on larger and more complex technical challenges compared to someone at a tech consultancy for corner shop software. A head of engineering at let's say Spotify likely is responsible for management of processes covering a few hundred engineers where a head at a mid-sized company doing the same for a team of 30 people is vastly different. This is all to say that title means nothing and the types of problems/challenges one faces in a role are dependent on the context.
Companies experience different classes of engineering challenges and if yours doesn't have or doesn't allow you to work on the type of thing that interests you, you should probably consider moving.
Gatekeepers to career progress
I really want to emphasise how the environment is much more important to your goals than your individual determination, grit, intelligence, etc. I recently re-watched the movie The Social Network which covers the origin story of Facebook and I really liked a scene where the Harvard President has a meeting with the Winklevoss twins. He tells them:
Harvard undergraduates believe that inventing a job is better than finding a job
I think this perfectly encapsulates the spirit of software engineers who want to build things and are eager to learn. Mark Zuckerberg, Dustin Moskovitz (CTO) and the other founders wanted to build things. They didn't sit around reading about building products, asking for advice or got hired by some company in the effort to learn how to build products. What they did is they deliberately placed themselves in an environment where they learned how to build products by building one themselves.
I didn't learn about the complexities and technical challenges of distributed systems by staying at my first job where we had a simple webapp with a handful of users, I learned that by moving into jobs where building distributed systems was actually happening.
In the beginning I referenced my previous article. There I talked about a class of companies and people where the Middle Management phenomenon is what really drives the company, its processes and your career development. People try to reduce complexity in a business to simple and neat boundaries between responsibilities and teams try to define hierarchies of skill. You are classified in a box according to some skill matrix that the company makes up and are judged based on. This is especially problematic with companies that try to make a separation between 'tech' and 'business'. In reality this is nonsense because you can't draw a boundary between the two. Tech is what facilitates the business. Tech is an implementation of some abstract business transaction like moving money from one account to another. One cannot exist without the other. A software engineer solves problems for a living but the Middle Manager class will have you believe that you only need to care about the tech. So why do companies still try to distinguish between them? Fundamentally, I think it has to do with people's aversion to the unknown. It's an attempt to find order in a chaotic and complex world.
One of the biggest issues corporations experience is a direct consequence of this reductive worldview - companies get separated based on function and role. Multiple levels are created with different bars to pass and progress through this hierarchy as if gaining skills and progressing through a career is a linear function where number of years correlate with how skilled you are. Have you got 5 years of experience with Apache Kafka? No? Rejected. But what if you worked at a startup that scaled from 10 to 10,000 users in the span of a year and had to face all the tricky little nuance of dealing with a distributed system? What if you had to lead a project with multiple developers, make product choices and gained a crash course in a bunch of skills as a result? The Middle Manager doesn't care - they can't see non-linear progress and they only see a list of items to test you against. They don't value curiosity, eagerness or drive.
Another fundamental problem with the Middle Manager is that due to poor team structure, arranged based around knowledge silos, they constantly need to coordinate between teams. That leads to a major problem - they don't really know the work that's happening at the lower levels, they just don't have time for it. What they do, instead, is send commands down the hierarchy. Wait...?
Politics
If they don't know what work you've been doing, because they're so busy with meetings and sending commands... How do they know if you're performing? Ah, performance reviews. Another standard practice in the business world. How does it work? It's pretty much a sales skill. You need to carefully craft a story about all the amazing things you've done for the company and how much impact it really had. Was it really true? Nobody can verify. Your next set of tasks is directly dependent on what higher-ups know and perceive about your work as well as how well they know you. If you fail at that - tough luck, you won't get to work on that interesting new project the company is taking on.
This is where politics come into play. Everyone wants to pretend like meritocracy exists and some businesses do try genuinely. What's often overlooked is the effect of relationships you have in the company. Middle Managers are not even aware of how their subordinates are gaming the system. And the people who game it are themselves are often unaware of the problem! This is another consequence of incentives, goals and performance evaluations on an individual level. In the reductionist worldview of those people they've attained a certain level of 'expert skills' and knowledge without the awareness of how incompetent they would be in a different organisation that asks them to do real problem solving instead of sending calendar invites and emails. It's only natural to want to preserve their livelihood - in today's status hierarchies, this is what drives behaviour. A Middle Manager who sees themselves as an expert would not hesitate to dismiss someone who doesn't fit their own mental model of what experience someone should have or has different ideas than theirs. After all, a new idea might be better than their way of running things and that can make them redundant. There's no incentive for them to allow it.
This is what people refer to as politics at the workplace. If you 'vibe' with your colleagues and managers, tell them a good story about all the things you've achieved, you'll do well. Maybe you're the type of person who instantly clicks with everyone and that's excellent. If you don't... you don't get to work on the interesting projects that will allow you to learn more of the things you aim towards. There are many studies about things like framing effect and heuristics that people use to make decisions. For example, one of the most common biases is that people mistake confidence with competence.
Is there a better way?
I don't know... The problems outlined above exist for a reason - they're just manifestations of characteristics that are literally a product of evolution for very specific cases. All of these behaviours are completely normal and human networks function like this in most contexts. The fact that some of those reasons are not relevant today is besides the point and you can't easily override evolution on a mass scale. Clearly the last half-century of businesses proves that companies can still thrive regardless of these systemic issues. Maybe they're just missing out on a bit more profit? Maybe the difference in potential gain isn't significant enough? One can at least hope that people in the future would learn to design systems of work and interactions with intent by creating and encouraging behaviours that are a bit closer to the meritocracy ideal everyone talks about. Until then, you have to take charge of your own career path and if you are not let through the gate of opportunities, create one for yourself.