The Problem
In my short but lively engineering career I’ve encountered situations where peers, leaders, and myself jump into solutions too quickly. I don’t blame them. How could you?
Over time, I’ve realized that spending more time defining the problem leads to a solution faster than if that time were spent architecting the solution. In my experience I’ve seen two frustrating scenarios when this is neglected:
Scenario 1: A frustrated engineer
An engineer is tasked with a vague product problem and immediately jumps into the solution. They spend a few days coming up with the perfect solution. Excited they present their work to the team only to be met with rigid push back. Concerns are flagged and they’re encouraged to pursue alternatives.
Over the next few days they investigate alternatives and present again only to meet rigid push back once again. The result? A frustrated engineer and inefficient time spent. Few iterations may result in the project being dropped, TL stepping in, or an escalation upwards.
Scenario 2: A micro-managing leader
A leader is delegating problems to engineers. Without defining the problem properly its given to an engineer
The leader patiently waits for the engineer to come up with a solution. A week later the solution is presented but it’s unclear whether it will solve the underlying problem. The leader becomes anxious and starts to post more frequent check-ins with the engineer with good intentions. Eventually a passable solution is aligned on. But the result is an engineer who is frustrated by a micro-managing leader and a frustrated leader with their engineer’s performance.
Takeaway
I’ve been in both situations on both ends of the stick. It doesn’t feel great. The misalignment of effort and results quickly leads to growing frustration, tension, and burnout.
Spending time on the problem might be unnecessary in situations. But counter intuitively I’ve found that spending the majority of your time on the problem and less time on the solution often leads to alignment faster and to a more elegant solution.
What is a Problem?
I like to think of a problem as a set of constraints and assumptions that delineate a solution space — the set of all possible outcomes that satisfy those constraints. The more precisely a problem is defined, the more simple it is to navigate and understand the solution space.
Each solution within the solution space has a different value function. A value function here is some arbitrary value function.
On the note of local vs global optima. I consider one aspect of seniority in an engineer to be a function of the distance that they can explore within a solution space. Assigning the wrong solution space to the wrong engineer will lead to them maximizing towards a local optima, instead of a global one. This is why delegation is hard and is a core skill of a good leader.
For example, consider a recent problem of mine working at a startup:
We had recently built out an MVP and were gearing up for production launch. The question was “how can we solve any bugs that appeared in production?”
Here’s the initial constraints
- We need to be alerted for every error and paged if error count reaches past a certain amount
- We need to be able to dive into the bugs.
- Our backend operates on GCP with serverless functions
- Our frontend operates on Vercel as a Create React app.
There’s a variety of solutions that could work here from self-hosted solutions to SaaS. Applying another set of constraints:
- Timeline is tight. We only have a few days to implement with limited bandwidth (a single engineer)
- We have a budget of $1,000 per month
Now we’re talking! We can throw away any self-hosted and custom solutions. Now we’re left with SaaS solutions since we can pay $ to accelerate. But do we use GCP and Vercel’s built-in logging and have fragmented logging or try to unify with a centralized reporting dashboard?
Further iteration of the constraints of the problem will lead you to the answer. In many cases refining the problem will have you answer tradeoff questions early, like budget vs timeline above. These are meant to be tough questions. The key is to set expectations with stakeholders early and align before diving into a solution.
Who can define the Problem?
Constraints are informed by
- Context of the current systems and tech debt
- Goals of the organization
- Direction or vision of the organization [vision definition by Will Larson].
The richer context one has, the better they can define its constraints and assumptions.
A Framework
Consider the following iterative framework to define a problem:
- State the Problem: Clearly articulate what needs to be solved, avoiding vague or overly broad statements.
- Identify Constraints: What are the limits (e.g., resources, time, technical feasibility) within which a solution must operate?
- Seek Context: What additional knowledge or context can help narrow the solution space?
- Iterate: Revisit the problem definition as new information emerges or as initial solutions reveal hidden complexities.