The Practical Engineer
0001-01-01
This series has spent a lot of time saying what LLM’s aren’t and what they can’t do, pointing out pitfalls and observing limits. But, obviously, this technology can do a lot that wasn’t possible before. So to a software engineer (or leadership), what should we be trying to do with all this?
Note, this page differs in tone from the rest of the series, which focuses on observable things. The author here is moving into the realm of opinion, based on those observations.
FIM is an obvious win
Fill-In-Middle works as you write code normally. Write a few characters, and the model takes the context of your current file and project, then tries to anticipate what you want to write. Sometimes it just completes a line, sometimes it implements a loop, sometimes it can get a whole method. It’s a completely uncontroversial feature, but surprisingly overlooked by those whose introduction to all this was agentic.
The advantage here is that the practical engineer does not need to tackle problems like [[Documentation is Always Wrong]] or [[Userland is Not Enough]], and he’s actively not running into problems where [[Production Kills the Vibe]]. He is still writing code, but he’s 2-3x faster, and the code is generally of higher quality since LLM’s won’t accidentally off-by-one or use the wrong variable name. And because he is the one who’s directing where each line is going, he doesn’t need to worry about how [[Alignment is hard actually]].
That’s not very exciting, what about agentic?
Agentic is excellent at building small personal tools without supervision. The practical engineer will use it to create an editor extension, a domain-specific log parser, or some very-specific internal tool that makes his life easier. You don’t have to be very specific, you don’t have to worry much about alignment, if it screws up that’s fine. It’s a completely throwaway project, and if you’re not trying to sell it to the rest of the company, or the public, or open-source it - then doesn’t really matter how it works.
But what about production systems? What if you want to write features into, say, your payment processor? Or product-specific user data (game progression, social connections, workspace stuff, etc)? These are the beating heart of your company, outages or data loss risks your entire product and company.
Don’t explain how you want it done, show how you want it done
This should be obvious in retrospect. When you define work for other engineers, you don’t create a bible of do’s and don’t’s, you point at prior art that has demonstrated success. How should I call X service? Look at Y service, they already do it. There can be room for improvement, or perhaps the needs of each project are different and there’s some tuning to be done - but the job of high-level engineers is to develop the prior art that other engineers can see and benefit from. You write the pattern, the agent follows that lead.
src
If you can get an agent to do what you want in one turn, you’re spending less time (and brainpower) than if you had to go back and forth a dozen times with it over the course of a couple hours. A mess of markdown files and plugins won’t reduce the turns required; actual code will.
Choose a tech stack that minimizes tech debt
The practical engineer reduces the number of possible ways the LLM can screw up. The more "magic" that exists in a codebase (Spring, untyped code, annotations, config that alters the framework’s core behavior, infrastructure-injected agents that change how your program runs, complex language-specific build systems, etc), the less likely the LLM will do the right thing. LLM’s and engineers both succeed when the tech stack is very clear about how, when, and what will happen. The practical engineer knows that the best part is no part.
Use typed and compiled languages. Avoid holistic frameworks. Remove parts of the stack that are not absolutely required in every case. Create libraries, not monolithic repos. It’s ok to write more code, as long as the code helps a reader with no context understand what’s happening. The practical engineer knows that LLM’s increase velocity, make sure that speed can’t take you off a cliff.
Very importantly, understand that we’re in a moment where aguments for inertia are moot. Saying "we’ve always run on the JVM" is pointless now. Why did you do that? Was that decision made in 2005, when there were no other options? How does this decision benefit us in a post-LLM world? Many managers might cling to these old stacks, because it’s what they did in their youth. And because [[Software Managers have Dunning-Kruegar]] they will be tempted to keep everyone on something they still understand. This is not the time to indulge that.
