May 6, 2025

Augmented engineering: learnings of an AI skeptic

Since ChatGPT burst onto the scene in November 2022 it and its competitors have taken software development by storm. We've seen companies go all in on AI and set expectations that new hires will only be prioritised if AI can't do the job. This has caused a general level of anxiety amongst software engineers that their roles and disciplines could be under threat.

Avoiding the initial hype train

I mostly watched the bandwagon grow from a distance; staying focused on my work without a real desire to reach into the AI world. My early experiments were not fruitful, the code generated by AI (by which I mean ChatGPT, Claude, Gemini - I tried all of them!) was mostly OK, but I lost just as much time to tracking down subtle bugs in their generated code than I saved having them generate code. Of course, this assumes that I never write subtle bugs in my own code, so one could argue that it was still a time saver. But I didn't see the magic that others were seeing in their day to day workflows.

Dipping my toes with curves and canvas

The first time I began to see a productivity boost was from working on my browser based game OnTrack. The game is represented as a 2D map drawn on a canvas, and I wanted to draw nice curved track pieces. The problem was that I had long removed my school trigonometry lessons from my head, and I wasn't about to open that can of worms - at least, as much as I could avoid it!

On a whim I decided to ask ChatGPT and see if it could give me a head start. So I asked it (in one chat) the following 3 questions (I've omitted some of the response to focus on the important parts):

What angles do I use to draw a 90 degree curve that goes clockwise from 12 o'clock to 3 o'clock: So, you would start your curve at 0 degrees and end it at -90 degrees to achieve the desired effect.
What is 0 degrees and -90 degrees in radians? The curve from 0 degrees to -90 degrees is equivalent to a curve from 0 radians to -pi/2 radians.
Can you give me the HTML canvas code to draw this?

And the code it gave me...worked! In about 2 minutes I had drawn a curve. This is code that I could have got from an example on MDN but the biggest productivity boost now was what I could do next. Because my chat with ChatGPT contained this context, I could now begin to feed it more information about my game, how the 2D grid is structured, and have it help me further.

Over the course of the next hour I worked through the problem, telling ChatGPT about:

How my grid is structured and stored in memory.
How I represent pieces of track in code - which side of the tile it starts and ends on.
The parameters required to adjust the size of the curves (for example, depending on the user's zoom level the width of the curves changes).

And asking it to take the example code for one curve and build a generic solution.

The code it produced still needed some tweaks to fit into my codebase, but at the end of it I was left with the ability to draw curved track in any direction and it also helped me understand logically about arcs and radians to a level that would have taken me much longer via traditional resources like StackOverflow, MDN and others.

Automating the boilerplate

It was around this time that I began experimenting with AI auto completion in my editor. I hadn't been compelled by the demos where you type // write me a function that does X and have it completed by AI; I found unless you were writing a very generic function, its implementation would not be specific because it was lacking the context of your codebase and application.

This still remains a challenge; I work on a large enough codebase at work that means we can't load it all into a model's context window (yet?), but what I have found is that AI excels at repetitive code changes.

During the implementation of curved track drawing for OnTrack, I had to write a lot of code that looks like this:

if (entryDirection === 'EAST' && exitDirection === 'NORTH') {
  // draw the curve with the right angles
} else if (entryDirection === 'NORTH' && exitDirection === 'WEST') {
  // and so on
}

And once I'd written one of these branches, I started getting completion from Copilot from the rest, and it was accurate! It was even able to infer the right context.arc calls in the canvas code and adjust the angles correctly for each direction.

Repeated boilerplate with clear patterns is where AI shines for me right now. This is what I want to automate away; the mundane, repetitive code changes that I have to spend time on. I want to focus my energy higher up on design patterns, architecture and connecting all the parts of my system together.

Overall with ChatGPT to answer my initial questions and an AI auto complete in my editor, I estimate it took 2-3 hours to build my curved track implementation. I am very confident it would have taken at least 5-6 hours without any AI assistance.

AI is a confident rubber duck and search engine hybrid

The key to unlocking productivity gains with AI was to figure out the scenarios where it shines. I like thinking about AI as a very powerful but slightly flawed rubber duck which is there to provoke my brain, particularly when I'm at an impasse or feeling creatively blocked. I've also noticed that it has nearly exclusively replaced search engines for me when programming.

Here's some scenarios where I'll routinely reach for AI:

When I'm starting a new piece of work and I'm not sure on the best implementation LLMs can really get me off the ground. In this instance I like to ask the AI to give me options (something that Simon Willison suggests in his post on how he uses LLMs) to solve the problem. It will often suggest things that I haven't thought about. What's great here is that I don't have to worry as much about the hallucination problem. I don't care if the AI isn't perfectly accurate as I'm using it to spark ideas and kick off a process.
When I know I need a particular solution, but I can't remember some detail, or I want an example implementation to get me started. This is particularly true in languages I am less familiar with. I write a fair amount of Lua to configure my dotfiles and I'm still getting up to speed with the syntax and the data structures. "Write a function to loop over a table and find the first match" is a question I would never ask if I was working in JavaScript, but in an unfamiliar or new language it's very helpful.
For one-off scripts or prototypes AI shines. I think this use case is very powerful because it removes some of the inherent weaknesses or pitfalls of using AI. These scripts often have no existing context because they are new, or self contained. I typically care slightly less about the code quality and style for helper scripts - and for prototypes I don't care at all. My what.mjs script, which helps me look up command line aliases I know I've defined, was written almost entirely via AI and prompting. The caching logic, which takes a bunch of files, calculates hashes for them, and combines them into a cache key, was entirely AI written. Sure, I could have written this myself, but I couldn't remember the exact APIs (I rarely work directly with CLI interfaces now) and because I knew the code wasn't going to be reused elsewhere, I didn't care too much about its interface or implementation.
For figuring out obscure command line interface arguments. I love using jq for parsing out big JSON files (I work on Performance Tracing in Chrome, which get delivered to DevTools as very large JSON arrays) but I can never remember the syntax and how the arguments work. But now I don't have to! "Give me the jq command to find the first 10 events with a args.data.url property of example.com" is pretty flawless. And if it doesn't work, I feed the error back into the AI, and it can often try again and spot its mistake.
To review my work as a virtual spellchecker or code reviewer. I love using AI to provoke my brain and asking it to review my work is a great way for it to give me a bunch of things to think about. Some of them won't make sense, some I might disagree with, but if I find one I like and action, it's easily worth the 20 seconds to ask the question.

Embrace the mistakes

In an episode of The pragmatic Engineer podcast with Simon Willison Simon mentions that anecdotally most developers he knows who have written off AI do so because they try it and find that the output wasn't flawless. They are then reluctant to embrace it into their workflow because they do not trust it.

I can relate to this, because this was my early experience. Even now I still will accept an autocomplete suggestion before realising that it wasn't correct. But I think that developers who think that AI is useless because they cannot trust it are missing a trick; if you assume you cannot trust AI, it becomes a very powerful tool.

The output from AI presents itself confidently and this is the crux of the problem. It continues to frustrate me that all the major AI surfaces contain the tiniest X may make mistakes in the footer, and I wish this fact was more prominently presented to users. However, once you know and understand this, you can use it with a freedom because you accept it will provide guidance and starting points not completed, perfect solutions.

Remember, at this point AI is a very fancy auto-complete. AI has no fundamental understanding of the concepts you are asking about.

This is why I rarely ask AI to write tests; I trust neither the AI nor my own review process to catch every subtle error in generated tests. I don't trust the AI or myself to generate and review tests without missing a mistake. And you don't write tests for your tests, so there's no safety net. This is the reason I feel more content letting the AI write a basic solution, and I write the required unit tests. What I will do though if ask the AI to generate a list of test cases it thinks I should consider. Most of the time it will suggest at least one that I haven't thought of.

If you are a software engineer reading this I am willing to bet that you have googled a problem, copied an answer from Stack Overflow, got frustrated that it didn't work, and fixed it yourself. You'll have to do this lots with AI too, but the starting point will be far superior.

How to embrace an AI augmented workflow

If you're reading this blog post and it's inspiring you to revisit AI in your daily work, here's the strategies I've used to make it work for mine. Everyone works differently, so see these as a starting point to think about rather than a strict guide you must follow.

Get it into your editor. I use a combination of codecompanion.nvim and parrot.nvim to integrate AI into Neovim, my editor of choice. But there's a lot of options here for every editor, so try a few and find what works best. Or, alternatively, don't! Lawrence Jones, an ex-colleague and fantastic AI engineering blogger and I discussed how he prefers to use AI in the browser, finding the "gap" between his editor and AI helpful for him. Whilst this doesn't work for me, I will often fire up a chat interface in Chrome rather than in my editor sometimes.
Experiment, lots. It's taken me a long time to build up a very small sense of intuition around how best to prompt these things. They remain a black box in many ways. You'll find that certain models respond best to certain styles of prompts. I've found that formatting is very important - a well formatted prompt (Markdown seems reliable) will really help it. And be specific - another tip from Simon Willison - when you want a very specific output. By trying different strategies and engaging with AI at different points, you will build up experience and intuition about what works, and what doesn't. I always refer back to Joshua Wood's blog post on LLM debugging as an inspiring read of one case where things didn't go entirely to plan, but the author was able to reflect and learn from it.
Read and watch how others do it. I've found a lot of tips out of seeing others embrace AI to be inspiring and helpful. Simon Willison, Lawrence Jones and the Incident.io blog have been a large source of content, but I'm sure there are many others (please share them with me!)
Don't trust it. If it generates code for you, review that code incredibly carefully, and test it, lots. Check its answers seem reasonable and use your common sense when blindly copying and pasting its output.
Share with your colleagues. On my team at work people often do presentations or share examples where they found the use of AI a major productivity boost. People also show examples of it getting it completely wrong! Compare notes and tricks from your colleagues; you may even be able to work together to help the AI understand your particular product and codebase to improve its suggestions for all of you.

Embracing AI augmented engineering

AI is here to stay in software development. We don't know long-term what it will look like, and we don't know how capable they will be or where their limits are, but it's clear that even if no progress was made from this day forward, the tools provide clear benefit.

The immediate threat to engineering jobs is likely overstated but what's clearer is that in a world where AI can offer tangible productivity benefits - provided you use it thoughtfully and critically - staying on the sidelines means potentially missing out. Don't make the same mistake I initially did; consider how these tools can augment your workflow and embrace them.