Skip to main content

The middle way for LLMs

Depending on your circle of friends and day-to-day environment, LLMs can be a controversial topic. In the geospatial tech world I inhabit, we have seen communities such as QGIS and GDAL implement outright bans on their usage, due to the harms inflicted on long-standing open-source projects. Depending on your beliefs, discussions around LLMs are either trite and mentally fatiguing, or about the best thing since sliced bread…

As somewhat of a pragmatist, I’ll try to cover some of my reflections over time on this topic.

Shovels and excavators

As mentioned in the Responsible AI Guide and my follow-up post here, there are many concerns around using these tools in the wild. Amongst the increasing number of risks, we have the cybersecurity threat being played out in real-time at the tail end of 2026.

A friend sent me an interesting article by Baldur Bjarnason recently, which explains how standardisation of development around ‘popular frameworks’ makes the role of a skilled software developer easily replaceable with less skilled developers, in particular from developing countries, and in recent years by LLM agents. While I wouldn’t be entirely opposed to more of the software industry shifting to developing countries, giving developers elsewhere in the world a chance to shine, I agree that this pattern does put more power in the hands of managers / organisations, as they assess positions for their ‘LLM replacement factor’.

He makes a useful distinction between tools that act like a shovel and tools that act like an excavator. Some tools need more expertise to use well, and so reinforce the value of the skilled person holding them. Others remove the need for expertise altogether, and so make that person interchangeable. The first kind resists labour arbitrage, while the second enables it. It’s worth noting the article is from May 2024, before agentic coding tools went mainstream, and if anything the intervening years have proved his point: the industry has largely pushed LLMs towards the excavator end, sold on the promise of replacing developers rather than augmenting them. This distinction is the crux of what I’ll call “the middle way” later on: using LLMs in the way that requires expertise, rather than the way that replaces it.

The middle way

While I would love to agree with the viewpoint that LLMs, though not useless, aren’t worth the harm they cause, I’m sad to say I think it’s not the whole story.

I wish we could uninvent these systems, and roll back their proliferation worldwide. Unfortunately we can’t. Of course, unfettered usage of these tools without a critical look into their functioning and issues is a bad thing. But I think it’s pretty clear to many working in software development by now that these things are here to stay (in some form or another), and are pretty darn useful in many scenarios (see the appendix for where I’ve found them helpful, and harmful). Now that to ‘ChatGPT’ a question is as embedded in our vocabulary as to ‘Google’ one, I think we have lost the race. And I just can’t see a world where governments get their act together, and successfully regulate them out of existence. So rather than asking whether LLMs are worth the harm in the abstract, I think the more useful question is: how do we use them so that the value we get out actually justifies the cost?

For me, the answer comes down to staying in the driver’s seat: control of the decisions taken, and control of the tools themselves.

Control of the decisions

In my opinion, if the LLM is mostly implementing your decisions, about architecture, trade-offs, what to build and what not to, then your expertise is still doing the real work. Someone simply letting the LLM decide will produce worse results in the long term (maintainability matters, despite what management might think). This is what keeps your skills valuable, and makes your contribution genuinely good rather than adding to the pile of slop.

Documentation is a good example. An LLM can do a decent job of explaining what code does. But why it was built that way is firmly in the domain of the engineer architecting it, and worth writing down, so the next AI crap cannon that comes along doesn’t make a PR trouncing over the design.

Of course, managers will still buy excavators, however carefully you wield your shovel. And that raises an uncomfortable question about the next generation: if the entry-level work goes to LLMs, where does the struggle that builds expertise come from, now that communities like StackOverflow, where many of us learnt, are drying up?

Control of the tools

Even if you are taking every decision yourself, relying on the API of some multinational corporation undermines your position: when your power can be stripped away at the click of a button. Owning the whole chain, from the model to the hardware it runs on, is a far stronger position. It’s in the spirit of the local-first software movement, which I can really get behind.

2026 was really the year that self-hosting LLMs became viable. Open-weight models, whose trained weights are published for anyone to download and run (unlike the locked-down proprietary models behind ChatGPT and co.), have caught up fast. Qwen3.8-27B and Gemma 4 run well on a single consumer GPU or a decent laptop, and Alibaba’s Qwen3.8-Flash-Next brought something close to frontier coding ability to a well-specced home machine. In reality though, the bigger models still involve owning quite a lot of compute power, in particular expensive GPUs that are now in short supply (unless you own a latest-gen MacBook, and can run a model quite efficiently using its unified memory). Unless you are a power user, this is slightly out of reach for many consumers, so for now we must rely on these godawful companies to provide their proprietary “frontier models” to us via API.

That said, not every task needs a frontier model. Picking the appropriately sized model for the job is part of using these tools skilfully (in the words of the teacher on a meditation retreat I attended recently).

I am biding my time, keeping abreast of open-source / open-weight model developments. As APUs become more prevalent into 2027 and beyond, self-hosting on laptops etc. will start to become a much better option. There are also many parties interested in this, in particular China and the EU, who are pursuing their own AI sovereignty, to remove reliance on US-centric companies. Mistral’s Large 4 is a good example: an open-weight European model competing with the frontier, albeit far too large to run locally. In the long term, I hope we can all have access to good local AI that works, without reliance on external vendors, and in particular, lock-in to an enshittified service from providers that are beholden to market forces and have no moral scruples.

Closing thoughts

So that’s my middle way. Acknowledge how useful these tools can be in certain scenarios, and push for responsible usage within certain guardrails, with a degree of ethical satisficing. Keep your skills, so you stay in control of the decisions. And stay abreast of developments in the sector, as we try to unshackle ourselves from the digital barons in control of our data and services.

I think the same question of control comes up when we talk about AI sovereignty, in particular in developing countries. Without it, they risk the worst of both worlds: locked into renting their AI from US companies, repeating the dependency of so many misguided tech for good projects, while being treated as the expendable labour that AI is here to replace. Avoiding that means upskilling developers globally to use LLMs well, but on a solid foundation of software engineering (struggle and all), so they’re the ones wielding the shovel rather than being replaced by the excavator.

Appendix: where LLMs help (and harm)

For reference, here’s a rundown of some common applications of LLMs, and how useful I’ve found them in different scenarios.

Note

First of all, I would like to demarcate two particular types of software I am interested in, for the purpose of this discussion:

  • (A) Mission-critical open-source software, such as the Linux kernel, libraries integrated into many machines globally (e.g. OpenSSH), low-level performance-critical programming, embedded systems that run important hardware, etc. Importance of fast iteration is low, while the risk of regressions and bugs introduced is high. They generally exist for the long term, including multi-decade projects. A good example could be GDAL.
  • (B) User-facing applications, such as web and mobile apps. The consequences for these things failing are generally pretty low stakes (a few disgruntled users), and the benefits of appropriately fast iteration are generally pretty high, with (if approached realistically) improvements directly benefiting users. These projects come and go, with user whims and as the usefulness of particular projects rises and falls over time. As examples, we pretty much have every tool we work on at HOT.

1. Software engineering

Effectiveness:

Possible harms:

2. Software debugging

Effectiveness:

Possible harms:

3. Writing technical documentation

Effectiveness:

Possible harms:

4. Writing other documents

Effectiveness:

Possible harms:

5. Learning, exploring, prototyping

Effectiveness:

Possible harms:

6. Translation

Effectiveness:

Possible harms: