My AI Journey: Why I Keep a Human in the Loop

My AI Journey: Why I Keep a Human in the Loop

Posted
By Nathan
13 min read
My AI Journey: Why I Keep a Human in the Loop

Like many people, I use AI tools every day. I enjoy experimenting with everything from frontier models to smaller models that I can run locally, and AI has become a significant part of both my work and personal life.

I have been using these tools since the release of ChatGPT with GPT-3.5. I remember being blown away by the capabilities of this seemingly intelligent chatbot. My curiosity soon led me down a path of trying to understand more about what was going on under the hood of large language models, usually shortened to LLMs.

Since then, I have watched the industry change at a remarkable pace. AI now shapes conversations everywhere from the workplace to mainstream news. The genie is well and truly out of the bottle, and there is no going back.

I believe AI will sit alongside the most significant technological breakthroughs in human history. The race to dominate this new intelligence era is already under way, and the models are only going to become more capable and influence more of how we live and work.

The consequences of that acceleration are still uncertain. I remain hopeful about the scientific and societal breakthroughs AI could help us achieve, but I am not naive about the dangers. AI-assisted warfare, cyber security, misinformation and the concentration of power are concerns we should take seriously.

For me, that makes the human role more important, not less.

The Moment AI Became Mainstream

Artificial intelligence as a field has existed for decades, and LLMs are only one part of it. However, AI has become a catch-all term for tools such as ChatGPT because ChatGPT was the product that brought this technology into everyday life.

I remember being on a train to work one morning in late 2024. Instead of everybody doomscrolling through TikTok, I could see several people using ChatGPT. That was a small moment, but it stuck with me. AI was no longer something confined to technical people. Ordinary people were making it part of their day.

Around the same time, I spoke to somebody who worked in sales. He told me that he got ChatGPT to do everything for him: emails, reports and marketing campaigns. He showed me examples of his chats on his laptop, and when I asked how it had changed the way he worked, he said that he felt more organised and confident.

That answer helped me understand why ChatGPT had people from so many different backgrounds hooked.

  • It removed the fear of the blank page and gave people somewhere to start.
  • It made specialist help feel instantly available without requiring specialist software skills.
  • It could turn rough thoughts into something structured and presentable within seconds.
  • It responded without judgement, which made it easier to ask questions people might otherwise keep to themselves.
  • It gave people a feeling of momentum, confidence and control over work that had previously felt difficult or time-consuming.

The conversational interface also mattered. You did not need to learn a complicated application. You could simply explain what you wanted in ordinary language and receive an immediate response.

Of course, convenience can easily become over-reliance. Seeing how other people used these tools made me think much more seriously about my own relationship with AI.

What Did I Actually Want From AI?

I kept seeing people use AI in creative ways, with well known users in the AI industry posting on X about new tools, workflows and use cases. It made me ask myself three important questions:

  • What do I want from this relationship with AI?
  • How am I genuinely benefiting from these tools?
  • What does all of this mean for my career?

For a while, I did not have clear answers.

It seemed as though a new tool appeared every day, and it was incredibly difficult to keep up. AI was becoming its own attention economy. The fear of missing out was real, and I did not want to fall behind.

I became obsessed with finding the best tools. I wanted to try them all, and I did try a lot of them. Some no longer exist, which was always a risk when many early products were little more than thin wrappers around frontier models from companies such as OpenAI and Anthropic.

“Ship fast” became the dominant message. Rapid improvements in model intelligence made it possible to build products quickly, but they also created an unforgiving market. If a company did not move quickly, it risked becoming obsolete. Timing the market and being first seemed to matter at all costs. Some products crashed and burned, while others adapted and thrived.

Watching that happen was exciting, but it was also exhausting.

Getting Lost in the Language of AI

Then there was all the terminology you were apparently expected to know if you wanted to be considered an expert:

  • Prompt engineering
  • Hallucinations
  • Tokens
  • Context windows
  • Transformer models
  • Retrieval-augmented generation, or RAG
  • Parameters
  • Fine-tuning

Each term describes something useful, but together they can make AI feel far less approachable than it needs to be.

I became particularly interested in prompt-engineering best practices. I wanted to craft the perfect prompt, and in some cases I did get better results by being more deliberate. The problem was that I sometimes spent more time designing the prompt than learning about the tool or completing the work.

The paralysis of trying to get the prompt exactly right was real for me.

I created a custom GPT to help improve my prompts, and I even built a prompt library to store the ones I wanted to reuse. Those experiments taught me a lot, but these days I do not worry nearly as much about prompt craft. Modern models are much better at understanding natural instructions than the earlier ones were.

Context is still extremely important. So are clear expectations, verification and iteration. However, I now usually talk to a model in plain language, explain what I am trying to achieve and refine the result through conversation. That has served me well and removed much of the stress of trying to design one perfect prompt.

Moving From FOMO to Conscious Use

So where am I now?

I am much more relaxed. I have stopped obsessing over every model release, benchmark, tool and new piece of terminology. I know I cannot try everything, and I no longer think I need to.

Instead, I try to be more conscious about how I use AI. I focus on getting hands-on with tools that provide a real benefit to me, my learning and my career.

Codex is my AI harness of choice for most of my work, so I will refer to it throughout this post. However, the principles apply to any AI harness or LLM.

As a DevOps engineer, much of what I do is knowledge work. I investigate problems, interpret evidence, understand systems, compare options, write documentation and explain technical ideas. Access to frontier models through Codex has dramatically improved my productivity and confidence.

Most importantly, it has helped me learn new skills and retain knowledge.

That final point matters more to me than simply completing tasks faster.

AI Should Support My Thinking, Not Replace It

I have always believed in keeping a human in the loop. I see Codex as a tool that helps me get things done more efficiently, but I do not want it to replace my thinking.

I love learning new skills, and Codex has helped me understand concepts in a way that suits my learning style. I can ask follow-up questions, request a simpler explanation, test my understanding and approach the same idea from a different angle. That has dramatically improved my ability to retain what I learn.

The things I learn will often become runbooks in my Obsidian vault. I review everything and make sure I understand it before it earns a place there. I wrote more about that system in Why I Switched to Obsidian.

Codex has read-only access to my vault. That means I can ask about something I have previously documented and it can find the relevant notes, giving its response context grounded in my own knowledge. Read-only access is an intentional boundary: it can use the vault to help me, but it cannot quietly rewrite the knowledge I rely on.

That is the key for me. AI is most useful when it strengthens my ability to think, learn and act.

Codex challenges my understanding and helps me look at problems from different perspectives. It can feel like having somebody available at any time to discuss an idea, but the relationship still needs clear expectations and boundaries based on what I want to achieve.

I also need to push back, challenge assumptions and verify responses. Sometimes I need to step away from the machine and process an idea for myself. Human thought is not an inconvenience to be automated away. It is the part that gives the work purpose, judgement and accountability.

What a Human in the Loop Actually Does

Keeping a human in the loop does not mean manually doing every task or watching every generated word appear on the screen. It means keeping human judgement at the points where it matters.

In my own workflow, that usually means:

  1. I define the outcome. I decide what problem I am trying to solve and what a useful result should look like.
  2. I provide the context. I point Codex towards the relevant files, documentation, constraints and previous decisions.
  3. I set boundaries. I decide what it can access, what it may change and which actions need approval.
  4. I challenge the response. I ask questions when something seems vague, surprising or too confident.
  5. I verify the evidence. I check commands, sources, calculations, changes and test results rather than accepting a confident answer as proof.
  6. I make the final decision. The consequences belong to me, so the decision must remain mine.
  7. I learn from the process. I make sure I can explain the result and capture useful knowledge in my own words.

The amount of oversight should also match the risk. Asking AI to suggest a title for a blog post is very different from letting it change production infrastructure, handle private data or send a message on your behalf.

The higher the consequence of a mistake, the stronger the checks, permissions and approvals should be.

The Risk of Giving Away Our Agency

There are many people who worry that AI is weakening our capacity to think. I understand that concern, particularly when somebody hands over an entire task without applying any judgement of their own.

We have all seen social media posts where somebody has clearly copied an LLM response without reading it. In the early days of ChatGPT, the words “As an AI language model…” appearing in a supposedly personal post became a familiar giveaway.

It was funny, but it also exposed a serious problem. The person had published text in their own name without taking responsibility for what it said.

We have probably all encountered somebody who treats a model’s answer as gospel but would struggle to explain the same idea without the tool. This is where AI becomes dangerous without a human in the loop. A fluent response can sound authoritative while being incomplete, outdated or simply wrong.

If we stop questioning the output, we give away more than a task. We give away our agency.

This is why I believe it is our responsibility to educate ourselves, verify responses and actions, and use these tools thoughtfully. AI literacy is no longer only for people working in the industry. Anybody using these systems should have a basic understanding of what they can do, where they can fail and when extra care is required.

You Do Not Need to Keep Up With Everything

AI education is important, but I know how hard it is to keep up. The field moves quickly, and the pressure to know every new tool or piece of terminology can make people feel as though they have already fallen behind.

You have to accept that you will not have time to try every tool. It is also fine if you do not yet know all the terminology. We all have to start somewhere.

My advice is to choose one useful tool, get hands-on and learn as you go. Start with a task you already understand so that you can judge the quality of the result. Ask the model to explain its reasoning in plain language, check important claims, and make sure you can describe what you have learnt afterwards.

Even Andrej Karpathy has written about the challenge of keeping up, which is a useful reminder that nobody has a complete view of such a fast-moving field.

If you want to build a foundation, there is a great deal of free material available. I have found the following useful:

The aim is not to consume everything. It is to learn enough to use these systems with confidence, curiosity and appropriate scepticism.

Final Thoughts

AI has already changed the way I live, learn and work. I expect its influence to grow as models become more capable and the tools around them become more autonomous.

I remain optimistic about what that could mean for scientific discovery, productivity and access to knowledge. I am also conscious of the risks, from convincing misinformation and cyber attacks to people gradually surrendering their own judgement.

For me, the answer is not to reject AI or hand everything over to it. It is to build a healthy working relationship with it.

I want AI to help me think, not think for me. I want it to help me learn, not create the appearance of knowledge. I want it to make me more capable without making me less responsible.

That is why the human in the loop matters. The model can generate, analyse and suggest, but the human provides the purpose, boundaries, judgement and accountability.

Use the tools, experiment with them and allow them to challenge you. Just do not remove yourself from the process.

ai llm codex productivity learning
This post is licensed under CC BY 4.0 by the author.