I've found that the job title AI Engineer to be pretty murky at the moment and it gets conflated with several other similar but different titles, so I'd like to describe how I think about the role of an AI engineer. Answering the question, what do AI engineers do?

The Short Answer

AI engineers are software engineers that learned the new skills and toolset for building software that leverage the power of LLMs behind the scenes.

The Long Answer (what you're looking for)

What It's Not

This is where I see quite a bit of divergence in job postings and reddit threads. There’s no single definition yet, but we can agree on what it isn’t:

• Not ML research — you don’t need a PhD or to train models from scratch.
• Not data science — the focus isn’t analytics or dashboards.
• Not just prompt engineering — prompts are part of the toolkit, not the whole job.
• Not MLOps — you’re not running training pipelines or infra.
• Not plain backend — it builds on backend skills but adds new tools like retrieval and evaluations.

Mostly Backend Engineering

I'd say 80% of the skills needed for AI engineering are just plain old backend engineering skills. At the end of the day were still building software the same way we always have. I like to think about this new evolution in a similar way to leveraging an external API or microservice within an app, say payment processing. Just because we decided to outsource payment processing to stripe doesn't mean the job is done, obviously. We still have to build the application and glue code, but we can outsource the business logic complexity to the external service.

Leveraging AI is the same in a lot of ways, you outsource parts of your application logic to AI (usually via API calls) to increase power andflexibility while also reducing development and maintenance time.

Build Lean Flexible Software

The biggest shift with AI isn’t that we suddenly write software differently — it’s that business logic itself becomes variable, stored in prompts and context instead of code. That makes software far more flexible and reactive. Let’s unpack that.

Why Flexibility Matters

Building and shipping production software takes time. Making changes takes even longer — code reviews, deployments, migrations, tests. Engineering orgs spend enormous energy trying to shorten this cycle (CICD pipelines, isolated testing, AI-assisted IDEs, code review bots, etc.).

One way to accelerate development is to avoid code changes altogether by moving logic into data. This is the foundation of flexible software.

Data vs. Code: A Simple Example

Let's look at an intentionally basic example - a trivial content filter that checks for banned words. While overly simplistic, it illustrates our point about storing rules as data vs code:

• Option 1: Hard-code the rules (the inflexible way):

def check_content(text):
    # Every rule change needs a code deployment 🤦
    banned_words = ["spam", "scam", "offensive"]
    for word in banned_words:
        if word in text.lower():
            return "rejected"
    return "approved"

• Option 2: Store rules as data (the flexible way):

def check_content(text):
    # Rules can be updated instantly via database 👍
    rules = db.get_banned_words()
    for word in rules:
        if word in text.lower():
            return "rejected"
    return "approved"

Yes, this is a super basic example (real content moderation is far more complex!). But it demonstrates a key principle: Option 1 needs a full code deployment to update the banned word list, while Option 2 just needs a quick database update. Both work, but one is more flexible because the rules live in data, not code.

The Pre-AI Toolkit

This same principle has guided software for decades:

• State lives in the database.
• Logic lives in the code.

By combining the two, we built flexible applications that could adapt without constant redeployments.

The AI Shift

Now, with LLMs, we have a new layer of flexibility. Prompts and context can serve as variable business logic:

• Prompts = the rules of operation (logic).
• Context = the inputs/state that logic operates on.

Instead of coding every condition, we can offload parts of the logic to a model. The application code becomes leaner — just the glue that connects database state and prompts/context to the LLM.

Here's how software flexibility has evolved:

The Payoff

This is what makes AI engineering different. By treating prompts and context as first-class application state, engineers can build lean, adaptable systems that evolve faster than traditional code-heavy software.

New Tools, New Challenges

As more of the software is driven by prompts, context, and calls to LLMs we needed new tools to solve the new challenges. AI engineers need to get up to speed on these new tools and techniques.

• Tool calling — extend LLMs with custom functions.
• Prompt engineering — use effective techniques to get reliable results.
• Model selection — balance cost, performance, speed, and quality. Every model has different characteristics.
• Context search (RAG) — retrieve knowledge via embeddings and vector databases.
• Evals — test and monitor non-deterministic logic.
• Agent frameworks — frameworks for abstract and orchestrate the complexity above.

Why It Matters

If you’re a software engineer today, learning these AI engineering skills is how you stay ahead.
You don’t need a PhD or deep ML research background — you already have 80% of what you need in your backend skills. The difference is picking up the new tools (prompts, context, retrieval, evaluations) and learning how to use them to build flexible, production-ready systems.

The engineers who adapt will be the ones shaping the next generation of software.

Summary

AI engineers are software engineers who combine core backend skills with new tools for working with LLMs. They don’t train models from scratch—they build production-ready systems that use prompts, context, and APIs to outsource parts of business logic.

The result is leaner, more flexible software that adapts faster and delivers more power with less code.