The AI-era software paradigms every engineer should know

The hype around the AI revolution was already picking up speed before I began my software engineering career. The first time it came onto my radar was almost 10 years ago, when DeepMind’s AlphaGo beat the world champion in a game of Go. At first, it felt distant from my daily work as a software engineer - primarily adding APIs for computer vision, natural language processing and some content generation.

But in this past year it made another big leap, and this one seems truly revolutionary. What began as a “smarter” autocomplete and test implementer quickly evolved into a good-enough coding assistant, now becoming fully fledged independent agents showing signs of autonomous problem-solving and decision-making. While still expensive and flawed, the rate of improvement is stunning. Even if progress stops now, it’s clear that the effects on our profession are huge.

These grand leaps forward not only change the way code is written, but are reshaping the nature of software itself. This was explained well in Andrej Karpathy’s recent talk “Software in the era of AI”, which I highly recommend watching. Before discussing its content, It’s worth briefly introducing Andrej Karpathy: he is one of the leading figures in the field of modern AI - PhD specializing in neural networks, one of the founders of OpenAI, lead the computer vision team in Tesla Autopilot (full self-driving) and currently works on an AI+Education startup. He also recently coined the term “vibe coding” in a now-famous tweet:

There's a new kind of coding I call "vibe coding", where you fully give in to the vibes, embrace exponentials, and forget that the code even exists. It's possible because the LLMs (e.g. Cursor Composer w Sonnet) are getting too good. Also I just talk to Composer with SuperWhisper…

— Andrej Karpathy (@karpathy) February 2, 2025

His lecture to students starting their career gives a convincing argument against the “software engineering is dead” doomsayers, showing the many opportunities large language models offer, and shares insights on designing LLM-based applications.
In this blog post, I’ll summarize Andrej’s argument about the fundamental change in the nature of software from the view point of a “traditional” software engineer, and highlight the emerging building blocks and paradigms which power AI-era software.

Neural networks are a completely different type of “software”

Andrej defines three “types” of software: the classic code that programs computers he calls “software 1.0”, the weights which “program” a neural network through deep learning techniques he calls “software 2.0” and the most recent change - prompts which “program” an LLM - he calls “software 3.0”.

There’s a strong point here, but I don’t fully agree with the “software 3.0” concept. Neural networks are clearly a fundamentally different type of software than traditional computer code - they are assembled and trained rather than explicitly programmed, which makes creating and maintaining them a completely different skill set. You can’t debug and fix or even understand weights like you do with code. They are also utilized differently:

• Traditional computer code is best at creating deterministic, automated workflows in a domain you can clearly understand and define programmatically.
• Neural networks are best at deep pattern-recognition and creating effective good-enough approximations (already better than a human in many cases) for complex and ambiguous tasks.

An interesting claim Andrej makes is that “Software 2.0 is eating software 1.0”. He gives the example of Tesla Autopilot’s codebase: over time tons of functionality moved from C++ code to the neural net itself which grew in complexity and size - from image recognition only to stitching images across sensors and time all on its own.

This makes sense as someone who follows this evolving field from the sidelines - as neural networks become bigger, faster and more advanced, their data manipulation strengths increase and don’t you need to perform as much pre- and post-processing around them: the better network can just handle the more complicated data by itself.

He predicts that “software 3.0” will consume “software 2.0” in much the same way. Here I disagree: prompts don’t seem to me like an entirely new category of software, but as a programming layer on top of neural networks - essentially a domain-specific language for LLMs. Writing complex prompts (or “prompt engineering”) is similar to conventional declarative programming: you precisely define what you want the LLM to do, handling various states and execution flows and providing external capabilities as modules. They’re not “software 3.0”, but blend the pattern-recognition power of neural networks (“software 2.0”) with the structured flexibility and precision of traditional code (“software 1.0”) - a powerful convergence.

Traditional “Software 1.0” paradigms are here to stay

Software paradigms are great. They create a generic “pattern” for building software, abstracting the complexity behind it and letting us focus on what really matters - adapting and combining well-engineered tools to solve real-world problems. What’s even more amazing is that you can use them to create new and improved paradigms to solve bigger problems, leading to huge stacks of recursive abstractions.
The foundation of “software 1.0” is the programming paradigm stack:

Object-Oriented Abstractions (classes, generics)
    ↓
High-Level Language Constructs (types, garbage collection)
    ↓
Procedural Logic and Function-Based Code (c, intermediate languages)
    ↓
Assembly Language (human-readable instructions)
    ↓
Machine Code (CPU opcodes)
    ↓
Physical Hardware (transistors and logic gates)

When defining classes and their interactions, we can abstract away the physical hardware and all the intermediate layers that make them possible - we trust robust compilers and interpreters to seamlessly mediate between them. We then built more sophisticated software paradigms on top of that: data structures, design patterns, reusable components, architectures and software frameworks. Each additional paradigm layer further encapsulates complexity, making hard problems easier to solve, and impossible problems possible.

What’s fundamental to all of “Software 1.0” paradigms is that they are (mostly) deterministic and predictable - you run the same code on the same inputs twice and you expect to get the same result. This might not be the case if you introduce randomness or external I/O, and sometimes this stack of abstractions can fail in unexpected ways, but those cases are the exceptions not the norm. The hardest bugs to solve are usually those where an abstraction breaks unexpectedly, the lower it is on the software chain the worse it gets.

Neural networks are by design unpredictable black boxes. The same exact input will give you the same exact output, but it’s only as good as the data it was trained on. Also, they fail in completely unintuitive ways - for example, slight and sometimes undetectable perturbations (even changing a single pixel for older models) can make them fail spectacularly. On top of that, in the case of LLMs they are non-deterministic by design - controlled randomness is introduced (like temperature) to mimic human-like text generation and creativity, so the same input will more likely than not lead to different output.

It’s clear to me that traditional software paradigms will not go anywhere as long as you care about getting predictable outputs and understanding what went wrong when something like that eventually happens. We might not be the ones writing or verifying it in the future, at least not directly and painstakingly like before, but that only means we have more energy to focus on what really matters - solving the problem. The cycle continues!

When should we consider using “software 2.0”?

Modern neural networks and deep learning techniques give such amazing results they’re practically magic. On the one hand they seem like just another software paradigm stack analogous to the one before - from GPUs crunching matrix math through layers of linear algebra and attention mechanisms, up to prompts turning human language into a high-level programming interface. On the other hand, how they work is still a mystery - the bigger and more dense they become the harder it is to understand what happens inside, like our own brain.

The magical and mysterious aspect of neural networks shouldn’t concern us as software engineers. Like engineers before us who built airplanes before knowing the science of turbulence and aerodynamics, we follow the Engineering Method of solving problems: using “rules of thumb” (heuristics, best practices gained from observation, trial & error) in a poorly understood situation to give the best solution. From my observations and experience with LLMs, it seems like we reached a point of maturity that integrating them into our software building toolset is useful in certain cases.

So let’s be practical and treat neural networks and specifically LLMs as powerful black boxes, focusing on what they do and where they fail and not on how they do it. This doesn’t mean we can trust them to work like we “trust” traditional software - they can fail in new and unexpected ways, with no compile-time or exhaustive testing guarantees, so guardrails and monitoring are essential to limit the cost and impact of their mistakes.

With that in mind, when should we consider using neural networks in our software?

• When simpler solutions (using traditional software) are very hard or impossible to implement with your available resources
• When we’re fine trading latency and cost for better task performance: for example reaching new heights in OCR capabilities.
• When a task solution is easy to verify, but hard to solve using classic software techniques: for example, advanced game AIs (verifying legal moves is easy, creating the best one is hard) and AI coding agents (with tests as automatic verification).
• When you want to add some “human” flavor to your software: personalization (see this cool personalized terminal browser mini-project), content generation, automated decision making etc.

The new paradigms of software: programs + neural nets = agents?

For years, neural networks didn’t affect our engineering workflows - they just felt like fancy packages we can plug into our systems with traditional APIs, without changing the way we think about our code.

But with enhanced LLMs things are different. With their human-like text generation capabilities, strong steering and versatility using prompts, and their ability to use tools, we can now use them as a software framework, with prompts as our programming interface. Integrating a plugin or tool is simple. You can instruct the LLM to use an XML tag, such as <web_search>{search query here}</web_search>. Then, you search for this structure in the response string and perform the action (in this case, searching the web) using a standard API. You don’t even have to create the instrumentation in your code yourself, you can use anthropic’s Model Context Protocol SDKs which will handle the integration for you. You can also use existing MCPs - many companies now ship MCPs integrating LLMs with their product.

A great resource which details the newly emerging paradigms using LLMs is Building Effective Agents by Anthropic engineering. It includes explanations of successful production use-cases for LLM-integrated software (customer support and coding agents) and the design patterns behind them. They call this set of software paradigms Agentic systems, as in systems which have varying level of “agency” to decide how to perform tasks. The types of agentic systems which they discuss in the article are:

• The augmented LLM - an LLM enhanced by retrieval, tool calls and memory. This is the most basic agentic system, capable of generating its own search queries, selecting appropriate tools and determining what information to retain. Modern AI assistants are “augmented LLMs” - they perform web searches, write code and execute it in the response, use tools like canvases and update memory. As an engineer you can control it by adapting the prompt (prompt engineering) or by creating or integrating custom tools (MCP).

• Workflows - systems where LLMs and tools are orchestrated through predefined code paths. This can be done in many ways, some common design patterns are:
  • Prompt chaining - turning a complex task into a sequence of LLM calls and intermediate processing.
  • Routing - classifying an input and directing it to a specialized followup task.
  • Parallelization - working simultaneously on a task and aggregating the outputs.
  • Multi-role patterns like Orchestrator-workers and Evaluator-optimizer

• Agents - systems where LLMs dynamically direct their own processes and tool usage, maintaining control over how they accomplish tasks. This involves LLMs capable of reasoning, planning and recovering from errors using tools based on a feedback loop from their environment.

Using these new paradigms, which continue to evolve and might seem trivial by the time you read this, we can build software which we never imagined being possible before - software which solves problems automatically, applying the right tool for the job on its own. You can find working examples in Anthropic’s Agents Cookbook, and start trying them out in low-risk contexts like building your first “agentic” automation.
Did you write an AI automation? congratulations, you’re an AI-era software engineer!

Final words

Like many software engineers, as AI coders became better than me at writing code and AI-excused layoffs emerge, I began worrying for the future of my short career and started planning my approaching pivot into manual labor (I was always interested in locksmithing, this might be the chance to finally try it out!).

It’s hard to predict how the AI boom will affect the software engineering career in the upcoming years, or all computer-based careers for that matter. By now it’s clear that if the only value you provide is in generating “more of the same” digital content for someone who knows exactly what they want - code, documents, presentations - then your near-term fate is similar to those whose only job was generating images (graphic designers, contracted artists) before the image generation boom: in many cases you will be replaced with a “good enough” AI generator.

But the role of software engineering is not just about writing code. It’s about solving real-world problems using software - we write code, design architectures, find the right tools for the job, convert ambiguous and contradicting demands into clear definitions, and make sure it all fits together in a nice and reliable package. So, instead of writing code ourselves, we will manage the agents that write it for us using “prompt engineered” rules (more code in a different level of abstraction). When things ultimately break down, we will be the ones to understand what went wrong and fix it, probably relying on other agents to help us.

One thing is clear - new software paradigms allow us to solve bigger and more ambitious problems, which usually bring more opportunities and not less. Embrace the change!

Thank you for reading my first technical blogpost! If you found it useful or have some suggestions, let me know!
And if you enjoyed my writing and want to read something more self-reflective and personal, see my actual first blog post: Starting a blog in the age of endless content