AI Engineer Roadmap

What an AI Engineer Actually Builds beginner: The real scope of the role in 2026: application architecture and orchestration, not model training.

• AI Engineer vs ML Engineer: Building on top of foundation models versus training custom ones.
• Foundation Models: Large pre-trained models like GPT and Claude used as a base.
• LLM App Architecture: The typical layers: prompt, retrieval, tools, model, output.

How LLMs Work Under the Hood beginner: Enough of the mechanics to reason about model behavior, without needing the training math.

• Tokens & Tokenization: How text is broken into the units a model actually processes.
• Context Window: The maximum amount of text a model can consider at once.
• Next-Token Prediction: The core mechanism generating one token at a time based on context.
• Temperature & Sampling: Parameters controlling how random or deterministic outputs are.

Embeddings Fundamentals beginner: How text becomes vectors, and why that's the foundation of both search and RAG.

• Vector Embeddings: Numerical representations of text that capture semantic meaning.
• Cosine Similarity: The standard way to measure how close two embeddings are.
• Embedding Models: Models specifically trained to produce useful text embeddings.

OpenAI API Fundamentals beginner: Making raw calls to GPT models, the base layer every framework you'll learn later sits on top of.

• Chat Completions API: OpenAI's core endpoint for sending messages and getting responses.
• System vs User Messages: How message roles shape a model's behavior and persona.
• Streaming Responses: Receiving output token by token instead of waiting for the full reply.

Anthropic Claude API Fundamentals beginner: Claude's Messages API and the ways it differs from OpenAI's, since production apps often use both.

• Messages API: Claude's request format for conversational and single-turn calls.
• System Prompts: Claude's dedicated field for persistent instructions and behavior.
• Extended Thinking: Claude's mode for showing intermediate reasoning before an answer.

Prompt Engineering Fundamentals beginner: The techniques that reliably improve output quality before reaching for any other tool.

• Zero-Shot vs Few-Shot: Prompting with no examples versus including sample input-output pairs.
• Chain of Thought: Prompting a model to reason step by step before answering.
• Structured Output Prompting: Getting a model to reliably return JSON or a defined schema.
• Prompt Injection Risks: How untrusted input can hijack a model's instructions.

Advanced Prompting Patterns intermediate: Patterns that show up in production prompts once basic prompting isn't enough.

• ReAct Prompting: Interleaving reasoning and tool actions within a single prompt loop.
• Prompt Chaining: Breaking one complex task into a sequence of smaller prompts.
• Self-Consistency: Sampling multiple reasoning paths and picking the most common answer.

Function Calling Fundamentals intermediate: How to let a model trigger real code instead of only generating text.

• Tool Schemas: Describing a function's name, parameters and purpose to a model.
• Tool Call Parsing: Reading a model's structured request to call a specific function.
• Parallel Tool Calls: A model requesting multiple tool calls in a single turn.

Building Custom Tools for LLM Apps intermediate: Designing tools that are reliable and hard for a model to misuse.

• Tool Design Principles: Narrow, well-described tools a model can call correctly and predictably.
• Error Handling in Tools: Returning useful errors a model can recover from instead of failing silently.
• MCP (Model Context Protocol): A standard protocol for connecting models to external tools and data.

RAG Pipeline Fundamentals intermediate: The core loop of retrieval augmented generation, from raw documents to grounded answers.

• Chunking Strategies: Splitting documents into pieces small enough to embed and retrieve well.
• Retrieval & Reranking: Fetching candidate chunks, then reordering them by relevance.
• Context Injection: Inserting retrieved chunks into a prompt before generation.

Vector Databases: Pinecone, Weaviate & pgvector intermediate: Choosing and operating the storage layer that makes retrieval fast at scale.

• Approximate Nearest Neighbor Search: The indexing technique vector databases use to search fast at scale.
• Pinecone: A managed, purpose-built vector database for production RAG.
• Weaviate: An open source vector database with built-in hybrid search.
• pgvector: A Postgres extension adding vector search to an existing database.

Advanced RAG Techniques advanced: The patterns that fix RAG's most common failure, retrieving the wrong or incomplete context.

• Hybrid Search: Combining keyword and semantic search for better retrieval accuracy.
• Query Rewriting: Reformulating a user's question to improve retrieval quality.
• Parent-Child Chunking: Retrieving small chunks but returning their larger surrounding context.
• Multi-Hop Retrieval: Chaining multiple retrieval steps to answer complex questions.

LangChain Fundamentals intermediate: The framework abstractions for chaining prompts, models and tools together.

• Chains & Runnables: LangChain's composable building blocks for multi-step logic.
• Memory: Persisting conversation history across turns in an application.
• Document Loaders & Retrievers: LangChain's abstractions for ingesting and querying documents.

AI Agents Fundamentals intermediate: What actually makes a system an agent rather than a single prompt-response call.

• Agent Loop: The plan, act, observe cycle an agent repeats to complete a task.
• Planning & Reasoning: How an agent decides what steps a task actually requires.
• Agent Memory: Short and long term state an agent uses across steps and sessions.

LangGraph for Agent Orchestration advanced: Building agents as explicit state graphs instead of implicit loops, for control and reliability.

• State Graphs: Modeling an agent's flow as nodes and edges instead of a black box loop.
• Conditional Edges: Routing execution based on the agent's current state or output.
• Human-in-the-Loop: Pausing an agent for human approval before a critical action.
• Checkpointing: Saving agent state so a run can pause, resume or be inspected.

Multi-Agent Systems advanced: Coordinating multiple specialized agents instead of asking one agent to do everything.

• Agent Handoffs: Passing a task from one specialized agent to another.
• Supervisor Pattern: A coordinating agent that delegates work to worker agents.
• Shared State Between Agents: How multiple agents read and write a common context.

Choosing Between Prompting, RAG & Fine-Tuning intermediate: The decision framework for picking the cheapest technique that actually solves the problem.

• Knowledge vs Behavior Problems: Whether the model lacks information or needs to act differently.
• Cost vs Control Tradeoffs: Weighing effort and cost against how much control each approach gives.

Fine-Tuning Fundamentals for App Builders advanced: Enough fine-tuning knowledge to use it as a tool, without becoming an ML researcher.

• When Fine-Tuning Beats RAG: Cases where changing model behavior matters more than added knowledge.
• LoRA & Parameter-Efficient Tuning: Lightweight fine-tuning methods that update far fewer parameters.
• Fine-Tuning APIs: Managed services that fine-tune a model without custom training infra.

LLM Evaluation Fundamentals advanced: Testing whether outputs are actually good, systematically, instead of eyeballing a few examples.

• Eval Datasets: Curated test cases used to measure an LLM app's output quality.
• LLM-as-Judge: Using a model to score another model's outputs against criteria.
• Regression Testing for Prompts: Catching quality drops when a prompt or model changes.

Tracing & Debugging with LangSmith advanced: Seeing exactly what happened inside a chain or agent run, not just the final output.

• Trace Inspection: Viewing every step, prompt and tool call inside an execution.
• Dataset-Based Evals: Running LangSmith evals against a saved set of test cases.
• Feedback Collection: Capturing user or automated ratings on production outputs.

Guardrails & Safety for LLM Apps advanced: Preventing an app from generating harmful, off-topic or leaked content in production.

• Input & Output Filtering: Screening content before it reaches the model or the user.
• Jailbreak Resistance: Designing prompts and checks resistant to instruction override attempts.
• PII Redaction: Detecting and removing sensitive personal data from inputs or outputs.

Deploying LLM Applications advanced: Taking a working script and turning it into a service that handles real traffic.

• API Wrapping with FastAPI: Exposing an LLM app's logic as a callable HTTP service.
• Async Request Handling: Serving multiple concurrent LLM calls without blocking.
• Rate Limiting & Retries: Handling API limits and transient failures gracefully.

Cost & Latency Optimization advanced: Keeping an LLM app fast and affordable once real usage volume shows up.

• Prompt Caching: Reusing repeated prompt segments to cut cost and latency.
• Model Routing: Sending easy queries to a cheaper, faster model automatically.
• Token Usage Monitoring: Tracking spend and usage patterns across an application.

Monitoring LLM Apps in Production advanced: Knowing when an app's quality degrades in the wild, not just when it crashes.

• Output Quality Drift: Tracking whether response quality degrades silently over time.
• Structured Logging for LLM Calls: Recording prompts, outputs and metadata for later debugging.