Building Blocks

8 min read

AI apps are not chatbots. They’re distributed systems that call models, tools, and data across dozens of services — and they fail like distributed systems too.

Before you write a single line of application code, it helps to understand the full stack you’re building on. This lesson maps the layers of a modern AI application, from infrastructure at the bottom to the user-facing experience at the top.

The AI application stack

The stack typically flows from bottom to top: infrastructure (compute, storage, networking) → platform (MLOps, orchestration) → framework (pre-built models and algorithms) → services (API layers) → applications (end-user solutions).

A production AI application touches up to 13 distinct layers:

  1. Interaction & control plane — the application layer where your UI and APIs live
  2. Core application & hosting infrastructure — where your app runs (serverless, containers, edge)
  3. Data ingestion & semantic data foundation — how raw data enters and gets prepared
  4. Business context & semantic modeling — domain knowledge that grounds the model
  5. Memory & state management — conversation history, session state, long-term recall
  6. Tools & integration layer — MCP, A2A, domain-specific tools the model can call
  7. Execution & workflow orchestration — durable, event-driven pipelines for multi-step tasks
  8. Model gateway & semantic caching — routing requests to the right model, caching repeated queries
  9. Safety & guardrails — input/output filtering, policy enforcement, abuse prevention
  10. Prompt & interaction design — how you structure requests to the model
  11. Evaluation & telemetry — measuring quality, latency, cost, and drift
  12. Experimentation & continuous improvement — A/B testing, prompt versioning, model upgrades
  13. Security, compliance & governance — auth, audit trails, data residency, regulatory requirements

What this module covers

The remaining lessons in this module dive into the building blocks you’ll use most often as a web developer:

  • Structured data — getting reliable, typed output from models instead of free-form text
  • Streaming — delivering responses token-by-token for responsive UX
  • Tool calling — letting models invoke functions, APIs, and external systems
  • Evals — testing and measuring whether your AI features actually work

Each of these patterns appears in nearly every production AI application. Master them here, and you’ll have the foundation for everything that follows — from agents to RAG to multi-modal workflows.

Further reading