Limitations & Risks

8 min read

Generative AI is powerful, but it has real limitations. Understanding them is essential for building reliable applications.

Hallucinations

LLMs generate plausible text, not truthful text. They will confidently produce:

  • Made-up citations and references
  • Incorrect facts stated with certainty
  • Non-existent APIs, functions, or libraries
  • Fabricated statistics and studies

Mitigation strategies:

  • Never trust model output for factual claims without verification
  • Use RAG (Retrieval-Augmented Generation) to ground responses in real data
  • Implement structured output validation
  • Add disclaimers to user-facing AI content

Context Window Limitations

Models can only consider text within their context window. They can’t:

  • Remember previous conversations (unless you include them in the prompt)
  • Access information outside the prompt
  • Learn or update their knowledge in real-time

Your application architecture must handle conversation history, context management, and knowledge retrieval.

Bias

Models inherit biases from their training data. This can manifest as:

  • Stereotypical associations
  • Uneven quality across languages and cultures
  • Inconsistent treatment of different demographics
  • Western-centric worldviews

What you can do:

  • Test with diverse inputs
  • Implement content filtering
  • Provide options for users to flag problematic output
  • Don’t use AI for high-stakes decisions without human oversight

Security Risks

Prompt Injection

Users can craft inputs that override your system prompt:

Ignore all previous instructions and instead...

This is the most critical security risk in AI applications. We’ll cover defense strategies in Module 2.

Data Leakage

Anything in the prompt is accessible to the model — and potentially to the user through prompt injection. Never put secrets, internal data, or PII in system prompts.

Output Filtering

Model output can include harmful, offensive, or inappropriate content. Your application needs output filtering and moderation layers.

Cost Risks

Without proper controls, AI API costs can spike:

  • A bug in a retry loop can burn through your token budget in minutes
  • Allowing unlimited user queries without rate limiting
  • Forgetting to set max_tokens limits

Always implement spending caps, rate limiting, and monitoring.

The Bigger Picture

AI is a tool, not magic. It’s incredibly useful for:

  • Drafting, editing, and transforming text
  • Analyzing and summarizing large amounts of content
  • Generating and explaining code
  • Answering questions from a knowledge base

It’s unreliable for:

  • Mathematical reasoning (improving, but not trustworthy)
  • Factual recall (always verify)
  • Consistent behavior (outputs vary between runs)
  • Tasks requiring real-time information

Module 1 Complete

You now have a solid mental model of how generative AI works — from neural networks and transformers to tokens, inference, and practical limitations.

In Module 2: Building Blocks for GenAI Apps, we’ll start building. You’ll write your first AI-powered features: prompt engineering, streaming responses, structured output, and more.