Collection · 446 entries

AI Concepts

The ideas, methods and training techniques behind modern AI systems — from attention and embeddings to alignment and evaluation.

• 3D AI

  3D Gaussian Splatting

  Real-time 3D radiance field rendering using anisotropic 3D Gaussians for photorealistic scene reconstruction.

  Advanced
• Practice

  A/B Testing for AI

  Comparing model or prompt variants on live traffic with randomised assignment to measure real-world impact.

  Intermediate
• Foundations

  Activation Function

  The non-linear function applied after each layer that lets a network represent more than a linear mapping.

  Intermediate
• Safety & Control

  Activation Steering

  Modifies internal activation vectors during inference to control model tone or bias.

  Advanced
• Practice

  Active Learning

  Letting the model choose which examples to have labelled next, so annotation effort goes where it changes the model most.

  Advanced
• Optimization

  AdamW Optimizer Mechanics

  Adaptive weight decay optimization algorithm standard for Transformer model training.

  Intermediate
• RAG Systems

  Adaptive RAG

  Uses an LLM classifier to dynamically choose between direct response, simple RAG, or complex iterative search.

  Advanced
• Agentic AI

  Advanced Agentic Workflows

  Design frameworks for orchestrating autonomous AI agents through iterative planning, tool execution, state management, and self-reflection.

  Advanced
• Risks

  Adversarial Attack

  Inputs deliberately crafted to make a model produce a wrong output, often with changes imperceptible to people.

  Advanced
• AI Security

  Adversarial Robustness

  Building models resilient to adversarial attacks and perturbations designed to trigger failure.

  Advanced
• RAG Systems

  Agentic RAG Architectures

  RAG systems where autonomous agents dynamically decide when, where, and how to search for information.

  Advanced
• Agentic AI

  Agentic Reflection Loops

  A specialized technique in agentic ai providing self-correction after code execution capabilities for advanced enterprise AI applications.

  Intermediate
• Agentic AI

  Agentic Web Scraping Protocols

  A specialized technique in agentic ai providing llm-driven dom traversal and data extraction capabilities for advanced enterprise AI applications.

  Intermediate
• AI Agents

  Agentic Workflow Design

  Architecting multi-agent loops with planning, tool invocation, and reflection stages.

  Intermediate
• Agents

  AI Agents

  Systems where a model plans a sequence of steps, calls tools, observes results and iterates towards a goal with limited human input.

  Intermediate
• Safety

  AI Alignment

  Making an AI system pursue the goals its developers and users actually intend, including implicit expectations they never stated.

  Advanced
• Infrastructure

  AI and the Internet of Things

  Combining networked sensors and devices with machine learning that interprets their data, often at the edge.

  Intermediate
• Infrastructure

  AI API

  A hosted interface that lets applications send prompts and receive model outputs without running any infrastructure.

  Beginner
• Practice

  AI Benchmark

  A standard task and dataset used to compare models on a shared yardstick.

  Intermediate
• Practice

  AI Citations

  Attaching verifiable source references to generated claims so a reader can check them.

  Beginner
• Applications

  AI Code Generation

  Producing, completing or refactoring source code from natural-language descriptions or surrounding context.

  Beginner
• Infrastructure

  AI Compute

  The total processing capacity used to train and run AI models, and one of the field's defining constraints.

  Intermediate
• Applications

  AI Content Moderation

  Using models to detect policy-violating content at a scale human review cannot reach.

  Intermediate
• Risks

  AI Data Exfiltration

  Using an AI system as a channel to extract confidential data it can access but the requester should not see.

  Advanced
• Ethics

  AI Energy and Environmental Impact

  The electricity, water and hardware footprint of training and running AI systems.

  Intermediate
• Practice

  AI Engineering

  The discipline of building production applications on top of AI models, as distinct from training the models themselves.

  Intermediate
• Ethics

  AI Ethics

  The study of how AI should be built and used, covering fairness, transparency, accountability, privacy, labour and environmental impact.

  Beginner
• Ethics

  AI Governance

  The policies, roles and controls an organisation uses to decide what AI it deploys and under what conditions.

  Intermediate
• AI Policy

  AI Governance Frameworks

  Standardized organizational risk management procedures for responsible AI deployment.

  Beginner
• Safety

  AI Guardrails

  Programmatic constraints around a model that block unsafe, off-topic or malformed inputs and outputs.

  Intermediate
• Risks

  AI Hallucination

  When a generative model produces fluent output that is factually wrong or entirely invented while sounding confident.

  Beginner
• Applications

  AI Image Generation

  Creating original images from text descriptions, reference images or both.

  Beginner
• Industry

  AI in Cybersecurity

  Using AI to detect, investigate and respond to security threats — and defending against attackers who use it too.

  Intermediate
• Industry

  AI in Education

  Using AI for tutoring, lesson preparation, feedback, accessibility and administrative work in learning settings.

  Beginner
• Industry

  AI in Finance

  Applying AI to risk, fraud, compliance, research, servicing and operations in financial institutions.

  Intermediate
• Industry

  AI in Healthcare

  Applying AI to diagnosis support, imaging, documentation, triage, operations and drug discovery.

  Intermediate
• Industry

  AI in Marketing

  Applying AI to research, content production, targeting, personalisation and performance analysis in marketing.

  Beginner
• Applications

  AI in Robotics

  Applying perception, planning and learned control so machines can act usefully in the physical world.

  Advanced
• Infrastructure

  AI Infrastructure

  The hardware, storage, networking and platform software that AI workloads run on.

  Intermediate
• Infrastructure

  AI Latency

  The delay between a request and a useful response, usually split into time to first token and generation speed.

  Intermediate
• Practice

  AI Literacy

  The practical ability to use AI tools effectively, judge their output critically and understand their limits.

  Beginner
• Architecture Patterns

  AI Memory

  Mechanisms that let an assistant retain useful information across turns and sessions beyond the context window.

  Intermediate
• Practice

  AI Observability

  Instrumenting AI systems so you can see what was asked, what was retrieved, what the model returned, and what it cost.

  Intermediate
• Architecture Patterns

  AI Orchestration

  Coordinating models, prompts, tools, retrieval and control flow into a reliable end-to-end application.

  Intermediate
• Applications

  AI Personalization

  Adapting content, pricing, layout or messaging to an individual using behavioural and contextual signals.

  Beginner
• Safety

  AI Red Teaming

  Deliberately attacking your own AI system to find harmful, insecure or embarrassing behaviour before users or attackers do.

  Intermediate
• Ethics

  AI Regulation

  Laws and binding rules governing how AI systems may be developed, sold and used.

  Intermediate
• Safety

  AI Safety

  The field concerned with ensuring AI systems behave as intended and do not cause harm.

  Intermediate
• Safety

  AI Security

  Protecting AI systems from attacks on their models, data, prompts, tools and supply chain.

  Intermediate
• Applications

  AI Summarization

  Condensing longer content into a shorter version that preserves the important information.

  Beginner
• Practice

  AI User Experience

  Designing interfaces for systems that are probabilistic, occasionally wrong and sometimes slow.

  Intermediate
• Ethics

  AI Watermarking

  Embedding a detectable signal into generated content so it can later be identified as machine-produced.

  Advanced
• Applications

  AI Workflow Automation

  Using AI inside automated business processes to handle the judgement steps that rules alone could not.

  Beginner
• Practice

  AI-Assisted Software Engineering

  The changed practice of building software when a large share of code is drafted, reviewed or refactored with AI assistance.

  Intermediate
• Tasks

  Anomaly Detection

  Identifying records or events that deviate meaningfully from normal patterns.

  Intermediate
• Foundations

  Artificial General Intelligence

  A hypothetical system able to learn and perform any intellectual task a human can, across domains, without task-specific retraining.

  Intermediate
• Foundations

  Artificial Intelligence

  The field of building computer systems that perform tasks normally associated with human intelligence, such as perception, reasoning, language and decision-making.

  Beginner
• Hardware & Accelerators

  Asynchronous GPU Compute Operations

  A specialized technique in hardware & accelerators providing overlapping gemm and communications capabilities for advanced enterprise AI applications.

  Intermediate
• Architectures

  Attention Mechanism

  A learned way for a model to weigh which other parts of the input matter most when representing each position.

  Advanced
• Core Architecture

  Attention with Linear Biases

  A specialized technique in core architecture providing alibi positional bias vectors capabilities for advanced enterprise AI applications.

  Intermediate
• Audio AI

  Audio Tokenization

  Converts continuous audio waveforms into discrete tokens for autoregressive modeling.

  Advanced
• Architectures

  Autoencoder

  A network trained to reconstruct its own input through a narrow bottleneck, learning a compressed representation on the way.

  Intermediate
• Prompt Engineering

  Automatic Prompt Optimization

  Using algorithms or LLMs to automatically refine and optimize prompts for peak metric scores.

  Intermediate
• Agentic AI

  Autonomous Goal Decomposition

  A specialized technique in agentic ai providing hierarchical task planning in ai agents capabilities for advanced enterprise AI applications.

  Intermediate
• Applications

  Autonomous Vehicles

  Vehicles that sense their environment and drive with partial or full independence from a human operator.

  Intermediate
• NLP Fundamentals

  Autoregressive Decoding

  Generating text token by token where each new token depends on all previously generated tokens.

  Beginner
• Foundations

  Backpropagation

  The algorithm that computes how much each parameter contributed to the error, by applying the chain rule backwards through the network.

  Intermediate
• Evaluation

  Benchmark Decontamination

  Filtering pre-training corpora to ensure test benchmarks are not memorized in model weights.

  Advanced
• Ethics

  Bias in AI

  Systematic unfairness in AI outputs that disadvantages particular groups, usually inherited from data or design choices.

  Beginner
• Core Architecture

  Causal Masking in Attention Layers

  A specialized technique in core architecture providing autoregressive unidirectional attention capabilities for advanced enterprise AI applications.

  Intermediate
• Prompt Engineering

  Chain of Density Prompting

  A specialized technique in prompt engineering providing progressive summarization refinement capabilities for advanced enterprise AI applications.

  Intermediate
• Prompt Engineering & Reasoning

  Chain-of-Thought (CoT) Prompting

  A technique that prompts language models to break down complex reasoning problems into intermediate logical steps before producing an answer.

  Beginner
• Practice

  Chain-of-Thought Prompting

  Prompting a model to work through intermediate steps before giving a final answer, which improves accuracy on reasoning tasks.

  Intermediate
• Prompt Engineering

  Chain-of-Verification (CoVe)

  Prompts models to draft answers, formulate verification questions, answer them independently, and revise.

  Intermediate
• Foundations

  Chinese Room Argument

  John Searle's thought experiment arguing that manipulating symbols by rules does not amount to understanding them.

  Intermediate
• Inference & Serving

  Chunked Context Prefill

  An inference optimization technique that breaks long prompt prefill phases into manageable chunks executed alongside decoding batches.

  Advanced
• Architecture Patterns

  Chunking

  Splitting documents into retrievable passages sized so that each one carries enough context to answer a question.

  Intermediate
• RAG Engineering

  Chunking Strategies

  Splitting large documents into optimal token segments for vector embedding and retrieval.

  Intermediate
• Tasks

  Classification

  Predicting which of a fixed set of categories an input belongs to.

  Beginner
• Generative Media

  Classifier-Free Guidance (CFG)

  Balances image quality and prompt adherence in diffusion models by scaling unconditional deltas.

  Intermediate
• Tasks

  Clustering

  Grouping records so that items inside a group are more similar to each other than to items in other groups.

  Beginner
• Agentic AI

  Code Execution Sandbox Safety

  A specialized technique in agentic ai providing e2b and docker agent sandboxes capabilities for advanced enterprise AI applications.

  Intermediate
• Algorithms

  Collaborative Filtering

  Recommending items by finding users with similar behaviour and surfacing what they engaged with.

  Intermediate
• Foundations

  Computer Vision

  The field of enabling machines to interpret images and video — detecting, classifying, segmenting and tracking what appears in them.

  Beginner
• Practice

  Confusion Matrix

  A table of true and false positives and negatives that shows exactly how a classifier is right and wrong.

  Beginner
• Alignment

  Constitutional AI

  An alignment approach where a model critiques and revises its own outputs against a written set of principles, reducing reliance on human harm labelling.

  Advanced
• Ethics

  Content Provenance

  Cryptographically signed metadata that records how a piece of media was created and edited.

  Intermediate
• Generative AI

  Context Window

  The maximum number of tokens a model can consider at once, covering the system prompt, conversation, retrieved documents and its own reply.

  Beginner
• Core Architecture

  Context Window Expansion Protocols

  A specialized technique in core architecture providing rope scaling and yarn interpolation capabilities for advanced enterprise AI applications.

  Intermediate
• Architecture

  Context Window Extension

  Methods such as YaRN and RoPE scaling to extend model token context post-pretraining.

  Advanced
• Inference & Serving

  Contextual Cache Management

  An inference engine optimization that saves and reuses KV cache states for shared prompt prefixes across multiple API calls.

  Intermediate
• RAG & Vector Search

  Contextual Chunking in RAG

  A specialized technique in rag & vector search providing preserving section titles in passage chunks capabilities for advanced enterprise AI applications.

  Intermediate
• RAG Systems

  Contextual Compression

  Filters out irrelevant tokens from retrieved documents before passing them to the LLM context.

  Intermediate
• RAG & Vector Search

  Contextual Retrieval Protocols

  A retrieval methodology where text chunks are pre-prepended with document-level context before embedding and indexing.

  Intermediate
• Multimodal AI

  Continuous Autoregressive Vision

  An architecture that generates images token-by-token or scale-by-scale using autoregressive transformer mechanics.

  Advanced
• Inference & Serving

  Continuous Batching in Inference

  A specialized technique in inference & serving providing dynamic slot allocation for llm servers capabilities for advanced enterprise AI applications.

  Intermediate
• Pre-training

  Continuous Pre-training

  Continuing unsupervised autoregressive training on domain corpora before fine-tuning.

  Advanced
• Multimodal AI

  Contrastive Language-Image Pretraining (CLIP)

  Pairs text and image encoders using contrastive loss to map vision and text into a shared embedding space.

  Intermediate
• Learning Paradigms

  Contrastive Learning

  Training representations by pulling matching pairs together and pushing mismatched pairs apart in embedding space.

  Advanced
• Applications

  ControlNet and Conditioning

  Techniques that steer image generation with structural inputs such as edges, depth maps, poses or segmentation masks.

  Advanced
• Applications

  Conversational AI

  Systems designed to hold a useful back-and-forth dialogue with people through text or voice.

  Beginner
• Architectures

  Convolutional Neural Network

  A network architecture that slides learned filters across a grid-shaped input, making it efficient for images and other spatial data.

  Intermediate
• RAG Systems

  Corrective RAG (CRAG)

  Evaluates retrieved document quality and triggers web search fallbacks if retrieved context is poor.

  Advanced
• RAG & Vector Search

  Cross-Encoder Re-Ranking

  A retrieval technique that passes query-document pairs jointly through a transformer to score exact semantic relevance.

  Intermediate
• Search Engineering

  Cross-Encoder Reranking

  Scores query-document pairs jointly through a Transformer for high-precision re-ranking.

  Advanced
• RAG & Vector Search

  Cross-Lingual Vector Retrieval

  A specialized technique in rag & vector search providing multilingual embedding space matching capabilities for advanced enterprise AI applications.

  Intermediate
• Practice

  Cross-Validation

  Estimating model performance by repeatedly training and testing on different partitions of the data.

  Intermediate
• Generative Media

  DALL-E 3 Architecture

  Uses dense, detailed synthetic image captions to dramatically improve prompt adherence in image generation.

  Advanced
• AI Security

  Data Exfiltration via AI

  Exploiting AI tool access to unauthorizedly transmit sensitive internal data externally.

  Advanced
• Data

  Data Labeling

  Attaching ground-truth answers to raw examples so a supervised model has something to learn from.

  Beginner
• Data

  Data Leakage

  When information unavailable at prediction time leaks into training, producing offline scores that collapse in production.

  Intermediate
• Distributed Training

  Data Parallelism (DDP / FSDP)

  Replicating models across GPUs while sharding optimizer states, gradients, and model parameters.

  Advanced
• AI Security

  Data Poisoning Detection

  Filtering malicious or backdoor training samples from pre-training and fine-tuning datasets.

  Advanced
• Ethics

  Data Privacy in AI

  Protecting personal information across the AI lifecycle — collection, training, prompting, logging and output.

  Beginner
• Data

  Dataset Curation

  Selecting, filtering and balancing the data that goes into training, which increasingly matters more than raw volume.

  Advanced
• Algorithms

  Decision Tree

  A model that splits data by feature thresholds into a tree of if-then rules ending in a prediction.

  Beginner
• Foundations

  Deep Learning

  Machine learning built on neural networks with many layers, which learn their own representations of raw data rather than relying on hand-designed features.

  Intermediate
• Risks

  Deepfake

  Synthetic audio, image or video that convincingly depicts a real person saying or doing something they did not.

  Beginner
• AI Safety

  Deepfake Detection Methods

  Algorithms inspecting frequency artifacts and biological markers to identify synthetic media.

  Intermediate
• Information Retrieval

  Dense Passage Retrieval (DPR)

  Retrieves text passages using dual-encoder neural embeddings rather than BM25 keyword matching.

  Intermediate
• RAG & Vector Search

  Dense Passage Retrieval Architectures

  A specialized technique in rag & vector search providing dual-encoder embedding search capabilities for advanced enterprise AI applications.

  Advanced
• Computer Vision

  Depth Anything Architecture

  Estimates dense 3D depth maps from arbitrary single 2D images.

  Intermediate
• Data

  Differential Privacy

  A mathematical guarantee that the presence or absence of any single individual's record barely changes a system's output.

  Advanced
• AI Privacy

  Differential Privacy in AI

  Adding mathematical noise during training to guarantee individual data privacy.

  Advanced
• Generative AI

  Diffusion Model

  A generative model that learns to reverse a gradual noising process, turning random noise into a coherent image, audio clip or video.

  Advanced
• Generative Media

  Diffusion Transformers (DiT)

  Combines Transformer backbones with diffusion processes to achieve state-of-the-art image and video synthesis.

  Advanced
• Applications

  Digital Twin

  A continuously updated virtual model of a physical asset, process or system used for monitoring and what-if analysis.

  Intermediate
• Tasks

  Dimensionality Reduction

  Compressing high-dimensional data into fewer dimensions while preserving as much meaningful structure as possible.

  Intermediate
• Alignment & Preference Tuning

  Direct Alignment from Preference Pools

  Post-training methods that integrate preference alignment directly into supervised fine-tuning loss functions without pair data.

  Advanced
• Alignment

  Direct Preference Optimization

  A preference-tuning method that optimises a language model directly on preferred-versus-rejected response pairs, without training a separate reward model.

  Advanced
• Alignment & Preference Tuning

  Direct Preference Optimization (DPO)

  A stable, implicit reward alignment algorithm for fine-tuning language models directly from human feedback without training a separate reward model.

  Advanced
• Alignment & Preference Tuning

  Direct Preference Optimization Extensions

  Advanced algorithmic extensions of DPO, such as IPO, KTO, and ORPO, that optimize preference alignment with higher stability.

  Advanced
• Alignment & Preference Tuning

  Direct Preference Optimization Loss

  A specialized technique in alignment & preference tuning providing dpo implicit loss formulation capabilities for advanced enterprise AI applications.

  Intermediate
• Alignment & Preference Tuning

  Direct Preference Optimization Stability

  A specialized technique in alignment & preference tuning providing ipo and identity preference loss capabilities for advanced enterprise AI applications.

  Intermediate
• Applications

  Document AI

  Extracting structure and meaning from documents such as PDFs, scans, invoices and forms.

  Intermediate
• Alignment

  DPO Variants (KTO, IPO, ORPO)

  Modern loss functions for direct preference alignment without explicit reward modeling.

  Advanced
• Practice

  Dropout

  A regularisation method that randomly zeroes a fraction of units during training so the network cannot rely on any single pathway.

  Intermediate
• Infrastructure

  Edge AI

  Running AI models directly on devices — phones, cameras, vehicles, sensors — instead of sending data to a server.

  Intermediate
• Search Engineering

  Embedding Fine-Tuning

  Adapting general embedding models to domain-specific jargon using contrastive loss.

  Intermediate
• Foundations

  Embeddings

  Numeric vector representations of text, images or other data where distance corresponds to semantic similarity.

  Intermediate
• Agentic AI

  Embodied Action Trajectory Generation

  A specialized technique in agentic ai providing predicting robot motor controls capabilities for advanced enterprise AI applications.

  Intermediate
• Foundations

  Emergent Abilities

  Capabilities that appear only above a certain model scale and are largely absent in smaller models.

  Advanced
• Prompt Engineering

  Emotion Prompting

  Adding psychological importance or urgency phrases to prompts to enhance model performance.

  Beginner
• Practice

  Ensemble Learning

  Combining several models so their errors partly cancel and the aggregate outperforms any single member.

  Intermediate
• Agentic AI

  Episodic Agent Memory Systems

  Architectural memory modules that store, summarize, and retrieve past agent actions, environment states, and user feedback.

  Intermediate
• AI Policy

  EU AI Act Compliance

  Regulatory compliance categories classifying AI systems by risk level and enforcement mandates.

  Intermediate
• AI Policy

  Executive Order on AI Safety

  United States policy standards for dual-use foundation model reporting and safety evaluations.

  Intermediate
• Core Architecture

  Expert Parallelism in Distributed Training

  A specialized technique in core architecture providing partitioning moe experts across gpus capabilities for advanced enterprise AI applications.

  Intermediate
• Foundations

  Expert System

  A rule-based program that encodes a specialist's decision logic to give advice in a narrow domain.

  Beginner
• Safety

  Explainable AI

  Methods that make a model's individual decisions understandable to the people affected by them.

  Intermediate
• Foundations

  Exploration vs Exploitation

  The trade-off between trying new options to learn and repeating the option currently believed best.

  Intermediate
• Practice

  Feature Engineering

  Turning raw data into the input variables a model can actually learn from.

  Intermediate
• Infrastructure

  Feature Store

  A shared system for computing, storing and serving model input features consistently between training and inference.

  Advanced
• Distributed AI

  Federated Learning (2)

  Training machine learning models across decentralized edge devices without centralizing private data.

  Advanced
• Practice

  Few-Shot Learning

  Including a handful of worked examples in the prompt so the model infers the pattern you want.

  Beginner
• Prompt Engineering

  Few-Shot Prompting

  Providing a small set of exemplar input-output pairs inside the prompt to guide task formatting.

  Beginner
• Practice

  Fine-Tuning

  Continuing training of a pretrained model on a smaller, targeted dataset to specialise its behaviour, format or domain.

  Intermediate
• Model Architecture & GPU Optimization

  FlashAttention (2)

  An IO-aware exact attention algorithm that speeds up Transformer training and inference while reducing memory usage.

  Advanced
• Hardware & Accelerators

  FlashAttention-3 Architecture

  The third generation of hardware-aware attention algorithms designed for NVIDIA Hopper GPUs with FP8 low-precision support.

  Advanced
• Generative Media

  Flow Matching Generative Models

  Generates continuous probability paths connecting noise to data for fast, high-quality image generation.

  Advanced
• Generative AI

  Foundation Model

  A large model pretrained on broad data that serves as a reusable base for many downstream applications.

  Intermediate
• Hardware & Accelerators

  FP8 Precision Training Workflows

  A specialized technique in hardware & accelerators providing e4m3 and e5m2 floating point formats capabilities for advanced enterprise AI applications.

  Intermediate
• Agents

  Function Calling

  The API mechanism where a model returns a structured call to a declared function instead of free text.

  Intermediate
• Evaluation

  Function Calling Evaluations

  Testing LLM accuracy in invoking external tools with correct parameter types and structures.

  Intermediate
• Inference & Serving

  Function Schema Validation

  A specialized technique in inference & serving providing json-schema validation for function calls capabilities for advanced enterprise AI applications.

  Intermediate
• Architectures

  Generative Adversarial Network

  Two networks trained against each other — a generator producing samples and a discriminator judging whether they are real.

  Advanced
• Generative AI

  Generative AI

  AI that produces new content — text, images, audio, video, code or data — rather than only classifying or predicting.

  Beginner
• Applications

  Generative Engine Optimization

  Making content easy for AI answer engines to retrieve, quote and attribute correctly.

  Intermediate
• AI Safety & Ethics

  Generative Image Watermarking Protocols

  A specialized technique in ai safety & ethics providing synthid latent space watermarks capabilities for advanced enterprise AI applications.

  Intermediate
• Multimodal AI

  Generative Physics Simulation

  A specialized technique in multimodal ai providing neural world models for physics capabilities for advanced enterprise AI applications.

  Intermediate
• Infrastructure

  GPU Cluster Interconnects (NVLink / InfiniBand)

  High-bandwidth GPU-to-GPU networking infrastructure required for distributed pre-training and multi-node inference.

  Advanced
• Infrastructure

  GPU for AI

  Graphics processors whose massively parallel arithmetic makes them the standard hardware for training and serving neural networks.

  Beginner
• Training Acceleration

  Gradient Accumulation

  Accumulates gradients over multiple small micro-batches before taking an optimizer step.

  Beginner
• Algorithms

  Gradient Boosting

  An ensemble technique that adds models sequentially, each one correcting the residual errors of the combination so far.

  Intermediate
• Training Acceleration

  Gradient Checkpointing

  Trades compute for memory by recomputing intermediate activations during backward passes.

  Advanced
• Foundations

  Gradient Descent

  The optimisation procedure that trains almost all neural networks by repeatedly stepping parameters in the direction that reduces loss.

  Intermediate
• LLM Engineering

  Grammar-Constrained Decoding

  Restricting valid next-token logits to strictly follow JSON, XML, or SQL grammars.

  Intermediate
• Inference & Serving

  Grammar-Guided Structured Decoding

  An inference-time technique that restricts model token logits to enforce strict JSON, XML, or code syntax matching a defined grammar.

  Intermediate
• Architecture

  Graph Neural Networks (GNN)

  Neural network architectures designed to learn representations directly over graph-structured data.

  Advanced
• Information Retrieval & Knowledge Graphs

  Graph RAG

  A retrieval methodology that combines knowledge graphs with vector search to capture complex entity relationships across large document sets.

  Advanced
• Practice

  Grounding

  Tying a model's answers to verifiable external sources or data rather than its parametric memory.

  Intermediate
• Alignment & Preference Tuning

  Group Relative Policy Optimization

  A reinforcement learning algorithm that optimizes policy outputs against a group of sampled completions without a critic model.

  Advanced
• Core Architecture

  Grouped Query Attention Scaling

  An attention mechanism that groups multiple query heads to share single key and value heads, balancing speed and model quality.

  Intermediate
• Architecture

  Grouped-Query Attention (GQA)

  Blends multi-head and multi-query attention to reduce KV cache size while preserving quality.

  Advanced
• Evaluation

  GSM8K Math Benchmark

  A dataset of grade school math word problems used to benchmark multi-step reasoning capabilities.

  Beginner
• LLM Engineering

  Guaranteed Structured Outputs

  Forces LLM sampling to strictly conform to JSON schemas or CFG grammars via token masking.

  Intermediate
• AI Safety & Ethics

  Guardrailing Classifier Architecture

  Dedicated lightweight neural classifiers deployed alongside primary LLMs to inspect incoming prompts and outgoing generations.

  Intermediate
• AI Safety

  Hallucination Mitigation

  Techniques and architectures designed to eliminate ungrounded or fabricated model outputs.

  Intermediate
• Hardware & Accelerators

  Hardware Acceleration for AI Routing

  A specialized technique in hardware & accelerators providing cerebras and groq spatial architectures capabilities for advanced enterprise AI applications.

  Intermediate
• Core Architecture

  Hierarchical Attention Mechanisms

  A specialized technique in core architecture providing multi-level context processing capabilities for advanced enterprise AI applications.

  Intermediate
• Vector Indexing

  Hierarchical Navigable Small World (HNSW)

  A multi-layer graph data structure enabling fast approximate nearest neighbor search.

  Advanced
• Hardware & Accelerators

  High-Bandwidth Memory Architecture

  A specialized technique in hardware & accelerators providing hbm3e memory integration in gpus capabilities for advanced enterprise AI applications.

  Advanced
• Practice

  Human in the Loop

  Designing AI systems so a person reviews, approves or corrects the output at defined points.

  Beginner
• Evaluation

  HumanEval Coding Benchmark

  OpenAI's benchmark evaluating Python code generation capabilities using pass@k metrics.

  Intermediate
• Search Engineering

  Hybrid Search

  Combines BM25 keyword matching with vector semantic search for optimal retrieval accuracy.

  Intermediate
• Core Architecture

  Hybrid SSM-Transformer Models

  A specialized technique in core architecture providing jamba architecture integration capabilities for advanced enterprise AI applications.

  Intermediate
• RAG & Vector Search

  Hybrid Vector Keyword Search

  Search architecture that combines dense vector embeddings with sparse BM25 keyword matching using reciprocal rank fusion.

  Intermediate
• Practice

  Hyperparameter

  A configuration value chosen before training — such as learning rate, batch size or number of layers — that is not learned from data.

  Intermediate
• Generative Media

  Image Diffusion Models

  Generates images by iteratively denoising Gaussian noise guided by text embeddings.

  Intermediate
• Applications

  Image Recognition

  Identifying what an image depicts by assigning it one or more labels.

  Beginner
• Generative AI

  In-Context Learning

  A model's ability to adapt to a new task from information in the prompt alone, without any weight updates.

  Intermediate
• AI Security

  Indirect Prompt Injection

  Hiding malicious instructions inside external content (web pages, PDFs) read by an AI assistant.

  Intermediate
• Infrastructure

  Inference

  Running a trained model to produce outputs, as opposed to training it.

  Beginner
• Practice

  Instruction Tuning

  Fine-tuning a base model on instruction-and-response pairs so it follows requests rather than merely continuing text.

  Intermediate
• RAG & Vector Search

  Instruction-Tuned Vector Embeddings

  A specialized technique in rag & vector search providing query-task conditioned embeddings capabilities for advanced enterprise AI applications.

  Intermediate
• Hardware & Accelerators

  Inter-GPU Interconnect Fabric

  A specialized technique in hardware & accelerators providing nvlink 4 and infiniband fabrics capabilities for advanced enterprise AI applications.

  Intermediate
• Safety

  Interpretability

  Research into understanding what is actually happening inside a neural network's computations.

  Advanced
• Vector Indexing

  Inverted File Index (IVF)

  Clustering vector space into Voronoi cells to narrow search scope during vector lookup.

  Advanced
• RAG & Vector Search

  Inverted File Index with Product Quantization

  A specialized technique in rag & vector search providing ivfpq vector indexing capabilities for advanced enterprise AI applications.

  Intermediate
• AI Safety & Ethics

  Jailbreak Resistance Benchmarking

  A specialized technique in ai safety & ethics providing stress-testing system safety prompts capabilities for advanced enterprise AI applications.

  Intermediate
• AI Security

  Jailbreaking Techniques

  Adversarial prompts crafted to bypass model safety filters and alignment guardrails.

  Intermediate
• Alignment

  Kahneman-Tversky Optimization (KTO)

  Aligns models directly from binary signal (good/bad) without requiring paired preferences.

  Advanced
• Model Training

  Knowledge Distillation

  Transfers knowledge from a large teacher model into a smaller, faster student model.

  Intermediate
• Data

  Knowledge Graph

  A structured network of entities and typed relationships that machines can traverse and reason over.

  Intermediate
• RAG & Vector Search

  Knowledge Graph Augmented Generation

  A specialized technique in rag & vector search providing kg-rag for structured entity queries capabilities for advanced enterprise AI applications.

  Intermediate
• Infrastructure

  KV Cache

  The stored key and value tensors from previous tokens that let a model generate each new token without recomputing the whole sequence.

  Advanced
• Inference

  KV Cache Optimization

  Caches key and query activations to prevent redundant computation during autoregressive generation.

  Intermediate
• Generative AI

  Large Language Model

  A very large neural network trained on text to predict the next token, which yields broad language, reasoning and coding ability.

  Beginner
• RAG & Vector Search

  Late Interaction Embedding Models

  A specialized technique in rag & vector search providing colbert token-level vector matching capabilities for advanced enterprise AI applications.

  Intermediate
• Generative Media

  Latent Consistency Models (LCM)

  Accelerates image generation to 1-4 steps by distilling diffusion models in latent space.

  Advanced
• Multimodal AI

  Latent Diffusion Noise Schedulers

  A specialized technique in multimodal ai providing ddim and euler noise step control capabilities for advanced enterprise AI applications.

  Intermediate
• Generative AI

  Latent Space Manipulation

  Traversing and editing feature vectors in latent space to control output characteristics.

  Intermediate
• Practice

  Learning Rate

  The step size that controls how far parameters move on each optimisation update.

  Intermediate
• Core Architecture

  Linear Attention Formulations

  A specialized technique in core architecture providing sub-quadratic complexity transformers capabilities for advanced enterprise AI applications.

  Intermediate
• Practice

  LLM Evaluation

  Measuring the quality, safety and cost of language-model outputs where there is rarely one correct answer.

  Advanced
• LLM Engineering

  LLM Memory Management

  Architecting chat history summarization, sliding context windows, and long-term vector retrieval.

  Intermediate
• Evaluation & Benchmarking

  LLM-as-a-Judge

  Uses frontier language models to automatically evaluate and score response quality.

  Intermediate
• Evaluation & Benchmarks

  LLM-as-a-Judge Evaluation

  An evaluation methodology that uses state-of-the-art LLMs like GPT-4 or Claude 3.5 to grade candidate model responses.

  Intermediate
• Practice

  LLMOps (2)

  Operating language-model applications: managing prompts, evaluations, cost, latency, safety and provider changes.

  Intermediate
• Evaluation

  LMSYS Chatbot Arena Elo

  A crowdsourced open platform benchmarking LLMs through anonymous side-by-side human preference battles.

  Beginner
• Architectures

  Long Short-Term Memory

  A recurrent cell with input, forget and output gates that lets useful information persist across many time steps.

  Advanced
• Practice

  LoRA (3)

  A parameter-efficient fine-tuning method that trains small low-rank matrices while keeping the original weights frozen.

  Advanced
• Foundations

  Loss Function

  The mathematical definition of how wrong a prediction is, and therefore what the model is actually being optimised to do.

  Intermediate
• Model Compression & Fine-Tuning

  Low-Rank Adaptation (LoRA)

  A parameter-efficient fine-tuning technique that freezes pre-trained model weights and injects trainable rank decomposition matrices.

  Intermediate
• Foundations

  Machine Learning

  A branch of AI where systems improve at a task by learning statistical patterns from data instead of following hand-written rules.

  Beginner
• Applications

  Machine Translation

  Automatically converting text or speech from one language into another.

  Beginner
• Architecture

  Mamba Architecture

  A selective state-space model offering linear-time context scaling as an alternative to Transformers.

  Advanced
• Self-Supervised Learning

  Masked Autoencoders (MAE)

  Self-supervised vision model that reconstructs missing image patches from high masking ratios.

  Advanced
• AI Safety

  Mechanistic Circuit Analysis

  Tracing exact sub-networks inside models responsible for specific behaviors like induction.

  Advanced
• AI Safety & Ethics

  Mechanistic Feature Steering

  A safety and control technique that modifies internal LLM hidden activations along specific sparse autoencoder feature directions.

  Advanced
• AI Safety & Interpretability

  Mechanistic Interpretability

  The reverse-engineering of neural network weights and activations into understandable algorithms and circuit diagrams.

  Advanced
• AI Safety & Ethics

  Mechanistic Model Editing

  A specialized technique in ai safety & ethics providing modifying internal knowledge vectors capabilities for advanced enterprise AI applications.

  Intermediate
• AI Security

  Membership Inference Attacks

  Determining whether a specific individual record was included in an AI model's training dataset.

  Advanced
• Agents

  Memory Networks in Agents

  Provides short-term, working, and episodic memory to autonomous agents across sessions.

  Intermediate
• LLM Sampling

  Min-P Sampling

  Filters out tokens whose probability is below a threshold relative to the top token's probability.

  Intermediate
• Training Acceleration

  Mixed-Precision Training (FP16 / BF16)

  Executing neural calculations in 16-bit float format while maintaining 32-bit master weights for speed.

  Intermediate
• Architecture

  Mixture of Depths (MoD)

  Dynamically allocates compute depth per token, skipping layers for simpler tokens.

  Advanced
• Architectures

  Mixture of Experts

  An architecture where a router activates only a few specialised sub-networks per token, so total parameters can grow without proportionally growing compute.

  Advanced
• Model Architecture & Scaling

  Mixture of Experts (MoE)

  A sparse neural network architecture that routes inputs to specialized sub-networks (experts) to scale parameter count without matching compute overhead.

  Advanced
• Practice

  MLOps (2)

  The discipline of deploying, monitoring and maintaining machine learning systems reliably in production.

  Intermediate
• Evaluation

  MMLU-Pro Benchmark

  An updated multi-task language understanding benchmark with harder multi-choice reasoning questions.

  Intermediate
• Evaluation & Benchmarks

  Model Anonymization in Benchmarking

  A specialized technique in evaluation & benchmarks providing double-blind elo evaluation capabilities for advanced enterprise AI applications.

  Intermediate
• AI Transparency

  Model Card Documentation

  Standardized documentation detailing model architecture, training data, evaluations, and intended uses.

  Beginner
• Agents

  Model Context Protocol

  An open protocol for connecting AI assistants to external tools, data sources and prompts through a standard interface.

  Advanced
• AI Infrastructure

  Model Context Protocol Standard

  An open protocol standard created to securely connect AI models to external data sources, developer tools, and local environments.

  Intermediate
• Practice

  Model Distillation

  Training a smaller student model to reproduce the behaviour of a larger teacher model.

  Advanced
• Risks

  Model Drift

  The gradual decay of model performance as the world changes away from the data it was trained on.

  Intermediate
• Practice

  Model Evaluation

  Measuring whether a model actually works, on data it has never seen, using metrics matched to the decision it supports.

  Intermediate
• AI Security

  Model Inversion Attacks

  Reconstructing private training data samples by probing a model's output confidence scores.

  Advanced
• Practice

  Model Leaderboard

  A ranked public table comparing models on benchmarks or head-to-head human preference votes.

  Beginner
• Model Engineering

  Model Merging

  Combines weight matrices from multiple fine-tuned models into a single superior checkpoint.

  Intermediate
• Fine-Tuning & Optimization

  Model Merging via Task Vectors

  A specialized technique in fine-tuning & optimization providing adding and subtracting fine-tuned tasks capabilities for advanced enterprise AI applications.

  Intermediate
• Foundations

  Model Parameters

  The learned weights inside a model, whose count is the usual shorthand for model size.

  Beginner
• Model Compression

  Model Quantization

  Reduces weight precision from FP16 to INT8 or INT4 to decrease VRAM footprint.

  Intermediate
• Infrastructure

  Model Serving

  The systems that expose a trained model as a reliable, scalable, monitored endpoint.

  Advanced
• Data Engineering

  Model-Assisted Data Annotation

  A specialized technique in data engineering providing automated labeling via foundation models capabilities for advanced enterprise AI applications.

  Intermediate
• AI Infrastructure

  Molecular Property Prediction AI

  A specialized technique in ai infrastructure providing graph neural networks for chemistry capabilities for advanced enterprise AI applications.

  Intermediate
• AI Safety & Ethics

  Monosemantic Feature Disentanglement

  A specialized technique in ai safety & ethics providing decomposing dense neurons via saes capabilities for advanced enterprise AI applications.

  Intermediate
• Agents

  Multi-Agent Collaboration

  Orchestrates specialized AI agents working together with defined roles and communication protocols.

  Advanced
• Agentic AI

  Multi-Agent Debate Protocols

  Orchestration protocols where multiple specialized AI agents critique, defend, and refine answers collaboratively to eliminate errors.

  Intermediate
• Agents

  Multi-Agent System

  An architecture where several specialised agents collaborate, delegate or debate to complete a task.

  Advanced
• Core Architecture

  Multi-Head Latent Attention

  A specialized technique in core architecture providing mla architecture in deepseek models capabilities for advanced enterprise AI applications.

  Intermediate
• Architecture

  Multi-Query Attention (MQA)

  Shares a single key and value head across all query heads to minimize KV cache RAM.

  Advanced
• RAG & Vector Search

  Multi-Query Expansion Retrieval

  A specialized technique in rag & vector search providing generating multiple search variants capabilities for advanced enterprise AI applications.

  Intermediate
• RAG Systems

  Multi-Vector Retrieval

  Stores multiple embedding vectors per document (e.g. summaries, key questions, tables) to maximize recall.

  Advanced
• Generative AI

  Multimodal AI

  Models that accept or produce more than one data type — text, images, audio, video — within a single system.

  Intermediate
• Multimodal AI

  Multimodal Alignment

  Projects vision, audio, and text embeddings into a shared semantic space.

  Advanced
• Multimodal AI

  Multimodal Patch Embeddings

  A specialized technique in multimodal ai providing converting images into vision tokens capabilities for advanced enterprise AI applications.

  Intermediate
• RAG Systems

  Multimodal RAG

  Retrieving and contextualizing images, charts, and slide decks alongside text documents for LLM context.

  Advanced
• Core Architecture

  Multimodal Rotary Position Embeddings

  An extension of RoPE that encodes 1D text positions alongside 2D image spatial coordinates and 3D video temporal frames.

  Advanced
• Applications

  Named Entity Recognition

  Identifying and labelling spans of text that refer to people, organisations, places, dates, amounts or domain-specific entities.

  Intermediate
• Foundations

  Narrow AI

  AI built for a specific task or domain, which is what every deployed system today actually is.

  Beginner
• Foundations

  Natural Language Processing

  The field concerned with getting computers to process, understand and generate human language.

  Beginner
• Multimodal AI

  Neural Audio Codec Tokenization

  A specialized technique in multimodal ai providing converting audio waves to discrete tokens capabilities for advanced enterprise AI applications.

  Intermediate
• Foundations

  Neural Network

  A model made of layers of connected units, each computing a weighted sum of its inputs followed by a non-linear function.

  Beginner
• AI Infrastructure

  Neural Partial Differential Equation Solvers

  A specialized technique in ai infrastructure providing pinn and fno differential solvers capabilities for advanced enterprise AI applications.

  Intermediate
• 3D AI

  Neural Radiance Fields (NeRF)

  Represents 3D scenes as continuous neural volumetric functions trained from 2D camera views.

  Advanced
• Foundations

  Neuro-Symbolic AI

  Approaches that combine neural pattern recognition with symbolic reasoning, logic or structured knowledge.

  Advanced
• Fine-Tuning & Optimization

  NormalFloat4 Quantization Format

  A specialized technique in fine-tuning & optimization providing nf4 data type in qlora capabilities for advanced enterprise AI applications.

  Intermediate
• Applications

  Object Detection

  Locating and classifying multiple objects within an image or video frame, usually with bounding boxes.

  Intermediate
• Alignment & Preference Tuning

  Odds Ratio Preference Optimization

  A specialized technique in alignment & preference tuning providing orpo monolithic sft-alignment capabilities for advanced enterprise AI applications.

  Intermediate
• Data

  Ontology

  A formal specification of the concepts in a domain and the relationships allowed between them.

  Advanced
• AI Policy

  Open Source vs Open Weights

  Distinguishing between fully open training datasets/code and distributing trained weight files.

  Beginner
• Generative AI

  Open-Weight Model

  A model whose trained parameters are published for download, allowing self-hosting, inspection and modification under a licence.

  Intermediate
• Practice

  Overfitting

  When a model memorises noise and specifics of the training set instead of learning patterns that generalise.

  Beginner
• Fine-Tuning & Optimization

  Paged Optimizer Memory Management

  A specialized technique in fine-tuning & optimization providing preventing oom during fine-tuning capabilities for advanced enterprise AI applications.

  Intermediate
• Inference

  PagedAttention

  Allocates KV cache memory in non-contiguous virtual pages to eliminate VRAM fragmentation.

  Advanced
• Fine-Tuning & Optimization

  Parameter-Efficient Fine-Tuning Protocols

  A specialized technique in fine-tuning & optimization providing peft methodologies overview capabilities for advanced enterprise AI applications.

  Intermediate
• RAG Systems

  Parent Document Retrieval

  Retrieves small text chunks for vector matching but feeds larger parent context chunks to the LLM.

  Intermediate
• Foundations

  Perceptron

  The earliest neural unit: a weighted sum of inputs passed through a threshold to produce a binary decision.

  Beginner
• Distributed Training

  Pipeline Parallelism

  Partitioning model layers sequentially across different GPUs in a pipeline setup.

  Advanced
• Core Architecture

  Pipeline Parallelism Optimization

  A specialized technique in core architecture providing stage-wise gpu model partitioning capabilities for advanced enterprise AI applications.

  Intermediate
• Learning Paradigms

  Policy Gradient Methods

  Reinforcement learning algorithms that adjust the action-selection distribution directly using gradients of expected reward.

  Advanced
• Data Engineering

  Pre-Training Data Deduplication

  A specialized technique in data engineering providing minhash lsh for massive text corpora capabilities for advanced enterprise AI applications.

  Intermediate
• Practice

  Precision and Recall

  Precision is how many flagged items were correct; recall is how many of the real cases were caught.

  Beginner
• Alignment

  Preference Fine-Tuning

  Aligns model outputs with human preferences using pairwise comparison datasets.

  Intermediate
• Practice

  Pretraining

  The initial, most expensive training phase where a model learns general structure from a very large unlabelled corpus.

  Advanced
• Alignment & Preference Tuning

  Process-Supervised Reward Models

  Reward models trained to evaluate and provide scalar feedback on individual reasoning steps rather than just the final output.

  Advanced
• Vector Indexing

  Product Quantization (PQ)

  Compresses high-dimensional vectors into compact byte codes to save vector DB RAM.

  Advanced
• Practice

  Prompt Engineering

  The practice of designing inputs that reliably get useful, correct and well-formatted output from a generative model.

  Beginner
• Risks

  Prompt Injection

  An attack where malicious instructions hidden in content the model reads cause it to ignore its real instructions.

  Intermediate
• AI Security

  Prompt Injection Defense

  Security controls preventing malicious user inputs from overriding system instructions.

  Intermediate
• AI Infrastructure

  Protein Language Model Embeddings

  A specialized technique in ai infrastructure providing esm evolutionary protein sequences capabilities for advanced enterprise AI applications.

  Intermediate
• Reinforcement Learning

  Proximal Policy Optimization (PPO)

  An actor-critic RL algorithm used to align language models while preventing drastic policy updates.

  Advanced
• Learning Paradigms

  Q-Learning

  A value-based reinforcement learning algorithm that learns the expected long-term reward of taking an action in a given state.

  Advanced
• Infrastructure

  Quantization

  Storing and computing model weights at lower numerical precision to cut memory use and speed up inference.

  Advanced
• Inference & Serving

  Quantized KV Cache Compression

  A specialized technique in inference & serving providing int4 and fp8 kv cache storage capabilities for advanced enterprise AI applications.

  Intermediate
• Model Compression & Fine-Tuning

  Quantized Low-Rank Adaptation (QLoRA)

  An advanced PEFT method that quantizes the base model to 4-bit NormalFloat precision while training LoRA adapters on top.

  Advanced
• AI Infrastructure

  Quantum Neural Network Hybrid Architectures

  A specialized technique in ai infrastructure providing qml kernels for classical models capabilities for advanced enterprise AI applications.

  Advanced
• RAG Systems

  Query Rewriting

  Transforms raw user queries into multiple search-optimized queries for improved retrieval recall.

  Intermediate
• Agentic AI

  ReAct Agent Execution Loop

  A specialized technique in agentic ai providing interleaving reasoning and tool action capabilities for advanced enterprise AI applications.

  Intermediate
• AI Agents & Tool Integration

  ReAct Framework (Reasoning + Acting)

  A paradigm that combines verbal reasoning traces with task-specific actions to enable autonomous LLM agent execution.

  Intermediate
• Generative AI

  Reasoning Model

  A language model trained to deliberate internally before answering, trading latency for accuracy on hard problems.

  Intermediate
• Search Engineering

  Reciprocal Rank Fusion (RRF)

  Re-ranks multi-source retrieval results by combining reciprocal rank positions.

  Intermediate
• Applications

  Recommender System

  A system that ranks items for a specific user based on past behaviour, item attributes and context.

  Intermediate
• Generative Media

  Rectified Flow Matching

  Generative formulation that connects noise and data via straight trajectories for fast sampling.

  Advanced
• Architectures

  Recurrent Neural Network

  A network that processes sequences one step at a time while carrying a hidden state forward.

  Intermediate
• Agents

  Reflection & Self-Correction

  Prompts agents to evaluate their previous code or output and fix identified bugs autonomously.

  Intermediate
• Inference & Serving

  Regex-Constrained Token Generation

  A specialized technique in inference & serving providing outlines regex softmax masking capabilities for advanced enterprise AI applications.

  Intermediate
• Tasks

  Regression

  Predicting a continuous numeric value from input features.

  Beginner
• Practice

  Regularization

  Techniques that constrain a model so it generalises better rather than fitting the training set perfectly.

  Intermediate
• Learning Paradigms

  Reinforcement Learning

  Learning a policy by acting in an environment and adjusting behaviour based on reward signals rather than labelled answers.

  Advanced
• Alignment

  Reinforcement Learning from AI Feedback (RLAIF)

  Using AI models to generate preference signals for alignment training, reducing human annotation costs.

  Advanced
• Alignment & Preference Tuning

  Reinforcement Learning from Execution Feedback

  A specialized technique in alignment & preference tuning providing rlef in coding agents capabilities for advanced enterprise AI applications.

  Intermediate
• Alignment

  Reinforcement Learning from Human Feedback

  A tuning method where human preference comparisons train a reward model, which then guides reinforcement learning of the assistant's behaviour.

  Advanced
• Alignment

  Reinforcement Learning from Human Feedback (RLHF)

  Fine-tunes models using a reward model trained on human preference ratings.

  Advanced
• LLM Sampling

  Repetition Penalty Decoding

  Penalizes previously generated tokens to prevent infinite text loops in autoregressive sampling.

  Beginner
• Fine-Tuning & Optimization

  Representation Fine-Tuning

  A parameter-efficient fine-tuning approach that learns interventions on hidden representations rather than modifying model weights.

  Advanced
• Architecture Patterns

  Reranking

  A second-stage model that re-scores retrieved candidates for relevance before they are passed to the generator.

  Advanced
• Infrastructure

  Response Streaming

  Sending a model's output token by token as it is generated instead of waiting for the full response.

  Beginner
• Ethics

  Responsible AI

  The operational practice of building and running AI systems that are fair, transparent, accountable and safe.

  Intermediate
• Architecture Patterns

  Retrieval-Augmented Generation

  An architecture that retrieves relevant documents at query time and gives them to a language model as context for its answer.

  Intermediate
• Information Retrieval & Memory

  Retrieval-Augmented Generation (RAG)

  An architecture that enhances LLM responses by fetching relevant external knowledge from databases before generating text.

  Intermediate
• Alignment

  Reward Model

  A model trained to score outputs the way human raters would, used as the optimisation target during preference tuning.

  Advanced
• Alignment

  Reward Modeling

  Training a neural network to score model completions based on human preference choices.

  Advanced
• Alignment

  RLAIF (RL from AI Feedback)

  Replaces human annotators with frontier AI models to evaluate and reward model behavior.

  Advanced
• Architecture

  RMSNorm Layer Normalization

  A computationally efficient alternative to LayerNorm that scales activations using root mean square.

  Intermediate
• Core Architecture

  Root Mean Square Normalization

  A specialized technique in core architecture providing rmsnorm layer normalization technique capabilities for advanced enterprise AI applications.

  Intermediate
• Model Architecture & Transformers

  Rotary Position Embedding (RoPE)

  A relative positional encoding method that rotates query and key vectors in complex space to preserve distance relationships across long contexts.

  Advanced
• Core Architecture

  Rotary Position Embeddings

  A specialized technique in core architecture providing rope relative positional encoding capabilities for advanced enterprise AI applications.

  Intermediate
• Foundations

  Scaling Laws

  Empirical relationships showing that model loss improves predictably as parameters, data and compute increase together.

  Advanced
• Computer Vision

  Segment Anything Model Architecture

  Promptable segmentation model capable of isolating any object in an image with zero-shot generalization.

  Intermediate
• Practice

  Self-Consistency

  Sampling several independent reasoning paths for the same question and taking the most common answer.

  Advanced
• Prompt Engineering

  Self-Consistency Prompting

  A decoding technique that samples multiple diverse Chain of Thought reasoning paths from an LLM and selects the most consistent answer.

  Intermediate
• Prompt Engineering

  Self-Consistency Sampling Protocols

  A specialized technique in prompt engineering providing majority vote reasoning extraction capabilities for advanced enterprise AI applications.

  Intermediate
• Data Engineering

  Self-Instruct

  Uses an LLM to generate its own instruction-following training datasets from seed prompts.

  Intermediate
• RAG Systems

  Self-Querying Retrievers

  Uses an LLM to separate natural language semantic queries from structural metadata filters.

  Intermediate
• RAG Systems

  Self-RAG

  Trainable framework where the model outputs special reflection tokens to decide when to retrieve and critique results.

  Advanced
• Alignment & Preference Tuning

  Self-Rewarding Language Models

  A specialized technique in alignment & preference tuning providing llms generating their own alignment data capabilities for advanced enterprise AI applications.

  Intermediate
• Learning Paradigms

  Self-Supervised Learning

  Training on unlabelled data by turning the data itself into a prediction task, such as predicting the next token or a masked patch.

  Advanced
• LLM Operations

  Semantic Caching

  Caches previous LLM responses based on query embedding vector similarity to save latency and cost.

  Intermediate
• Applications

  Semantic Search

  Search that matches meaning rather than exact keywords, using embeddings to compare intent.

  Beginner
• Learning Paradigms

  Semi-Supervised Learning

  Learning from a small labelled set combined with a much larger pool of unlabelled examples.

  Advanced
• Applications

  Sentiment Analysis

  Classifying the emotional tone or opinion expressed in a piece of text.

  Beginner
• Practice

  Simulation for AI

  Training and testing AI systems in synthetic environments before exposing them to the real world.

  Advanced
• Prompt Engineering

  Skeleton-of-Thought (SoT)

  Guides LLMs to draft an answer outline first and then expand points in parallel for fast latency.

  Intermediate
• Architecture

  Sliding Window Attention (SWA)

  Restricts attention computation to a local fixed-size context window to achieve linear time complexity.

  Intermediate
• Architecture

  Sliding Window Attention Mechanisms

  Restricts token attention to a fixed local window size to achieve linear compute scaling.

  Intermediate
• Generative AI

  Small Language Model

  A compact language model, typically under roughly ten billion parameters, designed for cheap, fast or on-device use.

  Intermediate
• Prompting

  Soft Prompting

  Prepends continuous trainable embedding vectors to prompts without changing model weights.

  Intermediate
• LLM Sampling

  Softmax Temperature Scaling

  Controls output randomness by scaling logit values before applying the softmax function.

  Beginner
• Architecture

  Sparse Attention

  Reduces Transformer quadratic attention complexity to linear or sub-quadratic scaling.

  Advanced
• AI Safety & Ethics

  Sparse Autoencoder Dictionary Learning

  A mechanistic interpretability method that decomposes dense LLM activations into millions of sparse, human-interpretable concepts.

  Advanced
• Core Architecture

  Sparse Mixture of Experts Routing

  A specialized technique in core architecture providing top-k gating mechanisms in moe capabilities for advanced enterprise AI applications.

  Intermediate
• RAG & Vector Search

  Sparse Vector Representation

  A specialized technique in rag & vector search providing splade lexical-semantic embeddings capabilities for advanced enterprise AI applications.

  Intermediate
• Multimodal AI

  Spatial-Temporal Video Attention

  A specialized technique in multimodal ai providing processing multi-frame video inputs capabilities for advanced enterprise AI applications.

  Intermediate
• Risks

  Specification Gaming

  When a system satisfies the literal objective it was given while defeating the intent behind it.

  Advanced
• Inference Optimization & Latency

  Speculative Decoding

  An inference acceleration technique where a small draft model predicts multiple tokens and a large target model verifies them in parallel.

  Advanced
• Inference & Serving

  Speculative Decoding Pipelines

  An inference acceleration technique that uses a smaller draft model to propose tokens validated in parallel by a larger target LLM.

  Advanced
• Inference

  Speculative Sampling

  Using small draft models to predict sequence tokens for fast parallel verification.

  Advanced
• Applications

  Speech Recognition

  Converting spoken audio into written text, optionally with speaker labels and timestamps.

  Beginner
• Applications

  Speech Synthesis

  Converting written text into spoken audio with natural prosody and timing.

  Beginner
• Audio AI

  Speech Synthesis Modeling

  Architectures generating natural human speech from text input with voice cloning capabilities.

  Intermediate
• Multimodal AI

  Speech-to-Speech Latency Optimization

  A specialized technique in multimodal ai providing sub-200ms voice agent architecture capabilities for advanced enterprise AI applications.

  Intermediate
• Model Architecture

  Spherical Linear Interpolation Merging

  A mathematical technique used to combine two fine-tuned checkpoint weights by interpolating along a spherical vector trajectory.

  Advanced
• Architectures

  State Space Model

  A sequence architecture that maintains a compressed continuous state, offering near-linear scaling with sequence length.

  Advanced
• Core Architecture

  State Space Model Foundations

  A specialized technique in core architecture providing mamba and selective ssm logic capabilities for advanced enterprise AI applications.

  Intermediate
• Architecture

  State Space Models (SSM)

  Sequence models processing long contexts with linear time complexity via state space formulations.

  Advanced
• Foundations

  Stochastic Parrot

  A critical term for large language models, arguing they stitch together training text by statistical likelihood without grounded understanding.

  Intermediate
• Data

  Structured Data and Schema

  Machine-readable markup that tells search engines and AI systems what a page's content actually represents.

  Beginner
• Practice

  Structured Output

  Constraining a model to emit data that conforms to a specified schema so downstream code can rely on it.

  Intermediate
• RAG Systems

  Sub-Query Decomposition

  Breaks complex multi-part questions into simpler sub-queries executed in parallel.

  Intermediate
• Foundations

  Superintelligence

  A hypothetical intelligence substantially exceeding the best human performance across essentially all domains.

  Advanced
• Fine-Tuning

  Supervised Fine-Tuning (SFT)

  Trains a pre-trained base model on high-quality instruction-response pairs.

  Intermediate
• Learning Paradigms

  Supervised Learning

  Training a model on input-output pairs so it can predict the correct label for new, unlabelled inputs.

  Beginner
• Agentic AI

  Swarm Agent Architecture

  An orchestration pattern where numerous specialized, lightweight agents interact through dynamic handoffs and shared state.

  Advanced
• Evaluation & Coding AI

  SWE-bench Evaluation

  A benchmark testing AI software agents on resolving real-world GitHub issues in Python repositories.

  Advanced
• Architecture

  SwiGLU Activation Function

  A gated linear unit activation function combining Swish and GLU used in modern LLMs like Llama.

  Intermediate
• Core Architecture

  Swish Gated Linear Units

  A specialized technique in core architecture providing swiglu activation in transformer ffns capabilities for advanced enterprise AI applications.

  Intermediate
• AI Safety

  Sycophancy Mitigation

  Techniques preventing models from falsely agreeing with user bias instead of truth.

  Intermediate
• Foundations

  Symbolic AI

  AI built from explicit symbols, logic and hand-written rules rather than learned statistical patterns.

  Intermediate
• Data

  Synthetic Data

  Artificially generated examples used to augment or replace real data for training and testing.

  Intermediate
• Data Engineering & Pre-training

  Synthetic Data Generation

  The creation of artificial training data using generative models to train downstream AI models when real-world data is scarce or sensitive.

  Intermediate
• Data Engineering

  Synthetic Data Quality Filtering

  Automated pipelines that evaluate, filter, and score synthetic text datasets to ensure only high-quality data enters training loops.

  Intermediate
• AI Safety & Ethics

  Synthetic Watermarking in Text Models

  A specialized technique in ai safety & ethics providing statistical provenance tracing capabilities for advanced enterprise AI applications.

  Intermediate
• Practice

  System Prompt

  The persistent instruction block that sets an assistant's role, rules and output conventions before any user message.

  Beginner
• Prompt Engineering

  System Prompt Engineering

  Crafting core persona, boundaries, and formatting instructions at the root of conversational LLMs.

  Beginner
• AI Safety & Ethics

  System Prompt Leak Mitigation

  A specialized technique in ai safety & ethics providing defending against system prompt extraction capabilities for advanced enterprise AI applications.

  Intermediate
• Practice

  Temperature

  A sampling setting that controls how random a model's token choices are.

  Beginner
• RAG Systems

  Temporal Graph RAG

  Incorporates timestamped relationship edges into knowledge graphs for time-sensitive query retrieval.

  Advanced
• RAG & Vector Search

  Temporal Vector Search

  A specialized technique in rag & vector search providing time-aware document embeddings capabilities for advanced enterprise AI applications.

  Intermediate
• Distributed Training

  Tensor Parallelism

  Splitting individual matrix multiplications in neural layers across multiple GPUs.

  Advanced
• Core Architecture

  Tensor Parallelism in LLM Training

  A specialized technique in core architecture providing splitting matrix operations across gpus capabilities for advanced enterprise AI applications.

  Intermediate
• Generative AI

  Test-Time Compute

  Improving answer quality by spending more computation at inference — thinking longer, sampling more, or verifying — rather than by training a bigger model.

  Advanced
• Inference & Serving

  Test-Time Compute Scaling

  The strategy of increasing computational resources at inference time (via search, sampling, or verification) to boost model accuracy.

  Advanced
• Applications

  Text Analytics

  Deriving measurable insight from bodies of text such as reviews, tickets, surveys and transcripts.

  Beginner
• Applications

  Text-to-Video Generation

  Generating moving footage from a text prompt, an image, or a combination of both.

  Intermediate
• Tasks

  Time Series Forecasting

  Predicting future values of a sequence measured over time, respecting the ordering of observations.

  Intermediate
• Generative AI

  Tokenization

  Splitting text into the sub-word units a language model actually reads, and the basis of context limits and pricing.

  Beginner
• Agents

  Tool Use

  Letting a model invoke external functions, APIs or code so it can act and fetch facts instead of guessing.

  Intermediate
• Agents

  Tool Use & Function Calling

  Enables LLMs to structure arguments and trigger external APIs or code execution.

  Intermediate
• LLM Sampling

  Top-K & Top-P (Nucleus) Sampling

  Truncates probability distributions to the top K tokens or cumulative top P probability threshold.

  Beginner
• Data

  Training Data

  The examples a model learns from, and the single biggest determinant of what it can and cannot do.

  Beginner
• Practice

  Transfer Learning

  Reusing a model trained on one large task as the starting point for a different, usually smaller, task.

  Intermediate
• Architectures

  Transformer

  The neural architecture, introduced in 2017, that uses self-attention to relate every position in a sequence to every other position.

  Intermediate
• Practice

  Tree of Thought

  A prompting strategy that explores multiple reasoning branches, evaluates them and backtracks rather than committing to one linear chain.

  Advanced
• Prompt Engineering & Search

  Tree of Thoughts (ToT)

  A framework that generalizes Chain-of-Thought by allowing LLMs to explore multiple reasoning paths and self-evaluate intermediate choices.

  Intermediate
• Prompt Engineering

  Tree of Thoughts Prompting

  A deliberate prompting framework that generalizes Chain of Thought by allowing models to explore multiple reasoning paths in parallel.

  Intermediate
• GPU Acceleration

  Triton Kernel Programming

  Open-source Python-based programming language for writing custom high-performance GPU code.

  Advanced
• Foundations

  Turing Test

  Alan Turing's 1950 proposal that a machine could be considered intelligent if a human judge could not reliably distinguish it from a person in conversation.

  Beginner
• Learning Paradigms

  Unsupervised Learning

  Finding structure in data that has no labels, typically through clustering, dimensionality reduction or density estimation.

  Intermediate
• Infrastructure

  Vector Database

  A database optimised for storing embeddings and finding the nearest vectors to a query at scale.

  Intermediate
• Infrastructure

  Vector Index Sharding

  Distributing large-scale vector indexes across multiple server nodes for horizontal scaling.

  Advanced
• Applications

  Vibe Coding

  Building software mainly by describing intent to an AI system and iterating on what it produces, rather than writing most code by hand.

  Beginner
• Architectures

  Vision Transformer

  A transformer applied to images by splitting them into patches and treating each patch as a token.

  Advanced
• Computer Vision

  Vision Transformer (ViT)

  Applies standard Transformer self-attention blocks directly to image patches for visual recognition.

  Intermediate
• Multimodal AI

  Vision Transformer Projection Layers

  A specialized technique in multimodal ai providing bridging vision encoders to llms capabilities for advanced enterprise AI applications.

  Intermediate
• Applications

  Voice Cloning

  Synthesising speech that reproduces a specific person's voice from a short reference sample.

  Intermediate
• Optimization

  Warmup & Cosine Decay Schedulers

  Learning rate schedules that gradually warm up learning rates before applying cosine decay curves.

  Intermediate
• AI Safety

  Watermarking AI Images

  Injecting imperceptible noise patterns into image pixels to verify synthetic media origin.

  Intermediate
• AI Safety

  Watermarking AI-Generated Text

  Embedding subtle mathematical patterns into output token distributions to detect synthetic text.

  Intermediate
• Model Compression

  Weight Pruning

  Removes non-essential neural connections to reduce model size with minimal accuracy loss.

  Advanced
• Fine-Tuning & Optimization

  Weight-Decomposed Low-Rank Adaptation

  A specialized technique in fine-tuning & optimization providing dora parameter adaptation capabilities for advanced enterprise AI applications.

  Intermediate
• Core Architecture

  ZeRO Memory Optimization Protocol

  A specialized technique in core architecture providing deepspeed memory stage 1-3 capabilities for advanced enterprise AI applications.

  Intermediate
• Distributed Training

  Zero Redundancy Optimizer (ZeRO)

  Memory optimization technique sharding memory states across data parallel processes.

  Advanced
• Practice

  Zero-Shot Learning

  Asking a model to perform a task with only an instruction and no worked examples.

  Beginner