Home

Note

Notes here are not meant to be comprehensive. Use it for revision or quick reference.

• Deep Generative Models

  • Transformer Architectures
    • Llama
    • Mistral
  • GANs
  • VAEs
  • Top NLP papers
  • Fine Tuning
  • RAG (To Update)

• Transformer Toolkit

  • Attention Mechanisms and Its Variants

    • MHA: Multi-Head Attention
    • MQA: Multi-Query Attention
    • GQA: Grouped Query Attention
    • Sliding Window Attention

  • KV Cache

  • Position Encodings

    • Absolute Positional Embeddings

    • Rotary Positional Embeddings

  • Tokenization Techniques (TODO)

  • Inference Techniques (TODO)
  • Training Techniques (TODO)

• Basic Concepts

  • Activation Functions
  • Loss Functions (TODO)
  • Optimization Algorithms
  • Regularization Techniques
  • Normalization Techniques
  • Evaluation Metrics
    • Classification Metrics (TODO)
    • Regression Metrics (TODO)
    • Clustering Metrics (TODO)
    • NLP Evaluation Metrics
    • Computer Vision Evaluation Metrics (TODO)
    • Generative Models Evaluation Metrics (TODO)