Attacking and Defending AI

AI & Machine Learning Essentials

• AI, ML and Deep Learning overview

• Supervised vs. Unsupervised learning

• Neural networks and model training

• Generative vs Discriminative models

• Practical applications in cybersecurity

Large Language Models (LLMs)

• What LLMs are and how they work

• Transformer architecture (encoder, decoder types)

• Key capabilities: NLP, text generation, reasoning

• Context windows and system prompts

Prompt Engineering

• Elements of effective prompts

• Prompting techniques

  • Zero-shot, few-shot, chain-of-thought

  • Generated knowledge and emotional prompting

• Iterative refinement strategies

Secure AI System Design

• Open WebUI architecture and components

• Retrieval-Augmented Generation (RAG) flow

• Tools and pipelines in Open WebUI

• Deploying filters (e.g. prompt injection, PII, toxicity)

AI Security Threats

• AI safety vs. security concerns

• OWASP LLM Top 10 risks

• Common attack vectors:

  • Prompt injection, system prompt leaks, jailbreaking

  • Role deception, confusion tactics, custom encoding

  • External malicious content, escalation chains

Offensive AI Examples with hands on Capture the Flag Challenge

• Adversarial LLM prompt design

• Bypassing safeguards

• Leveraging agentic and interpreter capabilities

• Attacking RAG

• Tooling to assist with attacks and assessments