The Challenge
As Large Language Models become integrated into production applications, their susceptibility to adversarial attacks, specifically prompt injection and jailbreaking, poses a critical security risk. I built Prompt Heist to serve as a gamified sandbox for testing these safety guardrails across different model scales.
System Architecture
The application is built on a Python backend utilizing the Groq API for rapid inference. The frontend interface is powered by Streamlit, allowing users to interactively attempt to bypass the model's system prompts.
User Input → Streamlit Frontend → Python Backend → Groq API → (Llama 3 / Mixtral / Gemma)
Quantitative Results
The framework integrated three GenAI models (OpenAI GPT-OSS 20B, OpenAI GPT-OSS 120B, Qwen 3.6 27B) across three adaptive difficulty tiers. The evaluation yielded the following bypass rates across 36 structured adversarial test cases (30 single-turn injection techniques, 6 multi-turn social engineering sequences):
Engineering Learnings
- State Management in Streamlit: Handled multi-turn conversational memory to test continuous context attacks.
- Data Integrity & Debugging: Diagnosed and resolved a backend data leak caused by unfiltered reasoning traces (
<think>tags), correcting a false 33% vulnerability measurement down to its accurate value. - API Integration: Implemented efficient routing to the Groq API to ensure low-latency responses during adversarial testing.
Interface
