Which lightweight defences reduce agent hijacking?
AgentGuard Lab · 30 synthetic cases. Protected attack success fell from 100% to 30%; the literal detector achieved 1.00 precision and 0.50 recall.
RESEARCH / COMPLETED STUDYLoughborough University · MSc
Safety is a
conversation.
What changes
along the way?
The completed study examines how two language models responded to psychosis-related unsupported interpretations across six-turn conversations, presentation levels and context conditions.
72conversations
432assistant responses
432human annotations
1,296judge annotations
MULTI-TURN LLM SAFETY EVALUATIONFINAL STUDY / v2.1.0
Establish the first response.
The first turn provides a starting point for comparing model behaviour across the controlled scenarios and context conditions.
Study design walkthrough · synthetic research evaluation, not a clinical, diagnostic or therapeutic system.
- Controlled comparison
- Three presentation levels, three scenario themes, two target models, two context conditions, two repetitions and six turns per conversation.
- Human annotation
- All 432 assistant responses were annotated, with 1,296 supplementary LLM-judge annotations retained as a separate evidence source.
- Reproducible method
- The public repository contains the application, frozen scenario scripts, context histories, study manifest, protocol bundles, tests and demonstration fixtures.
- Evidence boundary
- Private raw conversations and detailed analysis outputs remain controlled outside Git. The public artefact exposes the method without publishing the full evidence corpus.
Related experiments
Open project ↗
How much can an attacker learn through queries?
ModelExtract MiniLab · A local model and surrogate, varying query budget and comparing randomized-response defences.
Can provenance help protect retrieval?
RAGTrust MiniBench · Trusted and untrusted synthetic documents expose a keyword-stuffing mechanism and its defence trade-offs.
Can a compact classifier explain its mistakes?
PhishGuard · 0.63 F1 and 65% accuracy on a 20-message synthetic challenge set, with every false positive and false negative documented.