Five Agents, One Verdict: Multi-Agent Fraud Detection
A single model sees a transaction and decides: fraud or not. But fraud is not a single signal — it is behavioural, geographic, linguistic, and statistical. So we built five specialised agents and made them argue.
5
AI Agents
5
Data Sources
LightGBM + GPT-4.1
Built With
3
Team Members
The team and the challenge
This was built for the ReplyMirror Challenge with Sanya Khan and Owes Mehboob — team Masala Techii. The challenge gave us transaction data enriched with user profiles, geolocation, SMS messages, and emails. The goal: detect fraud. The catch: real fraud is not a single pattern. It is a person using a card in two countries within an hour. It is a phishing email followed by an unusual transaction. It is a behaviour shift that no single feature captures. We decided early that a monolithic model was the wrong abstraction.
The five-agent architecture
Each agent has one job and does it well. The FeatureAgent extracts transaction-level and contextual signals. The TextAgent uses GPT-4.1 Mini through OpenRouter to analyse SMS and email content for phishing patterns and social engineering language — things a traditional model would miss entirely. The ProfileAgent builds behavioural baselines for each user and flags deviations. The RiskAgent aggregates all signals into a unified fraud score. And the DecisionAgent applies adaptive thresholding to make the final call. The pipeline flows linearly: data in, features extracted, profiles checked, anomalies detected, risk computed, verdict delivered.
LLMs as forensic readers
The TextAgent was the most interesting piece to build. Traditional fraud systems ignore message content or reduce it to keyword matching. We fed full SMS and email text into an LLM and asked it to identify specific fraud indicators: urgency language, impersonation of authority, requests for credentials, suspicious links. The LLM outputs structured flags that feed directly into the risk computation. This is not a chatbot — it is an LLM used as a forensic text analyser, and it catches patterns that regex-based systems never would.
Isolation Forest meets adaptive thresholds
The anomaly detection layer uses Isolation Forest to identify transactions that are statistically unusual within a user's behavioural profile. But a hard threshold on anomaly scores is too rigid — what is unusual for a student is normal for a frequent traveller. So the DecisionAgent uses adaptive thresholding: the fraud cutoff adjusts based on the user's historical variance and the confidence of the other agents. Every decision is traceable through Langfuse, giving us full observability over which agent contributed what to each verdict.