Regional bank · US Southeast · $4.2B AUM
The bank was experiencing $3.1M in annual fraud losses — above peer-group benchmarks for an institution of its size. The existing rule-based fraud detection system had a false positive rate of 31%, meaning that for every genuine fraud transaction flagged, nearly one legitimate transaction was also blocked. False positives were generating significant customer service cost and increasing account abandonment at a rate that concerned the retail banking leadership.
The compliance team also had a mandate from the OCC to improve the auditability of fraud decisioning. The existing system produced binary outputs — fraud or not fraud — with no explanation of why a transaction had been flagged. This was inadequate for regulatory review.
Shelorve designed a two-component fraud detection architecture. The first component is a batch-trained SageMaker ML model, trained on three years of transaction history including confirmed fraud cases identified by the bank's investigation team. The model learns the combination of transaction characteristics, account behaviour patterns, and contextual signals that predict fraud — rather than applying fixed rules.
The second component is a real-time scoring pipeline built on AWS Kinesis. Every transaction is streamed through Kinesis and scored by the model within 50 milliseconds — fast enough to intervene before the transaction completes. Fraud scores are returned to the transaction processing system in real time, with a configurable threshold that allows the compliance team to tune the balance between fraud prevention and false positive rate.
To satisfy the OCC audit requirement, Shelorve built model explainability into the pipeline using SageMaker Clarify. Every flagged transaction generates a feature importance report — showing which transaction characteristics drove the fraud score — that is logged to DynamoDB with a tamper-resistant timestamp. The compliance team can retrieve a complete audit trail for any transaction on any date.
Annual fraud losses reduced from $3.1M to $0.8M — a saving of $2.3M in the first full year of operation. The false positive rate fell from 31% to 18% — a 42% reduction — which the retail banking team attributed to an measurable improvement in customer satisfaction scores for the debit card product. The OCC review of the model governance framework was completed without findings.
"We had been running rule-based fraud detection for eleven years. The rules were written by analysts who have since left and nobody fully understood them. The SageMaker model learned from actual fraud patterns — and the explainability output means we can finally answer the regulator's questions about why a transaction was flagged."
SageMaker · SageMaker Clarify · Kinesis · Lambda · DynamoDB
The SageMaker pipeline includes an automated retraining schedule — monthly by default, triggered immediately if model performance metrics drop below defined thresholds. SageMaker Model Monitor tracks data drift and prediction drift in production and alerts the operations team when retraining is needed ahead of schedule. The retraining pipeline runs without manual intervention; the new model is deployed only after automated evaluation confirms it outperforms the current production model.
Shelorve worked with the fraud team and the retail banking team to define an acceptable false positive budget — how many legitimate transactions could be blocked per 1,000 fraud detections before customer impact became significant. The threshold was set to optimise within this budget, with a dashboard that allows the compliance team to adjust it in real time as the business need changes.
The model is trained on behavioural patterns rather than specific transaction signatures, which gives it better generalisation to novel fraud tactics than rule-based systems. Known fraud patterns that the model might not detect are handled by a complementary rule layer — the two systems run in parallel, and a transaction is flagged if either layer generates a score above threshold. The monthly retraining cycle also incorporates newly confirmed fraud cases, giving the model ongoing exposure to emerging patterns.
Tell us what you are trying to fix. We will tell you whether Shelorve is the right partner — and if we are not, we will tell you that too.