A Neural Network Redefines Fraud Detection

Innovation in the field of artificial intelligence continues to push the boundaries of what we thought was possible. A recent experiment demonstrated that a neural network can learn to generate its own fraud detection rules without direct human intervention. This approach uses differentiable rule induction to extract IF-THEN rules from a pre-trained neural network. The goal is to create auditable and interpretable fraud rules without having to code them manually.

In this experiment, a hybrid neural network was enhanced with a differentiable rule learning module. This module has the capability to automatically extract fraud rules during the training process. Using the Kaggle credit card fraud dataset, which has a fraud rate of 0.17%, the model successfully learned interpretable rules. For example, a typical rule might be: IF V14 < −1.5σ AND V4 > +0.5σ → Fraud, where σ represents the standard deviation of the feature after normalization.

The rule learning module achieved a ROC-AUC score of 0.933 ± 0.029, while maintaining a fidelity of 99.3% to the predictions of the neural network. What is particularly remarkable is that the model independently rediscovered the feature V14, a variable long recognized by analysts as being strongly correlated with fraud, without being explicitly instructed to look for it.

What the Model Discovered

Before diving into the technical details of the architecture or the loss function, it is essential to understand the final discoveries of the model. After training for up to 80 epochs, with early stopping, most seeds converged between epochs 56-78. The rule learning module produced significant results in two specific seeds where the rules were clearly defined.

In the case of Seed 42, the clearest rule had five conditions with a confidence of 0.95. The learned rule was: IF V14 < −1.5σ. For Seed 7, a complementary rule with eight conditions and a confidence of 0.74 was identified: IF V14 < −1.6σ. In both scenarios, the low values of V14 were central to the fraud detection logic, illustrating a striking convergence in the absence of prior directives.

The model never received specific information about the importance of features. Yet, it independently rediscovered the same feature that human analysts had identified for years. This ability of a neural network to discover its own fraud rules embodies the promise of neuro-symbolic AI: combining statistical learning with human-readable logic. The rest of this article explores how this discovery was possible and why the gradient continued to identify V14 even without prior information.

From Injected Rules to Learned Rules — Why This Matters

Every fraud detection model has a decision boundary. However, fraud teams work with rules. The gap between what the model learns and what analysts can understand, audit, and defend before a regulator is crucial for compliance.

In a previous article in this series, two analyst rules were directly encoded into the loss function: if the transaction amount is abnormally high and if the PCA signature is abnormal, the sample is treated as suspicious. This approach worked, allowing the hybrid model to match the detection performance of a pure neural network while remaining interpretable.

However, an obvious limitation remained: these rules were handwritten. They were based on features chosen for their intuitive sense. Hand-coded rules are effective when fraud patterns are stable and domain knowledge is deep. But they are inadequate when patterns evolve, features are anonymized (as in this dataset), or when one wants the model to highlight unexpected signals.

The natural question arises: which features would the gradient choose if it had the freedom to choose? This scheme extends beyond fraud. Medical diagnostic systems require rules that can be verified by doctors before making decisions. Cybersecurity models need rules that can be audited by engineers. Anti-money laundering systems operate under regulatory frameworks that demand explainable decisions. In any field combining rare events, domain expertise, and compliance requirements, the ability to extract auditable IF-THEN rules from a trained neural network is directly valuable.

Architecturally, the change is surprisingly simple. It is not about replacing the MLP, but rather adding a second path that learns to express the MLP's decisions in the form of human-readable symbolic rules. The MLP trains normally, while the rules module learns to align with it in symbolic form. This is the subject of this article: differentiable rule induction in about 250 lines of PyTorch, without prior knowledge of important features.

“You do not replace the neural network. You teach it to explain itself.”

The Architecture: Three Learnable Elements

The architecture retains a standard neural network intact but adds a second path that learns symbolic rules explaining the network's decisions. Both paths operate in parallel from the same input, and their outputs are combined by a learnable weight α:

• The Hybrid Rule Learner operates with two parallel paths from the same input of 30 features. The MLP path handles detection; the rules path learns to explain it. α is a trainable scalar — not a hyperparameter.

The MLP path is identical to the previous article: three fully connected layers with batch normalization. The rules path is new. Alpha is a learnable scalar that the model uses to weight the two paths, starting at 0.5 and trained via gradient descent like any other parameter. After training, α converged to about 0.88 on average across the seeds (range: 0.80–0.94). The model learned to weight the neural path at about 88% and the rules path at 12% on average. The rules do not replace the MLP; they are a structured symbolic summary of what the MLP has learned.

1. Learnable Discretizer

Rules require binary inputs — is V14 below a threshold? yes or no. Neural networks need continuous and differentiable operations. The soft sigmoid threshold bridges the two.

For each feature f and each learnable threshold t:

bf,t=σ ⁣(xf−θf,tτ)

• xf is the value of feature f for this transaction
• θf,t is a learnable threshold, randomly initialized, trained via backpropagation
• τ is the temperature — high at the beginning of training (exploratory), low later (precise)

bf,t is the soft binary output: “is feature f above threshold t?”

The model learns three thresholds per feature, giving it three “cuts” per dimension. Each threshold is independent — the model can spread them across the feature range or concentrate them around the most discriminative cut point.

2. Rule Learning Layer

Each rule is a weighted combination of binarized features, passed through a sigmoid:

ruler(x)=σ ⁣(∑iwr,i⋅biτ)

The sign of each weight has a direct interpretation after being squashed by tanh:

• w>+0.5 → the feature must be HIGH for this rule to activate
• w<−0.5 → the feature must be LOW for this rule to activate
• |w|<0.5 → the feature is unimportant for this rule

Rule extraction follows directly: threshold the absolute values of the weights after training to identify which features each rule uses. This is how IF-THEN statements emerge from continuous parameters — by reading the weight matrix.

3. Annealing Temperature

The temperature follows an exponential decay schedule:

τ(t)=τstart⋅(τendτstart)t/T

With τ_start=5.0, τ_end=0.1, T=80 epochs:

• Epoch 0: τ=5.00 — Fully soft rules — the gradient flows everywhere
• Epoch 40: τ=0.69 — Rules tightening — thresholds engaging
• Epoch 79: τ=0.10 — Almost crisp rules — readable as IF-THEN

The temperature τ decays exponentially over 80 epochs, transitioning from exploratory softness (τ=5.0) to nearly binary crispness (τ=0.1). The shaded area shows the region where gradients are still informative.

Without annealing, the model remains soft, and the rules never crystallize into something a fraud analyst can read or a compliance team can approve. Annealing is what converts continuous optimization into symbolic output.