Feature-based machine-learning detectors are the most widely deployed defence against phishing websites, and recent work routinely reports them at 97 to 99 percent accuracy. This paper asks whether that number means anything once a real attacker starts editing a page. It does not. The study evaluates six detectors, four tree ensembles and two deep networks, on two public datasets of 10,000 and 57,405 web pages, under a constrained evasion model in which the attacker may change only the parts of a URL and its markup that an attacker actually authors, and every candidate is projected back to a valid feature vector. Under that model, detection on the first dataset falls from 98.50 percent to between 66.30 and 69.30 percent under a perturbation that leaves the second dataset unmoved, and the same convolutional architecture is the most resistant model on one dataset and the least resistant on the other. Clean accuracy predicts none of it. What separates a fragile detector from a robust one is not the algorithm but where the model finds its signal: how much of its decision rests on features the attacker can rewrite.
The Short Version
Machine-learning phishing detectors look solved on the usual benchmarks. This study tests whether that survives contact with an adversary who edits only what an attacker can author. On one dataset detection collapses from 98.50 percent to the high sixties under the smallest perturbation tested; on the other the same attack does nothing at all. The same architecture is the most robust model on one dataset and the least robust on the other. The explanation is not the model but the feature representation: the fragile detector draws roughly three-quarters of its decision from attacker-controllable features, the robust one about a quarter. And there is a number you can measure before you deploy, the share of a model's importance sitting on features an attacker owns, that tells you which case you are in.
Why This Study
Phishing is still one of the most common ways attackers get a foothold, and feature-based machine-learning detectors are the countermeasure most organisations actually run. The literature reports them at 97 to 99 percent accuracy, and on the standard benchmarks the problem can look finished. Three things undercut that figure. Benchmarks have saturated, so competing methods can no longer be told apart on shared data. Datasets carry artefacts, where a single engineered feature makes the whole set almost trivially separable and inflates every score. And there is little evidence on adversarial robustness, even though an attacker can rewrite a URL to evade a detector while keeping the page malicious.
This work takes the third problem seriously and tests what the accuracy number is worth when an adversary is present. The key move is to be realistic about what an attacker can and cannot change. Editing markup and a URL string is free. Changing the domain's age, its hosting, or its TLS certificate costs money and time. So the attacks here are permitted to touch only the features an attacker authors, and every generated page is projected back into the space of valid feature vectors before it counts. The question is not whether a detector can be fooled with unlimited edits; it is how detection degrades as an attacker makes the cheap, realistic changes first.
"Every model scores 87 to 99 percent on clean data, and that number predicts almost nothing about what happens when an attacker starts editing. What explains the difference is not the algorithm but where the model finds its signal."
Accuracy Is Not RobustnessResearch questions
- Does clean accuracy predict how far a detector degrades under a realistic, constrained evasion attack?
- Does it predict which models hold up best, across different model families and datasets?
- What property of a trained model explains the difference in vulnerability?
- Does adversarial training harden a detector, and does that hold up against an attack it was not trained on?
The Experiment
The study runs in five stages: prepare the data and split every feature by who controls it, train six detectors plus a surrogate, attack them two independent ways, analyse where each model's signal sits, then harden the fragile model and re-attack it with an attacker it has never seen.
The idea the whole study rests on
Every feature goes into one of two groups. A feature is mutable if an attacker can set it just by writing the page, the number of dots in a URL, the path length, the ratio of external links, the presence of keywords. It is immutable if it reflects infrastructure the attacker cannot fake by authoring markup, the domain's age, its name servers, its TLS certificate, its hosting ASN. The attacks are allowed to edit the first group and must leave the second alone.
The split is not even across the two datasets, which turns out to matter. Dataset A has 19 of 48 features mutable and 29 immutable; Dataset B has 76 of 111 mutable and 35 immutable. The formal constraint is simple: a generated sample equals the original plus a perturbation that is zero on every immutable feature and bounded in size, then passed through a validity projection that clips each feature to the range seen in training and rounds the ones that must be integers. The projection enforces per-feature ranges and integrality; it does not encode every dependency between related features, so the generated samples are admissible under the constraints imposed rather than claimed to be fully realistic pages in every respect.
Models and attackers
Four tree ensembles were chosen because they are the models most often recommended for this task: Random Forest, XGBoost, LightGBM, and CatBoost. Two deep networks, a multilayer perceptron and a one-dimensional CNN, were added so the findings could not be blamed on a single model family. A separate differentiable surrogate, an MLP with 128 hidden units, provides the gradients for the transfer attack.
The two attackers are deliberately different. The transfer attack trains the differentiable surrogate on comparable data and runs projected gradient descent for 30 steps, bounded in the L-infinity norm by a budget epsilon, then relies on the adversarial examples carrying over to the real detectors, which it cannot query. The decision-based boundary attack is strictly black-box: it sees only the hard label the target returns, and it measures the L-2 length of the perturbation, bounded by tau, needed to cross the boundary. The second is the more conservative test for a defender, because its result does not depend on how well the surrogate happens to match the target; any degradation it reports is attributable to the target itself. On Dataset A, an epsilon of 0.05 corresponds to changing the number of dots in a URL by about one, or its length by roughly two characters; a tau of 2.0 corresponds to editing on the order of ten such features at once.
Two Public Datasets, Different Depths
| Dataset A | Dataset B | |
|---|---|---|
| Source | Tan (2018), Mendeley Data | Vrbancic et al. (2020), Data in Brief 33:106438 |
| Instances | 10,000 | 58,645 → 57,405 after de-duplication |
| Features | 48 | 111 |
| Schema | URL and page content | URL plus host, DNS and TLS |
| Mutable / immutable | 19 / 29 | 76 / 35 |
The difference in schema is the whole story in miniature. Dataset A describes a page mostly through its URL and content, which is exactly the territory an attacker authors. Dataset B adds host, DNS, and TLS signals, which an attacker cannot fake by editing markup. Both are near-balanced, which is worth keeping in mind, since phishing is rare in real operation and a deployment base rate would shift the absolute numbers. Data was preprocessed with de-duplication, an 80/20 stratified split, and standardisation so that a budget in standardised units means the same thing across features on different scales.
On Clean Data, Everything Looks Fine
Trained and measured the usual way, all six detectors land where the literature says they should. The four tree ensembles sit inside a band narrower than seed-to-seed noise, which is the benchmark-saturation problem in one line: on clean accuracy alone, you cannot separate them.
| Model | Dataset A | Dataset B |
|---|---|---|
| Random Forest | 98.50 | 95.71 |
| XGBoost | 98.75 | 94.98 |
| LightGBM | 98.55 | 95.80 |
| CatBoost | 98.65 | 94.55 |
| Deep MLP | 97.40 | 94.87 |
| 1D-CNN | 93.45 | 87.43 |
Clean accuracy (%). Figures as reported in the paper. The tree ensembles are statistically indistinguishable on this metric.
Let the Attacker Edit, and the Datasets Split Apart
The same detectors, now under the transfer attack at the smallest budget tested, epsilon of 0.05. Dataset A collapses. Dataset B does not move. This is the central result: one number, clean accuracy, said these were the same calibre of model, and they are not.
| Model | A: clean → ε=0.05 | B: clean → ε=0.05 |
|---|---|---|
| Random Forest | 98.2 → 68.4 | 96.5 → 96.5 |
| XGBoost | 98.9 → 69.3 | 95.3 → 95.3 |
| LightGBM | 98.5 → 69.3 | 96.0 → 96.0 |
| CatBoost | 98.9 → 66.5 | 95.1 → 95.1 |
Phishing detection rate (%) under the transfer attack. On Dataset A detection falls into the high sixties at the smallest non-zero budget; the paper reports the range across models as 66.30 to 69.30 percent. Dataset B is unaffected.
Almost all of the degradation on Dataset A happens at the very first non-zero budget, which is why the budget grid is spaced finely near zero. A coarser grid would have stepped straight over the collapse and reported a gentler curve that never happened.
The Independent Attacker Agrees
The boundary attack shares no gradient information with the transfer attack and cannot query a surrogate; it sees only hard labels. It reaches the same conclusion, which is what makes the finding credible rather than an artefact of one attack. The figure that matters here is the median L-2 cost: how large a perturbation the attacker must spend to evade. Higher is more robust.
| Model | A: evadable | A: median L2 | B: evadable | B: median L2 |
|---|---|---|---|---|
| Random Forest | 94.8% | 2.64 | 57.8% | 5.86 |
| XGBoost | 99.0% | 1.85 | 90.0% | 5.85 |
| LightGBM | 99.0% | 2.03 | 80.8% | 5.71 |
| CatBoost | 95.0% | 1.58 | 81.2% | 5.96 |
| Deep MLP | 98.0% | 1.85 | 81.8% | 4.38 |
| 1D-CNN | 70.0% | 2.50 | 99.0% | 3.54 |
Boundary attack. "Evadable" is the share of phishing pages the attacker can push across the boundary; median L2 is what it costs. On Dataset B more than 40 percent of phishing pages cannot be evaded by Random Forest at all.
Same architecture, opposite verdicts. The 1D-CNN is the most robust model on Dataset A, only 70 percent of pages evadable, and the least robust on Dataset B, 99 percent evadable. No property of the algorithm changed between those two runs. Only the data it learned from did.
Why One Model Breaks and Another Holds
The reason is where each model concentrates its decision. Measured by impurity-based importance, the LightGBM detector draws 76.60 percent of its signal from mutable URL-lexical features on the fragile Dataset A, against a median L-2 of just 2.03, and only 26.90 percent on Dataset B, where it needs 5.71 and leans instead on host, DNS, registration, and TLS signals. Where the discriminative power sits in features the attacker controls, the input can be walked across the boundary cheaply; where it does not, evasion is bounded no matter how the URL is rewritten.
| Measure | Dataset A | Dataset B |
|---|---|---|
| LightGBM, impurity importance | 76.60% | 26.90% |
| LightGBM, permutation importance | 66.20% | 46.00% |
| Random Forest, impurity importance | 59.50% | 63.00% |
| Random Forest, permutation importance | 55.80% | 33.20% |
Share of the model's decision sitting on attacker-controllable features. Permutation importance is reported as the conservative check; it shrinks the contrast and, on Dataset B, reverses the impurity ordering, but the central comparison survives: LightGBM still falls from 66.20 to 46.00 percent.
Random Forest is the instructive exception. Its mutable share is similar on both datasets, yet it is the most resistant model on Dataset A. That fits the wider claim rather than breaking it: robustness is governed by where the signal concentrates, and a forest spreads importance thinly across many features while boosting concentrates it into a few. Two importance measures disagreeing on the magnitude is itself a reason to treat any single importance-based indicator with caution, which is why the paper reports both.
A Real Evasion, and a Defence That Partly Backfired
The edits are ones an attacker could actually make
A feature-space perturbation only matters if it describes a page someone could build. Starting from a Dataset A page the detector scored at 0.987 phishing probability, grouped edits moved mutable features toward the median of legitimate training pages. Removing sensitive keywords changed nothing, because that feature already sat at the median. Normalising the external-link and self-redirect ratios then dropped the probability to 0.005 in a single step and flipped the verdict, three values changed in total, every one of them under the attacker's control at authoring time.
Adversarial training, and the case it hid
Adversarial training was then tested as a mitigation. On Dataset A it worked: clean accuracy held, transfer detection at epsilon of 0.3 recovered from 69.30 to 99.70 percent, and, most importantly, the median L-2 demanded by the independent boundary attack rose from 2.03 to 2.84, and that attack played no part in training. On Dataset B the same procedure raised transfer detection to 99.90 percent but reduced the median black-box L-2 from 5.71 to 3.70, pushing the decision surface into a region a decision-based attacker could exploit at lower cost.
| Dataset | Clean acc. (%) | Transfer det. (%) @ ε=0.3 | Black-box median L2 |
|---|---|---|---|
| A: Tan | 98.55 → 98.65 | 69.3 → 99.7 | 2.03 → 2.84 |
| B: Vrbancic | 95.80 → 95.03 | 89.8 → 99.9 | 5.71 → 3.70 ⚠ |
Measuring only the trained-against attack would have reported a clean win on both datasets. The black-box number on Dataset B, evasion getting cheaper, is only visible because the defence was tested with an attack it had never seen. A robustness claim becomes credible only after adaptive, or at least independent, evaluation.
Four Things a Practitioner Should Do Differently
- Stop quoting accuracy on its own. Across two datasets it predicted neither the size of the drop, nor which model held up best, nor whether the defence would help. Report robustness under a stated threat model alongside accuracy, never inferred from it.
- Treat robustness as a property of the feature representation. The 1D-CNN was the most robust model on one dataset and the least robust on the other. Same architecture, opposite verdicts, because the signal sat in different places.
- Measure the mutable-importance share before you deploy. It is computable from a trained model with no attacker in the loop. Where it is high, add signals the attacker cannot fabricate, host, DNS, registration, TLS, or regularise the reliance away.
- Test a defence with an attack it has never seen. Otherwise you are measuring memorisation of one attack, not hardening, and you can miss a case where evasion has quietly become cheaper.
One deployment tension is worth naming. PhreshPhish (Dalton et al., 2025) excludes host-based features because fetching them adds latency, which suits real-time blocking. On this evidence, that same choice reduces robustness, because it strips out exactly the immutable signals that bound a lexical attacker. Speed and robustness pull against each other here, and the trade should be made deliberately.
Every Number Is Reproducible
The whole study is seeded and runs end to end. A full pass takes about 83 seconds on Dataset A and 209 seconds on Dataset B on a single CPU core, and the accompanying notebook runs top to bottom in roughly fifteen minutes, with Dataset A embedded as a compressed, checksummed blob so it cannot go missing and Dataset B downloading itself at runtime with a retry. Every measured value is written out as JSON, and every trained detector is saved, including the adversarially trained variants.
One deviation is stated plainly rather than buried: on Dataset B the 1D-CNN was trained on a 10,000-instance stratified subsample of the training split to bound its cost, so its absolute accuracy on that dataset is a lower bound rather than a converged estimate. Every other number comes from the full training split. Naming that limit is part of the same discipline the paper argues for everywhere else, report the threat model, report the conservative measure, and do not let a single favourable number stand in for the whole picture.
The repository ships the notebook, the LaTeX source and compiled PDF, every result as JSON, publication-quality figures, and the trained models with their scalers. The clean results match the wider literature, which is the point: the divergence appears only once an adversary is allowed to modify the features under their control.
Conclusion
This study evaluated six phishing detectors, four tree ensembles and two deep networks, under a constrained evasion model that only lets an attacker change what an attacker can author. The headline is in the title. Clean accuracy, the number the field leans on, conveyed almost nothing of practical value: it gave no indication of how far detection would fall, of how the models would rank once attacked, or of whether a mitigation would help. Selecting a detector on accuracy alone can mean picking one that is highly vulnerable.
The difference between a fragile detector and a robust one came down to where each model draws its signal. Features from the URL and markup are exactly what an adversary rewrites, so a model that concentrates its decision there needs only a small, cheap edit to evade. Features grounded in hosting, registration history, and TLS state cannot be produced by authoring a page, and they bound what a lexical-only attacker can achieve. That is why the same architecture can be the most robust model on one dataset and the least robust on another: the algorithm did not change, the location of the signal did.
The practical recommendations follow directly. Measure robustness under a stated threat model and report it next to accuracy. Audit the share of a model's importance that sits on attacker-controllable features, and where it is high, add signals the attacker cannot fabricate or regularise the reliance away. And test any defence against an attack it has not seen, because the one case here where hardening made black-box evasion cheaper would have been invisible to an evaluation that only re-ran the attack it trained against. Robustness is not something accuracy implies; it is something you have to measure, under an adversary, before you trust it.
References
- Tan, C.L. (2018). Phishing dataset for machine learning: feature evaluation. Mendeley Data, V1.
- Vrbancic, G., Fister, I., Podgorelec, V. (2020). Datasets for phishing websites detection. Data in Brief 33, 106438.
- Brendel, W., Rauber, J., Bethge, M. (2018). Decision-based adversarial attacks. ICLR.
- Madry, A. et al. (2018). Towards deep learning models resistant to adversarial attacks. ICLR.
- Papernot, N. et al. (2017). Practical black-box attacks against machine learning. ACM AsiaCCS.
- Pierazzi, F. et al. (2020). Intriguing properties of adversarial ML attacks in the problem space. IEEE S&P.
- Apruzzese, G., Conti, M., Yuan, Y. (2022). SpacePhish. ACSAC.
- Montaruli, B. et al. (2023). Raze to the ground. ACM AISec.
- Carlini, N., Wagner, D. (2017). Towards evaluating the robustness of neural networks. IEEE S&P.