Testing and acceptance
Measuring DPI classification accuracy on your traffic
Measure DPI classification accuracy per application class, on a sample of your own traffic where you already know the right answer, and report both detection and false positives for each class rather than one headline figure. Any accuracy figure a vendor quotes is a starting point: subscriber mix, application mix, the share of encrypted traffic and how recently the classifier was updated all change the result. This page sets out a method your test and acceptance team can use.
A Saudi product, built by Interkey
For test, acceptance and network engineering teams at telecom operators, ISPs and regulators. Reviewed November 2026.
- Per class, not one number:detection and false positives for every application class that matters to you.
- On your traffic:a classifier tuned on one operator's traffic will not behave identically on another's.
- Against verified ground truth:traffic generated by known applications on devices and subscriptions you control.
- Out-of-band, and repeated:test without touching the forwarding path, and re-test after significant updates.
Why one accuracy number is not enough
A single figure averages over every class. If a few large classes are classified well, the average can look high while the classes your policies depend on are classified poorly. It also says nothing about false positives: traffic wrongly placed in a class. A false positive matters as much as a miss, because it means a policy acts on the wrong traffic.
DPI classification accuracy: the measures to report
| Measure | What it tells you | Why it matters |
|---|---|---|
| Detection rate (per class) | Of the traffic that truly belongs to a class, the share the engine placed in that class | Shows what a policy or report on that class would miss |
| False positives (per class) | Traffic from other classes that the engine placed in this class | Shows what a policy on that class would wrongly act on |
| Precision (per class) | Of everything the engine placed in a class, the share that truly belongs there | Combines both effects into what the class label is worth |
| Unclassified share | Traffic the engine did not assign to any class | Shows how much traffic sits outside every policy and report |
| Confusion between classes | Which classes are mistaken for which | Points to the classes that need attention |
| Stability over time | Whether results hold after updates and as applications change | Shows whether today's result will still hold next month |
Decide in advance whether you count flows or bytes. Both are valid, but they give different answers: a few large flows can dominate a byte count.
Building a verified sample
Ground truth is traffic where you know the right answer independently of the engine under test.
- Generate it yourself.Run known applications on devices and subscriptions you control, so the sample contains no subscriber data and every flow has a known label.
- Cover the classes that matter.Include every class your policies and reports depend on, and the encrypted and tunnelled versions of them.
- Include changing applications.Use current application versions, and repeat later with newer versions.
- Keep it separable.Make sure the sample can be told apart from other traffic in the results, for example by the test devices' addresses.
- Test at realistic load.Run part of the test during your busy hour, so the engine is measured under the load it will face.
The test method, step by step
1. Write the acceptance criteria first.For each class, agree the detection and false-positive thresholds, the unit (flows or bytes) and the reports to be produced, before the test starts.
2. Classify out-of-band.The platform observes traffic without sitting in the forwarding path, so the test cannot affect subscribers.
3. Compare with ground truth.For each class, count correct results, misses and false positives, and record the unclassified share.
4. Report per class.Produce the measures above for every class, with the confusion between classes.
5. Repeat after updates.Re-run the test after significant model or policy updates, and on a regular schedule as applications change.
Setting thresholds by what the class is used for
The threshold for a class should follow what will be done with it:
| Use of the class | Threshold emphasis |
|---|---|
| Reporting and capacity planning only | Detection and the unclassified share, so trends are reliable |
| A policy in report-only mode | Both measures, to see what the rule would have done |
| A policy that acts on traffic, such as blocking where your policy and applicable regulation require it | The strictest false-positive limit, because a false positive acts on the wrong traffic |
On Interkey's accuracy figure
Interkey's product material states 99% recognition accuracy, including for VPN and other adversarial encrypted applications. The figure has not been independently benchmarked and is not a specification for any particular network. Results depend on subscriber mix, application mix, the share of encrypted traffic and how recently the classifier was updated, which is why we recommend measuring it on your traffic with the method above.
Interkey Intelligent DPI classifies from flow behaviour (handshake structure, packet sizing and timing, and endpoint behaviour) without decrypting traffic. Its model is retrained automatically on features extracted from live traffic, and Live View results feed back into models and policies, so re-checking accuracy after significant updates belongs in your change control. More on the engine: Network traffic classification engine for telecom operators.
Where this fits in a proof of concept
This method is the second of the three proof-of-concept stages Interkey recommends: written acceptance criteria first; out-of-band classification measured against verified traffic, with false positives measured as carefully as detection; then policies in report-only mode before the inline decision. How Interkey runs a DPI proof of concept
To make the method binding in a tender, see the DPI RFP requirements checklist.
Questions about DPI accuracy testing
How do you measure DPI classification accuracy?
What is a false positive in DPI?
Why do results differ between networks?
Can we test without affecting subscribers?
How often should accuracy be re-tested?
Related pages
- How Interkey runs a DPI proof of concept
- Network traffic classification engine
- Encrypted traffic classification without decryption
- DPI RFP requirements checklist
- Inline vs out-of-band DPI
- Bandwidth management for ISPs
- Policy control and PCRF
- Quality of experience monitoring
- Tethering detection
- Traffic shaping
- Video optimization
- VPN detection
- Zero rating
Proof of concept in three phases
Written acceptance criteria.Agree which rules are tested, on which traffic and subscriber groups, how results are measured and what counts as a pass.
Out-of-band classification against verified traffic.Measure false positives as carefully as detection for every class the rules depend on.
Report-only policy, then the inline decision.Each rule shows what it would have done; enforcement follows only after classification has proven itself.
Plan an accuracy test on your traffic
Tell us which application classes matter for your policies and reports, whether you already have a verified sample, and your timeline. Our team in Riyadh replies in Arabic or English.
- Our engineers read your request
- We agree the scope of a technical session
- You see the product on a use case you choose