Align CyberPII-Bench with the rest of the benchmarks

Signed-off-by: Víctor Mayoral Vilches <v.mayoralv@gmail.com>
This commit is contained in:
Víctor Mayoral Vilches 2025-06-03 08:22:21 +00:00
parent f0bffe88f9
commit f518d6b1de
4 changed files with 3 additions and 2 deletions

View File

@ -11,7 +11,8 @@ Currently, this are the benchmarks included:
| [SecEval](https://github.com/XuanwuAI/SecEval) | Benchmark designed to evaluate large language models (LLMs) on security-related tasks. It includes various real-world scenarios such as phishing email analysis, vulnerability classification, and response generation. |
| [CyberMetric](https://github.com/CyberMetric) | Benchmark framework that focuses on measuring the performance of AI systems in cybersecurity-specific question answering, knowledge extraction, and contextual understanding. It emphasizes both domain knowledge and reasoning ability. |
| [CTIBench](https://github.com/xashru/cti-bench) | Benchmark focused on evaluating LLM models' capabilities in understanding and processing Cyber Threat Intelligence (CTI) information. |
| [PentestPerf](https://gitlab.com/aliasrobotics/alias_research/caiextensions/pentestperf) | Internal Benchmark to evaluate the security capabilities. It is built as a composition of individual benchmarks, each represented by a Docker container. Each container scenario (CTF) can contain multiple challenges and tasks. The system is designed to be modular and extensible, allowing for the addition of new benchmarks and challenges. |
| [PentestPerf](https://gitlab.com/aliasrobotics/alias_research/caiextensions/pentestperf) | An internal benchmarking framework that measures penetration testing capabilities of LLM models in a proprietary set of IT, OT and robotics scenarios. Reach out if you wish to cooperate in this direction. |
| [CyberPII-Bench](https://github.com/aliasrobotics/cai/tree/main/benchmarks/cyberPII-bench/) | Benchmark designed to evaluate the ability of LLM models to maintain privacy and handle **Personally Identifiable Information (PII)** in cybersecurity contexts. Built from real-world data generated during offensive hands-on exercises conducted with **CAI (Cybersecurity AI)**. |
The goal is to consolidate diverse evaluation tasks under a single framework to support rigorous, standardized testing.

View File

@ -1,6 +1,6 @@
## 🧠 CyberPII-Bench
**CyberPII-Bench** is a benchmark designed to evaluate the ability of language models and automated systems to detect, analyze, and handle **Personally Identifiable Information (PII)** in cybersecurity contexts. The benchmark is built from real-world data generated during hands-on exercises conducted with **CAI (Cybersecurity AI)**, focusing on attack simulation, security testing, and reconnaissance scenarios.
**CyberPII-Bench** is a benchmark designed to evaluate the ability of LLM models to maintain privacy and handle **Personally Identifiable Information (PII)** in cybersecurity contexts. Built from real-world data generated during offensive hands-on exercises conducted with **CAI (Cybersecurity AI)**, focusing on attack simulation, security testing, and reconnaissance scenarios.
### 📁 Dataset: `memory01_80/`