mirror of https://github.com/aliasrobotics/cai.git
Align CyberPII-Bench with the rest of the benchmarks
Signed-off-by: Víctor Mayoral Vilches <v.mayoralv@gmail.com>
This commit is contained in:
parent
a44412c7da
commit
c73446b847
|
|
@ -11,7 +11,8 @@ Currently, this are the benchmarks included:
|
|||
| [SecEval](https://github.com/XuanwuAI/SecEval) | Benchmark designed to evaluate large language models (LLMs) on security-related tasks. It includes various real-world scenarios such as phishing email analysis, vulnerability classification, and response generation. |
|
||||
| [CyberMetric](https://github.com/CyberMetric) | Benchmark framework that focuses on measuring the performance of AI systems in cybersecurity-specific question answering, knowledge extraction, and contextual understanding. It emphasizes both domain knowledge and reasoning ability. |
|
||||
| [CTIBench](https://github.com/xashru/cti-bench) | Benchmark focused on evaluating LLM models' capabilities in understanding and processing Cyber Threat Intelligence (CTI) information. |
|
||||
| [PentestPerf](https://gitlab.com/aliasrobotics/alias_research/caiextensions/pentestperf) | Internal Benchmark to evaluate the security capabilities. It is built as a composition of individual benchmarks, each represented by a Docker container. Each container scenario (CTF) can contain multiple challenges and tasks. The system is designed to be modular and extensible, allowing for the addition of new benchmarks and challenges. |
|
||||
| [PentestPerf](https://gitlab.com/aliasrobotics/alias_research/caiextensions/pentestperf) | An internal benchmarking framework that measures penetration testing capabilities of LLM models in a proprietary set of IT, OT and robotics scenarios. Reach out if you wish to cooperate in this direction. |
|
||||
| [CyberPII-Bench](https://github.com/aliasrobotics/cai/tree/main/benchmarks/cyberPII-bench/) | Benchmark designed to evaluate the ability of LLM models to maintain privacy and handle **Personally Identifiable Information (PII)** in cybersecurity contexts. Built from real-world data generated during offensive hands-on exercises conducted with **CAI (Cybersecurity AI)**. |
|
||||
|
||||
|
||||
The goal is to consolidate diverse evaluation tasks under a single framework to support rigorous, standardized testing.
|
||||
|
|
|
|||
|
|
@ -1,6 +1,6 @@
|
|||
## 🧠 CyberPII-Bench
|
||||
|
||||
**CyberPII-Bench** is a benchmark designed to evaluate the ability of language models and automated systems to detect, analyze, and handle **Personally Identifiable Information (PII)** in cybersecurity contexts. The benchmark is built from real-world data generated during hands-on exercises conducted with **CAI (Cybersecurity AI)**, focusing on attack simulation, security testing, and reconnaissance scenarios.
|
||||
**CyberPII-Bench** is a benchmark designed to evaluate the ability of LLM models to maintain privacy and handle **Personally Identifiable Information (PII)** in cybersecurity contexts. Built from real-world data generated during offensive hands-on exercises conducted with **CAI (Cybersecurity AI)**, focusing on attack simulation, security testing, and reconnaissance scenarios.
|
||||
|
||||
### 📁 Dataset: `memory01_80/`
|
||||
|
||||
Loading…
Reference in New Issue