808 lines
30 KiB
Markdown
808 lines
30 KiB
Markdown
# Extension Challenges
|
||
|
||
You've built a basic concurrent TCP port scanner. Now make it production-ready with features that professional tools like Nmap have spent decades perfecting.
|
||
|
||
These challenges are ordered by difficulty. Start with the easier ones to build confidence, then tackle the harder ones when you want to dive deeper.
|
||
|
||
## Easy Challenges
|
||
|
||
### Challenge 1: CSV Output Format
|
||
|
||
**What to build:**
|
||
Add a command line flag `-o output.csv` that writes results to CSV instead of printing to terminal.
|
||
|
||
**Why it's useful:**
|
||
Security teams need machine-readable output for feeding into other tools. CSV loads into Excel, imports to databases, and processes with Python/awk scripts for reporting.
|
||
|
||
**What you'll learn:**
|
||
- File I/O in C++
|
||
- Structured output formats
|
||
- Making CLI tools pipeline-friendly
|
||
|
||
**Hints:**
|
||
- Add new option in `main.cpp` around line 14: `("output,o", po::value<std::string>(), "CSV output file")`
|
||
- Modify `PortScanner::scan()` to write to a file stream instead of `printf`
|
||
- CSV format: `port,state,service,banner` with proper escaping for quotes/commas in banners
|
||
- Don't forget to close the file when scanning completes
|
||
|
||
**Test it works:**
|
||
```bash
|
||
./simplePortScanner -i scanme.nmap.org -p 1-1024 -o results.csv
|
||
cat results.csv
|
||
# Should see: 22,OPEN,SSH,SSH-2.0-OpenSSH_...
|
||
```
|
||
|
||
### Challenge 2: Progress Indicator
|
||
|
||
**What to build:**
|
||
Show percentage completion during scans so users know it's working and how long to wait.
|
||
|
||
**Why it's useful:**
|
||
Full TCP scans of 65535 ports take minutes. Without feedback, users think the tool hung. Progress bars reduce anxiety and support requests.
|
||
|
||
**What you'll learn:**
|
||
- Terminal control codes for overwriting lines
|
||
- Calculating completion percentage with concurrent workers
|
||
- Balancing UI updates with performance (don't update every port, batch it)
|
||
|
||
**Hints:**
|
||
- Track `scanned_count` (ports finished) and `total_ports` (from start/end range)
|
||
- Update display every N ports (not every port - too slow): `if (scanned_count % 100 == 0)`
|
||
- Use `\r` to overwrite the current line: `printf("\rProgress: %d/%d (%.1f%%)", scanned, total, percent);`
|
||
- Flush output after printing: `fflush(stdout);`
|
||
- Look at `PortScanner.cpp:147,156` where statistics increment - add progress calculation there
|
||
|
||
**Test it works:**
|
||
```bash
|
||
./simplePortScanner -i scanme.nmap.org -p 1-10000
|
||
# Should show: Progress: 1000/10000 (10.0%)
|
||
# Progress: 2000/10000 (20.0%)
|
||
# ...updating in place
|
||
```
|
||
|
||
### Challenge 3: Scan Multiple Hosts
|
||
|
||
**What to build:**
|
||
Accept multiple targets: `./simplePortScanner -i 192.168.1.1,192.168.1.2,192.168.1.3 -p 80,443`
|
||
|
||
**Why it's useful:**
|
||
Pentesting requires scanning entire subnets. Rerunning the tool 254 times for a /24 network is tedious. Batch scanning is essential.
|
||
|
||
**What you'll learn:**
|
||
- Parsing comma-separated values
|
||
- Managing multiple endpoint targets
|
||
- Coordinating async operations across different hosts
|
||
|
||
**Hints:**
|
||
- Modify `parse_port()` pattern to create `parse_hosts()` that splits on commas
|
||
- Store vector of endpoints instead of single endpoint
|
||
- Outer loop over hosts, inner loop over ports (or vice versa - try both and compare performance)
|
||
- Print host IP/name with each result so you can tell which host a port belongs to
|
||
|
||
**Test it works:**
|
||
```bash
|
||
./simplePortScanner -i 8.8.8.8,1.1.1.1 -p 53
|
||
# Should show:
|
||
# 8.8.8.8 53 OPEN DNS ...
|
||
# 1.1.1.1 53 OPEN DNS ...
|
||
```
|
||
|
||
## Intermediate Challenges
|
||
|
||
### Challenge 4: JSON Output for Tool Integration
|
||
|
||
**What to build:**
|
||
Add `-o output.json --format json` to produce structured JSON output compatible with security tool chains.
|
||
|
||
**Real world application:**
|
||
CI/CD pipelines run port scans and check results programmatically. JSON integrates with Python security scripts, Splunk, ELK stack. This makes your scanner suitable for automated security testing.
|
||
|
||
**What you'll learn:**
|
||
- JSON serialization in C++ (use a library like nlohmann/json)
|
||
- Nested data structures for representing scan results
|
||
- Output format negotiation via CLI flags
|
||
|
||
**Implementation approach:**
|
||
|
||
1. **Add JSON library dependency** to CMakeLists.txt
|
||
- Download nlohmann/json: `https://github.com/nlohmann/json`
|
||
- Add include path or use FetchContent in CMake
|
||
|
||
2. **Collect results during scan** instead of immediate printing
|
||
- Create `std::vector<ScanResult>` where `ScanResult` has port, state, service, banner fields
|
||
- In completion handlers, append to vector instead of `printf`
|
||
- Print JSON at end in `run()`
|
||
|
||
3. **Structure JSON output:**
|
||
```json
|
||
{
|
||
"target": "192.168.1.1",
|
||
"scan_time": "2024-01-30T15:23:45Z",
|
||
"ports_scanned": 1024,
|
||
"results": [
|
||
{"port": 22, "state": "open", "service": "ssh", "banner": "SSH-2.0-..."},
|
||
{"port": 80, "state": "closed", "service": "http", "banner": null}
|
||
]
|
||
}
|
||
```
|
||
|
||
**Hints:**
|
||
- Don't try to write JSON manually with string concatenation (error-prone)
|
||
- Use library: `json j; j["port"] = 22; j["state"] = "open";`
|
||
- Handle special characters in banners (newlines, quotes)
|
||
|
||
**Extra credit:**
|
||
Support multiple output formats simultaneously: print human-readable to stdout and write JSON to file.
|
||
|
||
### Challenge 5: Service Version Detection
|
||
|
||
**What to build:**
|
||
Beyond basic banner grabbing, send protocol-specific probes to identify exact software versions even when services don't announce themselves.
|
||
|
||
**Real world application:**
|
||
Many hardened servers disable banners. HTTP servers configured with `ServerTokens Prod` just say "Apache" without version. FTP might not announce at all. Active probing extracts version info for vulnerability assessment.
|
||
|
||
**What you'll learn:**
|
||
- Application layer protocols (HTTP GET requests, SMTP EHLO commands)
|
||
- Protocol-specific fingerprinting techniques
|
||
- Managing multiple round-trips per port
|
||
|
||
**Implementation approach:**
|
||
|
||
1. **Create probe database** mapping ports to probe sequences
|
||
- Port 80: Send "GET / HTTP/1.0\r\n\r\n", parse Server header
|
||
- Port 21: Read banner, send "SYST\r\n", parse system type
|
||
- Port 25: Read banner, send "EHLO scanner\r\n", parse capabilities
|
||
|
||
2. **Extend banner grab logic** in `PortScanner.cpp:143`
|
||
- After reading initial banner, check if we have a probe for this port
|
||
- If yes, send probe via `async_write`
|
||
- Read response via another `async_read_some`
|
||
- Parse response to extract version
|
||
|
||
3. **Parse version strings:**
|
||
- HTTP: Extract from `Server:` header
|
||
- FTP: Parse `220 ProFTPD 1.3.5 Server` format
|
||
- SSH: Already in banner (SSH-2.0-OpenSSH_X.Y)
|
||
|
||
**Hints:**
|
||
- Look at Nmap's `nmap-service-probes` file for probe inspiration
|
||
- Handle protocols that need specific responses (FTP expects USER after connection)
|
||
- Some probes trigger IDS alerts (be careful with aggressive fingerprinting)
|
||
|
||
**Extra credit:**
|
||
Implement version matching against CPE database to map versions to CVEs automatically.
|
||
|
||
## Advanced Challenges
|
||
|
||
### Challenge 6: SYN Scan (Stealth Scanning)
|
||
|
||
**What to build:**
|
||
Implement half-open SYN scanning that doesn't complete the TCP handshake, making it stealthier than our current connect scan.
|
||
|
||
**Why this is hard:**
|
||
Requires raw sockets (root privileges), manual packet construction, handling responses at IP layer. You're bypassing the kernel's TCP stack entirely.
|
||
|
||
**What you'll learn:**
|
||
- Raw socket programming in Linux
|
||
- TCP packet structure (SYN flags, sequence numbers, checksums)
|
||
- Privilege escalation requirements and security implications
|
||
- Packet crafting with libraries like libnet or raw POSIX sockets
|
||
|
||
**Architecture changes needed:**
|
||
```
|
||
Current:
|
||
┌───────────┐
|
||
│ Kernel │ ← Handles TCP handshake
|
||
│ TCP Stack │
|
||
└───────────┘
|
||
↑
|
||
┌───────────┐
|
||
│ Scanner │ ← Calls connect()
|
||
└───────────┘
|
||
|
||
SYN Scan:
|
||
┌───────────┐
|
||
│ Kernel │ ← Bypassed for send, used for receive
|
||
└───────────┘
|
||
↑
|
||
┌───────────┐
|
||
│ Raw Socket│ ← Crafts SYN packets manually
|
||
└───────────┘
|
||
↑
|
||
┌───────────┐
|
||
│ Scanner │ ← Builds packets, listens for SYN-ACK
|
||
└───────────┘
|
||
```
|
||
|
||
**Implementation steps:**
|
||
|
||
1. **Research phase**
|
||
- Read TCP RFC 793 sections on SYN handshake
|
||
- Study Nmap's SYN scan implementation (open source)
|
||
- Understand TCP checksum calculation (pseudo-header + TCP header + data)
|
||
|
||
2. **Design phase**
|
||
- Decide: Use raw sockets or libnet/libpcap?
|
||
- Raw sockets = more control but harder. libnet = easier but dependency.
|
||
- Plan packet structure: IP header + TCP header with SYN flag
|
||
- Consider: Do you send from random source ports? (Nmap does for evasion)
|
||
|
||
3. **Implementation phase**
|
||
- Create raw socket: `socket(AF_INET, SOCK_RAW, IPPROTO_TCP)` (requires root)
|
||
- Build TCP SYN packet:
|
||
```cpp
|
||
struct tcphdr syn;
|
||
syn.th_sport = htons(random_port);
|
||
syn.th_dport = htons(target_port);
|
||
syn.th_seq = htonl(random_seq);
|
||
syn.th_flags = TH_SYN;
|
||
// ... set other fields
|
||
syn.th_sum = tcp_checksum(&syn);
|
||
```
|
||
- Send via `sendto()`
|
||
- Listen for response with `recvfrom()` or pcap filter
|
||
- Parse response:
|
||
- SYN-ACK = port open
|
||
- RST = port closed
|
||
- Nothing = filtered (or packet lost)
|
||
|
||
4. **Testing phase**
|
||
- Test against localhost first (easier to debug)
|
||
- Use Wireshark to verify packets are correct
|
||
- Compare results with connect scan (should match)
|
||
- Test filtering detection (timeout logic still applies)
|
||
|
||
**Gotchas:**
|
||
- **Kernel sends RST after SYN-ACK:** When you get SYN-ACK, kernel sends RST automatically (it doesn't know about your raw socket connection). This is normal but leaves traces in logs.
|
||
- **Checksum calculation is tricky:** TCP checksum includes pseudo-header with source/dest IPs. Get this wrong and packets are dropped silently.
|
||
- **IDS detection:** SYN scans without completing handshake trigger alerts on modern IDS. Less stealthy than you might think.
|
||
|
||
**Resources:**
|
||
- RFC 793 - TCP specification
|
||
- Nmap source code - `scan_engine.cc` has SYN scan logic
|
||
- libnet documentation - easier than raw sockets
|
||
|
||
### Challenge 7: OS Fingerprinting via TCP/IP Stack Differences
|
||
|
||
**What to build:**
|
||
Identify target operating system by analyzing TCP/IP implementation quirks (initial TTL, window size, TCP options, fragmentation handling).
|
||
|
||
**Why this is hard:**
|
||
Requires deep knowledge of OS-specific TCP behaviors, statistical analysis of multiple probes, and maintaining signature databases. You're exploiting implementation differences, not protocol vulnerabilities.
|
||
|
||
**What you'll learn:**
|
||
- TCP/IP stack implementation differences between OS families
|
||
- Passive vs active fingerprinting techniques
|
||
- Statistical classification from network behavior
|
||
- How tools like p0f and Nmap's os-detection work
|
||
|
||
**Implementation steps:**
|
||
|
||
**Phase 1: Data Collection** (2-4 hours)
|
||
- Capture TCP/IP characteristics from responses:
|
||
- Initial TTL (Linux: 64, Windows: 128, Cisco: 255)
|
||
- TCP window size (varies by OS and version)
|
||
- TCP options order (MSS, SACK, Timestamps, Window Scale)
|
||
- IPID behavior (incremental, random, zero)
|
||
- Don't Fragment (DF) bit usage
|
||
- Send crafted packets to elicit responses:
|
||
- SYN with unusual window sizes
|
||
- SYN with specific TCP options
|
||
- Empty ACK to closed port (RST response reveals info)
|
||
|
||
**Phase 2: Signature Database** (3-5 hours)
|
||
- Create OS fingerprint database:
|
||
```json
|
||
{
|
||
"Linux 5.x": {
|
||
"ttl": 64,
|
||
"window_size": 29200,
|
||
"tcp_options": "M*,S,T,N,W*",
|
||
"df_bit": true
|
||
},
|
||
"Windows 10": {
|
||
"ttl": 128,
|
||
"window_size": 8192,
|
||
"tcp_options": "M*,N,W*,S,T",
|
||
"df_bit": true
|
||
}
|
||
}
|
||
```
|
||
- Test against known systems to validate signatures
|
||
- Handle version variations (Windows 7 vs 10, Ubuntu 18.04 vs 22.04)
|
||
|
||
**Phase 3: Matching Logic** (4-6 hours)
|
||
- Implement fuzzy matching (exact matches are rare):
|
||
- TTL might have been decremented by routers
|
||
- Window sizes can be configured
|
||
- Weight different signals (TTL is most reliable)
|
||
- Calculate confidence scores:
|
||
```cpp
|
||
int score = 0;
|
||
if (ttl_matches) score += 50;
|
||
if (window_matches) score += 30;
|
||
if (options_match) score += 20;
|
||
return score >= 70 ? "High confidence" : "Uncertain";
|
||
```
|
||
|
||
**Phase 4: Integration** (2-3 hours)
|
||
- Hook into existing scanner after banner grab
|
||
- Send additional probes for fingerprinting
|
||
- Display OS guess with confidence: "Linux 2.6.X - 5.X (95%)"
|
||
|
||
**Testing strategy:**
|
||
- Test against VMs with known OSes (Ubuntu, Windows, FreeBSD)
|
||
- Test through NAT (TTL changes complicate things)
|
||
- Compare results with Nmap: `nmap -O target` should agree with your guess
|
||
|
||
**Known challenges:**
|
||
|
||
1. **TTL ambiguity**
|
||
- Problem: TTL 64 could be Linux (initial=64) or Windows (initial=128, crossed 64 hops)
|
||
- Hint: Use other signals to disambiguate, or probe with traceroute first
|
||
|
||
2. **Virtualization masking**
|
||
- Problem: VMs might mimic different OS at IP layer
|
||
- Hint: Combine with banner analysis (kernel version strings) for confirmation
|
||
|
||
**Success criteria:**
|
||
Your implementation should:
|
||
- [ ] Correctly identify Linux vs Windows vs macOS > 90% of time
|
||
- [ ] Distinguish major versions (Windows 10 vs 11, CentOS 7 vs 8)
|
||
- [ ] Handle ambiguous cases with "multiple possibilities" output
|
||
- [ ] Avoid false positives (don't confidently guess wrong OS)
|
||
- [ ] Process 10+ test fingerprints in < 5 seconds
|
||
|
||
## Expert Challenges
|
||
|
||
### Challenge 8: Full Nmap-Style Scan Engine
|
||
|
||
**What to build:**
|
||
A production-grade scanner supporting multiple scan types (SYN, ACK, FIN, Xmas, NULL), timing templates (Paranoid to Insane), OS detection, service versioning, and NSE-like scripting. This is a multi-week project.
|
||
|
||
**Estimated time:**
|
||
4-6 weeks of focused development for a basic implementation. 3-6 months for production quality.
|
||
|
||
**Prerequisites:**
|
||
You should have completed Challenges 1-7 first because this builds on SYN scanning, version detection, OS fingerprinting, and output formats.
|
||
|
||
**What you'll learn:**
|
||
- Production scanner architecture
|
||
- IDS evasion techniques
|
||
- Advanced network timing and congestion control
|
||
- Extensible plugin systems
|
||
- Real world network edge cases
|
||
|
||
**Planning this feature:**
|
||
|
||
Before you code, think through:
|
||
- How does scan type selection change packet crafting? (SYN vs FIN scans use different flags)
|
||
- What are the performance implications of 10,000+ concurrent operations? (File descriptor limits, memory)
|
||
- How do you migrate from simple port states to rich service metadata? (Database schema change)
|
||
- What's your rollback plan if timing templates overload the network? (Rate limiting, adaptive backoff)
|
||
|
||
**High level architecture:**
|
||
```
|
||
┌──────────────────────────────────────┐
|
||
│ CLI / Configuration │
|
||
│ (Scan type, timing, output format) │
|
||
└──────────────┬───────────────────────┘
|
||
│
|
||
┌─────────┼─────────┐
|
||
▼ ▼ ▼
|
||
┌─────────┐ ┌─────────┐ ┌─────────┐
|
||
│SYN Scan │ │ACK Scan │ │FIN Scan │
|
||
│ Engine │ │ Engine │ │ Engine │
|
||
└────┬────┘ └────┬────┘ └────┬────┘
|
||
│ │ │
|
||
└───────────┼───────────┘
|
||
▼
|
||
┌───────────────────────┐
|
||
│ Packet Constructor │
|
||
│ (TCP header building) │
|
||
└───────────┬───────────┘
|
||
│
|
||
┌───────────┼───────────┐
|
||
▼ ▼ ▼
|
||
┌─────────┐ ┌─────────┐ ┌──────────┐
|
||
│ Timer │ │Raw Sock │ │ Filter │
|
||
│ Engine │ │ I/O │ │ (BPF) │
|
||
└─────────┘ └─────────┘ └──────────┘
|
||
```
|
||
|
||
**Implementation phases:**
|
||
|
||
**Phase 1: Foundation** (1-2 weeks)
|
||
- Refactor existing code into modular scan engine interface
|
||
- Abstract packet construction (currently hardcoded for connect scan)
|
||
- Implement scan type registry (map scan names to implementations)
|
||
- Create unified result storage (database or in-memory structure)
|
||
|
||
**Phase 2: Scan Types** (2-3 weeks)
|
||
- Implement SYN scan (Challenge 6)
|
||
- Add FIN scan (send FIN instead of SYN, open ports don't respond)
|
||
- Add Xmas scan (FIN+PSH+URG flags set, looks like Christmas tree in Wireshark)
|
||
- Add NULL scan (no flags set, RFC violation triggers different responses)
|
||
- Add ACK scan (firewall mapping, not port state detection)
|
||
|
||
**Phase 3: Timing Templates** (1 week)
|
||
- T0 Paranoid: 5-minute delays between probes (IDS evasion, glacially slow)
|
||
- T1 Sneaky: Serialized scanning with pauses (evades basic detection)
|
||
- T2 Polite: Reduces network load (good for production systems)
|
||
- T3 Normal: Our current default (balance speed and stealth)
|
||
- T4 Aggressive: Faster timeouts, more parallelism
|
||
- T5 Insane: Maximum speed, assumes fast local network
|
||
|
||
**Phase 4: Advanced Features** (1-2 weeks)
|
||
- Integrate OS fingerprinting (Challenge 7)
|
||
- Add service version detection (Challenge 5)
|
||
- Implement output formats (JSON, XML, grepable) (Challenge 4)
|
||
- Add scan resumption (save state, restart interrupted scans)
|
||
|
||
**Testing strategy:**
|
||
- **Unit tests**: Mock network responses for each scan type
|
||
- **Integration tests**: Scan test VMs with known configurations
|
||
- **Performance tests**: Scan 10,000 ports, measure time and resource usage
|
||
- **Evasion tests**: Run against Snort IDS, measure detection rate
|
||
|
||
**Known challenges:**
|
||
|
||
1. **Packet Loss Handling**
|
||
- Problem: UDP scans lose packets, need retries
|
||
- Hint: Exponential backoff, cap max retries per port
|
||
|
||
2. **Network Congestion Detection**
|
||
- Problem: Aggressive scanning floods network, drops legitimate traffic
|
||
- Hint: Monitor RTT variance, back off when network slows
|
||
|
||
**Success criteria:**
|
||
Your implementation should:
|
||
- [ ] Support 5+ scan types (SYN, ACK, FIN, Xmas, NULL)
|
||
- [ ] Implement timing templates T0-T5 with measurable speed differences
|
||
- [ ] Correctly handle scan type selection via CLI flags
|
||
- [ ] Detect and adapt to network congestion (drop packet rate)
|
||
- [ ] Pass comparison tests against Nmap on identical targets
|
||
- [ ] Process full /24 subnet (254 hosts × 1000 ports) in < 10 minutes (T4)
|
||
|
||
### Challenge 9: IDS Evasion Techniques
|
||
|
||
**What to build:**
|
||
Implement fragmentation, decoy scans, source port manipulation, and timing randomization to evade intrusion detection systems.
|
||
|
||
**Estimated time:**
|
||
2-3 weeks (requires understanding IDS internals first)
|
||
|
||
**Prerequisites:**
|
||
Complete SYN scan implementation (Challenge 6) since these techniques modify packet-level behavior.
|
||
|
||
**What you'll learn:**
|
||
- How IDS systems like Snort detect scans
|
||
- IP fragmentation and reassembly
|
||
- Spoofing techniques and limitations
|
||
- The cat-and-mouse game between attackers and defenders
|
||
|
||
**Implementation steps:**
|
||
|
||
**Phase 1: Research IDS Detection Signatures** (3-5 hours)
|
||
Read Snort rules for port scan detection:
|
||
```
|
||
alert tcp any any -> any any (flags:S; threshold: type both, track by_src, count 10, seconds 60; msg:"Possible SYN scan";)
|
||
```
|
||
This triggers on 10+ SYN packets to different ports from one source in 60 seconds. Our scanner easily exceeds this.
|
||
|
||
**Phase 2: Packet Fragmentation** (1 week)
|
||
Split TCP SYN packets across multiple IP fragments:
|
||
```cpp
|
||
// Normal packet: [IP Header][TCP Header][Options]
|
||
|
||
// Fragmented:
|
||
// Packet 1: [IP Header (MF=1, offset=0)][TCP Header partial]
|
||
// Packet 2: [IP Header (MF=0, offset=8)][TCP Header remaining][Options]
|
||
```
|
||
Many older IDS can't reassemble fragments, so they miss the scan. Modern IDS handles this, but it's still useful against legacy systems.
|
||
|
||
**Phase 3: Decoy Scanning** (4-5 days)
|
||
Send scans from fake source IPs mixed with your real IP:
|
||
```
|
||
Real scanner: 10.0.0.100
|
||
Decoys: 10.0.0.50, 10.0.0.75, 10.0.0.125
|
||
|
||
Target sees SYN packets from:
|
||
10.0.0.50:12345 -> target:80
|
||
10.0.0.75:12346 -> target:80
|
||
10.0.0.100:12347 -> target:80 ← Real scanner
|
||
10.0.0.125:12348 -> target:80
|
||
```
|
||
IDS sees scanning from multiple sources, can't determine which is real. Only you see SYN-ACK responses (sent to your IP).
|
||
|
||
**Gotchas:**
|
||
- Decoy IPs must be alive (respond to pings) or target might filter "dead" sources
|
||
- Too many decoys = obvious attack pattern
|
||
- Asymmetric routing breaks this (target might respond via different path)
|
||
|
||
**Phase 4: Timing Randomization** (2-3 days)
|
||
Add jitter to probe timing:
|
||
```cpp
|
||
// Bad: Regular 100ms intervals
|
||
send_probe(); sleep(0.1);
|
||
send_probe(); sleep(0.1);
|
||
|
||
// Good: Random intervals between 50-150ms
|
||
send_probe(); sleep(random(0.05, 0.15));
|
||
send_probe(); sleep(random(0.05, 0.15));
|
||
```
|
||
Defeats timing-based detection (burst of regular probes = scanner signature).
|
||
|
||
**Success criteria:**
|
||
- [ ] Snort default ruleset doesn't alert on your scans
|
||
- [ ] Fragmentation bypasses basic IDS (test with tcpdump reassembly)
|
||
- [ ] Decoy scans hide your real IP in logs (confirmed via target logs)
|
||
- [ ] Randomization defeats threshold-based detection (burst detector doesn't trigger)
|
||
|
||
## Mix and Match
|
||
|
||
Combine features for bigger projects:
|
||
|
||
**Project Idea 1: Cloud Security Scanner**
|
||
- Combine Challenge 3 (multiple hosts) + Challenge 4 (JSON output) + Challenge 5 (version detection)
|
||
- Add AWS/GCP cloud integration (scan entire VPCs)
|
||
- Result: Feed results into lambda functions for automated CVE checking
|
||
|
||
**Project Idea 2: Continuous Monitoring Dashboard**
|
||
- Challenge 2 (progress bars) + Challenge 4 (JSON) + web UI
|
||
- Run scans periodically, store results in database
|
||
- Visualize port changes over time (new ports = potential compromise)
|
||
|
||
## Real World Integration Challenges
|
||
|
||
### Integrate with Metasploit for Automated Exploitation
|
||
|
||
**The goal:**
|
||
After scanning, automatically launch Metasploit modules against discovered vulnerable services.
|
||
|
||
**What you'll need:**
|
||
- Metasploit Framework installed
|
||
- RPC API access to msfconsole
|
||
- Version detection implemented (Challenge 5)
|
||
|
||
**Implementation plan:**
|
||
1. Output scan results with service versions to JSON
|
||
2. Map service versions to Metasploit modules (MSF database lookup)
|
||
3. Use MSF RPC to launch exploits:
|
||
```ruby
|
||
client = Msf::RPC::Client.new(...)
|
||
client.call('module.execute', 'exploit', 'exploit/linux/ssh/...')
|
||
```
|
||
4. Collect exploitation results
|
||
|
||
**Watch out for:**
|
||
- Ethics: Only run on systems you own or have written permission to test
|
||
- False positives: Version detection isn't perfect, might target wrong systems
|
||
- Rate limiting: Don't launch 100 exploits simultaneously
|
||
|
||
### Deploy on AWS Lambda for Serverless Scanning
|
||
|
||
**The goal:**
|
||
Run distributed scans from Lambda functions across different regions.
|
||
|
||
**What you'll learn:**
|
||
- Serverless architecture patterns
|
||
- Network restrictions in Lambda (no raw sockets)
|
||
- Distributing work across cloud functions
|
||
|
||
**Steps:**
|
||
1. Package scanner as Lambda deployment (zip with dependencies)
|
||
2. Configure IAM role for network access
|
||
3. Trigger Lambda with target list (SQS queue)
|
||
4. Collect results in S3 or DynamoDB
|
||
5. Aggregate from Lambda results processor
|
||
|
||
**Production checklist:**
|
||
- [ ] Error handling for Lambda timeouts (15 min limit)
|
||
- [ ] VPC configuration if scanning private networks
|
||
- [ ] Cost estimation (Lambda + data transfer can get expensive)
|
||
- [ ] Rate limiting to avoid overwhelming targets
|
||
|
||
## Performance Challenges
|
||
|
||
### Challenge: Handle 100,000 Concurrent Connections
|
||
|
||
**The goal:**
|
||
Scan 1000 hosts × 1000 ports each = 1,000,000 ports without crashing.
|
||
|
||
**Current bottleneck:**
|
||
File descriptor limits. Linux defaults to 1024 open files per process. Our scanner creates socket + timer per port = 2 FDs per concurrent operation. At 100 threads, we use ~200 FDs. At 100,000 we'd need 200,000 (impossible).
|
||
|
||
**Optimization approaches:**
|
||
|
||
**Approach 1: Increase FD Limit**
|
||
- How: `ulimit -n 100000` (temporary), modify `/etc/security/limits.conf` (permanent)
|
||
- Gain: Supports more concurrent connections
|
||
- Tradeoff: Kernel memory for tracking FDs, still capped by system-wide limit
|
||
|
||
**Approach 2: Socket Pooling and Reuse**
|
||
- How: Close sockets immediately after results, reuse FD
|
||
- Implementation: In completion handler, close socket before calling `scan()` again
|
||
- Gain: Only need FDs for active probes
|
||
- Tradeoff: Slightly more complex lifecycle management
|
||
|
||
**Approach 3: Hybrid Batch Processing**
|
||
- How: Scan in batches of 10k ports, process results, scan next batch
|
||
- Gain: Bounded memory usage
|
||
- Tradeoff: Doesn't leverage full concurrency potential
|
||
|
||
**Benchmark it:**
|
||
```bash
|
||
# Monitor FD usage
|
||
watch -n 0.1 'ls -l /proc/$(pgrep simplePortScanner)/fd | wc -l'
|
||
|
||
# Run large scan
|
||
./simplePortScanner -i target -p 65535 -t 10000
|
||
```
|
||
|
||
Target metrics:
|
||
- FD usage stays below system limit
|
||
- Memory usage < 1GB even at high concurrency
|
||
- Scan completes without crashes
|
||
|
||
### Challenge: Reduce Network Bandwidth Usage
|
||
|
||
**The goal:**
|
||
Cut bandwidth by 50% while maintaining scan accuracy.
|
||
|
||
**Profile first:**
|
||
```bash
|
||
# Monitor bandwidth
|
||
iftop -i eth0
|
||
|
||
# Current usage: ~5 Mbps for 100 concurrent scans
|
||
```
|
||
|
||
**Common optimization areas:**
|
||
- Reduce timeout from 2s to 1s (fewer retries on slow networks)
|
||
- Only grab banners for interesting ports (80, 443, 22) not every open port
|
||
- Implement adaptive timeout based on RTT measurements
|
||
|
||
## Security Challenges
|
||
|
||
### Challenge: Implement Port Knock Sequence Detection
|
||
|
||
**What to implement:**
|
||
Before scanning, knock on specific ports in sequence to signal "friendly" scanner and avoid triggering alerts.
|
||
|
||
**Threat model:**
|
||
This protects against:
|
||
- Automated IDS blocking your scanner IP
|
||
- Admin annoyance at legitimate security testing
|
||
- Revealing your scanning activity to casual log reviewers
|
||
|
||
**Implementation:**
|
||
```cpp
|
||
void knock_sequence(const std::string& target, const std::vector<int>& sequence) {
|
||
for (int port : sequence) {
|
||
tcp::socket s(io);
|
||
s.connect(tcp::endpoint(address, port));
|
||
s.close();
|
||
sleep(0.5); // Delay between knocks
|
||
}
|
||
// Now run actual scan
|
||
}
|
||
|
||
// Usage:
|
||
knock_sequence("target.com", {1234, 5678, 9012}); // Secret sequence
|
||
```
|
||
|
||
**Testing the security:**
|
||
- Configure target server with port knock daemon (knockd on Linux)
|
||
- Scan without knocking - should be blocked/logged aggressively
|
||
- Scan with knock sequence - should proceed without alerts
|
||
- Verify logs show different behavior
|
||
|
||
### Challenge: Add Scan Attribution Watermark
|
||
|
||
**The goal:**
|
||
Make this project compliant with responsible disclosure by embedding scanner identity in packets.
|
||
|
||
**Threat model:**
|
||
This protects against:
|
||
- Your scanner being mistaken for malicious attacker
|
||
- Difficulty identifying scanning source during incident response
|
||
- Ethical issues with anonymous security testing
|
||
|
||
**Implementation:**
|
||
Add custom TCP option or banner request that identifies your scanner:
|
||
```cpp
|
||
// HTTP probe includes User-Agent
|
||
"GET / HTTP/1.1\r\n"
|
||
"Host: " + target + "\r\n"
|
||
"User-Agent: PortScanner-Learning-Project/1.0 (Educational; Contact: your@email.com)\r\n"
|
||
"\r\n"
|
||
```
|
||
|
||
Now when admins investigate, logs clearly show educational scanning with contact info.
|
||
|
||
## Contribution Ideas
|
||
|
||
Finished a challenge? Share it back:
|
||
|
||
1. **Fork the repo** (if this was hosted on GitHub)
|
||
2. **Implement your extension** in a new branch: `git checkout -b feature/syn-scan`
|
||
3. **Document it** - Add section to this file explaining your implementation
|
||
4. **Submit a PR** with:
|
||
- Code changes with comments
|
||
- Unit tests if applicable
|
||
- Updated README.md mentioning new feature
|
||
- Example usage in documentation
|
||
|
||
Good extensions might get merged into the main project and help future learners.
|
||
|
||
## Challenge Yourself Further
|
||
|
||
### Build Something New
|
||
|
||
Use the concepts you learned here to build:
|
||
|
||
- **Vulnerability Scanner** - After port scan, run checks for known vulns (Heartbleed, ShellShock) on discovered services
|
||
- **Network Topology Mapper** - Use traceroute + port scanning to visualize network structure and firewall boundaries
|
||
- **Continuous Security Monitor** - Scheduled scanning with alerting when new ports open (indicator of compromise)
|
||
|
||
### Study Real Implementations
|
||
|
||
Compare your implementation to production tools:
|
||
|
||
- **Nmap** - Read source code at https://github.com/nmap/nmap - see how they handle edge cases you haven't thought of
|
||
- **masscan** - Asynchronous scanner that can scan the entire internet (4 billion IPs). Study their packet rate limiting.
|
||
- **ZMap** - Similar to masscan but simpler architecture. Good for learning high-performance scanning patterns.
|
||
|
||
Read their code, understand their tradeoffs, adapt their techniques to your scanner.
|
||
|
||
### Write About It
|
||
|
||
Document your extension:
|
||
- Blog post: "Building a SYN Scanner from Scratch in C++"
|
||
- Tutorial: "Port Scanning 101: From Theory to Implementation"
|
||
- Comparison: "Connect Scan vs SYN Scan: Performance and Detection Analysis"
|
||
|
||
Teaching others forces you to truly understand the concepts. If you can't explain it simply, you don't understand it well enough.
|
||
|
||
## Getting Help
|
||
|
||
Stuck on a challenge?
|
||
|
||
1. **Debug systematically**
|
||
- What did you expect to happen?
|
||
- What actually happened?
|
||
- What's the smallest code change that reproduces the issue?
|
||
|
||
2. **Read existing implementations**
|
||
- How does Nmap handle this? (Source is open)
|
||
- Look at Boost.Asio examples for async patterns
|
||
- Search for "TCP SYN scan implementation C++" if doing Challenge 6
|
||
|
||
3. **Search for similar problems**
|
||
- Stack Overflow tag: [boost-asio]
|
||
- Reddit: r/netsec, r/cpp
|
||
- GitHub issues on Nmap/masscan repos
|
||
|
||
4. **Ask for help constructively**
|
||
- Show what you tried: code snippets, error messages
|
||
- Explain your understanding: "I think this should work because..."
|
||
- Be specific: "SYN packets aren't triggering responses" not "it doesn't work"
|
||
|
||
## Challenge Completion Tracker
|
||
|
||
Track your progress:
|
||
|
||
- [ ] Easy Challenge 1: CSV Output
|
||
- [ ] Easy Challenge 2: Progress Indicator
|
||
- [ ] Easy Challenge 3: Multiple Hosts
|
||
- [ ] Intermediate Challenge 4: JSON Output
|
||
- [ ] Intermediate Challenge 5: Service Version Detection
|
||
- [ ] Advanced Challenge 6: SYN Scan
|
||
- [ ] Advanced Challenge 7: OS Fingerprinting
|
||
- [ ] Expert Challenge 8: Full Scan Engine
|
||
- [ ] Expert Challenge 9: IDS Evasion
|
||
|
||
Completed all of them? You've gone from beginner port scanner to advanced network reconnaissance tool. You understand async I/O, network protocols, and security fundamentals at a deep level. Time to build something entirely new or contribute to open source security tools like Nmap or Metasploit.
|