Users can now better control desired program runtime and scanning depth.

README.md

@@ -13,9 +13,11 @@ AutoRecon was inspired by three tools which the author used during the OSCP labs
 ## Features
 
 * Supports multiple targets in the form of IP addresses, IP ranges (CIDR notation), and resolvable hostnames.
+* Targets can either be specified via the command line or read from a file.
 * Can scan targets concurrently, utilizing multiple processors if they are available.
 * Customizable port scanning profiles for flexibility in your initial scans.
 * Customizable service enumeration commands and suggested manual follow-up commands.
+* Support for different complexity levels to better balance runtime and scanning depth during service enumeration.
 * An intuitive directory structure for results gathering.
 * Full logging of commands that were run, along with errors if they fail.
 * Global and per-scan pattern matching so you can highlight/extract important information from the noise.
@@ -62,15 +64,20 @@ whatweb
 wkhtmltoimage
 ```
 
+AutoRecon performs some checks during startup and will indicate any program that is missing and will potentially affect the result gathering process.
+
+
 ## Usage
 
 AutoRecon uses Python 3 specific functionality and does not support Python 2.
 
 ```
 usage: autorecon.py [-h] [-ct <number>] [-cs <number>] [--profile PROFILE]
-                    [-o OUTPUT] [--nmap NMAP | --nmap-append NMAP_APPEND] [-v]
+                    [-o OUTPUT] [--nmap NMAP | --nmap-append NMAP_APPEND]
-                    [--disable-sanity-checks]
+                    [--skip-service-scan]
-                    targets [targets ...]
+                    [--run-level RUN_LEVEL [RUN_LEVEL ...]] [--run-only]
+                    [-r TARGET_FILE] [-v] [--disable-sanity-checks]
+                    [targets [targets ...]]
 
 Network reconnaissance tool to port scan and automatically enumerate services
 found on multiple targets.
@@ -93,9 +100,18 @@ optional arguments:
   -o OUTPUT, --output OUTPUT
                         The output directory for results. Default: results
   --nmap NMAP           Override the {nmap_extra} variable in scans. Default:
-                        -vv --reason -Pn
+                        --reason -Pn
   --nmap-append NMAP_APPEND
                         Append to the default {nmap_extra} variable in scans.
+  --skip-service-scan   Do not perfom extended service scanning but only
+                        document commands.
+  --run-level RUN_LEVEL [RUN_LEVEL ...]
+                        During extended service scanning, only run scanners of
+                        a certain complexity level or below.
+  --run-only            If enabled, only run scanners of the specified
+                        complexity level during extended service scanning.
+  -r TARGET_FILE, --read TARGET_FILE
+                        Read targets from file.
   -v, --verbose         Enable verbose output. Repeat for more verbosity.
   --disable-sanity-checks
                         Disable sanity checks that would otherwise prevent the
@@ -227,31 +243,43 @@ AutoRecon supports three levels of verbosity:
 By default, results will be stored in the ./results directory. A new sub directory is created for every target. The structure of this sub directory is:
 
 ```
-.
+├── exploit
-├── exploit/
+├── loot
-├── loot/
+├── privilege_escalation
-├── report/
+├── report
 │   ├── local.txt
-│   ├── notes.txt
 │   ├── proof.txt
-│   └── screenshots/
+│   └── screenshots
-└── scans/
+└── scans
     ├── _commands.log
-    ├── _manual_commands.txt
+    ├── _errors.log
-    └── xml/
+    ├── _full_tcp_nmap.txt
+    ├── _manual_commands.log
+    ├── _notes.txt
+    ├── _patterns.log
+    ├── _quick_tcp_nmap.txt
+    ├── <service>
+    ├── <service>
+    ├── (...)
+    └── xml
+        ├── <service>
+        ├── <service>
+        ├── (...)
 ```
 
 The exploit directory is intended to contain any exploit code you download / write for the target.
 
 The loot directory is intended to contain any loot (e.g. hashes, interesting files) you find on the target.
 
+The privilege_escalation directory is intended to contain any code for escalating the privileges on the target once initial access has been gained
+
 The report directory contains some auto-generated files and directories that are useful for reporting:
 * local.txt can be used to store the local.txt flag found on targets.
 * notes.txt should contain a basic template where you can write notes for each service discovered.
 * proof.txt can be used to store the proof.txt flag found on targets.
 * The screenshots directory is intended to contain the screenshots you use to document the exploitation of the target.
 
-The scans directory is where all results from scans performed by AutoRecon will go. This includes port scans / service detection scans, as well as any service enumeration scans. It also contains two other files:
+The scans directory (and respective subdirectories) is where all results from scans performed by AutoRecon will go. This includes port scans / service detection scans, as well as any service enumeration scans. It also contains two other files:
 * \_commands.log contains a list of every command AutoRecon ran against the target. This is useful if one of the commands fails and you want to run it again with modifications.
 * \_manual_commands.txt contains any commands that are deemed "too dangerous" to run automatically, either because they are too intrusive, require modification based on human analysis, or just work better when there is a human monitoring them.
 
@@ -259,6 +287,8 @@ If a scan results in an error, a file called \_errors.log will also appear in th
 
 If output matches a defined pattern, a file called \_patterns.log will also appear in the scans directory with details about the matched output.
 
+The file \_notes.txt contains information about any open ports discovered during scanning.
+
 The scans/xml directory stores any XML output (e.g. from Nmap scans) separately from the main scan outputs, so that the scans directory itself does not get too cluttered.
 
 ### Port Scan profiles
@@ -381,6 +411,7 @@ service-names = [
     run_once = true
     ports.tcp = [139, 389, 445]
     ports.udp = [137]
+    level = 2
 
     [[smb.scan]]
     name = 'nbtscan'
@@ -426,6 +457,19 @@ In fact, enum4linux will always try these ports when it is run. So if the SMB se
 
 * If run_once is set to true, the command will only ever run once for that target, even if the SMB service is found on multiple ports.
 
+* The level variable indicates the complexity of performing the scan. For instance, the nikto security scanner is able to examine a running web server in depth. At the same time, the scanning process is also lengthy and significantly increases the runtime of AutoRecon when evaluating multiple targets. 
+
+With the help of the --run-level parameter, the user has the possibility of better balancing required scanning time and scanning depth. Programs that require a longer runtime can also be run individually by making use of the --run-only parameter. Thereby, the user may run a quick scan first, followed by a more in-depth scan, e.g., 
+
+# run any extended service enumeration program with a complexity level of 2 or lower
+python3 autorecon.py 127.0.0.1 --run-level 2
+
+# only run programs with a complexity level of 3
+python3 autorecon.py 127.0.0.1 --run-level 3 --run-only
+
+The user may also skip extended service scanning entirely by specifying the --skip-service-scan parameter. In this case, respective commands will be only documented but not executed, giving the tester a good check list for later service analysis and planning the attack more carefully.
+
+
 ## Testimonials
 
 > AutoRecon was invaluable during my OSCP exam, in that it saved me from the tedium of executing my active information gathering commands myself.  I was able to start on a target with all of the information I needed clearly laid in front of me.  I would strongly recommend this utility for anyone in the PWK labs, the OSCP exam, or other environments such as VulnHub or HTB.  It is a great tool for both people just starting down their journey into OffSec and seasoned veterans alike.  Just make sure that somewhere between those two points you take the time to learn what's going on "under the hood" and how / why it scans what it does.

autorecon.py

@@ -23,13 +23,24 @@ import toml
 verbose = 0
 nmap_default_options = '--reason -Pn'
 srvname = ''
-port_scan_profile = None
 
+# number of possible complexity levels for scanners
+max_level = 3
+
+port_scan_profile = None
 port_scan_profiles_config = None
 service_scans_config = None
 global_patterns = []
 applications = {}
 
+files = {
+            'commands'          :   '_commands.log',
+            'manual_commands'   :   '_manual_commands.log',
+            'patterns'          :   '_patterns.log',
+            'notes'             :   '_notes.txt',
+            'errors'            :   '_errors.log',
+        }
+
 username_wordlist = '/usr/share/seclists/Usernames/top-usernames-shortlist.txt'
 password_wordlist = '/usr/share/seclists/Passwords/darkweb2017-top100.txt'
 
@@ -181,14 +192,14 @@ async def read_stream(stream, target, tag='?', patterns=[], color=Fore.BLUE):
                         if verbose >= 1:
                             info('Task {bgreen}{tag}{rst} on {byellow}{address}{rst} - {bmagenta}' + p['description'].replace('{match}', '{bblue}{match}{crst}{bmagenta}') + '{rst}')
                         async with target.lock:
-                            with open(os.path.join(target.scandir, '_patterns.log'), 'a') as file:
+                            with open(os.path.join(target.scandir, files['patterns']), 'a') as file:
                                 file.writelines(e('{tag} - ' + p['description'] + '\n\n'))
                 else:
                     for match in matches:
                         if verbose >= 1:
                             info('Task {bgreen}{tag}{rst} on {byellow}{address}{rst} - {bmagenta}Matched Pattern: {bblue}{match}{rst}')
                         async with target.lock:
-                            with open(os.path.join(target.scandir, '_patterns.log'), 'a') as file:
+                            with open(os.path.join(target.scandir, files['patterns']), 'a') as file:
                                 file.writelines(e('{tag} - Matched Pattern: {match}\n\n'))
 
             for p in patterns:
@@ -198,30 +209,34 @@ async def read_stream(stream, target, tag='?', patterns=[], color=Fore.BLUE):
                         if verbose >= 1:
                             info('Task {bgreen}{tag}{rst} on {byellow}{address}{rst} - {bmagenta}' + p['description'].replace('{match}', '{bblue}{match}{crst}{bmagenta}') + '{rst}')
                         async with target.lock:
-                            with open(os.path.join(target.scandir, '_patterns.log'), 'a') as file:
+                            with open(os.path.join(target.scandir, files['patterns']), 'a') as file:
                                 file.writelines(e('{tag} - ' + p['description'] + '\n\n'))
                 else:
                     for match in matches:
                         if verbose >= 1:
                             info('Task {bgreen}{tag}{rst} on {byellow}{address}{rst} - {bmagenta}Matched Pattern: {bblue}{match}{rst}')
                         async with target.lock:
-                            with open(os.path.join(target.scandir, '_patterns.log'), 'a') as file:
+                            with open(os.path.join(target.scandir, files['patterns']), 'a') as file:
                                 file.writelines(e('{tag} - Matched Pattern: {match}\n\n'))
         else:
             break
 
-async def run_cmd(semaphore, cmd, target, tag='?', patterns=[]):
+async def run_cmd(semaphore, cmd, target, category='?', tag='?', patterns=[]):
     async with semaphore:
         address = target.address
         scandir = target.scandir
 
+        if len(category) == 0: category = 'all'
+        category = category.strip('/')
+
         info('Running task {bgreen}{tag}{rst} on {byellow}{address}{rst}' + (' with {bblue}{cmd}{rst}.' if verbose >= 1 else '.'))
 
         async with target.lock:
-            with open(os.path.join(scandir, '_commands.log'), 'a') as file:
+            with open(os.path.join(scandir, files['commands']), 'a') as file:
-                file.writelines(e('{cmd}\n\n'))
+                file.writelines(e('{category} - {cmd}\n\n'))
         
-        # TODO: check extended service scanning requested?
+        # skip extended service scanning if only respective commands should be documented 
+        if args.skip_service_scan: return {'returncode': 0, 'name': 'run_cmd'}
         
         process = await asyncio.create_subprocess_shell(cmd, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, executable='/bin/bash')
 
@@ -235,7 +250,7 @@ async def run_cmd(semaphore, cmd, target, tag='?', patterns=[]):
     if process.returncode != 0:
         error('Task {bred}{tag}{rst} on {byellow}{address}{rst} returned non-zero exit code: {process.returncode}.')
         async with target.lock:
-            with open(os.path.join(scandir, '_errors.log'), 'a') as file:
+            with open(os.path.join(scandir, files['errors']), 'a') as file:
                 file.writelines(e('[*] Task {tag} returned non-zero exit code: {process.returncode}. Command: {cmd}\n'))
     else:
         info('Task {bgreen}{tag}{rst} on {byellow}{address}{rst} finished successfully.')
@@ -264,14 +279,14 @@ async def parse_port_scan(stream, tag, target, pattern):
                         if verbose >= 1:
                             info('Task {bgreen}{tag}{rst} on {byellow}{address}{rst} - {bmagenta}' + p['description'].replace('{match}', '{bblue}{match}{crst}{bmagenta}') + '{rst}')
                         async with target.lock:
-                            with open(os.path.join(target.scandir, '_patterns.log'), 'a') as file:
+                            with open(os.path.join(target.scandir, files['patterns']), 'a') as file:
                                 file.writelines(e('{tag} - ' + p['description'] + '\n\n'))
                 else:
                     for match in matches:
                         if verbose >= 1:
                             info('Task {bgreen}{tag}{rst} on {byellow}{address}{rst} - {bmagenta}Matched Pattern: {bblue}{match}{rst}')
                         async with target.lock:
-                            with open(os.path.join(target.scandir, '_patterns.log'), 'a') as file:
+                            with open(os.path.join(target.scandir, files['patterns']), 'a') as file:
                                 file.writelines(e('{tag} - Matched Pattern: {match}\n\n'))
         else:
             break
@@ -299,14 +314,14 @@ async def parse_service_detection(stream, tag, target, pattern):
                         if verbose >= 1:
                             info('Task {bgreen}{tag}{rst} on {byellow}{address}{rst} - {bmagenta}' + p['description'].replace('{match}', '{bblue}{match}{crst}{bmagenta}') + '{rst}')
                         async with target.lock:
-                            with open(os.path.join(target.scandir, '_patterns.log'), 'a') as file:
+                            with open(os.path.join(target.scandir, files['patterns']), 'a') as file:
                                 file.writelines(e('{tag} - ' + p['description'] + '\n\n'))
                 else:
                     for match in matches:
                         if verbose >= 1:
                             info('Task {bgreen}{tag}{rst} on {byellow}{address}{rst} - {bmagenta}Matched Pattern: {bblue}{match}{rst}')
                         async with target.lock:
-                            with open(os.path.join(target.scandir, '_patterns.log'), 'a') as file:
+                            with open(os.path.join(target.scandir, files['patterns']), 'a') as file:
                                 file.writelines(e('{tag} - Matched Pattern: {match}\n\n'))
         else:
             break
@@ -328,7 +343,7 @@ async def run_portscan(semaphore, tag, target, service_detection, port_scan=None
             info('Running port scan {bgreen}{tag}{rst} on {byellow}{address}{rst}' + (' with {bblue}{command}{rst}.' if verbose >= 1 else '.'))
 
             async with target.lock:
-                with open(os.path.join(scandir, '_commands.log'), 'a') as file:
+                with open(os.path.join(scandir, files['commands']), 'a') as file:
                     file.writelines(e('{command}\n\n'))
 
             process = await asyncio.create_subprocess_shell(command, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, executable='/bin/bash')
@@ -345,7 +360,7 @@ async def run_portscan(semaphore, tag, target, service_detection, port_scan=None
             if process.returncode != 0:
                 error('Port scan {bred}{tag}{rst} on {byellow}{address}{rst} returned non-zero exit code: {process.returncode}')
                 async with target.lock:
-                    with open(os.path.join(scandir, '_errors.log'), 'a') as file:
+                    with open(os.path.join(scandir, files['errors']), 'a') as file:
                         file.writelines(e('[*] Port scan {tag} returned non-zero exit code: {process.returncode}. Command: {command}\n'))
                 return {'returncode': process.returncode}
             else:
@@ -363,7 +378,7 @@ async def run_portscan(semaphore, tag, target, service_detection, port_scan=None
         info('Running service detection {bgreen}{tag}{rst} on {byellow}{address}{rst}' + (' with {bblue}{command}{rst}.' if verbose >= 1 else '.'))
 
         async with target.lock:
-            with open(os.path.join(scandir, '_commands.log'), 'a') as file:
+            with open(os.path.join(scandir, files['commands']), 'a') as file:
                 file.writelines(e('{command}\n\n'))
 
         process = await asyncio.create_subprocess_shell(command, stdout=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE, executable='/bin/bash')
@@ -380,7 +395,7 @@ async def run_portscan(semaphore, tag, target, service_detection, port_scan=None
         if process.returncode != 0:
             error('Service detection {bred}{tag}{rst} on {byellow}{address}{rst} returned non-zero exit code: {process.returncode}')
             async with target.lock:
-                with open(os.path.join(scandir, '_errors.log'), 'a') as file:
+                with open(os.path.join(scandir, files['errors']), 'a') as file:
                     file.writelines(e('[*] Service detection {tag} returned non-zero exit code: {process.returncode}. Command: {command}\n'))
         else:
             info('Service detection {bgreen}{tag}{rst} on {byellow}{address}{rst} finished successfully.')
@@ -429,8 +444,8 @@ async def scan_services(loop, semaphore, target):
 
                         info('Port {bmagenta}{protocol} {port}{rst} ({bmagenta}{service}{rst}) open on target {byellow}{address}{rst}.')
 
-                        with open(os.path.join(target.reportdir, 'notes.txt'), 'a') as file:
+                        with open(os.path.join(target.scandir, files['notes']), 'a') as file:
-                            file.writelines(e('[*] Port {protocol} {port} ({service}) open on {address}.\n\n\n\n'))
+                            file.writelines(e('[*] Port {protocol} {port} ({service}) open on {address}.\n\n'))
 
                         if protocol == 'udp':
                             nmap_extra = nmap_default_options + " -sU"
@@ -471,7 +486,7 @@ async def scan_services(loop, semaphore, target):
                             if not matched_service:
                                 continue
 
-                            # NOTE: change for saving results in directories per service
+                            # INFO: change for saving results in directories per service
                             if not service_scan == 'all-services':
                                 category = '{0}/'.format(service_scan) 
                             else:
@@ -487,7 +502,7 @@ async def scan_services(loop, semaphore, target):
 
                             if 'manual' in service_scans_config[service_scan]:
                                 heading = False
-                                with open(os.path.join(scandir, '_manual_commands.txt'), 'a') as file:
+                                with open(os.path.join(scandir, files['manual_commands']), 'a') as file:
                                     for manual in service_scans_config[service_scan]['manual']:
                                         if 'description' in manual:
                                             if not heading:
@@ -507,9 +522,16 @@ async def scan_services(loop, semaphore, target):
 
                             if 'scan' in service_scans_config[service_scan]:
                                 for scan in service_scans_config[service_scan]['scan']:
-
                                     if 'name' in scan:
                                         name = scan['name']
+                                        
+                                        # INFO: change for supporting different complexity levels during service scanning
+                                        run_level = scan['level'] if 'level' in scan else 0
+                                        if (not args.run_only and run_level > max(args.run_level)) or (args.run_only and not run_level in args.run_level):    
+                                            if verbose >= 1:
+                                                info('Scan profile {bgreen}{name}{rst} is at a {bgree}different complexity level{rst} and is ignored.')
+                                            continue
+
                                         if 'command' in scan:
                                             tag = e('{protocol}/{port}/{name}')
                                             command = scan['command']
@@ -553,7 +575,7 @@ async def scan_services(loop, semaphore, target):
                                             if 'pattern' in scan:
                                                 patterns = scan['pattern']
 
-                                            pending.add(asyncio.ensure_future(run_cmd(semaphore, e(command), target, tag=tag, patterns=patterns)))
+                                            pending.add(asyncio.ensure_future(run_cmd(semaphore, e(command), target, category=category, tag=tag, patterns=patterns)))
 
 def scan_host(target, concurrent_scans):
     info('Scanning target {byellow}{target.address}{rst}.')
@@ -574,8 +596,6 @@ def scan_host(target, concurrent_scans):
     reportdir = os.path.abspath(os.path.join(basedir, 'report'))
     target.reportdir = reportdir
     os.makedirs(reportdir, exist_ok=True)
-    f = open(os.path.join(reportdir, 'notes.txt'), 'w')
-    f.close()
 
     screenshotdir = os.path.abspath(os.path.join(reportdir, 'screenshots'))
     os.makedirs(screenshotdir, exist_ok=True)
@@ -583,6 +603,7 @@ def scan_host(target, concurrent_scans):
     scandir = os.path.abspath(os.path.join(basedir, 'scans'))
     target.scandir = scandir
     os.makedirs(scandir, exist_ok=True)
+    prepare_log_files(scandir, target)
 
     os.makedirs(os.path.abspath(os.path.join(scandir, 'xml')), exist_ok=True)
 
@@ -604,10 +625,18 @@ def scan_host(target, concurrent_scans):
     except KeyboardInterrupt:
         sys.exit(1)
 
+def prepare_log_files(scandir, target):
 
-''' Reads a list of targets from a file
+    for filename in files:
+        try:
+            caption = 'Log session started for host {0} - {1}\n'.format(target.address, datetime.now().strftime('%B %d, %Y - %H:%M:%S'))
+            with open(os.path.join(scandir, files[filename]), 'a') as f:
+                f.write('\n{}\n'.format('=' * len(caption)))
+                f.write(caption)
+                f.write('{}\n\n'.format('=' * len(caption)))
+        except OSError:
+            fail('Error while setting up log file {filename}.')
 
-'''
 def read_targets_from_file(filename, targets, disable_sanity_checks):
 
     if not os.path.isfile(filename):
@@ -621,7 +650,6 @@ def read_targets_from_file(filename, targets, disable_sanity_checks):
         error('The file {filename} with target information could not be read.')
         return (targets, True)
 
-
     error = False
     for ip in entries.split('\n'):
         if ip.startswith('#') or len(ip) == 0: continue
@@ -631,7 +659,6 @@ def read_targets_from_file(filename, targets, disable_sanity_checks):
     
     return (targets, error)
 
-
 def get_ip_address(target, targets, disable_sanity_checks):
 
     errors = False
@@ -661,7 +688,6 @@ def get_ip_address(target, targets, disable_sanity_checks):
 
     return (targets, errors)
 
-
 class Target:
     def __init__(self, address):
         self.address = address
@@ -682,13 +708,18 @@ if __name__ == '__main__':
     nmap_group = parser.add_mutually_exclusive_group()
     nmap_group.add_argument('--nmap', action='store', default=nmap_default_options, help='Override the {nmap_extra} variable in scans. Default: %(default)s')
     nmap_group.add_argument('--nmap-append', action='store', default='', help='Append to the default {nmap_extra} variable in scans.')
+    parser.add_argument('--skip-service-scan', action='store_true', default=False, help='Do not perfom extended service scanning but only document commands.')
+    parser.add_argument('--run-level', action='store', type=int, default=0, nargs="+", help='During extended service scanning, only run scanners of a certain complexity level or below.')
+    parser.add_argument('--run-only', action='store_true', default=False, help='If enabled, only run scanners of the specified complexity level during extended service scanning.')
     parser.add_argument('-r', '--read', action='store', type=str, default='', dest='target_file', help='Read targets from file.')
     parser.add_argument('-v', '--verbose', action='count', default=0, help='Enable verbose output. Repeat for more verbosity.')
     parser.add_argument('--disable-sanity-checks', action='store_true', default=False, help='Disable sanity checks that would otherwise prevent the scans from running.')
-    parser.add_argument('--skip-service-scan', action='store_true', default=False, help='Do not perfom extended service scanning but only protocol commands.')
     parser.error = lambda s: fail(s[0].upper() + s[1:])
     args = parser.parse_args()
 
+    if not os.getuid() == 0:
+        warn('Warning: You are not running the program with superuser privileges. Service scanning may be impacted.')
+
     config_loaded = get_configuration()
     if not config_loaded: sys.exit(-1)
 
@@ -704,6 +735,10 @@ if __name__ == '__main__':
         error('Argument -ct/--concurrent-scans: must be at least 1.')
         errors = True
 
+    if min(args.run_level) < 0 or max(args.run_level) > max_level:
+        error('Argument --run-level: must be between 0 (default) and {}.'.format(max_level))
+        errors = True
+
     port_scan_profile = args.profile
 
     found_scan_profile = False

config/config.toml

@@ -3,12 +3,15 @@
 
     nmap = '/usr/bin/nmap'
 
+    # system programs
+    tee = '/usr/bin/tee'
+
     # password brute-forcing programs
     hydra = '/usr/bin/hydra'
     medusa = '/usr/bin/medusa'
     patator = '/usr/bin/patator'
 
-    # programs for web servers and application
+    # programs for web servers and applications
     sslscan = '/usr/bin/sslscan'
     curl = '/usr/bin/curl'
     dirb = '/usr/bin/dirb'
@@ -19,14 +22,22 @@
     wpscan = '/usr/local/bin/wpscan'
     dirsearch = ''
     
-    # Oracle programs
+    # WebDAV programs
+    davtest = '/usr/bin/davtest'
+    cadaver = '/usr/bin/cadaver'
+
+    # database programs
     tnscmd10g = '/usr/bin/tnscmd10g'
     oscanner = '/usr/bin/oscanner'
     odat = '/usr/bin/python odat.py'
+    sqsh = '/usr/bin/sqsh'
 
     # RPC programs
     rpcclient = '/usr/bin/rpcclient'
 
+    # SIP programs
+    svwar = '/usr/bin/svwar'
+
     # SMB programs
     enum4linux = '/usr/bin/enum4linux'
     smbclient = '/usr/bin/smbclient'
@@ -34,7 +45,7 @@
     smb_version = '/usr/local/bin/get_smb_version'
 
     # SMTP programs
-    smtp-user-enum = '/usr/bin/smtp-user-enum'
+    smtp_user_enum = '/usr/bin/smtp-user-enum'
 
     # SNMP programs
     onesixtyone = '/usr/bin/onesixtyone'

config/port-scan-profiles.toml

@@ -50,7 +50,7 @@
     [debug.nmap-quick]
 
         [debug.nmap-quick.service-detection]
-        command = '{nmap} {nmap_extra} --top-ports 10 -oN "{scandir}/_quick_tcp_nmap.txt" -oX "{scandir}/xml/_quick_tcp_nmap.xml" {address}'
+        command = '{nmap} {nmap_extra} -p 21,22,80,139,445,8000,8080 -oN "{scandir}/_quick_tcp_nmap.txt" -oX "{scandir}/xml/_quick_tcp_nmap.xml" {address}'
         pattern = '^(?P<port>\d+)\/(?P<protocol>(tcp|udp))(.*)open(\s*)(?P<service>[\w\-\/]+)(\s*)(.*)$'
 
 

config/service-scans.toml

@@ -10,7 +10,7 @@ service-names = [
 
     [[all-services.scan]]
     name = 'sslscan'
-    command = 'if [ "{secure}" == "True" ]; then {sslscan} --show-certificate --no-colour {address}:{port} 2>&1 | tee "{scandir}/{protocol}_{port}_sslscan.txt"; fi'
+    command = 'if [ "{secure}" == "True" ]; then {sslscan} --show-certificate --no-colour {address}:{port} 2>&1 | {tee} "{scandir}/{protocol}_{port}_sslscan.txt"; fi'
 
 [cassandra]
 
@@ -74,7 +74,7 @@ service-names = [
     command = '{nmap} {nmap_extra} -sV -p {port} --script="(ftp* or ssl*) and not (brute or broadcast or dos or external or fuzzer)" -oN "{scandir}/{category}{protocol}_{port}_ftp_nmap.txt" -oX "{scandir}/xml/{category}{protocol}_{port}_ftp_nmap.xml" {address}'
 
         [[ftp.scan.pattern]]
-        description = 'Anonymous FTP Enabled!'
+        description = 'Anonymous FTP enabled!'
         pattern = 'Anonymous FTP login allowed'
 
     [[ftp.manual]]
@@ -91,6 +91,7 @@ service-names = [
 ]
 
 ignore-service-names = [
+    # Micrsoft Windows RPC over HTTP (IIS)
     '^nacn_http$'
 ]
 
@@ -99,23 +100,23 @@ ignore-service-names = [
     command = '{nmap} {nmap_extra} -sV -p {port} --script="(http* or ssl*) and not (broadcast or dos or external or http-slowloris* or fuzzer)" -oN "{scandir}/{category}{protocol}_{port}_http_nmap.txt" -oX "{scandir}/xml/{category}{protocol}_{port}_{scheme}_nmap.xml" {address}'
 
         [[http.scan.pattern]]
-        description = 'Identified HTTP Server: {match}'
+        description = 'HTTP server identified: {match}'
         pattern = 'Server: ([^\n]+)'
 
         [[http.scan.pattern]]
-        description = 'WebDAV is enabled'
+        description = 'WebDAV is enabled.'
         pattern = 'WebDAV is ENABLED'
          
     [[http.scan]]
     name = 'curl-index'
-    command = 'curl -sSik {scheme}://{address}:{port}/ -m 10 2>&1 | tee "{scandir}/{category}{protocol}_{port}_{scheme}_index.html"'
+    command = '{curl} -sSik {scheme}://{address}:{port}/ -m 10 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_{scheme}_index.html"'
 
         [[http.scan.pattern]]
         pattern = '(?i)Powered by [^\n]+'
 
     [[http.scan]]
     name = 'curl-robots'
-    command = 'curl -sSik {scheme}://{address}:{port}/robots.txt -m 10 2>&1 | tee "{scandir}/{category}{protocol}_{port}_{scheme}_robots.txt"'
+    command = '{curl} -sSik {scheme}://{address}:{port}/robots.txt -m 10 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_{scheme}_robots.txt"'
 
     [[http.scan]]
     name = 'wkhtmltoimage'
@@ -123,41 +124,50 @@ ignore-service-names = [
 
     [[http.scan]]
     name = 'whatweb'
-    command = '{whatweb} --color=never --no-errors -a 3 -v {scheme}://{address}:{port} 2>&1 | tee "{scandir}/{category}{protocol}_{port}_{scheme}_whatweb.txt"'
+    command = '{whatweb} --color=never --no-errors -a 3 -v {scheme}://{address}:{port} 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_{scheme}_whatweb.txt"'
-
-    [[http.scan]]
-    name = 'nikto'
-    command = '{nikto} -ask=no -h {scheme}://{address}:{port} 2>&1 | tee "{scandir}/{category}{protocol}_{port}_{scheme}_nikto.txt"'
 
     [[http.scan]]
     name = 'gobuster'
-    command = '{gobuster} -u {scheme}://{address}:{port}/ -w /usr/share/seclists/Discovery/Web-Content/common.txt -e -k -l -s "200,204,301,302,307,403" -x "txt,html,php,asp,aspx" 2>&1 | tee "{scandir}/{category}{protocol}_{port}_{scheme}_gobuster.txt"'
+    command = '{gobuster} -u {scheme}://{address}:{port}/ -w /usr/share/seclists/Discovery/Web-Content/common.txt -e -k -l -s "200,204,301,302,307,403" -x "txt,html,htm,php,pl,asp,aspx" 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_{scheme}_gobuster.txt"'
+    level = 2
+
+    [[http.scan]]
+    name = 'nikto'
+    command = '{nikto} -ask=no -h {scheme}://{address}:{port} 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_{scheme}_nikto.txt"'
+    level = 3
 
     [[http.manual]]
     description = '(dirsearch) Multi-threaded recursive directory/file enumeration for web servers using various wordlists:'
     commands = [
-        'dirsearch -u {scheme}://{address}:{port}/ -t 16 -r -e txt,html,php,asp,aspx -f -w /usr/share/seclists/Discovery/Web-Content/big.txt --plain-text-report="{scandir}/{category}{protocol}_{port}_{scheme}_dirsearch_big.txt"',
+        '{dirsearch} -u {scheme}://{address}:{port}/ -t 16 -r -e txt,html,php,asp,aspx -f -w /usr/share/seclists/Discovery/Web-Content/big.txt --plain-text-report="{scandir}/{category}{protocol}_{port}_{scheme}_dirsearch_big.txt"',
-        'dirsearch -u {scheme}://{address}:{port}/ -t 16 -r -e txt,html,php,asp,aspx -f -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt --plain-text-report="{scandir}/{category}{protocol}_{port}_{scheme}_dirsearch_dirbuster.txt"'
+        '{dirsearch} -u {scheme}://{address}:{port}/ -t 16 -r -e txt,html,php,asp,aspx -f -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt --plain-text-report="{scandir}/{category}{protocol}_{port}_{scheme}_dirsearch_dirbuster.txt"'
     ]
 
     [[http.manual]]
     description = '(dirb) Recursive directory/file enumeration for web servers using various wordlists (same as dirsearch above):'
     commands = [
-        'dirb {scheme}://{address}:{port}/ -o "{scandir}/{category}{protocol}_{port}_{scheme}_dirb_big.txt" /usr/share/seclists/Discovery/Web-Content/big.txt',
+        '{dirb} {scheme}://{address}:{port}/ -o "{scandir}/{category}{protocol}_{port}_{scheme}_dirb_big.txt" /usr/share/seclists/Discovery/Web-Content/big.txt',
-        'dirb {scheme}://{address}:{port}/ -o "{scandir}/{category}{protocol}_{port}_{scheme}_dirb_dirbuster.txt" /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt'
+        '{dirb} {scheme}://{address}:{port}/ -o "{scandir}/{category}{protocol}_{port}_{scheme}_dirb_dirbuster.txt" /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt'
     ]
 
     [[http.manual]]
     description = '(gobuster) Directory/file enumeration for web servers using various wordlists (same as dirb above):'
     commands = [
-        '{gobuster} -u {scheme}://{address}:{port}/ -w /usr/share/seclists/Discovery/Web-Content/big.txt -e -k -l -s "200,204,301,302,307,403,500" -x "txt,html,php,asp,aspx" -o "{scandir}/{category}{protocol}_{port}_{scheme}_gobuster_big.txt"',
+        '{gobuster} -u {scheme}://{address}:{port}/ -w /usr/share/seclists/Discovery/Web-Content/big.txt -e -k -l -s "200,204,301,302,307,403,500" -x "txt,html,htm,php,pl,asp,aspx" -o "{scandir}/{category}{protocol}_{port}_{scheme}_gobuster_big.txt"',
-        '{gobuster} -u {scheme}://{address}:{port}/ -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt -e -k -l -s "200,204,301,302,307,403,500" -x "txt,html,php,asp,aspx" -o "{scandir}/{category}{protocol}_{port}_{scheme}_gobuster_dirbuster.txt"'
+        '{gobuster} -u {scheme}://{address}:{port}/ -w /usr/share/wordlists/dirbuster/directory-list-2.3-medium.txt -e -k -l -s "200,204,301,302,307,403,500" -x "txt,html,htm,php,pl,asp,aspx" -o "{scandir}/{category}{protocol}_{port}_{scheme}_gobuster_dirbuster.txt"'
     ]
 
     [[http.manual]]
     description = '(wpscan) WordPress Security Scanner (useful if WordPress is found):'
     commands = [
-        '{wpscan} --url {scheme}://{address}:{port}/ --no-update -e vp,vt,tt,cb,dbe,u,m -f cli-no-color 2>&1 | tee "{scandir}/{category}{protocol}_{port}_{scheme}_wpscan.txt"'
+        '{wpscan} --url {scheme}://{address}:{port}/ --no-update -e vp,vt,tt,cb,dbe,u,m -f cli-no-color 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_{scheme}_wpscan.txt"'
+    ]
+    
+    [[http.manual]]
+    description = '(cadaver) WebDAV command-line client (useful if WebDAV is enabled):'
+    commands = [
+        '{davtest} -cleanup -nocreate -url {scheme}://{address}:{port}/ 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_{scheme}_davtest.txt"',
+        '{cadaver} {scheme}://{address}:{port}/'
     ]
 
     [[http.manual]]
@@ -202,7 +212,7 @@ service-names = [
 
     [[ldap.scan]]
     name = 'enum4linux'
-    command = '{enum4linux} -a -M -l -d {address} 2>&1 | tee "{scandir}/{category}enum4linux.txt"'
+    command = '{enum4linux} -a -M -l -d {address} 2>&1 | {tee} "{scandir}/{category}enum4linux.txt"'
     run_once = true
     ports.tcp = [139, 389, 445]
     ports.udp = [137]
@@ -231,7 +241,7 @@ service-names = [
     [[mssql.manual]]
     description = '(sqsh) interactive database shell'
     commands = [
-        'sqsh -U <username> -P <password> -S {address}:{port}'
+        '{sqsh} -U <username> -P <password> -S {address}:{port}'
     ]
 
 [mysql]
@@ -277,19 +287,19 @@ service-names = [
 
     [[oracle.scan]]
     name = 'oracle-tnscmd-ping'
-    command = 'tnscmd10g ping -h {address} -p {port} 2>&1 | tee "{scandir}/{category}{protocol}_{port}_oracle_tnscmd_ping.txt"'
+    command = '{tnscmd10g} ping -h {address} -p {port} 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_oracle_tnscmd_ping.txt"'
 
     [[oracle.scan]]
     name = 'oracle-tnscmd-version'
-    command = 'tnscmd10g version -h {address} -p {port} 2>&1 | tee "{scandir}/{category}{protocol}_{port}_oracle_tnscmd_version.txt"'
+    command = '{tnscmd10g} version -h {address} -p {port} 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_oracle_tnscmd_version.txt"'
 
     [[oracle.scan]]
     name = 'oracle-tnscmd-version'
-    command = 'tnscmd10g version -h {address} -p {port} 2>&1 | tee "{scandir}/{category}{protocol}_{port}_oracle_tnscmd_version.txt"'
+    command = '{tnscmd10g} version -h {address} -p {port} 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_oracle_tnscmd_version.txt"'
 
     [[oracle.scan]]
     name = 'oracle-scanner'
-    command = 'oscanner -v -s {address} -P {port} 2>&1 | tee "{scandir}/{category}{protocol}_{port}_oracle_scanner.txt"'
+    command = '{oscanner} -v -s {address} -P {port} 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_oracle_scanner.txt"'
 
     [[oracle.manual]]
     description = 'Brute-force SIDs using Nmap'
@@ -382,7 +392,7 @@ service-names = [
 
     [[sip.scan]]
     name = 'svwar'
-    command = 'svwar -D -m INVITE -p {port} {address}'
+    command = '{svwar} -D -m INVITE -p {port} {address}'
 
 [ssh]
 
@@ -414,38 +424,38 @@ service-names = [
 
     [[smb.scan]]
     name = 'smbclient'
-    command = '{smbclient} -L\\ -N -I {address} 2>&1 | tee "{scandir}/{category}smbclient.txt"'
+    command = '{smbclient} -L\\ -N -I {address} 2>&1 | {tee} "{scandir}/{category}smbclient.txt"'
     run_once = true
     ports.tcp = [139, 445]
     
     [[smb.scan]]
     name = 'smb-version'
-    command = '{smb_version} {address} -P {port} | head -n 1 | tee -a "{scandir}/{category}smb-version.txt"'
+    command = '{smb_version} {address} -P {port} | head -n 1 | {tee} -a "{scandir}/{category}smb-version.txt"'
     run_once = true
     ports.tcp = [139, 445]
 
     [[smb.scan]]
     name = 'smbmap-share-permissions'
-    command = '{smbmap} -H {address} -P {port} 2>&1 | tee -a "{scandir}/{category}smbmap-share-permissions.txt"; {smbmap} -u null -p "" -H {address} -P {port} 2>&1 | tee -a "{scandir}/{category}smbmap-share-permissions.txt"'
+    command = '{smbmap} -H {address} -P {port} 2>&1 | {tee} -a "{scandir}/{category}smbmap-share-permissions.txt"; {smbmap} -u null -p "" -H {address} -P {port} 2>&1 | {tee} -a "{scandir}/{category}smbmap-share-permissions.txt"'
 
     [[smb.scan]]
     name = 'smbmap-list-contents'
-    command = '{smbmap} -H {address} -P {port} -R 2>&1 | tee -a "{scandir}/{category}smbmap-list-contents.txt"; {smbmap} -u null -p "" -H {address} -P {port} -R 2>&1 | tee -a "{scandir}/{category}smbmap-list-contents.txt"'
+    command = '{smbmap} -H {address} -P {port} -R 2>&1 | {tee} -a "{scandir}/{category}smbmap-list-contents.txt"; {smbmap} -u null -p "" -H {address} -P {port} -R 2>&1 | {tee} -a "{scandir}/{category}smbmap-list-contents.txt"'
 
     [[smb.scan]]
     name = 'smbmap-execute-command'
-    command = '{smbmap} -H {address} -P {port} -x "ipconfig /all" 2>&1 | tee -a "{scandir}/{category}smbmap-execute-command.txt"; {smbmap} -u null -p "" -H {address} -P {port} -x "ipconfig /all" 2>&1 | tee -a "{scandir}/{category}smbmap-execute-command.txt"'
+    command = '{smbmap} -H {address} -P {port} -x "ipconfig /all" 2>&1 | {tee} -a "{scandir}/{category}smbmap-execute-command.txt"; {smbmap} -u null -p "" -H {address} -P {port} -x "ipconfig /all" 2>&1 | {tee} -a "{scandir}/{category}smbmap-execute-command.txt"'
    
     [[smb.scan]]
     name = 'enum4linux'
-    command = '{enum4linux} -a -M -l -d {address} 2>&1 | tee "{scandir}/{category}enum4linux.txt"'
+    command = '{enum4linux} -a -M -l -d {address} 2>&1 | {tee} "{scandir}/{category}enum4linux.txt"'
     run_once = true
     ports.tcp = [139, 389, 445]
     ports.udp = [137]
 
     [[smb.scan]]
     name = 'nbtscan'
-    command = '{nbtscan} -rvh {address} 2>&1 | tee "{scandir}/{category}nbtscan.txt"'
+    command = '{nbtscan} -rvh {address} 2>&1 | {tee} "{scandir}/{category}nbtscan.txt"'
     run_once = true
     ports.udp = [137]
 
@@ -470,7 +480,7 @@ service-names = [
 
     [[smtp.scan]]
     name = 'smtp-user-enum'
-    command = 'smtp-user-enum -M VRFY -U "{username_wordlist}" -t {address} -p {port} 2>&1 | tee "{scandir}/{category}{protocol}_{port}_smtp_user-enum.txt"'
+    command = '{smtp_user_enum} -M VRFY -U "{username_wordlist}" -t {address} -p {port} 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_smtp_user-enum.txt"'
 
 [snmp]
 
@@ -484,55 +494,55 @@ service-names = [
 
     [[snmp.scan]]
     name = 'onesixtyone'
-    command = 'onesixtyone -c /usr/share/seclists/Discovery/SNMP/common-snmp-community-strings_onesixtyone.txt -dd {address} 2>&1 | tee "{scandir}/{category}{protocol}_{port}_snmp_onesixtyone.txt"'
+    command = '{onesixtyone} -c /usr/share/seclists/Discovery/SNMP/common-snmp-community-strings_onesixtyone.txt -dd {address} 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_snmp_onesixtyone.txt"'
     run_once = true
     ports.udp = [161]
 
     [[snmp.scan]]
     name = 'snmpwalk'
-    command = '{snmpwalk} -c public -v 1 {address} 2>&1 | tee "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk.txt"'
+    command = '{snmpwalk} -c public -v 1 {address} 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk.txt"'
     run_once = true
     ports.udp = [161]
 
     [[snmp.scan]]
     name = 'snmpwalk-system-processes'
-    command = '{snmpwalk} -c public -v 1 {address} 1.3.6.1.2.1.25.1.6.0 2>&1 | tee "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk_system_processes.txt"'
+    command = '{snmpwalk} -c public -v 1 {address} 1.3.6.1.2.1.25.1.6.0 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk_system_processes.txt"'
     run_once = true
     ports.udp = [161]
 
     [[snmp.scan]]
     name = 'snmpwalk-running-processes'
-    command = '{snmpwalk} -c public -v 1 {address} 1.3.6.1.2.1.25.4.2.1.2 2>&1 | tee "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk_running_processes.txt"'
+    command = '{snmpwalk} -c public -v 1 {address} 1.3.6.1.2.1.25.4.2.1.2 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk_running_processes.txt"'
     run_once = true
     ports.udp = [161]
 
     [[snmp.scan]]
     name = 'snmpwalk-process-paths'
-    command = '{snmpwalk} -c public -v 1 {address} 1.3.6.1.2.1.25.4.2.1.4 2>&1 | tee "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk_process_paths.txt"'
+    command = '{snmpwalk} -c public -v 1 {address} 1.3.6.1.2.1.25.4.2.1.4 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk_process_paths.txt"'
     run_once = true
     ports.udp = [161]
 
     [[snmp.scan]]
     name = 'snmpwalk-storage-units'
-    command = '{snmpwalk} -c public -v 1 {address} 1.3.6.1.2.1.25.2.3.1.4 2>&1 | tee "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk_storage_units.txt"'
+    command = '{snmpwalk} -c public -v 1 {address} 1.3.6.1.2.1.25.2.3.1.4 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk_storage_units.txt"'
     run_once = true
     ports.udp = [161]
 
     [[snmp.scan]]
     name = 'snmpwalk-software-names'
-    command = '{snmpwalk} -c public -v 1 {address} 1.3.6.1.2.1.25.6.3.1.2 2>&1 | tee "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk_software_names.txt"'
+    command = '{snmpwalk} -c public -v 1 {address} 1.3.6.1.2.1.25.6.3.1.2 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk_software_names.txt"'
     run_once = true
     ports.udp = [161]
 
     [[snmp.scan]]
     name = 'snmpwalk-user-accounts'
-    command = '{snmpwalk} -c public -v 1 {address} 1.3.6.1.4.1.77.1.2.25 2>&1 | tee "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk_user_accounts.txt"'
+    command = '{snmpwalk} -c public -v 1 {address} 1.3.6.1.4.1.77.1.2.25 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk_user_accounts.txt"'
     run_once = true
     ports.udp = [161]
 
     [[snmp.scan]]
     name = 'snmpwalk-tcp-ports'
-    command = '{snmpwalk} -c public -v 1 {address} 1.3.6.1.2.1.6.13.1.3 2>&1 | tee "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk_tcp_ports.txt"'
+    command = '{snmpwalk} -c public -v 1 {address} 1.3.6.1.2.1.6.13.1.3 2>&1 | {tee} "{scandir}/{category}{protocol}_{port}_snmp_snmpwalk_tcp_ports.txt"'
     run_once = true
     ports.udp = [161]