Added packaging constructs to allow installation using `pip` or `pipx` (#48)

* Restructured program to enable packaging. Moved main into its own function and resolved global dependency issues. Configured pyproject.toml

* Restructured configuration management such that configuration files are stored in the default user configuration directory for the OS they're using, utilising appdirs

* Added exclusion for byte compiled python files

* Removed build files from version control

* Restructured config file initialisation into a function so that it can be called by the entrypoint main() function

* Resolved issues updating global variables from within function scope. Added global identifiers to global variables to make them more readily identifiable as such

* Formatting revision

* Updated documentation to reflect package installation instructions and usage

* Grammatical revision to improve instructional clarity

* Added default sudo for udp scans to prompt for password to elevate privileges

* Added an installation command for all required packages to the README

* Added manual installation instructions, clarification of oscp vm limitations installing pip3, and instructions on how to elevate privileges when installed with pipx

* Formatting revision to improve clarity of instructions and added requirements.txt

* Removed hardcoded sudo commands from default scan configurations

* Changed suggestion to add alias to .bashrc to .profile as this is what's used by default in kali to set initial temrinal conditions

* Fixed error in manual installation that would have resulted in installing python2 packages
This commit is contained in:
Samuel Dowling 2020-05-01 12:18:23 +09:30 committed by GitHub
parent d5b3122d8f
commit 0b03f7b801
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
11 changed files with 408 additions and 64 deletions

139
.gitignore vendored Normal file
View File

@ -0,0 +1,139 @@
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class
*.pyc
# C extensions
*.so
# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
.installed.cfg
*.egg
MANIFEST
# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec
# Installer logs
pip-log.txt
pip-delete-this-directory.txt
# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
# PyBuilder
.pybuilder/
target/
# Jupyter Notebook
.ipynb_checkpoints
# IPython
profile_default/
ipython_config.py
# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
dmypy.json
# Pyre type checker
.pyre/
# pytype static type analyzer
.pytype/
# Cython debug symbols
cython_debug/

102
README.md
View File

@ -26,24 +26,66 @@ AutoRecon was inspired by three tools which the author used during the OSCP labs
## Requirements
* Python 3
* colorama
* toml
- Python 3
- `python3-pip`
- `pipx` (optional, but recommended)
Once Python 3 is installed, pip3 can be used to install the other requirements:
### Python 3
If you don't have these installed, and are running Kali Linux, you can execute the following:
```bash
$ pip3 install -r requirements.txt
$ sudo apt install python3
$ sudo apt install python3-pip
```
Several people have indicated that installing pip3 via apt on the OSCP Kali version makes the host unstable. In these cases, pip3 can be installed by running the following commands:
Additionally, if you experience any issues with the stability of the `python3-pip` installation (as reported by a number of people installing `pip3` via `apt` on the OSCP distribution of Kali), you can install it manually as follows:
```bash
$ curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
$ python3 get-pip.py
```
The "pip3" command should now be usable.
The `pip3` command should now be usable.
### `pipx`
Further, it's recommended you use `pipx` to manage your python packages; this installs each python package in it's own virtualenv, and makes it available in the global context, which avoids conflicting package dependencies and the resulting instability. To summarise the installation instructions:
```bash
$ python3 -m pip install --user pipx
$ python3 -m pipx ensurepath
```
Note that if you want to elevate privileges to run a program installed with `pipx`, with `sudo`, you have two options:
1. Append the appropriate path to your execution command, using _one_ of the following examples (recommended):
```bash
$ sudo env "PATH=$PATH" autorecon [OPTIONS]
$ sudo $(which autorecon) [OPTIONS]
```
To make this easier, you could add the following alias to your `~/.profile` (or equivalent):
```
alias sudo="sudo env \"PATH=$PATH\""
```
2. Add the `pipx` binary path to the `secure_path` set in `/etc/sudoers`
```bash
sudo visudo /etc/sudoers
```
Update the `secure_path` directive as follows:
```
Defaults secure_path="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/kali/.local/bin"
```
If you're not using Kali Linux, make sure to adjust the path to the relevant user. Further detail on the installation of `pipx` is available in their installation instructions available [here](https://pipxproject.github.io/pipx/installation/). Please refer to this for any issues you experience.
### Supporting packages
Several commands used in AutoRecon reference the SecLists project, in the directory /usr/share/seclists/. You can either manually download the SecLists project to this directory (https://github.com/danielmiessler/SecLists), or if you are using Kali Linux (**highly recommended**) you can run the following:
@ -75,12 +117,50 @@ whatweb
wkhtmltoimage
```
On Kali Linux, you can ensure these are all installed using the following command:
```bash
$ sudo apt install curl enum4linux gobuster nbtscan nikto nmap onesixtyone oscanner smbclient smbmap smtp-user-enum snmp sslscan sipvicious tnscmd10g whatweb wkhtmltopdf
```
## Installation
Ensure you have all of the requirements installed as per the previous section.
### Using `pipx` (recommended)
```bash
$ pipx install git+https://github.com/Tib3rius/AutoRecon.git
```
### Using `pip`
```bash
$ python3 -m pip install git+https://github.com/Tib3rius/AutoRecon.git
```
### Manual
If you'd prefer not to use `pip` or `pipx`, you can always still install and execute `autorecon.py` manually as a script. First install the dependencies:
```bash
$ python3 -m pip install -r requirements.txt
```
You will then be able to run the `autorecon.py` script (from `<AUTORECON_ROOT_DIR>/src/autorecon`):
```bash
$ python3 autorecon.py [OPTIONS] 127.0.0.1
```
See detailed usage options below.
## Usage
AutoRecon uses Python 3 specific functionality and does not support Python 2.
```
usage: autorecon.py [-h] [-t TARGET_FILE] [-ct <number>] [-cs <number>]
usage: autorecon [-h] [-t TARGET_FILE] [-ct <number>] [-cs <number>]
[--profile PROFILE_NAME] [-o OUTPUT_DIR] [--single-target]
[--only-scans-dir] [--heartbeat HEARTBEAT]
[--nmap NMAP | --nmap-append NMAP_APPEND] [-v]
@ -135,7 +215,7 @@ optional arguments:
**Scanning a single target:**
```
python3 autorecon.py 127.0.0.1
$ autorecon 127.0.0.1
[*] Scanning target 127.0.0.1
[*] Running service detection nmap-full-tcp on 127.0.0.1
[*] Running service detection nmap-top-20-udp on 127.0.0.1
@ -183,7 +263,7 @@ Note that the actual command line output will be colorized if your terminal supp
**Scanning multiple targets**
```
python3 autorecon.py 192.168.1.100 192.168.1.1/30 localhost
$ autorecon 192.168.1.100 192.168.1.1/30 localhost
[*] Scanning target 192.168.1.100
[*] Scanning target 192.168.1.1
[*] Scanning target 192.168.1.2
@ -208,7 +288,7 @@ AutoRecon supports multiple targets per scan, and will expand IP ranges provided
**Scanning multiple targets with advanced options**
```
python3 autorecon.py -ct 2 -cs 2 -vv -o outputdir 192.168.1.100 192.168.1.1/30 localhost
$ autorecon -ct 2 -cs 2 -vv -o outputdir 192.168.1.100 192.168.1.1/30 localhost
[*] Scanning target 192.168.1.100
[*] Scanning target 192.168.1.1
[*] Running service detection nmap-quick on 192.168.1.100 with nmap -vv --reason -Pn -sV -sC --version-all -oN "/root/outputdir/192.168.1.100/scans/_quick_tcp_nmap.txt" -oX "/root/outputdir/192.168.1.100/scans/_quick_tcp_nmap.xml" 192.168.1.100

42
poetry.lock generated Normal file
View File

@ -0,0 +1,42 @@
[[package]]
category = "main"
description = "A small Python module for determining appropriate platform-specific dirs, e.g. a \"user data dir\"."
name = "appdirs"
optional = false
python-versions = "*"
version = "1.4.3"
[[package]]
category = "main"
description = "Cross-platform colored terminal text."
name = "colorama"
optional = false
python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*"
version = "0.4.3"
[[package]]
category = "main"
description = "Python Library for Tom's Obvious, Minimal Language"
name = "toml"
optional = false
python-versions = "*"
version = "0.10.0"
[metadata]
content-hash = "7359d313673568959d8011a312ea74ee6c3c3f3a2a5d81f690b87f6236d051ec"
python-versions = "^3.7"
[metadata.files]
appdirs = [
{file = "appdirs-1.4.3-py2.py3-none-any.whl", hash = "sha256:d8b24664561d0d34ddfaec54636d502d7cea6e29c3eaf68f3df6180863e2166e"},
{file = "appdirs-1.4.3.tar.gz", hash = "sha256:9e5896d1372858f8dd3344faf4e5014d21849c756c8d5701f78f8a103b372d92"},
]
colorama = [
{file = "colorama-0.4.3-py2.py3-none-any.whl", hash = "sha256:7d73d2a99753107a36ac6b455ee49046802e59d9d076ef8e47b61499fa29afff"},
{file = "colorama-0.4.3.tar.gz", hash = "sha256:e96da0d330793e2cb9485e9ddfd918d456036c7149416295932478192f4436a1"},
]
toml = [
{file = "toml-0.10.0-py2.7.egg", hash = "sha256:f1db651f9657708513243e61e6cc67d101a39bad662eaa9b5546f789338e07a3"},
{file = "toml-0.10.0-py2.py3-none-any.whl", hash = "sha256:235682dd292d5899d361a811df37e04a8828a5b1da3115886b73cf81ebc9100e"},
{file = "toml-0.10.0.tar.gz", hash = "sha256:229f81c57791a41d65e399fc06bf0848bab550a9dfd5ed66df18ce5f05e73d5c"},
]

2
poetry.toml Normal file
View File

@ -0,0 +1,2 @@
[virtualenvs]
create = true

28
pyproject.toml Normal file
View File

@ -0,0 +1,28 @@
[tool.poetry]
name = "autorecon"
version = "1.0.0"
description = "A multi-threaded network reconaissance tool which performs automated enumeration of services"
authors = ["Tib3rius"]
license = "GPL-3.0-only"
readme = "README.md"
homepage = "https://github.com/Tib3rius/AutoRecon"
repository = "https://github.com/Tib3rius/AutoRecon"
packages = [
{include = "autorecon", from = "src"},
]
[tool.poetry.dependencies]
python = "^3.7"
toml = "^0.10.0"
colorama = "^0.4.3"
appdirs = "^1.4.3"
[tool.poetry.dev-dependencies]
[tool.poetry.scripts]
autorecon = 'autorecon.autorecon:main'
[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"

View File

@ -1,2 +1,3 @@
colorama
toml
appdirs

View File

View File

@ -23,32 +23,103 @@ import sys
import time
import toml
import termios
import appdirs
import shutil
def _quit():
termios.tcsetattr(sys.stdin.fileno(), termios.TCSADRAIN, TERM_FLAGS)
atexit.register(_quit)
TERM_FLAGS = termios.tcgetattr(sys.stdin.fileno())
# Globals
verbose = 0
nmap = '-vv --reason -Pn'
srvname = ''
nmap = "-vv --reason -Pn"
srvname = ""
heartbeat_interval = 60
port_scan_profile = None
port_scan_profiles_config = None
service_scans_config = None
global_patterns = []
username_wordlist = '/usr/share/seclists/Usernames/top-usernames-shortlist.txt'
password_wordlist = '/usr/share/seclists/Passwords/darkweb2017-top100.txt'
rootdir = os.path.dirname(os.path.realpath(__file__))
username_wordlist = "/usr/share/seclists/Usernames/top-usernames-shortlist.txt"
password_wordlist = "/usr/share/seclists/Passwords/darkweb2017-top100.txt"
single_target = False
only_scans_dir = False
def _quit():
TERM_FLAGS = termios.tcgetattr(sys.stdin.fileno())
termios.tcsetattr(sys.stdin.fileno(), termios.TCSADRAIN, TERM_FLAGS)
def _init():
global port_scan_profiles_config
global service_scans_config
global global_patterns
atexit.register(_quit)
appname = "AutoRecon"
rootdir = os.path.dirname(os.path.realpath(__file__))
default_config_dir = os.path.join(rootdir, "config")
config_dir = appdirs.user_config_dir(appname)
port_scan_profiles_config_file = os.path.join(config_dir, "port-scan-profiles.toml")
service_scans_config_file = os.path.join(config_dir, "service-scans.toml")
global_patterns_config_file = os.path.join(config_dir, "global-patterns.toml")
# Confirm this directory exists; if not, populate it with the default configurations
if not os.path.exists(config_dir):
os.makedirs(config_dir, exist_ok=True)
shutil.copy(
os.path.join(default_config_dir, "port-scan-profiles-default.toml"),
port_scan_profiles_config_file,
)
shutil.copy(
os.path.join(default_config_dir, "service-scans-default.toml"),
service_scans_config_file,
)
shutil.copy(
os.path.join(default_config_dir, "global-patterns-default.toml"),
global_patterns_config_file,
)
with open(port_scan_profiles_config_file, "r") as p:
try:
port_scan_profiles_config = toml.load(p)
if len(port_scan_profiles_config) == 0:
fail(
"There do not appear to be any port scan profiles configured in the {port_scan_profiles_config_file} config file."
)
except toml.decoder.TomlDecodeError as e:
fail(
"Error: Couldn't parse {port_scan_profiles_config_file} config file. Check syntax and duplicate tags."
)
with open(service_scans_config_file, "r") as c:
try:
service_scans_config = toml.load(c)
except toml.decoder.TomlDecodeError as e:
fail(
"Error: Couldn't parse service-scans.toml config file. Check syntax and duplicate tags."
)
with open(global_patterns_config_file, "r") as p:
try:
global_patterns = toml.load(p)
if "pattern" in global_patterns:
global_patterns = global_patterns["pattern"]
else:
global_patterns = []
except toml.decoder.TomlDecodeError as e:
fail(
"Error: Couldn't parse global-patterns.toml config file. Check syntax and duplicate tags."
)
if "username_wordlist" in service_scans_config:
if isinstance(service_scans_config["username_wordlist"], str):
username_wordlist = service_scans_config["username_wordlist"]
if "password_wordlist" in service_scans_config:
if isinstance(service_scans_config["password_wordlist"], str):
password_wordlist = service_scans_config["password_wordlist"]
def e(*args, frame_index=1, **kvargs):
frame = sys._getframe(frame_index)
@ -146,40 +217,6 @@ def calculate_elapsed_time(start_time):
return ', '.join(elapsed_time)
port_scan_profiles_config_file = 'port-scan-profiles.toml'
with open(os.path.join(rootdir, 'config', port_scan_profiles_config_file), 'r') as p:
try:
port_scan_profiles_config = toml.load(p)
if len(port_scan_profiles_config) == 0:
fail('There do not appear to be any port scan profiles configured in the {port_scan_profiles_config_file} config file.')
except toml.decoder.TomlDecodeError as e:
fail('Error: Couldn\'t parse {port_scan_profiles_config_file} config file. Check syntax and duplicate tags.')
with open(os.path.join(rootdir, 'config', 'service-scans.toml'), 'r') as c:
try:
service_scans_config = toml.load(c)
except toml.decoder.TomlDecodeError as e:
fail('Error: Couldn\'t parse service-scans.toml config file. Check syntax and duplicate tags.')
with open(os.path.join(rootdir, 'config', 'global-patterns.toml'), 'r') as p:
try:
global_patterns = toml.load(p)
if 'pattern' in global_patterns:
global_patterns = global_patterns['pattern']
else:
global_patterns = []
except toml.decoder.TomlDecodeError as e:
fail('Error: Couldn\'t parse global-patterns.toml config file. Check syntax and duplicate tags.')
if 'username_wordlist' in service_scans_config:
if isinstance(service_scans_config['username_wordlist'], str):
username_wordlist = service_scans_config['username_wordlist']
if 'password_wordlist' in service_scans_config:
if isinstance(service_scans_config['password_wordlist'], str):
password_wordlist = service_scans_config['password_wordlist']
async def read_stream(stream, target, tag='?', patterns=[], color=Fore.BLUE):
address = target.address
@ -595,7 +632,7 @@ async def scan_services(loop, semaphore, target):
pending.add(asyncio.ensure_future(run_cmd(semaphore, e(command), target, tag=tag, patterns=patterns)))
def scan_host(target, concurrent_scans):
def scan_host(target, concurrent_scans, outdir):
start_time = time.time()
info('Scanning target {byellow}{target.address}{rst}')
@ -655,8 +692,18 @@ class Target:
self.lock = None
self.running_tasks = []
if __name__ == '__main__':
def main():
global single_target
global only_scans_dir
global port_scan_profile
global heartbeat_interval
global nmap
global srvname
global verbose
_init()
parser = argparse.ArgumentParser(description='Network reconnaissance tool to port scan and automatically enumerate services found on multiple targets.')
parser.add_argument('targets', action='store', help='IP addresses (e.g. 10.0.0.1), CIDR notation (e.g. 10.0.0.1/24), or resolvable hostnames (e.g. foo.bar) to scan.', nargs="*")
parser.add_argument('-t', '--targets', action='store', type=str, default='', dest='target_file', help='Read targets from file.')
@ -814,7 +861,7 @@ if __name__ == '__main__':
for address in targets:
target = Target(address)
futures.append(executor.submit(scan_host, target, concurrent_scans))
futures.append(executor.submit(scan_host, target, concurrent_scans, outdir))
try:
for future in as_completed(futures):
@ -827,3 +874,8 @@ if __name__ == '__main__':
elapsed_time = calculate_elapsed_time(start_time)
info('{bgreen}Finished scanning all targets in {elapsed_time}!{rst}')
if __name__ == '__main__':
main()