Docs: rewrite README for fork; add quick developer setup and changelog
This commit is contained in:
mazennafee
parent
73fb23553f
commit
8eebb04dad
270
README.md
270
README.md
|
|
@ -1,183 +1,153 @@
|
|||
## About Sublist3r
|
||||
# Sublist3r (fork by Mazen Nafee)
|
||||
|
||||
Sublist3r is a python tool designed to enumerate subdomains of websites using OSINT. It helps penetration testers and bug hunters collect and gather subdomains for the domain they are targeting. Sublist3r enumerates subdomains using many search engines such as Google, Yahoo, Bing, Baidu and Ask. Sublist3r also enumerates subdomains using Netcraft, Virustotal, ThreatCrowd, DNSdumpster and ReverseDNS.
|
||||
**Fork of** aboul3la/Sublist3r — maintained by Mazen Nafee.
|
||||
This fork focuses on robustness, tests, and developer ergonomics:
|
||||
|
||||
[subbrute](https://github.com/TheRook/subbrute) was integrated with Sublist3r to increase the possibility of finding more subdomains using bruteforce with an improved wordlist. The credit goes to TheRook who is the author of subbrute.
|
||||
- Defensive DNSdumpster CSRF extraction (regex + BeautifulSoup fallback).
|
||||
- Validation and safe handling of nameserver entries to avoid dnspython errors.
|
||||
- Unit tests and small repro scripts for the fixes.
|
||||
- GitHub Actions CI to run tests on push and pull requests.
|
||||
|
||||
## Screenshots
|
||||
---
|
||||
|
||||
|
||||
## About
|
||||
|
||||
Sublist3r is a Python tool for enumerating subdomains using OSINT. It helps penetration testers and bug hunters collect subdomains for a target domain by querying search engines and public services. Sublist3r supports multiple sources including Google, Yahoo, Bing, Baidu, Ask, Netcraft, VirusTotal, ThreatCrowd, DNSdumpster, and ReverseDNS.
|
||||
|
||||
Subbrute is integrated to increase coverage via bruteforce using an improved wordlist. Credit for subbrute goes to TheRook.
|
||||
|
||||
---
|
||||
|
||||
## Quick start
|
||||
|
||||
```bash
|
||||
# clone the fork
|
||||
git clone https://github.com/mazennafee/Sublist3r.git
|
||||
cd Sublist3r
|
||||
|
||||
# create and activate a virtual environment
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate
|
||||
|
||||
# install editable and test dependencies
|
||||
pip install --upgrade pip
|
||||
pip install -e .
|
||||
pip install -r requirements.txt || true
|
||||
|
||||
# run tests
|
||||
pytest -q
|
||||
|
||||
Usage
|
||||
|
||||
Command line
|
||||
|
||||
# basic usage
|
||||
python3 ./sublist3r.py -d example.com
|
||||
|
||||
# use the console script after activating venv
|
||||
sublist3r -d example.com -t 10 -o results.txt
|
||||
|
||||
Options
|
||||
|
||||
-d, --domain Domain name to enumerate subdomains of
|
||||
|
||||
-b, --bruteforce Enable the subbrute bruteforce module
|
||||
|
||||
-p, --ports Scan found subdomains against specific TCP ports
|
||||
|
||||
-v, --verbose Enable verbose mode and display results in realtime
|
||||
|
||||
-t, --threads Number of threads to use for bruteforce or lookups
|
||||
|
||||
-e, --engines Comma separated list of search engines to use
|
||||
|
||||
-o, --output Save results to a text file
|
||||
|
||||
-h, --help Show help and exit
|
||||
|
||||
Examples
|
||||
|
||||
# show help
|
||||
python3 sublist3r.py -h
|
||||
|
||||
# enumerate subdomains
|
||||
python3 sublist3r.py -d example.com
|
||||
|
||||
# enumerate and scan ports 80 and 443
|
||||
python3 sublist3r.py -d example.com -p 80,443
|
||||
|
||||
# enable bruteforce
|
||||
python3 sublist3r.py -b -d example.com
|
||||
|
||||
|
||||
## Installation
|
||||
Using Sublist3r as a module
|
||||
|
||||
```
|
||||
git clone https://github.com/aboul3la/Sublist3r.git
|
||||
```
|
||||
import sublist3r
|
||||
|
||||
## Recommended Python Version:
|
||||
subdomains = sublist3r.main(
|
||||
domain="example.com",
|
||||
no_threads=40,
|
||||
savefile="example_subdomains.txt",
|
||||
ports=None,
|
||||
silent=False,
|
||||
verbose=False,
|
||||
enable_bruteforce=False,
|
||||
engines=None
|
||||
)
|
||||
|
||||
Sublist3r currently supports **Python 2** and **Python 3**.
|
||||
print(len(subdomains), "unique subdomains found")
|
||||
|
||||
* The recommended version for Python 2 is **2.7.x**
|
||||
* The recommended version for Python 3 is **3.4.x**
|
||||
Return value The main function returns a set of unique subdomain strings.
|
||||
|
||||
## Dependencies:
|
||||
Dependencies and supported Python versions
|
||||
Supported Python This fork targets Python 3.8 or newer.
|
||||
|
||||
Sublist3r depends on the `requests`, `dnspython` and `argparse` python modules.
|
||||
Key dependencies
|
||||
|
||||
These dependencies can be installed using the requirements file:
|
||||
requests
|
||||
|
||||
- Installation on Windows:
|
||||
```
|
||||
c:\python27\python.exe -m pip install -r requirements.txt
|
||||
```
|
||||
- Installation on Linux
|
||||
```
|
||||
sudo pip install -r requirements.txt
|
||||
```
|
||||
dnspython
|
||||
|
||||
Alternatively, each module can be installed independently as shown below.
|
||||
beautifulsoup4
|
||||
|
||||
#### Requests Module (http://docs.python-requests.org/en/latest/)
|
||||
Install dependencies with:
|
||||
pip install -r requirements.txt
|
||||
Add or update requirements.txt with pinned minimums for reproducible installs.
|
||||
|
||||
- Install for Windows:
|
||||
```
|
||||
c:\python27\python.exe -m pip install requests
|
||||
```
|
||||
Tests and CI
|
||||
This fork includes unit tests and small repro scripts under tests/ and a GitHub Actions workflow at .github/workflows/ci.yml that runs pytest on push and pull requests.
|
||||
|
||||
- Install for Ubuntu/Debian:
|
||||
```
|
||||
sudo apt-get install python-requests
|
||||
```
|
||||
Run tests locally:
|
||||
pytest -q
|
||||
|
||||
- Install for Centos/Redhat:
|
||||
```
|
||||
sudo yum install python-requests
|
||||
```
|
||||
License
|
||||
This project is licensed under the GNU GPL v2. See the LICENSE file for details.
|
||||
|
||||
- Install using pip on Linux:
|
||||
```
|
||||
sudo pip install requests
|
||||
```
|
||||
Credits
|
||||
|
||||
#### dnspython Module (http://www.dnspython.org/)
|
||||
TheRook — subbrute integration and bruteforce approach
|
||||
|
||||
- Install for Windows:
|
||||
```
|
||||
c:\python27\python.exe -m pip install dnspython
|
||||
```
|
||||
Bitquark — inspiration for the subbrute wordlist
|
||||
|
||||
- Install for Ubuntu/Debian:
|
||||
```
|
||||
sudo apt-get install python-dnspython
|
||||
```
|
||||
Ahmed Aboul-Ela — original Sublist3r project
|
||||
|
||||
- Install using pip:
|
||||
```
|
||||
sudo pip install dnspython
|
||||
```
|
||||
Contributing
|
||||
|
||||
#### argparse Module
|
||||
If you want to contribute, please:
|
||||
|
||||
- Install for Ubuntu/Debian:
|
||||
```
|
||||
sudo apt-get install python-argparse
|
||||
```
|
||||
Fork the repository and create a feature branch.
|
||||
|
||||
- Install for Centos/Redhat:
|
||||
```
|
||||
sudo yum install python-argparse
|
||||
```
|
||||
Add tests for new behavior.
|
||||
|
||||
- Install using pip:
|
||||
```
|
||||
sudo pip install argparse
|
||||
```
|
||||
Open a pull request with a clear description and reproduction steps.
|
||||
|
||||
**for coloring in windows install the following libraries**
|
||||
```
|
||||
c:\python27\python.exe -m pip install win_unicode_console colorama
|
||||
```
|
||||
See CONTRIBUTING.md for more details.
|
||||
|
||||
## Usage
|
||||
Changelog
|
||||
Unreleased
|
||||
|
||||
Short Form | Long Form | Description
|
||||
------------- | ------------- |-------------
|
||||
-d | --domain | Domain name to enumerate subdomains of
|
||||
-b | --bruteforce | Enable the subbrute bruteforce module
|
||||
-p | --ports | Scan the found subdomains against specific tcp ports
|
||||
-v | --verbose | Enable the verbose mode and display results in realtime
|
||||
-t | --threads | Number of threads to use for subbrute bruteforce
|
||||
-e | --engines | Specify a comma-separated list of search engines
|
||||
-o | --output | Save the results to text file
|
||||
-h | --help | show the help message and exit
|
||||
Defensive DNSdumpster CSRF extraction and fallback parsing.
|
||||
|
||||
### Examples
|
||||
Nameserver validation to avoid dnspython errors.
|
||||
|
||||
* To list all the basic options and switches use -h switch:
|
||||
|
||||
```python sublist3r.py -h```
|
||||
|
||||
* To enumerate subdomains of specific domain:
|
||||
|
||||
``python sublist3r.py -d example.com``
|
||||
|
||||
* To enumerate subdomains of specific domain and show only subdomains which have open ports 80 and 443 :
|
||||
|
||||
``python sublist3r.py -d example.com -p 80,443``
|
||||
|
||||
* To enumerate subdomains of specific domain and show the results in realtime:
|
||||
|
||||
``python sublist3r.py -v -d example.com``
|
||||
|
||||
* To enumerate subdomains and enable the bruteforce module:
|
||||
|
||||
``python sublist3r.py -b -d example.com``
|
||||
|
||||
* To enumerate subdomains and use specific engines such Google, Yahoo and Virustotal engines
|
||||
|
||||
``python sublist3r.py -e google,yahoo,virustotal -d example.com``
|
||||
|
||||
|
||||
## Using Sublist3r as a module in your python scripts
|
||||
|
||||
**Example**
|
||||
|
||||
```python
|
||||
import sublist3r
|
||||
subdomains = sublist3r.main(domain, no_threads, savefile, ports, silent, verbose, enable_bruteforce, engines)
|
||||
```
|
||||
The main function will return a set of unique subdomains found by Sublist3r
|
||||
|
||||
**Function Usage:**
|
||||
* **domain**: The domain you want to enumerate subdomains of.
|
||||
* **savefile**: save the output into text file.
|
||||
* **ports**: specify a comma-sperated list of the tcp ports to scan.
|
||||
* **silent**: set sublist3r to work in silent mode during the execution (helpful when you don't need a lot of noise).
|
||||
* **verbose**: display the found subdomains in real time.
|
||||
* **enable_bruteforce**: enable the bruteforce module.
|
||||
* **engines**: (Optional) to choose specific engines.
|
||||
|
||||
Example to enumerate subdomains of Yahoo.com:
|
||||
```python
|
||||
import sublist3r
|
||||
subdomains = sublist3r.main('yahoo.com', 40, 'yahoo_subdomains.txt', ports= None, silent=False, verbose= False, enable_bruteforce= False, engines=None)
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
Sublist3r is licensed under the GNU GPL license. take a look at the [LICENSE](https://github.com/aboul3la/Sublist3r/blob/master/LICENSE) for more information.
|
||||
|
||||
|
||||
## Credits
|
||||
|
||||
* [TheRook](https://github.com/TheRook) - The bruteforce module was based on his script **subbrute**.
|
||||
* [Bitquark](https://github.com/bitquark) - The Subbrute's wordlist was based on his research **dnspop**.
|
||||
|
||||
## Thanks
|
||||
|
||||
* Special Thanks to [Ibrahim Mosaad](https://twitter.com/ibrahim_mosaad) for his great contributions that helped in improving the tool.
|
||||
|
||||
## Version
|
||||
**Current version is 1.0**
|
||||
Unit tests and CI added.
|
||||
|
|
|
|||
Loading…
Reference in New Issue