.github | ||
cmd | ||
common | ||
integration_tests | ||
internal/testutils | ||
runner | ||
scripts | ||
static | ||
.gitignore | ||
.goreleaser.yml | ||
Dockerfile | ||
go.mod | ||
go.sum | ||
LICENSE.md | ||
Makefile | ||
README.md |
Features • Installation • Usage • Running httpx • Notes • Join Discord
httpx is a fast and multi-purpose HTTP toolkit allow to run multiple probers using retryablehttp library, it is designed to maintain the result reliability with increased threads.
Features
- Simple and modular code base making it easy to contribute.
- Fast And fully configurable flags to probe multiple elements.
- Supports multiple HTTP based probings.
- Smart auto fallback from https to http as default.
- Supports hosts, URLs and CIDR as input.
- Handles edge cases doing retries, backoffs etc for handling WAFs.
Supported probes:-
Probes | Default check | Probes | Default check |
---|---|---|---|
URL | true | IP | true |
Title | true | CNAME | true |
Status Code | true | Raw HTTP | false |
Content Length | true | HTTP2 | false |
TLS Certificate | true | HTTP 1.1 Pipeline | false |
CSP Header | true | Virtual host | false |
Location Header | true | CDN | false |
Web Server | true | Path | false |
Web Socket | true | Ports | false |
Response Time | true | Request method | false |
Installation Instructions
httpx requires go1.14+ to install successfully. Run the following command to get the repo -
GO111MODULE=on go get -v github.com/projectdiscovery/httpx/cmd/httpx
Usage
httpx -h
This will display help for the tool. Here are all the switches it supports.
👉 httpx help menu 👈
Usage:
httpx [flags]
Flags:
TARGET:
-vhost-input Get a list of vhosts as input
-H string[] Custom Header to send with request
-ports string[] Port ranges to scan (nmap syntax: eg 1,2-10,11)
-http-proxy string HTTP Proxy, eg http://127.0.0.1:8080
-l string Input file containing list of hosts to process
-x string Request Methods to use, use 'all' to probe all HTTP methods
-path string Request path/file (example '/api')
-paths string Command separated paths or file containing one path per line (example '/api/v1,/apiv2')
-body string Content to send in body with HTTP request
TEMPLATE:
-tls-grab Perform TLS(SSL) data grabbing
-tech-detect Perform wappalyzer based technology detection
-threads int Number of threads (default 50)
-retries int Number of retries
-timeout int Timeout in seconds (default 5)
-vhost Check for VHOSTs
-follow-redirects Follow HTTP Redirects
-follow-host-redirects Only Follow redirects on the same host
-max-redirects int Max number of redirects to follow per host (default 10)
-tls-probe Send HTTP probes on the extracted TLS domains
-csp-probe Send HTTP probes on the extracted CSP domains
-unsafe Send raw requests skipping golang normalization
-pipeline HTTP1.1 Pipeline probe
-http2 HTTP2 probe
-no-fallback Probe both protocol (HTTPS and HTTP)
-no-fallback-scheme Probe with input protocol scheme
-random-agent Use randomly selected HTTP User-Agent header value (default true)
-allow string[] Allow list of IP/CIDR's to process (file or comma separated)
-deny string[] Deny list of IP/CIDR's to process (file or comma separated)
-response-size-to-save int Max response size to save in bytes (default - unlimited) (default 2147483647)
-response-size-to-read int Max response size to read in bytes (default - unlimited) (default 2147483647)
-resume Resume scan using resume.cfg
-exclude-cdn Skip full port scans for CDNs (only checks for 80,443)
-max-host-error int Max error count per host before skipping remaining path/s (default 30)
FILTERING:
-mc string Match response with specific status code (-mc 200,302)
-ml string Match response with specific content length (-ml 102)
-fc string Filter response with specific status code (-fc 403,401)
-fl string Filter response with specific content length (-fl 23)
-filter-string string Filter response with specific string
-match-string string Match response with specific string
-filter-regex string Filter response with specific regex
-match-regex string Match response with specific regex
-extract-regex string Display response content with matched regex
RATE-LIMIT:
-rate-limit int Maximum requests to send per second (default 150)
OUTPUT:
-o string File to write output to (optional)
-status-code Display HTTP response status code
-title Display page title
-location Display location header
-content-length Display HTTP response content length
-sr Store HTTP response to directory (default 'output')
-srd string Custom directory to store HTTP responses (default "output")
-json Display output in JSON format
-csv Display output in CSV format
-method Display request method
-silent Silent mode
-version Show version of httpx
-verbose Verbose Mode
-no-color Disable colored output
-web-server Display server header
-websocket Display server using websocket
-response-in-json Show Raw HTTP response In Output (-json only) (deprecated)
-include-response Show Raw HTTP response In Output (-json only)
-include-chain Show Raw HTTP Chain In Output (-json only)
-content-type Display content-type header
-ip Display Host IP
-request string File containing raw request
-debug Debug mode
-cname Display Host cname
-cdn Display CDN
-response-time Display the response time
-stats Enable statistic on keypress (terminal may become unresponsive till the end)
-store-chain Save chain to file (default 'output')
-probe Display probe status
Running httpX
Running httpx with stdin
This will run the tool against all the hosts and subdomains in hosts.txt
and returns URLs running HTTP webserver.
▶ cat hosts.txt | httpx
__ __ __ _ __
/ /_ / /_/ /_____ | |/ /
/ __ \/ __/ __/ __ \| /
/ / / / /_/ /_/ /_/ / |
/_/ /_/\__/\__/ .___/_/|_| v1.0
/_/
projectdiscovery.io
[WRN] Use with caution. You are responsible for your actions
[WRN] Developers assume no liability and are not responsible for any misuse or damage.
https://mta-sts.managed.hackerone.com
https://mta-sts.hackerone.com
https://mta-sts.forwarding.hackerone.com
https://docs.hackerone.com
https://www.hackerone.com
https://resources.hackerone.com
https://api.hackerone.com
https://support.hackerone.com
Running httpx with file input
This will run the tool with the probe
flag against all of the hosts in hosts.txt and return URLs with probed status.
▶ httpx -l hosts.txt -silent -probe
http://ns.hackerone.com [FAILED]
https://docs.hackerone.com [SUCCESS]
https://mta-sts.hackerone.com [SUCCESS]
https://mta-sts.managed.hackerone.com [SUCCESS]
http://email.hackerone.com [FAILED]
https://mta-sts.forwarding.hackerone.com [SUCCESS]
http://links.hackerone.com [FAILED]
https://api.hackerone.com [SUCCESS]
https://www.hackerone.com [SUCCESS]
http://events.hackerone.com [FAILED]
https://support.hackerone.com [SUCCESS]
https://gslink.hackerone.com [SUCCESS]
http://o1.email.hackerone.com [FAILED]
http://info.hackerone.com [FAILED]
https://resources.hackerone.com [SUCCESS]
http://o2.email.hackerone.com [FAILED]
http://o3.email.hackerone.com [FAILED]
http://go.hackerone.com [FAILED]
http://a.ns.hackerone.com [FAILED]
http://b.ns.hackerone.com [FAILED]
Running httpx with CIDR input
▶ echo 173.0.84.0/24 | httpx -silent
https://173.0.84.29
https://173.0.84.43
https://173.0.84.31
https://173.0.84.44
https://173.0.84.12
https://173.0.84.4
https://173.0.84.36
https://173.0.84.45
https://173.0.84.14
https://173.0.84.25
https://173.0.84.46
https://173.0.84.24
https://173.0.84.32
https://173.0.84.9
https://173.0.84.13
https://173.0.84.6
https://173.0.84.16
https://173.0.84.34
Running httpx with subfinder
subfinder -d hackerone.com | httpx -title -tech-detect -status-code
__ __ __ _ __
/ /_ / /_/ /_____ | |/ /
/ __ \/ __/ __/ __ \| /
/ / / / /_/ /_/ /_/ / |
/_/ /_/\__/\__/ .___/_/|_|
/_/ v1.0.6
projectdiscovery.io
Use with caution. You are responsible for your actions
Developers assume no liability and are not responsible for any misuse or damage.
https://mta-sts.managed.hackerone.com [404] [Page not found · GitHub Pages] [Varnish,GitHub Pages,Ruby on Rails]
https://mta-sts.hackerone.com [404] [Page not found · GitHub Pages] [Varnish,GitHub Pages,Ruby on Rails]
https://mta-sts.forwarding.hackerone.com [404] [Page not found · GitHub Pages] [GitHub Pages,Ruby on Rails,Varnish]
https://docs.hackerone.com [200] [HackerOne Platform Documentation] [Ruby on Rails,jsDelivr,Gatsby,React,webpack,Varnish,GitHub Pages]
https://support.hackerone.com [301,302,301,200] [HackerOne] [Cloudflare,Ruby on Rails,Ruby]
https://resources.hackerone.com [301,301,404] [Sorry, no Folders found.]
📋 Notes
- As default, httpx checks for
HTTPS
probe and fall-back toHTTP
only ifHTTPS
is not reachable. - For printing both HTTP/HTTPS results,
no-fallback
flag can be used. - Custom scheme for ports can be defined, for example
-ports http:443,http:80,https:8443
vhost
,http2
,pipeline
,ports
,csp-probe
,tls-probe
andpath
are unique flag with different probes.- Unique flags should be used for specific use cases instead of running them as default with other flags.
- When using
json
flag, all the information (default probes) included in the JSON output.
Thanks
httpx is made with 🖤 by the projectdiscovery team. Community contributions have made the project what it is. See the Thanks.md file for more details. Do also check out these similar awesome projects that may fit in your workflow:
Probing feature is inspired by @tomnomnom/httprobe work ❤️