httpx is a fast and multi-purpose HTTP toolkit that allows running multiple probes using the retryablehttp library.
Go to file
2020-06-01 21:23:17 +05:30
cmd/httpx fixed empty title in json 2020-05-29 22:20:15 +02:00
common adding error check on request 2020-05-29 21:59:18 +02:00
static added httpx run example 2020-05-29 15:09:34 +05:30
go.mod Initial commit 2020-05-29 09:37:55 +00:00
go.sum Initial commit 2020-05-29 09:37:55 +00:00
LICENSE.md added LICENSE file 2020-05-29 15:44:39 +05:30
README.md Update README.md 2020-06-01 21:23:17 +05:30

httpx

License Go Report Card contributions welcome

httpx is a fast and multi-purpose HTTP toolkit allow to run multiple probers using retryablehttp library, it is designed to maintain the result reliability with increased threads.

Resources

Features

httpx

  • Simple and modular code base making it easy to contribute.
  • Fast And fully configurable flags to probe mutiple elements.
  • Supports vhost, urls, ports, title, content-length, status-code, response-body probbing.
  • Smart auto fallback between http and https.
  • Supports hosts and URLs as input.
  • Handles edge cases doing retries, backoffs etc for handling WAFs.

Usage

httpx -h

This will display help for the tool. Here are all the switches it supports.

Flag Description Example
-H Custom Header input httpx -H 'x-bug-bounty: hacker'
-follow-redirects Follow URL redirects (default false) httpx -follow-redirects
-http-proxy URL of the proxy server httpx -http-proxy hxxp://proxy-host:80
-l File containing host/urls to process httpx -l hosts.txt
-no-color Disable colors in the output. httpx -no-color
-o File to save output result (optional) httpx -o output.txt
-json Prints all the probes in JSON format (default false) httpx -json
-vhost Probes to detect vhost from list of subdomains httpx -vhost
-threads Number of threads (default 50) httpx - threads 100
-ports Ports ranges to probe (nmap syntax: eg 1,2-10,11) httpx -ports 80,443,100-200
-title Prints title of page if avaiable httpx -title
-content-length Prints content length in the output httpx -content-length
-status-code Prints status code in the output httpx -status-code
-store-response Store response as domain.txt httpx -store-response
-store-response-dir Directory to store response (default current path) httpx -store-response-dir output
-retries Number of retries httpx -retries
-silent Prints only results in the output httpx -silent
-timeout Timeout in seconds (default 5) httpx -timeout 10
-verbose Verbose Mode httpx -verbose
-version Prints current version of the httpx httpx -version
-x Request Method (default 'GET') httpx -x HEAD

Installation Instructions

From Binary

The installation is easy. You can download the pre-built binaries for your platform from the Releases page. Extract them using tar, move it to your $PATHand you're ready to go.

> tar -xvf httpx-linux-amd64.tar
> mv httpx-linux-amd64 /usr/bin/httpx
> httpx -h

From Source

httpx requires go1.13+ to install successfully. Run the following command to get the repo -

> GO111MODULE=on go get -u -v github.com/projectdiscovery/httpx/cmd/httpx

In order to update the tool, you can use -u flag with go get command.

Running httpX to probe 7614 hosts

> chaos -d uber.com -count -silent 

7614

> time chaos -d uber.com -silent | httpx -status-code -content-length -title -store-response -threads 100 -json | wc 

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   / 
 / / / / /_/ /_/ /_/ /   |  
/_/ /_/\__/\__/ .___/_/|_|  
             /_/            

		projectdiscovery.io

[WRN] Use with caution. You are responsible for your actions
[WRN] Developers assume no liability and are not responsible for any misuse or damage.

210

real	0m36.952s
user	0m7.976s
sys	0m7.884s

Running httpx with stdin

This will run the tool against all the hosts in hosts.txt and returns the matched results.

> cat hosts.txt | httpx 

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   / 
 / / / / /_/ /_/ /_/ /   |  
/_/ /_/\__/\__/ .___/_/|_|  
             /_/            

		projectdiscovery.io

[WRN] Use with caution. You are responsible for your actions
[WRN] Developers assume no liability and are not responsible for any misuse or damage.
https://mta-sts.managed.hackerone.com
https://mta-sts.hackerone.com
https://mta-sts.forwarding.hackerone.com
https://docs.hackerone.com
https://www.hackerone.com
https://resources.hackerone.com
https://api.hackerone.com
https://support.hackerone.com

Running httpx with file input

This will run the tool against all the hosts in hosts.txt and returns the matched results.

> httpx -l hosts.txt

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   / 
 / / / / /_/ /_/ /_/ /   |  
/_/ /_/\__/\__/ .___/_/|_|  
             /_/            

		projectdiscovery.io

[WRN] Use with caution. You are responsible for your actions
[WRN] Developers assume no liability and are not responsible for any misuse or damage.
https://docs.hackerone.com
https://mta-sts.hackerone.com
https://mta-sts.managed.hackerone.com
https://mta-sts.forwarding.hackerone.com
https://www.hackerone.com
https://resources.hackerone.com
https://api.hackerone.com
https://support.hackerone.com

Using httpX with subfinder/chaos and any other similar tool.

> subfinder -d hackerone.com -silent | httpx -title -content-length -status-code


    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   / 
 / / / / /_/ /_/ /_/ /   |  
/_/ /_/\__/\__/ .___/_/|_|  
             /_/            

		projectdiscovery.io

[WRN] Use with caution. You are responsible for your actions
[WRN] Developers assume no liability and are not responsible for any misuse or damage.
https://mta-sts.forwarding.hackerone.com [404] [9339] [Page not found · GitHub Pages]
https://mta-sts.hackerone.com [404] [9339] [Page not found · GitHub Pages]
https://mta-sts.managed.hackerone.com [404] [9339] [Page not found · GitHub Pages]
https://docs.hackerone.com [200] [65444] [HackerOne Platform Documentation]
https://www.hackerone.com [200] [54166] [Bug Bounty - Hacker Powered Security Testing | HackerOne]
https://support.hackerone.com [301] [489] []
https://api.hackerone.com [200] [7791] [HackerOne API]
https://hackerone.com [301] [92] []
https://resources.hackerone.com [301] [0] []

Running httpX with json output

> chaos -d hackerone.com -silent | httpx -json

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   / 
 / / / / /_/ /_/ /_/ /   |  
/_/ /_/\__/\__/ .___/_/|_|  
             /_/            

		projectdiscovery.io

[WRN] Use with caution. You are responsible for your actions
[WRN] Developers assume no liability and are not responsible for any misuse or damage.

{"url":"https://mta-sts.forwarding.hackerone.com","content-length":9339,"status-code":404,"title":"","error":null,"vhost":false}
{"url":"https://mta-sts.hackerone.com","content-length":9339,"status-code":404,"title":"","error":null,"vhost":false}
{"url":"https://docs.hackerone.com","content-length":65444,"status-code":200,"title":"","error":null,"vhost":false}
{"url":"https://mta-sts.managed.hackerone.com","content-length":9339,"status-code":404,"title":"","error":null,"vhost":false}
{"url":"https://support.hackerone.com","content-length":489,"status-code":301,"title":"","error":null,"vhost":false}
{"url":"https://resources.hackerone.com","content-length":0,"status-code":301,"title":"","error":null,"vhost":false}
{"url":"https://api.hackerone.com","content-length":7791,"status-code":200,"title":"","error":null,"vhost":false}
{"url":"https://www.hackerone.com","content-length":54166,"status-code":200,"title":"","error":null,"vhost":false}

You can simply use jq to filter out the json results as per your interest.

Todo

Thanks

httpX is made with 🖤 by the projectdiscovery team. Community contributions have made the project what it is. See the Thanks.md file for more details. Do also check out these similar awesome projects that may fit in your workflow:

Probing feature is inspired by @tomnomnom/httprobe work ❤️