projectdiscovery/httpx
1
1
Fork 0
mirror of https://github.com/projectdiscovery/httpx.git synced 2024-11-28 13:04:02 +03:00
Code Issues Packages Projects Releases Wiki Activity
1,516 Commits 10 Branches 56 Tags 43 MiB
8b5dc7bc0f
Commit Graph

220 Commits

Author SHA1 Message Date
Mzack9999
8b5dc7bc0f merge conflict 2023-09-11 16:58:31 +02:00
Ramana Reddy
76adadfeb4
Add flag to preview first N characters (#1316) 
* Add flag to preview first N characters

* use dynamic var for body-preview flag

* strip html tags from response

* add stripe option to remove html or xml tags

* typo

* resolve merge conflicts

* add test and update readme

* update readme

* typo

* typo no 2

* small refactor

* include body preview in jsonl output

* minor changes

---------

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
 2023-09-07 15:26:54 +05:30
JoMar
179678581d
Exclude WAF (#1341) 
* Exclude WAF

* Add missing 's'

* Revert "Add missing 's'"

This reverts commit 467ceca320.
 2023-09-05 19:53:06 +05:30
Stefan Kraxberger
964cc3bbd0 Implemented a redirect strategy which respects HSTS headers and modifies the URL accordingly if necessary 2023-08-31 10:58:51 +02:00
Stefan Kraxberger
0b702c5713 Revert "Added redirect policy which takes HSTS into account, so that even if the location would go from HTTPs to HTTP it would follow to HTTPs if an HTSTS header is present" 
This reverts commit c4810ffbb7.
 2023-08-31 10:44:09 +02:00
Stefan Kraxberger
c4810ffbb7 Added redirect policy which takes HSTS into account, so that even if the location would go from HTTPs to HTTP it would follow to HTTPs if an HTSTS header is present 2023-08-31 10:41:20 +02:00
Stefan Kraxberger
13f78bc302 Modified storing of response chains, because it was not correct 2023-08-30 16:06:45 +02:00
Doğan Can Bakır
2718a9aab7 use mlutils 2023-07-17 06:54:00 +00:00
mzack
e991ff1908 commenting out train logic 2023-06-28 11:08:09 +02:00
mzack
4ca3089d5b Merge branch 'dev' into pr/1245 2023-06-28 10:38:14 +02:00
Ramana Reddy
5f8e5d0dc5
fix tls-grab option with ztls mode (#1259) 
* fix tls-grab option with ztls mode

* using canonical concatenation

* mod tidy

---------

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
 2023-06-27 20:37:47 +05:30
mzack
ae1d9511df memory optimization 2023-06-22 07:27:16 +02:00
mzack
4c8ac6703e Merge branch 'dev' into pr/1245 2023-06-22 06:11:46 +02:00
Doğan Can Bakır
900f5623e8 add error page classifier 2023-06-20 14:13:48 +03:00
Mzack9999
a14af24496
Skip body read with status code 304 (#1231) 
* skip body read with status code 304

* dep update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
 2023-06-20 14:01:51 +05:30
Mzack9999
8ea85e901d
Adding prototype of tls impersonate (#1199) 
* Adding prototype of tls impersonate

* updating docs

* misc update

* dep update

---------

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
 2023-06-17 16:34:19 +05:30
dependabot[bot]
7d41db16f9
chore(deps): bump github.com/projectdiscovery/cdncheck from 0.0.4-0.20220413175814-b47bc2d578b1 to 1.0.0 (#1118) 
* chore(deps): bump github.com/projectdiscovery/cdncheck

Bumps [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) from 0.0.4-0.20220413175814-b47bc2d578b1 to 1.0.0.
- [Release notes](https://github.com/projectdiscovery/cdncheck/releases)
- [Changelog](https://github.com/projectdiscovery/cdncheck/blob/main/.goreleaser.yaml)
- [Commits](https://github.com/projectdiscovery/cdncheck/commits/v1.0.0)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/cdncheck
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

* cdncheck update fixes

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ramana Reddy <ramanaredy.manda@gmail.com>
 2023-04-27 12:46:28 +05:30
Mzack9999
00d0977963
Adding Screenshot support (#1097) 
* Adding Screenshot support

* adding headless body

* moving example into compilable file

* updating docs

* enabling store response with screenshot

* fixing output dir path

* fixing mod

* using native leakless

* splitting response+screenshot folders

* better handling json output

* readme update

* rel => abs path

* Added chromium into docker

* “Wipe them out. All of them.” – Darth Sidious

* utils bump

* go mod tidy

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
 2023-04-27 01:07:06 +05:30
Mzack9999
1defd0a9b7
Add support to skip decode (#1055) 
add support to skip decode
 2023-03-29 19:06:47 +05:30
Tarun Koyalwar
c7456929b4 fix integration tests and upgrade deps 2023-02-08 19:50:03 +05:30
Tarun Koyalwar
3a93d2d1e8 bump url utils to v0.0.7 2023-02-07 20:15:23 +05:30
Mzack9999
db1bc2ebba
Reworking favicon hash calculation (#906) 
* Reworking favicon hash calculation

* removing debug code

* bumping go to 1.19

* removing automatic follow redirect

* more edge cases

* added favicon path info
 2023-01-06 18:28:08 +05:30
xm1k3
88c49b812b
added ztls flag (#896) 
* added ztls to httpx

* adding missing dialtls

* updated readme.md

* add ztls functional test

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
 2022-12-28 23:40:22 +05:30
Tarun Koyalwar
20a9a930c2
adds base64 encoding support (#813) (#885) 
* adds base64 encoding support (#813)

* rename b64 to base64
 2022-11-24 21:50:37 +05:30
Mzack9999
f327e1cc77 adding generic CL test cases 2022-11-09 17:02:18 +01:00
Mzack9999
9d4f6afa8a Fixing content length bytes count 2022-11-09 16:32:56 +01:00
vrenzolaverace
686a2cced8
Update helpers libraries #850 (#851) 
Use utils helpers libraries (#850)
 2022-11-07 01:52:47 +05:30
Mzack9999
8af2f59d96
Fixing https with burp (#753) 
* Fixing https with burp

* solving by making error non-fatal

* adding missing error comparison
 2022-10-27 11:04:22 +02:00
mzack
25122cee34 fixing func signature 2022-10-25 03:22:17 +02:00
Sami
12fee4f3ee
include dsl tag for all json keys (#812) 
* include dsl tag for all json keys

* removed dsl tag and built list from json tag

* asn interface changed

* refactoring expression engine

- Using mapstructure for uniform mapping
- Flat key:value in expressions/variables list

* adding fields to ignore

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
 2022-10-20 13:18:01 +05:30
Sami
c90b897eba
list-dsl flag to list json output keys which can be further used with matchers / filters (#768) 
* flag list-dsl

* variable name update

* used reflect to dnymically access DSL variables

* aligned with the changes in dev branch

Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
 2022-09-19 22:50:40 +05:30
Ice3man
674b792770
Fixed connection pooling with cancelable timeout (#763) 2022-09-15 22:57:26 +05:30
Ice3man
9b4a2ecb0f
Added tlsx integration for probing and grabbing (#767) 
* Added tlsx integration for probing and grabbing

* Added additional TLS required fields
 2022-09-15 20:33:51 +05:30
Sami
d491201eff
Issue 735+issue 739 (#757) 
* adding strings.ToLower to headers

* added validation logic for flags

* validated silent flag with incompatible flags

* silent flag bug fix

* added test case

* updated test case to include multiple headers
 2022-09-13 16:46:38 +05:30
forgedhallpass
c48909f64f
Minor fixes and improvements (#726) 
* Added .idea to .gitignore

* Updated the README

* Added more verbose information on preset extractors

* removing deprecated ioutil

Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
 2022-08-17 03:16:06 +04:00
Mzack9999
21ca2621bd
Fixing and improving custom ports handling (#714) 
* Fixing and improving custom ports

* adding test case from https://github.com/projectdiscovery/httpx/pull/713
 2022-07-31 20:54:34 +05:30
Ice3man
90a32dd65f
Added connection pooling for jarm conn + fastdialer (#707) 
Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
 2022-07-25 18:40:06 +05:30
Its Hex !
8d0a34f28a
Update go.mod (#689) 
* Exposing regex matches to json output (#668)

* Update runner.go

Fix regex json output.

* small refactor

Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>

* Update go.mod

* fixing jarm-go reference

* removing redundant declaration

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: Dothelm <49758889+st3rv04ka@users.noreply.github.com>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
 2022-07-11 03:09:43 +05:30
Sami
89cf7396b7
modified error message with invalid ports (#677) 
* modified error message with invalid ports

* updated ValidateCustomPorts function
 2022-07-07 20:04:31 +05:30
Mzack9999
dfab811ff9
removing redundant regex (#676) 2022-07-05 23:37:08 +05:30
Mzack9999
310294649f
Improving HTML decoding (#673) 2022-06-27 13:36:42 +05:30
M09Ic
cd18edc689
support multi -extract-regex flag and json output (#529) 
* Add hashes flag, to support multi body hash type. issue:489 and issue:488

* go mod update

* Add hash type validate for -hash flag

* support multi extract regexps

* support extract regexps matches for json output

* support extract regexp preset

* extract result deduplicate

* small changes

* lint ignore

Co-authored-by: sandeep <sandeep@projectdiscovery.io>
Co-authored-by: mzack <marco.rivoli.nvh@gmail.com>
 2022-06-13 17:48:00 +05:30
Mzack9999
ee655bbdcd
Adding Client TLS1.0 (#645) 2022-05-31 03:43:31 +05:30
Mzack9999
daf4ad6c5a
Fixing all ips probe with ports (#626) 
* Fixing all ips probe with standard ports

* using custom type for targets
 2022-05-19 14:59:07 +05:30
mzack
f0430675d8 Fixing SNI for HTTP transport 2022-05-10 19:43:58 +02:00
Miguel Ángel Jimeno
032c535292 httpx: consider CDN check error as non-fatal 
Fixes #599
 2022-05-03 10:31:41 +02:00
mzack
06f1c2c4c5 Adding support for custom SNI name 2022-04-15 17:41:57 +02:00
Sami
ccf0c9ddc5
Issue 549 jarm algorithm (#567) 
* jarm hash

* lint error fix

* remove default jarm value if jarm not calculated

* jarm cli boolean flag

* removed unused hash type

* readme update

* example update

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
 2022-04-08 18:50:03 +05:30
mzack
987131a20b Adding custom cookies supports in redirects 2022-03-29 19:34:55 +02:00
mzack
ffb2079dfb Adding CSP parsing form HTML meta 2022-03-22 16:47:55 +01:00