Commit Graph

2074 Commits

Author SHA1 Message Date
sandeep
842e59ed05 Merge branch 'dev' of https://github.com/projectdiscovery/httpx into dev 2024-06-24 19:12:54 +05:30
sandeep
cba8ecbbd9 version update 2024-06-24 19:12:46 +05:30
Dogan Can Bakir
2821731fc2
bump wappalyzer version (#1777)
Co-authored-by: sandeep <8293321+ehsandeep@users.noreply.github.com>
2024-06-24 19:11:11 +05:30
Ice3man
3b5554af36
feat: added enhancements to favicon + made API public (#1774)
* more additions and enhancements to httpx

* feat: added enhancements to favicon + made API public

* update deps

* misc additions

* fix hasChain with 1 status code not working

* Revert "fix hasChain with 1 status code not working"

This reverts commit fabadcddbc.

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
2024-06-24 19:07:26 +05:30
dependabot[bot]
d2fcc0fd9c
Merge pull request #1779 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/wappalyzergo-0.1.8 2024-06-24 10:37:05 +00:00
dependabot[bot]
ae3490f923
chore(deps): bump github.com/projectdiscovery/wappalyzergo
Bumps [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) from 0.1.6 to 0.1.8.
- [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases)
- [Commits](https://github.com/projectdiscovery/wappalyzergo/compare/v0.1.6...v0.1.8)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/wappalyzergo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-24 10:32:00 +00:00
dependabot[bot]
48c9c3895d
Merge pull request #1782 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/useragent-0.0.57 2024-06-24 10:31:14 +00:00
dependabot[bot]
41109a0c2a
chore(deps): bump github.com/projectdiscovery/useragent
Bumps [github.com/projectdiscovery/useragent](https://github.com/projectdiscovery/useragent) from 0.0.56 to 0.0.57.
- [Release notes](https://github.com/projectdiscovery/useragent/releases)
- [Commits](https://github.com/projectdiscovery/useragent/compare/v0.0.56...v0.0.57)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/useragent
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-24 10:26:40 +00:00
dependabot[bot]
cdb5541c43
Merge pull request #1780 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/utils-0.1.5 2024-06-24 10:26:04 +00:00
dependabot[bot]
6ee165bd3f
Merge pull request #1781 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/cdncheck-1.1.0 2024-06-24 10:25:26 +00:00
dependabot[bot]
f3af434678
Merge pull request #1778 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/rawhttp-0.1.55 2024-06-24 10:25:21 +00:00
dependabot[bot]
1f269567a7
chore(deps): bump github.com/projectdiscovery/cdncheck
Bumps [github.com/projectdiscovery/cdncheck](https://github.com/projectdiscovery/cdncheck) from 1.0.9 to 1.1.0.
- [Release notes](https://github.com/projectdiscovery/cdncheck/releases)
- [Changelog](https://github.com/projectdiscovery/cdncheck/blob/main/.goreleaser.yaml)
- [Commits](https://github.com/projectdiscovery/cdncheck/compare/v1.0.9...v1.1.0)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/cdncheck
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-24 10:20:18 +00:00
dependabot[bot]
3fbc7f005c
chore(deps): bump github.com/projectdiscovery/utils from 0.1.4 to 0.1.5
Bumps [github.com/projectdiscovery/utils](https://github.com/projectdiscovery/utils) from 0.1.4 to 0.1.5.
- [Release notes](https://github.com/projectdiscovery/utils/releases)
- [Changelog](https://github.com/projectdiscovery/utils/blob/main/CHANGELOG.md)
- [Commits](https://github.com/projectdiscovery/utils/compare/v0.1.4...v0.1.5)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/utils
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-24 10:20:14 +00:00
dependabot[bot]
ca2319975b
chore(deps): bump github.com/projectdiscovery/rawhttp
Bumps [github.com/projectdiscovery/rawhttp](https://github.com/projectdiscovery/rawhttp) from 0.1.54 to 0.1.55.
- [Release notes](https://github.com/projectdiscovery/rawhttp/releases)
- [Commits](https://github.com/projectdiscovery/rawhttp/compare/v0.1.54...v0.1.55)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/rawhttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-24 10:20:04 +00:00
sandeep
1ecf09747c version update 2024-06-22 12:51:57 -07:00
Ramana Reddy
9330887a58
Extract body_domains and body_fqdns to jsonl (#1750)
* Extract body-domains and body-fqdns

* remvove port in domains

* Add test for domains extraction

* misc update

* improve domain regex

* fix test

* extract domain inside quotes

* sanitize urls

* fix test

* minor

* do not embed

* remove js variables fp + improve regex

---------

Co-authored-by: Tarun Koyalwar <tarun@projectdiscovery.io>
2024-06-23 01:01:43 +05:30
dependabot[bot]
72f4c2cef4
Merge pull request #1765 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/retryablehttp-go-1.0.65 2024-06-17 16:50:32 +00:00
dependabot[bot]
ebb1df4175
Merge pull request #1766 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/wappalyzergo-0.1.6 2024-06-17 16:50:10 +00:00
dependabot[bot]
80a7e69c74
chore(deps): bump github.com/projectdiscovery/retryablehttp-go
Bumps [github.com/projectdiscovery/retryablehttp-go](https://github.com/projectdiscovery/retryablehttp-go) from 1.0.64 to 1.0.65.
- [Release notes](https://github.com/projectdiscovery/retryablehttp-go/releases)
- [Commits](https://github.com/projectdiscovery/retryablehttp-go/compare/v1.0.64...v1.0.65)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/retryablehttp-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-17 16:39:32 +00:00
dependabot[bot]
647d156c33
chore(deps): bump github.com/projectdiscovery/wappalyzergo
Bumps [github.com/projectdiscovery/wappalyzergo](https://github.com/projectdiscovery/wappalyzergo) from 0.1.4 to 0.1.6.
- [Release notes](https://github.com/projectdiscovery/wappalyzergo/releases)
- [Commits](https://github.com/projectdiscovery/wappalyzergo/compare/v0.1.4...v0.1.6)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/wappalyzergo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-17 16:39:31 +00:00
dependabot[bot]
74c1237eb6
Merge pull request #1764 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/useragent-0.0.56 2024-06-17 16:38:43 +00:00
dependabot[bot]
534ae56420
chore(deps): bump github.com/projectdiscovery/useragent
Bumps [github.com/projectdiscovery/useragent](https://github.com/projectdiscovery/useragent) from 0.0.54 to 0.0.56.
- [Release notes](https://github.com/projectdiscovery/useragent/releases)
- [Commits](https://github.com/projectdiscovery/useragent/compare/v0.0.54...v0.0.56)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/useragent
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-17 16:32:35 +00:00
dependabot[bot]
ac6c4f8ecb
Merge pull request #1767 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/rawhttp-0.1.54 2024-06-17 16:31:36 +00:00
dependabot[bot]
acda5d4823
Merge pull request #1768 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/goflags-0.1.56 2024-06-17 16:31:04 +00:00
dependabot[bot]
2faaeda99d
chore(deps): bump github.com/projectdiscovery/rawhttp
Bumps [github.com/projectdiscovery/rawhttp](https://github.com/projectdiscovery/rawhttp) from 0.1.52 to 0.1.54.
- [Release notes](https://github.com/projectdiscovery/rawhttp/releases)
- [Commits](https://github.com/projectdiscovery/rawhttp/compare/v0.1.52...v0.1.54)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/rawhttp
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-17 16:26:23 +00:00
dependabot[bot]
0cab380c6d
chore(deps): bump github.com/projectdiscovery/goflags
Bumps [github.com/projectdiscovery/goflags](https://github.com/projectdiscovery/goflags) from 0.1.54 to 0.1.56.
- [Release notes](https://github.com/projectdiscovery/goflags/releases)
- [Commits](https://github.com/projectdiscovery/goflags/compare/v0.1.54...v0.1.56)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/goflags
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-17 16:26:04 +00:00
Ice3man
355814014a
Merge pull request #1769 from projectdiscovery/use-tech-networkpolicy-singletons
feat: use networkpolicy + wappalyzer singletons from library
2024-06-17 21:55:13 +05:30
Ice3man
80d5532b15 misc fixing tests 2024-06-17 19:58:12 +05:30
Ice3man
ca3e696834 feat: use networkpolicy + wappalyzer singletons from library 2024-06-17 18:11:01 +05:30
SAEED
736ed3299c
Added the Functionality of storing raw request response headers (#1671)
* implemented storing header

* Revert "chore(deps): bump github.com/quic-go/quic-go from 0.37.7 to 0.42.0"

This reverts commit a88d85acd6.

* implemented req/resp headers storing feature

* implemented req/resp headers storing feature

* update readme

* commit to last commit

* add `-ob, -omit-body` flag

---------

Co-authored-by: Sandeep Singh <sandeep@projectdiscovery.io>
Co-authored-by: Doğan Can Bakır <dogancanbakir@protonmail.com>
Co-authored-by: Mzack9999 <mzack9999@protonmail.com>
2024-06-13 02:37:14 +05:30
Mzack9999
e432123e95
Merge pull request #1761 from projectdiscovery/support_multiple_values
Support multiple values
2024-06-12 15:45:18 +02:00
Mzack9999
6811b18e85 removing redundant code + update readme 2024-06-12 15:36:16 +02:00
Mzack9999
8f9bb51b64
Merge pull request #1762 from projectdiscovery/http_based_speed_control
add http based speed control
2024-06-12 15:31:56 +02:00
Mzack9999
1db018b469 adding tests 2024-06-12 15:10:53 +02:00
Mzack9999
746b821388 using std waitgroup for non-changing output 2024-06-12 14:57:04 +02:00
Doğan Can Bakır
dece6fce63 commit to last commit 2024-06-12 14:34:30 +03:00
Doğan Can Bakır
d77348b8da set threads to 2 2024-06-12 14:26:00 +03:00
Doğan Can Bakır
6adf02ea9c minor 2024-06-12 14:15:44 +03:00
Doğan Can Bakır
a91cce13ae add http based speed control 2024-06-12 13:53:07 +03:00
Mzack9999
a636cf0e86
Merge pull request #1759 from projectdiscovery/fix_empty_host
fix empty host
2024-06-11 18:44:29 +02:00
Mzack9999
0729980017 stripping port 2024-06-11 18:39:35 +02:00
Doğan Can Bakır
812d7d892f update README 2024-06-11 12:07:28 +03:00
Doğan Can Bakır
608d6d40d7 support multiple values 2024-06-11 12:04:54 +03:00
Dogan Can Bakir
64ab42a4c8
do not omit status code (#1760) 2024-06-10 21:11:00 +05:30
Doğan Can Bakır
3e0593b81b fix empty host 2024-06-10 16:44:54 +03:00
dependabot[bot]
da34530300
Merge pull request #1757 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/fastdialer-0.1.3 2024-06-10 10:44:37 +00:00
dependabot[bot]
1e91ab5d22
chore(deps): bump github.com/projectdiscovery/fastdialer
Bumps [github.com/projectdiscovery/fastdialer](https://github.com/projectdiscovery/fastdialer) from 0.1.2 to 0.1.3.
- [Release notes](https://github.com/projectdiscovery/fastdialer/releases)
- [Commits](https://github.com/projectdiscovery/fastdialer/compare/v0.1.2...v0.1.3)

---
updated-dependencies:
- dependency-name: github.com/projectdiscovery/fastdialer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2024-06-10 10:39:45 +00:00
dependabot[bot]
35d5a664fd
Merge pull request #1755 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/hmap-0.0.46 2024-06-10 10:38:20 +00:00
dependabot[bot]
fc17089bc1
Merge pull request #1754 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/retryablehttp-go-1.0.64 2024-06-10 10:38:08 +00:00
dependabot[bot]
54f9065597
Merge pull request #1756 from projectdiscovery/dependabot/go_modules/dev/github.com/projectdiscovery/dsl-0.1.2 2024-06-10 10:38:03 +00:00