microsoft/playwright
1
1
Fork 0
mirror of https://github.com/microsoft/playwright.git synced 2024-11-30 23:45:33 +03:00
Code Issues Projects Releases Wiki Activity

fix(har timing): record connect timing for proxied connections (#32855)

Browse Source 
Fixes a bug discovered in
https://github.com/microsoft/playwright/pull/32647. When using http
proxy, the `connect` event isn't emitted so we don't populate
`tcpConnectionAt`. The updated version of `https-proxy-agent` emits a
`proxyConnect` as a replacement, so this PR updates and listens to that
event.
For socks proxies, the `on("socket")` event is emitted once the SOCKS
connection is established, which is the equivalent of having a TCP
connection available.

---------

Signed-off-by: Simon Knott <info@simonknott.de>
Co-authored-by: Max Schmitt <max@schmitt.mx>
This commit is contained in:
Simon Knott 2024-10-07 09:59:13 +02:00 committed by GitHub
parent ddeabdf770
commit de4a4d1ce1
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
7 changed files with 94 additions and 432 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

392
packages/playwright-core/ThirdPartyNotices.txt
View File

@ -6,7 +6,7 @@ This project incorporates components from the projects listed below. The origina
- @types/node@17.0.24 (https://github.com/DefinitelyTyped/DefinitelyTyped)
- @types/yauzl@2.10.0 (https://github.com/DefinitelyTyped/DefinitelyTyped)
- agent-base@6.0.2 (https://github.com/TooTallNate/node-agent-base)
- agent-base@7.1.1 (https://github.com/TooTallNate/proxy-agents)
- balanced-match@1.0.2 (https://github.com/juliangruber/balanced-match)
- brace-expansion@1.1.11 (https://github.com/juliangruber/brace-expansion)
- buffer-crc32@0.2.13 (https://github.com/brianloveswords/buffer-crc32)
@ -23,7 +23,7 @@ This project incorporates components from the projects listed below. The origina
- fd-slicer@1.1.0 (https://github.com/andrewrk/node-fd-slicer)
- get-stream@5.2.0 (https://github.com/sindresorhus/get-stream)
- graceful-fs@4.2.10 (https://github.com/isaacs/node-graceful-fs)
- https-proxy-agent@5.0.0 (https://github.com/TooTallNate/node-https-proxy-agent)
- https-proxy-agent@7.0.5 (https://github.com/TooTallNate/proxy-agents)
- ip-address@9.0.5 (https://github.com/beaugunderson/ip-address)
- is-docker@2.2.1 (https://github.com/sindresorhus/is-docker)
- is-wsl@2.2.0 (https://github.com/sindresorhus/is-wsl)
@ -42,7 +42,7 @@ This project incorporates components from the projects listed below. The origina
- retry@0.12.0 (https://github.com/tim-kos/node-retry)
- signal-exit@3.0.7 (https://github.com/tapjs/signal-exit)
- smart-buffer@4.2.0 (https://github.com/JoshGlazebrook/smart-buffer)
- socks-proxy-agent@6.1.1 (https://github.com/TooTallNate/node-socks-proxy-agent)
- socks-proxy-agent@8.0.4 (https://github.com/TooTallNate/proxy-agents)
- socks@2.8.3 (https://github.com/JoshGlazebrook/socks)
- sprintf-js@1.1.3 (https://github.com/alexei/sprintf.js)
- stack-utils@2.0.5 (https://github.com/tapjs/stack-utils)
@ -103,128 +103,11 @@ MIT License
=========================================
END OF @types/yauzl@2.10.0 AND INFORMATION
%% agent-base@6.0.2 NOTICES AND INFORMATION BEGIN HERE
%% agent-base@7.1.1 NOTICES AND INFORMATION BEGIN HERE
=========================================
agent-base
==========
### Turn a function into an [`http.Agent`][http.Agent] instance
[![Build Status](https://github.com/TooTallNate/node-agent-base/workflows/Node%20CI/badge.svg)](https://github.com/TooTallNate/node-agent-base/actions?workflow=Node+CI)
This module provides an `http.Agent` generator. That is, you pass it an async
callback function, and it returns a new `http.Agent` instance that will invoke the
given callback function when sending outbound HTTP requests.
#### Some subclasses:
Here's some more interesting uses of `agent-base`.
Send a pull request to list yours!
* [`http-proxy-agent`][http-proxy-agent]: An HTTP(s) proxy `http.Agent` implementation for HTTP endpoints
* [`https-proxy-agent`][https-proxy-agent]: An HTTP(s) proxy `http.Agent` implementation for HTTPS endpoints
* [`pac-proxy-agent`][pac-proxy-agent]: A PAC file proxy `http.Agent` implementation for HTTP and HTTPS
* [`socks-proxy-agent`][socks-proxy-agent]: A SOCKS proxy `http.Agent` implementation for HTTP and HTTPS
Installation
------------
Install with `npm`:
``` bash
$ npm install agent-base
```
Example
-------
Here's a minimal example that creates a new `net.Socket` connection to the server
for every HTTP request (i.e. the equivalent of `agent: false` option):
```js
var net = require('net');
var tls = require('tls');
var url = require('url');
var http = require('http');
var agent = require('agent-base');
var endpoint = 'http://nodejs.org/api/';
var parsed = url.parse(endpoint);
// This is the important part!
parsed.agent = agent(function (req, opts) {
var socket;
// `secureEndpoint` is true when using the https module
if (opts.secureEndpoint) {
socket = tls.connect(opts);
} else {
socket = net.connect(opts);
}
return socket;
});
// Everything else works just like normal...
http.get(parsed, function (res) {
console.log('"response" event!', res.headers);
res.pipe(process.stdout);
});
```
Returning a Promise or using an `async` function is also supported:
```js
agent(async function (req, opts) {
await sleep(1000);
// etc…
});
```
Return another `http.Agent` instance to "pass through" the responsibility
for that HTTP request to that agent:
```js
agent(function (req, opts) {
return opts.secureEndpoint ? https.globalAgent : http.globalAgent;
});
```
API
---
## Agent(Function callback[, Object options]) → [http.Agent][]
Creates a base `http.Agent` that will execute the callback function `callback`
for every HTTP request that it is used as the `agent` for. The callback function
is responsible for creating a `stream.Duplex` instance of some kind that will be
used as the underlying socket in the HTTP request.
The `options` object accepts the following properties:
* `timeout` - Number - Timeout for the `callback()` function in milliseconds. Defaults to Infinity (optional).
The callback function should have the following signature:
### callback(http.ClientRequest req, Object options, Function cb) → undefined
The ClientRequest `req` can be accessed to read request headers and
and the path, etc. The `options` object contains the options passed
to the `http.request()`/`https.request()` function call, and is formatted
to be directly passed to `net.connect()`/`tls.connect()`, or however
else you want a Socket to be created. Pass the created socket to
the callback function `cb` once created, and the HTTP request will
continue to proceed.
If the `https` module is used to invoke the HTTP request, then the
`secureEndpoint` property on `options` _will be set to `true`_.
License
-------
(The MIT License)
Copyright (c) 2013 Nathan Rajlich &lt;nathan@tootallnate.net&gt;
Copyright (c) 2013 Nathan Rajlich <nathan@tootallnate.net>
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
@ -244,14 +127,8 @@ IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
[http-proxy-agent]: https://github.com/TooTallNate/node-http-proxy-agent
[https-proxy-agent]: https://github.com/TooTallNate/node-https-proxy-agent
[pac-proxy-agent]: https://github.com/TooTallNate/node-pac-proxy-agent
[socks-proxy-agent]: https://github.com/TooTallNate/node-socks-proxy-agent
[http.Agent]: https://nodejs.org/api/http.html#http_class_http_agent
=========================================
END OF agent-base@6.0.2 AND INFORMATION
END OF agent-base@7.1.1 AND INFORMATION
%% balanced-match@1.0.2 NOTICES AND INFORMATION BEGIN HERE
=========================================
@ -629,124 +506,11 @@ IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
=========================================
END OF graceful-fs@4.2.10 AND INFORMATION
%% https-proxy-agent@5.0.0 NOTICES AND INFORMATION BEGIN HERE
%% https-proxy-agent@7.0.5 NOTICES AND INFORMATION BEGIN HERE
=========================================
https-proxy-agent
================
### An HTTP(s) proxy `http.Agent` implementation for HTTPS
[![Build Status](https://github.com/TooTallNate/node-https-proxy-agent/workflows/Node%20CI/badge.svg)](https://github.com/TooTallNate/node-https-proxy-agent/actions?workflow=Node+CI)
This module provides an `http.Agent` implementation that connects to a specified
HTTP or HTTPS proxy server, and can be used with the built-in `https` module.
Specifically, this `Agent` implementation connects to an intermediary "proxy"
server and issues the [CONNECT HTTP method][CONNECT], which tells the proxy to
open a direct TCP connection to the destination server.
Since this agent implements the CONNECT HTTP method, it also works with other
protocols that use this method when connecting over proxies (i.e. WebSockets).
See the "Examples" section below for more.
Installation
------------
Install with `npm`:
``` bash
$ npm install https-proxy-agent
```
Examples
--------
#### `https` module example
``` js
var url = require('url');
var https = require('https');
var HttpsProxyAgent = require('https-proxy-agent');
// HTTP/HTTPS proxy to connect to
var proxy = process.env.http_proxy || 'http://168.63.76.32:3128';
console.log('using proxy server %j', proxy);
// HTTPS endpoint for the proxy to connect to
var endpoint = process.argv[2] || 'https://graph.facebook.com/tootallnate';
console.log('attempting to GET %j', endpoint);
var options = url.parse(endpoint);
// create an instance of the `HttpsProxyAgent` class with the proxy server information
var agent = new HttpsProxyAgent(proxy);
options.agent = agent;
https.get(options, function (res) {
console.log('"response" event!', res.headers);
res.pipe(process.stdout);
});
```
#### `ws` WebSocket connection example
``` js
var url = require('url');
var WebSocket = require('ws');
var HttpsProxyAgent = require('https-proxy-agent');
// HTTP/HTTPS proxy to connect to
var proxy = process.env.http_proxy || 'http://168.63.76.32:3128';
console.log('using proxy server %j', proxy);
// WebSocket endpoint for the proxy to connect to
var endpoint = process.argv[2] || 'ws://echo.websocket.org';
var parsed = url.parse(endpoint);
console.log('attempting to connect to WebSocket %j', endpoint);
// create an instance of the `HttpsProxyAgent` class with the proxy server information
var options = url.parse(proxy);
var agent = new HttpsProxyAgent(options);
// finally, initiate the WebSocket connection
var socket = new WebSocket(endpoint, { agent: agent });
socket.on('open', function () {
console.log('"open" event!');
socket.send('hello world');
});
socket.on('message', function (data, flags) {
console.log('"message" event! %j %j', data, flags);
socket.close();
});
```
API
---
### new HttpsProxyAgent(Object options)
The `HttpsProxyAgent` class implements an `http.Agent` subclass that connects
to the specified "HTTP(s) proxy server" in order to proxy HTTPS and/or WebSocket
requests. This is achieved by using the [HTTP `CONNECT` method][CONNECT].
The `options` argument may either be a string URI of the proxy server to use, or an
"options" object with more specific properties:
* `host` - String - Proxy host to connect to (may use `hostname` as well). Required.
* `port` - Number - Proxy port to connect to. Required.
* `protocol` - String - If `https:`, then use TLS to connect to the proxy.
* `headers` - Object - Additional HTTP headers to be sent on the HTTP CONNECT method.
* Any other options given are passed to the `net.connect()`/`tls.connect()` functions.
License
-------
(The MIT License)
Copyright (c) 2013 Nathan Rajlich &lt;nathan@tootallnate.net&gt;
Copyright (c) 2013 Nathan Rajlich <nathan@tootallnate.net>
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
@ -766,10 +530,8 @@ IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
[CONNECT]: http://en.wikipedia.org/wiki/HTTP_tunnel#HTTP_CONNECT_Tunneling
=========================================
END OF https-proxy-agent@5.0.0 AND INFORMATION
END OF https-proxy-agent@7.0.5 AND INFORMATION
%% ip-address@9.0.5 NOTICES AND INFORMATION BEGIN HERE
=========================================
@ -1207,141 +969,11 @@ CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
=========================================
END OF smart-buffer@4.2.0 AND INFORMATION
%% socks-proxy-agent@6.1.1 NOTICES AND INFORMATION BEGIN HERE
%% socks-proxy-agent@8.0.4 NOTICES AND INFORMATION BEGIN HERE
=========================================
socks-proxy-agent
================
### A SOCKS proxy `http.Agent` implementation for HTTP and HTTPS
[![Build Status](https://github.com/TooTallNate/node-socks-proxy-agent/workflows/Node%20CI/badge.svg)](https://github.com/TooTallNate/node-socks-proxy-agent/actions?workflow=Node+CI)
This module provides an `http.Agent` implementation that connects to a
specified SOCKS proxy server, and can be used with the built-in `http`
and `https` modules.
It can also be used in conjunction with the `ws` module to establish a WebSocket
connection over a SOCKS proxy. See the "Examples" section below.
Installation
------------
Install with `npm`:
``` bash
$ npm install socks-proxy-agent
```
Examples
--------
#### TypeScript example
```ts
import https from 'https';
import { SocksProxyAgent } from 'socks-proxy-agent';
const info = {
host: 'br41.nordvpn.com',
userId: 'your-name@gmail.com',
password: 'abcdef12345124'
};
const agent = new SocksProxyAgent(info);
https.get('https://jsonip.org', { agent }, (res) => {
console.log(res.headers);
res.pipe(process.stdout);
});
```
#### `http` module example
```js
var url = require('url');
var http = require('http');
var SocksProxyAgent = require('socks-proxy-agent');
// SOCKS proxy to connect to
var proxy = process.env.socks_proxy || 'socks://127.0.0.1:1080';
console.log('using proxy server %j', proxy);
// HTTP endpoint for the proxy to connect to
var endpoint = process.argv[2] || 'http://nodejs.org/api/';
console.log('attempting to GET %j', endpoint);
var opts = url.parse(endpoint);
// create an instance of the `SocksProxyAgent` class with the proxy server information
var agent = new SocksProxyAgent(proxy);
opts.agent = agent;
http.get(opts, function (res) {
console.log('"response" event!', res.headers);
res.pipe(process.stdout);
});
```
#### `https` module example
```js
var url = require('url');
var https = require('https');
var SocksProxyAgent = require('socks-proxy-agent');
// SOCKS proxy to connect to
var proxy = process.env.socks_proxy || 'socks://127.0.0.1:1080';
console.log('using proxy server %j', proxy);
// HTTP endpoint for the proxy to connect to
var endpoint = process.argv[2] || 'https://encrypted.google.com/';
console.log('attempting to GET %j', endpoint);
var opts = url.parse(endpoint);
// create an instance of the `SocksProxyAgent` class with the proxy server information
var agent = new SocksProxyAgent(proxy);
opts.agent = agent;
https.get(opts, function (res) {
console.log('"response" event!', res.headers);
res.pipe(process.stdout);
});
```
#### `ws` WebSocket connection example
``` js
var WebSocket = require('ws');
var SocksProxyAgent = require('socks-proxy-agent');
// SOCKS proxy to connect to
var proxy = process.env.socks_proxy || 'socks://127.0.0.1:1080';
console.log('using proxy server %j', proxy);
// WebSocket endpoint for the proxy to connect to
var endpoint = process.argv[2] || 'ws://echo.websocket.org';
console.log('attempting to connect to WebSocket %j', endpoint);
// create an instance of the `SocksProxyAgent` class with the proxy server information
var agent = new SocksProxyAgent(proxy);
// initiate the WebSocket connection
var socket = new WebSocket(endpoint, { agent: agent });
socket.on('open', function () {
console.log('"open" event!');
socket.send('hello world');
});
socket.on('message', function (data, flags) {
console.log('"message" event! %j %j', data, flags);
socket.close();
});
```
License
-------
(The MIT License)
Copyright (c) 2013 Nathan Rajlich &lt;nathan@tootallnate.net&gt;
Copyright (c) 2013 Nathan Rajlich <nathan@tootallnate.net>
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
@ -1362,7 +994,7 @@ CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
=========================================
END OF socks-proxy-agent@6.1.1 AND INFORMATION
END OF socks-proxy-agent@8.0.4 AND INFORMATION
%% socks@2.8.3 NOTICES AND INFORMATION BEGIN HERE
=========================================

69
packages/playwright-core/bundles/utils/package-lock.json generated
View File

@ -13,7 +13,7 @@
"debug": "^4.3.4",
"dotenv": "^16.4.5",
"graceful-fs": "4.2.10",
"https-proxy-agent": "5.0.0",
"https-proxy-agent": "7.0.5",
"jpeg-js": "0.4.4",
"mime": "^3.0.0",
"minimatch": "^3.1.2",
@ -23,7 +23,7 @@
"proxy-from-env": "1.1.0",
"retry": "0.12.0",
"signal-exit": "3.0.7",
"socks-proxy-agent": "6.1.1",
"socks-proxy-agent": "8.0.4",
"stack-utils": "2.0.5",
"ws": "8.17.1"
},
@ -130,14 +130,15 @@
}
},
"node_modules/agent-base": {
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/agent-base/-/agent-base-6.0.2.tgz",
"integrity": "sha512-RZNwNclF7+MS/8bDg70amg32dyeZGZxiDuQmZxKLAlQjr3jGyLx+4Kkk58UO7D2QdgFIQCovuSuZESne6RG6XQ==",
"version": "7.1.1",
"resolved": "https://registry.npmjs.org/agent-base/-/agent-base-7.1.1.tgz",
"integrity": "sha512-H0TSyFNDMomMNJQBn8wFV5YC/2eJ+VXECwOadZJT554xP6cODZHPX3H9QMQECxvrgiSOP1pHjy1sMWQVYJOUOA==",
"license": "MIT",
"dependencies": {
"debug": "4"
"debug": "^4.3.4"
},
"engines": {
"node": ">= 6.0.0"
"node": ">= 14"
}
},
"node_modules/balanced-match": {
@ -224,15 +225,16 @@
"integrity": "sha512-9ByhssR2fPVsNZj478qUUbKfmL0+t5BDVyjShtyZZLiK7ZDAArFFfopyOTj0M05wE2tJPisA4iTnnXl2YoPvOA=="
},
"node_modules/https-proxy-agent": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-5.0.0.tgz",
"integrity": "sha512-EkYm5BcKUGiduxzSt3Eppko+PiNWNEpa4ySk9vTC6wDsQJW9rHSa+UhGNJoRYp7bz6Ht1eaRIa6QaJqO5rCFbA==",
"version": "7.0.5",
"resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-7.0.5.tgz",
"integrity": "sha512-1e4Wqeblerz+tMKPIq2EMGiiWW1dIjZOksyHWSUm1rmuvw/how9hBHZ38lAGj5ID4Ik6EdkOw7NmWPy6LAwalw==",
"license": "MIT",
"dependencies": {
"agent-base": "6",
"agent-base": "^7.0.2",
"debug": "4"
},
"engines": {
"node": ">= 6"
"node": ">= 14"
}
},
"node_modules/ip-address": {
@ -382,16 +384,17 @@
}
},
"node_modules/socks-proxy-agent": {
"version": "6.1.1",
"resolved": "https://registry.npmjs.org/socks-proxy-agent/-/socks-proxy-agent-6.1.1.tgz",
"integrity": "sha512-t8J0kG3csjA4g6FTbsMOWws+7R7vuRC8aQ/wy3/1OWmsgwA68zs/+cExQ0koSitUDXqhufF/YJr9wtNMZHw5Ew==",
"version": "8.0.4",
"resolved": "https://registry.npmjs.org/socks-proxy-agent/-/socks-proxy-agent-8.0.4.tgz",
"integrity": "sha512-GNAq/eg8Udq2x0eNiFkr9gRg5bA7PXEWagQdeRX4cPSG+X/8V38v637gim9bjFptMk1QWsCTr0ttrJEiXbNnRw==",
"license": "MIT",
"dependencies": {
"agent-base": "^6.0.2",
"debug": "^4.3.1",
"socks": "^2.6.1"
"agent-base": "^7.1.1",
"debug": "^4.3.4",
"socks": "^2.8.3"
},
"engines": {
"node": ">= 10"
"node": ">= 14"
}
},
"node_modules/sprintf-js": {
@ -523,11 +526,11 @@
}
},
"agent-base": {
"version": "6.0.2",
"resolved": "https://registry.npmjs.org/agent-base/-/agent-base-6.0.2.tgz",
"integrity": "sha512-RZNwNclF7+MS/8bDg70amg32dyeZGZxiDuQmZxKLAlQjr3jGyLx+4Kkk58UO7D2QdgFIQCovuSuZESne6RG6XQ==",
"version": "7.1.1",
"resolved": "https://registry.npmjs.org/agent-base/-/agent-base-7.1.1.tgz",
"integrity": "sha512-H0TSyFNDMomMNJQBn8wFV5YC/2eJ+VXECwOadZJT554xP6cODZHPX3H9QMQECxvrgiSOP1pHjy1sMWQVYJOUOA==",
"requires": {
"debug": "4"
"debug": "^4.3.4"
}
},
"balanced-match": {
@ -588,11 +591,11 @@
"integrity": "sha512-9ByhssR2fPVsNZj478qUUbKfmL0+t5BDVyjShtyZZLiK7ZDAArFFfopyOTj0M05wE2tJPisA4iTnnXl2YoPvOA=="
},
"https-proxy-agent": {
"version": "5.0.0",
"resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-5.0.0.tgz",
"integrity": "sha512-EkYm5BcKUGiduxzSt3Eppko+PiNWNEpa4ySk9vTC6wDsQJW9rHSa+UhGNJoRYp7bz6Ht1eaRIa6QaJqO5rCFbA==",
"version": "7.0.5",
"resolved": "https://registry.npmjs.org/https-proxy-agent/-/https-proxy-agent-7.0.5.tgz",
"integrity": "sha512-1e4Wqeblerz+tMKPIq2EMGiiWW1dIjZOksyHWSUm1rmuvw/how9hBHZ38lAGj5ID4Ik6EdkOw7NmWPy6LAwalw==",
"requires": {
"agent-base": "6",
"agent-base": "^7.0.2",
"debug": "4"
}
},
@ -696,13 +699,13 @@
}
},
"socks-proxy-agent": {
"version": "6.1.1",
"resolved": "https://registry.npmjs.org/socks-proxy-agent/-/socks-proxy-agent-6.1.1.tgz",
"integrity": "sha512-t8J0kG3csjA4g6FTbsMOWws+7R7vuRC8aQ/wy3/1OWmsgwA68zs/+cExQ0koSitUDXqhufF/YJr9wtNMZHw5Ew==",
"version": "8.0.4",
"resolved": "https://registry.npmjs.org/socks-proxy-agent/-/socks-proxy-agent-8.0.4.tgz",
"integrity": "sha512-GNAq/eg8Udq2x0eNiFkr9gRg5bA7PXEWagQdeRX4cPSG+X/8V38v637gim9bjFptMk1QWsCTr0ttrJEiXbNnRw==",
"requires": {
"agent-base": "^6.0.2",
"debug": "^4.3.1",
"socks": "^2.6.1"
"agent-base": "^7.1.1",
"debug": "^4.3.4",
"socks": "^2.8.3"
}
},
"sprintf-js": {

4
packages/playwright-core/bundles/utils/package.json
View File

@ -14,7 +14,7 @@
"debug": "^4.3.4",
"dotenv": "^16.4.5",
"graceful-fs": "4.2.10",
"https-proxy-agent": "5.0.0",
"https-proxy-agent": "7.0.5",
"jpeg-js": "0.4.4",
"mime": "^3.0.0",
"minimatch": "^3.1.2",
@ -24,7 +24,7 @@
"proxy-from-env": "1.1.0",
"retry": "0.12.0",
"signal-exit": "3.0.7",
"socks-proxy-agent": "6.1.1",
"socks-proxy-agent": "8.0.4",
"stack-utils": "2.0.5",
"ws": "8.17.1"
},

34
packages/playwright-core/src/server/fetch.ts
View File

@ -20,7 +20,6 @@ import http from 'http';
import https from 'https';
import type { Readable, TransformCallback } from 'stream';
import { pipeline, Transform } from 'stream';
import url from 'url';
import zlib from 'zlib';
import type { HTTPCredentials } from '../../types/types';
import { TimeoutSettings } from '../common/timeoutSettings';
@ -489,11 +488,11 @@ export abstract class APIRequestContext extends SdkObject {
// happy eyeballs don't emit lookup and connect events, so we use our custom ones
const happyEyeBallsTimings = timingForSocket(socket);
dnsLookupAt = happyEyeBallsTimings.dnsLookupAt;
tcpConnectionAt = happyEyeBallsTimings.tcpConnectionAt;
tcpConnectionAt ??= happyEyeBallsTimings.tcpConnectionAt;
// non-happy-eyeballs sockets
socket.on('lookup', () => { dnsLookupAt = monotonicTime(); });
socket.on('connect', () => { tcpConnectionAt = monotonicTime(); });
socket.on('connect', () => { tcpConnectionAt ??= monotonicTime(); });
socket.on('secureConnect', () => {
tlsHandshakeAt = monotonicTime();
@ -509,11 +508,21 @@ export abstract class APIRequestContext extends SdkObject {
}
});
// when using socks proxy, having the socket means the connection got established
if (agent instanceof SocksProxyAgent)
tcpConnectionAt ??= monotonicTime();
serverIPAddress = socket.remoteAddress;
serverPort = socket.remotePort;
});
request.on('finish', () => { requestFinishAt = monotonicTime(); });
// http proxy
request.on('proxyConnect', () => {
tcpConnectionAt ??= monotonicTime();
});
progress.log(`→ ${options.method} ${url.toString()}`);
if (options.headers) {
for (const [name, value] of Object.entries(options.headers))
@ -680,17 +689,16 @@ export class GlobalAPIRequestContext extends APIRequestContext {
}
export function createProxyAgent(proxy: types.ProxySettings) {
const proxyOpts = url.parse(proxy.server);
if (proxyOpts.protocol?.startsWith('socks')) {
return new SocksProxyAgent({
host: proxyOpts.hostname,
port: proxyOpts.port || undefined,
});
}
const proxyURL = new URL(proxy.server);
if (proxyURL.protocol?.startsWith('socks'))
return new SocksProxyAgent(proxyURL);
if (proxy.username)
proxyOpts.auth = `${proxy.username}:${proxy.password || ''}`;
// TODO: We should use HttpProxyAgent conditional on proxyOpts.protocol instead of always using CONNECT method.
return new HttpsProxyAgent(proxyOpts);
proxyURL.username = proxy.username;
if (proxy.password)
proxyURL.password = proxy.password;
// TODO: We should use HttpProxyAgent conditional on proxyURL.protocol instead of always using CONNECT method.
return new HttpsProxyAgent(proxyURL);
}
function toHeadersArray(rawHeaders: string[]): types.HeadersArray {

2
packages/playwright-core/src/server/socksClientCertificatesInterceptor.ts
View File

@ -98,7 +98,7 @@ class SocksProxyConnection {
async connect() {
if (this.socksProxy.proxyAgentFromOptions)
this.target = await this.socksProxy.proxyAgentFromOptions.callback(new EventEmitter() as any, { host: rewriteToLocalhostIfNeeded(this.host), port: this.port, secureEndpoint: false });
this.target = await this.socksProxy.proxyAgentFromOptions.connect(new EventEmitter() as any, { host: rewriteToLocalhostIfNeeded(this.host), port: this.port, secureEndpoint: false });
else
this.target = await createSocket(rewriteToLocalhostIfNeeded(this.host), this.port);

2
packages/playwright-core/src/utils/network.ts
View File

@ -50,7 +50,7 @@ export function httpRequest(params: HTTPRequestParams, onResponse: (r: http.Inco
const proxyURL = getProxyForUrl(params.url);
if (proxyURL) {
const parsedProxyURL = url.parse(proxyURL);
const parsedProxyURL = new URL(proxyURL);
if (params.url.startsWith('http:')) {
options = {
path: parsedUrl.href,

23
tests/library/har.spec.ts
View File

@ -24,9 +24,9 @@ import type { Log } from '../../packages/trace/src/har';
import { parseHar } from '../config/utils';
const { createHttp2Server } = require('../../packages/playwright-core/lib/utils');
async function pageWithHar(contextFactory: (options?: BrowserContextOptions) => Promise<BrowserContext>, testInfo: any, options: { outputPath?: string } & Partial<Pick<BrowserContextOptions['recordHar'], 'content' | 'omitContent' | 'mode'>> = {}) {
async function pageWithHar(contextFactory: (options?: BrowserContextOptions) => Promise<BrowserContext>, testInfo: any, options: { outputPath?: string, proxy?: BrowserContextOptions['proxy'] } & Partial<Pick<BrowserContextOptions['recordHar'], 'content' | 'omitContent' | 'mode'>> = {}) {
const harPath = testInfo.outputPath(options.outputPath || 'test.har');
const context = await contextFactory({ recordHar: { path: harPath, ...options }, ignoreHTTPSErrors: true });
const context = await contextFactory({ recordHar: { path: harPath, ...options }, ignoreHTTPSErrors: true, proxy: options.proxy });
const page = await context.newPage();
return {
page,
@ -858,6 +858,25 @@ it('should respect minimal mode for API Requests', async ({ contextFactory, serv
expect(entry.response.bodySize).toBe(-1);
});
it('should include timings when using http proxy', async ({ contextFactory, server, proxyServer }, testInfo) => {
proxyServer.forwardTo(server.PORT, { allowConnectRequests: true });
const { page, getLog } = await pageWithHar(contextFactory, testInfo, { proxy: { server: `localhost:${proxyServer.PORT}` } });
const response = await page.request.get(server.EMPTY_PAGE);
await response.body();
await expect(response).toBeOK();
const log = await getLog();
expect(log.entries[0].timings.connect).toBeGreaterThan(0);
});
it('should include timings when using socks proxy', async ({ contextFactory, server, socksPort }, testInfo) => {
const { page, getLog } = await pageWithHar(contextFactory, testInfo, { proxy: { server: `socks5://localhost:${socksPort}` } });
const response = await page.request.get(server.EMPTY_PAGE);
await response.body();
await expect(response).toBeOK();
const log = await getLog();
expect(log.entries[0].timings.connect).toBeGreaterThan(0);
});
it('should include redirects from API request', async ({ contextFactory, server }, testInfo) => {
server.setRedirect('/redirect-me', '/simple.json');
const { page, getLog } = await pageWithHar(contextFactory, testInfo);