hurl/docs/capturing-response.md

Capturing Response

Captures

Captures are optional values captured from the HTTP response, in a named variable. Captures can be the response status code, part of or the entire the body, and response headers.

Captured variables are available through a run session; each new value of a given variable overrides the last value.

Captures allow using data from one request in another request, when working with CSRF tokens for instance. Variables can also be initialized at the start of the session, by passing variable values, or can be used in templates.

# An example to show how to pass a CSRF token from one request
# to another:

# First GET request to get CSRF token value:
GET https://example.org

HTTP/1.1 200
# Capture the CSRF token value from html body.
[Captures]
csrf_token: xpath "normalize-space(//meta[@name='_csrf_token']/@content)"

# Do the login !
POST https://acmecorp.net/login?user=toto&password=1234
X-CSRF-TOKEN: {{csrf_token}}

HTTP/1.1 302

Structure of a capture:

my_varvariable : xpath "string(//h1)"query

A capture consists of a variable name, followed by : and a query. The captures section starts with [Captures].

Query

Query can be of the following type:

• status
• header
• cookie
• body
• bytes
• xpath
• jsonpath
• regex
• variable
• duration

Status capture

Capture the received HTTP response status code. Status capture consists of a variable name, followed by a :, and the keyword status.

GET https://example.org

HTTP/1.1 200
[Captures]
my_status: status

Header capture

Capture a header from the received HTTP response headers. Header capture consists of a variable name, followed by a :, then the keyword header and a header name.

POST https://example.org/login
[FormParams]
user: toto
password: 12345678

HTTP/1.1 302
[Captures]
next_url: header "Location"

Cookie capture

Capture a Set-Cookie header from the received HTTP response headers. Cookie capture consists of a variable name, followed by a :, then the keyword cookie and a cookie name.

GET https://example.org/cookies/set

HTTP/1.0 200
[Captures]
session-id: cookie "LSID"

Cookie attributes value can also be captured by using the following format: <cookie-name>[cookie-attribute]. The following attributes are supported: Value, Expires, Max-Age, Domain, Path, Secure, HttpOnly and SameSite.

GET https://example.org/cookies/set

HTTP/1.0 200
[Captures]
value1: cookie "LSID"
value2: cookie "LSID[Value]"     # Equivalent to the previous capture
expires: cookie "LSID[Expires]"
max-age: cookie "LSID[Max-Age]"
domain: cookie "LSID[Domain]"
path: cookie "LSID[Path]"
secure: cookie "LSID[Secure]"
http-only: cookie "LSID[HttpOnly]"
same-site: cookie "LSID[SameSite]"

Body capture

Capture the entire body (decoded as text) from the received HTTP response

GET https://example.org/home

HTTP/1.1 200
[Captures]
my_body: body

Bytes capture

Capture the entire body (as a raw bytestream) from the received HTTP response

GET https://example.org/data.bin

HTTP/1.1 200
[Captures]
my_data: bytes

XPath capture

Capture a XPath query from the received HTTP body decoded as a string. Currently, only XPath 1.0 expression can be used.

GET https://example.org/home

# Capture the identifier from the dom node <div id="pet0">5646eaf23</div
HTTP/1.1 200
[Captures]
ped-id: xpath "normalize-space(//div[@id='pet0'])"

# Open the captured page.
GET https://example.org/home/pets/{{pet-id}}

HTTP/1.1 200

XPath captures are not limited to node values (like string, or boolean); any valid XPath can be captured and asserted with variable asserts.

# Test that the XML endpoint return 200 pets
GET https://example.org/api/pets
HTTP/* 200
[Captures]
pets: xpath "//pets"
[Asserts]
variable "pets" count == 200

JSONPath capture

Capture a JSONPath query from the received HTTP body.

POST https://example.org/api/contact
[FormParams]
token: {{token}}
email: toto@rookie.net

HTTP/1.1 200
[Captures]
contact-id: jsonpath "$['id']"

Explain that the value selected by the JSONPath is coerced to a string when only one node is selected.

As with XPath captures, JSONPath captures can be anything from string, number, to object and collections. For instance, if we have a JSON endpoint that returns the following JSON:

{
  "a_null": null,
  "an_object": {
    "id": "123"
  },
  "a_list": [
    1,
    2,
    3
  ],
  "an_integer": 1,
  "a float": 1.1,
  "a_bool": true,
  "a_string": "hello"
}

We can capture the following paths:

GET https://example.org/captures-json

HTTP/1.0 200
[Captures]
an_object:  jsonpath "$['an_object']"
a_list:     jsonpath "$['a_list']"
a_null:     jsonpath "$['a_null']"
an_integer: jsonpath "$['an_integer']"
a_float:    jsonpath "$['a_float']"
a_bool:     jsonpath "$['a_bool']"
a_string:   jsonpath "$['a_string']"
all:        jsonpath "$"

Regex capture

Capture a regex pattern from the HTTP received body, decoded as text.

GET https://example.org/helloworld

HTTP/1.0 200
[Captures]
id_a: regex "id_a:([0-9]+)!"
id_b: regex "id_b:(\\d+)!"
name: regex "Hello ([a-zA-Z]+)!"

Pattern of the regex query must have at least one capture group, otherwise the capture will fail. Metacharacters beginning with a backslash in the pattern (like \d, \s) must be escaped: regex "(\\d+)!" will capture one or more digit.

Variable capture

Capture the value of a variable into another.

GET https://example.org/helloworld

HTTP/1.0 200
[Captures]
in: body
name: variable "in" regex "Hello ([a-zA-Z]+)!"

Duration capture

Capture the response time of the request in ms.

GET https://example.org/helloworld

HTTP/1.0 200
[Captures]
duration_in_ms: duration

Subquery

Optionally, query can be refined using subqueries regex and count.

my_varvariable : xpath "string(//h1)"query regex "(\\d+)"subquery (optional)

Regex subquery

GET https://pets.org/cats/cutest

HTTP/1.0 200
# Cat name are structured like this `meow + id`: for instance `meow123456` 
[Captures]
id: jsonpath "$.cats[0].name" regex "meow(\\d+)"

Pattern of the regex subquery must have at least one capture group, otherwise the capture will fail. Metacharacters beginning with a backslash in the pattern (like \d, \s) must be escaped: regex "(\\d+)!" will capture one or more digit.

Count subquery

Returns the count of a collection.

GET https://pets.org/cats/cutest

HTTP/1.0 200
[Captures]
cats_size: jsonpath "$.cats" count