Orange-OpenSource/hurl
1
1
Fork 0
mirror of https://github.com/Orange-OpenSource/hurl.git synced 2024-12-24 03:21:40 +03:00
Code Issues Packages Projects Releases Wiki Activity
458c780238
hurl/docs/asserting-response.md

769 lines
21 KiB
Markdown
Raw Blame History

# Asserting Response
## Version - Status
Expected protocol version and status code of the HTTP response.
Protocol version is one of `HTTP/1.0`, `HTTP/1.1`, `HTTP/2` or
`HTTP/*`; `HTTP/*` describes any version. Note that there are no status text following the status code.
```hurl
GET https://example.org/404.html
HTTP/1.1 404
```
Wildcard keywords (`HTTP/*`, `*`) can be used to disable tests on protocol version and status:
```hurl
GET https://example.org/api/pets
HTTP/1.0 *
# Check that response status code is > 400 and <= 500
[Asserts]
status > 400
status <= 500
```
## Headers
Optional list of the expected HTTP response headers that must be in the received response.
A header consists of a name, followed by a `:` and a value.
For each expected header, the received response headers are checked. If the received header is not equal to the expected,
or not present, an error is raised. Note that the expected headers list is not fully descriptive: headers present in the response
and not in the expected list doesn't raise error.
```hurl
# Check that user toto is redirected to home after login.
POST https://example.org/login
[FormParams]
user: toto
password: 12345678
HTTP/1.1 302
Location: https://example.org/home
```
> Quotes in the header value are part of the value itself.
>
> This is used by the [ETag](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag) Header
> ```
> ETag: W/"<etag_value>"
> ETag: "<etag_value>"
> ```
Testing duplicated headers is also possible.
For example with the `Set-Cookie` header:
```
Set-Cookie: theme=light
Set-Cookie: sessionToken=abc123; Expires=Wed, 09 Jun 2021 10:18:14 GMT
```
You can either test the two header values:
```hurl
GET https://example.org/index.html
Host: example.net
HTTP/1.0 200
Set-Cookie: theme=light
Set-Cookie: sessionToken=abc123; Expires=Wed, 09 Jun 2021 10:18:14 GMT
```
Or only one:
```hurl
GET https://example.org/index.html
Host: example.net
HTTP/1.0 200
Set-Cookie: theme=light
```
If you want to test specifically the number of headers returned for a given header name, or
if you want to test header value with [predicates] (like `startsWith`, `contains`, `exists`)
you can use the explicit [header assert].
## Asserts
Optional list of assertions on the HTTP response. Assertions can describe checks
on status code, on the received body (or part of it) and on response headers.
Structure of an assert:
<div class="schema-container schema-container u-font-size-1 u-font-size-2-sm u-font-size-3-md">
<div class="schema">
<span class="schema-token schema-color-2">jsonpath "$.book"<span class="schema-label">query</span></span>
<span class="schema-token schema-color-1">contains<span class="schema-label">predicate type</span></span>
<span class="schema-token schema-color-3">"Dune"<span class="schema-label">predicate value</span></span>
</div>
</div>
<div class="schema-container schema-container u-font-size-1 u-font-size-2-sm u-font-size-3-md">
<div class="schema">
<span class="schema-token schema-color-2">body<span class="schema-label">query</span></span>
<span class="schema-token schema-color-1">matches<span class="schema-label">predicate type</span></span>
<span class="schema-token schema-color-3">/\d{4}-\d{2}-\d{2}/<span class="schema-label">predicate value</span></span>
</div>
</div>
An assert consists of a query followed by a predicate. The format of the query
is shared with [captures], and can be one of :
- [`status`](#status-assert)
- [`header`](#header-assert)
- [`url`](#url-assert)
- [`cookie`](#cookie-assert)
- [`body`](#body-assert)
- [`bytes`](#bytes-assert)
- [`xpath`](#xpath-assert)
- [`jsonpath`](#jsonpath-assert)
- [`regex`](#regex-assert)
- [`sha256`](#sha-256-assert)
- [`md5`](#md5-assert)
- [`variable`](#variable-assert)
- [`duration`](#duration-assert)
Queries, as in captures, can be refined with subqueries. [`count`] subquery can be used
with various predicates to add tests on collections sizes.
### Predicates
Predicates consist of a predicate function, and a predicate value. Predicate functions are:
- `==` (`equals`): check equality of query and predicate value
- `!=`: check that query and predicate value are different
- `>` (`greaterThan`): check that query number is greater than predicate value
- `>=` (`greaterThanOrEquals`): check that query number is greater than or equal to the predicate value
- `<` (`lessThan`): check that query number is less than that predicate value
- `<=` (`lessThanOrEquals`): check that query number is less than or equal to the predicate value
- `startsWith`: check that query starts with the predicate value (query can return a string or a binary content)
- `endsWith`: check that query ends with the predicate value (query can return a string or a binary content)
- `contains`: check that query contains the predicate value (query can return a string or a binary content)
- `includes`: check that query collections includes the predicate value
- `matches`: check that query string matches the regex pattern described by the predicate value
- `exists`: check that query returns a value
- `isInteger`: check that query returns an integer
- `isFloat`: check that query returns a float
- `isBoolean`: check that query returns a boolean
- `isString`: check that query returns a string
- `isCollection`: check that query returns a collection
Each predicate can be negated by prefixing it with `not` (for instance, `not contains` or `not exists`)
<div class="schema-container schema-container u-font-size-1 u-font-size-2-sm u-font-size-3-md">
<div class="schema">
<span class="schema-token schema-color-2">jsonpath "$.book"<span class="schema-label">query</span></span>
<span class="schema-token schema-color-1">not contains<span class="schema-label">predicate type</span></span>
<span class="schema-token schema-color-3">"Dune"<span class="schema-label">predicate value</span></span>
</div>
</div>
A predicate values is typed, and can be a string, a boolean, a number, a bytestream, `null` or a collection. Note that
`"true"` is a string, whereas `true` is a boolean.
For instance, to test the presence of a h1 node in an HTML response, the following assert can be used:
```hurl
GET https://example.org/home
HTTP/1.1 200
[Asserts]
xpath "boolean(count(//h1))" == true
xpath "//h1" exists # Equivalent but simpler
```
As the XPath query `boolean(count(//h1))` returns a boolean, the predicate value in the assert must be either
`true` or `false` without double quotes. On the other side, say you have an article node and you want to check the value of some
[data attributes]:
```xml
<article
id="electric-cars"
data-visible="true"
...
</article>
```
The following assert will check the value of the `data-visible` attribute:
```hurl
GET https://example.org/home
HTTP/1.1 200
[Asserts]
xpath "string(//article/@data-visible)" == "true"
```
In this case, the XPath query `string(//article/@data-visible)` returns a string, so the predicate value must be a
string.
The predicate function `equals` can work with string, number or boolean while `matches`, `startWith` and `contains` work
only on string. If a query returns a number, a `contains` predicate will raise a runner error.
```hurl
# A really well tested web page...
GET https://example.org/home
HTTP/1.1 200
[Asserts]
header "Content-Type" contains "text/html"
header "Last-Modified" == "Wed, 21 Oct 2015 07:28:00 GMT"
xpath "//h1" exists # Check we've at least one h1
xpath "normalize-space(//h1)" contains "Welcome"
xpath "//h2" count == 13
xpath "string(//article/@data-id)" startsWith "electric"
```
### Status assert
Check the received HTTP response status code. Status assert consists of the keyword `status` followed by a predicate
function and value.
```hurl
GET https://example.org
HTTP/1.1 *
[Asserts]
status < 300
```
### Header assert
Check the value of a received HTTP response header. Header assert consists of the keyword `header` followed by a predicate
function and value.
```hurl
GET https://example.org
HTTP/1.1 302
[Asserts]
header "Location" contains "www.example.net"
```
If there are multiple headers with the same name, the header assert returns a collection, so `count`, `includes` can be
used in this case to test the header list.
Let's say we have this request and response:
```
> GET /hello HTTP/1.1
> Host: example.org
> Accept: */*
> User-Agent: hurl/1.8.0-SNAPSHOT
>
* Response: (received 12 bytes in 11 ms)
*
< HTTP/1.0 200 OK
< Vary: Content-Type
< Vary: User-Agent
< Content-Type: text/html; charset=utf-8
< Content-Length: 12
< Server: Flask Server
< Date: Fri, 07 Oct 2022 20:53:35 GMT
```
One can use explicit header asserts:
```hurl
GET https://example.org/hello
HTTP/* 200
[Asserts]
header "Vary" count == 2
header "Vary" includes "User-Agent"
header "Vary" includes "Content-Type"
```
Or implicit header asserts:
```hurl
GET https://example.org/hello
HTTP/* 200
Vary: User-Agent
Vary: Content-Type
```
### URL assert
Check the last fetched URL. This is most meaningful if you have told Hurl to follow redirection (see [`[Options]`section][options] or
[`--location` option]). URL assert consists of the keyword `url` followed by a predicate function and value.
```hurl
GET https://example.org/redirecting
[Options]
location: true
HTTP/* 200
[Asserts]
url == "https://example.org/redirected"
```
### Cookie assert
Check value or attributes of a [`Set-Cookie`] response header. Cookie assert
consists of the keyword `cookie`, followed by the cookie name (and optionally a
cookie attribute), a predicate function and value.
Cookie attributes value can be checked by using the following format:
`<cookie-name>[cookie-attribute]`. The following attributes are supported: `Value`,
`Expires`, `Max-Age`, `Domain`, `Path`, `Secure`, `HttpOnly` and `SameSite`.
```hurl
GET http://localhost:8000/cookies/set
HTTP/1.0 200
# Explicit check of Set-Cookie header value. If the attributes are
# not in this exact order, this assert will fail.
Set-Cookie: LSID=DQAAAKEaem_vYg; Expires=Wed, 13 Jan 2021 22:23:01 GMT; Secure; HttpOnly; Path=/accounts; SameSite=Lax;
Set-Cookie: HSID=AYQEVnDKrdst; Domain=.localhost; Expires=Wed, 13 Jan 2021 22:23:01 GMT; HttpOnly; Path=/
Set-Cookie: SSID=Ap4PGTEq; Domain=.localhost; Expires=Wed, 13 Jan 2021 22:23:01 GMT; Secure; HttpOnly; Path=/
# Using cookie assert, one can check cookie value and various attributes.
[Asserts]
cookie "LSID" == "DQAAAKEaem_vYg"
cookie "LSID[Value]" == "DQAAAKEaem_vYg"
cookie "LSID[Expires]" exists
cookie "LSID[Expires]" contains "Wed, 13 Jan 2021"
cookie "LSID[Max-Age]" not exists
cookie "LSID[Domain]" not exists
cookie "LSID[Path]" == "/accounts"
cookie "LSID[Secure]" exists
cookie "LSID[HttpOnly]" exists
cookie "LSID[SameSite]" equals "Lax"
```
> `Secure` and `HttpOnly` attributes can only be tested with `exists` or `not exists` predicates
> to reflect the [Set-Cookie header] semantics (in other words, queries `<cookie-name>[HttpOnly]`
> and `<cookie-name>[Secure]` don't return boolean).
### Body assert
Check the value of the received HTTP response body when decoded as a string.
Body assert consists of the keyword `body` followed by a predicate function and
value. The encoding used to decode the body is based on the `charset` value in the
`Content-Type` header response.
```hurl
GET https://example.org
HTTP/1.1 200
[Asserts]
body contains "<h1>Welcome!</h1>"
```
> Precise the encoding used to decode the text body.
### Bytes assert
Check the value of the received HTTP response body as a bytestream. Body assert
consists of the keyword `bytes` followed by a predicate function and value.
```hurl
GET https://example.org/data.bin
HTTP/* 200
[Asserts]
bytes startsWith hex,efbbbf;
```
### XPath assert
Check the value of a [XPath] query on the received HTTP body decoded as a string.
Currently, only XPath 1.0 expression can be used. Body assert consists of the
keyword `xpath` followed by a predicate function and value. Values can be string,
boolean or number depending on the XPath query.
Let's say we want to check this HTML response:
```plain
$ curl -v https://example.org
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=UTF-8
...
<!doctype html>
<html>
<head>
<title>Example Domain</title>
...
</head>
<body>
<div>
<h1>Example</h1>
<p>This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.</p>
<p><a href="https://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>
```
With Hurl, we can write multiple XPath asserts describing the DOM content:
```hurl
GET https://example.org
HTTP/1.1 200
Content-Type: text/html; charset=UTF-8
[Asserts]
xpath "string(/html/head/title)" contains "Example" # Check title
xpath "count(//p)" == 2 # Check the number of <p>
xpath "//p" count == 2 # Similar assert for <p>
xpath "boolean(count(//h2))" == false # Check there is no <h2>
xpath "//h2" not exists # Similar assert for <h2>
```
XML Namespaces are also supported. Let's say you want to check this XML response:
```xml
<?xml version="1.0"?>
<!-- both namespace prefixes are available throughout -->
<bk:book xmlns:bk='urn:loc.gov:books'
xmlns:isbn='urn:ISBN:0-395-36341-6'>
<bk:title>Cheaper by the Dozen</bk:title>
<isbn:number>1568491379</isbn:number>
</bk:book>
```
This XML response can be tested with the following Hurl file:
```hurl
GET http://localhost:8000/assert-xpath
HTTP/1.0 200
[Asserts]
xpath "string(//bk:book/bk:title)" == "Cheaper by the Dozen"
xpath "string(//*[name()='bk:book']/*[name()='bk:title'])" == "Cheaper by the Dozen"
xpath "string(//*[local-name()='book']/*[local-name()='title'])" == "Cheaper by the Dozen"
xpath "string(//bk:book/isbn:number)" == "1568491379"
xpath "string(//*[name()='bk:book']/*[name()='isbn:number'])" == "1568491379"
xpath "string(//*[local-name()='book']/*[local-name()='number'])" == "1568491379"
```
The XPath expressions `string(//bk:book/bk:title)` and `string(//bk:book/isbn:number)` are written with `bk` and `isbn`
namespaces.
> For convenience, the first default namespace can be used with `_`
### JSONPath assert
Check the value of a [JSONPath] query on the received HTTP body decoded as a JSON
document. Body assert consists of the keyword `jsonpath` followed by a predicate
function and value.
Let's say we want to check this JSON response:
```plain
curl -v http://httpbin.org/json
< HTTP/1.1 200 OK
< Content-Type: application/json
...
{
"slideshow": {
"author": "Yours Truly",
"date": "date of publication",
"slides": [
{
"title": "Wake up to WonderWidgets!",
"type": "all"
},
...
],
"title": "Sample Slide Show"
}
}
```
With Hurl, we can write multiple JSONPath asserts describing the DOM content:
```hurl
GET http://httpbin.org/json
HTTP/1.1 200
[Asserts]
jsonpath "$.slideshow.author" == "Yours Truly"
jsonpath "$.slideshow.slides[0].title" contains "Wonder"
jsonpath "$.slideshow.slides" count == 2
jsonpath "$.slideshow.date" != null
jsonpath "$.slideshow.slides[*].title" includes "Mind Blowing!"
```
> Explain that the value selected by the JSONPath is coerced to a string when only
> one node is selected.
In `matches` predicates, metacharacters beginning with a backslash (like `\d`, `\s`) must be escaped.
Alternatively, `matches` predicate support [Javascript-like Regular expression syntax] to enhance
the readability:
```hurl
GET https://sample.org/hello
HTTP/1.0 200
[Asserts]
# Predicate value with matches predicate:
jsonpath "$.date" matches "^\\d{4}-\\d{2}-\\d{2}$"
jsonpath "$.name" matches "Hello [a-zA-Z]+!"
# Equivalent syntax:
jsonpath "$.date" matches /^\d{4}-\d{2}-\d{2}$/
jsonpath "$.name" matches /Hello [a-zA-Z]+!/
```
### Regex assert
Check that the HTTP received body, decoded as text, matches a regex pattern.
```hurl
GET https://sample.org/hello
HTTP/1.0 200
[Asserts]
regex "^\\d{4}-\\d{2}-\\d{2}$" == "2018-12-31"
```
### SHA-256 assert
Check response body [SHA-256] hash.
```hurl
GET https://example.org/data.tar.gz
HTTP/* *
[Asserts]
sha256 == hex,039058c6f2c0cb492c533b0a4d14ef77cc0f78abccced5287d84a1a2011cfb81;
```
### MD5 assert
Check response body [MD5] hash.
```hurl
GET https://example.org/data.tar.gz
HTTP/* *
[Asserts]
md5 == hex,ed076287532e86365e841e92bfc50d8c;
```
### Variable assert
```hurl
# Test that the XML endpoint return 200 pets
GET https://example.org/api/pets
HTTP/* 200
[Captures]
pets: xpath "//pets"
[Asserts]
variable "pets" count == 200
```
### Duration assert
Check the total duration (sending plus receiving time) of the HTTP transaction.
```hurl
GET https://sample.org/helloworld
HTTP/1.0 200
[Asserts]
duration < 1000 # Check that response time is less than one second
```
## Filters
Optionally, asserts can be refined using filters `count` and `regex`.
### Count filter
```hurl
GET https://pets.org/cats/cutest
HTTP/1.0 200
[Asserts]
jsonpath "$.cats" count == 12
```
### Regex filter
```hurl
GET https://pets.org/cats/cutest
HTTP/1.0 200
# Cat name are structured like this `meow + id`: for instance `meow123456`
[Asserts]
jsonpath "$.cats[0].name" regex /meow(\d+)/ == "123456"
```
## Body
Optional assertion on the received HTTP response body. Body section can be seen
as syntactic sugar over [body asserts] (with `equals` predicate function). If the
body of the response is a [JSON] string or a [XML] string, the body assertion can
be directly inserted without any modification. For a text based body that is not JSON nor XML,
one can use multiline string that starts with <code>&#96;&#96;&#96;</code> and ends
with <code>&#96;&#96;&#96;</code>. For a precise byte control of the response body,
a [Base64] encoded string or an input file can be used to describe exactly
the body byte content to check.
### JSON body
```hurl
# Get a doggy thing:
GET https://example.org/api/dogs/{{dog-id}}
HTTP/1.1 200
{
"id": 0,
"name": "Frieda",
"picture": "images/scottish-terrier.jpeg",
"age": 3,
"breed": "Scottish Terrier",
"location": "Lisco, Alabama"
}
```
### XML body
~~~hurl
GET https://example.org/api/catalog
HTTP/1.1 200
<?xml version="1.0" encoding="UTF-8"?>
<catalog>
<book id="bk101">
<author>Gambardella, Matthew</author>
<title>XML Developer's Guide</title>
<genre>Computer</genre>
<price>44.95</price>
<publish_date>2000-10-01</publish_date>
<description>An in-depth look at creating applications with XML.</description>
</book>
</catalog>
~~~
### Multiline string body
~~~hurl
GET https://example.org/models
HTTP/1.1 200
```
Year,Make,Model,Description,Price
1997,Ford,E350,"ac, abs, moon",3000.00
1999,Chevy,"Venture ""Extended Edition""","",4900.00
1999,Chevy,"Venture ""Extended Edition, Very Large""",,5000.00
1996,Jeep,Grand Cherokee,"MUST SELL! air, moon roof, loaded",4799.00
```
~~~
The standard usage of a multiline string is :
~~~
```
line1
line2
line3
```
~~~
is evaluated as "line1\nline2\nline3\n".
To construct an empty string :
~~~
```
```
~~~
or
~~~
``````
~~~
Finally, multiline can be used without any newline:
~~~
```line```
~~~
is evaluated as "line".
### Base64 body
Base64 body assert starts with `base64,` and end with `;`. MIME's Base64 encoding
is supported (newlines and white spaces may be present anywhere but are to be
ignored on decoding), and `=` padding characters might be added.
```hurl
GET https://example.org
HTTP/1.1 200
base64,TG9yZW0gaXBzdW0gZG9sb3Igc2l0IGFtZXQsIGNvbnNlY3RldHVyIG
FkaXBpc2NpbmcgZWxpdC4gSW4gbWFsZXN1YWRhLCBuaXNsIHZlbCBkaWN0dW0g
aGVuZHJlcml0LCBlc3QganVzdG8gYmliZW5kdW0gbWV0dXMsIG5lYyBydXRydW
0gdG9ydG9yIG1hc3NhIGlkIG1ldHVzLiA=;
```
### File body
To use the binary content of a local file as the body response assert, file body
can be used. File body starts with `file,` and ends with `;``
```hurl
GET https://example.org
HTTP/1.1 200
file,data.bin;
```
File are relative to the input Hurl file, and cannot contain implicit parent
directory (`..`). You can use [`--file-root` option] to specify the root directory
of all file nodes.
[predicates]: #predicates
[header assert]: #header-assert
[captures]: /docs/capturing-response.md#query
[data attributes]: https://developer.mozilla.org/en-US/docs/Learn/HTML/Howto/Use_data_attributes
[`Set-Cookie`]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie
[Set-Cookie header]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie
[XPath]: https://en.wikipedia.org/wiki/XPath
[JSONPath]: https://goessner.net/articles/JsonPath/
[body asserts]: #body-assert
[JSON]: https://www.json.org
[XML]: https://en.wikipedia.org/wiki/XML
[Base64]: https://en.wikipedia.org/wiki/Base64
[`--file-root` option]: /docs/manual.md#file-root
[`count`]: /docs/capturing-response.md#count-subquery
[Javascript-like Regular expression syntax]: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Regular_Expressions
[MD5]: https://en.wikipedia.org/wiki/MD5
[SHA-256]: https://en.wikipedia.org/wiki/SHA-2
[options]: /docs/request.md#options
[`--location` option]: /docs/manual.md#location
Reference in New Issue View Git Blame Copy Permalink