Orange-OpenSource/hurl
1
1
Fork 0
mirror of https://github.com/Orange-OpenSource/hurl.git synced 2024-11-26 11:43:08 +03:00
Code Issues Packages Projects Releases Wiki Activity
c11c5ed16e
hurl/docs/asserting-response.md

21 KiB
Raw Blame History

Asserting Response

Version - Status

Expected protocol version and status code of the HTTP response.

Protocol version is one of HTTP/1.0, HTTP/1.1, HTTP/2 or HTTP/*; HTTP/* describes any version. Note that there are no status text following the status code.

GET https://example.org/404.html

HTTP/1.1 404

Wildcard keywords (HTTP/*, *) can be used to disable tests on protocol version and status:

GET https://example.org/api/pets

HTTP/1.0 *
# Check that response status code is > 400 and <= 500
[Asserts]
status > 400
status <= 500

Headers

Optional list of the expected HTTP response headers that must be in the received response.

A header consists of a name, followed by a : and a value.

For each expected header, the received response headers are checked. If the received header is not equal to the expected, or not present, an error is raised. Note that the expected headers list is not fully descriptive: headers present in the response and not in the expected list doesn't raise error.

# Check that user toto is redirected to home after login.
POST https://example.org/login
[FormParams]
user: toto
password: 12345678

HTTP/1.1 302
Location: https://example.org/home

Quotes in the header value are part of the value itself.

This is used by the ETag Header

ETag: W/"<etag_value>"
ETag: "<etag_value>"

Testing duplicated headers is also possible.

For example with the Set-Cookie header:

Set-Cookie: theme=light
Set-Cookie: sessionToken=abc123; Expires=Wed, 09 Jun 2021 10:18:14 GMT

You can either test the two header values:

GET https://example.org/index.html
Host: example.net

HTTP/1.0 200
Set-Cookie: theme=light
Set-Cookie: sessionToken=abc123; Expires=Wed, 09 Jun 2021 10:18:14 GMT

Or only one:

GET https://example.org/index.html 
Host: example.net

HTTP/1.0 200
Set-Cookie: theme=light

If you want to test specifically the number of headers returned for a given header name, or if you want to test header value with predicates (like startsWith, contains, exists) you can use the explicit header assert.

Asserts

Optional list of assertions on the HTTP response. Assertions can describe checks on status code, on the received body (or part of it) and on response headers.

Structure of an assert:

jsonpath "$.book"query containspredicate type "Dune"predicate value
bodyquery matchespredicate type /\d{4}-\d{2}-\d{2}/predicate value

An assert consists of a query followed by a predicate. The format of the query is shared with captures, and can be one of :

Queries, as in captures, can be refined with subqueries. count subquery can be used with various predicates to add tests on collections sizes.

Predicates

Predicates consist of a predicate function, and a predicate value. Predicate functions are:

  • == (equals): check equality of query and predicate value
  • !=: check that query and predicate value are different
  • > (greaterThan): check that query number is greater than predicate value
  • >= (greaterThanOrEquals): check that query number is greater than or equal to the predicate value
  • < (lessThan): check that query number is less than that predicate value
  • <= (lessThanOrEquals): check that query number is less than or equal to the predicate value
  • startsWith: check that query starts with the predicate value (query can return a string or a binary content)
  • endsWith: check that query ends with the predicate value (query can return a string or a binary content)
  • contains: check that query contains the predicate value (query can return a string or a binary content)
  • includes: check that query collections includes the predicate value
  • matches: check that query string matches the regex pattern described by the predicate value
  • exists: check that query returns a value
  • isInteger: check that query returns an integer
  • isFloat: check that query returns a float
  • isBoolean: check that query returns a boolean
  • isString: check that query returns a string
  • isCollection: check that query returns a collection

Each predicate can be negated by prefixing it with not (for instance, not contains or not exists)

jsonpath "$.book"query not containspredicate type "Dune"predicate value

A predicate values is typed, and can be a string, a boolean, a number, a bytestream, null or a collection. Note that "true" is a string, whereas true is a boolean.

For instance, to test the presence of a h1 node in an HTML response, the following assert can be used:

GET https://example.org/home

HTTP/1.1 200
[Asserts]
xpath "boolean(count(//h1))" == true
xpath "//h1" exists # Equivalent but simpler

As the XPath query boolean(count(//h1)) returns a boolean, the predicate value in the assert must be either true or false without double quotes. On the other side, say you have an article node and you want to check the value of some data attributes:

<article
  id="electric-cars"
  data-visible="true"
...
</article>

The following assert will check the value of the data-visible attribute:

GET https://example.org/home

HTTP/1.1 200
[Asserts]
xpath "string(//article/@data-visible)" == "true"

In this case, the XPath query string(//article/@data-visible) returns a string, so the predicate value must be a string.

The predicate function equals can work with string, number or boolean while matches, startWith and contains work only on string. If a query returns a number, a contains predicate will raise a runner error.

# A really well tested web page...
GET https://example.org/home

HTTP/1.1 200
[Asserts]
header "Content-Type" contains "text/html"
header "Last-Modified" == "Wed, 21 Oct 2015 07:28:00 GMT"
xpath "//h1" exists  # Check we've at least one h1
xpath "normalize-space(//h1)" contains "Welcome"
xpath "//h2" count == 13
xpath "string(//article/@data-id)" startsWith "electric"

Status assert

Check the received HTTP response status code. Status assert consists of the keyword status followed by a predicate function and value.

GET https://example.org

HTTP/1.1 *
[Asserts]
status < 300

Header assert

Check the value of a received HTTP response header. Header assert consists of the keyword header followed by a predicate function and value.

GET https://example.org

HTTP/1.1 302
[Asserts]
header "Location" contains "www.example.net"

If there are multiple headers with the same name, the header assert returns a collection, so count, includes can be used in this case to test the header list.

Let's say we have this request and response:

> GET /hello HTTP/1.1
> Host: example.org
> Accept: */*
> User-Agent: hurl/1.8.0-SNAPSHOT
>
* Response: (received 12 bytes in 11 ms)
*
< HTTP/1.0 200 OK
< Vary: Content-Type
< Vary: User-Agent
< Content-Type: text/html; charset=utf-8
< Content-Length: 12
< Server: Flask Server
< Date: Fri, 07 Oct 2022 20:53:35 GMT

One can use explicit header asserts:

GET https://example.org/hello

HTTP/* 200
[Asserts]
header "Vary" count == 2
header "Vary" includes "User-Agent"
header "Vary" includes "Content-Type"

Or implicit header asserts:

GET https://example.org/hello

HTTP/* 200
Vary: User-Agent
Vary: Content-Type

URL assert

Check the last fetched URL. This is most meaningful if you have told Hurl to follow redirection (see [Options]section or --location option). URL assert consists of the keyword url followed by a predicate function and value.

GET https://example.org/redirecting
[Options]
location: true

HTTP/* 200
[Asserts]
url == "https://example.org/redirected"

Cookie assert

Check value or attributes of a Set-Cookie response header. Cookie assert consists of the keyword cookie, followed by the cookie name (and optionally a cookie attribute), a predicate function and value.

Cookie attributes value can be checked by using the following format: <cookie-name>[cookie-attribute]. The following attributes are supported: Value, Expires, Max-Age, Domain, Path, Secure, HttpOnly and SameSite.

GET http://localhost:8000/cookies/set

HTTP/1.0 200

# Explicit check of Set-Cookie header value. If the attributes are
# not in this exact order, this assert will fail. 
Set-Cookie: LSID=DQAAAKEaem_vYg; Expires=Wed, 13 Jan 2021 22:23:01 GMT; Secure; HttpOnly; Path=/accounts; SameSite=Lax;
Set-Cookie: HSID=AYQEVnDKrdst; Domain=.localhost; Expires=Wed, 13 Jan 2021 22:23:01 GMT; HttpOnly; Path=/
Set-Cookie: SSID=Ap4PGTEq; Domain=.localhost; Expires=Wed, 13 Jan 2021 22:23:01 GMT; Secure; HttpOnly; Path=/

# Using cookie assert, one can check cookie value and various attributes.
[Asserts]
cookie "LSID" == "DQAAAKEaem_vYg"
cookie "LSID[Value]" == "DQAAAKEaem_vYg"
cookie "LSID[Expires]" exists
cookie "LSID[Expires]" contains "Wed, 13 Jan 2021"
cookie "LSID[Max-Age]" not exists
cookie "LSID[Domain]" not exists
cookie "LSID[Path]" == "/accounts"
cookie "LSID[Secure]" exists
cookie "LSID[HttpOnly]" exists
cookie "LSID[SameSite]" equals "Lax"

Secure and HttpOnly attributes can only be tested with exists or not exists predicates to reflect the Set-Cookie header semantics (in other words, queries <cookie-name>[HttpOnly] and <cookie-name>[Secure] don't return boolean).

Body assert

Check the value of the received HTTP response body when decoded as a string. Body assert consists of the keyword body followed by a predicate function and value. The encoding used to decode the body is based on the charset value in the Content-Type header response.

GET https://example.org

HTTP/1.1 200
[Asserts]
body contains "<h1>Welcome!</h1>"

Precise the encoding used to decode the text body.

Bytes assert

Check the value of the received HTTP response body as a bytestream. Body assert consists of the keyword bytes followed by a predicate function and value.

GET https://example.org/data.bin

HTTP/* 200
[Asserts]
bytes startsWith hex,efbbbf;

XPath assert

Check the value of a XPath query on the received HTTP body decoded as a string. Currently, only XPath 1.0 expression can be used. Body assert consists of the keyword xpath followed by a predicate function and value. Values can be string, boolean or number depending on the XPath query.

Let's say we want to check this HTML response:

$ curl -v https://example.org

< HTTP/1.1 200 OK
< Content-Type: text/html; charset=UTF-8
...
<!doctype html>
<html>
  <head>
    <title>Example Domain</title>
    ...
  </head>

  <body>
    <div>
      <h1>Example</h1>
      <p>This domain is for use in illustrative examples in documents. You may use this domain in literature without prior coordination or asking for permission.</p>
      <p><a href="https://www.iana.org/domains/example">More information...</a></p>
    </div>
  </body>
</html>

With Hurl, we can write multiple XPath asserts describing the DOM content:

GET https://example.org

HTTP/1.1 200
Content-Type: text/html; charset=UTF-8

[Asserts]
xpath "string(/html/head/title)" contains "Example" # Check title
xpath "count(//p)" == 2                             # Check the number of <p>
xpath "//p" count == 2                              # Similar assert for <p>
xpath "boolean(count(//h2))" == false               # Check there is no <h2>  
xpath "//h2" not exists                             # Similar assert for <h2>

XML Namespaces are also supported. Let's say you want to check this XML response:

<?xml version="1.0"?>
<!-- both namespace prefixes are available throughout -->
<bk:book xmlns:bk='urn:loc.gov:books'
         xmlns:isbn='urn:ISBN:0-395-36341-6'>
    <bk:title>Cheaper by the Dozen</bk:title>
    <isbn:number>1568491379</isbn:number>
</bk:book>

This XML response can be tested with the following Hurl file:

GET http://localhost:8000/assert-xpath

HTTP/1.0 200
[Asserts]

xpath "string(//bk:book/bk:title)" == "Cheaper by the Dozen"
xpath "string(//*[name()='bk:book']/*[name()='bk:title'])" == "Cheaper by the Dozen"
xpath "string(//*[local-name()='book']/*[local-name()='title'])" == "Cheaper by the Dozen"

xpath "string(//bk:book/isbn:number)" == "1568491379"
xpath "string(//*[name()='bk:book']/*[name()='isbn:number'])" == "1568491379"
xpath "string(//*[local-name()='book']/*[local-name()='number'])" == "1568491379"

The XPath expressions string(//bk:book/bk:title) and string(//bk:book/isbn:number) are written with bk and isbn namespaces.

For convenience, the first default namespace can be used with _

JSONPath assert

Check the value of a JSONPath query on the received HTTP body decoded as a JSON document. Body assert consists of the keyword jsonpath followed by a predicate function and value.

Let's say we want to check this JSON response:

curl -v http://httpbin.org/json

< HTTP/1.1 200 OK
< Content-Type: application/json
...

{
  "slideshow": {
    "author": "Yours Truly",
    "date": "date of publication",
    "slides": [
      {
        "title": "Wake up to WonderWidgets!",
        "type": "all"
      },
       ...
    ],
    "title": "Sample Slide Show"
  }
}

With Hurl, we can write multiple JSONPath asserts describing the DOM content:

GET http://httpbin.org/json

HTTP/1.1 200
[Asserts]
jsonpath "$.slideshow.author" == "Yours Truly"
jsonpath "$.slideshow.slides[0].title" contains "Wonder"
jsonpath "$.slideshow.slides" count == 2
jsonpath "$.slideshow.date" != null
jsonpath "$.slideshow.slides[*].title" includes "Mind Blowing!"

Explain that the value selected by the JSONPath is coerced to a string when only one node is selected.

In matches predicates, metacharacters beginning with a backslash (like \d, \s) must be escaped. Alternatively, matches predicate support Javascript-like Regular expression syntax to enhance the readability:

GET https://sample.org/hello

HTTP/1.0 200
[Asserts]
# Predicate value with matches predicate:
jsonpath "$.date" matches "^\\d{4}-\\d{2}-\\d{2}$"
jsonpath "$.name" matches "Hello [a-zA-Z]+!"
# Equivalent syntax:
jsonpath "$.date" matches /^\d{4}-\d{2}-\d{2}$/
jsonpath "$.name" matches /Hello [a-zA-Z]+!/

Regex assert

Check that the HTTP received body, decoded as text, matches a regex pattern.

GET https://sample.org/hello

HTTP/1.0 200
[Asserts]
regex "^\\d{4}-\\d{2}-\\d{2}$" == "2018-12-31"

SHA-256 assert

Check response body SHA-256 hash.

GET https://example.org/data.tar.gz

HTTP/* *
[Asserts]
sha256 == hex,039058c6f2c0cb492c533b0a4d14ef77cc0f78abccced5287d84a1a2011cfb81;

MD5 assert

Check response body MD5 hash.

GET https://example.org/data.tar.gz

HTTP/* *
[Asserts]
md5 == hex,ed076287532e86365e841e92bfc50d8c;

Variable assert

# Test that the XML endpoint return 200 pets 
GET https://example.org/api/pets
HTTP/* 200
[Captures]
pets: xpath "//pets"
[Asserts]
variable "pets" count == 200

Duration assert

Check the total duration (sending plus receiving time) of the HTTP transaction.

GET https://sample.org/helloworld

HTTP/1.0 200
[Asserts]
duration < 1000   # Check that response time is less than one second

Filters

Optionally, asserts can be refined using filters count and regex.

Count filter

GET https://pets.org/cats/cutest

HTTP/1.0 200
[Asserts]
jsonpath "$.cats" count == 12

Regex filter

GET https://pets.org/cats/cutest

HTTP/1.0 200
# Cat name are structured like this `meow + id`: for instance `meow123456` 
[Asserts]
jsonpath "$.cats[0].name" regex /meow(\d+)/ == "123456"

Body

Optional assertion on the received HTTP response body. Body section can be seen as syntactic sugar over body asserts (with equals predicate function). If the body of the response is a JSON string or a XML string, the body assertion can be directly inserted without any modification. For a text based body that is not JSON nor XML, one can use multiline string that starts with ``` and ends with ```. For a precise byte control of the response body, a Base64 encoded string or an input file can be used to describe exactly the body byte content to check.

JSON body

# Get a doggy thing:
GET https://example.org/api/dogs/{{dog-id}}

HTTP/1.1 200
{
    "id": 0,
    "name": "Frieda",
    "picture": "images/scottish-terrier.jpeg",
    "age": 3,
    "breed": "Scottish Terrier",
    "location": "Lisco, Alabama"
}

XML body

GET https://example.org/api/catalog

HTTP/1.1 200
<?xml version="1.0" encoding="UTF-8"?>
<catalog>
   <book id="bk101">
      <author>Gambardella, Matthew</author>
      <title>XML Developer's Guide</title>
      <genre>Computer</genre>
      <price>44.95</price>
      <publish_date>2000-10-01</publish_date>
      <description>An in-depth look at creating applications with XML.</description>
   </book>
</catalog>

Multiline string body

GET https://example.org/models

HTTP/1.1 200
```
Year,Make,Model,Description,Price
1997,Ford,E350,"ac, abs, moon",3000.00
1999,Chevy,"Venture ""Extended Edition""","",4900.00
1999,Chevy,"Venture ""Extended Edition, Very Large""",,5000.00
1996,Jeep,Grand Cherokee,"MUST SELL! air, moon roof, loaded",4799.00
```

The standard usage of a multiline string is :

```
line1
line2
line3
```

is evaluated as "line1\nline2\nline3\n".

To construct an empty string :

```
```

or

``````

Finally, multiline can be used without any newline:

```line```

is evaluated as "line".

Base64 body

Base64 body assert starts with base64, and end with ;. MIME's Base64 encoding is supported (newlines and white spaces may be present anywhere but are to be ignored on decoding), and = padding characters might be added.

GET https://example.org

HTTP/1.1 200
base64,TG9yZW0gaXBzdW0gZG9sb3Igc2l0IGFtZXQsIGNvbnNlY3RldHVyIG
FkaXBpc2NpbmcgZWxpdC4gSW4gbWFsZXN1YWRhLCBuaXNsIHZlbCBkaWN0dW0g
aGVuZHJlcml0LCBlc3QganVzdG8gYmliZW5kdW0gbWV0dXMsIG5lYyBydXRydW
0gdG9ydG9yIG1hc3NhIGlkIG1ldHVzLiA=;

File body

To use the binary content of a local file as the body response assert, file body can be used. File body starts with file, and ends with `;``

GET https://example.org

HTTP/1.1 200
file,data.bin;

File are relative to the input Hurl file, and cannot contain implicit parent directory (..). You can use --file-root option to specify the root directory of all file nodes.