hurl/docs/tutorial/captures.md

Captures

We have seen how to chain requests in a Hurl file. In some use cases, you want to use data from one request and inject it in another one. That's what captures are all about.

Capturing a CSRF Token

In our website, a user can login at http://localhost:3000/login. The HTML page is a form where the user can input:

• a required username
• a required password

If we look at the page HTML content, we can see an HTML form:

<form class="login-form" method="post" action="/login">
    <input type="hidden" name="_csrf" value="0fSk7gRA-UTkS25Fbsyal0dgLPBjVy1YIoNg">
    ...
    <input type="text" name="username" id="username" autocomplete="off" minlength="3" maxlength="32" pattern="[a-zA-Z0-9_-]{3,32}" title="Username must use a-z, A-Z and 0-9" required="">
    ...
    <input type="password" name="password" id="password" autocomplete="off" minlength="6" maxlength="32" required="">
    ...
    <input type="submit" value="Login">
    ...
</form>

When the user clicks on 'Login' button, a POST request is sent with form values: the username and a password. Our server implements a Post / Redirect / Get pattern: if the POST submission is successful, the user is redirected to his favorites movies page.

Let's try to test it!

Form values can be sent using a Form parameters section, with each key followed by its corresponding value.

1. Create a new file named login.hurl:

POST http://localhost:3000/login
[FormParams]
username: fab
password: 12345678
HTTP 302

When sending form data with a Form parameters section, you don't need to set the Content-Type HTTP header: Hurl infers that the content type of the request is application/x-www-form-urlencoded.

2. Run login.hurl:

login.hurl: Running [1/1]
error: Assert status code
  --> login.hurl:5:6
   |
 5 | HTTP 302
   |      ^^^ actual value is <403>
   |

login.hurl: Failure (1 request(s) in 9 ms)
--------------------------------------------------------------------------------
Executed files:  1
Succeeded files: 0 (0.0%)
Failed files:    1 (100.0%)
Duration:        10 ms

This is unexpected! Our test is failing, we're not redirected to the favorite movies page.

The reason is quite simple, let's look more precisely at our HTML form:

<form class="login-form" method="post" action="/login">
    <input type="hidden" name="_csrf" value="0fSk7gRA-UTkS25Fbsyal0dgLPBjVy1YIoNg">
    ...
</form>

The server login page is protected by a CSRF token. In a browser, when the user wants to log in by sending a POST request, a token is sent along the username/password values. This token is generated server-side, and embedded in the HTML. When the POST request is made, our server expects that the request includes a valid token, and will reject the request if the token is missing or invalid.

In our Hurl file, we're not sending any token, so the server is rejecting our request with a 403 Forbidden HTTP response.

Unfortunately, we can't hard code the value of a token in our [FormParams] section because the token is dynamically generated on each request, and a certain fixed value would be valid only during a small period of time.

We need to dynamically capture the value of the CSRF token and pass it to our form. To do so, we are going to:

• perform a first GET request to http://localhost:3000/login and capture the CSRF token
• chain with a POST request that contains our username/password value, and our captured CSRF token
• check that the POST response is a redirection, i.e. a 302 Found to the favorites page

So, let's go!

How to capture values

1. Modify login.hurl:

# First, display the login page to capture
# the CSRF token (see https://en.wikipedia.org/wiki/Cross-site_request_forgery)
GET http://localhost:3000/login
HTTP 200
[Captures]
csrf_token: xpath "string(//input[@name='_csrf']/@value)"

Captures are defined in a [Captures] section. Captures are composed of a variable name and a query. We have already seen queries in Adding asserts tutorial part. Since we want to capture value from an HTML document, we can use a XPath capture.

Every query can be used in assert or in capture. You can capture value from JSON response with a JSONPath capture, or capture cookie value with the same queries that you use in asserts.

In this capture, csrf_token is a variable and xpath "string(//input[@name='_csrf']/@value)" is the XPath query.

Now that we have captured the CSRF token value, we can inject it in the POST request.

2. Add a POST request using csrf_token variable in login.hurl:

# First, display the login page to capture
# the CSRF token (see https://en.wikipedia.org/wiki/Cross-site_request_forgery)
GET http://localhost:3000/login
HTTP 200
[Captures]
csrf_token: xpath "string(//input[@name='_csrf']/@value)"


# Log in user, using the captured CSRF token:
POST http://localhost:3000/login
[FormParams]
username: fab
password: 12345678
_csrf: {{csrf_token}}
HTTP 302

3. Run login.hurl and verify everything is ok:

$ hurl --test login.hurl
login.hurl: Running [1/1]
login.hurl: Success (2 request(s) in 14 ms)
--------------------------------------------------------------------------------
Executed files:  1
Succeeded files: 1 (100.0%)
Failed files:    0 (0.0%)
Duration:        16 ms

Follow Redirections

Like its HTTP engine curl, Hurl doesn't follow redirection by default: if a response has a 302 Found status code, Hurl doesn't implicitly run requests until a 200 OK is reached. This can be useful if you want to validate each redirection step.

After having logged it, we would like to test the page where the user has been redirected. This is really simple and can be achieved with a header assert: on the response to the POST creation request, we are going to assert the Location header, which indicates the redirection URL target.

1. Add a new header assert to test the Location header:

# First, display the login page to capture
# ...

# Log in user, using the captured CSRF token:
POST http://localhost:3000/login
[FormParams]
username: fab
password: 12345678
_csrf: {{csrf_token}}
HTTP 302
[Asserts]
header "Location" == "/my-movies"

2. Add a request to get the favorites page that the user has been redirected to:

# First, display the login page to capture
# ...

# Log in user, using the captured CSRF token:
# ...

# Follow redirection and open favorites:
GET http://localhost:3000/my-movies
HTTP 200
[Asserts]
xpath "string(//title)" == "My Movies"

3. Run login.hurl and verify everything is ok:

$ hurl --test login.hurl
login.hurl: Running [1/1]
login.hurl: Success (3 request(s) in 17 ms)
--------------------------------------------------------------------------------
Executed files:  1
Succeeded files: 1 (100.0%)
Failed files:    0 (0.0%)
Duration:        19 ms

You can force Hurl to follow redirection by using -L / --location option or using an [Options] section. In this case, asserts and captures will be run against the last redirection step.

A login workflow is surprisingly hard to do well. You can try to add more test on our login.hurl test. With Hurl, try now to test the following usecase:

• when a user is not authenticated and goes to http://localhost:3000/my-movies, he is redirected to the login page,
• what's happen if the user try to log in with a wrong password,
• after a user log out, he can open the login page again.

You can see a more complete login.hurl on the GitHub repo.

Recap

So, our test file login.hurl is now:

# First, display the login page to capture
# the CSRF token (see https://en.wikipedia.org/wiki/Cross-site_request_forgery)
GET http://localhost:3000/login
HTTP 200
[Captures]
csrf_token: xpath "string(//input[@name='_csrf']/@value)"


# Log in user, using the captured CSRF token:
POST http://localhost:3000/login
[FormParams]
username: fab
password: 12345678
_csrf: {{csrf_token}}
HTTP 302


# Follow redirection and open favorites:
GET http://localhost:3000/my-movies
HTTP 200
[Asserts]
xpath "string(//title)" == "My Movies"

We have seen how to capture response data in a variable and use it in others request. Captures and asserts share the same queries, and can be inter-mixed in the same response. Finally, Hurl doesn't follow redirect by default, but captures can be used to run each step of a redirection.