Fastly BDD

Introduction

Fastly is a popular CDN based on the open-source Varnish. Since it supports VCL, a lot of custom “logic” to handle incoming requests can be added, right at the edge. This improves the user experience as well as reduces the overhead on origin servers.

Let’s take an example of redirects. Traditionally, they are applied on webservers and often involve not-so-straightforward syntax for complex rules (multiple checks for various headers, cookies, referrers, urls etc.). All of these are fairly straight forward to achieve in the VCL. Thanks to a lot of customization by Fastly on default VCL, you even have various functions OOTB for use. It makes redirects one of the easiest to offload to Fastly edge. This is what we will look at in this post.

Problem

As the rules begin to increase on edge, you need to make changes reliably and also need to test before pushing them off to the live environment. You can create multiple services and deploy changes first on a lower environment, test it and then promote to live. As for automating these tests, you can use anything - even a simple curl command (have covered some of it here)

While curl maybe fine to test a few redirects, it’s not ideal if you want to run through hundreds of them, and inspect various different sections of the response. A lot of plumbing would be needed in shell to make it work. Ideally, we need programatic control and constructs.

Solution

Enter behave. It is a BDD¹ framework which uses tests written in a natural language style, backed up by Python code.

The tests can easily be executed in any CI/CD pipeline as an initial step of deploying a live service. The overall flow can be as follows with AWS Codepipeline and Codebuild -

1. Any commits on github triggers the AWS Codepipeline.
2. First step in the pipeline deploys a test service with the same vcl.
3. Second step runs the behave tests against this test service and destroy the temporary service afterwards.
4. If the tests are successful, the pipeline then runs a tarraform plan against the live service.
5. Waits for a manual approval. A member of the team can review the plan output and approve it.
6. After approval, it deploys the new vcl to the live service.

I will cover the Fastly CI/CD in detail in a separate post later.

Let’s look at an example now.

Feature Test Cases

This defines the functionality that we need to implement and can be written by someone from the business, QA or PM. I am going to use the following structure and layout for this project.

1+--tests/
2    +--features/
3    |    +-- steps/
4    |    |    +-- steps.py
5    |    +-- redirect.feature

Consider the first test. It has a feature redirect and within this, there are 2 scenarios. One, to test when actual users hit the site. Second, to test when some crawlers hit the site.

 1Feature: redirect
 2
 3  Scenario: 
 4    Given I am a human
 5    And I visit https://fastly-bdd-example.com.global.prod.fastly.net/status/200
 6    Then Response is redirect
 7    And Response reason is REDIRECT
 8    And Status code is 302
 9    And Redirected url is https://fastly-bdd-example.com/gateway
10
11  Scenario: 
12    Given I am a googlebot
13    And I visit https://fastly-bdd-example.com.global.prod.fastly.net/status/200
14    Then Status code is 200
15    And Response reason is OK

VCL

Now that we have the feature described above, we need to implement this using VCLs. We are going to use vcl snippets which are quicker to setup and use. These can be deployed by the CI/CD pipeline or if you are just testing, from the Fastly console as well.

recv vcl snippet

1if (req.http.User-Agent ~ "googlebot"){
2    return (lookup);
3}
4
5if (req.url ~ "^/status/200") {
6    error 902;
7}

error vcl snippet

1if (obj.status == 902) {
2    set obj.status = 302;
3    set obj.response = "REDIRECT";
4    set obj.http.Location = "https://" req.http.host "/gateway";
5    return (deliver);
6}

Step Files

The backend code for running the test is implemented in the file steps.py. You can split this across different files as your tests grow.

 1from behave import *
 2import requests
 3
 4@given(u'I am a {useragent}')
 5def step_impl(context,useragent):
 6    if "bot" in useragent:
 7        context.useragent = useragent 
 8    else:
 9        context.useragent = "pybehave"
10
11@given(u'I visit {url}')
12def step_impl(context,url):
13    headers = {'User-Agent': context.useragent}
14    context.resp = requests.get(url, headers=headers, allow_redirects=False, verify=False )
15    print("response_headers :"+str(context.resp.headers))
16
17@then(u'Response is redirect')
18def step_impl(context):
19    assert context.resp.is_redirect
20
21@then(u'Response reason is {resp_reason}')
22def step_impl(context,resp_reason):
23    print("response_reason : "+context.resp.reason)
24    assert context.resp.reason == resp_reason
25
26@then(u'Status code is {resp_status}')
27def step_impl(context,resp_status):
28    print("response_status : "+str(context.resp.status_code))
29    assert context.resp.status_code == int(resp_status)
30
31@then(u'Redirected url is {resp_url}')
32def step_impl(context,resp_url):
33    print("response_url : "+context.resp.headers['Location'])
34    assert context.resp.headers['Location'] == resp_url

Executing Tests

Once you have the above files ready in your project, it is very simple to run the tests. You can use a lot of command line options as per your need. These are described in detail here

1$ behave
2.
3.
41 feature passed, 0 failed, 0 skipped
51 scenario passed, 0 failed, 0 skipped
66 steps passed, 0 failed, 0 skipped, 0 undefined
7Took 0m0.051s

To generate test reports in the junit format use the --junit command-line option. The so generated reports can then be used by any existing junit visualization frameworks you have, to create visualizations.

Conclusion

Apart from the obvious benefits of BDD, this workflow ensures issues are caught early and the deployments are error free. A side benefit of deploying it via a CI/CD pipeline is that it also syntax checks the actual VCL snippets before they make it to the live service. Since Fastly’s vcl implementation is quite different from open source Varnish - it is near impossible to syntax check quickly using a varnish container.

1. Behave 
2. About Vcl Snippets