Automated Performance Testing
We believe in goal oriented and automated load testing. That's why we built k6 to work well in such environments, integrating nicely with services like CircleCI, the continuous integration and delivery platform.
This guide will help you get up and running with k6, CircleCI and Load Impact Insights (with build notifications sent to Slack). This guide assumes you are familiar with k6 and CircleCI.
It also assumes you have a Load Impact account. If not, go get one here – it’s free. Once you have your account setup the integration with CircleCI is only three more steps.
It starts with code. This is the load test example script we'll be using in this guide. It tests our example site test.loadimpact.com:
To describe the load test script in words:
Front pageinside which, we make a request, check the response status code and sleep for 10s, before letting the user loop from the top of the function.
Below is the barebones circle.yml config file needed to run k6 and the above load test from CircleCI, nothing more!
To describe the config in words:
main.jslocated in the
loadtestsdirectory in the root of the repository
Note that the k6 command includes the
-o cloud option which means the results will be streamed to Insights (exclude that part of the command if that's not what you want). For that to work we'll need to make sure k6 has access to our Load Impact API token in the environment variable
K6_CLOUD_TOKEN, so let's set this up next!
Right, so to have the results of our load tests streamed to Insights we need to add the K6_CLOUD_TOKEN environment variable. Navigate to your project's settings in CircleCI and find the "Environment variables" section under "Build settings" in the menu to the left. Add a new environment variable with name K6_CLOUD_TOKEN and copy/paste the API token from your account as the value:
We're soon ready to run our first CI triggered load test, promise! But first, let's hook up Slack to our CircleCI project so that we get notified when our load tests finish. For this you'll need to setup an "Incoming Webhook" integration in Slack. Once that is done, copy/paste the Webhook URL into the input field in the "Chat Notifications" settings:
Let's trigger a build by making a small change to the load test script (lowering the threshold to 100ms perhaps). Push the change to Github, CircleCI will pick up the repo change and run the through the circle.yml, and soon enough trigger the load test run:
As we can see above a lowering of the threshold to 100ms triggered a failed build step and thus a failed build, and the notification was quick to let us know:
A passing build is not much fun to look at. But, when builds fail, you want to understand why. No exception when it's a load test failing the build. We've built Insights to be a perfect companion to k6 for analysing results! The analysis workflow is error driven, meaning the goal is to help you quickly finding the cause of the failed load test. Read more about Insights functionality on this and adjacent pages in our Knowledge Base.
That's it, happy automated testing!
You'd need to grant CircleCI access to your system by adding the necessary IP ranges to your firewall. If you're using AWS you can temporarily grant access by adding a security group rule pre-test:
and then removing the grant post-test:
See the modified circle.yml and shell scripts to accomplish this in the example repository.
There's no one answer here. We recommend against running load tests on every code commit though. Load tests are different in nature from unit, functional or integration tests. They need a longer time to complete and a target environment that is closer to real life (production) than the types of tests that are more functional in nature.
Instead, we recommend that you run your test at a frequency that makes sense for your organization. You could for example run tests when merging code into a release branch or as part of a nightly build so you get your results fresh in the morning!