Automated Load Testing Results Analysis
Stockholm, Sweden - 26 November, 2018 - Load Impact, the leading provider of SaaS solutions for load testing of websites, apps and APIs, is excited to announce Performance Alerts for automated load testing results analysis. One of the challenges DevOps teams face is understanding the results of a performance test run. Load Impact Insights Performance Alerts allow developers, performance engineers and QA teams to quickly see where they have a performance issue, either in the system under test, or in their test scripts.
Automated results analysis can really cut down on the time you spend poring over your test data to figure out where the bottlenecks are. In our user survey, we found that automated results analysis is the second most wanted capability for a load testing solution.
Load Impact automatically analyzes your k6 performance test results when the output is streamed to Load Impact Insights from the command line (k6 run -o cloud script.js), or when the test is executed in the Load Impact SaaS application.
There are two categories of alerts: Smart Result Performance Alerts and Test Health Performance Alerts.
Smart Result Performance Alerts analyze the performance test results and alert you to performance issues in the system under test. There are currently 2 alerts in this category:
Throughput Limit Alert
The target system (website, SaaS / web app, API, or microservice) has hit a throughput limit, meaning the request rate has flatlined. Your system is overloaded.
Throughput Limit Performance Alert
High HTTP Failure Rate Alert
The HTTP error rate for some period during the load test is at least 10% higher than at the beginning of the test. It usually means the target system is close to its performance limit.
These alerts highlight test script related issues. These issues can either skew your performance test results or make analysis harder. They can often be quickly resolved by making changes to your test script. There are currently 4 alerts in this category:
Third Party Content Alert
Insights detects more than 3 different domains in a test. This is typically caused by your test script containing requests to 3rd party resources such as CDNs, social media scripts, analytic tools, etc. That should usually be avoided.
Too Many URLs
Insights detects more than 500 unique URLs in your test results. Commonly caused by a URL that contains a query parameter that is unique per iteration, e.g. tokens, session IDs, etc. Use the URL grouping feature in k6.
High Load Generator CPU / Memory Usage (2 separate alerts)
These alerts tell you if your test configuration is causing over utilization of the load generator resources (CPU and Memory), which can skew your test results.
“Performance Alerts are another key feature that allows DevOps teams to bring high quality software to market faster and at lower cost,” said Haakan Tylen, CEO at Load Impact.
DevOps teams can try out k6, Load Impact Insights and Cloud Execution for free. k6 will always be freely available (see the GitHub link below) while Load Impact Insights and Cloud Execution are available through a 30-day free trial.