Load Impact

Load Impact Newsletter #1 - March 25, 2009

Welcome to the first newsletter describing news and events around Load Impact!

First of all, let me say that it is great fun to see so many people interested in Load Impact. Blogs are writing nice things about us, there are visitors dropping in from all over the world and the number of registered users is increasing very fast. It seems we are doing something right, and we would like to do things even better in the future, so don't hesitate to give us your thoughts on where we might improve, new ideas etc. Email me at ragnar@loadimpact.com if you have any ideas, comments or feedback.

And now on to the real news...

  /Ragnar



** New site update **

Load Impact 1.35 was installed on the site on the 25th of March. Main highlights are:

- New metric: user load time
- New metric: bandwidth usage
- Note that old metric "Test Summary" has been renamed to "Accumulated load time" - Acc. load time
- New knowledgebase Wiki
- Improved information and help features
- Improved security features (better functionality and feedback)

Further below you can read details about everything that's new in Load Impact 1.35.



** New knowledgebase Wiki **

With Load Impact 1.35 we integrate a Wiki into the site. This Wiki is intended to become a valuable, free knowledgebase with information on load testing, performance testing and optimization. We will start supplying it with articles and information, but our hope is to also get some assistance from the Internet community with this effort. The address to the wiki is http://loadimpact.com/kb

We will strive to keep the Wiki neutral and fact-based. Commercial (or otherwise) products will be listed and reviewed, not hyped. In the end we hope to amass a certain number of people who are interested in helping maintain the Wiki, and who will be the ones to run it on a daily basis. We realize, however, that a lot of work is needed to reach that stage. A community doesn't start itself. We are prepared to spend time and effort building up this load/performance testing knowledgebase until it is self-powered.

The wiki will not be open for public editing, only viewing. If you are interested in becoming a contributor, please contact us at kb@loadimpact.com



** Accounts and subscriptions **

There has been some confusion around our account options. Why should you buy an account and how long do you have to keep subscribing to it?

Let's answer the last question first: when you buy a subscription account, you pay for each month in advance, and you can terminate your subscription whenever you want. This means that whenever you cancel your subscription, no more payments will be drawn after that.

Buying a premium account - i.e. Load Impact BASIC, PROFESSIONAL or ADVANCED - lets you run tests with higher load levels and more features. It also allows you to run tests more frequently. An anonymous (i.e. not logged in) Load Impact user will not be allowed to start a test for a certain site, if that site has already been tested twice during the last 24 hours. A Load Impact LIGHT user, on the other hand, will be allowed to start a test if the site s/he wants to test has been tested less than 4 times during the last 24 hours.

This means that in a situation where an anonymous user gets denied starting a test, getting the error message "This site contains addresses that have been tested too many times the last 24 hours", a Load Impact LIGHT user might still be able to start the exact same test. Simply by Load Impact trusting a registered user a little bit more than it trusts an anonymous user.

The current allowances are as follows:

- Anonymous users can load a specific site/IP up to 2 times/day
- Load Impact LIGHT users can load a specific site/IP up to 4 times/day
- Load Impact BASIC users can load a specific site/IP up to 10 times/day
- Load Impact PROFESSIONAL users can load a specific site/IP up to 20 times/day
- Load Impact ADVANCED users can load a specific site/IP up to 40 times/day

Note also: with a valid loadimpact.txt file on your site, these (and other) allowances are doubled



** Load Impact 1.35 **

This is a list (a changelog) of most of the observable changes from the previous version. It is also the first such list that we write, but we intend to make it a habit from now on, making sure we communicate new features and changes to all our users, to better allow you to make use of them!

Load Impact 1.35 contains a lot of bug fixes and minor improvements, as well as a few major ones. Our primary aim with this release has been to improve usability. This is achieved through the addition of more supporting information - help texts and tooltips, better error messages, etc - as well as user interface and functionality fixes that make the system more understandable and easier to use. We have also fine-tuned the security features to make them less of an obstacle to legitimate testers.


1. New measurement metrics!

We have introduced two new metrics (test result types) and renamed an old metric. The old Test summary metric that has been the default value plotted for all tests has now become a collapsable folder:



Here is an explanation of what the three different metrics are:

Accumulated Load Time (used to be Test Summary)
This is the value that used to be plotted when a user selected Test summary. It was renamed to better reflect what it is actually measuring. The Accumulated Load Time is the combined load time of all objects loaded by the clients during the test. It does not reflect the user experienced load time of a web page as it does not take into account that objects are loaded across multiple connections simultaneously. Many people have assumed the old Test summary result to be the page load time as experienced by a user, which is why we have decided to rename the metric. This metric is good to use for performance optimization work - i.e. when you want to clearly see if a change you made to your system affects load time or not.

User Load Time (new default metric plotted in graphs and snapshots)
This is the load time as experienced by a user when loading a page with a web browser. I.e. it is the time the user has to wait before the page is fully loaded by the user's browser. This has now been made the default value that is plotted in all graphs, because it is what most people expect to see in the graphs. Note that actual time before a page is rendered in a browser may still deviate from the User Load Time metric, because rendering a page takes a little time, and javascript execution can also slow things down for the browser. Most of the time, however, User Load Time is fairly close to page rendering time.

Bandwidth Usage
This is the amount of network bandwidth used by the simulated clients (all of them combined). It concerns HTTP bandwidth, which means there will be some overhead for TCP and IP header data also, but that overhead is usually one or two percent, so it is fairly negligible.

.

2. New Knowledgebase Wiki!

We now have an integrated Wiki that is going to serve as a repository for articles and information on load testing and performance optimization. It is accessible from the main menu, under the heading "Knowledgebase", or directly at http://loadimpact.com/kb

This knowledgebase is available to anyone on the Internet, not just Load Impact users. The aim is to create the best online information resource for load testing, performance testing and performance optimization. To achieve this goal we will of course try to keep the information neutral - i.e. not pushing certain products or companies but try to be objective in what we publish and provide real user value. We hope to enlist the aid of the Internet community in order to fill the Wiki with information, so if you would like to contribute and write an article you are welcome to contact us at kb@loadimpact.com

.

3. Better information when a user is denied to start a test

This configuration contains addresses that have been tested too many times the last 24 hours/month is a common error that people get when they try to start tests. It is a security feature that is meant to prevent people from testing any one specific site too often. Unfortunately, many legitimate tests load objects from the same sources (sites) which means that it is very common to get this error.

Previously, the error message you got was simply the text above - "This configuration contains addresses..." - which didn't state what addresses Load Impact was objecting to. Now you get an error message that tells you what sites/IP addresses were the offending ones. This means that you can then edit your load script and remove the offending URLs from it, which will allow you to start your test.

.

4. New FAQ article about how to interpret different graph types

The view test page links (through the example graphs at the bottom of the screen) to a new FAQ article that explains how to interpret several common types of graphs. I.e. a graph with a falling curve, or a graph with a straight curve (line). Direct link to the article: http://loadimpact.com/forum/viewtopic.php?id=87

.

5. loadimpact.txt improvements

Usernames are now supported in loadimpact.txt - i.e. you put in the file the email address that you have registered on loadimpact.com with. A valid loadimpact.txt will allow a user to run twice the number of tests, twice the amount of data, etc during a 24-hour, or 30-day period.

It is also necessary to be able to run large tests (for those of you who have Load Impact PROFESSIONAL or Load Impact ADVANCED subscriptions)

Check the last section of this article for more information about loadimpact.txt.

.

6. Configurable client bandwidth has been removed!

Due to incompatibilities in the backend components, we have had to temporarily remove the "configurable client bandwidth" feature. All simulated clients will now use as much bandwidth as they can when downloading items during a test. This feature will be back in a later release of Load Impact.

.

7. Userlevels and permissions changed

Anonymous users now have a separate user level, with lower permission settings than a LIGHT user.

Also, the "daily site runs" permission setting has been changed for all userlevels. "daily site runs" is a setting that controls how many times in a 24-hour period users may start tests that load things from a certain IP address (server). Load Impact remembers how many tests it has executed that have loaded items from a particular IP, and if the number is greater than or equal to the current user's "daily site runs" permission setting, then the user may not start a new test. This means e.g. that if five different users have started tests that have loaded items from www.domain.com, then a sixth user whose "daily site runs" permission setting is set to 5 will not be able to start yet another test that loads things from www.domain.com.

The old settings were somewhat tight and didn't allow people to run enough tests to use Load Impact as a performance optimization tool, so we have increased the limits for all userlevels. The new "daily site runs" permissions for the different userlevels are:

Anonymous user: 2
LIGHT user: 4
BASIC user: 10
PROFESSIONAL user: 20
ADVANCED user: 40

.

8. More connections per simulated client

Max number of connections used by simulated clients is now 4.

Earlier, simulated clients would use max 2 concurrent connections when loading things from a target webserver. Newer browser versions have increased the number of concurrent connections they use when loading objects from webservers, so to better emulate IE7+ and FF3+ we have increased the number of connections a client may use, from 2 to 4. This means that generated traffic patterns will be more realistic and results, and especially the new "User Load Time" metric, will match real observed load times better.

Note that you can still run tests with the old number of connections (2), by editing your load script

.

9. New press page

We now have a press page, with press images etc, at http://loadimpact.com/info/press.php

.

10. Miscellaneous changes

* Tooltips when hovering with mouse pointer over a point in the graphs
Hovering with the mouse pointer over a point in a graph will now show you a small tooltip with the exact value plotted for that point.

* Userlevel is now checked on each page load
Users who upgrade their account, or otherwise change their user level, no longer need to logout and login again for the system to recognize their new user level

* Email notifications when tests are completed
Users may now choose to get notified by email when their tests are completed.

* Improved information when a test is queued
Users who get placed in a queue when they start a test will now receive more information about their place in the queue

* Improved help features
There is now tooltip help for the various test detail headers on the test result page

* Top menu has been changed
Support & Knowledgebase have been moved to the right side

* Better IE8 compatibility
IE8 browsers are now forced to render pages in IE7 compatibility mode, which means both IE7 and IE8 users should get pages properly rendered

* Many small fixes/changes
  - Old tests that were sometimes left in the "running" state should now be properly marked as finished/aborted/failed
  - Some UI bugfixes on the edit test page
  - Long URLs are handled better on the view test page
  - The public URL box has been made bigger
  - Running tests can no longer be deleted from the test results list
  - Uppercase file extensions no longer automatically ends up in the "misc files" section on the view test page
  - It is not possible to start tests to reserved IP ranges (private networks, multicast, etc)
  - The two statistics boxes for running tests on view test page have been combined
  - Email addresses on the site have been obfuscated through javascript code, to make life a little harder for spam bots
  - Snapshot graphs will now always use the same scale as the viewtest graphs
  - Blacklist now properly handles URLs with port numbers
  - Users who get "test denied" now get a link to the forum with explanations
  - Queuetest page now avoids reloading the page (could sometimes cause a browser to loop and try to queue a test many times)
  - Timeout tests now have their client levels displayed in the test result list
  - The running tests box now correctly displays links to runnings tests that were started with URL parameters

Last edited by Ragnar (2009-03-25 19:27:24)