Can't donate to charity?
Volunteer computer time
or Support SETI!
R&D Sponsorship Center
June 2011

Home Page
Feature Archive
A&I Column Archive
Production Tools
State Marketing Data
US Marketing Data
World Marketing
Service Directory
Quality Assurance
3D Printing

Subscribe to Advertising & Marketing Review!
Contact Ken Custer at 303-277-9840.

Affordable Ways to Load Test Your Website

by Glen Emerson Morris
Related Columns
Desktop Manufacturing
Hits the Home Market

It's the biggest revolution of the 21st century. And it's going on now.
The Fab Future of
Desktop Manufacturing

Desktop manufacturing is finally getting affordable, at least to businesses.
The Object of 3D Printing
Print anything you need, from cameras to cars, and do your own recycling.
The Future of Arthur C. Clarke
One of the greatest science writers predicts what this century will hold for us.
Recommended Columns
The Greening of Expectations
It's not a fad, it's critical to our survival.
The Learning Curve to Prosperity
Buckminster Fuller predicted the resource crunch now hitting us. He also gave us the tools to deal with it.

Load testing Websites has always been problematic. It can cost a business thousands in lost sales if a new Website can't handle the projected level of traffic, but it can also cost thousands to load test properly, sometimes well into the five figure range. Unfortunately, load testing tends to be expensive for three reasons; it requires a lot of computers, it uses complex software, and it usually takes highly skilled programmers to execute.

Until recently there were just two primary approaches to load testing; buying a commercial product like LoadRunner, or using an open source program like JMeter. Either option is expensive. LoadRunner can run north of $50.000, and JMeter and similar programs require computer programmers who can cost thousands in man hours. Fortunately, a new approach has emerged based on cloud technology.

Shortly after Amazon launched a service allowing businesses to use Amazon's surplus computer processing capability as a cloud resource, several companies sprang up using Amazon to offer load testing services. Some offer do the complete job for you. Others offer a do it yourself approach. Both of these types of services open up Website load testing to thousands of businesses that simply could not have afforded it before. And neither requires a degree in programming to use. They just require a basic understanding of what load testing is, and it's really not that complicated.

What load testing is - Load testing is a testing process to determine how quickly and accurately a Website responds at high levels of traffic. For complex Websites it's a critical part of the development process. Once the basic programming is done, a series of load tests is run, with modifications made after each load test. Getting a Website, particularly an e-commerce site, to perform close to its theoretical limits takes some fine tuning, and this can go on for days, sometimes weeks.

Load testing software - Load testing software is software that is designed to simulate user activities on a Website and monitor how long it takes the Website to respond to the simulated activities at varying levels of traffic. The load testing software runs scripts, which tell it what user activities to simulate.

Load testing hardware - By definition, load testing requires a lot of computers. Most load testing programs can only simulate 25 simultaneous users with one computer. If you want to test a Website's performance with 250 simultaneous users, you'll have to use 10 different computers. Realistically, most business sites won't have to handle 250 simultaneous online customers, but 50 to 100 isn't out of the question for a local advertiser following a successful TV or radio promotion.

User simulation - Usually a load test involves running a series of somewhat different scripts, which are simply lists of actions the average users will be doing on the Website. Scripts can be created by recording the actions of a real person using a computer, or written from scratch in a programming language. For example, one script would simply be a recording of a person logging in to the Website. Another script might cover a person doing a product search on the Website. Another might be of a person loading a randomly chosen page, like a Contact Us page. Most importantly there will be scripts that cover the purchase and billing process. At any given point in time an e-commerce Website will be handling some combination of those different user actions, so any valid load test would have to include them to be accurate.Generating high traffic - There are two ways to use the scripts to create a high level of traffic, you can either repeat the same scripts over and over again, or you can program the scripts to use different information each time for what's known as data driven testing. This approach uses data from a data table like an Excel spreadsheet. (Data driven load testing opens up a lot of options, because it allows you to test a lot of basic functionality on a very large scale.)

Running the load test - It's standard practice to start off a load test with just a few simulated users, and then increase the number every few minutes until the full load is reached, usually 15 to 30 minutes later. This lets you see exactly at what level of traffic performance starts falling off. It's not uncommon for different problems to show up at different levels of traffic. (The really basic problems tend to show up at even low traffic levels.) After the load test is run, the programmers analyze the results, make modifications, and run the test again. The process is repeated until the Website reaches it's best theoretical performance, or the deadline to go live occurs, which ever comes first (and it's never the theoretical limits).

One of the leading cloud based load testing service is BrowserMob. To use BrowserMob, you just record user activities in the free automation tool Selenium, upload the scripts to the BrowserMob system, and then begin the tests. A basic report on response time is provided for each test run.

BrowserMob offers load testing at varying prices, based on testing volume level and on a weekly, monthly, or yearly time frame. The price covers setup costs and up to a certain amount of traffic. Beyond those limits and you have to pay "cloud dollars," otherwise known as hard cash. A 100 browser test for one hour costs 15 cloud dollars A 100 virtual user test for one hour costs 1.5 cloud dollars.

The "Basic" package goes for $499 a week with a maximum of 100 real browsers and 500 virtual users and a credit of $75 cloud dollars. The Advanced package at $999 a week handles 500 real browsers and 5000 virtual users and includes a credit of $185 cloud dollars. Monthly rates are about three times the weekly rate and come with about twice the cloud dollars.

Is it worth it? You be the judge. A few years ago I worked on a project that spent $10,000 for 30 days use of LoadRunner with just 250 simultaneous users. Today, twice the traffic level could be tested using BrowserMob instead, and for just one third the cost (if you bought the Advancecd package @ $3000 a month). Granted, LoadRunner would have provided a much more detailed performance report, but for many projects, the numbers BrowserMob provides would be enough.

Glen Emerson Morris was a senior QA Consultant for SAP working on a new product to help automate compliance with the Sarbanes-Oxley law, an attempt to make large corporations at least somewhat accountable to stockholders and the law. He has worked as a technology consultant for Yahoo!, Ariba, WebMD, Inktomi, Adobe, Apple and Radius.

Copyright 1994 - 2011 by Glen Emerson Morris All Rights Reserved ' keywords: Internet advertising, Internet marketing, business, advertising, Internet, marketing. For more advertising and marketing help, news, resources and information visit our Home Page.

Back to top

Economic Indicators
Census 2010
Census Bureau
Health   Labor
Commerce Dept.

It's Time to Let
A Robot
Make Your Sales Pitch!
Roy the Robot
Funded by Kickstarter