Simple Test - User Guide

SimpleTest is a new service by CloudIntelligence aimed to serve people that are not Load-Testing professionals.

If you are a Load-Testing professional, SimpleTest will bore you. We highly recommend using JMeter in the Cloud, which is a complete Load-Testing platform for the use of Load-Testing professionals.

If you are not a professional but looking for a complete turn-key solution for your Load-Testing project, including script creation, test execution and results analysis, you should use LoadIntelligence.

To use the SimpleTest service, you need to open an account at www.cloud-intelligence.com, register and choose Simple-Test service.

Choose the required landing pages that will participate in the test and the delay between each page request. Each landing page must include:

• A title that will assist in identifying the page in the reports
• A URL that represents the web address of the page. The URL should include the full address. For example: http://my.domain.com/landing/page?with=some&var=iables

You can include up to six landing pages.

 You need to specify the delay in seconds between each page request.

Each simulated user will call one page at a time from the list of landing pages. The user will pause for the specified delay between each page request.

Choose one of four global geographic locations from which the load will originate. At present we support East & West USA, Europe and Singapore.

Specify the maximum number of virtual users.

Once the test has started, users will ramp up to the maximum number in a period of 30 minutes. This number derives the number of servers that will participate in the test.

Using CloudIntelligence Load-Testing platform you can run unlimited size tests. However, at this time, we allow to run tests of size not greater than 4,200 concurrent virtual users. Soon we will allow unlimited size.

If you wish to execute a test of size greater than 4,200, please consider using JMeter in the Cloud.

Prior to starting the test you will need to purchase a test credit. Please follow the on-screen instructions that walk you through the purchase process.

Tests in a capacity of up to 100 concurrent users are free of charge.

We support various load scenarios. Each load scenario is different in the users ramp-up time and the continuous load after the ramp-up period.

Ramp-up time is the time it takes for users to ramp up from zero to the maximum capacity. The ramp-up time has a high influence over the website performance as the action of spawning a new thread/process to deal with a new user is very expensive in terms of resources.

Load Scenario - Users are gradually ramping up from zero to a full load capacity in a period of 30 minutes. The load then continues for additional 10 minutes and then stops.

Stress Scenario - Users are quickly ramping up from zero to a full load capacity in a period of 10 minutes. The load then continues for additional 30 minutes and then stops. In this case you are stressing your system for a period of 10 minutes, then the system is tested under continuous load.

Extreme Stress Scenario - Users are ramping up from zero to a full load capacity in a period of 5 minutes. The load then continues for additional 35 minutes and then stops. In this case you are testing your website under extreme stress conditions, where all users are ramping up with-in 5 minutes, then the system is tested under continuous load.

For example, imagine having a TV campaign, that encourages users to go to your website at a specific time. At that time your website can experience heavy traffic with-in a short period of time. This is a stress scenario, where users are ramping up from zero to a high volume in a short period of time.

If you just want to test your website under a load scenario than you do not want a short ramp-up period as it can overwhelm your server.

Embedded resources include all CSSs, JSs, Images and other resources that are part of a landing page. By downloading all embedded resources of each landing page, the response time of each landing page will be the total sum of response times for all embedded resources.

Disable this option if you want a request to be made only to the URL of the landing page and not to retrieve the embedded resources.

For example in case you are using a Content Distribution Network or smart caching of any sort for the static content, you should disable this option.

If you don't know if this is important to you or not, ignore it altogether.

Once you save the test you can start it by pressing on the Start link.

Once the test has started, please follow the instructions that appear on the page. The instructions will basically tell you to do nothing. After few minutes the reports will appear.

There is no hurry, as the test takes about 40 minutes to execute and the reports need some data to accumulate before they can generate.

Before running a load test on a website which is in a capacity greater than 100 users, we need first to make sure that you are the owner of this website. For obvious reasons we cannot allow to run a high capacity load test on a website owned by someone else.

To ensure website ownership, we require that you put a simple html file named cloudintelligence.html at the top level path of your website.

For example, if you are testing http://www.example.com/landing/page, a URL of the name: http://www.example.com/cloudintelligence.html must exist.

This action provides us with a proof that you are the owner of the website and therefore can run a load test on it.

Prior to running the load, we automatically check the availability of this file at all the domains specified in the test. If this check returns a false value on any domain, the load test will not run.

If for some reason you are unable to do so, please contact us and we will authenticate you and allow the tests to run.

A sample test (a test of up to 10 users) does not require these security/ownership measures and can run without the file being present on the domains under test.

Few minutes after the test has started, reports will begin to appear. Completed reports will be available only after it has ended.

Reports are generated in real time. To see an example of a live reports screen, please visit our demo section.

There are two types of reports:

1. Aggregated Reports
2. Detailed Reports

Aggregated Reports

Aggregated reports present various distributions of response times, latencies and errors throughout the entire test. These reports can draw the entire picture in a single snapshot.

Aggregated Reports include:

1. Two graphs: Response time distribution by percentage from the entire test and  according to the number of samples
2. Two graphs: Latency distribution by percentage from the entire test and  according to the number of samples
3. Two graphs: Error/Response Code distribution by percentage from the entire test and  according to the number of samples
4. One graph: Throughput distribution

Detailed Reports

Detailed reports present the average value of each measure per minute throughout the entire test.

Measures include:

1. Response time
2. Latency
3. Hits per minute
4. Number of concurrent users (Active Threads)
5. Errors
6. Throughput (in Kilobytes)

There is one view for all URLs that were included in the test and detailed views for each unique URL.

Detailed reports include:

1. Response Time and Latency Vs Users
2. Response Time and Latency Vs Hits
3. Response Time and Latency Vs Errors
4. Response Time and Latency Vs Throughput
5. Hits Vs Users
6. Hits Vs Errors
7. Hits Vs Throughput

To see an example of a live reports screen, please visit our demo section.

If you wish to see a different type of report, contact us and let us know.

SimpleTest is not a complete Load-Testing solution. It is not a substitute for a full test or for fixing any problem resulting from that test.

SimpleTest should be considered as limited performance checkup. In that sense, you will still need an expert to translate the findings. You will need an expert to solve any problems found. An expert may also recommend further steps.

A full test should include simulation of numerous business processes, different load scenarios over multiple iterations. All these are possible using JMeter in the Cloud Load-Testing platform.

Reports should give an initial picture of the status of your website performance and load endurance.

All the scripts and reports that were automatically generated are kept for you with-in your account in case you want to do some serious testing by using  JMeter in the Cloud.  

Please contact us if you want to use our professional services or have us recommend a service provider in your region.