Load testing: Gatling "Tips & Tricks"

english
gatling
load-testing
methodology

When you work on a high-traffic application, or simply deploy a new application in production, you need to be able to identify the load your application and infrastructure are ready to absorb.

It is also very important to test the behaviour of your application well: indeed, if part of it has cache, for example, it is important to take it into account and try to simulate a "real" load.

Gatling Load Test

To test this load, different tools exist on the market allowing you to "script" scenarios, depending on the language you like, among them: Locust if you want to develop your scenarios in Python, or Vegeta developed in Go but allowing to create scenarios quite simply.

Overview of Gatling

I chose to use the tool https://gatling.io/ for my load testing because I find it quite complete both in the different needs of the scenarios but also in its output reports.

Gatling is a tool written in https://www.scala-lang.org/, which requires a JVM for its execution, so you will need Java tools for its execution. It is of course an open-source product even if an enterprise (and therefore paid) offer is available in case you need to obtain reports on a robust application quickly and simply.

The objective of this article is to demonstrate to you technically the advantages that Gatling brings to the management of your scenarios.

Query factor per second / Duration factor

When you run a scenario, you will certainly want to be able to quickly and easily adapt the number of users you want to simulate as well as the duration of the scenario. To do this, you can simply set up two variables in your scenario that will allow you to perform a multiplier factor. For example:

val rps_factor = Integer.getInteger("rpsFactor", 1).toInt
val time_factor  = Integer.getInteger("timeFactor", 1).toInt

Then, assign to your test scenarios the load values you want by passing the multiplier coefficient:

  setUp(
    scn.inject(
      rampUsers(50 * rps_factor) over(5 * time_factor seconds),
      rampUsers(100 * rps_factor) over(10 * time_factor seconds),
    )
    .protocols(httpConf)
  )

This way, you can pass the value 1 for your scenario compilation tests (so that the tests do not run for too long), and higher values for your production tests.

Feeders

This is usually one of the first concepts you will need to run your tests but we will still talk about it: feeders, being able to provide your test case with a set of values. Let's take a simple case: you want to test the load of an API in production and to avoid caching, you want to make calls to get a different resource each time. In this case, simply create a CSV file with a list of valid identifiers.

So let's create the user-files/data/my-identifiers.csv file:

myIdentifiers
e9fbd24b-31f8-498f-ba03-7d758d4d2a17
2a012137-ec39-4d37-b2b7-0fc3186f78a0
507a036e-a946-4e82-ae52-305306981694

Then load this file into your code and give it to your scenario to use as follows:

val myIdentifiersFeeder = csv("my-identifiers.csv").random

val scn = scenario("FTPPubSimulation")
  .feed(feeder)
  .exec(http("Call to obtain my ressource")
    .get("/my-ressource/${myIdentifiers}")
    .queryParam("id_diffusion", "${metaId}"))

Until then, nothing complicated and this is already explained in the documentation. Let's now look at some other tips that I hope will be useful to you if you are not very familiar with Gatling and the Scala language.

Condition based on a previous request

Imagine that you make a request to an API, and that you want to make a second request only in case a value is defined in your first request. This is possible with the following syntax:

.exec(
    http("GET /api/conditionner/{conditionID}")
    .get("/api/conditionner/${conditionID}")
    .check(
        jsonPath("$..purchaseId").findAll.saveAs("purchaseID")
    )
    .doIf("${purchaseID.exists()}") {
        exec(http("GET /api/purchase/{purchaseID}")
        .get("/api/purchase/${purchaseID(0)}")
      )
    }
)

Here, the second request (/api/purchase/...) will only be executed in the case where a "purchaseId" field is returned by the first request. For more information on the various conditions you can put in place, I invite you to go to the following page: https://gatling.io/docs/2.3/general/scenario/#conditional-statements.

Polling: waiting for an asynchronous task

If you have workers or simply tasks that are processed asynchronously in your architecture, you will understand that you will have to wait for them to be executed. Gatling also allows you to manage this case with the functions tryMax and check :

.tryMax(100) {
    pause(1)
    .exec(http("GET /api/registration/{registrationID}")
        .get("/api/registration/${registrationID(0)}")
        .check(
            jsonPath("$..purchaseId").findAll.saveAs("purchaseID")
        )
    )
}
...

Here, we will call the HTTP request every second (because we pause for one second) up to 100 times, unless the "purchaseID" field is found in the response, in which case the check will be noted as positive and the scenario will proceed to the next execution.

Random query

You may want to spread the load over two different types of requests: for example, accepting or rejecting a registration (randomly). To do this, you will be able to generate a random number (0 or 1) and use doIfEqualsOrElse to call the corresponding query:

exec(session => {
    val rnd = new scala.util.Random
    session.set("randomInt", 0 + rnd.nextInt((1 - 0) + 1))
})
.doIfEqualsOrElse("${randomInt}", 1) {
    exec(
        http("PUT /api/contract/{contractID}/accept")
        .put("/api/contract/" + contractID + "/accept")
        .header("Cookie", "_token=" + token)
        .check(status.not(404), status.not(500))
    )
} {
    exec(
        http("PUT /api/contract/{contractID}/refuse")
        .put("/api/contract/" + contractID + "/refuse")
        .header("Cookie", "_token=" + token)
        .check(status.not(404), status.not(500))
    )
}

Thus, rather than testing only one API, you can simulate a query load that is more consistent with the reality of your application.

Loop on a list of items

The keyword foreach will allow you to loop on a list of elements in order to perform actions on these elements:

exec(
    http("GET /api/contract/{contractID}/users")
    .get("/api/contract/${contractID}/users")
    .queryParam("limit", "10")
    .check(
        jsonPath("$.results[*].id").findAll.saveAs("userID")
    )
)
.foreach("${userID}", "elementID") {
    exec(
        http("PUT /api/user/{elementID}")
        .put("/api/user/${elementID}")
        .header("Cookie", "_token=" + token)
        .header("Content-Type", "application/json")
        .body(StringBody("""{"status": "status_has_contract"}""")).asJSON
        .check(status.not(404), status.not(500))
    )
}

In this example, the PUT request will therefore be executed on all user elements returned by the first request.

Play tests on several nodes

In case you need to generate a high load, a single machine may not be enough, either in terms of available resources (CPU, RAM) or bandwidth. Gatling allows you to aggregate data from several test reports. Thus, the idea is to play the reports on several machines at the same time and to aggregate the reports to obtain only one. Of course, this fact must be taken into account in the user values and time of your scenarios.

To automate this, I took the time to make the following bash script that runs one of my scenarios and connects to the different servers to run them simultaneously. Let's start by preparing the various variables of our script:

#!/bin/bash

# Assuming we use this user for all hosts
USER_NAME='root'

# Remote hosts list
HOSTS=( 1.1.1.1 2.2.2.2 3.3.3.3 4.4.4.4 )

# Assuming all Gatling installation are in the same path (with write permissions)
GATLING_HOME=/opt/gatling/my-project
GATLING_SIMULATIONS_DIR=$GATLING_HOME/user-files/simulations
GATLING_RUNNER=$GATLING_HOME/bin/gatling.sh

# Simulation class name
SIMULATION_NAME='mynamespace.MyTestSimulation'

GATLING_REPORT_DIR=$GATLING_HOME/results/
GATHER_REPORTS_DIR=/gatling/reports/

Before starting, we will also clean up any previous test report that may remain locally and on remote servers:

echo "Cleaning previous runs from localhost"
rm -rf reports
rm -rf $GATHER_REPORTS_DIR
mkdir $GATHER_REPORTS_DIR
rm -rf $GATLING_REPORT_DIR

for HOST in "${HOSTS[@]}"
do
  echo "Cleaning previous runs from host: $HOST"
  ssh -n -f $USER_NAME@$HOST "sh -c 'rm -rf $GATLING_REPORT_DIR'"
done

Now it's time to update the scenarios on the servers and run them:

for HOST in "${HOSTS[@]}"
do
  echo "Copying simulations to host: $HOST"
  scp -r $GATLING_SIMULATIONS_DIR/* $USER_NAME@$HOST:$GATLING_SIMULATIONS_DIR
done

for HOST in "${HOSTS[@]}"
do
  echo "Running simulation on host: $HOST"
  ssh -n -f $USER_NAME@$HOST "sh -c 'nohup $GATLING_RUNNER -nr -s $SIMULATION_NAME > /gatling/run.log 2>&1 &'"
done

Once the tests are completed, all you have to do is retrieve the log files and aggregate them:


for HOST in "${HOSTS[@]}"
do
  echo "Gathering result file from host: $HOST"
  ssh -n -f $USER_NAME@$HOST "sh -c 'ls -t $GATLING_REPORT_DIR | head -n 1 | xargs -I {} mv ${GATLING_REPORT_DIR}{} ${GATLING_REPORT_DIR}report'"
  scp $USER_NAME@$HOST:${GATLING_REPORT_DIR}report/simulation.log ${GATHER_REPORTS_DIR}simulation-$HOST.log
done

mv $GATHER_REPORTS_DIR $GATLING_REPORT_DIR
echo "Aggregating simulations"
$GATLING_RUNNER -ro reports

And that's it, your report is now available. You can aggregate all these commands to create a bash script that automatically executes these steps.

Conclusion

Gatling is a complete tool that allows you to create test scenarios tailored to your application. The Scala language is not very complicated to learn in this case and you can write your scenarios quite simply. Also, the reports provided by Gatling are quite pleasant to read and its log file operation allows logs from several sources to be aggregated, so it is possible to run it on several nodes in order to simulate a high load.