sdlc.dev

QA automation path · Trust at scale · Topic 11 of 12

Performance and load testing

Not started

A load test sends many requests at once to find out how latency, errors and throughput change as traffic grows. Measure percentiles, not averages: the 95th and 99th percentile latencies are what the slowest one in twenty and one in a hundred users actually get. Ramp the load up in steps, and set thresholds so the run passes or fails like any other test.

Why it matters: Functional tests run one request at a time. Users don't. Pools running out, lock contention and slow queries only show up under realistic load.

  1. 1Measure the tail

    Report p95 and p99 latency next to the error rate and throughput. The mean hides the slow requests that users remember.

  2. 2Pick the test type

    Smoke (does the script work), average load, stress (above normal), spike, soak (hours), breakpoint (find the limit).

  3. 3Ramp, don't jump

    Increase load in steps and hold it, so you can see at what level latency and errors start to climb.

  4. 4Thresholds make it a test

    Write the goal down, such as p95 under 300 ms and under 1% errors, so the run passes or fails like any other test.

1

How it works

The explanation, with examples you can run.

Why the mean lies

A thousand latencies, 2% of which take a slow path (a cache miss, a lock, a cold connection):

⌘/Ctrl + Enter
OutputNot run yet

Press Run to see what your code prints, in order.

Runs in this browser, in an isolated frame with no access to this page. console.log, info, warn and error are shown; code runs as a classic script, so use an async function for await.

The mean looks healthy and so does p95. Only p99 shows that one request in a hundred takes over two seconds. Which percentile you set a goal on depends on how many calls make up one user action: the more calls, the further into the tail you need to look.

What to measure

  • Latency percentiles (p50, p95, p99), with successful and failed requests counted separately. A fast error is not a fast success.
  • Error rate, including wrong answers that come back with a 200.
  • Throughput: requests or completed journeys per second.
  • Saturation on the server: CPU, memory, connection pools, queue depth. That is where you find out why latency climbs.

A k6 script with a ramp and thresholds

// load.js: run with  k6 run -e BASE_URL=https://staging.example.com load.jsimport http from "k6/http"import { check, sleep } from "k6" export const options = {  stages: [    { duration: "2m", target: 50 }, // ramp up to 50 virtual users    { duration: "5m", target: 50 }, // hold    { duration: "1m", target: 0 },  // ramp down  ],  thresholds: {    http_req_failed: ["rate<0.01"],                   // under 1% errors    http_req_duration: ["p(95)<300", "p(99)<800"],    // milliseconds  },} export default function () {  const res = http.get(`${__ENV.BASE_URL}/api/products?page=1`)  check(res, { "status is 200": (r) => r.status === 200 })  sleep(1) // think time between a user's actions}

If a threshold fails, k6 exits with a non-zero code, so a CI job fails the same way it would for a broken unit test.

Closed and open load

Virtual users that wait for each response form a closed model: when the server slows down, they send less, and the test eases off exactly when it should push. To hold a fixed arrival rate whatever the response times, use an open model:

export const options = {  scenarios: {    checkout: {      executor: "constant-arrival-rate",      rate: 100, timeUnit: "1s",      // 100 new iterations every second      duration: "5m",      preAllocatedVUs: 50, maxVUs: 300,    },  },}

Making results mean something

Test an environment sized like production, with realistic data volumes (an empty database is fast), and a realistic mix of requests rather than one URL. Start with a smoke run at minimal load to check the script works. Watch the system under test, not just the load generator. A load generator that runs out of CPU produces numbers about itself, not about your service.

2

Check yourself

3 questions. Get them all right to finish this topic.

  1. 1.Mean latency is 150 ms but p99 is 2.4 s. What does that tell you?
  2. 2.In a closed model (a fixed number of virtual users, each waiting for its response), the server slows down. What happens to the load?
  3. 3.Why ramp up gradually instead of starting at full load?

0 of 3 right so far. Get all three to finish this topic.

Go deeper