Performance Engineering Interview Questions

Load testing checks behavior at an expected, realistic traffic level to validate SLAs. Stress testing pushes past that level deliberately to find the breaking point and confirm the system degrades gracefully (or to find the bottleneck). Soak testing runs a moderate, sustained load for hours or days to surface issues averages and short tests miss — memory leaks, connection pool exhaustion, log disk fill-up, clock drift.

L = λW: the average number of concurrent requests in a system (L) equals the arrival rate (λ) times the average time each request spends in the system (W). It lets you solve for any one variable given the other two — e.g. how many concurrent users you need to hit a target throughput at a known response time, or what throughput a fixed pool of workers can sustain. It's a conservation law, not a model with parameters to fit, so it holds for any stable system.

An average is dominated by the bulk of fast requests and barely moves when a tail of users has a bad experience. Percentiles describe the shape of the distribution, including the tail real users feel — p99 tells you what your slowest 1% of users actually experience, which is often what triggers complaints or SLA breaches even when the mean looks healthy.

Throughput is requests completed per unit time; concurrency is how many requests are in flight at once. They're related by Little's Law, not interchangeable — you can have high concurrency with low throughput (slow backend, requests pile up) or low concurrency with high throughput (fast backend, requests don't linger). Confusing them leads to sizing decisions based on the wrong number, e.g. provisioning for peak concurrent users when the system is actually throughput-bound.

A closed model has a fixed number of virtual users, each waiting for a response before sending the next request — throughput is self-limiting because slow responses automatically reduce request rate. An open model generates requests at a fixed arrival rate regardless of how fast the system responds, which is what real internet traffic looks like and is far more likely to expose queueing collapse under overload. Most load testing tools default to closed-model; this is a common reason load tests look fine but production falls over.

k6 is code-first (JavaScript), lightweight, scripts version well in Git, and is efficient per-machine because it's compiled Go under the hood — good fit for CI pipelines and teams comfortable writing test logic as code. JMeter is GUI-first with a huge plugin ecosystem and protocol support (JDBC, JMS, LDAP, etc.), which suits teams that need broad protocol coverage or prefer a visual test plan, at the cost of being heavier per virtual user and clunkier to keep in version control.

Correlation is extracting a dynamic value from one response (a session token, CSRF token, order ID) and substituting it into subsequent requests. It's necessary because a recorded script with hardcoded values will replay against stale state and fail or hit a different code path (e.g. always operating on the same already-completed order) — the script needs to behave like a real client that adapts to what the server just returned.

JMeter's GUI mode and its JVM threading model add significant per-thread overhead, plus its default samplers and listeners aren't tuned for high concurrency. Common fixes: run in non-GUI/CLI mode, disable or minimize listeners during the run, increase heap (-Xmx), and if a single machine still can't generate enough load, distribute across multiple injector machines.

Pacing (or think time) is the deliberate delay between a virtual user's iterations to mimic real user behavior. A script that hammers requests back-to-back with no delay creates unrealistic concurrency for a given throughput target — you'll either need far fewer virtual users than reality to hit your target rate, or you'll generate far more load than real users would, both of which produce numbers that don't transfer to production capacity planning.

Use CSV data sets, randomized generators, or a parameterization library so each virtual user (or iteration) uses distinct data rather than one shared value — reusing the same login or order ID across thousands of virtual users can mask concurrency bugs (row locking, duplicate-key handling) and trips caching layers in ways that don't reflect real traffic diversity.

This pattern (vs. a flat plateau) points to resource accumulation rather than pure undercapacity: connection or thread pool leaks, unbounded caches/queues, GC pressure from accumulating garbage, or a slow memory leak. Check GC log overhead and heap trend, open file descriptor / connection counts over time, and queue depths — a leak shows monotonic growth in one of these even at constant load, which a single short test would never reveal.

CPU-bound: high CPU utilization (often pegged near 100%) with low wait time in thread dumps, threads mostly RUNNABLE. I/O-bound: CPU is low/moderate while threads sit in WAITING/TIMED_WAITING on socket or disk operations — check disk iowait and network round-trip times. Lock-bound: high BLOCKED thread counts in dumps, with one identifiable thread holding the contended monitor; CPU may look fine because threads are queued, not computing.

It's the total time the JVM spends paused for garbage collection divided by total wall-clock run time. Above roughly 5% you should start watching it; above ~10% it's usually costing meaningful throughput and worth tuning (heap sizing, generation sizing, or collector choice) before looking elsewhere, since no amount of application-level optimization fixes time lost to GC pauses.

This usually indicates a queueing or contention effect with a hard limit, not gradual resource exhaustion — e.g. a connection pool maxing out and new requests timing out instead of queueing, a lock that becomes a bottleneck only once enough threads contend for it simultaneously, or hitting a hard OS limit (file descriptors, ephemeral ports). The Universal Scalability Law models exactly this: contention and coherency costs cause throughput to peak and then decline, not just plateau, as concurrency increases.

Look for request latency that suddenly spikes once concurrency crosses a threshold, paired with the application/connection-pool metrics showing active connections pinned at the configured max and a growing wait queue for connections. Thread dumps will show many threads blocked waiting to acquire a connection from the pool rather than doing work. The fix is rarely just 'increase pool size' — first check if connections are being held too long (slow queries, missing timeouts, connections not returned on error paths).

An SLI (indicator) is a measured metric, like request latency or error rate. An SLO (objective) is the target you set for that metric, e.g. '99.9% of requests under 300ms.' An SLA (agreement) is the SLO turned into a contractual promise with consequences (credits, penalties) if breached — SLOs are usually set stricter than SLAs to leave margin before contractual exposure.

An error budget is the allowed amount of unreliability implied by an SLO — e.g. a 99.9% SLO allows ~43 minutes of downtime a month. Once the budget is spent, the team is expected to halt risky releases and focus on reliability work until the budget recovers; while budget remains, teams can ship faster. It reframes reliability as a shared, quantified resource instead of an abstract 'be careful' directive, and gives both product and SRE teams an objective trigger for slowing down.

A single threshold (e.g. 'alert if error rate > 1%') either fires too late for fast, severe outages or too often for brief blips. Multi-window burn-rate alerting checks the same SLO at multiple time windows (e.g. 5 minutes and 1 hour) with different severity thresholds, so a sudden severe spike pages immediately while a slow, sustained burn that would exhaust the monthly budget also gets flagged before it's catastrophic, without flapping on momentary noise.

Start from current per-instance throughput at acceptable latency (from load tests, not guesses), use Little's Law to translate the new expected concurrency/throughput into required instance count, and load-test at the projected scale rather than extrapolating linearly — non-linear bottlenecks (a shared DB, a rate-limited third-party API, a single-instance cache) often don't scale with the rest of the fleet. Always test the actual constraint, not just the easiest layer to scale (web tier).

When a downstream service slows down or partially fails, naive retries amplify load on it — every failed request becomes 2+ requests, compounding the original problem and potentially turning a brief blip into a full outage. Mitigations: exponential backoff with jitter, retry budgets/circuit breakers that stop retrying once a failure rate threshold is hit, and ensuring retries aren't unbounded in count or fan-out.

Output length varies per request (latency scales with tokens generated, which isn't known in advance), batching behavior on the serving side means a request's latency depends on what else is co-scheduled, and cold model loads or KV-cache eviction under memory pressure add irregular spikes. This makes p99 latency for LLM endpoints far less predictable than for a typical CRUD API, and averages are even less representative than usual.

Estimate tokens per request (input + expected output) from representative prompts, multiply by the per-token price for the chosen model/tier, then multiply by expected request volume — and separately model worst-case cost from long-tail prompts (very long context, retries) since those can dominate spend even at low frequency. Always re-validate against real usage after launch since prompt patterns in production rarely match design-time assumptions.

Throughput (tokens/sec, or requests/sec) measures overall serving capacity; TTFT measures how long a user waits before seeing the first streamed token, which dominates perceived responsiveness for interactive use cases. A system can have excellent throughput under batching but poor TTFT if requests queue behind a large batch, so the two need to be tuned and reported separately rather than collapsed into one number.

GPU instances are expensive to keep warm, so autoscaling-to-zero is tempting for cost — but loading model weights onto a GPU (often gigabytes) takes far longer than a typical serverless cold start, sometimes tens of seconds, which is unacceptable for latency-sensitive requests. Mitigations include provisioned/minimum warm instances, smaller models for latency-critical paths, or accepting cold starts only for batch/async workloads.