Order Statistics & Quantiles theory

Let X_1, X_2, ..., X_n be a sample of n independent and identically distributed continuous random variables. When the sample values are sorted in ascending algebraic order: X_{(1)} ≤ X_{(2)} ≤ ... ≤ X_{(n)}, the random variable X_{(r)} is called the r-th order statistic. X_{(1)} = min(X_1, ..., X_n) is the sample minimum, X_{(n)} = max(X_1, ..., X_n) is the sample maximum, and X_{((n+1)/2)} (for odd n) is the sample median. Because the parent distribution is continuous, the probability of tied values P(X_i = X_j) is zero, guaranteeing strict ordering X_{(1)} < X_{(2)} < ... < X_{(n)} almost surely.

For an observation to fall in [x, x + dx] as the r-th order statistic, exactly (r - 1) observations must be strictly less than x, exactly 1 observation must fall in [x, x + dx], and the remaining (n - r) observations must be strictly greater than x. By the multinomial counting rule, the probability density function is f_{(r)}(x) = [n! / ((r - 1)! 1! (n - r)!)] · [F(x)]^(r - 1) · [1 - F(x)]^(n - r) · f(x). For the sample minimum (r = 1): f_{(1)}(x) = n [1 - F(x)]^(n-1) f(x). For the sample maximum (r = n): f_{(n)}(x) = n [F(x)]^(n-1) f(x).

For a standard uniform distribution X ~ U[0, 1], F(x) = x and f(x) = 1 for 0 ≤ x ≤ 1. Substituting these into the general order statistic density formula yields: f_{(r)}(x) = [n! / ((r - 1)! (n - r)!)] x^(r - 1) (1 - x)^(n - r) = [1 / B(r, n - r + 1)] x^(r - 1) (1 - x)^((n - r + 1) - 1). This is precisely the probability density function of a Beta distribution with parameters α = r and β = n - r + 1: X_{(r)} ~ Beta(r, n - r + 1). Consequently, its theoretical expectation is E[X_{(r)}] = α / (α + β) = r / (r + n - r + 1) = r / (n + 1).

The joint density of the original unordered sample (X_1, ..., X_n) is ∏_{i=1}^n f(x_i). There are exactly n! distinct permutations of n distinct values that sort to the same ordered sequence x_1 < x_2 < ... < x_n. Therefore, the joint probability density function of the vector of order statistics (X_{(1)}, ..., X_{(n)}) is f_{(1),...,(n)}(x_1, ..., x_n) = n! ∏_{i=1}^n f(x_i) for x_1 < x_2 < ... < x_n, and 0 elsewhere. For standard uniform variables, f(x) = 1, so the joint density is constant: f(x_1, ..., x_n) = n! over the simplex 0 < x_1 < ... < x_n < 1.

Define the sample spacings as D_0 = X_{(1)}, D_r = X_{(r+1)} - X_{(r)} for r = 1, ..., n - 1, and D_n = 1 - X_{(n)}. These n + 1 spacings represent the lengths of the sub-intervals into which the n order statistics partition the unit interval [0, 1]. Notice that ∑_{r=0}^n D_r = 1. Remarkably, for standard uniform data, all n + 1 spacings are identically distributed random variables, each following a Beta(1, n) distribution with common expectation E[D_r] = 1 / (n + 1). This explains why the expected value of X_{(r)} increases by exactly 1/(n + 1) for each increment of rank r.

Extreme Value Theory (EVT), formalized by the Fisher-Tippett-Gnedenko theorem, studies the asymptotic distribution of the sample maximum X_{(n)} = max(X_1, ..., X_n). As n → ∞, properly normalized maxima converge in distribution to one of three extreme value families: Gumbel (type I, light exponential tails), Fréchet (type II, heavy power-law tails), or Weibull (type III, bounded upper support), unified under the Generalized Extreme Value (GEV) distribution. EVT allows engineers to estimate the 100-year flood level or financial risk managers to calculate extreme Value-at-Risk (VaR).

Let m be the population median such that F(m) = 0.5. If the density f(m) > 0 is continuous at m, then by the Bahadur representation, the sample median X̃_n is asymptotically normally distributed: √n (X̃_n - m) →^d N(0, 1 / [4 f(m)^2]). For a standard normal distribution N(0, 1), f(0) = 1 / √(2π), so the asymptotic variance of the sample median is 1 / [4 (1/(2π))] = π / 2 ≈ 1.5708. Thus, the asymptotic relative efficiency (ARE) of the sample median relative to the sample mean is 1 / (π/2) = 2/π ≈ 63.66% for normal data, but exceeds 100% for heavy-tailed distributions like Laplace or Student's t.