← Lecture 13 | ↑ All Lectures | Lecture 15 →

Lecture 14: Determinantal Point Processes (DPPs)

Introduction

Last time:

We defined Gibbs point processes were defined via a probability density w.r.t. a homogeneous Poisson process.
Gibbs point processes form a large class of point processes with a wide range of repulsive (or attractive) behavior.
However, one major downside is that the normalizing constant has no closed form in general, and can only be approximately sampled with MCMC.
Determinantal point processes (DPPs) avoid this issue: DPPs are repulsive point processes with closed form densities
DPPs also arise naturally in other settings such as the distribution of eigenvalues of random matrixices.

Today: DPPs on $\mathbb{R}^d$.
For DPPs on finite discrete spaces, see [kulesza2012determinantal].

Determinantal Point Process

Recall: a point process $\Phi$ on $\mathbb{R}^d$ has product densities $\rho^{(k)}$ satisfying \[ \alpha^{(k)}(B) := \mathbb{E}\Big[ \Phi^{(k)}(B) \Big] = \int_{B} \rho^{(k)}(x_1,\ldots,x_k)\, dx_1\cdots dx_k, \qquad B\in\mathcal{B}((\mathbb{R}^d)^k). \]

Definition. A point process $\Phi$ on $\mathbb{R}^d$ is a determinantal point process with kernel
$K:\mathbb{R}^d\times\mathbb{R}^d\to\mathbb{C}$ if its product densities satisfy \[ \rho^{(k)}(x_1,\ldots,x_k) = \det\big[ K(x_i,x_j) \big]_{i,j=1}^k, \qquad \forall k\in\mathbb{N}. \] We write $\Phi\sim \mathrm{DPP}(K)$.

Note: $\rho^{(1)}(x)=\lambda(x)=K(x,x).$

Remark: If $\Phi\sim\mathrm{DPP}(K)$ and $B\subset\mathbb{R}^d$ is compact, then the restriction $\Phi|_B$ is distributed as $\mathrm{DPP}(K_B)$, where $K_B$ is the restriction of $K$ to $B\times B$.

Proposition. If $\Phi\sim\mathrm{DPP}(K)$ and $\widetilde{\Phi}$ is an independent thinning of $\Phi$ with retention probability $p:\mathbb{R}^d\to[0,1]$, then
\[ \widetilde{\Phi}\sim \mathrm{DPP}(\widetilde{K}), \] where \[ \widetilde{K}(x,y)=\sqrt{p(x)}\, K(x,y)\, \sqrt{p(y)}. \]

Existence Theorem for DPPs

Assumption (A): Assume the kernel $K:\mathbb{R}^d\times\mathbb{R}^d\to\mathbb{C}$ is:

Hermitian,
positive semidefinite,
continuous.

Using restriction of DPPs and Kolmogorov consistency, one can show (see Themorem 5.2.17 in [baccelli2024random]): \[ K \text{ determines a DPP on }\mathbb{R}^d \quad\Longleftrightarrow\quad K_S \text{ determines a DPP on every compact } S\subset\mathbb{R}^d. \]

Under Assumption (A), Mercer’s theorem implies that for any compact $S \subset \mathbb{R}^d$, $K_S$ has a spectral representation: \[ K_S(x,y)=\sum_{k=1}^{\infty} \lambda^S_k\, \phi_k^S(x)\, \overline{\phi^S_k(y)}, \qquad x,y\in B, \] where:

$\{\phi_k^S\}$ is an orthonormal basis for $L^2(B)$ consisting of eigenfunctions of the integral operator \[ T_S f(x)=\int_B K(x,y) f(y)\, dy, \]
$\{\lambda^S_k\}$ are eigenvalues of $T_S$ with $\sum_k \lambda_k <\infty$ and $\lambda_k\ge 0$.

Theorem. A kernel $K$ satisfying (A) defines a DPP on $\mathbb{R}^d$ iff for all compact $B\subset\mathbb{R}^d$, \[ 0\le \lambda^S_k \le 1 \qquad \forall k. \]

Construction on a Compact Set

Let $S\subset\mathbb{R}^d$ be compact.
Assume the kernel has finite-rank form

\[\begin{equation} K(x,y)=\sum_{k=1}^{n} \phi_k(x)\, \overline{\phi_k(y)}, \qquad x,y\in S, \end{equation}\]

where $\{\phi_k\}_{k=1}^n$ is orthonormal in $L^2(S)$.

Then $K$ is the kernel of a projection operator onto $\mathrm{span}\{\phi_1,\ldots,\phi_n\}$.

Definition. A DPP $\Phi\sim \mathrm{DPP}(K)$ for such a kernel is called a projection DPP.

Lemma (Number of Points). If $\Phi\sim\mathrm{DPP}(K)$ with finite-rank $K$ as above, then $\Phi(S)=n$ almost surely.

Proof. We have \[ \mathbb{E}\big[\Phi(S)\big] = \int_S K(x,x)\, dx = \sum_{k=1}^n \int_S |\varphi_k(x)|^2 \, dx = n. \]

Moreover, for any $m>n$, the rank of the matrix $ _{i,j=1}^m $ is at most $n$, hence its determinant satisfies $ ^{(m)} = 0, , m>n. $ Thus $\Phi(S) \le n$ almost surely; combined with $\mathbb{E}[\Phi(S)] = n$, this implies $\Phi(S)=n$ almost surely.

$\square$

Geometric Interpretation

Let $v(x)=(\phi_1(x),\ldots,\phi_n(x))\in\mathbb{C}^n.$ Then, $K(x,y)=\langle v(x), v(y)\rangle,$ and \[ \rho^{(n)}(x_1,\ldots,x_n)=\det[K(x_i,x_j)]_{i,j=1}^n = \mathrm{vol}(P(v(x_1),\ldots,v(x_n)))^2. \]

Algorithm (Sampling projection DPP). Given $K(x,y)=\sum_{k=1}^{n}\phi_k(x)\overline{\phi_k(y)}$, define $v(x)$ as above.

Sample $X_1$ from density
\[ p_1(x)=\frac{\|v(x)\|^2}{n}, \qquad x\in S. \]
Let $w_1=v(X_1)$ and $e_1=w_1/\|w_1\|$.
For $i=2,\ldots,n$: \[\begin{align*} p_i(x) &= \frac{1}{n-i+1}\Big\| v(x)-\sum_{j=1}^{i-1} \mathrm{proj}_{e_j} v(x)\Big\|^2, \\ w_i &= v(X_i) - \sum_{j=1}^{i-1}\mathrm{proj}_{e_j} v(X_i), \\ e_i &= \frac{w_i}{\|w_i\|}. \end{align*}\]
Output $\{X_1,\ldots,X_n\}$.

Proposition. The sample $\{X_1,\ldots,X_n\}$ returned by the above algorithm is distributed as $\mathrm{DPP}(K)$ for $K$ as in Eq.~[ref].

Proof. The joint density of the ordered vector $(X_1,\ldots,X_n)$ is \[ p(x_1,\ldots,x_n) = \frac{1}{n!}\, \mathrm{Vol}\!\big( P(v(x_1),\ldots,v(x_n)) \big)^2 = \frac{1}{n!}\, \det\!\big( K(x_i,x_j) \big)_{i,j=1}^n . \]

For the unordered collection $\{X_1,\ldots,X_n\}$, we have \[ \rho^{(n)}(x_1,\ldots,x_n) = n!\, p(x_1,\ldots,x_n) = \det\!\big( K(x_i,x_j) \big)_{i,j=1}^n . \]

$\square$

General DPP Kernels via Bernoulli Thinning

Theorem. Suppose $\Phi \sim \mathrm{DPP}(K)$ on $S$ with kernel \[ K(x,y) = \sum_{k=1}^{\infty} \lambda_k\, \varphi_k(x)\overline{\varphi_k(y)}, \qquad x,y\in S, \] where $\{\varphi_k\}_{k\ge 1}$ is an orthonormal set in $L^2(S)$,
$\{\lambda_k\}_{k\ge 1}$ satisfies $\sum_k \lambda_k < \infty$ and $\lambda_k\in[0,1]$.

Let $\{I_k\}_{k\ge1}$ be independent random variables with
\[ I_k \sim \mathrm{Bernoulli}(\lambda_k), \] and define \[ K_I(x,y) := \sum_{k=1}^{\infty} I_k\, \varphi_k(x)\overline{\varphi_k(y)}. \] Then, if $\Phi_I \sim \mathrm{DPP}(K_I)$, we have \[ \Phi_I \stackrel{d}{=} \Phi. \]

Note: $\Phi(S) \sim \sum_{k=1}^{\infty} I_k$, so $\mathbb{E}[\Phi(S)] = \sum_{k\ge1} \lambda_k$, $\mathrm{Var}(\Phi(S)) = \sum_{k\ge1} \lambda_k(1-\lambda_k)$, and $\mathbb{P}(\Phi(S)=0) = \prod_{k\ge1} (1-\lambda_k).$

Proof. [Proof idea] Show that the product densities of $\Phi_I$ and $\Phi$ are equal, i.e. \[ \mathbb{E}\!\left[ \det\big( K_I(x_i,x_j) \big)_{i,j=1}^k \right] = \det\big( K(x_i,x_j) \big)_{i,j=1}^k . \] See the proof of Theorem 7 in [hough2006determinantal].

$\square$

Proof of the Existence Theorem

Proof. Proof of the existence theorem

[$(\Leftarrow)$] This direction is proved by constructing $\Phi_I$ and applying the previous result.
[$(\Rightarrow)$]
Suppose there exists a compact set $S \subset \mathbb{R}^d$ such that \[ K_S(x,y) = \sum_{k=1}^{\infty} \lambda_k \varphi_k(x)\overline{\varphi_k(y)}, \] and let the maximum eigenvalue satisfy $\lambda_1 > 1$. Let $\Phi \sim \mathrm{DPP}(K_S)$, and let $\Phi_1$ be an independent thinning of $\Phi$ with retention probability $1/\lambda_1$.

Then
\[ \Phi_1 \sim \mathrm{DPP}\!\left(\tfrac{1}{\lambda_1} K_S\right). \]

Since $\tfrac{1}{\lambda_1} < 1$ and $\Phi$ has finitely many points almost surely,
\[ \mathbb{P}\big[ \Phi_1(S)=0 \big] > 0 \qquad\text{(there is a positive probability that thinning removes all points)}. \]

However all eigenvalues of $\tfrac{1}{\lambda_1} K_S$ lie in $[0,1]$, and at least one equals 1.
Hence \[ \mathbb{P}\big[ \Phi_1(S) \ge 1 \big] = 1, \] a contradiction.

$\square$

← Lecture 13 | ↑ All Lectures | Lecture 15 →