Basics

Mathematical Notation

• \(\sum_{i=1}^{n} a_i\) : summation over terms from \(i=1\) to \(n\)
• \(\prod_{i=1}^{n} a_i\) : product over terms from \(i=1\) to \(n\)
• \(\mathbf{x} \in \mathbb{R}^{d}\) : vector \(\mathbf{x}\) lives in a \(d\)-dimensional real space
• \(\mathbf{W} \in \mathbb{R}^{d \times k}\) : matrix \(\mathbf{W}\) has shape \(d \times k\)
• \(\mathbf{x}^\top\) : transpose of vector or matrix \(\mathbf{x}\)
• \(\mathbf{x}^\top \mathbf{y}\) : inner product / dot product
• \(\lVert \mathbf{x} \rVert_2\) : Euclidean norm of \(\mathbf{x}\)
• \(\langle \mathbf{x}, \mathbf{y} \rangle\) : inner product notation
• \(\mathbf{x} \odot \mathbf{y}\) : element-wise (Hadamard) product
• \(\mathbb{E}[X]\) : expectation of random variable \(X\)
• \(\mathrm{Var}(X)\) : variance of random variable \(X\)
• \(\mu\) : mean
• \(\sigma\) : standard deviation
• \(\sigma^2\) : variance
• \(P(A \mid B)\) : probability of \(A\) given \(B\)
• \(\max(x, y)\) : maximum of two values
• \(\arg\max_x f(x)\) : value of \(x\) that maximizes \(f(x)\)
• \(\mathrm{sign}(x)\) : sign of \(x\)
• \(\sqrt{x}\) : principal square root
• \(\lfloor x \rfloor\) : floor function
• \(\frac{d}{dx} f(x)\) : derivative of \(f\) with respect to \(x\)
• \(\frac{\partial f}{\partial x}\) : partial derivative of \(f\) with respect to \(x\)
• \(\nabla_{\mathbf{x}} f\) : gradient of \(f\) with respect to \(\mathbf{x}\)
• \(f(x) \propto g(x)\) : \(f(x)\) is proportional to \(g(x)\)
• \(a \approx b\) : \(a\) is approximately equal to \(b\)