top of page

Why Bayesian methods are important for quantum machine learning?

Updated: Mar 17, 2023

Continue updating….


What is Bayesian machine leaning?

Bayesian Machine Learning (BML) is a branch of machine learning that uses Bayesian statistical methods to model and make predictions from data. In Bayesian statistics, we use probability theory to represent uncertainty and update our beliefs about a model as we observe more data.

The assets of BML is incorporating prior, which is a probability distribution that represents the initial belief or knowledge about the model parameters before observing any data, and likelihood, which measures the quantity of the model parameters with data. The focus is to estimate the posterior distribution of the model parameters rather than finding the optima single set of the parameters. The updated probability distribution of model parameters after seeing the data is called posterior. The Bayes’ theorem is the fundamental principle that describe this process,

where this is for supervised learning. The denominator is the marginal likelihood, which is the normalization of the posterior, has similarities compared to partition function in statistical physics. The marginal likelihood (also known as the model evidennce or integrated likelihood) is the probability of the observed data given a specific model, averaging over all possible values of the model parameters. It is used to evaluate the relative goodness-of-fit of different models, as it naturally incorporates a penalty for model complexity, avoiding overfitting.

There serval popular Bayesian machine learning model as,

Bayesian neural networks

Bayesian network, also known as a Bayes network or belief network, is a probabilistic graphical model that represents a set of variables and their conditional dependencies through a directed acyclic graph (DAG). Each node in the graph represents a random variable, while the directed edges between nodes signify conditional dependencies. The strength of these dependencies is quantified using conditional probability tables associated with each node. Bayesian networks are used for various tasks, including reasoning, learning, and decision-making under uncertainty. They provide a compact and intuitive representation of complex probability distributions, allowing efficient inference and prediction in a wide range of applications.

Gaussian processes

Gaussian processes (GPs) are a non-parametric, probabilistic modeling approach used in machine learning and statistics, particularly for regression and classification tasks. They define a distribution over functions, rather than parameters, enabling them to capture the complexity of the underlying data.

Mathematically, a GP is defined by a mean function, μ(x), and a covariance (kernel) function, k(x, x'), as

f(x) ~ GP(μ(x), k(x, x')),

where x and x' are input points. The kernel function encodes the similarity between input points and determines the shape and smoothness of the functions in the GP. Popular kernel choices include the squared exponential (RBF), Matérn, and periodic kernels.

For a set of input points X and their corresponding function values f(X), the GP model specifies that

f(X) ~ N(μ(X), K(X, X)),

where μ(X) is the mean function evaluated at X, K(X, X) is the covariance matrix with elements K_ij = k(x_i, x_j), and N denotes the multivariate normal distribution. Given observed data D = {X, y}, where y = f(X) + ε and ε is the observation noise, Gaussian process regression (GPR) involves conditioning the GP on the data to make predictions at new input points X*. The predictive distribution is given by

f(X*) | D, X* ~ N(μ(X*), Σ(X*)),

where the posterior mean μ(X*) and covariance Σ(X*) can be computed using the kernel matrix and the observed data

μ(X*) = K(X*, X) [K(X, X) + σ^2 I]^(-1) y,

Σ(X*) = K(X*, X*) - K(X*, X) [K(X, X) + σ^2 I]^(-1) K(X, X*),

with σ^2 representing the observation noise variance and I being the identity matrix.

Bayesian optimization

An important optimization method based on Bayesian machine learning method is called Bayesian optimization. Bayesian optimization is an efficient, global optimization technique used for optimizing expensive-to-evaluate, black-box functions. It is particularly suitable for functions that have unknown gradients, are noisy, or have expensive evaluations. Bayesian optimization is widely used in hyperparameter tuning for machine learning models and other optimization problems where direct evaluation is costly.

The core idea of Bayesian optimization is to build a probabilistic model of the objective function and then use this model to make decisions about where to sample next. The most common choice for the probabilistic model is a GP, which provides a distribution over functions and allows for uncertainty quantification.

Bayesian optimization consists of two main components: a surrogate model and an acquisition function. The surrogate model, often a GP, is used to model the underlying function. The acquisition function, guides the search for the next sampling point by trading off exploration (sampling points with high uncertainty) and exploitation (sampling points with high predicted values). Some popular acquisition functions include Expected Improvement (EI), Probability of Improvement (PI), and Upper Confidence Bound (UCB).

Bayesian optimization has been successful in various applications due to its ability to efficiently explore the search space and find global optima with a limited number of evaluations.


What is quantum machine learning?

Quantum machine learning

45 views0 comments

Recent Posts

See All


bottom of page