Information field theory (IFT) is a Bayesian statistical field theory relating to signal reconstruction, cosmography, and other related areas.[1][2] IFT summarizes the information available on a physical field using Bayesian probabilities. It uses computational techniques developed for quantum field theory and statistical field theory to handle the infinite number of degrees of freedom of a field and to derive algorithms for the calculation of field expectation values. For example, the posterior expectation value of a field generated by a known Gaussian process and measured by a linear device with known Gaussian noise statistics is given by a generalized Wiener filter applied to the measured data. IFT extends such known filter formula to situations with nonlinear physics[disambiguation needed], nonlinear devices, non-Gaussian field or noise statistics, dependence of the noise statistics on the field values, and partly unknown parameters of measurement. For this it uses Feynman diagrams, renormalisation flow equations, and other methods from mathematical physics.[3]
Motivation
Fields play an important role in science, technology, and economy. They describe the spatial variations of a quantity, like the air temperature, as a function of position. Knowing the configuration of a field can be of large value. Measurements of fields, however, can never provide the precise field configuration with certainty. Physical fields have an infinite number of degrees of freedom, but the data generated by any measurement device is always finite, providing only a finite number of constraints on the field. Thus, an unambiguous deduction of such a field from measurement data alone is impossible and only probabilistic inference remains as a means to make statements about the field. Fortunately, physical fields exhibit correlations and often follow known physical laws. Such information is best fused into the field inference in order to overcome the mismatch of field degrees of freedom to measurement points. To handle this, an information theory for fields is needed, and that is what information field theory is.
Concepts
Bayesian inference
is a field value at a location in a space . The prior knowledge about the unknown signal field is encoded in the probability distribution . The data provides additional information on via the likelihood that gets incorporated into the posterior probability
Information Hamiltonian
In IFT Bayes theorem is usually rewritten in the language of a statistical field theory,
Fields
As fields have an infinite number of degrees of freedom, the definition of probabilities over spaces of field configurations has subtleties. Identifying physical fields as elements of function spaces provides the problem that no Lebesgue measure is defined over the latter and therefore probability densities can not be defined there. However, physical fields have much more regularity than most elements of function spaces, as they are continuous and smooth at most of their locations. Therefore, less general, but sufficiently flexible constructions can be used to handle the infinite number of degrees of freedom of a field.
A pragmatic approach is to regard the field to be discretized in terms of pixels. Each pixel carries a single field value that is assumed to be constant within the pixel volume. All statements about the continuous field have then to be cast into its pixel representation. This way, one deals with finite dimensional field spaces, over which probability densities are well definable.
In order for this description to be a proper field theory, it is further required that the pixel resolution can always be refined, while expectation values of the discretized field converge to finite values:
Path integrals
If this limit exists, one can talk about the field configuration space integral or path integral
Gaussian prior
The simplest prior for a field is that of a zero mean Gaussian probability distribution
A Gaussian probability distribution requires the specification of the field two point correlation function with coefficients
The corresponding prior information Hamiltonian reads
Measurement equation
The measurement data was generated with the likelihood . In case the instrument was linear, a measurement equation of the form
where a vector component notation was also introduced for signal and data vectors.
If the noise follows a signal independent zero mean Gaussian statistics with covariance , then the likelihood is Gaussian as well,
Free theory
Free Hamiltonian
The joint information Hamiltonian of the Gaussian scenario described above is
Generalized Wiener filter
The posterior mean
In IFT, is called the information source, as it acts as a source term to excite the field (knowledge), and the information propagator, as it propagates information from one location to another in
Interacting theory
Interacting Hamiltonian
If any of the assumptions that lead to the free theory is violated, IFT becomes an interacting theory, with terms that are of higher than quadratic order in the signal field. This happens when the signal or the noise are not following Gaussian statistics, when the response is non-linear, when the noise depends on the signal, or when response or covariances are uncertain.
In this case, the information Hamiltonian might be expandable in a Taylor-Fréchet series,
Classical field
The classical field minimizes the information Hamiltonian,
Critical filter
The Wiener filter problem requires the two point correlation of a field to be known. If it is unknown, it has to be inferred along with the field itself. This requires the specification of a hyperprior . Often, statistical homogeneity (translation invariance) can be assumed, implying that is diagonal in Fourier space (for being a dimensional Cartesian space). In this case, only the Fourier space power spectrum needs to be inferred. Given a further assumption of statistical isotropy, this spectrum depends only on the length of the Fourier vector and only a one dimensional spectrum has to be determined. The prior field covariance reads then in Fourier space coordinates .
If the prior on is flat, the joint probability of data and spectrum is
The resulting non-linear filter is called the critical filter.[4] The generalization of the power spectrum estimation formula as
The critical filter, extensions thereof to non-linear measurements, and the inclusion of non-flat spectrum priors, permitted the application of IFT to real world signal inference problems, for which the signal covariance is usually unknown a priori.
IFT application examples
The generalized Wiener filter, that emerges in free IFT, is in broad usage in signal processing. Algorithms explicitly based on IFT were derived for a number of applications. Many of them are implemented using the Numerical Information Field Theory (NIFTy) library.
- D³PO is a code for Denoising, Deconvolving, and Decomposing Photon Observations. It reconstructs images from individual photon count events taking into account the Poisson statistics of the counts and an instrument response function. It splits the sky emission into an image of diffuse emission and one of point sources, exploiting the different correlation structure and statistics of the two components for their separation. D³PO has been applied to data of the Fermi and the RXTE satellites.
- RESOLVE is a Bayesian algorithm for aperture synthesis imaging in radio astronomy. RESOLVE is similar to D³PO, but it assumes a Gaussian likelihood and a Fourier space response function. It has been applied to data of the Very Large Array.
- PySESA is a Python framework for Spatially Explicit Spectral Analysis for spatially explicit spectral analysis of point clouds and geospatial data.
Advanced theory
Many techniques from quantum field theory can be used to tackle IFT problems, like Feynman diagrams, effective actions, and the field operator formalism.
Feynman diagrams
In case the interaction coefficients in a Taylor-Fréchet expansion of the information Hamiltonian
Effective action
In order to have a stable numerics for IFT problems, a field functional that if minimized provides the posterior mean field is needed. Such is given by the effective action or Gibbs free energy of a field. The Gibbs free energy can be constructed from the Helmholtz free energy via a Legendre transformation. In IFT, it is given by the difference of the internal information energy
The Gibbs free energy is then
Minimizing the Gibbs free energy provides approximatively the posterior mean field
Operator formalism
The calculation of the Gibbs free energy requires the calculation of Gaussian integrals over an information Hamiltonian, since the internal information energy is
By the usage of the field operator formalism the Gibbs free energy can be calculated, which permits the (approximate) inference of the posterior mean field via a numerical robust functional minimization.
History
The book of Norbert Wiener[7] might be regarded as one of the first works on field inference. The usage of path integrals for field inference was proposed by a number of authors, e.g. Edmund Bertschinger[8] or William Bialek and A. Zee.[9] The connection of field theory and Bayesian reasoning was made explicit by Jörg Lemm.[10] The term information field theory was coined by Torsten Enßlin.[11] See the latter reference for more information on the history of IFT.
See also
References
- ^ Enßlin, Torsten (2013). "Information field theory". AIP Conference Proceedings. 1553 (1): 184–191. arXiv:1301.2556. Bibcode:2013AIPC.1553..184E. doi:10.1063/1.4819999.
- ^ Enßlin, Torsten A. (2019). "Information theory for fields". Annalen der Physik. 531 (3): 1800127. arXiv:1804.03350. Bibcode:2019AnP...53100127E. doi:10.1002/andp.201800127.
- ^ "Information field theory". Max Planck Society. Retrieved 13 Nov 2014.
- ^ Enßlin, Torsten A.; Frommert, Mona (2011-05-19). "Reconstruction of signals with unknown spectra in information field theory with parameter uncertainty". Physical Review D. 83 (10): 105014. arXiv:1002.2928. Bibcode:2011PhRvD..83j5014E. doi:10.1103/PhysRevD.83.105014.
- ^ Enßlin, Torsten A. (2010). "Inference with minimal Gibbs free energy in information field theory". Physical Review E. 82 (5): 051112. arXiv:1004.2868. Bibcode:2010PhRvE..82e1112E. doi:10.1103/physreve.82.051112. PMID 21230442.
- ^ Leike, Reimar H.; Enßlin, Torsten A. (2016-11-16). "Operator calculus for information field theory". Physical Review E. 94 (5): 053306. arXiv:1605.00660. Bibcode:2016PhRvE..94e3306L. doi:10.1103/PhysRevE.94.053306. PMID 27967173.
- ^ Wiener, Norbert (1964). Extrapolation, interpolation, and smoothing of stationary time series with engineering applications (Fifth printing ed.). Cambridge, Mass.: Technology Press of the Massachusetts Institute of Technology. ISBN 0262730057. OCLC 489911338.
- ^ Bertschinger, Edmund (December 1987). "Path integral methods for primordial density perturbations - Sampling of constrained Gaussian random fields". The Astrophysical Journal. 323: L103–L106. Bibcode:1987ApJ...323L.103B. doi:10.1086/185066. ISSN 0004-637X.
- ^ Bialek, William; Zee, A. (1988-09-26). "Understanding the Efficiency of Human Perception". Physical Review Letters. 61 (13): 1512–1515. Bibcode:1988PhRvL..61.1512B. doi:10.1103/PhysRevLett.61.1512. PMID 10038817.
- ^ Lemm, Jörg C. (2003). Bayesian field theory. Baltimore, Md.: Johns Hopkins University Press. ISBN 9780801872204. OCLC 52762436.
- ^ Enßlin, Torsten A.; Frommert, Mona; Kitaura, Francisco S. (2009-11-09). "Information field theory for cosmological perturbation reconstruction and nonlinear signal analysis". Physical Review D. 80 (10): 105005. arXiv:0806.3474. Bibcode:2009PhRvD..80j5005E. doi:10.1103/PhysRevD.80.105005.