Complete Characterization of Gorbunov and Pinsker Nonanticipatory Epsilon Entropy of Multivariate Gaussian Sources: Structural Properties
Loading...
Access rights
openAccess
publishedVersion
URL
Journal Title
Journal ISSN
Volume Title
A1 Alkuperäisartikkeli tieteellisessä aikakauslehdessä
This publication is imported from Aalto University research portal.
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
View publication in the Research portal (opens in new window)
View/Open full text file from the Research portal (opens in new window)
Other link related to publication (opens in new window)
Date
2022-03-01
Major/Subject
Mcode
Degree programme
Language
en
Pages
25
Series
IEEE Transactions on Information Theory, Volume 68, issue 3, pp. 1440-1464
Abstract
This paper derives the optimal test channel distribution and the complete characterization of the classical Gorbunov and Pinsker (1973), Gorbunov and Pinsker (1974) nonanticipatory epsilon entropy of multivariate Gaussian Markov sources with square-error fidelity, which remained an open problem since 1974. The paper also formulates a state dependent nonanticipatory epsilon entropy, in which past reproductions are available to the decoder and not to the encoder, the test channel is specified with respect to an auxiliary (state) random process, and the reproduction process is a causal function of past reproduction and the auxiliary random process. This variation is analogous to the Wyner and Ziv (1976) and Wyner (1978) rate distortion function (RDF), of memoryless sources. It is shown that the operational rate of zero-delay codes, with past reproductions available to the decoder but not to the encoder is bounded below by the state dependent nonanticipatory epsilon entropy rate. For the case of multivariate Gaussian Markov sources with square-error fidelity, the optimal test channel distribution and the complete characterization of the state dependent of nonanticipatory epsilon entropy are derived, and also shown that that the two nonanticipatory epsilon entropies coincide. The derivations are new; they are based on structural properties of the stochastic realizations of the reproduction process that induce the optimal test channel distributions. They are derived using, achievable lower bounds on information theoretic measures, properties of mean-square estimation theory, Hadamard's inequality, and canonical correlation coefficients of a tuple of multivariate jointly Gaussian random processes. Applications of the nonanticipatory epsilon entropy and its state dependent variation are discussed to the areas of control of unstable Gaussian systems over limited memory channels, design of causal estimators for Gaussian Markov sources with a fidelity criterion, computation of the rate loss of causal and zero-delay codes of Gaussian Markov sources with respect to non-causal codes.Description
Publisher Copyright: Author
Keywords
Channel estimation, Codes, Decoding, Distortion, Entropy, Markov processes, multivariate Gaussian processes, Nonanticipatory epsilon entropy, Resource description framework, square-error fidelity criterion
Other note
Citation
Charalambous, C D, Charalambous, T, Kourtellaris, C & van Schuppen, J H 2022, ' Complete Characterization of Gorbunov and Pinsker Nonanticipatory Epsilon Entropy of Multivariate Gaussian Sources : Structural Properties ', IEEE Transactions on Information Theory, vol. 68, no. 3, pp. 1440-1464 . https://doi.org/10.1109/TIT.2021.3124888