## 2010 |

Olmos, Pablo M; Murillo-Fuentes, Juan Jose; Perez-Cruz, Fernando Tree-Structure Expectation Propagation for Decoding LDPC Codes over Binary Erasure Channels Inproceedings 2010 IEEE International Symposium on Information Theory, pp. 799–803, IEEE, Austin, TX, 2010, ISBN: 978-1-4244-7892-7. Abstract | Links | BibTeX | Tags: belief propagation, binary erasure channels, Bipartite graph, BP decoder, Capacity planning, Channel Coding, codeword, computational complexity, Decoding, Finishing, graph theory, H infinity control, LDPC code decoding, LDPC Tanner graph, Maxwell decoder, parity check codes, Performance analysis, tree structure expectation propagation, trees (mathematics), Upper bound @inproceedings{Olmos2010, title = {Tree-Structure Expectation Propagation for Decoding LDPC Codes over Binary Erasure Channels}, author = {Pablo M Olmos and Juan Jose Murillo-Fuentes and Fernando Perez-Cruz}, url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5513636}, isbn = {978-1-4244-7892-7}, year = {2010}, date = {2010-01-01}, booktitle = {2010 IEEE International Symposium on Information Theory}, pages = {799--803}, publisher = {IEEE}, address = {Austin, TX}, abstract = {Expectation Propagation is a generalization to Belief Propagation (BP) in two ways. First, it can be used with any exponential family distribution over the cliques in the graph. Second, it can impose additional constraints on the marginal distributions. We use this second property to impose pair-wise marginal distribution constraints in some check nodes of the LDPC Tanner graph. These additional constraints allow decoding the received codeword when the BP decoder gets stuck. In this paper, we first present the new decoding algorithm, whose complexity is identical to the BP decoder, and we then prove that it is able to decode codewords with a larger fraction of erasures, as the block size tends to infinity. The proposed algorithm can be also understood as a simplification of the Maxwell decoder, but without its computational complexity. We also illustrate that the new algorithm outperforms the BP decoder for finite block-size codes.}, keywords = {belief propagation, binary erasure channels, Bipartite graph, BP decoder, Capacity planning, Channel Coding, codeword, computational complexity, Decoding, Finishing, graph theory, H infinity control, LDPC code decoding, LDPC Tanner graph, Maxwell decoder, parity check codes, Performance analysis, tree structure expectation propagation, trees (mathematics), Upper bound}, pubstate = {published}, tppubtype = {inproceedings} } Expectation Propagation is a generalization to Belief Propagation (BP) in two ways. First, it can be used with any exponential family distribution over the cliques in the graph. Second, it can impose additional constraints on the marginal distributions. We use this second property to impose pair-wise marginal distribution constraints in some check nodes of the LDPC Tanner graph. These additional constraints allow decoding the received codeword when the BP decoder gets stuck. In this paper, we first present the new decoding algorithm, whose complexity is identical to the BP decoder, and we then prove that it is able to decode codewords with a larger fraction of erasures, as the block size tends to infinity. The proposed algorithm can be also understood as a simplification of the Maxwell decoder, but without its computational complexity. We also illustrate that the new algorithm outperforms the BP decoder for finite block-size codes. |

Koch, Tobias; Lapidoth, Amos Gaussian Fading Is the Worst Fading Journal Article IEEE Transactions on Information Theory, 56 (3), pp. 1158–1165, 2010, ISSN: 0018-9448. Abstract | Links | BibTeX | Tags: Additive noise, channel capacity, channels with memory, Distribution functions, ergodic fading processes, Fading, fading channels, flat fading, flat-fading channel capacity, Gaussian channels, Gaussian fading, Gaussian processes, H infinity control, high signal-to-noise ratio (SNR), Information technology, information theory, multiple-input single-output fading channels, multiplexing gain, noncoherent, noncoherent channel capacity, peak-power limited channel capacity, Signal to noise ratio, signal-to-noise ratio, single-antenna channel capacity, spectral distribution function, time-selective, Transmitters @article{Koch2010a, title = {Gaussian Fading Is the Worst Fading}, author = {Tobias Koch and Amos Lapidoth}, url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5429105}, issn = {0018-9448}, year = {2010}, date = {2010-01-01}, journal = {IEEE Transactions on Information Theory}, volume = {56}, number = {3}, pages = {1158--1165}, abstract = {The capacity of peak-power limited, single-antenna, noncoherent, flat-fading channels with memory is considered. The emphasis is on the capacity pre-log, i.e., on the limiting ratio of channel capacity to the logarithm of the signal-to-noise ratio (SNR), as the SNR tends to infinity. It is shown that, among all stationary and ergodic fading processes of a given spectral distribution function and whose law has no mass point at zero, the Gaussian process gives rise to the smallest pre-log. The assumption that the law of the fading process has no mass point at zero is essential in the sense that there exist stationary and ergodic fading processes whose law has a mass point at zero and that give rise to a smaller pre-log than the Gaussian process of equal spectral distribution function. An extension of these results to multiple-input single-output (MISO) fading channels with memory is also presented.}, keywords = {Additive noise, channel capacity, channels with memory, Distribution functions, ergodic fading processes, Fading, fading channels, flat fading, flat-fading channel capacity, Gaussian channels, Gaussian fading, Gaussian processes, H infinity control, high signal-to-noise ratio (SNR), Information technology, information theory, multiple-input single-output fading channels, multiplexing gain, noncoherent, noncoherent channel capacity, peak-power limited channel capacity, Signal to noise ratio, signal-to-noise ratio, single-antenna channel capacity, spectral distribution function, time-selective, Transmitters}, pubstate = {published}, tppubtype = {article} } The capacity of peak-power limited, single-antenna, noncoherent, flat-fading channels with memory is considered. The emphasis is on the capacity pre-log, i.e., on the limiting ratio of channel capacity to the logarithm of the signal-to-noise ratio (SNR), as the SNR tends to infinity. It is shown that, among all stationary and ergodic fading processes of a given spectral distribution function and whose law has no mass point at zero, the Gaussian process gives rise to the smallest pre-log. The assumption that the law of the fading process has no mass point at zero is essential in the sense that there exist stationary and ergodic fading processes whose law has a mass point at zero and that give rise to a smaller pre-log than the Gaussian process of equal spectral distribution function. An extension of these results to multiple-input single-output (MISO) fading channels with memory is also presented. |

## 2009 |

Fresia, Maria; Perez-Cruz, Fernando; Poor, Vincent H Optimized Concatenated LDPC Codes for Joint Source-Channel Coding Inproceedings 2009 IEEE International Symposium on Information Theory, pp. 2131–2135, IEEE, Seoul, 2009, ISBN: 978-1-4244-4312-3. Abstract | Links | BibTeX | Tags: approximation theory, asymptotic behavior analysis, Channel Coding, combined source-channel coding, Concatenated codes, Decoding, Entropy, EXIT chart, extrinsic information transfer, H infinity control, Information analysis, joint belief propagation decoder, joint source-channel coding, low-density-parity-check code, optimized concatenated independent LDPC codes, parity check codes, Redundancy, source coding, transmitter, Transmitters @inproceedings{Fresia2009, title = {Optimized Concatenated LDPC Codes for Joint Source-Channel Coding}, author = {Maria Fresia and Fernando Perez-Cruz and Vincent H Poor}, url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5205766}, isbn = {978-1-4244-4312-3}, year = {2009}, date = {2009-01-01}, booktitle = {2009 IEEE International Symposium on Information Theory}, pages = {2131--2135}, publisher = {IEEE}, address = {Seoul}, abstract = {In this paper a scheme for joint source-channel coding based on low-density-parity-check (LDPC) codes is investigated. Two concatenated independent LDPC codes are used in the transmitter: one for source coding and the other for channel coding, with a joint belief propagation decoder. The asymptotic behavior is analyzed using EXtrinsic Information Transfer (EXIT) charts and this approximation is corroborated with illustrative experiments. The optimization of the degree distributions for our sparse code to maximize the information transmission rate is also considered.}, keywords = {approximation theory, asymptotic behavior analysis, Channel Coding, combined source-channel coding, Concatenated codes, Decoding, Entropy, EXIT chart, extrinsic information transfer, H infinity control, Information analysis, joint belief propagation decoder, joint source-channel coding, low-density-parity-check code, optimized concatenated independent LDPC codes, parity check codes, Redundancy, source coding, transmitter, Transmitters}, pubstate = {published}, tppubtype = {inproceedings} } In this paper a scheme for joint source-channel coding based on low-density-parity-check (LDPC) codes is investigated. Two concatenated independent LDPC codes are used in the transmitter: one for source coding and the other for channel coding, with a joint belief propagation decoder. The asymptotic behavior is analyzed using EXtrinsic Information Transfer (EXIT) charts and this approximation is corroborated with illustrative experiments. The optimization of the degree distributions for our sparse code to maximize the information transmission rate is also considered. |

## 2008 |

Perez-Cruz, Fernando Kullback-Leibler Divergence Estimation of Continuous Distributions Inproceedings 2008 IEEE International Symposium on Information Theory, pp. 1666–1670, IEEE, Toronto, 2008, ISBN: 978-1-4244-2256-2. Abstract | Links | BibTeX | Tags: Convergence, density estimation, Density measurement, Entropy, Frequency estimation, H infinity control, information theory, k-nearest-neighbour density estimation, Kullback-Leibler divergence estimation, Machine learning, Mutual information, neuroscience, Random variables, statistical distributions, waiting-times distributions @inproceedings{Perez-Cruz2008, title = {Kullback-Leibler Divergence Estimation of Continuous Distributions}, author = {Fernando Perez-Cruz}, url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4595271}, isbn = {978-1-4244-2256-2}, year = {2008}, date = {2008-01-01}, booktitle = {2008 IEEE International Symposium on Information Theory}, pages = {1666--1670}, publisher = {IEEE}, address = {Toronto}, abstract = {We present a method for estimating the KL divergence between continuous densities and we prove it converges almost surely. Divergence estimation is typically solved estimating the densities first. Our main result shows this intermediate step is unnecessary and that the divergence can be either estimated using the empirical cdf or k-nearest-neighbour density estimation, which does not converge to the true measure for finite k. The convergence proof is based on describing the statistics of our estimator using waiting-times distributions, as the exponential or Erlang. We illustrate the proposed estimators and show how they compare to existing methods based on density estimation, and we also outline how our divergence estimators can be used for solving the two-sample problem.}, keywords = {Convergence, density estimation, Density measurement, Entropy, Frequency estimation, H infinity control, information theory, k-nearest-neighbour density estimation, Kullback-Leibler divergence estimation, Machine learning, Mutual information, neuroscience, Random variables, statistical distributions, waiting-times distributions}, pubstate = {published}, tppubtype = {inproceedings} } We present a method for estimating the KL divergence between continuous densities and we prove it converges almost surely. Divergence estimation is typically solved estimating the densities first. Our main result shows this intermediate step is unnecessary and that the divergence can be either estimated using the empirical cdf or k-nearest-neighbour density estimation, which does not converge to the true measure for finite k. The convergence proof is based on describing the statistics of our estimator using waiting-times distributions, as the exponential or Erlang. We illustrate the proposed estimators and show how they compare to existing methods based on density estimation, and we also outline how our divergence estimators can be used for solving the two-sample problem. |

Koch, Tobias; Lapidoth, Amos Multipath Channels of Unbounded Capacity Inproceedings 2008 IEEE 25th Convention of Electrical and Electronics Engineers in Israel, pp. 640–644, IEEE, Eilat, 2008, ISBN: 978-1-4244-2481-8. Abstract | Links | BibTeX | Tags: channel capacity, discrete-time capacity, Entropy, Fading, fading channels, Frequency, H infinity control, Information rates, multipath channels, multipath fading channels, noncoherent, noncoherent capacity, path gains decay, Signal to noise ratio, statistics, Transmitters, unbounded capacity @inproceedings{Koch2008b, title = {Multipath Channels of Unbounded Capacity}, author = {Tobias Koch and Amos Lapidoth}, url = {http://ieeexplore.ieee.org/articleDetails.jsp?arnumber=4736611}, isbn = {978-1-4244-2481-8}, year = {2008}, date = {2008-01-01}, booktitle = {2008 IEEE 25th Convention of Electrical and Electronics Engineers in Israel}, pages = {640--644}, publisher = {IEEE}, address = {Eilat}, abstract = {The capacity of discrete-time, noncoherent, multipath fading channels is considered. It is shown that if the variances of the path gains decay faster than exponentially, then capacity is unbounded in the transmit power.}, keywords = {channel capacity, discrete-time capacity, Entropy, Fading, fading channels, Frequency, H infinity control, Information rates, multipath channels, multipath fading channels, noncoherent, noncoherent capacity, path gains decay, Signal to noise ratio, statistics, Transmitters, unbounded capacity}, pubstate = {published}, tppubtype = {inproceedings} } The capacity of discrete-time, noncoherent, multipath fading channels is considered. It is shown that if the variances of the path gains decay faster than exponentially, then capacity is unbounded in the transmit power. |

Santiago-Mozos, Ricardo; Fernandez-Lorenzana, R; Perez-Cruz, Fernando; Artés-Rodríguez, Antonio On the Uncertainty in Sequential Hypothesis Testing Inproceedings 2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro, pp. 1223–1226, IEEE, Paris, 2008, ISBN: 978-1-4244-2002-5. Abstract | Links | BibTeX | Tags: binary hypothesis test, Biomedical imaging, Detectors, H infinity control, likelihood ratio, Medical diagnostic imaging, medical image application, medical image processing, Medical tests, patient diagnosis, Probability, Random variables, Sequential analysis, sequential hypothesis testing, sequential probability ratio test, Signal processing, Testing, tuberculosis diagnosis, Uncertainty @inproceedings{Santiago-Mozos2008, title = {On the Uncertainty in Sequential Hypothesis Testing}, author = {Ricardo Santiago-Mozos and R Fernandez-Lorenzana and Fernando Perez-Cruz and Antonio Artés-Rodríguez}, url = {http://ieeexplore.ieee.org/articleDetails.jsp?arnumber=4541223}, isbn = {978-1-4244-2002-5}, year = {2008}, date = {2008-01-01}, booktitle = {2008 5th IEEE International Symposium on Biomedical Imaging: From Nano to Macro}, pages = {1223--1226}, publisher = {IEEE}, address = {Paris}, abstract = {We consider the problem of sequential hypothesis testing when the exact pdfs are not known but instead a set of iid samples are used to describe the hypotheses. We modify the classical test by introducing a likelihood ratio interval which accommodates the uncertainty in the pdfs. The test finishes when the whole likelihood ratio interval crosses one of the thresholds and reduces to the classical test as the number of samples to describe the hypotheses tend to infinity. We illustrate the performance of this test in a medical image application related to tuberculosis diagnosis. We show in this example how the test confidence level can be accurately determined.}, keywords = {binary hypothesis test, Biomedical imaging, Detectors, H infinity control, likelihood ratio, Medical diagnostic imaging, medical image application, medical image processing, Medical tests, patient diagnosis, Probability, Random variables, Sequential analysis, sequential hypothesis testing, sequential probability ratio test, Signal processing, Testing, tuberculosis diagnosis, Uncertainty}, pubstate = {published}, tppubtype = {inproceedings} } We consider the problem of sequential hypothesis testing when the exact pdfs are not known but instead a set of iid samples are used to describe the hypotheses. We modify the classical test by introducing a likelihood ratio interval which accommodates the uncertainty in the pdfs. The test finishes when the whole likelihood ratio interval crosses one of the thresholds and reduces to the classical test as the number of samples to describe the hypotheses tend to infinity. We illustrate the performance of this test in a medical image application related to tuberculosis diagnosis. We show in this example how the test confidence level can be accurately determined. |