## 2015 |

## Inproceedings |

Martino, Luca; Elvira, Victor; Luengo, David; Artés-Rodríguez, Antonio; Corander, Jukka Smelly Parallel MCMC Chains Inproceedings 2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4070–4074, IEEE, Brisbane, 2015, ISBN: 978-1-4673-6997-8. Abstract | Links | BibTeX | Tags: Bayesian inference, learning (artificial intelligence), Machine learning, Markov chain Monte Carlo, Markov chain Monte Carlo algorithms, Markov processes, MC methods, MCMC algorithms, MCMC scheme, mean square error, mean square error methods, Monte Carlo methods, optimisation, parallel and interacting chains, Probability density function, Proposals, robustness, Sampling methods, Signal processing, Signal processing algorithms, signal sampling, smelly parallel chains, smelly parallel MCMC chains, Stochastic optimization @inproceedings{Martino2015a, title = {Smelly Parallel MCMC Chains}, author = {Luca Martino and Victor Elvira and David Luengo and Antonio Artés-Rodríguez and Jukka Corander}, url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=7178736 http://www.tsc.uc3m.es/~velvira/papers/ICASSP2015_martino.pdf}, doi = {10.1109/ICASSP.2015.7178736}, isbn = {978-1-4673-6997-8}, year = {2015}, date = {2015-04-01}, booktitle = {2015 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)}, pages = {4070--4074}, publisher = {IEEE}, address = {Brisbane}, abstract = {Monte Carlo (MC) methods are useful tools for Bayesian inference and stochastic optimization that have been widely applied in signal processing and machine learning. A well-known class of MC methods are Markov Chain Monte Carlo (MCMC) algorithms. In this work, we introduce a novel parallel interacting MCMC scheme, where the parallel chains share information, thus yielding a faster exploration of the state space. The interaction is carried out generating a dynamic repulsion among the “smelly” parallel chains that takes into account the entire population of current states. The ergodicity of the scheme and its relationship with other sampling methods are discussed. Numerical results show the advantages of the proposed approach in terms of mean square error, robustness w.r.t. to initial values and parameter choice.}, keywords = {Bayesian inference, learning (artificial intelligence), Machine learning, Markov chain Monte Carlo, Markov chain Monte Carlo algorithms, Markov processes, MC methods, MCMC algorithms, MCMC scheme, mean square error, mean square error methods, Monte Carlo methods, optimisation, parallel and interacting chains, Probability density function, Proposals, robustness, Sampling methods, Signal processing, Signal processing algorithms, signal sampling, smelly parallel chains, smelly parallel MCMC chains, Stochastic optimization}, pubstate = {published}, tppubtype = {inproceedings} } Monte Carlo (MC) methods are useful tools for Bayesian inference and stochastic optimization that have been widely applied in signal processing and machine learning. A well-known class of MC methods are Markov Chain Monte Carlo (MCMC) algorithms. In this work, we introduce a novel parallel interacting MCMC scheme, where the parallel chains share information, thus yielding a faster exploration of the state space. The interaction is carried out generating a dynamic repulsion among the “smelly” parallel chains that takes into account the entire population of current states. The ergodicity of the scheme and its relationship with other sampling methods are discussed. Numerical results show the advantages of the proposed approach in terms of mean square error, robustness w.r.t. to initial values and parameter choice. |

## 2012 |

## Inproceedings |

Taborda, Camilo G; Perez-Cruz, Fernando Mutual Information and Relative Entropy over the Binomial and Negative Binomial Channels Inproceedings 2012 IEEE International Symposium on Information Theory Proceedings, pp. 696–700, IEEE, Cambridge, MA, 2012, ISSN: 2157-8095. Abstract | Links | BibTeX | Tags: Channel estimation, conditional mean estimation, Entropy, Estimation, estimation theoretical quantity, estimation theory, Gaussian channel, Gaussian channels, information theory concept, loss function, mean square error methods, Mutual information, negative binomial channel, Poisson channel, Random variables, relative entropy @inproceedings{Taborda2012a, title = {Mutual Information and Relative Entropy over the Binomial and Negative Binomial Channels}, author = {Camilo G Taborda and Fernando Perez-Cruz}, url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6284304}, issn = {2157-8095}, year = {2012}, date = {2012-01-01}, booktitle = {2012 IEEE International Symposium on Information Theory Proceedings}, pages = {696--700}, publisher = {IEEE}, address = {Cambridge, MA}, abstract = {We study the relation of the mutual information and relative entropy over the Binomial and Negative Binomial channels with estimation theoretical quantities, in which we extend already known results for Gaussian and Poisson channels. We establish general expressions for these information theory concepts with a direct connection with estimation theory through the conditional mean estimation and a particular loss function.}, keywords = {Channel estimation, conditional mean estimation, Entropy, Estimation, estimation theoretical quantity, estimation theory, Gaussian channel, Gaussian channels, information theory concept, loss function, mean square error methods, Mutual information, negative binomial channel, Poisson channel, Random variables, relative entropy}, pubstate = {published}, tppubtype = {inproceedings} } We study the relation of the mutual information and relative entropy over the Binomial and Negative Binomial channels with estimation theoretical quantities, in which we extend already known results for Gaussian and Poisson channels. We establish general expressions for these information theory concepts with a direct connection with estimation theory through the conditional mean estimation and a particular loss function. |

Pastore, Adriano; Koch, Tobias; Fonollosa, Javier Rodriguez Improved Capacity Lower Bounds for Fading Channels with Imperfect CSI Using Rate Splitting Inproceedings 2012 IEEE 27th Convention of Electrical and Electronics Engineers in Israel, pp. 1–5, IEEE, Eilat, 2012, ISBN: 978-1-4673-4681-8. Abstract | Links | BibTeX | Tags: channel capacity, channel capacity lower bounds, conditional entropy, Decoding, Entropy, Fading, fading channels, Gaussian channel, Gaussian channels, Gaussian random variable, imperfect channel-state information, imperfect CSI, independent Gaussian variables, linear minimum mean-square error, mean square error methods, Medard lower bound, Mutual information, Random variables, rate splitting approach, Resource management, Upper bound, wireless communications @inproceedings{Pastore2012, title = {Improved Capacity Lower Bounds for Fading Channels with Imperfect CSI Using Rate Splitting}, author = {Adriano Pastore and Tobias Koch and Javier Rodriguez Fonollosa}, url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6377031}, isbn = {978-1-4673-4681-8}, year = {2012}, date = {2012-01-01}, booktitle = {2012 IEEE 27th Convention of Electrical and Electronics Engineers in Israel}, pages = {1--5}, publisher = {IEEE}, address = {Eilat}, abstract = {As shown by Medard (“The effect upon channel capacity in wireless communications of perfect and imperfect knowledge of the channel,” IEEE Trans. Inform. Theory, May 2000), the capacity of fading channels with imperfect channel-state information (CSI) can be lower-bounded by assuming a Gaussian channel input X, and by upper-bounding the conditional entropy h(XY, Ĥ), conditioned on the channel output Y and the CSI Ĥ, by the entropy of a Gaussian random variable with variance equal to the linear minimum mean-square error in estimating X from (Y, Ĥ). We demonstrate that, by using a rate-splitting approach, this lower bound can be sharpened: we show that by expressing the Gaussian input X as as the sum of two independent Gaussian variables X(1) and X(2), and by applying Medard's lower bound first to analyze the mutual information between X(1) and Y conditioned on Ĥ while treating X(2) as noise, and by applying the lower bound then to analyze the mutual information between X(2) and Y conditioned on (X(1), Ĥ), we obtain a lower bound on the capacity that is larger than Medard's lower bound.}, keywords = {channel capacity, channel capacity lower bounds, conditional entropy, Decoding, Entropy, Fading, fading channels, Gaussian channel, Gaussian channels, Gaussian random variable, imperfect channel-state information, imperfect CSI, independent Gaussian variables, linear minimum mean-square error, mean square error methods, Medard lower bound, Mutual information, Random variables, rate splitting approach, Resource management, Upper bound, wireless communications}, pubstate = {published}, tppubtype = {inproceedings} } As shown by Medard (“The effect upon channel capacity in wireless communications of perfect and imperfect knowledge of the channel,” IEEE Trans. Inform. Theory, May 2000), the capacity of fading channels with imperfect channel-state information (CSI) can be lower-bounded by assuming a Gaussian channel input X, and by upper-bounding the conditional entropy h(XY, Ĥ), conditioned on the channel output Y and the CSI Ĥ, by the entropy of a Gaussian random variable with variance equal to the linear minimum mean-square error in estimating X from (Y, Ĥ). We demonstrate that, by using a rate-splitting approach, this lower bound can be sharpened: we show that by expressing the Gaussian input X as as the sum of two independent Gaussian variables X(1) and X(2), and by applying Medard's lower bound first to analyze the mutual information between X(1) and Y conditioned on Ĥ while treating X(2) as noise, and by applying the lower bound then to analyze the mutual information between X(2) and Y conditioned on (X(1), Ĥ), we obtain a lower bound on the capacity that is larger than Medard's lower bound. |

## 2010 |

## Journal Articles |

Perez-Cruz, Fernando; Rodrigues, Miguel R D; Verdu, Sergio MIMO Gaussian Channels With Arbitrary Inputs: Optimal Precoding and Power Allocation Journal Article IEEE Transactions on Information Theory, 56 (3), pp. 1070–1084, 2010, ISSN: 0018-9448. Abstract | Links | BibTeX | Tags: Collaborative work, Equations, fixed-point equation, Gaussian channels, Gaussian noise channels, Gaussian processes, Government, Interference, linear precoding, matrix algebra, mean square error methods, mercury-waterfilling algorithm, MIMO, MIMO communication, MIMO Gaussian channel, minimum mean-square error, minimum mean-square error (MMSE), multiple-input-multiple-output channel, multiple-input–multiple-output (MIMO) systems, Mutual information, nondiagonal precoding matrix, optimal linear precoder, optimal power allocation policy, optimal precoding, optimum power allocation, Phase shift keying, precoding, Quadrature amplitude modulation, Telecommunications, waterfilling @article{Perez-Cruz2010a, title = {MIMO Gaussian Channels With Arbitrary Inputs: Optimal Precoding and Power Allocation}, author = {Fernando Perez-Cruz and Miguel R D Rodrigues and Sergio Verdu}, url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5429131}, issn = {0018-9448}, year = {2010}, date = {2010-01-01}, journal = {IEEE Transactions on Information Theory}, volume = {56}, number = {3}, pages = {1070--1084}, abstract = {In this paper, we investigate the linear precoding and power allocation policies that maximize the mutual information for general multiple-input-multiple-output (MIMO) Gaussian channels with arbitrary input distributions, by capitalizing on the relationship between mutual information and minimum mean-square error (MMSE). The optimal linear precoder satisfies a fixed-point equation as a function of the channel and the input constellation. For non-Gaussian inputs, a nondiagonal precoding matrix in general increases the information transmission rate, even for parallel noninteracting channels. Whenever precoding is precluded, the optimal power allocation policy also satisfies a fixed-point equation; we put forth a generalization of the mercury/waterfilling algorithm, previously proposed for parallel noninterfering channels, in which the mercury level accounts not only for the non-Gaussian input distributions, but also for the interference among inputs.}, keywords = {Collaborative work, Equations, fixed-point equation, Gaussian channels, Gaussian noise channels, Gaussian processes, Government, Interference, linear precoding, matrix algebra, mean square error methods, mercury-waterfilling algorithm, MIMO, MIMO communication, MIMO Gaussian channel, minimum mean-square error, minimum mean-square error (MMSE), multiple-input-multiple-output channel, multiple-input–multiple-output (MIMO) systems, Mutual information, nondiagonal precoding matrix, optimal linear precoder, optimal power allocation policy, optimal precoding, optimum power allocation, Phase shift keying, precoding, Quadrature amplitude modulation, Telecommunications, waterfilling}, pubstate = {published}, tppubtype = {article} } In this paper, we investigate the linear precoding and power allocation policies that maximize the mutual information for general multiple-input-multiple-output (MIMO) Gaussian channels with arbitrary input distributions, by capitalizing on the relationship between mutual information and minimum mean-square error (MMSE). The optimal linear precoder satisfies a fixed-point equation as a function of the channel and the input constellation. For non-Gaussian inputs, a nondiagonal precoding matrix in general increases the information transmission rate, even for parallel noninteracting channels. Whenever precoding is precluded, the optimal power allocation policy also satisfies a fixed-point equation; we put forth a generalization of the mercury/waterfilling algorithm, previously proposed for parallel noninterfering channels, in which the mercury level accounts not only for the non-Gaussian input distributions, but also for the interference among inputs. |

## 2009 |

## Journal Articles |

Murillo-Fuentes, Juan Jose; Perez-Cruz, Fernando Gaussian Process Regressors for Multiuser Detection in DS-CDMA Systems Journal Article IEEE Transactions on Communications, 57 (8), pp. 2339–2347, 2009, ISSN: 0090-6778. Abstract | Links | BibTeX | Tags: analytical nonlinear multiuser detectors, code division multiple access, communication systems, Detectors, digital communication, digital communications, DS-CDMA systems, Gaussian process for regressi, Gaussian process regressors, Gaussian processes, GPR, Ground penetrating radar, least mean squares methods, maximum likelihood, maximum likelihood detection, maximum likelihood estimation, mean square error methods, minimum mean square error, MMSE, Multiaccess communication, Multiuser detection, nonlinear estimator, nonlinear state-ofthe- art solutions, radio receivers, Receivers, regression analysis, Support vector machines @article{Murillo-Fuentes2009, title = {Gaussian Process Regressors for Multiuser Detection in DS-CDMA Systems}, author = {Juan Jose Murillo-Fuentes and Fernando Perez-Cruz}, url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5201027}, issn = {0090-6778}, year = {2009}, date = {2009-01-01}, journal = {IEEE Transactions on Communications}, volume = {57}, number = {8}, pages = {2339--2347}, abstract = {In this paper we present Gaussian processes for Regression (GPR) as a novel detector for CDMA digital communications. Particularly, we propose GPR for constructing analytical nonlinear multiuser detectors in CDMA communication systems. GPR can easily compute the parameters that describe its nonlinearities by maximum likelihood. Thereby, no cross-validation is needed, as it is typically used in nonlinear estimation procedures. The GPR solution is analytical, given its parameters, and it does not need to solve an optimization problem for building the nonlinear estimator. These properties provide fast and accurate learning, two major issues in digital communications. The GPR with a linear decision function can be understood as a regularized MMSE detector, in which the regularization parameter is optimally set. We also show the GPR receiver to be a straightforward nonlinear extension of the linear minimum mean square error (MMSE) criterion, widely used in the design of these receivers. We argue the benefits of this new approach in short codes CDMA systems where little information on the users' codes, users' amplitudes or the channel is available. The paper includes some experiments to show that GPR outperforms linear (MMSE) and nonlinear (SVM) state-ofthe- art solutions.}, keywords = {analytical nonlinear multiuser detectors, code division multiple access, communication systems, Detectors, digital communication, digital communications, DS-CDMA systems, Gaussian process for regressi, Gaussian process regressors, Gaussian processes, GPR, Ground penetrating radar, least mean squares methods, maximum likelihood, maximum likelihood detection, maximum likelihood estimation, mean square error methods, minimum mean square error, MMSE, Multiaccess communication, Multiuser detection, nonlinear estimator, nonlinear state-ofthe- art solutions, radio receivers, Receivers, regression analysis, Support vector machines}, pubstate = {published}, tppubtype = {article} } In this paper we present Gaussian processes for Regression (GPR) as a novel detector for CDMA digital communications. Particularly, we propose GPR for constructing analytical nonlinear multiuser detectors in CDMA communication systems. GPR can easily compute the parameters that describe its nonlinearities by maximum likelihood. Thereby, no cross-validation is needed, as it is typically used in nonlinear estimation procedures. The GPR solution is analytical, given its parameters, and it does not need to solve an optimization problem for building the nonlinear estimator. These properties provide fast and accurate learning, two major issues in digital communications. The GPR with a linear decision function can be understood as a regularized MMSE detector, in which the regularization parameter is optimally set. We also show the GPR receiver to be a straightforward nonlinear extension of the linear minimum mean square error (MMSE) criterion, widely used in the design of these receivers. We argue the benefits of this new approach in short codes CDMA systems where little information on the users' codes, users' amplitudes or the channel is available. The paper includes some experiments to show that GPR outperforms linear (MMSE) and nonlinear (SVM) state-ofthe- art solutions. |