## 2016 |

## Journal Articles |

Martino, Luca; Elvira, Victor; Luengo, David; Corander, Jukka; Louzada, Francisco Orthogonal Parallel MCMC Methods for Sampling and Optimization (Journal Article) Digital Signal Processing, 58 , pp. 64–84, 2016, ISSN: 10512004. (Abstract | Links | BibTeX | Tags: Bayesian inference, Block Independent Metropolis, Journal, Optimization, Parallel Markov Chain Monte Carlo, Parallel Multiple Try Metropolis, Parallel Simulated Annealing, Recycling samples) @article{Martino2016b, title = {Orthogonal Parallel MCMC Methods for Sampling and Optimization}, author = {Martino, Luca and Elvira, Victor and Luengo, David and Corander, Jukka and Louzada, Francisco}, url = {http://www.sciencedirect.com/science/article/pii/S1051200416300987}, doi = {10.1016/j.dsp.2016.07.013}, issn = {10512004}, year = {2016}, date = {2016-11-01}, journal = {Digital Signal Processing}, volume = {58}, pages = {64--84}, abstract = {Monte Carlo (MC) methods are widely used for Bayesian inference and optimization in statistics, signal processing and machine learning. A well-known class of MC methods are Markov Chain Monte Carlo (MCMC) algorithms. In order to foster better exploration of the state space, specially in high-dimensional applications, several schemes employing multiple parallel MCMC chains have been recently introduced. In this work, we describe a novel parallel interacting MCMC scheme, called orthogonal MCMC (O-MCMC), where a set of “vertical” parallel MCMC chains share information using some “horizontal” MCMC techniques working on the entire population of current states. More specifically, the vertical chains are led by random-walk proposals, whereas the horizontal MCMC techniques employ independent proposals, thus allowing an efficient combination of global exploration and local approximation. The interaction is contained in these horizontal iterations. Within the analysis of different implementations of O-MCMC, novel schemes in order to reduce the overall computational cost of parallel multiple try Metropolis (MTM) chains are also presented. Furthermore, a modified version of O-MCMC for optimization is provided by considering parallel simulated annealing (SA) algorithms. Numerical results show the advantages of the proposed sampling scheme in terms of efficiency in the estimation, as well as robustness in terms of independence with respect to initial values and the choice of the parameters.}, keywords = {Bayesian inference, Block Independent Metropolis, Journal, Optimization, Parallel Markov Chain Monte Carlo, Parallel Multiple Try Metropolis, Parallel Simulated Annealing, Recycling samples}, pubstate = {published}, tppubtype = {article} } Monte Carlo (MC) methods are widely used for Bayesian inference and optimization in statistics, signal processing and machine learning. A well-known class of MC methods are Markov Chain Monte Carlo (MCMC) algorithms. In order to foster better exploration of the state space, specially in high-dimensional applications, several schemes employing multiple parallel MCMC chains have been recently introduced. In this work, we describe a novel parallel interacting MCMC scheme, called orthogonal MCMC (O-MCMC), where a set of “vertical” parallel MCMC chains share information using some “horizontal” MCMC techniques working on the entire population of current states. More specifically, the vertical chains are led by random-walk proposals, whereas the horizontal MCMC techniques employ independent proposals, thus allowing an efficient combination of global exploration and local approximation. The interaction is contained in these horizontal iterations. Within the analysis of different implementations of O-MCMC, novel schemes in order to reduce the overall computational cost of parallel multiple try Metropolis (MTM) chains are also presented. Furthermore, a modified version of O-MCMC for optimization is provided by considering parallel simulated annealing (SA) algorithms. Numerical results show the advantages of the proposed sampling scheme in terms of efficiency in the estimation, as well as robustness in terms of independence with respect to initial values and the choice of the parameters. |

## 2015 |

## Inproceedings |

Vazquez-Vilar, Gonzalo; Martinez, Alfonso; Guillen i Fabregas, Albert A derivation of the Cost-Constrained Sphere-Packing Exponent (Inproceeding) 2015 IEEE International Symposium on Information Theory (ISIT), pp. 929–933, IEEE, Hong Kong, 2015, ISBN: 978-1-4673-7704-1. (Links | BibTeX | Tags: Channel Coding, channel-coding cost-constrained sphere-packing exp, continuous channel, continuous memoryless channel, cost constraint, error probability, hypothesis testing, Lead, Memoryless systems, Optimization, per-codeword cost constraint, reliability function, spherepacking exponent, Testing) @inproceedings{Vazquez-Vilar2015, title = {A derivation of the Cost-Constrained Sphere-Packing Exponent}, author = {Vazquez-Vilar, Gonzalo and Martinez, Alfonso and Guillen i Fabregas, Albert}, url = {http://ieeexplore.ieee.org/articleDetails.jsp?arnumber=7282591}, doi = {10.1109/ISIT.2015.7282591}, isbn = {978-1-4673-7704-1}, year = {2015}, date = {2015-06-01}, booktitle = {2015 IEEE International Symposium on Information Theory (ISIT)}, pages = {929--933}, publisher = {IEEE}, address = {Hong Kong}, keywords = {Channel Coding, channel-coding cost-constrained sphere-packing exp, continuous channel, continuous memoryless channel, cost constraint, error probability, hypothesis testing, Lead, Memoryless systems, Optimization, per-codeword cost constraint, reliability function, spherepacking exponent, Testing}, pubstate = {published}, tppubtype = {inproceedings} } |

## 2012 |

## Inproceedings |

Montoya-Martinez, Jair; Artés-Rodríguez, Antonio; Hansen, Lars Kai; Pontil, Massimiliano Structured Sparsity Regularization Approach to the EEG Inverse Problem (Inproceeding) 2012 3rd International Workshop on Cognitive Information Processing (CIP), pp. 1–6, IEEE, Baiona, 2012, ISBN: 978-1-4673-1878-5. (Abstract | Links | BibTeX | Tags: BES, brain electrical sources matrix, Brain modeling, EEG inverse problem, Electrodes, Electroencephalography, good convergence, Inverse problems, large nonsmooth convex problems, medical signal processing, optimisation, Optimization, proximal splitting optimization methods, Sparse matrices, spatio-temporal source space, structured sparsity regularization approach, undetermined ill-posed problem) @inproceedings{Montoya-Martinez2012, title = {Structured Sparsity Regularization Approach to the EEG Inverse Problem}, author = {Montoya-Martinez, Jair and Artés-Rodríguez, Antonio and Hansen, Lars Kai and Pontil, Massimiliano}, url = {http://ieeexplore.ieee.org/articleDetails.jsp?arnumber=6232898}, isbn = {978-1-4673-1878-5}, year = {2012}, date = {2012-01-01}, booktitle = {2012 3rd International Workshop on Cognitive Information Processing (CIP)}, pages = {1--6}, publisher = {IEEE}, address = {Baiona}, abstract = {Localization of brain activity involves solving the EEG inverse problem, which is an undetermined ill-posed problem. We propose a novel approach consisting in estimating, using structured sparsity regularization techniques, the Brain Electrical Sources (BES) matrix directly in the spatio-temporal source space. We use proximal splitting optimization methods, which are efficient optimization techniques, with good convergence rates and with the ability to handle large nonsmooth convex problems, which is the typical scenario in the EEG inverse problem. We have evaluated our approach under a simulated scenario, consisting in estimating a synthetic BES matrix with 5124 sources. We report results using ℓ1 (LASSO), ℓ1/ℓ2 (Group LASSO) and ℓ1 + ℓ1/ℓ2 (Sparse Group LASSO) regularizers.}, keywords = {BES, brain electrical sources matrix, Brain modeling, EEG inverse problem, Electrodes, Electroencephalography, good convergence, Inverse problems, large nonsmooth convex problems, medical signal processing, optimisation, Optimization, proximal splitting optimization methods, Sparse matrices, spatio-temporal source space, structured sparsity regularization approach, undetermined ill-posed problem}, pubstate = {published}, tppubtype = {inproceedings} } Localization of brain activity involves solving the EEG inverse problem, which is an undetermined ill-posed problem. We propose a novel approach consisting in estimating, using structured sparsity regularization techniques, the Brain Electrical Sources (BES) matrix directly in the spatio-temporal source space. We use proximal splitting optimization methods, which are efficient optimization techniques, with good convergence rates and with the ability to handle large nonsmooth convex problems, which is the typical scenario in the EEG inverse problem. We have evaluated our approach under a simulated scenario, consisting in estimating a synthetic BES matrix with 5124 sources. We report results using ℓ1 (LASSO), ℓ1/ℓ2 (Group LASSO) and ℓ1 + ℓ1/ℓ2 (Sparse Group LASSO) regularizers. |

## 2011 |

## Inproceedings |

Olmos, Pablo; Murillo-Fuentes, Juan Jose; Perez-Cruz, Fernando Capacity Achieving LDPC Ensembles for the TEP Decoder in Erasure Channels (Inproceeding) 2011 IEEE International Symposium on Information Theory Proceedings, pp. 2398–2402, IEEE, St. Petersburg, 2011, ISSN: 2157-8095. (Abstract | Links | BibTeX | Tags: BP threshold, Complexity theory, Decoding, Differential equations, erasure channels, fixed-rate code, Iterative decoding, LDPC, low-density parity-check codes, MAP capacity, MAP threshold, optimisation, Optimization, optimization problem, parity check codes, TEP decoder, tree-expectation propagation decoder) @inproceedings{Olmos2011b, title = {Capacity Achieving LDPC Ensembles for the TEP Decoder in Erasure Channels}, author = {Olmos, Pablo M. and Murillo-Fuentes, Juan Jose and Perez-Cruz, Fernando}, url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=6033993}, issn = {2157-8095}, year = {2011}, date = {2011-01-01}, booktitle = {2011 IEEE International Symposium on Information Theory Proceedings}, pages = {2398--2402}, publisher = {IEEE}, address = {St. Petersburg}, abstract = {In this work we address the design of degree distributions (DD) of low-density parity-check (LDPC) codes for the tree-expectation propagation (TEP) decoder. The optimization problem to find distributions to maximize the TEP decoding threshold for a fixed-rate code can not be analytically solved. We derive a simplified optimization problem that can be easily solved since it is based in the analytic expressions of the peeling decoder. Two kinds of solutions are obtained from this problem: we either design LDPC ensembles for which the BP threshold equals the MAP threshold or we get LDPC ensembles for which the TEP threshold outperforms the BP threshold, even achieving the MAP capacity in some cases. Hence, we proved that there exist ensembles for which the MAP solution can be obtained with linear complexity even though the BP threshold does not achieve the MAP threshold.}, keywords = {BP threshold, Complexity theory, Decoding, Differential equations, erasure channels, fixed-rate code, Iterative decoding, LDPC, low-density parity-check codes, MAP capacity, MAP threshold, optimisation, Optimization, optimization problem, parity check codes, TEP decoder, tree-expectation propagation decoder}, pubstate = {published}, tppubtype = {inproceedings} } In this work we address the design of degree distributions (DD) of low-density parity-check (LDPC) codes for the tree-expectation propagation (TEP) decoder. The optimization problem to find distributions to maximize the TEP decoding threshold for a fixed-rate code can not be analytically solved. We derive a simplified optimization problem that can be easily solved since it is based in the analytic expressions of the peeling decoder. Two kinds of solutions are obtained from this problem: we either design LDPC ensembles for which the BP threshold equals the MAP threshold or we get LDPC ensembles for which the TEP threshold outperforms the BP threshold, even achieving the MAP capacity in some cases. Hence, we proved that there exist ensembles for which the MAP solution can be obtained with linear complexity even though the BP threshold does not achieve the MAP threshold. |

Maiz, Cristina; Miguez, Joaquin On the Optimization of Transportation Routes with Multiple Destinations in Random Networks (Inproceeding) 2011 IEEE Statistical Signal Processing Workshop (SSP), pp. 349–352, IEEE, Nice, 2011, ISBN: 978-1-4577-0569-4. (Abstract | Links | BibTeX | Tags: Approximation algorithms, communication networks, Estimation, graph theory, Histograms, intelligent transportation, Monte Carlo algorithm, Monte Carlo methods, multiple destinations, optimisation, Optimization, random networks, route optimization, routing, Sequential Monte Carlo, Signal processing algorithms, stochastic graph, Stochastic processes, telecommunication network routing, time-varying graph, transportation routes) @inproceedings{Maiz2011, title = {On the Optimization of Transportation Routes with Multiple Destinations in Random Networks}, author = {Maiz, Cristina S. and Miguez, Joaquin}, url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5967701}, isbn = {978-1-4577-0569-4}, year = {2011}, date = {2011-01-01}, booktitle = {2011 IEEE Statistical Signal Processing Workshop (SSP)}, pages = {349--352}, publisher = {IEEE}, address = {Nice}, abstract = {Various practical problems in transportation research and routing in communication networks can be reduced to the computation of the best path that traverses a certain graph and visits a set of D specified destination nodes. Simple versions of this problem have received attention in the literature. Optimal solutions exist for the cases in which (a) D >; 1 and the graph is deterministic or (b) D = 1 and the graph is stochastic (and possibly time-dependent). Here, we address the general problem in which both D >; 1 and the costs of the edges in the graph are stochastic and time-varying. We tackle this complex global optimization problem by first converting it into an equivalent estimation problem and then computing a numerical solution using a sequential Monte Carlo algorithm. The advantage of the proposed technique over some standard methods (devised for graphs with time-invariant statistics) is illustrated by way of computer simulations.}, keywords = {Approximation algorithms, communication networks, Estimation, graph theory, Histograms, intelligent transportation, Monte Carlo algorithm, Monte Carlo methods, multiple destinations, optimisation, Optimization, random networks, route optimization, routing, Sequential Monte Carlo, Signal processing algorithms, stochastic graph, Stochastic processes, telecommunication network routing, time-varying graph, transportation routes}, pubstate = {published}, tppubtype = {inproceedings} } Various practical problems in transportation research and routing in communication networks can be reduced to the computation of the best path that traverses a certain graph and visits a set of D specified destination nodes. Simple versions of this problem have received attention in the literature. Optimal solutions exist for the cases in which (a) D >; 1 and the graph is deterministic or (b) D = 1 and the graph is stochastic (and possibly time-dependent). Here, we address the general problem in which both D >; 1 and the costs of the edges in the graph are stochastic and time-varying. We tackle this complex global optimization problem by first converting it into an equivalent estimation problem and then computing a numerical solution using a sequential Monte Carlo algorithm. The advantage of the proposed technique over some standard methods (devised for graphs with time-invariant statistics) is illustrated by way of computer simulations. |

## 2008 |

## Inproceedings |

Rodrigues, Miguel; Perez-Cruz, Fernando; Verdu, Sergio Multiple-Input Multiple-Output Gaussian Channels: Optimal Covariance for Non-Gaussian Inputs (Inproceeding) 2008 IEEE Information Theory Workshop, pp. 445–449, IEEE, Porto, 2008, ISBN: 978-1-4244-2269-2. (Abstract | Links | BibTeX | Tags: Binary phase shift keying, covariance matrices, Covariance matrix, deterministic MIMO Gaussian channel, fixed-point equation, Gaussian channels, Gaussian noise, Information rates, intersymbol interference, least mean squares methods, Magnetic recording, mercury-waterfilling power allocation policy, MIMO, MIMO communication, minimum mean-squared error, MMSE, MMSE matrix, multiple-input multiple-output system, Multiple-Input Multiple-Output Systems, Mutual information, Optimal Input Covariance, Optimization, Telecommunications) @inproceedings{Rodrigues2008, title = {Multiple-Input Multiple-Output Gaussian Channels: Optimal Covariance for Non-Gaussian Inputs}, author = {Rodrigues, Miguel R. D. and Perez-Cruz, Fernando and Verdu, Sergio}, url = {http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=4578704}, isbn = {978-1-4244-2269-2}, year = {2008}, date = {2008-01-01}, booktitle = {2008 IEEE Information Theory Workshop}, pages = {445--449}, publisher = {IEEE}, address = {Porto}, abstract = {We investigate the input covariance that maximizes the mutual information of deterministic multiple-input multipleo-utput (MIMO) Gaussian channels with arbitrary (not necessarily Gaussian) input distributions, by capitalizing on the relationship between the gradient of the mutual information and the minimum mean-squared error (MMSE) matrix. We show that the optimal input covariance satisfies a simple fixed-point equation involving key system quantities, including the MMSE matrix. We also specialize the form of the optimal input covariance to the asymptotic regimes of low and high snr. We demonstrate that in the low-snr regime the optimal covariance fully correlates the inputs to better combat noise. In contrast, in the high-snr regime the optimal covariance is diagonal with diagonal elements obeying the generalized mercury/waterfilling power allocation policy. Numerical results illustrate that covariance optimization may lead to significant gains with respect to conventional strategies based on channel diagonalization followed by mercury/waterfilling or waterfilling power allocation, particularly in the regimes of medium and high snr.}, keywords = {Binary phase shift keying, covariance matrices, Covariance matrix, deterministic MIMO Gaussian channel, fixed-point equation, Gaussian channels, Gaussian noise, Information rates, intersymbol interference, least mean squares methods, Magnetic recording, mercury-waterfilling power allocation policy, MIMO, MIMO communication, minimum mean-squared error, MMSE, MMSE matrix, multiple-input multiple-output system, Multiple-Input Multiple-Output Systems, Mutual information, Optimal Input Covariance, Optimization, Telecommunications}, pubstate = {published}, tppubtype = {inproceedings} } We investigate the input covariance that maximizes the mutual information of deterministic multiple-input multipleo-utput (MIMO) Gaussian channels with arbitrary (not necessarily Gaussian) input distributions, by capitalizing on the relationship between the gradient of the mutual information and the minimum mean-squared error (MMSE) matrix. We show that the optimal input covariance satisfies a simple fixed-point equation involving key system quantities, including the MMSE matrix. We also specialize the form of the optimal input covariance to the asymptotic regimes of low and high snr. We demonstrate that in the low-snr regime the optimal covariance fully correlates the inputs to better combat noise. In contrast, in the high-snr regime the optimal covariance is diagonal with diagonal elements obeying the generalized mercury/waterfilling power allocation policy. Numerical results illustrate that covariance optimization may lead to significant gains with respect to conventional strategies based on channel diagonalization followed by mercury/waterfilling or waterfilling power allocation, particularly in the regimes of medium and high snr. |