We consider the problem of model selection for high-dimensional linear regressions in the context of support recovery with multiple measurement vectors available. Here, we assume that the regression coefficient vectors have a common support and the elements of the additive noise vector are potentially correlated. Accordingly, to estimate the support, we propose a non-negative Lasso estimator that is based on covariance matching techniques. We provide deterministic conditions under which the support estimate of our method is guaranteed to match the true support. Further, we use the extended Fisher information criterion to select the tuning parameter in our non-negative Lasso. We also prove that the extended Fisher information criterion can find the true support with probability one as the number of rows in the design matrix grows to infinity. The numerical simulations confirm that our support estimate is asymptotically consistent. Finally, the simulations also show that the proposed method is robust to high correlation between columns of the design matrix.

Statistical model selection is a great challenge when the number of accessible measurements is much smaller than the dimension of the parameter space. We study the problem of model selection in the context of subset selection for high-dimensional linear regressions. Accordingly, we propose a new model selection criterion with the Fisher information that leads to the selection of a parsimonious model from all the combinatorial models up to some maximum level of sparsity. We analyze the performance of our criterion as the number of measurements grows to infinity, as well as when the noise variance tends to zero. In each case, we prove that our proposed criterion gives the true model with a probability approaching one. Additionally, we devise a computationally affordable algorithm to conduct model selection with the proposed criterion in practice. Interestingly, as a side product, our algorithm can provide the ideal regularization parameter for the Lasso estimator such that Lasso selects the true variables. Finally, numerical simulations are included to support our theoretical findings.

Time-frequency distributions (TFDs) allow direction of arrival (DOA) estimation algorithms to be used in scenarios when the total number of sources are more than the number of sensors. The performance of such time-frequency (t-f) based DOA estimation algorithms depends on the resolution of the underlying TFD as a higher resolution TFD leads to better separation of sources in the t-f domain. This paper presents a novel DOA estimation algorithm that uses the adaptive directional t-f distribution (ADTFD) for the analysis of close signal components. The ADTFD optimizes the direction of kernel at each point in the t-f domain to obtain a clear t-f representation, which is then exploited for DOA estimation. Moreover, the proposed methodology can also be applied for DOA estimation of sparse signals. Experimental results indicate that the proposed DOA algorithm based on the ADTFD outperforms other fixed and adaptive kernel based DOA algorithms.

In this paper, a subspace identification method is proposed for Hammerstein systems under periodic disturbance. By using the linear superposition principle to decompose the periodic disturbance response from the deterministic system response, an orthogonal projection is established to eliminate the disturbance effect. The unknown disturbance period can be estimated by defining an objective function of output prediction error for minimization. Correspondingly, a singular value decomposition (SVD) based algorithm is given to estimate the observability matrix and the lower triangular block-Toeplitz matrix. The state matrices A and C are subsequently retrieved from the estimated observability matrix via a shift-invariant algorithm, while the input matrix B and the nonlinear input function parameters are retrieved from the estimated lower triangular block-Toeplitz matrix by an SVD approach. Consistent estimation of the observability matrix and the lower triangular block-Toeplitz matrix is analyzed. An illustrative example is shown to demonstrate the effectiveness of the proposed identification method. 

Graphs are naturally sparse objects that are used to study many problems involving networks, for example, distributed learning and graph signal processing. In some cases, the graph is not given, but must be learned from the problem and available data. Often it is desirable to learn sparse graphs. However, making a graph highly sparse can split the graph into several disconnected components, leading to several separate networks. The main difficulty is that connectedness is often treated as a combinatorial property, making it hard to enforce in e.g. convex optimization problems. In this article, we show how connectedness of undirected graphs can be formulated as an analytical property and can be enforced as a convex constraint. We especially show how the constraint relates to the distributed consensus problem and graph Laplacian learning. Using simulated and real data, we perform experiments to learn sparse and connected graphs from data.

We show that the recently proposed (enhanced) principal-singular-vector utilization for modal analysis (PUMA) estimator for array processing [C. Qian, L. Huang, N. Sidiropoulos, and H. C. So, "Enhanced PUMA for direction-of-arrival estimation and its performance analysis," IEEE Trans. Signal Process., vol. 64, no. 16, pp. 4127-4137, Aug. 2016], minimizes the same criterion function as the well-established method of direction estimation (MODE) estimator.

This paper focuses on the problem of reconstructing low-rank matrices from underdetermined measurements using alternating optimization strategies. We endeavour to combine an alternating least-squares based estimation strategy with ideas from the alternating direction method of multipliers (ADMM) to recover low-rank matrices with linear parameterized structures, such as Hankel matrices. The use of ADMM helps to improve the estimate in each iteration due to its capability of incorporating information about the direction of estimates achieved in previous iterations. We show that merging these two alternating strategies leads to a better performance and less consumed time than the existing alternating least squares (ALS) strategy. The improved performance is verified via numerical simulations with varying sampling rates and real applications.

In many fields, and especially in the medical and social sciences and in recommender systems, data are gathered through clinical studies or targeted surveys. Participants are generally reluctant to respond to all questions in a survey or they may lack information to respond adequately to some questions. The data collected from these studies tend to lead to linear regression models where the regression vectors are only known partially: some of their entries are either missing completely or replaced randomly by noisy values. In this work, assuming missing positions are replaced by noisy values, we examine how a connected network of agents, with each one of them subjected to a stream of data with incomplete regression information, can cooperate with each other through local interactions to estimate the underlying model parameters in the presence of missing data. We explain how to adjust the distributed diffusion strategy through (de)regularization in order to eliminate the bias introduced by the incomplete model. We also propose a technique to recursively estimate the (de)regularization parameter and examine the performance of the resulting strategy. We illustrate the results by considering two applications: one dealing with a mental health survey and the other dealing with a household consumption survey.

In this paper, based on a successively accuracy-increasing approximation of the l(0) norm, we propose a new algorithm for recovery of sparse vectors from underdetermined measurements. The approximations are realized with a certain class of concave functions that aggressively induce sparsity and their closeness to the l(0) norm can be controlled. We prove that the series of the approximations asymptotically coincides with the l(1) and l(0) norms when the approximation accuracy changes from the worst fitting to the best fitting. When measurements are noise-free, an optimization scheme is proposed that leads to a number of weighted l(1) minimization programs, whereas, in the presence of noise, we propose two iterative thresholding methods that are computationally appealing. A convergence guarantee for the iterative thresholding method is provided, and, for a particular function in the class of the approximating functions, we derive the closed-form thresholding operator. We further present some theoretical analyses via the restricted isometry, null space, and spherical section properties. Our extensive numerical simulations indicate that the proposed algorithm closely follows the performance of the oracle estimator for a range of sparsity levels wider than those of the state-of-the-art algorithms.

We develop a Bayesian learning method for robust principal component analysis where the main task is to estimate a low-rank matrix from noisy and outlier contaminated measurements. To promote low-rank, we use a structured Gaussian prior that induces correlations among column vectors as well as row vectors of the matrix under estimation. In our method, the noise and outliers are modeled by a combined noise model. The method is evaluated and compared to other methods using synthetic data as well as data from the MovieLens 100K dataset. Comparisons show that the method empirically provides a significant performance improvement over existing methods.