Our work on Information Theoretic Learning uncovered the central role that moments of the PDF, instead of the moments of the data, assume in the definition of cost functions for advanced adaptive filtering. An obvious link to Information Theory thru Renyi’s entropy exists and has provided further understanding. This talk will seek a linkage between these apparent disparate concepts and a functional analysis framework for non parametric estimation. Applications to system identification (channel equalization), blind deconvolution and matched filtering will be presented.

Time permitting, this talk will also present a new similarity function called correntropy. The name was coined to show that it is similar to correlation but its mean value across delays (or dimensions) is the argument of Renyi’s quadratic entropy. This similarity function has the potential to change the way we design nonlinear signal processing algorithms.