The Double-Sided Information Bottleneck Function †

A double-sided variant of the information bottleneck method is considered. Let [Formula: see text] be a bivariate source characterized by a joint pmf [Formula: see text]. The problem is to find two independent channels [Formula: see text] and [Formula: see text] (setting the Markovian structure [Formula: see text]), that maximize [Formula: see text] subject to constraints on the relevant mutual information expressions: [Formula: see text] and [Formula: see text]. For jointly Gaussian [Formula: see text] and [Formula: see text] , we show that Gaussian channels are optimal in the low-SNR regime but not for general SNR. Similarly, it is shown that for a doubly symmetric binary source, binary symmetric channels are optimal when the correlation is low and are suboptimal for high correlations. We conjecture that Z and S channels are optimal when the correlation is 1 (i.e., [Formula: see text]) and provide supporting numerical evidence. Furthermore, we present a Blahut–Arimoto type alternating maximization algorithm and demonstrate its performance for a representative setting. This problem is closely related to the domain of biclustering.