The IB method finds compressed representation T of data X that preserves relevant information about target Y.
Trade-off: Low β emphasizes compression, high β preserves more task-relevant information.
• Deep learning (understanding neural networks)
• Feature extraction and dimensionality reduction
• Clustering and categorization
• Lossy compression with relevance