Channel-Wise MLPs Boost RCN Generalization

This document presents a research paper that investigates how channel-wise mixing using multi-layer perceptrons (MLPs) impacts the generalization capabilities of recurrent convolutional networks. The authors introduce two architectures: DARC, a standard recurrent convolutional network, and DAMP, which enhances DARC by adding a gated MLP for explicit channel mixing. Through experiments on the Re-ARC benchmark, the paper demonstrates that DAMP significantly outperforms DARC, especially in out-of-distribution generalization, suggesting that MLPs enable the learning of more robust computational patterns. The findings have implications for neural program synthesis, positioning DAMP as a promising target architecture for hypernetwork approaches.