ConvLSTM

ConvLSTM is the spatiotemporal sequence predictor: it integrates Convolutional operations into the LSTM cell to capture both spatial features and temporal dependencies in grid-structured data.

This architecture is a direct upgrade to the standard Fully Connected LSTM (FC-LSTM) for handling multi-dimensional data, specifically sequences of images or grids. The key modification: replacing the internal matrix multiplications with convolution operations. This design ensures that the cell state and hidden state retain critical spatial structure, a necessity for tasks like precipitation nowcasting. Introduced by Xingjian Shi et al. in 2015, ConvLSTM consistently outperforms FC-LSTM and traditional algorithms (e.g., ROVER) on spatiotemporal forecasting problems, making it the go-to model for video prediction, weather modeling, and radar echo extrapolation.

https://arxiv.org/abs/1506.04214