Do VLMs Need Vision Transformers? Evaluating State Space Models as Vision Encoders
arXiv:2603.19209v1 Announce Type: cross Abstract: Large vision–language models (VLMs) often use a frozen vision backbone, whose image features are mapped into a large language model through a lightweight connector. While transformer-based encoders are the standard visual backbone, we ask whether…
