Functional organisation in learned sensorimotor representations: A lesion study approach to multimodal autoencoders

POURFANNAN, Hamed, MA, Ruidong, RODRIGUEZ, Alejandro and DI NUOVO, Alessandro (2026). Functional organisation in learned sensorimotor representations: A lesion study approach to multimodal autoencoders. Biomimetic Intelligence and Robotics: 100378. [Article]

Documents
38194:1445190
[thumbnail of Proof, replace with VoR once available]
Preview
PDF (Proof, replace with VoR once available)
Pourfannan-FunctionalOrganisationInLearned(Proof).pdf - Published Version
Available under License Creative Commons Attribution.

Download (16MB) | Preview
Abstract
In biological systems, cross-modal processes, such as temporal and spatial synchronisation between sensory modalities, information transfer across senses, and compensatory plasticity, are fundamental to coherent perception and motor control. Humans, in particular, exhibit a strong, partly innate tendency to form such associations. In artificial neural networks, similar correspondences have been deliberately engineered to align two modalities (e.g., CLIP) or multiple modalities (e.g., ImageBind) through shared embedding spaces. However, the emergence of structured cross-modal organisation in embodied artificial systems, such as robots, remains poorly understood. In this study, we investigate the functional organisation of latent representations in a Multimodal Variational Auto-encoder (MVAE) trained on robotic motor babbling data. Using systematic ablation methods inspired by neuroscience lesion studies, we demonstrate that: (1) the MVAE latent space develops spatially organised representations in which specific dimensions encode distinct regions of the robot’s workspace despite the absence of explicit spatial supervision; (2) these representations exhibit heterogeneous tuning properties, including highly selective and highly specific dimensions alongside more broadly tuned units; (3) ablating region-specific dimensions shows that highly selective dimensions produce selective reconstruction impairment; and (4) multimodal integration yields distributed representations that are substantially more resistant to targeted ablation than either vision-only or proprioception-only unimodal baselines, a property functionally analogous to Redundancy Gain (RG) in biological systems.
More Information
Statistics

Downloads

Downloads per month over past year

View more statistics

Metrics

Altmetric Badge

Dimensions Badge

Share
Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email

Actions (login required)

View Item View Item