Deep learning systems for estimating visual attention in robot-assisted therapy of children with autism and intellectual disability

DI NUOVO, Alessandro, CONTI, Daniela, TRUBIA, Grazia, BUONO, Serafino and DI NUOVO, Santo (2018). Deep learning systems for estimating visual attention in robot-assisted therapy of children with autism and intellectual disability. Robotics, 7 (2), p. 25.

robotics-07-00025.pdf - Published Version
Creative Commons Attribution.

Download (1MB) | Preview
Official URL:
Link to published version::


Recent studies suggest that some children with autism prefer robots as tutors for improving their social interaction and communication abilities which are impaired due to their disorder. Indeed, research has focused on developing a very promising form of intervention named Robot-Assisted Therapy. This area of intervention poses many challenges, including the necessary flexibility and adaptability to real unconstrained therapeutic settings, which are different from the constrained lab settings where most of the technology is typically tested. Among the most common impairments of children with autism and intellectual disability is social attention, which includes difficulties in establishing the correct visual focus of attention. This article presents an investigation on the use of novel deep learning neural network architectures for automatically estimating if the child is focusing their visual attention on the robot during a therapy session, which is an indicator of their engagement. To study the application, the authors gathered data from a clinical experiment in an unconstrained setting, which provided low-resolution videos recorded by the robot camera during the child–robot interaction. Two deep learning approaches are implemented in several variants and compared with a standard algorithm for face detection to verify the feasibility of estimating the status of the child directly from the robot sensors without relying on bulky external settings, which can distress the child with autism. One of the proposed approaches demonstrated a very high accuracy and it can be used for off-line continuous assessment during the therapy or for autonomously adapting the intervention in future robots with better computational capabilities.

Item Type: Article
Additional Information: This article belongs to the Special Issue Intelligent Systems in Robotics
Research Institute, Centre or Group: Materials and Engineering Research Institute > Centre for Automation and Robotics Research > Mobile Machine and Vision Laboratory
Departments: Faculty of Science, Technology and Arts > Computing
Identification Number:
Depositing User: Alessandro Di Nuovo
Date Deposited: 04 Jun 2018 08:55
Last Modified: 04 Jun 2018 17:51

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics