Developing formative evaluation for complex interaction

© 2019 IADIS Press. All rights reserved. This paper reports on the development, refinement and use of a design instrument for supporting the exploration of interactive tool design. The distinctive focus of the instrument is that it is intended for interactive tools that go beyond the aim of enabling direct interaction. It is intended to tools that by necessity mediate access to rich complex functionality. We argue that the majority of evaluative instruments that HCI designers have at-hand are not particularly useful in this context. We proposed that alternative concepts are of more relevance, specifically the concepts developed in the Cognitive Dimensions Framework. We have developed an instrument to enable and support the critical assessment of alternative complex interaction designs that is motivated by these concepts. Having described the origins of the tool, the paper then describes how it has been applied in three real world design and development settings. The paper concludes with reflections upon how best to refine, encapsulate and further improve the instrument.