Discovering the hidden knowledge in transaction data through formal concept analysis

The aim of this research is to discover if hitherto hidden knowledge exists in transaction data and how it can be exposed through the application of Formal Concept Analysis (FCA).

Enterprise systems capture data in a transaction structure so that they can provide information that seeks to align with the knowledge that decision-makers use to achieve business goals. With the emergence of service-oriented architecture and developments in business intelligence, data in its own right is becoming significant, suggesting that data in itself may be capable of capturing human behaviour and offerer novel insights from a `bottom-up' perspective. The constraints of hard-coded top-down analysis can thus be addressed by agile systems that use components based on the discovery of the hidden knowledge in the transaction data.

There is a need to connect the user's human-oriented approach to problem solving with the formal structures that computer applications need to bring their productivity to bear. FCA offers a natural approach that meets these requirements as it provides a mathematical theory based on concepts, logical relationships that can be represented and understood by humans.

By taking an action research and case study approach an experimental environment was designed along two avenues. The first was a study in an educational setting that would combine the generation of the data with the behaviour of the users (students) at the time, thereby capturing their actions as reflected in the transaction data. To create a representative environment, the students used an industry standard SAP enterprise system with the business simulator ERPsim. This applied study provided an evaluation of FCA and contemporary tools while maintaining a relevant pedagogic outcome for the students.

The second avenue was a discovery experiment based on user activity logs from an actual organisations productive system, applying and developing the methods applied previously. Analysis of user logs from this system using FCA revealed the hitherto hidden knowledge in its transaction data by discovering patterns and relationships made visible through the multi dimensional representation of data.

The evidence gathered by this research supports FCA for exposing and discovering hidden knowledge from transactional data, it can contribute towards systems and humans working together more effectively.