Understanding student progression for data mining analysis

As Higher Education (HE) funding continues to be reduced, understanding student progression and identifying ways to improve the student experience are vital to the student, institution and state (York and Longden, 2008). In 2008, a Parliamentary report found that 8.4% first year full time UK (United Kingdom) degree students, who enrolled in 2004-05, failed to progress into their second year (Parliamentary, 2008). Whilst this is a slight improvement to 1999-2000, this still represents a significant financial loss to the student, institution and state. Yorke and Longden (2008) estimate that the cost of non progression is £110 million per annum. We believe that all of the HE stakeholders have an active part to play in improving student progression. Over the years, there has been increased pressure placed upon institutions to widen participation and increase access to HE (York and Longden, 2008). In the current economic climate the numbers of university places available, financing and employment opportunities are a major concern for institutions, students and the government. Therefore, understanding how to improve student progression and the student experience is vital, as financial pressures on all HE stakeholders increase Indeed, finding ways to understand student progression and improve student experience have become integral to a universities corporate plan. Our research is seeks to introduce Business Intelligence (BI) (specifically Data Warehousing (DW) and Data Mining (DM)) tools and techniques to the problem of student progression in HE to help improve student progression and the student experience for all undergraduate students. It follows on from a previous study that looked into the applicability of using of Data Mining to identify student progression problems in HE (Burley, 2007 and 2008). There are two main strands to our research. The first is concerned with gathering the opinions of Sheffield Hallam University (SHU) students from all faculties through face-to-face interviews and an online questionnaire. The second is concerned with creating intelligent user profiles of SHU undergraduate students to predict attainment, non achievement, employment type and progression on to postgraduate studies. This paper discusses the findings of our face-to-face interviews with undergraduate students, at SHU, which were later used to inform a university wide online survey. We go on to discuss the gathering of undergraduate student data, from the university’s Student Information Systems databases, and the development of a number of DM Marts. These marts were developed to predict attainment, non achievement, employment type and progression on to postgraduate studies.