A collaborative internship program between the University of Liverpool and Cubewise—industry experts in IBM Planning Analytics (TM1)—gave students hands-on experience analysing real-world data using enterprise technology. In partnership with academic leaders in mathematics and data science, students took part in a remote internship analysing over 9 million rows of university application data using TM1 and TM1py.

The remote format enabled flexible communication and regular check-ins with Cubewise mentors, while also giving students a taste of working in distributed professional teams. Through this experience, students developed technical skills and gained insights into how data informs strategic decisions.

Two student teams tackled real university challenges:

• Group 1 focused on student recruitment, identifying application trends and demographic patterns.
• Group 2 investigated student retention, uncovering factors—particularly from low participation areas—that contributed to student withdrawal.

(disclaimer: this transcript has been automatically transcribed so it may contain errors)

Okay. Let’s let’s get started. Shall we welcome everyone? My name’s Tom. And I’m coming to you from London, UK. I’ve got with me Corina Constantinescu. Who’s the professor of mathematics and also the director of Institute of financial and actuarial mathematics with us. We’ve also got Phillip Hopwood, who’s their director of strategic planning at the University of Liverpool. And so there, we got Niall Booth. Who’s the head of data science at the University of Liverpool. So what they’re going to take you through is a talk on inspiring the next generation student internship, with TM1 and TM1py.

So let me just share my screen and then over to you guys, just to, just as a method of asking questions in the chat, if you want to put any questions in the chat, I’ll be monitoring that. And then at the end of the presentation, then we’ll go through those and we’ll ask, ask the team your questions. So feel free to ask your questions in the chat.

Okay. So over to you Corina.

So it’s my pleasure to speak about the student internships experience at University of Liverpool and Tom if you can share the next slide. At University of Liverpool, the mathematics department works for a number of years now with industry data where undergraduate students work in groups of five under the supervision of graduate students. And they simply provide data input data. So that data insights in data sets from companies that are willing to work with us and pose questions that, that interest them.

And why do they work with us? And why is it exciting? Because students provide always an untaped untainted point of view. They have fresh eyes. They love to work with real data. And to try to answer real questions. This year, this last summer was the first time our own university provided us with a data set. And in addition, they were generous enough to provide us with a software that goes with it. And our students had the chance to work with TM1 and TM1py.

I just want to mention that it was unique as well, that this was first time we conducted this internships remotely, which proved to have some benefits, as in asking easy, setting easy meetings with the Cubewise for students, if they had any questions. But of course, missed the vibe in the room that the students get when they work together in a classroom.

So among the ten groups that worked this summer, which is usually the number of groups that we have in a normal summer project, two groups looked at recruitment—one at recruitment, one at retention.

So the question is posed by our strategic planning, our own Philip that you’ll hear from in a minute: What factors influence decision-making in recruitment, and what actions can we take as a university to improve recruitment, recruiting the best of students? And also in terms of retention—why are students withdrawing? And again, as a university, what interventions can we make, and when?

So over to you, Tom, to tell us a bit of about what’s behind.

Thanks Corina. Yeah. So the Cubewise went to the University of Liverpool to develop TM1 model that used TM1py to enable us to analyze the data that we had on applications and completions.

So some of the challenges that we had around the data was the sheer volume of data on applications—there was sort of over 9 million rows of data. That meant it was impossible to sort of view that data without use of a software like TM1, to be able to allow users to analyze and explore the data.

So we went about loading the data into three different cubes: the applications data, which was students signing up or applying for university places, went into an Applications Cube; the sort of weekly status of the application—that data went into our Weekly Applications Cube; and the completion data, which was the passage throughout the years of their study at the university—that went into a separate Completions Cube.

What it enabled us to do with TM1 was to use its natural ability to aggregate the data so that you could provide the number of students and metrics on the number of students that applied with different attributes to do with those students—like where they came from, what type of background—and that data was summarized in those three cubes and then made available to the students that needed to extract the data using TM1py.

So our TM1py team used Jupyter notebooks and took the students through how to extract the data from TM1 using TM1py to enable them to do that analysis and then feed their reports that they fed back on the course.

So yeah, really interesting use case to be able to see the ability for TM1 to handle vast amounts of data and how TM1py can connect to that data to help you sort of extract and analyze further.

Okay. That’s kind of it on the TM1 model. So back to you Corina on student experiences.

I have the opportunity to share what the students thought of TM1 and TM1py, which might be of interest to many other TM1 users. Especially because this was their first time they’ve seen TM1 and TM1py, and the entire project took six weeks.

So I asked them their opinions, and it was that it’s very easy to use for beginners because the interface is user-friendly.

It was really helpful that TM1py has a very comprehensive Python library, which they used for building their model. What they really appreciated and kept telling me a lot was that using TM1 they got cleaner data, and transfer between data sets and reading data was much better than from Excel files.

Also, these are second year students going into third year. You see them in the pictures—the ones that were involved in these two projects. They had or had not Python experience coming into the project. So they all were happy that they did not need prior Python experience and knowledge, but they felt inspired to learn Python after using TM1py and seeing all the features and facilities.

So overall, this was one of the most excited groups with their project, both because the question was meaningful in terms of helping their own university make better decisions, and also because the software they used and the new methods they learned were very new and exciting.

So I’m really happy to hear what Niall and Phil had to say about their results.

So back to you.

Thanks Corina. So I’m going to talk about the first group who looked at student recruitment. So we’ve asked the students to look at what factors influence the decision-making of our applicants who are looking to come to the university, and what actions we could possibly take to get involved in the application process to improve the number of students—and the number and quality of those students—who end up coming to Liverpool.

We provided the group with five years of application data: 2016 to 2020. So it was well over a hundred thousand applicants worth of data. So we could be confident that their findings would be robust.

We focused on the undergraduate applicants. We did that because in the UK there’s a standard application process for undergraduates—all undergraduate applicants have to go through it—so we can be confident that the data will be high quality and replicable.

We shared with the group all the applicant demographics: where they are from, what part of the UK, what subjects they were studying, what ethnicity they were, if they had any disabilities—all kinds of things like that.

We also shared with them the standard application decision process that every applicant to a UK university goes through. And we also gave them competitor win-loss data. So if an applicant decides not to come to Liverpool, that data tells the students which one of our competitors they ended up actually attending.

Every year we make tens of thousands of offers to students. Only around 20% of those offer holders actually select Liverpool as their first choice, 13% select us as their second choice, and 67%—so around two-thirds of all offer holders—actually decline our offer.

So we look at that and we think, you know, that’s tens of thousands of high-quality applicants who have taken the time to apply to Liverpool, get offered a place at Liverpool, and then decide they don’t want to come here. So what can we do to get a nice slice of that 67% pie?

So these are—we’re just going to go through—a few of the factors the students managed to identify as having an influence.

The first thing is the grounds of our applicants. So you see this chart on the left-hand side—it looks at Polar 1, 2, 3, 4, 5. Polar is a measure supplied to each postcode within the UK, and it’s a measure of how many young people in that postcode actually end up going to university.

So Polar 1 are those areas in the country where the fewest number of young people end up going into higher education. And we see the Polar groups 1 and 2—those low participation areas—students are much more likely to withdraw from the application process. Around 4% of all applicants from those areas withdrew, compared to less than 2% from Polar areas 4 or 5, which are those areas of higher participation.

That’s perhaps not surprising, but it’s certainly concerning, because we want to make sure that students from these more deprived backgrounds are confident to attend a high-quality university.

On the right-hand side, the students looked at the region. Region 1 is the region closest to Liverpool and Region 4 is the region furthest away. Again, we can see—perhaps not surprisingly but certainly very interesting—that students from regions further away from Liverpool are about three times more likely to withdraw from the application process than students from the region close to Liverpool.

Interestingly, that intermediate Region 2—which is areas moderately far away from Liverpool, not in the direct city area but just a bit further away—students were half as likely to drop from the process. Only 1.5% of students from Region 2 ended up withdrawing from the process.

Next slide, please, Tom.

Okay, and the final factor we found really interesting was the time it takes for the university to respond to an applicant’s application. We can see there that if an applicant had a response from us within a week of submitting their application, only 62% of them withdrew—compared to if an applicant had to wait between 15 and 30 days for a response, 74% withdrew.

So a big 12% difference just based on how long it takes us to reply to an application. That’s giving us some key insight: the quicker we can reply to an application, the more likely it is that the applicant will end up coming to the University of Liverpool.

On the right-hand side, we’ve got a bit of a decision structure that the student interns advised to us. You can see that they’ve highlighted some key interventions—some key points at which we could get involved in the application process to try and improve the number of applicants who decide to come to Liverpool. They put in actions at each stage as well.

So when we receive the student’s information, we should classify them by that POLAR score and region to see if there’s anything we can do to help to make sure students from those critical areas end up coming here.

Then, when we receive a student’s reply, make sure that we make an appropriate offer and that they’re aware of scholarships and other options available if they come to the University of Liverpool—all the way through to registration. So that is really useful for us to look at and to know where we can start making those interventions.

And now I’ll hand over to Phil.

Thanks, Niall. The other half of this is once we’ve already got the students—the students have got through, they’re the 20% that have come to us—do we keep them, or do they leak out?

For this, we looked at seven years of data—again undergraduate, as we did with Niall’s piece. We looked at 125 different factors, including (as with admissions) the student demographics: whether they came from low participation neighbourhoods or high participation neighbourhoods, whether they came in with great qualifications, good qualifications, or where they were coming in with lower qualifications, and also the subjects that they were studying. Were there any particular subjects that had lower or higher rates of retention?

One thing that we found here is that roughly every year, 500 students chose not to continue their degree. That translated, in financial terms, to a loss in income of £4.7 million. Now, I know that within the last year we had a surplus of somewhere around £17 million, so gaining some of that £4.7 million back would be incredibly useful for our surplus, to reinvest it in areas for the university.

Next slide.

This time we looked at three different areas. What you’re seeing here is the marks that students get as they go through their three terms of the year at various points—when they were more likely to withdraw.

You might expect that those students who are failing would withdraw at the end of the year, but this is kind of showing that we are getting withdrawals of those that start to fail individual modules as they go through the year. They are much more likely to withdraw than others.

Now, good for the university—over recent years that’s got better, as we’ve managed to put targeted interventions in place. But it’s still showing that behaviour: even those that are just passing or coming up with third-class results—they are showing a propensity to withdraw as well.

Next slide, please, Tom.

The other perception is about the age of students. Most of our students come in at the age of 18—they’re directly leaving school. But quite often, we get mature students—those who are over 21. Sometimes with them, they don’t necessarily have the learning or study skills that are as current as those leaving school at 18.

We see this in the data. Perhaps there’s a tension there when they come in that leads them not to perform as well, and therefore more likely to withdraw. Maybe they’ve even got an alternative career option that they wanted to take. But they have a greater propensity to withdraw. So again, this is getting under the skin of that one and targeting it is something that we as a university need to focus on.

Next slide, please, Tom.

The final aspect, which was what Niall was talking about: what about socioeconomic class? Those that come from low participation neighbourhoods—again, we see this as being areas where we are leaking more students than we should do. Yes, we’ve improved it over recent years, but it’s still there as a main driver. Probably the same reasons that apply to those groups not selecting us as their first choice or not coming to us before they arrive are still prevalent after they arrive.

Those are the sorts of main perspectives that we got from the study. We presented it to a number of our other admissions colleagues within the university. And as you might expect, one of the reasons we’re doing this is we probably thought that these were the answers we’d get—we just had never quantified them with such a degree of robustness as we have before.

So it was very useful. A small group were quite taken aback at how we were able to substantiate some of the main drivers of these behaviours. It’s clearly within the data.

And that draws us to an end. So I’ll pass back to Tom and we’ll take any questions that you might have.

Okay, great. Thanks Phil, Niall and Corina. Fascinating to hear some of the results that have come out of that data analysis that the students did.

If anyone’s got any questions, then put them in the chat now.

I’ve got a question about—now that you’ve seen that and you’ve seen the results that have come out of being able to use TM1py to allow students to analyse it—has it sparked any new use cases that you might consider in the future in terms of analysis?

I’ll take that, if I may.

When we presented this to our admissions colleagues, what we got back was: “Have you thought about this? Have you thought about that?” Loads of drivers that we hadn’t put into the original questions for our students—because we wanted to keep it as simple as possible.

So yes, it told our colleagues much of what they thought, but in a lot more detail. But it also raised a number of questions that we would try and put into the next set of analysis when we get interns next year.

So hopefully, we’re progressing to a richer and richer understanding of which students come to us, where we can put key interventions—and also how we retain them once they get here.

Great. So I don’t have any more questions coming through at the moment.

So thank you all for attending and presenting that. Really fascinating to see how you can extend TM1 using TM1py and see a real use case of how it can drive the change in behaviours going forward.