Challenge: Sophisticated cohort tracking and time series analysis is difficult
Colleges are being called upon to conduct more sophisticated analyses than ever, including comparing and tracking student cohorts. However, rarely are the data needed to conduct such analyses available within the college’s data systems. As a result, it is up to Institutional Research staff to generate their own calculated variables, a difficult and time-consuming task with a high potential for error. In addition, BI tools are not designed for time-series analyses, forcing IR/IT to do a lot of data prep and still ending up with a report that is largely static.
For example, here are some challenging time-series questions:
Once a student passes class X, what is the probability that he/she will graduate in 3 more years? (i.e. to identify gatekeeper classes)
Once a student enrolls in class X, how long does it take him/her to pass it? How long before he/she will enroll in/pass the subsequent class?
What percentage of students in a particular cohort (academic program, gender, ethnicity) graduated within 8 terms of enrolling?
In response, ZogoTech has created a data model as well as a number of front-end tools that make it easy to do sophisticated time-series analyses.
Example
Let’s dig deeper into one example: identifying the most important gatekeeper class. One way to approach this is to ask “What is the likelihood that students will graduate within n terms of completing each of the biggest gateway courses (English Composition, History, College-Level Algebra)?” Cohort comparisons such as these are difficult because students complete the course in different semesters. Here are some of the steps needed:
Get a list of all students who ever completed English 1301 (or perhaps limit it to those who completed in particular terms or those who had certain characteristics)
Decide whether to use the first time or last time the student completed the course since students could have finished the course multiple times
Calculate the number of terms from the term the student completed the course to the term when the student graduated
Decide whether to use the first or last time the student graduated since students could graduate multiple times.
Decide whether to look at just the terms the student was enrolled, whether to count intervening terms, and whether to just look at long terms (i.e., Spring and Fall)
Update a field indicating the number of terms calculated in #3
Repeat all steps for all students who ever completed History 1301
Repeat all steps for all students who ever completed Math 1314
Not only would it be difficult to write this report by hand, but it would also be largely static. If you wanted to use a different cohort, choose different classes, change the decisions you made, you would need to re-do the entire process. If you wanted to look at other outcomes such as transfer, you would need to add additional steps to parse and integrate the NSC files. You would have to take into account students who reverse transfer, swirl, etc.
Solution: Data platform and applications
With the ZogoTech data platform and front-end tools, it’s easy to run all of these analyses in just a few minutes.
ZogoTech’s data platform includes multiple cohort definitions (e.g., FTIC, IPEDS, starting, ending). The Student Navigator allows end-users to track any arbitrary group of students. For example, students could be grouped and tracked based on certain criteria (e.g. any students who are taking both MATH-1314 and ENGL-1301 at the same time) or can be arbitrary lists of students who don’t fit any criteria stored in the SIS (e.g. a list of students in an Excel spreadsheet who signed in at a success skills workshop). Users can compare groups that receive an intervention to those that have not, and further disaggregate by dozens of variables such as gender, ethnicity, high school attended, classes attended, etc.
Outcomes are very flexible as well. The analytics platform computes outcomes such as performance-based funding momentum points, integrates NSC files for transfer outcomes, integrates degree audit information for progression outcomes.
Completion Point Reporting
For more sophisticated cohort analysis, ZogoTech has developed a powerful tool for calculating conditional probabilities and time-series analyses.
Comparing cohorts can be very difficult without the appropriate transformations in the data warehouse. An important transformation shown in the example above is the Term Index: the number of terms relative to the student’s starting cohort. This chart shows the Associate Degree graduation rate over time comparing students who started in each of three fall semesters. After 10 terms, the 2002FA cohort graduated the largest number of students.
Users can easily modify these reports. For example, users can select any starting point (starting cohort, obtained a certain number of hours) or any ending point (graduated, transferred, completed a certain class). This allows users to answer questions like “once a student has 15 hours how long does it take to reach 45 hours? and what’s the probability of that happening?” Terms can be measured in Elapsed / Enrolled terms, or Long / Short terms (i.e. include summers). Users can combine modules to pick out arbitrary groups of students using the Student Navigator and then track them using Completing Point reporting.
The following example shows how long it took students to graduate once they had completed each of 3 courses: college-level English, college-level math, or history. Easily we can see that College Level Algebra (Math-1314 in green) is by far the biggest gatekeeper course.
We see that approximately 20% of students will graduate within 6 terms of completing College Level Algebra. The chart shows over time that not only is Math 1314 the biggest gateway course, it is as big as both of the others combined.
Table detail below:
