Peter Ewell -- The Accountability Scene for Community Colleges: New Challenges, New Measures - Presentation at NCCBP forum

Note slide 18 which shows Milestone Events in a Student Enrollment Pathway -- All of the rates there are great measurements of success, but can be very difficult to obtain.  ZogoTech developed a general-purpose engine for generating these with a few clicks.  It allows time-series analysis with arbitrary starting and ending events.  i.e. so you can determine how long it takes for students to graduate once they have achieved X credits or how long it takes them to pass through the developmental sequence once they have passed their first class.  For more applications, see the Completion Point Reporting Engine

This is a fascinating way to quantify student progress / momentum.  Points are assigned to each student based on how far they have moved through the pathway.  In this case it is used for performance-based state funding, but it could also be used in an individual institution to monitor progress in a more fine-grained manner (rather than just looking at graduation rates).  It allows institutions to focus on gaps between the various stages / momentum points.

      Email from Tina Bloomer, Director, Student Achievement Project:

We keep information about the Student Achievement Initiative at:  http://www.sbctc.ctc.edu/college/e_studentachievement.aspx

We have a one page document that describes momentum points at:  http://www.sbctc.ctc.edu/college/education/momentum_point_calculation_mar07.pdf 

While we identified the actual points we are using for this initiative and decided to call them momentum points, the concept of momentum actually comes from Cliff Adelman, a veteran researcher formerly of the U.S. Department of Education, who developed the concept of the academic momentum perspective based on the Department of Education staff review of the national databases

The Maryland Model proposes a more accurate model for measuring student success in a community college.

 

Carroll Community College provides the following example: their Federal Student Right to Know (SRK) 3 year graduation rate is about 18%, but filtering out students by intent, using a longer time period, and broadening the definition of success to be more appropriate for community colleges, the rate of "success" can be as high as 70-80%

 

There are several interesting things the model does:

• The model looks at the student's behavior after the fact to measure intent.  If the student did not attempt 18 hours during the first 2 years of study, then Maryland Model take the position that the student was not "degree-seeking" and those students are not counted  (debatable, but perhaps a reasonable assumption)
• The model looks at a 4 year time period instead of 3 years 
• The model considers transfer as success
• The model considers 30 hours credit with a 2.0 GPA as success (even if the student does not get a degree)

The Impact of Postsecondary Remediation Using a Regression Discontinuity Approach: Addressing Endogenous Sorting and Noncompliance (An NCPR Working Paper)
By: Juan Carlos Calcagno and Bridget Terry Long (April 2008)

This paper looks at the performance of students just above and below the state cutoff test score for remediation (i.e. students who just barely tested out of remediation vs. those who just barely missed testing out of remediation).  It finds that remediation did not give any additional benefit for those students just below the cutoff, but it did slow them down by a semester.

One of our Achieving the Dream schools applied the same approach in their analysis of students completing gatekeeper courses.  They created learning communities for students who scored just below the cutoff score for college-level math.  Students took both developmental math and college-level algebra in the same semester.  Completion rates were similar to students not in learning communities and the students passed a major gatekeeper to graduation.

An overview of the NCPR paper can be found on Inside Higher Ed

Written in 1994, this article shows a variety of measurements for tracking

It is based in part on the work done as part of the LONESTAR project with David Preston, Brazosport College.  The types of analysis and research presented here are still applicable today (and most data warehouses still can't do them)

Visualizing Data

Tufte is a genius at displaying complex, multivariate data

Visual Display of Quantitative Information

Envisioning Information

Visual Explanations: Images, Quantities, Evidence and Narrative

Early Alert

Estudias uses indicators developed by Noel-Levitz and others to identify students who are at-risk before the semester starts.

This includes indicators such as:  Student has significantly increased his / her course load, Student is retaking a class, Student is remedial in more than one area, Student's GPA has been decreasing over the past 3 terms.  When most schools talk about Early Alert, they are talking about intervening in the middle of the semester.  However has these indicators, they are at-risk as soon as the semester starts.