A Summary of the Procedure to Create Special Indexes and Indicators in a Retrospective

Cohort Study of Science, Technology, Engineering, and Mathematics Students

Creating a Retrospective Cohort Study

Select the sample

Request from the Registrar the number of copies of the transcripts of randomly selected students you decide on, who fit the criteria of the study, covering any personal identification material in the copy. For example, first-time, full-time students entering in the fall of 1994 who were admitted in Chemistry, or with a preference for Chemistry, or who declared as Chemistry majors at their first opportunity (if the institution does not admit by major). Make sure to set the entrance date sufficiently far in the past to allow for resolution of their academic career, remembering that today’s students may take up to seven years to graduate in STEM fields.

Back to Top

Collect the data

Decide what data to record, and have reliable workers do it manually from transcripts provided by the Registrar if a computerized system does not yet exist at the institution.

Extract from the transcripts data relating to the pertinent variables relating to the final disposition of the student (creating a worksheet such as that presented in the manual): graduation date and status at time of graduation (major, average in major, and GPA), or year leaving and average in major and GPA at time of leaving.

Review the transcripts again, noting all courses where students obtained an unsatisfactory grade. The compilation of these courses constitutes a list of candidates for problem courses.

When all data on student performance in these courses is collected, it will be evident which courses are truly problem courses -- those causing significant disruption in student pursuit of timely graduation. This may take into account how many students were affected, what percentage of students were affected, how seriously students were affected, etc.

Compile a list of the results of the first attempt in these problem courses (in a worksheet).

Place those results in the appropriate columns (2, 3, 5, 7, and 8) of a Student Performance in Problem Courses spreadsheet.

Enter the appropriate formulas in the remaining columns to calculate the inferred values*, completing the Student Performance in Problem Courses table.

Back to Top

Analyze the data

Determine what results are significant, developing hypotheses and/or explanations, and providing a meaningful description of the data. The manual offers some ideas on how to do this. A review of one important area of conclusions with a substantial subjective component is the “Brain Drain.”

Calculating the Brain Drain

Identifying a “brain drain” is relatively subjective and speculative. Constructing the data base requires only that for each STEM student who leaves before graduating, data is registered on the major, the GPA, and the number of credits of the departing student. Then the cases are ranked (within majors, if so desired), to see if a pattern emerges.

Based on subjective judgment, assisted by the patterns, groups of students are categorized as possessing modest, good, or outstanding GPAs and minimal, modest, or substantial numbers of credits to create categories. They can then be classified as “highly probable,” “likely,” “possible,” or “unlikely” to be participants in a brain drain from the institution in question.

*Calculating the inferred values:

Stymie Rate

To calculate the Stymie Rate take the number of students who took a particular course (head count, counting individuals only once, no matter how many times they took the course) and divide this sum by the number who never passed it in satisfactory fashion (usually defined as passing with an A, B, or C).

ICE Number

For each (high-risk) course, divide the number of students taking the course by the number that complete the course with a satisfactory grade (usually A, B, or C). Each time a student takes a course counts separately. Student equivalents rather than one-time head count are appropriate, since each attempt represents a cost to the institution in terms of course space.

Back to Top

www.prlsamp.org