Analyzing Data

The next step is to analyze your data and look for trends.

Finding Trends in the Gradebook (Quantitative)

One way to do this is to view the gradebook and look for patterns. Do you see that students typically submit their writing assignments late? Are students consistently performing below the mastery level you had hoped for on their homework assignments? Is there a sudden drop in the class average on a particular quiz?

If you see a pattern emerge, analyze it more closely. In the example above where there is a sudden drop in the class average for a quiz, you may need to check whether quiz questions align properly with the objectives for the week, or whether the lectures and readings cover the material adequately. Was the quiz scheduled at a time during the semester when students are likely to miss the quiz? If you have an idea of what may have caused this issue, you’ll need to decide how to take action (see below). If you’re unsure of the cause you could implore qualitative data techniques or survey the students to investigate the pattern further.

If you are interested in learning more, you can sign up for the Using e-Learning Data to Improve Learning training (details below).

Analyzing and Improving Quiz Questions (Quantitative)

It’s a good idea to complete an item analysis on every new multiple choice quiz or test you give. An item analysis will identify questions that could be rewritten to be made more effective. Your goal should be to write valid questions that will help you reliably discriminate between students who know the material you are testing them on, and students who do not.

The AT Handbook on Testing and Grading explains how to calculate the difficulty, discrimination index, and effectiveness of each alternative for each multiple choice question in your assessment. Your LMS may already calculate some of this information for you. For more information on how to use the statistics available to you within the UF e-Learning system, read the CITT Blog post What a Test Can Tell You? Or, sign up for the Using e-Learning Data to Improve Learning training (details below).

Finding Trends in Survey Results (Quantitative and Qualitative)

Once students have completed a survey, try sorting students’ responses based on their expected letter grade or current average. Do the ‘A’ students feel differently about some aspects of the course than the ‘C’ students? Look for trends. Can you hypothesize why your lower achieving students may feel differently about one aspect of the course than your higher achievers? The next step will be to decide how and when to intervene.

Discussion Boards (Qualitative)

Review your Stop, Start, Continue board and/or FAQ/General Discussion board to identify areas where students needed more clarification. Additionally, review the weekly graded discussion topic boards for quantity (frequency of posts beyond the minimum) and quality to see how you could improve your prompts to make them more engaging.