Collecting Data

There are a number of ways you can collect and analyze data from your online courses to engage in learning analytics. Once you have collected data, you will use it to make a prediction, change an aspect of the course based on the prediction, and then collect data again to see whether your change has had a measurable effect on the course. Below is an overview of both quantitative and qualitative data collection methods. Both can add value to your course evaluation because you capture the majority opinion as well as those of the individual.

Quantitative Data

Data is considered quantitative any time a number is generated by the data collection process. For example, students provide feedback by selecting a satisfaction level on a scale, or data is aggregated based on students’ performance on an assessment. You may collect quantitative data through surveys, the learning management system (LMS), or by entering test data by hand into a spreadsheet. Quantitative data is powerful because it quickly reveals the majority trend, allowing you to identify potential issues promptly.

  • You can build and administer a questionnaire to your students through an online survey platform like Qualtrics to collect data on how they view different aspects of the course. For example, you may want to collect information on students’ satisfaction with the following aspects of your course:
    • Hours spent per week
    • Opportunities for instructor-student interaction (e.g., office hour availability)
    • Assignment weighting/course structure
    • Lecture quality and length
  • If you are using an LMS to administer quizzes or tests, there is most likely a way to access and download quiz statistics from within the LMS. Search the instructor guides for your LMS to find information on how to view quiz statistics. The resources below will help you find this information in the e-Learning system at UF.
  • Data can also be collected by hand. The Cornell Test Construction Manual provides an excellent step-by-step guide to creating an effective test, analyzing responses, and revisiting it for improved validity and reliability.
  • You could also review the gradebook to look for patterns (lower than average performance, high percentage of late submissions, etc.).

Qualitative Data

Feedback can be a valuable asset in evaluating your content delivery and general course efficacy. In certain cases performance feedback is instant. When a band’s performance is lack luster, the crowd is despondent. When a comedian has a bad set, the audience doesn’t laugh. But for educators, especially those instructing online classes, it can be very difficult to tell what is landing with your students and what bombs. Enter, qualitative data. One of the benefits of qualitative feedback is that it allows you to collect specific data and experiences that may otherwise fall through the cracks of quantitative data collection. A student can’t answer a question that was never asked, so the goal of qualitative data collection is more exploratory than quantitative data collection as it gives them the opportunity to report on their experience and can provide more information on “why” it matters. Below are some qualitative data gathering techniques you can use in an individual course.

  • During the semester, students are asked to list specific things you should STOP doing, START doing and CONTINUE doing.
  • At the beginning of the semester, create an FAQ/General Discussion board.
  • Include free response questions in your end of semester surveys to allow students the opportunity to provide suggestions or comments that would otherwise go unreported.
    • Sample: We value your feedback on your online learning experience. Please use the space to add any additional comments and/or suggestions beyond what was asked. Please be as specific as possible with your constructive criticism.
    • Encourage survey responses by sending out reminders (2 weeks out, at least 2 reminders) explaining how you have used previous survey responses.