October 7, 2020
Written By: Deb Spearing
Even prior to the COVID-19 pandemic, many public schools nationwide were using online Learning Management Systems (LMS) to manage and enhance student education. Still, some recent reports have shown that up to 40% of K-12 schools have been using one or more LMS platforms for less than a year, and a good number of those confirmed that their adoption of one or more LMS tools was due to COVID-19. Understandably, many districts are still learning the ropes about what their LMS can do besides provide a way for students and teachers to share classroom resources, assignments, and communication.
Whether your district is using Canvas, Schoology, Moodle, Blackboard, Google Classroom, or one of the other hundreds of LMS platforms currently available, there’s a good chance you aren’t using the features to their full potential. Only about 1/3 of districts say they are using the reporting and analytics features of their LMS at all.1 As a team of professionals that is obsessed with helping our clients make data-informed decisions, VIA Evaluation wanted to share some of the most useful data collection and reporting tools that most LMS platforms have available to teachers and administrators.
Why Should We Care About Analytics?
Teachers have been logging TONS of hours creating new content and amending standard content to be accessible and engaging in an online environment. Despite their tremendous effort, some lessons just aren’t going to be as engaging in a virtual setting as they are in person. The good news is that there are reporting tools embedded within the LMS that can be very helpful in directing attention towards what’s working and what could work better.
LMS platforms hold a wealth of data that can be used to develop insight about student performance, engagement, time requirements, and even the utility of the platform itself. Using just a few of the reporting and analytics resources available can help your district track student success and develop better management for the valuable resources you’ve put towards operating in a new environment this year.
Although the pivot towards more virtual learning was foisted upon us, perhaps involuntarily, let us not overlook the benefits of this unique experience to capture information about students’ learning and study habits in a way that isn’t possible in a traditional classroom environment. Educators are suddenly equipped with insight they never had before into how our students are spending their time interacting with course materials when teachers aren’t present to walk them through exercises.
What Should I Be Looking At?
The majority of the important reports fall under the broad category of “User Activity Details” — essentially looking at if and when your students are logging on the various LMS platforms, what they’re doing when they get there, and how long they spend on activities. In addition to the basic review of how long students are spending on assignments, here are a few specific things that we recommend keeping an eye on.
Assignment submission times
Something that many students appreciate about virtual learning is that they can complete their work on their own time, but patterns of students who are completing assignments long before they are due or getting them in right under the wire could provide valuable insight into how much time and effort students are putting into their work. Looking at a connection between assignment submission times and similar work being submitted could also be an indication (although definitely not proof) that students are working together on assignments designed to be completed independently.
Number of attempts at submissions
We’ve heard anecdotally from a number of parents and teachers that many assignment and quizzes in virtual classrooms are being left open for multiple submissions or attempts. Between teachers and students both working out the nuances of learning with new technology, leaving the assignments open for resubmission can save a lot of headaches if students are accidentally submitting the wrong version or realize that they have time to revise a draft. High numbers of resubmissions can mean a variety of things, from students submitting work too hastily, to a potential lack of organizational skills, or students being diligent about reviewing their work. Teachers can use this information to make decisions about what types of assignments are most susceptible for re-submits and inquire about this activity with students.
Individual test item answers
As a former instructor of college courses who was often responsible for building my own exam questions, my absolute favorite feature of online exams is the huge time saving from grading by hand. My second favorite advantage is tracking patterns of student responses. Most LMS platforms can flag when a certain number of students gets a question right or wrong, easily identifying which questions were too easy (or just taught very well) or too difficult (or not appropriately covered in the teaching material). I’ve found that tremendous amounts of trust can be built from a teacher admitting, “I see that 90% of the class answered question 5 incorrectly, perhaps I didn’t do a very good job of explaining the concept” and even exploring the option to refund points deducted or offer extra credit.
Rapid exits
When students enter a lesson, document, or assignment and quickly close out of it, there could be more behind that click than accidentally opening the wrong link. Students could be reacting to a feeling that the material is overwhelming or something they don’t feel prepared to tackle just yet. If you notice a trend that multiple students are behaving with your materials in this way, it may be worth addressing the assignment to provide some additional support or motivation for following through with that work.
Total length of time logged in
This is probably one of the weakest metrics to attach valuable insight to, because students often log into a course and then walk away. However, it’s likely to be one of the easiest reports to run so I’ll address how you might use this information. At a very basic level, the fact that a student has logged into a course shows that they are able to access the course and have some awareness of the materials therein. If you don’t have time to look at the specifics of how students are interacting with individual assignments, folders, or activities, the length of time logged in can be a red flag alerting teachers to students who have not logged into a course at all over a period of time or have not logged into the course for long enough to open all of the required documents.
Most viewed course materials
Most of your viewed course materials will have obvious reasons for their popularity – those that hold the keys to completing an assignment, for example, are going to be viewed because students can’t succeed without accessing those files. However, there will always be students who ignore the requirement aspect of opening a folder because it doesn’t look interesting to them. Looking at this report could also be a key to seeing what types of materials are most engaging to students or simply understanding that the teacher has done something to emphasize the importance of that material.
Least viewed course materials
I’ll never forget the day this past Spring when we found out that our 9th grader had missed a number of assignments, when he had previously been performing very well in his new online classes. We quickly realized that the teacher had embedded some assignments in the folders that had been previously used to only to hold notes. Turns out, he had only been accessing the folder where the assignments lived, completing the assignments with his own research, and never reviewing any of the actual course material. Although we were somewhat impressed with his creative approach to time saving, finding out that he had essentially been “skipping class” in the educational sense was a problem.
When you run a full report of the times and dates that folders or course materials were accessed, pay attention to the content that isn’t being viewed, regardless of how well your students are doing. If no one is opening a particular file that you’ve put time and effort into creating, ask yourself why. And if students are succeeding in the class even without reviewing that material, it may not be the most useful content for teaching in a virtual environment.
Other Helpful Things Your LMS Can Probably Do:
Customized reporting
As an administrator, we know this is an extremely difficult time to be asking more of teachers. If you can determine the 4-5 things that you want reviewed on a daily or weekly basis, your LMS may be able to save everyone time with templates for custom reports. Once account administrators or individual teachers set up their reporting template one time, they can run the same report on a consistent basis and monitor the areas that they’d most like to pay attention to based on overall goals for virtual learning.
Automatic emailing of reports
Depending on how streamlined your school’s use is of various LMS platforms, you may be able to set up a feature where all teachers opt to receive customized reports with metrics to review directly in their inbox. From this point, reminders can be set up to open the reports and note very specific elements – like outliers of students who spent a very long time or a very short time logged into a particular course or section of notes.
Create rich data visualizations of student performance
Essentially, what I mean by “student performance” is grades. Naturally, you’re already looking at your students’ grades so this isn’t a new report that you should be paying attention to. However, it might be worth looking into some of the creative things that your LMS can do in terms of creating graphs, charts, and other visualizations for student grades on specific assignments in a way that your traditional methods for reviewing grades didn’t do automatically.
Keep In Mind…
When reviewing your LMS metrics, it’s important to consider what your data points really mean. There are obviously A LOT of other reasons besides motivation, interest, or discouragement that influence how long, how often, or how early students access course material. Still, taking regular glances at your platforms’ data reports can provide a variety of different ways to spot red flags that require some additional intervention to keep students on track with their learning.
If you’re interested in using metrics to get the most out of your Learning Management Systems, VIA Evaluation can help. Contact us for help developing customized plans for gaining insight into your students’ success with virtual learning.
References
1. https://www.capterra.com/learning-management-system-software/user-research/
