A typical instructor workflow for manual grading is currently this:
Instructor creates an assignment with a mixture of manual and automatically graded questions.
Students take the assessment and are shown the message “Waiting for instructor grading” at the end of the attempt.
Instructors enter speed grader and need to check through every student, navigate to the ungraded questions and change the scores.
After all grading, instructors send all grades to the gradebook.
Some feedback we have received with the current implementation (release 3.19 and earlier):
The instructor has to remember to disable automatically submit to gradebook to prevent students from seeing partial scores.
When the instructor first enters speedgrader for an attempt, there is no partial score shown at the top. The expectation is that the partial score from auto-scored questions would show up.
When an instructor changes any question grade, the system then computes the student total score and shows it to both the instructor in speed grader and the student on the review screen. This likely happens before the grading is completed and can be misleading to the student.
The default score displayed on an ungraded question is 0, which means it’s awkward for an instructor to change the score to 0 and update the scores. (This is because the submission code only runs when the grade changes away from 0.)
There is no way for instructors to see the list of all attempts that still need grading.
There is no way for instructors to efficiently navigate to ungraded questions in speed grader.
After a grade change in speedgrader, the grades go through a process of updating in Learnosity and Canvas. The instructor may feel that it’s necessary to wait 5-10 seconds for the scores to update.
The system does not correctly identify autogradeable questions. For example a feedback essay that is scored out of 0 pts is considered not autoscorable, and a short text question which is marked as not autoscorable (i.e. no answer given) is considered autoscorable.
Changes made in 3.20:
We maintain a graded state on attempts which keeps track of attempts that need manual grading. A completed attempt becomes graded when Learnosity returns a score for all questions. An attempt is also marked graded when submitted to the gradebook.
Auto-gradeable assignments are marked as graded immediately after scoring. For assignments with manual graded questions, this will happen either after an instructor finishes scoring all ungraded questions, or the attempt is submitted to the gradebook.
When viewing an ungraded attempt in speedgrader:
The current total score will always be visible at the top together with an indication that manual grading is necessary.
An index of ungraded questions with direct links will appear at the top of the page.
All ungraded questions will show a blank score rather than 0 so that scoring as 0 is a detectable change.
Grading changes will cause an immediate “Save” and then “Save done” message. This should happen within a second or two. The learnosity scoring job will launch at some point later and will rescore the attempt and update the score at the top of the page. It will be clear to the user that it’s not necessary to wait for this to happen.
Any question that learnosity can’t grade will be considered manually-graded. This could be any question type that doesn’t have a provided answer, even questions like multiple-choice. It also won’t consider any question with max score 0 as manually graded, even if it’s a typically manually-graded question.
When viewing the list of all attempt in the connector:
A new status of “Needs grading” will show for attempts in the grading state. This will allow instructors to immediately view unscored attempts.
Graded attempts will continue to show up as “Completed”.
When a student views an attempt:
An attempt with manual-graded questions will always show “Waiting for instructor review” until the score is sent to the gradebook. (Alternatively we could wait for the graded state, but waiting for submit to gradebook would be a more common release pattern for grades.)
If the primary attempt has been sent to the gradebook, and a new ungraded attempt has been made, then ideally only the new attempt will show as waiting for instructor grading.
When an instructor edits assignment settings:
If an assignment is manually graded questions and autosubmit to gradebook is enabled, a warning indicator should be displayed.