Comments feature for grading #789
Conversation
* Comments field added to GradingResult, below Grade and XP * Comments editor added to GradingEditor, using react-mde Markdown editor * react-mde updated to latest version * Optional 'comments' field added to the relevant types * Redux actions updated to handle 'comments' field * Test comments added to the mock assessments and gradings * Updated tests to test for new 'comments' field * Refactor of 'Markdown' component to pass down more props * CSS styles for elements in GradingResult and GradingEditor updated
* Implemented draft saving and loading mechanism using local storage * New buttons for 'Save Draft' and 'Load Draft' * Old 'Save' functionality replaced with 'Submit and Continue' * Unique logic for each button to determine applied CSS styles * Modified CSS styles for editor interface * Fixed issue where submitting would raise a 'You have unsaved changes' dialog box * Confirmation box appears when loading draft will overwrite unsaved changes
…tend into reinstate-comments
* Fix for "<td> cannot appear as a child of <tbody>" warning * Fix grading editor checking for grade and XP boundaries on every state change * Tweaked CSS to clean up grey lines on grading editor interface
…tend into reinstate-comments
Pull Request Test Coverage Report for Build 2530
💛 - Coveralls |
There was a problem hiding this comment.
I don't see the benefit of using localStorage to save a draft.
-
We should use the database as the source of truth. Using localStorage makes it difficult to discern whether the data in the database or localStorage should be used.
-
Other graders might have written their own comments. This again creates an issue with the source of truth.
I propose we remove the use of localStorage. Save should just save to the database.
The use of local storage is meant to be temporary, allowing graders to pause their grading and resume where they left off later, without exposing the unfinished grades to students. On issue #739, @blackening mentions that graders may input temporary marking schemes before they actually begin to grade. This draft saving feature would help with that, serving as a convenient way to allow individual graders to store data that will only be available to themselves It should function similarly to copying the grade out to a local text file and saving it, then copying it back into the editor once a grader is ready to finish the grading. The local storage saving makes it much easier as it tracks which grade belongs to which student and which question automatically, and loading fills in all the fields (grade, XP, comments) with a single click
It should not cause an issue with the source of truth, as the current comment that the student can see is retrieved from the backend and always loaded by default. Only if there is a draft in the local storage will the 'Load Draft' option appear, prompting the grader to manually select if personal saved data is to be used Possibly, one change to be made could be an option to append the local draft to the bottom of the document, to allow for ease of use when graders want to add on to something that is already graded
Currently, the backend does not discern the difference between unfinished comments and comments that are ready for view, meaning that once something like a marking scheme is "saved" to the database students will be able to access it, which may confuse them or mislead them with the incorrect information Having said that, a 'drafts' column could be added to the database, but there is no need to unnecessarily complicate the system or crowd the database with temporary data that only need to be accessed by a single user. Use of local storage resolves both of these issues, making sure that the backend system remains simple to understand for developers, and also that the drafts are only accessible by the creators. One change could be to make it very clear that the draft is saved locally; so graders using publicly accessible computers will understand that if they do not submit the grading, there is no way to persist draft data to another grading session later on (on another computer) |
Ok, a little comment here. For the practical exam, i'm not sure how we will grade it. On the other hand, what i'm sure is that it's very unlikely to be a single grader. Usually what happens in distributed grading environments is
In this case, using the local storage may be detrimental to the grading experience.
Yes, this is a problem, i consider this as a future WIP.
The main issue is that there's no finalize grading. It's an implicit process. WIP grading leaks out during the grading process. For now, this is acceptable. So long as they don't get spammed with notifications when grading its fine. In the future we should upgrade the system to prevent leaking information in the middle of grading, but this is fine. |
Okay, I see it may not be used much for grading the practical For Missions and Quests however, how useful would this feature be? I don't see the harm of having this feature on the side, it should be useful for those assignments that can be graded by a single grader |
This is fine. The majority use case is mostly single user. The only question is how much complexity is introduced vs how much value the system adds. |
I don't think there's much value. It's not a form that takes a long time to complete, nor is the data so big that it justifies saving it locally over saving it on the database. |
I agree. Let's take out local storage of drafts. Just save is fine. Once saved, the grader can go back, edit and save again, right? |
The issue I have with just "Save" is that it is not just storing the grade in the database, but also making them available for students to view. For answers that are more difficult to grade, and that graders would like to reference other materials (like other students' answers) the "Save Draft" would be more useful, allowing the grader to type a comment halfway and then return to it Ideally, the backend should be set up to receive the drafts, but this will make it much more complicated as there will have to be logic to clear the drafts from the system once the grading is finalised and available for view. Also, the database may be storing a lot of unnecessary data if the drafts were sent to it, as there could possibly be one draft per question, per student, per grader, leading to many permutations of stored unique drafts Collaboration on the same draft could also be done, but if this is required the grading should probably be discussed in detail by all graders at a meeting, before the grading process begins However, in any case making grades available for students to view should be more permanent, especially with the notification system. It feels really bad as a student to look at your grade one day and then realise it has completely changed on the next, and there is no indication of when you can be certain that the grade is final. Once the grades are made available for view by the students, changes should only be made on a case-by-case basis Overall, I think it comes down to the style of grading for individual graders. For weaker students, graders may prefer lengthy, descriptive comments which may be more useful. These comments would take a longer time to create and so the "Save Draft" may come in handy. But for stronger students, they might leave many short comments like "Good job", which may even be handled in the chat instead of using the grading comments. Providing more options instead of less would make for a better experience in my opinion Perhaps we should set up the backend to handle drafts, but this will require more development time. |
Just put a isGraded flag. It's not hard to implement. Writing test cases will take most of the time. Do not need to optimize unless you can show that it needs optimization. In this case it's 1 text field per question which is acceptable. |
|
Thanks, guys, for reasoning this out so carefully. I understand that it would be nice to have a "save draft" feature. However, it would make the system more complicated and harder to maintain, and is a source of error. For now, I'm arguing for a simple and robust implementation. I'd rather go to the Avengers and tell them: (1) Be careful with Save: Students immediately see the comments. |
We can just not notify students until we are done grading. Unless students check immediately, which is unlikely (no email notifications), this is fine for the moment. |
This sounds like a good suggestion! One question is, what should be shown in the grading editor when you want to edit grading that has already been submitted. At the top, instead of "Currently Grading Student_X", maybe it should show "Editing Grade of Student_X" instead?
Ok, I understand this point of view. I will remove the drafts feature for now. Maybe we should log this as an issue, and then we can come back to it later on. What do you think?
Actually as a student I did check the Source Academy quite regularly. Some weeks there are a lot of assignments available and I would log in and check for updates often, including my grades |
Yes, definitely log as an issue. We shall then label it as _postponed and re-visit after the semester, according to: |
* 'Save Draft' and 'Load Draft' replaced with 'Save Changes' and 'Discard Changes' * Styles for new buttons updated * 'Submit and Continue' renamed to 'Save and Continue' * New Redux Action to support 'Save and Continue' * Updated CSS stylesheet for grading editor * Added tests for 'Save and Continue'
|
Okay, I have pushed a new commit! Now instead of saving drafts, we have "Save Changes" and "Discard Changes"
Of course, the "Save" now refers to actually submitting the grading entry into the database for students to see |
|
Looks great! |
|
What does the discard change function do? Does it delete all unsaved changes, reverting to the previous saved data (from the database) or does it set it blank? |
Yes, it reverts the state to the previous saved data from the database. It never sets it to blank unless the grading did not originally exist on the database Of course, you could just refresh the page to do this as well |
|
There is a dialog box set up before the discard actually takes place, so graders should not accidentally throw away their work |
After some recent discussions, @martin-henz has decided that the comments feature should be re-implemented. This will ensure that grading is not too reliant on the Chatkit API.
Hence this commit will be focused on adding comments and comment editing to the grading system.
Key changes:
The 'react-mde' Markdown editor (documentation here) has been put back in the grading editor, and updated to the latest version v7.6.2 (was previously v5.6.0)
The grading report shown to the student will now display comments:
Editor preview uses the same component as the comments displayed to students - compare the screenshot below to the one above
The 'Save' button has been replaced with 'Submit and Continue' - see issue Grading: Pressing "Save" should go to next question #739
New: a draft feature where graders can save drafts of their grade adjustments/XP adjustments/comments to their browser storage (without making the grading available for view for the student):
For clarity, this is how the grading editor will work in this version:
Enter the grading overview page and click 'Edit' to enter the grading workspace
The default text loaded into the editor will be the current comment for the student's answer, if any
The website will look for a grading draft corresponding to that specific student's answer that is stored on the local system. If such a draft exists, a notification pops up and 'Load Draft' is highlighted until it is clicked:
Proceed with grading. Once any of the values change, 'Save Draft' and 'Submit and Continue' will light up.
If the grade or XP are out of bounds, a notification will appear and you will be unable to save or submit:
Select 'Save Draft' and the draft will be saved into the local browser storage. It can be loaded again with 'Load Draft'
If 'Load Draft' is clicked, and the saved values are different from the editor values, there will be a dialog box asking for confirmation:
Select 'Submit and Continue' when done with the grading. The website will attempt to send the grading results to the backend, and when this is successful it will automatically navigate to the next page:
To accommodate these changes, the backend had to be modified to store comments in the database. Refer to this Pull Request for backend commits: source-academy/backend#461
Please leave comments on either Pull Request to suggest changes!