Cross-check

Process

Step #1 Submitting your work for review

To submit your work, follow these steps:

1. Visit the website at https://app.rs.school/.
2. Click on the “Cross-Check: Submit” link.
3. Depending on the task requirements, provide either:
   • A link to a fully operational and deployed website, or
   • A link to a prepared pull request.
4. The link can be submitted multiple times; the latest submission will be saved.
5. Be aware that any work not submitted within the given timeframe will receive a score of zero.
6. Note that students who do not participate in the cross-checking process will also receive a score of zero.
7. The deadline for submission is stated in the schedule.

Step #2 Review pairs generation

After the deadline for submitting the work, pairs will be generated for reviewing the assignments. Each student needs to review four other students' works.

Step #3 Checking each other’s work

Students who have submitted work should also review others' work. To do this, visit https://app.rs.school/ and follow the Cross-Check: Review link. Here's the review process:

1. Select the assignment (e.g., Portfolio).
2. Choose a student to review.
3. A link to the student's work should appear under their name after selection. If the link is broken, contact the student to provide a correct link.
4. Review the work according to the verification criteria specified in the task.
5. Assign a grade in the form.
6. Leave a comment explaining the given grade.
7. Submit the form.
8. Each student reviews four other students' works.

You can check your review history by visiting app.rs.school, clicking the Cross-Check: Review link, and selecting the assignment and the student you have reviewed.

Step #4 Getting Grades and Comments

• The overall grade is the average of the grades given by all reviewers for your work.
• If four people reviewed your work, the lowest grade is omitted, and the average of the remaining three grades is calculated.
• If fewer than four people reviewed your work, all received grades are considered when calculating the average.
• If you have reviewed all four works but don't have a grade, it means that less than two people have reviewed your work. Don't panic; just be a little patient and wait.
• If you haven't reviewed all the assigned works of other students before the cross-check deadline, your grade for the assignment will be 0 points.

Quality of Work Review in Cross-Check

The purpose of the cross-check is not only to review the work of other students and assign grades to them but also to gain valuable insights for yourself from this reviewing process and provide benefits to those whose work you are reviewing. When conducting a review, try to adhere to the following guidelines:

• If you come across excellent work during your review, let the author know and share your thoughts on their work.
• If you believe that the work you are reviewing deserves attention from other students, add a link to it in the form for the best works. You can also add a link to your own work in the form for the best works if you believe that you have done an exceptional job.
• If you notice flaws in the project you are reviewing, don't just list them. Give the author some advice on how these flaws could have been avoided and how to correct the identified errors.
• If a student who reviewed your work has provided detailed feedback with suggestions for improving your application and the review was done effectively, benefiting you, express your gratitude through the RS app. Note that gratitude should be expressed not only for high grades but also for valuable comments and advice.
• Share the most useful comments received during cross-checking in the Discord channel #crosscheck (ensure to hide the reviewer's contact information).

Principles of evaluating work in cross-check assessment

• If you believe that a criterion has been fully met, assign 100% of the points.
• If there are significant deviations that are clearly not intentional features, assign 50% of the points.
• If a criterion is not fulfilled at all, assign 0 points.
• If a criterion represents an indivisible action, either assign 100% of the points or 0 points.
• In the presence of non-critical inaccuracies, the evaluation should be in favor of the student.
• If you do not assign the maximum possible number of points to the work being reviewed, please:
  • Provide a detailed comment explaining why you have done so and indicate the score you have assigned.
  • Provide a way to contact you through Discord, Telegram, etc., so that the student has a chance to explain their decision and the opportunity to change your evaluation if you missed or didn't noticed something.
  • You can always submit a new evaluation before the assessment deadline, with a comment explaining why you have changed your decision.
• If you suspect that the the student cheated, you should:
  • Inform the course moderator or RSS AI via private message.
  • Evaluate the work according to all requirements, regardless of suspicion.

Example 1: make up a prototype that behaves correctly at any resolution

• layout of the application prototype adaptive or responsive - 20 points
• if the layout behaves incorrectly at a resolution of 320px-1440px (in case of adaptive layout, part of the data may be hidden at a resolution less than desktop) - 10 points
• layout of the prototype application is not adaptive and not responsive - 0 points

Example 2: search for weather data by geographic location.

• implemented search - 60 points
• not updated map || forecast for today || 3 day forecast || coordinates - 30 points
• search is not implemented - 0 points

Example 3: implement the function of changing the background image

• when refreshing the page, the background image changes - 10 points
• when refreshing the page the background image does not change - 0 points

In the presence of uncritical inaccuracies - the assessment is put in favor of the student.

Dear students, remember to maintain a respectful attitude when communicating. Cases of inappropriate communication in private messages or chats (aggression, profanity, insults, personal attacks, sexism, etc.) will be strictly dealt with by the administration, based on screenshots or other evidence, up to the complete removal of offenders from RS School (regardless of the fairness of their work assessment).

Responsibility for poor conduct of cross-check assessment

Poor conduct of assessment includes:

• Undervaluing points without explanatory comments.
• Undervaluing points with comments that do not describe specific deductions.
• Undervaluing points with comments based on reasons not evaluated in the assignment.
• Lack of assessment while assigning a score.
• Overvaluing points.

Admins have the ability to review the evaluations given during cross-check assessments (admins can see the GitHub profiles of anonymous reviewers). Cross-checkers who conduct poor assessments will face the following consequences:

• First offense: Warning in the #moderation channel and zeroing of points for the task.
• Second offense: Disqualification.

Use of special assessment forms

To facilitate the assessment of all necessary criteria and the calculation of the final score, special forms are available for each task. These forms allow you to:

• Obtain an evaluation of the work by indicating the completeness of fulfilling the corresponding requirements.
• Generate and receive a comprehensive comment on the work, including:
  • Overall score.
  • List of requirements not fully met (with comments for each, if applicable).
  • List of partially met requirements (with comments for each, if applicable).
  • List of fully met requirements (with comments for each, if applicable).

Rules of communication in the Discord channel #cross-check

If you believe that the evaluation given by the reviewer is incorrect and want to confirm it, you can do so in the #cross-check channel.

Your message should include:

• The link to your work.
• Self-assessment on the disputed points.
• Evaluation given by the reviewer and their feedback.

By doing this, your fellow participants in the chat can help you identify any errors if they truly exist or confirm your point of view.

Messages in the #cross-check channel should be focused on discussing the works, specifically the implementation details of the tasks, rather than discussing the reviewers or those being reviewed.

Appeal

the appeal is only possible if:

• there are activists who are willing to drive the process.
• you haven't made any changes to your work after the deadline.
• all the reviews of your work have been completed and step #4 (see above) is finished.
• the expected grade and the average grade obtained in step #4 differ by 10% or more from the maximum grade for the assignment.

The grade can be decreased or increased without further discussion.

To file an appeal:

1. Create a new issue https://github.com/rolling-scopes-school/support/issues.

2. Name it according to the template: Cross-Check 'task name' - 'Your name on Github'

3. Fill in the issue according to the following template (if the issue is named or filled out incorrectly, it will not be processed):

   • Link to your project's deployment
   • Link to the repository with the project on GitHub.
   • Link to the task.
   • Screenshot of the cross-check grades (the reviewers' contact information should be hidden).
   • Evaluation as a result of self-examination, with its argumentation.
   • Score for cross-check from your Score.

Appeal review process:

• The process starts after the cross-check deadline.
• Valid issues are marked as "help wanted" (Labels > "help wanted").
• A volunteer activist chooses an issue from https://github.com/rolling-scopes-school/support/issues.
• The activist adds the "review started" label to the chosen issue (Labels > "review started").
• The activist reviews the work according to the criteria specified in the assignment. They leave their comments and grade. Everything is visible and transparent.
• The activist adds the "review completed" label (Labels > "review completed").
• The activist mentions the trainer who assigned the task in the issue comment.
• The trainer submits the grade in the RS APP.
• The grade replaces the current grade in the score.

Complaint

• It will be considered only if the administrators and RSS-AI have free time.
• It can only be submitted after the cross-check.
• It is used only to report unfair reviewers.
• Before filing a complaint, it is recommended to contact the reviewer and attempt to discuss the results of the review, providing arguments for your position.
• A complaint cannot be withdrawn.
• The student who filed the complaint will not be notified of the outcome of the complaint review.
• If the complaint is approved, the student who was complained about will receive a warning. Subsequent warnings may result in disqualification.
• Even after a complaint is satisfied, the grade received from the cross-check remains unchanged.

To file a complaint:

• Wait for the cross-check to finish.
• Fill out the complaint form, providing the following information:
  • Your exact GitHub username (not a link).
  • Task name.
  • Score given by the reviewer.
  • Your comment regarding the complaint.

Complaints submitted before the end of the cross-check or those filled out incorrectly will not be considered.

Why cross-check?

Advantages

• Checking all the student's works within the specified deadlines.
• Understanding exactly how many students completed the task (100% comprehension).
• Approximately 95% of students review the works carefully. Responsible students provide valuable feedback based on the review's results.
• Excellent way to review tasks related to web development.
• Cross-check allows comparing one's own work with others, understanding one's level compared to others.
• Opportunity to see how different individuals approach the same task.
• Evaluating works teaches students to thoroughly read the task requirements.
• Students learn to review works, which is especially relevant as many of them will return to RS School as mentors.
• Opportunity to view the application from a user's perspective and learn something new about one's own work, such as untested behavior.
• Paying closer attention to the task requirements when knowing that another student will review the work, fostering a more responsible approach to its implementation.
• Practicing application testing skills that can be applied for debugging one's own projects in the future.

Disadvantages

• Evaluation of works is not always objective.
• Lack of opportunity to communicate with the reviewing student and discuss disputed aspects.
• Not all students approach the review responsibly. Some provide only a few lines of feedback, while others provide extensive comments.
• Reviewing takes a significant amount of time.
• Clear evaluation criteria are needed.

Objectives of the second stage of training

• Students gaining Core Js knowledge
• Preparing students for technical interviews
• English student submission
• Students completing a course project