OSM Auto Evaluation (AI-assisted answer-sheet grading)

How to use OSM Auto Evaluation (AI-assisted answer-sheet grading)

OSM Auto Evaluation extends standard Onscreen Marking with AI: scanned answer sheets are OCR-processed, evaluated against a model answer using a domain-tuned LLM, and a draft score is presented to the faculty for review and override.

When to use Auto Evaluation

• Large-volume semester exams where manual evaluation is slow.
• Multiple-choice + descriptive answer sheets where MCQ can be fully auto-graded and descriptive responses get an AI draft score.
• Exams where consistency across many evaluators matters (the AI scores uniformly; human reviewers focus on edge cases).

Prerequisites

1. Booklets must be scanned and uploaded. See university result processing.
2. The exam must have an answer key / model answer uploaded per question.
3. Auto Evaluation must be enabled on your organization plan — contact help@eklavvya.com if the menu is not visible.

Step 1 — Enable Auto Evaluation on the exam

1. Log in as admin or COE.
2. Open the exam and go to OSM Settings → Auto Evaluation.
3. Toggle Enable AI Auto Evaluation.
4. Pick an evaluation mode:

   Mode	When to use
   Easy	Lenient — awards partial marks for partial understanding. Use for early-grade or skill-development assessments.
   Default	Balanced — matches typical examiner standards. Use this unless you have a specific reason to deviate.
   Moderate	Stricter adherence to the model answer with clear deductions. Use for university semester exams.
   Strict	Rigorous — full marks only for complete, accurate answers. Use for high-stakes certification exams.

5. Optional: turn on Identity Masking so faculty reviewers see anonymous booklets.
6. Save.

Step 2 — Run the AI evaluation

1. Go to OSM → Auto Evaluation Queue.
2. Click Process Booklets. The system OCRs each scanned page, extracts text, and scores each answer against the model answer.
3. Wait for processing — you can leave and come back. The queue shows progress per booklet.
4. When done, each booklet has a draft AI score per question and an overall total.

Step 3 — Faculty review and override

1. Examiners log in and open their assigned booklets from the OSM bucket as usual.
2. For each question, the faculty sees the candidate's answer, the AI's draft score, and the AI's reasoning ("Why this score").
3. Examiner clicks Accept (keep AI score), Override (enter their own mark), or Comment (add feedback without changing the score).
4. Annotations, marks, and any blank-page cross-marks work the same as standard OSM.
5. When every page is green, click Finish Paper Checking.

Step 4 — Moderation

For double-evaluated booklets the moderator workflow is unchanged — the moderator sees both examiners' final marks (whether AI-accepted or examiner-overridden) and consolidates. See moderation flow.

What the AI looks at

• Accuracy — how closely the answer matches the model.
• Keyword detection — presence of expected terms and concepts.
• Structure — logical flow, paragraph organization.
• Semantic relevance — meaning and context, not just keyword matching.
• Partial credit — awards proportional marks for incomplete but correct reasoning.

Multi-language

Auto Evaluation works for English, Hindi, Marathi, Tamil, Telugu, and other supported languages. The model answer language must match the candidate response language.

Common questions

How accurate is the AI compared to a human?
The AI is a draft scorer — it's intended to be reviewed by a human examiner. In Default mode, AI scores typically land within a small range of an experienced examiner's score; the examiner's role is to catch the edge cases where context, partial credit reasoning, or unique phrasing matters.

Can I run Auto Evaluation on only some booklets?
Yes. From the Auto Evaluation Queue, you can filter by exam, subject, or evaluator and process a subset.

What if a candidate disputes their AI score?
Same as standard OSM — assign the booklet to a re-evaluator. The original AI score, the examiner's override, and the re-evaluator's score are all in the audit trail.