# Deterministic scorer

Two small programs, `scorer.py` (Python 3, standard library only) and `scorer.js` (Node 18 or newer, no packages), that score an NGO's answers the way the site does and say why each verdict came out as it did. They give the same output for the same input, character for character.

Use them when you want the numbers without the web app: inside your own AI environment, in a spreadsheet pipeline, or on a laptop with no network.

In this pack the folders are `scorer/` and `data/` side by side, so from the pack's top folder the command is `python3 scorer/scorer.py --answers answers.json --data data > result.json`. Nothing is sent anywhere. The programs read two things, your answers file and the data folder, and print JSON. They do not call a model.

## Run it

```
python3 scorer.py --answers answers.json --data ../data > result.json
node scorer.js    --answers answers.json --data ../data > result.json
```

| Flag | Meaning |
|---|---|
| `--answers FILE` | Your answers (shape below). Required. |
| `--data FOLDER` | The `v1.0-ngo` data folder. Required. It holds the use-case catalogue, the scoring map and the question files. |
| `--out FILE` | Write to a file instead of printing. |
| `--compact` | One line, no indentation. |
| `--precise` | Keep full-precision numbers. The default rounds to three decimal places. |

## Input

The answers file uses the shape the site keeps in the browser's local storage. Keys are the storage names without the assessment id prefix (a 36-character id followed by a colon is stripped if present). Each value is either the JSON itself or a string holding JSON.

| Key | What it holds |
|---|---|
| `ngo_assessment_0`, `_1`, `_2`, `_4` | `{ "answers": { "<questionId>": "<answer>" } }` for Phases 0, 1, 2 and 4. |
| `irf_ngo_selected_problem_areas` | List of category codes, for example `["A","B"]`. Nothing is ranked without it. |
| `irf_p3_starred_usecases` | List of the use case ids the organisation starred. These are the ones that get a Phase 4 verdict. |
| `irf_p3_starred_priority_order` | Optional. Orders the starred list. |
| `irf_act0` | Optional. `{ "sector": "humanitarian" }`, one of `humanitarian`, `environmental`, `healthcare`, `microfinance`, `education`, `agriculture`, `human_rights`, `wash`, `community`. The Conversation asks the sector outright and keeps it here. It wins over the words found in the mission and organisation type answers when the sector minimums are applied. |
| `ngo.interagencyContext` | Optional. `central`, `partial`, `unsure` or `no`. |
| `ngo_evaluation_answers` | Optional. Answers to the detailed per-use-case evaluation: `{ "<ucId>": { "<questionId>": { "value": "...", "flagged": false } } }`. |
| `ngo_eval_results` | Optional. Results already computed by the site. Used in preference to re-scoring `ngo_evaluation_answers`. |

Multi-select answers are stored as one string with the separator `|~|` between options. A plain list also works.

A demo file such as `demo-ngo-example.json` (answers under `answers.phase0` to `phase4`, each entry `{questionId, answer}`) is also accepted. Selected categories and starred use cases are then taken from `selectedCategories` and `selectedUseCases` if the file has them, or from the use cases that have Part B answers. `inputNotes` in the output says what was assumed.

Answers must be the exact option text from the question files. The scorer does not interpret free text and does not guess at an answer the person did not give. A scoring signal on a Phase 0, Phase 1 or Tadamon question carries a `when` list in the scoring map: it rates 4 only when the answer (split on `|~|` for a pick-all question) shares a value with that list, and 0 otherwise. A few questions rate the answer from a table, from how many countries were chosen, or from the two staff ranges. No signal rates 4 for an answer of any kind. If an AI environment has filled the file from a conversation, checking that each answer was confirmed by the person is that environment's job.

## Output

```
{
  "scorerVersion": "1.0.0",
  "inputNotes": [ ... ],                  // what was assumed or missing in the input
  "context": { selectedCategories, starredUseCases, starredSource, interagencyContext,
               detectedSector, programme, organisation{ sizeBucket, staffCount, ... } },
  "tiers": { top10, highNeed, mediumNeed, lowNeed, minimal },   // lists of use case ids
  "useCases": [ ... ],
  "notEnoughInformation": [ ... ]
}
```

Each entry in `useCases`:

| Field | Meaning |
|---|---|
| `problemMagnitude` | The ranking score, 0 to 5: 60% need, 40% scale, plus a small bump when the interagency context applies. |
| `need`, `scale` | The two parts of the problem score, from the answers the scoring map links to this use case. |
| `blockchainFit` | How well a shared ledger suits the problem, 0 to 5. |
| `readiness` | Organisational readiness for this use case on the same scale, with `readinessSource` saying whether it came from answers or from the neutral default of 2.5. |
| `gates` | `fit`, `readiness` and `overall`, each `pass`, `borderline`, `fail` or `no-data`, with the threshold used. |
| `tier` | `high` (3.5 and above), `medium` (2.0), `low` (1.0), `minimal`, `not-enough-information` or `not-ranked`. |
| `rank`, `inTop10` | Position among use cases that have any signal, and whether it is in the top ten (at least two and at most four per selected category). |
| `evidence` | Up to five answers that moved the score most. |
| `floorApplied`, `floorReason` | Present when a sector or programme minimum raised the score. |
| `phase4` | The readiness verdict, for starred use cases only (below). |

`phase4`:

| Field | Meaning |
|---|---|
| `outcome` | `PROCEED`, `PROCEED_WITH_CONDITIONS`, `NOT_NOW`, `EXPLORE_ALTERNATIVES` or `NOT_ASSESSED`. |
| `rule` | `{ id, sentence, inputs }`. The sentence is the one rule that produced the verdict, in plain English, for example "Only your organisation is involved, so a shared ledger adds little." `inputs` holds the numbers behind it. |
| `dimensionScores`, `rawDimensionScores`, `adjustments` | The five readiness dimensions (governance, financial, operational, technical, data) before and after context adjustments. |
| `thresholds`, `importance` | What this use case needs in each dimension, and how much each dimension counts. |
| `complexity` | The three complexity factors for this use case, adjusted for the organisation. |
| `partB`, `p2Severity` | The use-case questions as answered, and the severity rating of the deep-dive answers for the category. |
| `conditions` | What to address, with level `significant`, `moderate` or `minor`. |
| `steppingStone` | A simpler use case in the same category worth doing first, when one exists. |

`notEnoughInformation` lists every use case the scorer could not judge, with a `stage` (`ranking` or `readiness`) and a `reason`. Nothing is dropped silently. A use case with no answered question behind its score is listed here, and it is kept out of `rank`. A use case that only reaches a score through a sector or programme minimum carries a note saying so.

## What it does not do

- It does not produce a brief, a tender or any document. It produces numbers and reasons.
- It scores Phases 0 to 4. Later phases (matching, design, pilot, implementation) are not scored.
- The detailed evaluation counts only if its answers are in the file.
- It does not ask questions, check that an answer was given by a person, or read free text.
- It does not decide for the organisation. A verdict is an input to a conversation with the people who know the work.
- It uses the data folder it is given. Run it against the same folder version the answers were collected with.

## Where it differs from the site

The scorer follows the site's engine rule for rule. The behaviours below are reproduced as they are, because changing them would change what organisations already saw, and each needs a decision from the framework owner:

1. The partner count for the agility score reads the partnership model question first, so it is always zero. The band "Over 50" would also parse as zero.
2. The compliance-staff flag reads the governance question (`NGO-P0-S2-Q9`), so only the IT-unit answer can set it.
3. Skipping the questions about the organisation gives "Proceed with conditions" as the verdict code. The scorer keeps the code but the sentence says readiness could not be judged, and the use case is also listed under `notEnoughInformation`.

One engine fault was fixed in the site code rather than copied. `getP2SeverityForCategory` in `site/js/modules/ngo-p0p1p2.js` used to run `parseInt` on the answer text, so only 46 of 742 deep-dive option strings could ever be rated. It now reads each question's own severity mapping, and Phase 4 loads the deep-dive questions so the mapping is there on a direct page load.

## Checking it

In the source repository, `pack/scorer/test-parity.py` runs the site's own engine modules under Node and compares them with `scorer.py` over the two demo files and 50 generated profiles, then compares `scorer.py` with `scorer.js`. That test is not part of this pack. The pack's own test runs both scripts on `templates/answers-example.json` and checks that they print the same output.