Metadata-Version: 2.3
Name: cr_proc
Version: 0.3.0
Summary: A tool for processing BYU CS code recording files.
Author: Ethan Dye
Author-email: Ethan Dye <mrtops03@gmail.com>
Requires-Python: >=3.11
Description-Content-Type: text/markdown

# `code_recorder_processor`

`code_recorder_processor` processes `*.recording.jsonl.gz` files produced by
the current `jetbrains-recorder` and `vscode-recorder` implementations. It
reconstructs the edited document, compares that reconstruction to a template,
and reports suspicious activity (large external pastes, rapid AI-style paste
bursts, time-limit violations), transcription-style typing (hand-typing that
looks like *reproducing* existing text rather than composing it), and engagement
metrics (active editing time, time away from the editor, and typed-vs-pasted
character counts) to help TAs and instructors review student work. An
approved-pastes allowlist lets you exempt code students are expected to paste. See
[Reading the Output](#reading-the-output-for-tas-and-instructors) for an
annotated example of what a run prints.

## Scope

The processor is designed around the current recorder implementations, not
around the historical examples in this repository.

Current schema expectations:

- Modern edit events use `type: "edit"`.
- Status events use typed records such as `type: "focusStatus"`.
- Events include `timestamp`, `document`, `offset`, `oldFragment`, and
  `newFragment`.
- Newer recorders also include `recorderVersion`.

Compatibility behavior:

- Older recordings that omit `type` on edit events are still accepted.
- If a mixed recording contains both modern typed edits and later stale legacy
  untyped edits, the processor prefers the typed stream.
- Example recordings in `recordings/` are fixtures, not the schema source of
  truth.

The canonical schema reference for this repository is
[`docs/recording-schema.md`](docs/recording-schema.md).

## Installation

For development inside this repository:

```bash
uv sync --dev
```

For running commands in the repo without a global install, prefer:

```bash
uv run cr_proc --help
```

To install the CLI globally from a local checkout:

```bash
uv tool install .
```

After that, the `cr_proc` command is available directly:

```bash
cr_proc --help
```

If you want the global command to track local source changes while developing:

```bash
uv tool install --editable .
```

## Quick Start

The simplest invocation is to pass only recordings. When `--template` is
omitted, the processor looks for a matching template file next to each
recording.

Single recording:

```bash
uv run cr_proc path/to/student.recording.jsonl.gz
```

Multiple recordings:

```bash
uv run cr_proc recordings/*.recording.jsonl.gz
```

Explicit template file:

```bash
uv run cr_proc student.recording.jsonl.gz --template template.py
```

Template directory:

```bash
uv run cr_proc recordings/*.recording.jsonl.gz --template templates/
```

Write reconstructed output:

```bash
uv run cr_proc student.recording.jsonl.gz --write reconstructed.py
uv run cr_proc recordings/*.recording.jsonl.gz --write output/
```

Compare to submitted files:

```bash
uv run cr_proc student.recording.jsonl.gz --submitted submitted.py
uv run cr_proc recordings/*.recording.jsonl.gz --submitted submissions/
```

Write JSON results:

```bash
uv run cr_proc recordings/*.recording.jsonl.gz --output-json results.json
```

Playback mode:

```bash
uv run cr_proc student.recording.jsonl.gz --playback
```

This opens a windowed viewer. Use the left/right arrow keys to step through
edits, `Space` to play or pause, and `Home`/`End` to jump to the beginning or
final state. The viewer is generated as a local HTML page and opened in your
default browser.

Select a specific document from a multi-document recording:

```bash
uv run cr_proc multi-file.recording.jsonl.gz --document src/main.py
```

## CLI Reference

Core inputs:

- `inputs`: One or more recording files or glob patterns.
- `--template PATH`: Optional template file or template directory.
- `--document NAME`: Optional override for which document inside the recording
  should be processed. This matches the recorded document path or filename and
  is not another local file input.

Outputs:

- `--write PATH`: Write reconstructed code. In single-file mode this can be a
  file or a directory. In batch mode it must be a directory.
- `--output-json PATH`: Write structured JSON results.
- `--submitted PATH`: Compare reconstructed code to a submitted file or a
  directory of submitted files.

Verification and filtering:

- `--time-limit MINUTES`: Flag recordings whose active editing time exceeds the
  limit.
- `--filter-file FILE`: Exclude recordings matching a path, filename, or base
  filename.
- `--filter-function-generation`: Suppress suspicious autocomplete findings
  that are recognized as IDE-generated boilerplate function stubs.
- `--approved-pastes FILE`: Treat the contents of FILE as approved code that
  students may legitimately paste (provided helpers, lecture/reading snippets).
  A flagged paste is suppressed when its content (whitespace-insensitive) is
  contained in the approved material. Repeatable; pass several files as needed.
  Suppressed pastes are removed from the suspicious list but reported as an
  audit count (`Approved pastes suppressed: N`, and `approved_paste_suppressions`
  in JSON output).
- `--transcription-threshold FRACTION`: Flag a document as transcription when at
  least this fraction of its typed content looks transcribed (default `0.20`).
  Raise it to surface only the most heavily-reproduced files; lower it to widen
  review. See [Transcription Detection](#transcription-detection).

Playback:

- `--playback`: Open a browser-based windowed playback viewer.
- `--playback-speed FLOAT`: Playback speed multiplier.
- `--playback-start-event N`: Start playback from a later applied-event index.

Compatibility aliases:

- Legacy positional-template usage still works.
- `--template-dir`, `--output-file`, `--output-dir`, `--submitted-file`, and
  `--submitted-dir` are still accepted as compatibility aliases.

## Template Resolution

When the processor needs a template, it resolves it in this order:

1. `--template <file>` uses that exact file.
2. `--template <directory>` searches that directory for the best filename or
   stem match to the recorded document.
3. If `--template` is omitted, the processor searches the recording's parent
   directory.
4. Legacy positional-template mode treats the last positional argument as a
   template file when it does not look like a recording path.

`--document` affects this process by telling the processor which recorded
document to treat as the target before template matching happens. It only
selects data already present in the recording.

If no matching template is found, processing still continues by falling back to
the recording snapshot as the reconstruction seed.

## Output Behavior

Normal user-facing output goes to `stderr`:

- time summaries
- character metrics (typed vs. pasted vs. deleted characters, and a typed
  fraction)
- time away from editor (window unfocused time)
- suspicious-event summaries
- transcription review (typed content that looks reproduced, with the localized
  line spans and the behavioral signals behind each)
- template mismatch diffs
- submitted-file comparison summaries
- warnings

Character metrics classify each inserted character as typed or pasted using a
width-based rule (single keystrokes, auto-indent whitespace, and IDE auto-paired
brackets/quotes count as typed; larger inserts count as pasted). The counts are
independent of the suspicious-event heuristics, so they stay stable as detectors
change. They appear per file and as a combined roll-up, and in JSON as
`char_metrics` / `combined_char_metrics`. The transcription analysis reuses this
same typed-vs-pasted rule, so its "typed content" means the same thing here as it
does for the typed fraction: pastes and IDE completions are excluded, and only
hand-typed keystrokes are assessed.

### Time measures (three distinct numbers)

The tool reports three independent time concepts; do not conflate them:

1. **Time span (first to last edit)** — raw wall-clock from the first to the last
   recorded event. Excludes nothing.
2. **Elapsed editing time** (`time_info.minutes_elapsed`) — the span minus the
   idle blocks where the **file was closed and reopened** (each reopen emits a
   file-open snapshot; the gap before it is dropped). It does *not* deduct idle
   time while the file stayed open, whether focused or switched away.
3. **Time away from editor** (`focus_info` / `combined_focus_info`) — total time
   the editor window was unfocused (blurred), computed by pairing window
   blur/focus events. The file stays open during these; this is "switched to
   something else" time. Reported as `unfocused_seconds` + `num_unfocused_intervals`.
   It is a standalone signal and does **not** affect measures 1 or 2.

**Known item flagged for review:** `--time-limit` currently compares the limit
against measure 2 (file-closed-deducted **editing time**), not measure 1 (the
wall-clock span). For timed in-class work the file stays open so the two are
~equal; on take-homes they diverge sharply. Whether the limit should gate on the
span instead is an open question — behavior is intentionally left unchanged for
now. See `check_time_limit` in `src/cr_proc/api/verify.py`.

Reconstructed code is written only when `--write` is used.

JSON output is written only when `--output-json` is used.

## Reading the Output (for TAs and instructors)

Every run prints a human-readable summary to `stderr`. Here is a real
single-recording run that was flagged:

```
Elapsed editing time: 15.742 minutes
Time span (first to last edit): 19.99 minutes
Characters: 127 typed, 2286 pasted, 272 deleted (typed fraction 5%)
Editors: jetbrains

Suspicious events detected:
  Events #52-#114 (AI rapid paste): 44 lines, 2140 chars
  Events #131-#136 (AI rapid paste): 3 lines, 146 chars
```

Line by line:

- **Elapsed editing time / Time span** — two of the three time measures (see
  [Time measures](#time-measures-three-distinct-numbers) above). Here the
  student was active for ~16 minutes within a ~20-minute window.
- **Characters: … (typed fraction 5%)** — only 5% of inserted characters were
  hand-typed; 2286 of 2413 arrived as pastes. A very low typed fraction is a
  strong, at-a-glance triage signal.
- **Editors** — which IDE recorder(s) produced the recording.
- **Suspicious events detected** — one line per flagged event: the event index
  (or index range for a cluster), the reason, and its size. `Events #52-#114
  (AI rapid paste): 44 lines` means 44 lines were pasted in rapid succession —
  the JetBrains AI-autocomplete acceptance pattern. The indices let you locate
  the event in `--playback` or the JSON.

**Transcription** is a different signal from pasting. Pasting flags content that
arrived all at once; transcription flags content the student *typed by hand* but
in a way that looks like reproducing an existing source rather than composing.
A student who retypes an answer character-by-character trips **no** paste flag
(their typed fraction may even be 100%) yet still shows up here. A flagged run
looks like this:

```
Characters: 1341 typed, 0 pasted, 12 deleted (typed fraction 100%)
Success! No suspicious events detected.

Transcription review: FLAGGED - 73% of typed content looks transcribed
  Lines 1-22 (532 chars, score 0.88): regular-cadence, no-deliberation, fast-steady
  Lines 22-31 (176 chars, score 0.83): regular-cadence, no-deliberation, fast-steady
  Lines 35-42 (165 chars, score 0.92): few-corrections, no-deliberation, fast-steady
  Note: this reflects typing behavior only - a TA confirms the source from playback.
```

How to read it:

- **`FLAGGED - 73% of typed content looks transcribed`** — 73% of the hand-typed
  characters fall inside runs that read as reproduction. High coverage is the
  triage signal; a genuine composition is usually near 0%.
- **Each `Lines A-B` span** is a place to jump to in `--playback`, with its size,
  a 0–1 confidence `score`, and the **behavioral signals** that fired there:
  `forward-linear` (text laid down top-to-bottom), `few-corrections`,
  `no-deliberation` (no thinking pauses), `regular-cadence` (metronomic keystroke
  timing), `fast-steady`, and `focus-correlated` (a fluent burst right after
  tabbing back — as if reading from another window). See
  [Transcription Detection](#transcription-detection) for the full glossary.
- **The Note is not boilerplate.** This detects *behavior*, not source. Typing
  from memory, reproducing a solution the class was taught, or copying one's own
  prior homework all look identical to copying a neighbor. **Open the playback and
  judge the source; the flag is only a pointer to where to look.**

Other lines you may see, depending on the recording and flags:

- `Time away from editor: 86.92 minutes across 54 switches` — the third time
  measure (window unfocused time); appears only when the recording contains
  window focus/blur events.
- `Approved pastes suppressed: 1 (events #80)` — a paste matched the
  `--approved-pastes` allowlist and was **not** flagged. Shown for transparency
  so you can confirm the allowlist is doing what you expect.
- `Success! No suspicious events detected.` — nothing was flagged.
- `Transcription review: nothing flagged (3% of typed content).` — the typing
  read as genuine composition; the percentage is the reproduction coverage.
- `Transcription review: not scoreable - the recording captured the editor page
  (web IDE), not the code file; span localization is unavailable.` — some web-IDE
  recordings capture the whole editor page rather than the `.py` file. These are
  excluded from the batch roll-up rather than scored on the wrong content.
- `Warning: …` — e.g. a template mismatch, a recovered initial state, or "the
  user likely has AI autocomplete enabled" (transient insert/delete suggestion
  previews were seen — a hint, not a flag).

A batch run (multiple recordings) ends with a combined summary:

```
BATCH SUMMARY: Processed 4 files
================================================================================
Verified: 4/4

COMBINED TIME REPORT (4 recordings):
Total elapsed editing time: 82.737 minutes
Overall time span: 18739.29 minutes
Combined characters: 4267 typed, 5144 pasted, 2441 deleted (typed fraction 45%)
Combined transcription: 1/4 documents flagged (18% of typed content)
```

The `Combined transcription` line rolls the per-document results up: how many
documents were flagged, and the pooled reproduction coverage across all
*scoreable* documents (unscoreable web-IDE recordings are excluded from the
pool but counted separately, e.g. `… (18% of typed content, 2 not scoreable)`).

### JSON output

`--output-json PATH` writes the same information in machine-readable form. Below
is a real single-recording result, abbreviated (the reconstructed source and one
larger cluster are trimmed):

```json
{
  "version": "0.3.0",
  "batch_mode": false,
  "total_files": 1,
  "verified_count": 1,
  "all_verified": true,
  "files": [
    {
      "jsonl_file": ".../cs111-homework0-CPT.recording.jsonl.gz",
      "document": "/.../homework0.py",
      "verified": true,
      "time_info": {
        "minutes_elapsed": 15.742,
        "first_timestamp": "2026-01-15T01:21:35.360168Z",
        "last_timestamp": "2026-01-15T01:41:34.626218Z",
        "exceeds_limit": false,
        "time_limit_minutes": null
      },
      "suspicious_events": [
        {
          "event_index": 131,
          "event_indices": [131, 135, 136],
          "line_count": 3,
          "char_count": 146,
          "reason": "AI rapid paste",
          "newFragment": "3 pasted lines in under one second"
        }
      ],
      "char_metrics": {
        "typed_chars": 127,
        "pasted_chars": 2286,
        "deleted_chars": 272,
        "typed_fraction": 0.0526
      },
      "transcription": {
        "flag": true,
        "score": 1.0,
        "transcribed_fraction": 0.732,
        "typed_chars": 1341,
        "transcribed_chars": 982,
        "unscoreable": false,
        "spans": [
          {
            "line_start": 1, "line_end": 22,
            "offset_start": 0, "offset_end": 532,
            "char_count": 532, "score": 0.88,
            "signals": {
              "regular_cadence": 0.9, "no_deliberation": 1.0,
              "fast_steady": 0.71, "forward_linear": 0.8,
              "few_corrections": 0.75, "focus_correlated": 0.0
            }
          }
        ]
      },
      "editors": ["jetbrains"],
      "reconstructed_code": "<full reconstructed source ...>"
    }
  ],
  "combined_char_metrics": {
    "typed_chars": 127, "pasted_chars": 2286,
    "deleted_chars": 272, "typed_fraction": 0.0526
  },
  "combined_transcription": {
    "documents": 1, "unscoreable": 0, "flagged": 1, "any_flagged": true,
    "typed_chars": 1341, "transcribed_chars": 982, "transcribed_fraction": 0.732
  }
}
```

Field reference:

- Top level: `version`, `batch_mode`, `total_files`, `verified_count`,
  `all_verified`, and the batch roll-ups `combined_time_info`,
  `combined_char_metrics`, `combined_focus_info`, `combined_transcription`,
  `total_approved_paste_suppressions` (each present when applicable).
- `files[]` per recording: `jsonl_file`, `document`, `verified`, `time_info`,
  `suspicious_events`, `template_diff`, `reconstructed_code`, `editors`, and —
  when present — `char_metrics`, `transcription`, `focus_info`,
  `approved_paste_suppressions`, `recorder_versions`, `warnings`,
  `submitted_comparison`.
- `time_info`: `minutes_elapsed` (the **editing time**, measure 2 above),
  `first_timestamp`/`last_timestamp` (whose difference is the **span**, measure
  1), `exceeds_limit`, `time_limit_minutes`.
- `char_metrics`: `typed_chars`, `pasted_chars`, `deleted_chars`,
  `typed_fraction` (0–1).
- `transcription`: `flag` (bool), `score` (0–1), `transcribed_fraction` (0–1,
  the coverage the flag is based on), `typed_chars`, `transcribed_chars`,
  `unscoreable` (bool), and `spans[]`. Each span has `line_start`/`line_end`,
  `offset_start`/`offset_end`, `char_count`, `score`, and `signals` (the 0–1
  behavioral signals). Absent when there is too little edit history to analyze.
- `combined_transcription`: `documents` (scoreable count), `unscoreable`,
  `flagged`, `any_flagged`, `typed_chars`, `transcribed_chars`,
  `transcribed_fraction` (pooled over scoreable documents; `null` if none typed).
- `focus_info`: `{ "unfocused_seconds": 5215.2, "num_unfocused_intervals": 54 }`
  — present only when the recording has focus events.
- `suspicious_events[]`: `event_index` (plus `event_indices` for a cluster),
  `line_count`, `char_count`, `reason` (see glossary below), and `newFragment`
  (or `diff` for a conflicting snapshot).
- `approved_paste_suppressions`: `{ "count": 1, "event_indices": [80] }` —
  present only when `--approved-pastes` suppressed something.

## Suspicious Activity Detection

Each suspicious event carries a `reason`. The labels you will see:

- **`external paste`** — a large block inserted at once (≥8 lines; or ≥6 lines
  and ≥250 chars; or ≥3 lines and ≥350 chars) that does not already appear in
  the document or the approved-pastes material. Typical of pasting from an
  external source. VS Code emits a paste as one such event.
- **`AI rapid paste`** — 3 or more multi-line inserts within one second,
  reported as a cluster (with its event index range). This is the JetBrains
  AI-autocomplete acceptance pattern (whole lines appearing in rapid
  succession). Note: this timing-based check fires regardless of whether the
  content is reused, so rapidly duplicating one's own code 3+ times within a
  second can still appear here.
- **`complete statement auto-complete (AI assistance)`** — a small (10–300 char)
  multi-line insert that suddenly completes a structure such as a `def`/`class`.
  Typical of an IDE/AI completing a whole statement at once.
- **`conflicting snapshot`** — a file-open snapshot whose contents do not match
  the replayed document state; reported with a `diff` instead of `newFragment`.

The processor also reports **time-limit violations** for single recordings and
combined batch activity (see [Time measures](#time-measures-three-distinct-numbers)).

With `-g/--filter-function-generation`, IDE-generated boilerplate is recognized
and suppressed rather than flagged: `def …: pass`/`...` stubs and
`if __name__ == "__main__":` skeletons (empty body or a lone entry-point call).

Reuse awareness suppresses the `external paste` and `complete statement
auto-complete` flags for content that already appeared earlier in the same
recording or an overlapping recording in the batch — including code that was
typed and later moved or re-pasted, even when it was typed and deleted
character-by-character. Moving one's own existing code is not flagged. (The
`AI rapid paste` timing check is the exception noted above.)

A separate **warning** — "the user likely has AI autocomplete enabled" — is
emitted when transient insert/delete suggestion previews are seen (a JetBrains
suggestion shown then withdrawn). This is context, not a flagged event.

These checks are heuristic. They are intended to surface recordings for review,
not to act as a standalone disciplinary decision engine.

## Transcription Detection

Paste detection catches content that *arrived* all at once. Transcription
detection catches content the student *typed by hand* but in a way that looks
like reproducing an existing source rather than composing. Because pasting is
handled separately (and is often legitimate), hand-**re**typing is the behavior
this surfaces — a student reading an answer off another screen and typing it in
leaves no paste trace but types with a tell-tale fluency.

**What it measures.** Within the hand-typed content only (pastes and IDE
completions are masked out, as for `typed_fraction`), a sliding window scores how
much each run of typing looks like reproduction. Contiguous flagged runs become
**spans** (line ranges you can jump to in playback). The file's
`transcribed_fraction` is the share of typed characters inside those spans;
`--transcription-threshold` (default `0.20`) is the coverage at which the file is
flagged. Genuine composition sits near zero; full retyping runs high.

**The behavioral signals** (each 0–1, shown per span):

- `few_corrections` — little backspacing/deleting; the text comes out right the
  first time.
- `forward_linear` — characters laid down front-to-back, not jumping around.
- `no_deliberation` — no thinking pauses between bursts.
- `regular_cadence` — metronomic inter-keystroke timing (low variation).
- `fast_steady` — sustained fast characters-per-second.
- `focus_correlated` — a fluent burst begins right after the window regains focus
  (tabbing back from another window, then typing). A strong "reading from
  elsewhere" tell; only available when the recording has focus events.

**`unscoreable`.** Some web-IDE recordings (e.g. the CS 110 "Bit" environment)
capture the whole editor **page** rather than the code file. Those are marked
`unscoreable`: still reported, but their span line numbers would reference page
chrome, so they are excluded from the batch roll-up rather than scored on the
wrong content.

**How to act on it — important.** This is a *behavioral* signal and is
deliberately source-agnostic: it sees *how* the code was entered, never *what* it
says or where it came from. High coverage is not proof of misconduct. All of the
following look identical to the detector and are common, legitimate causes:

- typing a solution from memory,
- reproducing a pattern the class was explicitly taught (convergent solutions —
  e.g. a standard file-reading or dictionary-inversion idiom),
- retyping the student's own earlier work.

Treat a flag as **a pointer to a span to watch in `--playback`**, then judge the
source yourself. The tool triages where to look; the human makes the call.

## Development

Run tests:

```bash
uv run pytest -q
```

Run the bundled example recording:

```bash
uv run cr_proc recordings/cs111-homework0/cs111-homework0-ISC.recording.jsonl.gz
```

## CI and Release

GitHub Actions uses `uv`, not Poetry.

- CI installs dependencies with `uv sync --locked --dev`.
- CI currently runs on Python `3.11` and `3.14`.
- The publish workflow builds distributions with `uv build`.

## Repository Fixtures

The bundled recordings are documented in
[`recordings/README.md`](recordings/README.md).
Those files are useful for regression tests and examples, but some were created
with older recorder versions and intentionally exercise compatibility paths.
