From 35b58755096e579e5ec4b04f7dbf290fdf47c460 Mon Sep 17 00:00:00 2001 From: Bob Date: Fri, 13 Feb 2026 08:42:11 +0100 Subject: [PATCH 1/4] docs: tiny transformer implementation plan for scam/spam classifier MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Targets existing classifier: scam, clean, topic_crypto (multi-label, ~4k samples) - Teacher: cardiffnlp/twitter-roberta-large-2022-154m (primary), vinai/bertweet-large (alt) - Ensemble strategy: 3 seeds at 4k, single teacher at 100k-1M scale - Dual-head: softmax (scam/clean) + sigmoid (topics) - Distillation: logit-based, T=2-4, intermediate-layer matching, DAPT - Student: 4-layer BERT, hidden 192, 4 heads, int8 ONNX ≤5MB - Daily log entry for 2026-02-13 --- docs/logs/2026-02-13.md | 36 +++ docs/plans/tiny-transformer-plan.md | 360 ++++++++++++++++++++++++++++ 2 files changed, 396 insertions(+) create mode 100644 docs/logs/2026-02-13.md create mode 100644 docs/plans/tiny-transformer-plan.md diff --git a/docs/logs/2026-02-13.md b/docs/logs/2026-02-13.md new file mode 100644 index 0000000..a811ed1 --- /dev/null +++ b/docs/logs/2026-02-13.md @@ -0,0 +1,36 @@ +--- +title: Daily Log 2026-02-13 +author: bob +date: 2026-02-13 +tz: Europe/Berlin +created: 2026-02-13T08:26:22+01:00 +last_section: 2026-02-13T09:08:18+01:00 +updated: 2026-02-13T09:09:50+01:00 +--- + +## 08:26 + +Planning tiny-transformer implementation doc for janitr. Gathered current context from scripts/prepare_data.py, scripts/prepare_hf.py, scripts/evaluate.py, extension/offscreen.js, extension/background.js, and data/replies\*.jsonl. Key constraint observed: only 14 ground-truth ai_generated_reply rows currently in data/replies.jsonl, so plan will include data expansion + teacher distillation before student deployment. + +Created docs/plans/tiny-transformer-plan.md with a concrete implementation plan for the tiny-transformer option. + +Included: + +- explicit student architecture (4 layers, hidden 192, 4 heads, seq len 96) +- tokenizer strategy (custom WordPiece 8k) +- data-prep workflow grounded on data/replies.jsonl, data/replies_inferred.jsonl, data/sample.jsonl +- teacher training + distillation + quantization pipeline +- ONNX export acceptance criteria +- extension integration points (background/offscreen/content-script/manifest) +- offline + runtime evaluation gates +- model/bundle size estimates + +Ran `npx @simpledoc/simpledoc check` after writing; it failed only on filename convention and suggested renaming to a date-prefixed filename. Kept the user-requested path `docs/plans/tiny-transformer-plan.md` intentionally. + +## 09:08 + +Reviewed current tiny-transformer plan and baseline Janitr fastText pipeline. Preparing full rewrite of docs/plans/tiny-transformer-plan.md to target existing scam/topic_crypto/clean classifier, incorporate teacher-model research, and remove all ai_reply references. + +Rewrote docs/plans/tiny-transformer-plan.md to target Janitr's existing scam/topic_crypto/clean classifier instead of ai_reply. Updated dataset baseline counts, teacher model choices (TimeLM primary, BERTweet alternative), distillation strategy (calibration, T=2-4, alpha schedule, hidden-state matching), DAPT/logit-caching plan, extension integration references, script names, evaluation gates, and added updated timestamp. + +Ran npx -y @simpledoc/simpledoc check after rewrite. Check failed because docs/plans/tiny-transformer-plan.md is not date-prefixed per current SimpleDoc naming convention; migration would rename file and update docs/logs references. Kept filename unchanged this turn because user requested direct rewrite of that exact path. diff --git a/docs/plans/tiny-transformer-plan.md b/docs/plans/tiny-transformer-plan.md new file mode 100644 index 0000000..95ac494 --- /dev/null +++ b/docs/plans/tiny-transformer-plan.md @@ -0,0 +1,360 @@ +--- +date: 2026-02-13 +author: Bob +title: Tiny Transformer Implementation Plan +tags: [janitr, scam_classifier, transformer, onnx, extension] +updated: 2026-02-13T08:08:20Z +--- + +2026-02-13T08:08:20Z + +# Tiny Transformer Option Plan (Janitr) + +## Goal + +Add an optional tiny-transformer inference path for Janitr's existing classification task (`scam`, `topic_crypto`, `clean`) in the browser extension, while keeping the current fastText path as default. + +Target outcomes: + +- Keep false positives low (`FPR <= 2%` on holdout). +- Keep local inference practical in extension runtime. +- Keep model artifacts small enough for extension delivery. + +## Current Repo Baseline (Grounding) + +- Existing production path is fastText in browser offscreen flow: + - `extension/background.js` + - `extension/offscreen.js` + - `extension/fasttext/scam-detector.js` +- Current training scripts are fastText-focused: + - `scripts/prepare_data.py` + - `scripts/train_fasttext.py` + - `scripts/evaluate.py` +- Current JSONL dataset snapshot for transformer work: + - `data/train.jsonl`: `3421` rows + - `data/valid.jsonl`: `428` rows + - `data/holdout.jsonl`: `214` rows + - `data/sample.jsonl`: `4281` rows total +- Current training labels are multi-label at source and consolidated for production into: + - `scam` (`783` in train) + - `clean` (`1621` in train) + - `topic_crypto` (`1636` in train) +- Expected data growth: `~4k` now to `100k-1M` over time. + +## Target Architecture + +### Student model (deployed) + +Use a compact BERT-style encoder fine-tuned for Janitr's existing task. + +- Layers: `4` +- Hidden size: `192` +- Attention heads: `4` +- FFN/intermediate size: `768` +- Max sequence length: `96` +- Dropout: `0.1` +- Tokenizer: WordPiece vocab `8192` +- Inference heads: + - Head A: 2-class softmax (`scam` vs `clean`) + - Head B: sigmoid topic head (`topic_crypto`, future topic labels) +- Final class decision for current production surface: + - If `scam` exceeds threshold -> `scam` + - Else if `topic_crypto` exceeds threshold -> `topic_crypto` + - Else -> `clean` + +Rationale: + +- Fits short X post text and local browser constraints. +- Keeps parameter count in ~3-4M range, suitable for int8 ONNX deployment. +- Preserves compatibility with current Janitr collapse strategy while allowing topic-head expansion later. + +### Teacher model (train-time only) + +Use a larger tweet-native teacher for distillation and pseudo-labeling. + +- Primary teacher: `cardiffnlp/twitter-roberta-large-2022-154m` (TimeLM 2022 Large). + - Tweet-native pretraining on 2018-2022 data. + - Fits Jetson-class training hardware with fp16 + gradient checkpointing. +- Alternative teacher: `vinai/bertweet-large`. + - Strong TweetEval history and large tweet corpus. + - Slightly older temporal coverage (through 2019). +- Multilingual option: `jhu-clsp/bernice` (skip unless multilingual support becomes a requirement). +- Teacher is train-time only and never shipped in extension. + +Teacher scaling strategy: + +- At current `~4k`: train 3 seeds of the same teacher and average logits for soft targets. +- At `100k-1M`: use a single teacher; ensemble no longer required for stability/cost. + +## Tokenizer Choice + +Use a custom WordPiece tokenizer trained on repo data. + +- Vocabulary size: `8192` +- Normalization: NFKC + lowercase + zero-width cleanup (match behavior in `scripts/prepare_data.py`) +- Special tokens: `[PAD] [UNK] [CLS] [SEP] [MASK]` +- Reason: 8k vocab keeps embedding table small while still covering X slang/handles/URLs. + +Tokenizer artifacts to ship with model: + +- `tokenizer.json` +- `special_tokens_map.json` +- `tokenizer_config.json` +- `vocab.txt` (if needed by runtime tooling) + +## Data Preparation Plan (Janitr labels and splits) + +### 1) Build transformer prep script aligned with current datasets + +Add `scripts/prepare_transformer_data.py` to produce model-ready JSONL from: + +- `data/train.jsonl` +- `data/valid.jsonl` +- `data/holdout.jsonl` + +Preparation rules: + +- Preserve raw multi-label payload in output metadata. +- Generate collapsed teacher/student targets: + - `y_scam_clean` for Head A (`scam` vs `clean`) + - `y_topics` multi-hot vector for Head B (`topic_crypto`, optional future labels) +- Reuse normalization behavior from `scripts/prepare_data.py` (NFKC, zero-width cleanup, lowercase defaults). +- Keep holdout untouched (no pseudo-labeling, no oversampling). + +### 2) Optional unlabeled corpus for DAPT + +Create `scripts/build_unlabeled_corpus.py` to build MLM text for domain-adaptive pretraining (DAPT): + +- Source from unlabeled X snapshots over the same pipeline as Janitr collection. +- Deduplicate and clean noisy boilerplate. +- Target initial corpus size: `50k-500k` texts; expand with dataset growth. + +### 3) Split strategy + +- Keep provided `train/valid/holdout` files as primary split source. +- For regenerated splits at scale, use time-aware partitioning and reduce handle leakage. +- Keep soft-label generation confined to train/valid; holdout remains hard-label only. + +### 4) Data scale gates + +- At `~4k`, allow transformer experiments but treat results as pre-scale. +- Promote to default-candidate training track once `>=100k` labeled rows are available. +- At `>=100k`, enable logit caching pipeline by default. + +## Training Pipeline + +### Phase A: DAPT (optional but recommended) + +Add `scripts/train_teacher_dapt.py`: + +1. Continue pretraining the chosen teacher with MLM on unlabeled tweet corpus. +2. Use short runs first (for example 10k-50k steps) and evaluate transfer gains. +3. Save adapted checkpoint under `models/teacher_dapt/`. + +Output: + +- `models/teacher_dapt/` + +### Phase B: Teacher supervised training + calibration + +Add `scripts/train_transformer_teacher.py`: + +1. Fine-tune primary teacher (`cardiffnlp/twitter-roberta-large-2022-154m`) on Janitr train/valid. +2. Train Head A + Head B jointly with class-weighted objective. +3. At `~4k`, run 3 seeds and retain best checkpoints for ensemble logits. +4. Save predictions on valid/holdout for analysis. + +Add `scripts/calibrate_teacher.py`: + +1. Perform temperature scaling per label/head on validation split. +2. Save calibrated temperatures for soft-label generation. +3. Verify calibration with ECE/Brier diagnostics per label. + +Output: + +- `models/teacher/` +- `models/teacher_calibration.json` +- `models/teacher_valid_preds.jsonl` + +### Phase C: Distillation to tiny student + +Add `scripts/train_transformer_student_distill.py`: + +- Initialize student with target architecture (4L/192H/4 heads). +- Train with mixed objective: + - Hard-label loss + soft-label KL distillation. + - Distillation temperature `T=2-4`. + - `alpha=0.5` at `~4k` (hard/soft balanced), moving toward `alpha=0.8` at `100k+`. +- Add intermediate-layer distillation: + - Project teacher hidden states to 192 via trainable projection. + - Apply L2 matching loss on aligned intermediate layers. +- Use calibrated teacher outputs only (never raw logits) for soft labels. + +At scale (`100k-1M`): + +- Precompute and cache teacher logits to float16 with `scripts/cache_teacher_logits.py`. +- Train student from cached logits to reduce teacher forward-pass cost. + +Output: + +- `models/student/` +- `models/student_eval.json` + +### Phase D: Quantization + +Add `scripts/quantize_transformer_student.py`: + +1. Export ONNX fp32 first. +2. Run dynamic int8 quantization. +3. Optionally run static int8 quantization with calibration set if dynamic int8 hurts FPR. + +Keep both artifacts: + +- `models/student.onnx` (fp32 baseline) +- `models/student.int8.onnx` (extension deployment candidate) + +## ONNX Export Plan + +Add `scripts/export_transformer_student_onnx.py` (or use Optimum CLI through wrapper script). + +Implementation target: + +- Opset: `17` +- Inputs: `input_ids`, `attention_mask` +- Dynamic axes for batch size +- Validate parity (`torch` vs `onnxruntime`) on at least 1k examples + +Acceptance criteria for export: + +- Mean absolute probability delta <= `0.01` +- Label agreement >= `99%` on validation slice + +## Browser Extension Integration Plan + +Keep fastText path unchanged; add transformer as optional mode. + +### 1) New extension assets + +Add under `extension/transformer/`: + +- `student.int8.onnx` +- tokenizer files (`tokenizer.json`, etc.) +- `scam-detector-transformer.js` (ONNX runtime wrapper) + +### 2) Offscreen inference integration + +Update: + +- `extension/offscreen.js` to dispatch by model mode: + - existing `ic-infer-offscreen` fastText path + - new transformer path (for example `ic-infer-offscreen` with `engine: "transformer"`) +- `extension/background.js` to forward engine selection and preserve backward-compatible messaging + +### 3) Content script integration + +Update `extension/content-script.js`: + +- Add optional transformer detector pass for existing Janitr classes. +- Keep existing highlight behavior and thresholds semantics. +- Start with feature flag (`const ENABLE_TINY_TRANSFORMER = false`) and flip only after eval gates pass. + +### 4) Manifest and resources + +Update `extension/manifest.json` `web_accessible_resources` for transformer files and runtime assets. + +### 5) Runtime dependency + +Vendor ONNX Runtime Web assets into `extension/vendor/` (similar to existing fastText vendoring) to avoid remote fetches. + +## Evaluation Strategy + +### Offline metrics (required) + +Add `scripts/evaluate_transformer.py` to report: + +- Per-class Precision / Recall / F1 for `scam`, `topic_crypto`, `clean` +- Per-class FPR/FNR and support +- Exact-match accuracy +- Micro and macro Precision/Recall/F1 +- PR-AUC for `scam` +- Calibration table (confidence bins) +- Metrics by subgroup: + - short posts (< 40 chars) + - with URL vs without URL + - seen vs unseen handles + +Primary release gates: + +- `scam` FPR on holdout `<= 0.02` +- `scam` precision on holdout `>= 0.90` +- `scam` recall on holdout no worse than current fastText baseline by more than `2` points +- `topic_crypto` F1 non-regression vs fastText baseline + +Threshold policy: + +- Tune per-label thresholds with `scripts/tune_thresholds_fpr.py`. +- Store transformer thresholds under `config/thresholds.transformer.json`. +- Block promotion if thresholds cannot satisfy `scam` FPR gate. + +### Browser/runtime metrics (required) + +Add extension smoke benchmark similar to `extension/tests/wasm-smoke.spec.ts`: + +- P50 latency <= `25 ms` per tweet (desktop) +- P95 latency <= `60 ms` +- No memory growth across 1,000 inferences in offscreen worker + +### Shadow rollout plan + +1. Ship behind feature flag, score-only mode (no UI action). +2. Compare transformer output vs existing fastText predictions on identical tweet batches. +3. Promote to warn/highlight mode only after 2 consecutive holdout runs pass gates. + +## Estimated Model and Bundle Size + +Estimated for student (4L, 192H, 4 heads, vocab 8k, seq 96): + +- FP32 ONNX: ~`13-16 MB` +- INT8 ONNX: ~`3.3-4.2 MB` +- Tokenizer artifacts: ~`0.3-0.6 MB` +- Total model payload in extension: ~`3.6-4.8 MB` + +Expected extension impact including ORT runtime assets: + +- Additional shipped size: ~`4.8-6.2 MB` + +This fits Janitr's stated browser model target range when using int8. + +## Implementation Checklist (Executable) + +### Milestone 1: Data + training scripts + +- Create `scripts/prepare_transformer_data.py` +- Create `scripts/build_unlabeled_corpus.py` +- Create `scripts/train_teacher_dapt.py` +- Create `scripts/train_transformer_teacher.py` +- Create `scripts/calibrate_teacher.py` +- Create `scripts/train_transformer_student_distill.py` +- Create `scripts/cache_teacher_logits.py` +- Create `scripts/export_transformer_student_onnx.py` +- Create `scripts/quantize_transformer_student.py` +- Create `scripts/evaluate_transformer.py` + +### Milestone 2: Model artifacts + +- Train/calibrate teacher, generate soft labels +- Distill student with hidden-state matching +- Export + quantize ONNX +- Record metrics report under `docs/reports/` + +### Milestone 3: Extension wiring + +- Add `extension/transformer/scam-detector-transformer.js` +- Wire `background.js` + `offscreen.js` + `content-script.js` +- Add smoke/latency test in `extension/tests/` + +### Milestone 4: Release decision + +- Run full offline + extension benchmarks against current fastText baseline +- Confirm scam FPR gate and latency gate +- Enable feature flag by default only after both gates pass From c69e081120f9b39418d65c7425056c09267c5b62 Mon Sep 17 00:00:00 2001 From: Bob Date: Fri, 13 Feb 2026 09:16:01 +0100 Subject: [PATCH 2/4] chore: fill in simpledoc skill with actual spec and preferences --- skills/simpledoc/SKILL.md | 126 +++++++++++++++----------------------- 1 file changed, 49 insertions(+), 77 deletions(-) diff --git a/skills/simpledoc/SKILL.md b/skills/simpledoc/SKILL.md index 5cc10e7..d68de70 100644 --- a/skills/simpledoc/SKILL.md +++ b/skills/simpledoc/SKILL.md @@ -1,23 +1,10 @@ --- name: simpledoc -description: Create or update documentation in this repo following SimpleDoc conventions. +description: Create or update documentation in this repo following SimpleDoc conventions. Use when creating docs, plans, logs, or any markdown files. --- # SimpleDoc Documentation Skill -Use this skill whenever you are asked to create or update documentation in this repository. - -## MUST DO SETUP - -**Attention agent!** This section is a template for creating instructions on how to create documentation in this repository. The placeholder sections must be filled in before anything else. - -1. Fetch the most recent SimpleDoc specification from https://github.com/osolmaz/SimpleDoc and copy it below, replacing the placeholder content. -2. Ask the user about their preferences in documentation style (tone, technicality, planning style, etc.). Do a few rounds of back-and-forth to get the user's preferences, then replace the placeholder content in the Preferences section. - -After finishing the setup, DELETE ONLY this section between dividers `---`. The other agent instruction below is to be kept as is. - ---- - **Attention agent!** Complete every item below before touching documentation work: 1. **Read this file in full for the current session.** No shortcuts. @@ -31,22 +18,48 @@ If the name and email are not available for some reason, ask the user to provide ## SimpleDoc Specification -``` - -``` +SimpleDoc defines two types of files: + +1. **Date-prefixed files**: SHOULD be used for most documents, e.g. `docs/2025-12-22-an-awesome-doc.md`. +2. **Capitalized files**: SHOULD be used for general documents that are not tied to a specific time, e.g. `README.md`. + +### 1. Date-prefixed files + +- MUST put date-prefixed files in a top level `docs/` folder, or a subfolder `docs//`. Subfolders MAY be nested indefinitely. +- MUST use ISO 8601 date prefixes (`YYYY-MM-DD`) — the date MUST contain dashes. +- After the date prefix, lowercase filenames SHOULD use dashes (`-`) as word delimiters (kebab-case). Avoid spaces and underscores. +- The date prefix MAY be the entire filename (for example, daily logs like `docs/logs/2026-02-04.md`). +- MUST NOT use capital letters in filename for Latin, Greek, Cyrillic and other writing systems that have lowercase/uppercase distinction. +- MAY use non-ASCII characters. +- Date-prefixed files SHOULD contain YAML frontmatter with at least `title`, `author` and `date` fields: + ```yaml + --- + title: Implementation Plan + author: John Doe + date: 2025-12-22 + --- + ``` +- If present in YAML frontmatter, author SHOULD be of `Name ` per the RFC 5322 name-addr mailbox format and date SHOULD be ISO 8601 `YYYY-MM-DD` format. + +### 2. Capitalized files + +- For general documents not tied to a specific time, e.g. `README.md`, `AGENTS.md`, `INSTALL.md`, `HOW_TO_DEBUG.md`. +- Multi-word filenames SHOULD use underscores (`CODE_OF_CONDUCT.md`). ## Preferences in Documentation Style -``` - -``` +- Tone: casual, clear, technically precise but not academic +- Planning docs: concrete and actionable, include checklists +- Keep docs concise — no fluff, no filler paragraphs +- Use ISO timestamps where relevant +- Prefer bullet points over prose for technical content ## Before You Start 1. Run `date +%Y-%m-%d` and use the output for both filename prefix and `date` field. 2. Identify where the document belongs: - Keep general documentation at the root of `docs/`. - - If exists, use the dedicated subdirectories for specialized content. + - Use dedicated subdirectories for specialized content (plans, logs, reports). 3. Check for existing, related docs to avoid duplicates and to link to prior work. ## File Naming @@ -58,10 +71,9 @@ If the name and email are not available for some reason, ask the user to provide ### Timeless vs. Dated -- Docs fall into two buckets: - - **Timeless general documents** describe enduring processes or repo-wide rules. They do not carry a date prefix and keep their canonical names. - - **All other content** (design notes, incidents, feature guides, migrations, meeting notes, etc.) must use the date-prefixed naming pattern above with a lower-case, hyphenated title. -- When adding or reviewing documentation, decide which bucket applies. If the doc is not a long-lived reference, rename or relocate it so the filename uses the `YYYY-MM-DD-…` form before merging. +- **Timeless general documents** describe enduring processes or repo-wide rules. They do not carry a date prefix and keep their canonical names. +- **All other content** (design notes, incidents, feature guides, migrations, meeting notes, plans, etc.) must use the date-prefixed naming pattern above. +- When adding or reviewing documentation, decide which bucket applies. ## Required Front Matter @@ -76,54 +88,22 @@ tags: [tag1, tag2] # Optional but recommended --- ``` -SimpleDoc Guidelines: - -- Keep the `date` value in sync with the filename prefix. -- Use a real contact in `author` (`Name `). -- Choose a concise, action-oriented `title`. -- Populate `tags` when it improves discoverability; omit the line if not needed. - ## Daily Logs (SimpleLog) -Use SimpleLog for daily logs. The spec lives at `docs/SIMPLELOG_SPEC.md`. - -### Where logs live - -- Default location: `docs/logs/YYYY-MM-DD.md`. -- The CLI writes to `/docs/logs/` by default when inside a git repo. -- You can set a shared default in `simpledoc.json` and override locally in `.simpledoc.local.json` (see `docs/SIMPLEDOC_CONFIG_SPEC.md`). - -### Create a daily log entry (recommended) - -Use the CLI to create the file and append entries: - -```bash -simpledoc log "Entry text here" -``` - -Notes: - -- The CLI creates the daily log file if missing, including required frontmatter. -- It adds a new session section only when the threshold is exceeded (default 5 minutes). -- It preserves the text you type and inserts a blank line before each new entry. - -### Multiline entries +Default location: `docs/logs/YYYY-MM-DD.md`. -Pipe or heredoc input (stdin) for multiline entries: +### Create a daily log entry ```bash -cat <<'EOF' | simpledoc log -Multiline entry. -- line two -- line three -EOF +npx -y @simpledoc/simpledoc log "Entry text here" ``` -You can also use `--stdin` explicitly: - +For multiline: ```bash -simpledoc log --stdin <<'EOF' -Another multiline entry. +cat <<'EOF' | npx -y @simpledoc/simpledoc log --stdin +Multiline entry here +- point one +- point two EOF ``` @@ -133,21 +113,13 @@ EOF - Ensure a blank line separates entries. - Session sections must be `## HH:MM` (local time of the first entry in that section). -### Ongoing logging (agent behavior) - -Once this skill is active in a repo, the agent SHOULD log anything worth noting as it goes. This includes: - -- Significant changes, decisions, discoveries, tradeoffs, and assumptions. -- Ongoing progress and small but real steps (changes, commands, tests, doc updates). -- Errors, failures, workarounds, and clarifications. +### Ongoing logging -Log each entry after completing the step or realizing the insight. +Log anything worth noting: significant changes, decisions, errors, workarounds, progress. Log each entry after completing the step. No exceptions. ## Final Checks Before Submitting -- [ ] Filename follows the `YYYY-MM-DD-…` pattern (date uses dashes) and lives in the correct directory. -- [ ] Capitalized multi-word filenames use underscores (e.g., `CODE_OF_CONDUCT.md`). -- [ ] Timeless vs. dated classification is correct and filenames reflect the choice. +- [ ] Filename follows the `YYYY-MM-DD-…` pattern and lives in the correct directory. - [ ] Front matter is complete and accurate. - [ ] Links to related documentation exist where applicable. -- [ ] Run `npx -y @simpledoc/simpledoc check` (or `simpledoc check`) to verify SimpleDoc conventions. +- [ ] Run `npx -y @simpledoc/simpledoc check` to verify SimpleDoc conventions. From b47d3a7b57b41959abc0d79f19bcbf7abcea319e Mon Sep 17 00:00:00 2001 From: Bob Date: Fri, 13 Feb 2026 09:17:48 +0100 Subject: [PATCH 3/4] chore: move skills to .agent/skills, update AGENTS.md references --- {skills => .agent/skills}/public-metrics-reporting/SKILL.md | 0 .../skills}/public-metrics-reporting/agents/openai.yaml | 0 .../references/public-copy-templates.md | 0 {skills => .agent/skills}/simpledoc/SKILL.md | 0 AGENTS.md | 4 ++-- 5 files changed, 2 insertions(+), 2 deletions(-) rename {skills => .agent/skills}/public-metrics-reporting/SKILL.md (100%) rename {skills => .agent/skills}/public-metrics-reporting/agents/openai.yaml (100%) rename {skills => .agent/skills}/public-metrics-reporting/references/public-copy-templates.md (100%) rename {skills => .agent/skills}/simpledoc/SKILL.md (100%) diff --git a/skills/public-metrics-reporting/SKILL.md b/.agent/skills/public-metrics-reporting/SKILL.md similarity index 100% rename from skills/public-metrics-reporting/SKILL.md rename to .agent/skills/public-metrics-reporting/SKILL.md diff --git a/skills/public-metrics-reporting/agents/openai.yaml b/.agent/skills/public-metrics-reporting/agents/openai.yaml similarity index 100% rename from skills/public-metrics-reporting/agents/openai.yaml rename to .agent/skills/public-metrics-reporting/agents/openai.yaml diff --git a/skills/public-metrics-reporting/references/public-copy-templates.md b/.agent/skills/public-metrics-reporting/references/public-copy-templates.md similarity index 100% rename from skills/public-metrics-reporting/references/public-copy-templates.md rename to .agent/skills/public-metrics-reporting/references/public-copy-templates.md diff --git a/skills/simpledoc/SKILL.md b/.agent/skills/simpledoc/SKILL.md similarity index 100% rename from skills/simpledoc/SKILL.md rename to .agent/skills/simpledoc/SKILL.md diff --git a/AGENTS.md b/AGENTS.md index 2996ff6..6ba6543 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -74,7 +74,7 @@ Janitr/ │ └── logs/ # Daily development logs │ ├── tests/ # Integration tests -└── skills/ # Agent skills (SimpleDoc) +└── .agent/skills/ # Agent skills (SimpleDoc, etc.) ``` --- @@ -159,7 +159,7 @@ Thresholds are tuned per-label to maintain FPR < 2%. The `promo` label is curren ## SimpleDoc Conventions -**Attention agent!** Before creating ANY documentation, use the `simpledoc` skill in `skills/simpledoc/SKILL.md`. +**Attention agent!** Before creating ANY documentation, use the `simpledoc` skill in `.agent/skills/simpledoc/SKILL.md`. Key rules: From ebfb8c0ea7be0fcaea783350b6d2b18449ad8ddd Mon Sep 17 00:00:00 2001 From: Bob Date: Fri, 13 Feb 2026 09:19:21 +0100 Subject: [PATCH 4/4] chore: rename plan to simpledoc date-prefix convention --- docs/logs/2026-02-13.md | 10 +++++----- ...mer-plan.md => 2026-02-13-tiny-transformer-plan.md} | 0 2 files changed, 5 insertions(+), 5 deletions(-) rename docs/plans/{tiny-transformer-plan.md => 2026-02-13-tiny-transformer-plan.md} (100%) diff --git a/docs/logs/2026-02-13.md b/docs/logs/2026-02-13.md index a811ed1..1a9d49f 100644 --- a/docs/logs/2026-02-13.md +++ b/docs/logs/2026-02-13.md @@ -12,7 +12,7 @@ updated: 2026-02-13T09:09:50+01:00 Planning tiny-transformer implementation doc for janitr. Gathered current context from scripts/prepare_data.py, scripts/prepare_hf.py, scripts/evaluate.py, extension/offscreen.js, extension/background.js, and data/replies\*.jsonl. Key constraint observed: only 14 ground-truth ai_generated_reply rows currently in data/replies.jsonl, so plan will include data expansion + teacher distillation before student deployment. -Created docs/plans/tiny-transformer-plan.md with a concrete implementation plan for the tiny-transformer option. +Created docs/plans/2026-02-13-tiny-transformer-plan.md with a concrete implementation plan for the tiny-transformer option. Included: @@ -25,12 +25,12 @@ Included: - offline + runtime evaluation gates - model/bundle size estimates -Ran `npx @simpledoc/simpledoc check` after writing; it failed only on filename convention and suggested renaming to a date-prefixed filename. Kept the user-requested path `docs/plans/tiny-transformer-plan.md` intentionally. +Ran `npx @simpledoc/simpledoc check` after writing; it failed only on filename convention and suggested renaming to a date-prefixed filename. Kept the user-requested path `docs/plans/2026-02-13-tiny-transformer-plan.md` intentionally. ## 09:08 -Reviewed current tiny-transformer plan and baseline Janitr fastText pipeline. Preparing full rewrite of docs/plans/tiny-transformer-plan.md to target existing scam/topic_crypto/clean classifier, incorporate teacher-model research, and remove all ai_reply references. +Reviewed current tiny-transformer plan and baseline Janitr fastText pipeline. Preparing full rewrite of docs/plans/2026-02-13-tiny-transformer-plan.md to target existing scam/topic_crypto/clean classifier, incorporate teacher-model research, and remove all ai_reply references. -Rewrote docs/plans/tiny-transformer-plan.md to target Janitr's existing scam/topic_crypto/clean classifier instead of ai_reply. Updated dataset baseline counts, teacher model choices (TimeLM primary, BERTweet alternative), distillation strategy (calibration, T=2-4, alpha schedule, hidden-state matching), DAPT/logit-caching plan, extension integration references, script names, evaluation gates, and added updated timestamp. +Rewrote docs/plans/2026-02-13-tiny-transformer-plan.md to target Janitr's existing scam/topic_crypto/clean classifier instead of ai_reply. Updated dataset baseline counts, teacher model choices (TimeLM primary, BERTweet alternative), distillation strategy (calibration, T=2-4, alpha schedule, hidden-state matching), DAPT/logit-caching plan, extension integration references, script names, evaluation gates, and added updated timestamp. -Ran npx -y @simpledoc/simpledoc check after rewrite. Check failed because docs/plans/tiny-transformer-plan.md is not date-prefixed per current SimpleDoc naming convention; migration would rename file and update docs/logs references. Kept filename unchanged this turn because user requested direct rewrite of that exact path. +Ran npx -y @simpledoc/simpledoc check after rewrite. Check failed because docs/plans/2026-02-13-tiny-transformer-plan.md is not date-prefixed per current SimpleDoc naming convention; migration would rename file and update docs/logs references. Kept filename unchanged this turn because user requested direct rewrite of that exact path. diff --git a/docs/plans/tiny-transformer-plan.md b/docs/plans/2026-02-13-tiny-transformer-plan.md similarity index 100% rename from docs/plans/tiny-transformer-plan.md rename to docs/plans/2026-02-13-tiny-transformer-plan.md