Commit graph

5 commits

Author SHA1 Message Date
17f7458d19 Sprint 9: cloze cards, plurals deck, project reorg, lint tooling
- Cloze card pipeline: 924 cards from 2,296 AI-vetted Hebrew book sentences
- Plurals deck: 375 notes (144 irregular + 231 regular from 86 mishkal patterns)
- Ktiv male forms expanded to 20,711 entries for sentence matching
- Project reorg: helpers.py (deduped strip_nikkud from 10 files), scripts/ for
  one-off tools, tests/ with smoke tests, deleted 3 dead files
- Lint tooling: pyproject.toml with ruff/vulture/bandit/pytest config, .editorconfig,
  fixed all 129 ruff errors (B023 closure fix, SIM103, unused vars)
- validate_apkg.py: card count range check for optional cloze template
- Data caches committed: vetted_sentences, ktiv_male_forms, noun_plurals,
  noun_slug_map, vocab_sentence_matches, epub_sentence_index

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-07 08:09:39 +00:00
372680be3c Add missing 70th verb להתקלח (to shower, Hitpa'el)
להתלקלח in the original source was a typo for להתקלח (1896-lehitkaleach),
not for להתקלקל as previously assumed — it's a completely different word.
Conjugation deck now has the correct 70 paradigm verbs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-04 06:46:07 +00:00
d26e4c8ce5 feat: Sprint 3 — passive/active separation, random card order, card UX fixes
Conjugation extraction:
- Active entries now extract active forms only (no auto passive partner)
- Passive (# 3ms:) entries extract passive section only via new
  _extract_passive_from_active_slug(); search-based fallback also uses
  this path so no active forms leak into passive entries
- # slug: VERB SLUG override syntax for search-ambiguous active verbs
- # 3ms: FORM ACTIVE-SLUG syntax for passive entries with known active page
- Fixed verb spellings: בוטל (was בותל), slug overrides for תואם →
  2344-letaem, זוכה → 503-lezakot, לָשִׂים → 45-lasim, העבר → 1442-lehaavir

Card UX:
- Passive card front: shows active partner infinitive (e.g. לְבַטֵּל) with
  (סָבִיל) inline in smaller font instead of bare 3ms past form
- Removed פָּעִיל label from active cards; only passive cards carry voice label
- New cards introduced in random order (new.order=0 via _RandomOrderPackage)
- Frequency badge: words outside top 50k show "50k+" instead of blank

README: updated CLI options, output files table, pipeline list, card
descriptions to reflect Sprint 3 state

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 10:16:50 +00:00
b018f21b1d feat: Sprint 2 + Sprint 3 — verb list, audio, passive forms, CSS/UX, validation, Heebo font, images
Sprint 2:
- extract_verb_list.py (NEW): downloads Coffin & Bolozky PDF, extracts
  71-verb paradigm list from Appendix 1 with hardcoded fallback.
  Pu'al/Huf'al use '# 3ms:' prefix for 3ms search.
- conjugation_extract.py: audio URL capture per form, passive forms
  parsing (Pu'al/Huf'al partner tables), 3ms search support.
- benyehuda.py: nikkud corpus (txt.zip), index by nikkud word form,
  single best example (longest ≤200 chars), --refresh-examples rebuild.
- apkg_builder.py: Hebrew labels, centered dark Hebrew text, freq-badge,
  related words grouped by PoS. Conjugation: Voice/Audio fields,
  present-tense 12-card expansion, 2fp/3fp modern fallback with
  classical in parens, פָּעִיל/סָבִיל voice labels.
- README.md: rewritten — learner-first structure, data sources.
- run.py: --refresh-examples flag, conjugation audio download (step 4b).
- data/conjugations.json: rebuilt with 70 verbs, audio URLs, passive
  partner data.

Sprint 3:
- validate_verb_list.py (NEW): queries pealim.com for all entries in
  verb input list, classifies as OK/3ms/REVIEW/NOT_FOUND, writes
  cleaned verbs_input.txt. Results: 51 OK, 15 3ms-past, 4 REVIEW.
- apkg_builder.py: binyan in Hebrew (BINYAN_TO_HEBREW map) on its own
  line; remove "דוגמה:" label; "Other" related-words shown unlabeled;
  "50k+" freq display for unlisted words; Image field in VOCAB_MODEL.
- image_fetch.py (NEW): Wikipedia/Commons thumbnails for concrete nouns,
  caches in data/image_cache.json, downloads to data/images/.
- Heebo variable font TTF bundled in both .apkg files via @font-face.
- run.py: step_fonts(), step_images(), --skip-images flag.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 08:36:51 +00:00
b086123bec feat: add apkg builder, frequency, Ben Yehuda examples, conjugation deck
Implements four major improvements to the Pealim Anki deck pipeline:

1. Automated .apkg generation (genanki) — no more manual Anki Desktop step.
   Both vocabulary and conjugation decks are built programmatically.

2. Word frequency ranking from hermitdave/FrequencyWords he_50k corpus.
   Notes sorted by rank so Anki presents most common words first.

3. Example sentences from Ben Yehuda public domain corpus (not pealim.com).
   Downloads txt_stripped.zip, indexes 25k texts, ~89% coverage on test set.

4. Conjugation drill deck — one card per form × verb.
   Input: verbs_input.txt (Hebrew infinitives). Initial set: 7 verbs (one
   per binyan). Extracts 28 forms each via pealim.com/search/ + table parse.

New files:
  apkg_builder.py     — genanki deck builder for both decks
  benyehuda.py        — Ben Yehuda corpus downloader + sentence indexer
  frequency_lookup.py — FrequencyWords downloader + rank lookup
  verbs_input.txt     — verb input list (7 test verbs, one per binyan)
  data/               — baseline CSVs + generated caches

Updated:
  conjugation_extract.py — rewritten: reads verbs_input.txt, searches
                           /search/?q= for slug, parses table by row labels
  requirements.txt       — add genanki, beautifulsoup4, lxml
  run.py                 — full orchestration pipeline with CLI flags
  .gitignore             — exclude venv/, benyehuda_index.json, audio/, output/

CLI:
  python run.py --skip-scrape --skip-audio --test 20  (quick test)
  python run.py --skip-scrape                          (full build)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-03 01:58:31 +00:00