docs: record Basque tokenizer fix and Cordn UX polish
This commit is contained in:
parent
4009dd3a72
commit
7a7b1c4950
1 changed files with 44 additions and 8 deletions
52
AGENTS.md
52
AGENTS.md
|
|
@ -1470,6 +1470,23 @@ session with **no other memory of this project**. So:
|
|||
(`?tab=events|chat|mls`; an explicit param wins over the auto-default). This is
|
||||
the cheap "pointer" integration — the two transports are NOT merged (DMs stay
|
||||
NIP-17; the coordinator is only the crew MLS group).
|
||||
**UX polish (2026-10-27):** after "Ask to join" the panel auto-checks the
|
||||
coordinator every 10s→30s (silent signers only; Amber stays manual), and stops
|
||||
when the welcome lands. The empty state has 3 plain steps + "only crew members
|
||||
can read this"; Cordn/group ref/key package moved under **Advanced**. The
|
||||
status chip is actionable (Retry), failures show a plain message + details.
|
||||
A device with no key package gets **"Set up this device"** (`requestJoin`
|
||||
publishes a fresh package). The member list shows avatars + founder/admin/
|
||||
member roles; join requests load on open and show avatars. Messages show
|
||||
Sending…/Sent/Failed with **Retry** (optimistic pending bubble). Invite opens
|
||||
a dialog with QR + copy/share and a `?tab=mls` link; on native it uses
|
||||
`https://skate-spots.org` (not `https://localhost`). The founder banner is
|
||||
dismissible per crew (`skatespots:cordn-banner-dismissed:<pk>/<id>`).
|
||||
`useCordnNative` adds `retry`, `keyPackageReady`, boolean
|
||||
`send`/`createChat`/`requestJoin`/`approve`, and **filters welcomes by gid** so
|
||||
one crew can't adopt another crew's chat. Coordinator announce relays are now
|
||||
`["wss://relay.skate-spots.org"]` only (removed purplepag.es/nostr.band/nos.lol
|
||||
timeouts; logs show `local[10002=ok]`).
|
||||
- **Map site-links strip (2026-10-05):** the desktop footer is hidden on the map
|
||||
(full-bleed), so `components/MapFooterStrip.tsx` surfaces the same links
|
||||
(Newsletter/License/Privacy/Collaborators + "not sponsored by OpenSats") as an
|
||||
|
|
@ -2244,6 +2261,15 @@ Rules:
|
|||
25 codes), regional flag SVGs + English/Portuguese variants, single-world
|
||||
map clamp with horizontal wrap to the Pacific, menu hardening; APK sha256
|
||||
`846925af…`, versionCode 10701, released with `scripts/release.sh`.
|
||||
- **1.7.2 (2026-10-27):** Basque translation repaired (Marian tokenizer IDs +
|
||||
EOS + one-time cache eviction), Cordn private-chat polish (auto-check,
|
||||
3-step onboarding, status/retry, new-device setup, member roles, send
|
||||
states, invite QR/deep link, dismissible banner), coordinator announce
|
||||
list trimmed to the community relay. APK sha256 `a25ad33c…`, versionCode
|
||||
10702.
|
||||
- **1.7.3 (2026-10-27):** native invite QR uses the public
|
||||
`https://skate-spots.org` URL instead of Capacitor's `https://localhost`.
|
||||
APK sha256 `0448a1c7…`, versionCode 10703.
|
||||
|
||||
### Relay archive (dataset backup)
|
||||
- **Transfer content between keys (`scripts/transfer-content.mjs`, 2026-10-04):**
|
||||
|
|
@ -2454,11 +2480,20 @@ Rules:
|
|||
- Hosted at `https://relay.skate-spots.org/models/…` (Caddy
|
||||
`handle_path /models/*` → `/srv/models`, CORS `*`; backup
|
||||
`/etc/caddy/Caddyfile.bak-…`).
|
||||
- **Verified from the host:** en↔es, ca↔gl, hi↔en. The Basque packs are
|
||||
converted and hosted (ONNX q8). Fixed: the ONNX **IR 9→8** mismatch and the
|
||||
Marian tokenizer's **target-side** `tokenizer.json`. The decoder now emits
|
||||
Basque, but it **loops** (EOS/generation config), so Basque is still **not
|
||||
usable**. Next: the `eos_token_id`/generation loop.
|
||||
- **Verified from the host:** en↔es, ca↔gl, hi↔en, and **eu↔en after the
|
||||
2026-10-27 fix**. The Basque packs are converted and hosted (ONNX q8).
|
||||
Tested in Node against the same hosted tokenizer/ONNX bytes; outputs match
|
||||
the original Python model. Two root causes: the ONNX **IR 9→8** mismatch,
|
||||
and the converted `tokenizer.json` using SentencePiece IDs (special tokens
|
||||
0/1/2) instead of the Marian model's `vocab.json` IDs — so `</s>` was never
|
||||
recognized and generation looped. Rebuilt both tokenizer.json files in
|
||||
model-vocab order with source-spm scores, special tokens at the model IDs,
|
||||
and an EOS post-processor; also set Metaspace `add_prefix_space` so the first
|
||||
word gets `▁`. Backups are on the model host.
|
||||
- **Cache migration:** `src/lib/translationModelCache.ts` deletes only the old
|
||||
cached `opus-mt-{eu-en,en-eu}` `tokenizer.json` entries on first Basque use
|
||||
(the 390 MB ONNX stays cached). Unit-tested; the worker calls it before
|
||||
loading a Basque translator.
|
||||
- Runtime: **`@xenova/transformers` 2.17.2** + ONNX Runtime **1.14** WASM
|
||||
self-hosted in `public/ort/` (37 MB, copied from `onnxruntime-web`), loaded
|
||||
from `../ort/`; worker `src/workers/translate.worker.ts`. M2M100 is used
|
||||
|
|
@ -2481,9 +2516,10 @@ Rules:
|
|||
- **Conversion:** Docker Python 3.11 + torch 2.4.1 + transformers 4.46.3 +
|
||||
optimum 1.24; dynamic q8; `tokenizer.json` generated from the SentencePiece
|
||||
models (needs protobuf + `sentencepiece_model_pb2.py`).
|
||||
- **Still to do:** fix the Basque EOS/generation loop, verify all 24 pairs in
|
||||
the browser/APK, measure speed/memory, and ship **1.7.2** (the phone is on
|
||||
1.7.1). No SpotChat.
|
||||
- **Still to do:** verify all 24 pairs in the browser/APK and measure
|
||||
speed/memory on low-end phones. **1.7.2 and 1.7.3 are released** (the phone
|
||||
needs the Zapstore update; the web deploy timer batches for 24h unless forced).
|
||||
No SpotChat.
|
||||
- **Keyboard caveat:** a malicious OS keyboard or rooted device is outside the
|
||||
app’s control. Warn/recommend a trusted offline keyboard, and avoid the OS
|
||||
keyboard for **secrets** (custom secure input / QR / bunker/Amber,
|
||||
|
|
|
|||
Loading…
Reference in a new issue