docs: record Basque tokenizer fix and Cordn UX polish

This commit is contained in:
SkateSpots 2026-10-27 16:42:47 +07:00
parent 4009dd3a72
commit 7a7b1c4950

View file

@ -1470,6 +1470,23 @@ session with **no other memory of this project**. So:
(`?tab=events|chat|mls`; an explicit param wins over the auto-default). This is
the cheap "pointer" integration — the two transports are NOT merged (DMs stay
NIP-17; the coordinator is only the crew MLS group).
**UX polish (2026-10-27):** after "Ask to join" the panel auto-checks the
coordinator every 10s→30s (silent signers only; Amber stays manual), and stops
when the welcome lands. The empty state has 3 plain steps + "only crew members
can read this"; Cordn/group ref/key package moved under **Advanced**. The
status chip is actionable (Retry), failures show a plain message + details.
A device with no key package gets **"Set up this device"** (`requestJoin`
publishes a fresh package). The member list shows avatars + founder/admin/
member roles; join requests load on open and show avatars. Messages show
Sending…/Sent/Failed with **Retry** (optimistic pending bubble). Invite opens
a dialog with QR + copy/share and a `?tab=mls` link; on native it uses
`https://skate-spots.org` (not `https://localhost`). The founder banner is
dismissible per crew (`skatespots:cordn-banner-dismissed:<pk>/<id>`).
`useCordnNative` adds `retry`, `keyPackageReady`, boolean
`send`/`createChat`/`requestJoin`/`approve`, and **filters welcomes by gid** so
one crew can't adopt another crew's chat. Coordinator announce relays are now
`["wss://relay.skate-spots.org"]` only (removed purplepag.es/nostr.band/nos.lol
timeouts; logs show `local[10002=ok]`).
- **Map site-links strip (2026-10-05):** the desktop footer is hidden on the map
(full-bleed), so `components/MapFooterStrip.tsx` surfaces the same links
(Newsletter/License/Privacy/Collaborators + "not sponsored by OpenSats") as an
@ -2244,6 +2261,15 @@ Rules:
25 codes), regional flag SVGs + English/Portuguese variants, single-world
map clamp with horizontal wrap to the Pacific, menu hardening; APK sha256
`846925af…`, versionCode 10701, released with `scripts/release.sh`.
- **1.7.2 (2026-10-27):** Basque translation repaired (Marian tokenizer IDs +
EOS + one-time cache eviction), Cordn private-chat polish (auto-check,
3-step onboarding, status/retry, new-device setup, member roles, send
states, invite QR/deep link, dismissible banner), coordinator announce
list trimmed to the community relay. APK sha256 `a25ad33c…`, versionCode
10702.
- **1.7.3 (2026-10-27):** native invite QR uses the public
`https://skate-spots.org` URL instead of Capacitor's `https://localhost`.
APK sha256 `0448a1c7…`, versionCode 10703.
### Relay archive (dataset backup)
- **Transfer content between keys (`scripts/transfer-content.mjs`, 2026-10-04):**
@ -2454,11 +2480,20 @@ Rules:
- Hosted at `https://relay.skate-spots.org/models/…` (Caddy
`handle_path /models/*` → `/srv/models`, CORS `*`; backup
`/etc/caddy/Caddyfile.bak-…`).
- **Verified from the host:** en↔es, ca↔gl, hi↔en. The Basque packs are
converted and hosted (ONNX q8). Fixed: the ONNX **IR 9→8** mismatch and the
Marian tokenizer's **target-side** `tokenizer.json`. The decoder now emits
Basque, but it **loops** (EOS/generation config), so Basque is still **not
usable**. Next: the `eos_token_id`/generation loop.
- **Verified from the host:** en↔es, ca↔gl, hi↔en, and **eu↔en after the
2026-10-27 fix**. The Basque packs are converted and hosted (ONNX q8).
Tested in Node against the same hosted tokenizer/ONNX bytes; outputs match
the original Python model. Two root causes: the ONNX **IR 9→8** mismatch,
and the converted `tokenizer.json` using SentencePiece IDs (special tokens
0/1/2) instead of the Marian model's `vocab.json` IDs — so `</s>` was never
recognized and generation looped. Rebuilt both tokenizer.json files in
model-vocab order with source-spm scores, special tokens at the model IDs,
and an EOS post-processor; also set Metaspace `add_prefix_space` so the first
word gets `▁`. Backups are on the model host.
- **Cache migration:** `src/lib/translationModelCache.ts` deletes only the old
cached `opus-mt-{eu-en,en-eu}` `tokenizer.json` entries on first Basque use
(the 390 MB ONNX stays cached). Unit-tested; the worker calls it before
loading a Basque translator.
- Runtime: **`@xenova/transformers` 2.17.2** + ONNX Runtime **1.14** WASM
self-hosted in `public/ort/` (37 MB, copied from `onnxruntime-web`), loaded
from `../ort/`; worker `src/workers/translate.worker.ts`. M2M100 is used
@ -2481,9 +2516,10 @@ Rules:
- **Conversion:** Docker Python 3.11 + torch 2.4.1 + transformers 4.46.3 +
optimum 1.24; dynamic q8; `tokenizer.json` generated from the SentencePiece
models (needs protobuf + `sentencepiece_model_pb2.py`).
- **Still to do:** fix the Basque EOS/generation loop, verify all 24 pairs in
the browser/APK, measure speed/memory, and ship **1.7.2** (the phone is on
1.7.1). No SpotChat.
- **Still to do:** verify all 24 pairs in the browser/APK and measure
speed/memory on low-end phones. **1.7.2 and 1.7.3 are released** (the phone
needs the Zapstore update; the web deploy timer batches for 24h unless forced).
No SpotChat.
- **Keyboard caveat:** a malicious OS keyboard or rooted device is outside the
app’s control. Warn/recommend a trusted offline keyboard, and avoid the OS
keyboard for **secrets** (custom secure input / QR / bunker/Amber,