docs: record Basque tokenizer fix and Cordn UX polish
This commit is contained in:
parent
4009dd3a72
commit
7a7b1c4950
1 changed files with 44 additions and 8 deletions
52
AGENTS.md
52
AGENTS.md
|
|
@ -1470,6 +1470,23 @@ session with **no other memory of this project**. So:
|
||||||
(`?tab=events|chat|mls`; an explicit param wins over the auto-default). This is
|
(`?tab=events|chat|mls`; an explicit param wins over the auto-default). This is
|
||||||
the cheap "pointer" integration — the two transports are NOT merged (DMs stay
|
the cheap "pointer" integration — the two transports are NOT merged (DMs stay
|
||||||
NIP-17; the coordinator is only the crew MLS group).
|
NIP-17; the coordinator is only the crew MLS group).
|
||||||
|
**UX polish (2026-10-27):** after "Ask to join" the panel auto-checks the
|
||||||
|
coordinator every 10s→30s (silent signers only; Amber stays manual), and stops
|
||||||
|
when the welcome lands. The empty state has 3 plain steps + "only crew members
|
||||||
|
can read this"; Cordn/group ref/key package moved under **Advanced**. The
|
||||||
|
status chip is actionable (Retry), failures show a plain message + details.
|
||||||
|
A device with no key package gets **"Set up this device"** (`requestJoin`
|
||||||
|
publishes a fresh package). The member list shows avatars + founder/admin/
|
||||||
|
member roles; join requests load on open and show avatars. Messages show
|
||||||
|
Sending…/Sent/Failed with **Retry** (optimistic pending bubble). Invite opens
|
||||||
|
a dialog with QR + copy/share and a `?tab=mls` link; on native it uses
|
||||||
|
`https://skate-spots.org` (not `https://localhost`). The founder banner is
|
||||||
|
dismissible per crew (`skatespots:cordn-banner-dismissed:<pk>/<id>`).
|
||||||
|
`useCordnNative` adds `retry`, `keyPackageReady`, boolean
|
||||||
|
`send`/`createChat`/`requestJoin`/`approve`, and **filters welcomes by gid** so
|
||||||
|
one crew can't adopt another crew's chat. Coordinator announce relays are now
|
||||||
|
`["wss://relay.skate-spots.org"]` only (removed purplepag.es/nostr.band/nos.lol
|
||||||
|
timeouts; logs show `local[10002=ok]`).
|
||||||
- **Map site-links strip (2026-10-05):** the desktop footer is hidden on the map
|
- **Map site-links strip (2026-10-05):** the desktop footer is hidden on the map
|
||||||
(full-bleed), so `components/MapFooterStrip.tsx` surfaces the same links
|
(full-bleed), so `components/MapFooterStrip.tsx` surfaces the same links
|
||||||
(Newsletter/License/Privacy/Collaborators + "not sponsored by OpenSats") as an
|
(Newsletter/License/Privacy/Collaborators + "not sponsored by OpenSats") as an
|
||||||
|
|
@ -2244,6 +2261,15 @@ Rules:
|
||||||
25 codes), regional flag SVGs + English/Portuguese variants, single-world
|
25 codes), regional flag SVGs + English/Portuguese variants, single-world
|
||||||
map clamp with horizontal wrap to the Pacific, menu hardening; APK sha256
|
map clamp with horizontal wrap to the Pacific, menu hardening; APK sha256
|
||||||
`846925af…`, versionCode 10701, released with `scripts/release.sh`.
|
`846925af…`, versionCode 10701, released with `scripts/release.sh`.
|
||||||
|
- **1.7.2 (2026-10-27):** Basque translation repaired (Marian tokenizer IDs +
|
||||||
|
EOS + one-time cache eviction), Cordn private-chat polish (auto-check,
|
||||||
|
3-step onboarding, status/retry, new-device setup, member roles, send
|
||||||
|
states, invite QR/deep link, dismissible banner), coordinator announce
|
||||||
|
list trimmed to the community relay. APK sha256 `a25ad33c…`, versionCode
|
||||||
|
10702.
|
||||||
|
- **1.7.3 (2026-10-27):** native invite QR uses the public
|
||||||
|
`https://skate-spots.org` URL instead of Capacitor's `https://localhost`.
|
||||||
|
APK sha256 `0448a1c7…`, versionCode 10703.
|
||||||
|
|
||||||
### Relay archive (dataset backup)
|
### Relay archive (dataset backup)
|
||||||
- **Transfer content between keys (`scripts/transfer-content.mjs`, 2026-10-04):**
|
- **Transfer content between keys (`scripts/transfer-content.mjs`, 2026-10-04):**
|
||||||
|
|
@ -2454,11 +2480,20 @@ Rules:
|
||||||
- Hosted at `https://relay.skate-spots.org/models/…` (Caddy
|
- Hosted at `https://relay.skate-spots.org/models/…` (Caddy
|
||||||
`handle_path /models/*` → `/srv/models`, CORS `*`; backup
|
`handle_path /models/*` → `/srv/models`, CORS `*`; backup
|
||||||
`/etc/caddy/Caddyfile.bak-…`).
|
`/etc/caddy/Caddyfile.bak-…`).
|
||||||
- **Verified from the host:** en↔es, ca↔gl, hi↔en. The Basque packs are
|
- **Verified from the host:** en↔es, ca↔gl, hi↔en, and **eu↔en after the
|
||||||
converted and hosted (ONNX q8). Fixed: the ONNX **IR 9→8** mismatch and the
|
2026-10-27 fix**. The Basque packs are converted and hosted (ONNX q8).
|
||||||
Marian tokenizer's **target-side** `tokenizer.json`. The decoder now emits
|
Tested in Node against the same hosted tokenizer/ONNX bytes; outputs match
|
||||||
Basque, but it **loops** (EOS/generation config), so Basque is still **not
|
the original Python model. Two root causes: the ONNX **IR 9→8** mismatch,
|
||||||
usable**. Next: the `eos_token_id`/generation loop.
|
and the converted `tokenizer.json` using SentencePiece IDs (special tokens
|
||||||
|
0/1/2) instead of the Marian model's `vocab.json` IDs — so `</s>` was never
|
||||||
|
recognized and generation looped. Rebuilt both tokenizer.json files in
|
||||||
|
model-vocab order with source-spm scores, special tokens at the model IDs,
|
||||||
|
and an EOS post-processor; also set Metaspace `add_prefix_space` so the first
|
||||||
|
word gets `▁`. Backups are on the model host.
|
||||||
|
- **Cache migration:** `src/lib/translationModelCache.ts` deletes only the old
|
||||||
|
cached `opus-mt-{eu-en,en-eu}` `tokenizer.json` entries on first Basque use
|
||||||
|
(the 390 MB ONNX stays cached). Unit-tested; the worker calls it before
|
||||||
|
loading a Basque translator.
|
||||||
- Runtime: **`@xenova/transformers` 2.17.2** + ONNX Runtime **1.14** WASM
|
- Runtime: **`@xenova/transformers` 2.17.2** + ONNX Runtime **1.14** WASM
|
||||||
self-hosted in `public/ort/` (37 MB, copied from `onnxruntime-web`), loaded
|
self-hosted in `public/ort/` (37 MB, copied from `onnxruntime-web`), loaded
|
||||||
from `../ort/`; worker `src/workers/translate.worker.ts`. M2M100 is used
|
from `../ort/`; worker `src/workers/translate.worker.ts`. M2M100 is used
|
||||||
|
|
@ -2481,9 +2516,10 @@ Rules:
|
||||||
- **Conversion:** Docker Python 3.11 + torch 2.4.1 + transformers 4.46.3 +
|
- **Conversion:** Docker Python 3.11 + torch 2.4.1 + transformers 4.46.3 +
|
||||||
optimum 1.24; dynamic q8; `tokenizer.json` generated from the SentencePiece
|
optimum 1.24; dynamic q8; `tokenizer.json` generated from the SentencePiece
|
||||||
models (needs protobuf + `sentencepiece_model_pb2.py`).
|
models (needs protobuf + `sentencepiece_model_pb2.py`).
|
||||||
- **Still to do:** fix the Basque EOS/generation loop, verify all 24 pairs in
|
- **Still to do:** verify all 24 pairs in the browser/APK and measure
|
||||||
the browser/APK, measure speed/memory, and ship **1.7.2** (the phone is on
|
speed/memory on low-end phones. **1.7.2 and 1.7.3 are released** (the phone
|
||||||
1.7.1). No SpotChat.
|
needs the Zapstore update; the web deploy timer batches for 24h unless forced).
|
||||||
|
No SpotChat.
|
||||||
- **Keyboard caveat:** a malicious OS keyboard or rooted device is outside the
|
- **Keyboard caveat:** a malicious OS keyboard or rooted device is outside the
|
||||||
app’s control. Warn/recommend a trusted offline keyboard, and avoid the OS
|
app’s control. Warn/recommend a trusted offline keyboard, and avoid the OS
|
||||||
keyboard for **secrets** (custom secure input / QR / bunker/Amber,
|
keyboard for **secrets** (custom secure input / QR / bunker/Amber,
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue