fix(stats): resolve derived regions via BigDataCloud, not Nominatim

Nominatim returned 429 for our IP because the read-time derivation runs from
every viewer on every load — its usage policy forbids that, and a 429 makes
reverseGeocodeRegions() (which returns empty on !res.ok) fail silently, freezing
the resolve counter and stalling the region card.

The stats fallback now uses BigDataCloud's reverse-geocode-client (no key, CORS
on both hops, English countryName/countryCode/city), spaced ~300ms. Nominatim
stays for create/edit geocoding only (author-initiated, one call per spot).
/privacy now names both providers.
This commit is contained in:
SkateSpots 2026-10-01 14:31:49 +07:00
parent 6091a0acd8
commit 8fd5bab3f1
3 changed files with 58 additions and 18 deletions

View file

@ -910,13 +910,20 @@ session with **no other memory of this project**. So:
of an untagged pin's own coordinates** (`lib/derivedRegions.ts`, localStorage of an untagged pin's own coordinates** (`lib/derivedRegions.ts`, localStorage
`skatespots:derived-regions-v1`, keyed by ~110 m-rounded coords). A `skatespots:derived-regions-v1`, keyed by ~110 m-rounded coords). A
`useDeriveRegions()` hook mounted on `/stats` scans the community relay for `useDeriveRegions()` hook mounted on `/stats` scans the community relay for
untagged spots/shops, fills cache misses at **1.1 s spacing** (Nominatim untagged spots/shops, fills cache misses, and invalidates `growth-stats` every
1 req/s) and invalidates `growth-stats`; the card shows "Resolving N…" until 5 resolved (and at the end); the card shows a live "Resolving N…" counter.
done. ⚠️ This geocodes pins **when the stats page is viewed**, so Nominatim ⚠️ **Do NOT use Nominatim for this** — it is per-VIEWER volume (every device,
sees the viewer's IP + public pin coords — disclosed on `/privacy` every reload), Nominatim's policy forbids it and it **429'd us** (2026-10-01),
("Spot/shop geocoding"). It is a **display fallback only**; the backfill above which silently broke every remaining lookup (`reverseGeocodeRegions()` returns
is what writes real `city`/`country` tags. Either way the card's `Uncategorised` empty on `!res.ok`, so the counter froze and the card stalled). The fallback
row now means "couldn't be tagged OR resolved", not "no tag". now uses **BigDataCloud** `reverse-geocode-client` (no key, CORS `*` on both
`api.bigdatacloud.net` → 307 → `api-bdc.io`, English `countryName`/`countryCode`/
`city`), spaced ~300 ms. Nominatim stays for **create/edit** only (the author's
own pin, one call per spot). ⚠️ This geocodes pins when the stats page is
viewed, so the provider sees the viewer's IP + public pin coords — disclosed on
`/privacy` ("Spot/shop geocoding", names both providers). It is a **display
fallback only**; the backfill above writes real `city`/`country` tags. Either
way the card's `Uncategorised` row now means "couldn't be tagged OR resolved".
- **NIP-05 names — `you@skate-spots.org` (2026-09-28):** SkateSpots hands out - **NIP-05 names — `you@skate-spots.org` (2026-09-28):** SkateSpots hands out
free handles so a new skater has a human name from minute one. The registry is free handles so a new skater has a human name from minute one. The registry is
the Netlify Function **`netlify/functions/nip05.mjs`** the Netlify Function **`netlify/functions/nip05.mjs`**

View file

@ -11,15 +11,18 @@
* cached in localStorage (keyed by ~110 m-rounded coordinates), so each pin is * cached in localStorage (keyed by ~110 m-rounded coordinates), so each pin is
* reverse-geocoded at most once per device. * reverse-geocoded at most once per device.
* *
* ⚠️ Privacy: unlike create/edit geocoding (which is the author's own pin), this * ⚠️ Uses **BigDataCloud** (`reverse-geocode-client`, no key, CORS, designed for
* runs when the STATS PAGE IS VIEWED, so Nominatim sees the viewer's IP and the * browsers) — NOT Nominatim. Nominatim's usage policy forbids this kind of
* (public) coordinates of untagged pins. Disclosed on /privacy. * per-viewer volume and returned 429 (see AGENTS.md). Nominatim stays for
* create/edit geocoding (author-initiated, one call per spot).
*
* ⚠️ Privacy: unlike create/edit geocoding (the author's own pin), this runs when
* the STATS PAGE IS VIEWED, so BigDataCloud sees the viewer's IP and the (public)
* coordinates of untagged pins. Disclosed on /privacy.
*/ */
import ngeohash from 'ngeohash'; import ngeohash from 'ngeohash';
import { reverseGeocodeRegions } from '@/lib/geocodeRegions';
export interface DerivedRegion { export interface DerivedRegion {
city: string; city: string;
country: string; country: string;
@ -27,8 +30,12 @@ export interface DerivedRegion {
} }
const STORAGE_KEY = 'skatespots:derived-regions-v1'; const STORAGE_KEY = 'skatespots:derived-regions-v1';
/** Nominatim's usage policy allows at most one request per second. */ /**
const GEOCODE_SPACING_MS = 1100; * BigDataCloud tolerates far more than Nominatim's 1 req/s, but keep it polite:
* ~3 req/s. The whole list is one pass per device, then cached.
*/
const GEOCODE_SPACING_MS = 300;
const REQUEST_TIMEOUT_MS = 8000;
let memoryCache: Record<string, DerivedRegion> | null = null; let memoryCache: Record<string, DerivedRegion> | null = null;
@ -87,9 +94,35 @@ export function coordsOf(event: { tags: string[][] }): { lat: number; lng: numbe
return null; return null;
} }
/** Best-effort reverse geocode via BigDataCloud. Never throws. */
async function geocodeClient(lat: number, lng: number): Promise<DerivedRegion> {
const empty: DerivedRegion = { city: '', country: '', countryCode: '' };
try {
const res = await fetch(
`https://api.bigdatacloud.net/data/reverse-geocode-client?latitude=${lat}&longitude=${lng}&localityLanguage=en`,
{ signal: AbortSignal.timeout(REQUEST_TIMEOUT_MS) },
);
if (!res.ok) return empty;
const data = (await res.json()) as {
city?: string;
locality?: string;
countryName?: string;
countryCode?: string;
};
return {
city: data.city || data.locality || '',
country: data.countryName || '',
countryCode: String(data.countryCode ?? '').trim().toUpperCase(),
};
} catch {
return empty;
}
}
/** /**
* Reverse-geocode a list of coordinates and fill the cache. Sequential and * Reverse-geocode a list of coordinates and fill the cache. Sequential and
* spaced for Nominatim's 1 req/s policy. Returns how many NEW regions were added. * lightly spaced. `onProgress` fires after each item (so a UI counter can tick).
* Returns how many NEW regions were added.
*/ */
export async function fillDerivedRegions( export async function fillDerivedRegions(
coords: Array<{ lat: number; lng: number }>, coords: Array<{ lat: number; lng: number }>,
@ -103,9 +136,9 @@ export async function fillDerivedRegions(
// Skip duplicates / already-cached pins, but still advance the progress bar. // Skip duplicates / already-cached pins, but still advance the progress bar.
if (!seen.has(key) && !getDerivedRegion(lat, lng)) { if (!seen.has(key) && !getDerivedRegion(lat, lng)) {
seen.add(key); seen.add(key);
const geo = await reverseGeocodeRegions(lat, lng); const geo = await geocodeClient(lat, lng);
if (geo.city || geo.country || geo.countryCode) { if (geo.city || geo.country || geo.countryCode) {
setDerivedRegion(lat, lng, { city: geo.city, country: geo.country, countryCode: geo.countryCode }); setDerivedRegion(lat, lng, geo);
filled += 1; filled += 1;
} }
if (i < coords.length - 1) { if (i < coords.length - 1) {

View file

@ -148,7 +148,7 @@ export default function PrivacyPage() {
</p> </p>
<ul className="list-disc pl-5 space-y-1 text-gray-400"> <ul className="list-disc pl-5 space-y-1 text-gray-400">
<li><strong className="text-white">Map tiles</strong> (OpenStreetMap) — see your IP and which area of the map you look at.</li> <li><strong className="text-white">Map tiles</strong> (OpenStreetMap) — see your IP and which area of the map you look at.</li>
<li><strong className="text-white">Spot/shop geocoding</strong> (OpenStreetMap <strong className="text-white">Nominatim</strong>) — when you create a spot or shop we send its coordinates to fill in the city/country tag, and when you open <strong className="text-white">Community Stats</strong> we may send the coordinates of pins that have no location tag yet, so they can be grouped by region. Each is cached on your device, so it happens at most once per pin. Nominatim sees your IP. These are the <em>pin you placed / public pin coordinates</em>, never your device's live GPS.</li> <li><strong className="text-white">Spot/shop geocoding</strong> — when you <em>create or edit</em> a spot or shop, its coordinates go to OpenStreetMap <strong className="text-white">Nominatim</strong> to fill in the city/country tag. When you open <strong className="text-white">Community Stats</strong>, the coordinates of pins that have no location tag go to <strong className="text-white">BigDataCloud</strong> so they can be grouped by region (cached on your device — at most once per pin). Both see your IP. These are the <em>pin you placed / public pin coordinates</em>, never your device's live GPS.</li>
<li><strong className="text-white">Bitcoin places</strong> (BTCMap, when you switch that layer on) — see your IP and the area you're viewing.</li> <li><strong className="text-white">Bitcoin places</strong> (BTCMap, when you switch that layer on) — see your IP and the area you're viewing.</li>
<li><strong className="text-white">Nostr relays</strong> — see your IP when the app reads or publishes.</li> <li><strong className="text-white">Nostr relays</strong> — see your IP when the app reads or publishes.</li>
<li><strong className="text-white">Cashu mints</strong> — see your IP whenever your wallet sends, receives or melts.</li> <li><strong className="text-white">Cashu mints</strong> — see your IP whenever your wallet sends, receives or melts.</li>