Merge 3c71c7e074 into 759652207f
This commit is contained in:
Malek
GitHub
commit
f0a22244b0
|
|
@ -5,6 +5,15 @@ of current runtime behavior, see [`docs/CURRENT-STATE.md`](docs/CURRENT-STATE.md
|
|||
|
||||
## [Unreleased]
|
||||
|
||||
### Added
|
||||
|
||||
- CCTV mesh now includes an **Ontario 511 (MTO)** city pack: ~250 public
|
||||
traffic cameras across Ontario (anchored on Toronto and Ottawa), fetched
|
||||
keylessly from the `511on.ca` developer API with frames served through the
|
||||
existing CCTV proxy. Licensed under the Open Government Licence – Ontario
|
||||
(attribution registered in the Data attribution popover). Toggle with
|
||||
`CCTV_ONTARIO_ENABLED` / cap with `CCTV_ONTARIO_MAX_SOURCES`.
|
||||
|
||||
### Fixed
|
||||
|
||||
- Mapped-site outages show their scheduled retry countdown and distinguish
|
||||
|
|
|
|||
|
|
@ -34,6 +34,7 @@ How to read this:
|
|||
| **City of Austin Open Data** | CCTV camera catalog + frames | City of Austin Open Data Terms of Use | "City of Austin, TX — data.austintexas.gov" |
|
||||
| **Caltrans (cwwp2.dot.ca.gov)** | CCTV camera catalogs + frames, California districts | Public Caltrans traffic camera data | "Caltrans — cwwp2.dot.ca.gov" (courtesy) |
|
||||
| **TfL Open Data (JamCams)** | CCTV camera catalog + frames, London | [TfL Open Data terms](https://tfl.gov.uk/info-for/open-data-users/) — attribution REQUIRED | "Powered by TfL Open Data. Contains OS data © Crown copyright and database rights" |
|
||||
| **Ontario 511 (MTO)** | CCTV camera catalog + frames, Ontario | [Open Government Licence – Ontario](https://www.ontario.ca/page/open-government-licence-ontario) — attribution REQUIRED; keyless developer API (throttled 10 calls/60 s) | "Contains information licensed under the Open Government Licence – Ontario" |
|
||||
| **GBFS (Lyft / BCycle)** | Bikeshare availability | Per-feed (attribution-only) | Credit the operator (e.g. Austin BCycle) + its `license_url` |
|
||||
| **Radio Browser** | Geolocated internet-radio station directory and station-level tags | Public-domain directory data under PDDL 1.0; individual broadcaster stream terms apply | "Radio Browser" plus a link to the selected broadcaster |
|
||||
| **Re:Earth Terrain** (Mapterhorn) | Terrain (keyless globe stacks — OSM etc. — + `/api/terrain/heights` ellipsoidal-height lookups) | Terrain mesh: CC BY 4.0; geoid: EGM2008 (NGA, public domain) | "Terrain (keyless globe stacks): Re:Earth Terrain / Mapterhorn (CC BY 4.0) / EGM2008 (NGA)" |
|
||||
|
|
@ -45,6 +46,7 @@ How to read this:
|
|||
- **adsb.lol flight fallback.** When OpenSky is unavailable and no last-good OpenSky response exists, the server requests a cached, capped 250 nm adsb.lol point snapshot around the current camera subpoint. This is regional observed context, not worldwide completeness; provenance is exposed in the Flights stats/context row. Military ICAOs remain reconciled through the existing dedicated military registry rather than duplicated.
|
||||
- **Launch Library 2.** `/api/launches` makes a server-side rolling-30-day query against the supported v2.3 detailed launch endpoint, caches successful responses for 15 minutes in memory and on disk, and serves the last successful response during a throttle or transient outage. Anonymous access is limited to 15 calls/hour; deployments can provide `LL2_API_TOKEN` for authenticated access. The Space Devs' published terms permit using and sharing the API data in any form, ask users not to forward it without adding value, disclaim complete accuracy, and encourage—but do not require—attribution. This app keeps a courtesy credit. Payload and stage/recovery records are shown only when supplied. Failed launches expose their source status and never receive fallback orbit geometry or a live/estimated marker. LL2 supplies launch context and event timing, not continuous ascent telemetry or live orbital state.
|
||||
- **TfL JamCams.** The camera list comes from the keyless `api.tfl.gov.uk` endpoint (an optional `TFL_APP_KEY` raises its rate limit); frames come from TfL's public S3 bucket. The "Powered by TfL Open Data" attribution is required by TfL's terms and is registered in the Data attribution popover.
|
||||
- **Ontario 511 (MTO).** The camera list comes from the keyless `511on.ca` developer API (`/api/v2/get/cameras`, throttled to 10 calls/60 s — the 15-minute catalog cache keeps hits far below that); each still frame is a direct image on `511on.ca` (JPEG, occasionally a PNG placeholder; CloudFront, ~20 s cache). Only enabled cameras with an official-host image URL are kept, distance-prioritized to the Toronto/Ottawa anchors and capped (`CCTV_ONTARIO_MAX_SOURCES`, default 250). The feed reports no camera heading (Direction is "Unknown" province-wide), so poses start from a low-confidence prior you calibrate. The "Open Government Licence – Ontario" attribution is required and registered in the Data attribution popover. Disable with `CCTV_ONTARIO_ENABLED=0`.
|
||||
- **Radio Browser.** `/api/radio/stations` discovers official API mirrors, makes bounded and coalesced healthy/geolocated HTTPS-station queries, caches the normalized public-domain directory for 45 minutes, and may serve the last good catalog for up to seven days during an outage. Refreshes must meet minimum accepted-query and station coverage before replacing a warm catalog; schema-valid responses whose rows all fail the product's health policy do not count as successful queries. A usable partial cold catalog is explicitly `DEGRADED`, and malformed or empty successful payloads are rejected atomically. Every directory and click-count request rejects redirects, validates all resolved addresses as globally routable (including reserved/documentation IPv4 and special/non-global IPv6 exclusions), and pins the TLS connection to a validated address. Only MP3/AAC non-HLS directory rows with public HTTPS stream targets are returned; favicons are intentionally omitted. Pressing play connects one browser audio element directly to the selected broadcaster and calls the directory's click counter through known-ID-only `POST /api/radio/click/:uuid`. GEV never proxies, caches, records, bundles, or redistributes audio. Radio Browser supplies station-level tags, not dependable current-song or upcoming-program metadata, so Radio filtering never claims either. Direct playback exposes the listener's IP address to the broadcaster, whose own stream terms apply.
|
||||
- **TomTom Traffic.** Optional and BYOK: without `TOMTOM_API_KEY` the traffic layer runs its built-in simulation and no TomTom data (or attribution) appears. With a key, flow vector tiles are fetched through the server-side `/api/tomtom` proxy (120 s cache + a daily tile-budget governor — `TOMTOM_DAILY_TILE_BUDGET`, default 40,000, a configurable application safety ceiling, not a guarantee of staying within TomTom's monthly free allowance; TomTom's [current pricing](https://docs.tomtom.com/pricing/) lists 200K free tile requests per month) and the "Traffic flow data © TomTom" credit is registered in the Data attribution popover the moment live mode activates. TomTom data is served live and cached only transiently (≤120 s TTL under `.gev-cache/`, gitignored) — it is not bundled or redistributed. One 23 KB point-in-time tile snapshot is committed as a decode-test fixture (`src/data/fixtures/`, © TomTom, never served to the app).
|
||||
- **Re:Earth Terrain.** Keyless (no API key). Used two ways: (1) `src/mapStackController.js` swaps in a `Cesium.CesiumTerrainProvider` pointed at Re:Earth's `cesium-mesh/ellipsoid` quantized-mesh endpoint for globe stacks without a Cesium ion token (e.g. OSM), replacing a flat `EllipsoidTerrainProvider`; falls back to the flat provider if the endpoint can't be reached. (2) The server-side `/api/terrain/heights` proxy (disk-cached, serve-stale) resolves per-point ellipsoidal ground height for entity placement. Both are best-effort with a keyless-safe fallback (bundled EGM96 geoid math) if Re:Earth is unreachable.
|
||||
|
|
|
|||
|
|
@ -0,0 +1,82 @@
|
|||
// normalizeOntario511Camera — the pure record→source mapping for the Ontario
|
||||
// 511 (MTO) CCTV pack. The feed reports no heading (Direction is "Unknown"
|
||||
// province-wide), so poses fall back to the low-confidence id-hash prior; only
|
||||
// enabled views whose still-image URL is on the official 511on.ca host are kept
|
||||
// (defense-in-depth origin pin). Fixture below is a real live-sampled record.
|
||||
import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { normalizeOntario511Camera } from '../../vite.config.js';
|
||||
|
||||
const TORONTO_CAMERA = Object.freeze({
|
||||
Id: 806,
|
||||
Source: 'City of Toronto',
|
||||
SourceId: 'Camera 9303',
|
||||
Roadway: 'Lake Shore Boulevard',
|
||||
Direction: 'Unknown',
|
||||
Latitude: 43.644334,
|
||||
Longitude: -79.375966,
|
||||
Location: 'Lake Shore Boulevard near Yonge Street',
|
||||
Views: [
|
||||
{ Id: 1457, Url: 'https://511on.ca/map/Cctv/1457', Status: 'Enabled', Description: '' },
|
||||
],
|
||||
});
|
||||
|
||||
test('normalizeOntario511Camera maps a live camera record to a canonical source', () => {
|
||||
const s = normalizeOntario511Camera(TORONTO_CAMERA);
|
||||
assert.ok(s, 'expected a normalized source');
|
||||
assert.equal(s.id, 'on-806');
|
||||
assert.equal(s.name, 'Lake Shore Boulevard near Yonge Street');
|
||||
assert.equal(s.city, 'Ontario');
|
||||
assert.equal(s.cityId, 'ontario');
|
||||
assert.equal(s.provider, 'Ontario 511 (MTO)');
|
||||
assert.equal(s.lat, 43.644334);
|
||||
assert.equal(s.lon, -79.375966);
|
||||
assert.equal(s.feedType, 'image');
|
||||
assert.equal(s.sourceKind, 'ontario511-open-data');
|
||||
assert.equal(s.url, 'https://511on.ca/map/Cctv/1457');
|
||||
assert.equal(s.snapshotUrl, 'https://511on.ca/map/Cctv/1457');
|
||||
assert.match(s.license, /Open Government Licence . Ontario/);
|
||||
// No heading signal in the feed → low-confidence id-hash prior in [0, 360).
|
||||
assert.equal(s.headingConfidence, 'low');
|
||||
assert.ok(Number.isFinite(s.headingDeg) && s.headingDeg >= 0 && s.headingDeg < 360);
|
||||
});
|
||||
|
||||
test('normalizeOntario511Camera skips records without finite coordinates', () => {
|
||||
assert.equal(normalizeOntario511Camera({ ...TORONTO_CAMERA, Latitude: null }), null);
|
||||
assert.equal(normalizeOntario511Camera({ ...TORONTO_CAMERA, Longitude: 'n/a' }), null);
|
||||
assert.equal(normalizeOntario511Camera(undefined), null);
|
||||
});
|
||||
|
||||
test('normalizeOntario511Camera requires an enabled, official-host view (origin pin)', () => {
|
||||
// Disabled view → skip.
|
||||
assert.equal(normalizeOntario511Camera({
|
||||
...TORONTO_CAMERA,
|
||||
Views: [{ Id: 1, Url: 'https://511on.ca/map/Cctv/1', Status: 'Disabled' }],
|
||||
}), null);
|
||||
// Enabled but off-host URL → skip (defense-in-depth against a spoofed catalog).
|
||||
assert.equal(normalizeOntario511Camera({
|
||||
...TORONTO_CAMERA,
|
||||
Views: [{ Id: 1, Url: 'https://evil.example.com/map/Cctv/1', Status: 'Enabled' }],
|
||||
}), null);
|
||||
// No views at all → skip.
|
||||
assert.equal(normalizeOntario511Camera({ ...TORONTO_CAMERA, Views: [] }), null);
|
||||
});
|
||||
|
||||
test('normalizeOntario511Camera prefers the first enabled official view among several', () => {
|
||||
const s = normalizeOntario511Camera({
|
||||
...TORONTO_CAMERA,
|
||||
Views: [
|
||||
{ Id: 1, Url: 'https://511on.ca/map/Cctv/1', Status: 'Disabled' },
|
||||
{ Id: 2, Url: 'https://evil.example.com/x', Status: 'Enabled' },
|
||||
{ Id: 3, Url: 'https://511on.ca/map/Cctv/3', Status: 'Enabled' },
|
||||
],
|
||||
});
|
||||
assert.equal(s.snapshotUrl, 'https://511on.ca/map/Cctv/3');
|
||||
});
|
||||
|
||||
test('normalizeOntario511Camera derives a name when Location is absent', () => {
|
||||
const roadwayOnly = normalizeOntario511Camera({ ...TORONTO_CAMERA, Location: '' });
|
||||
assert.equal(roadwayOnly.name, 'Lake Shore Boulevard — Ontario 511');
|
||||
const idOnly = normalizeOntario511Camera({ ...TORONTO_CAMERA, Location: '', Roadway: '' });
|
||||
assert.equal(idOnly.name, 'Ontario 511 Camera 806');
|
||||
});
|
||||
|
|
@ -127,6 +127,13 @@ export const DATA_CREDITS = [
|
|||
'<a href="https://tfl.gov.uk/info-for/open-data-users/" target="_blank" rel="noopener">Powered by TfL Open Data</a>. ' +
|
||||
'Contains OS data © Crown copyright and database rights.',
|
||||
},
|
||||
{
|
||||
key: 'ontario511-cctv',
|
||||
html:
|
||||
'CCTV cameras & frames (Ontario): ' +
|
||||
'<a href="https://511on.ca" target="_blank" rel="noopener">Ontario 511 (MTO)</a> — ' +
|
||||
'Contains information licensed under the Open Government Licence – Ontario.',
|
||||
},
|
||||
{
|
||||
key: 'gbfs',
|
||||
html: 'Bikeshare availability: GBFS operator feeds (e.g. Austin BCycle)',
|
||||
|
|
|
|||
125
vite.config.js
125
vite.config.js
|
|
@ -3543,6 +3543,17 @@ const TFL_JAMCAM_URL = 'https://api.tfl.gov.uk/Place/Type/JamCam';
|
|||
const TFL_IMAGE_ORIGIN = 'https://s3-eu-west-1.amazonaws.com/jamcams.tfl.gov.uk/';
|
||||
const DEFAULT_TFL_MAX_SOURCES = 250;
|
||||
const LONDON_CENTER = { lat: 51.5074, lon: -0.1278 };
|
||||
/** Ontario 511 (MTO) CCTV: one keyless province-wide list endpoint; each still
|
||||
* frame is a direct image on 511on.ca (JPEG, occasionally PNG). Open
|
||||
* Government Licence – Ontario. */
|
||||
const ONTARIO511_CCTV_URL = 'https://511on.ca/api/v2/get/cameras';
|
||||
const ONTARIO511_IMAGE_ORIGIN = 'https://511on.ca/map/Cctv/';
|
||||
const DEFAULT_ONTARIO511_MAX_SOURCES = 250;
|
||||
/** Prioritization anchors: the two largest Ontario metros the feed covers. */
|
||||
const ONTARIO_ANCHORS = [
|
||||
{ lat: 43.6532, lon: -79.3832 }, // Toronto
|
||||
{ lat: 45.4215, lon: -75.6972 }, // Ottawa
|
||||
];
|
||||
/** Camera CATALOGS change rarely; 15 min keeps multi-megabyte upstream list refetches (Austin rows.json + 4 Caltrans districts + TfL) infrequent. Frames are fetched per-request and are unaffected. */
|
||||
const CCTV_SOURCE_CACHE_MS = 15 * 60 * 1000;
|
||||
/** Per-provider catalog-fetch timeout. Bounds the worst-case refresh so one
|
||||
|
|
@ -4168,6 +4179,112 @@ async function loadTflSourcesFromOpenData() {
|
|||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Normalize one Ontario 511 camera record into a canonical CCTV source, or
|
||||
* null to skip it. Pure (no I/O) so it is unit-testable. Keeps only cameras
|
||||
* with finite coords and an ENABLED view whose still-image URL sits on the
|
||||
* official 511on.ca host (defense-in-depth origin pin, mirroring Caltrans/TfL:
|
||||
* the proxy only ever fetches catalog URLs, and this pins the catalog host).
|
||||
* Ontario's feed reports no camera heading (Direction is "Unknown" across the
|
||||
* whole province), so the pose starts from the id-hash prior at low confidence
|
||||
* — the client's one-shot ground snap + manual gizmo own the truth.
|
||||
*
|
||||
* @param {object} raw - One element of the /api/v2/get/cameras array.
|
||||
* @returns {object|null} Normalized source, or null when unusable.
|
||||
*/
|
||||
export function normalizeOntario511Camera(raw) {
|
||||
// Coordinates must be genuine finite numbers. Number()/toFiniteNumber would
|
||||
// coerce null and '' to 0 (a null-island "0,0"), so reject non-number input
|
||||
// outright — the Ontario feed always sends numeric lat/lon.
|
||||
const lat = raw?.Latitude;
|
||||
const lon = raw?.Longitude;
|
||||
if (typeof lat !== 'number' || !Number.isFinite(lat)
|
||||
|| typeof lon !== 'number' || !Number.isFinite(lon)) return null;
|
||||
|
||||
const views = Array.isArray(raw?.Views) ? raw.Views : [];
|
||||
const view = views.find((v) =>
|
||||
String(v?.Status).toLowerCase() === 'enabled'
|
||||
&& String(v?.Url || '').startsWith(ONTARIO511_IMAGE_ORIGIN));
|
||||
if (!view) return null;
|
||||
|
||||
const rawId = String(raw?.Id ?? '').trim();
|
||||
if (!rawId) return null;
|
||||
const cameraId = `on-${rawId}`;
|
||||
|
||||
// raw.Direction is a dedicated field, but the feed leaves it "Unknown"
|
||||
// province-wide; parse it defensively, then fall back to the id-hash prior.
|
||||
const heading = directionToHeading(raw?.Direction, true);
|
||||
const hasHeading = Number.isFinite(heading);
|
||||
const location = String(raw?.Location || '').trim();
|
||||
const roadway = String(raw?.Roadway || '').trim();
|
||||
const name = location || (roadway ? `${roadway} — Ontario 511` : `Ontario 511 Camera ${rawId}`);
|
||||
|
||||
return {
|
||||
id: cameraId,
|
||||
name,
|
||||
city: 'Ontario',
|
||||
cityId: 'ontario',
|
||||
provider: 'Ontario 511 (MTO)',
|
||||
lat,
|
||||
lon,
|
||||
headingDeg: hasHeading ? heading : fallbackHeadingFromId(cameraId),
|
||||
headingConfidence: hasHeading ? 'high' : 'low',
|
||||
// Same two fabricated pose personalities as Austin/Caltrans (design §1a):
|
||||
// RAW PRIOR starting points; the client's one-shot ground snap + manual
|
||||
// calibration own the truth.
|
||||
pitchDeg: hasHeading ? -24 : -18,
|
||||
fovDeg: hasHeading ? 56 : 44,
|
||||
rangeM: hasHeading ? 210 : 145,
|
||||
mountHeightM: hasHeading ? 10 : 8,
|
||||
groundElevationM: 100, // Southern-Ontario prior; one-shot snap corrects it.
|
||||
feedType: 'image',
|
||||
url: String(view.Url),
|
||||
snapshotUrl: String(view.Url),
|
||||
sourceKind: 'ontario511-open-data',
|
||||
license: 'Contains information licensed under the Open Government Licence – Ontario',
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Fetch Ontario 511 (MTO) CCTV cameras — one keyless, province-wide list
|
||||
* endpoint; each still frame is a direct image on 511on.ca (JPEG, occasionally
|
||||
* a PNG placeholder; CloudFront, ~20 s cache). Only enabled cameras with finite
|
||||
* coords and an official-host image
|
||||
* URL are kept, then distance-prioritized to the Toronto/Ottawa anchors. The
|
||||
* 15-min source cache keeps list hits far below the API's 10-calls/60 s throttle.
|
||||
* Attribution: "Open Government Licence – Ontario" (registered in
|
||||
* src/data/dataCredits.js). CCTV_ONTARIO_MAX_SOURCES caps the served count;
|
||||
* CCTV_ONTARIO_ENABLED=0 disables the pack.
|
||||
*
|
||||
* @returns {Promise<Array<object>>} Normalized camera source objects.
|
||||
*/
|
||||
async function loadOntario511SourcesFromOpenData() {
|
||||
try {
|
||||
const resp = await fetch(ONTARIO511_CCTV_URL, { headers: { Accept: 'application/json' }, signal: AbortSignal.timeout(CCTV_SOURCE_FETCH_TIMEOUT_MS) });
|
||||
if (!resp.ok) {
|
||||
console.warn('[CCTV] Ontario 511 download failed:', resp.status);
|
||||
return [];
|
||||
}
|
||||
const rows = await resp.json();
|
||||
if (!Array.isArray(rows)) return [];
|
||||
|
||||
const cameras = [];
|
||||
for (const raw of rows) {
|
||||
const camera = normalizeOntario511Camera(raw);
|
||||
if (camera) cameras.push(camera);
|
||||
}
|
||||
|
||||
const maxRaw = Number(process.env.CCTV_ONTARIO_MAX_SOURCES || DEFAULT_ONTARIO511_MAX_SOURCES);
|
||||
const maxCount = Number.isFinite(maxRaw) ? Math.max(8, Math.min(600, Math.floor(maxRaw))) : DEFAULT_ONTARIO511_MAX_SOURCES;
|
||||
const prioritized = prioritizeSources(cameras, maxCount, ONTARIO_ANCHORS);
|
||||
console.log(`[CCTV] Loaded Ontario 511 camera sources: ${cameras.length} enabled (using nearest ${prioritized.length})`);
|
||||
return prioritized;
|
||||
} catch (error) {
|
||||
console.warn('[CCTV] Ontario 511 download error:', error?.message || error);
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Normalize a raw CCTV source item into a canonical shape with safe defaults.
|
||||
*
|
||||
|
|
@ -4244,22 +4361,26 @@ async function refreshCctvSources() {
|
|||
// Austin-only fetch, now governing all three. Each pack fails independently.
|
||||
const needsLiveSources = forceAustin || ((fromFile.length + fromEnv.length) === 0 && preferAustin);
|
||||
const tflEnabled = String(process.env.CCTV_TFL_ENABLED || '1').trim() !== '0';
|
||||
const ontarioEnabled = String(process.env.CCTV_ONTARIO_ENABLED || '1').trim() !== '0';
|
||||
|
||||
let fromAustin = [];
|
||||
let fromCaltrans = [];
|
||||
let fromTfl = [];
|
||||
let fromOntario = [];
|
||||
if (needsLiveSources) {
|
||||
const [austinResult, caltransResult, tflResult] = await Promise.allSettled([
|
||||
const [austinResult, caltransResult, tflResult, ontarioResult] = await Promise.allSettled([
|
||||
loadAustinSourcesFromOpenData(),
|
||||
loadCaltransSourcesFromOpenData(),
|
||||
tflEnabled ? loadTflSourcesFromOpenData() : Promise.resolve([]),
|
||||
ontarioEnabled ? loadOntario511SourcesFromOpenData() : Promise.resolve([]),
|
||||
]);
|
||||
fromAustin = austinResult.status === 'fulfilled' ? austinResult.value : [];
|
||||
fromCaltrans = caltransResult.status === 'fulfilled' ? caltransResult.value : [];
|
||||
fromTfl = tflResult.status === 'fulfilled' ? tflResult.value : [];
|
||||
fromOntario = ontarioResult.status === 'fulfilled' ? ontarioResult.value : [];
|
||||
}
|
||||
// Live sources first so file/env overrides win on duplicate IDs (Map last-write).
|
||||
const merged = [...fromAustin, ...fromCaltrans, ...fromTfl, ...fromFile, ...fromEnv];
|
||||
const merged = [...fromAustin, ...fromCaltrans, ...fromTfl, ...fromOntario, ...fromFile, ...fromEnv];
|
||||
|
||||
// Deduplicate by camera ID (last-write wins because of Map.set)
|
||||
const byId = new Map();
|
||||
|
|
|
|||
Loading…
Reference in New Issue