NEXUS Wayback Machine

Technical summary

NEXUS OSINT plugin for website history via the Wayback Machine (Internet Archive): latest snapshot lookup, capture listing through the CDX API with date filters, a yearly visual timeline, batch checks of up to 50 domains, CSV export and query history — with 7-day local caching and real-time service status.

Executed scope

• backend.php backend (461 lines): WAYBACK_CACHE_TTL = 604800 (7 days); no authentication — public Internet Archive APIs.
• Tables (created by wayback_install() on load): wayback_cache (id, kind, query_key VARCHAR(512), status, http_code, data LONGTEXT, error, task_id, updated_at; UNIQUE uk_kind_key (kind, query_key(255)), idx_updated_at) and wayback_history (id, query_type, query_val VARCHAR(512), status, task_id, created_at; idx_created_at).
• Cache: wayback_cache_get/put — upsert per kind+key with 7-day TTL; every query is also logged to history; the force parameter bypasses cache.
• HTTP: wayback_http_get — 5s connect / 12s timeout (20s on CDX queries), Accept: application/json and up to 5 redirects.
• wayback_check — GET archive.org/wayback/available?url=; returns the latest snapshot (archived_snapshots.closest with url, timestamp and status).
• wayback_snapshots — CDX web.archive.org/cdx/search/cdx with output=json, fl=timestamp,original,statuscode,mimetype,digest, collapse=timestamp:8 (1 capture per day), filter=statuscode:200, from/to and limit (default 50); CDX JSON parsing with raw fallback.
• wayback_timeline — CDX limit=500 with collapse=timestamp:6 and statuscode:200 filter; groups by year (total, first_seen, last_seen) exposing total_years; each year links to web.archive.org/web/{year}*/{url}.
• wayback_bulk_check — up to 50 domains per call; reuses wayback_check (cache-aware) returning domain, available, timestamp, url, status and cached per domain.
• wayback_status — GET (not HEAD) on wayback/available?url=example.com (4s connect / 6s timeout); services.wayback_available and services.wayback_cdx (online = 2xx) + has_sample_snapshot.
• wayback_history — page and limit ≤ 50, ordered by id DESC (query_type, query_val, status, task_id, created_at).
• API: 6 actions via PluginManager::registerApiAction (wayback_status, wayback_check, wayback_snapshots, wayback_timeline, wayback_bulk_check, wayback_history) — no placeholders.
• No Intelligence Hub enricher — on-demand lookup plugin (direct OSINT, no automatic enrichment signals).
• Alpine.js UI (tab.php, 690 lines): cyan theme; API status badge (pulsing green / red); Lookup tab (latest snapshot with "View" link and dd/mm/yyyy hh:mm timestamps), Snapshots (date filters + limit, timestamp/date/original URL/MIME/status table and CSV export), Timeline (yearly bars with capture totals and first_seen/last_seen), Batch (up to 50 domains, available/no-snapshot badge, cache indicator and CSV) and History (type/query/status/date); raw JSON toggle.
• Fastr: 3 commands — /wayback <url> (url), /wayback-snaps <url> (url), /wayback-timeline <url> (url); all arg_required=true.
• No cronjob.json — on-demand plugin with 7-day cache.

Stack and tools

• PHP 8 (no framework)
• MySQL 8 (wayback_cache + wayback_history)
• Alpine.js + Tailwind CSS (premium light, cyan ring)
• NEXUS plugin system (PluginManager)
• cURL → archive.org Wayback Machine (Availability API + CDX API)

Operational tags

• OSINT
• Internet Archive
• Wayback Machine
• CDX
• Snapshots
• Digital Preservation
• Plugin NEXUS

Operational outcome

• Access to the largest web history archive (800B+ pages since 1996): quick domain presence check and a direct link to the latest replay.
• Snapshot listing with date filters and CSV export for offline analysis of a site's evolution.
• Yearly timeline that reveals, in seconds, since when and how often a domain has been archived.
• Batch checks of up to 50 domains for OSINT triage (e.g. spotting squatting or abandoned domains still archived).
• 7-day cache: repeated lookups do not re-hit the Internet Archive APIs and the query history stays auditable.