docs(research): capture search migration findings

Save the Ubuntu 24 Stork migration notes and the Pagefind viability assessment so the production decision and source trail live in the repo.
This commit is contained in:
ThatGuySam 2026-03-15 11:58:22 -05:00
parent 1c2ce65f7d
commit d6057857fb
2 changed files with 195 additions and 0 deletions

View file

@ -0,0 +1,77 @@
# Netlify Ubuntu 24 Stork Migration
Date: 2026-03-15
## Scope
Investigate why `doesitarm` fails on Netlify's Noble/Ubuntu 24 image and
identify whether the fix belongs in this repo or `../doesitarm-functions/`.
## Short Answer
The failing production build is blocked in the Stork setup stage, not in Astro
or the companion functions repo.
The immediate break is that `doesitarm` downloads Stork's
`stork-ubuntu-20-04` binary, which is linked against `libssl.so.1.1`. Netlify's
Noble/Ubuntu 24 image no longer ships that library, so the binary exits before
Astro starts building.
There is also a separate Node 22 regression in this repo: `isBrowserContext()`
uses `navigator` to detect browsers, but Node 22 exposes a global
`navigator`. That causes Node processes on macOS to be misdetected as browser
contexts and pushes local Stork downloads onto the Linux binary path.
## What The Evidence Says
- Confirmed from the user-provided Netlify build log:
`./stork-executable: error while loading shared libraries: libssl.so.1.1`
- Confirmed from the latest Stork release metadata:
`v1.6.0` ships both `stork-ubuntu-20-04` and `stork-ubuntu-22-04`.
- Confirmed from local binary inspection:
`stork-ubuntu-20-04` references `libssl.so.1.1` and `libcrypto.so.1.1`
while `stork-ubuntu-22-04` references `libssl.so.3` and `libcrypto.so.3`.
- Confirmed from Node 22 docs and local runtime checks:
Node 22 exposes a global `navigator`, so `typeof navigator !== 'undefined'`
is no longer a safe browser-only check.
- Confirmed locally on 2026-03-15:
`CI=1 mise exec node@22 -- pnpm run netlify-build` succeeds after switching
the Stork target and fixing runtime detection.
## What Works
- Use Stork's `stork-ubuntu-22-04` binary on Linux/Netlify.
- Use `stork-macos-13-arm` on Apple Silicon Macs.
- Detect browser context with `window` and `document`, not `navigator`.
- Keep the fix in `doesitarm`; `../doesitarm-functions/` is an external API
dependency referenced via `VFUNCTIONS_URL` and `PUBLIC_API_DOMAIN`, not part
of the failing Netlify build path.
## What To Avoid
- Do not keep using `stork-ubuntu-20-04` on Noble/Ubuntu 24.
- Do not use the `stork-amazon-linux` artifact as a Netlify fallback; its
binary references `libssl.so.10`, which is also not a fit for Ubuntu 24.
- Do not use `navigator` as the only browser-runtime signal in Node 22+ code.
## Recommendation
Keep the Stork fix minimal and repo-local:
1. Detect the Stork binary target from `process.platform` and `process.arch`.
2. Use `stork-ubuntu-22-04` for Linux builds.
3. Use `stork-macos-13-arm` for Apple Silicon.
4. Add tests for runtime detection and Stork target selection.
5. Leave `../doesitarm-functions/` unchanged unless its own deployment starts
failing independently.
## Source Links
- Stork latest release metadata:
https://api.github.com/repos/jameslittle230/stork/releases/latest
- Stork install docs:
https://stork-search.net/docs/install
- Stork CI/Netlify guide:
https://stork-search.net/docs/stork-and-netlify
- Node 22 globals docs:
https://nodejs.org/dist/latest-v22.x/docs/api/globals.html

View file

@ -0,0 +1,118 @@
# Pagefind Viability For doesitarm
Date: 2026-03-15
## Scope
Investigate whether Pagefind is a good replacement for Stork in `doesitarm`,
given the current Astro 2 + Netlify server build and the existing custom search
pipeline.
## Short Answer
Pagefind is viable for this repo, but not as a drop-in replacement.
The lowest-risk production path today is to keep the Stork fix and ship it.
If `doesitarm` later moves to Pagefind, the right migration path is a
behind-feature-flag prototype using Pagefind's Node API with
`addCustomRecord()`, not a simple `pagefind --site dist` crawl.
## What The Evidence Says
- Current repo shape:
`doesitarm` builds with `output: "server"` in Astro and only prerenders a
small subset of routes (`/`, `/categories`, `/games`). Most searchable
listing pages are SSR routes, not static HTML files in `dist/`.
- Current Stork shape:
[helpers/stork/toml.js](/Users/athena/Code/doesitarm/helpers/stork/toml.js)
generates a structured index from sitemap payloads, and
[components/search-stork.vue](/Users/athena/Code/doesitarm/components/search-stork.vue)
renders a custom search UI over those records.
- Pagefind official docs:
the Node API supports `addDirectory()`, `addHTMLFile()`, and
`addCustomRecord()`. The docs explicitly describe `addCustomRecord()` as a
way to build an index from non-HTML content.
- Pagefind browser API:
supports custom JS search UIs, per-result lazy loading with `result.data()`,
excerpts, filters, and sorting.
- Pagefind latest release:
`v1.4.0`, published 2025-09-01.
- Relevant GitHub issues:
`#163` shows Astro + Netlify usage is workable but can need selector/root
troubleshooting.
`#198` and `#277` show demand for indexing non-HTML/custom data, which is now
covered by the Node API docs.
`#574` remains open for an `npx` wrapper failure on `ubuntu-latest`, which is
a reason to prefer a pinned dependency and explicit integration over a casual
CLI-only swap.
## What Works
- Pagefind can support this repo's filters and sorts.
- Pagefind can support a custom UI instead of the stock widget.
- Pagefind can index structured records directly with `addCustomRecord()`,
which matches `doesitarm` better than crawling built HTML.
- A feature-flagged migration is feasible:
1. build Pagefind assets from the existing sitemap payload data
2. expose them under `/pagefind/`
3. add a Pagefind-backed client alongside the existing Stork component
4. switch between them with a runtime/build flag
## What To Avoid
- Do not treat Pagefind as a trivial `postbuild` swap in this repo.
A plain HTML crawl would miss most of the real searchable surface because the
site is primarily SSR on Netlify.
- Do not attempt the production migration by replacing Stork first and figuring
out the UI later.
- Do not rely on `npx pagefind` alone in CI without pinning and testing the
binary/package path on the target image.
## Recommendation
For production now:
1. Ship the Stork Ubuntu 24 fix.
2. Merge that branch to `master`.
3. verify the Netlify deploy is green.
For a Pagefind migration later:
1. Add `pagefind` as a pinned dependency.
2. Create a build script that maps the same sitemap payloads into
`addCustomRecord()` calls.
3. Write Pagefind output to `static/pagefind/` or `dist/pagefind/`.
4. Add a feature flag that swaps the current Stork client for a Pagefind
adapter in the search UI.
5. Only remove Stork after the Pagefind path has parity on excerpts, filters,
and result URLs.
Inference:
Pagefind is probably the cleaner long-term search engine here, but because this
repo already has a data-first indexing pipeline, the migration cost is more
about adapter work than about search quality.
## Source Links
- Pagefind repo:
https://github.com/Pagefind/pagefind
- Pagefind latest release:
https://github.com/Pagefind/pagefind/releases/tag/v1.4.0
- Pagefind Node API docs:
https://pagefind.app/docs/node-api/
- Pagefind browser API docs:
https://pagefind.app/docs/api/
- Pagefind filtering docs:
https://pagefind.app/docs/filtering/
- Pagefind sorts docs:
https://pagefind.app/docs/sorts/
- Pagefind issue `#163`:
https://github.com/Pagefind/pagefind/issues/163
- Pagefind issue `#198`:
https://github.com/Pagefind/pagefind/issues/198
- Pagefind issue `#277`:
https://github.com/Pagefind/pagefind/issues/277
- Pagefind issue `#574`:
https://github.com/Pagefind/pagefind/issues/574
- HN result set for Pagefind launches:
https://hn.algolia.com/?q=Pagefind