mirror of
https://github.com/ThatGuySam/doesitarm.git
synced 2026-05-15 06:35:20 -07:00
docs(research): capture search migration findings
Save the Ubuntu 24 Stork migration notes and the Pagefind viability assessment so the production decision and source trail live in the repo.
This commit is contained in:
parent
1c2ce65f7d
commit
d6057857fb
2 changed files with 195 additions and 0 deletions
77
docs/research/netlify-ubuntu-24-stork-2026-03-15.md
Normal file
77
docs/research/netlify-ubuntu-24-stork-2026-03-15.md
Normal file
|
|
@ -0,0 +1,77 @@
|
||||||
|
# Netlify Ubuntu 24 Stork Migration
|
||||||
|
|
||||||
|
Date: 2026-03-15
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
Investigate why `doesitarm` fails on Netlify's Noble/Ubuntu 24 image and
|
||||||
|
identify whether the fix belongs in this repo or `../doesitarm-functions/`.
|
||||||
|
|
||||||
|
## Short Answer
|
||||||
|
|
||||||
|
The failing production build is blocked in the Stork setup stage, not in Astro
|
||||||
|
or the companion functions repo.
|
||||||
|
|
||||||
|
The immediate break is that `doesitarm` downloads Stork's
|
||||||
|
`stork-ubuntu-20-04` binary, which is linked against `libssl.so.1.1`. Netlify's
|
||||||
|
Noble/Ubuntu 24 image no longer ships that library, so the binary exits before
|
||||||
|
Astro starts building.
|
||||||
|
|
||||||
|
There is also a separate Node 22 regression in this repo: `isBrowserContext()`
|
||||||
|
uses `navigator` to detect browsers, but Node 22 exposes a global
|
||||||
|
`navigator`. That causes Node processes on macOS to be misdetected as browser
|
||||||
|
contexts and pushes local Stork downloads onto the Linux binary path.
|
||||||
|
|
||||||
|
## What The Evidence Says
|
||||||
|
|
||||||
|
- Confirmed from the user-provided Netlify build log:
|
||||||
|
`./stork-executable: error while loading shared libraries: libssl.so.1.1`
|
||||||
|
- Confirmed from the latest Stork release metadata:
|
||||||
|
`v1.6.0` ships both `stork-ubuntu-20-04` and `stork-ubuntu-22-04`.
|
||||||
|
- Confirmed from local binary inspection:
|
||||||
|
`stork-ubuntu-20-04` references `libssl.so.1.1` and `libcrypto.so.1.1`
|
||||||
|
while `stork-ubuntu-22-04` references `libssl.so.3` and `libcrypto.so.3`.
|
||||||
|
- Confirmed from Node 22 docs and local runtime checks:
|
||||||
|
Node 22 exposes a global `navigator`, so `typeof navigator !== 'undefined'`
|
||||||
|
is no longer a safe browser-only check.
|
||||||
|
- Confirmed locally on 2026-03-15:
|
||||||
|
`CI=1 mise exec node@22 -- pnpm run netlify-build` succeeds after switching
|
||||||
|
the Stork target and fixing runtime detection.
|
||||||
|
|
||||||
|
## What Works
|
||||||
|
|
||||||
|
- Use Stork's `stork-ubuntu-22-04` binary on Linux/Netlify.
|
||||||
|
- Use `stork-macos-13-arm` on Apple Silicon Macs.
|
||||||
|
- Detect browser context with `window` and `document`, not `navigator`.
|
||||||
|
- Keep the fix in `doesitarm`; `../doesitarm-functions/` is an external API
|
||||||
|
dependency referenced via `VFUNCTIONS_URL` and `PUBLIC_API_DOMAIN`, not part
|
||||||
|
of the failing Netlify build path.
|
||||||
|
|
||||||
|
## What To Avoid
|
||||||
|
|
||||||
|
- Do not keep using `stork-ubuntu-20-04` on Noble/Ubuntu 24.
|
||||||
|
- Do not use the `stork-amazon-linux` artifact as a Netlify fallback; its
|
||||||
|
binary references `libssl.so.10`, which is also not a fit for Ubuntu 24.
|
||||||
|
- Do not use `navigator` as the only browser-runtime signal in Node 22+ code.
|
||||||
|
|
||||||
|
## Recommendation
|
||||||
|
|
||||||
|
Keep the Stork fix minimal and repo-local:
|
||||||
|
|
||||||
|
1. Detect the Stork binary target from `process.platform` and `process.arch`.
|
||||||
|
2. Use `stork-ubuntu-22-04` for Linux builds.
|
||||||
|
3. Use `stork-macos-13-arm` for Apple Silicon.
|
||||||
|
4. Add tests for runtime detection and Stork target selection.
|
||||||
|
5. Leave `../doesitarm-functions/` unchanged unless its own deployment starts
|
||||||
|
failing independently.
|
||||||
|
|
||||||
|
## Source Links
|
||||||
|
|
||||||
|
- Stork latest release metadata:
|
||||||
|
https://api.github.com/repos/jameslittle230/stork/releases/latest
|
||||||
|
- Stork install docs:
|
||||||
|
https://stork-search.net/docs/install
|
||||||
|
- Stork CI/Netlify guide:
|
||||||
|
https://stork-search.net/docs/stork-and-netlify
|
||||||
|
- Node 22 globals docs:
|
||||||
|
https://nodejs.org/dist/latest-v22.x/docs/api/globals.html
|
||||||
118
docs/research/pagefind-viability-2026-03-15.md
Normal file
118
docs/research/pagefind-viability-2026-03-15.md
Normal file
|
|
@ -0,0 +1,118 @@
|
||||||
|
# Pagefind Viability For doesitarm
|
||||||
|
|
||||||
|
Date: 2026-03-15
|
||||||
|
|
||||||
|
## Scope
|
||||||
|
|
||||||
|
Investigate whether Pagefind is a good replacement for Stork in `doesitarm`,
|
||||||
|
given the current Astro 2 + Netlify server build and the existing custom search
|
||||||
|
pipeline.
|
||||||
|
|
||||||
|
## Short Answer
|
||||||
|
|
||||||
|
Pagefind is viable for this repo, but not as a drop-in replacement.
|
||||||
|
|
||||||
|
The lowest-risk production path today is to keep the Stork fix and ship it.
|
||||||
|
If `doesitarm` later moves to Pagefind, the right migration path is a
|
||||||
|
behind-feature-flag prototype using Pagefind's Node API with
|
||||||
|
`addCustomRecord()`, not a simple `pagefind --site dist` crawl.
|
||||||
|
|
||||||
|
## What The Evidence Says
|
||||||
|
|
||||||
|
- Current repo shape:
|
||||||
|
`doesitarm` builds with `output: "server"` in Astro and only prerenders a
|
||||||
|
small subset of routes (`/`, `/categories`, `/games`). Most searchable
|
||||||
|
listing pages are SSR routes, not static HTML files in `dist/`.
|
||||||
|
- Current Stork shape:
|
||||||
|
[helpers/stork/toml.js](/Users/athena/Code/doesitarm/helpers/stork/toml.js)
|
||||||
|
generates a structured index from sitemap payloads, and
|
||||||
|
[components/search-stork.vue](/Users/athena/Code/doesitarm/components/search-stork.vue)
|
||||||
|
renders a custom search UI over those records.
|
||||||
|
- Pagefind official docs:
|
||||||
|
the Node API supports `addDirectory()`, `addHTMLFile()`, and
|
||||||
|
`addCustomRecord()`. The docs explicitly describe `addCustomRecord()` as a
|
||||||
|
way to build an index from non-HTML content.
|
||||||
|
- Pagefind browser API:
|
||||||
|
supports custom JS search UIs, per-result lazy loading with `result.data()`,
|
||||||
|
excerpts, filters, and sorting.
|
||||||
|
- Pagefind latest release:
|
||||||
|
`v1.4.0`, published 2025-09-01.
|
||||||
|
- Relevant GitHub issues:
|
||||||
|
`#163` shows Astro + Netlify usage is workable but can need selector/root
|
||||||
|
troubleshooting.
|
||||||
|
`#198` and `#277` show demand for indexing non-HTML/custom data, which is now
|
||||||
|
covered by the Node API docs.
|
||||||
|
`#574` remains open for an `npx` wrapper failure on `ubuntu-latest`, which is
|
||||||
|
a reason to prefer a pinned dependency and explicit integration over a casual
|
||||||
|
CLI-only swap.
|
||||||
|
|
||||||
|
## What Works
|
||||||
|
|
||||||
|
- Pagefind can support this repo's filters and sorts.
|
||||||
|
- Pagefind can support a custom UI instead of the stock widget.
|
||||||
|
- Pagefind can index structured records directly with `addCustomRecord()`,
|
||||||
|
which matches `doesitarm` better than crawling built HTML.
|
||||||
|
- A feature-flagged migration is feasible:
|
||||||
|
1. build Pagefind assets from the existing sitemap payload data
|
||||||
|
2. expose them under `/pagefind/`
|
||||||
|
3. add a Pagefind-backed client alongside the existing Stork component
|
||||||
|
4. switch between them with a runtime/build flag
|
||||||
|
|
||||||
|
## What To Avoid
|
||||||
|
|
||||||
|
- Do not treat Pagefind as a trivial `postbuild` swap in this repo.
|
||||||
|
A plain HTML crawl would miss most of the real searchable surface because the
|
||||||
|
site is primarily SSR on Netlify.
|
||||||
|
- Do not attempt the production migration by replacing Stork first and figuring
|
||||||
|
out the UI later.
|
||||||
|
- Do not rely on `npx pagefind` alone in CI without pinning and testing the
|
||||||
|
binary/package path on the target image.
|
||||||
|
|
||||||
|
## Recommendation
|
||||||
|
|
||||||
|
For production now:
|
||||||
|
|
||||||
|
1. Ship the Stork Ubuntu 24 fix.
|
||||||
|
2. Merge that branch to `master`.
|
||||||
|
3. verify the Netlify deploy is green.
|
||||||
|
|
||||||
|
For a Pagefind migration later:
|
||||||
|
|
||||||
|
1. Add `pagefind` as a pinned dependency.
|
||||||
|
2. Create a build script that maps the same sitemap payloads into
|
||||||
|
`addCustomRecord()` calls.
|
||||||
|
3. Write Pagefind output to `static/pagefind/` or `dist/pagefind/`.
|
||||||
|
4. Add a feature flag that swaps the current Stork client for a Pagefind
|
||||||
|
adapter in the search UI.
|
||||||
|
5. Only remove Stork after the Pagefind path has parity on excerpts, filters,
|
||||||
|
and result URLs.
|
||||||
|
|
||||||
|
Inference:
|
||||||
|
Pagefind is probably the cleaner long-term search engine here, but because this
|
||||||
|
repo already has a data-first indexing pipeline, the migration cost is more
|
||||||
|
about adapter work than about search quality.
|
||||||
|
|
||||||
|
## Source Links
|
||||||
|
|
||||||
|
- Pagefind repo:
|
||||||
|
https://github.com/Pagefind/pagefind
|
||||||
|
- Pagefind latest release:
|
||||||
|
https://github.com/Pagefind/pagefind/releases/tag/v1.4.0
|
||||||
|
- Pagefind Node API docs:
|
||||||
|
https://pagefind.app/docs/node-api/
|
||||||
|
- Pagefind browser API docs:
|
||||||
|
https://pagefind.app/docs/api/
|
||||||
|
- Pagefind filtering docs:
|
||||||
|
https://pagefind.app/docs/filtering/
|
||||||
|
- Pagefind sorts docs:
|
||||||
|
https://pagefind.app/docs/sorts/
|
||||||
|
- Pagefind issue `#163`:
|
||||||
|
https://github.com/Pagefind/pagefind/issues/163
|
||||||
|
- Pagefind issue `#198`:
|
||||||
|
https://github.com/Pagefind/pagefind/issues/198
|
||||||
|
- Pagefind issue `#277`:
|
||||||
|
https://github.com/Pagefind/pagefind/issues/277
|
||||||
|
- Pagefind issue `#574`:
|
||||||
|
https://github.com/Pagefind/pagefind/issues/574
|
||||||
|
- HN result set for Pagefind launches:
|
||||||
|
https://hn.algolia.com/?q=Pagefind
|
||||||
Loading…
Add table
Add a link
Reference in a new issue