diff --git a/docs/research/netlify-ubuntu-24-stork-2026-03-15.md b/docs/research/netlify-ubuntu-24-stork-2026-03-15.md new file mode 100644 index 0000000..00f15b6 --- /dev/null +++ b/docs/research/netlify-ubuntu-24-stork-2026-03-15.md @@ -0,0 +1,77 @@ +# Netlify Ubuntu 24 Stork Migration + +Date: 2026-03-15 + +## Scope + +Investigate why `doesitarm` fails on Netlify's Noble/Ubuntu 24 image and +identify whether the fix belongs in this repo or `../doesitarm-functions/`. + +## Short Answer + +The failing production build is blocked in the Stork setup stage, not in Astro +or the companion functions repo. + +The immediate break is that `doesitarm` downloads Stork's +`stork-ubuntu-20-04` binary, which is linked against `libssl.so.1.1`. Netlify's +Noble/Ubuntu 24 image no longer ships that library, so the binary exits before +Astro starts building. + +There is also a separate Node 22 regression in this repo: `isBrowserContext()` +uses `navigator` to detect browsers, but Node 22 exposes a global +`navigator`. That causes Node processes on macOS to be misdetected as browser +contexts and pushes local Stork downloads onto the Linux binary path. + +## What The Evidence Says + +- Confirmed from the user-provided Netlify build log: + `./stork-executable: error while loading shared libraries: libssl.so.1.1` +- Confirmed from the latest Stork release metadata: + `v1.6.0` ships both `stork-ubuntu-20-04` and `stork-ubuntu-22-04`. +- Confirmed from local binary inspection: + `stork-ubuntu-20-04` references `libssl.so.1.1` and `libcrypto.so.1.1` + while `stork-ubuntu-22-04` references `libssl.so.3` and `libcrypto.so.3`. +- Confirmed from Node 22 docs and local runtime checks: + Node 22 exposes a global `navigator`, so `typeof navigator !== 'undefined'` + is no longer a safe browser-only check. +- Confirmed locally on 2026-03-15: + `CI=1 mise exec node@22 -- pnpm run netlify-build` succeeds after switching + the Stork target and fixing runtime detection. + +## What Works + +- Use Stork's `stork-ubuntu-22-04` binary on Linux/Netlify. +- Use `stork-macos-13-arm` on Apple Silicon Macs. +- Detect browser context with `window` and `document`, not `navigator`. +- Keep the fix in `doesitarm`; `../doesitarm-functions/` is an external API + dependency referenced via `VFUNCTIONS_URL` and `PUBLIC_API_DOMAIN`, not part + of the failing Netlify build path. + +## What To Avoid + +- Do not keep using `stork-ubuntu-20-04` on Noble/Ubuntu 24. +- Do not use the `stork-amazon-linux` artifact as a Netlify fallback; its + binary references `libssl.so.10`, which is also not a fit for Ubuntu 24. +- Do not use `navigator` as the only browser-runtime signal in Node 22+ code. + +## Recommendation + +Keep the Stork fix minimal and repo-local: + +1. Detect the Stork binary target from `process.platform` and `process.arch`. +2. Use `stork-ubuntu-22-04` for Linux builds. +3. Use `stork-macos-13-arm` for Apple Silicon. +4. Add tests for runtime detection and Stork target selection. +5. Leave `../doesitarm-functions/` unchanged unless its own deployment starts + failing independently. + +## Source Links + +- Stork latest release metadata: + https://api.github.com/repos/jameslittle230/stork/releases/latest +- Stork install docs: + https://stork-search.net/docs/install +- Stork CI/Netlify guide: + https://stork-search.net/docs/stork-and-netlify +- Node 22 globals docs: + https://nodejs.org/dist/latest-v22.x/docs/api/globals.html diff --git a/docs/research/pagefind-viability-2026-03-15.md b/docs/research/pagefind-viability-2026-03-15.md new file mode 100644 index 0000000..69c2a27 --- /dev/null +++ b/docs/research/pagefind-viability-2026-03-15.md @@ -0,0 +1,118 @@ +# Pagefind Viability For doesitarm + +Date: 2026-03-15 + +## Scope + +Investigate whether Pagefind is a good replacement for Stork in `doesitarm`, +given the current Astro 2 + Netlify server build and the existing custom search +pipeline. + +## Short Answer + +Pagefind is viable for this repo, but not as a drop-in replacement. + +The lowest-risk production path today is to keep the Stork fix and ship it. +If `doesitarm` later moves to Pagefind, the right migration path is a +behind-feature-flag prototype using Pagefind's Node API with +`addCustomRecord()`, not a simple `pagefind --site dist` crawl. + +## What The Evidence Says + +- Current repo shape: + `doesitarm` builds with `output: "server"` in Astro and only prerenders a + small subset of routes (`/`, `/categories`, `/games`). Most searchable + listing pages are SSR routes, not static HTML files in `dist/`. +- Current Stork shape: + [helpers/stork/toml.js](/Users/athena/Code/doesitarm/helpers/stork/toml.js) + generates a structured index from sitemap payloads, and + [components/search-stork.vue](/Users/athena/Code/doesitarm/components/search-stork.vue) + renders a custom search UI over those records. +- Pagefind official docs: + the Node API supports `addDirectory()`, `addHTMLFile()`, and + `addCustomRecord()`. The docs explicitly describe `addCustomRecord()` as a + way to build an index from non-HTML content. +- Pagefind browser API: + supports custom JS search UIs, per-result lazy loading with `result.data()`, + excerpts, filters, and sorting. +- Pagefind latest release: + `v1.4.0`, published 2025-09-01. +- Relevant GitHub issues: + `#163` shows Astro + Netlify usage is workable but can need selector/root + troubleshooting. + `#198` and `#277` show demand for indexing non-HTML/custom data, which is now + covered by the Node API docs. + `#574` remains open for an `npx` wrapper failure on `ubuntu-latest`, which is + a reason to prefer a pinned dependency and explicit integration over a casual + CLI-only swap. + +## What Works + +- Pagefind can support this repo's filters and sorts. +- Pagefind can support a custom UI instead of the stock widget. +- Pagefind can index structured records directly with `addCustomRecord()`, + which matches `doesitarm` better than crawling built HTML. +- A feature-flagged migration is feasible: + 1. build Pagefind assets from the existing sitemap payload data + 2. expose them under `/pagefind/` + 3. add a Pagefind-backed client alongside the existing Stork component + 4. switch between them with a runtime/build flag + +## What To Avoid + +- Do not treat Pagefind as a trivial `postbuild` swap in this repo. + A plain HTML crawl would miss most of the real searchable surface because the + site is primarily SSR on Netlify. +- Do not attempt the production migration by replacing Stork first and figuring + out the UI later. +- Do not rely on `npx pagefind` alone in CI without pinning and testing the + binary/package path on the target image. + +## Recommendation + +For production now: + +1. Ship the Stork Ubuntu 24 fix. +2. Merge that branch to `master`. +3. verify the Netlify deploy is green. + +For a Pagefind migration later: + +1. Add `pagefind` as a pinned dependency. +2. Create a build script that maps the same sitemap payloads into + `addCustomRecord()` calls. +3. Write Pagefind output to `static/pagefind/` or `dist/pagefind/`. +4. Add a feature flag that swaps the current Stork client for a Pagefind + adapter in the search UI. +5. Only remove Stork after the Pagefind path has parity on excerpts, filters, + and result URLs. + +Inference: +Pagefind is probably the cleaner long-term search engine here, but because this +repo already has a data-first indexing pipeline, the migration cost is more +about adapter work than about search quality. + +## Source Links + +- Pagefind repo: + https://github.com/Pagefind/pagefind +- Pagefind latest release: + https://github.com/Pagefind/pagefind/releases/tag/v1.4.0 +- Pagefind Node API docs: + https://pagefind.app/docs/node-api/ +- Pagefind browser API docs: + https://pagefind.app/docs/api/ +- Pagefind filtering docs: + https://pagefind.app/docs/filtering/ +- Pagefind sorts docs: + https://pagefind.app/docs/sorts/ +- Pagefind issue `#163`: + https://github.com/Pagefind/pagefind/issues/163 +- Pagefind issue `#198`: + https://github.com/Pagefind/pagefind/issues/198 +- Pagefind issue `#277`: + https://github.com/Pagefind/pagefind/issues/277 +- Pagefind issue `#574`: + https://github.com/Pagefind/pagefind/issues/574 +- HN result set for Pagefind launches: + https://hn.algolia.com/?q=Pagefind diff --git a/helpers/environment.js b/helpers/environment.js index 5b081a5..e9a77ba 100644 --- a/helpers/environment.js +++ b/helpers/environment.js @@ -6,7 +6,10 @@ export function isNuxt( VueThis ) { } export function isBrowserContext () { - if ( typeof navigator === 'undefined' ) return false + // Node 22 exposes a global navigator, so use window/document instead. + if ( typeof window === 'undefined' ) return false + + if ( typeof document === 'undefined' ) return false return true } @@ -14,6 +17,8 @@ export function isBrowserContext () { export function hasProcesGlobal () { if ( typeof process === 'undefined' ) return false + if ( !process.versions?.node ) return false + return true } diff --git a/helpers/stork/executable.js b/helpers/stork/executable.js index e402943..cc86b65 100644 --- a/helpers/stork/executable.js +++ b/helpers/stork/executable.js @@ -2,7 +2,6 @@ import fs from 'fs-extra' import execa from 'execa' -import { isDarwin } from '~/helpers/environment.js' import { storkVersion, storkExecutableName, @@ -11,15 +10,25 @@ import { storkIndexPath } from '~/helpers/stork/config.js' +// Netlify's Ubuntu 24 (Noble) image needs the OpenSSL 3 compatible binary. +export function getStorkExecutableTarget ( { + platform = process.platform, + arch = process.arch +} = {} ) { + if ( platform === 'darwin' ) { + if ( arch === 'arm64' ) return 'stork-macos-13-arm' + + return 'stork-macos-10-15' + } + + return 'stork-ubuntu-22-04' +} + // https://stork-search.net/docs/install -const execDownloadUrls = { - darwin: `https://files.stork-search.net/releases/v${ storkVersion }/stork-macos-10-15`, - default: `https://files.stork-search.net/releases/v${ storkVersion }/stork-ubuntu-20-04` +export function getStorkExecutableDownloadUrl ( options = {} ) { + const target = getStorkExecutableTarget( options ) - // Stork 2.0 - // darwin: `https://files.stork-search.net/releases/v${ storkVersion }/stork-macos-12`, - - // default: `https://files.stork-search.net/releases/v${ storkVersion }/stork-amazon-linux` + return `https://files.stork-search.net/releases/v${ storkVersion }/${ target }` } // Check if a file is executable @@ -33,9 +42,7 @@ async function isExecutable ( path ) { // 👩‍💻 Bash Download example - https://github.com/jmooring/hugo-stork/blob/main/build.sh export async function downloadStorkExecutable () { - const envKey = isDarwin() ? 'darwin' : 'default' - - const execDownloadUrl = execDownloadUrls[ envKey ] + const execDownloadUrl = getStorkExecutableDownloadUrl() // console.log( { execDownloadUrl } ) @@ -46,6 +53,7 @@ export async function downloadStorkExecutable () { // Download the binary await execa( `curl`, [ + '-fsSL', execDownloadUrl, // Set filename @@ -60,7 +68,7 @@ export async function downloadStorkExecutable () { // console.log( 'isExecutable', isExecutable ) - if ( !isExecutable( storkExecutablePath ) ) throw new Error( `Downloaded binary at ${ storkExecutablePath } is not executable.` ) + if ( !(await isExecutable( storkExecutablePath )) ) throw new Error( `Downloaded binary at ${ storkExecutablePath } is not executable.` ) // Check Stork version @@ -78,7 +86,7 @@ export async function downloadStorkExecutable () { export async function buildIndex () { - if ( !isExecutable( storkExecutablePath ) ) throw new Error( `Binary at ${ storkExecutablePath } is not executable.` ) + if ( !(await isExecutable( storkExecutablePath )) ) throw new Error( `Binary at ${ storkExecutablePath } is not executable.` ) // Check Stork version // so we know our binary is working diff --git a/scripts/stork-netlify.sh b/scripts/stork-netlify.sh index 4e5f52b..714a28b 100755 --- a/scripts/stork-netlify.sh +++ b/scripts/stork-netlify.sh @@ -2,11 +2,11 @@ # Hugo Bash Example https://github.com/jmooring/hugo-stork/blob/main/build.sh -# curl https://files.stork-search.net/releases/latest/stork-amazon-linux -o stork-executable -# curl https://files.stork-search.net/releases/v1.4.3/stork-macos-latest -o stork-executable +# Netlify's Noble/Ubuntu 24 image needs the Ubuntu 22.04 Stork build. +# curl -fsSL https://files.stork-search.net/releases/v1.6.0/stork-macos-13-arm -o stork-executable -curl https://files.stork-search.net/releases/v1.4.2/stork-amazon-linux -o stork-executable -# curl https://files.stork-search.net/releases/v1.4.2/stork-macos-10-15 -o stork-executable +curl -fsSL https://files.stork-search.net/releases/v1.6.0/stork-ubuntu-22-04 -o stork-executable +# curl -fsSL https://files.stork-search.net/releases/v1.6.0/stork-macos-10-15 -o stork-executable chmod +x stork-executable ./stork-executable build --input static/stork.toml --output static/search-index.st diff --git a/test/prebuild/environment.test.js b/test/prebuild/environment.test.js new file mode 100644 index 0000000..e3be839 --- /dev/null +++ b/test/prebuild/environment.test.js @@ -0,0 +1,24 @@ +import { describe, expect, it } from 'vitest' + +import { + isBrowserContext, + isDarwin, + isLinux +} from '~/helpers/environment.js' + +describe( 'environment helpers', () => { + it( 'does not treat Node 22 navigator as a browser runtime', () => { + expect( isBrowserContext() ).toBe( false ) + }) + + it( 'detects darwin directly from process.platform', () => { + expect( isDarwin() ).toBe( process.platform === 'darwin' ) + }) + + it( 'detects linux-like runtimes directly from process.platform', () => { + expect( isLinux() ).toBe([ + 'linux', + 'openbsd' + ].includes( process.platform ) ) + }) +}) diff --git a/test/prebuild/stork-executable.test.js b/test/prebuild/stork-executable.test.js new file mode 100644 index 0000000..3a44c1e --- /dev/null +++ b/test/prebuild/stork-executable.test.js @@ -0,0 +1,29 @@ +import { describe, expect, it } from 'vitest' + +import { + getStorkExecutableTarget, + getStorkExecutableDownloadUrl +} from '~/helpers/stork/executable.js' + +describe( 'stork executable selection', () => { + it( 'uses the Ubuntu 22.04 binary for Linux builds', () => { + expect( getStorkExecutableTarget({ + platform: 'linux', + arch: 'x64' + }) ).toBe( 'stork-ubuntu-22-04' ) + }) + + it( 'uses the Apple Silicon macOS binary on arm64 Macs', () => { + expect( getStorkExecutableTarget({ + platform: 'darwin', + arch: 'arm64' + }) ).toBe( 'stork-macos-13-arm' ) + }) + + it( 'builds the download URL from the selected target', () => { + expect( getStorkExecutableDownloadUrl({ + platform: 'linux', + arch: 'x64' + }) ).toContain( '/stork-ubuntu-22-04' ) + }) +})