Wire extracted packages into docs site library nav

Message

- Add ensureextractedpkgmanifest: synthesize a pkg/<name>/manifest.json for extracted repos whose custom build tasks skip copy-package-docs (e.g. sigil-crypto, sigil-tls, sigil-nrepl), so they appear in the Libraries sidebar and render a page instead of 404ing. Guarded to only run when the repo actually built modules attributed to that package, avoiding bogus empty entries for workspace/multi-package repos. - Add core general-purpose libraries to EXTRACTED_REPOS: sigil-websocket, sigil-oauth, sigil-jwt, sigil-irc, sigil-xmpp, sigil-org, sigil-tui. - Clear the public/ output dir before regenerating so pages for removed or renamed content do not linger and ship stale to production.

Changed

 publish.sh | 82 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 82 insertions(+)

Diff

publish.shmodified

@@ -116,6 +116,17 @@ EXTRACTED_REPOS=(

116

sigil-format sigil-hooks sigil-test

117

sigil-peg sigil-sxml sigil-markdown sigil-css sigil-fp sigil-match

118

sigil-r7rs sigil-transducers sigil-web sigil-yaml sigil-later

+119

# Core, general-purpose libraries added by the 2026-07 completeness audit.

+120

# These are published sigil/* library packages that were missing from the

+121

# docs site. (SaaS/API-client packages: forgejo, caldav, jmap, discourse,

+122

# postmark, ses, lemonsqueezy, telegram, twitch, youtube, wise, plus the

+123

# graphics/audio/game stack, are intentionally excluded; see the audit note.)

+124

# sigil-rss was also identified but is NOT release-ready (no tags, its

+125

# package.sgl lacks the required `sigil:` field and fails to build), so it

+126

# is held back until it is fixed and released. sigil-ledger was evaluated

+127

# and intentionally left out as too specialized.

+128

sigil-websocket sigil-oauth sigil-jwt sigil-irc sigil-xmpp

+129

sigil-org sigil-tui

130

)

131

132

# ---------------------------------------------------------------------------

@@ -271,6 +282,61 @@ MANIFEST

282

echo " Generated manifest for $name"

283

}

284

+285

# Ensure a _pkg manifest exists for an EXTRACTED repo package.

+286

# Extracted repos whose package.sgl defines a custom tasks: build task override

+287

# the default build (which runs copy-package-docs), and if they also ship no

+288

# docs/ dir they never emit their own _pkg/<name>/manifest.json. Without that

+289

# manifest, Press's scan-library-docs skips the package entirely: no Libraries

+290

# sidebar entry and the /docs/lib/<name>/ page 404s (as happened to

+291

# sigil-crypto, sigil-tls, and sigil-nrepl). This is the same failure mode

+292

# ensure_pkg_manifest handles for MONOREPO_PACKAGES; generate the manifest from

+293

# the repo's own package.sgl. The package name is the repo name (matches the

+294

# convention fix_extracted_json relies on for single-package repos).

+295

ensure_extracted_pkg_manifest() {

+296

local name="$1"

+297

local repo_dir="$PARENT_DIR/$name"

+298

local pkg_sgl="$repo_dir/package.sgl"

+299

local build_lib="$repo_dir/build/dev/lib"

+300

+301

if [[ ! -f "$pkg_sgl" ]]; then

+302

return

+303

fi

+304

+305

local manifest_dir="$COMBINED_LIB/_pkg/$name"

+306

local manifest_path="$manifest_dir/manifest.json"

+307

+308

# Skip if a manifest was already merged from the repo's own build output

+309

if [[ -f "$manifest_path" ]]; then

+310

return

+311

fi

+312

+313

# Only synthesize a manifest when the repo actually built modules attributed

+314

# to this package. This guards against workspace/multi-package repos (e.g.

+315

# sigil-lang, whose package.sgl name is the workspace name "sigil-lang" but

+316

# which ships sigil-stdlib/sigil-lib) and against unbuilt/empty repos --

+317

# either would otherwise produce a bogus empty package in the sidebar.

+318

if [[ ! -d "$build_lib" ]]; then

+319

return

+320

fi

+321

if ! grep -rls "\"package\": \"$name\"" "$build_lib" --include='*.json' >/dev/null 2>&1; then

+322

return

+323

fi

+324

+325

local version description

+326

version=$(grep 'version:' "$pkg_sgl" | head -1 | sed 's/.*version: *"$.*$".*/\1/' || true)

+327

description=$(grep 'description:' "$pkg_sgl" | head -1 | sed 's/.*description: *"$.*$".*/\1/' || true)

+328

+329

mkdir -p "$manifest_dir"

+330

cat > "$manifest_path" <<MANIFEST

+331

{

+332

"name": "$name",

+333

"version": "$version",

+334

"description": "$description"

+335

}

+336

MANIFEST

+337

echo " Generated manifest for extracted package $name"

+338

}

+339

340

# ---------------------------------------------------------------------------

341

# Step 1: Ensure sigil monorepo is present and built

342

# ---------------------------------------------------------------------------

@@ -417,6 +483,13 @@ for pkg in "${MONOREPO_PACKAGES[@]}"; do

483

ensure_pkg_manifest "$pkg"

484

done

485

+486

# Generate _pkg manifests for extracted repos whose custom build tasks skip

+487

# copy-package-docs (e.g. sigil-crypto, sigil-tls, sigil-nrepl). Without this

+488

# they never appear in the Libraries sidebar and their pages 404.

+489

for repo in "${EXTRACTED_REPOS[@]}"; do

+490

ensure_extracted_pkg_manifest "$repo"

+491

done

+492

493

echo " Combined lib ready."

494

495

# ---------------------------------------------------------------------------

@@ -438,6 +511,15 @@ fi

511

echo " Compiling site sources..."

512

$SIGIL build

513

+514

# Clear stale output before regenerating. The site build writes pages into

+515

# public/ but never removes ones that no longer exist, so pages for packages

+516

# dropped from the arrays (or any renamed/removed content) linger as stale

+517

# files. Because the deploy step copies public/ verbatim into the fresh pages

+518

# branch, those stale pages would ship to production on every republish, not

+519

# just locally. Starting from an empty output tree guarantees public/ reflects

+520

# exactly the current build.

+521

rm -rf "$PUBLIC_DIR"

+522

523

# Build and run the site

524

# -L adds combined-lib to module search paths so Press's load-module-details

525

# can find .json doc files for packages not in the site's dependency tree