sitegen/readme.md

# clover sitegen framework

this repository contains clover's "sitegen" framework, which is a set of tools
that assist building websites. these tools power <https://paperclover.net>.

- **HTML ("Server Side Rendering") engine written from scratch.** (~500 lines)
  - A more practical JSX runtime (`class` instead of `className`, built-in
    `clsx`, `html()` helper over `dangerouslySetInnerHTML` prop, etc).
  - Integration with [Marko] for concisely written components.
  - TODO: MDX-like compiler for content-heavy pages like blogs.
  - Different languages can be used at the same time. Supports `async function`
    components, `<Suspense />`, and custom extensions.
- **Incremental static site generator and build system.**
  - Build both development and production sites on startup start. Incremental
    generator rebuilds changed pages; Build system state survives coding sessions.
  - Multiple backend support.
  - (TODO) Tests, Lints, and Type-checking is run alongside, and only re-runs
    checks when the files change. For example, changing a component re-tests
    only pages that use that component and re-lints only the changed file.
- **Integrated libraries for building complex, content heavy web sites.**
  - Static asset serving with automatic ETag and build-time compression.
  - Dynamic pages. Static pages can be regenerated (and cached) at runtime.
  - Databases with a minimal SQLite wrapper.
  - Meta Tags and Open Graph generation.
  - Font subsetting to reduce page bandwidth.
- **Built on the stable and battle-tested Node.js runtime.**

none of these tools are complex or revolutionary. rather, this project is the
sum of many years of experience on managing content heavy websites, and an
example on how other over-complicate other frameworks.

[Marko]: https://next.markojs.com

Included is `src`, which contains `paperclover.net`. Website highlights:

- TODO: flashy homepage.
- [Question/Answer board, custom markdown parser and components][q+a].
- [File viewer with fast ui/ux + optimized media streaming][file].
- [Personal, friends-only blog with password protection][friends].
- TODO: digital garden styled blog.

[q+a]: https://paperclover.net/q+a
[file]: https://paperclover.net/file
[friends]: https://paperclover.net/friends

## Development

minimum system requirements:

- a cpu with at least 1 core.
- random access memory.
- windows 7 or later, macos, or other operating system.

required software:

- node.js v24
- python v3

my development machine, for example, is Dell Inspiron 7348 with Core i7

    npm install

    # build site using 'run.js' to enable runtime plugins
    node run generate
    # the built site runs in regular node.js
    node --enable-source-maps .clover/o/backend

    # watch-rebuild mode
    node run watch

for unix systems, the provided `flake.nix` can be used with `nix develop` to
open a shell with all needed system dependencies.

## Deployment

there are two primary server components to be deployed: the web server and the
sourth of truth server. The latter is a singleton that runs on Clover's NAS,
which holds the full contents of the file storage. The web server pulls data
from the source of truth and renders web pages, and can be duplicated to
multiple cloud hosts without issue.

Deployment of the source of truth can be done with Docker Compose:

    services:
      backend:
        container_name: backend
        build:
          # this uses loopback to hit the self-hosted git server,
          # docker will cache the image to not re-fetch on reboot.
          context: http://127.0.0.1:3000/clo/sitegen.git
          dockerfile: src/source-of-truth.dockerfile
        environment:
          # configuration
          - PORT=43200
          - CLOVER_DB=/data
          - CLOVER_FILE_RAW=/published
          - CLOVER_FILE_DERIVED=/data/derived
          - CLOVER_SOT_KEY=... # guards private/unreleased content
        ports:
          - '43200:43200'
        restart: unless-stopped
        volumes:
          - /mnt/storage1/clover/Documents/Config/paperclover:/data
          - /mnt/storage1/clover/Published:/published

Due to caching, images may need to be purged via `docker image rm {image} -f`
when an update is desired. Some docker GUIs support force pulls, some are buggy.

The web server performs rendering. A `Dockerfile` for it is present in
`src/web.dockerfile` but it is currently unused. Deployments are done by
building the project locally, and then using `rsync` to copy files.

    node run generate
    rsync .clover/o "$REMOTE_USER:~/paperclover" --exclude=.clover --exclude=.env \
      --delete-after --progress --human-readable
    ssh "$REMOTE_USER" /bin/bash << EOF
        set -e
        cd ~/paperclover
        npm ci
        pm2 restart site
        echo "-> https://paperclover.net"
    EOF

## Contributions

No contributions to `src` accepted, only `framework`.