# Workspace semantics

The workspace is the canonical Knowl artifact. `wiki/` is directly portable as
an Open Knowledge Format (OKF) v0.2 bundle, alongside immutable source files in
`raw/`; the SQL adapter is only an operational index and projection.

## Ownership and layout

```text
<workspace>/
├── schema.md
├── raw/
│   └── <scope/source identity>/<version>/
│       ├── source
│       └── manifest.yaml
├── wiki/
│   ├── index.md
│   ├── log.md
│   ├── catalogs/
│   │   └── <semantic-domain>/index.md
│   ├── entities/*.md
│   ├── concepts/*.md
│   └── syntheses/*.md
└── .knowl/
    ├── staging/
    ├── recovery/
    └── knowl.sqlite
```

The filesystem adapter safely tokenizes raw-source path components, so the
physical directory names need not equal the source ID. `manifest_ref` in an
accepted source identifies the stored manifest. The logical identity is always
the tuple `(scope, adapter, source ID, version)`.

Ownership is intentionally split:

- `schema.md` is required, operator-controlled Markdown policy supplied to the
  maintainer as untrusted input. It guides semantic choices but is not an
  executable schema or validation DSL. Maintainer plans may read it, but edit
  validation rejects schema changes.
- `raw/` contains accepted source bytes and a manifest. A source version is
  immutable and idempotent: replaying the same digest returns the existing
  record; the same identity/version with a different digest is a conflict.
- `wiki/**/*.md` is the accumulated semantic knowledge artifact. Pages,
  catalogs, and the append-only `log.md` are readable and Git-compatible. It
  contains no configured-source copies.
- `.knowl/staging/` and `.knowl/recovery/` are implementation state used for
  preview, atomic commit, and interruption recovery. They are not knowledge
  content. The SQLite file is also rebuildable operational state.

## Export for publication

Export the canonical `wiki/` directory as a standalone OKF v0.2 bundle:

```bash
knowl export okf --output ./public
```

The output directory must not already exist. Its root contains `index.md`,
`log.md`, catalogs, pages, and bundle assets; it does not contain `schema.md`,
`raw/`, `.knowl/`, configuration, or database files. There is no separate
`export wiki` alias because `wiki/` is already the OKF bundle, and this command
does not create an archive.

Generate an [llms.txt](https://llmstxt.org/) navigation file beside those
pages:

```bash
knowl export llms-txt --output ./public/llms.txt
```

Without `--output`, the command writes only the Markdown document to stdout.
Links are bundle-relative by default, so the two commands above produce a
directory that can be published as one unit. Use `--base-url` when the final
public URL is already known:

```bash
knowl export llms-txt \
  --base-url https://docs.example.com/knowledge/ \
  --output ./llms.txt
```

`--title` overrides the root catalog title and `--summary` adds a blockquote
summary. Rendering follows the evolving llms.txt v2 open proposal rather than
claiming a formal standard. Knowl creates these files but does not upload,
serve, compress, or otherwise publish them.

`knowl init` creates the required directories and starter `schema.md`,
`wiki/index.md`, and `wiki/log.md` without replacing existing files. The root
index declares `okf_version: "0.2"`.
`knowl validate` checks the required workspace paths. Full migration,
projection, and recovery checks happen during `start` or host construction.

## Page contract

The initial workspace groups pages under `entities`, `concepts`, and
`syntheses`, but the application accepts safe Markdown paths under `wiki/` and
the operator-owned schema defines the intended page types. A normal page is an
OKF concept with bounded YAML frontmatter followed by Markdown:

```markdown
---
title: Example
type: entity
description: A portable example.
tags: [example]
knowl:
  id: entities/example
  source_refs:
    - fixture:source-1@1
---
# Example

The page body is human-readable and cites the accepted source above.

See the workspace root index for navigation.
```

The enforced and linted conventions are:

- the safe bundle-relative path without `.md` is the concept identity;
- `type` is required; `title`, `description`, `resource`, `tags`, `sources`,
  lifecycle, generation, and verification fields follow OKF v0.2;
- unknown producer fields round-trip as OKF extensions;
- internal links must resolve to a page in the current bundle;
- the `knowl` namespaced extension carries Knowl provenance. Its `source_refs`
  contains stable citations in the form
  `adapter:source-id@version`; every maintained page must cite at least one
  accepted raw source.
- Curated Knowl pages use double-bracket wiki links whose target is an existing
  bundle-relative page identity; these links remain strict. Imported OKF
  concepts use standard Markdown links such as `[Related](related.md)`. Local
  concept targets are resolved deterministically; unresolved internal targets
  are rejected, while external URLs and assets are excluded from the concept
  graph.
- Maintainer edit plans may target safe semantic `wiki/**/*.md` except
  `wiki/log.md` and the reserved legacy `wiki/sources/**` boundary. The plan must carry the
  current schema digest, cite the accepted source, use unique paths, and stay
  within the configured file/count limits.

Accepted raw manifests carry structured `source_document` metadata containing
`source_id`, `document_id`, immutable `revision`, and canonical `uri`. Curated
pages cite stable raw refs; snapshots resolve all supporting refs into a sorted
`source_documents` collection without copying source bodies into the page.
A successful complete scan may tombstone a deleted document, while its raw
revisions and curated semantic pages remain. An interrupted scan never
authorizes deletion.

The starter `schema.md` is a concise usable policy with `schema_version: 1`, a
page taxonomy, metadata and provenance guidance, link and organization rules,
and contradiction/supersession handling. Operators can specialize those choices
for their domain. The version is descriptive: Knowl reads the first
`schema_version:` line, defaults older files without one to version `1`, and does
not parse the remaining prose as machine-enforced rules.

Knowl code remains authoritative for the enforced and linted conventions above,
including OKF structure, safe paths, provenance, links, bounds, and protected
control files. The schema's exact byte digest is recorded with every operation
and must match the provider plan. It is rechecked before staging and commit so
an operator policy change invalidates work planned against older bytes.

## Reading pages and accepted sources in the browser

The optional [web UI](web-ui.md) reads the current canonical `wiki/` page and
its saved source references. **Page sources** identifies each immutable accepted
revision; **Read saved source** displays its accepted text and digest from
`raw/`, without fetching the original source again. If that revision is missing,
unavailable, too large, or not displayable as text, the UI reports the failure.
It never substitutes a newer upstream document.

An **Open original** link appears only for a safe, credential-free HTTP/HTTPS
URI recorded in provenance, with user info, query strings, and fragments removed.
Filesystem paths, credentials, and unsafe schemes
are not exposed as clickable originals; a source can remain readable from its
accepted revision when no original URL is available. Links require a deliberate
click and carry no operator bearer token.

Source references describe support for the page as a whole, not individual
sentences. A search evidence card preserves the retrieved snippet and source
references. **Open current page** reads the page as it exists when opened, which
may have changed since retrieval. Page digests and snapshot versions identify
current reads; the browser does not provide historical wiki-page storage.
Accepted raw revisions retain their immutable identity independently.

## Control pages

`wiki/index.md` is the OKF root catalog and declares version `0.2`. Nested
`index.md` files are also reserved catalogs but cannot redeclare the bundle
version. Catalog controls are validated and included in canonical digests, but
never become retrieval evidence.

`wiki/log.md` is maintained by the application. Each committed operation adds
one structured JSON metadata line containing the operation ID, generation,
schema digest, cited source references, and committed file paths. A maintainer
cannot rewrite it as an arbitrary edit; its prior digest is part of staging and
commit preconditions.

Every `log.md` is a reserved, newest-first ISO-date-grouped OKF control. Source
OKF logs remain immutable raw evidence and are not imported into the semantic
wiki. Attested Computation metadata on semantic pages is parsed, preserved,
projected, and returned, but never executed or dereferenced.

## Explicit semantic hierarchy reconciliation

A valid flat root catalog remains compatible and readable. Knowl reorganizes it
only when an operator explicitly runs:

```bash
knowl hierarchy reconcile
```

The maintainer receives a structured, deterministically ordered description of
ordinary pages: canonical ID/path/digest, OKF type/title/description/tags, a
bounded excerpt of user content, and current catalog membership. It does not
receive raw source bodies, source-native paths, frontmatter, or provenance
envelopes as taxonomy input. The provider returns a graph; application code
validates it and renders the OKF Markdown.

The single provider call follows a generic subject-first planning contract.
Subject domains are the primary navigation axis; page type, document kind, and
implementation technology are supporting evidence rather than automatic root
categories. Broad heterogeneous subjects are recursively decomposed into
cohesive subdomains. Every page receives a primary placement, while a page may
appear in an additional catalog only when its bounded semantic fields clearly
span subjects. Suitable existing semantic paths and unrelated memberships are
retained when possible so an unchanged snapshot can converge without churn.
These instructions constrain the plan shape, but semantic quality is still
provider-dependent and no numeric balance or optimal taxonomy is promised.

The hierarchy input carries the current schema digest to bind the operation; it
does not carry `schema.md` content or derive taxonomy policy from the schema.
The current planning contract has durable identity `hierarchy-v3`. Catalogs
created by an earlier planner remain valid and untouched until an operator runs
the explicit reconcile command.

Hierarchy reconciliation owns only `wiki/index.md` and generated
`wiki/catalogs/**/index.md`. It may create, replace, or remove those managed
catalog controls, but it cannot target ordinary pages, `wiki/log.md`, `raw/`,
`wiki/sources/**`, or operator-created catalogs elsewhere. Every ordinary page
must remain reachable from the single root, and catalog edges must stay inside
`wiki/`, resolve, and remain acyclic. Reserved controls remain excluded from
retrieval. Every generated non-root catalog must contain at least one child; an
empty root is accepted only for an empty wiki. Cross-cutting pages may be linked
from distinct catalogs, while a duplicate child edge within one catalog is
invalid.

The current conservative limits are 1,024 ordinary pages, 1,024 catalogs,
16,384 edges, graph depth 16, 4 MiB total planner input, 4,096 excerpt
characters per page, 1 MiB structured plan output, 1,024 managed edits, 256 KiB
per generated catalog, and a 1 MiB stage manifest. Paths use the existing safe
edit-path bound. The complete input must fit: Knowl fails before invoking the
provider rather than silently classifying a partial wiki.

The command reserves and claims one durable hierarchy operation, stages catalog
changes with schema/snapshot/target digest preconditions, commits through the
normal recovery journal, appends the audit log, updates the projection, and
validates the resulting workspace. Its JSON result reports `operation_id`,
`status`, `changed`, and, for a canonical change, `generation` and `files`.
Re-running a converged snapshot is a byte-stable no-op. Stale snapshots, unsafe
paths, invalid or incomplete graphs, exceeded limits, staging failures, and
projection failures retain a specific wrapped cause; interrupted canonical
commits are replayed from the durable stage and receipt.

The command does not start the HTTP listener, the general scheduler, source
jobs, or `sync.on_start`. Startup, bootstrap, validate, retrieve, operation
reads, source list, and source status do not invoke hierarchy reconciliation.

## Explicit OKF migration

Startup and read-only commands never migrate content. To upgrade a legacy
canonical workspace, stop writers, take a backup, and run:

```bash
knowl migrate okf-v0.2
knowl validate
```

The command preflights the whole change, uses the normal recovery journal,
archives the exact legacy log, writes its marker only after canonical commit,
and rebuilds the configured projection. Re-running it is a no-op. If interrupted,
the next invocation deterministically rolls back or completes from the journal.

## Recovery and Git

Content commit writes a staging manifest, records preimages in a recovery
journal, atomically replaces the target files, and marks the journal committed.
Startup recovery runs before projection readiness:

- a `prepared` journal restores preimages and records a rolled-back operation;
- a `committed` journal is cleaned after the canonical files are complete;
- incomplete staging is discarded as uncommitted work.

Shutdown preserves interrupted operations for startup recovery and inspection
through redacted operation status. Do not hand-edit staging or
recovery files while a host is running. For a backup, stop the host first and
copy the complete workspace; the SQL projection can be rebuilt from its
Markdown snapshot.

Source synchronization uses the same staged canonical writer and recovery
journal. On restart, filesystem recovery completes before resumable source runs
and projection preparation. One source failure does not remove another source's
raw evidence or make the prior successful semantic snapshot unavailable. Sync
success covers raw acceptance and durable maintenance reservation; the required
maintainer provider runs asynchronously through the operation scheduler. Bounded
maintenance outcomes are reported separately in source status.

Legacy `wiki/sources/<source_id>/**` files are derived migration input, not
active knowledge. The next successful reconciliation deletes only the matching
source subtree through staged recovery and leaves raw history and curated pages
unchanged.

The workspace is suitable for Git review. Knowl synchronizes configured remote
Git sources by fetching their tracked refs and accepting documents from an
immutable commit. These read-only fetches never execute repository content or
modify the remote. Knowl does not commit or push the generated workspace to a
remote repository; operators own workspace version control and publication.
Operators commonly version `schema.md`, `raw/`, and `wiki/`, while treating
`.knowl/` as local operational state. If `.knowl/` is versioned for a specific
workflow, exclude credentials and review
the database/staging contents deliberately. Deletion or forgetting is not
implicit: absence from search is not an audited delete operation.
