--- title: "GitHub and Git forge source" description: "Load Markdown from a repository on GitHub, GitLab, Codeberg, Bitbucket, or another Git forge." canonical_url: "https://content.comark.dev/sources/forges" --- # GitHub and Git forge source > Load Markdown from a repository on GitHub, GitLab, Codeberg, Bitbucket, or another Git forge. The `forge` source reads content from a Git repository on GitHub, GitLab, Codeberg, Bitbucket, or any other forge that [forges](https://forges.link) supports. Content lives apart from your app, so a push to the content repository goes live without a rebuild. ## `forge(options)`{lang="ts"} Install `forges` next to `comark-content`: ```bash npm install forges ``` Then pass a provider for your forge and a repository. For a GitHub repository: ```ts [content.ts] import { comarkContent } from 'comark-content' import forge from 'comark-content/sources/forge' import { github } from 'forges/github' const content = comarkContent({ source: forge({ provider: github(), repo: 'comarkdown/comark', ref: 'main', path: 'docs/content', }), }) ``` **Parameters:** - `options` - A [`ForgeSource`](https://content.comark.dev/reference/types/content#forgesource) object **Returns:** [`Source`](https://content.comark.dev/reference/types/content#source). Pass it to `comarkContent({ source })`. ## Connect to your forge Each forge has its own provider in forges. Import it from its own subpath, such as `forges/gitlab`, so the other forges stay out of your server build. Public repositories work without a token, at the forge's anonymous rate limit. For a private repository, or for a higher rate limit, pass a token that can read the repository's contents. ### GitHub Without a token, GitHub allows 60 API requests per hour for each IP address. For a private repository, a [fine-grained personal access token](https://github.com/settings/tokens) with the `Contents: read` permission is enough: ```ts [content.ts] import { comarkContent } from 'comark-content' import forge from 'comark-content/sources/forge' import { github } from 'forges/github' const content = comarkContent({ source: forge({ provider: github({ auth: { type: 'token', token: process.env.GITHUB_TOKEN! } }), repo: 'acme/private-docs', ref: 'main', path: 'content', }), }) ``` For GitHub Enterprise Server, pass the API URL of your instance as `baseUrl`, for example `github({ baseUrl: 'https://ghe.example.com/api/v3' })`. ### GitLab On GitLab, the owner part of `repo` can contain nested groups, such as `docs-team/handbook/website`. For a self-managed instance, pass its root URL as `baseUrl`: ```ts [content.ts] import forge from 'comark-content/sources/forge' import { gitlab } from 'forges/gitlab' forge({ provider: gitlab({ baseUrl: 'https://gitlab.example.com', auth: { type: 'token', token: process.env.GITLAB_TOKEN! }, }), repo: 'docs-team/handbook/website', path: 'content', }) ``` ### Codeberg, Forgejo, and Gitea Codeberg is the default instance of the Forgejo provider. For another Forgejo instance, pass its root URL as `baseUrl`. For Gitea, use `gitea()` from `forges/gitea`, which takes the same options. ```ts [content.ts] import forge from 'comark-content/sources/forge' import { forgejo } from 'forges/forgejo' forge({ provider: forgejo({ auth: { type: 'token', token: process.env.CODEBERG_TOKEN! } }), repo: 'acme/docs', path: 'content', }) ``` ### Other forges Bitbucket, Azure DevOps, Gitee, Sourcehut, Tangled, and Cloudflare Artifacts work the same way, each with its own provider. The [forges providers page](https://forges.link/providers) lists every provider with its import path and options. ## Options | Option | Type | Default | Description | | ----------------------- | ------------------- | -------------- | -------------------------------------------------------------------- | | [`provider`](#provider) | `ForgeProvider` | - | A forges provider for the forge that hosts the repository. Required. | | [`repo`](#repo) | `string \| RepoRef` | - | The repository, as `owner/name`. Required. | | [`ref`](#ref) | `string` | default branch | Branch, tag, or commit SHA to read from. | | [`path`](#path) | `string` | `''` | Directory inside the repo to use as the content root. | | [`prefix`](#prefix) | `string` | `undefined` | Prepended to each entry's `path` (for example `/blog`). | | [`exclude`](#exclude) | `string[]` | `undefined` | Picomatch globs of keys to drop before parsing. | | [`schema`](#schema) | `JsonSchema` | `undefined` | Explicit document `data` schema for types and query columns. | ### `provider` The forges provider carries the forge's address and the credentials. [Connect to your forge](#connect-to-your-forge) shows the setup for each forge. ### `repo` The repository as `owner/name`. You can also pass a forges `RepoRef` that you already have. ### `ref` The branch, tag, or commit SHA to read from. Defaults to the repository's default branch. Pin `ref` to a commit SHA when every instance must render the same content. [`content.withRef(sha)`](https://content.comark.dev/reference/content/with-ref) returns an instance pinned to another ref, and `provider.contents.resolveRef(repo, 'main')` returns the SHA at the tip of a branch. The [production pattern](#production-serve-one-commit-and-follow-the-branch) combines the two. ### `path` The directory inside the repo to use as the content root. Keys and public paths are relative to it, so `content/guide/start.md` is served at `/guide/start` when `path` is `content`: ```ts [scoped.ts] forge({ provider: github(), repo: 'nuxt/nuxt', ref: 'main', path: 'docs/1.getting-started', }) await content.list() // only entries under docs/1.getting-started/** ``` ### `prefix` Prepended to every entry's public `path`. Combine it with [`path`](#path) to remap the public path space: ```ts [mounted.ts] forge({ provider: github(), repo: 'org/handbook', ref: 'main', path: 'content/policies', prefix: '/policies', }) await content.get('/policies/security') // reads content/policies/security.md in org/handbook ``` The prefix changes only the public `path` returned by [`content.get()`](https://content.comark.dev/reference/content/get) and [`content.list()`](https://content.comark.dev/reference/content/list). It doesn't affect [`meta.key`](https://content.comark.dev/guide/files-and-paths#internal-identifiers), which stays `/`, for example `default/security.md`. ### `exclude` Picomatch globs matched against each entry's key. Excluded files are dropped **before** parsing, so they never enter the manifest: ```ts [drafts.ts] forge({ provider: github(), repo: 'org/docs', ref: 'main', path: 'content', exclude: ['drafts/**', '**/*.draft.md'], }) ``` ### `schema` An explicit JSON Schema for the source's `data`. When set, the Content instance reports Markdown, JSON, and YAML frontmatter validation issues during source load and on direct updates, and uses the schema to generate types and query columns. ## How files are read The source lists the directory's tree in one request where the forge supports it. Sourcehut and Tangled list one directory level at a time, and GitHub, Gitee, and Forgejo truncate very large trees. In those cases, the source walks the tree with one request per directory. Each file body takes one request to the forge's cheapest endpoint for raw files. On [github.com](http://github.com), that endpoint is GitHub's raw CDN, which doesn't count against the API rate limit. A load on [github.com](http://github.com) then costs one API request: the tree listing. Every read is a network request, so the source sets [`expensiveReads`](https://content.comark.dev/sources/custom#expensivereads). A partial `init()` downloads each file to read its frontmatter and keeps the raw body in the [cache](https://content.comark.dev/advanced/caching). The first `get()` of a page parses that cached body instead of downloading the file again. Symbolic links are followed, up to three in a row, including links that point outside `path`. The source skips a link whose target is missing, and it leaves out submodules. ## Production: serve one commit and follow the branch A branch moves; an instance reads one source. Rather than telling a running instance about every push, production serves the content at a **commit** and replaces the instance when the branch advances. [`content.withRef(sha)`](https://content.comark.dev/reference/content/with-ref) gives you an instance pinned to that commit, with its own cache namespace. The rest is a small amount of application code. The following snippets are Nitro server code. `yourSharedDriver()` and `verifySignature()` stand for your own cache driver and webhook signature check. ::steps{level="3"} ### Resolve the commit to serve Ask the forge for the latest commit that touched the content directory on the target branch (GitHub here), and cache the answer in a store every function instance reads, with a short TTL. Filtering by path means a code-only commit doesn't move the pointer. ```ts [server/utils/pointer.ts] import { createStorage } from 'unstorage' const refs = createStorage({ driver: yourSharedDriver({ base: 'content:refs', ttl: 60 }) }) export async function resolveContentSha(branch = 'main', dir = 'docs/content', refresh = false) { const key = `${branch}:${dir}` if (!refresh) { const cached = await refs.getItem(key) if (cached) return cached } const [commit] = await $fetch>('https://api.github.com/repos/org/docs/commits', { query: { sha: branch, path: dir, per_page: 1 }, headers: { Authorization: `Bearer ${process.env.GITHUB_TOKEN}` }, }) await refs.setItem(key, commit.sha) return commit.sha } ``` ### Namespace the cache by commit A commit is immutable, so its parsed content can be cached for a long time. Pass a driver whose base includes a parser version and let `withRef()` add the commit. Bump the version when you change plugins or parser options that affect cached output; unrelated deployments then keep reusing warm namespaces. ```ts [server/utils/content.ts] import { comarkContent } from 'comark-content' import forge from 'comark-content/sources/forge' import { github } from 'forges/github' const base = comarkContent({ source: forge({ provider: github(), repo: 'org/docs', ref: 'main', path: 'docs/content' }), cache: { driver: yourSharedDriver({ base: 'content:v3', ttl: 60 * 60 * 24 }) }, }) ``` ### Swap the instance when the pointer moves Resolve the pointer on each request. If it differs from the commit the current instance serves, drop the instance and build a new one. Keep the promise, not the instance, so two requests on a cold function don't build twice, and don't memoize a failed build. ```ts [server/utils/content.ts] let headSha: string | undefined let current: Promise> | undefined export async function getContent() { const sha = await resolveContentSha() if (sha !== headSha) { headSha = sha const previous = current current = undefined void previous?.then((instance) => instance.dispose()) } current ??= (async () => { const instance = base.withRef(sha) await instance.init() return instance })().catch((error) => { current = undefined throw error }) return current } ``` Preview deployments skip the resolution and stay pinned to the commit they were built at. ### Warm the new commit from the webhook The push webhook verifies its signature, refreshes the pointer, and warms the new namespace so the first request after the push doesn't pay for the walk. Everything here is idempotent, so a retried or duplicated delivery repeats work and changes nothing. ```ts [server/api/revalidate.post.ts] export default defineEventHandler(async (event) => { await verifySignature(event, process.env.WEBHOOK_SECRET!) const sha = await resolveContentSha('main', 'docs/content', true) waitUntil(base.withRef(sha).init({ partial: false })) // Then purge the rendered pages that depend on content. return { ok: true, sha } }) ``` :: ### What this gives you - **One commit per request.** Listings, navigation, and bodies come from the same instance, so a push between two calls can't mix versions. - **No coordination.** Each writer fills its own commit's namespace with deterministic content. Duplicates are harmless; a delayed job can't overwrite a newer one. - **Visibility delay equals the pointer TTL**, or zero right after the webhook's forced refresh. - **Recovery reads at the commit**, never at the moving branch head. - **Old namespaces expire** by the driver's TTL. To delete one sooner, call [`clean()`](https://content.comark.dev/reference/content/clean) on that ref's instance. ::warning A regional cache confines the webhook's forced refresh to its region; other regions pick up the new pointer when their cached copy expires. For a multi-region deployment, put the pointer in a globally replicated store. :: ## Refresh a single instance Without refs, a running instance learns about a push only when you tell it. Forge sources don't watch for changes, and there's no response cache to wait out: each `keys()` call lists the tree again. Call [`content.refresh()`](https://content.comark.dev/reference/content/refresh) to re-read the origin and reconcile the index: ```ts [webhook.ts] export default defineEventHandler(async () => { await content.refresh() return { revalidated: true } }) ``` This updates the instance that handles the webhook. Other running instances keep their index until they restart, which is why the [production pattern](#production-serve-one-commit-and-follow-the-branch) replaces instances instead. ## Warm start from a build-time seed Every cold start walks the repository: one API request to list the tree, then one raw read per file to collect frontmatter. On a serverless platform, where instances start often, that is the slowest thing your application does. Ship a snapshot as a seed instead. [`withSnapshot()`](https://content.comark.dev/sources/snapshot#withsnapshot) keeps the repository as the authority while cold boots serve from parsed data: ```ts [server/content.ts] import { comarkContent } from 'comark-content' import forge from 'comark-content/sources/forge' import { github } from 'forges/github' import { withSnapshot } from 'comark-content/sources/snapshot' export const content = comarkContent({ source: withSnapshot( forge({ provider: github(), repo: 'org/docs', ref: 'main', path: 'content' }), () => loadSeed('snapshot.json'), () => loadSeed('manifest.json') ), // A cache that outlives the instance is what carries runtime updates forward. cache: { driver: runtimeCacheDriver() }, }) ``` Write the seed during the build with [`comark-content snapshot`](https://content.comark.dev/reference/cli#comark-content-snapshot), and load it however your platform ships files — see [Nuxt](https://content.comark.dev/deployment/with-a-snapshot#nuxt) and [Next.js](https://content.comark.dev/deployment/with-a-snapshot#nextjs) for the two bundling mechanisms. The repository stays reachable throughout, which is the difference from a plain [`snapshot()`](https://content.comark.dev/sources/snapshot) source: `refresh()`, [`watch()`](https://content.comark.dev/reference/content/watch), media bytes, and `comark-content snapshot` all still use it. With [pinned instances](#production-serve-one-commit-and-follow-the-branch), build the seed on a pinned instance too (`content.withRef(sha).snapshot()`). The seed then carries its commit. At a later commit, the seed is never used as the index. Every body whose source text didn't change is still taken from it without a parse, and only the changed files are read at the new commit. See [which bodies hydrate](https://content.comark.dev/advanced/artifacts-and-hydration#which-bodies-hydrate). ### Which index wins A seed is a starting point, not the truth. The index and the seed compete by timestamp, and the newest one wins: | Scenario | Result | | --------------------------------------------- | ---------------------------------------------------------------------- | | First boot of a deployment | The seed serves. The repository isn't walked. | | A runtime update or removal, then a cold boot | The runtime index wins. The edit survives without a redeploy. | | A deployment ships a newer seed | The seed wins, replacing older runtime state. | | `refresh()` | The repository wins, and the result is stamped so later boots keep it. | [`update()`](https://content.comark.dev/reference/content/update), [`remove()`](https://content.comark.dev/reference/content/remove), and [`refresh()`](https://content.comark.dev/reference/content/refresh) re-stamp the persisted index as they write it. Without that, a deployment built *after* the seed an edit was applied to would look newer than the edit and silently revert it. Ties keep the persisted index, so re-deploying the same content never discards runtime state. #### The stamps come from two clocks The seed's `time` is the **build machine's** clock at build time. A runtime edit's `time` is the **host's** clock. The comparison assumes the two agree. If the build machine runs ahead, a seed built shortly *before* an edit can carry a stamp *after* it, win the comparison, and revert the edit on the next cold boot. If it runs behind, a genuinely newer seed can lose to an older edit. Cloud runners and hosts are NTP-synced, which keeps the disagreement under a second, so this needs an edit and a build within the same second to matter. It is a real assumption all the same: a self-hosted runner with a wrong clock will produce exactly this. There is no fix inside the library — two machines have no shared monotonic clock — so keep the clocks synced and know where to look if an edit ever "comes back". ::warning This depends on a cache whose driver outlives the instance — a Redis, KV, or platform runtime cache. With the default in-memory cache, every instance starts from the seed and runtime updates die with the instance that made them. :: ## Migrate from `github()` Comark Content 0.5 removes the built-in `github()` source, `comark-content/sources/github`. The `forge()` source replaces it: forges reads GitHub and every other forge through one API, so Comark Content no longer ships a source for a single forge. To migrate from 0.4: 1. Install `forges`: ```bash npm install forges ``` 2. Replace the `github()` source with `forge()` and the GitHub provider. For example, a private repository on the `main` branch: ```diff [content.ts] -import github from 'comark-content/sources/github' +import forge from 'comark-content/sources/forge' +import { github } from 'forges/github' const content = comarkContent({ - source: github({ + source: forge({ + provider: github({ auth: { type: 'token', token: process.env.GITHUB_TOKEN! } }), repo: 'acme/docs', - branch: 'main', - token: process.env.GITHUB_TOKEN, + ref: 'main', path: 'content', }), }) ``` 3. Map any other options with the following table: | `github()` option | `forge()` equivalent | | ----------------------------- | --------------------------------------------------------------------------------------------------- | | `repo` | `repo`, unchanged. | | `branch` | `ref`. Defaults to the repository's default branch instead of `main`. | | `path` | `path`. Always a directory: a path to a single file is no longer supported. | | `token` | `provider: github({ auth: { type: 'token', token } })`. | | `prefix`, `exclude`, `schema` | Unchanged. | | `ttl`, `cdnURL` | Removed, with no replacement. | | `apiURL` | Removed. For GitHub Enterprise Server, use `github({ baseUrl: 'https://ghe.example.com/api/v3' })`. | The new source also sets `expensiveReads` and implements `withRef()`, so [`content.withRef(sha)`](https://content.comark.dev/reference/content/with-ref) works as before. ## Sitemap See the full [sitemap](https://content.comark.dev/sitemap.md) for all pages.