---
title: "Create a custom source"
description: "Load content from any backend you control by implementing three methods."
canonical_url: "https://content.comark.dev/sources/custom"
---
# Create a custom source

> Load content from any backend you control by implementing three methods.

If no built-in source or [unstorage driver](https://unstorage.unjs.io/drivers) fits your backend, implement the [`Source`](https://content.comark.dev/reference/types/content#source) interface yourself. Three methods do the work: list the keys, read one key as text, read one key as bytes. The rest is optional.

This is the contract the [built-in sources](https://content.comark.dev/sources) implement. You don't need it to use them.

## Interface

```ts [Source.ts]
interface Source {
  prefix?: string
  schema?: JsonSchema
  expensiveReads?: boolean
  withRef?: (ref: string) => Source
  keys: () => Promise<string[]> | string[]
  getItem: (key: string) => Promise<string> | string
  getItemRaw: (key: string) => Promise<unknown> | unknown
  watch?: Driver['watch']
}
```

| Member                                        | Type                                   | Description                                                                                       |
| --------------------------------------------- | -------------------------------------- | ------------------------------------------------------------------------------------------------- |
| [`keys`](#interface-keys)                     | `() => string[] \| Promise<string[]>`  | **Required.** List every content key in the source.                                               |
| [`getItem`](#interface-getitem)               | `(key) => string \| Promise<string>`   | **Required.** Return a key's text content.                                                        |
| [`getItemRaw`](#interface-getitemraw)         | `(key) => unknown \| Promise<unknown>` | **Required.** Return a key's raw bytes, or `null`.                                                |
| [`prefix`](#interface-prefix)                 | `string`                               | Prepended to each entry's public `path`.                                                          |
| [`schema`](#interface-schema)                 | `JsonSchema`                           | Explicit `data` schema for types and query columns.                                               |
| [`expensiveReads`](#interface-expensivereads) | `boolean`                              | Reads are network fetches — cache raw bodies from partial init for reuse by `get()`.              |
| [`withRef`](#interface-withref)               | `(ref) => Source`                      | Return an equivalent source pinned to a branch, tag, commit, or other reference.                  |
| [`watch`](#interface-watch)                   | `Driver['watch']`                      | Forward change events to [`content.watch()`](https://content.comark.dev/reference/content/watch). |

### `keys`

List every content key the source exposes, as forward-slash paths with no leading slash (for example `posts/hello.md`). Keys must round-trip: whatever `keys()` returns is exactly what `getItem` and `getItemRaw` receive. Keys ending in `.md` / `.markdown` are parsed as documents out of the box; other extensions need a parser registered by a [plugin](https://content.comark.dev/plugins).

### `getItem`

Return a key's text content: the raw Markdown (or JSON/YAML) that Comark parses. Throw if the key is unreachable; the Content instance surfaces the error to the caller rather than swallowing it.

### `getItemRaw`

Return a key's raw bytes, as a `Uint8Array`, for media files. Required even if you only ship documents: return `null` when a key has no binary form.

### `prefix`

Prepended to every entry's public `path` (for example `/blog`). It does **not** affect the `key`, only the `path` returned by [`content.get()`](https://content.comark.dev/reference/content/get) and [`content.list()`](https://content.comark.dev/reference/content/list).

### `schema`

An explicit JSON Schema for the source's `data`. When set, the Content instance reports Markdown, JSON, and YAML validation issues during source load and on direct updates, and uses the schema to generate types and query columns.

### `expensiveReads`

Optional. Set it when every `getItem` is a network fetch (a SaaS API, a remote repo, …). A partial [`init()`](https://content.comark.dev/reference/content/init) must read each full file only to extract the frontmatter — with this flag the raw bodies are kept in the [cache](https://content.comark.dev/advanced/caching) under `<instance name>:__raw__:<file key>`, so the first full parse per file ([`get()`](https://content.comark.dev/reference/content/get)) reads from the cache instead of fetching the source again. The raw entry is dropped once the parsed body is cached. Leave it unset for local sources (like the built-in `fs`), where re-reading is cheaper than the cache churn. The built-in [GitHub source](https://content.comark.dev/sources/github) sets it.

### `withRef`

Optional. Return a new source with the same configuration, pinned to the requested ref. [`content.withRef(ref)`](https://content.comark.dev/reference/content/with-ref) calls it when it creates a pinned instance. Omit it when your backend isn't versioned; the source is then reused unchanged, and only the instance's cache namespace changes.

```ts [api-source.ts]
export function apiSource(options: { baseURL: string; ref?: string }): Source {
  return {
    withRef: (ref) => apiSource({ ...options, ref }),
    keys: () => listKeys(options),
    getItem: (key) => readItem(options, key),
    getItemRaw: (key) => readBytes(options, key),
  }
}
```

The built-in [GitHub source](https://content.comark.dev/sources/github) implements it by rebuilding itself with the ref as its branch.

### `watch`

Optional. When present, `content.watch()` forwards your source's change events. It receives a `(event, key)` callback (`event` is `'update'` or `'remove'`) and returns an unsubscribe function. See [Source with watch support](#examples-watch-support) for a worked example.

## Examples

Three minimal sources, from a plain HTTP backend to one with watch support:

### HTTP source

```ts [http-source.ts]
import type { Source } from 'comark-content'

export function httpSource(baseUrl: string): Source {
  return {
    async keys() {
      const res = await fetch(`${baseUrl}/index.json`)
      if (!res.ok) throw new Error(`index.json: ${res.status}`)
      return (await res.json()) as string[]
    },
    async getItem(key) {
      const res = await fetch(`${baseUrl}/${key}`)
      if (!res.ok) throw new Error(`${key}: ${res.status}`)
      return await res.text()
    },
    async getItemRaw(key) {
      const res = await fetch(`${baseUrl}/${key}`)
      if (!res.ok) throw new Error(`${key}: ${res.status}`)
      return new Uint8Array(await res.arrayBuffer())
    },
  }
}
```

### Database source

```ts [db-source.ts]
import type { Source } from 'comark-content'

interface Row { path: string; content: string; binary: Buffer | null }

export function dbSource(query: (sql: string, params?: any[]) => Promise<Row[]>): Source {
  return {
    async keys() {
      const rows = await query('SELECT path FROM documents')
      return rows.map(r => r.path)
    },
    async getItem(key) {
      const [row] = await query('SELECT content FROM documents WHERE path = ?', [key])
      if (!row) throw new Error(`${key}: not found`)
      return row.content
    },
    async getItemRaw(key) {
      const [row] = await query('SELECT binary FROM documents WHERE path = ?', [key])
      return row?.binary ?? null
    },
  }
}
```

### Watch support

`watch` lets `content.watch()` forward your source's change events. It returns an unsubscribe function:

```ts [pollable-source.ts]
import type { Source } from 'comark-content'

export function pollSource(loader: () => Promise<Map<string, string>>, intervalMs = 5000): Source {
  let cache = new Map<string, string>()
  let initialized = false

  async function ensureCache() {
    if (!initialized) {
      cache = await loader()
      initialized = true
    }
  }

  return {
    async keys() {
      await ensureCache()
      return [...cache.keys()]
    },
    async getItem(key) {
      await ensureCache()
      return cache.get(key) ?? ''
    },
    async getItemRaw(key) {
      await ensureCache()
      return cache.get(key) ?? null
    },
    watch(callback) {
      const handle = setInterval(async () => {
        const next = await loader()
        for (const [key, value] of next) {
          if (cache.get(key) !== value) callback('update', key)
        }
        for (const key of cache.keys()) {
          if (!next.has(key)) callback('remove', key)
        }
        cache = next
      }, intervalMs)
      return () => clearInterval(handle)
    },
  }
}
```

::tip{to="https://content.comark.dev/reference/content/watch"}
The watcher contract (event names, key shape, callback signature) is described in the [Watcher guide](https://content.comark.dev/reference/content/watch).
::


## Sitemap

See the full [sitemap](https://content.comark.dev/sitemap.md) for all pages.