> ## Documentation Index > Fetch the complete documentation index at: https://vitruvio.sh/docs/llms.txt > Use this file to discover all available pages before exploring further. # Declared sources > Declaring where material comes from, pulling only what is new, and why a pull cannot resurrect redacted bytes. [Registering evidence](/docs/sources/registering-evidence) takes one file at a time. That is right for a paper you just read and wrong for material that keeps arriving: a course platform that publishes a new practical every week, a browser extension dropping arXiv PDFs into a folder, a transcript per video. A **source** is a declaration of where material comes from. `vitruvio source pull` acquires from it and registers what is new into **canonical** memory. Interpretation is unchanged and still separate — `vitruvio task` and `vitruvio ingest run` are where a model gets to say what the evidence *means*. ```console theme={null} vitruvio source add papers --kind directory --path ~/Downloads/arxiv --option glob='*.pdf' vitruvio source pull papers --dry-run # what it would take, fetching nothing vitruvio source pull papers # registers vitruvio source pull papers # skipped by origin; nothing is fetched ``` ```toml theme={null} [brain.sources.papers] kind = "directory" path = "~/Downloads/arxiv" normalize_with = "pdf-text" [brain.sources.papers.options] glob = "*.pdf" [brains.optics] path = "./brains/optics" [brains.optics.sources.course] kind = "courseware" # a plugin you wrote options = { course_id = 77 } [brains.statics] path = "./brains/statics" [brains.statics.sources.course] kind = "courseware" # the same kind and source name, different brain defaults options = { course_id = 202 } ``` Sources are brain configuration, not project configuration. A single-brain project declares them under `[brain.sources.]`; a named project declares them under `[brains..sources.]`. The same source kind and name may appear under several brains with different paths and options, and selecting a brain selects only its declarations. Project-level `[sources.]` declarations are invalid. To migrate an older declaration, move the whole source table under its destination brain and delete the old `brain` field: ```toml theme={null} # before [sources.course] kind = "courseware" brain = "statics" # after [brains.statics.sources.course] kind = "courseware" ``` ## Defaults in the declaration, overrides on one pull `options` is the open, kind-specific part of a source. The declaration provides defaults, and a named pull may override them for that invocation without rewriting `vitruvio.toml`: ```console theme={null} vitruvio --project coursework --brain statics source pull course \ --option course_id=101 --option 'sections=Theory|Bibliography' --dry-run vitruvio --project coursework --brain dynamics source pull course \ --option course_id=202 --option 'sections=Theory|Bibliography' --dry-run ``` The kind receives the merged values through `self.options`; command-line values win. Its constructor validates the result exactly as it validates declared options. Values are parsed as booleans, integers or strings by the same rules as `source add`. Persistent differences belong in each brain's declaration. `--option` remains useful for a one-off override and never rewrites that declaration. It cannot be combined with `--all`, because one override set has no unambiguous meaning across different source kinds in the selected brain. Overrides are acquisition parameters, not hidden state. A plugin must include every value that changes a remote item's identity in that item's stable `origin` — for example both `course_id` and `resource_id`. Otherwise two courses that reuse an id could cause a false origin skip. ## What stops the same thing being registered twice Three layers, in the order they run: 1. **The origin index.** Before anything is fetched, one hash-map lookup answers "have I acquired this?". This is the cheap layer: pulling a folder of a hundred unchanged files does a hundred lookups and no reads. 2. **The redaction guard.** After the fetch, before the register. See below. 3. **Content addressing.** Identical bytes compute the same block identity, so `register` reports a duplicate. The backstop for a source that cannot produce a stable origin: one wasted download, never a wrong result. Nothing is persisted for any of this. The origin index is derived state under `.vitruvio/`, regenerated by `vitruvio index build` like every other index — which is why there is no cursor file. Changing a source's `media_type` or `normalize_with` **re-registers** rather than being skipped. Both are part of a canonical block's identity, so a silent skip would make the correction do nothing and leave the wrong block in place. Likewise, an item listed without a type does not trust an existing `application/octet-stream` registration: it is fetched once so a `FetchResult` can replace that generic type. Once the origin holds a specific MIME type, later pulls skip it before downloading again. ### A pull can never restore redacted bytes `vitruvio retain redact` destroys bytes under policy — for personal data, credentials, licensed material. A source that still lists the same item would re-fetch exactly those bytes, and a scheduled pull would undo every redaction on a schedule. So a digest that was tombstoned is refused, out loud, and `--refetch` does not override it: ```console theme={null} vitruvio retain redact --memory-type canonical --reason "personal data" --yes vitruvio source pull papers --refetch # skip paper.pdf sha256:a0f459... was redacted; re-registering would restore the bytes a # retention policy destroyed. Undo the redaction deliberately if that is what you want ``` Undoing a redaction is a deliberate act: register the file by hand with `vitruvio source register`. ## Several brains Select a brain before inspecting, adding, removing, or pulling its sources. `--all` means every source belonging to that selected brain; it never crosses into another brain: ```console theme={null} vitruvio --project coursework --brain optics source pull --all # ok course 4 registered # ok papers 12 registered, 3 skipped ``` `--all` keeps going past a failure and exits 11 if any source failed. Being told which one of six went wrong is better than stopping at the first and leaving four that would have worked unpulled and unmentioned. This containment removes the old conflict between a project-level source's `brain` field and the selected `--brain`: there is no destination field to disagree with the source's location. A multi-brain project with no selected brain refuses every source command before it can acquire or register anything. ## Where a pull can go wrong ```console theme={null} vitruvio --project coursework --brain optics source status # ok papers directory ~/Downloads/arxiv # -- course courseware courseware is not on PATH ``` `status` reports an unusable source as a row rather than failing, because one broken declaration must not hide the five that are fine. One risk worth stating plainly: a `directory` source composes with `vitruvio dist push` into a way to publish something you did not mean to. Point one at the wrong folder and a private key becomes a canonical block, content-addressed and Merkle-committed in a public repository — which cannot be cleanly retracted. `source add` warns when a path sits outside the project directory. Read that warning. ## Exit codes | code | meaning | | - | - | | 0 | pulled, or everything was already held | | 2 | the invocation was wrong: an undeclared source, `--all` with a name | | 3 | the declaration is wrong: no `path` on a directory source, a kind that is not installed | | 11 | a source was unreachable or refused: a tool missing, a timeout, a directory gone | Sources are plugins. What the configuration may name, and how to write a kind of your own, is on [Writing a source kind](/docs/sources/writing-a-source-kind). This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.