# DreamLake — Full documentation > DreamLake py is a Python SDK for ML data management — upload any file, organize it into a project/episode/folder tree, and pull it back by path, folder, or curated bundle. Generated from https://py.dreamlake.ai. 16 pages. --- Source: https://py.dreamlake.ai # DreamLake Python A Python SDK for ML data management. Upload any file, organize it into a project/episode/folder tree, and pull it back — by path, by folder, or as a curated bundle. ## What it is DreamLake stores ML artifacts (video, audio, images, label tracks, transcripts, parquet, csv — anything) as nodes in a hierarchical tree. The Python SDK is a thin client that handles the upload/download protocol and exposes a small surface: | Function | Purpose | | --- | --- | | `dl.upload(file, path=...)` | Upload any file. Type auto-detected from extension. | | `dl.download(path_or_id, to=...)` | Download a file, folder, or episode (recursive). | | `dl.bindr(name, members=[...])` | Create a curated collection across the tree. | | `dl.download_bindr(name, to=...)` | Download every member of a bindr into one tree. | | `dl.Prefix(project=..., prefix=...)` | Context manager for scoping subsequent calls. | ## Hello, DreamLake ```python file="hello_dreamlake.py" import dreamlake as dl with dl.Prefix(project="robotics@alice", prefix="/session-042"): dl.upload("./video.mp4", path="camera/front") dl.upload("./labels.jsonl", path="detections/yolo") dl.download("camera/front/video.mp4", to="./out/") ``` Three things happen in this snippet: 1. The `Prefix` context scopes everything to the `robotics` project (owned by `alice`) and the `session-042` episode. 2. Each `upload` is auto-routed: extension → kind (`video`, `label-track`), MIME type, BSS staging path. The file is hashed (SHA-256, 16-char prefix), streamed directly to S3 in multipart chunks, then registered as a node in the tree. Folders along the path (`camera/front`, `detections/yolo`) are created on demand. 3. `download` resolves the relative path against the current `Prefix` and streams the file back. ## How files are organized Everything is a node in one tree, distinguished by `kind`: ``` robotics ← kind="project" └─ session-042 ← kind="episode" ├─ camera/front/video.mp4 ← kind="video" └─ detections/yolo/labels.jsonl ← kind="label-track" ``` The `kind` field is free-form — `parquet`, `csv`, `npy`, `pickle`, or anything else works the same way as the built-in media kinds. There's no separate registry to extend. ## Bindrs — curated collections by path A bindr groups any set of nodes (files or whole folders) into a named collection. Useful for assembling a training set out of scattered data, sharing a slice of a project, or downloading "everything I need for this experiment." ```python with dl.Prefix(project="robotics@alice", prefix="/session-042"): dl.bindr("training-set", members=[ "camera/front", "detections/yolo", ]) # Later — pull every file referenced by the bindr dl.download_bindr("training-set", project="robotics@alice", to="./train/") ``` Bindrs don't copy data. They store path references; downloads stream from the original location. ## What it isn't - **Not a processing engine.** Uploads don't trigger HLS splitting, transcript chunking, or feature extraction. Those run as separate pipelines you opt into. - **Not a versioning system.** Files are content-addressed by hash for dedup, but there's no commit graph. If you need versioning, encode it in your paths. - **Not local-only.** Everything is cloud-backed (S3 + a metadata service). Local caching is on the roadmap. ## Where to next - [Upload Files](/examples-upload-files.md) — the basic upload flow with auto-detection - [Upload an Episode](/examples-upload-episode.md) — bulk upload a directory of mixed files - [Bindrs](/examples-bindrs.md) — curated cross-tree collections - [Download Files](/examples-download.md) — files, folders, episodes, bindrs --- Source: https://py.dreamlake.ai/dev-tools # Internal pages You're reading this because **dev mode is on** (the `DEV` chip is lit in the top right). Pages tagged `hidden: true` only appear in the sidebar in this mode; otherwise they're invisible. Direct URLs still work — sharing a link to a hidden page doesn't depend on the toggle. Use this section for things first-time visitors shouldn't trip over: internal scratch, decision logs, half-written specs, dev-only how-tos. ## Toggle | | | |---|---| | Keybinding | **Cmd+Shift+D** (macOS) / **Ctrl+Shift+D** (Linux/Win) | | State | `localStorage` key `doc:show-hidden`, persisted across reloads + tabs | | Indicator | `DEV` chip in the topbar (click to turn back off) | | Per-page mark | Open-lock icon next to the entry in the sidebar | ## Marking a page hidden ```mdx file="pages/my-internal/+Page.mdx" --- title: My internal page section: Developer notes order: 1 hidden: true --- # whatever ``` That's it. The page disappears from the default sidebar; reappears with an open-lock icon when dev mode is on. ## `hidden` vs `draft` | Frontmatter | Where it shows | When to use | |---|---|---| | `draft: true` | Always in sidebar, with a `DRAFT` chip | You want reviewers to find it | | `hidden: true` | Only with dev mode on, with an open-lock icon | Internal-only — scratch, decisions, dev-tools docs | | Both | Only with dev mode on, both chips | Internal page that's still in flux | ## Where the code lives - `lib/use-hidden-toggle.ts` — the chord handler + `useLocalStorage` - `lib/navigation.ts` — `groupedPages` (all) vs `groupedVisiblePages` (filtered) - `components/Sidebar.tsx` — picks the filter based on the toggle - `components/Topbar.tsx` — the `DEV` chip (also a click target) --- Source: https://py.dreamlake.ai/examples-upload-files # Upload Files The simplest case: one file, one project, one folder. ```python file="upload.py" import dreamlake as dl with dl.Prefix(project="robotics@alice", prefix="/session-042"): dl.upload("./video.mp4", path="camera/front") ``` The SDK does five things behind the scenes: 1. Hashes the file (SHA-256, 16 hex chars) — used as the S3 staging key. 2. Detects `kind="video"` and `Content-Type: video/mp4` from the `.mp4` extension. 3. Streams the bytes to S3 in 10 MB multipart chunks via presigned URLs. Bytes never traverse the BSS server. 4. Registers a `File` record in BSS. 5. Creates a `kind="video"` Node in the tree at `/session-042/camera/front/video.mp4`, auto-creating the `camera/front` folders along the way. ## Multiple uploads share the context The `Prefix` context manager scopes every call inside the block — project, episode (= last segment of `prefix`), and base path. Three uploads, three different sub-folders, one episode: ```python with dl.Prefix(project="robotics@alice", prefix="/session-042"): dl.upload("./video.mp4", path="camera/front") dl.upload("./mic.wav", path="audio/mic-1") dl.upload("./labels.jsonl", path="detections/yolo") ``` Resulting tree: ``` robotics ← kind="project" └─ session-042 ← kind="episode" ├─ camera/front/video.mp4 ← kind="video" ├─ audio/mic-1/mic.wav ← kind="audio" └─ detections/yolo/labels.jsonl ← kind="label-track" ``` ## Auto-detected kinds | Extension | Detected `kind` | | --- | --- | | `.mp4` `.mkv` `.mov` `.webm` | `video` | | `.wav` `.mp3` `.aac` `.opus` `.flac` | `audio` | | `.jpg` `.png` `.webp` `.gif` `.bmp` `.tiff` | `image` | | `.vtt` `.srt` | `text-track` | | `.jsonl` | `label-track` | | `.parquet` | `parquet` | | `.csv` | `csv` | | `.npy` `.npz` | `npy` | | `.json` | `json` | | `.txt` `.md` `.log` | `text` | | anything else | `file` | The `kind` string is free-form. Pass `type="my-kind"` to override detection or label something the SDK doesn't know about: ```python with dl.Prefix(project="robotics@alice", prefix="/session-042"): dl.upload("./checkpoint.bin", path="models/v3", type="model-weights") ``` ## Without a Prefix context Pass `project` and `prefix` directly. Useful for one-off scripts. ```python dl.upload( "./video.mp4", project="robotics@alice", prefix="/session-042", ) ``` The episode is taken from the `Prefix` context (or, equivalently, the last segment of the `prefix=` arg above). Without a `Prefix` context the file lands directly under the project — `prefix=` only sets the folder path, not an episode wrapper. For the episode + sub-folder pattern, use `Prefix`: ```python with dl.Prefix(project="robotics@alice", prefix="/session-042"): dl.upload("./video.mp4", path="camera/front") ``` ## What you can do next - [Upload an Episode](/examples-upload-episode.md) — bulk-upload a directory of mixed files - [Bindrs](/examples-bindrs.md) — curate files across folders into named collections - [Download Files](/examples-download.md) — pull files, folders, or whole episodes back --- Source: https://py.dreamlake.ai/memos {/* GENERATED: scripts/sync-sdk-notes.mjs; edit the SDK owning docs first. */} # Memos Memos are shared documents for people and agents. From Python 0.41.0, use the canonical `memos` facade and `Memo` types: ```python import dreamlake as dl from dreamlake.api.memos import Memo, MemoSnapshot # Opening a handle is local; read() makes the request. memo = dl.memos("team/6ae3f318608512b4ba34860c") snapshot = memo.read() print(snapshot.content) ``` An explicit client provides the same interface: ```python from dreamlake.api._client import DreamLakeClient client = DreamLakeClient() memo = client.memos("6ae3f318608512b4ba34860c", namespace="team") snapshot = memo.read() receipt = memo.append(text="\nReviewed.", at=snapshot, mode="merge") ``` Use the snapshot returned by the original read as the edit baseline. The rename does not change command identity, failure outcomes, file permissions, or the exact source submitted for an edit. Collection operations are `dl.memos.list(namespace="team")` and `dl.memos.create(namespace="team", name="Review")`. The lower-level API exposes `memo`, `create_memo`, `list_memos`, `search_memos`, and `grep_memos` from either `dreamlake` or `dreamlake.api.memos`. ## Phased compatibility `dl.notes`, `client.notes`, `Note`, and the old function names remain aliases. Imports from `dreamlake.api.notes`, `_notes_facade`, and `_notes_mapped` resolve to the canonical implementations; there is one editing engine. New code should use `memos`, `Memo`, and `dreamlake.api.memos`. This SDK release uses the existing server routes and response fields. Physical database names and RTC wire changes are separate coordinated rollout steps. Existing `:note[...]` content remains readable and stays unchanged during unrelated edits. New references use `:memo[]`, never a title or slug and never a fallback to `:note`. After `created = dl.memos.create(...)`, insert `created.directive`. A handle opened by slug resolves its immutable ID before producing `.directive`; duplicate titles and renames cannot retarget it. If creation has no confirmed ID, inspect the existing Memo before retrying. ## Baseline references Python 0.41.2 accepts complete room-bound `memo-rtc:v1:` references as well as historical native `rtc:` revisions. Treat either as opaque: carry the original snapshot through edits, pinned reads and command reconciliation. Do not truncate a reference or substitute the content hash. The matching server decides whether a saved draft belongs to its current collaboration room; preserve the draft and read a fresh baseline if it reports a session change. Publishing this SDK does not activate the server's RTC migration. ## Visibility Memos support `private`, `unlisted` and `public`. Unlisted memos are readable by anyone with their direct URL, without signing in, but stay out of public profiles and discovery. Private memos require existing access; Public memos are listed. ```python memo = dl.memos.create(namespace="team", name="Review", visibility="unlisted") memo.set_visibility(visibility="unlisted") assert memo.metadata().visibility == "unlisted" ``` `dl.notes` remains an alias. `set_visibility` reads current management permission before sending one metadata-only PATCH. It preserves content, history, attachments and share links. It never retries an uncertain write; read metadata before deciding whether to try again. Only callers with `permissions.setVisibility` may change it. Changing visibility does not revoke existing share grants or grant anonymous editing. ## Public article URLs For public or unlisted memos, get or allocate a URL from the saved H1, then edit its public slug independently of the internal slug or display name: ```python memo = dl.memos("/") current = memo.permalink() changed = memo.permalink(slug="my-article") print(changed["href"], changed["aliases"]) ``` The dictionary contains `noteId`, `namespaceSlug`, `slug`, `href` (relative to the web app), and `aliases` (previous slugs). Metadata management permission is required. Title edits preserve the URL. Old slugs remain reserved aliases within the namespace; `MemoPermalinkConflict` means an explicit slug is already reserved. Choose another slug rather than retrying. Automatic allocation suffixes collisions. Copies allocate fresh URLs without inheriting alias history. Private visibility revokes anonymous access through current and previous URLs. These operations use the same REST permalink endpoint as the web UI and TypeScript SDK. ## Presence and selections Read the live roster without joining or renewing a lease: ```python roster = memo.presence(selections=True) print(roster.status, roster.selection_status, roster.observed_at) if roster.selection_status == "available": print(roster.selection_hash, roster.participants) ``` `selections` defaults to false and requires a bool. Matching servers return `status` (`available` or `inactive`), `selection_status` (`available`, `inactive` or `not_requested`), and `selection_hash` when source-bound selections are available. Older servers yield `unknown` / `unsupported`. Transport and access failures raise; they never become an empty successful roster. Participant dictionaries retain the server's `selection`: null, unresolved, or resolved with `anchor`, `head`, `start`, `end`, `unit`, `text` and `truncated`. Offsets are Unicode code points at `selection_hash`. A collapsed selection is a caret. The preview may be truncated; selection data does not prove attention or human edit authorship. Use `memo.diff(since=prior_revision)` for recent source changes and retain the normal source snapshot/revision guards for edits. ## Cover images From Python 0.44.0, set a memo's header cover independently of its collaborative body. Upload an image through the existing attachment API, then select its ID: ```python memo = dl.memos("team/") image = memo.files.upload("cover.png", path="covers/cover.png") memo.set_cover(file_id=image.id) assert memo.metadata().cover_file_id == image.id memo.set_cover(file_id=None) # Clear the header; retain the attachment. ``` Only an existing live image attachment from this memo may be selected. The server enforces metadata management permission. Readers receive the same cover metadata; images remain subject to memo access. An unconfirmed write raises `WriteOutcomeUnknown`; inspect metadata before retrying. No body write, history reset or collaboration-room replacement is involved. Source: [Python SDK guide](https://github.com/fortyfive-labs/dreamlake/blob/3e27204b95f97fcafeaa05993fa3d74f6274057c/docs/memos.md). --- Source: https://py.dreamlake.ai/examples-upload-episode # Upload an Episode A typical robot experiment run produces a mix of files in nested folders. Mirror the local layout into a DreamLake **episode** in one pass. An episode is the unit of an experiment run — a contiguous capture of related data. Everything inside it shares one logical scope. ## Local layout ``` ~/captures/run-100/ ├── camera/ │ ├── front/clip.mp4 │ └── rear/clip.mp4 ├── audio/ │ ├── mic-1/sample.wav │ └── mic-2/sample.wav ├── labels/yolo/detections.jsonl ├── data/train.parquet └── meta.json ``` ## One call — `dl.upload_folder` ```python file="upload_episode.py" import dreamlake as dl with dl.Prefix(project="robotics@alice", prefix="/run-100"): dl.upload_folder("~/captures/run-100") ``` That single line: - Walks the local directory tree recursively - Uploads each file in parallel (default 4 workers) - Mirrors the local folder structure on the server — a file at `~/captures/run-100/camera/front/clip.mp4` lands at `/run-100/camera/front/clip.mp4` in DreamLake - Auto-detects kind from extension for each file - Skips hidden / temp files (`.DS_Store`, `*.tmp`, `*.swp`, `.*`) by default - Returns a summary: `{"total", "uploaded", "failed", "errors"}` ### Tuning ```python dl.upload_folder( "~/captures/run-100", project="robotics@alice", # no Prefix context needed prefix="/run-100", parallel=8, # bump for fast networks ignore=["*.bak", "checkpoints/*"], # custom skip patterns recursive=False, # flat — only direct children on_file=lambda f, i, n: print(f" [{i+1}/{n}] {f.name}"), ) ``` ## Resulting tree ``` robotics ← project └─ run-100 ← episode ├─ camera/front/clip.mp4 ├─ camera/rear/clip.mp4 ├─ audio/mic-1/sample.wav ├─ audio/mic-2/sample.wav ├─ labels/yolo/detections.jsonl ├─ data/train.parquet └─ meta.json ``` ## Dedup is automatic Re-running the same script is safe — uploads are content-addressed by hash. A file that's already in staging at `{owner}/{project}/staging/{hash}` won't be re-uploaded. Folders along the path are upsert-only (no duplicates). ## When to drop down to the loop For special cases — selective filters, custom kinds per file, attaching tags — write the loop yourself: ```python file="upload_loop.py" from pathlib import Path import dreamlake as dl ROOT = Path("~/captures/run-100").expanduser() with dl.Prefix(project="robotics@alice", prefix=f"/{ROOT.name}"): for file in ROOT.rglob("*"): if not file.is_file(): continue if file.suffix == ".npy" and file.stat().st_size > 1e9: continue # skip huge tensors rel = file.parent.relative_to(ROOT) sub = "" if str(rel) == "." else str(rel) # top-level files dl.upload(str(file), path=sub) ``` ## Best practices - **One episode per experiment run.** Use a stable name (`run-100`, `2026-04-15-trial-01`) so re-uploads land in the same place. - **Treat the path as the only schema you need.** Folder structure encodes meaning (`camera/front/`, `labels/yolo/`) without needing tags or metadata. - **Don't fight kind detection.** If the SDK calls your file `text` because it's `.txt`, that's fine — kind doesn't gate storage, only display. - **Use `ignore=` for local clutter.** `dl.upload_folder` filters `.DS_Store`, `*.tmp`, `*.lock`, `*.swp`, and hidden files by default. Add your own patterns for project-specific clutter. ## What you can do next - [Bindrs](/examples-bindrs.md) — bundle subsets of this episode into named collections - [Download Files](/examples-download.md) — pull the whole episode back to a different machine --- Source: https://py.dreamlake.ai/notes {/* GENERATED: scripts/sync-sdk-notes.mjs; edit the SDK owning docs first. */} # Memo diffs ## MERGE and EXACT patches (Python 0.20.0) Read an original RTC snapshot, prepare a patch against its exact source, and keep that snapshot while submitting the patch. MERGE is the default and sends no `If-Match` header. The server validates the original source, compiles edits against its original RTC identities, and uses ordinary CRDT synchronization. Concurrent edits are handled by the existing CRDT behavior, without fuzzy rebasing onto the newest text. ```python import dreamlake as dl note = dl.note("/") snapshot = note.read_snapshot() # This illustrative patch requires snapshot.content == "Hello world." patch = "@@ chars 0:12 @@\n~ Hello [-world-]{+team+}.\n" receipt = note.patch(patch, base_revision=snapshot.revision) assert receipt.mode == "merge" print(receipt.to_dict()) # Optional verification of the revision acknowledged by this request: verified = note.read_snapshot(at=receipt.revision) ``` For a separate request that must refuse any intervening revision, select EXACT: ```python receipt = note.patch(patch, base_revision=snapshot.revision, exact=True) ``` EXACT is per request, never a permanent document mode. `if_match=revision` is an explicit compatibility spelling for EXACT. It can supply the original baseline when `base_revision` is absent; when both are supplied they must match. `format="inline-dff"` is the default; `format="diff"` accepts the line format. The Python client transports the patch without applying DMP or fetching a newer baseline. The API must retain the original native identity snapshot; an old content hash alone is not an RTC baseline. `NoteSnapshot` is immutable and carries `note`, `content`, `hash`, `revision`. A v2 `PatchReceipt` is a separate immutable value carrying `note`, `mode`, `base_revision`, `hash`, and `revision`. Its `to_dict()` uses the same JSON fields as CLI/API receipts: `note`, `mode`, `baseRevision`, `hash`, `revision`. `hash` describes content; `revision` identifies an RTC state. The receipt is not a string and does not pretend that these two references are interchangeable. Snapshot reads do not alter the legacy `Note`/`Doc` cache. Missing or expired original baselines, mismatched patch source, and failed EXACT checks are explicit errors. No HTTP patch request is automatically retried. Preserve the draft and original snapshot on any error. For an ambiguous lost acknowledgment, inspect the current state and reconcile before resubmitting; separate HTTP requests do not share an idempotency receipt. Unknown baselines raise `NoteNotFound`; EXACT conflicts raise `NoteChanged`; source mismatches raise `PatchFailed`; unavailable RTC responses raise `NoteBusy`. Python 0.20.0 provides this contract with the matching Memos v2 API. Use `legacy=True` for the earlier ETag patch interface. Package and deployed acceptance receipts are tracked in [workspace #706](https://github.com/dreamlake-ai/dreamlake-workspace/issues/706). ## Concurrent edit walkthrough Use a private test Memo containing exactly `Hello world.`. Run the first part, then prepend `Human: ` in the browser and wait for that edit to sync. Keep the original immutable snapshot and patch unchanged. ```python import dreamlake as dl from dreamlake import NoteChanged note = dl.note("/") original = note.read_snapshot() assert original.content == "Hello world." patch = "@@ chars 0:12 @@\n~ Hello [-world-]{+team+}.\n" input('Prepend "Human: " in the browser, wait for sync, then press Enter: ') try: note.patch(patch, base_revision=original.revision, exact=True) except NoteChanged as error: print(type(error).__name__) # NoteChanged: HTTP 412, no patch operations else: raise AssertionError("the intervening revision should reject EXACT") receipt = note.patch(patch, base_revision=original.revision) print(receipt.mode) # merge print(receipt.base_revision) # original.revision, unchanged print(receipt.hash) # content SHA-256 of the acknowledged observation print(receipt.revision) # RTC revision of that same observation print(receipt.to_dict()) # note, mode, baseRevision, hash, revision assert note.read_snapshot().content == "Human: Hello team." assert original.content == "Hello world." ``` The SDK itself prints nothing: values are returned, failures are exceptions. The example prints only when requested. Receipt `hash` and `revision` describe one coherent observation after acknowledgment, not a guarantee that all peers are synchronized or that the state remains current forever. Original native identity preservation and zero patch writes after EXACT rejection are checked by the paired API/RTC acceptance fixture. The SDK does not fall back from EXACT to MERGE or overwrite the saved snapshot after a failure. ### Successful EXACT and the next MERGE request Continue from the merged fixture, taking a new snapshot for this separate edit. If a writer intervenes again, EXACT raises `NoteChanged`; retain this snapshot and draft and inspect the memo before deciding on another request. ```python exact_base = note.read_snapshot() assert exact_base.content == "Human: Hello team." exact_patch = "@@ chars 18:18 @@\n~ {+!+}\n" exact_receipt = note.patch(exact_patch, base_revision=exact_base.revision, exact=True) print(exact_receipt.mode) # exact print(exact_receipt.base_revision) # exact_base.revision print(exact_receipt.hash) # content hash of the post-ACK observation print(exact_receipt.revision) # RTC revision of that same observation print(exact_receipt.to_dict()) # note, mode, baseRevision, hash, revision assert note.read_snapshot().content == "Human: Hello team.!" # EXACT applies to one request. This separate no-op uses the MERGE default. next_base = note.read_snapshot() next_receipt = note.patch("", base_revision=next_base.revision) assert next_receipt.mode == "merge" assert next_receipt.base_revision == next_base.revision assert next_receipt.revision == next_base.revision print(next_receipt.to_dict()) ``` ### Recorded Python fixture values The following values were replayed through the candidate SDK from an isolated API + RTC + Mongo fixture. Authentication, catalog, and S3 projections were mocked. This is test evidence, not a released-package or production transcript. ```text receipt.mode == "merge" receipt.base_revision == "rtc:9a67f239fc8b862aa557dd10dd6512a7669861287ac8fe262d9c80afebcd18e7" receipt.hash == "sha256:163af32ec4bc25f27e3b9ae68fe85c75e5b4436a769cc82a4050692643ce92cf" receipt.revision == "rtc:94c1a7696f6bcd27fa880e4b38b3f73cdd3971f28b44edf9018fadf817df0f3f" ``` `receipt.to_dict()` has the same field names as CLI/API JSON: ```json { "note": "507f1f77bcf86cd799439099", "mode": "merge", "baseRevision": "rtc:9a67f239fc8b862aa557dd10dd6512a7669861287ac8fe262d9c80afebcd18e7", "hash": "sha256:163af32ec4bc25f27e3b9ae68fe85c75e5b4436a769cc82a4050692643ce92cf", "revision": "rtc:94c1a7696f6bcd27fa880e4b38b3f73cdd3971f28b44edf9018fadf817df0f3f" } ``` EXACT raised `NoteChanged` with this message, after HTTP 412 and zero patch journal writes. The SDK emitted no stdout or stderr: ```text patch fixture/507f1f77bcf86cd799439099: The RTC baseline changed ``` The fixture's missing-baseline response was HTTP 404 with `error="revision_not_found"`, `message="Original RTC baseline is not retained"`. Malformed inline input was HTTP 422 with `error="patch_failed"`, `message="Expected inline header and one record"`. The SDK raises `NoteNotFound` and `PatchFailed` respectively; it does not turn either response into a MERGE retry against newer source. ### Recorded successful EXACT return value A separate run of the same isolated API/RTC/Mongo fixture produced this receipt, replayed through the candidate SDK. The SDK emitted no stdout or stderr; these are returned attributes, not a published-package or production transcript. ```text exact_receipt.mode == "exact" exact_receipt.base_revision == "rtc:71371be6e3227abca241161ebb7d8d65e2d851b8872b5a5d51ce7ec80369d95e" exact_receipt.hash == "sha256:2b8c0f5475f23494272f3800abb11a7680b3360bc2e927763c10aec9cf4398f5" exact_receipt.revision == "rtc:63834c3821f7b76779815f3a670449268711811e69f86f6b208fb52236713484" ``` `exact_receipt.to_dict()`: ```json { "note": "507f1f77bcf86cd799439099", "mode": "exact", "baseRevision": "rtc:71371be6e3227abca241161ebb7d8d65e2d851b8872b5a5d51ce7ec80369d95e", "hash": "sha256:2b8c0f5475f23494272f3800abb11a7680b3360bc2e927763c10aec9cf4398f5", "revision": "rtc:63834c3821f7b76779815f3a670449268711811e69f86f6b208fb52236713484" } ``` The next empty patch omitted `exact=True`, returned `mode == "merge"`, and kept the same hash and revision. Its `to_dict()` was: ```json { "note": "507f1f77bcf86cd799439099", "mode": "merge", "baseRevision": "rtc:63834c3821f7b76779815f3a670449268711811e69f86f6b208fb52236713484", "hash": "sha256:2b8c0f5475f23494272f3800abb11a7680b3360bc2e927763c10aec9cf4398f5", "revision": "rtc:63834c3821f7b76779815f3a670449268711811e69f86f6b208fb52236713484" } ``` ## Legacy ETag compatibility `read()` still returns the editable `Doc`; `Doc.save()`, whole-body writes, and section writes retain their previous ETag behavior. For old unified-diff callers, select `patch(..., legacy=True)` explicitly. That path continues to return `PatchResult(str)` with `.etag` and `.size_bytes`, and retains the previous cached ETag precondition. A legacy content ETag must not be passed as a v2 RTC baseline. ## Legacy comparisons against your last read or edit The legacy `note.read()` body read returns `doc.etag`, a quoted SHA-256 hash of the complete text. Keep it as a revision reference, including across processes: ```python import dreamlake as dl note = dl.note("") doc = note.read() ref = doc.etag # Later, including edits made by collaborators: print(note.diff(since=ref)) print(note.diff()) # defaults to this handle's last read/edit ETag # Apply your patch against the exact revision you used to prepare it: result = note.patch(my_diff, legacy=True, if_match=ref) ref = result.etag ``` The server retains note-scoped content snapshots for body/section reads and API writes. Identical content has the same hash; a ref identifies text, not an RTC operation or author. Partial body reads return the hash of the complete body. Diff inspection does not advance the handle's cached revision or write precondition. Use `read()`/`refresh()` when you want to adopt the latest state. `GET /namespaces/:slug/notes/:noteId/diff?since=` returns `diff`, `from`, `to`, and `etag` (the current ref). `context` accepts 0–100 lines, default 3. The endpoint enforces current memo read permissions. Unknown or unavailable snapshots return 404, never an empty-baseline diff. Old ETags issued before snapshot retention was deployed may be unavailable. Retention requires the matching server deployment; the SDK does not infer historical bytes. ## Inspect and patch a local draft Read a memo once, inspect edits, apply unified diffs locally, and save against that snapshot's revision: ```python note = dl.note("") doc = note.read() doc.replace("Published", query="Draft", all=True) print(doc.diff(since="last_edit")) # the latest local change print(doc.diff()) # all changes since read or successful save doc.patch(unified_diff) # edits the draft; returns its full text doc.save() # conditional remote write ``` `since="last_edit"` compares the current draft with its contents immediately before its latest content-changing operation, including element edits, patches, and `revert()`. Before any edit it returns an empty string. No-op edits and successful saves preserve this comparison; a new `read()` creates a new draft. It does not retrieve other collaborators' history or persisted server revisions. `doc.patch(diff)` accepts a single-file unified diff, including diffs from `doc.diff()`, `diff -u`, and git. Hunk positions and context must match exactly; there is no fuzzy relocation. Invalid patches raise `EditError` and leave both the draft and last-edit comparison unchanged. Empty diffs do nothing. Diffs preserve Unicode, CRLF, and missing final newlines. `context=0` is supported; the default for `diff()` is three context lines. To apply a patch directly to the remote memo, use the existing API: ```python result = note.patch(unified_diff, legacy=True, if_match=doc.etag) ``` The CLI equivalent is: ```bash dreamlake memos patch "$NOTE_ID" --legacy --file change.patch --if-match '""' ``` Remote writes reject stale revisions. A local patch is committed only when `doc.save()` succeeds; a rejected save retains the draft for inspection. ## Original-revision append (Python 0.26.0 release candidate) This candidate changes `note.append(text)` from a cached whole-body PUT returning an ETag string to a native original-revision insertion returning `PatchReceipt`. Requires Python 0.26.0; publication acceptance is tracked separately. Read exact source once, retain it, and append: ```python original = note.read_snapshot() receipt = note.append("New paragraph", at=original) verified = note.read_snapshot(at=receipt.revision) assert verified.hash == receipt.hash ``` Without `at`, append reads one fresh hash-verified snapshot. With `at`, it validates Memo identity, source hash and the original opaque revision before any network request; it never substitutes current source. The insertion uses Unicode code point positions and escapes inline framing and delimiter punctuation (`+`, `-`) in the canonical form the CLI and server use, preserving CRLF and LF bytes. An existing nonempty body lacking a final LF receives one LF before the addition, including when the addition is empty. A completely unchanged MERGE returns an observational receipt without PATCH or a new commit; it does not claim current revision verification. `exact=True` submits even an unchanged patch so the server checks the saved baseline. No request is automatically replayed. Retained `read_snapshot(at=revision)` verifies the returned revision and source hash even if another edit has since arrived. `if_match=revision` instead checks current source; combining the two is rejected. A successful patch updates the legacy cached ETag only in its content-hash domain, never to an RTC revision. Unknown write outcomes preserve caches and drafts: keep the original baseline and draft, inspect retained/current source, and reconcile before resubmitting. For previous cached-text/LF/ETag PUT behavior and string return values, choose `note.append(text, legacy=True)`. Existing `force=True` also selects that legacy path. Combining either with `at` or `exact` is rejected before HTTP. `Doc.save()`, whole-body writes and section APIs retain their existing behavior. ## Native body drafts (Python 0.27.0 release candidate) Python 0.27.0 changes `note.write(text)` from whole-body PUT to disjoint Unicode-codepoint patches against original RTC source. `at=original_snapshot` pins identity and source without GET; without it one verified snapshot is read before planning. Unchanged gaps remain untouched, and alignment work/trace/source limits fail before PATCH instead of replacing a coarse region. No automatic replay occurs. Explicit `legacy=True`, existing `force=True`, or legacy `if_match=etag` retain the ETag PUT/string contract; modern `at`/`exact` cannot be mixed with those compatibility options. `note.read()` creates a modern editable Doc retaining its original immutable snapshot. Edits remain local. `doc.save()` submits against that same snapshot, returns `PatchReceipt`, and reads the acknowledged retained revision to adopt its coherent merged content as the next draft/original. A different write through the Memo handle does not move this draft's baseline. `exact=True` checks even unchanged drafts; unchanged MERGE returns an observational receipt without a new commit or a current-revision claim. `note.read(at=revision)` opens retained source. If post-ACK readback fails, the raised `NoteError.receipt` contains the known acknowledgment; draft and original source stay available. Further save calls are refused until the caller reconciles: inspect `note.read_snapshot(at=receipt.revision)` and current source, then explicitly open a new draft and reapply intended edits. This failure is not a rollback and must not be blindly resubmitted. Ambiguous PATCH failures retain drafts and snapshots without adopting a newer baseline. For legacy editable drafts, choose `note.read(legacy=True)` or the legacy ETag verification `if_match=etag`; their save behavior and `SaveResult` are retained. `doc.save(legacy=True)` / `force=True` explicitly choose legacy body behavior. Publication acceptance remains separate. Partial reads, section writes and section selector mapping remain legacy and are a separate pending migration; their ranges must not be guessed into original native snapshot coordinates. ## Full source reads and remote inspection (Python 0.28.0 release candidate) Default `note.refresh()` and `note.text` read full source through verified Memos v2 snapshots. Cached ETags stay in the quoted content-hash domain; RTC revisions stay on immutable snapshots. `note.read()` also populates that verified cache, while existing Docs retain their own original snapshot even after another refresh. `read_snapshot()` itself still leaves caches unchanged. Explicit `refresh(legacy=True)` and `get_text(legacy=True)` preserve older body-read behavior. Legacy append and legacy editable reads select that path explicitly on cache miss. No v2 failure silently falls back to a legacy read. `note.diff(since=content_hash)` uses the shipped v2 body inspection endpoint and continues returning a diff string. It checks Memo, base hash, format and revision, then reads the retained acknowledged revision to verify its source hash. Inspection updates no cache or draft baseline. `since` is a content hash (raw, quoted ETag or sha256-prefixed), never an RTC revision or a native patch baseRevision. Omitting since uses this handle's last full read/edit content-hash ETag. The returned string is inspection material; retaining only its hash does not establish a writable original identity snapshot. The v2 endpoint has fixed diff formatting and no context-selection parameter. Explicit nondefault context selects the retained compatibility `/diff` endpoint before transport, preserving existing calls; `legacy=True` also selects it. This is option-based compatibility, never a fallback after a failed v2 response. V2 context selection remains an unresolved Namespace-owned endpoint capability. There is no existing Python DiffResult value; no new protocol or parser is introduced. Sections and partial reads still await their retained range/discovery contracts and remain an explicit separate migration. Publication acceptance for Python 0.28.0 remains separate. ## The Memos API facade (Python 0.34.0) `dl.notes` implements the Python tabs of the [Memos API](https://docs.dreamlake.ai/notes/api/) design pages. Stable IDs are primary, every operation takes keyword arguments, and the facade is an adapter over the retained-baseline contract above: edits compile to one `inline-dff` record against the original revision and go through the same validated `PATCH`. `dl.note()`, `read_snapshot()` and the released `note.append()` separator behavior are unchanged. ```python import dreamlake as dl note = dl.notes(note_id) # local; the server resolves the owner on first use snapshot = note.read() # NoteSnapshot {note, content, hash, revision} receipt = note.append(text="\nHello\n", at=snapshot) # verbatim: no separator is added acknowledged = note.read(at=receipt.revision) page = dl.notes.list(namespace="your-workspace", limit=20, offset=0) created = dl.notes.create(namespace="your-workspace", name="Review", text="# Review\n") slug = dl.notes("design-doc", namespace="your-workspace") # a slug needs namespace context ``` ### Reading `read()` returns exact source. `read(view="mapped")` returns a `MappedRead`: the same exact `content`, its absolute code-point `chars`, the observed `elements` (`address`, `chars`, `lines`, `section`) and the reading view in `text` — generated `` hints that are never source. It is the snapshot the CLI's `--view html` reads (`view=html&contentFormat=literal-markdown`), validated the same way. Exactly one of `lines=(a, b)` (1-based, inclusive; past-the-end is refused), `section="s1"` or `address="s1.p2"` scopes a read; the last two need the mapped view. `at=` reads a retained revision and verifies it. `outline(at=)` lists headings with addresses; `find(text=, case_sensitive=False, at=)` searches exact source and returns absolute ranges; `diff(since=revision, format="diff")` returns base/result revisions and hashes with the patch. A `FocusedRead`/scoped `MappedRead` carries the whole memo's `hash` and `revision` but `complete=False`; it baselines a targeted edit inside its span and is refused (`SnapshotRequired`) as a whole-body baseline. A complete mapped read is accepted wherever a snapshot is. ### Editing ```python original = note.read() note.replace(lines=(40, 42), text="- Updated item\n", at=original) note.insert(after="s1.p2", text="- New item\n", at=original) note.remove(lines=(40, 42), at=original) note.write(text="Hello team.\n", at=original, mode="exact") note.patch(patch=unified_diff, format="diff", at=original, mode="exact") ``` `replace`/`insert`/`remove` take exactly one selector or placement (`lines`, `address`; `before`, `after`, `cursor`) and default to `mode="merge"`; an address must have been observed at `at`'s revision (a `MappedRead` resolves it without a request). `write` and `patch` require `mode`; `patch` requires `format`. Replacement text is verbatim: no newline, indentation or marker is added, and input containing `