RFC: Add listByType(typeHash) to Store interface with fs index #9

New Issue

2026-05-18T14:09:20Z

xiaoju commented

2026-05-18 14:09:20 +00:00

Summary

Add listByType(typeHash): Hash[] to the Store interface, backed by an append-only file index in json-cas-fs.

Motivation

Currently, finding all nodes of a given type requires iterating every node in the store (store.list() + filter). This is O(n) over all CAS nodes — unacceptable as the store grows.

The immediate use case is uwf cas schema list which needs all nodes whose type is the meta-schema hash.

Design

1. Store interface (`json-cas/src/types.ts`)

export type Store = {
  put(typeHash: Hash | null, payload: unknown): Promise<Hash>;
  get(hash: Hash): CasNode | null;
  has(hash: Hash): boolean;
  list(): Hash[];
  listByType(typeHash: Hash): Hash[];  // NEW
};

2. FS index (`json-cas-fs`)

Index files stored at <cas-dir>/_index/<typeHash>, one hash per line, append-only:

cas/_index/056WS17E5F9E5   # all nodes with type = 056WS17E5F9E5
cas/_index/7M9ESCS02BX94   # all nodes with type = 7M9ESCS02BX94

put(): after writing the .bin file, appendFileSync(indexPath, hash + \"\\n\")
listByType(typeHash): readFileSync(indexPath) → split by newline
loadDir(): also build an in-memory Map<Hash, Hash[]> type index from existing index files (or rebuild from .bin files if index is missing)
Index is append-only, no locking needed
Missing index file → return []

3. In-memory store (`json-cas/src/store.ts`)

The in-memory createStore() should also implement listByType using a Map<Hash, Set<Hash>> maintained on put().

4. Migration

For existing CAS directories without _index/, on first loadDir() build the index from the loaded .bin files and write the index files. This is a one-time cost.

Tasks

Add listByType(typeHash): Hash[] to Store type in json-cas/src/types.ts
Implement in-memory index in json-cas/src/store.ts
Implement fs index in json-cas-fs/src/store.ts with _index/<typeHash> files
Handle migration: rebuild index from existing nodes on first load
Update tests
Update json-cas exports if needed

## Summary Add `listByType(typeHash): Hash[]` to the `Store` interface, backed by an append-only file index in `json-cas-fs`. ## Motivation Currently, finding all nodes of a given type requires iterating every node in the store (`store.list()` + filter). This is O(n) over all CAS nodes — unacceptable as the store grows. The immediate use case is `uwf cas schema list` which needs all nodes whose type is the meta-schema hash. ## Design ### 1. Store interface (`json-cas/src/types.ts`) ```typescript export type Store = { put(typeHash: Hash | null, payload: unknown): Promise<Hash>; get(hash: Hash): CasNode | null; has(hash: Hash): boolean; list(): Hash[]; listByType(typeHash: Hash): Hash[]; // NEW }; ``` ### 2. FS index (`json-cas-fs`) Index files stored at `<cas-dir>/_index/<typeHash>`, one hash per line, append-only: ``` cas/_index/056WS17E5F9E5 # all nodes with type = 056WS17E5F9E5 cas/_index/7M9ESCS02BX94 # all nodes with type = 7M9ESCS02BX94 ``` - **`put()`**: after writing the `.bin` file, `appendFileSync(indexPath, hash + \"\\n\")` - **`listByType(typeHash)`**: `readFileSync(indexPath)` → split by newline - **`loadDir()`**: also build an in-memory `Map<Hash, Hash[]>` type index from existing index files (or rebuild from `.bin` files if index is missing) - Index is append-only, no locking needed - Missing index file → return `[]` ### 3. In-memory store (`json-cas/src/store.ts`) The in-memory `createStore()` should also implement `listByType` using a `Map<Hash, Set<Hash>>` maintained on `put()`. ### 4. Migration For existing CAS directories without `_index/`, on first `loadDir()` build the index from the loaded `.bin` files and write the index files. This is a one-time cost. ## Tasks - [ ] Add `listByType(typeHash): Hash[]` to `Store` type in `json-cas/src/types.ts` - [ ] Implement in-memory index in `json-cas/src/store.ts` - [ ] Implement fs index in `json-cas-fs/src/store.ts` with `_index/<typeHash>` files - [ ] Handle migration: rebuild index from existing nodes on first load - [ ] Update tests - [ ] Update `json-cas` exports if needed

xiaoju referenced a pull request that will close this issue

2026-05-18 14:12:36 +00:00

feat: add listByType(typeHash) to Store interface #10

xiaomo closed this issue

2026-05-18 14:13:16 +00:00

xiaoju commented

2026-05-18 16:20:41 +00:00

Closing: listByType implemented (PR #10), published as 0.2.0

— 小橘 🍊（NEKO Team）

Closing: listByType implemented (PR #10), published as 0.2.0 — 小橘 🍊（NEKO Team）

This repo is archived. You cannot comment on issues.

1 Participants

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: uncaged/json-cas#9

RFC: Add listByType(typeHash) to Store interface with fs index #9