@mantlebee/ts-data
    Preparing search index...

    2.3.0 — SQL null semantics, explicit lookup pages, ESM declarations

    Version 2.3.0 lets the in-memory reader treat unset fields as SQL does, on request; makes the paging constraint of the reader-based lookup sources explicit with a RangeError; ships separate type declarations for ES module consumers and compiles to ES2020 instead of ES6, as @mantlebee/ts-core 1.5.0 does. The release is additive: the defaults are unchanged, and code that compiles and runs against 2.2.0 compiles and runs against 2.3.0 in every environment listed in section 3.


    The in-memory reader is the fake of a backend in specs and stories, so it should answer what the backend answers. Up to 2.2 an unset field (null or undefined) is a value like any other: it differs from every set value, so status notEqual "archived" keeps the items whose status is null. SQL (Postgres, PostgREST) answers differently: NULL <> 'archived' is unknown, and the row is dropped. A spec written against the fake then passes where the backend fails, or the other way round.

    InMemoryQueryItemsReaderOptions has a new option, read by filterItems, itemMatchesFilter, itemMatchesFilterExpression, queryItems, countItems, createInMemoryQueryItemsReader and createInMemoryLookupSource:

    type InMemoryNullSemantics = "javascript" | "sql";

    type InMemoryQueryItemsReaderOptions<TItem> = {
    caseSensitive?: boolean;
    /** `"javascript"` by default. */
    nullSemantics?: InMemoryNullSemantics;
    toComparable?: InMemoryQueryItemsComparer<TItem>;
    };
    const { countReader, reader } = createInMemoryQueryItemsReader(() => users, {
    nullSemantics: "sql",
    });

    "javascript", the default, keeps the behaviour of 2.2. With "sql" an unset field never matches notEqual, notIn nor notContains, and still matches equal null / in [null]. A field is unset when its comparable is null: null, undefined, a missing property, or whatever a custom toComparable turns into null. countItems reads the same option, so the count agrees with the pages.

    How each filter matches an unset field:

    Filter "javascript" "sql"
    equal null, equalInsensitive null ✅ ✅
    equal x, equalInsensitive x ❌ ❌
    notEqual x ✅ ❌
    notEqual null ❌ ❌
    in [x, null] ✅ ✅
    in [x], in [] ❌ ❌
    notIn [x], notIn [] ✅ ❌
    notIn [x, null] ❌ ❌
    contains x ❌ ❌
    notContains x ✅ ❌
    startsWith x, endsWith x ❌ ❌
    greaterThan, greaterThanOrEqual, lessThan, lessThanOrEqual ❌ ❌
    range (closed, half-open or open) ❌ ❌
    • The ordered and string operators never matched an unset field, in 2.2 as well: only the three negated operators change.
    • A set field matches the same way with both semantics: notEqual null keeps every set field, notIn [x, null] keeps the fields other than x. Strict SQL would make x NOT IN (a, NULL) never true; the reader keeps a null in the list meaning "unset", as in [x, null] does.
    • Sorting is not affected: unset values come first ascending and last descending, as before (MySQL and SQLite order them the same way; Postgres puts them last ascending).
    • To keep the unset items under "sql", say so, as in SQL: status notEqual "archived" or status equal null.

    Every row of the table is covered by a test, for null, undefined and a missing property.


    createLookupSource and createInMemoryLookupSource translate the { limit, offset } of search into the 1-based, page-based Pagination of the payload: { itemsPerPage: limit, page: offset / limit + 1 }. That only works when offset is a multiple of limit, as it is when a panel pages by its own limit from 0. 2.2 already rejected the other offsets, with a plain Error; 2.3 rejects them with a RangeError, still without calling the reader, and documents the constraint on LookupPage:

    search(query, page) 2.2.0 2.3.0
    { limit: 20, offset: 40 } page 3 page 3
    { limit: 20, offset: 30 } rejects Error rejects RangeError
    { limit: 0, offset: 0 } rejects Error rejects RangeError
    { limit: 20, offset: -20 }, 0.5 rejects Error rejects RangeError

    The messages are unchanged, e.g. The offset of a lookup page must be a non-negative multiple of its limit (20), got 30. RangeError extends Error, so a catch or a rejects.toThrow() written for 2.2 still holds. A hand-written ILookupSource may accept any offset.


    The package has no "type": "module", so its .d.ts files are CommonJS declarations for TypeScript. Up to 2.2 exports["."] had a single types condition, shared by import and require: with moduleResolution node16 / nodenext, an ES module importing the package got CommonJS declarations for an ES module file ("masquerading as CJS" in arethetypeswrong). The build now runs scripts/emit-esm-types.mjs after tsc-alias, which copies every dist/src/**/*.d.ts to dist/esm/**/*.d.mts adding the .mjs extensions that ES modules require, and each condition points at its own declarations, as in @mantlebee/ts-core 1.5.0 and @mantlebee/ts-random:

    "exports": {
    ".": {
    "import": {
    "types": "./dist/esm/index.d.mts",
    "default": "./dist/esm/index.mjs"
    },
    "require": {
    "types": "./dist/src/index.d.ts",
    "default": "./dist/index.js"
    }
    }
    }

    The top-level main, module and types fields are unchanged, for tools that don't read exports. With moduleResolution: "bundler" (Vite, esbuild) TypeScript picks the import condition, so it now reads the .d.mts, which declare the same API. arethetypeswrong reports no problem for node10, node16 (from CommonJS and from ESM) and bundler.

    The JavaScript is compiled to ES2020 instead of ES6. Optional chaining, nullish coalescing, async / await, exponentiation and object spread are emitted as written instead of being rewritten into helpers: no more __awaiter / __generator state machines, so the output is smaller, debuggable and its stack traces follow the source; the tslib helpers inlined into dist/esm/vendor are gone. Class fields are still compiled as before (they are ES2022), so their semantics are unchanged: the declare fields of a DataSource subclass (see 2.2.0) behave as they did.

    Minimum environment: browsers released since 2020 (Chrome / Edge 80, Firefox 74, Safari 13.1) and Node.js 14 or later.

    This is not a breaking change because the actual minimum was already ES2020: the peer @mantlebee/ts-core 1.4.0 uses globalThis (ES2020) at runtime, and the read cycle of DataSource depends on its AbortableDelegate, hence on AbortController (Safari 12.1, Node 15). The one setup to check is a toolchain whose parser predates ES2020 and that parses node_modules without transpiling them (webpack 4, whose acorn 6 rejects ?.): transpile the package with Babel or upgrade to webpack 5. Vite, esbuild, Rollup, webpack 5, Jest and Node.js parse ES2020 natively.

    Item 2.2.0 2.3.0
    exports["."] types one types for both import.types → .d.mts, require.types → .d.ts
    ESM declarations — dist/esm/**/*.d.mts
    build script … && tsc-alias … && tsc-alias && node scripts/emit-esm-types.mjs
    Compilation target ES6 ES2020
    dist/esm/vendor (tslib) inlined helpers removed, no helpers needed
    @mantlebee/ts-core peer ^1.4.0 peer ^1.4.0, developed against 1.5.0

    New, from the package root: the InMemoryNullSemantics type. New option: InMemoryQueryItemsReaderOptions.nullSemantics. Nothing else changes.


    1. Update the dependency: npm install @mantlebee/ts-data@^2.3.0.
    2. Where an in-memory reader fakes a SQL backend, pass nullSemantics: "sql" and drop the notEqual x or equal null workarounds that only the fake needed.
    3. Where a spec asserts the error of a misaligned lookup page, it may now check RangeError.
    4. If you compile with moduleResolution node16 / nodenext and had workarounds for the CommonJS declarations of the ESM entry (a // @ts-ignore, a paths override), remove them.
    5. If you run on a browser older than 2020 or on Node.js 12, or bundle with webpack 4 without transpiling node_modules, transpile the package or stay on 2.2.0 (but see section 3: the data source already required newer environments).