Version 2.0.0 makes the extensions of a data source part of its type.
Until 1.2.0 IDataSource<TItem> promised nothing about its extensions: a
consumer discovered them at runtime with getExtension(Class) and
instanceof, imported the concrete classes to probe them, and failed on
mount when the one it needed was missing. Now a source built with
createDataSource(reader, { filtering, pagination, distinct }) carries its
extensions key by key, a consumer declares what it needs in a signature
(source: IDataSourceWith<User, "filtering" | "pagination">) and the
compiler checks it.
The release is breaking: the extensions are given by key instead of as an
ordered list (still in the order you write them), the DataSource class is
no longer exported (createDataSource is the only way to build a source),
getExtension is gone, and the two positional hooks of an extension become
buildPayload and readItems, the second one wrapping the reading
(1.6).
The paging of the payload is renamed to pagination (Paging is
Pagination, payload.paging is payload.pagination). read(), the events
and everything under query-items keep their semantics.
IDataSource<TItem, TExt>IDataSourceWith<TItem, TCapability>createDataSourceextensions/ folder and the exportsPaging is PaginationA source takes its extensions as a record, a key per extension, the way a
Subscribable of ts-core takes its events as an EventsDefinition
(DataSourceExtensionsDefinition). Any key holds any extension of the item
type: the data source knows no key.
type DataSourceExtensionsDefinition<TItem> = Dictionary<
IDataSourceExtension<TItem> | undefined
>;
Every extension given is exposed under its key as a readonly own
property of the source, set only when given: "sorting" in source is false
for a source without sorting.
The four extensions of the package have a conventional key, each one typed
with its interface (DataSourceCapabilities): a
FilterExtensionWithFixedFilters is a filtering, a SingleSortExtension
is a sorting. The convention lives in IDataSourceWith
(1.3), where a consumer asks for
them; the record itself does not enforce it.
type DataSourceCapabilities<TItem> = {
filtering: IFilterExtension<TItem>;
pagination: IPaginationExtension<TItem>;
projection: IProjectionExtension<TItem>;
sorting: ISortExtension<TItem>;
};
type DataSourceCapability = keyof DataSourceCapabilities; // "filtering" | "pagination" | "projection" | "sorting"
IDataSource gains a second type parameter, the record of its extensions:
type IDataSource<
TItem = Any,
TExt extends DataSourceExtensionsDefinition<TItem> = Record<never, never>,
> = IDataSourceBase<TItem> & DataSourceExtensionProperties<TExt>;
undefined is a required readonly property.undefined (sorting: maybeSorting, or
sorting?: in a type you write) is an optional one, without the
undefined once narrowed.IDataSource carries no key of its own.IDataSourceWith<User, "filtering" | "pagination"> →
IDataSourceWith<User, "filtering"> → IDataSource<User>); the opposite
direction is a compile error.IDataSource<User> accepts every source and exposes no extension.type UsersSource = IDataSource<
User,
{ filtering: IFilterExtension<User>; distinct?: DistinctExtension<User> }
>;
IDataSource is now a type alias (an intersection) instead of an
interface: TypeScript does not let an interface extend a mapped type over a
generic record. Nothing changes for an annotation, an import or an
implements IDataSource<User> with concrete type arguments; only a
declaration merging on IDataSource would stop working. The members that do
not depend on the extensions (items, isReading, read()) live in the new
interface IDataSourceBase<TItem>.
The shortcut for the extensions of the package, the common case of a
signature: a union of keys of DataSourceCapabilities resolved to the record
of their interfaces.
type IDataSourceWith<
TItem,
TCapability extends DataSourceCapability,
> = IDataSource<TItem, Pick<DataSourceCapabilities<TItem>, TCapability>>;
// IDataSource<User, { filtering: IFilterExtension<User>; pagination: IPaginationExtension<User> }>
function useTable(source: IDataSourceWith<User, "filtering" | "pagination">) {
source.filtering.addFilter(adult); // required
source.pagination.goToNextPage();
}
useTable(createDataSource(readUsers, { filtering, pagination, sorting })); // more than asked: fits
useTable(createDataSource(readUsers, { filtering })); // compile error: no `pagination`
useTable(createDataSource(readUsers, { filtering: sorting, pagination })); // compile error: a sorting is not a filtering
A record is needed only to name a custom key or an optional one, and the two compose by intersection:
type UsersSource = IDataSourceWith<User, "filtering"> &
IDataSource<User, { distinct: DistinctExtension<User> }>;
createDataSource(reader, extensions) is the way to build a source: it
returns the IDataSource typed with the record it is given, custom keys
included, inferring everything from the arguments.
function createDataSource<
TItem,
TExt extends DataSourceExtensionsDefinition<TItem>,
>(reader: DataSourceReader<TItem>, extensions: TExt): IDataSource<TItem, TExt>;
const users = createDataSource(readUsers, { filtering, pagination, distinct });
// IDataSource<User, { filtering: FilterExtension<User>; pagination: PaginationExtension<User>; distinct: DistinctExtension<User> }>
users.filtering.addFilter(adult);
users.distinct; // DistinctExtension<User>
const maybeSorting: SortExtension<User> | undefined = sortable
? new SortExtension<User>()
: undefined;
const products = createDataSource(readProducts, {
filtering,
sorting: maybeSorting,
});
products.sorting; // SortExtension<User> | undefined
The record is checked against the extension contract: every value must be an
IDataSourceExtension of the item type. The keys are not: { filtering: sorting } compiles, and fails where a consumer asks for
IDataSourceWith<User, "filtering">. Write the shape of a record as a type
literal, not an interface: an interface has no implicit index signature and
does not satisfy the constraint.
The extensions run in the order of the keys of the record, the order you
write them in (Object.keys: JavaScript lists integer-like keys first, so
name the extensions), custom keys and keys of the package alike. The library
imposes no order: as in 1.2.0 it is yours to choose and it matters. Write
filtering and sorting before pagination, so the count reader receives
them and the page reset watches them; whatever you write before pagination
is part of the count and of the hash, whatever you write after is not. The
hooks, the reader and the signal are invoked exactly as in 1.2.0, in that
order.
createDataSource(readUsers, { filtering, sorting, pagination, projection });
// getItemsBefore: filtering, sorting, pagination (count with filters and sorts), projection
createDataSource(readUsers, { pagination, filtering });
// getItemsBefore: pagination (count without filters), filtering: almost never what you want
Until 1.2.0 an extension had two positional hooks, getItemsBefore(payload, signal?) building the payload and getItemsAfter(items, payload, signal?)
transforming the items read, with the reader invoked by the data source in a
fixed position between the two. An extension could neither skip the reader
nor call it more than once, which ruled out a cache (a hit reads nothing) and
a batching (a backend taking one sort or one filter per call: N reads,
results merged). The hooks are now split by responsibility:
buildPayload(payload, signal?) is getItemsBefore renamed: the build
phase, in the order of the keys, completed before any item is read. The
count of the pagination stays here.readItems(payload, next, signal?) wraps the reading, like a
middleware. The extensions with the hook form an onion around the reader:
the first key is the outermost layer, the reader the core, and
next(payload), a DataSourceReadNext<TItem>, runs the layers inside. An
extension without the hook is skipped. next may be called zero times (a
cache hit: the reader is never invoked), once, with this or another
payload, or N times (a batching), every call running the inner layers and
the reader again, so the hooks and the events of the inner extensions may
fire N times. The result may be transformed before being returned.getItemsAfter is removed: const items = await next(payload); return transform(items); covers it. Mind the change of semantics: the
transformations run in reverse order of the keys, the outermost
extension transforming last, as in every middleware (until 1.2.0 they ran in
list order).
Two phases instead of one "around" hook so that readItems always sees the
whole payload, whatever the position of the extension: with a single hook
a cache written first would not see the pagination added by a later key, and
would have to be last.
/** A read-through cache: a hit skips `next`, hence the reader. */
class CacheExtension<TItem>
extends Subscribable<EventsDefinition>
implements IDataSourceExtension<TItem>
{
private readonly _cache = new Map<string, TItem[]>();
public async readItems(
payload: DataSourcePayload<TItem>,
next: DataSourceReadNext<TItem>,
): Promise<TItem[]> {
const key = createHash(payload);
const hit = this._cache.get(key);
if (hit) return hit;
const items = await next(payload);
this._cache.set(key, items);
return items;
}
}
/** A backend taking one sort per call: one read per sort, results merged. */
class OneSortPerCallExtension<TItem>
extends Subscribable<EventsDefinition>
implements IDataSourceExtension<TItem>
{
public async readItems(
payload: DataSourcePayload<TItem>,
next: DataSourceReadNext<TItem>,
): Promise<TItem[]> {
const sorts = payload.sorts ?? [];
if (sorts.length < 2) return next(payload);
const reads = await Promise.all(
sorts.map((sort) => next({ ...payload, sorts: [sort] })),
);
return merge(reads);
}
}
The signal. next has the signal of the read bound already: nothing to
forward (forgetting it would lose the cancellation in silence), hence its
own type with the payload as only parameter. The signal parameter of the
hook stays for the reads of the extension itself, as the count reader of the
pagination. The latest read wins holds inside the chain: every layer
built by the data source, the reader included, checks the signal before and
after running and rejects with the AbortedException of ts-core once
aborted. So a next called after the supersession rejects, a reader
resolving late rejects the next awaiting it, and a result returned in
silence by an extension swallowing that rejection never reaches the outer
layers nor the source. The cycle still tells the abort (no event, items
untouched, read() rejected with AbortedException at once) from a real
error (error event and rejection).
Not in this release, and worth knowing before writing those extensions: a
cache in readItems does not intercept the count of the pagination, which
runs in the build phase with a reader of its own (decorate the count reader
on the consumer side, or wait for an option of the pagination recounting only
when the hash of the upstream query changes); and a batching does not
compose with the pagination for free, since merging N results already
paginated does not give the right page.
The DataSource class is no longer exported: createDataSource is the
only way to build a source. A class cannot type its members from a generic
record, so new DataSource(reader, { filtering }) would have exposed no key
in its type, and two ways to build the same thing, one of them untyped, is
one too many. Behind the factory the class takes (reader, extensions), the
record of 1.1, runs the extensions in the order of
the keys and sets every one as an own property; a key that is already a
member of the source (items, read, isReading, on, dispose, …)
throws. Annotate with IDataSource / IDataSourceWith
(1.2, 1.3);
a source of your own implements IDataSource<User> instead of extending the
class.
A custom extension gets a key of its own in the record, nothing else: no registration, no module augmentation, and the key is scoped to the source that uses it, so two sources can give the same key different types.
class DistinctExtension<TItem>
extends Subscribable<EventsDefinition>
implements IDataSourceExtension<TItem>
{
public async buildPayload(payload: DataSourcePayload<TItem>) {
payload.distinct = true;
return payload;
}
public async readItems(
payload: DataSourcePayload<TItem>,
next: DataSourceReadNext<TItem>,
) {
return Array.from(new Set(await next(payload)));
}
}
const users = createDataSource(readUsers, {
filtering,
distinct: new DistinctExtension<User>(),
pagination,
}); // distinct runs between filtering and pagination: the count asks for distinct items
users.distinct; // DistinctExtension<User>
key in source tells whether the extension is there.undefined is absent: no property, nothing in the
pipeline, optional in the type.buildPayload
of each one, then readItems of each one as an onion around the reader,
the first key outermost, the result flowing outward.next runs the inner layers and the reader every time it is called, zero,
one or N times; once the read is superseded every layer rejects with
AbortedException.dispose() disposes every extension of the pipeline.read(), isReading, items, the events and the latest-read-wins
behaviour are those of 1.2.0.extensions/ folder and the exportsEverything about the extensions as such moves from the files of
data-source/ to data-source/extensions/, next to the extensions shipped
with the package; the barrel re-exports it, so the import paths of the
package do not change.
New:
createDataSource (data-source/utils.ts)IDataSourceBase (data-source/interfaces.ts)IDataSourceWith (data-source/extensions/interfaces.ts)DataSourceCapabilities, DataSourceCapability,
DataSourceExtensionsDefinition, DataSourceExtensionProperties,
DataSourceReadNext (data-source/extensions/types.ts)Moved: IDataSourceExtension (data-source/extensions/interfaces.ts),
DataSourceExtension and DataSourceExtensionChangeDelayDefault
(data-source/extensions/models.ts), DataSourceExtensionOptions
(data-source/extensions/types.ts).
Changed: IDataSource is a type alias with a second, defaulted type
parameter; IDataSourceExtension.getItemsBefore is renamed buildPayload,
and the extensions of the package implement it under that name; Paging is
renamed Pagination (query-items/pagination/types.ts).
Removed: the DataSource class (no longer exported, see
1.7), IDataSource.getExtension and
IDataSourceExtension.getItemsAfter (replaced by readItems, see
1.6).
npm run lint runs ESLint (@eslint/js and typescript-eslint recommended
rules, Prettier conflicts off), prettier --check and the type-check;
npm run lint:fix and npm run format fix what they can. The configuration
is eslint.config.mjs and .prettierignore; Prettier runs with its defaults.
Paging is PaginationThe type and the property that carry the page requested are renamed after the
name the rest of the package already uses (PaginationExtension,
IPaginationExtension, the pagination key of the extensions, the
pagination/ folders):
type Pagination = {
itemsPerPage: number;
page: number;
totalItemsCount: number;
};
type QueryItemsPayload<TItem> = {
// ...
pagination?: Pagination; // was `paging?: Paging`
};
Nothing else changes: pageItems(items, pagination?) keeps its name, and the
PaginationExtension still adds the pagination to the payload after reading
the total count. resetPageOnQueryChange: true hashes every property of the
payload except pagination.
new DataSource(...) no longer compiles: the class is not exported.
Build with createDataSource(reader, { filtering, pagination }), the
record replacing the list, the keys in the order you want the extensions to
run (1.5). A subclass of DataSource
implements IDataSource<User> instead.getExtension(Class) is removed: read the keyed property
(source.filtering, source.pagination, source.projection,
source.sorting, or your own key).Paging is Pagination and payload.paging is payload.pagination:
an import of the type, a reader that reads payload.paging and a
resetPageOnQueryChange list naming "paging" no longer compile or match.IDataSource is a type alias: a declaration merging on it, or an
implements IDataSource<T, X> with a generic X, no longer compiles.items, read, …) throws
at construction.getItemsBefore is buildPayload and getItemsAfter is gone: an
extension transforming the items implements
readItems(payload, next, signal?) and transforms the result of
next(payload). The transformations run in reverse order of the keys, the
first key last, where they ran in list order.npm install @mantlebee/ts-data@^2.0.0 @mantlebee/ts-core@^1.4.0
createDataSource(reader, { filtering, pagination, ... })
instead of new DataSource(reader, [filtering, pagination, ...]), every
custom extension under a key of your choice, the keys in the order the
list had.source.getExtension<FilterExtension<User>>(FilterExtension) with
source.filtering (PaginationExtension → pagination, SortExtension /
SingleSortExtension → sorting, ProjectionExtension → projection).
Where the source may lack the extension, type it with an optional key
(sorting?: ISortExtension<User>) and narrow: if (source.sorting).source: IDataSourceWith<User, "filtering" | "pagination"> or a record
with your own keys, and let the compiler reject a source that cannot do
it, instead of throwing at runtime.Paging to Pagination and payload.paging to
payload.pagination in your readers, in the payloads you build by hand
and in the in-memory helpers you call with them.getItemsBefore to buildPayload, and
turn getItemsAfter(items, payload, signal?) into
readItems(payload, next, signal?):
const items = await next(payload); return transform(items);. If two of
your extensions transform the items, check their order: the first key now
transforms last.