Scan

Use the Scan command to retrieve multiple items from a DynamoDB table. This is the Document Builder command for a Scan operation.

Basic Usage

Scan a table to retrieve all items.

const todoEntity = new DynamoEntity({
  table: parentTable,
  schema: z.object({
    todoId: z.string(),
    userId: z.string(),
    title: z.string(),
    completed: z.boolean(),
  }),
  partitionKey: todo => key('USER', todo.userId),
  sortKey: todo => key('TODO', todo.todoId),
});

const { items, lastEvaluatedKey } = await todoEntity.send(new Scan());

Scan Filtering

Apply filter conditions to reduce the items returned from a scan. Note that filters are applied after reading items, so you still pay for all scanned items.

const productEntity = new DynamoEntity({
  table: parentTable,
  schema: z.object({
    productId: z.string(),
    name: z.string(),
    price: z.number(),
    inStock: z.boolean(),
  }),
  partitionKey: product => key('PRODUCT', product.productId),
});

const scanCommand = new Scan({
  filter: and(
    { price: greaterThan(10) },
    { price: lessThan(100) },
    { inStock: true }
  ),
});

const result = await productEntity.send(scanCommand);

Scans are Expensive

Scan operations read every item in the table or index, making them expensive. Use Query operations when possible. Filters don’t reduce read capacity costs—they only reduce what’s returned to you.

Defining Conditions

Document Builder provides a expressive API for defining conditions over DynamoDB’s base ConditionExpression, see Conditions for more details.

Secondary Indexes Scans

Scan a Global Secondary Index or Local Secondary Index by specifying the indexName parameter.

const todoEntity = new DynamoEntity({
  table: parentTable,
  schema: z.object({
    userId: z.string(),
    todoId: z.string(),
    status: z.string(),
    priority: z.number(),
    title: z.string(),
  }),
  partitionKey: todo => key('USER', todo.userId),
  sortKey: todo => key('TODO', todo.todoId),
  globalSecondaryIndexes: {
    StatusIndex: {
      partitionKey: todo => key('STATUS', todo.status),
      sortKey: todo => todo.priority,
    },
  },
});

const scanCommand = new Scan({
  indexName: 'StatusIndex',
  filter: { priority: greaterThan(5) },
});

const result = await todoEntity.send(scanCommand);

Projected Attributes in Secondary Indexes

When scanning a Secondary Index, only attributes projected into that index are available. If you need attributes not in the projection, you’ll need to fetch the full items separately.

Scan Limiting

Use the limit parameter to restrict the maximum number of items evaluated during a scan.

const scanCommand = new Scan({
  limit: 100,
});

const result = await todoEntity.send(scanCommand);
// Evaluates at most 100 items
console.log(result.items.length); // <= 100

if (result.lastEvaluatedKey) {
  // More items are available
  const nextScan = new Scan({
    limit: 100,
    exclusiveStartKey: result.lastEvaluatedKey,
  });
  const nextResult = await todoEntity.send(nextScan);
}

Limit Evaluation

The limit parameter sets the maximum number of items to evaluate (before filtering), not necessarily the number returned. If you use a filter condition, the actual number of items returned may be less than the limit.

Consistent Reads

Set consistent to true for strongly consistent reads when you need the most up-to-date data.

const scanCommand = new Scan({
  consistent: true,
});

const result = await todoEntity.send(scanCommand);

Consistent Scans are Super Expensive

Consistent reads consume twice as much read capacity as eventually consistent reads and are not supported on Global Secondary Indexes. Only use when absolutely necessary (this is quite literally one of the most expensive operations you can do with DynamoDB).

Scan Pagination

Document Builder provides automatic pagination support using async generators to process large tables efficiently.

const scan = new Scan({
  filter: { completed: false },
  pageSize: 100,
});

for await (const page of todoEntity.paginate(scan)) {
  console.log(`Page with ${page.count} items`);
  await processItems(page.items);

  // Break early if needed
  if (page.items.length === 0) {
    break;
  }
}

Note the important difference of using paginate() on the entity instead of send().

Page Size

Use the pageSize parameter to control how many items are evaluated per request. This helps manage memory usage and allows for processing items in batches.

Projection Scans

Use the ProjectedScan command to retrieve only specific attributes from items.

const projectedScanCommand = new ProjectedScan({
  projection: ['title', 'completed'],
  projectionSchema: z.object({
    title: z.string(),
    completed: z.boolean(),
  }),
  filter: {
    completed: false,
  },
});

const result = await todoEntity.send(projectedScanCommand);
// result.items is typed as Array<{ title: string; completed: boolean; }>
console.log(result.items[0]?.title);

Reducing costs by using Projections

Projection expressions are useful when working with large items but only needing a subset of attributes. This reduces the amount of data transferred and can lower read capacity costs.

Parallel Scans

For large tables, use segment and totalSegments to perform a parallel scan.

async function parallelScan(totalSegments: number)
  const allItems = [];

  async function scanSegment(segmentNumber: number) {
    const scan = new Scan({
      segment: segmentNumber,
      totalSegments: totalSegments,
    });

    for await (const page of todoEntity.paginate(scan)) {
      allItems.push(...page.items);
    }
  }

  const scanPromises = Array.from({ length: totalSegments }, (_, i) =>
    scanSegment(i, totalSegments)
  );

  const results = await Promise.all(scanPromises);
  return allItems;
}

Scan Command Config

The Scan command expects the following input config:

{
  indexName?: string;
  filter?: Condition;
  limit?: number;
  selectAttributes?: Select;
  consistent?: boolean;
  validationConcurrency?: number;
  segment?: number;
  totalSegments?: number;
  exclusiveStartKey?: Partial<Schema>;
  pageSize?: number;
  skipValidation?: boolean;
  timeoutMs?: number;
  abortController?: AbortController;
  returnConsumedCapacity?: ReturnConsumedCapacity;
}

All config properties are optional.

Property	Type	Description
indexName	string	The name of a secondary index to scan. If not provided, the table will be scanned.
filter	Condition	An optional filter condition to apply to the results of the scan. See Conditions for more details on building conditions.
limit	number	The maximum number of items to return in the scan.
selectAttributes	Select	Specify a subset of attributes to retrieve from the items returned by the scan. Valid values are ALL_ATTRIBUTES, ALL_PROJECTED_ATTRIBUTES, COUNT, and SPECIFIC_ATTRIBUTES.
consistent?	boolean	If set to true, DynamoDB will ensure a strongly consistent read. Defaults to false.
validationConcurrency	number	The number of concurrent validations to perform when validating items against the entity schema. Internally p-map is used to manage concurrency, default is 64.
segment	number	For parallel scans, the segment number to scan (0-based). Must be used with totalSegments.
totalSegments	number	For parallel scans, the total number of segments to divide the scan into. Must be used with segment.
exclusiveStartKey	Partial<Schema>	The primary key of the item from which to continue an earlier scan. This is used for pagination when a previous scan returned a lastEvaluatedKey.
pageSize	number	The number of items per paginated request to DynamoDB. Only applies when using Document Builder’s pagination.
skipValidation?	boolean	If set to true, schema validation is bypassed entirely. Defaults to false.
timeoutMs?	number	Number of milliseconds to wait before the operation times out and auto-cancels.
abortController?	AbortController	If you need to abort the commands operation, you can use the abort controller to signal cancellation.
returnConsumedCapacity?	ReturnConsumedCapacity	Determines the level of detail about provisioned throughput consumption that is returned in the response. Valid values are TOTAL, INDEXES, and NONE.

Scan Command Result

The Scan command returns the following result:

{
  items: Schema[];
  count: number;
  scannedCount: number;
  lastEvaluatedKey?: Partial<Schema>;
  responseMetadata?: ResponseMetadata;
  consumedCapacity?: ConsumedCapacity;
}

Property	Type	Description
items	Schema[]	An array of items matching the scan. Will be an empty array if no items matched.
count	number	The number of items returned in the items array.
scannedCount	number	The number of items that were scanned in DynamoDB to produce the results. This may be higher than count if a filter condition was applied.
lastEvaluatedKey?	Partial<Schema>	If present, indicates that there are more items to retrieve for the scan. This key can be used as the exclusiveStartKey in a subsequent scan to continue retrieving results.
responseMetadata?	ResponseMetadata	Metadata about the response from DynamoDB.
consumedCapacity?	ConsumedCapacity	Information about the capacity units consumed by the operation, if requested via the returnConsumedCapacity config.

Projected Scan Command Config

The ProjectedScan command expects the following input config:

{
  projection: string[];
  projectionSchema: ZodObject;
  indexName?: string;
  filter?: Condition;
  limit?: number;
  selectAttributes?: Select;
  consistent?: boolean;
  validationConcurrency?: number;
  segment?: number;
  totalSegments?: number;
  exclusiveStartKey?: Partial<Schema>;
  pageSize?: number;
  skipValidation?: boolean;
  timeoutMs?: number;
  abortController?: AbortController;
  returnConsumedCapacity?: ReturnConsumedCapacity;
}

Property	Type	Description
projection (required)	string[]	An array of attribute names to include in the returned items.
projectionSchema (required)	ZodObject	A Zod schema defining the shape of the projected items.
indexName	string	The name of a secondary index to scan. If not provided, the table will be scanned.
filter	Condition	An optional filter condition to apply to the results of the scan. See Conditions for more details on building conditions.
limit	number	The maximum number of items to return in the scan.
selectAttributes	Select	Specify a subset of attributes to retrieve from the items returned by the scan. Valid values are ALL_ATTRIBUTES, ALL_PROJECTED_ATTRIBUTES, COUNT, and SPECIFIC_ATTRIBUTES.
consistent?	boolean	If set to true, DynamoDB will ensure a strongly consistent read. Defaults to false.
validationConcurrency	number	The number of concurrent validations to perform when validating items against the entity schema. Internally p-map is used to manage concurrency, default is 64.
segment	number	For parallel scans, the segment number to scan (0-based). Must be used with totalSegments.
totalSegments	number	For parallel scans, the total number of segments to divide the scan into. Must be used with segment.
exclusiveStartKey	Partial<Schema>	The primary key of the item from which to continue an earlier scan. This is used for pagination when a previous scan returned a lastEvaluatedKey.
pageSize	number	The number of items per paginated request to DynamoDB. Only applies when using Document Builder’s pagination.
skipValidation?	boolean	If set to true, schema validation is bypassed entirely. Defaults to false.
timeoutMs?	number	Number of milliseconds to wait before the operation times out and auto-cancels.
abortController?	AbortController	If you need to abort the commands operation, you can use the abort controller to signal cancellation.
returnConsumedCapacity?	ReturnConsumedCapacity	Determines the level of detail about provisioned throughput consumption that is returned in the response. Valid values are TOTAL, INDEXES, and NONE.

Projected Scan Command Result

The ProjectedScan command returns the following result:

{
  items: ProjectionSchema[];
  count: number;
  scannedCount: number;
  lastEvaluatedKey?: Partial<Schema>;
  responseMetadata?: ResponseMetadata;
  consumedCapacity?: ConsumedCapacity;
}

Property	Type	Description
items	ProjectionSchema[]	An array of items matching the scan. Will be an empty array if no items matched. Unlike the standard Scan, these items will be typed according to the provided projection schema.
count	number	The number of items returned in the items array.
scannedCount	number	The number of items that were scanned in DynamoDB to produce the results. This may be higher than count if a filter condition was applied.
lastEvaluatedKey?	Partial<Schema>	If present, indicates that there are more items to retrieve for the scan. This key can be used as the exclusiveStartKey in a subsequent scan to continue retrieving results.
responseMetadata?	ResponseMetadata	Metadata about the response from DynamoDB.
consumedCapacity?	ConsumedCapacity	Information about the capacity units consumed by the operation, if requested via the returnConsumedCapacity config.

Tree-shakable Imports

import { Scan } from 'dynamo-document-builder/commands/scan';
import { ProjectedScan } from 'dynamo-document-builder/commands/projected-scan';

API Reference

• Scan
• ScanConfig
• ScanResult
• ProjectedScan
• ProjectedScanConfig
• ProjectedScanResult