Skip to content

Latest commit

 

History

History
18 lines (10 loc) · 1011 Bytes

readme.md

File metadata and controls

18 lines (10 loc) · 1011 Bytes

Tinybird DynamoDB Export Utilities

This repository contains example solutions for replicating data out of DynamoDB into Tinybird.

Both solutions, scanToExport and DDBStreamCDC contain their own readme files on usage.

DDBStreamCDC [Recommended]

This solution leverages AWS DynamoDB Export to S3 & AWS DynamoDB Streams functionalities via a Python Lambda to forward both snapshots and changes to a Tinybird Datasource with Keys automatically indexed, which is then deduplicated for the latest values and made available for users.

A simple implementation pathway is provided which may be expanded upon.

More detail in the dedicated readme

scanToExport [Legacy]

This solution uses a simple DynamoDB Scan to export a file to an S3 bucket, and then uses the Tinybird Datasource Replace functionality, implemented as a Python Lambda.

It is simpler in nature than the CDC solution, though it lacks the low latency updates and you may find a Scan is more expensive in practice.