Reader
A class for reading ETL files from the sync-output directory. This is the recommended way to read ETL files as it provides consistent error handling and metadata access.Basic Usage
Key Methods
get(stream: string, options): DataFrame
Read a specific stream as a Polars DataFrameUsage
Parameters
stream
(str): Name of stream to readoptions
(obj)catalogTypes
: (boolean) - Use catalog for automatic type inference- Other options will be passed through to Polars when reading. See ReadCSV and ReadParquet options for more information
getPk(stream: string): string[]
Gets primary key(s) for a stream from catalogUsage
Parameters
stream
(string): Name of stream to get primary keys for
Returns
string[]
: Array of primary key column names
keys(): string[]
Gets all available stream names from the input files.Usage
Returns
string[]
: Array of all available stream names
Common Patterns
Iterating through multiple streams
etl.ts