toExport

​

toExport

​

Basic Usage

import * as gs from '@hotglue/gluestick-ts';

const df = pl.DataFrame({...});
gs.toExport(df, "users", "./etl-output");

​

Key Methods

​

toExport(data, name, outputDir, options)

​

Usage

// Basic singer export
gs.toExport(df, "users", "./etl-output");

// Export with custom options
gs.toExport(df, "users", "./etl-output", {
    exportFormat: 'parquet',
    keys: ['user_id'],
    outputFilePrefix: 'processed_'
});

​

Parameters

• data (pl.DataFrame): Polars DataFrame to export
• name (string): Name for the output file
• outputDir (string): Directory path for output files
• options (ExportOptions): Optional configuration object

​

ExportOptions Interface

​

Properties

• keys? (string[]): Primary keys for the dataset (used in singer format)
• exportFormat? (‘csv’ | ‘json’ | ‘jsonl’ | ‘parquet’ | ‘singer’): Output format (defaults to ‘singer’ or DEFAULT_EXPORT_FORMAT env var)
• outputFilePrefix? (string): Prefix to add to output filenames
• stringifyObjects? (boolean): Whether to stringify object values
• reservedVariables? (Record<string, string>): Custom variables for prefix formatting
• allowObjects? (boolean): Whether to allow object values in singer format (defaults to true)
• schema? (SingerHeaderMap): Singer schema definition

​

Common Patterns

​

Iterating through multiple streams and exporting

import * as gs from '@hotglue/gluestick-ts';

// iterate through each stream in the input directory
const input = new gs.Reader();
const availableStreams = input.keys();

for (const key of availableStreams) {
    const inputDf = input.get(key, { catalogTypes: true });
    
    // Export the transformed data
    gs.toExport(inputDf, key, "./etl-output");
}