getCatalogSchema

A function for retrieving and processing schema information from a Singer catalog for a specific stream. This function extracts the schema definition and ensures proper formatting for array types.

Basic Usage

import * as gs from '@hotglue/gluestick-ts';

const schema = gs.getCatalogSchema("users");
console.log(schema.properties);

Key Methods


getCatalogSchema(stream: string): SingerHeaderMap

Retrieves the schema definition for a specific stream from the Singer catalog and processes it for proper formatting.

Usage

// Get schema for a specific stream
const userSchema = gs.getCatalogSchema("users");

// Use schema with export functions
gs.toExport(df, "users", "./output", {
    exportFormat: 'singer',
    schema: userSchema
});

Parameters

  • stream (string): Name of the stream to get schema for (can be stream name or tap_stream_id)

Returns

  • SingerHeaderMap: Processed schema object with proper type definitions and array handling

Schema Structure

The returned schema object has the following structure:
{
  type: ["object", "null"],
  properties: {
    // Stream properties with their types and definitions
  }
}

Error Handling

The function throws errors in the following cases:
  • No catalog found: When the catalog file cannot be read or doesn’t exist
  • Stream not found: When the specified stream doesn’t exist in the catalog
try {
    const schema = gs.getCatalogSchema("nonexistent_stream");
} catch (error) {
    console.error(error.message);
    // "No schema found in catalog for stream nonexistent_stream"
}

Common Patterns

Get Schema for Multiple Streams

import * as gs from '@hotglue/gluestick-ts';

const reader = new gs.Reader();
const availableStreams = reader.keys();

const schemas = {};
for (const stream of availableStreams) {
    try {
        schemas[stream] = gs.getCatalogSchema(stream);
    } catch (error) {
        console.warn(`No schema found for ${stream}:`, error.message);
    }
}

Notes

  • Requires a valid Singer catalog file in the project directory
  • The function accesses the catalog through the Reader class’s private readCatalog method
  • Array types are automatically normalized to include items properties
  • The returned schema is compatible with Singer export formats
  • Schema properties are preserved as-is from the original catalog