JSON & Object Handling

​

explode_json_to_cols

Converts JSON array columns into separate columns, with one column per array value.

​

Usage

from gluestick.pandas_utils import explode_json_to_cols

# Basic column explosion
df = explode_json_to_cols(df, 'array_column')

# With options
df = explode_json_to_cols(
    df,
    'array_column',
    drop=True,
    inplace=False
)

​

Parameters

• df (pd.DataFrame): Input DataFrame
• column_name (str): Column containing JSON arrays
• drop (bool): Whether to drop original column
• inplace (bool): Modify DataFrame in place
• parser (function): Custom parsing function (optional)

​

Returns

DataFrame with array values expanded into separate columns

​

explode_json_to_rows

Takes a column containing an array of objects and expands it into multiple rows, creating columns for each object property.

​

Usage

from gluestick.pandas_utils import explode_json_to_rows

# Basic explosion
df = explode_json_to_rows(df, 'array_column')

# With nesting control
df = explode_json_to_rows(
    df, 
    'array_column',
    drop=True,
    max_level=2
)

​

Parameters

• df (pd.DataFrame): Input DataFrame
• column_name (str): Column containing JSON arrays
• drop (bool): Whether to drop original column
• max_level (int): Maximum nesting level for flattening
• parser (function): Custom parsing function (optional)

​

Returns

DataFrame with array objects expanded into separate rows

​

json_tuple_to_cols

Convert JSON tuple columns into separate columns based on key-value pairs. Useful for transforming nested JSON data into a flattened DataFrame structure.

​

Installation

from gluestick.pandas_utils import json_tuple_to_cols

​

Basic Usage

# Basic conversion with default settings
df = json_tuple_to_cols(df, 'json_column')

# Custom key-value mapping
config = {
    'cols': {'key_prop': 'CategoryName', 'value_prop': 'CategoryValue'},
    'look_up': {'key_prop': 'name', 'value_prop': 'value'}
}
df = json_tuple_to_cols(df, 'json_column', col_config=config)

​

Parameters

• df (pd.DataFrame): Input DataFrame
• column_name (str): Column containing JSON tuples
• col_config (dict): Configuration for key-value mapping
  • cols: Output column names
  • look_up: Input property names

​

Returns

DataFrame with JSON tuple column split into separate columns

​

compress_rows_to_col

Compresses previously exploded rows back into a single column containing array data.

​

Usage

from gluestick.pandas_utils import compress_rows_to_col

# Basic compression
df = compress_rows_to_col(df, 'line_items', 'invoice_id')

# With CSV storage
df = compress_rows_to_col(
    df,
    'line_items',
    'invoice_id',
    use_csv=True
)

​

Parameters

• df (pd.DataFrame): Input DataFrame
• column_prefix (str): Prefix of columns to compress
• pk (str): Primary key for grouping rows
• use_csv (bool): Use CSV format for storage

​

Returns

DataFrame with specified columns compressed into array format

​

array_to_dict_reducer

Creates a reducer function that converts arrays into dictionaries using specified key-value properties.

​

Usage

from gluestick.pandas_utils import array_to_dict_reducer
from functools import reduce

# Create reducer
reducer = array_to_dict_reducer(
    key_prop='name',
    value_prop='value'
)

# Apply to array
result = reduce(reducer, array_data, {})

​

Parameters

• key_prop (str): Property to use as dictionary key
• value_prop (str): Property to use as dictionary value

​

Returns

Function that reduces arrays to dictionaries

​

Notes

• Returns a reducer function to be used with functools.reduce
• Raises AttributeError if values aren’t dictionaries
• Handles both specified key-value pairs and full dictionary merging

​

clean_obj_null_values

Replaces null values with None in stringified objects for further processing.

​

Usage

from gluestick.etl_utils import clean_obj_null_values

# Clean stringified object
cleaned = clean_obj_null_values(json_string)

​

Parameters

• obj (str): Stringified dictionary/list with null values

​

Returns

String with ‘null’ values replaced with ‘None’

​

Notes

• Returns empty dict () for pandas NA values
• Particularly useful before using explode functions
• Preserves original object structure for non-null values

​

Common Patterns

​

Processing Nested JSON Data

import gluestick as gs
import pandas as pd

# Example with nested invoice data
def process_invoice_lines(df):
    # Clean nulls first
    df['line_items'] = df['line_items'].apply(gs.clean_obj_null_values)
    
    # Explode to rows
    df_lines = gs.explode_json_to_rows(df, 'line_items')
    
    # Process and transform
    # ... apply transformations ...
    
    # Compress back to column if needed
    df = gs.compress_rows_to_col(df_lines, 'line_items', 'invoice_id')
    
    return df