Skip to main content
Programmatically manage your transformation scripts using the etl commands below.

​
ETL Download

​
Description

Clones the remote ETL script saved in hotglue to your local machine.

​
Sample


$ hotglue etl download [--overwrite] [--downloadTo]
✔ Finished: Verifying user and authorizing.
✔ Finished: Scanning for downloadable files.
ℹ Info: Downloading script files to ./scripts/tap.
✔ Finished: Downloading file: etl.ipynb.
┌───────────┬────────────┐
│ File      │ Status     │
├───────────┼────────────┤
│ etl.ipynb │ Downloaded │
└───────────┴────────────┘

​
Parameters

OptionDefaultDescription
--overwrite
-ofalseWhen enabled, overwrites any files that already exist locally in the download to directory.
--downloadTo
-d.The directory to download the ETL to. Defaults to the local directory.

​
ETL Deploy

​
Description

Deploys the local ETL script to hotglue.

​
Sample

$ hotglue etl deploy [--sourceFolder]
✔ Finished: Verifying user and authorizing.
✔ Finished: Validating flow and tap location.
✔ Finished: Preparing deployment target.
ℹ Info: Deploying ETL scripts.
✔ Finished: Pushing file: default/flows/bTHIweD0W/taps/cin7/etl/etl.ipynb.
┌─────────────────────────────────────────────────┬──────────┐
│ File                                            │ Status   │
├─────────────────────────────────────────────────┼──────────┤
│ default/flows/bTHIweD0W/taps/cin7/etl/etl.ipynb │ Deleted  │
├─────────────────────────────────────────────────┼──────────┤
│ default/flows/bTHIweD0W/taps/cin7/etl/etl.ipynb │ Deployed │
└─────────────────────────────────────────────────┴──────────┘

​
Parameters

OptionDefaultDescription
--sourceFolder
-s.The directory to upload the ETL script from. Defaults to the local directory.

​
ETL Delete

​
Description

Deletes a deployed ETL script on hotglue.

​
Sample

$ hotglue etl delete
ℹ Info: Deleting ETL scripts for Tenant tenantId Flow flowId and Tap tapId to envId.
✔ Finished: Verifying user and authorizing.
✔ Finished: Deleting ETL scripts.
┌───────────┬─────────┐
│ File      │ Status  │
├───────────┼─────────┤
│ etl.ipynb │ Deleted │
└───────────┴─────────┘

​
ETL Set up Local Job Data

​
Description

Clones hotglue job data to your local machine and creates a .env file with the job’s environment variables. The file structure and content is identical to the file system your ETL script ran in. The downloaded etl-output folder will be renamed to etl-output-reference. We recommend using this command to reproduce ETL failures or back test again successful jobs.

​
Sample


$ hotglue etl setup-local-run tenant123/flows/At_kHalC/jobs/2026/02/09/05/7Y6iUA [--include-configs] [--overwrite] [--downloadTo]
✔ Finished: Verifying user and authorizing.
✔ Finished: Scanning for downloadable files.
┌───────────────────────────────┬─────┬──────────────┐
│ File                                            │ Size    │ LastModified         │
├───────────────────────────────┼─────┼──────────────┤
│ catalog.json                                │ 571     │ 1/9/2026, 6:29:30 PM │
├───────────────────────────────┼─────┼──────────────┤
│ sync-output/products-20220209T222727.csv        │ 3226419 │ 1/9/2026, 5:27:34 PM │
└───────────────────────────────┴─────┴──────────────┘
ℹ Info: Downloading files to `.`
✔ Finished: Downloading file: catalog.json
✔ Finished: Downloading file: sync-output/products-20220209T222727.csv

​
Parameters

OptionDefaultDescription
--include-configsfalseWhen enabled, also downloads target-config.json, source-config.json, tenant-config.json and sets the API_KEY environment variable.
--overwrite
-ofalseWhen enabled, overwrites any files that already exist locally in the download to directory.
--downloadTo
-d.The directory to download the job data to. Defaults to the local directory.

​
Running the ETL with the jobs environment variables

After running the setup-local-run command a .env file will be created containing the same environment variables that were available when the job ran in the hotglue environment.
In order to run the ETL with those same environment variables use one of the following methods:

​
For VSCode and it’s variants

Open the launcher file {project_folder}/.vscode/launch.json and add the envFile entry for launch configuration: 
{
    "version": "0.2.0",
    "configurations": [
        ...,
        {
            "name": "Run ETL script",
            "type": "debugpy",
            "request": "launch",
            "program": "${workspaceFolder}/etl.py",
            "console": "integratedTerminal",
            "cwd": "${workspaceFolder}",
            "python": "${workspaceFolder}/.venv/bin/python",
            "envFile": "${workspaceFolder}/.env"
        }
    ]
}

​
For Linux/macOS/Git bash/WSL terminal

Run the following: 
source .env
python etl.py