Do I need Python installed to use NetCDF Studio?

Only if you use Offline mode or run a Compute Node. Python 3.9 or later is required for local processing. Online mode handles everything in the cloud with no local installation required.

Welcome to NetCDF Studio

Q: What file formats can I upload to NetCDF Studio?

NetCDF Studio accepts .zip archives containing NetCDF (.nc) files from the IMDAA reanalysis dataset. Google Earth Engine jobs do not require a file upload — the server fetches the data directly using your GEE credentials.

Q: What is the maximum upload size?

In Online mode, the limit is 3 GB per file. In Offline mode, there is no limit — processing happens locally on your machine.

Q: Does NetCDF Studio charge me for Google Earth Engine usage?

No. NetCDF Studio does not charge you for GEE API calls, data retrieval, or Earth Engine compute credits. GEE usage is billed directly by Google to your GEE account, subject to Google's own free tier and quotas. Your NetCDF Studio plan covers the server-side computation (CPU, memory, workers) used to process the data once it is fetched.

Q: What is a Compute Node in NetCDF Studio?

A compute node is a secondary machine you connect to your NetCDF Studio server to add processing capacity. It runs the netcdf-studio-node agent, which dials out to the server and receives jobs over WebSocket. No port forwarding is required.

Your guide to converting IMDAA reanalysis archives into research-ready CSV datasets.

What Can You Do?

Upload & Convert

Upload raw IMDAA ZIP archives and convert them into structured CSV files with live progress tracking.

Apply Corrections

Run scientific correction engines on your CSV data — temperature, radiation, humidity, wind speed, and more.

Convert Units

Transform units across your datasets — Kelvin to Celsius, W/m² to MJ/m², and other scientific conversions.

Go Offline

Install the PWA app and process unlimited files locally without any upload size limits.

Online vs Offline Mode

Feature	Online (Cloud)	Offline (Local)
Upload Limit	3 GB per file	Unlimited
Processing	Cloud server (+ optional compute nodes)	Your machine
Requires Login	Yes (Email or Google Sign-In)	Optional
Requires Install	No	Yes (PWA + Node.js)
Internet Required	Yes	No
Google Earth Engine	Yes (Pro / Max plans)	Yes (with local GEE setup)

Uploading Archives

How to upload IMDAA reanalysis ZIP files for conversion.

Step-by-Step

1

Go to the Dashboard

Sign in to your account and navigate to the Home page. You'll see the upload area at the center of the screen.

2

Select Your Archive

Drag and drop your IMDAA .zip archive onto the upload zone, or click to browse. The file must contain NetCDF (.nc) files from IMDAA reanalysis datasets.

3

Configure & Submit

After the archive is inspected, you can choose your output format and variable selections. Click Submit to begin processing.

4

Download Results

Track progress in the My Requests page. Once complete, download your CSV files directly from the dashboard.

💡

You can re-use previously uploaded archives. Go to the upload step and select from your existing archive list instead of uploading again.

Tracking Requests

Monitor your conversion jobs in real time.

Request Statuses

Status	Meaning
Queued	Your request is waiting to be processed.
Processing	The server is actively converting your archive. You'll see a live progress bar.
Completed	All CSV files are ready for download.
Failed	Something went wrong. Check the error message and try again.

What You Can Do

View Progress — Live percentage and ETA updates as your job runs.
Download Files — Once complete, download individual CSVs or all at once.
Delete Requests — Remove completed or failed requests from your history.
Re-run — Use the same archive to create new CSV variants with different settings.

Offline Mode

Process unlimited files locally on your own computer — no internet required.

📱

Offline mode is only available when you install NetCDF Studio as a PWA app. Visit the site in Chrome or Edge, then click "Install" in the browser's address bar.

How It Works

When you switch to Offline mode in the dashboard, NetCDF Studio connects to a local server running on your machine instead of the cloud. This removes the 3GB upload limit and processes everything locally.

Setting Up Offline Mode

1

Install the PWA App

Open NetCDF Studio in Chrome or Edge. Click the install icon in the address bar (or the "Install App" prompt). This adds the app to your desktop.

2

Install Node.js

Download and install Node.js (v18 or later) from nodejs.org.

3

Install NetCDF Studio Package

Open a terminal and run:

$ npm i -g netcdf-studio

4

Start the Local Server

$ netcdf-studio start

This starts the local server. Open the NetCDF Studio PWA app and switch to Offline mode using the toggle in the header.

⚠️

Keep the terminal window open while using offline mode. Closing it will stop the local server. To restart, run netcdf-studio start again.

Correction Engines

Apply scientific correction pipelines to your processed CSV data.

What Are Correction Engines?

IMDAA provides raw 3-hourly reanalysis data. Correction engines transform this raw data into daily, scientifically standardized products that are ready for research — converting units, aggregating timesteps, and applying domain-specific formulas.

How to Apply Corrections

1

Navigate to Corrections

Go to the Corrections page from the sidebar menu.

2

Select a Source File

Choose a processed CSV from your completed requests. The system will show which correction modules are compatible with your data.

3

Choose a Module

Select the correction engine to apply. Each module shows what it does and what outputs you'll receive.

4

Review & Download

Once processing completes, preview the results and download the corrected CSV files.

Available Modules

Module	What It Does	Output
🌡️ Temperature	Converts 3-hourly Kelvin data to daily °C products	Tmean, Tmax, Tmin
☀️ Radiation	Integrates W/m² flux into daily energy totals	SW, LW, Net Radiation (MJ/m²/day)
💧 Relative Humidity	Converts fractional RH to daily percentage stats	RH mean, max, min (%)
☁️ Vapor Pressure	Derives actual vapor pressure from dewpoint	ea (kPa)
🌬️ Wind Speed	Computes magnitude from U/V components at 2m height	Daily mean wind speed (m/s)
🌧️ Rainfall Erosivity	Estimates R-factor from sub-daily rainfall	Erosivity index (MJ·mm/ha/hr)
🌱 Soil Moisture	Processes 4-layer Noah LSM soil moisture data	Layer-wise daily mean, SWS (mm), Root Zone, Surface

Unit Conversion

Transform units across your CSV datasets with built-in scientific formulas.

How to Use

1

Open the Unit Converter

Navigate to the Unit Conversion page from the sidebar. You'll see the converter interface.

2

Upload Your CSV

Upload a processed CSV file. The system reads the columns and lets you select which ones to convert.

3

Choose Conversions

Select the source unit and target unit for each column. The formula used is shown transparently so you can verify the transformation.

4

Download Converted CSV

Click Download to get your unit-converted CSV with standardized column names and clear unit labels.

Common Conversions

From	To	Formula
Kelvin (K)	Celsius (°C)	T(°C) = T(K) − 273.15
W/m²	MJ/m²/day	E = flux × 0.0864
Pa	kPa	P(kPa) = P(Pa) / 1000
m/s	km/h	v(km/h) = v(m/s) × 3.6
kg/m²/s	mm/day	P(mm) = P(kg/m²/s) × 86400

File Management

Organize, download, and manage your processed archives and CSV outputs.

Your Workspace

The Files page shows all your archives and their associated outputs organized by upload. Each archive can have multiple CSV variants from different processing runs.

What You Can Do

Download — Download individual CSV files or all outputs from an archive at once.
Preview — Quick-view the first rows of any CSV to verify the data before downloading.
Re-process — Select an existing archive and run it again with different settings.
Delete — Remove files you no longer need to free up storage.

💡

Your storage quota depends on your membership plan. Check your Account page to see how much storage you have remaining.

Scientific Modules Reference

How each correction module transforms your IMDAA data.

🌡️ Temperature

Converts 3-hourly temperatures from Kelvin to Celsius, then produces daily aggregates.

UNIT CONVERSION

T(°C) = T(K) − 273.15

Products: Tmean (daily mean), Tmax (daily maximum), Tmin (daily minimum).

☀️ Radiation

Integrates instantaneous flux values (W/m²) into daily energy totals (MJ/m²/day).

FLUX INTEGRATION

R_daily = Σ(i=1 to 8) [ R_i × 0.03888 ]
where 0.03888 = 10800 seconds / 10⁶ (J → MJ)

Products: Net Shortwave, Net Longwave, Net Radiation.

💧 Relative Humidity

Converts fractional RH (0–1) to percentage and computes daily statistics.

CONVERSION

RH(%) = RH(fraction) × 100

Products: RH mean, RH max, RH min.

☁️ Vapor Pressure

Derives actual vapor pressure from dewpoint temperature using the Magnus formula.

MAGNUS FORMULA

ea = 0.6108 × exp[ (17.27 × Td) / (Td + 237.3) ]

Products: Daily mean actual vapor pressure (kPa).

🌬️ Wind Speed

Computes wind speed from U and V vector components with FAO-56 height adjustment.

VECTOR MAGNITUDE + HEIGHT ADJ.

WS = √(U² + V²)
WS_2m = WS_z × [ 4.87 / ln(67.8z − 5.42) ]

Products: Daily mean wind speed at 2m (m/s).

🌧️ Rainfall Erosivity

Estimates the R-factor (erosive potential) from sub-daily rainfall intensity.

R-FACTOR

R = Σ(EI₃₀) where EI₃₀ = kinetic energy × max 30-min intensity

Products: Daily precipitation (mm), Erosivity index.

🌱 Soil Moisture

Processes 4-layer Noah LSM soil moisture data (m³/m³) to compute layer-wise daily means, total soil water storage, and root zone averages.

SOIL WATER STORAGE

SWS = Σ(θᵢ × Zᵢ) → mm of water
Layers: L1 (0–10cm, 100mm) · L2 (10–35cm, 250mm) · L3 (35–100cm, 650mm) · L4 (100–300cm, 2000mm)

ROOT ZONE MOISTURE

θ_root = (θ₁×Z₁ + θ₂×Z₂ + θ₃×Z₃) / (Z₁ + Z₂ + Z₃) → m³/m³

Products: Layer-wise daily mean, Total SWS (mm), Root Zone moisture (0–1 m), Surface layer (0–10 cm), Monthly summaries.

Build Custom Correction Engines

Create your own scientific correction modules and share them with the research community.

🔧

Custom engines let you define your own scientific pipeline — input variables, formulas, output schema, and documentation — all uploadable from the Corrections page in the dashboard.

What You Need

Every custom correction module requires two files:

File	Format	Purpose
logic.py	Python (.py)	The processing script with your scientific logic
theory.md	Markdown (.md)	Scientific documentation shown to users in the wizard

Input File Requirements

Your module will receive a processed IMDAA CSV file. These CSVs always contain:

Column	Type	Description
`lat`	Float	Latitude coordinate
`lon`	Float	Longitude coordinate
`time`	Datetime	Timestamp of the observation
`station`	String	Station name or ID
`height`	Float	Station elevation (m)
`grid`	String	Grid cell identifier
`loc_id`	String	Location identifier (lat_lon)
Variable columns	Float	The actual data values (e.g., temperature, wind speed)

Variable Mapping & Unit Conversion Flow

Before your apply_correction() runs, the engine performs two preprocessing steps automatically:

1

Column Mapping

The user maps each of your REQUIRED_VARIABLES to a CSV column. The engine sends the column indices (0-based) to the Python worker. Your module receives only the mapped columns plus the standard metadata columns (lat, lon, time, station, height, grid, loc_id, date).

2

Unit Conversion

If the user specifies source → target unit conversions (e.g., Kelvin → Celsius), the engine applies the conversion formula before calling your apply_correction(). Your function receives already-converted data. You do not need to handle unit conversion in your logic.

The complete data flow from user input to your function:

/* Data preprocessing pipeline */ User selects CSV file → User maps REQUIRED_VARIABLES to CSV columns (by index) → User sets source unit → target unit for each variable → Engine selects mapped columns from DataFrame → Engine applies unit formulas (e.g., "x - 273.15" for K→°C) → Engine generates loc_id column from lat/lon if missing → Engine calls apply_correction(df, frequency, aggregations, output_prefix, transpose) → Your function receives the converted DataFrame

💡

The "original" baseline option gives the user access to the pre-conversion data, while "converted" gives the post-conversion data. Both are handled by the engine — your module only sees the converted data.

Python Script Structure

Your logic.py must define two things:

1. Metadata Constants

These tell the UI what your module needs and what it produces:

# Variables the user must map from their CSV REQUIRED_VARIABLES = ["Temperature", "Dewpoint"] # Output schema — what the module produces AGGREGATION_SCHEMA = { "baseline": { "options": [ {"id": "original", "label": "Original Values", "default": True} ] }, "correctionOutputs": { "note": "Optional description shown in the wizard", "aggregations": [ { "temporal": "sub-daily", "label": "Sub-Daily", "defaultEnabled": False, "types": [ {"id": "instant", "label": "Instantaneous Value", "default": True} ] }, { "temporal": "daily", "label": "Daily", "defaultEnabled": True, "types": [ {"id": "mean", "label": "Daily Mean", "default": True}, {"id": "max", "label": "Daily Max", "default": False}, {"id": "min", "label": "Daily Min", "default": False} ] }, { "temporal": "monthly", "label": "Monthly", "defaultEnabled": False, "types": [ {"id": "mean", "label": "Monthly Mean", "default": True} ] } ] } }

2. How the Output Selection Plan Works

When a user selects your module in the correction wizard, the Output Selection Plan section is dynamically generated from your AGGREGATION_SCHEMA. This means the checkboxes, group labels, and default states are entirely driven by what you define — no UI code changes are needed.

Data Flow

/* How the schema reaches the UI */ Python AGGREGATION_SCHEMA constant → inspect_module.py extracts it at server boot → Cached in server settings (sysSettings.corrections[].aggregationSchema) → Served via GET /api/settings/public → Frontend renderAggregations(schema) generates DOM checkboxes

Schema Structure Breakdown

Field	Level	Purpose
`baseline.options`	Root	Defines baseline checkboxes (e.g., "Original Values", "Unit Converted"). Each option has `id`, `label`, and `default` (checked state).
`correctionOutputs.note`	Root	A descriptive string shown above the aggregation checkboxes (e.g., "Temperature Product Suite: Generates Tmax, Tmin, Tmean").
`aggregations[]`	Output group	Each entry defines a temporal group (row of checkboxes).
`temporal`	Group	The temporal key — `"sub-daily"`, `"daily"`, `"monthly"`, or `"yearly"`. This value is passed to `apply_correction()` in the `aggregations` payload.
`label`	Group	Display label for the group (e.g., "DAILY PRODUCTS", "Monthly").
`defaultEnabled`	Group	If `true`, the group's checkboxes are checked by default. If `false` or omitted, they start unchecked.
`types[]`	Group	Individual output checkboxes within the group.
`types[].id`	Type	Unique identifier for the output type (e.g., `"max"`, `"sws"`). This `id` is what your `apply_correction()` receives in the `aggregations` payload.
`types[].label`	Type	User-visible label (e.g., "Tmax (Max)", "Total Soil Water Storage (mm)").
`types[].default`	Type	If `true`, this checkbox starts checked.

Supported Temporal Values

Value	Use Case	Example
`"sub-daily"`	Instantaneous or same-resolution outputs	10m wind speed from hourly U/V
`"daily"`	Daily aggregations (mean, max, min, sum)	Daily mean temperature
`"monthly"`	Monthly summaries	Monthly mean soil water storage
`"yearly"`	Annual summaries	Mean annual erosivity
`"none"`	Baseline only (no temporal grouping)	Original or unit-converted data

What the User's Selection Becomes

When the user clicks Execute Pipeline, the checked checkboxes are collected into an aggregations array and sent to apply_correction():

// If user checks: Daily → Tmax, Tmin | Monthly → Tmean aggregations = [ { "temporal": "daily", "types": ["max", "min"] }, { "temporal": "monthly", "types": ["avg"] } ]

Your apply_correction() function iterates over this array to produce the requested outputs. The temporal and types[].id values are what you defined in your schema — they are the contract between your UI declaration and your processing logic.

💡

If no AGGREGATION_SCHEMA is defined, the system falls back to a default schema with a single "none" temporal group producing "original" and "converted" baseline outputs.

3. The apply_correction Function

This is the entry point the engine calls with the user's mapped data:

# This function is called by the engine def apply_correction(df, frequency="hourly", aggregations=None, output_prefix="Output", transpose=True, **kwargs): """ df: Pandas DataFrame with metadata + mapped columns frequency: Data frequency ("hourly", "3hourly", "daily") aggregations: List of user-selected outputs from AGGREGATION_SCHEMA output_prefix: Prefix for output filenames (default: "Output") transpose: If True, pivots output to station-per-column format """ # 1. Get variable columns (everything except metadata) _META = {"lat", "lon", "time", "station", "height", "grid", "loc_id", "date"} var_cols = [c for c in df.columns if c.lower() not in _META] # 2. Validate required variables exist if len(var_cols) < 1: return {"error": "No variable columns found"} # 3. Your scientific computation here result = df[var_cols].mean() # Example # 4. Return a dict of filename → DataFrame return { f"{output_prefix}_Daily_Mean.csv": result_df }

Output Schema

Your function must return a dictionary where:

Keys = output filenames (e.g., "Daily_Mean.csv")
Values = Pandas DataFrames with the computed results

Each DataFrame should include lat, lon, date columns plus your computed values. The engine handles storage and download links automatically.

Output File Naming Convention

Every correction run generates one or more CSV files. Understanding the naming convention helps you identify outputs programmatically and build downstream automation.

How Filenames Are Resolved

The engine uses a two-stage resolution process to produce final filenames:

1

Your apply_correction() returns a dict

Each dict key is either a complete filename (ending in .csv) or a suffix (without .csv). The engine treats these differently.

2

write_results() resolves the final name

If the key ends with .csv → used as-is (e.g., "Output_Tmax_Daily.csv")
If the key does not end with .csv → the engine prepends {output_prefix}_ and appends .csv (e.g., "Daily_Mean" becomes "Output_Daily_Mean.csv")

Baseline Output Names

Baseline outputs are generated automatically by the engine (not by your module):

Baseline Option	Filename
`"original"`	`{output_prefix}_Original.csv`
`"converted"`	`{output_prefix}_UnitConverted.csv`

Built-in Module Naming Patterns

The table below shows the naming convention used by each built-in module, so you know what to expect when running them:

Module	Dict Key Pattern	Final Filename Example	Has .csv in key?
Temperature	`{prefix}_{Label}_{Temporal}.csv`	`Output_Tmax_Daily.csv`	Yes
Relative Humidity	`{prefix}_{Label}_{Temporal}.csv`	`Output_RHmean_Daily.csv`	Yes
Radiation	`{prefix}_{Product}_Daily_MJ.csv`	`Output_NetSW_Daily_MJ.csv`	Yes
Rainfall Erosivity	`{prefix}_{Product}_{Temporal}.csv`	`Output_Rainfall_Daily.csv`	Yes
Soil Moisture	`{prefix}_{Product}_{Temporal}`	`Output_SoilWaterStorage_Daily.csv`	No — engine adds .csv
Wind Speed	`{Product}`	`Output_Daily_Wind_Speed_10m.csv`	No — engine adds prefix + .csv
Vapor Pressure	`{Product}`	`Output_Daily_Saturation_Vapor_Pressure_ES.csv`	No — engine adds prefix + .csv

Recommended Pattern for Custom Modules

For consistency with the built-in modules, we recommend including both output_prefix and .csv in your dict keys. This makes filenames predictable and self-documenting:

# Recommended: include prefix + .csv in the key return { f"{output_prefix}_Daily_Mean.csv": daily_df, f"{output_prefix}_Monthly_Sum.csv": monthly_df, }

ℹ️

The output_prefix comes from the user's "Output Prefix" field in the wizard (defaults to "Output"). If multiple files are generated, each gets its own name — no index suffixes are added by the engine.

Transpose Output Format

The transpose parameter controls the output format:

transpose=True (default): Output is pivoted — rows are timestamps, columns are station/location IDs. Best for time-series analysis.
transpose=False: Output is in long format — each row is a single observation with lat, lon, time, and value columns.

ℹ️

The engine automatically handles the loc_id column creation from lat/lon coordinates when needed for grouping.

Error Handling

To signal errors, return a dictionary with an "error" key:

if len(var_cols) < 2: return {"error": "Wind Speed module requires both U and V components."}

The engine will catch this and display the error message to the user instead of generating output files.

Writing Theory Documentation

The theory.md file is shown to users in the correction wizard. It should include:

1

Overview

What the module does, which variables it processes, and what scientific products it generates.

2

Calculation Logic

The formulas used. Use LaTeX notation: $$ T(°C) = T(K) - 273.15 $$

3

Expected Outputs

List every output file the module can generate, with column names and units.

Getting Started Template

Copy this starter template and modify it for your use case:

# my_custom_module/logic.py import pandas as pd import numpy as np REQUIRED_VARIABLES = ["MyVariable"] AGGREGATION_SCHEMA = { "baseline": { "options": [ {"id": "original", "label": "Original Data", "default": True} ] }, "correctionOutputs": { "aggregations": [ { "temporal": "daily", "label": "Daily Products", "defaultEnabled": True, "types": [ {"id": "daily_mean", "label": "Daily Mean", "default": True} ] } ] } } def apply_correction(df, frequency="hourly", aggregations=None, output_prefix="Output", transpose=True, **kwargs): # Extended metadata columns _META = {"lat", "lon", "time", "station", "height", "grid", "loc_id", "date"} var_cols = [c for c in df.columns if c.lower() not in _META] # Validate required variables if not var_cols: return {"error": "No variable columns found"} # Parse time and create location ID df = df.copy() df["time"] = pd.to_datetime(df["time"]) df["loc_id"] = df["lat"].round(4).astype(str) + "_" + df["lon"].round(4).astype(str) # Pivot for time-series operations ts_pivot = df.pivot_table(index="time", columns="loc_id", values=var_cols[0]) # Compute daily mean daily = ts_pivot.resample("D").mean() # Format output based on transpose flag if not transpose: daily = daily.reset_index().melt(id_vars="time", var_name="loc_id", value_name="MyVariable_daily_mean") daily[["lat", "lon"]] = daily["loc_id"].str.split("_", expand=True).astype(float) daily = daily.drop(columns=["loc_id"]) return {f"{output_prefix}_Daily_Mean.csv": daily}

How to Upload

1

Go to Corrections Page

Navigate to Corrections in the dashboard sidebar.

2

Fill in Module Details

Enter a label (e.g., "Bias Correction v2"), upload your logic.py and theory.md files.

3

Share (Optional)

Check "Share with Community" to make your module available in the global research library for other users.

4

Upload

Click UPLOAD_SCIENTIFIC_MODULE. Your module will appear in the correction engine dropdown immediately.

⚠️

Your Python script runs in a sandboxed environment. Only pandas, numpy, and scipy are available. If you need additional packages, contact the admin.

Google Earth Engine

Fetch satellite and climate datasets from Google Earth Engine directly into your processing jobs.

ℹ️

GEE integration is available on Pro and Max plans. You must hold your own Google Earth Engine account — NetCDF Studio does not provide GEE access.

What Is GEE Integration?

Instead of uploading a local IMDAA archive, you can point a job at a Google Earth Engine collection. NetCDF Studio will authenticate with GEE using your credentials, fetch the requested data, and run the same processing pipeline — correction engines, unit conversion, and CSV export — on the result.

Billing Clarification

⚠️

You are not charged by NetCDF Studio for Google Earth Engine API usage. GEE data retrieval is billed by Google directly to your GEE account (subject to Google's own free tier and quotas). What your NetCDF Studio plan covers is the server-side computation: CPU, memory, and workers used to process the fetched data.

Cost Item	Who Pays
GEE API calls & data retrieval quota	Google bills your GEE account
Server computation (conversion, corrections)	Covered by your NetCDF Studio plan
Output storage	Covered by your plan quota

Setting Up Your GEE Credential

1

Go to Credentials Settings

In the Dashboard, open Settings → Credentials. You'll see the Google Earth Engine section.

2

Paste Your Service Account Key

Paste the JSON service account key from your Google Cloud Console. NetCDF Studio stores it encrypted (AES-256-GCM) — the key is never returned to the client after saving; only the last 4 characters are shown for reference.

3

Start a GEE Job

When submitting a new request, choose Google Earth Engine as the data source, specify the collection and date range, then submit. The job runs identically to a local archive job.

Supported Data Sources

Any publicly accessible GEE ImageCollection your account can read
CMIP6 collections available via GEE catalogue
ERA5, MODIS, Landsat, Sentinel — anything your GEE quota allows

💡

ESGF direct-download (the separate ESGF federation) is not currently supported as a GEE source — use the IMDAA upload flow for ESGF datasets instead.

Compute Nodes

Lend additional machines to your NetCDF Studio server to scale processing capacity.

What Are Compute Nodes?

A compute node is any machine — a workstation, a spare laptop, a cloud VM — that runs the netcdf-studio-node agent. The agent dials out to your server over WebSocket and receives jobs from it. No firewall changes or port forwarding required on the node side.

🔒

Nodes connect outbound only — the server never initiates a connection to the node machine. Job credentials are passed through the authenticated socket, never via command-line arguments (which are visible in the process list).

Installing the Node Agent

1

Install Node.js (v18+)

The agent requires Node.js 18 or later on the remote machine.

2

Install the Agent Package

$ npm install -g netcdf-studio-node

3

Enrol the Node in Admin

In the Admin Panel → Nodes page, click Enrol Node. Fill in the node name, roles (Compute, Storage, or both), cores to utilise, and max parallel jobs. The panel will show a one-time token — copy it before closing.

4

Start the Agent

$ netcdf-studio-node --server https://your-server.example.com --token <token>

Or use the environment variable to keep the token out of your shell history:

$ NETCDF_NODE_TOKEN=<token> netcdf-studio-node --server https://your-server.example.com

Node Roles

Role	What It Does
Compute	Receives and runs processing jobs (Python worker). Requires Python 3.9+ on the node machine.
Storage	Holds cold archive files for the server. Frees up primary server disk. Files are never deleted on agent restart — the server treats this as the durable cold copy.

Node Options

Flag	Description
`--server <url>`	Base URL of the NetCDF Studio server. Required.
`--token <token>`	Node token from the Admin enrol dialog. Prefer `NETCDF_NODE_TOKEN` env var.
`--name <label>`	Display name shown in the Admin Nodes panel. Defaults to hostname.
`--concurrency <n>`	Max simultaneous jobs (informational — the server enforces the limit set at enrolment).
`--python <path>`	Python interpreter to use. Defaults to `python3` (or `python` on Windows).
`--worker <path>`	Path to `worker_entry.py`. Only needed if the agent is not installed from the same release as the server.

Security Notes

Node tokens are shown once only at enrolment and stored HMAC-hashed on the server. If lost, enrol a new node.
Job scratch directories are deleted immediately after each job completes — no data lingers on the node machine.
Credentials passed to jobs (GEE keys, CDS tokens) arrive over the encrypted socket and go into the worker's environment — never written to disk or passed via argv.

Developer Reference

Complete technical specification for the IMDAA Correction Engine. Use this to build custom correction modules.

📄

This page loads the live technical specification from /api/docs/correction-engine. The content is the same markdown served to the dashboard for module development.

Loading technical specification...

Frequently Asked Questions

Common questions about using NetCDF Studio.

What file formats can I upload?

NetCDF Studio accepts .zip archives containing NetCDF (.nc) files from the IMDAA reanalysis dataset. Google Earth Engine jobs do not require a file upload — the server fetches the data directly using your GEE credentials.

What is the maximum upload size?

In Online mode, the limit is 3 GB per file. In Offline mode, there is no limit — processing happens locally on your machine.

Does NetCDF Studio charge me for Google Earth Engine usage?

No. NetCDF Studio does not charge you for GEE API calls, data retrieval, or Earth Engine compute credits. GEE usage is billed directly by Google to your GEE account, subject to Google's own free tier and quotas. Your NetCDF Studio plan covers the server-side computation (CPU, memory, workers) used to process the data once it is fetched. See the Google Earth Engine page for a full breakdown.

Do I need Python installed?

Only if you use Offline mode or run a Compute Node. The local server and node agent use Python for scientific data processing. Python 3.9 or later is required. Online mode handles everything on the cloud.

What is a Compute Node?

A compute node is a secondary machine that you connect to your NetCDF Studio server to add processing capacity. It runs the netcdf-studio-node agent, which dials out to the server and receives jobs over WebSocket. No port forwarding is required. See the Compute Nodes page for setup instructions.

Can I re-use uploaded archives?

Yes. Previously uploaded archives are saved in your workspace. You can select them again during the upload step to create new CSV variants without re-uploading.

What happens to my data?

In Online mode, your data is stored securely on our servers during processing and is only accessible from your authenticated account. In Offline mode, all data stays on your local machine. On Compute Nodes, job scratch data is automatically deleted as soon as the job completes.

How do I switch between Online and Offline mode?

Use the mode toggle in the dashboard header. Offline mode only appears if you've installed the NetCDF Studio PWA app.

What if the offline server won't start?

Run netcdf-studio doctor in your terminal to diagnose common issues. Make sure Node.js v18+ and Python 3.9+ are installed. For compute node agents, confirm the token is correct and the server URL does not have a trailing slash.