6 agents · 6 skills · macOS, Windows, Linux

A modular data engineering team, orchestrated end to end.

Ingest, normalize, cleanse, validate, persist, and analyze. Six specialized subagents, each with its own toolchain and reusable agent skills, coordinated by a single orchestrator that hands a progressively refined dataset down the pipeline.

Get started Meet the agents View on GitHub

What this is

Give the orchestrator a data task. It identifies the agents required, announces each active agent with [Active Agent: X], applies that agent's tools and skills, then hands off downstream, routing any invalid records to a dead letter queue (DLQ).

1 Ingest
→
2 Schema
→
3 Cleanse
→
4 Guard
→
5 DBOps
→
6 Analytics
→
Reports

Format-agnostic ingest

Encoding detection and messy-workbook parsing into clean Arrow or Parquet.

Strict schemas

Semantic column mapping, JSON and XML flattening, ISO-8601 dates.

Privacy and hygiene

Fuzzy dedupe, salted-hash PII masking, imputation, outlier flags.

Governed loads

Contract assertions, drift detection, DLQ routing before the warehouse.

Warehouse ops

Bulk COPY FROM, MERGE INTO upserts across dialects.

Reporting

Window-function SQL and formatted multi-tab Excel workbooks.

Install and run

The installer is pure Python standard library (no pip install needed) and runs the same on macOS, Windows, and Linux. Pick your OS, copy the command, and paste it into your terminal.

Terminal 
# from the folder containing install_data_agents.py + claude_data_agents.zip
cd ~/Documents/AGENTDEV
python3 install_data_agents.py --project    # install to ./.claude
# or:  python3 install_data_agents.py         # install to ~/.claude (global)
# preview first:  python3 install_data_agents.py --dry-run
PowerShell 
# from the folder containing install_data_agents.py + claude_data_agents.zip
cd $HOME\Documents\AGENTDEV
python install_data_agents.py --project     # install to .\.claude
# or:  python install_data_agents.py          # install to ~\.claude (global)
# preview first:  python install_data_agents.py --dry-run
python install_data_agents.py --help
Why isn't there a "run" button? Browsers cannot launch your terminal or run local programs; that is a security boundary no static site can cross. Copy the command above and paste it into Terminal (macOS or Linux) or PowerShell (Windows). Existing agents and skills are never overwritten: collisions get a -de suffix unless you pass --force or --skip-existing.

Orchestration examples

Ways to drive the whole team, from a single fire-and-forget prompt to fine-grained manual control. Click a tab, then copy the prompt.

Prompt
Project: Q3 Sales Consolidation.
Input: /data/raw/q3_sales_*.xlsx (12 regional workbooks, merged headers).
Goal: clean, deduplicate customers, load into DuckDB table fact_sales,
and produce an executive summary workbook.
Run the full pipeline. Mask customer PII. Reject rows where
total != qty * unit_price to the DLQ.

Flow: Ingest (+excel-unstacker) → Schema → Cleanse (+pii-masker, +fuzzy-deduplicator) → Guard (+data-contract-validator) → DBOps (+bulk-db-upsert) → Analytics (+excel-report-generator).

Prompt
Let's run this project stage by stage. Pause after each agent and show me
its report before continuing.
Start with IngestAgent on /data/raw/q3_sales_*.xlsx.

Then continue each stage, for example: "Proceed to SchemaAgent, map to the canonical sales schema."

Prompt
The data is already typed and clean in warehouse.stg_sales.
Skip Ingest/Schema/Cleanse. Run GuardAgent to validate against the sales
contract, then DBOpsAgent to MERGE INTO fact_sales, then AnalyticsAgent
for the summary workbook.

Single-agent one-off: "[AnalyticsAgent only] top 10 customers by YoY growth via window functions, export to Excel."

Prompt
Two independent inputs:
  A) /data/raw/salesforce_export.csv
  B) /data/raw/erp_dump.xlsx
Run Ingest -> Schema -> Cleanse on each branch independently, then hand both
to GuardAgent to validate a shared contract, union the verified sets,
and load once via DBOpsAgent. Report which branch each DLQ row came from.

Fan out per source, then join at the Guard stage before a single load.

Prompt
Project: Legacy CRM to Snowflake migration.
Source: 3 years of monthly exports, mixed encodings (some ISO-8859-1),
multi-tab workbooks with a pivot layout on the "Activity" tab.
Contract: customer_id not null and unique; email valid or null;
signup_date ISO-8601; status in (active, churned, trial).

Orchestrate the full team:
- IngestAgent: sniff encoding per file; use excel-unstacker on Activity tabs.
- SchemaAgent: unify all months to one schema; flatten the nested contacts JSON.
- CleanseAgent: hash email + phone (deterministic); fuzzy-dedupe customers;
  impute missing region via KNN.
- GuardAgent: enforce the contract; KS-test signup_date drift vs 2024 baseline;
  DLQ violations with reasons.
- DBOpsAgent: Alembic migration; COPY FROM staging; MERGE INTO dim_customer
  on customer_id (Snowflake dialect).
- AnalyticsAgent: monthly cohort retention with recursive CTEs; export
  migration_report.xlsx (Summary + DLQ + Metadata tabs).
Checkpoint after GuardAgent so I can review the DLQ before we load.

A fuller, adapt-me template covering encoding, contracts, drift, dialect, and checkpoints.

The data workers

Six specialized agents. Each card lists what it does, the skills it carries, and the kinds of work it handles. Hit "Template prompts" to reveal a copy-and-paste starter.