Skip to main content
Version: Next

AI Development Guide for Watchmen Developers

Purpose​

This guide helps developers start using AI effectively when working with Watchmen metadata and related development tasks. It is written as a standalone onboarding document and does not require any prior knowledge of internal skill definitions.

The main goal is simple:

  • use AI to accelerate repetitive development work,
  • keep changes small and reviewable,
  • rely on existing metadata and server state instead of guessing,
  • treat AI as a development assistant, not as the source of truth.

What AI Is Good At​

AI is most useful for the following development workflows:

  • discovering existing Topics, Pipelines, Enums, Semantic Models, Metrics, and Ingestion configs,
  • generating small YAML changes based on existing files,
  • explaining metadata structure and dependencies,
  • scaffolding new metadata from examples,
  • converting manual CLI workflows into repeatable steps,
  • helping debug command failures and configuration issues.

AI is less reliable when asked to invent large configurations without context. Always prefer starting from an existing resource or template.

Use this workflow whenever you ask AI to help with Watchmen development:

  1. Identify the exact target entity.
  2. Discover the entity from the server by name or ID.
  3. Pull the minimal required resource locally.
  4. Edit the smallest necessary part.
  5. Review the diff.
  6. Push only the changed file or target.

This approach is safer and faster than bulk pulling everything and editing blindly.

Prerequisites​

Before using AI for development, make sure the local environment is ready:

  • Python 3.10+
  • network access to the Watchmen server
  • a working local vault directory
  • valid credentials such as PAT, or username/password if applicable

Example setup:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install -U pip setuptools wheel
python -m pip install watchmen-agent-cli
agent-cli init --vault <vault> --host <host> --pat <token>

Useful verification commands:

agent-cli tenant --vault <vault>
agent-cli config --vault <vault>

Core Development Principles​

When working with AI, follow these rules:

  • prefer discovery over assumption,
  • prefer small local edits over full regeneration,
  • prefer pull-by-name or pull-by-ID over pulling all metadata,
  • prefer explicit mappings and explicit names,
  • keep one change focused on one business goal,
  • review generated YAML before pushing,
  • preserve real IDs on existing resources,
  • use placeholder IDs only for new local resources.

Typical Developer Workflows​

1. Discover Existing Metadata​

Ask AI to discover remote resources first.

Examples:

agent-cli topic list-remote --vault <vault>
agent-cli pipeline list-remote --vault <vault>
agent-cli enum list-remote --vault <vault>
agent-cli semantic list-remote --vault <vault>
agent-cli metric list-remote --vault <vault>

Good prompt examples:

  • "Find the remote pipeline related to customer enrichment."
  • "List remote topics and identify the one used for claims."
  • "Check whether a metric named total_claim_cases already exists."

2. Pull Only the Resource You Need​

Once the target is known, pull only that resource.

Examples:

agent-cli topic pull-name "Customer 360" --vault <vault>
agent-cli pipeline pull-name "Customer Enrichment" --vault <vault>
agent-cli enum pull-name "ClaimStatus" --vault <vault>
agent-cli semantic pull-name "customer_semantic" --vault <vault>
agent-cli metric pull-name "total_claim_cases" --vault <vault>

This keeps the workspace clean and reduces unnecessary context.

3. Edit Local YAML Safely​

After pulling a resource, let AI help edit the local YAML file.

Recommended usage:

  • ask AI to explain the file structure first,
  • ask AI to change only one block at a time,
  • ask AI to show the exact diff before push,
  • keep names and IDs aligned with the existing object.

Good prompt examples:

  • "Add one factor named customer_age_group to this topic YAML."
  • "Insert one insert-row action into this pipeline."
  • "Rename the metric label but keep the existing metric ID."

4. Push the Smallest Possible Change​

Prefer pushing one file instead of all resources.

Examples:

agent-cli topic push-file <file_path> --vault <vault>
agent-cli pipeline push-file <file_path> --vault <vault>
agent-cli enum push-file <file_path> --vault <vault>
agent-cli semantic push-file <file_path> --vault <vault>
agent-cli metric push-file <file_path> --vault <vault>

Use broader sync only when that is truly the intent:

agent-cli push --target topic --vault <vault>
agent-cli push --target pipeline --vault <vault>

File and ID Conventions​

Local files should follow the standard naming rule:

{name}__{id}.yml

Examples:

customer_360__1486457832676011008.yml
customer_enrichment__1486457832676011055.yml

Directory layout usually follows this structure:

vault/
├── .agent-cli/
│   └── config.json
├── ingest/
│   ├── tables/
│   ├── models/
│   └── modules/
├── transformation/
│   ├── topics/
│   ├── pipelines/
│   ├── subjects/
│   └── enums/
└── metrics/
    ├── semantics/
    └── metric/

ID guidance:

  • existing resources should keep their real server-generated IDs,
  • new local resources may use placeholder IDs such as f-topic_customer_001,
  • after push, the server may replace placeholder IDs with real IDs.

Pipeline Development Recommendations​

When using AI for pipeline development:

  • start from an existing pipeline whenever possible,
  • keep pipeline names clear and unique,
  • use explicit mappings rather than implicit assumptions,
  • validate referenced topic IDs and factor names,
  • keep a new pipeline disabled while drafting if needed.

A simple workflow looks like this:

agent-cli pipeline pull-name "Customer Enrichment" --vault <vault>
# edit local YAML
agent-cli pipeline push-file <vault>/transformation/pipelines/customer_enrichment__<id>.yml --vault <vault>

Ingestion Development Recommendations​

For ingestion development:

  • model one root table only,
  • ensure child tables use parentName,
  • keep module, model, and table names consistent,
  • use AI to generate starter YAML from an existing pattern,
  • review join keys, object keys, and dependencies carefully.

Useful commands:

agent-cli ingest module list-remote --vault <vault>
agent-cli ingest model pull "CustomerProfile" --all --vault <vault>
agent-cli ingest table push-file <file_path> --vault <vault>
agent-cli ingest model create-raw-topic "CustomerProfile" --vault <vault>

How to Prompt AI Well​

The quality of the result depends heavily on the prompt. A good prompt should include:

  • the business goal,
  • the entity type,
  • the exact entity name or ID,
  • the vault path,
  • whether the task is discovery, pull, edit, validation, or push,
  • any constraint such as "show diff only" or "do not push yet".

Strong prompt examples:

  • "Pull the pipeline named Customer Enrichment into vault /data/project."
  • "Update the local topic YAML to add one factor named claim_age_bucket, then show the diff only."
  • "Create a starter ingestion model YAML for customer orders based on the existing customer templates."
  • "List remote enums and identify whether ClaimStatus already exists."
  • "Do not push anything. Only prepare the local YAML change."

Weak prompt examples:

  • "Create a pipeline for me."
  • "Fix this metadata."
  • "Make it work."

Review Checklist Before Push​

Before pushing AI-generated changes, verify:

  • the resource name is correct,
  • the target file path is correct,
  • IDs are preserved for existing resources,
  • placeholder IDs are used only for newly created resources,
  • referenced topics, enums, metrics, or models exist,
  • factor names and labels are valid,
  • the change is limited to the intended scope,
  • the YAML format is clean and consistent.

Troubleshooting​

Command not found​

Install or verify the package:

python -m pip install watchmen-agent-cli
python -m pip show watchmen-agent-cli

Vault config missing​

Initialize first:

agent-cli init --vault <vault> --host <host> --pat <token>

Names with spaces break the command​

Wrap names with quotes:

agent-cli pipeline pull-name "Policy Admin System" --vault <vault>

PAT can read but cannot push​

Check tenant and permissions:

agent-cli tenant --vault <vault>

Ingestion table push fails due to root table conflict​

One model should have exactly one root table. All other tables must set parentName.

Suggested Team Practices​

To make AI adoption effective across the team:

  • standardize prompts for common workflows,
  • keep example YAML files clean and reusable,
  • prefer templates over free-form generation,
  • require human review before pushing production changes,
  • store successful prompt patterns in team documentation,
  • use AI for speed, but keep validation in developer hands.

Example End-to-End Session​

agent-cli topic list-remote --vault <vault>
agent-cli topic pull-name "Customer 360" --vault <vault>
agent-cli pipeline pull-name "Customer Enrichment" --vault <vault>
# edit local files
agent-cli pipeline push-file <vault>/transformation/pipelines/customer_enrichment__<id>.yml --vault <vault>

Summary​

The best way to use AI in Watchmen development is to keep the workflow precise:

  • discover first,
  • pull the smallest possible target,
  • edit locally,
  • review carefully,
  • push only the intended change.

That combination gives the highest development speed with the lowest operational risk.