# Helix Engage — CI/CD & Operations Dashboard

## Overview

Three services on EC2 provide CI/CD and operational visibility:

- **Gitea** (`git.healix360.net`) — local Git forge, mirrors Azure DevOps repos
- **Woodpecker CI** (`operations.healix360.net`) — build dashboard, runs pipelines
- **MinIO** (internal) — stores test reports, served via Caddy

## URLs

| Service | URL | Auth |
|---|---|---|
| Build Dashboard | `https://operations.healix360.net` | Gitea OAuth (helix-admin / Global@2026) |
| Test Reports | `https://operations.healix360.net/reports/{run}/index.html` | Basic auth (helix-admin / Global@2026) |
| Git Forge | `https://git.healix360.net` | helix-admin / Global@2026 |

## How It Works

```
Azure DevOps (push)
    ↓ mirror sync (every 15min or manual)
Gitea (git.healix360.net)
    ↓ webhook
Woodpecker CI (operations.healix360.net)
    ↓ runs pipeline steps in Docker containers
    ├── typecheck (node:20, yarn tsc)
    ├── e2e-tests (playwright, 40 smoke tests)
    ├── publish-report (S3 plugin → MinIO)
    └── notify-teams (curl → Power Automate → Teams channel)
```

## Pipelines

### helix-engage (frontend)

Triggers on push to any branch or manual run.

**Steps:**
1. **typecheck** — `yarn install` + `tsc --noEmit` (node:20 image)
2. **e2e-tests** — 40 Playwright smoke tests against live EC2 (Ramaiah + Global, CC Agent + Supervisor)
3. **publish-report** — uploads Playwright HTML report to MinIO via S3 plugin
4. **notify-teams** — sends Adaptive Card to Teams "Deployment updates" channel with pipeline link + report link

**Report URL:** `https://operations.healix360.net/reports/{pipeline-number}/index.html`

### helix-engage-server (sidecar)

Triggers on push to any branch or manual run.

**Steps:**
1. **unit-tests** — `npm ci` + `jest --ci --forceExit` (node:20 image)
2. **notify-teams** — sends Adaptive Card to Teams with pipeline link

## Mirrored Repos

| Azure DevOps Repo | Gitea Mirror | Branch |
|---|---|---|
| `globalhealthx/EMR/_git/helix-engage` | `helix-admin/helix-engage` | feature/omnichannel-widget |
| `globalhealthx/EMR/_git/helix-engage-server` | `helix-admin/helix-engage-server` | master |

Mirror syncs every 15 minutes automatically. To force sync:

```bash
curl -s -X POST "https://git.healix360.net/api/v1/repos/helix-admin/helix-engage/mirror-sync" \
  -u "helix-admin:Global@2026"

curl -s -X POST "https://git.healix360.net/api/v1/repos/helix-admin/helix-engage-server/mirror-sync" \
  -u "helix-admin:Global@2026"
```

## Teams Notifications

Notifications go to the "Deployment updates" channel via Power Automate Workflow webhook.

Each notification includes:
- Project name and build number
- Branch name
- Commit message
- "View Pipeline" button (links to Woodpecker)
- "View Report" button (links to Playwright HTML report, frontend only)

## Secrets (Woodpecker)

Configured per-repo in Woodpecker Settings → Secrets:

| Secret | Used by | Purpose |
|---|---|---|
| `s3_access_key` | publish-report | MinIO access key (`minio`) |
| `s3_secret_key` | publish-report | MinIO secret key |
| `teams_webhook` | notify-teams | Power Automate webhook URL |

## Docker Containers

| Container | Image | Purpose |
|---|---|---|
| `ramaiah-prod-gitea-1` | `gitea/gitea:latest` | Git forge |
| `ramaiah-prod-woodpecker-server-1` | `woodpeckerci/woodpecker-server:v3` | CI dashboard + pipeline engine |
| `ramaiah-prod-woodpecker-agent-1` | `woodpeckerci/woodpecker-agent:v3` | Executes pipeline steps in Docker |

## Agent Configuration

The Woodpecker agent is configured to:
- Run pipeline containers on the `ramaiah-prod_default` Docker network (so they can reach Gitea and MinIO)
- Allow up to 2 concurrent workflows

## Troubleshooting

### Pipeline fails at git clone

Check that Gitea's `REQUIRE_SIGNIN_VIEW` is `false` (public repos must be cloneable without auth):

```bash
ssh -i /tmp/ramaiah-ec2-key ubuntu@13.234.31.194 \
  "docker exec ramaiah-prod-gitea-1 grep REQUIRE_SIGNIN /data/gitea/conf/app.ini"
```

### npm install crashes with "Exit handler never called"

Known npm bug in CI containers. Use `yarn` instead of `npm` for the frontend. The sidecar's lockfile is clean so `npm ci` works.

### Pipeline says "pipeline definition not found"

The `.woodpecker.yml` file is missing or has invalid YAML. Check:

```bash
curl -s "https://git.healix360.net/api/v1/repos/helix-admin/helix-engage/contents/.woodpecker.yml?ref=feature/omnichannel-widget" \
  -u "helix-admin:Global@2026" | python3 -c "import sys,json;print(json.load(sys.stdin).get('name','NOT FOUND'))"
```

### Teams notification not arriving

Verify the webhook secret is set in Woodpecker and the Power Automate workflow is active.

### Test reports not loading (403/XML error)

Caddy must strip the Authorization header before proxying to MinIO. Check:

```bash
ssh -i /tmp/ramaiah-ec2-key ubuntu@13.234.31.194 \
  "grep -A8 'handle_path /reports' /opt/fortytwo/Caddyfile"
```

Should include `header_up -Authorization`.

### Manually trigger a pipeline

```bash
WP_TOKEN="<woodpecker-api-token>"
curl -s -X POST "https://operations.healix360.net/api/repos/1/pipelines" \
  -H "Authorization: Bearer $WP_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"branch":"feature/omnichannel-widget"}'
```

### Delete old pipeline runs

```bash
WP_TOKEN="<woodpecker-api-token>"
for i in $(seq 1 20); do
  curl -s -X DELETE "https://operations.healix360.net/api/repos/1/pipelines/$i" \
    -H "Authorization: Bearer $WP_TOKEN"
done
```

## E2E Test Coverage

40 tests across 2 hospitals, 3 roles:

**Login (4):** branding, invalid creds, supervisor login, auth guard

**Ramaiah CC Agent (10):** landing, call desk, call history, patients (list + search), appointments, my performance (API + KPI), sidebar, sign-out modal, sign-out complete

**Ramaiah Supervisor (12):** landing, team performance, live monitor, leads, patients, appointments, call log, recordings, missed calls, campaigns, settings, sidebar

**Global CC Agent (7):** landing, call history, patients, appointments, my performance, sidebar, sign-out

**Global Supervisor (5):** landing, patients, appointments, campaigns, settings

**Auto-cleanup:** Last CC Agent test completes sign-out to release agent session. Setup steps call `/api/maint/unlock-agent` to clear stale locks.