跳到正文
AI NEWS JOURNAL
Hacker News Show HNAI NEWS / JOURNAL

Show HN: ShadowDiff – Shadow testing and SQL diffing for AI-migrated back ends

Article URL: https://github.com/vishalrajofficial/ShadowDiff Comments URL: https://news.ycombinator.com/item?id=50504643 Points: 1 # Comments: 0

翻译服务暂未运行,当前显示原文。

Dual-Traffic Shadow Testing Engine for AI-Refactored Legacy Code Eliminating the 5% risk that stops enterprise CTOs from deploying AI-modernized backends to production. 🌐 Interactive Live Cloud Sandbox: https://shadowdiff-demo.fly.dev (Self-driving dual-traffic simulator active) 🎬 Live Terminal Walkthrough (vhs) ⚡ The Problem Enterprises are actively modernizing mission-critical legacy backends (Java 7/8 Spring monoliths, Python 2 Django, PHP 5.6 backends) into modern cloud-native services (Go, TypeScript/Node, FastAPI) using AI code migration engines. While AI generates ~95% of refactored code accurately, the remaining 5% of hidden edge cases make production deployment an unacceptable risk: Subtle Serialization Discrepancies: Jackson omitting empty lists vs Go serializing null, datetime timezone truncations, float precision drift. Implicit Framework Quirks: Legacy servlet filter quirks, undocumented fallback headers, HTTP 200 responses with embedded error payloads. Database State Regressions: Silent differences in database transaction boundaries, missing audit writes, or N+1 query loop explosions. 💡 The Solution: ShadowDiff ShadowDiff is a high-performance network proxy and analysis engine that duplicates real staging or production HTTP traffic, sending identical payloads to both the legacy service and the AI-refactored service simultaneously. [ Live Inbound Traffic ] │ ▼ ┌─────────────────────────────┐ │ Envoy Gateway / Mesh │ │ (request_mirror_policies) │ └──────────────┬──────────────┘ │ ┌───────────────────┴───────────────────┐ ▼ ▼ ┌───────────────────┐ ┌───────────────────┐ │ Legacy Service │ │ ShadowDiff Core │ │ (Target) │ │ (Analysis Engine)│ └─────────┬─────────┘ └─────────┬─────────┘ │ │ │ (Live Response to Client) ▼ │ ┌───────────────────┐ │ │ AI-Refactored │ │ │ Candidate (S) │ │ └─────────┬─────────┘ │ │ └───────────────────┬───────────────────┘ ▼ ┌─────────────────────────────┐ │ 3-Way Diff & SQL Comparator │ │ - P1 ⊕ P2 Noise Filter │ │ - SQL Mutation Interceptor │ │ - N+1 Query Loop Detector │ └──────────────┬──────────────┘ │ (Pure Regressions Isolated) ▼ ┌─────────────────────────────┐ │ Automated AI Test Generator │ │ - Powered by Google Gemini │ │ - Synthesizes Go/Jest Tests │ │ - Root Cause Code Fixes │ └─────────────────────────────┘ 🚀 Key Architectural Capabilities 1. ⚡ Network-Level Envoy Traffic Mirroring ShadowDiff integrates directly with Envoy Proxy and Istio Service Mesh using request_mirror_policies. Traffic is cloned asynchronously at line rate in the network layer with zero application latency penalty. 2. 🎯 3-Way Differential Noise Cancellation (Diffy Model) Automatically filters out non-deterministic runtime noise (timestamps, UUIDs, session nonces) by evaluating twin baseline instances: $$\mathcal{N} = P_1 \oplus P_2$$ $$\mathcal{D} = (P_1 \oplus S) \setminus \mathcal{N}$$ Only verified contract regressions reach developer triage. 3. 💾 Database Mutation & SQL Side-Effect Diffing Captures and correlates SQL statements executed by both services for each HTTP trace: Missing Writes: Flags when legacy wrote to audit_logs or ledger tables that modern code omitted. Transaction Leaks: Warns if legacy used BEGIN...COMMIT and modern ran autocommit statements. N+1 Query Explosions: Detects when modern ORMs loop and execute $\ge 3\times$ more queries than legacy. 4. 🤖 Google Gemini Regression Test Synthesis Every isolated edge case is analyzed by Google Gemini (gemini-2.0-flash), which writes a turnkey integration test (Go httptest or Vitest) and diagnoses the code fix. 5. 🐙 Autonomous Closed-Loop GitHub PR Bot Eliminates manual triaging by directly integrating with the GitHub API: Automatic Branching: Spins up shadowdiff/fix- from the repository's default branch. Test File Commitment: Automatically commits the synthesized regression test directly to the repository (e.g. tests/regression/test_drift_.go). Rich Pull Request Generation: Opens a detailed PR with side-by-side JSON diff tables, SQL mutation waterfall analysis, N+1 query warnings, and local verification commands. 1-Click Triage in Console: Developers can inspect the incident in the web console and click "🚀 Open GitHub PR" to immediately open the PR on GitHub. 6. 🚦 CI/CD "Shadow Gating" CLI (shadowdiff gate) A native deployment blocker that evaluates traffic quality during pre-release soak tests or integration pipelines: Automated Evaluation: Assesses semantic parity, database mutations, and N+1 query loops over a specified evaluation window. Zero False Failure: Honors Diffy noise cancellation ($\mathcal{N} = P_1 \oplus P_2$), only failing on genuine regressions. Pipeline Integration: Returns standard exit codes (0 for PASS, 1 for FAIL/violation, 2 for error) with high-impact ANSI colored terminal summaries and optional JSON format. # Evaluate quality gate in your CI/CD pipeline: shadowdiff gate \ -dashboard-url=http://localhost:8083 \ -duration=15s \ -min-parity=100.0 \ -min-requests=50 \ -max-drifts=0 \ -allow-n-plus-one=false \ -allow-db-drift=false 7. 🛡️ Ephemeral Copy-on-Write (CoW) Database Sandboxing Allows safe shadow mirroring of mutating HTTP methods (POST, PUT, DELETE, PATCH) with zero database pollution: Transactional Savepoint Rollbacks: Injects boundary headers to ensure shadow writes execute within an isolated savepoint and automatically roll back. Ephemeral Schema Sandboxing: Dynamically creates disposable schemas per test run and drops them upon completion. Serverless CoW Branching: Leverages Neon / Supabase serverless branching to clone database state instantly via API for mutating candidate runs. 🚦 GitHub Actions CI/CD Quality Gating Workflow Incorporate ShadowDiff directly into your automated deployment pipeline (.github/workflows/shadowdiff.yml): name: Dual-Traffic Shadow Gating on: [pull_request, push] jobs: shadow-gate: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-go@v5 with: go-version: '1.22' - name: Build & Start Services run: | go build -o bin/shadowdiff ./cmd/shadowdiff/main.go ./bin/legacy-service -port=:8081 & ./bin/refactored-service -port=:8082 & ./bin/shadowdiff -proxy-port=:8080 -primary-url=http://localhost:8081 -shadow-url=http://localhost:8082 -sandbox-mode=rollback & sleep 2 - name: Replay Traffic run: | for i in {1..25}; do curl -s "http://localhost:8080/api/users?id=$i" > /dev/null curl -s -X POST "http://localhost:8080/api/orders" -d '{"amount":99.50}' > /dev/null done - name: Run ShadowDiff Quality Gate run: | ./bin/shadowdiff gate \ -dashboard-url=http://localhost:8083 \ -min-parity=100.0 \ -min-requests=20 \ -allow-n-plus-one=false \ -allow-db-drift=false 🛠️ Quickstart with Envoy & Docker Compose Launch the complete 5-container service mesh testbed (Envoy Gateway + ShadowDiff + Legacy Target + Twin Replica + AI Candidate): # 1. Clone the repository git clone https://github.com/vishalrajofficial/ShadowDiff.git cd ShadowDiff # 2. Launch the Envoy Service Mesh topology docker compose -f docker-compose.envoy.yml up -d Send Ingress Traffic Through Envoy (Port 10000): # Send test traffic to the Envoy Gateway (mirrored to ShadowDiff automatically): curl http://localhost:10000/api/users/1 curl -X POST http://localhost:10000/api/orders Open the telemetry dashboard at http://localhost:8083 to inspect the live 3-way diffs, SQL mutation timeline, and Gemini-generated tests! ☸️ Kubernetes & Istio Service Mesh Deployment For production Kubernetes clusters running Istio or native Envoy, deploy the included EnvoyFilter to mirror traffic dynamically: kubectl apply -f deploy/k8s/envoy-shadow-sidecar.yaml See docs/ARCHITECTURE.md for full architectural specifications. 📄 License This project is licensed under the MIT License.

资料来源

阅读原文 ↗

相关主题

← 继续阅读全部动态