-
Notifications
You must be signed in to change notification settings - Fork 65
Add Node dependency matrix sample and Compose setup #93
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
officialasishkumar
wants to merge
22
commits into
main
Choose a base branch
from
codex/node-dependency-matrix
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 3 commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
52f42a8
Add node dependency matrix sample
officialasishkumar 4759af3
Add compose setup for node matrix sample
officialasishkumar 36ceeac
docs: tighten node dependency matrix k8s flow
officialasishkumar 1a7a45a
Stabilize node dependency matrix recording flows
officialasishkumar 8b5cb4f
Document node dependency matrix staging runbook
officialasishkumar 0f32218
Implement async catalog synchronization and deduplication endpoints i…
officialasishkumar fa22767
Update Kubernetes traffic recording instructions and enhance scripts …
officialasishkumar e058e3f
Refactor Kafka and SQS scenario functions to improve error handling a…
officialasishkumar 503c04a
Enhance deploy-kind.sh to support custom node images for Kind cluster…
officialasishkumar faf87b6
fix: update expected-values.json to match actual UI dedup behavior
officialasishkumar 93048b8
fix: update requiredKinds to include PostgresV2 and Redis
officialasishkumar 90ef1d0
fix: address Copilot review feedback for node-dependency-matrix
officialasishkumar c09e5b3
fix: reduce requiredKinds to Http only — mock kinds vary between runs
officialasishkumar 106d008
fix: sync package-lock.json with package.json
officialasishkumar d5c74f0
fix: use ECR public mirrors to avoid Docker Hub rate limits
officialasishkumar a083455
fix: use docker pull + kind load instead of crictl pull
officialasishkumar aaf64db
fix: configure Kind with GCR mirror to avoid all registry rate limits
officialasishkumar 616949a
fix: use Keploy's local registry mirror (192.168.116.165:5000)
officialasishkumar d7fadf6
fix: load Kind images one at a time to avoid all-or-nothing failure
officialasishkumar 4c88fc1
fix(node-dependency-matrix): allow Mongo TLS MITM hostname mismatch
officialasishkumar 894935e
fix(node-dependency-matrix): pin Mongo TLS to direct endpoint
officialasishkumar 5f03052
fix(node-dependency-matrix): poll Mongo server monitor through proxy
officialasishkumar File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Some comments aren't visible on the classic Files Changed page.
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| node_modules | ||
| dist | ||
| .generated | ||
| README.md | ||
| k8s | ||
| scripts | ||
| keploy |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| dist | ||
| node_modules | ||
| .generated | ||
| keploy |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| FROM node:20-bookworm-slim AS build | ||
|
|
||
| WORKDIR /app | ||
|
|
||
| COPY package*.json ./ | ||
| RUN npm ci | ||
|
|
||
| COPY tsconfig.json ./ | ||
| COPY proto ./proto | ||
| COPY fixtures ./fixtures | ||
| COPY src ./src | ||
| COPY entrypoint.sh ./entrypoint.sh | ||
|
|
||
| RUN npm run build && chmod +x /app/entrypoint.sh | ||
|
|
||
| FROM node:20-bookworm-slim | ||
|
|
||
| WORKDIR /app | ||
|
|
||
| RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| COPY --from=build /app/node_modules ./node_modules | ||
| COPY --from=build /app/dist ./dist | ||
| COPY --from=build /app/proto ./proto | ||
| COPY --from=build /app/fixtures ./fixtures | ||
| COPY --from=build /app/entrypoint.sh ./entrypoint.sh | ||
|
|
||
| ENV NODE_ENV=production | ||
| ENV PORT=8080 | ||
| ENV GRPC_PORT=9090 | ||
|
|
||
| ENTRYPOINT ["./entrypoint.sh"] | ||
| CMD ["node", "dist/bin/app.js"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| # Node Dependency Matrix | ||
|
|
||
| This sample is a TypeScript application built to validate Keploy cloud record/replay against a broad dependency matrix. | ||
|
|
||
| What it exposes: | ||
|
|
||
| - incoming HTTP | ||
| - incoming gRPC | ||
| - outgoing HTTPS | ||
| - outgoing HTTP/2 | ||
| - outgoing gRPC | ||
| - outgoing MySQL | ||
| - outgoing Postgres | ||
| - outgoing Mongo | ||
| - outgoing Redis | ||
| - outgoing Kafka | ||
| - outgoing SQS-over-HTTPS | ||
| - outgoing generic TLS traffic | ||
| - a noisy endpoint | ||
| - an expected replay failure endpoint | ||
| - duplicate-friendly endpoints for static dedup verification | ||
|
|
||
| ## Quick start | ||
|
|
||
| ```bash | ||
| cd samples-typescript/node-dependency-matrix | ||
| npm install | ||
| npm run build | ||
| bash k8s/deploy-kind.sh | ||
| APP_URL=http://localhost:30081 bash scripts/record_traffic.sh | ||
| ``` | ||
|
|
||
| If the default host ports are already occupied, override them: | ||
|
|
||
| ```bash | ||
| HOST_PROXY_PORT=31080 HOST_APP_PORT=31081 HOST_GRPC_PORT=31090 bash k8s/deploy-kind.sh | ||
| APP_URL=http://localhost:31081 bash scripts/record_traffic.sh | ||
| ``` | ||
|
|
||
| Optional incoming gRPC traffic: | ||
|
|
||
| ```bash | ||
| GRPC_TARGET=localhost:30090 bash scripts/send_grpc_traffic.sh | ||
| ``` | ||
|
|
||
| ## Docker Compose | ||
|
|
||
| For a non-Kubernetes local stack: | ||
|
|
||
| ```bash | ||
| cd samples-typescript/node-dependency-matrix | ||
| npm install | ||
| bash scripts/compose_up.sh | ||
| APP_URL=http://localhost:38081 bash scripts/record_traffic.sh | ||
| GRPC_TARGET=localhost:39090 bash scripts/send_grpc_traffic.sh | ||
| ``` | ||
|
|
||
| To stop it: | ||
|
|
||
| ```bash | ||
| bash scripts/compose_down.sh | ||
| ``` | ||
|
|
||
| If `38081` or `39090` are occupied: | ||
|
|
||
| ```bash | ||
| APP_HTTP_PORT=48081 APP_GRPC_PORT=49090 bash scripts/compose_up.sh | ||
| APP_URL=http://localhost:48081 bash scripts/record_traffic.sh | ||
| GRPC_TARGET=localhost:49090 bash scripts/send_grpc_traffic.sh | ||
| ``` | ||
|
|
||
| ## Keploy expectations | ||
|
|
||
| The machine-readable contract is: | ||
|
|
||
| ```bash | ||
| fixtures/expected-values.json | ||
| ``` | ||
|
|
||
| Use it later in Playwright or manual verification for: | ||
|
|
||
| - testcase counts | ||
| - required and acceptable mock kinds | ||
| - scenario-to-mock-kind mapping | ||
| - the exact HTTP and gRPC traffic plans | ||
| - hosted UI + Helm expectations for Kind | ||
| - total mock count ranges | ||
| - dedup expectations | ||
| - expected replay pass/noisy/fail scenarios | ||
|
|
||
| ## Kubernetes | ||
|
|
||
| The full Kind and hosted UI flow is in: | ||
|
|
||
| ```bash | ||
| k8s/README.md | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,205 @@ | ||
| name: node-dependency-matrix | ||
|
|
||
| x-node-matrix-image: &node_matrix_image | ||
| build: | ||
| context: . | ||
| image: node-dependency-matrix:compose | ||
|
|
||
| services: | ||
| app: | ||
| <<: *node_matrix_image | ||
| depends_on: | ||
| fixture-service: | ||
| condition: service_started | ||
| mysql: | ||
| condition: service_healthy | ||
| postgres: | ||
| condition: service_healthy | ||
| mongo-tls: | ||
| condition: service_started | ||
| redis-tls: | ||
| condition: service_started | ||
| redpanda: | ||
| condition: service_started | ||
| sqs-tls: | ||
| condition: service_started | ||
| environment: | ||
| PORT: "8080" | ||
| GRPC_PORT: "9090" | ||
| FIXTURE_HTTPS_BASE: https://fixture-service:8443 | ||
| FIXTURE_HTTP2_ORIGIN: https://fixture-service:9443 | ||
| FIXTURE_GRPC_TARGET: fixture-service:50051 | ||
| FIXTURE_GENERIC_HOST: fixture-service | ||
| FIXTURE_GENERIC_PORT: "9445" | ||
| MYSQL_URL: mysql://root@mysql:3306/matrix | ||
| POSTGRES_URL: postgresql://postgres@postgres:5432/matrix | ||
| MONGO_URL: mongodb://mongo-tls:27017/matrix?tls=true | ||
| REDIS_URL: rediss://redis-tls:6380 | ||
| KAFKA_BROKERS: redpanda:9092 | ||
| KAFKA_TOPIC: matrix-events | ||
| SQS_ENDPOINT: https://sqs-tls:4567 | ||
| SQS_QUEUE_URL: https://sqs-tls:4567/000000000000/dependency-matrix | ||
| SAMPLE_CA_CERT_PATH: /etc/sample-certs/ca.crt | ||
| COMBINED_CA_CERT_PATH: /tmp/node-dependency-matrix-ca-bundle.crt | ||
| NODE_EXTRA_CA_CERTS: /tmp/node-dependency-matrix-ca-bundle.crt | ||
| SSL_CERT_FILE: /tmp/node-dependency-matrix-ca-bundle.crt | ||
| ports: | ||
| - "${APP_HTTP_PORT:-38081}:8080" | ||
| - "${APP_GRPC_PORT:-39090}:9090" | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro | ||
|
|
||
| fixture-service: | ||
| <<: *node_matrix_image | ||
| command: ["node", "dist/bin/dependencyFixture.js"] | ||
| environment: | ||
| TLS_CERT_PATH: /etc/sample-certs/proxy.crt | ||
| TLS_KEY_PATH: /etc/sample-certs/proxy.key | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro | ||
|
|
||
| mysql: | ||
| image: mysql:8.0 | ||
| environment: | ||
| MYSQL_ALLOW_EMPTY_PASSWORD: "yes" | ||
| MYSQL_DATABASE: matrix | ||
| healthcheck: | ||
| test: ["CMD", "mysqladmin", "ping", "-h", "127.0.0.1", "-uroot"] | ||
| interval: 5s | ||
| timeout: 5s | ||
| retries: 20 | ||
|
|
||
| postgres: | ||
| image: postgres:16 | ||
| environment: | ||
| POSTGRES_HOST_AUTH_METHOD: trust | ||
| POSTGRES_DB: matrix | ||
| POSTGRES_USER: postgres | ||
| healthcheck: | ||
| test: ["CMD-SHELL", "pg_isready -U postgres -d matrix"] | ||
| interval: 5s | ||
| timeout: 5s | ||
| retries: 20 | ||
|
|
||
| mongo: | ||
| image: mongo:7 | ||
| command: ["--bind_ip_all"] | ||
|
|
||
| redis: | ||
| image: redis:7-alpine | ||
|
|
||
| redpanda: | ||
| image: docker.redpanda.com/redpandadata/redpanda:v25.1.2 | ||
| command: | ||
| - redpanda | ||
| - start | ||
| - --overprovisioned | ||
| - --smp | ||
| - "1" | ||
| - --memory | ||
| - 512M | ||
| - --reserve-memory | ||
| - 0M | ||
| - --check=false | ||
| - --node-id | ||
| - "0" | ||
| - --kafka-addr | ||
| - PLAINTEXT://0.0.0.0:9092 | ||
| - --advertise-kafka-addr | ||
| - PLAINTEXT://redpanda:9092 | ||
| - --set | ||
| - redpanda.auto_create_topics_enabled=true | ||
|
|
||
| localstack: | ||
| image: localstack/localstack:3.3 | ||
| environment: | ||
| SERVICES: sqs | ||
| AWS_DEFAULT_REGION: us-east-1 | ||
| volumes: | ||
| - ./scripts/localstack-init:/etc/localstack/init/ready.d:ro | ||
|
|
||
| mysql-tls: | ||
| <<: *node_matrix_image | ||
| command: ["node", "dist/bin/tlsProxy.js"] | ||
| depends_on: | ||
| mysql: | ||
| condition: service_healthy | ||
| environment: | ||
| LISTEN_PORT: "3306" | ||
| TARGET_HOST: mysql | ||
| TARGET_PORT: "3306" | ||
| TLS_CERT_PATH: /etc/sample-certs/proxy.crt | ||
| TLS_KEY_PATH: /etc/sample-certs/proxy.key | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro | ||
|
|
||
| postgres-tls: | ||
| <<: *node_matrix_image | ||
| command: ["node", "dist/bin/tlsProxy.js"] | ||
| depends_on: | ||
| postgres: | ||
| condition: service_healthy | ||
| environment: | ||
| LISTEN_PORT: "5432" | ||
| TARGET_HOST: postgres | ||
| TARGET_PORT: "5432" | ||
| TLS_CERT_PATH: /etc/sample-certs/proxy.crt | ||
| TLS_KEY_PATH: /etc/sample-certs/proxy.key | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro | ||
|
|
||
| mongo-tls: | ||
| <<: *node_matrix_image | ||
| command: ["node", "dist/bin/tlsProxy.js"] | ||
| depends_on: | ||
| - mongo | ||
| environment: | ||
| LISTEN_PORT: "27017" | ||
| TARGET_HOST: mongo | ||
| TARGET_PORT: "27017" | ||
| TLS_CERT_PATH: /etc/sample-certs/proxy.crt | ||
| TLS_KEY_PATH: /etc/sample-certs/proxy.key | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro | ||
|
|
||
| redis-tls: | ||
| <<: *node_matrix_image | ||
| command: ["node", "dist/bin/tlsProxy.js"] | ||
| depends_on: | ||
| - redis | ||
| environment: | ||
| LISTEN_PORT: "6380" | ||
| TARGET_HOST: redis | ||
| TARGET_PORT: "6379" | ||
| TLS_CERT_PATH: /etc/sample-certs/proxy.crt | ||
| TLS_KEY_PATH: /etc/sample-certs/proxy.key | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro | ||
|
|
||
| kafka-tls: | ||
| <<: *node_matrix_image | ||
| command: ["node", "dist/bin/tlsProxy.js"] | ||
| depends_on: | ||
| - redpanda | ||
| environment: | ||
| LISTEN_PORT: "9094" | ||
| TARGET_HOST: redpanda | ||
| TARGET_PORT: "9092" | ||
| TLS_CERT_PATH: /etc/sample-certs/proxy.crt | ||
| TLS_KEY_PATH: /etc/sample-certs/proxy.key | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro | ||
|
|
||
| sqs-tls: | ||
| <<: *node_matrix_image | ||
| command: ["node", "dist/bin/tlsProxy.js"] | ||
| depends_on: | ||
| - localstack | ||
| environment: | ||
| LISTEN_PORT: "4567" | ||
| TARGET_HOST: localstack | ||
| TARGET_PORT: "4566" | ||
| TLS_CERT_PATH: /etc/sample-certs/proxy.crt | ||
| TLS_KEY_PATH: /etc/sample-certs/proxy.key | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| #!/bin/sh | ||
| set -eu | ||
|
|
||
| COMBINED_CA_CERT_PATH="${COMBINED_CA_CERT_PATH:-/tmp/node-dependency-matrix-ca-bundle.crt}" | ||
|
|
||
| rm -f "${COMBINED_CA_CERT_PATH}" | ||
|
|
||
| if [ -n "${SAMPLE_CA_CERT_PATH:-}" ] && [ -f "${SAMPLE_CA_CERT_PATH}" ]; then | ||
| cat "${SAMPLE_CA_CERT_PATH}" > "${COMBINED_CA_CERT_PATH}" | ||
| cp "${SAMPLE_CA_CERT_PATH}" /usr/local/share/ca-certificates/node-dependency-matrix.crt | ||
| update-ca-certificates >/dev/null 2>&1 || true | ||
| fi | ||
|
|
||
| if [ -f /tmp/keploy-tls/ca.crt ]; then | ||
| cat /tmp/keploy-tls/ca.crt >> "${COMBINED_CA_CERT_PATH}" | ||
| fi | ||
|
|
||
| exec "$@" | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.