ci: Add GitHub workflows to build and publish coordinator runtime, worker runtime, and worker development environment container images. #26
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,88 @@ | ||
| name: prestocpp-worker-with-clp-connector-runtime-image-build | ||
|
Member
There was a problem hiding this comment. How about prestissimo-worker-images-build? |
||
|
|
||
| on: | ||
| # TODO: specifiy the branch to the release-0.293 when finalize the PR | ||
| pull_request: | ||
| push: | ||
|
|
||
|
There was a problem hiding this comment. 🧹 Nitpick (assertive) Stale TODO & typo; tighten branch filters instead of post-build conditions The comment still says “specifiy” and the TODO is no longer needed—the branch name is already hard-coded further down. -# TODO: specifiy the branch to the release-0.293 when finalize the PR
+# Workflow only runs for the release branch
+
+push:
+ branches:
+ - release-0.293-clp-connector
🤖 Prompt for AI AgentsThere was a problem hiding this comment. 🧹 Nitpick (assertive) TODO & broad trigger – restrict the workflow before merge The comment is still TODO and the on:
push:
branches: [ "release-0.293-clp-connector" ]
pull_request:
branches: [ "release-0.293-clp-connector" ]or drop the TODO entirely. 🤖 Prompt for AI Agents |
||
| jobs: | ||
| prestocpp-worker-with-clp-connector-runtime-image: | ||
|
Member
There was a problem hiding this comment. How about prestissimo-worker-images-build? |
||
| name: prestocpp-worker-with-clp-connector-runtime-image | ||
| runs-on: ubuntu-22.04 | ||
| steps: | ||
| - uses: "actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683" | ||
| with: | ||
| submodules: "recursive" | ||
|
|
||
| - name: "Login to Image Registry" | ||
|
anlowee marked this conversation as resolved.
Outdated
|
||
| uses: "docker/login-action@74a5d142397b4f367a81961eba4e8cd7edddf772" | ||
| with: | ||
| registry: ghcr.io | ||
| username: ${{github.actor}} | ||
| password: ${{secrets.GITHUB_TOKEN}} | ||
|
|
||
| - name: "Update Metadata for Dependency Image" | ||
|
Member
There was a problem hiding this comment. Can we reorder these steps so the order is:
anlowee marked this conversation as resolved.
Outdated
|
||
| id: "meta-dependency" | ||
|
Member
There was a problem hiding this comment. How about metadata-deps-image? |
||
| uses: "docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804" | ||
| with: | ||
| images: >- | ||
| ghcr.io/${{github.repository}}/prestissimo-with-clp-connector-dependency | ||
|
anlowee marked this conversation as resolved.
Outdated
|
||
| tags: | | ||
|
anlowee marked this conversation as resolved.
Outdated
|
||
| type=raw,value=ubuntu-22.04 | ||
|
There was a problem hiding this comment. Image repository is not lower-cased – GHCR rejects mixed-case paths - ghcr.io/${{github.repository}}/prestissimo-with-clp-connector-dependency
+ ghcr.io/${{ github.repository | toLower }} /prestissimo-with-clp-connector-dependency(The same change is needed for the runtime image block below.) env:
REPO_LOWER: ${{ github.repository }}
...
images: ghcr.io/${{ env.REPO_LOWER,, }}/prestissimo-with-clp-connector-dependencyPlease adopt a lower-casing strategy before merging. Also applies to: 48-51 🤖 Prompt for AI Agents
anlowee marked this conversation as resolved.
Outdated
|
||
|
|
||
| - name: "Update Metadata for Runtime Image" | ||
|
anlowee marked this conversation as resolved.
Outdated
|
||
| id: "meta-runtime" | ||
|
Member
There was a problem hiding this comment. How about metadata-runtime-image? |
||
| uses: "docker/metadata-action@902fa8ec7d6ecbf8d84d538b9b233a880e428804" | ||
| with: | ||
| images: >- | ||
| ghcr.io/${{github.repository}}/prestissimo-with-clp-connector-runtime | ||
|
anlowee marked this conversation as resolved.
Outdated
|
||
| tags: | | ||
|
anlowee marked this conversation as resolved.
Outdated
|
||
| type=raw,value=ubuntu-22.04 | ||
|
anlowee marked this conversation as resolved.
Outdated
|
||
|
|
||
|
There was a problem hiding this comment. Repository path may be rejected by GHCR – force lowercase
-images: ghcr.io/${{github.repository}}/prestissimo-with-clp-connector-dependency
+images: ghcr.io/${{ env.REPO_LOWER }}/prestissimo-with-clp-connector-dependencyAdd a job-level env: env:
REPO_LOWER: ${{ github.repository }}and reference with Bash lowercase 🤖 Prompt for AI Agents |
||
| - name: Check for dependency changes in presto-native-execution | ||
| id: dependency-changes | ||
| run: | | ||
| git fetch origin ${{ github.event.before }} --depth=1 || true | ||
| dep_changed=false | ||
| if git diff --name-only ${{ github.event.before }} ${{ github.sha }} | grep -q '^presto-native-execution/scripts'; then | ||
| dep_changed=true | ||
| fi | ||
| cd presto-native-execution/velox | ||
| if git diff --name-only ${{ github.event.before }} ${{ github.sha }} | grep -q '^scripts'; then | ||
| dep_changed=true | ||
| fi | ||
| echo "dep-changed=${dep_changed}" >> $GITHUB_OUTPUT | ||
|
|
||
|
There was a problem hiding this comment.
Hyphens are not allowed in output names ( -echo "dep-changed=${dep_changed}" >> $GITHUB_OUTPUT
+echo "dep_changed=${dep_changed}" >> $GITHUB_OUTPUTand -if: steps.dependency-changes.outputs.dep-changed == 'true'
+if: steps.dependency-changes.outputs.dep_changed == 'true'Without this, the job errors with “Error: Invalid workflow file”.
🤖 Prompt for AI Agents |
||
| - name: "Build and push dependency image" | ||
| if: steps.dependency-changes.outputs.dep-changed == 'true' | ||
| uses: "docker/build-push-action@471d1dc4e07e5cdedd4c2171150001c434f0b7a4" | ||
| with: | ||
| context: "./presto-native-execution/" | ||
| file: "./presto-native-execution/scripts/dockerfiles/ubuntu-22.04-dependency.dockerfile" | ||
| push: >- | ||
| ${{github.event_name != 'pull_request' | ||
| && github.ref == 'refs/heads/release-0.293-clp-connector'}} | ||
| tags: "${{steps.meta-dependency.outputs.tags}}" | ||
| labels: "${{steps.meta-dependency.outputs.labels}}" | ||
|
|
||
|
There was a problem hiding this comment. 🛠️ Refactor suggestion Multi-line tag list passed as
build-args: |
DEPENDENCY_IMAGE=${{ steps.meta-dependency.outputs.tags %% $'\n'* }}or capture it in a preceding step: - run: echo "dep_tag=$(echo "${{ steps.meta-dependency.outputs.tags }}" | head -n1)" >> $GITHUB_OUTPUT
id: select-tag
...
DEPENDENCY_IMAGE=${{ steps.select-tag.outputs.dep_tag }}🤖 Prompt for AI Agents |
||
| - name: Get number of CPU | ||
|
anlowee marked this conversation as resolved.
Outdated
|
||
| id: get-cores | ||
| run: | | ||
|
anlowee marked this conversation as resolved.
Outdated
|
||
| echo "num-threads=$(nproc)" >> $GITHUB_OUTPUT | ||
|
|
||
|
|
||
| - name: "Build and push runtime image" | ||
| uses: "docker/build-push-action@471d1dc4e07e5cdedd4c2171150001c434f0b7a4" | ||
| with: | ||
| build-args: |- | ||
| NUM_THREADS=${{ steps.get-cores.outputs.num-threads }} | ||
| DEPENDENCY_IMAGE=${{ steps.meta-dependency.outputs.tags }} | ||
|
anlowee marked this conversation as resolved.
Outdated
|
||
| BASE_IMAGE=ubuntu:22.04 | ||
| OSNAME=ubuntu | ||
| EXTRA_CMAKE_FLAGS=-DPRESTO_ENABLE_TESTING=OFF -DPRESTO_ENABLE_PARQUET=ON -DPRESTO_ENABLE_S3=ON | ||
|
There was a problem hiding this comment. 🧹 Nitpick (assertive) Unused/unsafe build args – consider quoting and sanitising
EXTRA_CMAKE_FLAGS="-DPRESTO_ENABLE_TESTING=OFF -DPRESTO_ENABLE_PARQUET=ON -DPRESTO_ENABLE_S3=ON"Also ensure downstream 🤖 Prompt for AI Agents
Member
There was a problem hiding this comment. Can we alphabetize these so they're easier to read later? |
||
| context: "./presto-native-execution/" | ||
| file: "./presto-native-execution/scripts/dockerfiles/prestissimo-runtime.dockerfile" | ||
| push: >- | ||
| ${{github.event_name != 'pull_request' | ||
| && github.ref == 'refs/heads/release-0.293-clp-connector'}} | ||
| tags: "${{steps.meta-runtime.outputs.tags}}" | ||
| labels: "${{steps.meta-runtime.outputs.labels}}" | ||
Uh oh!
There was an error while loading. Please reload this page.