Skip to content

Overture docs update - #911

Open
MitchellShiell wants to merge 8 commits into
developfrom
overtureDocsUpdate
Open

Overture docs update#911
MitchellShiell wants to merge 8 commits into
developfrom
overtureDocsUpdate

Conversation

@MitchellShiell

@MitchellShiell MitchellShiell commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Documentation for the Overture docs site.

The docs still described the pre-5.3.0 data model: donor/specimen/sample entities in the base schema, ID management delegated to an external service. They now describe the consolidated 5.3.0 schema and its migration path. Separately, docs/02-Usage/docs/02-Reference/ to match the other components, and the setup guide is rewritten around this repo's own Compose stack instead of the retired Prelude quickstart branch.

Source changes are limited to Makefile and docker-compose.yml: dev tooling, no application logic, no Java.

Song 5.3.0 data model

  • 04-data-model-management.md: base schema corrected to the three fields it actually requires (studyId, analysisType, files); the old text listed donor/specimen/sample IDs and "essential cancer sample characteristics". Adds a Schema Options section for options.fileTypes and options.externalValidations, including that omitting an option preserves its previous value while an empty list clears it.
  • 06-id-management.md: rewritten, replacing a 184-line page. Song generates both IDs it tracks, with no external ID service: analysis IDs are random UUIDs, file IDs deterministic UUID5 hashes of the analysis ID and file name. IDs are immutable, so changing a contributing value means UNPUBLISH + SUPPRESS + resubmit.
  • 11-database-migration.md (new): pre-5.3.0 data spanned multiple tables, 5.3.0 writes to one, so existing databases must be migrated. Documents the TypeScript migration script, opening with a back-up warning.

Each carries a version note recording the old behaviour, including the retired "federated" ID mode.

Setup guide

The old guide opened with git clone -b quickstart .../prelude.git. That branch is retired, so it now uses the docker-compose.yml and Makefile already in this repo.

  • make start-deps → Keycloak 9082, keycloak-db 9444, Score 8087, Minio 8085. make start-song-server adds song-db 8432 and song-server 8080 (debug 5006).
  • Every port in the old table was the Quickstart's (8180, 9000, 5433), not ours. Corrected, the Conductor row is gone, and songDev.svg, which drew the Quickstart topology, is dropped.
  • CLIENT_ACCESS_TOKEN=68fb42b4-… and CLIENT_STUDY_ID=demo are now placeholders. Both were Quickstart values that worked nowhere else, and this stack starts with an empty database, so there is no demo study to query.
  • New walkthrough for getting an API key: set the realm admin password in Keycloak, get a token via the system client's direct access grant, exchange it at apikey/api_key for a song.WRITE/score.WRITE key.
  • Also noted: Keycloak imports myrealm and downloads keycloak-apikeys at start-up; the targets build with ./mvnw, so a JDK is needed even for the supporting services alone. make clean and make log-song-server documented.

Source changes

Makefile. The guide does not work without these:

  • Compose is detected, not assumed: probes docker compose version, falls back to the standalone binary. V1 no longer ships with Docker Desktop, so every Compose target failed outright on a current install.
  • MVN_EXE prefers the bundled ./mvnw over a system mvn.
  • ROOT_DIR moves above the detection block that now depends on it.

docker-compose.yml. Without this the stack could not pull at all. bitnami/keycloak:22 and bitnami/postgresql:11bitnamilegacy/*; Bitnami archived its versioned tags in 2025. Commented in place.

Expect a dirty tree after your first build. fmt-maven-plugin binds format to install, so the documented ./mvnw clean install reformats nineteen committed files under song-server/. Deliberately kept out of this PR. Worth a separate formatting pass on develop.

docs/02-Usage/docs/02-Reference/

Every page moves; 02-Usage.mdx becomes 02-Reference.mdx (retitled "Reference", glossary retained). docs/custom-schemas.md is deleted, because its Options content is now the Schema Options section of 04-data-model-management.md, where anyone configuring an analysis type already is.

One real fix came along in the move: 01-retrieving-metadata.md and 02-updating-metadata.md showed a multi-analysis update querying analysis/search/id?submitterDonorId=<sample>, an endpoint that does not serve submitter sample lookups. Song has no server-side search for submitter sample or donor IDs, so the example now pages through the study's analyses, filters client-side, and patches each match by analysis ID.

Overview and README fixes

  • 00-overview.md: the repo tree said /song-servers (actual: /song-server); the song-java-sdk entry described a package that no longer exists; song-python-sdk was described as being for "java applications".
  • 01-setup.md: the services table credited Keycloak with OAuth2 "for Score" on the Song page, and the Song-server application.yml link pointed into the score repo.
  • README.md: stale docs/core-software/Song/* links → /develop/Song/*, contributing → /develop/contributing, getting started → /deploy.

The docs site's under-development section has been removed; Lyric (the
tabular data submission service) is now the canonical destination for the
'data submission system' reference, so link to its core-software overview
instead of the retired /docs/under-development/ page.
The site renamed /docs/core-software/* to /build/* to /develop/*, moved
/docs/contribution to /develop/contributing, /guides/getting-started to
/deploy, and /guides/administration-guides to /use. Each old path only
resolved through a redirect hop. Verified every new target against a
local site build.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant