Wikibase Suite (WBS) Deploy is a containerized, production-ready Wikibase system that allows you to self-host a knowledge graph similar to Wikidata. In addition to Wikibase on MediaWiki, WBS Deploy includes the Wikidata Query Service (WDQS) and its front end, along with QuickStatements, Elasticsearch and a Traefik reverse proxy with SSL termination and ACME support. The service orchestration is implemented using Docker Compose V2.
🔧 This document is for people wanting to self-host the full Wikibase Suite using Wikibase Suite Deploy. If you are looking for individual WBS images, head over to hub.docker.com/u/wikibase.
💡 This document presumes familiarity with basic Linux administration tasks and with Docker and Docker Compose.
WBS Deploy consists of the following services:
- Wikibase MediaWiki packaged with the Wikibase extension and other commonly used extensions.
- Job Runner The MediaWiki JobRunner service which uses the same Wikibase container as above.
- MariaDB Database service for MediaWiki and Wikibase.
- Elasticsearch Search service used by MediaWiki.
- WDQS Wikidata Query Service to process SPARQL queries.
- WDQS Frontend Web front end for SPARQL queries.
- WDQS Updater Keeps the WDQS data in sync with Wikibase.
- Quickstatements A web-based tool to import and manipulate large amounts of data.
- Traefik A reverse proxy that handles TLS termination and SSL certificate renewal through ACME.
- Network connection with a public IP address
- x86_64 (AMD64) architecture
- 8 GB RAM
- 4 GB free disk space
- Docker 22.0 (or greater)
- Docker Compose 2.10 (or greater)
- git
You need two DNS records that resolve to your machine's IP address:
- Wikibase, e.g., "wikibase.example"
- QueryService, e.g., "query.wikibase.example"
💡 If you want to run a quick test on a machine that has no public IP address (such as your local machine), check our FAQ entry below.
Check out the files from Github, move to the subdirectory deploy.
git clone https://github.com/wmde/wikibase-release-pipeline
cd wikibase-release-pipeline/deployMake a copy of the configuration template in the wikibase-release-pipeline/deploy directory.
cp template.env .envFollow the instructions in the comments in your newly created .env file to set domain names, usernames and passwords.
Run the following command from within wikibase-release-pipeline/deploy:
docker compose upThe first start can take a couple of minutes. You can check the status of the stack by running docker ps from another terminal. When your WBS Deploy instance is ready, the wbs-deploy-wikibase-1 container will be marked healthy.
🎉 Congratulations! You can now access your instance via https://wikibase.example. Make sure to adjust your domain name accordingly.
💡 If anything goes wrong, you can run
docker logs <CONTAINER_NAME>to see some helpful error messages.
To stop, use
docker compose stopMost values set in .env are written into the respective containers after you run docker compose up for the first time.
If you want to reset the configuration while retaining your existing data:
- Make any needed changes to the values in
.env. NOTE: Do not changeDB_*values unless you are also re-creating the database. - Remove your
LocalSettings.phpfile from thedeploy/configdirectory. (Create a backup if you made any changes.) - Remove and re-create containers:
docker compose down
docker compose upOn first launch, WBS Deploy will create files in the deploy/config directory. This is your instance configuration. You own and control these files. Be sure to include them in your backups.
This file is generated by the MediaWiki installer script and supplemented by the Wikibase container's entrypoint.sh script on first launch. Once this file has been generated, you own and control it. You may need to make changes to it for MediaWiki major version updates.
If config/LocalSettings.php is missing, it triggers the Wikibase container to run the MediaWiki installer script. If you need to run the installer again, you can remove the deploy/config/LocalSettings.php file (but keep a backup just in case!) and restart your instance.
This is Wikibase's php.ini override file, a good place for tuning PHP configuration values. It gets loaded by the Mediawiki Wikibase web server's PHP interpreter.
This configuration file allows you to control wdqs-frontend, the GUI for the query service.
It is possible to add extensions to Wikibase Suite Deploy's MediaWiki. To learn how this works, consult the README in deploy/config/extensions.
The Wikibase Suite Wikibase Image has a Call Back feature. This initiative will help maintain an index of Wikibases. The goal of this index is to gather more quantitative data to learn more about how Wikibase is being used. It eventually also aims to be a central hub for data re-use and federation initiatives between Wikibases, where users can discover other Wikibases easily. In the near future, we expect to have a proper showcase of all the Wikibases that have opted in so as to increase discoverability. For now, however, this data will remain only with Wikimedia Deutschland.
You can join this initiative by setting METADATA_CALLBACK=true or disable the feature by setting METADATA_CALLBACK=false in your .env file. If you enable the feature, your hostnames configured in .env will be shared and added to the list. We will then be able to periodically analyze publicly available information on your Wikibase instance. It is important to note that we can only access publicly visible information. If your Wikibase instance requires a login to view data, we will not be able to collect statistics.
You can disable the feature at any time by setting METADATA_CALLBACK=false in your .env file and by sending an E-Mail to wikibase-suite-support@wikimedia.de containing your hostname to remove your instance from the listing and stop periodic analysis.
Let's build the Linked Open Data Web together!
To further customize your instance, you can also make changes to docker-compose.yml. To ease updating to newer versions of WBS Deploy, consider putting your customizations into a new file called, for example, docker-compose.override.yml. If you do this, you'll need to start using the following commands to restart your instance:
docker compose -f docker-compose.yml -f docker-compose.override.yml down
docker compose -f docker-compose.yml -f docker-compose.override.yml upThis way, your changes are kept separate from the original WBS Deploy code.
Besides your configuration, it's your data that makes your instance unique. All instance data is stored in Docker volumes.
wikibase-image-data: MediaWiki image and media file uploadsmysql-data: MediaWiki/Wikibase MariaDB raw databasewdqs-data: Wikidata Query Service raw databaseelasticsearch-data: Elasticsearch raw databasequickstatements-data: generated Quickstatements OAuth binding for this MediaWiki instancetraefik-letsencrypt-data: Traefik SSL certificates generated by letsencrypt
To back up your data, shut down the instance and dump the contents of all Docker volumes into .tar.gz files.
docker compose down
for v in \
wbs-deploy_wikibase-image-data \
wbs-deploy_mysql-data \
wbs-deploy_wdqs-data \
wbs-deploy_elasticsearch-data \
wbs-deploy_quickstatements-data \
wbs-deploy_traefik-letsencrypt-data \
; do
docker run --rm --volume $v:/backup debian:12-slim tar cz backup > $v.tar.gz
doneTo restore the volume backups, ensure your instance has been shut down by running docker compose down and populate the Docker volumes with data from your .tar.gz files.
docker compose down
for v in \
wbs-deploy_wikibase-image-data \
wbs-deploy_mysql-data \
wbs-deploy_wdqs-data \
wbs-deploy_elasticsearch-data \
wbs-deploy_quickstatements-data \
wbs-deploy_traefik-letsencrypt-data \
; do
docker volume rm $v 2> /dev/null
docker volume create $v
docker run -i --rm --volume $v:/backup debian:12-slim tar xz < $v.tar.gz
doneWBS uses semantic versioning. WBS Deploy and each WBS image have individual version numbers.
WBS Deploy always references the latest minor and patch releases of the compatible WBS images' major versions using the Docker images' major version tag. For example, WBS Deploy 2.0.1 might reference wikibase/wikibase:3, a tag that always points to the latest image Wikibase 3.x.x image.
Because WBS Deploy always references the latest minor and patch releases of compatible WBS images, non-breaking changes (including security updates) can be pulled at any time.
This is always safe to do. Simply run:
docker compose down
docker compose pull
docker compose up💡 In order to automatically update images on every start, you can also use
docker compose up --pull alwaysto start your WBS Deploy stack.
If you installed User Defined Extensions in config/extensions, they might have updates too. Make sure to update them regularly too. See User Defined Extension Docs for more information.
WBS Deploy versions are tagged in git with tags such as deploy@2.0.1. Switching to a tag with the same major version will never trigger breaking changes. These updates are always considered safe. If you made no changes to docker-compose.yml, you may update simply by switching the git tag.
git remote update
git checkout deploy@2.0.2💡 If you made any changes to
docker-compose.yml, commit them. Merge with upstream changes as you see fit.
Major version upgrades are performed by updating WBS Deploy's major version. This is done by changing your git checkout to the new major version tag. This may reference new major versions of WBS images and involve breaking changes. In turn, those may require additional steps as described below.
WBS only supports updating from one major version to the next version in sequence. In order to upgrade from 1.x.x to 3.x.x, you must first upgrade from 1.x.x to 2.x.x and then to 3.x.x.
docker compose downCreate a backup of your data.
Back up your ./config directory as well using:
cp -r ./config ./config-$(date +%Y%M%d%H%M%S)
WBS Deploy versions are tagged, such as deploy@2.0.0 or deploy@3.0.3. To update, switch to a more recent tag.
git remote update
git checkout deploy@3.0.3💡 If you made changes to
docker-compose.yml, merge them as you see fit.
Look for changes in the new template.env that you might want to apply to your .env file.
WBS Deploy 4.x.x to 5.x.x
Wikibase Image switched from version 4.x.x to 5.x.x; this upgrades MediaWiki from 1.43 to 1.44. Please read the MediaWiki UPGRADE file.
Please, note that the .env file now requires setting METADATA_CALLBACK. Find more details about it in template.env.
WBS Deploy 3.x.x to 4.x.x
Wikibase Image switched from version 3.x.x to 4.x.x; this upgrades MediaWiki from 1.42 to 1.43. Please read the MediaWiki UPGRADE file.
Note that URLs changed with Deploy 4 to the following defaults:
- https://wikibase.example MediaWiki with Wikibase extension
- https://wikibase.example/w/rest.php MediaWiki REST API including Wikibase REST API
- https://query.wikibase.example Front end for WDQS (Query GUI)
- https://query.wikibase.example/sparql SPARQL API endpoint for WDQS
- https://wikibase.example/tools/quickstatements QuickStatements tool
Note that the wdqs-proxy image has been removed. Routing of WDQS HTTP traffic is now done by central Traefik.
Note that wdqs-frontend environment variables changed. Read more on https://github.com/wmde/wikibase-release-pipeline/tree/main/build/wdqs-frontend#environment-variables
WBS Deploy 2.x.x to 3.x.x
Read the MediaWiki UPGRADE file.
No Wikibase-specific migrations are necessary.
WBS Deploy 1.x.x to 2.x.x
Read the MediaWiki UPGRADE file.
No Wikibase-specific migrations are necessary.
If you installed User Defined Extensions in config/extensions, they might require updates in order to be compatible with the new MediaWiki version too. See User Defined Extension Docs for more information.
docker compose up
At the moment, WBS Deploy does not support automatic updates. To automatically deploy minor and patch updates including security fixes to your WBS images, restart your instance on a regular basis with a systemd timer, cron job, or similar.
Downgrades are not supported. In order to revert an update, restore your data from a backup made prior to the upgrade.
To reset the configuration and data, remove the Docker containers, Docker volumes and the generated deploy/config files.
docker compose down --volumes
rm -vf config/{LocalSettings.php,wikibase-php.ini,wdqs-frontend-config.json}Removing the traefik-letsencrypt-data volume will request a new certificate from LetsEncrypt on the next launch of your instance. Certificate generation on LetsEncrypt is rate-limited; eventually you may be blocked from generating new certificates for multiple days. To avoid that outcome, change to the LetsEncrypt staging server by appending the following line to the traefik command stanza of your docker-compose.yml file:
--certificatesresolvers.letsencrypt.acme.caserver=https://acme-staging-v02.api.letsencrypt.org/directoryTo interact with the WDQS via the web front end, navigate to the URL defined as WDQS_PUBLIC_HOST in the .env file. By default, this is set to query.wikibase.example.
Alternatively, send GET requests with your SPARQL query to the WDQS API endpoint: https://query.wikibase.example/sparql?query={SPARQL}
Due to the OAuth configuration for MediaWiki and QuickStatements, along with the automatic SSL certification generated by Traefik, you must specify a value for WIKIBASE_PUBLIC_HOST in your .env file. This value should be a domain name that resolves to the IP address of the server hosting these services and must be accessible on the Internet.
However, for testing purposes WBS Deploy can be run locally or on a server that is not accessible to the Internet, with the following caveats:
- In this configuration, you will still need to set
WIKIBASE_PUBLIC_HOSTto hostnames that resolve locally to the IP address of the machine running the services. Configuring locally resolving DNS entries differs depending on your environment (Linux, MacOS, Windows), so setting this up correctly will require knowledge of or additional research about your specific setup. - Any SSL certificates generated in this setup will be invalid, though you can optionally bypass the warning about these invalid certificates when first loading the Wikibase site in the browser.
- QuickStatements will not function in this setup, as OAuth will not authorize against a local, non-Internet-accessible Wikibase installation.
It is possible to migrate an existing Wikibase installation to WBS Deploy. The general procedure is as follows:
- Back up your MediaWiki
- Install Wikibase Suite as described above
- Re-apply any changes to
config/LocalSettings.php - Import your database dump
- Regenerate the WDQS database
- Regenerate the Elasticsearch database
Check out the known issue in the WDQS README. You may find your solution there in the form of a workaround.
As of this writing, we can offer no specific recommendations for VPS providers to host Wikibase Suite. The suite has been tested successfully on various providers; as long as the minimum technical requirements are met, it should run as expected.
If you have questions not listed above or need help, use this bug report form to start a conversation with the engineering team.