Migrate to monaco editor

[vemonet]

Description

I looked into building an LSP-independent editor, which means a lot of breaking changes to the API: instead of just having Yasqe as object to configure everything related to language checks (lint/formatting/diagnostics, completion), we will now have Yasqe that will just handle display, and the LSP (external dependency) will handle the language checks, and will be passed to Yasqe

Important

Tl;dr: Monaco editor is better with LSP than CodeMirror 6 and look nicer, but larger bundle (~11MB vs 1.2MB, still loads fast in production for me though)

Monaco editor with the qlue-ls language server is deployed on my fork: https://vemonet.github.io/Yasgui

@ajuvercr would you be interested to help with implementing Yasgui with swls? I have created a swls.html in dev/ to test it. We can have a call to check this together

Migration to CodeMirror 6

I checked migrating to CodeMirror v6 with language servers, I have a prototype working with qlue-ls here, and most features available (diagnostics, code actions, completion, syntax highlight, format): https://github.com/rdfjs/Yasgui/tree/migrate-to-codemirror6

npm i && npm run dev

The official CodeMirror LSP-client is missing a few features (released really recently in summer 2025), such as no support for semantic tokens highlight, no support for pull notifications (which qlue-ls is using), so we need to reimplement these on top of the codemirror lsp-client for each LS: https://github.com/rdfjs/Yasgui/blob/migrate-to-codemirror6/dev/qlueLs.ts

Migration to the Monaco editor

In term of looks and feels CodeMirror 6 is decent, but I still prefer Monaco/VSCode. Monaco has custom ctrl+f search, and custom right click menu built-in

The limitations we are hitting with @codemirror/lsp-client are absent from the Monaco editor, we have full LSP support out of the box (with qlue-ls at least) without having to do to much middleware. Which makes sense because afaik the LSP came with VSCode (microsoft development)

I managed to configure vite to properly package the Monaco wasm workers. Fully replaced the CodeMirror in Yasqe by Monaco editor (configurable, with sane defaults), it is language server agnostic, the LS needs to be passed at Yasgui/Yasqe creation.

If the language server provides semantic highlight then it will be used. If no language server is provided, syntax highlighting will still work, but no diagnostics or completion.

I deployed it on my fork to make the whole thing was properly working:

• main page is just the YASGUI editor: vemonet.github.io/Yasgui
• I have started a markdown based docs website using vitepress (so the main page is served by vitepress): https://vemonet.github.io/Yasgui/docs/introduction

The dark part is that to get the Monaco wasm workers resolved properly in Yasqe/Yasgui nested packages we need to inline all workers in the file, resulting in a final yasgui.js of 11MB (instead of 1.2MB previously with codemirror), 2.7MB gzipped, without the Language server (add a few MBs more, but separated, so less impactful)

For Yasgui deployer UX, in the future we might want to consider having packages with a preloaded/preconfigured language server for convenience, e.g. @rdfjs/sparql-studio-qluels and @rdfjs/sparql-studio-swls

For now, I focused on implementing using the qlue-ls language server, but it would be interesting if @ajuvercr can add the equivalent needed to use swls (swls.html + swls.ts in dev/ folder)

This way we better understand the final API we need to support different language servers

Conclusion

If we want to go down the language server road, then I think it would be better to go with Monaco editor over CodeMirror:

• Better support for LSP
• More modern UI, the most popular among devs
• Bigger community (most modern IDEs are VSCode forks)

The main drawback is a large bundle size (11MB vs 1.2MB), but that can be improved, and does not seems that impactful

There is also value in taking different approaches. Matdata fork currently still uses CodeMirror 5, and I think he is more interested in CodeMirror 6 than Monaco. If people wants to use a maintained CodeMirror based Yasgui editor, they can use the @matdata/Yasgui package, and we cover the Monaco crowd

Adding a configurable LSP will anyway increase complexity for final user who deploy Yasgui. But we can’t make an omelet without breaking eggs, a more capable system often brings some complexity, especially if we can to keep modularity. And anyway deployment complexity is mitigated by coding agents that can easily wire the different components for the devs using Yasgui

@ludovicm67 @rubensworks @ktk

Changes

• Migrate from CodeMirror 5 to Monaco editor (VSCode) with a language server
  • Yasqe now extends EventEmitter instead of CodeMirror
  • Use a wasm language server for diagnostics, syntax highlight, autocompletion, code actions, etc
  • In the dev/index.html implement a demo using the Qlue-LS language server
  • Delete code in yasqe related to built-in autocomplete and grammar (since this now handled by the language server)
  • Update some of the puppeteer tests
• Add docs website with VitePress
  • main page is a bare Yasgui with qlue-ls language server
  • documentation to use yasgui is defined in markdown files available at /docs
  • API references docs auto generated from code
• Add support for light/dark theme
• Enable to import the main JS and CSS files from @zazuko/yasgui and @zazuko/yasgui/style.css
• Update CodeMirror used by the YASR component to display raw response JSON from v5 to v6
• Update default SPARQL endpoint from dbpedia to https://sparql.dblp.org/sparql (faster for completion queries)

Checklist

• Tests added or updated as needed
• Documentation added or updated as needed
• Changeset added (npx changeset add) if this change warrants a release

[ludovicm67]

Do you think it might be relevant to have a package for Monaco, and a second once for CodeMirror, and let the user pick the one they want?
We can include an opinionated combo for the global package.

[vemonet]

Do you think it might be relevant to have a package for Monaco, and a second once for CodeMirror, and let the user pick the one they want? We can include an opinionated combo for the global package.

Note that for each language server we will need a connector to each editor (right now just 2x2), with maybe some caveats to overcome and wrapping to do. Then users will be coming with issues for the combination of editors X language servers

We will need to make sure Yasqe monaco and CodeMirror surface the same API to Yasgui (which might limit us)

So possible but some additional work, not sure if it is worth it.

I would be down to go for it if there are request, but I suggest people first try the 2 versions in practice (monaco from my fork deployment, and local deployment of the codemirror branch) before deciding if it is worth to pursue

I would say

• 1st we figure out the API over Yasqe/Yasgui for language servers (while integrating swls)
• We might look into having packages with the language servers bundled
• Then we explore having support for CodeMirror over the stabilized APIs (especially if there are explicit request for it)

[ajuvercr]

I'll give it a spin today, indeed the monaco version looks quite nice already!

[vemonet]

I have added a @yasqe-codemirror package (names will change in the future)

We can try it there: https://vemonet.github.io/Yasgui/codemirror

It is indeed faster to load (almost instant with codemirror for me)

I moved a lot of Yasqe to the utils package (the shared IYasqe interface + the functions to send SPARQL requests to endpoints)

The LS-CodeMirror connector is a bit verbose because:

• Qlue-ls was not accepting doc change request without a range, which was sent by the @codemirror/lsp-client, this will be fixed at next release [BUG] Server rejects rangeless didChange content changes IoannisNezis/Qlue-ls#313
• We need to manually wire diagnostics from the language server to codemirror to get code actions to work
• @codemirror/lsp-client does not support semantic token highlighting (a modern way to have theme-independant syntax hightlighting). To make this work we add CSS classes to the tokens, then control color from CSS.

There is still a lot to improve (especially regarding the language server implementation), but that works!

I would like to see how would look a connector for swls with @ajuvercr, once we have this we can think more seriously about how we want to deal with these language servers

[ajuvercr]

It was really easy to add swls to this setup, and I'm quite impressed by your general approach @vemonet .

I've added a single commit to migrate-to-monaco-editor-swls but please don't look at my worker code, I'm fixing this to be more in line with Qlue-ls as that is elegantly implemented.

[vemonet]

It could be interesting to enable to have multiple language servers made available to the user to choose from directly in the UI.

We let the user switch language server at the click of a button in Yasqe, this way they can use the best language server for each endpoint. Then we store language servers preference per endpoint

Typically qlue-ls completion works well for Qlever endpoints, but can be a bit unstable on large virtuoso like dbpedia. In this case the user can use swls

So instead of just passing 1 language server, consumer pass a list of language servers

Then we need also to think of a nice way to get a configuration UI for the language servers directly from Yasgui (ideally, without requiring each deployment to setup config panels for each LS).

We could use a generic form that takes a JSON-schema describing the language server config to render the config UI, and have a callback that enables to deal with the generated JSON config file (to apply it to the LS). Would need to see how swls does settings, to find the right approach

I have implemented the array of language servers in this PR, sorry @ajuvercr there will be a little change to do, but the previous worker should work as it was, you just need to provide it in a list of objects, with per server onReady and onEndpointChange hooks, e.g. (do it directly in dev/index.html):

const yasqe = new Yasqe(el, { ...conf, languageServers: [
    {
        label: "swls",
        description: "Ontology-based completions",
        worker: createSwlsWorker,
        onReady: (client: any) => {
            swls.configureSettings(client, settings);
            swls.configureBackend(client, yasgui?.getTab()?.getEndpoint() ?? DEMO_ENDPOINT);
        },
        onEndpointChange: (client: any, endpoint: string) => qlueLs.configureBackend(client, endpoint),
    },
]})

If only 1 language servers it will just be used, no way to change it. If 2+ you get a dropdown list, and language server preference stored per endpoint.

A question I have is: how should we present the menu to switch language server in monaco editor?

• 👍 A submenu in the right click menu?
  • Cleaner UI, but users needs to know they can right click to switch language server
• 👎 Same dropdown menu as for CodeMirror next to the execute/share query buttons?
  • Add a bit of clutter to the UI, but more consistent with CodeMirror UI and more obvious

[rubensworks]

For reference, @ajuvercr implemented a prototype for adding @jitsedesmet's Traqula using this LSP architecture: https://github.com/ajuvercr/Yasgui/blob/migrate-to-monaco-editor-traqula/dev/traqula.worker.ts

[vemonet]

I merged in @ajuvercr implementations for swls and traqula, thanks a lot!

• All 3 language servers are available to choose at runtime:
  • Qlue-ls: full support (semantic highlighting, completions)
  • swls: there seems to be some semantic highlighting, but not complete, and could not make the completions work besides keyword completion. I am seeing error about retrieving some namespaces from the linked open vocabularies in the console
  • Traqula: no semantic highlighting, we use a default syntax highlighting (monarch for monaco, lezer for codemirror)
• When initializing Yasqe with the language server you can pass the JSON schema of the language server settings, and provide a callback to apply the settings to your language server. This way we have a standard user-friendly UI to configure language server, and it can adapt to most language server settings approach
• Since the next release will be under the new name I started renaming Yasgui/Yasqe/Yasr packages to @rdfjs/sparql-studio|sparql-editor-monaco|sparql-editor-codemirror|sparql-results
• While renaming I found the address sparql.studio was available which perfectly fits the lib, so I deployed the website there

Regarding the renaming a few questions are left:

• Do we keep yasgui- for the CSS classes? Changing it will break compability, especially with 3rd party Yasr plugins that might use the yasr- prefix for CSS classes. Keeping it will slightly increase comlexity for dev using sparql-studio, but not a big problem if well documented, it's something the final user don't feel
• Do we keep yasgui for the localStorage key? Changing it will cause every existing user to lose their currently saved tabs. We could also add a migration logic, but would add a bit more of complexity

[rubensworks]

Very nice @vemonet!
I also like https://sparql.studio/ :-)

So do I understand correctly that the deployment on https://sparql.studio/ has all 3 language servers enabled by default?
Because I see 1.2 queries are supported there, which I suspect is due to Traqula being enabled as well (as fallback)?

Do we keep yasgui- for the CSS classes?

I would not keep the yasgui classes.
What we could however do for backwards-compat, is dynamically add the correspond yasgui classes via javascript. (I don't know the code that well, but I'd suspect something like that should be possible here)

Do we keep yasgui for the localStorage key?

I would also use a new name for this.
It's a new package, so it may even be preferrable to have a different key to avoid collisions.

[vemonet]

So do I understand correctly that the deployment on https://sparql.studio/ has all 3 language servers enabled by default?

yes, qlue-ls and traqula supports 1.2 queries, swls I don't know

I would not keep the yasgui classes.
What we could however do for backwards-compat, is dynamically add the correspond yasgui classes via javascript

Doing this would mean a lot of changes in the current styling approach, making it more complex (from my experience css-in-js can be messy). To be fair I am saying that will "break compatibility", but that's not going to be that dramatic

For studio and editor, that will not change much. Just that people who already deployed yasgui and customized CSS using .yasgui would need to replace their stuff with .sparql-studio

For results, that might create styling issues if some external plugins we use are styling stuff using .yasr classes. It should not break the thing, just that style might be a bit off, which can always be patched manually in a few lines of CSS.

Some we might as well just make the move and add fixes later if really needed