Skip to content

refactor(export): migrate from puppeteer to satori#4343

Draft
darentanrw wants to merge 10 commits into
nusmodifications:masterfrom
darentanrw:improved-exports
Draft

refactor(export): migrate from puppeteer to satori#4343
darentanrw wants to merge 10 commits into
nusmodifications:masterfrom
darentanrw:improved-exports

Conversation

@darentanrw
Copy link
Copy Markdown

@darentanrw darentanrw commented Mar 7, 2026

Context

The current export service uses Puppeteer (via puppeteer-core and @sparticuz/chromium) to render timetable images and PDFs.

This approach has several drawbacks, the biggest of which being slow performance due to cold starts and chromium load time. This has resulted in issues such as 504 Errors (#4290) where function time outs, and fixes to increase timeout to 15 seconds such as (#4255)

Implementation

Instead, I propose to replace Puppeteer with Satori + resvg-js for server-side timetable rendering:

  • Satori converts React JSX → SVG entirely in Node.js
  • resvg-js converts SVG → PNG
  • PDFKit composites the rendered PNG into a properly formatted A4 PDF

ExportData plus modules are parsed into buildRenderableTimetable, which produces a RenderableTimetable view model.

That model is fed into TimetableImage in view.tsx, which is then rendered from JSX → SVG → PNG → PDF

Drawbacks and Limitations

Because Satori does not run a real browser, It only accepts JSX elements that are pure and stateless, and a subset of layout and styling (inline styles, no CSS classes, no full DOM). The NUSMods timetable uses SCSS, class-based styling, so it can’t be rendered directly by Satori. view.tsx hence is a separate React tree that mirrors the timetable layout using only Satori-compatible inline styles (thanks Claude).

However:

  1. This is not currently pixel-perfect with the website (scroll to bottom to see UI changes). In particular, the font is not the same as NUSMods currently uses system font?
  2. There is duplicated logic and data:
  • export/src/types.ts- ExportData, Module, SemTimetableConfig, etc. (from website/src/types/export.ts, types/timetables.ts, types/modules.ts)
  • export/src/satori/theme.ts - Theme color palettes (from website/src/styles/constants.scss $nusmods-theme-colors)
  • export/src/satori/render-model.ts — Time conversion, lesson arrangement, week formatting, color mapping (from website/src/utils/timify.ts, utils/timetables.ts, utils/colors.ts)

This has potential future painful implication - as any change to the timetable UI (layout, themes, week formatting, etc.) must be reflected in both the website components and the export view.tsx / render-model.ts, increasing the chance of inconsistency.

Note: This was also done from my assumption of that you are deploying your monorepo separately onto different vercel instances - there are ways we can reduce this data duplication with packages and etc, but it will require a more significant rewrite?

Future work: Should v4 migration (#4314) be implemented and Next.js is chosen, this feature also can be implemented via Next.js ImageResponse as well.

Performance Improvements

Performance improvements of 60~70% 🥳!

Wrote a shell script, at export/utils/endpoint-performance-test.sh which ran 10 simulated timetables against both the old (Puppeteer) and new (Satori) endpoints. I deployed the new export service at https://nusmods-export.vercel.app, and latency was measured on an M3 MacBook Pro over Ethernet under typical usage.

CleanShot 2026-03-08 at 00 56 20@2x Full log here: 080326-1255-performance-test-output.txt

For more precise numbers, we should inspect Vercel function logs to compare cold starts and execution time between the two implementations.

Visual Differences

New Export (Satori) Old Export (Puppeteer)
1-new-image 1-old-image
2-new-image 2-old-image
4-new-image
PDF Version
4-old-image
PDF Version
8-new-image
PDF Version
8-old-image PDF Failed to Render (Error Here)

Misc

First PR to nusmods, any feedback will be appreciated!

I found many edge cases, but I'm sure that there are more that I have yet to consider, so would love your inputs and edits on this PR.

Thanks a lot! Especially to @ravern and @leslieyip02 for planting this idea in my head.

@vercel
Copy link
Copy Markdown

vercel Bot commented Mar 7, 2026

@darentanrw is attempting to deploy a commit to the modsbot's projects Team on Vercel.

A member of the Team first needs to authorize it.

@codecov
Copy link
Copy Markdown

codecov Bot commented Mar 7, 2026

Codecov Report

❌ Patch coverage is 50.00000% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 56.39%. Comparing base (988c6fd) to head (1c7b5d2).
⚠️ Report is 236 commits behind head on master.

Files with missing lines Patch % Lines
website/src/apis/export.ts 0.00% 3 Missing ⚠️
website/src/views/timetable/ExportMenu.tsx 80.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #4343      +/-   ##
==========================================
+ Coverage   54.52%   56.39%   +1.87%     
==========================================
  Files         274      317      +43     
  Lines        6076     6970     +894     
  Branches     1455     1682     +227     
==========================================
+ Hits         3313     3931     +618     
- Misses       2763     3039     +276     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Comment thread export/package.json Outdated
Comment thread export/package.json
@yangshun
Copy link
Copy Markdown
Member

yangshun commented Mar 8, 2026

Note: This was also done from my assumption of that you are deploying your monorepo separately onto different vercel instances - there are ways we can reduce this data duplication with packages and etc, but it will require a more significant rewrite?

Sounds like a great idea. I'll be starting on Phase 2 of #4314 soon and we can tackle this extraction.

Comment thread export/src/types.ts
Comment thread export/package.json
Comment thread export/package.json
Comment thread export/package.json Outdated
Comment thread export/package.json Outdated
@yangshun
Copy link
Copy Markdown
Member

yangshun commented Mar 8, 2026

Some thoughts:

  1. The current export service acts like a general screenshotting service, which is in some sense better because it doesn't need to contain any rendering or layout logic. The export service will work regardless of how the website looks, which may or may not be desired.
  2. If we're going to share the data model between website and export, we will need to ensure that either data model changes are backwards-compatible, or both apps are deployed at the same time. The current export already is coupled to the data model, but not as much as the new way.

Lastly, can we also explore if https://www.renoun.dev/screenshot works? I think it runs on the client and maybe we can use it for images and avoid a server round-trip.

I'm torn between the export approaches. Decoupling the rendering has good benefits too, so I guess the question is how much the visual consistency between website and export matters.

@darentanrw
Copy link
Copy Markdown
Author

Thanks for the review! I've fixed the low hanging fruits that you commented on.

I will take a further look into https://www.renoun.dev/screenshot, however a potential problem that I foresee is that it might not be easy to select only the timetable and module info as a PNG export.

The current workflow with Puppeteer screenshots https://nusmods.vercel.app/timetable-only after injecting it with the timetable data. However, what is currently rendered to the user also includes the TimetableActions and ModuleSelectContainer components. Will need to figure out a way to exclude them from the local screenshot, or to redirect to timetable-only, then screenshot, which is not an ideal user experience.

In this case of using Serverless functions for export, I believe that a cleaner approach to couple the data models is by shifting the export serverless handlers into the main website APIs (to website/api/export/*).

Personally, I don't think that the visual consistency between website and export matters that much (See Visual Differences in original PR), and I think that we can make it more similar (except for fonts) - but ultimately this is a decision for the maintainers to make.

@yangshun
Copy link
Copy Markdown
Member

yangshun commented Mar 8, 2026

Thanks for the review! I've fixed the low hanging fruits that you commented on.

I will take a further look into https://www.renoun.dev/screenshot, however a potential problem that I foresee is that it might not be easy to select only the timetable and module info as a PNG export.

The current workflow with Puppeteer screenshots https://nusmods.vercel.app/timetable-only after injecting it with the timetable data. However, what is currently rendered to the user also includes the TimetableActions and ModuleSelectContainer components. Will need to figure out a way to exclude them from the local screenshot, or to redirect to timetable-only, then screenshot, which is not an ideal user experience.

In this case of using Serverless functions for export, I believe that a cleaner approach to couple the data models is by shifting the export serverless handlers into the main website APIs (to website/api/export/*).

Personally, I don't think that the visual consistency between website and export matters that much (See Visual Differences in original PR), and I think that we can make it more similar (except for fonts) - but ultimately this is a decision for the maintainers to make.

Those are great points! I'm convinced this is a good approach. Like you said, we can also co-locate the exporting functions into the web app, which is a good idea for the future.

Btw I've started extracting the types in #4346, depending on whose PR merges first, someone will have to resolve the conflicts, but it shouldn't be hard.

@jloh02
Copy link
Copy Markdown
Member

jloh02 commented Mar 21, 2026

Personally, I don't think that the visual consistency between website and export matters that much (See Visual Differences in original PR), and I think that we can make it more similar (except for fonts) - but ultimately this is a decision for the maintainers to make.

Agree on this. I think this is fine. Ultimately, my biggest barrier is still the configuration/duplicated logic issues which have been lightly addressed for now.

In this case of using Serverless functions for export, I believe that a cleaner approach to couple the data models is by shifting the export serverless handlers into the main website APIs (to website/api/export/*).

I am agreeable to this since technically having 2 separate Vercel projects doesn't yield much benefit if that can ultimately reduce duplicated code. @ravern @leslieyip02 can weigh in on this too.

# Conflicts:
#	export/package.json
#	export/src/handler.ts
#	export/src/render-serverless.ts
#	pnpm-lock.yaml
@ravern
Copy link
Copy Markdown
Member

ravern commented May 24, 2026

Btw I've started extracting the types in #4346, depending on whose PR merges first, someone will have to resolve the conflicts, but it shouldn't be hard.

Since we've stopped the effort to extract/share types, let's not include that into the scope of this PR.

I also personally don't think that the visual differences don't matter too much.

I do like the effort here in bringing in a new approach to exporting timetables. And I do think there are pros and cons, so why not we merge this in to sit alongside the current export behind a beta, and then see what users think?

Export is pure-function-ish and we can just have both current and beta endpoints without much hassle.

I can make the changes to support both.

@ravern
Copy link
Copy Markdown
Member

ravern commented May 24, 2026

@greptileai review

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 24, 2026

Greptile Summary

This PR replaces the Puppeteer/Chromium-based export pipeline with Satori + resvg-js + PDFKit for server-side timetable rendering, delivering a ~65% latency improvement by eliminating cold-start browser overhead. The new beta endpoints (/api/export/beta/image and /api/export/beta/pdf) are gated behind a beta settings flag in the UI, so production traffic is unaffected during the rollout.

  • A new render-model layer (render-model.ts) transforms ExportData + Module[] into a RenderableTimetable view model, with a matching Satori-compatible JSX tree (view.tsx) that mirrors the timetable layout using inline styles only.
  • The handler.ts is refactored to extract shared error handling into handleExportError and introduce makeExportHandler for Satori functions alongside the renamed makeBrowserExportHandler for the existing Puppeteer path.
  • The ModuleLessonConfig type in export/src/types.ts is updated to match the website's current LessonIndex[] format (not breaking — the website already used this representation), and the ta field is simplified from TaModulesConfig to Array<ModuleCode> to align with website/src/types/export.ts.

Confidence Score: 5/5

Safe to merge: the new Satori pipeline is gated behind the beta flag and production traffic continues through the unchanged Puppeteer path.

All production Vercel functions validate their own inputs correctly via makeExportHandler. The two findings are confined to the local Koa dev server and a type-bypass pattern that is already error-handled at runtime. No data-loss, auth, or correctness issues found in the rendering pipeline.

No files require special attention for the production rollout; the only rough edges are in the local development server (app.ts) and the type casts in the Vercel beta handlers.

Sequence Diagram

sequenceDiagram
    participant Browser
    participant ExportMenu
    participant BetaImageHandler as api/export/beta/image
    participant makeExportHandler
    participant renderImage
    participant renderPdf
    participant getModules
    participant buildRenderableTimetable
    participant satori
    participant resvg
    participant PDFKit

    Browser->>ExportMenu: click beta image/pdf (beta testers only)
    ExportMenu->>ExportMenu: extractStateForExport() → JSON.stringify → URL
    ExportMenu-->>Browser: "href with ?data=..."

    Browser->>BetaImageHandler: "GET /api/export/beta/image?data=..."
    BetaImageHandler->>makeExportHandler: JSON.parse(data) + validateExportData
    makeExportHandler->>renderImage: exportData, options
    renderImage->>getModules: fetch modules from API/filesystem
    getModules-->>renderImage: Module[]
    renderImage->>buildRenderableTimetable: exportData + modules
    buildRenderableTimetable-->>renderImage: RenderableTimetable model
    renderImage->>satori: TimetableImage JSX + fonts
    satori-->>renderImage: SVG string
    renderImage->>resvg: SVG → render().asPng()
    resvg-->>renderImage: PNG Buffer
    renderImage-->>BetaImageHandler: PNG Buffer
    BetaImageHandler-->>Browser: image/png

    Browser->>BetaImageHandler: "GET /api/export/beta/pdf?data=..."
    BetaImageHandler->>makeExportHandler: JSON.parse(data) + validateExportData
    makeExportHandler->>renderPdf: exportData
    renderPdf->>getModules: fetch modules
    getModules-->>renderPdf: Module[]
    renderPdf->>buildRenderableTimetable: exportData + modules
    buildRenderableTimetable-->>renderPdf: RenderableTimetable model
    renderPdf->>satori: TimetableImage JSX + fonts
    satori-->>renderPdf: SVG string
    renderPdf->>resvg: SVG → render().asPng() (2× zoom)
    resvg-->>renderPdf: PNG Buffer
    renderPdf->>PDFKit: image(pngBuffer, x, y, dimensions)
    PDFKit-->>renderPdf: PDF Buffer (via stream)
    renderPdf-->>BetaImageHandler: PDF Buffer
    BetaImageHandler-->>Browser: application/pdf
Loading
Prompt To Fix All With AI
Fix the following 3 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 3
export/api/export/beta/image.ts:14-15
The `as never` cast bypasses TypeScript's knowledge that `request.query.data` is `string | string[] | undefined`. If the caller sends duplicate `?data=` query params, Vercel parses it as `string[]`, and `JSON.parse` silently calls `.toString()` on the array (producing `"item1,item2"`) which almost certainly fails to parse and throws a 422 — but the failure reason is opaque. A `as string` cast makes intent explicit and lets TypeScript warn if the type ever narrows differently.

```suggestion
    if (typeof request.query.data !== 'string') {
      throw new Error('Expected data to be a single string query parameter');
    }
    const exportData = JSON.parse(request.query.data);
    validateExportData(exportData);
```

### Issue 2 of 3
export/api/export/beta/pdf.ts:8-9
Same `as never` cast as the image handler — same concern applies. If `request.query.data` arrives as a `string[]` (duplicate query params), `JSON.parse` will silently mangle the value. Prefer an explicit type-check before parsing.

```suggestion
    if (typeof request.query.data !== 'string') {
      throw new Error('Expected data to be a single string query parameter');
    }
    const exportData = JSON.parse(request.query.data);
    validateExportData(exportData);
```

### Issue 3 of 3
export/src/app.ts:35-50
**Beta routes silently pass `undefined` data in the dev server**

The `parseExportData` middleware guards against a missing `?data` param only for paths matching `EXPORT_ROUTES` (`/api/export/image` and `/api/export/pdf`). The new `/beta/image` and `/beta/pdf` paths don't match that regex, so when the dev server receives a request without `?data`, `ctx.state.data` stays `undefined`. Both handlers immediately destructure it (`const { data } = ctx.state`) and pass it to `renderImage`/`renderPdf`, which crashes with a `TypeError: Cannot read properties of undefined (reading 'timetable')` rather than returning a clean 422. This only affects the local Koa dev server — the Vercel production functions validate inside `makeExportHandler` — but it makes debugging difficult during development.

Reviews (2): Last reviewed commit: "fix(export): address satori beta review ..." | Re-trigger Greptile

Comment thread export/src/satori/render-model.ts Outdated
Comment thread export/src/render-pdf.ts Outdated
Comment thread export/src/satori/render-satori.tsx Outdated
@ravern
Copy link
Copy Markdown
Member

ravern commented May 24, 2026

@greptileai review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants