Skip to content

Prerender only max version of crates' readmes, lazily render/cache older versions' as requested #1025

@carols10cents

Description

@carols10cents

It takes a long time to rerender all the readmes for all the versions on crates.io :(

I think a way to cut down on this time by a lot is just to render fewer versions upfront, and render the rest on-demand. What I'm envisioning is:

  • Change the readme rendering to be proxied through crates.io's backend rather than served from s3 directly. That way, we can check to see if it exists on s3 first, and try and render and serve it if it doesn't (hmmmm we don't want to keep trying to render readmes that are missing over and over, hmmmm)

Once the proxying/request rendering if it isn't on s3 yet, the render-readmes script should:

  • Remove all rendered readmes saved on s3
  • For each crate, render and save to s3 the readme for the latest version semvertically (that is, whatever version shows up when you go to crates.io/crates/cratename, not necessarily the most recently published version)

As new versions are published, the publish process will render their readme to s3. The next time we need to rerender readmes, old versions will get cleared out.

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-readmeC-internal 🔧Category: Nonessential work that would make the codebase more consistent or clear

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions