[server] incremental workspaces #14461

svenefftinge · 2022-11-07T01:13:48Z

Description

Introduces a project setting that allows for starting workspaces on the successful prebuild on the direct linear commit history. Enabling this will always skip the "prebuild-in-progress" view.
It also allows specifying the number of commits to skip in between prebuilds.

Related Issue(s)

Fixes #12583

How to test

create a team & project
go to the settings and enable 'incremental prebuilds' set the number of commits to 1
start a workspace.
push a commit (no config change!) and verify that no prebuild is running
start a workspace and verify that it is based on the first prebuild
push a commit (no config change!) and verify that a prebuild gets triggered
push a commit that changes the init task in the gitpod.yml and verify that a prebuild gets triggered

Release Notes

Introduced a new project setting that allows starting workspaces based on the last successful prebuild in the git commit history.

Documentation

Werft options:

/werft with-local-preview
If enabled this will build install/preview
/werft with-preview
/werft with-large-vm
/werft with-integration-tests=all
Valid options are all, workspace, webapp, ide

werft-gitpod-dev-com · 2022-11-07T01:14:10Z

started the job as gitpod-build-se-inc-ws2.1 because the annotations in the pull request description changed
(with .werft/ from main)

werft-gitpod-dev-com · 2022-11-07T01:14:12Z

started the job as gitpod-build-se-inc-ws2.2 because the annotations in the pull request description changed
(with .werft/ from main)

AlexTugarev · 2022-11-07T10:49:59Z

This issue is related: #14237

I would make sense to decouple receiving push events from actually triggering a prebuild. We could introduce a control loop (on quorum leader) to go through the recorded webhook events and then call the prebuild manager periodically.

svenefftinge · 2022-11-07T10:56:47Z

This issue is related: #14237

I think we can and should tackle this issue separately, to avoid making this PR bigger than it needs to be.

gtsiolis · 2022-11-07T16:41:07Z

Looking at this now! 👀

gtsiolis

Thanks for reaching out, @svenefftinge! Left some comments below!

components/dashboard/src/projects/ProjectSettings.tsx

gtsiolis · 2022-11-07T17:17:24Z

components/dashboard/src/projects/ProjectSettings.tsx

+            </div>
+            <div className="flex mt-4 max-w-2xl">
+                <div className="flex flex-col ml-6">
+                    <input


suggestion: This could also benefit from a button to update the number here, as we do with dotfiles.

svenefftinge · 2022-11-08T06:35:33Z

/werft run with-clean-slate-deployment

👍 started the job as gitpod-build-se-inc-ws2.10
(with .werft/ from main)

jankeromnes

Awesome work, many thanks @svenefftinge! 💯

I read through the code changes twice, and tested various cases, and everything looks good so far, so approving to unblock this. 🚢

(I feel like I'd need to spend more time to fully understand and test all the implications of the workspace creation mode changes, but this provides sufficient value and seems low-risk enough that we can also go ahead with this, and we can adjust and look for bugs async / or worst case, revert at the first sign of trouble 😋)

Let's do this! 🏀

jankeromnes · 2022-11-08T08:08:40Z

components/dashboard/src/projects/ProjectSettings.tsx

+                        // we are disabling prebuild cancellation when incremental workspaces are enabled
+                        keepOutdatedPrebuildsRunning: target.checked || project?.settings?.keepOutdatedPrebuildsRunning,


I feel like we'll eventually want to remove this feature. It's not ideal / has too many unfortunate side effects.

jankeromnes · 2022-11-08T08:14:51Z

components/dashboard/src/projects/ProjectSettings.tsx

+                    <div className="text-gray-500 dark:text-gray-400 text-sm mt-2">
+                        The number of commits that are skipped between prebuilds.
+                    </div>


We'll likely iterate on this feature and the UX more later, but I wonder: Why bundle these two features together? (I.e. "run prebuilds every N commits" and "always skip prebuild in progress").

I feel like these should be two separate features, and I'm not 100% sure we can do a good job explaining to users exactly what this "number of skipped commits" does and how to use it properly for their project.

jankeromnes · 2022-11-08T08:23:58Z

components/server/ee/src/prebuilds/bitbucket-app.ts

+            if (!ws.done) {
+                await this.webhookEvents.updateEvent(event.id, {
+                    prebuildStatus: "prebuild_triggered",
+                    status: "processed",
+                    prebuildId: ws.prebuildId,
+                });
+                return ws;
+            }


I'm not sure I fully understand this change. What does it mean when ws.done is true? And does it always mean that we did not in fact trigger a prebuild? 💭

roboquat added do-not-merge/work-in-progress do-not-merge/release-note-label-needed size/XL labels Nov 7, 2022

svenefftinge force-pushed the se/inc-ws2 branch from 43440fd to 4a8557a Compare November 7, 2022 03:20

roboquat added size/XXL release-note-none and removed size/XL do-not-merge/release-note-label-needed labels Nov 7, 2022

svenefftinge marked this pull request as ready for review November 7, 2022 03:27

svenefftinge requested a review from a team November 7, 2022 03:27

roboquat removed the do-not-merge/work-in-progress label Nov 7, 2022

github-actions bot added the team: webapp Issue belongs to the WebApp team label Nov 7, 2022

svenefftinge changed the title ~~Skip prebuilds for incremental workspaces~~ [server] incremental workspaces (2 of 2) - Skip prebuilds Nov 7, 2022

svenefftinge mentioned this pull request Nov 7, 2022

[server] incremental workspaces (1 of 2) #14303

Closed

4 tasks

svenefftinge force-pushed the se/inc-ws2 branch from 4a8557a to 5d7094a Compare November 7, 2022 15:38

roboquat added size/XL and removed size/XXL labels Nov 7, 2022

svenefftinge force-pushed the se/inc-ws2 branch from 5d7094a to d49aea0 Compare November 7, 2022 15:56

gtsiolis reviewed Nov 7, 2022

View reviewed changes

svenefftinge force-pushed the se/inc-ws2 branch from d49aea0 to fa6fb9d Compare November 7, 2022 18:04

roboquat added release-note and removed release-note-none labels Nov 7, 2022

svenefftinge marked this pull request as draft November 7, 2022 18:11

roboquat added the do-not-merge/work-in-progress label Nov 7, 2022

svenefftinge marked this pull request as ready for review November 7, 2022 18:11

roboquat removed the do-not-merge/work-in-progress label Nov 7, 2022

svenefftinge changed the title ~~[server] incremental workspaces (2 of 2) - Skip prebuilds~~ [server] incremental workspaces Nov 7, 2022

svenefftinge force-pushed the se/inc-ws2 branch from fa6fb9d to ffdad13 Compare November 7, 2022 20:27

Incremental workspaces

ea82c11

svenefftinge force-pushed the se/inc-ws2 branch from ffdad13 to f0793ee Compare November 7, 2022 20:27

[inc ws] skip prebuilding

09152d3

svenefftinge force-pushed the se/inc-ws2 branch from f0793ee to 09152d3 Compare November 8, 2022 06:21

jankeromnes approved these changes Nov 8, 2022

View reviewed changes

roboquat merged commit c18efbd into main Nov 8, 2022

roboquat deleted the se/inc-ws2 branch November 8, 2022 08:33

roboquat added deployed: webapp Meta team change is running in production deployed Change is completely running in production labels Nov 8, 2022

geropl mentioned this pull request Nov 8, 2022

Prebuild runs only when opening new workspace #9564

Closed

AlexTugarev mentioned this pull request Nov 16, 2022

[prebuilds] workspace starts with derived config, wrong image, and no prebuild #14726

Closed

filiptronicek mentioned this pull request Nov 12, 2024

[server] Add more tests around "prebuild selection" #20358

Merged

15 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[server] incremental workspaces #14461

[server] incremental workspaces #14461

Uh oh!

svenefftinge commented Nov 7, 2022 •

edited

Loading

Uh oh!

werft-gitpod-dev-com bot commented Nov 7, 2022

Uh oh!

werft-gitpod-dev-com bot commented Nov 7, 2022

Uh oh!

AlexTugarev commented Nov 7, 2022

Uh oh!

svenefftinge commented Nov 7, 2022

Uh oh!

gtsiolis commented Nov 7, 2022 •

edited by werft-gitpod-dev-com bot

Loading

Uh oh!

gtsiolis left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gtsiolis Nov 7, 2022

Uh oh!

svenefftinge commented Nov 8, 2022 •

edited by werft-gitpod-dev-com bot

Loading

Uh oh!

jankeromnes left a comment •

edited

Loading

Uh oh!

jankeromnes Nov 8, 2022

Uh oh!

jankeromnes Nov 8, 2022

Uh oh!

jankeromnes Nov 8, 2022

Uh oh!

Uh oh!

		// we are disabling prebuild cancellation when incremental workspaces are enabled
		keepOutdatedPrebuildsRunning: target.checked \|\| project?.settings?.keepOutdatedPrebuildsRunning,

[server] incremental workspaces #14461

[server] incremental workspaces #14461

Uh oh!

Conversation

svenefftinge commented Nov 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related Issue(s)

How to test

Release Notes

Documentation

Werft options:

Uh oh!

werft-gitpod-dev-com bot commented Nov 7, 2022

Uh oh!

werft-gitpod-dev-com bot commented Nov 7, 2022

Uh oh!

AlexTugarev commented Nov 7, 2022

Uh oh!

svenefftinge commented Nov 7, 2022

Uh oh!

gtsiolis commented Nov 7, 2022 • edited by werft-gitpod-dev-com bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gtsiolis left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

gtsiolis Nov 7, 2022

Choose a reason for hiding this comment

Uh oh!

svenefftinge commented Nov 8, 2022 • edited by werft-gitpod-dev-com bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jankeromnes left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jankeromnes Nov 8, 2022

Choose a reason for hiding this comment

Uh oh!

jankeromnes Nov 8, 2022

Choose a reason for hiding this comment

Uh oh!

jankeromnes Nov 8, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

svenefftinge commented Nov 7, 2022 •

edited

Loading

gtsiolis commented Nov 7, 2022 •

edited by werft-gitpod-dev-com bot

Loading

svenefftinge commented Nov 8, 2022 •

edited by werft-gitpod-dev-com bot

Loading

jankeromnes left a comment •

edited

Loading