[UI] - introduce Posthog for Vault Dedicated managed clusters #30425

emoncuso · 2025-04-28T19:33:05Z

Description

This PR enables basic analytics tracking in the Web UI. We're using posthog as a provider, but only sending limited information back to the data warehouse. As written, this feature is only enabled on HVD managed clusters.

Implementation details

The activation of this feature is handled by two variables: (ember build) environment and the isHvdManaged feature flag.

This PR adds a new ember service (analytics) to the app
- the service attempts to start itself on application load (in the route:application)
- the service will attempt identify the user with an identifier once a user logs in
- the service will add an event listener for route transitions
The service is an abstraction on top of the PostHog provider itself. If needed, the provider should be swappable with no impact on the in-code implementation
We're filtering out many of the identifying fields posthog collects by default. It's goofy code, but the safer option for not collecting things we don't want. We're specifically filtering our geolocation and urls, but see the specifics here
This includes an update to the vault-reporting addon to track custom events. These code changes were already reviewed in the shared-secure-ui repo so reviewing the generated dist files is not reccomended. If you are curious about the changes on that side the PR can be found here

From an end user perspective, this app is always run in production mode. Dev mode is purely for development purposes

How to test

run the app locally using ENABLE_POSTHOG=true yarn start, this will run the app in development mode with an explicit call to send events to posthog
- This is an HVD only feature (and thus enterprise only), but the functionality should work on community edition as well. For our testing, run against an ent vault backend.
open the web app
open the web console
do some stuff!
- identify - the service will send events even if you're not logged in. So try logging in with your root token or another user. This activity should be visible in the console.
- trackPageView - we're collecting pageview information on every route transition. Navigating to any page will trigger this event. You can see the intended routename in the console
  - NOTE: we're not collecting urls as those can contain sensitive information on a user's Vault setup. This is intentional
- trackEvent - custom events are triggered by user interactions. As an example, an event has been added to the REPL toggle button in the sidebar. Try toggling the REPL and note the message in the console.
You should also see posthog events being sent over the network, this is the final confirmation that things are workin!

Enterprise tests pass? ✅

TODO only if you're a HashiCorp employee

Backport Labels: If this fix needs to be backported, use the appropriate backport/ label that matches the desired release branch. Note that in the CE repo, the latest release branch will look like backport/x.x.x, but older release branches will be backport/ent/x.x.x+ent.
- LTS: If this fixes a critical security vulnerability or severity 1 bug, it will also need to be backported to the current LTS versions of Vault. To ensure this, use all available enterprise labels.
ENT Breakage: If this PR either 1) removes a public function OR 2) changes the signature
of a public function, even if that change is in a CE file, double check that
applying the patch for this PR to the ENT repo and running tests doesn't
break any tests. Sometimes ENT only tests rely on public functions in CE
files.
Jira: If this change has an associated Jira, it's referenced either
in the PR description, commit message, or branch name.
RFC: If this change has an associated RFC, please link it in the description.
ENT PR: If this change has an associated ENT PR, please link it in the
description. Also, make sure the changelog is in this PR, not in your ENT PR.

github-actions · 2025-04-28T19:51:48Z

CI Results:
All Go tests succeeded! ✅

github-actions · 2025-05-22T18:47:03Z

Build Results:
All builds succeeded! ✅

emoncuso · 2025-05-22T19:21:54Z

ui/app/components/sidebar/frame.js

+  trackReplToggle = () => {
+    this.analytics.trackEvent(TOGGLE_WEB_REPL);
+  };


This is an example of how to track custom events in the code.

We could also do something like {{on "click" (fn this.analytics.trackEvent MY_EVENT)}} in the template, but I think the component file is a little cleaner.

This is neat! Is the long-term plan to figure out where we want to add these?

Yeah! The plan is add event tracking to the toolkit for new features as we release them. It's something we can discuss with product and design as well. "How do we know this feature is successful?" can be answered with the data coming back from this work.

The first thing we're aiming to track is the reporting work, but all new features (and probably many old one) can be added as devs see fit!

emoncuso · 2025-05-22T19:25:13Z

ui/app/routes/vault/cluster.js

+        const token = await this.api.auth.tokenLookUpSelf();
+        const entity = token.data.entity_id ? token.data.entity_id : `root_${crypto.randomUUID()}`;
+
+        if (model.license?.state) {


License is only available to for ent customers, so do some guarding in case this somehow gets run in CE mode

Is state just an empty string for CE cluster?

The whole model.license is undefined. Which makes sense

Ah okay, I was wondering if this should be mode?.license?.state then

hellobontempo

This is neat! Excited to see the data from it. My main concern is with crypto.randomUUID do to historical issues with it. The other suggestions are minor - like avoiding making an extra network request to lookup token data since we already should have that info after a user is auth'd.

Most everything else is a question because I'm trying to understand what's going on 😂

hellobontempo · 2025-05-22T22:22:44Z

ui/app/utils/analytics-providers/dummy.ts

+ */
+
+/*
+  Normally, the default export of this file would _do something_. In our case, the mthods for the provider simply log the 


It looks like this comment was cut off - you left me on the edge of my seat! 🪑

Whoops! Fixed

hellobontempo · 2025-05-22T22:33:29Z

ui/app/utils/analytics-providers/README.md

Is there a plan to add a readme?

There was! Sorry I've been working on this PR over the course of like two months and I've definitely just forgotten some things. I'll remove this file for now and add real docs soon!

No worries at all! I'm just genuinely curious 😄

hellobontempo · 2025-05-22T22:34:03Z

ui/app/components/sidebar/frame.js

+  trackReplToggle = () => {
+    this.analytics.trackEvent(TOGGLE_WEB_REPL);
+  };


This is neat! Is the long-term plan to figure out where we want to add these?

hellobontempo · 2025-05-22T22:43:04Z

ui/app/routes/vault/cluster.js

+      try {
+        let license;
+        const token = await this.api.auth.tokenLookUpSelf();
+        const entity = token.data.entity_id ? token.data.entity_id : `root_${crypto.randomUUID()}`;


A while ago there was a bug (linked to this PR: #19410) because crypto.randomUUID() requires secure context and so the UI broke for users accessing it in an insecure context. There was a whole back and forth about enforcing a secure context...because yes - Vault is a secure product 😅 . But essentially using uuid4() (example) circumvents this issue all together.

There was a whole back and forth about enforcing a secure context...because yes - Vault is a secure product 😅

Haha I have questions. But that's an easy swap!

hellobontempo · 2025-05-22T22:46:31Z

ui/app/routes/vault/cluster.js

+        const token = await this.api.auth.tokenLookUpSelf();
+        const entity = token.data.entity_id ? token.data.entity_id : `root_${crypto.randomUUID()}`;


Would you consider getting this from the auth service instead and save making another request?
this.auth.authData.entity_id?

hellobontempo · 2025-05-22T22:53:40Z

ui/config/content-security-policy.js

+    'form-action': ["'none'"],
+  };
+
+  policy['connect-src'].push('https://eu.i.posthog.com');


So does this happen regardless of whether analytics are enabled or not?

Also - what is this doing exactly? 😅

It is enabled regardless. It's letting things connect to that posthog url from the web app.

Since the analytics client is inactive when running on HVD, this will be fine.

hellobontempo · 2025-05-22T22:55:10Z

ui/config/environment.js

+      ENV.APP.ANALYTICS_CONFIG = {
+        provider: 'posthog',
+        enabled: true,
+        project_id: 'phc_zPQ9fPlFj4ZTYKJmThG1C8AE4J4RgPQx8dJJ7agg4SG',


Is this id sensitive at all or something that we can have exposed like this? Sorry - no idea how this works 😂

Nope! These are the client side ids, so okay to have exposed/hardcoded.

hellobontempo · 2025-05-22T23:03:00Z

ui/tests/unit/services/analytics-test.js

+  });
+
+  module('#start', function () {
+    skip('#start does nothing when no provider is given', function (assert) {


is there a plan to un-skip these at some point?

I wanted to test the start mechanism a differently than I got time for. I'll remove these for now and come back to them after code freeze

hellobontempo · 2025-05-22T23:03:22Z

ui/tests/unit/services/analytics-test.js

+
+    this.service.identifyUser(identifier, traits);
+
+    assert.true(providerStub.identify.calledOnce, 'the service calls idenfiy on the provider');


Suggested change

assert.true(providerStub.identify.calledOnce, 'the service calls idenfiy on the provider');

assert.true(providerStub.identify.calledOnce, 'the service calls identfiy on the provider');

hellobontempo · 2025-05-22T23:04:09Z

ui/tests/unit/services/analytics-test.js

+
+    test('logging is not shown when inactive', function (assert) {
+      this.service.debug = false;
+      // for the next few lines, console.log WILL NOT WORK AS EXPECTED


😂 appreciate this comment

lane-wetmore · 2025-05-23T16:27:38Z

ui/app/utils/analytics-providers/posthog.ts

+  if (cr === null) return cr;
+  const { properties } = cr;
+
+  // extract ONLY what we need


Just curious, would there ever be potential for Posthog adding a new field that they track that we could miss here? Or would that be gated behind an explicit upgrade of some sort?

Good question! My goal was to avoid that, so that's why I went for the "opt in" approach vs deleting the pieces we didn't want.

I destructured the properties we want from the original cr.properties object, then rebuilt it a couple lines down without the other data points

* add dummy provider and wire it into the app * add tests for analytics service * add posthog provider * wire in posthog * add HVD limitation for analytics and add unit test * filter out sensitive event properties * add changelog * run copywrite headers * update logging tests for analytics service * update changelog format * disable telemetry in test mode * remove unnecessary test * self review * Update vault-reporting addon with analytics tracking changes * address review feedback --------- Co-authored-by: Jim Wright <[email protected]>

github-actions bot added the hashicorp-contributed-pr If the PR is HashiCorp (i.e. not-community) contributed label Apr 28, 2025

emoncuso added 6 commits May 21, 2025 07:30

add dummy provider and wire it into the app

782f742

add tests for analytics service

2089a05

add posthog provider

112e1ff

wire in posthog

088502c

add HVD limitation for analytics and add unit test

13838f3

filter out sensitive event properties

58ca822

emoncuso force-pushed the feat/posthog branch from 33a4fa1 to 58ca822 Compare May 22, 2025 04:27

emoncuso added this to the 1.20.0-rc milestone May 22, 2025

emoncuso added 4 commits May 21, 2025 21:43

add changelog

9b1e4b5

run copywrite headers

7058bc9

update logging tests for analytics service

a5a5b75

update changelog format

86b1096

emoncuso added the ui label May 22, 2025

disable telemetry in test mode

b39f643

emoncuso marked this pull request as ready for review May 22, 2025 18:42

emoncuso requested review from a team as code owners May 22, 2025 18:42

emoncuso requested a review from j--w May 22, 2025 18:42

Merge branch 'main' into feat/posthog

355d029

vercel bot deployed to Preview May 22, 2025 18:50 View deployment

emoncuso changed the title ~~WIP - Feat/posthog~~ [UI] - introduce Posthog for HVD May 22, 2025

emoncuso changed the title ~~[UI] - introduce Posthog for HVD~~ [UI] - introduce Posthog for Vault Dedicated managed clusters May 22, 2025

remove unnecessary test

b6ab5af

emoncuso commented May 22, 2025

View reviewed changes

emoncuso and others added 2 commits May 22, 2025 12:27

self review

065f964

Update vault-reporting addon with analytics tracking changes

d4a8081

hellobontempo reviewed May 22, 2025

View reviewed changes

address review feedback

1b750b6

lane-wetmore approved these changes May 23, 2025

View reviewed changes

emoncuso enabled auto-merge (squash) May 23, 2025 16:38

Merge branch 'main' into feat/posthog

d2a4786

vercel bot deployed to Preview May 23, 2025 16:45 View deployment

emoncuso added 5 commits May 23, 2025 09:46

Merge branch 'main' into feat/posthog

9ca8d5b

Merge branch 'main' into feat/posthog

522bb9e

Merge branch 'main' into feat/posthog

377d125

Merge branch 'main' into feat/posthog

aee95f8

Merge branch 'main' into feat/posthog

af44a21

emoncuso disabled auto-merge May 23, 2025 18:27

emoncuso enabled auto-merge (squash) May 23, 2025 18:27

emoncuso and others added 2 commits May 23, 2025 11:54

Intentionally empty - bumping GHA

4ef5352

Merge branch 'main' into feat/posthog

a881c84

vercel bot deployed to Preview May 23, 2025 19:22 View deployment

emoncuso merged commit 689ede2 into main May 23, 2025
33 of 34 checks passed

emoncuso deleted the feat/posthog branch May 23, 2025 19:40

Monkeychip mentioned this pull request Jul 23, 2025

UI Bug Fix: Skip analytics init on DR secondary clusters due to async breaking the DR transition flow #31359

Draft

2 tasks

		const token = await this.api.auth.tokenLookUpSelf();
		const entity = token.data.entity_id ? token.data.entity_id : `root_${crypto.randomUUID()}`;


		this.service.identifyUser(identifier, traits);

		assert.true(providerStub.identify.calledOnce, 'the service calls idenfiy on the provider');

[UI] - introduce Posthog for Vault Dedicated managed clusters #30425

[UI] - introduce Posthog for Vault Dedicated managed clusters #30425

Uh oh!

Conversation

emoncuso commented Apr 28, 2025 • edited by j--w Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Implementation details

How to test

TODO only if you're a HashiCorp employee

Uh oh!

github-actions bot commented Apr 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented May 22, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

hellobontempo left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

emoncuso commented Apr 28, 2025 •

edited by j--w

Loading

github-actions bot commented Apr 28, 2025 •

edited

Loading