[cmd/opampsupervisor] chore: Store RemoteConfigStatus in persistent state

[dpaasman00]

Description

• The last reported RemoteConfigStatus should be stored in the persistent state so that on startup we can more accurately report this. Currently it just reports the hash of the remote config that's on disk and an "Applied" status which may not always be accurate.
• When we opt to not run the collector because we have an empty config map (performance optimization) we should also be reporting a "Healthy" status since this is what the collector would do if it were ran.
• Small refactor to the "loadAndWriteInitialMergedConfig" function pulling out the own telemetry config generation. Additionally this function is handling loading the remote config since we no longer need to do so on startup with the status being stored in the persistent state.

Link to tracking issue

Fixes

Testing

• Unit tests updated
• e2e updated

Documentation

[evan-bradley]

Currently it just reports the hash of the remote config that's on disk and an "Applied" status which may not always be accurate.

I see we pass the remote config status in the OpAMP client start settings. Is this reported right away without starting the Collector?

[dpaasman00]

Currently it just reports the hash of the remote config that's on disk and an "Applied" status which may not always be accurate.

I see we pass the remote config status in the OpAMP client start settings. Is this reported right away without starting the Collector?

Yep. This remote config status is reported as soon as the OpAMP client connects to the server. This is where it is defined in the OpAMP-Go code: https://github.com/open-telemetry/opamp-go/blob/main/client/types/startsettings.go#L35-L40

[evan-bradley]

Looks good to me. Thanks for the additional info.

[dpaasman00]

Marking as draft for a bit. Found one edge case issue this should fix but doesn't.

[dpaasman00]

This is ready to go again. Found an edge case when getting sent an empty config we'd report an unhealthy status when we don't want to. Updated an e2e test to make sure it's healthy.

[andrzej-stencel]

@evan-bradley can you take one last look after the latest change?

EDIT: I've marked this PR with do-not-merge, @evan-bradley please take the label off and merge when you take a look.

[dpaasman00]

This is ready for another look.