Add Invocation Id to Module Private Data

[bachuv]

This PR adds support to add Invocation Id information to Module Private Data so we can use it for distributed tracing in Start-DurableOrchestration.

This is a draft PR to discuss the design. It will be updated to get tracing information from the Durable PowerShell OpenTelemetry SDK based on Invocation Id in the Start-DurableOrchestration function.

[bachuv]

Where would I actually call this to set the invocation id? I tried to find how SetDurableClient() is getting called, but didn't find it in this SDK.

I looked at the internal SDK implementation and I see that it gets called in DurableController.cs

_powerShellServices.SetDurableClient(durableClient);

Do I need to update the Functions PowerShell worker?

[AnatoliB]

Yes, I believe you would need to move this method to the PowerShell worker and invoke it from there. In fact, the SetDurableClient and SetOrchestrationContext methods do not make sense in this repo, and they were probably just copied over from the worker repo by mistake.

[AnatoliB]

But wait, this will require us to deploy a PS worker. Perhaps we can find a way to avoid that. Two ideas:

1. Ask the customer to invoke Set-FunctionInvocationContext explicitly from their app code. Make this a part of DT enabling instructions (at least temporary, until we are able to update the worker). Do you know if the invocation id is available in the app code?
2. Perhaps we can make the OTel module invoke Set-FunctionInvocationContext automatically? @andystaples, any thoughts on this?

[AnatoliB]

Have we decided that the invocation id should come from the OTel module? In this case, we can bypass the worker completely by either making the OTel module invoke Set-FunctionInvocationContext directly, or make the Durable SDK module invoke a command from the OTel module, whatever is more convenient. These modules don't have to take a hard dependency on each other, they will just know that cmdlets with certain names may exist, so this will be a dynamic dependency.

[davidmrdavid]

In fact, the SetDurableClient and SetOrchestrationContext methods do not make sense in this repo, and they were probably just copied over from the worker repo by mistake.

Hmm, I seem to recall there was a reason - I recall Michael Peng telling me about this class at some point, but to be frank the reasons escape me now. My suggestion - add print statements / obvious failures in this codepath and run a few tests. If indeed it doesn't get called, then yeah just remove this.

[AnatoliB]

Let's be careful with this parameter. Unless we make the customer invoke this cmdlet explicitly, we need to remember that whatever invokes this cmdlet may be dealing with an older SDK version that didn't have this parameter yet. So the attempt to invoke it will fail, and we need to make sure that the caller tolerates the error. Nothing special to do in this PR, but please keep this in mind.