[Benchmarks] Update microbenchmarks (#6868)

## Summary of changes
This PR updates the microbenchmarks:
- The microbenchmarks now run a new fixed version of the tooling
- Benchmarks run on every commit and publish results only on PRs
- When a job is cancelled, the spawned infrastructure is deleted
(otherwise it hangs and creates extra cost before scheduled cleanup),
[example of cancelled job with destruction of spawned
infrastructure](https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-dotnet/-/jobs/901198114)
- Uses a more conventional naming for branch
## Reason for change
- Noticed some extra unplanned costs.
- Need to update tooling version
## Implementation details

## Test coverage

## Other details
<!-- Fixes #{issue} -->

<!-- ⚠️ Note: where possible, please obtain 2 approvals prior to
merging. Unless CODEOWNERS specifies otherwise, for external teams it is
typically best to have one review from a team member, and one review
from apm-dotnet. Trivial changes do not require 2 reviews. -->

Author: faydef

.gitlab-ci.yml

@@ -220,10 +220,4 @@ microbenchmarks:
   trigger:
     include: .gitlab/benchmarks/microbenchmarks.yml
   allow_failure: true
-  rules:
-    - if: '$CI_PIPELINE_SOURCE == "push" && $CI_COMMIT_BRANCH == "master"'
-      when: always
-    - if: '$CI_PIPELINE_SOURCE == "schedule" && $BENCHMARK_RUN == "true"'
-      when: always
-    - when: manual
 

.gitlab/benchmarks/microbenchmarks.yml

@@ -1,3 +1,11 @@
+.setup:
+  script:
+    - mkdir -p ~/.aws
+    - /app/infra/tools/fetch-ssm-parameter.sh $AWS_EPHEMERAL_INFRA_PROFILE_SSM_PARAMETER > ~/.aws/config || exit $?
+    - export AWS_PROFILE=ephemeral-infra-ci
+    - export BP_INFRA_KEY_PAIR_NAME=$(cat ~/.aws/key-pair-name.txt)
+    - export BP_INFRA_KEY_PAIR_PRIVATE_KEY_PATH=~/.aws/key-pair-private-key.pem
+
 stages:
   - benchmarks
 
@@ -7,11 +15,17 @@ run-benchmarks:
   interruptible: true
   timeout: 1h
   # Image created in the following job https://gitlab.ddbuild.io/DataDog/benchmarking-platform-tools/-/jobs/869830045
-  image: registry.ddbuild.io/images/benchmarking-platform-tools-ubuntu:dotnet-microbenchmarks
+  image: registry.ddbuild.io/images/benchmarking-platform-tools-ubuntu:dotnet-microbenchmarks-2
 
   script:
-    - git clone --branch fayssal/test-micro-delivery https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.ddbuild.io/DataDog/benchmarking-platform platform
+    - git clone --branch dd-trace-dotnet/micro https://gitlab-ci-token:${CI_JOB_TOKEN}@gitlab.ddbuild.io/DataDog/benchmarking-platform platform
     - ./platform/steps/launch-instance.sh
+  after_script:
+    - !reference [.setup, script]
+    - |
+      infra cleanup --provision ./platform/ephemeral-infra/ephemeral-instance-main.yaml \
+        --region "${AWS_REGION}" \
+        --bypass-stack-destroy
     - ./platform/steps/post-pr-comment.sh
     # Temporarely commented out pending issue resolution with sending files to backend
     # - ./platform/steps/upload-to-bp-ui.sh