Skip to content

[move][tracing] Compress traces + slight format change #20851

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 11, 2025

Conversation

tzakian
Copy link
Contributor

@tzakian tzakian commented Jan 10, 2025

Description

Adds compression to Move traces to make them smaller. It compresses traces using the zstd compression, as in benchmarking/testing this compression provided the best compression and speed for the generated traces.

This also updates the trace file format to be streamed. Before the trace format was a single json object something like

{
  version: 1,
  events: [
    event1, 
    event2,
    ...,
    event_n,
  ]
}

The new trace format is now a sequence of json objects

{ "version": 2 }
{ event1 }
{ event2 }
...
{ event3 }

This also updates the representation of Move values in the generated traces to be typed.

The base commit is the core Rust changes

The second commit are the changes to the debugger to support the new traces format (I'm a Typescript n00b so feel free to throw things at me here @awelc).

The third commit is just updating snapshot files.

These commits can't really be broken up as they should really be landed all together since they are both dependent on the trace format and output.

And before you ask: yes, yes, delta encoding will be coming soon™️

Test plan

Existing tests + added a couple smaller ones.


Release notes

Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.

For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.

  • Protocol:
  • Nodes (Validators and Full nodes):
  • gRPC:
  • JSON-RPC:
  • GraphQL:
  • CLI: Change in generated Move trace representation to be compressed. Existing Move test traces will need to be regenerated in order to me used.
  • Rust SDK:

@tzakian tzakian requested a review from awelc January 10, 2025 17:21
Copy link

vercel bot commented Jan 10, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
multisig-toolkit ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 11, 2025 8:27pm
sui-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 11, 2025 8:27pm
sui-kiosk ✅ Ready (Inspect) Visit Preview 💬 Add feedback Apr 11, 2025 8:27pm

@tzakian tzakian temporarily deployed to sui-typescript-aws-kms-test-env January 10, 2025 17:21 — with GitHub Actions Inactive
@tzakian tzakian force-pushed the tzakian/compress-traces branch from 86ee5cb to b78f8c5 Compare April 3, 2025 23:05
@tzakian tzakian had a problem deploying to sui-typescript-aws-kms-test-env April 3, 2025 23:05 — with GitHub Actions Failure
@tzakian tzakian requested review from amnn and cgswords April 3, 2025 23:09
@tzakian tzakian force-pushed the tzakian/compress-traces branch from b78f8c5 to 931afdc Compare April 3, 2025 23:21
@tzakian tzakian had a problem deploying to sui-typescript-aws-kms-test-env April 3, 2025 23:21 — with GitHub Actions Failure
@tzakian tzakian marked this pull request as ready for review April 3, 2025 23:22
@tzakian tzakian had a problem deploying to sui-typescript-aws-kms-test-env April 3, 2025 23:22 — with GitHub Actions Failure
@tzakian tzakian force-pushed the tzakian/compress-traces branch from 931afdc to 3aa0a1a Compare April 4, 2025 15:38
@tzakian tzakian had a problem deploying to sui-typescript-aws-kms-test-env April 4, 2025 15:38 — with GitHub Actions Failure
@tzakian tzakian force-pushed the tzakian/compress-traces branch from 3aa0a1a to ed44d69 Compare April 4, 2025 16:45
@tzakian tzakian had a problem deploying to sui-typescript-aws-kms-test-env April 4, 2025 16:45 — with GitHub Actions Failure
@tzakian tzakian force-pushed the tzakian/compress-traces branch from ed44d69 to 64d8649 Compare April 4, 2025 16:48
@tzakian tzakian temporarily deployed to sui-typescript-aws-kms-test-env April 4, 2025 16:48 — with GitHub Actions Inactive
Copy link
Contributor

@awelc awelc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Module some nits, LGTM

@tzakian tzakian force-pushed the tzakian/compress-traces branch from 64d8649 to 6fc75a5 Compare April 11, 2025 20:25
@tzakian tzakian temporarily deployed to sui-typescript-aws-kms-test-env April 11, 2025 20:25 — with GitHub Actions Inactive
@tzakian tzakian enabled auto-merge (squash) April 11, 2025 20:25
@tzakian tzakian merged commit ff4bf05 into main Apr 11, 2025
47 checks passed
@tzakian tzakian deleted the tzakian/compress-traces branch April 11, 2025 20:56
@awelc awelc mentioned this pull request Apr 14, 2025
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants