Skip to content

Reduce memory usage during index construction. #8834

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 2 commits into from
Jun 27, 2025
Merged

Reduce memory usage during index construction. #8834

merged 2 commits into from
Jun 27, 2025

Conversation

isoos
Copy link
Collaborator

@isoos isoos commented Jun 25, 2025 •
edited
Loading

  • The removal of _documentsByName (little use) and _topics (unused) is mostly symbolic, there isn't much memory overhead there.
  • The change in API doc page index construction (using separate strings instead of concatenating and the splitting the same string) reduces index building time, but more importantly reduces memory usage by several hundred megabytes.

@isoos isoos requested review from szakarias and sigurdm June 25, 2025 14:03
@isoos isoos changed the title Expose memory allocations in search benchmarks. @isoos Reduce memory usage during index construction. Jun 27, 2025
@isoos isoos changed the title @isoos Reduce memory usage during index construction. Reduce memory usage during index construction. Jun 27, 2025
@isoos isoos merged commit f4cf87f into dart-lang:master Jun 27, 2025
31 checks passed
@isoos isoos deleted the bench-mem branch June 27, 2025 09:45
@isoos
Copy link
Collaborator Author

isoos commented Jun 27, 2025

Note: I've confirmed the memory effect on staging too (it has a newer search snapshot).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sigurdm sigurdm sigurdm approved these changes

@szakarias szakarias Awaiting requested review from szakarias

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@isoos @sigurdm