Use an array for building up mappings in the sourcemap generator. #43785
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
typescript-bot
added
the
For Uncommitted Bug
|
@typescript-bot perf test this faster |
|
Heya @DanielRosenwasser, I've started to run the perf test suite on this PR at 4c86404. You can monitor the build here. |
|
Heya @DanielRosenwasser, I've started to run the abridged perf test suite on this PR at 4c86404. You can monitor the build here. Update: The results are in! |
|
@DanielRosenwasser Here they are:Comparison Report - master..43785
System
Hosts
Scenarios
Developer Information: |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
@typescript-bot perf test this |
|
Heya @DanielRosenwasser, I've started to run the perf test suite on this PR at 312ce16. You can monitor the build here. Update: The results are in! |
|
Heya @DanielRosenwasser, I've started to run the tarball bundle task on this PR at 312ce16. You can monitor the build here. |
|
@DanielRosenwasser Here they are:Comparison Report - master..43785
System
Hosts
Scenarios
Developer Information: |
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
@typescript-bot perf test this |
|
Heya @DanielRosenwasser, I've started to run the perf test suite on this PR at 799bbbe. You can monitor the build here. |
|
Heya @DanielRosenwasser, I've started to run the tarball bundle task on this PR at 799bbbe. You can monitor the build here. |
dmichon-msft
reviewed
May 6, 2021
| lastNameIndex = pendingNameIndex; | ||
| } | ||
| } | ||
|
|
||
| mappings = mappingSegmentsBuffer.join(""); |
There was a problem hiding this comment.
In local experimentation, I've found that I can get about a 40% speed up by batching the char code conversion:
function base64VLQFormatEncode2(value) {
if (value < 0) {
value = ((-value) << 1) + 1;
}
else {
value = value << 1;
}
// Encode 5 bits at a time starting from least significant bits
const result = [];
do {
let currentDigit = value & 31; // 11111
value = value >> 5;
if (value > 0) {
// There are still more digits to decode, set the msb (6th bit)
currentDigit = currentDigit | 32;
}
result.push(base64FormatEncode(currentDigit));
} while (value > 0);
return result;
}
function buildMappingStr() {
const charCodes = [];
// Mock representation of adding a segment, for benchmarking
for (let i = 0; i < target; i ++) {
charCodes.push(...base64VLQFormatEncode2(i));
}
let str = '';
const segmentLength = 4096;
for (let i = 0, len = charCodes.length; i < len; i += segmentLength) {
const segment = String.fromCharCode.apply(null, charCodes.slice(i, i + segmentLength));
str += segment;
}
return str;
}
There was a problem hiding this comment.
If you are encoding a very large number of mappings (>100000), it gets faster to use an encoder with its own buffer:
class SourceMapEncoder {
constructor() {
this._encoded = '';
this._buffer = new Uint8Array(1024);
this._bufferPos = 0;
}
get encoded() {
if (this._bufferPos > 0) {
this._commit();
}
return this._encoded;
}
/**
* @param {number} code
*/
appendCharCode(code) {
this._buffer[this._bufferPos] = code;
this._bufferPos++;
if (this._bufferPos === this._buffer.length) {
this._commit();
}
}
/**
* @param {number} value
*/
appendBase64VLQ(value) {
if (value < 0) {
value = ((-value) << 1) + 1;
}
else {
value = value << 1;
}
// Encode 5 bits at a time starting from least significant bits
do {
let currentDigit = value & 31; // 11111
value = value >> 5;
if (value > 0) {
// There are still more digits to decode, set the msb (6th bit)
currentDigit = currentDigit | 32;
}
this.appendCharCode(formatArr[currentDigit]);
} while (value > 0);
}
reset() {
this._encoded = '';
this._bufferPos = 0;
}
_commit() {
this._encoded += String.fromCharCode.apply(null, this._buffer.slice(0, this._bufferPos));
this._bufferPos = 0;
}
}
Call .appendBase64VLQ(value) for each mapping to append, then retrieve .encoded at the end.
Still trying to figure out why the overhead is so much higher for small counts
There was a problem hiding this comment.
I actually tried something in the same spirit with a Uint8Array - check out #43987
3 tasks
|
To help with PR housekeeping, I'm going to close this draft PR. It is pretty old now. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.