Skip to content

slow serialization for some cases (happens in Node.js 10) #3

Closed
@kaesonho

Description

@kaesonho

Hi @ericf as we discussed, there might be some case to cause a GC pause,

I'm guessing it's some V8 de-opt we're hitting because the replacer forces V8 to go from native-land to user-land and back. Whereas with JSON.stringify().replace() it can stay in native-land the whole time.
So it's probably causing a serialization/de-serialization of the massive string causing a GC pause.

Now we have data ~500KB to be serialized, and found that some case(typically when the data is complicated and big enough) str.replace takes a very long time after JSON.stringify with replacer

To reproduce it, here's the simple script with the data we have,
We will get rid of it with a gc() or run without replacer, it acts like what you mentioned.
https://drive.google.com/file/d/0B0ZB5_LvayjrRjV0U2hrMGVjMUE/view?usp=sharing
please run node --trace-gc --replacer --issue ./slowserialization.js to reproduce it and see gc solve it with node --trace-gc --expose-gc --replacer --issue --gc ./slowserialization.js

will see when slow serialization happens, there's a lot of sweep and cause time spend.

[41363]     3133 ms: Mark-sweep 331.0 (368.4) -> 327.2 (366.4) MB, 123 ms [last resort gc].
[41363]     3249 ms: Mark-sweep 327.2 (366.4) -> 327.2 (364.4) MB, 116 ms [last resort gc].
[41363]     3361 ms: Mark-sweep 327.2 (364.4) -> 327.2 (364.4) MB, 112 ms [last resort gc].
[41363]     3475 ms: Mark-sweep 327.2 (364.4) -> 327.2 (364.4) MB, 113 ms [last resort gc].
[41363]     3578 ms: Mark-sweep 327.2 (364.4) -> 327.2 (364.4) MB, 103 ms [last resort gc].
[41363]     3676 ms: Mark-sweep 327.2 (364.4) -> 327.2 (364.4) MB, 97 ms [last resort gc].
[41363]     3776 ms: Mark-sweep 327.2 (364.4) -> 327.2 (364.4) MB, 100 ms [last resort gc].

The slow serialization only happens with Node.js 10, 12 won't have that issue.

I don't think that makes sense to fix this only for Node.js 10, but let's discuss here, like can we try something else for handling the function/regexp or make that optional (for our main customer's case, we don't have function and regexp)?

@redonkulus @lingyan @mridgway @Vijar

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions