Generating large JSON (10+ MB) much slower than with Oj

[shnikola]

We've recently removed Oj from our production app and started using JSON. We are now noticing degraded performance in cases where we need we call to_json on very large objects. Here's a benchmark comparing serializing the attached file with JSON and Oj.

print "Testing JSON...\n"
obj = JSON.parse(File.read("large_example.json"))
objs = [obj] * 10

Benchmark.bm do |x|
  x.report { objs.to_json }
end

Output:

       user     system      total        real
   1.046600   0.061810   1.108410 (  1.107597)

print "Testing OJ...\n"
Oj.optimize_rails
obj = JSON.parse(File.read("large_example.json"))
objs = [obj] * 10

Benchmark.bm do |x|
  x.report { objs.to_json }
end

Output:

       user     system      total        real
   0.103054   0.006695   0.109749 (  0.109749)

large_example.json

[byroot]

Since you call Oj.optimize_rails I presume this is in the context of a Rails app with Active Support loaded.

Active Support replaces the to_json method with its own: https://github.com/rails/rails/blob/986d7dfe372430dabd95f71db2e1573b22e2b069/activesupport/lib/active_support/core_ext/object/json.rb#L34-L50

Which adds some overhead because it does some extra escaping: https://github.com/rails/rails/blob/986d7dfe372430dabd95f71db2e1573b22e2b069/activesupport/lib/active_support/json/encoding.rb#L41-L57

You can skip all that by instead using JSON.generate(objs):

require 'benchmark'
require 'json'
require 'oj'
require 'active_support/all'


print "Testing JSON...\n"
obj = JSON.parse(File.read("/tmp/large_example.json"))
objs = [obj] * 10

puts "=== vanilla ==="
Benchmark.bm do |x|
  x.report(".to_json") { objs.to_json }
  x.report("JSON.gen") { JSON.generate(objs) }
  x.report("Oj.dump") { Oj.dump(objs) }
end

puts "=== Oj.optimize_rails ==="
Oj.optimize_rails
Benchmark.bm do |x|
  x.report(".to_json") { objs.to_json }
  x.report("JSON.gen") { JSON.generate(objs) }
  x.report("Oj.dump") { Oj.dump(objs) }
end

Testing JSON...
=== vanilla ===
              user     system      total        real
.to_json  0.148308   0.005058   0.153366 (  0.153877)
JSON.gen  0.069562   0.002631   0.072193 (  0.072254)
Oj.dump   0.063983   0.004442   0.068425 (  0.068964)
=== Oj.optimize_rails ===
              user     system      total        real
.to_json  0.074912   0.003743   0.078655 (  0.079265)
JSON.gen  0.073253   0.003333   0.076586 (  0.076893)
Oj.dump   0.065512   0.003454   0.068966 (  0.071187)

As you can see on the result aboves, it's on par with Oj.optimize_rails.

The Active Support performance is also on my radar (rails/rails#54481 and rails/rails#54484) but it does more so will always be a bit slower.

That being said, on my machine I only see a 2x difference between .to_json and JSON.dump, what you posted suggest a 10x difference, which is really concerning.

But since you didn't provide a self contained executable script, I can't tell what it could be, hence this isn't actionable for me so I'll close.

If you provide a full repro script I'll reopen and investigate.

[shnikola]

Thank you @byroot for a super quick response!
After some investigation, it looks like the main reason for the 10x difference is us being on Rails 6. I've checked the benchmark on both 7 and 8 and it is closer to your numbers (although vanilla JSON.generate is still about 30% slower than Oj.dump for me).

Do you have any recommendations on how to prevent Rails from using to_json in the render json: call and instead using JSON.generate?

[byroot]

Do you have any recommendations on how to prevent Rails from using to_json in the render json: call and instead using JSON.generate?

Good question. I think you could try to replace the default :json renderer: https://api.rubyonrails.org/classes/ActionController.html#method-c-add_renderer

Something like

class ApplicationController < ...
  remove_renderer :json
  add_rendererer :json do |json, options|
    json = JSON.generate(json, options) unless json.kind_of?(String)

    if options[:callback].present?
      if media_type.nil? || media_type == Mime[:json]
        self.content_type = Mime[:js]
      end

      "/**/#{options[:callback]}(#{json})"
    else
      self.content_type = Mime[:json] if media_type.nil?
      json
    end
  end
end

NB: I haven't tested it, might need some adjustments.

[byroot]

although vanilla JSON.generate is still about 30% slower than Oj.dump for me

Could you give me more details about your setup?

• Ruby version
• JSON gem version
• Operating system and CPU type

[shnikola]

I am using ruby 3.3.6 (2024-11-05 revision 75015d4c1f) [arm64-darwin24] on Apple M2 Max
JSON gem version is 2.10.1

Running your benchmark with ruby test/benchmark.rb and ruby --enable-yjit test/benchmark.rb gives a similar result:

              user     system      total        real
.to_json  0.251084   0.010159   0.261243 (  0.261346)
JSON.gen  0.098276   0.004044   0.102320 (  0.102422)
Oj.dump   0.069650   0.004888   0.074538 (  0.074541)

[byroot]

Here's an improved version of the benchmark, that's less subject to small variance:

$VERBOSE = nil

require 'bundler/inline'
gemfile do
  gem "activesupport", "~> #{ENV['RAILS_VERSION'] || '6.1'}.0", require: false
  gem "json", "2.10.1"
  gem "oj"
  gem "benchmark-ips"
end

source = File.read("/tmp/large_example.json")
print "Generating #{source.bytesize}B of JSON...\n"
obj = JSON.parse(source)
objs = [obj] * 10

puts "=== vanilla ==="
Benchmark.ips do |x|
  x.report(".to_json") { objs.to_json }
  x.report("JSON.gen") { JSON.generate(objs) }
  x.report("Oj.dump") { Oj.dump(objs) }
  x.compare!(order: :baseline)
end

require 'active_support/all'
puts "=== active support #{ActiveSupport::VERSION::STRING} loaded ==="
Benchmark.ips do |x|
  x.report(".to_json") { objs.to_json }
  x.report("JSON.gen") { JSON.generate(objs) }
  x.report("Oj.dump") { Oj.dump(objs) }
  x.compare!(order: :baseline)
end

puts "=== Oj.optimize_rails ==="
Oj.optimize_rails
Benchmark.ips do |x|
  x.report(".to_json") { objs.to_json }
  x.report("JSON.gen") { JSON.generate(objs) }
  x.report("Oj.dump") { Oj.dump(objs) }
  x.compare!(order: :baseline)
end

You said Rails 6, but not sure which one, I assumed 6.1.

On my Air M3:

Generating 2148768B of JSON...
=== vanilla ===
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
            .to_json     1.000 i/100ms
            JSON.gen     1.000 i/100ms
             Oj.dump     1.000 i/100ms
Calculating -------------------------------------
            .to_json     13.118 (± 0.0%) i/s   (76.23 ms/i) -     66.000 in   5.031548s
            JSON.gen     13.090 (± 7.6%) i/s   (76.39 ms/i) -     66.000 in   5.062203s
             Oj.dump     14.235 (± 0.0%) i/s   (70.25 ms/i) -     72.000 in   5.058716s

Comparison:
            .to_json:       13.1 i/s
             Oj.dump:       14.2 i/s - 1.09x  faster
            JSON.gen:       13.1 i/s - same-ish: difference falls within error

=== active support 6.1.7.8 loaded ===
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
            .to_json     1.000 i/100ms
            JSON.gen     1.000 i/100ms
             Oj.dump     1.000 i/100ms
Calculating -------------------------------------
            .to_json      1.412 (± 0.0%) i/s  (708.42 ms/i) -      8.000 in   5.667971s
            JSON.gen     12.250 (± 0.0%) i/s   (81.63 ms/i) -     62.000 in   5.063245s
             Oj.dump     14.082 (± 0.0%) i/s   (71.01 ms/i) -     71.000 in   5.042828s

Comparison:
            .to_json:        1.4 i/s
             Oj.dump:       14.1 i/s - 9.98x  faster
            JSON.gen:       12.3 i/s - 8.68x  faster

=== Oj.optimize_rails ===
ruby 3.3.4 (2024-07-09 revision be1089c8ec) +YJIT [arm64-darwin23]
Warming up --------------------------------------
            .to_json     1.000 i/100ms
            JSON.gen     1.000 i/100ms
             Oj.dump     1.000 i/100ms
Calculating -------------------------------------
            .to_json     13.039 (± 0.0%) i/s   (76.69 ms/i) -     66.000 in   5.063831s
            JSON.gen     12.967 (± 0.0%) i/s   (77.12 ms/i) -     65.000 in   5.014628s
             Oj.dump     13.972 (± 0.0%) i/s   (71.57 ms/i) -     70.000 in   5.010640s

Comparison:
            .to_json:       13.0 i/s
             Oj.dump:       14.0 i/s - 1.07x  faster
            JSON.gen:       13.0 i/s - 1.01x  slower

So on my machine the performance is very much on par, as long as you don't fall into the ActiveSupport::JSON::Encoding trap (it's really brutal on these older versions of Rails, but got much better on recent ones).