Rails Response and Assets Compression on Heroku

[winston]

Update 10 Jan 2016: The earlier benchmarks were ran against a Rails 4.2.4 (with Sprockets v3.4.0) app where gzip compression was missing.

@schneems has since reintroduced gzip compression in v3.5.0 (see commit rails/sprockets@7faa6ed), and so I ran the baseline again with Rails 4.2.5 and more importantly with Sprockets v3.5.2. Results:

Baseline Updated with Sprockets v3.5.2

Demo at http://rails-heroku-baseline-updated.herokuapp.com

Let's take another look at our baseline - how a basic Rails 4.2.5 app performs out of the box.

As compared to the previous baseline (using Rails 4.2.4 and Sprockets 3.4.0), you can see that in this updated baseline, the application.css and application.js are both gzipped.

In total, 571KB was transferred and it took about 3.66s for the page to load.

When we run Page Speed Insight on this app, we get a score of 64/100 and Enable compression is top of the "Should Fix" list, but it's only for the web request (and not the assets).

This means that we only need to fix the problem of gzipping our web response. Read on!

---

@heroku is awesome, in that you can deploy a Ruby app in less than 5 minutes up into the internet. However, in exchange for that convenience, we are not able to configure web server settings easily (unless you launch your own Nginx buildpack, for example).

On the other hand, speed really is king and every website aims to be speedier for many, many reasons. Two being for a better user experience and for a better site ranking (according to Google).

One of the most commonly suggested advice in speeding up a website is to enable compression and serve gzipped responses and gzipped assets (JS and CSS) which can be easily configured on the server (Nginx) level.

However, we can't really do that on vanilla Heroku and so we have to explore alternatives.

There are a number of ways we can have content compression on vanilla Heroku, and this post is for exploring those different ways.

tl;dr You can use the heroku-deflater gem.

---

For the purpose of exploring different ways to achieve content compression on vanilla Heroku, I created a simple Rails 4.2 app with the following gems:

ruby '2.2.3'

gem 'puma'
gem 'pg'

gem 'slim-rails'

gem 'bootstrap-sass'
gem 'font-awesome-sass'

group :staging, :production do
  gem 'rails_12factor'
end

Next, I generated a scaffold for blog_post with title and content as attributes and populated the database 1500 blog posts using seeds.rb.

The source code is available here: https://github.com/winston/rails-heroku-compression

Goals

Our goal is to find out which method is better for achieving compression on:

• web response
• application.css
• application.js

Essentially, these responses should be gzipped and be small in size.

Baseline

Demo at http://rails-heroku-baseline.herokuapp.com

Let's first look at how a basic Rails app performs out of the box.

The size of the web response is about 431KB, application.css 148KB and application.js 156KB.

In the Content-Encoding column, you can see that all three are not encoded (gzipped) in anyway.

In total, 799KB was transferred and it took about 3.25s for the page to load.

When we run Page Speed Insight on this app, we get a score of 56/100 and Enable compression is top of the "Should Fix" list.

Rack Deflater

Demo at http://rails-heroku-rack-deflater.herokuapp.com

In this branch, we added a middleware that would perform runtime compression on the web response. However, it doesn't compress CSS or JavaScript.

# Added to config/application.rb

module RailsHerokuCompression
  class Application < Rails::Application
    # ...

    config.middleware.use Rack::Deflater
  end
end

Let's look at how it performs.

The size of the web response is now 24.5KB and "Content-Encoding" appears as gzip, while application.css and application.js remains unchanged.

That's a saving of about 94% in size!

In total, 392KB was transferred and it took about 3.52s for the page to load.

Even though the total size was reduced by about 50%, however on the average with Rack::Deflater, this branch seemed to have taken just a bit more time than the baseline to load. That's because compression was done during runtime, and that could have resulted in a slight slowdown, as shared by @thoughtbot too.

When we run Page Speed Insight on this app, we get a score of 70/100 which is an increase of 14 points over baseline.

Assets Gzip

Demo at http://rails-heroku-assets-gzip.herokuapp.com

In this branch, we are only concerned about compressing our assets.

This is important because compression has been removed from Sprockets 3 (affects Rails 4), so we need to do this "manually" for now, until maybe the next version of Sprockets.

Of course, other than doing this on the server, you can explore using a CDN like fastly that could do the compression of assets but we'll leave that to a separate discussion.

# Added to lib/assets.rake
# Source: https://github.com/mattbrictson/rails-template/blob/master/lib/tasks/assets.rake

namespace :assets do
  desc "Create .gz versions of assets"
  task :gzip => :environment do
    zip_types = /\.(?:css|html|js|otf|svg|txt|xml)$/

    public_assets = File.join(
      Rails.root,
      "public",
      Rails.application.config.assets.prefix)

    Dir["#{public_assets}/**/*"].each do |f|
      next unless f =~ zip_types

      mtime = File.mtime(f)
      gz_file = "#{f}.gz"
      next if File.exist?(gz_file) && File.mtime(gz_file) >= mtime

      File.open(gz_file, "wb") do |dest|
        gz = Zlib::GzipWriter.new(dest, Zlib::BEST_COMPRESSION)
        gz.mtime = mtime.to_i
        IO.copy_stream(open(f), gz)
        gz.close
      end

      File.utime(mtime, mtime, gz_file)
    end
  end

  # Hook into existing assets:precompile task
  Rake::Task["assets:precompile"].enhance do
    Rake::Task["assets:gzip"].invoke
  end
end

Let's look at how it performs.

The web response in this case remains un-gzipped at 431KB.

The size of application.css is now 26.4KB (down from 148KB) and "Content-Encoding" is gzip while the size of application.js is now 48.5KB (down from 156KB) and "Content-Encoding" is gzip too.

In total, 569KB was transferred and it took about 3.22s for the page to load.

When we run Page Speed Insight on this app, we get a score of 59/100 largely because the web response wasn't compressed.

Heroku Deflater

Demo at http://rails-heroku-heroku-deflater.herokuapp.com

In this branch, we will be using the heroku-deflater gem.

# Added to Gemfile

group: stagimg, :production do
  gem 'heroku-deflater'
end

Let's look at how it performs.

The web response is now 24.5KB (down from 431 KB), identical to when Rack::Deflater was used, while application.css is now 26.7KB and application.js is now 49.5KB.

All of them have "Content-Encoding" as gzip.

In total, 164KB was transferred which translates to a savings of 79% from the baseline measurement, and it took about 2.64s for the page to load.

When we run Page Speed Insight on this app, we get a score of 87/100 and it no longer complains about "Compression".

Optimized

Demo at http://rails-heroku-optimized.herokuapp.com

At this point, heroku-deflater has given us the best results so far with everything compressed.

Looking beneath the hood, heroku-deflater is actually simply using Rack::Deflater for "all" requests.
But if a gzipped version of the file already exists, then it would serve up that file immediately and not compressed it every single time.

With this in mind, I decided to try and combine both "Assets Gzip" and "Heroku Deflater" into this branch.

Let's look at how it performs.

The web response is still compressed at 24.5KB while application.css and application.js are both at the better compression of 26.5KB and 48.5KB (due to "Assets Gzip").

In total, there's also a slight reduction to 163KB sent and it took 2.91s to load the page.

When we run Page Speed Insight on this app, we get an even more impressive score of 89/100!

Conclusion

App	Web Response	application.css	application.js	Total Size	Total Time
baseline	431KB	148KB	156KB	799KB	3.25s
rack-deflater	24.5KB	148KB	156KB	392KB	3.52s
assets-gzip	431KB	26.4KB	48.5KB	569KB	3.22s
heroku-deflater	24.5KB	26.7KB	49.5KB	164KB	2.64s
optimized	24.5KB	26.5KB	48.5KB	163KB	2.91s

Rails doesn't do any compression out of the box, and if you are deploying on Heroku, a quick fix would be to use heroku-deflater.

If you are deploying your apps on non-Heroku boxes, then I am sure you will be able to tweak Nginx's server configurations to make compression work even more easily.

Besides doing such compression, it's also a good practice to put your apps behind CDNs too, as that would make your app even speedier.

In summary, don't forget to shrink your app before you deploy!

Notes:

• There is a CLI tool for Page Speed Insights too: https://github.com/addyosmani/psi
• Related thread between @vipulnsward and @schneems https://twitter.com/vipulnsward/status/654668379979014145

---

Thank you for reading.

@winston ✏️ Jolly Good Code

About Jolly Good Code

We specialise in Agile practices and Ruby, and we love contributing to open source.
Speak to us about your next big idea, or check out our projects.

[schneems]

Good stuff!

I would be curious to see the difference in backend time that it takes to "deflate" a response versus sending a regular response. i.e. even though it is faster for the browser to download and inflate the response, you lose something in the time it takes ruby to deflate the response. I'm curious what the delta is there.

-- 
Richard Schneeman
http://schneems.com

On October 20, 2015 at 4:41:43 AM, Winston (notifications@github.com) wrote:

@heroku is awesome, in that you can deploy a Ruby app in less than 5 minutes up into the internet. However, in exchange for that convenience, we are not able to configure web server settings easily (unless you launch your own Nginx buildpack, for example).

On the other hand, speed really is king and every website aims to be speedier for many, many reasons. Two being for a better user experience and for a better site ranking (according to Google).

One of the most commonly suggested advice in speeding up a website is to enable compression and serve gzipped responses and gzipped assets (JS and CSS) which can be easily configured on the server (Nginx) level.

However, we can't really do that on vanilla Heroku and so we have to explore alternatives.

There are a number of ways we can have content compression on vanilla Heroku, and this post is for exploring those different ways.

tl;dr You can use the heroku-deflater gem.

For the purpose of exploring different ways to achieve content compression on vanilla Heroku, I created a simple Rails 4.2 app with the following gems:

ruby '2.2.3'

gem 'puma'
gem 'pg'

gem 'slim-rails'

gem 'bootstrap-sass'
gem 'font-awesome-sass'

group :staging, :production do
gem 'rails_12factor'
end

Next, I generated a scaffold for blog_post with title and content as attributes and populated the database 1500 blog posts using seeds.rb.

The source code is available here: https://github.com/winston/rails-heroku-compression

Goals

Our goal is to find out which method is better for achieving compression on:

web response
application.css
application.js
Essentially, these responses should be gzipped and be small in size.

Baseline

Demo at http://rails-heroku-baseline.herokuapp.com

Let's first look at how a basic Rails app performs out of the box.

The size of the web response is about 431KB, application.css 148KB and application.js 156KB.

In the Content-Encoding column, you can see that all three are not encoded (gzipped) in anyway.

In total, 799KB was transferred and it took about 3.25s for the page to load.

When we run Page Speed Insight on this app, we get a score of 56/100 and Enable compression is top of the "Should Fix" list.

Rack Deflater

Demo at http://rails-heroku-rack-deflater.herokuapp.com

In this branch, we added a middleware that would perform runtime compression on the web response. However, it doesn't compress CSS or JavaScript.

Added to config/application.rb

module RailsHerokuCompression
class Application < Rails::Application
# ...

config.middleware.use Rack::Deflater

end
end

Let's look at how it performs.

The size of the web response is now 24.5KB and "Content-Encoding" appears as gzip, while application.css and application.js remains unchanged.

That's a saving of about 94% in size!

In total, 392KB was transferred and it took about 3.52s for the page to load.

Even though the total size was reduced by about 50%, however on the average with Rack::Deflater, this branch seemed to have taken just a bit more time than the baseline to load. That's because compression was done during runtime, and that could have resulted in a slight slowdown, as shared by @thoughtbot too.

When we run Page Speed Insight on this app, we get a score of 70/100 which is an increase of 14 points over baseline.

Assets Gzip

Demo at http://rails-heroku-assets-gzip.herokuapp.com

In this branch, we are only concerned about compressing our assets.

This is important because compression has been removed from Sprockets 3 (affects Rails 4), so we need to do this "manually" for now, until maybe the next version of Sprockets.

Of course, other than doing this on the server, you can explore using a CDN like fastly that could do the compression of assets but we'll leave that to a separate discussion.

Added to lib/assets.rake

Source: https://github.com/mattbrictson/rails-template/blob/master/lib/tasks/assets.rake

namespace :assets do
desc "Create .gz versions of assets"
task :gzip => :environment do
zip_types = /.(?:css|html|js|otf|svg|txt|xml)$/

public_assets = File.join(
  Rails.root,
  "public",
  Rails.application.config.assets.prefix)

Dir["#{public_assets}/**/*"].each do |f|
  next unless f =~ zip_types

  mtime = File.mtime(f)
  gz_file = "#{f}.gz"
  next if File.exist?(gz_file) && File.mtime(gz_file) >= mtime

  File.open(gz_file, "wb") do |dest|
    gz = Zlib::GzipWriter.new(dest, Zlib::BEST_COMPRESSION)
    gz.mtime = mtime.to_i
    IO.copy_stream(open(f), gz)
    gz.close
  end

  File.utime(mtime, mtime, gz_file)
end

end

Hook into existing assets:precompile task

Rake::Task["assets:precompile"].enhance do
Rake::Task["assets:gzip"].invoke
end
end

Let's look at how it performs.

The web response in this case remains un-gzipped at 431KB.

The size of application.css is now 26.4KB (down from 148KB) and "Content-Encoding" is gzip while the size of application.js is now 48.5KB (down from 156KB) and "Content-Encoding" is gzip too.

In total, 569KB was transferred and it took about 3.22s for the page to load.

When we run Page Speed Insight on this app, we get a score of 59/100 largely because the web response wasn't compressed.

Heroku Deflater

Demo at http://rails-heroku-heroku-deflater.herokuapp.com

In this branch, we will be using the heroku-deflater gem.

Added to Gemfile

group: stagimg, :production do
gem 'heroku-deflater'
end

Let's look at how it performs.

The web response is now 24.5KB (down from 431 KB), identical to when Rack::Deflater was used, while application.css is now 26.7KB and application.js is now 49.5KB.

All of them have "Content-Encoding" as gzip.

In total, 164KB was transferred which translates to a savings of 79% from the baseline measurement, and it took about 2.64s for the page to load.

When we run Page Speed Insight on this app, we get a score of 87/100 and it no longer complains about "Compression".

Optimized

Demo at http://rails-heroku-optimized.herokuapp.com

At this point, heroku-deflater has given us the best results so far with everything compressed.

Looking beneath the hood, heroku-deflater is actually simply using Rack::Deflater for "all" requests.
But if a gzipped version of the file already exists, then it would serve up that file immediately and not compressed it every single time.

With this in mind, I decided to try and combine both "Assets Gzip" and "Heroku Deflater" into this branch.

Let's look at how it performs.

The web response is still compressed at 24.5KB while application.css and application.js are both at the better compression of 26.5KB and 48.5KB (due to "Assets Gzip").

In total, there's also a slight reduction to 163KB sent and it took 2.91s to load the page.

When we run Page Speed Insight on this app, we get an even more impressive score of 89/100!

Conclusion

App Web Response application.css application.js Total Size Total Time
baseline 431KB 148KB 156KB 799KB 3.25s
rack-deflater 24.5KB 148KB 156KB 392KB 3.52s
assets-gzip 431KB 26.4KB 48.5KB 569KB 3.22s
heroku-deflater 24.5KB 26.7KB 49.5KB 164KB 2.64s
optimized 24.5KB 26.5KB 48.5KB 163KB 2.91s
Rails doesn't do any compression out of the box, and if you are deploying on Heroku, a quick fix would be to use heroku-deflater.

If you are deploying your apps on non-Heroku boxes, then I am sure you will be able to tweak Nginx's server configurations to make compression work even more easily.

Besides doing such compression, it's also a good practice to put your apps behind CDNs too, as that would make your app even speedier.

In summary, don't forget to shrink your app before you deploy!

Notes:

There is a CLI tool for Page Speed Insights too: https://github.com/addyosmani/psi
Related thread between @vipulnsward and @schneems https://twitter.com/vipulnsward/status/654668379979014145
Thank you for reading.

@winston Jolly Good Code

About Jolly Good Code

We specialise in Agile practices and Ruby, and we love contributing to open source.
Speak to us about your next big idea, or check out our projects.

—
Reply to this email directly or view it on GitHub.

[winston]

@schneems

Good stuff!

Thank you!

the difference in backend time that it takes to "deflate" a response versus sending a regular response

Yea. I was thinking about that too, because as you can see from the "Total Time" column, even though the final Total Size is about 79% smaller, but the time taken is only about 18% faster.. So the "deflate" definitely took some time..

Benchmarking the "deflate" should be doable. But the download i.e. "sending a regular response" depends on network latency so it may not be an apple to apple comparison. I'll see what I can dig up. 😄

[maia]

heroku-deflater has not been updated in the past two years – is it because it just works, or because people moved on to another solution? I assume heroku is hosting quite a lot of rails apps, so if this gem does not have any intolerable downsides, why would anyone not want to use it, why hasn't it become a de-facto-standard?

Also, does anyone know how it compares to heroku_rails_deflate? Both gems reference each other, both gems had their last commit a long time ago.

[winston]

@maia That's a good question. In fact, that was worrying me too which resulted in this post, because I wanted to make sure that @romanbsd https://github.com/romanbsd/heroku-deflater is still working. Therefore my conclusion is that it just works since the actual code is pretty simple.

Also there is an outstanding PR romanbsd/heroku-deflater#20 for Rails 5 compatibility, which means the code should be updated pretty soon.

why would anyone not want to use it, why hasn't it become a de-facto-standard?

Thing is, Heroku is not exclusive to Rails app, and Rails apps are not exclusive to Heroku, so probably it would be "wrong" to put such optimizations in either. Maybe @schneems has a better answer to that.

Also, does anyone know how it compares to heroku_rails_deflate?

Looking at the code. Both should be doing the same thing. I merely went with the one with more stars.

[romanbsd]

Just merged the PR and released 0.6.0 btw.

[winston]

Awesome. Thank you @romanbsd!

[winston]

Updated the article with a baseline benchmark using Rails 4.2.5 and Sprockets 3.5.2 in which gzip was reintroduced.

[vishalzambre]

Helpful information 👍