Iris is faster

[dcu]

perhaps gin can borrow some ideas to make the framework faster:

https://github.com/kataras/iris#benchmarks

[manucorporat]

@dcu maybe, it looks like a simple binary tree.
anyway I can't find how to run the benchmarks by myself, because the iris developer didn't forked go-http-bechmarks with iris included.

[manucorporat]

@dcu I just found a way to run the tests:

BenchmarkGin_GithubAll     20000         74933 ns/op           0 B/op          0 allocs/op
BenchmarkIris_GithubAll    20000         61231 ns/op           0 B/op          0 allocs/op

the difference is not that big.

anyway, I should check out eventual opportunities for optimizations.

[appleboy]

@manucorporat nice work

[CaptainCodeman]

It uses a cache to inflate it's results. They were substantially different when I added it into the go-http-router-benchmark and measured it without the cache (so actually measuring the router itself) but slower in either case.

Caching disabled
BenchmarkGin_GithubAll 50000 36235 ns/op 0 B/op 0 allocs/op
BenchmarkIris_GithubAll 10000 132241 ns/op 0 B/op 0 allocs/op

Caching enabled
BenchmarkGin_GithubAll 50000 35648 ns/op 0 B/op 0 allocs/op
BenchmarkIris_GithubAll 30000 42653 ns/op 0 B/op 0 allocs/op

https://www.reddit.com/r/golang/comments/4a8yit/is_this_the_fastest_go_web_framework/d0z0d42

[appleboy]

@CaptainCodeman I test the benchmark again.

Caching disabled

$ go test -bench=. -benchmem=true -benchtime=10s
   Gin: 52464 Bytes

#GithubAPI Routes: 203
   Iris: 80520 Bytes

PASS
BenchmarkGin_GithubAll    200000             63869 ns/op               0 B/op          0 allocs/op
BenchmarkIris_GithubAll   300000             47437 ns/op               0 B/op          0 allocs/op
ok      github.com/kataras/iris 28.160s

Caching enabled

$ go test -bench=. -benchmem=true -benchtime=10s
   Gin: 54256 Bytes

#GithubAPI Routes: 203
   Iris: 81784 Bytes

PASS
BenchmarkGin_GithubAll    200000             62737 ns/op               0 B/op          0 allocs/op
BenchmarkIris_GithubAll   500000             26655 ns/op               0 B/op          0 allocs/op
ok      github.com/kataras/iris 26.881s

[CaptainCodeman]

As I explained in the reddit thread several times, I was seeing a huge discrepancy between the results I got when I ran the tests in the Iris project vs when I ran them in the go-http-router-benchmark project - for both Gin and Iris so I didn't find them convincing. For a valid comparison to be made I think the author really needs to add his router to the go-http-router-benchmark project.

I'm also not interested in "caching enabled" results as those don't actually test the router, just the hash lookup (?) of the cache and IMO there are already better front-end caching solutions available for that.

[appleboy]

@CaptainCodeman

Could you provide go-http-router-benchmark results or testing source code? I am gonna get to know what's different between Iris project and go-http-router-benchmark.

[CaptainCodeman]

Sorry @appleboy, after the last update it started failing the unit tests and as the author was unwilling to accept the results anyway so I didn't see any point bothering with it anymore so deleted it all.

[appleboy]

@CaptainCodeman Thanks for your explanation.

[deankarn]

@manucorporat @CaptainCodeman @appleboy @dcu
It's not as fast as the author portrays, there is the pull request I just put in to bring it back in line:
kataras/iris#14

and updated the reddit posting:
https://www.reddit.com/r/golang/comments/4a8yit/is_this_the_fastest_go_web_framework/d15n2oa

[appleboy]

@kataras Why you closed kataras/iris#14 issues? If someone like me want to test iris framework, he/she will get wrong benchmark results.

By the way, you must to change benchmark result on iris project readme and update benchmark chart on iris website.

Thanks.

[otraore]

@kataras The handler for your benchmark (Iris) was still empty and you're comparing different things here. The benchmarks should be comparing routers, not routers with cached enabled. Regardless, I wish you the best with your project and I'm requesting we close this issue since the title is not correct. Maybe a better issue would be Implement optional caching like:

r := gin.Default()
r.SetCacheEnabled(true)

or something of that nature.

[CaptainCodeman]

I agree - this approach to caching is overly simplistic, is comparing apples and oranges and has already been solved in better ways (e.g. varnish or nginx, CDN + origin + http cache headers) in ways that won't fall over for non-trivial apps and traffic.

I'd still like to see the router added to the 'official' benchmark app for fairer comparison because the results I got were very different (even for gin) than running things from your codebase.

@kataras You also need to learn to accept criticism or people pointing out errors in your approach - people telling you the results they get doesn't make them liars or "paid" (who do you think is paying them exactly and why?)

[deankarn]

don't waste your breath @CaptainCodeman I gave the same advice in his repo and all he did was "locked and limited conversation to collaborators" then deleted his reddit account and edited his above comment to claim he doesn't have one.

sorry everyone for the noise.

[CaptainCodeman]

Yeah, I wanted to leave it as a warning for anyone who may be suckered by the bogus claims. There's no convincing some people that they haven't found a way to turn lead into gold. LOL