Graylog Grok Pattern Extractor issue #38
Description
Hi,
I have installed graylog1.1.5-1 through the vagrant image. I have set up a syslog input to grab log entries from linux iptables running on my linux firewall. All of that is working perfectly fine and the entries are being successfully captured.
I have tried to set up a Grok Pattern Extractor to capture some of the fields from the iptables log entries so I can do some indexing and searches on specific fields and i ran into some issues.
An entry looks like this:
[19348602.294727] New_Connection -- ACCEPT IN=eth1 OUT= MAC=ff:ff:ff:ff:ff:ff:00:1d:7d:0c:03:db:08:00 SRC=192.168.1.11 DST=192.168.1.255 LEN=78 TOS=0x00 PREC=0x00 TTL=128 ID=13989 PROTO=UDP SPT=137 DPT=137 LEN=58
and the Grok Pattern looks like this:
%{SYSLOG5424SD:time_stamp}.*%{WORD:action} IN=%{WORD:int_eth} OUT= MAC=%{IP}:%{MAC} SRC=%{IP:src_ip} DST=%{IP:dst_ip} LEN=%{INT:length}.*PROTO=%{WORD:proto} SPT=%{INT:src_port} DPT=%{INT:dst_port}.*
I have run these through the Grok debugger and it successfully captures the right fields, but when I set up an Extractor with these and try it on the web interface, it doesn't work.
The first time I tried it, it gave me a timeout message, so I increased the timeout to 10 seconds.
After that, I tried again and it gave me another error. I googled it, but can't find any information on it. I found this error in the /var/log/graylog/server/current log file:
2015-08-07_20:41:34.80474 ERROR [AnyExceptionClassMapper] Unhandled exception in REST resource
2015-08-07_20:41:34.80476 oi.thekraken.grok.api.exception.GrokException: Deep recursion pattern compilation of %{SYSLOG5424SD:time_stamp}.*%{WORD:action} IN=%{WORD:int_eth} OUT= MAC=%{IP}:%{MAC} SRC=%{IP:src_ip} DST=%{IP:dst_ip} LEN=%{INT:length}.*PROTO=%{WORD:proto} SPT=%{INT:src_port} DPT=%{INT:dst_port}.*
2015-08-07_20:41:34.80477 at oi.thekraken.grok.api.Grok.compile(Grok.java:344)
2015-08-07_20:41:34.80478 at org.graylog2.rest.resources.tools.GrokTesterResource.doTestGrok(GrokTesterResource.java:83)
2015-08-07_20:41:34.80478 at org.graylog2.rest.resources.tools.GrokTesterResource.testGrok(GrokTesterResource.java:72)
2015-08-07_20:41:34.80479 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2015-08-07_20:41:34.80479 at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
2015-08-07_20:41:34.80480 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2015-08-07_20:41:34.80480 at java.lang.reflect.Method.invoke(Method.java:497)
2015-08-07_20:41:34.80481 at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
2015-08-07_20:41:34.80482 at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:164)
2015-08-07_20:41:34.80483 at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:181)
2015-08-07_20:41:34.80483 at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:203)
2015-08-07_20:41:34.80484 at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:101)
2015-08-07_20:41:34.80484 at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
2015-08-07_20:41:34.80485 at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
2015-08-07_20:41:34.80485 at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
2015-08-07_20:41:34.80486 at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:305)
2015-08-07_20:41:34.80486 at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
2015-08-07_20:41:34.80487 at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
2015-08-07_20:41:34.80488 at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
2015-08-07_20:41:34.80489 at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
2015-08-07_20:41:34.80489 at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
2015-08-07_20:41:34.80490 at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
2015-08-07_20:41:34.80490 at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:288)
2015-08-07_20:41:34.80491 at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1110)
2015-08-07_20:41:34.80493 at org.graylog2.jersey.container.netty.NettyContainer.messageReceived(NettyContainer.java:356)
2015-08-07_20:41:34.80493 at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
2015-08-07_20:41:34.80494 at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
2015-08-07_20:41:34.80495 at org.jboss.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
2015-08-07_20:41:34.80495 at org.jboss.netty.handler.execution.ChannelUpstreamEventRunnable.doRun(ChannelUpstreamEventRunnable.java:43)
2015-08-07_20:41:34.80497 at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:67)
2015-08-07_20:41:34.80497 at com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:176)
2015-08-07_20:41:34.80498 at org.jboss.netty.handler.execution.MemoryAwareThreadPoolExecutor$MemoryAwareRunnable.run(MemoryAwareThreadPoolExecutor.java:622)
2015-08-07_20:41:34.80498 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
2015-08-07_20:41:34.80499 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
2015-08-07_20:41:34.80499 at java.lang.Thread.run(Thread.java:745)
I have done some googling on the deep recursion error message, but couldn't find anything, hence why I am posting this here.
I have opened an issue on github.com/graylog2/graylog2-server, but they said that it might be a bug in java-grok, hence why I am opening an issue here.
Any idea on how to solve this would be much appreciated.
Thanks a lot in advance,
Bertrand.