Fluentd v1.19.0

Full Changelog: v1.18.0...v1.19.0

Enhancement

New features:

• #4657 Add zstd compression support
  • Buffer: add zstd to compress option.
  • out_file: add zstd to compress option.
  • out_forward: add zstd to compress option.
  • in_forward: support zstd format. (Experimental)
• #4986 buffer: add feature to evacuate chunk files when retry limit
• #4859 out_http: TLS1.3 support
• #5036 out_stdout: support output to STDOUT independently of Fluentd logger by setting use_logger to false
• #4904 out_file: add symlink_path_use_relative option to use relative path instead of absolute path in symlink_path
• #5008 System configuration: Add forced_stacktrace_level to force the log level of stacktraces.
• #4893 System configuration: support built-in config files

Metrics:

• #4966 metrics: enable input metrics by default
• #4980 in_tail: add "tracked_file_count" metrics to see how many log files are being tracked
• #4971 output: add metrics for number of writing events in secondary
• #4981 output: add metrics for dropped oldest chunk count

Others:

• #5003 in_forward: enable skip_invalid_event by default not to process broken data
• #4998 buf_file: reinforce buffer file corruption check
• #4866 in_http: allow empty Origin header requests to pass CORS checks
• #4865 in_tail: add warning for directory permission
• #4858 Add logging for errors about loading dependencies on startup
• Performance improvements
  • #4759 #4760 #4763 #4764 #4769 #4813 #4817 #4835 #4845 #4881 #4884 #4886 #4930 #4931 #4932 #4933 #4934 #4995

Bug Fix

• #5010 in_tail: fixed where specifying only encoding parameter might cause data corruption (affects since v0.14.12).
• #4864 formatter_csv: fix memory leak
• #5026 server plugin helper: ensure to close all connections at shutdown
• #4836 Fixed a bug where the default umask was not set to 0 when using --daemon (td-agent, fluent-package) since v1.14.6.
• #4836 --umask command line option: Fixed so that it is applied when Fluentd runs with --daemon (fluent-package) as well as when Fluentd runs with --no-supervisor.
• #4909 Windows: Stop the service when the supervisor is dead
• #4782 Windows: Fixed an issue where stopping the service immediately after startup could leave the processes.
• #4782 Windows: Fixed an issue where stopping service sometimes can not be completed forever.

Misc

• #4962 in_monitor_agent: stop using CGI.parse due to support Ruby 3.5
• #4962 HTTP server plugin helper: stop using CGI.parse due to support Ruby 3.5
• #4914 config: change inspect format
• #4857 console_adapter: support console gem v1.30
• #5042 gemspec: fix io-event and io-stream version to avoid unstable behavior on Windows
• #4900 in_http: replace WEBrick::HTTPUtils.parse_query with URI
  • Note that at least, this makes it unable to use ; delimiter.
• #4899 http_server: stop fallback to WEBrick
• #4978 metrics: add getter method automatically
• #4903 http_server helper: add header method for Request
• #4979 multi_output: fix metrics name
• #4964 plugin_id: fix typo
• CI fixes
  • #4728 #4730 #4746 #4747 #4748 #4750 #4755 #4820 #4874 #4877
• #4928 Fixes RuboCop's remarks
• #4989 CI: Add benchmark scripts

Contributors to this release (Alphabetical order)

• Athish Pranav D
• Daijiro Fukuda
• Eduardo Silva Pereira
• Florian Dobener
• Kentaro Hayashi
• Richard Lee
• Shizuo Fujita
• Takuro Ashie
• Tomoaki KOBAYASHI

Fluentd v1.16.9

Full Changelog: v1.16.8...v1.16.9

Bug Fix

• winsvc: Fix bug where service accidentally stops after starting.
  The previous version (v1.16.8) should not be used for Windows Service.
  #4955

Misc

• CI improvemnts
  #4956

Contributors to this release (Alphabetical order)

• Daijiro Fukuda
• Kentaro Hayashi
• Takuro Ashie

Fluentd v1.16.8

Full Changelog: v1.16.7...v1.16.8

This version has a critical bug about Windows Service. Do not use this version.
(#4955)

Bug Fix

• winsvc: Stop the service when the supervisor is dead
  #4942
• formatter_csv: Fix memory leak
  #4920

Misc

• Add fiddle as dependency gem for Ruby 3.5 on Windows
  #4919
• Refactoring code
  #4921
  #4922
  #4926
  #4943
• CI improvemnts
  #4821
  #4850
  #4851
  #4862
  #4915
  #4923
  #4925
  #4927

Contributors to this release (Alphabetical order)

• Daijiro Fukuda
• Kentaro Hayashi
• Shizuo Fujita
• Takuro Ashie

Fluentd v1.16.7

Full Changelog: v1.16.6...v1.16.7

Bug fix

• Backport(v1.16) command: fix NoMethodError of --daemon under Windows (#4716) in #4796
• Backport(v1.16) Windows: Fix an issue where stopping the service immediately after startup could leave the processes (#4782) in #4802

Misc

• Backport(v1.16): tests: use never instead of dont_allow (#4671) in #4723
• Backport(v1.16) tests: fix unused_port (#4675) in #4788
• Backport(v1.16) test_in_udp: add timeout for message_length_limit test (#4676) in #4789
• Backport(v1.16) test_in_udp: Reduce execution time of message_length_limit test (#4682) in #4790
• Backport(v1.16) test_out_forward: remove unnecessary ack_response_timeout setting (#4685) in #4791
• Backport(v1.16) test_cat: use proper protocol in unused_port method (#4686) in #4793
• Backport(v1.16) test_ack_handler: use singleton method instead of stub for stability (#4698) in #4794
• Backport(v1.16) test_in_tail: fix flaky tests for file rotation (#4710) in #4795
• Backport(v1.16) README: remove deprecated google analytics beacon (#4722) in #4797
• Backport(v1.16) test_out_file: fix file open mode for binary file in #4798
• Backport(v1.16) test_cat: specify the reading size to read in binary mode (#4730) in #4799
• Backport(v1.16) windows: add workaround for unexpected exception (#4747) in #4800
• Backport(v1.16) test_out_exec_filter: add a sleep to ensure the stream is emitted (#4755) in #4801
• Backport(v1.16) ci: use latest rubygems again for Ruby 3.1 on windows (#4750) in #4803

Contributors to this release (Alphabetical order)

• bennettfalkenberg
• Daijiro Fukuda
• Kentaro Hayashi
• Shizuo Fujita
• Takuro Ashie

Fluentd v1.18.0

Full Changelog: v1.17.1...v1.18.0

Enhancement

• #4624 Add zero-downtime-restart feature for non-Windows
• #4661 Add with-source-only feature
  • fluentd command: Add --with-source-only option
  • System configuration: Add with_source_only option
• #4661 Embedded plugin: Add out_buffer plugin, which can be used for buffering and relabeling events
• #4580 Config File Syntax: Extend Embedded Ruby Code support for Hashes and Arrays
  • Example: key {"foo":"#{1 + 1}"} => key {"foo":"2"}
  • Please note that this is not backward compatible, although we assume that this will never affect to actual existing configs.
  • In case the behavior changes unintentionally, you can disable this feature by surrounding the entire value with single quotes.
    • key '{"foo":"#{1 + 1}"}' => key {"foo":"#{1 + 1}"}
• #4718 transport tls: Use SSL_VERIFY_NONE by default
• #4720 transport tls: Add ensure_fips option to ensure FIPS compliant mode
• #4649 plugin_helper/server: Add receive_buffer_size parameter in transport section
• #4620 filter_parser: Now able to handle multiple parsed results
• #4655 in_http: add add_tag_prefix option
• #4604 System configuration: add path option in log section

Bug Fix

• #4716 command: fix NoMethodError of --daemon under Windows
• #4605 fluentd command: fix --plugin (-p) option not to overwrite default value

Misc

• #4619 http_server: Ready to support Async 2.0 gem
• #4641 Minor code refactoring
• #4638 #4644 #4675 #4676 #4677 #4686 CI fixes

Contributors to this release (Alphabetical order)

• Athish Pranav D
• Daijiro Fukuda
• Kentaro Hayashi
• Takuro Ashie
• Watson

Fluentd v1.16.6

Full Changelog: v1.16.5...v1.16.6

Bug Fix

• YAML config syntax: Fix issue where $log_level element was not supported correctly
  #4486
• parser_json: Fix wrong LoadError warning
  #4592
• fluentd command: Fix --plugin (-p) option not to overwrite default value
  #4605

Misc

• out_file: Add warn message for symlink_path setting
  #4512
• Keep console gem v1.23 to avoid LoadError
  #4510

Fluentd v1.17.1

Full Changelog: v1.17.0...v1.17.1

Enhancement

• #4528 out_http: Add compress gzip option
• #4533 in_exec: Add encoding option to handle non-ascii characters
• #4578 in_tail: Add throttling metrics
• #4588 compat: Improve method call performance
• #4586 in_sample: Add reuse_record parameter to reuse the sample data
  • in_sample has changed to copy sample data by default to avoid the impact of destructive changes by subsequent plugins.
  • This increases the load when generating large amounts of sample data.
  • You can use this new parameter to have the same performance as before.

Bug Fix

• #4492 logger: Fix LoadError with console gem v1.25
• #4522 parser_json: Fix wrong LoadError warning
• #4530 in_tail: Fix an issue where a large single line could consume a large amount of memory even though max_line_size is set
• #4482 yaml_parser: Support $log_level element

Misc

• #4523 Comment out inappropriate default configuration about out_forward
• #4534 gemspec: Remove unnecessary files from released gem
• #4535 plugin-generator: Update gemspec to remove unnecessary files
• #4594 Suppress non-parenthesis warnings
• #4598 Fix FrozenError in http_server plugin helper
• #4589 Add logger gem dependency for Ruby 3.5
• #4502 out_file: Add warn message for symlink_path setting

Contributors to this release (Alphabetical order)

• Athish Pranav
• Daijiro Fukuda
• Harshita Sao
• Hiroshi Hatake
• Kentaro Hayashi
• Lewis Rockliffe
• Shingo Nakayama
• Taeseong Yu
• Takuro Ashie
• Watson

Fluentd v1.17.0

Full Changelog: v1.16.2...v1.17.0

Enhancement

• #4282 in_http: Recognize CSP reports as JSON data
• #4330 out_http: Add option to reuse connections
• #4401 in_tail: Expand glob capability for square brackets and one character matcher
• #4459 out_http: Support AWS Signature Version 4 authentication

Bug Fix

• #4474 Make sure parser_json and parser_msgpack return Hash.
  Make parser_json and parser_msgpack accept only Hash or Array of Hash.
• #4478 filter_parser: Add error event for multiple parsed results

Misc

• #4288 Raise minimum required ruby version
• #4411 Require missing dependent gems as of Ruby 3.4-dev
• #4294 #4299 #4302 #4320 Minor code refactoring
• #4369 #4433 #4452 #4477 CI fixes
• #4429 github: unify YAML file extension to .yml

Contributors to this release (Alphabetical order)

• Christian Norbert Menges
• Daijiro Fukuda
• David Venable
• Giedrius Statkevičius
• Hiroshi Hatake
• Kentaro Hayashi
• Šimon Lukašík
• Takuro Ashie

v1.16.5

Full Changelog: v1.16.4...v1.16.5

Bug Fix

• #4447 Buffer: Fix emit error of v1.16.4 sometimes failing to process large data exceeding chunk size limit

Contributors to this release (Alphabetical order)

• Daijiro Fukuda
• Luke Arntz (Reporting #4447 problem. Thanks!)

v1.16.4

Full Changelog: v1.16.3...v1.16.4

Bug Fix

• #4342 Fix to avoid processing discarded chunks in write_step_by_step.
  It fixes not to raise pile of IOError when many chunk bytes limit exceeds errors are occurred.
• #4334 in_tail: Fix tail watchers in rotate_wait state not being managed.

Misc

• #4331 buffer: Avoid unnecessary log processing. It will improve performance.

Contributors to this release (Alphabetical order)

• Christian Norbert Menges
• Daijiro Fukuda
• Kentaro Hayashi
• Takuro Ashie