OmHTTPFS, Another Rsyslog HDFS Output Plugin

Still in beta, merged into rsyslog’s official repo.

Known issues or features to be supported:
Error codes returned;
Connection detect;
http keep-alive;

Code here: https://github.com/rsyslog/rsyslog/tree/master/contrib/omhttpfs

Update:
I wrote a test script, but most requests were lost.

Here are queue parameters supported by action() by default: http://www.rsyslog.com/doc/master/rainerscript/queue_parameters.html
And updated config:


OmHTTPFS

Author: sskaje (sskaje@gmail.com, http://sskaje.me/)

OmHTTPFS is an Rsyslog plugin writting data to HDS via Hadoop HDFS over HTTP.

Hadoop HDFS over HTTP

Official site: Hadoop HDFS over HTTP

HTTPFS is not well documented. I tried to read its source and write an intro with examples, until I found Administering the file system by using HttpFS REST APIs by IBM.

OmHDFS for Rsyslog

Rsyslog provides a plugin named omHDFS which requires lots of work compiling and configuring, and it’s not that usable.
Here is what I tried before writting this omhttpfs: Build omhdfs for Rsyslog.

Rsyslog config

Legacy config NOT supported.

Example:

Tested with CDH 5.2.0 + Rsyslog 8.6.0 on CentOS 7

# EOF

OmHTTPFS, Another Rsyslog HDFS Output Plugin by @sskaje: https://sskaje.me/2014/12/omhttpfs-rsyslog-hdfs-output-plugin/

Incoming search terms: