aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorShulhan <ms@kilabit.info>2024-03-17 02:04:08 +0700
committerShulhan <ms@kilabit.info>2024-03-17 16:26:26 +0700
commit0ab8aa946072d711d309fb759871d610a8dc085e (patch)
treef2b1be9f7ea9c8e9c38884b314f4146f46c1cd17
parent125b571dc3815b41f4d36a1581532bc7ef973b06 (diff)
downloadhaminer-0ab8aa946072d711d309fb759871d610a8dc085e.tar.xz
all: reformat the README to markdown
Our primary host for the repository in sourcehut, unfortunately they do not support AsciiDoc format, unlike GitHub. To make both repositories can render README we changes it to markdown format.
l---------README.adoc1
-rw-r--r--README.md (renamed from README)141
l---------_doc/README.adoc1
l---------_doc/README.md1
-rw-r--r--_doc/index.adoc11
5 files changed, 71 insertions, 84 deletions
diff --git a/README.adoc b/README.adoc
deleted file mode 120000
index 100b938..0000000
--- a/README.adoc
+++ /dev/null
@@ -1 +0,0 @@
-README \ No newline at end of file
diff --git a/README b/README.md
index 000cb8c..1ca54ae 100644
--- a/README
+++ b/README.md
@@ -1,9 +1,4 @@
-// SPDX-FileCopyrightText: 2018 M. Shulhan <ms@kilabit.info>
-// SPDX-License-Identifier: GPL-3.0-or-later
-= haminer
-:toc:
-:sectanchors:
-:sectlinks:
+# haminer
Library and program to parse and forward HAProxy logs.
@@ -12,26 +7,27 @@ to backend.
In default format, it looks like these (split into multi lines, for
readability):
-----
+```
<158>Sep 4 17:08:47 haproxy[109530]: 185.83.144.103:46376
[04/Sep/2022:17:08:47.264] www~ be_kilabit/kilabit-0.0/0/1/2/3 200 89 - -
---- 5/5/0/0/0 0/0 "GET / HTTP/1.1"
-----
+```
See
-https://www.haproxy.com/documentation/hapee/1-8r1/onepage/#8.2.3[HTTP log format documentation]
+[HTTP log format
+documentation](https://www.haproxy.com/documentation/hapee/1-8r1/onepage/#8.2.3)
for more information.
Currently, there are two supported database where haminer can forward the
-parsed log: influxdb and questdb.
+parsed log: Influxdb and Questdb.
Haminer support Influxdb v1 and v2.
-----
+```
+---------+ UDP +---------+ +-----------+
- | HAProxy |------>| haminer |----->| influxdb |
- +---------+ +---------+ | / questdb |
+ | HAProxy |------>| haminer |----->| Influxdb |
+ +---------+ +---------+ | / Questdb |
+-----------+
-----
+```
In Influxdb, the log are stored as measurement called `haproxy`.
In Questdb, the log are stored as table called `haproxy`.
@@ -49,172 +45,167 @@ Once the log has been accumulated, we can query the data.
For example, with Questdb we can count each visited URL using the following
query,
-----
+```
select backend, http_url, count(*) as visit from 'haproxy'
group by backend, http_url
order by visit desc;
-----
+```
-== Installation
+## Installation
-=== Building from source
+### Building from source
-*Requirements*
+Requirements,
-* https://golang.org[Go^] for building from source code
-* https://git-scm.com/[git^] for downloading source code
+- [Go](https://golang.org) for building the source code
+- [git](https://git-scm.com) for downloading the source code
Get the source code using git,
-----
+```
$ git clone https://git.sr.ht/~shulhan/haminer
$ cd haminer
$ make
-----
+```
The binary name is `haminer` build in the current directory.
-=== Pre-build package
+### Pre-build package
The Arch Linux package is available at build.kilabit.info.
-Add the following repository to your pacman.conf,
+Add the following repository to your `pacman.conf`,
-----
+```
[build.kilabit.info]
Server = https://build.kilabit.info/aur
-----
+```
To install it,
$ sudo pacman -Sy --noconfirm haminer-git
-== Configuration
+## Configuration
haminer by default will load it's config from `/etc/haminer.conf`, if not
specified when running the program.
See
-https://git.sr.ht/~shulhan/haminer/tree/main/item/cmd/haminer/haminer.conf[haminer.conf^]
+[haminer.conf](https://git.sr.ht/~shulhan/haminer/tree/main/item/cmd/haminer/haminer.conf)
for an example of possible configuration and their explanation.
-=== Forwarders
+### Forwarders
Currently, there are two supported database where haminer can forward the
-parsed log: influxdb and questdb.
+parsed log: Influxdb and Questdb.
Haminer support Influxdb v1 and v2.
-==== Influxdb v1
+#### Influxdb v1
For v1, you need to create the user and database first,
-----
+```
$ influx
> CREATE USER "haminer" WITH PASSWORD 'haminer'
> CREATE DATABASE haminer
> GRANT ALL ON haminer TO haminer
-----
+```
Example of forwarder configuration,
-----
+```
[forwarder "influxd"]
version = v1
url = http://127.0.0.1:8086
bucket = haminer
user = haminer
password = haminer
-----
+```
-==== Influxdb v2
+#### Influxdb v2
For v2,
-----
+```
$ sudo influx bucket create \
--name haminer \
--retention 30d
-----
+```
For v2, the example configuration is
-----
+```
[forwarder "influxd"]
-version = v1
+version = v2
url = http://127.0.0.1:8086
org = $org
bucket = haminer
token = $token
-----
+```
-==== Questdb
+#### Questdb
-For questdb the configuration is quite simple,
+For Questdb the configuration is quite simple,
-----
+```
[forwarder "questdb"]
url = udp://127.0.0.1:9009
-----
+```
-We did not need to create the table, questdb will handled that automatically.
+We did not need to create the table, Questdb will handled that automatically.
-== Deployment
+## Deployment
-. Copy configuration from `$SOURCE/cmd/haminer/haminer/conf` to
+Copy configuration from `$SOURCE/cmd/haminer/haminer/conf` to
`/etc/haminer.conf`
-. Update haminer configuration in `/etc/haminer.conf`
-+
---
+Update haminer configuration in `/etc/haminer.conf`.
For example,
-----
+
+```
[haminer]
listen = 127.0.0.1:5140
...
-----
+```
Add one or more provider to the configuration as the example above.
---
-. Update HAProxy config to forward log to UDP port other than rsyslog.
-+
---
+Update HAProxy config to forward log to UDP port other than rsyslog.
For example,
-----
+
+```
global
...
log 127.0.0.1:5140 local3
...
-----
+```
+
Then reload or restart HAProxy.
---
-. Run the haminer program,
-+
---
-----
+Run the haminer program,
+
+```
$ haminer
-----
+```
+
or use a
-https://git.sr.ht/~shulhan/haminer/tree/main/item/cmd/haminer/haminer.service[systemd
-service^].
+[systemd service](https://git.sr.ht/~shulhan/haminer/tree/main/item/cmd/haminer/haminer.service).
-----
+```
$ sudo systemctl enable haminer
$ sudo systemctl start haminer
-----
---
+```
-== License
+## License
-----
+```
haminer - Library and program to parse and forward HAProxy logs.
-Copyright (C) 2018-2022 M. Shulhan <ms@kilabit.info>
+Copyright (C) 2018-2024 M. Shulhan <ms@kilabit.info>
This program is free software: you can redistribute it and/or modify it under
the terms of the GNU General Public License as published by the Free Software
@@ -228,4 +219,4 @@ details.
You should have received a copy of the GNU General Public License along with
this program.
If not, see <http://www.gnu.org/licenses/>.
-----
+```
diff --git a/_doc/README.adoc b/_doc/README.adoc
deleted file mode 120000
index 59a23c4..0000000
--- a/_doc/README.adoc
+++ /dev/null
@@ -1 +0,0 @@
-../README \ No newline at end of file
diff --git a/_doc/README.md b/_doc/README.md
new file mode 120000
index 0000000..32d46ee
--- /dev/null
+++ b/_doc/README.md
@@ -0,0 +1 @@
+../README.md \ No newline at end of file
diff --git a/_doc/index.adoc b/_doc/index.adoc
index b57d021..53f12f4 100644
--- a/_doc/index.adoc
+++ b/_doc/index.adoc
@@ -1,9 +1,6 @@
= haminer
-:toc:
-:sectanchors:
-:sectlinks:
-haminer is a library and program to parse and forward HAProxy HTTP logs.
+`haminer` is a library and program to parse and forward HAProxy HTTP logs.
The HTTP logs is HTTP request that received by HAProxy frontend and forwarded
to backend.
@@ -21,13 +18,13 @@ https://www.haproxy.com/documentation/hapee/1-8r1/onepage/#8.2.3[HTTP log format
for more information.
Currently, there are two supported database where haminer can forward the
-parsed log: influxdb and questdb.
+parsed log: Influxdb and Questdb.
Haminer support Influxdb v1 and v2.
----
+---------+ UDP +---------+ +-----------+
- | HAProxy |------>| haminer |----->| influxdb |
- +---------+ +---------+ | / questdb |
+ | HAProxy |------>| haminer |----->| Influxdb |
+ +---------+ +---------+ | / Questdb |
+-----------+
----