| Age | Commit message (Collapse) | Author |
|---|
|
**💧 Group documentation into _doc/ directory**.
This is so we can serve the documentation under
https://kilabit.info/project/haminer.
In the Makefile, we add task `serve-doc` to preview the documentation.
**💧 Replace module "share" with "pakakeh.go"**
The "share" module has been renamed to "pakakeh.go" and moved to different
git repository.
**💧 Realign struct HttpLog and questdbClient**
The HttpLog realign from 304 to 288 bytes.
The questdbClient realign from 56 to 24 bytes.
**🪵 Comply with all linters recommendations**
Some of breaking changes,
* Field [Config.HttpUrl] renamed to [Config.HTTPURL]
* Field [ConfigForwarder.Url] renamed to [ConfigForwarder.URL]
* Struct [HttpLog] renamed to [HTTPLog]
**💧 _ops/haminer-test: setup container for development using mkosi**
The container run HAProxy and PostgreSQL servers.
We also create new dummy backend to test backend in HAProxy.
The haminer-dummy-backend run in container and serve two ports in HTTP
and TCP modes.
**🌱 all: implement forwarder for Postgresql**
The Postgresql forwarder accept single option "URL",
[forwarder "postgresql"]
url = postgres://<user>:<pass>@<host>/<database>?sslmode=<>
The user and database must already created first, manually.
**🌼 Do not log ServerName with invalid connection '<NOSRV>'**
**🌼 _AUR: fix installation of binary in package function**
**Add linter gocheck**
Program gocheck implement go static analysis using [Analyzer] that are not
included in the default go vet.
See package [lib/goanalysis] for more information.
[Analyzer]: https://pkg.go.dev/golang.org/x/tools/go/analysis#hdr-Analyzer +
[lib/goanalysis]: https://pkg.go.dev/git.sr.ht/~shulhan/pakakeh.go/lib/goanalysis/
|
|
Convert the old ".reuse/dep5" format to REUSE.toml format using
"reuse convert-dep5" command.
For generated files, add the ".license" file.
Using "reuse annotate ..." it automatically add empty line after
"SPDX-FileCopyrightText:", and it also put "SPDX-FileCopyrightText" above
the "SPDX-License-Identifier".
Now, this project is compliant with version 3.3 of the REUSE
Specification.
|
|
The fieldalignment and shadow is a linter from golang.org/x/tools.
This program actually have an API that can be used.
The pakakeh.go/lib/goanalysis wrap those APIs into a single function
call that can be run inside a main.
This minimize and simplified our tools dependencies.
|
|
|
|
Also, make the README as index for project page.
|
|
Golangci-lint does not works when working with gotip, use too much memory,
and become red herring and subjective instead of correctness.
|
|
|
|
|
|
|
|
|
|
|
|
The Postgresql forwarder accept single option "URL",
[forwarder "postgresql"]
url = postgres://<user>:<pass>@<host>/<database>?sslmode=<>
The user and database must already created first, manually.
|
|
|
|
The function signature to parse the UDP packet changes to return HTTPLog
instead of as receiver.
|
|
|
|
The haminer-dummy-backend run in container and serve two ports in HTTP
and TCP modes.
|
|
The container run HAProxy and PostgreSQL servers.
|
|
Our primary host for the repository in sourcehut, unfortunately they do
not support AsciiDoc format, unlike GitHub.
To make both repositories can render README we changes it to markdown
format.
|
|
|
|
Some of breaking changes,
* Field [Config.HttpUrl] renamed to [Config.HTTPURL]
* Field [ConfigForwarder.Url] renamed to [ConfigForwarder.URL]
* Struct [HttpLog] renamed to [HTTPLog]
|
|
The HttpLog realign from 304 to 288 bytes.
The questdbClient realign from 56 to 24 bytes.
|
|
|
|
|
|
|
|
Show what the HTTP log looks like, how it stored, and how it can be used.
|
|
|
|
|
|
Without symlink, GitHub will render repository page as plain text
instead of AsciiDoc.
|
|
|
|
This release relicensing the software to GPLv3, add support for forwarding
logs to InfluxDB v2 and questdb [1].
[1] https://questdb.io.
Signed-off-by: Shulhan <ms@kilabit.info>
|
|
While at it, mention how to install it using pre-build Arch Linux
package through build.kilabit.info.
|
|
|
|
See https://kilabit.info/journal/2022/gpl/ for more information.
|
|
This changes convert the README to AsciiDoc format, add section about
configuring forwarder using questdb.
|
|
Questdb [1] is one the time-series database.
We experiment to forward the HTTP log using Influx Line Protocol (ILP).
[1]: https://questdb.io/
|
|
Instead of single forwarder, Influxd, the Config struct now can have
one or more forwarders.
The kind of forwarders is defined by it subsection name, for example
`[forwarder "influxd"]` defined a forwarder for influxd.
|
|
|
|
Later we will have multiple forwarders, not only influxd.
|
|
It is up to the user of haminer library (in this case the cmd/haminer)
on how to Start and Stop the process, not at the library level.
|
|
Halog contains parsed HTTP log, so its make more readable if we rename
the type name.
|
|
|
|
|
|
|
|
Since the influxdb v2, and since the flux query language introduced,
the way the log read and queried kinda changes.
Things that we can query on v1, is not possible (or maybe I forgotten)
anymore.
This changes move all haproxy log fields value that is not number (except
HTTP status code) to tags and left all numbers (like time, number of
connections, bytes read) in the fields.
|
|
Instead of using struct UDPPacket to read UDP packet from HAproxy log,
simplify it by using fixed, reusable size of []byte directly.
|
|
Previously, if no logs received after 15 seconds (or any interval user
defined in Config.ForwardInterval), the haminer process send an empty
halog to forwarders (empty halog is the one that use "-" for backend,
frontend, server name, and HTTP method).
This cause bogus traffic and need additional filter when analyzed.
|
|
This changes replace the "influxdb_api_write" with new section
`[forwarder "influxd"]`.
The section contains version, url, org, bucket, user, password, and
token.
The version field define the API version to be used when writing log
to Influxd.
|
|
Set the minimum Go to 1.18 and update the share modules.
When this program written, the ini library does not have the Unmarshal
function, so we load the configuration by reading each key and parse it
manually.
Now that we have Unmarshal function, the way we parse the configuration
is simplified so does the way configuration written.
This changes the accept_backend, capture_request_header written.
Instead of using comma to set multiple values, now it must be written
one key and one value on different line.
|
|
|
|
An administrator may need to monitor if the haminer service is running or
not through kapacitor. With heartbeat, we can tell if haminer is not
running if no data is send along, say, one hour.
|
