jarink, branch mainProgram to inspects and maintains web sites.http://git.kilabit.info/jarink/atom?h=main2026-02-12T02:05:42ZRelease jarink 0.3.0 (2026-02-12)2026-02-12T02:05:42ZShulhanms@kilabit.info2026-02-12T02:04:54Zurn:sha1:23bbad5b40b5940ca3dcb7839d09c0462cf6c45d
**🌼 brokenlinks: refactoring the logic, simplify the code**
Previously, we made the scan logic to run in multiple goroutine with
one channel to push and consume the result and another channel to push
and pop link to be processed.
The logic is a very complicated code, making it hard to read and debug.
These changes refactoring it to use single goroutine that push and pop
link from/to a slices, as queue.
Another refactoring is in where we store the link.
Previously, we have [jarink.Link], [brokenlinks.Broken], and
[brokenlinks.linkQueue] to store the metadata for a link.
These release unified them into struct [jarink.Link].
**🌱 brokenlinks: print the progress to stderr**
Each time the scan start, new queue add, fetching start, print the
message to stderr.
This remove the verbose options for better user experience.
**🌼 brokenlinks: improve fetch logging and decrease timeout to 10s**
When fetching, print log after the fetch completed.
If success, print the URL along with HTTP status code.
If fail, print the URL along with the error.
The timeout now reduce to 10 seconds to prevent long delay when working
with broken website.
**🌼 brokenlinks: mark the link in queue as seen with status code 0**
This is to fix double URL being pushed to queue.
Given the following queue and its parent,
----
/page2.html => /index.html
/brokenPage => /index.html
/brokenPage => /page2.html
----
Before scanning the second "/brokenPage" on parent page "/page2.html",
check if its seen first to get the status code before we run the scan.
This allow jarink report "/brokenPage" as broken link for both pages,
not just in "/index.html".
**🌼 brokenlinks: skip parsing non-HTML page**
If the response Content-type return other than "text/html", skip parsing
the content and return immediately.
We also skip processing "mailto\:" URL.
**🌼 brokenlinks: make link that return HTML always end with slash**
If parent URL like "/page" return the body as HTML page, the URL should
be end with slash to make the relative links inside it works when joined
with the parent URL.
**🌱 brokenlinks: store the anchor or image source in link**
In the struct `Link`, we add field `Value` that store the `href` from A
element or `src` from IMG element.
This allow us to debug any error during scan, especially joining path
and link.
**🌼 brokenlinks: fix possible panic in markAsBroken**
If the Link does not have `parentUrl`, set the parent URL using the link
URL itself.
This only happened if the target URL that we will scan return an error.
all: update the README2026-02-12T01:59:22ZShulhanms@kilabit.info2026-02-12T01:59:22Zurn:sha1:3aaf1dd2a070f04c81c58681b32186abb3a956b9
Rewording some paragraphs, formatting on code, and add INSTALL
section.
make: add `build` task2026-02-12T01:56:48ZShulhanms@kilabit.info2026-02-12T01:56:48Zurn:sha1:b9f7ca45432916f3cf19ff8b62e428531aa0fa8d
The build task set the Version information based on the latest
tag and number of commits.
brokenlinks: fix possible panic in markAsBroken2026-02-11T18:07:04ZShulhanms@kilabit.info2026-02-11T18:05:52Zurn:sha1:e4b3383d84cd181f4a9fec1097ec61e583d6f827
If the Link does not have parentUrl, set the parent URL using the link
URL itself.
This only happened if the target URL that we will scan return an error.
brokenlinks: store the anchor or image source in link2026-02-11T18:04:40ZShulhanms@kilabit.info2026-02-11T18:04:40Zurn:sha1:9c7ee77376294e9abd70ca356e26d0ab16ad7466
In the struct Link, we add field Value that store the href from A element
or src from IMG element.
This allow us to debug any error during scan, especially joining path
and link.
brokenlinks: make link that return HTML always end with slash2026-02-11T14:45:06ZShulhanms@kilabit.info2026-02-11T03:47:42Zurn:sha1:8100b3be0730173a77f1a64f9ac6bc8862a159ac
If parent URL like "/page" return the body as HTML page, the URL should
be end with slash to make the relative links inside it works when joined
with the parent URL.
go.mod: update all dependencies2026-02-11T14:45:06ZShulhanms@kilabit.info2026-02-11T02:00:13Zurn:sha1:e59724f4d701f8889167219b2ccc18f4e8954034brokenlinks: skip processing "mailto:" URL2026-02-11T14:45:06ZShulhanms@kilabit.info2026-02-10T21:45:04Zurn:sha1:c92d49c41e27d50a36fd6eba0824331606627132brokenlinks: test links that wrapped by other elements2026-02-11T14:45:06ZShulhanms@kilabit.info2026-02-10T21:40:35Zurn:sha1:ded0c89eea2ff0193f7916db35e3f600d24842ff
This is to see the behaviour of [Node.Descendants] when traversing
the element recursively.
brokenlinks: check if link has been seen before scan2026-02-10T21:38:20ZShulhanms@kilabit.info2026-02-10T21:38:20Zurn:sha1:61eb5351087be894a4bfeb71c99346b8065bb7f1
Given the following queue and its parent,
/page2.html => /index.html
/brokenPage => /index.html
/brokenPage => /page2.html
Before scanning the second "/brokenPage" on parent page "/page2.html",
check if its seen first to get the status code before we run the scan.
This allow jarink report "/brokenPage" as broken link for both pages,
not just in "/index.html".