diff options
| author | Shulhan <ms@kilabit.info> | 2024-01-06 17:49:56 +0700 |
|---|---|---|
| committer | Shulhan <ms@kilabit.info> | 2024-01-06 17:50:21 +0700 |
| commit | 84ee82ffe4bedf85aa3a69f45034d0d1e1b74e6a (patch) | |
| tree | 52907fb8c0dfc91d92b36ee2ac661249729c0e4c | |
| parent | bbefc9b4cd94a0688c45245ff2b2386ddb76d32e (diff) | |
| download | pakakeh.go-0.52.0.tar.xz | |
Release share v0.52.0 (2024-01-06)v0.52.0
=== New features
* ssh/config: add method MarshalText and WriteTo
* lib/ssh: implement method Output on Client
* ssh/sftp: implement method MkdirAll on Client
* cmd/httpdfs: implement [libhttp.Server] with [memfs.MemFS]
=== Breaking changes
* ssh/config: refactoring the Config merge
* ssh/config: add parameter Config to NewSection
* lib/ssh: add parameter context to Execute method
* lib/time: remove UnixMicro and UnixMilli
* lib/io: removed, this package has been merged into "lib/os"
* lib/parser: removed, this package has been merged into lib/strings
=== Bug fixes
* ssh/config: fix setting the default values
* ssh/config: set the Hostname if its not set on [Config.Get]
* http/sseclient: fix data race on [Client.Close]
* http/sseclient: fix Retry value not set to millisecond
* ssh/sftp: fix Stat on empty remote file name
* ssh/sftp: fix non-nil returned error on Close
=== Enhancements
* ssh/config: merge the Section slice values on [Section.merge]
* ssh/config: set the default UserKnownHostsFile in setDefaults
| -rw-r--r-- | CHANGELOG.adoc | 1319 | ||||
| -rw-r--r-- | _doc/CHANGELOG_2023.adoc | 1334 | ||||
| -rw-r--r-- | _doc/index.adoc | 3 | ||||
| -rw-r--r-- | share.go | 2 |
4 files changed, 1429 insertions, 1229 deletions
diff --git a/CHANGELOG.adoc b/CHANGELOG.adoc index 92dae827..c0f95b05 100644 --- a/CHANGELOG.adoc +++ b/CHANGELOG.adoc @@ -1,11 +1,12 @@ -= CHANGELOG -Shulhan <ms@kilabit.info> += CHANGELOG :toc: :sectanchors: -:sectlinks: This library is released every month, usually at the first week of month. +link:CHANGELOG_2023.html[Changelog in 2023^]. +This is changelog for share module since v0.43.0 until v0.51.0. + link:CHANGELOG_2022.html[Changelog in 2022^]. This is changelog for share module since v0.33.0 until v0.42.0. @@ -18,1317 +19,179 @@ This is changelog for share module since v0.12.0 until v0.21.0. link:CHANGELOG_2018-2019.html[Changelog from 2018 to 2019^]. This is changelog for share module since v0.1.0 until v0.11.0. +[#v0_52_0] +== share v0.52.0 (2024-01-06) -[#v0_51_0] -== share v0.51.0 (2023-12-06) - -[#v0_51_0__new_features] -=== New features - -lib/http: implement Server-Sent Events (SSE):: -+ --- -For server SSE, we add new type SSEEndpoint, for registering endpoint -that can handle SSE. - -For client SSE, we add it in new sub package "sseclient". --- - -lib/net: add method Read:: -+ --- -The Read method read packet from raw connection. - -If the conn parameter is nil it will return [net.ErrClosed]. - -The bufsize parameter set the size of buffer for each read operation, -default to 1024 if not set or invalid (less than 0 or greater than -65535). - -The timeout parameter set how long to wait for data before considering -it as failed. -If its not set, less or equal to 0, it will wait forever. -If no data received and timeout is set, it will return [ErrReadTimeout]. - -If there is data received and connection closed at the same time, it will -return the data first without error. -The subsequent Read will return empty packet with [ErrClosed]. --- - -lib/crypto: add support for reading passphrase using SSH_ASKPASS:: -+ --- -If the library failed to changes os.Stdin to raw, it will try to use -a program defined in SSH_ASKPASS environment variable. - -The SSH_ASKPASS is controlled by environment SSH_ASKPASS_REQUIRE. - -- If SSH_ASKPASS_REQUIRE is empty the passphrase will read from - terminal first, if not possible then using SSH_ASKPASS program. - -- If SSH_ASKPASS_REQUIRE is set to "never" the passphrase will read - from terminal only. - -- If SSH_ASKPASS_REQUIRE is set to "prefer", the passphrase will read - using SSH_ASKPASS program not from terminal, but require - DISPLAY environment to be set. - -- If SSH_ASKPASS_REQUIRE is set to "force", the passphrase will read - using SSH_ASKPASS program not from terminal, without checking DISPLAY - environment. --- - -This changes affect the [ssh.NewClientInteractive] indirectly. - -lib/memfs: add method JSON to Node:: -+ --- -The JSON method encode the Node into JSON. -This method provides an alternative to MarshalJSON with granular options, -depth and withoutModTime. - -The depth set the level of childs to be encoded. -The depth=0 only encode the Root Node itself (with its childs), depth=1 -encode the Root node and its subdirectories, and so on. - -If withoutModTime is set to true, all of the node ModTime will not be -included in output. --- - - -[#v0_51_0__enhancements] -=== Enhancements - -lib/ini: create the file if its not exist on Open:: -+ -Previously, Open will return an error if file not exist. -This changes create the file if not exist. - -lib/net: use [time.Ticker] to loop in WaitAlive:: -+ --- -The Timeout field in net.Dialer sometimes does not have any effect. -In the for-loop, we expect that each Dial at least consume 100 ms -before returning an error. - -Turns out, on several occasion, the loop run too quickly, less than the -passed timeout. -This is probably because the listening address has not been -active yet and we run it in the same machine, the OS (or Go?) return -immediately without waiting for timeout. --- - -lib/memfs: add sub file system:: -+ --- -This implemented simplified version of merging multiple MemFS instances. -The sub file system (subfs) can be added to the parent MemFS instance -by calling new method "Merge". - - parent.Merge(other *MemfS) - -When Get method called, each subfs will be evaluated in order of Merge -called. - -This deprecated the function Merge. --- - -lib/memfs: improve the refresh method:: -+ --- -Instead of refreshing from the Root, find the directory that closes -to the requested path. - -While at it, simplify the returned error. --- - -lib/ini: append variable into section before any empty lines:: -+ --- -Previously, appending a new variable into section always create an empty -line before after the variable. -This changes fix this by appending variable only after non-empty lines. -While at it, add empty space before variable value. --- - -lib/memfs: stop all Watcher spawn by DirWatcher:: -+ --- -Previously, the Watcher goroutine will stopped only if the file is -being deleted. -If the DirWatcher stopped manually, by calling Stop method, the -Watcher goroutine will still running in the background. - -This changes record all spawned Watcher and stop it when files inside -a deleted directory or when Stop called. --- - -lib/memfs: remove embedding os.FileInfo and http.File:: -+ -The original idea of embedding those interface is to guard the Node -implementation to always follow all methods in the os.FileInfo and -http.File, in case there is additional methods in the future. -But, embedding interfaces make the struct store it as field "FileInfo" -and File with "<nil>" values, which add storage cost to Node. - -lib/memfs: refactoring, simplify DirWatcher logic:: -+ -In this changes, we simplify re-scanning the directory content. -Instead of looping on node Child to detect new or delete files, use map -to store previous Child first. -If node path exist in map then add it again as child, otherwise delete it. - -email/dkim: set ExpiredAt to MaxInt64 if value is greater than 12 digits:: -+ --- -According to RFC 6376, - - To avoid denial-of-service attacks, implementations MAY consider any - value longer than 12 digits to be infinite. - -Since ExpiredAt type is uint64, we cannot set it to infinite, so set it -to maximum value int64 (not maximum of uint64). --- - - -[#v0_50_1] -== share v0.50.1 (2023-11-05) - -This release bring many enhancements to "lib/memfs", a library for caching -file system in memory. - -[#v0_50_1__enhancements] -=== Enhancements - -lib/memfs: return nil in AddChild if file not exist:: -+ --- -In case the node being added is a symlink, and the symlink is broken, -the AddChild will return an error "fs.ErrNotExist". - -In this changes we check for fs.ErrNotExist and return nil without an -error, so the user of AddChild can skip adding the node to the parent. --- - -lib/memfs: quote the path in the returned error:: -+ --- -Error message should at least contains 4W in the following order, - - WHEN WHO/WHERE WHAT WHY - -By quoting, user or machine can parse that the value inside quote -is the value that cause the error (the what), not part of the cause of -the error (the why). --- - -lib/memfs: add method Child to Node:: -+ -The Child method return the child node based on its node. - -lib/memfs: call the Init method in the embedded file:: -+ --- -By calling the Init method, the regular expression for include and -excludes are initialized, which allow using any method of MemFS instance -later, for example the Watch method. - -While at it, fix the fields alignment in template code format. --- - -lib/memfs: include empty directory:: -+ --- -Even thought empty directory does not contains file, from the parent -node _it is_ part of their content. - -Also, there is a use case where memfs use as virtual file system (VFS), -as a layer with file system, where user can view list of directory, -create a directory or file on the fly. -If we skip scanning empty directory, that directory will not be visible. --- - -lib/memfs: re-scan directory content on Node's Update:: -+ -Previously, only node content get updated. -In case in the MemFS, user set "Options.TryDirect" to true, the directory -content should be updated too on "MemFS.Get". - - -[#v0_50_0] -== share v0.50.0 (2023-10-04) - -This release bring many enhancements thanks to linters like revive [1], -fieldaligment [2], and shadow [3]. - -This release also replace "math/rand.Seed" with "crypto/rand". -Since Go 1.20 the "math/rand.Seed" is considered deprecated (the initial -value of rand is seeded automatically, not zero). -Now, it is the time to replace "math/rand" with more secure random number -generator, from "crypto/rand". -This changes affect tests in package "lib/email", "lib/http", and -"lib/stmp". - -[#v0_50_0__breaking_changes] -=== Breaking changes - -lib/test: refactoring, rename TestWriter to BufferWriter:: -+ --- -The name TestWriter is considered stutter if its called from external -package, test.TestWriter. - -While at it, implement the Error and Errorf in BufferWriter and add -comment to each exported methods. --- - -ssh/sftp: rename method Close to CloseFile:: -+ -Since the method accept FileHandle, that is returned from OpenFile, -then the method should changed with the same suffix. -This is also to make the method un-ambiguous later when we add Close method -to close client connection. - -[#v0_50_0__new_features] +[#v0_52_0__new_features] === New features -lib/errors: implement method Is:: -+ -The Is method will return true if the target error is instance of *E -and the value of field Code and Name match with values in e. - -lib/email: add method ID to Header:: -+ -The ID method return the Message-ID or empty if its not exist. - -test/mock: implement mock for testing io.ReadWriter or io.StringWriter:: -+ --- -The type ReadWriter provide two buffers, BufRead and BufWrite. -The BufRead is for io.Reader and BufWrite for io.Writer or -io.StringWriter. - -The type provide one method helper, Reset, to reset all buffers. --- - -lib/crypto: add function LoadPrivateKeyInteractive:: +ssh/config: add method MarshalText and WriteTo:: + -- -The LoadPrivateKeyInteractive load the private key from file. -If the private key file is encrypted, it will prompt for the passphrase -from terminal. +The MarshalText method encode the Section back to ssh_config format +with two spaces as indentation in key. -This function is taken from package "lib/ssh" with modification by adding -parameter "io.ReadWriter" and removing parameter maxAttempt. +The WriteTo method marshal the Section into text and write it to +[io.Writer] w. -- -lib/crypto: implement RSA encrypt and decryption for large message size:: -+ --- -The EncryptOaep extend the "rsa.EncryptOAEP" to make it able to encrypt a -message larger than its than (public modulus size - 2*hash.Size - 2). - -The function signature is the same with "rsa.EncryptOAEP" except the -name, to make it distinguishable. - -The DecryptOaep function extend the "rsa.DecryptOAEP" to make it able to -decrypt a message larger than its public modulus size. --- - -ssh/sftp: add method to close client connection:: -+ -The Close method close the client sftp session and release all its -resources. - -lib/ssh: add method Close to Client:: -+ -The Close method close the client connection and release all resources. - - -[#v0_50_0__enhancements] -=== Enhancements - -_AUR: include go-mod-tip.sh to package:: -+ -The "go-mod-tip.sh" is a shell script to get and print the latest Go module -version based on the latest tag and the latest commit hash from current -working git directory. - -lib/crypto: rewrite LoadPrivateKey as wrapper of ssh.ParseRawPrivate:: -+ --- -Previously, the LoadPrivateKey function only able to load private key -with PKCS#1 format. - -This changes make the function as a wrapper for ssh.ParseRawPrivate -that can load RSA, DSA, ECDSA, and Ed25519 in PKCS#1, PKCS#8, OpenSSL, -and OpenSSH formats. --- - -lib/ssh: store and use the connected identity file for ScpPut and ScpGet:: -+ -Previously, the ScpPut and ScpGet use the first IdentityFile in section -to use in "-i" argument. -This is not correct. -If the section contains two or more IdentityFile and the one that can -connect are the last one, the ScpPut and ScpGet method may return an -error when executing the "scp" command. - -[1] https://github.com/mgechev/revive - -[2] https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/fieldalignment - -[3] https://pkg.go.dev/golang.org/x/tools@v0.13.0/go/analysis/passes/shadow - - -[#v0_49_1] -== share v0.49.1 (2023-09-02) - -[#v0_49_1__bug_fixes] -=== Bug fixes - -lib/dns: fix leaking internal zone:: +lib/ssh: implement method Output on Client:: + -- -Previously, if the server have internal zone "my.internal" and the client -query "sub.my.internal" that does not exist in the zone, the server then -forward the query to parent name server. -This cause the internal zone and its domains leaked to parent name server. - -This changes fix this issue by checking if the query is subset of -internal zone Origin if domain does not exist, and response with error -code 3 (ERR_NAME) with the Zone SOA in Authority. +The Output method run the command and return its standard output and +error as is. +Any other error beside standard error, like connection, will be returned +as error. -- -[#v0_49_1__enhancements] -=== Enhancements - -lib/dns: update the SOA Serial when record added or removed from Zone:: -+ -Any call to Zone Add or Remove methods will update the Zone.SOA.Serial to -current epoch. - -lib/dns: add method AddAuthority to Message:: +ssh/sftp: implement method MkdirAll on Client:: + -- -The AddAuthority add the rr to list of Authority. -Calling this method mark the message as answer, instead of query. - -If the rr is SOA, it will replace the existing record if exist and set -the flag authoritative answer (IsAA) in header to true. -If the rr is NS, it will be added only if its not exist. - -It will return an error if the rr type is not SOA or NS or the size of -records in Authority is full, maximum four records. +The MkdirAll create directory on the server, from left to right. +Each directory is separated by '/', where the left part is the parent of +the right part. +This method is similar to [os.MkdirAll]. -- -lib/dns: add method to populate internal caches by Zone:: -+ -The InternalPopulateZone populate the internal caches from Zone's -messages. - -ssh/config: handle key UserKnownHostsFile:: -+ -The UserKnownHostsFile define list of the known_hosts files to be read, -separated by spaces. -If not set default to "~/.ssh/known_hosts" and "~/.ssh/known_hosts2". - - -lib/ssh: use UserKnownHostFile from configuration in NewClientInteractive:: +cmd/httpdfs: implement [libhttp.Server] with [memfs.MemFS]:: + -- -Previously, the ssh Client always use InsecureIgnoreHostKey in -HostKeyCallback. -This may post security issue, like man-in-the-middle attack, since we -did not check the server host key with one of key that known by client -from UserKnownHostFile (for example ~/.ssh/known_hosts). - -This changes use the SSH section UserKnownHostFile from configuration -(default to ~/.ssh/known_hosts) to check if the server host key is -valid. -The NewClientInteractive will return an error, "key is unknown", if host -key not exist in UserKnownHostFile or "key is mismatch" if host key -not match with one registered in UserKnownHostFile. - -This changes depends on patch of golang.org/x/crypto [1] that has not -reviewed yet, so we need to replace it with one that contains the patch. - -[1] https://go-review.googlesource.com/c/crypto/+/523555 +The httpdfs is a program to serve a directory under HTTP. -- - -[#v0_49_0] -== share v0.49.0 (2023-08-04) - -[#v0_49_0__breaking_changes] +[#v0_52_0__breaking_changes] === Breaking changes -lib/email: refactoring, replace field with type []byte to string:: -+ -Using string provide safety, guaranteed that if we pass it as parameter -the receiver will not be able to modify its content. - -ssh/config: refactoring, simplify the Section fields:: +ssh/config: refactoring the Config merge:: + -- -Instead of storing each Section value in separate field, store them -inside a map, Field. -This reduce the size of Section and simplify adding or getting the -key that we are not supported but maybe usable by user in the future. +This changes rename method [Config.Prepend] to [Config.Merge]. -This changes introduce several new methods as replacement of field: +The way that how the other Config merged is changed. +Instead of appending all of other's sections into the current Config, +append the other Config instance to the current instance of Config. -* CASignatureAlgorithms: a method that return list of signature - algorithms that Section set or the default -* CanonicalDomains: a method that return CanonicalDomains set in Section -* CanonicalizePermittedCNames: return the permitted CNAMEs set in Section, - from KeyCanonicalizePermittedCNames. -* CertificateFile: return list of certificate file -* Environments: return system and/or custom environment that will be - passed to remote machine. - The key and value is derived from "SendEnv" and "SetEnv". -* FieldBool: return field value as boolean -* FieldInt: return the field value as int -* Hostname: return the Hostname in this Section -* IdentityAgent: return the path to SSH agent socket to be used -* Port: return the remote machine port -* User: return the remote user name -* Set: set the Field using key and value +During [Config.Get] the top Config will be evaluated first, and then the +other Config is evaluated in order of Merge. -- -lib/ssh: refactoring NewClientFromConfig, renamed to NewClientInteractive:: +ssh/config: add parameter Config to NewSection:: + -- -Previously, the NewClientInteractive blindly use the signers from -Section.Signers. -If one of the IdentityFile valid, it will add all the keys in -IdentityFile to SSH agent. - -In this changes we try each IdentityFile independently. -If the key is valid, client connected to remote machine, then only that -key will be added to SSH agent. +This changes how the Section and parser initialized. -While at it we also rename the method to NewClientInteractive to -indicate that the function will prompt for passphrase if one of the -IdentityFile is encrypted. +Previously, the Config depends on the parser to set the workDir and +homeDir and Section depends on Config only on Get; now its the other +way around, from top to bottom. +Config initialized first, then parser initialized using Config instance, +and then Section initialized also using Config instance. -- -[#v0_49_0__bug_fixes] -=== Bug fixes - -lib/http: fix missing query when handling redirect in HandleFS:: -+ -In 06e6cbdd511c, we redirect request by adding end slash to the path -if the requested resource is directory, but somehow we miss adding the -original request query. -This changes fix this issue. - -lib/os: check for extract path in untar and unzip:: +lib/ssh: add parameter context to Execute method:: + -- -Once we Join-ed the directory output with the file name, we check if -the result of join is still under directory output, if its not, return -an error to prevent -https://cwe.mitre.org/data/definitions/22.html[Zip Slip vulnerability]. +This changes require the fork of our golang.org/x/crypto. -- -lib/smtp: format the passed data in NewMailTx:: +lib/time: remove UnixMicro and UnixMilli:: + -- -The following rules are applied to the data, - -* all lines must end with CRLF -* if the line start with period, additional period is inserted before - the line. This recommendation based on RFC 5321 section 4.5.2 [1] to - prevent data that contains CRLF "." CRLF does not corrupt the message, - causing the server terminate reading the message where it should not. - -[1] https://datatracker.ietf.org/doc/html/rfc5321#section-4.5.2 +Both of those methods has been added into standard library as +[Time.UnixMicro] and [Time.UnixMilli] since Go 1.17. -- -[#v0_49_0__new_features] -=== New features - -lib/os: add function PathFold and PathUnfold:: +lib/io: removed, this package has been merged into "lib/os":: + -- -The PathFold replace the path "in" with tilde "~" if its prefix match -with user's home directory from os.UserHomeDir. - -The PathUnfold expand the tilde "~/" prefix into user's home directory -using os.UserHomeDir and environment variables using os.ExpandEnv -inside the string path "in". +While some functions are merged to "lib/os", some are not used anymore +like io.Reader. -- -lib/os: add function Environments:: -+ -The Environments function return list of system environment as map of -key and value. - -lib/ssh: add function LoadPrivateKeyInteractive:: -+ -The LoadPrivateKeyInteractive load private key from file. -If key is encrypted, it will prompt the passphrase in terminal with -maximum maxAttempt times. -If the passphrase still invalid after maxAttempt it will return an error. +lib/parser: removed, this package has been merged into lib/strings:: -[#v0_49_0__enhancements] -=== Enhancements - -lib/smtp: set minimum Server TLS to v1.2:: -+ -Using the TLS v1.1 is considered insecure and should not be used in -server anymore. - -lib/memfs: check for refresh URL outside of Root SysPath:: -+ -The case when refresh URL outside of Root SysPath is only exist when -the memfs TryDirect is set to true, usually during development. -In the production, the TryDirect should be false, hence the refresh -always return nil Node. - -ssh/config: reorder struct fields for better alignment:: -+ --- -Changes, - -* Config: changes allocated size from 32 to 16 bytes (-8 bytes) -* parser: changes allocated size from 40 to 32 bytes (-8 bytes) -* Section: changes allocated size from 392 to 360 bytes (-32 bytes) - -The rest of changes that are not mentioned are from test files. --- - - -[#v0_48_0] -== share v0.48.0 (2023-07-07) - -This release bring many enhancements to lib/websocket including timeout, -handling upgrade and read/write concurrently using goroutine. - -[#v0_48_0__breaking_changes] -=== Breaking changes - -lib/net: changes the WaitRead/Event model on Poll:: -+ --- -Previously, the Pool's WaitRead and WaitReadEVent methods return list of -file descriptor (fd) and keeps the fd in the pool. -In case we want to process the returned fd concurrently, by running it -in different goroutine, the next call WaitRead may return the same fd -if its goroutine not fast enough to read from fd. - -This changes fix this issue by removing list of fd from poll and set the -fd flag to blocking mode again after returning it from WaitRead or -WaitReadEvent. - -This changes also remove the ReregisterRead and ReregisterEvent methods -since it is not applicable anymore. --- -[#v0_48_0__bug_fixes] +[#v0_52_0__bug_fixes] === Bug fixes -lib/websocket: call Quit when handshake contains close or invalid frame:: +ssh/config: fix setting the default values:: + -If the HTTP handshake response contains trailing frame, handle it -directly. -If the frame is invalid or contains control close operation, call Quit -directly to trigger the HandleQuit if its defined by user. - -lib/websocket: revert maxBuffer back to 1024:: -+ -In v0.47.0 we increase the maxBuffer to 4096 to try increasing the -performance when handling large payload. -Turns out increasing this break the autobahn test suite. - - -[#v0_48_0__new_features] -=== New features - -lib/ascii: add type Set:: -+ --- -The Set type is a bitmap that represent list of ASCII characters for faster -lookup. - -A Set is a 36-byte value, where each bit in the first 32-bytes represents -the presence of a given ASCII character in the set. -The remaining 4-bytes is a counter for the number of ASCII characters in the -set. -The 128-bits of the first 16 bytes, starting with the least-significant bit of -the lowest word to the most-significant bit of the highest word, map to the -full range of all 128 ASCII characters. -The 128-bits of the next 16 bytes will be zeroed, ensuring that any non-ASCII -character will be reported as not in the set. -- - -lib/net: implement generic PollEvent:: -+ +The field default value should be set on Get, after all the Host or +Match fields merged. +In this way, if the field key already set, its not overridden by the +default value or subsequent Host or Match value. -- -The PollEvent contains file descriptor and the underlying event -based on OS, unix.EpollEvent on Linux or unix.Kevent_t on BSD. -The Poll interface provides two APIs to works with PollEvent, -WaitReadEvents that return list of PollEvent ready for read, and -ReregisterEvent to register the event back to poll (only for Linux). --- - -[#v0_48_0__enhancements] -=== Enhancements - -lib/websocket: add option to set read/write timeout on Server:: +ssh/config: set the Hostname if its not set on [Config.Get]:: + -- -The ReadWriteTimeout define the maximum duration the server wait when -receiving/sending packet from/to client before considering the -connection as broken. +Per manual ssh_config(5) on Hostname, -Default read-write timeout is 30 seconds if not set. +[quote] +The default is the name given on the command line. -This changes affect the exported function Send and Recv by adding -additional parameter timeout to both of them. +So, if the requested host name match with one of Host or Match, but +Hostname is not set, it should be default to the requested parameter +name. -- -lib/websocket: handle concurrent upgrade using goroutine:: +http/sseclient: fix data race on [Client.Close]:: + -- -The maxGoroutineUpgrader define maximum goroutines running at the same -time to handle client upgrade. -The new goroutine only dispatched when others are full, so it will -run incrementally not all at once. -Default to defServerMaxGoroutineUpgrader (128) if its not set. +The data race happened when Close set conn to nil but the consume +method still on Read. +The fix is by waiting for 100ms so consume goroutine can check if closeq +is triggered from Close or not. -- -lib/websocket: handle concurrent Server read using goroutines:: -+ -The Server now dispatch a goroutine to consume event from poll reader -for each client connection that is ready to read. -The maximum number of goroutine is defined in ServerOptions -maxGoroutineReader, which currently set to 1024. - -lib/websocket: handle concurrent ping using goroutines:: -+ -The maximum goroutines is quarter of max queue. -The new goroutine for pinger will be dispatched when no goroutine can -consume the current processed connection. - - -[#v0_48_0__chores] -=== Chores - -websocket/testdata: rewrite autobahn test using container:: +http/sseclient: fix Retry value not set to millisecond:: + -- -Since the autobahn script can only run on Python 2, it become hard to -setup and run the test on distro that does not provide Python 2 anymore. -The autobahn repository recommend to use docker instead. - -When testing the server, we simplify it by using make task "test-server". -The test-server task run our test server in background, and then run the -autobahn fuzzingclient from container. -Once the tests completed, we trigger the server to shutdown by sending -text frame with payload "shutdown". +When client receive "retry:" message, the value is in millisecond, but +when we store it we only convert it to [time.Duration] which default +to nanosecond. -When testing the client, we simplify it by using make task "test-client". -The test-client task run the autobahn fuzzingserver and then -we run our client. -Once client finished, we trigger the server to generate the reports -and cleanup the container. +While at it, update comments on field [Client.Retry] and +[Client.Insecure]. -- - -[#v0_47_0] -== share v0.47.0 (2023-06-05) - -[#v0_47_0__breaking_changes] -=== Breaking changes - -email/maildir: major refactoring plus adding unit tests:: -+ -This changes remove all unneeded methods from Manager leave it with -four methods: Delete, FetchNew, Incoming, OutgoingQueue. -+ -Also, we add the type filename to generate file name for tmp and new -directory. - -lib/email: unexport the field ContentType in the field:: -+ -The field ContentType will be set only when the field Name is -"Content-Type" so it's not always exist on each field. -To get the field ContentType, use Header.ContentType(). - - -[#v0_47_0__bug_fixes] -=== Bug fixes - -lib/dns: fix zone parsing on SOA record with single line:: -+ -Due to refactoring in c376eccd25, parsing SOA record with single line -return an error: "parseSOA: line 2: incomplete SOA statement '0'". - -lib/memfs: ignore permission error when scanning directory content:: -+ -Instead of returning error, skip the directory that we cannot read and -continue to process the other. - -lib/memfs: fix panic when watched file deleted or renamed:: -+ -When the file being watched is deleted, sometimes it will cause panic. - -lib/email: fix parsing multiple parameters in ContentType:: -+ -While at it, also fix the ContentType String method to prefix ';' before -appending parameter key and value. - - -[#v0_47_0__new_features] -=== New features - -cmd/bcrypt: CLI to compare or generate hash using bcrypt:: -+ -The bcrypt command has two subcommand "compare" and "gen". -The "compare" subcommand accept two parameter the hash and plain text. -The "gen" subcommand accept only one parameter, the plain text to be hashed. - -lib/sql: add type DmlKind:: -+ -The DmlKind define the kind for Data Manipulation Language. - -email/maildir: implement Folder:: -+ -Folder is a directory under maildir that store messages per file. -A folder contains three directories: tmp, new, and cur; and an empty -file "maildirfolder". - -lib/net: add function WaitAlive:: -+ --- -WaitAlive try to connect to network at address until timeout reached. -If connection cannot established it will return an error. - -Unlike [net.DialTimeout], this function will retry not returning an error -immediately if the address has not ready yet. --- - -lib/smtp: implement Client SendEmail:: -+ --- -Somehow in 3a1a2715b25f, we include this method without implementing it. - -The SendEmail method simplify sending email by automatically create -[MailTx] for passing it to method Client.MailTx. - -The test right now use live connection since the Server is not ready yet. --- - -[#v0_47_0__enhancements] -=== Enhancements - -lib/dns: add option to set debug level in ServerOptions:: -+ -This options replace the global debug package. - -lib/dns: do not cache empty answers:: -+ -The use case if one use and switch between two different -networks with internal zone, frequently. -For example, if on network Y they have domain MY.Y and -current connection is X, request to MY.Y will return an -empty answers. -Once they connect to Y again, any request to MY.Y will not -be possible because rescached caches contains empty answer -for MY.Y. - -_bin/go-test-coverhtml: add parameter to run specific test:: -+ -The second parameter is optional. -It is passed to -run= argument in "go test". -Default value is ".", or all functions. - -lib/http: redirect path with slash if request is directory:: +ssh/sftp: fix Stat on empty remote file name:: + -- -If request path is a directory and it is not end with slash, redirect -request to location with slash to allow relative links works inside the -HTML content. +The implementation of SSH server (openssh) for Stat is not consistent with +the RFC. +The RFC mentioned that -For example, a "/page/index.html" contains links href="sub.html" (where -"sub.html" is inside "/page" directory). -If request to "/page" (without end with slash) return content of -"/page/index.html", then when user click on sub.html it will request to -"/sub.html" instead of "/page/sub.html". --- +[quote] +An empty path name is valid, and it refers to the user's default +directory (usually the user's home directory). -lib/email: handle obsolete white spaces and comment when unpacking date:: -+ +But this only working on some command, like Mkdir, but not Stat. -- -In the obsolete syntax, white space and comments can appear between many -more element, for example the folloing Date value are valid - - Date : Fri, 21 Nov 1997 09(comment): 55 : 06 -0600 -This changes handle this by sanitizing the Field value, removing comment -and merge multiple spaces into one, before parsing it. --- - -lib/email: set the Field Type and unpack its value on ParseField:: +ssh/sftp: fix non-nil returned error on Close:: + -- -Once the field Name has detected and its Value is valid, we can unpack -the Value based to type that it represent, for example to Date or Mailbox. - -This changes remove calling to unpack directly in some tests and check -an error when testing ParseHeader. +This changes fix the Close that always return an error. -- -lib/net: increase the maximum poll events:: -+ -The maxQueue define the number of events that can be read from poll at -one time. -Using 128 seems to small for high throughput networks. -Increasing this number also increase the memory consumed by process. -Maybe later we can export this function as option when creating poll. - -lib/websocket: increase the max buffer and queue for better throughput:: -+ --- -The maxBuffer increased from 1024 to 4096 bytes. -The reason that we use 1024 previously is related to MTU size and maximum -payload in TCP (although its higher, 1460 bytes). - -The maxQueue increase from 128 to 4096. --- - -[#v0_47_0__chores] -=== Chores - -all: remove any usage of debug.Value in all packages:: -+ -Using global debug value for all packages turns out is not a good -idea. - -lib/test: update documentation related to Assert and Data:: -+ -The documentation is based on the article published at -https://kilabit.local/journal/2023/go_test_data/ -after reviewing and explain how to use both of them to public. - -all: record the contributors of this module in file AUTHORS:: - - -[#v0_46_0] -== share v0.46.0 (2023-05-02) -This release deprecated lib/io and lib/parser. - - -[#v0_46_0__breaking_changes] -=== Breaking changes - -lib/reflect: remove the third return value from Marshal:: -+ -The third, boolean, return value is redundant with the second error value. - -lib/bytes: changes the DumpPrettyTable output format:: -+ -The change is to accommodate large bytes data, more than 0xFFFF. -The hex address in the first column is increased to 8 digits, the -characters compacted without space in between. - - -[#v0_46_0__new_features] -=== New features - -lib/os: merge some functions from lib/io:: -+ -Functions like Copy, IsBinary, IsDirEmpty, IsFileExist, RmdirEmptyAll -are read and operate on file and directory on operating system level, so -it is not correct to put it in package io. - -lib/strings: merge lib/parser here:: -+ --- -The first idea of parser is to provide generic parser for both bytes and -string. -After we introduce lib/parser there is not much changes to that package. -Also, since we create another Parser in lib/bytes that accept and -return token as []byte, the lib/parser is not unique anymore. - -The following function/methods changes to minimize conflict in the future, - -* Lines become LinesOfFile -* New become NewParser -* Open become OpenForParser -* Token become Read -* TokenEscaped become ReadEscaped -* TokenTrimSpace become ReadNoSpace --- - -lib/bytes: implement function ParseHexDump:: -+ --- -The ParseHexDump parse the default output of [hexdump](1) utility from -parameter in back into stream of byte. - -An example of default output of hexdump is - - 0000000 7865 5f70 6964 2f72 0000 0000 0000 0000 - 0000010 0000 0000 0000 0000 0000 0000 0000 0000 - * - 0000060 0000 0000 3030 3030 3537 0035 3030 3130 - -The first column is the address and the rest of the column is the data. -Each data column is 16-bit words in big-endian order, so in the above -example, the first byte would be 65, second byte is 78 and so on. -The asterisk "*" means that the address from 0000020 to 0000050 is equal to -the previous line, 0000010. - -[hexdump]: https://man.archlinux.org/man/hexdump.1 --- - -lib/bytes: implement tokenize Parser:: -+ -The Parser type parse stream of byte using one or more delimiters as -separator between token. - -lib/bytes: add function TrimNull:: -+ -The TrimNull function remove 0 value ("\0" or NULL in C) at leading -and trailing of input. - -lib/net: add method WriteTo to ResolvConf:: -+ -The WriteTo method write the ResolvConf as text. - - -[#v0_46_0__enhancements] +[#v0_52_0__enhancements] === Enhancements -lib/time: calculate the next event before notify the user on Scheduler run:: -+ -This allow user to call the Next method, to know the next time the -scheduler will be triggered, after receiving the event. - -lib/reflect: add option to skip processing struct field in Do/IsEqual:: -+ -A struct's field tagged with `noequal:""`, its value will not be processed for -equality. - - -[#v0_46_0__chores] -=== Chores - -lib/reflect: use doEqual inside IsEqual:: -+ -Previously, IsEqual internally use isEqual, which have the same logic as -doEqual. -This changes minimize duplicate code between IsEqual and DoEqual, by -calling doEqual for both of functions. - -lib/time: replace lib/io#Reader with lib/bytes#Parser:: - -lib/smtp: replace lib/io#Reader with lib/bytes#Parser:: - -lib/dns: replace lib/io#Reader with lib/bytes#Parser:: - -lib/http: replace lib/io#Reader with lib/bytes#Parser:: - -lib/email: replace lib/io#Reader with lib/bytes#Parser:: - -email/dkim: replace lib/io#Reader with lib/bytes#Parser:: - -lib/hunspell: replace lib/io with lib/os:: - -lib/hunspell: replace lib/parser with lib/strings:: - -lib/http: replace lib/parser with lib/strings:: - -lib/bytes: copy TokenFind to internal/bytes#TokenFind:: +ssh/config: merge the Section slice values on [Section.merge]:: + -This is to prevent import cycle later when we use lib/test in bytes. - - -[#v0_45_0] -== share v0.45.0 (2023-04-01) - -This release set the Go version to 1.19. - -[#v0_45_0__breaking_changes] -=== Breaking changes - -lib/net: changes the PopulateQuery logic:: -+ --- -Previously, PopulateQuery only add the passed dname if the number of dots -is greater than 0. -After inspecting the result from dig and getenv, the dots seems does not -affect the query. -For example, if we have A record for domain "kilabit", both of those tools -query name "kilabit" without adding local domain or domain in search. -- +Instead of using [Section.Set], set the key-value directly. -[#v0_45_0__new_features] -=== New features - -_bin: add shell script go-test-lint.sh to run test and lint sequentially:: -+ --- -The go-test-lint.sh run Go test and if its success it will run -predefined linter, in the current directory. - -Arg 1: the method or function to test, default to ".". - -The linter program and its argument is derived from environment variable -GO_LINT. -If its empty, it will try the following linter in order: revive and then -golangci-lint. - -To add additional arguments to Go test set the environment variable -GO_TEST_ARGS. --- - -lib/bytes: add function DumpPrettyTable:: -+ -The DumpPrettyTable write each byte in slice data as hexadecimal, ASCII -character, and integer with 8 columns width. - -lib/bytes: add function SplitEach:: -+ -The SplitEach funciton split the slice of byte into n number of bytes. -If n is less or equal than zero, it will return the data as chunks. - -lib/dns: add function ParseZone:: -+ --- -The ParseZone parse the content of zone from raw bytes. - -Now that we have ParseZone, all tests that use zoneParser now can be -replaced using combination of test.Data and ParseZone. --- - -lib/dns: add method WriteTo to Zone:: -+ --- -The WriteTo method write the zone as text into io.Writer. - -The result of WriteTo will be different with original content of zone -file, since it does not preserve comment and indentation. --- - -lib/http: add function to parse multipart Range response for Client:: -+ -The ParseMultipartRange parse the multipart/byteranges body or response -from HTTP Range request. -Each Content-Range position and body part in the multipart will be stored -under RangePosition. - -lib/http: add support for HTTP Range in Server:: -+ --- -For HTTP Server using HandleFS, the Range request is handled -automatically. -For other HTTP server, user can use the HandleRange function. - -The HandleRange function handle -https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests[HTTP Range] -request using "bytes" unit. -The body parameter contains the content of resource being requested that -accept Seek method. - -If the Request method is not GET, or no Range in header request it will -return all the body -https://datatracker.ietf.org/doc/html/rfc7233#section-3.1[RFC7233 S-3.1]. - -The contentType is optional, if its empty, it will detected by -http.ResponseWriter during Write. --- - -lib/io: add method ReplaceAll on Reader:: -+ -The ReplaceAll method behave like standard bytes.ReplaceAll but start -from current index. - -lib/parser: add method TokenTrimSpace:: -+ -The TokenTrimSpace read the next token until one of the delimiter found, -with leading and trailing spaces are ignored. - -lib/parser: add method SetDelimiters:: -+ -The SetDelimiters replace the current delimiters. - -lib/telemetry: package for collecting and forwarding metrics:: -+ -Package telemetry is a library for collecting various Metric, for example -from standard runtime/metrics, and send or write it to one or more -Forwarder. -Each Forwarder has capability to format the Metric before sending or -writing it using Formatter. - - -[#v0_45_0__bug_Fixes] -=== Bug fixes - -lib/dns: fix packing, parsing, and saving MINFO resource data:: -+ --- -Even thought the MINFO record not formally obsolete, according to -https://en.wikipedia.org/wiki/List_of_DNS_record_types#Obsolete_record_types[Wikipedia], -we still need to support this for backward compatibility. - -When packing the resource data length does not include total length. -When parsing, the RMailBox and EmailBox should be added the origin suffix -if its not end with dot. -When saving, the origin should be trimmed from RMailBox and EmailBox. --- - -lib/dns: fix packing and unpacking resource record HINFO:: -+ --- -The rdata for HINFO contains two character-strings: CPU and OS. -Previously, we pack the rdata section sequentially, without adding length -on each of them: <RDLEN><CPU><OS>. -The correct pack format should <RDLEN><LENGTH><CPU><LENGTH><OS>. --- - -lib/dns: fix parsing SRV record from zone file:: -+ --- -Previous parseSRV start by parsing the _Service from tok, but the -actual value of parameter tok is the Priority. - -This changes fix this and as testing we use the example from RFC 2782. --- - -[#v0_45_0__enhancements] -=== Enhancements - -lib/dns: allow parsing TXT rdata without quote in zone file:: -+ --- -Previously, the zone only parsing TXT record with double quote since -most of the example that we found during implementation all use double -quote. - -This changes allow non-double quoted text in zone file with consequence -that any spaces will terminated the rdata immediately. - -Fixes #6 +While at it, merge the certificateFile, IdentityFile, knownHostFiles, +and sendEnv. -- -lib/dns: handle zone file with CRLF line ending:: +ssh/config: set the default UserKnownHostsFile in setDefaults:: + -- -While at it, fix parsing multiline SOA record where closing parentheses -end on next lines. - -Fixes #6 +While at it, unfold each value of IdentityFile and UserKnownHostsFile +in setDefaults, by expanding "~" into user's home directory or joining +with "config" directory if its relative. -- - - - - -lib/test: simplify the string diff output from Assert:: -+ --- -In the output, instead of using %q we replace it with %s, because printing -string with double quote cause escaping and hard to read -This change may cause difference in white spaces not showed in the -terminal. - -In the diff changes, only print the Old and New, without printing each -chunk. --- - - -[#v0_44_0] -== share v0.44.0 (2023-03-02) - -[#v0_44_0__new_features] -=== New features - -lib/time: implement Scheduler:: -+ --- -Scheduler is a timer that run periodically based on calendar or day time. - -A schedule is divided into monthly, weekly, daily, hourly, and minutely. -An empty schedule is equal to minutely, a schedule that run every minute. --- - -lib/time: add new type Clock:: -+ -Clock represent 24 hours time with hour, minute, and second. -An hour value is from 0 to 23, a minute value is from 0 to 59, and -a second value is from 0 to 59. - -[#v0_44_0__bug_fixes] -=== Bug fixes - -lib/clise: fix potential data race between Push and Slice:: -+ -The data race may occur if Push is called, the .last field is incremented -and at the same time an other goroutine call Slice that access the .last -field. - -lib/memfs: minimize data race on DirWatcher:: -+ -Calling DirWatcher Stop while the start method set dw.ticker can cause -data race. This changes fix this issue. - -go.mod: update all dependencies:: -+ -This update use "go get all" which resolve to semver for each dependencies. - -[#v0_44_0__chores] -=== Chores - -all: set the test timeout to 1m:: -+ -Some flaky test, caused by waiting for channel, require waiting for 10m -before it considered fail. -This changes we cut the default timeout to 1 minute. - -all: access embedded field or methods using the type name:: -+ -This is for clarity in the code, better to be explicit by typing where -the field or methods come from. - -lib/email: add an example for Filter method on Header:: - -cmd: temporarily hide unused commands:: -+ -The cart, cascaded-random-forest, lnsmote, random-forest, and smote -are part of completing thesis and they never used anymore. - -AUR: add go-test-coverhtml.sh and go-bench.sh into package:: -+ -While at it, changes the go-bench count to 10. - -lib/websocket: replace math/rand.Read with crypto/rand.Read:: -+ -The math/rand will be deprecated in Go 1.20. - - -[#v0_43_0] -== share v0.43.0 (2023-02-07) - -[#v0_43_0__new_features] -=== New features - -lib/http: add function MarshalForm:: -+ --- -The MarshalForm marshal struct fields tagged with `form:` into url.Values. - -The rules for marshaling follow the same rules as in [UnmarshalForm]. - -It will return an error if the input is not pointer to or a struct. --- - -clise: implement io Closer, Writer, StringWriter, and ByteWriter:: - -clise: add method UnmarshalJSON:: -+ -The UnmarshalJSON convert JSON array into Clise. diff --git a/_doc/CHANGELOG_2023.adoc b/_doc/CHANGELOG_2023.adoc new file mode 100644 index 00000000..92dae827 --- /dev/null +++ b/_doc/CHANGELOG_2023.adoc @@ -0,0 +1,1334 @@ += CHANGELOG +Shulhan <ms@kilabit.info> +:toc: +:sectanchors: +:sectlinks: + +This library is released every month, usually at the first week of month. + +link:CHANGELOG_2022.html[Changelog in 2022^]. +This is changelog for share module since v0.33.0 until v0.42.0. + +link:CHANGELOG_2021.html[Changelog in 2021^]. +This is changelog for share module since v0.22.0 until v0.32.0. + +link:CHANGELOG_2020.html[Changelog in 2020^]. +This is changelog for share module since v0.12.0 until v0.21.0. + +link:CHANGELOG_2018-2019.html[Changelog from 2018 to 2019^]. +This is changelog for share module since v0.1.0 until v0.11.0. + + +[#v0_51_0] +== share v0.51.0 (2023-12-06) + +[#v0_51_0__new_features] +=== New features + +lib/http: implement Server-Sent Events (SSE):: ++ +-- +For server SSE, we add new type SSEEndpoint, for registering endpoint +that can handle SSE. + +For client SSE, we add it in new sub package "sseclient". +-- + +lib/net: add method Read:: ++ +-- +The Read method read packet from raw connection. + +If the conn parameter is nil it will return [net.ErrClosed]. + +The bufsize parameter set the size of buffer for each read operation, +default to 1024 if not set or invalid (less than 0 or greater than +65535). + +The timeout parameter set how long to wait for data before considering +it as failed. +If its not set, less or equal to 0, it will wait forever. +If no data received and timeout is set, it will return [ErrReadTimeout]. + +If there is data received and connection closed at the same time, it will +return the data first without error. +The subsequent Read will return empty packet with [ErrClosed]. +-- + +lib/crypto: add support for reading passphrase using SSH_ASKPASS:: ++ +-- +If the library failed to changes os.Stdin to raw, it will try to use +a program defined in SSH_ASKPASS environment variable. + +The SSH_ASKPASS is controlled by environment SSH_ASKPASS_REQUIRE. + +- If SSH_ASKPASS_REQUIRE is empty the passphrase will read from + terminal first, if not possible then using SSH_ASKPASS program. + +- If SSH_ASKPASS_REQUIRE is set to "never" the passphrase will read + from terminal only. + +- If SSH_ASKPASS_REQUIRE is set to "prefer", the passphrase will read + using SSH_ASKPASS program not from terminal, but require + DISPLAY environment to be set. + +- If SSH_ASKPASS_REQUIRE is set to "force", the passphrase will read + using SSH_ASKPASS program not from terminal, without checking DISPLAY + environment. +-- + +This changes affect the [ssh.NewClientInteractive] indirectly. + +lib/memfs: add method JSON to Node:: ++ +-- +The JSON method encode the Node into JSON. +This method provides an alternative to MarshalJSON with granular options, +depth and withoutModTime. + +The depth set the level of childs to be encoded. +The depth=0 only encode the Root Node itself (with its childs), depth=1 +encode the Root node and its subdirectories, and so on. + +If withoutModTime is set to true, all of the node ModTime will not be +included in output. +-- + + +[#v0_51_0__enhancements] +=== Enhancements + +lib/ini: create the file if its not exist on Open:: ++ +Previously, Open will return an error if file not exist. +This changes create the file if not exist. + +lib/net: use [time.Ticker] to loop in WaitAlive:: ++ +-- +The Timeout field in net.Dialer sometimes does not have any effect. +In the for-loop, we expect that each Dial at least consume 100 ms +before returning an error. + +Turns out, on several occasion, the loop run too quickly, less than the +passed timeout. +This is probably because the listening address has not been +active yet and we run it in the same machine, the OS (or Go?) return +immediately without waiting for timeout. +-- + +lib/memfs: add sub file system:: ++ +-- +This implemented simplified version of merging multiple MemFS instances. +The sub file system (subfs) can be added to the parent MemFS instance +by calling new method "Merge". + + parent.Merge(other *MemfS) + +When Get method called, each subfs will be evaluated in order of Merge +called. + +This deprecated the function Merge. +-- + +lib/memfs: improve the refresh method:: ++ +-- +Instead of refreshing from the Root, find the directory that closes +to the requested path. + +While at it, simplify the returned error. +-- + +lib/ini: append variable into section before any empty lines:: ++ +-- +Previously, appending a new variable into section always create an empty +line before after the variable. +This changes fix this by appending variable only after non-empty lines. +While at it, add empty space before variable value. +-- + +lib/memfs: stop all Watcher spawn by DirWatcher:: ++ +-- +Previously, the Watcher goroutine will stopped only if the file is +being deleted. +If the DirWatcher stopped manually, by calling Stop method, the +Watcher goroutine will still running in the background. + +This changes record all spawned Watcher and stop it when files inside +a deleted directory or when Stop called. +-- + +lib/memfs: remove embedding os.FileInfo and http.File:: ++ +The original idea of embedding those interface is to guard the Node +implementation to always follow all methods in the os.FileInfo and +http.File, in case there is additional methods in the future. +But, embedding interfaces make the struct store it as field "FileInfo" +and File with "<nil>" values, which add storage cost to Node. + +lib/memfs: refactoring, simplify DirWatcher logic:: ++ +In this changes, we simplify re-scanning the directory content. +Instead of looping on node Child to detect new or delete files, use map +to store previous Child first. +If node path exist in map then add it again as child, otherwise delete it. + +email/dkim: set ExpiredAt to MaxInt64 if value is greater than 12 digits:: ++ +-- +According to RFC 6376, + + To avoid denial-of-service attacks, implementations MAY consider any + value longer than 12 digits to be infinite. + +Since ExpiredAt type is uint64, we cannot set it to infinite, so set it +to maximum value int64 (not maximum of uint64). +-- + + +[#v0_50_1] +== share v0.50.1 (2023-11-05) + +This release bring many enhancements to "lib/memfs", a library for caching +file system in memory. + +[#v0_50_1__enhancements] +=== Enhancements + +lib/memfs: return nil in AddChild if file not exist:: ++ +-- +In case the node being added is a symlink, and the symlink is broken, +the AddChild will return an error "fs.ErrNotExist". + +In this changes we check for fs.ErrNotExist and return nil without an +error, so the user of AddChild can skip adding the node to the parent. +-- + +lib/memfs: quote the path in the returned error:: ++ +-- +Error message should at least contains 4W in the following order, + + WHEN WHO/WHERE WHAT WHY + +By quoting, user or machine can parse that the value inside quote +is the value that cause the error (the what), not part of the cause of +the error (the why). +-- + +lib/memfs: add method Child to Node:: ++ +The Child method return the child node based on its node. + +lib/memfs: call the Init method in the embedded file:: ++ +-- +By calling the Init method, the regular expression for include and +excludes are initialized, which allow using any method of MemFS instance +later, for example the Watch method. + +While at it, fix the fields alignment in template code format. +-- + +lib/memfs: include empty directory:: ++ +-- +Even thought empty directory does not contains file, from the parent +node _it is_ part of their content. + +Also, there is a use case where memfs use as virtual file system (VFS), +as a layer with file system, where user can view list of directory, +create a directory or file on the fly. +If we skip scanning empty directory, that directory will not be visible. +-- + +lib/memfs: re-scan directory content on Node's Update:: ++ +Previously, only node content get updated. +In case in the MemFS, user set "Options.TryDirect" to true, the directory +content should be updated too on "MemFS.Get". + + +[#v0_50_0] +== share v0.50.0 (2023-10-04) + +This release bring many enhancements thanks to linters like revive [1], +fieldaligment [2], and shadow [3]. + +This release also replace "math/rand.Seed" with "crypto/rand". +Since Go 1.20 the "math/rand.Seed" is considered deprecated (the initial +value of rand is seeded automatically, not zero). +Now, it is the time to replace "math/rand" with more secure random number +generator, from "crypto/rand". +This changes affect tests in package "lib/email", "lib/http", and +"lib/stmp". + +[#v0_50_0__breaking_changes] +=== Breaking changes + +lib/test: refactoring, rename TestWriter to BufferWriter:: ++ +-- +The name TestWriter is considered stutter if its called from external +package, test.TestWriter. + +While at it, implement the Error and Errorf in BufferWriter and add +comment to each exported methods. +-- + +ssh/sftp: rename method Close to CloseFile:: ++ +Since the method accept FileHandle, that is returned from OpenFile, +then the method should changed with the same suffix. +This is also to make the method un-ambiguous later when we add Close method +to close client connection. + +[#v0_50_0__new_features] +=== New features + +lib/errors: implement method Is:: ++ +The Is method will return true if the target error is instance of *E +and the value of field Code and Name match with values in e. + +lib/email: add method ID to Header:: ++ +The ID method return the Message-ID or empty if its not exist. + +test/mock: implement mock for testing io.ReadWriter or io.StringWriter:: ++ +-- +The type ReadWriter provide two buffers, BufRead and BufWrite. +The BufRead is for io.Reader and BufWrite for io.Writer or +io.StringWriter. + +The type provide one method helper, Reset, to reset all buffers. +-- + +lib/crypto: add function LoadPrivateKeyInteractive:: ++ +-- +The LoadPrivateKeyInteractive load the private key from file. +If the private key file is encrypted, it will prompt for the passphrase +from terminal. + +This function is taken from package "lib/ssh" with modification by adding +parameter "io.ReadWriter" and removing parameter maxAttempt. +-- + +lib/crypto: implement RSA encrypt and decryption for large message size:: ++ +-- +The EncryptOaep extend the "rsa.EncryptOAEP" to make it able to encrypt a +message larger than its than (public modulus size - 2*hash.Size - 2). + +The function signature is the same with "rsa.EncryptOAEP" except the +name, to make it distinguishable. + +The DecryptOaep function extend the "rsa.DecryptOAEP" to make it able to +decrypt a message larger than its public modulus size. +-- + +ssh/sftp: add method to close client connection:: ++ +The Close method close the client sftp session and release all its +resources. + +lib/ssh: add method Close to Client:: ++ +The Close method close the client connection and release all resources. + + +[#v0_50_0__enhancements] +=== Enhancements + +_AUR: include go-mod-tip.sh to package:: ++ +The "go-mod-tip.sh" is a shell script to get and print the latest Go module +version based on the latest tag and the latest commit hash from current +working git directory. + +lib/crypto: rewrite LoadPrivateKey as wrapper of ssh.ParseRawPrivate:: ++ +-- +Previously, the LoadPrivateKey function only able to load private key +with PKCS#1 format. + +This changes make the function as a wrapper for ssh.ParseRawPrivate +that can load RSA, DSA, ECDSA, and Ed25519 in PKCS#1, PKCS#8, OpenSSL, +and OpenSSH formats. +-- + +lib/ssh: store and use the connected identity file for ScpPut and ScpGet:: ++ +Previously, the ScpPut and ScpGet use the first IdentityFile in section +to use in "-i" argument. +This is not correct. +If the section contains two or more IdentityFile and the one that can +connect are the last one, the ScpPut and ScpGet method may return an +error when executing the "scp" command. + +[1] https://github.com/mgechev/revive + +[2] https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/fieldalignment + +[3] https://pkg.go.dev/golang.org/x/tools@v0.13.0/go/analysis/passes/shadow + + +[#v0_49_1] +== share v0.49.1 (2023-09-02) + +[#v0_49_1__bug_fixes] +=== Bug fixes + +lib/dns: fix leaking internal zone:: ++ +-- +Previously, if the server have internal zone "my.internal" and the client +query "sub.my.internal" that does not exist in the zone, the server then +forward the query to parent name server. +This cause the internal zone and its domains leaked to parent name server. + +This changes fix this issue by checking if the query is subset of +internal zone Origin if domain does not exist, and response with error +code 3 (ERR_NAME) with the Zone SOA in Authority. +-- + +[#v0_49_1__enhancements] +=== Enhancements + +lib/dns: update the SOA Serial when record added or removed from Zone:: ++ +Any call to Zone Add or Remove methods will update the Zone.SOA.Serial to +current epoch. + +lib/dns: add method AddAuthority to Message:: ++ +-- +The AddAuthority add the rr to list of Authority. +Calling this method mark the message as answer, instead of query. + +If the rr is SOA, it will replace the existing record if exist and set +the flag authoritative answer (IsAA) in header to true. +If the rr is NS, it will be added only if its not exist. + +It will return an error if the rr type is not SOA or NS or the size of +records in Authority is full, maximum four records. +-- + +lib/dns: add method to populate internal caches by Zone:: ++ +The InternalPopulateZone populate the internal caches from Zone's +messages. + +ssh/config: handle key UserKnownHostsFile:: ++ +The UserKnownHostsFile define list of the known_hosts files to be read, +separated by spaces. +If not set default to "~/.ssh/known_hosts" and "~/.ssh/known_hosts2". + + +lib/ssh: use UserKnownHostFile from configuration in NewClientInteractive:: ++ +-- +Previously, the ssh Client always use InsecureIgnoreHostKey in +HostKeyCallback. +This may post security issue, like man-in-the-middle attack, since we +did not check the server host key with one of key that known by client +from UserKnownHostFile (for example ~/.ssh/known_hosts). + +This changes use the SSH section UserKnownHostFile from configuration +(default to ~/.ssh/known_hosts) to check if the server host key is +valid. +The NewClientInteractive will return an error, "key is unknown", if host +key not exist in UserKnownHostFile or "key is mismatch" if host key +not match with one registered in UserKnownHostFile. + +This changes depends on patch of golang.org/x/crypto [1] that has not +reviewed yet, so we need to replace it with one that contains the patch. + +[1] https://go-review.googlesource.com/c/crypto/+/523555 +-- + + +[#v0_49_0] +== share v0.49.0 (2023-08-04) + +[#v0_49_0__breaking_changes] +=== Breaking changes + +lib/email: refactoring, replace field with type []byte to string:: ++ +Using string provide safety, guaranteed that if we pass it as parameter +the receiver will not be able to modify its content. + +ssh/config: refactoring, simplify the Section fields:: ++ +-- +Instead of storing each Section value in separate field, store them +inside a map, Field. +This reduce the size of Section and simplify adding or getting the +key that we are not supported but maybe usable by user in the future. + +This changes introduce several new methods as replacement of field: + +* CASignatureAlgorithms: a method that return list of signature + algorithms that Section set or the default +* CanonicalDomains: a method that return CanonicalDomains set in Section +* CanonicalizePermittedCNames: return the permitted CNAMEs set in Section, + from KeyCanonicalizePermittedCNames. +* CertificateFile: return list of certificate file +* Environments: return system and/or custom environment that will be + passed to remote machine. + The key and value is derived from "SendEnv" and "SetEnv". +* FieldBool: return field value as boolean +* FieldInt: return the field value as int +* Hostname: return the Hostname in this Section +* IdentityAgent: return the path to SSH agent socket to be used +* Port: return the remote machine port +* User: return the remote user name +* Set: set the Field using key and value +-- + +lib/ssh: refactoring NewClientFromConfig, renamed to NewClientInteractive:: ++ +-- +Previously, the NewClientInteractive blindly use the signers from +Section.Signers. +If one of the IdentityFile valid, it will add all the keys in +IdentityFile to SSH agent. + +In this changes we try each IdentityFile independently. +If the key is valid, client connected to remote machine, then only that +key will be added to SSH agent. + +While at it we also rename the method to NewClientInteractive to +indicate that the function will prompt for passphrase if one of the +IdentityFile is encrypted. +-- + +[#v0_49_0__bug_fixes] +=== Bug fixes + +lib/http: fix missing query when handling redirect in HandleFS:: ++ +In 06e6cbdd511c, we redirect request by adding end slash to the path +if the requested resource is directory, but somehow we miss adding the +original request query. +This changes fix this issue. + +lib/os: check for extract path in untar and unzip:: ++ +-- +Once we Join-ed the directory output with the file name, we check if +the result of join is still under directory output, if its not, return +an error to prevent +https://cwe.mitre.org/data/definitions/22.html[Zip Slip vulnerability]. +-- + +lib/smtp: format the passed data in NewMailTx:: ++ +-- +The following rules are applied to the data, + +* all lines must end with CRLF +* if the line start with period, additional period is inserted before + the line. This recommendation based on RFC 5321 section 4.5.2 [1] to + prevent data that contains CRLF "." CRLF does not corrupt the message, + causing the server terminate reading the message where it should not. + +[1] https://datatracker.ietf.org/doc/html/rfc5321#section-4.5.2 +-- + +[#v0_49_0__new_features] +=== New features + +lib/os: add function PathFold and PathUnfold:: ++ +-- +The PathFold replace the path "in" with tilde "~" if its prefix match +with user's home directory from os.UserHomeDir. + +The PathUnfold expand the tilde "~/" prefix into user's home directory +using os.UserHomeDir and environment variables using os.ExpandEnv +inside the string path "in". +-- + +lib/os: add function Environments:: ++ +The Environments function return list of system environment as map of +key and value. + +lib/ssh: add function LoadPrivateKeyInteractive:: ++ +The LoadPrivateKeyInteractive load private key from file. +If key is encrypted, it will prompt the passphrase in terminal with +maximum maxAttempt times. +If the passphrase still invalid after maxAttempt it will return an error. + +[#v0_49_0__enhancements] +=== Enhancements + +lib/smtp: set minimum Server TLS to v1.2:: ++ +Using the TLS v1.1 is considered insecure and should not be used in +server anymore. + +lib/memfs: check for refresh URL outside of Root SysPath:: ++ +The case when refresh URL outside of Root SysPath is only exist when +the memfs TryDirect is set to true, usually during development. +In the production, the TryDirect should be false, hence the refresh +always return nil Node. + +ssh/config: reorder struct fields for better alignment:: ++ +-- +Changes, + +* Config: changes allocated size from 32 to 16 bytes (-8 bytes) +* parser: changes allocated size from 40 to 32 bytes (-8 bytes) +* Section: changes allocated size from 392 to 360 bytes (-32 bytes) + +The rest of changes that are not mentioned are from test files. +-- + + +[#v0_48_0] +== share v0.48.0 (2023-07-07) + +This release bring many enhancements to lib/websocket including timeout, +handling upgrade and read/write concurrently using goroutine. + +[#v0_48_0__breaking_changes] +=== Breaking changes + +lib/net: changes the WaitRead/Event model on Poll:: ++ +-- +Previously, the Pool's WaitRead and WaitReadEVent methods return list of +file descriptor (fd) and keeps the fd in the pool. +In case we want to process the returned fd concurrently, by running it +in different goroutine, the next call WaitRead may return the same fd +if its goroutine not fast enough to read from fd. + +This changes fix this issue by removing list of fd from poll and set the +fd flag to blocking mode again after returning it from WaitRead or +WaitReadEvent. + +This changes also remove the ReregisterRead and ReregisterEvent methods +since it is not applicable anymore. +-- + +[#v0_48_0__bug_fixes] +=== Bug fixes + +lib/websocket: call Quit when handshake contains close or invalid frame:: ++ +If the HTTP handshake response contains trailing frame, handle it +directly. +If the frame is invalid or contains control close operation, call Quit +directly to trigger the HandleQuit if its defined by user. + +lib/websocket: revert maxBuffer back to 1024:: ++ +In v0.47.0 we increase the maxBuffer to 4096 to try increasing the +performance when handling large payload. +Turns out increasing this break the autobahn test suite. + + +[#v0_48_0__new_features] +=== New features + +lib/ascii: add type Set:: ++ +-- +The Set type is a bitmap that represent list of ASCII characters for faster +lookup. + +A Set is a 36-byte value, where each bit in the first 32-bytes represents +the presence of a given ASCII character in the set. +The remaining 4-bytes is a counter for the number of ASCII characters in the +set. +The 128-bits of the first 16 bytes, starting with the least-significant bit of +the lowest word to the most-significant bit of the highest word, map to the +full range of all 128 ASCII characters. +The 128-bits of the next 16 bytes will be zeroed, ensuring that any non-ASCII +character will be reported as not in the set. +-- + +lib/net: implement generic PollEvent:: ++ +-- +The PollEvent contains file descriptor and the underlying event +based on OS, unix.EpollEvent on Linux or unix.Kevent_t on BSD. + +The Poll interface provides two APIs to works with PollEvent, +WaitReadEvents that return list of PollEvent ready for read, and +ReregisterEvent to register the event back to poll (only for Linux). +-- + +[#v0_48_0__enhancements] +=== Enhancements + +lib/websocket: add option to set read/write timeout on Server:: ++ +-- +The ReadWriteTimeout define the maximum duration the server wait when +receiving/sending packet from/to client before considering the +connection as broken. + +Default read-write timeout is 30 seconds if not set. + +This changes affect the exported function Send and Recv by adding +additional parameter timeout to both of them. +-- + +lib/websocket: handle concurrent upgrade using goroutine:: ++ +-- +The maxGoroutineUpgrader define maximum goroutines running at the same +time to handle client upgrade. +The new goroutine only dispatched when others are full, so it will +run incrementally not all at once. +Default to defServerMaxGoroutineUpgrader (128) if its not set. +-- + +lib/websocket: handle concurrent Server read using goroutines:: ++ +The Server now dispatch a goroutine to consume event from poll reader +for each client connection that is ready to read. +The maximum number of goroutine is defined in ServerOptions +maxGoroutineReader, which currently set to 1024. + +lib/websocket: handle concurrent ping using goroutines:: ++ +The maximum goroutines is quarter of max queue. +The new goroutine for pinger will be dispatched when no goroutine can +consume the current processed connection. + + +[#v0_48_0__chores] +=== Chores + +websocket/testdata: rewrite autobahn test using container:: ++ +-- +Since the autobahn script can only run on Python 2, it become hard to +setup and run the test on distro that does not provide Python 2 anymore. +The autobahn repository recommend to use docker instead. + +When testing the server, we simplify it by using make task "test-server". +The test-server task run our test server in background, and then run the +autobahn fuzzingclient from container. +Once the tests completed, we trigger the server to shutdown by sending +text frame with payload "shutdown". + +When testing the client, we simplify it by using make task "test-client". +The test-client task run the autobahn fuzzingserver and then +we run our client. +Once client finished, we trigger the server to generate the reports +and cleanup the container. +-- + + +[#v0_47_0] +== share v0.47.0 (2023-06-05) + +[#v0_47_0__breaking_changes] +=== Breaking changes + +email/maildir: major refactoring plus adding unit tests:: ++ +This changes remove all unneeded methods from Manager leave it with +four methods: Delete, FetchNew, Incoming, OutgoingQueue. ++ +Also, we add the type filename to generate file name for tmp and new +directory. + +lib/email: unexport the field ContentType in the field:: ++ +The field ContentType will be set only when the field Name is +"Content-Type" so it's not always exist on each field. +To get the field ContentType, use Header.ContentType(). + + +[#v0_47_0__bug_fixes] +=== Bug fixes + +lib/dns: fix zone parsing on SOA record with single line:: ++ +Due to refactoring in c376eccd25, parsing SOA record with single line +return an error: "parseSOA: line 2: incomplete SOA statement '0'". + +lib/memfs: ignore permission error when scanning directory content:: ++ +Instead of returning error, skip the directory that we cannot read and +continue to process the other. + +lib/memfs: fix panic when watched file deleted or renamed:: ++ +When the file being watched is deleted, sometimes it will cause panic. + +lib/email: fix parsing multiple parameters in ContentType:: ++ +While at it, also fix the ContentType String method to prefix ';' before +appending parameter key and value. + + +[#v0_47_0__new_features] +=== New features + +cmd/bcrypt: CLI to compare or generate hash using bcrypt:: ++ +The bcrypt command has two subcommand "compare" and "gen". +The "compare" subcommand accept two parameter the hash and plain text. +The "gen" subcommand accept only one parameter, the plain text to be hashed. + +lib/sql: add type DmlKind:: ++ +The DmlKind define the kind for Data Manipulation Language. + +email/maildir: implement Folder:: ++ +Folder is a directory under maildir that store messages per file. +A folder contains three directories: tmp, new, and cur; and an empty +file "maildirfolder". + +lib/net: add function WaitAlive:: ++ +-- +WaitAlive try to connect to network at address until timeout reached. +If connection cannot established it will return an error. + +Unlike [net.DialTimeout], this function will retry not returning an error +immediately if the address has not ready yet. +-- + +lib/smtp: implement Client SendEmail:: ++ +-- +Somehow in 3a1a2715b25f, we include this method without implementing it. + +The SendEmail method simplify sending email by automatically create +[MailTx] for passing it to method Client.MailTx. + +The test right now use live connection since the Server is not ready yet. +-- + +[#v0_47_0__enhancements] +=== Enhancements + +lib/dns: add option to set debug level in ServerOptions:: ++ +This options replace the global debug package. + +lib/dns: do not cache empty answers:: ++ +The use case if one use and switch between two different +networks with internal zone, frequently. +For example, if on network Y they have domain MY.Y and +current connection is X, request to MY.Y will return an +empty answers. +Once they connect to Y again, any request to MY.Y will not +be possible because rescached caches contains empty answer +for MY.Y. + +_bin/go-test-coverhtml: add parameter to run specific test:: ++ +The second parameter is optional. +It is passed to -run= argument in "go test". +Default value is ".", or all functions. + +lib/http: redirect path with slash if request is directory:: ++ +-- +If request path is a directory and it is not end with slash, redirect +request to location with slash to allow relative links works inside the +HTML content. + +For example, a "/page/index.html" contains links href="sub.html" (where +"sub.html" is inside "/page" directory). +If request to "/page" (without end with slash) return content of +"/page/index.html", then when user click on sub.html it will request to +"/sub.html" instead of "/page/sub.html". +-- + +lib/email: handle obsolete white spaces and comment when unpacking date:: ++ +-- +In the obsolete syntax, white space and comments can appear between many +more element, for example the folloing Date value are valid + + Date : Fri, 21 Nov 1997 09(comment): 55 : 06 -0600 + +This changes handle this by sanitizing the Field value, removing comment +and merge multiple spaces into one, before parsing it. +-- + +lib/email: set the Field Type and unpack its value on ParseField:: ++ +-- +Once the field Name has detected and its Value is valid, we can unpack +the Value based to type that it represent, for example to Date or Mailbox. + +This changes remove calling to unpack directly in some tests and check +an error when testing ParseHeader. +-- + +lib/net: increase the maximum poll events:: ++ +The maxQueue define the number of events that can be read from poll at +one time. +Using 128 seems to small for high throughput networks. +Increasing this number also increase the memory consumed by process. +Maybe later we can export this function as option when creating poll. + +lib/websocket: increase the max buffer and queue for better throughput:: ++ +-- +The maxBuffer increased from 1024 to 4096 bytes. +The reason that we use 1024 previously is related to MTU size and maximum +payload in TCP (although its higher, 1460 bytes). + +The maxQueue increase from 128 to 4096. +-- + +[#v0_47_0__chores] +=== Chores + +all: remove any usage of debug.Value in all packages:: ++ +Using global debug value for all packages turns out is not a good +idea. + +lib/test: update documentation related to Assert and Data:: ++ +The documentation is based on the article published at +https://kilabit.local/journal/2023/go_test_data/ +after reviewing and explain how to use both of them to public. + +all: record the contributors of this module in file AUTHORS:: + + +[#v0_46_0] +== share v0.46.0 (2023-05-02) + +This release deprecated lib/io and lib/parser. + + +[#v0_46_0__breaking_changes] +=== Breaking changes + +lib/reflect: remove the third return value from Marshal:: ++ +The third, boolean, return value is redundant with the second error value. + +lib/bytes: changes the DumpPrettyTable output format:: ++ +The change is to accommodate large bytes data, more than 0xFFFF. +The hex address in the first column is increased to 8 digits, the +characters compacted without space in between. + + +[#v0_46_0__new_features] +=== New features + +lib/os: merge some functions from lib/io:: ++ +Functions like Copy, IsBinary, IsDirEmpty, IsFileExist, RmdirEmptyAll +are read and operate on file and directory on operating system level, so +it is not correct to put it in package io. + +lib/strings: merge lib/parser here:: ++ +-- +The first idea of parser is to provide generic parser for both bytes and +string. +After we introduce lib/parser there is not much changes to that package. +Also, since we create another Parser in lib/bytes that accept and +return token as []byte, the lib/parser is not unique anymore. + +The following function/methods changes to minimize conflict in the future, + +* Lines become LinesOfFile +* New become NewParser +* Open become OpenForParser +* Token become Read +* TokenEscaped become ReadEscaped +* TokenTrimSpace become ReadNoSpace +-- + +lib/bytes: implement function ParseHexDump:: ++ +-- +The ParseHexDump parse the default output of [hexdump](1) utility from +parameter in back into stream of byte. + +An example of default output of hexdump is + + 0000000 7865 5f70 6964 2f72 0000 0000 0000 0000 + 0000010 0000 0000 0000 0000 0000 0000 0000 0000 + * + 0000060 0000 0000 3030 3030 3537 0035 3030 3130 + +The first column is the address and the rest of the column is the data. +Each data column is 16-bit words in big-endian order, so in the above +example, the first byte would be 65, second byte is 78 and so on. +The asterisk "*" means that the address from 0000020 to 0000050 is equal to +the previous line, 0000010. + +[hexdump]: https://man.archlinux.org/man/hexdump.1 +-- + +lib/bytes: implement tokenize Parser:: ++ +The Parser type parse stream of byte using one or more delimiters as +separator between token. + +lib/bytes: add function TrimNull:: ++ +The TrimNull function remove 0 value ("\0" or NULL in C) at leading +and trailing of input. + +lib/net: add method WriteTo to ResolvConf:: ++ +The WriteTo method write the ResolvConf as text. + + +[#v0_46_0__enhancements] +=== Enhancements + +lib/time: calculate the next event before notify the user on Scheduler run:: ++ +This allow user to call the Next method, to know the next time the +scheduler will be triggered, after receiving the event. + +lib/reflect: add option to skip processing struct field in Do/IsEqual:: ++ +A struct's field tagged with `noequal:""`, its value will not be processed for +equality. + + +[#v0_46_0__chores] +=== Chores + +lib/reflect: use doEqual inside IsEqual:: ++ +Previously, IsEqual internally use isEqual, which have the same logic as +doEqual. +This changes minimize duplicate code between IsEqual and DoEqual, by +calling doEqual for both of functions. + +lib/time: replace lib/io#Reader with lib/bytes#Parser:: + +lib/smtp: replace lib/io#Reader with lib/bytes#Parser:: + +lib/dns: replace lib/io#Reader with lib/bytes#Parser:: + +lib/http: replace lib/io#Reader with lib/bytes#Parser:: + +lib/email: replace lib/io#Reader with lib/bytes#Parser:: + +email/dkim: replace lib/io#Reader with lib/bytes#Parser:: + +lib/hunspell: replace lib/io with lib/os:: + +lib/hunspell: replace lib/parser with lib/strings:: + +lib/http: replace lib/parser with lib/strings:: + +lib/bytes: copy TokenFind to internal/bytes#TokenFind:: ++ +This is to prevent import cycle later when we use lib/test in bytes. + + +[#v0_45_0] +== share v0.45.0 (2023-04-01) + +This release set the Go version to 1.19. + +[#v0_45_0__breaking_changes] +=== Breaking changes + +lib/net: changes the PopulateQuery logic:: ++ +-- +Previously, PopulateQuery only add the passed dname if the number of dots +is greater than 0. +After inspecting the result from dig and getenv, the dots seems does not +affect the query. +For example, if we have A record for domain "kilabit", both of those tools +query name "kilabit" without adding local domain or domain in search. +-- + +[#v0_45_0__new_features] +=== New features + +_bin: add shell script go-test-lint.sh to run test and lint sequentially:: ++ +-- +The go-test-lint.sh run Go test and if its success it will run +predefined linter, in the current directory. + +Arg 1: the method or function to test, default to ".". + +The linter program and its argument is derived from environment variable +GO_LINT. +If its empty, it will try the following linter in order: revive and then +golangci-lint. + +To add additional arguments to Go test set the environment variable +GO_TEST_ARGS. +-- + +lib/bytes: add function DumpPrettyTable:: ++ +The DumpPrettyTable write each byte in slice data as hexadecimal, ASCII +character, and integer with 8 columns width. + +lib/bytes: add function SplitEach:: ++ +The SplitEach funciton split the slice of byte into n number of bytes. +If n is less or equal than zero, it will return the data as chunks. + +lib/dns: add function ParseZone:: ++ +-- +The ParseZone parse the content of zone from raw bytes. + +Now that we have ParseZone, all tests that use zoneParser now can be +replaced using combination of test.Data and ParseZone. +-- + +lib/dns: add method WriteTo to Zone:: ++ +-- +The WriteTo method write the zone as text into io.Writer. + +The result of WriteTo will be different with original content of zone +file, since it does not preserve comment and indentation. +-- + +lib/http: add function to parse multipart Range response for Client:: ++ +The ParseMultipartRange parse the multipart/byteranges body or response +from HTTP Range request. +Each Content-Range position and body part in the multipart will be stored +under RangePosition. + +lib/http: add support for HTTP Range in Server:: ++ +-- +For HTTP Server using HandleFS, the Range request is handled +automatically. +For other HTTP server, user can use the HandleRange function. + +The HandleRange function handle +https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests[HTTP Range] +request using "bytes" unit. +The body parameter contains the content of resource being requested that +accept Seek method. + +If the Request method is not GET, or no Range in header request it will +return all the body +https://datatracker.ietf.org/doc/html/rfc7233#section-3.1[RFC7233 S-3.1]. + +The contentType is optional, if its empty, it will detected by +http.ResponseWriter during Write. +-- + +lib/io: add method ReplaceAll on Reader:: ++ +The ReplaceAll method behave like standard bytes.ReplaceAll but start +from current index. + +lib/parser: add method TokenTrimSpace:: ++ +The TokenTrimSpace read the next token until one of the delimiter found, +with leading and trailing spaces are ignored. + +lib/parser: add method SetDelimiters:: ++ +The SetDelimiters replace the current delimiters. + +lib/telemetry: package for collecting and forwarding metrics:: ++ +Package telemetry is a library for collecting various Metric, for example +from standard runtime/metrics, and send or write it to one or more +Forwarder. +Each Forwarder has capability to format the Metric before sending or +writing it using Formatter. + + +[#v0_45_0__bug_Fixes] +=== Bug fixes + +lib/dns: fix packing, parsing, and saving MINFO resource data:: ++ +-- +Even thought the MINFO record not formally obsolete, according to +https://en.wikipedia.org/wiki/List_of_DNS_record_types#Obsolete_record_types[Wikipedia], +we still need to support this for backward compatibility. + +When packing the resource data length does not include total length. +When parsing, the RMailBox and EmailBox should be added the origin suffix +if its not end with dot. +When saving, the origin should be trimmed from RMailBox and EmailBox. +-- + +lib/dns: fix packing and unpacking resource record HINFO:: ++ +-- +The rdata for HINFO contains two character-strings: CPU and OS. +Previously, we pack the rdata section sequentially, without adding length +on each of them: <RDLEN><CPU><OS>. +The correct pack format should <RDLEN><LENGTH><CPU><LENGTH><OS>. +-- + +lib/dns: fix parsing SRV record from zone file:: ++ +-- +Previous parseSRV start by parsing the _Service from tok, but the +actual value of parameter tok is the Priority. + +This changes fix this and as testing we use the example from RFC 2782. +-- + +[#v0_45_0__enhancements] +=== Enhancements + +lib/dns: allow parsing TXT rdata without quote in zone file:: ++ +-- +Previously, the zone only parsing TXT record with double quote since +most of the example that we found during implementation all use double +quote. + +This changes allow non-double quoted text in zone file with consequence +that any spaces will terminated the rdata immediately. + +Fixes #6 +-- + +lib/dns: handle zone file with CRLF line ending:: ++ +-- +While at it, fix parsing multiline SOA record where closing parentheses +end on next lines. + +Fixes #6 +-- + + + + +lib/test: simplify the string diff output from Assert:: ++ +-- +In the output, instead of using %q we replace it with %s, because printing +string with double quote cause escaping and hard to read +This change may cause difference in white spaces not showed in the +terminal. + +In the diff changes, only print the Old and New, without printing each +chunk. +-- + + +[#v0_44_0] +== share v0.44.0 (2023-03-02) + +[#v0_44_0__new_features] +=== New features + +lib/time: implement Scheduler:: ++ +-- +Scheduler is a timer that run periodically based on calendar or day time. + +A schedule is divided into monthly, weekly, daily, hourly, and minutely. +An empty schedule is equal to minutely, a schedule that run every minute. +-- + +lib/time: add new type Clock:: ++ +Clock represent 24 hours time with hour, minute, and second. +An hour value is from 0 to 23, a minute value is from 0 to 59, and +a second value is from 0 to 59. + +[#v0_44_0__bug_fixes] +=== Bug fixes + +lib/clise: fix potential data race between Push and Slice:: ++ +The data race may occur if Push is called, the .last field is incremented +and at the same time an other goroutine call Slice that access the .last +field. + +lib/memfs: minimize data race on DirWatcher:: ++ +Calling DirWatcher Stop while the start method set dw.ticker can cause +data race. This changes fix this issue. + +go.mod: update all dependencies:: ++ +This update use "go get all" which resolve to semver for each dependencies. + +[#v0_44_0__chores] +=== Chores + +all: set the test timeout to 1m:: ++ +Some flaky test, caused by waiting for channel, require waiting for 10m +before it considered fail. +This changes we cut the default timeout to 1 minute. + +all: access embedded field or methods using the type name:: ++ +This is for clarity in the code, better to be explicit by typing where +the field or methods come from. + +lib/email: add an example for Filter method on Header:: + +cmd: temporarily hide unused commands:: ++ +The cart, cascaded-random-forest, lnsmote, random-forest, and smote +are part of completing thesis and they never used anymore. + +AUR: add go-test-coverhtml.sh and go-bench.sh into package:: ++ +While at it, changes the go-bench count to 10. + +lib/websocket: replace math/rand.Read with crypto/rand.Read:: ++ +The math/rand will be deprecated in Go 1.20. + + +[#v0_43_0] +== share v0.43.0 (2023-02-07) + +[#v0_43_0__new_features] +=== New features + +lib/http: add function MarshalForm:: ++ +-- +The MarshalForm marshal struct fields tagged with `form:` into url.Values. + +The rules for marshaling follow the same rules as in [UnmarshalForm]. + +It will return an error if the input is not pointer to or a struct. +-- + +clise: implement io Closer, Writer, StringWriter, and ByteWriter:: + +clise: add method UnmarshalJSON:: ++ +The UnmarshalJSON convert JSON array into Clise. diff --git a/_doc/index.adoc b/_doc/index.adoc index 794f2755..39bfa6bd 100644 --- a/_doc/index.adoc +++ b/_doc/index.adoc @@ -41,6 +41,9 @@ This library is released every month, usually at the first week of month. link:CHANGELOG.html[Latest changelog^]. +link:CHANGELOG_2023.html[Changelog in 2023^]. +This is changelog for share module since v0.43.0 until v0.51.0. + link:CHANGELOG_2022.html[Changelog in 2022^]. This is changelog for share module since v0.33.0 until v0.42.0. @@ -8,5 +8,5 @@ package share var ( // Version of this module. - Version = `0.51.0` + Version = `0.52.0` ) |