diff options
Diffstat (limited to 'CHANGELOG.adoc')
| -rw-r--r-- | CHANGELOG.adoc | 1319 |
1 files changed, 91 insertions, 1228 deletions
diff --git a/CHANGELOG.adoc b/CHANGELOG.adoc index 92dae827..c0f95b05 100644 --- a/CHANGELOG.adoc +++ b/CHANGELOG.adoc @@ -1,11 +1,12 @@ -= CHANGELOG -Shulhan <ms@kilabit.info> += CHANGELOG :toc: :sectanchors: -:sectlinks: This library is released every month, usually at the first week of month. +link:CHANGELOG_2023.html[Changelog in 2023^]. +This is changelog for share module since v0.43.0 until v0.51.0. + link:CHANGELOG_2022.html[Changelog in 2022^]. This is changelog for share module since v0.33.0 until v0.42.0. @@ -18,1317 +19,179 @@ This is changelog for share module since v0.12.0 until v0.21.0. link:CHANGELOG_2018-2019.html[Changelog from 2018 to 2019^]. This is changelog for share module since v0.1.0 until v0.11.0. +[#v0_52_0] +== share v0.52.0 (2024-01-06) -[#v0_51_0] -== share v0.51.0 (2023-12-06) - -[#v0_51_0__new_features] -=== New features - -lib/http: implement Server-Sent Events (SSE):: -+ --- -For server SSE, we add new type SSEEndpoint, for registering endpoint -that can handle SSE. - -For client SSE, we add it in new sub package "sseclient". --- - -lib/net: add method Read:: -+ --- -The Read method read packet from raw connection. - -If the conn parameter is nil it will return [net.ErrClosed]. - -The bufsize parameter set the size of buffer for each read operation, -default to 1024 if not set or invalid (less than 0 or greater than -65535). - -The timeout parameter set how long to wait for data before considering -it as failed. -If its not set, less or equal to 0, it will wait forever. -If no data received and timeout is set, it will return [ErrReadTimeout]. - -If there is data received and connection closed at the same time, it will -return the data first without error. -The subsequent Read will return empty packet with [ErrClosed]. --- - -lib/crypto: add support for reading passphrase using SSH_ASKPASS:: -+ --- -If the library failed to changes os.Stdin to raw, it will try to use -a program defined in SSH_ASKPASS environment variable. - -The SSH_ASKPASS is controlled by environment SSH_ASKPASS_REQUIRE. - -- If SSH_ASKPASS_REQUIRE is empty the passphrase will read from - terminal first, if not possible then using SSH_ASKPASS program. - -- If SSH_ASKPASS_REQUIRE is set to "never" the passphrase will read - from terminal only. - -- If SSH_ASKPASS_REQUIRE is set to "prefer", the passphrase will read - using SSH_ASKPASS program not from terminal, but require - DISPLAY environment to be set. - -- If SSH_ASKPASS_REQUIRE is set to "force", the passphrase will read - using SSH_ASKPASS program not from terminal, without checking DISPLAY - environment. --- - -This changes affect the [ssh.NewClientInteractive] indirectly. - -lib/memfs: add method JSON to Node:: -+ --- -The JSON method encode the Node into JSON. -This method provides an alternative to MarshalJSON with granular options, -depth and withoutModTime. - -The depth set the level of childs to be encoded. -The depth=0 only encode the Root Node itself (with its childs), depth=1 -encode the Root node and its subdirectories, and so on. - -If withoutModTime is set to true, all of the node ModTime will not be -included in output. --- - - -[#v0_51_0__enhancements] -=== Enhancements - -lib/ini: create the file if its not exist on Open:: -+ -Previously, Open will return an error if file not exist. -This changes create the file if not exist. - -lib/net: use [time.Ticker] to loop in WaitAlive:: -+ --- -The Timeout field in net.Dialer sometimes does not have any effect. -In the for-loop, we expect that each Dial at least consume 100 ms -before returning an error. - -Turns out, on several occasion, the loop run too quickly, less than the -passed timeout. -This is probably because the listening address has not been -active yet and we run it in the same machine, the OS (or Go?) return -immediately without waiting for timeout. --- - -lib/memfs: add sub file system:: -+ --- -This implemented simplified version of merging multiple MemFS instances. -The sub file system (subfs) can be added to the parent MemFS instance -by calling new method "Merge". - - parent.Merge(other *MemfS) - -When Get method called, each subfs will be evaluated in order of Merge -called. - -This deprecated the function Merge. --- - -lib/memfs: improve the refresh method:: -+ --- -Instead of refreshing from the Root, find the directory that closes -to the requested path. - -While at it, simplify the returned error. --- - -lib/ini: append variable into section before any empty lines:: -+ --- -Previously, appending a new variable into section always create an empty -line before after the variable. -This changes fix this by appending variable only after non-empty lines. -While at it, add empty space before variable value. --- - -lib/memfs: stop all Watcher spawn by DirWatcher:: -+ --- -Previously, the Watcher goroutine will stopped only if the file is -being deleted. -If the DirWatcher stopped manually, by calling Stop method, the -Watcher goroutine will still running in the background. - -This changes record all spawned Watcher and stop it when files inside -a deleted directory or when Stop called. --- - -lib/memfs: remove embedding os.FileInfo and http.File:: -+ -The original idea of embedding those interface is to guard the Node -implementation to always follow all methods in the os.FileInfo and -http.File, in case there is additional methods in the future. -But, embedding interfaces make the struct store it as field "FileInfo" -and File with "<nil>" values, which add storage cost to Node. - -lib/memfs: refactoring, simplify DirWatcher logic:: -+ -In this changes, we simplify re-scanning the directory content. -Instead of looping on node Child to detect new or delete files, use map -to store previous Child first. -If node path exist in map then add it again as child, otherwise delete it. - -email/dkim: set ExpiredAt to MaxInt64 if value is greater than 12 digits:: -+ --- -According to RFC 6376, - - To avoid denial-of-service attacks, implementations MAY consider any - value longer than 12 digits to be infinite. - -Since ExpiredAt type is uint64, we cannot set it to infinite, so set it -to maximum value int64 (not maximum of uint64). --- - - -[#v0_50_1] -== share v0.50.1 (2023-11-05) - -This release bring many enhancements to "lib/memfs", a library for caching -file system in memory. - -[#v0_50_1__enhancements] -=== Enhancements - -lib/memfs: return nil in AddChild if file not exist:: -+ --- -In case the node being added is a symlink, and the symlink is broken, -the AddChild will return an error "fs.ErrNotExist". - -In this changes we check for fs.ErrNotExist and return nil without an -error, so the user of AddChild can skip adding the node to the parent. --- - -lib/memfs: quote the path in the returned error:: -+ --- -Error message should at least contains 4W in the following order, - - WHEN WHO/WHERE WHAT WHY - -By quoting, user or machine can parse that the value inside quote -is the value that cause the error (the what), not part of the cause of -the error (the why). --- - -lib/memfs: add method Child to Node:: -+ -The Child method return the child node based on its node. - -lib/memfs: call the Init method in the embedded file:: -+ --- -By calling the Init method, the regular expression for include and -excludes are initialized, which allow using any method of MemFS instance -later, for example the Watch method. - -While at it, fix the fields alignment in template code format. --- - -lib/memfs: include empty directory:: -+ --- -Even thought empty directory does not contains file, from the parent -node _it is_ part of their content. - -Also, there is a use case where memfs use as virtual file system (VFS), -as a layer with file system, where user can view list of directory, -create a directory or file on the fly. -If we skip scanning empty directory, that directory will not be visible. --- - -lib/memfs: re-scan directory content on Node's Update:: -+ -Previously, only node content get updated. -In case in the MemFS, user set "Options.TryDirect" to true, the directory -content should be updated too on "MemFS.Get". - - -[#v0_50_0] -== share v0.50.0 (2023-10-04) - -This release bring many enhancements thanks to linters like revive [1], -fieldaligment [2], and shadow [3]. - -This release also replace "math/rand.Seed" with "crypto/rand". -Since Go 1.20 the "math/rand.Seed" is considered deprecated (the initial -value of rand is seeded automatically, not zero). -Now, it is the time to replace "math/rand" with more secure random number -generator, from "crypto/rand". -This changes affect tests in package "lib/email", "lib/http", and -"lib/stmp". - -[#v0_50_0__breaking_changes] -=== Breaking changes - -lib/test: refactoring, rename TestWriter to BufferWriter:: -+ --- -The name TestWriter is considered stutter if its called from external -package, test.TestWriter. - -While at it, implement the Error and Errorf in BufferWriter and add -comment to each exported methods. --- - -ssh/sftp: rename method Close to CloseFile:: -+ -Since the method accept FileHandle, that is returned from OpenFile, -then the method should changed with the same suffix. -This is also to make the method un-ambiguous later when we add Close method -to close client connection. - -[#v0_50_0__new_features] +[#v0_52_0__new_features] === New features -lib/errors: implement method Is:: -+ -The Is method will return true if the target error is instance of *E -and the value of field Code and Name match with values in e. - -lib/email: add method ID to Header:: -+ -The ID method return the Message-ID or empty if its not exist. - -test/mock: implement mock for testing io.ReadWriter or io.StringWriter:: -+ --- -The type ReadWriter provide two buffers, BufRead and BufWrite. -The BufRead is for io.Reader and BufWrite for io.Writer or -io.StringWriter. - -The type provide one method helper, Reset, to reset all buffers. --- - -lib/crypto: add function LoadPrivateKeyInteractive:: +ssh/config: add method MarshalText and WriteTo:: + -- -The LoadPrivateKeyInteractive load the private key from file. -If the private key file is encrypted, it will prompt for the passphrase -from terminal. +The MarshalText method encode the Section back to ssh_config format +with two spaces as indentation in key. -This function is taken from package "lib/ssh" with modification by adding -parameter "io.ReadWriter" and removing parameter maxAttempt. +The WriteTo method marshal the Section into text and write it to +[io.Writer] w. -- -lib/crypto: implement RSA encrypt and decryption for large message size:: -+ --- -The EncryptOaep extend the "rsa.EncryptOAEP" to make it able to encrypt a -message larger than its than (public modulus size - 2*hash.Size - 2). - -The function signature is the same with "rsa.EncryptOAEP" except the -name, to make it distinguishable. - -The DecryptOaep function extend the "rsa.DecryptOAEP" to make it able to -decrypt a message larger than its public modulus size. --- - -ssh/sftp: add method to close client connection:: -+ -The Close method close the client sftp session and release all its -resources. - -lib/ssh: add method Close to Client:: -+ -The Close method close the client connection and release all resources. - - -[#v0_50_0__enhancements] -=== Enhancements - -_AUR: include go-mod-tip.sh to package:: -+ -The "go-mod-tip.sh" is a shell script to get and print the latest Go module -version based on the latest tag and the latest commit hash from current -working git directory. - -lib/crypto: rewrite LoadPrivateKey as wrapper of ssh.ParseRawPrivate:: -+ --- -Previously, the LoadPrivateKey function only able to load private key -with PKCS#1 format. - -This changes make the function as a wrapper for ssh.ParseRawPrivate -that can load RSA, DSA, ECDSA, and Ed25519 in PKCS#1, PKCS#8, OpenSSL, -and OpenSSH formats. --- - -lib/ssh: store and use the connected identity file for ScpPut and ScpGet:: -+ -Previously, the ScpPut and ScpGet use the first IdentityFile in section -to use in "-i" argument. -This is not correct. -If the section contains two or more IdentityFile and the one that can -connect are the last one, the ScpPut and ScpGet method may return an -error when executing the "scp" command. - -[1] https://github.com/mgechev/revive - -[2] https://pkg.go.dev/golang.org/x/tools/go/analysis/passes/fieldalignment - -[3] https://pkg.go.dev/golang.org/x/tools@v0.13.0/go/analysis/passes/shadow - - -[#v0_49_1] -== share v0.49.1 (2023-09-02) - -[#v0_49_1__bug_fixes] -=== Bug fixes - -lib/dns: fix leaking internal zone:: +lib/ssh: implement method Output on Client:: + -- -Previously, if the server have internal zone "my.internal" and the client -query "sub.my.internal" that does not exist in the zone, the server then -forward the query to parent name server. -This cause the internal zone and its domains leaked to parent name server. - -This changes fix this issue by checking if the query is subset of -internal zone Origin if domain does not exist, and response with error -code 3 (ERR_NAME) with the Zone SOA in Authority. +The Output method run the command and return its standard output and +error as is. +Any other error beside standard error, like connection, will be returned +as error. -- -[#v0_49_1__enhancements] -=== Enhancements - -lib/dns: update the SOA Serial when record added or removed from Zone:: -+ -Any call to Zone Add or Remove methods will update the Zone.SOA.Serial to -current epoch. - -lib/dns: add method AddAuthority to Message:: +ssh/sftp: implement method MkdirAll on Client:: + -- -The AddAuthority add the rr to list of Authority. -Calling this method mark the message as answer, instead of query. - -If the rr is SOA, it will replace the existing record if exist and set -the flag authoritative answer (IsAA) in header to true. -If the rr is NS, it will be added only if its not exist. - -It will return an error if the rr type is not SOA or NS or the size of -records in Authority is full, maximum four records. +The MkdirAll create directory on the server, from left to right. +Each directory is separated by '/', where the left part is the parent of +the right part. +This method is similar to [os.MkdirAll]. -- -lib/dns: add method to populate internal caches by Zone:: -+ -The InternalPopulateZone populate the internal caches from Zone's -messages. - -ssh/config: handle key UserKnownHostsFile:: -+ -The UserKnownHostsFile define list of the known_hosts files to be read, -separated by spaces. -If not set default to "~/.ssh/known_hosts" and "~/.ssh/known_hosts2". - - -lib/ssh: use UserKnownHostFile from configuration in NewClientInteractive:: +cmd/httpdfs: implement [libhttp.Server] with [memfs.MemFS]:: + -- -Previously, the ssh Client always use InsecureIgnoreHostKey in -HostKeyCallback. -This may post security issue, like man-in-the-middle attack, since we -did not check the server host key with one of key that known by client -from UserKnownHostFile (for example ~/.ssh/known_hosts). - -This changes use the SSH section UserKnownHostFile from configuration -(default to ~/.ssh/known_hosts) to check if the server host key is -valid. -The NewClientInteractive will return an error, "key is unknown", if host -key not exist in UserKnownHostFile or "key is mismatch" if host key -not match with one registered in UserKnownHostFile. - -This changes depends on patch of golang.org/x/crypto [1] that has not -reviewed yet, so we need to replace it with one that contains the patch. - -[1] https://go-review.googlesource.com/c/crypto/+/523555 +The httpdfs is a program to serve a directory under HTTP. -- - -[#v0_49_0] -== share v0.49.0 (2023-08-04) - -[#v0_49_0__breaking_changes] +[#v0_52_0__breaking_changes] === Breaking changes -lib/email: refactoring, replace field with type []byte to string:: -+ -Using string provide safety, guaranteed that if we pass it as parameter -the receiver will not be able to modify its content. - -ssh/config: refactoring, simplify the Section fields:: +ssh/config: refactoring the Config merge:: + -- -Instead of storing each Section value in separate field, store them -inside a map, Field. -This reduce the size of Section and simplify adding or getting the -key that we are not supported but maybe usable by user in the future. +This changes rename method [Config.Prepend] to [Config.Merge]. -This changes introduce several new methods as replacement of field: +The way that how the other Config merged is changed. +Instead of appending all of other's sections into the current Config, +append the other Config instance to the current instance of Config. -* CASignatureAlgorithms: a method that return list of signature - algorithms that Section set or the default -* CanonicalDomains: a method that return CanonicalDomains set in Section -* CanonicalizePermittedCNames: return the permitted CNAMEs set in Section, - from KeyCanonicalizePermittedCNames. -* CertificateFile: return list of certificate file -* Environments: return system and/or custom environment that will be - passed to remote machine. - The key and value is derived from "SendEnv" and "SetEnv". -* FieldBool: return field value as boolean -* FieldInt: return the field value as int -* Hostname: return the Hostname in this Section -* IdentityAgent: return the path to SSH agent socket to be used -* Port: return the remote machine port -* User: return the remote user name -* Set: set the Field using key and value +During [Config.Get] the top Config will be evaluated first, and then the +other Config is evaluated in order of Merge. -- -lib/ssh: refactoring NewClientFromConfig, renamed to NewClientInteractive:: +ssh/config: add parameter Config to NewSection:: + -- -Previously, the NewClientInteractive blindly use the signers from -Section.Signers. -If one of the IdentityFile valid, it will add all the keys in -IdentityFile to SSH agent. - -In this changes we try each IdentityFile independently. -If the key is valid, client connected to remote machine, then only that -key will be added to SSH agent. +This changes how the Section and parser initialized. -While at it we also rename the method to NewClientInteractive to -indicate that the function will prompt for passphrase if one of the -IdentityFile is encrypted. +Previously, the Config depends on the parser to set the workDir and +homeDir and Section depends on Config only on Get; now its the other +way around, from top to bottom. +Config initialized first, then parser initialized using Config instance, +and then Section initialized also using Config instance. -- -[#v0_49_0__bug_fixes] -=== Bug fixes - -lib/http: fix missing query when handling redirect in HandleFS:: -+ -In 06e6cbdd511c, we redirect request by adding end slash to the path -if the requested resource is directory, but somehow we miss adding the -original request query. -This changes fix this issue. - -lib/os: check for extract path in untar and unzip:: +lib/ssh: add parameter context to Execute method:: + -- -Once we Join-ed the directory output with the file name, we check if -the result of join is still under directory output, if its not, return -an error to prevent -https://cwe.mitre.org/data/definitions/22.html[Zip Slip vulnerability]. +This changes require the fork of our golang.org/x/crypto. -- -lib/smtp: format the passed data in NewMailTx:: +lib/time: remove UnixMicro and UnixMilli:: + -- -The following rules are applied to the data, - -* all lines must end with CRLF -* if the line start with period, additional period is inserted before - the line. This recommendation based on RFC 5321 section 4.5.2 [1] to - prevent data that contains CRLF "." CRLF does not corrupt the message, - causing the server terminate reading the message where it should not. - -[1] https://datatracker.ietf.org/doc/html/rfc5321#section-4.5.2 +Both of those methods has been added into standard library as +[Time.UnixMicro] and [Time.UnixMilli] since Go 1.17. -- -[#v0_49_0__new_features] -=== New features - -lib/os: add function PathFold and PathUnfold:: +lib/io: removed, this package has been merged into "lib/os":: + -- -The PathFold replace the path "in" with tilde "~" if its prefix match -with user's home directory from os.UserHomeDir. - -The PathUnfold expand the tilde "~/" prefix into user's home directory -using os.UserHomeDir and environment variables using os.ExpandEnv -inside the string path "in". +While some functions are merged to "lib/os", some are not used anymore +like io.Reader. -- -lib/os: add function Environments:: -+ -The Environments function return list of system environment as map of -key and value. - -lib/ssh: add function LoadPrivateKeyInteractive:: -+ -The LoadPrivateKeyInteractive load private key from file. -If key is encrypted, it will prompt the passphrase in terminal with -maximum maxAttempt times. -If the passphrase still invalid after maxAttempt it will return an error. +lib/parser: removed, this package has been merged into lib/strings:: -[#v0_49_0__enhancements] -=== Enhancements - -lib/smtp: set minimum Server TLS to v1.2:: -+ -Using the TLS v1.1 is considered insecure and should not be used in -server anymore. - -lib/memfs: check for refresh URL outside of Root SysPath:: -+ -The case when refresh URL outside of Root SysPath is only exist when -the memfs TryDirect is set to true, usually during development. -In the production, the TryDirect should be false, hence the refresh -always return nil Node. - -ssh/config: reorder struct fields for better alignment:: -+ --- -Changes, - -* Config: changes allocated size from 32 to 16 bytes (-8 bytes) -* parser: changes allocated size from 40 to 32 bytes (-8 bytes) -* Section: changes allocated size from 392 to 360 bytes (-32 bytes) - -The rest of changes that are not mentioned are from test files. --- - - -[#v0_48_0] -== share v0.48.0 (2023-07-07) - -This release bring many enhancements to lib/websocket including timeout, -handling upgrade and read/write concurrently using goroutine. - -[#v0_48_0__breaking_changes] -=== Breaking changes - -lib/net: changes the WaitRead/Event model on Poll:: -+ --- -Previously, the Pool's WaitRead and WaitReadEVent methods return list of -file descriptor (fd) and keeps the fd in the pool. -In case we want to process the returned fd concurrently, by running it -in different goroutine, the next call WaitRead may return the same fd -if its goroutine not fast enough to read from fd. - -This changes fix this issue by removing list of fd from poll and set the -fd flag to blocking mode again after returning it from WaitRead or -WaitReadEvent. - -This changes also remove the ReregisterRead and ReregisterEvent methods -since it is not applicable anymore. --- -[#v0_48_0__bug_fixes] +[#v0_52_0__bug_fixes] === Bug fixes -lib/websocket: call Quit when handshake contains close or invalid frame:: +ssh/config: fix setting the default values:: + -If the HTTP handshake response contains trailing frame, handle it -directly. -If the frame is invalid or contains control close operation, call Quit -directly to trigger the HandleQuit if its defined by user. - -lib/websocket: revert maxBuffer back to 1024:: -+ -In v0.47.0 we increase the maxBuffer to 4096 to try increasing the -performance when handling large payload. -Turns out increasing this break the autobahn test suite. - - -[#v0_48_0__new_features] -=== New features - -lib/ascii: add type Set:: -+ --- -The Set type is a bitmap that represent list of ASCII characters for faster -lookup. - -A Set is a 36-byte value, where each bit in the first 32-bytes represents -the presence of a given ASCII character in the set. -The remaining 4-bytes is a counter for the number of ASCII characters in the -set. -The 128-bits of the first 16 bytes, starting with the least-significant bit of -the lowest word to the most-significant bit of the highest word, map to the -full range of all 128 ASCII characters. -The 128-bits of the next 16 bytes will be zeroed, ensuring that any non-ASCII -character will be reported as not in the set. -- - -lib/net: implement generic PollEvent:: -+ +The field default value should be set on Get, after all the Host or +Match fields merged. +In this way, if the field key already set, its not overridden by the +default value or subsequent Host or Match value. -- -The PollEvent contains file descriptor and the underlying event -based on OS, unix.EpollEvent on Linux or unix.Kevent_t on BSD. -The Poll interface provides two APIs to works with PollEvent, -WaitReadEvents that return list of PollEvent ready for read, and -ReregisterEvent to register the event back to poll (only for Linux). --- - -[#v0_48_0__enhancements] -=== Enhancements - -lib/websocket: add option to set read/write timeout on Server:: +ssh/config: set the Hostname if its not set on [Config.Get]:: + -- -The ReadWriteTimeout define the maximum duration the server wait when -receiving/sending packet from/to client before considering the -connection as broken. +Per manual ssh_config(5) on Hostname, -Default read-write timeout is 30 seconds if not set. +[quote] +The default is the name given on the command line. -This changes affect the exported function Send and Recv by adding -additional parameter timeout to both of them. +So, if the requested host name match with one of Host or Match, but +Hostname is not set, it should be default to the requested parameter +name. -- -lib/websocket: handle concurrent upgrade using goroutine:: +http/sseclient: fix data race on [Client.Close]:: + -- -The maxGoroutineUpgrader define maximum goroutines running at the same -time to handle client upgrade. -The new goroutine only dispatched when others are full, so it will -run incrementally not all at once. -Default to defServerMaxGoroutineUpgrader (128) if its not set. +The data race happened when Close set conn to nil but the consume +method still on Read. +The fix is by waiting for 100ms so consume goroutine can check if closeq +is triggered from Close or not. -- -lib/websocket: handle concurrent Server read using goroutines:: -+ -The Server now dispatch a goroutine to consume event from poll reader -for each client connection that is ready to read. -The maximum number of goroutine is defined in ServerOptions -maxGoroutineReader, which currently set to 1024. - -lib/websocket: handle concurrent ping using goroutines:: -+ -The maximum goroutines is quarter of max queue. -The new goroutine for pinger will be dispatched when no goroutine can -consume the current processed connection. - - -[#v0_48_0__chores] -=== Chores - -websocket/testdata: rewrite autobahn test using container:: +http/sseclient: fix Retry value not set to millisecond:: + -- -Since the autobahn script can only run on Python 2, it become hard to -setup and run the test on distro that does not provide Python 2 anymore. -The autobahn repository recommend to use docker instead. - -When testing the server, we simplify it by using make task "test-server". -The test-server task run our test server in background, and then run the -autobahn fuzzingclient from container. -Once the tests completed, we trigger the server to shutdown by sending -text frame with payload "shutdown". +When client receive "retry:" message, the value is in millisecond, but +when we store it we only convert it to [time.Duration] which default +to nanosecond. -When testing the client, we simplify it by using make task "test-client". -The test-client task run the autobahn fuzzingserver and then -we run our client. -Once client finished, we trigger the server to generate the reports -and cleanup the container. +While at it, update comments on field [Client.Retry] and +[Client.Insecure]. -- - -[#v0_47_0] -== share v0.47.0 (2023-06-05) - -[#v0_47_0__breaking_changes] -=== Breaking changes - -email/maildir: major refactoring plus adding unit tests:: -+ -This changes remove all unneeded methods from Manager leave it with -four methods: Delete, FetchNew, Incoming, OutgoingQueue. -+ -Also, we add the type filename to generate file name for tmp and new -directory. - -lib/email: unexport the field ContentType in the field:: -+ -The field ContentType will be set only when the field Name is -"Content-Type" so it's not always exist on each field. -To get the field ContentType, use Header.ContentType(). - - -[#v0_47_0__bug_fixes] -=== Bug fixes - -lib/dns: fix zone parsing on SOA record with single line:: -+ -Due to refactoring in c376eccd25, parsing SOA record with single line -return an error: "parseSOA: line 2: incomplete SOA statement '0'". - -lib/memfs: ignore permission error when scanning directory content:: -+ -Instead of returning error, skip the directory that we cannot read and -continue to process the other. - -lib/memfs: fix panic when watched file deleted or renamed:: -+ -When the file being watched is deleted, sometimes it will cause panic. - -lib/email: fix parsing multiple parameters in ContentType:: -+ -While at it, also fix the ContentType String method to prefix ';' before -appending parameter key and value. - - -[#v0_47_0__new_features] -=== New features - -cmd/bcrypt: CLI to compare or generate hash using bcrypt:: -+ -The bcrypt command has two subcommand "compare" and "gen". -The "compare" subcommand accept two parameter the hash and plain text. -The "gen" subcommand accept only one parameter, the plain text to be hashed. - -lib/sql: add type DmlKind:: -+ -The DmlKind define the kind for Data Manipulation Language. - -email/maildir: implement Folder:: -+ -Folder is a directory under maildir that store messages per file. -A folder contains three directories: tmp, new, and cur; and an empty -file "maildirfolder". - -lib/net: add function WaitAlive:: -+ --- -WaitAlive try to connect to network at address until timeout reached. -If connection cannot established it will return an error. - -Unlike [net.DialTimeout], this function will retry not returning an error -immediately if the address has not ready yet. --- - -lib/smtp: implement Client SendEmail:: -+ --- -Somehow in 3a1a2715b25f, we include this method without implementing it. - -The SendEmail method simplify sending email by automatically create -[MailTx] for passing it to method Client.MailTx. - -The test right now use live connection since the Server is not ready yet. --- - -[#v0_47_0__enhancements] -=== Enhancements - -lib/dns: add option to set debug level in ServerOptions:: -+ -This options replace the global debug package. - -lib/dns: do not cache empty answers:: -+ -The use case if one use and switch between two different -networks with internal zone, frequently. -For example, if on network Y they have domain MY.Y and -current connection is X, request to MY.Y will return an -empty answers. -Once they connect to Y again, any request to MY.Y will not -be possible because rescached caches contains empty answer -for MY.Y. - -_bin/go-test-coverhtml: add parameter to run specific test:: -+ -The second parameter is optional. -It is passed to -run= argument in "go test". -Default value is ".", or all functions. - -lib/http: redirect path with slash if request is directory:: +ssh/sftp: fix Stat on empty remote file name:: + -- -If request path is a directory and it is not end with slash, redirect -request to location with slash to allow relative links works inside the -HTML content. +The implementation of SSH server (openssh) for Stat is not consistent with +the RFC. +The RFC mentioned that -For example, a "/page/index.html" contains links href="sub.html" (where -"sub.html" is inside "/page" directory). -If request to "/page" (without end with slash) return content of -"/page/index.html", then when user click on sub.html it will request to -"/sub.html" instead of "/page/sub.html". --- +[quote] +An empty path name is valid, and it refers to the user's default +directory (usually the user's home directory). -lib/email: handle obsolete white spaces and comment when unpacking date:: -+ +But this only working on some command, like Mkdir, but not Stat. -- -In the obsolete syntax, white space and comments can appear between many -more element, for example the folloing Date value are valid - - Date : Fri, 21 Nov 1997 09(comment): 55 : 06 -0600 -This changes handle this by sanitizing the Field value, removing comment -and merge multiple spaces into one, before parsing it. --- - -lib/email: set the Field Type and unpack its value on ParseField:: +ssh/sftp: fix non-nil returned error on Close:: + -- -Once the field Name has detected and its Value is valid, we can unpack -the Value based to type that it represent, for example to Date or Mailbox. - -This changes remove calling to unpack directly in some tests and check -an error when testing ParseHeader. +This changes fix the Close that always return an error. -- -lib/net: increase the maximum poll events:: -+ -The maxQueue define the number of events that can be read from poll at -one time. -Using 128 seems to small for high throughput networks. -Increasing this number also increase the memory consumed by process. -Maybe later we can export this function as option when creating poll. - -lib/websocket: increase the max buffer and queue for better throughput:: -+ --- -The maxBuffer increased from 1024 to 4096 bytes. -The reason that we use 1024 previously is related to MTU size and maximum -payload in TCP (although its higher, 1460 bytes). - -The maxQueue increase from 128 to 4096. --- - -[#v0_47_0__chores] -=== Chores - -all: remove any usage of debug.Value in all packages:: -+ -Using global debug value for all packages turns out is not a good -idea. - -lib/test: update documentation related to Assert and Data:: -+ -The documentation is based on the article published at -https://kilabit.local/journal/2023/go_test_data/ -after reviewing and explain how to use both of them to public. - -all: record the contributors of this module in file AUTHORS:: - - -[#v0_46_0] -== share v0.46.0 (2023-05-02) -This release deprecated lib/io and lib/parser. - - -[#v0_46_0__breaking_changes] -=== Breaking changes - -lib/reflect: remove the third return value from Marshal:: -+ -The third, boolean, return value is redundant with the second error value. - -lib/bytes: changes the DumpPrettyTable output format:: -+ -The change is to accommodate large bytes data, more than 0xFFFF. -The hex address in the first column is increased to 8 digits, the -characters compacted without space in between. - - -[#v0_46_0__new_features] -=== New features - -lib/os: merge some functions from lib/io:: -+ -Functions like Copy, IsBinary, IsDirEmpty, IsFileExist, RmdirEmptyAll -are read and operate on file and directory on operating system level, so -it is not correct to put it in package io. - -lib/strings: merge lib/parser here:: -+ --- -The first idea of parser is to provide generic parser for both bytes and -string. -After we introduce lib/parser there is not much changes to that package. -Also, since we create another Parser in lib/bytes that accept and -return token as []byte, the lib/parser is not unique anymore. - -The following function/methods changes to minimize conflict in the future, - -* Lines become LinesOfFile -* New become NewParser -* Open become OpenForParser -* Token become Read -* TokenEscaped become ReadEscaped -* TokenTrimSpace become ReadNoSpace --- - -lib/bytes: implement function ParseHexDump:: -+ --- -The ParseHexDump parse the default output of [hexdump](1) utility from -parameter in back into stream of byte. - -An example of default output of hexdump is - - 0000000 7865 5f70 6964 2f72 0000 0000 0000 0000 - 0000010 0000 0000 0000 0000 0000 0000 0000 0000 - * - 0000060 0000 0000 3030 3030 3537 0035 3030 3130 - -The first column is the address and the rest of the column is the data. -Each data column is 16-bit words in big-endian order, so in the above -example, the first byte would be 65, second byte is 78 and so on. -The asterisk "*" means that the address from 0000020 to 0000050 is equal to -the previous line, 0000010. - -[hexdump]: https://man.archlinux.org/man/hexdump.1 --- - -lib/bytes: implement tokenize Parser:: -+ -The Parser type parse stream of byte using one or more delimiters as -separator between token. - -lib/bytes: add function TrimNull:: -+ -The TrimNull function remove 0 value ("\0" or NULL in C) at leading -and trailing of input. - -lib/net: add method WriteTo to ResolvConf:: -+ -The WriteTo method write the ResolvConf as text. - - -[#v0_46_0__enhancements] +[#v0_52_0__enhancements] === Enhancements -lib/time: calculate the next event before notify the user on Scheduler run:: -+ -This allow user to call the Next method, to know the next time the -scheduler will be triggered, after receiving the event. - -lib/reflect: add option to skip processing struct field in Do/IsEqual:: -+ -A struct's field tagged with `noequal:""`, its value will not be processed for -equality. - - -[#v0_46_0__chores] -=== Chores - -lib/reflect: use doEqual inside IsEqual:: -+ -Previously, IsEqual internally use isEqual, which have the same logic as -doEqual. -This changes minimize duplicate code between IsEqual and DoEqual, by -calling doEqual for both of functions. - -lib/time: replace lib/io#Reader with lib/bytes#Parser:: - -lib/smtp: replace lib/io#Reader with lib/bytes#Parser:: - -lib/dns: replace lib/io#Reader with lib/bytes#Parser:: - -lib/http: replace lib/io#Reader with lib/bytes#Parser:: - -lib/email: replace lib/io#Reader with lib/bytes#Parser:: - -email/dkim: replace lib/io#Reader with lib/bytes#Parser:: - -lib/hunspell: replace lib/io with lib/os:: - -lib/hunspell: replace lib/parser with lib/strings:: - -lib/http: replace lib/parser with lib/strings:: - -lib/bytes: copy TokenFind to internal/bytes#TokenFind:: +ssh/config: merge the Section slice values on [Section.merge]:: + -This is to prevent import cycle later when we use lib/test in bytes. - - -[#v0_45_0] -== share v0.45.0 (2023-04-01) - -This release set the Go version to 1.19. - -[#v0_45_0__breaking_changes] -=== Breaking changes - -lib/net: changes the PopulateQuery logic:: -+ --- -Previously, PopulateQuery only add the passed dname if the number of dots -is greater than 0. -After inspecting the result from dig and getenv, the dots seems does not -affect the query. -For example, if we have A record for domain "kilabit", both of those tools -query name "kilabit" without adding local domain or domain in search. -- +Instead of using [Section.Set], set the key-value directly. -[#v0_45_0__new_features] -=== New features - -_bin: add shell script go-test-lint.sh to run test and lint sequentially:: -+ --- -The go-test-lint.sh run Go test and if its success it will run -predefined linter, in the current directory. - -Arg 1: the method or function to test, default to ".". - -The linter program and its argument is derived from environment variable -GO_LINT. -If its empty, it will try the following linter in order: revive and then -golangci-lint. - -To add additional arguments to Go test set the environment variable -GO_TEST_ARGS. --- - -lib/bytes: add function DumpPrettyTable:: -+ -The DumpPrettyTable write each byte in slice data as hexadecimal, ASCII -character, and integer with 8 columns width. - -lib/bytes: add function SplitEach:: -+ -The SplitEach funciton split the slice of byte into n number of bytes. -If n is less or equal than zero, it will return the data as chunks. - -lib/dns: add function ParseZone:: -+ --- -The ParseZone parse the content of zone from raw bytes. - -Now that we have ParseZone, all tests that use zoneParser now can be -replaced using combination of test.Data and ParseZone. --- - -lib/dns: add method WriteTo to Zone:: -+ --- -The WriteTo method write the zone as text into io.Writer. - -The result of WriteTo will be different with original content of zone -file, since it does not preserve comment and indentation. --- - -lib/http: add function to parse multipart Range response for Client:: -+ -The ParseMultipartRange parse the multipart/byteranges body or response -from HTTP Range request. -Each Content-Range position and body part in the multipart will be stored -under RangePosition. - -lib/http: add support for HTTP Range in Server:: -+ --- -For HTTP Server using HandleFS, the Range request is handled -automatically. -For other HTTP server, user can use the HandleRange function. - -The HandleRange function handle -https://developer.mozilla.org/en-US/docs/Web/HTTP/Range_requests[HTTP Range] -request using "bytes" unit. -The body parameter contains the content of resource being requested that -accept Seek method. - -If the Request method is not GET, or no Range in header request it will -return all the body -https://datatracker.ietf.org/doc/html/rfc7233#section-3.1[RFC7233 S-3.1]. - -The contentType is optional, if its empty, it will detected by -http.ResponseWriter during Write. --- - -lib/io: add method ReplaceAll on Reader:: -+ -The ReplaceAll method behave like standard bytes.ReplaceAll but start -from current index. - -lib/parser: add method TokenTrimSpace:: -+ -The TokenTrimSpace read the next token until one of the delimiter found, -with leading and trailing spaces are ignored. - -lib/parser: add method SetDelimiters:: -+ -The SetDelimiters replace the current delimiters. - -lib/telemetry: package for collecting and forwarding metrics:: -+ -Package telemetry is a library for collecting various Metric, for example -from standard runtime/metrics, and send or write it to one or more -Forwarder. -Each Forwarder has capability to format the Metric before sending or -writing it using Formatter. - - -[#v0_45_0__bug_Fixes] -=== Bug fixes - -lib/dns: fix packing, parsing, and saving MINFO resource data:: -+ --- -Even thought the MINFO record not formally obsolete, according to -https://en.wikipedia.org/wiki/List_of_DNS_record_types#Obsolete_record_types[Wikipedia], -we still need to support this for backward compatibility. - -When packing the resource data length does not include total length. -When parsing, the RMailBox and EmailBox should be added the origin suffix -if its not end with dot. -When saving, the origin should be trimmed from RMailBox and EmailBox. --- - -lib/dns: fix packing and unpacking resource record HINFO:: -+ --- -The rdata for HINFO contains two character-strings: CPU and OS. -Previously, we pack the rdata section sequentially, without adding length -on each of them: <RDLEN><CPU><OS>. -The correct pack format should <RDLEN><LENGTH><CPU><LENGTH><OS>. --- - -lib/dns: fix parsing SRV record from zone file:: -+ --- -Previous parseSRV start by parsing the _Service from tok, but the -actual value of parameter tok is the Priority. - -This changes fix this and as testing we use the example from RFC 2782. --- - -[#v0_45_0__enhancements] -=== Enhancements - -lib/dns: allow parsing TXT rdata without quote in zone file:: -+ --- -Previously, the zone only parsing TXT record with double quote since -most of the example that we found during implementation all use double -quote. - -This changes allow non-double quoted text in zone file with consequence -that any spaces will terminated the rdata immediately. - -Fixes #6 +While at it, merge the certificateFile, IdentityFile, knownHostFiles, +and sendEnv. -- -lib/dns: handle zone file with CRLF line ending:: +ssh/config: set the default UserKnownHostsFile in setDefaults:: + -- -While at it, fix parsing multiline SOA record where closing parentheses -end on next lines. - -Fixes #6 +While at it, unfold each value of IdentityFile and UserKnownHostsFile +in setDefaults, by expanding "~" into user's home directory or joining +with "config" directory if its relative. -- - - - - -lib/test: simplify the string diff output from Assert:: -+ --- -In the output, instead of using %q we replace it with %s, because printing -string with double quote cause escaping and hard to read -This change may cause difference in white spaces not showed in the -terminal. - -In the diff changes, only print the Old and New, without printing each -chunk. --- - - -[#v0_44_0] -== share v0.44.0 (2023-03-02) - -[#v0_44_0__new_features] -=== New features - -lib/time: implement Scheduler:: -+ --- -Scheduler is a timer that run periodically based on calendar or day time. - -A schedule is divided into monthly, weekly, daily, hourly, and minutely. -An empty schedule is equal to minutely, a schedule that run every minute. --- - -lib/time: add new type Clock:: -+ -Clock represent 24 hours time with hour, minute, and second. -An hour value is from 0 to 23, a minute value is from 0 to 59, and -a second value is from 0 to 59. - -[#v0_44_0__bug_fixes] -=== Bug fixes - -lib/clise: fix potential data race between Push and Slice:: -+ -The data race may occur if Push is called, the .last field is incremented -and at the same time an other goroutine call Slice that access the .last -field. - -lib/memfs: minimize data race on DirWatcher:: -+ -Calling DirWatcher Stop while the start method set dw.ticker can cause -data race. This changes fix this issue. - -go.mod: update all dependencies:: -+ -This update use "go get all" which resolve to semver for each dependencies. - -[#v0_44_0__chores] -=== Chores - -all: set the test timeout to 1m:: -+ -Some flaky test, caused by waiting for channel, require waiting for 10m -before it considered fail. -This changes we cut the default timeout to 1 minute. - -all: access embedded field or methods using the type name:: -+ -This is for clarity in the code, better to be explicit by typing where -the field or methods come from. - -lib/email: add an example for Filter method on Header:: - -cmd: temporarily hide unused commands:: -+ -The cart, cascaded-random-forest, lnsmote, random-forest, and smote -are part of completing thesis and they never used anymore. - -AUR: add go-test-coverhtml.sh and go-bench.sh into package:: -+ -While at it, changes the go-bench count to 10. - -lib/websocket: replace math/rand.Read with crypto/rand.Read:: -+ -The math/rand will be deprecated in Go 1.20. - - -[#v0_43_0] -== share v0.43.0 (2023-02-07) - -[#v0_43_0__new_features] -=== New features - -lib/http: add function MarshalForm:: -+ --- -The MarshalForm marshal struct fields tagged with `form:` into url.Values. - -The rules for marshaling follow the same rules as in [UnmarshalForm]. - -It will return an error if the input is not pointer to or a struct. --- - -clise: implement io Closer, Writer, StringWriter, and ByteWriter:: - -clise: add method UnmarshalJSON:: -+ -The UnmarshalJSON convert JSON array into Clise. |