| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
With help of spdxconv tool [1], we able to bulk update all files license
and copyright format to comply with SPDX formats.
[1] https://kilabit.info/project/spdxconv/
|
|
This changes move the following fields and methods from MemFS,
* incRE and excRE fields to Options,
* isIncluded and isExclude methods to Options,
* watchRE field to WatchOptions,
* isWatched method to WatchOptions.
The reason is to allow other type that use Options or WatchOptions to use
isIncluded, isExclude, or isWatched methods.
|
|
While at it, temporary disable gosec due to excessive report for G115,
which may be true, but may also break the current working program.
We should alter and fix once we can test and make sure that it does not
breaks.
|
|
In [MemFS.refresh], if the requested url is "/file1" and [Options.Root]
is ".", the path during refresh become "file1" and if passed to
[filepath.Dir] it will return ".".
This cause the loop on refresh never end because there is no PathNodes
equal with ".".
|
|
The MemFS always store directory without slash.
If caller request a directory node with slash, it will always return nil.
|
|
Some of warnings from those linter are false positives, so we just
annotated them.
|
|
The Merge function has been replaced with [memfs.MemFS.Merge] for
better API.
|
|
For HTTP server that use TLS, set the minimum TLS version and
ReadHeaderTimeout to mitigate slowloris attack.
For HTTP client or server that parameterize the use of InsecureSkipVerify,
annotate the line with "nolint:gosec" to allow the code pass the check.
Library that still use sha1, in example in DKIM and TOTP, skip the
warnings by annotating the line with "nolint:gosec".
A pointer variable now allocated their address before assigning its
value.
Any error that returned now wrapped using "%w".
Also, replace error checking using [errors.Is] or [errors.As] instead of
using equal or not-equal operators.
In "lib/http", replace any usage of "math/rand" with "crypto/rand".
Any call of [math/big.Rat.SetString] now annotated with "nolint:gosec"
since its false positive, the issue has been fixed in Go >= 1.17.7.
Any switch case that does not cover the rest of the possible values now
handled by adding the cases or by replacing the "default" case with
the rest of values.
|
|
HTTP request now implicitly create request with context.
Any false positive related to not closing HTTP response body has been
annotated with "nolint:bodyclose".
In the example code, use consistent "// Output:" comment format, by
prefixing with single space.
Any comment on code now also prefixing with single space.
An error returned without variables now use [errors.New] instead of
[fmt.Errorf].
Any error returned using [fmt.Errorf] now wrapped using "%w" instead of
"%s".
Also, replace error checking using [errors.Is] or [errors.As], instead
of using equal/not-equal operator.
Any statement like "x = x OP y" now replaced with "x OP= y".
Also, swap statement is simplified using "x, y = y, x".
Any switch statement with single case now replaced with if-condition.
Any call to defer on function or program that call [os.Exit], now
replaced by calling the deferred function directly.
Any if-else condition now replaced with switch statement, if possible.
|
|
There are several reasons that why we move from github.com.
First, related to the name of package.
We accidentally name the package with "share" a common word in English
that does not reflect the content of repository.
By moving to other repository, we can rename it to better and unique
name, in this "pakakeh.go".
Pakakeh is Minang word for tools, and ".go" suffix indicate that the
repository related to Go programming language.
Second, supporting open source.
The new repository is hosted under sourcehut.org, the founder is known
to support open source, and all their services are licensed under AGPL,
unlike GitHub that are closed sources.
Third, regarding GitHub CoPilot.
The GitHub Terms of Service [1], allow any public content that are hosted
there granted them to parse the content.
On one side, GitHub helps and flourish the open source, but on another
side have an issues regarding scraping the copyleft license [2].
[1]: https://docs.github.com/en/site-policy/github-terms/github-terms-of-service#4-license-grant-to-us
[2]: https://githubcopilotinvestigation.com
|
|
The test is flaky because we use range over map of [tdata.Input].
Since range over map will return random key, there is possibility that
the first test is writing "/dir-a/dir-b/file2" not "/dir-a/dir-b/file".
|
|
There are some reports that I disagree with revive, in example, code
should not declare the type after variables.
In my opinion, on some cases, declaring the type make the code more
readable and explicit.
Since I did not want to add new configuration file, we changes it and
follow revive for now.
|
|
If the parameter withContent is true, which is default in MarshalJSON,
then all of the node content will be included in JSON output.
This also changes the parameter withoutModTime to withModTime because
working with negation make code not easy to read.
|
|
|
|
Instead of refreshing from the Root, find the directory that closes
to the requested path.
While at it, simplify the returned error.
|
|
The original idea of mocking the os.Stat to allow running test without
adding Sleep, but this cause our tests a little bit flaky.
|
|
Even thought empty directory does not contains file, from the parent
node _it is_ part of their content.
Also, there is a use case where memfs use as virtual file system (VFS),
as a layer with file system, where user can view list of directory,
create a directory or file on the fly.
If we skip scanning empty directory, that directory will not be visible.
|
|
The Child method return the child node based on its node.
|
|
In the history of our tests, we got the following test that sometime
failed,
2023/05/02 14:27:32 DirWatcher: on file changes /subdir/testfile: file does not exist
2023/05/02 14:27:34 DirWatcher: on file changes /sub/index.html: file does not exist
panic: test timed out after 1m0s
...
goroutine 1 [chan receive]:
github.com/shuLhan/share/lib/memfs.ExampleDirWatcher()
share/lib/memfs/dirwatcher_example_test.go:114 +0x165f
One of possible cause is that in the DirWatcher or Watcher we have
the following condition
// Watcher.start
if w.node.ModTime().Equal(newInfo.ModTime()) {
continue
}
// DirWatcher.processSubdirs
if node.ModTime().Equal(newDirInfo.ModTime()) {
continue
that result to true if the node modified in the same second as it was
created, which cause changes not propagated to qchanges.
The fix is by mocking the os.Stat by returning the FileInfo with ModTime
always increased by 1 second on every call.
|
|
While at it change the isIncluded parameter from os.FileMode to
os.FileInfo.
|
|
Previously, calling Merge(...), set the merged MemFS Root.SysPath to
"..".
Since we allow the TryDirect to access the file directly (if its set
to true), this may cause the file system leaks if returned MemFS set
this flag to true.
To prevent that, we set the SysPath to the first MemFS SysPath.
|
|
Previously, we assume that the list of files being Watch-ed is equal
to list of files being in Includes.
This may not be correct.
For example, we may want to watch "*.ts" files only but did not want
it to be included during GoEmbed.
This changes introduce list of pattern for files to be watched in
the WatchOptions.Watches field.
If this field is empty, only files pass the Includes filter will be
watched.
|
|
The Init provided to initialize MemFS instance if its Options is set
directly, not through New() function.
|
|
The realign save storage spaces on struct,
* Node: from 240 to 224 bytes (-16 bytes)
* Options: from 112 to 104 bytes (-8 bytes)
* PathNode: from 16 to 8 bytes (-8 bytes)
|
|
Previously, the PathNode has two fields to store the node in memory,
one is "v" that store map of path to *Node and another is "f" that
store the map of path to function that return a *Node.
The "f" is used to handle embedding Go generated code, but since
the template now can handle generated dynamic path and Node this field
is not used anymore.
|
|
One of the hardest when multiple packages is used is to detect where
there error happened. For example, if a program return an error io.EOF,
it's hard to detect the exact method or function that caused its,
especially when processing multiple files with network connection.
|
|
The reason why the field named V is because it's short. It's come
from my C/C++ experience that got carried away when writing this
package.
Now, after having more time writing Go, I prefer clarity over
cleverity(?).
|
|
The createRoot should not open the file, only call to os.Stat.
The scanDir should only accept the node to be scanned and the call to
Open and Close should be inside the method of scanDir not outside of it.
While at it, prefix all returned errors with method names.
|
|
Previously, MarshalJSON on memfs will return an object of map
of all PathNodes and on Node it will return an object.
This changes make it the JSON response consistent. If its directory
it will return the node object with its childs, without "content".
If its file, it will return the node object with content.
While at it, use single "mod_time" with value is epoch and return
the node ContentType as "content_type".
|
|
The Save method will write the new content to file system and update
the content of Node using Encode().
|
|
Previously, we did not check if the file size is 0 before reading the
content or updating the content type, which cause the read on file
return io.EOF and the file not added to caches.
This commit fix this issue by checking for zero file size and for
io.EOF when reading the file content.
|
|
Previously, if file is symbolic link and has different name with their
original file, it will return an error when we tried to open the file
parentpath/filename: no such file or directory
because we use the original file name, not the symlinked file name.
This commit fix this issue by not replacing the original FileInfo for
symlink but by setting only the size and mode.
|
|
If someone clone this repository, all the files modTime in the directory
testdata will be set to the current time. This will cause the test
fail.
This changes fix this issue by setting the modTime of all Node to static,
fixed time value.
|
|
Previously, without MarshalJSON, encoding the MemFS or Node object will
result in incomplete information, for example, missing name, modification
time, and size.
This commit implement the json.Marshaler in MemFS which encode the
PathNode sorted by keys in ascending order.
|
|
While at it, rename the test functions by the function or method that
they tests.
|
|
|
|
The Merge function merge one or more instance of MemFS into single MemFS
instance.
If there are two instance of Node that have the same path, the last
instance will be ignored.
|
|
On Arch Linux with Go tip, the content-type for JavaScript file is
"text/javascript". While on Ubuntu with Go 1.16 the content-type for
JavaScript file is "application/javascript".
|
|
The latest Go set the content type for JavaScript file as
"text/javascript; charset=utf-8", not "application/javascript".
|
|
Previously, the test.Assert and test.AssertBench functions has the
boolean parameter to print the stack trace of test in case its not equal.
Since this parameter is not mandatory and its usually always set to
"true", we remove them from function signature to simplify the call
to Assert and AssertBench.
|
|
Previously, AddFile set the internal path equal to path of file to be
included. This may cause conflict if the file is already included
due to the same sys path but different internal path.
This commit add parameter internalPath to set custom internal path in
the memfs map.
|
|
The field WithContent is not necessary if we set MaxFileSize to negative
value.
|
|
Previously, the generated Go code from memfs can be used only once
on the package that use it. For example, if we have two instances of
memfs.MemFS and both of them call GoGenerate(), when we load them back
again only the last one will be active and set the global variable
memfs.GeneratedPathNode.
This changes refactoring on how we use memfs by storing the
generated path node into variable that is defined by user and pass
them to New Options.
This changes affect package http and io.
|
|
|
|
Previously, the Go generated code from call to GoGenerate() set only
mapping of path to node. Calling Readdir() using http.File using
eoot, err := mfs.Open("/")
fis, err := root.Readdir(0)
will return nil on fis.
This changes add all child nodes to generated node.
|
|
This changes add parameter directory to be mounted in New(), so user
did not need to call Mount manually
This cause the Mount method and its sibling, Unmount and IsMounted
become unneeded, so we remove them.
|
|
|
|
|
