| Age | Commit message (Collapse) | Author |
|---|
|
When the Play command executed from the web user interface, there is a
possibility that concurrent requests set the sshConfig field in Awwan
struct at the same time.
In the integration tests for TestAwwan_Play_withLocal and
TestExecLocal_sudo, the data race happens when writing to the same
buffer for stdout and stderr, so we split them into separate buffers.
There is also data race in SSE connection, when handler for ExecuteTail
write to the same buffer with worker keep alive.
This has been fixed on pakakeh.go module.
|
|
In case we needs to add another parameter, which will do later, the
argument will be too long.
Using parameters is acceptable only for 2 to 3 arguments.
|
|
The `-shutdown-idle` option set the duration when server will stop
accepting new connections and shutting down.
|
|
**๐ผ internal/cmd/www-awwan: implement socket based activation using systemd**
This allow us to run www-awwan only when its activated in the local
environment.
**๐ผ cmd/awwan: implement socket based activation on serve command**
The "awwan serve" command now can run based on socket activation under
systemd.
**๐ผ _wui/doc: use default ciigo style**
The default ciigo style provides consistent style with kilabit.info
and have support for light and dark themes.
**๐ง all: fix data race in tests and [httpServer.ExecuteTail]**
In the test for AwwanLocal, use buffer with lock, so each write and
read is safe.
In the httpServer, the test found data race during ExecuteTail when
accessing [ExecResponse.EndAt].
We fix it by locking the resource during call to end() and when
accessing the EndAt field.
**๐ง _ops/awwan-tour: fix build with latest mkosi**
The option "Incremental" and "CacheDirectory" in mkosi.conf has been moved
to section "[Build]".
**๐ง all: use separate buffer for stdout and stderr in TestAwwanLocal**
This is to fix flaky test on TestAwwanLocal.
**๐ผ all: improve the Server-Sent Events (SSE) output**
In the ExecResponse, store the event in the Output instead of message
data, so the server can iterate the Output directly and pass it to
WriteEvent directly.
The event ID now start at 1 with type "begin".
This is to minimize confusion when comparing empty Last-Event-ID from
client, which is equal to 0.
|
|
In the ExecResponse, store the event in the Output instead of message
data, so the server can iterate the Output directly and pass it to
WriteEvent directly.
The event ID now start at 1 with type "begin".
This is to minimize confusion when comparing empty Last-Event-ID from
client, which is equal to 0.
|
|
The "awwan serve" command now can run based on socket activation under
systemd.
|
|
[NEW FEATURES]
Add option "$noparse" for magic "#put" command.
The "$noparse" option allow copying file without reading and parsing
the input file.
[CHORES]
Fix "reuse lint" warnings due to embedded SPDX annotation.
Fix field alignment on struct Statement.
|
|
This release mostly chores, updating the dependencies, linters, and fix
on integration testing with mkosi and Arch Linux.
[CHORE]
The default linters now "go vet" and internal/cmd/gocheck.
The gocheck is collection of static Analyzers that are not included in
"go vet" from golang.org/x/tools.
[CHORE]
The replace directive on esbuild and golang.org/x/crypt in go.mod has
been removed.
Maintaining fork is not easy and take a lot of time.
As we seen in crypto fork, we are 10 minor versions behind upstream.
So now we are back to using upstream.
[CHORE]
The integration tests using mkosi and Arch Linux has been fixed and
updated.
|
|
=== Chores
* all: replace licensing format to REUSE.toml
Using ".reuse/dep5" has been deprecated since REUSE v3.2.0.
* _wui: update wui submodule URL
The wui submodule has been renamed to "pakakeh.ts".
|
|
This release replace module "share" with "pakakeh.go".
In the "_wui", we use shared static assets from Cloud Storage.
The idea is to minimize noise in the logs that does not related to page
access and minimize binary size.
In the "_ops", we use shared mkosi cache in userโs home ".cache".
This is to minimize duplicate files and allow us to find or grep files
without excluding certains directory.
We also apply some recommendations from linters.
|
|
|
|
The "share" project has been moved to SourceHut with new name
"pakakeh.go".
|
|
=== Breaking changes
* all: refactoring "env-set" arguments
Previously, the "env-set" take the file argument as the last argument
and optional.
This changes move the file argument to the first argument so the bash
completion can detect and print the completion for list of keys.
While at it, fix handling key with quoted in EnvSet and EnvGet.
* all: refactoring env-get command
This changes the order of arguments of env-get command to pass the
directory first before the key.
The reason is to simplify auto-completion later from the command line.
=== New features
* all: add command env-keys
The "env-keys" command print list of environment variables under a
directory.
This command is internal, not documented, used by bash completion.
* all: add bash completion script
Using awwan from CLI now can automatically complete the arguments based on
the command and current parameter number.
|
|
Previously, the "env-set" take the file argument as the last argument
and optional.
This changes move the file argument to the first argument so the bash
completion can detect and print the completion for list of keys.
While at it, fix handling key with quoted in EnvSet and EnvGet.
Closes: https://todo.sr.ht/~shulhan/awwan/10
|
|
The "env-keys" command print list of environment variables under a
directory.
This command is internal, not documented, used by bash completion.
|
|
In this release we create https://tour.awwan.org where user can try and
learn awwan using step-by-step tutorial.
In the web-user interface (WUI) we add functionality to stop the local or
SSH execution.
=== Breaking changes
* all: make the magic line "#put:" use explicit source for encrypted file
=== New features
* _wui: implement button to stop execution
* all: implement HTTP API to stop local or play execution
=== Bug fixes
* all: check script file is a directory
* all: fix panic due to out of range when running "#require" statement
=== Enhancements
* all: delete the execution response and context cancellation on finished
* all: change the remote temporary directory to "~/.cache/awwan"
|
|
The update on share module fix several issues in "lib/ssh/config".
Including,
* Fix on SSH Config the default values.
Using Host with different Port now working as expected.
* Refactoring the Config merge.
Using IdentityFile or UserKnownHostsFile with relative path in
".ssh/config", for example
UserKnownHostsFile known_hosts
will now load the known_hosts in the same directory as config file.
|
|
Passing context allow the command Local or Play to be cancelled when
running in asynchronous mode, in this case when awwan run with WUI.
|
|
|
|
|
|
If the file to be copied contains sensitive data, putting them in
"/tmp" considered a security risk, even though it will be moved to
destination later.
The issue is when the "#put" command failed, the plain file is left
on "/tmp" directory.
This changes add additional advantage where we did not need to remove
the temporary directory on remote when execution completed, since the
temporary directory should be accessible by user only.
Implements: https://todo.sr.ht/~shulhan/awwan/8
|
|
=== New features
internal/cmd: add flag "address" for command www-awwan::
This is to allow using different address when running on local,
without conflict with "serve-www" task in Makefile.
all: implement command "env-get" to get value from environment files::
The env-get command get the value from environment files.
Syntax,
<key> [dir]
The "key" argument define the key where value is stored in environment
using "section:sub:name" format.
The "dir" argument is optional, its define the directory where
environment files will be loaded, recursively, from BaseDir to dir.
If its empty default to the current directory.
all: implement command to set environment value with "env-set"::
The env-set command set the value of environment file. Syntax,
<key> <value> <file>
The "key" argument define the key to be set using "section:sub:name"
format.
The "value" argument define the value key.
The "file" argument define path to environment file.
For example, to set the value for "name" under section "host" to
"myhost" in file "awwan.env" run
----
$ awwan env-set host::name myhost awwan.env
----
To set the value for key "pass" under section "user" subsection
"database" to value "s3cret" in file "awwan.env" run
----
$ awwan env-set user:database:pass s3cret awwan.env
----
_wui: implement Encrypt::
In the right side of Save button we now have a button Encrypt that
allow user to Encrypt opened file.
This require the workspace has been setup with private key
(.ssh/awwan.key) and pass file (.ssh/awwan.pass).
_wui: implement Decrypt::
In the right side of Save button we now have a button Decrypt that
allow user to Decrypt file with ".vault" extension only.
This require the workspace has been setup with private key
(.ssh/awwan.key) and pass file (.ssh/awwan.pass).
=== Breaking changes
all: make the magic line "#local" works on "local" command too::
In case we have a script that manage local host and remote server,
calling "play" on "#local" lines only always open the connection to
remote server.
To minimize opening unused connections, let the "#local" command works
on both commands. Its up to user which part of lines that they want
to execute on remote or local.
_wui: use CTRL+Enter to trigger save instead of CTRL+s on editor::
Using CTRL+s sometimes cause pressing s only trigger the save, due to
fast typing (or keyboard error?).
=== Bug fixes
all: close the SSH connection once Play finished::
Previously, we used to run awwan as CLI so each connection is open and
closed once the command completed.
Since we now use awwan WUI frequently, any command that execute Play
does not close the session immediately once finished.
This cause many connections open in remote server.
This changes close the SSH connections immediately once the Play
command finished.
all: fix memfs excludes regex::
Previously, the regex does not contains "^" and "$" which makes
file like "multi-user.target.wants" considered as ".tar" file and being
excluded.
_wui: update editor component::
Changes,
- fix paste that always end with newline
- fix editor content that got wrapped due to width
- update layout without using float
- replace execCommand with Selection
all: fix excludes on HTTP server related to .git::
The HTTP server should excludes ".git" directory only, not the other
files, like ".gitignore" or ".gitconfig".
all: always load SSH config when running Play::
In case awwan run with "serve" and we modify the ".ssh/config", the
changes does not detected by awwan because we only read ".ssh/config"
once we Awwan instance created.
This changes fix this issue by always loading SSH config every time
the Play method executed so the user CLI and WUI has the same
experiences.
=== Enhancements
all: reduce the response on HTTP endpoint on GET fs::
Previously, the HTTP endpoint for "GET /awwan/api/fs" return the
content of files when the requested node is a directory.
This is cause unnecessary load because when requesting directory we
only need list of file names not the content.
This changes reduce the response by returning only list of node child
without its content, which require update on share module on
[Node.JSON].
_wui: use the output for displaying notification::
Previously, we use a quick "pop-up" to display notification for each
information or error from WUI.
Sometimes this is annoying, it overlap the buttons, make it hard to
Save and Encrypt at the same time.
In this changes we move the nofication message to be displayed in
the output, same with output of execution.
all: use the same date format between log and mlog package::
In this way, the date-time output from log.Xxx and mlog.Xxx are
consistent.
all: remove duplicate errors logged on Copy, Put, and SudoCopy::
While at it, replace all call of [log.Printf] with [Request.mlog] so
error both written to stderr and to [Request.Output].
_wui: do not clear output when executing another command::
This allow user to see the output of previous command without opening
the log file.
_wui: store and load the vfs width in local storage::
This is allow user to resize vfs width in one window and when new
window is opened the vfs width is restored with the same size.
_wui: disable button "Local" and "Play" when clicked::
Once the execution completed, both buttons will be enabled again.
While at it, add an icon to show the execution status.
_wui: rename "Remote" to "Play"::
This is to make command between the CLI and WUI consistent.
all: refactoring HTTP endpoint for Execute::
Previously, the Execute endpoint wait for command execution to finish.
In case the command takes longer than proxy or server write timeout, it
will return with a timeout error to client.
In this changes, we generate an execution ID for each request and return
it immediately.
The new API "GET /awwan/api/execute/tail" implement Server-sent
events, accept the execution ID from "/awwan/api/execute".
Once called with valid ID, it will streaming the command output
to client.
By using this new API, the WUI can receive the output of command
immediately without waiting for all commands to be completed.
_wui: add vertical resizer, to resize between VFS and editor::
_wui: allow all content type but decrease max file size to 1 MB::
Previously, only file with type json, message, octet-stream, script,
text, or XML that can be opened by editor.
In this changes we allow all files as long as the size is less than
1MB.
_wui: update vfs component::
This changes allow user browse the crumb in path and item in the list
using tab key.
While at it, fix the layout to make VFS and editor aligned.
_wui: implement file filter::
This changes move the text input for creating new file above the list.
Filling the text field will filter the list based on the node name
using regular expression.
Another changes is for node with type directory now suffixed with "/".
|
|
Previously, we used to run awwan as CLI so each connection is open and
closed (forcefully).
Since we now use awwan WUI frequently, any command that execute Play
does not close the session immediately once finished.
This cause many connections open in remote server.
This changes close the SSH connections immediately once the Play command
finished.
|
|
Better to fix and follow the reported warnings rather than adding
configuration file to add an exception.
|
|
|
|
Previously, the Execute endpoint wait for command execution to finish.
In case the command takes longer than proxy or server write timeout, it
will return with an timeout error to client.
In this changes, we generate an execution ID for each request and return
it immediately.
The next commit will implement HTTP endpoint to fetch the latest status
and/or output by execution ID.
References: https://todo.sr.ht/~shulhan/awwan/5
|
|
Although log print to stderr, this allow user to filter between awwan
output and command output, for example by piping stderr to /dev/null
$ awwan env-get "key" dir/ 2>/dev/null
The output env-get is not poluted by other logs.
|
|
The "file" argument is optional, its define path to environment file.
If its empty it will be set to "awwan.env" in the current directory.
|
|
The env-get command get the value from environment files.
Syntax,
----
<key> [dir]
----
The "key" argument define the key where value is stored in environment
using "section:sub:name" format.
The "dir" argument is optional, its define the directory where environment
files will be loaded, recursively, from BaseDir to dir.
If its empty default to the current directory.
|
|
The env-set command set the value of environment file. Syntax,
<file> <key> <value>
The "file" argument define path to environment file.
The "key" argument define the key to be set using "section:sub:name"
format.
The "value" argument define the value key.
For example, to set the value for "name" under section "host" to
"myhost" in file "awwan.env" run
$ awwan env-set awwan.env host::name myhost
To set the value for key "pass" under section "user" subsection "database"
to value "s3cret" in file "awwan.env" run
$ awwan env-set awwan.env user:database:pass s3cret
|
|
|
|
In case awwan run with "serve" and we modify the ".ssh/config", the
changes does not detected by awwan because we only read ".ssh/config"
once we Awwan instance created.
This changes fix this issue by always loading SSH config everytime
the Play method executed so the user CLI and WUI has the same experiences.
|
|
Awwan now have a website at https://awwan.org.
=== New features
* all: implement remote "#get!" and "#put!" with owner and mode
The magic command "#get" and "#put" now have an inline options to set
the owner and permission of copied file. Example of usage are,
#get:$USER:$GROUP+$PERM src dst
#put!$USER:$GROUP+$PERM src dst
The $USER, $GROUP and $PERM are optionals.
If $USER and/or $GROUP is set, a copied file will have owner set to
user $USER and/or group to $GROUP. If $PERM is set, a copied file will
have the mode permission set to $MODE.
* all: add magic command "#local"
The magic command "#local" define the command to be executed using
shell in local environment. Its have effect and can only be used in
script that executed using "play".
In script that is executed using "local" it does nothing.
* _www: replace button "Clear selection" with text input for line range
Instead of using mouse to select which lines to be executed, let user
input it manually like in the CLI.
* all: log all execution into file
For each script execution, a file suffixed with ".log" will be created
in the same directory with the same name as script file. For example,
if the script is path is "a/b/c.aww" then the log file would named
"a/b/c.aww.log".
This is to provides history and audit in the future.
* cmd/awwan: add option "-address" to command serve
The "-address" option allow defining the HTTP server address to serve
the web-user interface.
=== Bug fixes
* all: trim spaces in passphrase when its read from file
Using vim, or UNIX in general, the file always end with "\n".
If we read the whole file then the passphrase will end with it,
this cause the decryption may fail (or wrong encryption passphrase
used).
* _www: fix saving file content using CTRL+s
The issue is using "this.editorOnSave" result on undefined "this"
inside the editorOnSave.
* all: remove the node when requested from HTTP API /awwan/api/fs
Previously, the HTTP API for deleting node only remove the file but
not the node in the memfs.
This changes remove the child node from memfs, so the next refresh on
directory will not contains the removed file.
* all: return the error as reponse in HTTP API execute
Previously, when the command execution failed, we check the error and
return it as HTTP status code 500. In this way, user cannot view the
log and actual error.
In this changes, if the command failed, we store the error in separate
field "Error" and return to the caller with HTTP status code 200.
=== Enhancements
* all: fix printing the statement to be executed
This fix missing magic command not printed in stdout.
* all: use "mlog.MultiLogger" to log Request output and error
By using "mlog.MultiLogger" every output or error can be written to
stdout/stderr and additional log writer that can collect both of them,
buffered and returned to the caller.
This changes simplify the HttpResponse to use only single output that
combine both stdout and stderr.
* _www: add button to resize editor and output
The button can be dragged up and down to resize both the editor and output
panes.
* _www: show confirmation when user open other file with unsaved changes
If user modify the current file without saving it and then open another
file, it will show confirmation dialog to continue opening file or cancel
it.
* all: on file save, make sure file end with line-feed
On some application, like haproxy configuration, line-feed (LF or "n") are
required, otherwise the application would not start.
* script: respect spaces when joining multi lines command
If a multi lines command does not have spaces or have multiple spaces,
join them as is. For example,
a\\
b
should return the value as ab, while
a \\
b
should return "a<space><space>b".
* _wui: various enhancements
Changes,
- The "File" tag now highlighted to distinguish with file name
- The "Execute" action moved to replace the Output, so we have some
additional horizontal space
- The "Output" tag removed
- Fix layout on mobile devices where height is set to static
- editor: re-render content after save
- editor: handle paste event manually
|
|
The "-address" option allow defining the HTTP server address to serve
the web-user interface.
|
|
Instead of passing stdout and stderr as "io.Writer", pass the Request,
so we can log using the Request mlog.
While at it, return error from sshClient rmdirAll so the caller can log
the error.
|
|
Previously, if there is any error from Local or Play before any statement
get executed it will not logged to log, also the Request close() is
not called since we put it as defer in the middle.
This changes fix this by moving down error handler into single exit
goto label.
|
|
For each script execution, a file suffixed with ".log" will be created
in the same directory with the same name as script file.
For example, if the script is path is "a/b/c.aww" then the log file
would named "a/b/c.aww.log".
This is to provides history and audit in the future.
|
|
|
|
Using flag "-dev" is more explicit than using environment variable.
|
|
By using "mlog.MultiLogger" every output or error can be written to
stdout/stderr and additional log writer that can collect both of them,
buffered and returned to the caller.
This changes simplify the HttpResponse to use only single output that
combine both stdout and stderr.
|
|
|
|
In remote environment, using magic command "#get:" or "#put:" with owner
and mode like "#get:$OWNER+$MODE" or "#put:$OWNER+MODE" will changes
the file owner to $USER or $GROUP and/or permission to $MODE.
The file owner will not works if user does not have permission.
|
|
The release-tip task build the binary on the current tip of main branch
and upload it to external storage to be consumed by public.
|
|
Instead of breaking the statement when its failed to execute, return
the error to the caller, Local.
|
|
This changes minimize code duplication when parsing magic command
between "#get:" and "#get!" using single function parseStatementGetPut.
In the returned Statement, we changes where to store the source argument
into args[0] instead of in cmd for better readability and minimize
confusion in Copy/SudoCopy.
While at it, we add unit test for "#get:".
Unit test for "#get!" is not possible right now, because it will prompt
for password.
|
|
This release add support for encryption, with two new commands "encrypt"
and "decrypt" for encrypting and decrypting file with RSA private key.
The awwan command also can read encrypted environment file with the name
".awwan.env.vault", so any secret variables can stored there and the
script that contains '{{.Val "..."}}' works as usual.
Any magic put "#put" also can copy encrypted file without any changes, as
long as the source file with ".vault" extension exist.
For environment where awwan need to be operated automatically, for example
in build system, awwan can read the private key's passphrase automatically
from the file ".ssh/awwan.pass".
=== Bug fixes
* all: do not expand environment during parseScript
* all: fix #require does not get executed on the same start
* _www: fix execute request that still use "begin_at" and "end_at"
=== Enhancements
* all: make .Vars, .Val, and .Vals panic if values is empty
* all: print any path relative to base directory
* all: move field bufout and buferr out of httpServer struct
|
|
All fields that use to serve HTTP API now moved inside one struct,
including memfsBase, bufout, and buferr.
|
|
This is to minimize log length in stdout and stderr, which make it
more readable.
|
|
This is to prevent copying or executing command with value that are
not defined or typo which make the result empty and may result in
undefined behaviour.
For example if we have "app_dir = /data/app" and command in the script
that remove that directory recursively,
sudo rm -r {{.Val "::app_dir}}/bin
will result removing "/bin" entirely.
|
|
Using hidden file (with "." prefix in name) cluttering the workspace
directory.
Since we already have ".ssh" as indicator of workspace, we should use it,
the same principal that ~/.ssh/ store user's private key.
|
