git/t/lib-httpd/apache.conf, branch gitk-resize-error

Merge branch 'jk/http-push-status-fix'

2021-10-29T22:43:12Z

"git push" client talking to an HTTP server did not diagnose the lack of the final status report from the other side correctly, which has been corrected. * jk/http-push-status-fix: transport-helper: recognize "expecting report" error from send-pack send-pack: complain about "expecting report" with --helper-status

send-pack: complain about "expecting report" with --helper-status

2021-10-18T20:26:52Z

When pushing to a server which erroneously omits the final ref-status report, the client side should complain about the refs for which we didn't receive the status (because we can't just assume they were updated). This works over most transports like ssh, but for http we'll print a very misleading "Everything up-to-date". It works for ssh because send-pack internally sets the status of each ref to REF_STATUS_EXPECTING_REPORT, and then if the server doesn't tell us about a particular ref, it will stay at that value. When we print the final status table, we'll see that we're still on EXPECTING_REPORT and complain then. But for http, we go through remote-curl, which invokes send-pack with "--stateless-rpc --helper-status". The latter option causes send-pack to return a machine-readable list of ref statuses to the remote helper. But ever since its inception in de1a2fdd38 (Smart push over HTTP: client side, 2009-10-30), the send-pack code has simply omitted mention of any ref which ended up in EXPECTING_REPORT. In the remote helper, we then take the absence of any status report from send-pack to mean that the ref was not even something we tried to send, and thus it prints "Everything up-to-date". Fortunately it does detect the eventual non-zero exit from send-pack, and propagates that in its own non-zero exit code. So at least a careful script invoking "git push" would notice the failure. But sending the misleading message on stderr is certainly confusing for humans (not to mention the machine-readable "push --porcelain" output, though again, any careful script should be checking the exit code from push, too). Nobody seems to have noticed because the server in this instance has to be misbehaving: it has promised to support the ref-status capability (otherwise the client will not set EXPECTING_REPORT at all), but didn't send us any. If the connection were simply cut, then send-pack would complain about getting EOF while trying to read the status. But if the server actually sends a flush packet (i.e., saying "now you have all of the ref statuses" without actually sending any), then the client ends up in this confused situation. The fix is simple: we should return an error message from "send-pack --helper-status", just like we would for any other error per-ref error condition (in the test I included, the server simply omits all ref status responses, but a more insidious version of this would skip only some of them). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

http-backend: handle HTTP_GIT_PROTOCOL CGI variable

2021-09-10T22:34:59Z

When a client requests the v2 protocol over HTTP, they set the Git-Protocol header. Webservers will generally make that available to our CGI as HTTP_GIT_PROTOCOL in the environment. However, that's not sufficient for upload-pack, etc, to respect it; they look in GIT_PROTOCOL (without the HTTP_ prefix). Either the webserver or the CGI is responsible for relaying that HTTP header into the GIT_PROTOCOL variable. Traditionally, our tests have configured the webserver to do so, but that's a burden on the server admin. We can make this work out of the box by having the http-backend CGI copy the contents of HTTP_GIT_PROTOCOL to GIT_PROTOCOL. There are no new tests here. By removing the SetEnvIf line from our test Apache config, we're now relying on this behavior of http-backend to trigger the v2 protocol there (and there are numerous tests that fail if this doesn't work). There is one subtlety here: we copy HTTP_GIT_PROTOCOL only if there is no existing GIT_PROTOCOL variable. That leaves the webserver admin free to override the client's decision if they choose. This is unlikely to be useful in practice, but is more flexible. And indeed, it allows the v2-to-v0 fallback test added in the previous commit to continue working. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

t5551: test v2-to-v0 http protocol fallback

2021-09-10T22:34:58Z

Since we use the v2 protocol by default, the connection of a v2 client to a v2 server is well covered by the test suite. And with the GIT_TEST_PROTOCOL_VERSION knob, we can easily test a v0 client connecting to a v2-aware server (which will then just speak v0). But we have no regular tests that a v2 client, when encountering a non-v2-aware server, will correctly fall back to using v0. In theory this is a job for the cross-version tests in t/interop, but: - they cover only git:// and file:// clones - they are not part of the usual test suite, so nobody ever runs them anyway Since using v2 over http requires configuring the web server to pass along the Git-Protocol header, we can easily create a situation where the server does not respect the v2 probe, and the conversation falls back to v0. This works just fine. This new test is not about fixing any particular bug, but just making sure that the system works (and continues to work) as expected. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

remote-curl: error on incomplete packet

2020-05-24T23:26:00Z

Currently, remote-curl acts as a proxy and blindly forwards packets between an HTTP server and fetch-pack. In the case of a stateless RPC connection where the connection is terminated with a partially written packet, remote-curl will blindly send the partially written packet before waiting on more input from fetch-pack. Meanwhile, fetch-pack will read the partial packet and continue reading, expecting more input. This results in a deadlock between the two processes. For a stateless connection, inspect packets before sending them and error out if a packet line packet is incomplete. Helped-by: Jeff King Signed-off-by: Denton Liu Signed-off-by: Junio C Hamano

t/lib-httpd: avoid using macOS' sed

2020-02-27T17:58:41Z

Among other differences relative to GNU sed, macOS' sed always ends its output with a trailing newline, even if the input did not have such a trailing newline. Surprisingly, this makes three httpd-based tests fail on macOS: t5616, t5702 and t5703. ("Surprisingly" because those tests have been around for some time, but apparently nobody runs them on macOS with a working Apache2 setup.) The reason is that we use `sed` in those tests to filter the response of the web server. Apart from the fact that we use GNU constructs (such as using a space after the `c` command instead of a backslash and a newline), we have another problem: macOS' sed LF-only newlines while webservers are supposed to use CR/LF ones. Even worse, t5616 uses `sed` to replace a binary part of the response with a new binary part (kind of hoping that the replaced binary part does not contain a 0x0a byte which would be interpreted as a newline). To that end, it calls on Perl to read the binary pack file and hex-encode it, then calls on `sed` to prefix every hex digit pair with a `\x` in order to construct the text that the `c` statement of the `sed` invocation is supposed to insert. So we call Perl and sed to construct a sed statement. The final nail in the coffin is that macOS' sed does not even interpret those `\x` constructs. Let's just replace all of that by Perl snippets. With Perl, at least, we do not have to deal with GNU vs macOS semantics, we do not have to worry about unwanted trailing newlines, and we do not have to spawn commands to construct arguments for other commands to be spawned (i.e. we can avoid a whole lot of shell scripting complexity). The upshot is that this fixes t5616, t5702 and t5703 on macOS with Apache2. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano

t/lib-httpd: pass LSAN_OPTIONS through apache

2019-05-08T04:21:23Z

Just as we instruct Apache to pass through ASAN_OPTIONS (so that server-side Git programs it spawns will respect our options while running the tests), we should do the same with LSAN_OPTIONS. Otherwise trying to collect a list of leaks like: export LSAN_OPTIONS=exitcode=0:log_path=/tmp/lsan make SANITIZE=leak test won't work for http tests (the server-side programs won't log their leaks to the right place, and they'll prematurely die, producing a spurious test failure). Signed-off-by: Jeff King Acked-by: Josh Steadmon Signed-off-by: Junio C Hamano

t5551: test server-side ERR packet

2019-02-06T20:20:23Z

When a smart HTTP server sends an error message via pkt-line, we detect the error due to using PACKET_READ_DIE_ON_ERR_PACKET. This case was added by 2d103c31c2 (pack-protocol.txt: accept error packets in any context, 2018-12-29), but not covered by tests. Signed-off-by: Josh Steadmon Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Merge branch 'jt/fetch-v2-sideband'

2019-02-05T22:26:11Z

"git fetch" and "git upload-pack" learned to send all exchange over the sideband channel while talking the v2 protocol. * jt/fetch-v2-sideband: tests: define GIT_TEST_SIDEBAND_ALL {fetch,upload}-pack: sideband v2 fetch response sideband: reverse its dependency on pkt-line pkt-line: introduce struct packet_writer pack-protocol.txt: accept error packets in any context Use packet_reader instead of packet_read_line

tests: define GIT_TEST_SIDEBAND_ALL

2019-01-17T19:25:07Z

Define a GIT_TEST_SIDEBAND_ALL environment variable meant to be used from tests. When set to true, this overrides uploadpack.allowsidebandall to true, allowing the entire test suite to be run as if this configuration is in place for all repositories. As of this patch, all tests pass whether GIT_TEST_SIDEBAND_ALL is unset or set to 1. Signed-off-by: Jonathan Tan Signed-off-by: Junio C Hamano