aboutsummaryrefslogtreecommitdiff
path: root/ci
AgeCommit message (Collapse)Author
2025-04-09ci: upgrade `sparse` to supported build agentsJohannes Schindelin
The `sparse` job still uses the `ubuntu-20.04` runner pool, but that pool is about to go away, so let's stop using it. There is no `sparse-22.04` artifact provided by the "Build sparse for Ubuntu" Azure Pipeline, but that is not necessary anyway because Ubuntu 22.04 has the `sparse` package: https://packages.ubuntu.com/jammy/sparse Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-08tests: remove GIT_TEST_MERGE_ALGORITHM and test_expect_merge_algorithmElijah Newren
Both of these existed to allow us to reuse all the merge-related tests in the testsuite while easily flipping between the 'recursive' and the 'ort' backends. Now that we have removed merge-recursive and remapped 'recursive' to mean 'ort', we don't need this scaffolding anymore. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-08Merge branch 'ps/ci-meson-check-build-docs'Junio C Hamano
CI update. * ps/ci-meson-check-build-docs: ci: perform build and smoke tests for Meson docs
2025-03-21ci: add build checking for side-effects in assert() callsElijah Newren
It is a big no-no to have side-effects in an assertion, because if the assert() is compiled out, you don't get that side-effect, leading to the code behaving differently. That can be a large headache to debug. We have roughly 566 assert() calls in our codebase (my grep might have picked up things that aren't actually assert() calls, but most appeared to be). All but 9 of them can be determined by gcc to be free of side effects with a clever redefine of assert() provided by Bruno De Fraine (from https://stackoverflow.com/questions/10593492/catching-assert-with-side-effects), who upon request has graciously placed his two-liner into the public domain without warranty of any kind. The current 9 assert() calls flagged by this clever redefinition of assert() appear to me to be free of side effects as well, but are too complicated for a compiler/linker to figure that since each assertion involves some kind of function call. Add a CI job which will find and report these possibly problematic assertions, and have the job suggest to the user that they replace these with ASSERT() calls. Example output from running: ``` ERROR: The compiler could not verify the following assert() calls are free of side-effects. Please replace with ASSERT() calls. /home/newren/floss/git/diffcore-rename.c:1409 assert(!dir_rename_count || strmap_empty(dir_rename_count)); /home/newren/floss/git/merge-ort.c:1645 assert(renames->deferred[side].trivial_merges_okay && !strset_contains(&renames->deferred[side].target_dirs, path)); /home/newren/floss/git/merge-ort.c:794 assert(omittable_hint == (!starts_with(type_short_descriptions[type], "CONFLICT") && !starts_with(type_short_descriptions[type], "ERROR")) || type == CONFLICT_DIR_RENAME_SUGGESTED); /home/newren/floss/git/merge-recursive.c:1200 assert(!merge_remote_util(commit)); /home/newren/floss/git/object-file.c:2709 assert(would_convert_to_git_filter_fd(istate, path)); /home/newren/floss/git/parallel-checkout.c:280 assert(is_eligible_for_parallel_checkout(pc_item->ce, &pc_item->ca)); /home/newren/floss/git/scalar.c:244 assert(have_fsmonitor_support()); /home/newren/floss/git/scalar.c:254 assert(have_fsmonitor_support()); /home/newren/floss/git/sequencer.c:4968 assert(!(opts->signoff || opts->no_commit || opts->record_origin || should_edit(opts) || opts->committer_date_is_author_date || opts->ignore_date)); ``` Note that if there are possibly problematic assertions, not necessarily all of them will be shown in a single run, because the compiler errors may include something like "ld: ... more undefined references to `not_supposed_to_survive' follow" instead of listing each individually. But in such cases, once you clean up a few that are shown in your first run, subsequent runs will show (some of) the ones that remain, allowing you to iteratively remove them all. Helped-by: Bruno De Fraine <defraine@gmail.com> Signed-off-by: Elijah Newren <newren@gmail.com> Acked-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-03-12ci: perform build and smoke tests for Meson docsPatrick Steinhardt
Our "documentation" CI job performs a couple of tests against our documentation. Part of these tests is to check whether documentation builds at all and whether it spits out the expected set of files. We don't yet have such a test for Meson, which means that we wouldn't notice at all if building the documentation were to break. As a result, breakages as fixed by 87eccc3a81d (meson: fix building technical and howto docs, 2025-03-02) are easy to go unnoticed. Address this test gap by starting to build both manpages and HTML sites as part of the CI job. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-02-20ci: exercise credential helpersPatrick Steinhardt
Wire up credential helpers in our CI runs so that we can rest assured that they compile and (if tests are available) function correctly. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-02-20ci: fix propagating UTF-8 test locale in musl-based Meson jobPatrick Steinhardt
The musl-based Meson job is supposed to explicitly specify the UTF-8 locale used for testing, which has been introduced with 84bb5eeace7 (ci: switch linux-musl to use Meson, 2025-01-28). That commit had two issues though: - We continue to refer to "linux-musl", even though the job has been renamed in the same commit to "linux-musl-meson". - We use the wrong option name to specify the locale. This was not noticed though due to the first issue. Fix both of these issues by fixing both the job and option naems. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-02-06Merge branch 'ps/zlib-ng'Junio C Hamano
The code paths to interact with zlib has been cleaned up in preparation for building with zlib-ng. * ps/zlib-ng: ci: make "linux-musl" job use zlib-ng ci: switch linux-musl to use Meson compat/zlib: allow use of zlib-ng as backend git-zlib: cast away potential constness of `next_in` pointer compat/zlib: provide stubs for `deflateSetHeader()` compat/zlib: provide `deflateBound()` shim centrally git-compat-util: move include of "compat/zlib.h" into "git-zlib.h" compat: introduce new "zlib.h" header git-compat-util: drop `z_const` define compat: drop `uncompress2()` compatibility shim
2025-02-06Merge branch 'ps/ci-misc-updates'Junio C Hamano
CI updates (containerization, dropping stale ones, etc.). * ps/ci-misc-updates: ci: remove stale code for Azure Pipelines ci: use latest Ubuntu release ci: stop special-casing for Ubuntu 16.04 gitlab-ci: add linux32 job testing against i386 gitlab-ci: remove the "linux-old" job github: simplify computation of the job's distro github: convert all Linux jobs to be containerized github: adapt containerized jobs to be rootless t7422: fix flaky test caused by buffered stdout t0060: fix EBUSY in MinGW when setting up runtime prefix
2025-02-03Merge branch 'ps/build-meson-fixes'Junio C Hamano
More build fixes and enhancements on meson based build procedure. * ps/build-meson-fixes: ci: wire up Visual Studio build with Meson ci: raise error when Meson generates warnings meson: fix compilation with Visual Studio meson: make the CSPRNG backend configurable meson: wire up fuzzers meson: wire up generation of distribution archive meson: wire up development environments meson: fix dependencies for generated headers meson: populate project version via GIT-VERSION-GEN GIT-VERSION-GEN: allow running without input and output files GIT-VERSION-GEN: simplify computing the dirty marker
2025-02-03Merge branch 'ps/3.0-remote-deprecation'Junio C Hamano
Following the procedure we established to introduce breaking changes for Git 3.0, allow an early opt-in for removing support of $GIT_DIR/branches/ and $GIT_DIR/remotes/ directories to configure remotes. * ps/3.0-remote-deprecation: remote: announce removal of "branches/" and "remotes/" builtin/pack-redundant: remove subcommand with breaking changes ci: repurpose "linux-gcc" job for deprecations ci: merge linux-gcc-default into linux-gcc Makefile: wire up build option for deprecated features
2025-01-28ci: make "linux-musl" job use zlib-ngPatrick Steinhardt
We don't yet have any test coverage for the new zlib-ng backend as part of our CI. Add it by installing zlib-ng in Alpine Linux, which causes Meson to pick it up automatically. Note that we are somewhat limited with regards to where we run that job: Debian-based distributions don't have zlib-ng in their repositories, Fedora has it but doesn't run tests, and Alma Linux doesn't have the package either. Alpine Linux does have it available and is running our test suite, which is why it was picked. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-28ci: switch linux-musl to use MesonPatrick Steinhardt
Switch over the "linux-musl" job to use Meson instead of Makefiles. This is done due to multiple reasons: - It simplifies our CI infrastructure a bit as we don't have to manually specify a couple of build options anymore. - It verifies that Meson detects and sets those build options automatically. - It makes it easier for us to wire up a new CI job using zlib-ng as backend. One platform compatibility that Meson cannot easily detect automatically is the `GIT_TEST_UTF8_LOCALE` variable used in tests. Wire up a build option for it, which we set via a new "MESONFLAGS" environment variable. Note that we also drop the CC variable, which is set to "gcc". We already default to GCC when CC is unset in "ci/lib.sh", so this is not needed. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-22ci: raise error when Meson generates warningsPatrick Steinhardt
Meson prints warnings in several cases, like for example when using a feature supported by the current version of Meson, but not yet supported by the minimum required version as declared by the project. These warnings will not cause the setup to fail by default, which makes it quite easy to miss them. Improve this by passing `--fatal-meson-warnings` to `meson setup` so that our CI jobs will fail on warnings. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-22meson: wire up fuzzersPatrick Steinhardt
Meson does not yet know to build our fuzzers. Introduce a new build option "fuzzers" and wire up the fuzzers in case it is enabled. Adapt our CI jobs so that they build the fuzzers by default. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-22ci: repurpose "linux-gcc" job for deprecationsPatrick Steinhardt
The "linux-gcc" job isn't all that interesting by itself and can be considered more or less the "standard" job: it is running with a reasonably up-to-date image and uses GCC as a compiler, both of which we already cover in other jobs. There is one exception though: we change the default branch to be "main" instead of "master", so it is forging ahead a bit into the future to make sure that this change does not cause havoc. So let's expand on this a bit and also add the new "WITH_BREAKING_CHANGES" flag to the mix. Rename the job to "linux-breaking-changes" accordingly. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-22ci: merge linux-gcc-default into linux-gccPatrick Steinhardt
The "linux-gcc-default" job is mostly doing the same as the "linux-gcc" job, except for a couple of minor differences: - We use an explicit GCC version instead of the default version provided by the distribution. We have other jobs that test with "gcc-8", making this distinction pointless. - We don't set up the Python version explicitly, and instead use the default Python version. Python 2 has been end-of-life for quite a while now though, making this distinction less interesting. - We set up the default branch name to be "main" in "linux-gcc". We have other testcases that don't and also some that explicitly use "master". - We use "ubuntu:20.04" in one job and "ubuntu:latest" in another. We already have a couple other jobs testing these respectively. So overall, the job does not add much to our test coverage. Drop the "linux-gcc-default" job and adapt "linux-gcc" to start using the default GCC compiler, effectively merging those two jobs into one. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-10ci: remove stale code for Azure PipelinesPatrick Steinhardt
Support for Azure Pipelines has been retired in 6081d3898f (ci: retire the Azure Pipelines definition, 2020-04-11) in favor of GitHub Actions. Our CI library still has some infrastructure left for Azure though that is now unused. Remove it. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-10ci: stop special-casing for Ubuntu 16.04Patrick Steinhardt
With c85bcb5de1 (gitlab-ci: switch from Ubuntu 16.04 to 20.04, 2024-10-31) we have adapted the last CI job to stop using Ubuntu 16.04 in favor of Ubuntu 20.04. Remove the special-casing we still have in our CI scripts. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-10gitlab-ci: add linux32 job testing against i386Patrick Steinhardt
Add another job to GitLab CI that tests against the i386 architecture. This job is equivalent to the same job in GitHub Workflows. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-10github: simplify computation of the job's distroPatrick Steinhardt
We explicitly list the distro of Linux-based jobs, but it is equivalent to the name of the image in almost all cases, except that colons are replaced with dashes. Drop the redundant information and massage it in our CI scripts, which is equivalent to how we do it in GitLab CI. There are a couple of exceptions: - The "linux32" job, whose distro name is different than the image name. This is handled by adapting all sites to use the new name. - The "alpine" and "fedora" jobs, neither of which specify a tag for their image. This is handled by adding the "latest" tag. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-10github: adapt containerized jobs to be rootlessPatrick Steinhardt
The containerized jobs in GitHub Actions run as root, giving them special permissions to for example delete files even when the user shouldn't be able to due to file permissions. This limitation keeps us from using containerized jobs for most of our Ubuntu-based jobs as it causes a number of tests to fail. Adapt the jobs to create a separate user that executes the test suite. This follows similar infrastructure that we already have in GitLab CI. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-02Merge branch 'jk/lsan-race-ignore-false-positive'Junio C Hamano
CI jobs that run threaded programs under LSan has been giving false positives from time to time, which has been worked around. This is an alternative to the jk/lsan-race-with-barrier topic with much smaller change to the production code. * jk/lsan-race-ignore-false-positive: test-lib: ignore leaks in the sanitizer's thread code test-lib: check leak logs for presence of DEDUP_TOKEN test-lib: simplify leak-log checking test-lib: rely on logs to detect leaks Revert barrier-based LSan threading race workaround
2025-01-01Revert barrier-based LSan threading race workaroundJunio C Hamano
The extra "barrier" approach was too much code whose sole purpose was to work around a race that is not even ours (i.e. in LSan's teardown code). In preparation for queuing a solution taking a much-less-invasive approach, let's revert them.
2025-01-01Merge branch 'jk/lsan-race-with-barrier'Junio C Hamano
CI jobs that run threaded programs under LSan has been giving false positives from time to time, which has been worked around. * jk/lsan-race-with-barrier: grep: work around LSan threading race with barrier index-pack: work around LSan threading race with barrier thread-utils: introduce optional barrier type Revert "index-pack: spawn threads atomically" test-lib: use individual lsan dir for --stress runs
2025-01-01Merge branch 'ps/weak-sha1-for-tail-sum-fix'Junio C Hamano
An earlier "csum-file checksum does not have to be computed with sha1dc" topic had a few code paths that had initialized an implementation of a hash function to be used by an unmatching hash by mistake, which have been corrected. * ps/weak-sha1-for-tail-sum-fix: ci: exercise unsafe OpenSSL backend builtin/fast-import: fix segfault with unsafe SHA1 backend bulk-checkin: fix segfault with unsafe SHA1 backend
2024-12-30ci: exercise unsafe OpenSSL backendPatrick Steinhardt
In the preceding commit we have fixed a segfault when using an unsafe SHA1 backend that is different from the safe one. This segfault only went by unnoticed because we never set up an unsafe backend in our CI systems. Fix this ommission by setting `OPENSSL_SHA1_UNSAFE` in our TEST-vars job. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-30thread-utils: introduce optional barrier typeJeff King
One thread primitive we don't yet support is a barrier: it waits for all threads to reach a synchronization point before letting any of them continue. This would be useful for avoiding the LSan race we see in index-pack (and other places) by having all threads complete their initialization before any of them start to do real work. POSIX introduced a pthread_barrier_t in 2004, which does what we want. But if we want to rely on it: 1. Our Windows pthread emulation would need a new set of wrapper functions. There's a Synchronization Barrier primitive there, which was introduced in Windows 8 (which is old enough for us to depend on). 2. macOS (and possibly other systems) has pthreads but not pthread_barrier_t. So there we'd have to implement our own barrier based on the mutex and cond primitives. Those are do-able, but since we only care about avoiding races in our LSan builds, there's an easier way: make it a noop on systems without a native pthread barrier. This patch introduces a "maybe_thread_barrier" API. The clunky name (rather than just using pthread_barrier directly) should hopefully clue people in that on some systems it will do nothing. It's wired to a Makefile knob which has to be triggered manually, and we enable it for the linux-leaks CI jobs (since we know we'll have it there). There are some other possible options: - we could turn it on all the time for Linux systems based on uname. But we really only care about it for LSan builds, and there is no need to add extra code to regular builds. - we could turn it on only for LSan builds. But that would break builds on non-Linux platforms (like macOS) that otherwise should support sanitizers. - we could trigger only on the combination of Linux and LSan together. This isn't too hard to do, but the uname check isn't completely accurate. It is really about what your libc supports, and non-glibc systems might not have it (though at least musl seems to). So we'd risk breaking builds on those systems, which would need to add a new knob. Though the upside would be that running local "make SANITIZE=leak test" would be protected automatically. And of course none of this protects LSan runs from races on systems without pthread barriers. It's probably OK in practice to protect only our CI jobs, though. The race is rare-ish and most leak-checking happens through CI. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-23Merge branch 'ps/ci-meson'Junio C Hamano
The meson-build procedure is integrated into CI to catch and prevent bitrotting. * ps/ci-meson: ci: wire up Meson builds t: introduce compatibility options to clar-based tests t: fix out-of-tree tests for some git-p4 tests Makefile: detect missing Meson tests meson: detect missing tests at configure time t/unit-tests: rename clar-based unit tests to have a common prefix Makefile: drop -DSUPPRESS_ANNOTATED_LEAKS ci/lib: support custom output directories when creating test artifacts
2024-12-23Merge branch 'ps/ci-gitlab-update'Junio C Hamano
GitLab CI updates. * ps/ci-gitlab-update: ci/lib: fix "CI setup" sections with GitLab CI ci/lib: do not interpret escape sequences in `group ()` arguments ci/lib: remove duplicate trap to end "CI setup" group gitlab-ci: update macOS images to Sonoma
2024-12-15Merge branch 'ps/build'Junio C Hamano
Build procedure update plus introduction of Meson based builds. * ps/build: (24 commits) Introduce support for the Meson build system Documentation: add comparison of build systems t: allow overriding build dir t: better support for out-of-tree builds Documentation: extract script to generate a list of mergetools Documentation: teach "cmd-list.perl" about out-of-tree builds Documentation: allow sourcing generated includes from separate dir Makefile: simplify building of templates Makefile: write absolute program path into bin-wrappers Makefile: allow "bin-wrappers/" directory to exist Makefile: refactor generators to be PWD-independent Makefile: extract script to generate gitweb.js Makefile: extract script to generate gitweb.cgi Makefile: extract script to massage Python scripts Makefile: extract script to massage Shell scripts Makefile: use "generate-perl.sh" to massage Perl library Makefile: extract script to massage Perl scripts Makefile: consistently use PERL_PATH Makefile: generate doc versions via GIT-VERSION-GEN Makefile: generate "git.rc" via GIT-VERSION-GEN ...
2024-12-13Merge branch 'es/oss-fuzz'Junio C Hamano
Backport oss-fuzz tests for us to our codebase. * es/oss-fuzz: fuzz: port fuzz-url-decode-mem from OSS-Fuzz fuzz: port fuzz-parse-attr-line from OSS-Fuzz fuzz: port fuzz-credential-from-url-gently from OSS-Fuzz
2024-12-13ci: wire up Meson buildsPatrick Steinhardt
Wire up CI builds for both GitLab and GitHub that use the Meson build system. While the setup is mostly trivial, one gotcha is the test output directory used to be in "t/", but now it is contained in the build directory. To unify the logic across Makefile- and Meson-based builds we explicitly set up the `TEST_OUTPUT_DIRECTORY` variable so that it is the same for both build systems. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-13ci/lib: support custom output directories when creating test artifactsPatrick Steinhardt
Update `create_failed_test_artifacts ()` so that it can handle arbitrary test output directories. This fixes creation of these artifacts for macOS on GitLab CI, which uses a separate output directory already. This will also be used by our out-of-tree builds with Meson. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-12ci/lib: fix "CI setup" sections with GitLab CIPatrick Steinhardt
Whenever we source "ci/lib.sh" we wrap the directives in a separate group so that they can easily be collapsed in the web UI. And as we source the script multiple times during a single CI run we thus end up with the same section name reused multiple times, as well. This is broken on GitLab CI though, where reusing the same group name is not supported. The consequence is that only the last of these sections can be collapsed. Fix this issue by including the name of the sourcing script in the group's name. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-12ci/lib: do not interpret escape sequences in `group ()` argumentsPatrick Steinhardt
We use printf to set up sections with GitLab CI, which requires us to print a bunch of escape sequences via printf. The group name is controlled by the user and is expanded directly into the formatting string, which may cause problems in case the argument contains escape sequences or formatting directives. Fix this potential issue by using formatting directives to pass variable data. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-12ci/lib: remove duplicate trap to end "CI setup" groupPatrick Steinhardt
We exlicitly trap on EXIT in order to end the "CI setup" group. This isn't necessary though given that `begin_group ()` already sets up the trap for us. Remove the duplicate trap. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-12Merge branch 'ps/build' into ps/3.0-remote-deprecationJunio C Hamano
* ps/build: (24 commits) Introduce support for the Meson build system Documentation: add comparison of build systems t: allow overriding build dir t: better support for out-of-tree builds Documentation: extract script to generate a list of mergetools Documentation: teach "cmd-list.perl" about out-of-tree builds Documentation: allow sourcing generated includes from separate dir Makefile: simplify building of templates Makefile: write absolute program path into bin-wrappers Makefile: allow "bin-wrappers/" directory to exist Makefile: refactor generators to be PWD-independent Makefile: extract script to generate gitweb.js Makefile: extract script to generate gitweb.cgi Makefile: extract script to massage Python scripts Makefile: extract script to massage Shell scripts Makefile: use "generate-perl.sh" to massage Perl library Makefile: extract script to massage Perl scripts Makefile: consistently use PERL_PATH Makefile: generate doc versions via GIT-VERSION-GEN Makefile: generate "git.rc" via GIT-VERSION-GEN ...
2024-12-12Merge branch 'ps/build' into ps/ci-mesonJunio C Hamano
* ps/build: (24 commits) Introduce support for the Meson build system Documentation: add comparison of build systems t: allow overriding build dir t: better support for out-of-tree builds Documentation: extract script to generate a list of mergetools Documentation: teach "cmd-list.perl" about out-of-tree builds Documentation: allow sourcing generated includes from separate dir Makefile: simplify building of templates Makefile: write absolute program path into bin-wrappers Makefile: allow "bin-wrappers/" directory to exist Makefile: refactor generators to be PWD-independent Makefile: extract script to generate gitweb.js Makefile: extract script to generate gitweb.cgi Makefile: extract script to massage Python scripts Makefile: extract script to massage Shell scripts Makefile: use "generate-perl.sh" to massage Perl library Makefile: extract script to massage Perl scripts Makefile: consistently use PERL_PATH Makefile: generate doc versions via GIT-VERSION-GEN Makefile: generate "git.rc" via GIT-VERSION-GEN ...
2024-12-10Merge branch 'bc/ancient-ci'Junio C Hamano
Drop support for ancient environments in various CI jobs. * bc/ancient-ci: Add additional CI jobs to avoid accidental breakage ci: remove clause for Ubuntu 16.04 gitlab-ci: switch from Ubuntu 16.04 to 20.04
2024-12-07Makefile: refactor GIT-VERSION-GEN to be reusablePatrick Steinhardt
Our "GIT-VERSION-GEN" script always writes the "GIT-VERSION-FILE" into the current directory, where the expectation is that it should exist in the source directory. But other build systems that support out-of-tree builds may not want to do that to keep the source directory pristine, even though CMake currently doesn't care. Refactor the script such that it won't write the "GIT-VERSION-FILE" directly anymore, but instead knows to replace @PLACEHOLDERS@ in an arbitrary input file. This allows us to simplify the logic in CMake to determine the project version, but can also be reused later on in order to generate other files that need to contain version information like our "git.rc" file. While at it, change the format of the version file by removing the spaces around the equals sign. Like this we can continue to include the file in our Makefiles, but can also start to source it in shell scripts in subsequent steps. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-11-21test-lib: unconditionally enable leak checkingPatrick Steinhardt
Over the last two releases we have plugged a couple hundred of memory leaks exposed by the Git test suite. With the preceding commits we have finally fixed the last leak exposed by our test suite, which means that we are now basically leak free wherever we have branch coverage. From hereon, the Git test suite should ideally stay free of memory leaks. Most importantly, any test suite that is being added should automatically be subject to the leak checker, and if that test does not pass it is a strong signal that the added code introduced new memory leaks and should not be accepted without further changes. Drop the infrastructure around TEST_PASSES_SANITIZE_LEAK to reflect this new requirement. Like this, all test suites will be subject to the leak checker by default. This is being intentionally strict, but we still have an escape hatch: the SANITIZE_LEAK prerequisite. There is one known case in t5601 where the leak sanitizer itself is buggy, so adding this prereq in such cases is acceptable. Another acceptable situation is when a newly added test uncovers preexisting memory leaks: when fixing that memory leak would be sufficiently complicated it is fine to annotate and document the leak accordingly. But in any case, the burden is now on the patch author to explain why exactly they have to add the SANITIZE_LEAK prerequisite. The TEST_PASSES_SANITIZE_LEAK annotations will be dropped in the next patch. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-11-01Merge branch 'ss/duplicate-typos'Taylor Blau
Typofixes. * ss/duplicate-typos: global: Fix duplicate word typos
2024-11-01Add additional CI jobs to avoid accidental breakagebrian m. carlson
In general, we'd like to make sure Git works on the LTS versions of major Linux distributions. To do that, let's add CI jobs for the oldest regular (non-extended) LTS versions of the major distributions: Ubuntu 20.04, Debian 11, and RHEL 8. Because RHEL isn't available to the public at no charge, use AlmaLinux, which is binary compatible with it. Note that Debian does not offer the language-pack packages, but suitable locale support can be installed with the locales-all package. Otherwise, use the set of installation instructions which exist and are most similar to the existing supported distros. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2024-11-01ci: remove clause for Ubuntu 16.04brian m. carlson
We're no longer testing this version and it's well beyond regular LTS support now, so remove the stanza for it from the case statement in our CI code. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2024-10-25Merge branch 'ps/ci-gitlab-windows'Taylor Blau
Enable Windows-based CI in GitLab. * ps/ci-gitlab-windows: gitlab-ci: exercise Git on Windows gitlab-ci: introduce stages and dependencies ci: handle Windows-based CI jobs in GitLab CI ci: create script to set up Git for Windows SDK t7300: work around platform-specific behaviour with long paths on MinGW
2024-10-21global: Fix duplicate word typosSven Strickroth
Used regex to find these typos: (?<!struct )(?<=\s)([a-z]{1,}) \1(?=\s) Signed-off-by: Sven Strickroth <email@cs-ware.de> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2024-10-16fuzz: port fuzz-url-decode-mem from OSS-FuzzEric Sesterhenn
Git's fuzz tests are run continuously as part of OSS-Fuzz [1]. Several additional fuzz tests have been contributed directly to OSS-Fuzz; however, these tests are vulnerable to bitrot because they are not built during Git's CI runs, and thus breaking changes are much less likely to be noticed by Git contributors. Port one of these tests back to the Git project: fuzz-url-decode-mem This test was originally written by Eric Sesterhenn as part of a security audit of Git [2]. It was then contributed to the OSS-Fuzz repo in commit c58ac4492 (Git fuzzing: uncomment the existing and add new targets. (#11486), 2024-02-21) by Jaroslav Lobačevski. I (Josh Steadmon) have verified with both Eric and Jaroslav that they're OK with moving this test to the Git project. [1] https://github.com/google/oss-fuzz [2] https://ostif.org/wp-content/uploads/2023/01/X41-OSTIF-Gitlab-Git-Security-Audit-20230117-public.pdf Co-authored-by: Jaroslav Lobačevski <jarlob@gmail.com> Co-authored-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2024-10-16fuzz: port fuzz-parse-attr-line from OSS-FuzzEric Sesterhenn
Git's fuzz tests are run continuously as part of OSS-Fuzz [1]. Several additional fuzz tests have been contributed directly to OSS-Fuzz; however, these tests are vulnerable to bitrot because they are not built during Git's CI runs, and thus breaking changes are much less likely to be noticed by Git contributors. Port one of these tests back to the Git project: fuzz-parse-attr-line This test was originally written by Eric Sesterhenn as part of a security audit of Git [2]. It was then contributed to the OSS-Fuzz repo in commit c58ac4492 (Git fuzzing: uncomment the existing and add new targets. (#11486), 2024-02-21) by Jaroslav Lobačevski. I (Josh Steadmon) have verified with both Eric and Jaroslav that they're OK with moving this test to the Git project. [1] https://github.com/google/oss-fuzz [2] https://ostif.org/wp-content/uploads/2023/01/X41-OSTIF-Gitlab-Git-Security-Audit-20230117-public.pdf Co-authored-by: Jaroslav Lobačevski <jarlob@gmail.com> Co-authored-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>
2024-10-16fuzz: port fuzz-credential-from-url-gently from OSS-FuzzEric Sesterhenn
Git's fuzz tests are run continuously as part of OSS-Fuzz [1]. Several additional fuzz tests have been contributed directly to OSS-Fuzz; however, these tests are vulnerable to bitrot because they are not built during Git's CI runs, and thus breaking changes are much less likely to be noticed by Git contributors. Port one of these tests back to the Git project: fuzz-credential-from-url-gently This test was originally written by Eric Sesterhenn as part of a security audit of Git [2]. It was then contributed to the OSS-Fuzz repo in commit c58ac4492 (Git fuzzing: uncomment the existing and add new targets. (#11486), 2024-02-21) by Jaroslav Lobačevski. I (Josh Steadmon) have verified with both Eric and Jaroslav that they're OK with moving this test to the Git project. [1] https://github.com/google/oss-fuzz [2] https://ostif.org/wp-content/uploads/2023/01/X41-OSTIF-Gitlab-Git-Security-Audit-20230117-public.pdf Co-authored-by: Jaroslav Lobačevski <jarlob@gmail.com> Co-authored-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Taylor Blau <me@ttaylorr.com>