aboutsummaryrefslogtreecommitdiff
path: root/Makefile
AgeCommit message (Collapse)Author
2022-10-05fsmonitor: relocate socket file if .git directory is remoteEric DeCosta
If the .git directory is on a remote filesystem, create the socket file in 'fsmonitor.socketDir' if it is defined, else create it in $HOME. Signed-off-by: Eric DeCosta <edecosta@mathworks.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-10-05fsmonitor: refactor filesystem checks to common interfaceEric DeCosta
Provide a common interface for getting basic filesystem information including filesystem type and whether the filesystem is remote. Refactor existing code for getting basic filesystem info and detecting remote file systems to the new interface. Refactor filesystem checks to leverage new interface. For macOS, error-out if the Unix Domain socket (UDS) file is on a remote filesystem. Signed-off-by: Eric DeCosta <edecosta@mathworks.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-19version: fix builtin linking & documentationVictoria Dye
Like most builtins, 'version' is documented in a corresponding 'Documentation/git-version.txt' and can be invoked with 'git version'. However, the 'check-docs' Makefile target showed that it was "removed but documented: git-version." This was cause by the fact that it is not built as a standalone 'git-version' executable, therefore appearing "removed" to 'check-docs'. Without a precedent for documented builtins that aren't built into an executable *or* any clear reason why a standalone 'git-version' shouldn't exist, the 'check-docs' error appears to correctly identify an issue. To correct that mismatch, add 'git-version' to the 'BUILT_INS' list in the root Makefile (indicating that the 'cmd_version()' function appears in a file that is *not* 'builtin/version.c'). Additionally, to avoid the "no link" message in 'check-docs', list 'git-version' as an "ancilliaryinterrogator" (like 'git help') in 'command-list.txt'. Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-19Merge branch 'vd/scalar-to-main'Junio C Hamano
Hoist the remainder of "scalar" out of contrib/ to the main part of the codebase. * vd/scalar-to-main: Documentation/technical: include Scalar technical doc t/perf: add 'GIT_PERF_USE_SCALAR' run option t/perf: add Scalar performance tests scalar-clone: add test coverage scalar: add to 'git help -a' command list scalar: implement the `help` subcommand git help: special-case `scalar` scalar: include in standard Git build & installation scalar: fix command documentation section header
2022-09-19fuzz: reorganise the path for existing oss-fuzz fuzzersArthur Chan
In order to provide a better organisation for oss-fuzz fuzzers and to avoid top-level clustters in the git repository when more fuzzers are introduced, move the existing fuzzer-related sources to their own oss-fuzz/ hierarchy. Grouping the fuzzers into their own directory, separate their application on fuzz-testing from the core functionalities of the git code, prvides better and tidier structure the oss-fuzz fuzzing library to manage, locate, build and execute those fuzzers for fuzz-testing purposes in future development. Signed-off-by: Arthur Chan <arthur.chan@adalogics.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-13Merge branch 'ab/submodule-helper-prep'Junio C Hamano
Code clean-up of "git submodule--helper". * ab/submodule-helper-prep: (33 commits) submodule--helper: fix bad config API usage submodule--helper: libify even more "die" paths for module_update() submodule--helper: libify more "die" paths for module_update() submodule--helper: check repo{_submodule,}_init() return values submodule--helper: libify "must_die_on_failure" code paths (for die) submodule--helper update: don't override 'checkout' exit code submodule--helper: libify "must_die_on_failure" code paths submodule--helper: libify determine_submodule_update_strategy() submodule--helper: don't exit() on failure, return submodule--helper: use "code" in run_update_command() submodule API: don't handle SM_..{UNSPECIFIED,COMMAND} in to_string() submodule--helper: don't call submodule_strategy_to_string() in BUG() submodule--helper: add missing braces to "else" arm submodule--helper: return "ret", not "1" from update_submodule() submodule--helper: rename "int res" to "int ret" submodule--helper: don't redundantly check "else if (res)" submodule--helper: refactor "errmsg_str" to be a "struct strbuf" submodule--helper: add "const" to passed "struct update_data" submodule--helper: add "const" to copy of "update_data" submodule--helper: add "const" to passed "module_clone_data" ...
2022-09-09Merge branch 'ab/retire-ppc-sha1'Junio C Hamano
Remove the assembly version of SHA-1 implementation for PPC. * ab/retire-ppc-sha1: Makefile: use $(OBJECTS) instead of $(C_OBJ) Makefile + hash.h: remove PPC_SHA1 implementation
2022-09-02scalar: include in standard Git build & installationVictoria Dye
Move 'scalar' out of 'contrib/' and into the root of the Git tree. The goal of this change is to build 'scalar' as part of the standard Git build & install processes. This patch includes both the physical move of Scalar's files out of 'contrib/' ('scalar.c', 'scalar.txt', and 't9xxx-scalar.sh'), and the changes to the build definitions in 'Makefile' and 'CMakelists.txt' to accommodate the new program. At a high level, Scalar is built so that: - there is a 'scalar-objs' target (similar to those created in 029bac01a8 (Makefile: add {program,xdiff,test,git,fuzz}-objs & objects targets, 2021-02-23)) for debugging purposes. - it appears in the root of the install directory (rather than the gitexecdir). - it is included in the 'bin-wrappers/' directory for use in tests. - it receives a platform-specific executable suffix (e.g., '.exe'), if applicable. - 'scalar.txt' is installed as 'man1' documentation. - the 'clean' target removes the 'scalar' executable. Additionally, update the root level '.gitignore' file to ignore the Scalar executable. Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-02submodule--helper: move "is-active" to a test-toolÆvar Arnfjörð Bjarmason
Create a new "test-tool submodule" and move the "is-active" subcommand over to it. It was added in 5c2bd8b77ae (submodule--helper: add is-active subcommand, 2017-03-16), since a452128a36c (submodule--helper: introduce add-config subcommand, 2021-08-06) it hasn't been used by git-submodule.sh. Since we're creating a command dispatch similar to test-tool.c itself let's split out the "struct test_cmd" into a new test-tool-utils.h, which both this new code and test-tool.c itself can use. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Reviewed-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-09-01Merge branch 'ds/bundle-uri-clone'Junio C Hamano
Implement "git clone --bundle-uri". * ds/bundle-uri-clone: clone: warn on failure to repo_init() clone: --bundle-uri cannot be combined with --depth bundle-uri: add support for http(s):// and file:// clone: add --bundle-uri option bundle-uri: create basic file-copy logic remote-curl: add 'get' capability
2022-08-31Makefile: use $(OBJECTS) instead of $(C_OBJ)Ævar Arnfjörð Bjarmason
In the preceding commit $(C_OBJ) added in c373991375a (Makefile: list generated object files in OBJECTS, 2010-01-26) became synonymous with $(OBJECTS). Let's avoid the indirection and use the $(OBJECTS) variable directly instead. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-31Makefile + hash.h: remove PPC_SHA1 implementationÆvar Arnfjörð Bjarmason
Remove the PPC_SHA1 implementation added in a6ef3518f9a ([PATCH] PPC assembly implementation of SHA1, 2005-04-22). When this was added Apple consumer hardware used the PPC architecture, and the implementation was intended to improve SHA-1 speed there. Since it was added we've moved to using sha1collisiondetection by default, and anyone wanting hard-rolled non-DC SHA-1 implementation can use OpenSSL's via the OPENSSL_SHA1 knob. The PPC_SHA1 originally originally targeted 32 bit PPC, and later the 64 bit PPC 970 (a.k.a. Apple PowerPC G5). See 926172c5e48 (block-sha1: improve code on large-register-set machines, 2009-08-10) for a reference about the performance on G5 (a comment in block-sha1/sha1.c being removed here). I can't get it to do anything but segfault on both the BE and LE POWER machines in the GCC compile farm[1]. Anyone who's concerned about performance on PPC these days is likely to be using the IBM POWER processors. There have been proposals to entirely remove non-sha1collisiondetection implementations from the tree[2]. I think per [3] that would be a bit overzealous. I.e. there are various set-ups git's speed is going to be more important than the relatively implausible SHA-1 collision attack, or where such attacks are entirely mitigated by other means (e.g. by incoming objects being checked with DC_SHA1). But that really doesn't apply to PPC_SHA1 in particular, which seems to have outlived its usefulness. As this gets rid of the only in-tree *.S assembly file we can remove the small bits of logic from the Makefile needed to build objects from *.S (as opposed to *.c) The code being removed here was also throwing warnings with the "-pedantic" flag, it could have been fixed as 544d93bc3b4 (block-sha1: remove use of obsolete x86 assembly, 2022-03-10) did for block-sha1/*, but as noted above let's remove it instead. 1. https://cfarm.tetaneutral.net/machines/list/ Tested on gcc{110,112,135,203}, a mixture of POWER [789] ppc64 and ppc64le. All segfault in anything needing object hashing (e.g. t/t1007-hash-object.sh) when compiled with PPC_SHA1=Y. 2. https://lore.kernel.org/git/20200223223758.120941-1-mh@glandium.org/ 3. https://lore.kernel.org/git/20200224044732.GK1018190@coredump.intra.peff.net/ Acked-by: brian m. carlson" <sandals@crustytoothpaste.net> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-29Merge branch 'mt/rot13-in-c'Junio C Hamano
Test portability improvements. * mt/rot13-in-c: tests: use the new C rot13-filter helper to avoid PERL prereq t0021: implementation the rot13-filter.pl script in C t0021: avoid grepping for a Perl-specific string at filter output
2022-08-26grep: fix multibyte regex handling under macOSDiomidis Spinellis
The commit 29de20504e (Makefile: fix default regex settings on Darwin, 2013-05-11) fixed t0070-fundamental.sh under Darwin (macOS) by adopting Git's regex library. However, this library is compiled with NO_MBSUPPORT, which causes git-grep to work incorrectly on multibyte (e.g. UTF-8) files. Current macOS versions pass t0070-fundamental.sh with the native macOS regex library, which also supports multibyte characters. Adjust the Makefile to use the native regex library, and call setlocale(3) to set CTYPE according to the user's preference. The setlocale call is required on all platforms, but in platforms supporting gettext(3), setlocale was called as a side-effect of initializing gettext. Therefore, move the CTYPE setlocale call from gettext.c to common-main.c and the corresponding locale.h include into git-compat-util.h. Thanks to the global initialization of CTYPE setlocale, the test-tool regex command now works correctly with supported multibyte regexes, and is used to set the MB_REGEX test prerequisite by assessing a platform's support for them. Signed-off-by: Diomidis Spinellis <dds@aueb.gr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-25Merge branch 'vd/scalar-generalize-diagnose'Junio C Hamano
The "diagnose" feature to create a zip archive for diagnostic material has been lifted from "scalar" and made into a feature of "git bugreport". * vd/scalar-generalize-diagnose: scalar: update technical doc roadmap scalar-diagnose: use 'git diagnose --mode=all' builtin/bugreport.c: create '--diagnose' option builtin/diagnose.c: add '--mode' option builtin/diagnose.c: create 'git diagnose' builtin diagnose.c: add option to configure archive contents scalar-diagnose: move functionality to common location scalar-diagnose: move 'get_disk_info()' to 'compat/' scalar-diagnose: add directory to archiver more gently scalar-diagnose: avoid 32-bit overflow of size_t scalar-diagnose: use "$GIT_UNZIP" in test
2022-08-25Merge branch 'jk/pipe-command-nonblock'Junio C Hamano
Fix deadlocks between main Git process and subprocess spawned via the pipe_command() API, that can kill "git add -p" that was reimplemented in C recently. * jk/pipe-command-nonblock: pipe_command(): mark stdin descriptor as non-blocking pipe_command(): handle ENOSPC when writing to a pipe pipe_command(): avoid xwrite() for writing to pipe git-compat-util: make MAX_IO_SIZE define globally available nonblock: support Windows compat: add function to enable nonblocking pipes
2022-08-24Merge branch 'ds/bundle-uri-clone' into ds/bundle-uri-3Junio C Hamano
* ds/bundle-uri-clone: clone: warn on failure to repo_init() clone: --bundle-uri cannot be combined with --depth bundle-uri: add support for http(s):// and file:// clone: add --bundle-uri option bundle-uri: create basic file-copy logic remote-curl: add 'get' capability
2022-08-17compat: add function to enable nonblocking pipesJeff King
We'd like to be able to make some of our pipes nonblocking so that poll() can be used effectively, but O_NONBLOCK isn't portable. Let's introduce a compat wrapper so this can be abstracted for each platform. The interface is as narrow as possible to let platforms do what's natural there (rather than having to implement fcntl() and a fake O_NONBLOCK for example, or having to handle other types of descriptors). The next commit will add Windows support, at which point we should be covering all platforms in practice. But if we do find some other platform without O_NONBLOCK, we'll return ENOSYS. Arguably we could just trigger a build-time #error in this case, which would catch the problem earlier. But since we're not planning to use this compat wrapper in many code paths, a seldom-seen runtime error may be friendlier for such a platform than blocking compilation completely. Our test suite would still notice it. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-14Merge branch 'ab/tech-docs-to-help'Junio C Hamano
Expose a lot of "tech docs" via "git help" interface. * ab/tech-docs-to-help: docs: move http-protocol docs to man section 5 docs: move cruft pack docs to gitformat-pack docs: move pack format docs to man section 5 docs: move signature docs to man section 5 docs: move index format docs to man section 5 docs: move protocol-related docs to man section 5 docs: move commit-graph format docs to man section 5 git docs: add a category for file formats, protocols and interfaces git docs: add a category for user-facing file, repo and command UX git help doc: use "<doc>" instead of "<guide>" help.c: remove common category behavior from drop_prefix() behavior help.c: refactor drop_prefix() to use a "switch" statement"
2022-08-14t0021: implementation the rot13-filter.pl script in CMatheus Tavares
This script is currently used by three test files: t0021-conversion.sh, t2080-parallel-checkout-basics.sh, and t2082-parallel-checkout-attributes.sh. To avoid the need for the PERL dependency at these tests, let's convert the script to a C test-tool command. The following commit will take care of actually modifying the said tests to use the new C helper and removing the Perl script. The Perl script flushes the log file handler after each write. As commented in [1], this seems to be an early design decision that was later reconsidered, but possibly ended up being left in the code by accident: >> +$debug->flush(); > > Isn't $debug flushed automatically? Maybe, but autoflush is not explicitly enabled. I will enable it again (I disabled it because of Eric's comment but I re-read the comment and he is only talking about pipes). Anyways, this behavior is not really needed for the tests and the flush() calls make the code slightly larger, so let's avoid them altogether in the new C version. [1]: https://lore.kernel.org/git/7F1F1A0E-8FC3-4FBD-81AA-37786DE0EF50@gmail.com/ Helped-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-12builtin/diagnose.c: create 'git diagnose' builtinVictoria Dye
Create a 'git diagnose' builtin to generate a standalone zip archive of repository diagnostics. The "diagnose" functionality was originally implemented for Scalar in aa5c79a331 (scalar: implement `scalar diagnose`, 2022-05-28). However, the diagnostics gathered are not specific to Scalar-cloned repositories and can be useful when diagnosing issues in any Git repository. Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Helped-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-12scalar-diagnose: move functionality to common locationVictoria Dye
Move the core functionality of 'scalar diagnose' into a new 'diagnose.[c,h]' library to prepare for new callers in the main Git tree generating diagnostic archives. These callers will be introduced in subsequent patches. While this patch appears large, it is mostly made up of moving code out of 'scalar.c' and into 'diagnose.c'. Specifically, the functions - dir_file_stats_objects() - dir_file_stats() - count_files() - loose_objs_stats() - add_directory_to_archiver() are all copied verbatim from 'scalar.c'. The 'create_diagnostics_archive()' function is a mostly identical (partial) copy of 'cmd_diagnose()', with the primary changes being that 'zip_path' is an input and "Enlistment root" is corrected to "Repository root" in the archiver log. Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Victoria Dye <vdye@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-10bundle-uri: create basic file-copy logicDerrick Stolee
Before implementing a way to fetch bundles into a repository, create the basic logic. Assume that the URI is actually a file path. Future logic will make this more careful to other protocols. For now, we also only succeed if the content at the URI is a bundle file, not a bundle list. Bundle lists will be implemented in a future change. Note that the discovery of a temporary filename is slightly racy because the odb_mkstemp() relies on the temporary file not existing. With the current implementation being limited to file copies, we could replace the copy_file() with copy_fd(). The tricky part comes in future changes that send the filename to 'git remote-https' and its 'get' capability. At that point, we need the file descriptor closed _and_ the file unlinked. If we were to keep the file descriptor open for the sake of normal file copies, then we would pollute the rest of the code for little benefit. This is especially the case because we expect that most bundle URI use will be based on HTTPS instead of file copies. Reviewed-by: Josh Steadmon <steadmon@google.com> Signed-off-by: Derrick Stolee <derrickstolee@github.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-04git docs: add a category for user-facing file, repo and command UXÆvar Arnfjörð Bjarmason
Create a new "Repository, command and file interfaces" section in the main "git help git" manual page. Move things that belong under this new criteria from the generic "Guides" section. The "Guides" section was added in f442f28a81b (git.txt: add list of guides, 2020-08-05). It makes sense to have e.g. "giteveryday(7)" and "gitfaq(7)" listed under "Guides". But placing e.g. "gitignore(5)" in it is stretching the meaning of what a "guide" is, ideally that section should list things similar to "giteveryday(7)" and "gitcore-tutorial(7)". An alternate name that was considered for this new section was "User formats", for consistency with the nomenclature used for man section 5 in general. My man(1) lists it as "File formats and conventions, e.g. /etc/passwd". So calling this "git help --formats" or "git help --user-formats" would make sense for e.g. gitignore(5), but would be stretching it somewhat for githooks(5), and would seem really suspect for the likes of gitcli(7). Let's instead pick a name that's closer to the generic term "User interface", which is really what this documentation discusses: General user-interface documentation that doesn't obviously belong elsewhere. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-08-03Merge branch 'rs/mergesort'Junio C Hamano
Make our mergesort implementation type-safe. * rs/mergesort: mergesort: remove llist_mergesort() packfile: use DEFINE_LIST_SORT fetch-pack: use DEFINE_LIST_SORT commit: use DEFINE_LIST_SORT blame: use DEFINE_LIST_SORT test-mergesort: use DEFINE_LIST_SORT test-mergesort: use DEFINE_LIST_SORT_DEBUG mergesort: add macros for typed sort of linked lists mergesort: tighten merge loop mergesort: unify ranks loops
2022-07-18Merge branch 'bc/nettle-sha256'Junio C Hamano
Support for libnettle as SHA256 implementation has been added. * bc/nettle-sha256: sha256: add support for Nettle
2022-07-18Merge branch 'ab/cocci-unused'Junio C Hamano
Add Coccinelle rules to detect the pattern of initializing and then finalizing a structure without using it in between at all, which happens after code restructuring and the compilers fail to recognize as an unused variable. * ab/cocci-unused: cocci: generalize "unused" rule to cover more than "strbuf" cocci: add and apply a rule to find "unused" strbufs cocci: have "coccicheck{,-pending}" depend on "coccicheck-test" cocci: add a "coccicheck-test" target and test *.cocci rules Makefile & .gitignore: ignore & clean "git.res", not "*.res" Makefile: remove mandatory "spatch" arguments from SPATCH_FLAGS
2022-07-18Merge branch 'ab/build-gitweb'Junio C Hamano
Teach "make all" to build gitweb as well. * ab/build-gitweb: gitweb/Makefile: add a "NO_GITWEB" parameter Makefile: build 'gitweb' in the default target gitweb/Makefile: include in top-level Makefile gitweb: remove "test" and "test-installed" targets gitweb/Makefile: prepare to merge into top-level Makefile gitweb/Makefile: clear up and de-duplicate the gitweb.{css,js} vars gitweb/Makefile: add a $(GITWEB_ALL) variable gitweb/Makefile: define all .PHONY prerequisites inline
2022-07-17mergesort: remove llist_mergesort()René Scharfe
Now that all of its callers are gone, remove llist_mergesort(). Signed-off-by: René Scharfe <l.s.r@web.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-10sha256: add support for Nettlebrian m. carlson
For SHA-256, we currently have support for OpenSSL and libgcrypt because these two libraries contain optimized implementations that can take advantage of native processor instructions. However, OpenSSL is not suitable for linking against for Linux distros due to licensing incompatibilities with the GPLv2, and libgcrypt has been less favored by cryptographers due to some security-related implementation issues, which, while not affecting our use of hash algorithms, has affected its reputation. Let's add another option that's compatible with the GPLv2, which is Nettle. This is an option which is generally better than libgcrypt because on many distros GnuTLS (which uses Nettle) is used for HTTPS and therefore as a practical matter it will be available on most systems. As a result, prefer it over libgcrypt and our built-in implementation. Nettle also has recently gained support for Intel's SHA-NI instructions, which compare very favorably to other implementations, as well as assembly implementations for when SHA-NI is not available. A git gc on git.git sees a 12% performance improvement with Nettle over our block SHA-256 implementation due to general assembly improvements. With SHA-NI, the performance of raw SHA-256 on a 2 GiB file goes from 7.296 seconds with block SHA-256 to 1.523 seconds with Nettle. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-06cocci: have "coccicheck{,-pending}" depend on "coccicheck-test"Ævar Arnfjörð Bjarmason
Have the newly introduced "coccicheck-test" target run implicitly when "coccicheck" itself is run. As with e.g. the "check-chainlint" target (see [1]) it makes sense to run this unconditionally before we run other "spatch" rules as a basic sanity check. See 1. 803394459d4 (t/Makefile: add machinery to check correctness of chainlint.sed, 2018-07-11) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-06cocci: add a "coccicheck-test" target and test *.cocci rulesÆvar Arnfjörð Bjarmason
Add a "coccicheck-test" target to test our *.cocci rules, and as a demonstration add tests for the rules added in 39ea59a2570 (remove unnecessary NULL check before free(3), 2016-10-08) and 1b83d1251ed (coccinelle: add a rule to make "expression" code use FREE_AND_NULL(), 2017-06-15). I considered making use of the "spatch --test" option, and the choice of a "tests" over a "t" directory is to make these tests compatible with such a future change. Unfortunately "spatch --test" doesn't return meaningful exit codes, AFAICT you need to "grep" its output to see if the *.res is what you expect. There's "--test-okfailed", but I didn't find a way to sensibly integrate those (it relies on some in-between status files, but doesn't help with the status codes). Instead let's use a "--sp-file" pattern similar to the main "coccicheck" rule, with the difference that we use and compare the two *.res files with cmp(1). The --very-quiet and --no-show-diff options ensure that we don't need to pipe stdout and stderr somewhere. Unlike the "%.cocci.patch" rule we're not using the diff. The "cmp || git diff" is optimistically giving us better output on failure, but even if we only have POSIX cmp and no system git installed we'll still fail with the "cmp", just with an error message that isn't as friendly. The "2>/dev/null" is in case we don't have a "git" installed. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-06Makefile & .gitignore: ignore & clean "git.res", not "*.res"Ævar Arnfjörð Bjarmason
Adjust the overly broad .gitignore and "make clean" rule added in ce39c2e04ce (Provide a Windows version resource for the git executables., 2012-05-24). For now this is merely a correctness fix, but needed because a subsequent commit will want to check in *.res files elsewhere in the tree, which we shouldn't have to "git add -f". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-07-06Makefile: remove mandatory "spatch" arguments from SPATCH_FLAGSÆvar Arnfjörð Bjarmason
The "--patch ." part of SPATCH_FLAGS added in f57d11728d1 (coccinelle: put sane filenames into output patches, 2018-07-23) should have been added unconditionally to the "spatch" invocation instead, using it isn't optional. Let's also move the other mandatory flag to come after $(SPATCH_FLAGS), to ensure that our "--sp-file" overrides any provided in the environment, both --sp-file <arg> and --patch <arg> are last-option-wins as far as spatch(1) option parsing is concerned. The environment variable override was initially added in a9a884aea57 (coccicheck: use --all-includes by default, 2016-09-30). In practice there's probably nobody that's using SPATCH_FLAGS to try to intentionally break our invocations, but since we're changing this let's make it clear what (if anything) we expect to be overridden by user-supplied flags. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-28gitweb/Makefile: add a "NO_GITWEB" parameterÆvar Arnfjörð Bjarmason
From looking at the {Free,Net,Dragonfly}BSD packages for git[1] they've been monkeypatching "gitweb" out of the Makefile, let's be nicer and provide a NO_GITWEB=Y for their use. For the "all" target this allows for optionally restoring what's been the status quo before the preceding commit, but now we'll also behave correctly on the subsequent "make install". As before our installation of gitweb can be suppressed with NO_PERL. For backwards compatibility the NO_PERL=Y flag by itself still doesn't change whether or not we build gitweb, unlike the new NO_GITWEB=Y flag. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-28Makefile: build 'gitweb' in the default targetSZEDER Gábor
Our Makefile's default target used to build 'gitweb', though indirectly: the 'all' target depended on 'git-instaweb', which in turn depended on 'gitweb'. Then e25c7cc146 (Makefile: drop dependency between git-instaweb and gitweb, 2015-05-29) removed the latter dependency, and for good reasons (quoting its commit message): "1. git-instaweb has no build-time dependency on gitweb; it is a run-time dependency 2. gitweb is a directory that we want to recursively make in. As a result, its recipe is marked .PHONY, which causes "make" to rebuild git-instaweb every time it is run." Since then a simple 'make' doesn't build 'gitweb'. Luckily, installing 'gitweb' is not broken: although 'make install' doesn't depend on the 'gitweb' target, it has a dependency on the 'install-gitweb' target, which does generate all the necessary files for 'gitweb' and installs them. However, if someone runs 'make && sudo make install', then those files in the 'gitweb' directory will be generated and owned by root, which is not nice. List 'gitweb' as a direct dependency of the default target, so a plain 'make' will build it. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-28gitweb/Makefile: include in top-level MakefileÆvar Arnfjörð Bjarmason
Include the gitweb/Makefile in the top-level Makefile rather than calling it as a sub-Makefile. As noted in the thread starting at at [1] (in particular [2]) we'll pay a high cost on NOOP runs of "make" just to figure out that we have nothing to do for "make gitweb". The "gitweb" script also isn't maintained out-of-tree, unlike "gitk-git" or "git-gui", which both have their own "Makefile". Other parts of it are already integrated into our main Makefiles, e.g. the documentation is built by Documentation/Makefile since 07ea4df2780 (gitweb: Add gitweb(1) manpage for gitweb itself, 2011-10-16). 1. https://lore.kernel.org/git/20220525205651.825669-1-szeder.dev@gmail.com/ 2. https://lore.kernel.org/git/220526.86k0a96sv2.gmgdl@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-06-10Merge branch 'jh/builtin-fsmonitor-part3'Junio C Hamano
More fsmonitor--daemon. * jh/builtin-fsmonitor-part3: (30 commits) t7527: improve implicit shutdown testing in fsmonitor--daemon fsmonitor--daemon: allow --super-prefix argument t7527: test Unicode NFC/NFD handling on MacOS t/lib-unicode-nfc-nfd: helper prereqs for testing unicode nfc/nfd t/helper/hexdump: add helper to print hexdump of stdin fsmonitor: on macOS also emit NFC spelling for NFD pathname t7527: test FSMonitor on case insensitive+preserving file system fsmonitor: never set CE_FSMONITOR_VALID on submodules t/perf/p7527: add perf test for builtin FSMonitor t7527: FSMonitor tests for directory moves fsmonitor: optimize processing of directory events fsm-listen-darwin: shutdown daemon if worktree root is moved/renamed fsm-health-win32: force shutdown daemon if worktree root moves fsm-health-win32: add polling framework to monitor daemon health fsmonitor--daemon: stub in health thread fsmonitor--daemon: rename listener thread related variables fsmonitor--daemon: prepare for adding health thread fsmonitor--daemon: cd out of worktree root fsm-listen-darwin: ignore FSEvents caused by xattr changes on macOS unpack-trees: initialize fsmonitor_has_run_once in o->result ...
2022-06-03Merge branch 'tb/cruft-packs'Junio C Hamano
A mechanism to pack unreachable objects into a "cruft pack", instead of ejecting them into loose form to be reclaimed later, has been introduced. * tb/cruft-packs: sha1-file.c: don't freshen cruft packs builtin/gc.c: conditionally avoid pruning objects via loose builtin/repack.c: add cruft packs to MIDX during geometric repack builtin/repack.c: use named flags for existing_packs builtin/repack.c: allow configuring cruft pack generation builtin/repack.c: support generating a cruft pack builtin/pack-objects.c: --cruft with expiration reachable: report precise timestamps from objects in cruft packs reachable: add options to add_unseen_recent_objects_to_traversal builtin/pack-objects.c: --cruft without expiration builtin/pack-objects.c: return from create_object_entry() t/helper: add 'pack-mtimes' test-tool pack-mtimes: support writing pack .mtimes files chunk-format.h: extract oid_version() pack-write: pass 'struct packing_data' to 'stage_tmp_packfiles' pack-mtimes: support reading .mtimes files Documentation/technical: add cruft-packs.txt
2022-06-03Merge branch 'jx/l10n-workflow-change'Junio C Hamano
A workflow change for translators are being proposed. * jx/l10n-workflow-change: l10n: Document the new l10n workflow Makefile: add "po-init" rule to initialize po/XX.po Makefile: add "po-update" rule to update po/XX.po po/git.pot: don't check in result of "make pot" po/git.pot: this is now a generated file Makefile: remove duplicate and unwanted files in FOUND_SOURCE_FILES i18n CI: stop allowing non-ASCII source messages in po/git.pot Makefile: have "make pot" not "reset --hard" Makefile: generate "po/git.pot" from stable LOCALIZED_C Makefile: sort source files before feeding to xgettext
2022-05-26t/helper/hexdump: add helper to print hexdump of stdinJeff Hostetler
Co-authored-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-26fsmonitor--daemon: stub in health threadJeff Hostetler
Create another thread to watch over the daemon process and automatically shut it down if necessary. This commit creates the basic framework for a "health" thread to monitor the daemon and/or the file system. Later commits will add platform-specific code to do the actual work. The "health" thread is intended to monitor conditions that would be difficult to track inside the IPC thread pool and/or the file system listener threads. For example, when there are file system events outside of the watched worktree root or if we want to have an idle-timeout auto-shutdown feature. This commit creates the health thread itself, defines the thread-proc and sets up the thread's event loop. It integrates this new thread into the existing IPC and Listener thread models. This commit defines the API to the platform-specific code where all of the monitoring will actually happen. The platform-specific code for MacOS is just stubs. Meaning that the health thread will immediately exit on MacOS, but that is OK and expected. Future work can define MacOS-specific monitoring. The platform-specific code for Windows sets up enough of the WaitForMultipleObjects() machinery to watch for system and/or custom events. Currently, the set of wait handles only includes our custom shutdown event (sent from our other theads). Later commits in this series will extend the set of wait handles to monitor other conditions. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-26fsmonitor-settings: stub in Win32-specific incompatibility checkingJeff Hostetler
Extend generic incompatibility checkout with platform-specific mechanism. Stub in Win32 version. In the existing fsmonitor-settings code we have a way to mark types of repos as incompatible with fsmonitor (whether via the hook and IPC APIs). For example, we do this for bare repos, since there are no files to watch. Extend this exclusion mechanism for platform-specific reasons. This commit just creates the framework and adds a stub for Win32. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-26t/helper: add 'pack-mtimes' test-toolTaylor Blau
In the next patch, we will implement and test support for writing a cruft pack via a special mode of `git pack-objects`. To make sure that objects are written with the correct timestamps, and a new test-tool that can dump the object names and corresponding timestamps from a given `.mtimes` file. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-26pack-mtimes: support reading .mtimes filesTaylor Blau
To store the individual mtimes of objects in a cruft pack, introduce a new `.mtimes` format that can optionally accompany a single pack in the repository. The format is defined in Documentation/technical/pack-format.txt, and stores a 4-byte network order timestamp for each object in name (index) order. This patch prepares for cruft packs by defining the `.mtimes` format, and introducing a basic API that callers can use to read out individual mtimes. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-26Makefile: add "po-init" rule to initialize po/XX.poÆvar Arnfjörð Bjarmason
The core translation is the minimum set of work that must be done for a new language translation. There are over 5000 messages in the template message file "po/git.pot" that need to be translated. It is not a piece of cake for such a huge workload. So we used to define a small set of messages called "core translation" that a new l10n contributor must complete before sending pull request to the l10n coordinator. By pulling in some parts of the git-po-helper[^1] logic, we add a new rule to create this core translation message "po/git-core.pot": make po/git-core.pot To help new l10n contributors to initialized their "po/XX.pot" from "po/git-core.pot", we also add new rules "po-init": make po-init PO_FILE=po/XX.po [^1]: https://github.com/git-l10n/git-po-helper/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-26Makefile: add "po-update" rule to update po/XX.poJiang Xin
Since there is no longer a "po/git.pot" file in tree, a l10n team leader has to run several commands to update their "po/XX.po" file: $ make pot $ msgmerge --add-location --backup=off -U po/XX.po po/git.pot To make this process easier, add a new rule so that l10n team leaders can update their "po/XX.po" with one command. E.g.: $ make po-update PO_FILE=po/zh_CN.po Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-26po/git.pot: don't check in result of "make pot"Ævar Arnfjörð Bjarmason
Remove the "po/git.pot" file from being tracked, which started with dce37b66fb0 (l10n: initial git.pot for 1.7.10 upcoming release, 2012-02-13). The reason the po/git.pot started being checked in was because the po/*.po files were changed a schema where we'd generate them from a known-good snapshot of po/git.pot, instead of each translator running "make pot" themselves. This makes sense, but we don't need to carry this file in-tree just to achieve that aim, and doing so has resulted in a significant amount of "diff churn" since this method of doing it was introduced: $ git log -p --oneline -- po/git.pot|wc -l 553743 We can instead let l10n contributors to generate "po/git.pot" in runtime to update their own "po/XX.po", and the l10n coordinator can check pull requests using CI pipeline. This reverts to the schema introduced initially in cd5513a7168 (i18n: Makefile: "pot" target to extract messages marked for translation, 2011-02-22). The actual "git rm" of po/git.pot was in preceding commit to make this change easier to review, and to preempt the mailing list from blocking it due to it being too large. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-26Makefile: remove duplicate and unwanted files in FOUND_SOURCE_FILESJiang Xin
We get source files saved in "$(FOUND_SOURCE_FILES)" by running the command "git ls-files" or the command "find". We tried to have the both commands return the same list of files, but apparently the "find" command will return more files, such as the generated headers. We can filter out these generated headers to get closer results. In addition to this, "$(FOUND_SOURCE_FILES)" may contain duplicate files. E.g. "git-ls-files" may have duplicate entries for the same file in different staging areas if there are unresolved conflicts in the working tree. For this case, we can reduce duplicate entries by passing the option "--deduplicate" to git-ls-files. Junio reported that when running "make" in a working tree with unresolved conflicts, "make" may report warnings like below: Makefile:xxxx: target '.build/pot/po/FOO.c.po' given more than once in the same rule The duplicate targets are introduced by the following pattern rule we added in the preceding commit for incremental build of "po/git.pot". $(LOCALIZED_C_GEN_PO): .build/pot/po/%.po: % Although we have resolved this issue by sorting to create a unique $(LOCALIZED_C), other targets may benefit from this. Such as: tags, cscope.out, etc. Reported-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2022-05-26i18n CI: stop allowing non-ASCII source messages in po/git.potÆvar Arnfjörð Bjarmason
In the preceding commit we moved away from using xgettext(1) to both generate the po/git.pot, and to merge the incrementally generated po/git.pot+ file as we sourced translations from C, shell and Perl. Doing it this way, which dates back to my initial implementation[1][2][3] was conflating two things: With xgettext(1) the --from-code both controls what encoding is specified in the po/git.pot's header, and what encoding we allow in source messages. We don't ever want to allow non-ASCII in *source messages*, and doing so has hid e.g. a buggy message introduced in a6226fd772b (submodule--helper: convert the bulk of cmd_add() to C, 2021-08-10) from us, we'd warn about it before, but only when running "make pot", but the operation would still succeed. Now we'll error out on it when running "make pot". Since the preceding Makefile changes made this easy: let's add a "make check-pot" target with the same prerequisites as the "po/git.pot" target, but without changing the file "po/git.pot". Running it as part of the "static-analysis" CI target will ensure that we catch any such issues in the future. E.g.: $ make check-pot XGETTEXT .build/pot/po/builtin/submodule--helper.c.po xgettext: Non-ASCII string at builtin/submodule--helper.c:3381. Please specify the source encoding through --from-code. make: *** [.build/pot/po/builtin/submodule--helper.c.po] Error 1 1. cd5513a7168 (i18n: Makefile: "pot" target to extract messages marked for translation, 2011-02-22) 2. adc3b2b2767 (Makefile: add xgettext target for *.sh files, 2011-05-14) 3. 5e9637c6297 (i18n: add infrastructure for translating Git with gettext, 2011-11-18) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>