From 9d505b7b49c00c5fa99a25506e63a2ef326a0062 Mon Sep 17 00:00:00 2001 From: Jerry Zhang Date: Tue, 11 Jan 2022 13:39:41 -0800 Subject: git-rev-list: add --exclude-first-parent-only flag It is useful to know when a branch first diverged in history from some integration branch in order to be able to enumerate the user's local changes. However, these local changes can include arbitrary merges, so it is necessary to ignore this merge structure when finding the divergence point. In order to do this, teach the "rev-list" family to accept "--exclude-first-parent-only", which restricts the traversal of excluded commits to only follow first parent links. -A-----E-F-G--main \ / / B-C-D--topic In this example, the goal is to return the set {B, C, D} which represents a topic branch that has been merged into main branch. `git rev-list topic ^main` will end up returning no commits since excluding main will end up traversing the commits on topic as well. `git rev-list --exclude-first-parent-only topic ^main` however will return {B, C, D} as desired. Add docs for the new flag, and clarify the doc for --first-parent to indicate that it applies to traversing the set of included commits only. Signed-off-by: Jerry Zhang Signed-off-by: Junio C Hamano --- Documentation/rev-list-options.txt | 22 +++++++++++++++------- 1 file changed, 15 insertions(+), 7 deletions(-) (limited to 'Documentation') diff --git a/Documentation/rev-list-options.txt b/Documentation/rev-list-options.txt index 43a86fa562..fd4f4e26c9 100644 --- a/Documentation/rev-list-options.txt +++ b/Documentation/rev-list-options.txt @@ -122,19 +122,27 @@ again. Equivalent forms are `--min-parents=0` (any commit has 0 or more parents) and `--max-parents=-1` (negative numbers denote no upper limit). --first-parent:: - Follow only the first parent commit upon seeing a merge - commit. This option can give a better overview when - viewing the evolution of a particular topic branch, - because merges into a topic branch tend to be only about - adjusting to updated upstream from time to time, and - this option allows you to ignore the individual commits - brought in to your history by such a merge. + When finding commits to include, follow only the first + parent commit upon seeing a merge commit. This option + can give a better overview when viewing the evolution of + a particular topic branch, because merges into a topic + branch tend to be only about adjusting to updated upstream + from time to time, and this option allows you to ignore + the individual commits brought in to your history by such + a merge. ifdef::git-log[] + This option also changes default diff format for merge commits to `first-parent`, see `--diff-merges=first-parent` for details. endif::git-log[] +--exclude-first-parent-only:: + When finding commits to exclude (with a '{caret}'), follow only + the first parent commit upon seeing a merge commit. + This can be used to find the set of changes in a topic branch + from the point where it diverged from the remote branch, given + that arbitrary merges can be valid topic branch changes. + --not:: Reverses the meaning of the '{caret}' prefix (or lack thereof) for all following revision specifiers, up to the next `--not`. -- cgit v1.3-5-g9baa From 88078f543b769dc13ae9796372651178584a25a0 Mon Sep 17 00:00:00 2001 From: Victoria Dye Date: Tue, 11 Jan 2022 18:05:02 +0000 Subject: checkout-index: add --ignore-skip-worktree-bits option Update `checkout-index` to no longer refresh files that have the `skip-worktree` bit set, exiting with an error if `skip-worktree` filenames are directly provided to `checkout-index`. The newly-added `--ignore-skip-worktree-bits` option provides a mechanism to replicate the old behavior, checking out *all* files specified (even those with `skip-worktree` enabled). The ability to toggle whether files should be checked-out based on `skip-worktree` already exists in `git checkout` and `git restore` (both of which have an `--ignore-skip-worktree-bits` option). The change to, by default, ignore `skip-worktree` files is especially helpful for sparse-checkout; it prevents inadvertent creation of files outside the sparse definition on disk and eliminates the need to expand a sparse index when using the `--all` option. Internal usage of `checkout-index` in `git stash` and `git filter-branch` do not make explicit use of files with `skip-worktree` enabled, so `--ignore-skip-worktree-bits` is not added to them. Helped-by: Elijah Newren Signed-off-by: Victoria Dye Reviewed-by: Elijah Newren Signed-off-by: Junio C Hamano --- Documentation/git-checkout-index.txt | 10 ++++++++-- builtin/checkout-index.c | 13 +++++++++++++ t/t1092-sparse-checkout-compatibility.sh | 27 +++++++++++++++++---------- 3 files changed, 38 insertions(+), 12 deletions(-) (limited to 'Documentation') diff --git a/Documentation/git-checkout-index.txt b/Documentation/git-checkout-index.txt index 4d33e7be0f..01dbd5cbf5 100644 --- a/Documentation/git-checkout-index.txt +++ b/Documentation/git-checkout-index.txt @@ -12,6 +12,7 @@ SYNOPSIS 'git checkout-index' [-u] [-q] [-a] [-f] [-n] [--prefix=] [--stage=|all] [--temp] + [--ignore-skip-worktree-bits] [-z] [--stdin] [--] [...] @@ -37,8 +38,9 @@ OPTIONS -a:: --all:: - checks out all files in the index. Cannot be used - together with explicit filenames. + checks out all files in the index except for those with the + skip-worktree bit set (see `--ignore-skip-worktree-bits`). + Cannot be used together with explicit filenames. -n:: --no-create:: @@ -59,6 +61,10 @@ OPTIONS write the content to temporary files. The temporary name associations will be written to stdout. +--ignore-skip-worktree-bits:: + Check out all files, including those with the skip-worktree bit + set. + --stdin:: Instead of taking list of paths from the command line, read list of paths from the standard input. Paths are diff --git a/builtin/checkout-index.c b/builtin/checkout-index.c index e21620d964..615a118e2f 100644 --- a/builtin/checkout-index.c +++ b/builtin/checkout-index.c @@ -7,6 +7,7 @@ #define USE_THE_INDEX_COMPATIBILITY_MACROS #include "builtin.h" #include "config.h" +#include "dir.h" #include "lockfile.h" #include "quote.h" #include "cache-tree.h" @@ -17,6 +18,7 @@ #define CHECKOUT_ALL 4 static int nul_term_line; static int checkout_stage; /* default to checkout stage0 */ +static int ignore_skip_worktree; /* default to 0 */ static int to_tempfile; static char topath[4][TEMPORARY_FILENAME_LENGTH + 1]; @@ -65,6 +67,7 @@ static int checkout_file(const char *name, const char *prefix) int namelen = strlen(name); int pos = cache_name_pos(name, namelen); int has_same_name = 0; + int is_skipped = 1; int did_checkout = 0; int errs = 0; @@ -78,6 +81,9 @@ static int checkout_file(const char *name, const char *prefix) break; has_same_name = 1; pos++; + if (!ignore_skip_worktree && ce_skip_worktree(ce)) + break; + is_skipped = 0; if (ce_stage(ce) != checkout_stage && (CHECKOUT_ALL != checkout_stage || !ce_stage(ce))) continue; @@ -106,6 +112,9 @@ static int checkout_file(const char *name, const char *prefix) fprintf(stderr, "git checkout-index: %s ", name); if (!has_same_name) fprintf(stderr, "is not in the cache"); + else if (is_skipped) + fprintf(stderr, "has skip-worktree enabled; " + "use '--ignore-skip-worktree-bits' to checkout"); else if (checkout_stage) fprintf(stderr, "does not exist at stage %d", checkout_stage); @@ -125,6 +134,8 @@ static int checkout_all(const char *prefix, int prefix_length) ensure_full_index(&the_index); for (i = 0; i < active_nr ; i++) { struct cache_entry *ce = active_cache[i]; + if (!ignore_skip_worktree && ce_skip_worktree(ce)) + continue; if (ce_stage(ce) != checkout_stage && (CHECKOUT_ALL != checkout_stage || !ce_stage(ce))) continue; @@ -185,6 +196,8 @@ int cmd_checkout_index(int argc, const char **argv, const char *prefix) struct option builtin_checkout_index_options[] = { OPT_BOOL('a', "all", &all, N_("check out all files in the index")), + OPT_BOOL(0, "ignore-skip-worktree-bits", &ignore_skip_worktree, + N_("do not skip files with skip-worktree set")), OPT__FORCE(&force, N_("force overwrite of existing files"), 0), OPT__QUIET(&quiet, N_("no warning for existing files and files not in index")), diff --git a/t/t1092-sparse-checkout-compatibility.sh b/t/t1092-sparse-checkout-compatibility.sh index db7ad41109..434ef0433c 100755 --- a/t/t1092-sparse-checkout-compatibility.sh +++ b/t/t1092-sparse-checkout-compatibility.sh @@ -772,9 +772,14 @@ test_expect_success 'checkout-index inside sparse definition' ' test_expect_success 'checkout-index outside sparse definition' ' init_repos && - # File does not exist on disk yet for sparse checkouts, so checkout-index - # succeeds without -f - test_sparse_match git checkout-index -- folder1/a && + # Without --ignore-skip-worktree-bits, outside-of-cone files will trigger + # an error + test_sparse_match test_must_fail git checkout-index -- folder1/a && + test_i18ngrep "folder1/a has skip-worktree enabled" sparse-checkout-err && + test_path_is_missing folder1/a && + + # With --ignore-skip-worktree-bits, outside-of-cone files are checked out + test_sparse_match git checkout-index --ignore-skip-worktree-bits -- folder1/a && test_cmp sparse-checkout/folder1/a sparse-index/folder1/a && test_cmp sparse-checkout/folder1/a full-checkout/folder1/a && @@ -783,8 +788,8 @@ test_expect_success 'checkout-index outside sparse definition' ' run_on_sparse mkdir -p folder1 && run_on_all cp ../new-a folder1/a && - test_all_match test_must_fail git checkout-index -- folder1/a && - test_all_match git checkout-index -f -- folder1/a && + test_all_match test_must_fail git checkout-index --ignore-skip-worktree-bits -- folder1/a && + test_all_match git checkout-index -f --ignore-skip-worktree-bits -- folder1/a && test_cmp sparse-checkout/folder1/a sparse-index/folder1/a && test_cmp sparse-checkout/folder1/a full-checkout/folder1/a ' @@ -799,14 +804,16 @@ test_expect_success 'checkout-index with folders' ' test_all_match test_must_fail git checkout-index -f -- folder1/ ' -# NEEDSWORK: even in sparse checkouts, checkout-index --all will create all -# files (even those outside the sparse definition) on disk. However, these files -# don't appear in the percentage of tracked files in git status. -test_expect_failure 'checkout-index --all' ' +test_expect_success 'checkout-index --all' ' init_repos && test_all_match git checkout-index --all && - test_sparse_match test_path_is_missing folder1 + test_sparse_match test_path_is_missing folder1 && + + # --ignore-skip-worktree-bits will cause `skip-worktree` files to be + # checked out, causing the outside-of-cone `folder1` to exist on-disk + test_all_match git checkout-index --ignore-skip-worktree-bits --all && + test_all_match test_path_exists folder1 ' test_expect_success 'clean' ' -- cgit v1.3-5-g9baa From 9325285df4988a35ab376947cc3599aeec0a24b6 Mon Sep 17 00:00:00 2001 From: Philip Oakley Date: Thu, 3 Feb 2022 10:16:43 +0000 Subject: doc: check-ignore: code-quote an exclamation mark The plain quoted exclamation mark renders as italics in the Windows pdf help manual. Fix this with back-tick quoting and surrounding double quotes as exemplified by the gitignore.txt guide. While at it, fix the surrounding double quotes for the other special characters usages. Signed-off-by: Philip Oakley Signed-off-by: Junio C Hamano --- Documentation/git-check-ignore.txt | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) (limited to 'Documentation') diff --git a/Documentation/git-check-ignore.txt b/Documentation/git-check-ignore.txt index 0c3924a63d..2892799e32 100644 --- a/Documentation/git-check-ignore.txt +++ b/Documentation/git-check-ignore.txt @@ -33,7 +33,7 @@ OPTIONS Instead of printing the paths that are excluded, for each path that matches an exclude pattern, print the exclude pattern together with the path. (Matching an exclude pattern usually - means the path is excluded, but if the pattern begins with '!' + means the path is excluded, but if the pattern begins with "`!`" then it is a negated pattern and matching it means the path is NOT excluded.) + @@ -77,7 +77,7 @@ If `--verbose` is specified, the output is a series of lines of the form: is the path of a file being queried, is the matching pattern, is the pattern's source file, and is the line number of the pattern within that source. If the pattern -contained a `!` prefix or `/` suffix, it will be preserved in the +contained a "`!`" prefix or "`/`" suffix, it will be preserved in the output. will be an absolute path when referring to the file configured by `core.excludesFile`, or relative to the repository root when referring to `.git/info/exclude` or a per-directory exclude file. -- cgit v1.3-5-g9baa From 961b130d20c9aea322b94a639a63ec8cca9f14fc Mon Sep 17 00:00:00 2001 From: Glen Choo Date: Fri, 28 Jan 2022 16:04:45 -0800 Subject: branch: add --recurse-submodules option for branch creation To improve the submodules UX, we would like to teach Git to handle branches in submodules. Start this process by teaching "git branch" the --recurse-submodules option so that "git branch --recurse-submodules topic" will create the `topic` branch in the superproject and its submodules. Although this commit does not introduce breaking changes, it does not work well with existing --recurse-submodules commands because "git branch --recurse-submodules" writes to the submodule ref store, but most commands only consider the superproject gitlink and ignore the submodule ref store. For example, "git checkout --recurse-submodules" will check out the commits in the superproject gitlinks (and put the submodules in detached HEAD) instead of checking out the submodule branches. Because of this, this commit introduces a new configuration value, `submodule.propagateBranches`. The plan is for Git commands to prioritize submodule ref store information over superproject gitlinks if this value is true. Because "git branch --recurse-submodules" writes to submodule ref stores, for the sake of clarity, it will not function unless this configuration value is set. This commit also includes changes that support working with submodules from a superproject commit because "branch --recurse-submodules" (and future commands) need to read .gitmodules and gitlinks from the superproject commit, but submodules are typically read from the filesystem's .gitmodules and the index's gitlinks. These changes are: * add a submodules_of_tree() helper that gives the relevant information of an in-tree submodule (e.g. path and oid) and initializes the repository * add is_tree_submodule_active() by adding a treeish_name parameter to is_submodule_active() * add the "submoduleNotUpdated" advice to advise users to update the submodules in their trees Incidentally, fix an incorrect usage string that combined the 'list' usage of git branch (-l) with the 'create' usage; this string has been incorrect since its inception, a8dfd5eac4 (Make builtin-branch.c use parse_options., 2007-10-07). Helped-by: Jonathan Tan Signed-off-by: Glen Choo Reviewed-by: Jonathan Tan Signed-off-by: Junio C Hamano --- Documentation/config/advice.txt | 3 + Documentation/config/submodule.txt | 37 +++-- Documentation/git-branch.txt | 19 ++- advice.c | 1 + advice.h | 1 + branch.c | 141 ++++++++++++++++++ branch.h | 29 ++++ builtin/branch.c | 44 +++++- builtin/submodule--helper.c | 38 +++++ submodule-config.c | 61 ++++++++ submodule-config.h | 34 +++++ submodule.c | 11 +- submodule.h | 3 + t/t3207-branch-submodule.sh | 292 +++++++++++++++++++++++++++++++++++++ 14 files changed, 694 insertions(+), 20 deletions(-) create mode 100755 t/t3207-branch-submodule.sh (limited to 'Documentation') diff --git a/Documentation/config/advice.txt b/Documentation/config/advice.txt index 063eec2511..adee26fbbb 100644 --- a/Documentation/config/advice.txt +++ b/Documentation/config/advice.txt @@ -116,6 +116,9 @@ advice.*:: submoduleAlternateErrorStrategyDie:: Advice shown when a submodule.alternateErrorStrategy option configured to "die" causes a fatal error. + submodulesNotUpdated:: + Advice shown when a user runs a submodule command that fails + because `git submodule update --init` was not run. addIgnoredFile:: Advice shown if a user attempts to add an ignored file to the index. diff --git a/Documentation/config/submodule.txt b/Documentation/config/submodule.txt index ee454f8126..6490527b45 100644 --- a/Documentation/config/submodule.txt +++ b/Documentation/config/submodule.txt @@ -59,18 +59,33 @@ submodule.active:: submodule.recurse:: A boolean indicating if commands should enable the `--recurse-submodules` - option by default. - Applies to all commands that support this option - (`checkout`, `fetch`, `grep`, `pull`, `push`, `read-tree`, `reset`, - `restore` and `switch`) except `clone` and `ls-files`. + option by default. Defaults to false. ++ +When set to true, it can be deactivated via the +`--no-recurse-submodules` option. Note that some Git commands +lacking this option may call some of the above commands affected by +`submodule.recurse`; for instance `git remote update` will call +`git fetch` but does not have a `--no-recurse-submodules` option. +For these commands a workaround is to temporarily change the +configuration value by using `git -c submodule.recurse=0`. ++ +The following list shows the commands that accept +`--recurse-submodules` and whether they are supported by this +setting. + +* `checkout`, `fetch`, `grep`, `pull`, `push`, `read-tree`, +`reset`, `restore` and `switch` are always supported. +* `clone` and `ls-files` are not supported. +* `branch` is supported only if `submodule.propagateBranches` is +enabled + +submodule.propagateBranches:: + [EXPERIMENTAL] A boolean that enables branching support when + using `--recurse-submodules` or `submodule.recurse=true`. + Enabling this will allow certain commands to accept + `--recurse-submodules` and certain commands that already accept + `--recurse-submodules` will now consider branches. Defaults to false. - When set to true, it can be deactivated via the - `--no-recurse-submodules` option. Note that some Git commands - lacking this option may call some of the above commands affected by - `submodule.recurse`; for instance `git remote update` will call - `git fetch` but does not have a `--no-recurse-submodules` option. - For these commands a workaround is to temporarily change the - configuration value by using `git -c submodule.recurse=0`. submodule.fetchJobs:: Specifies how many submodules are fetched/cloned at the same time. diff --git a/Documentation/git-branch.txt b/Documentation/git-branch.txt index 731e340cbc..c8b4f9ce3c 100644 --- a/Documentation/git-branch.txt +++ b/Documentation/git-branch.txt @@ -16,7 +16,8 @@ SYNOPSIS [--points-at ] [--format=] [(-r | --remotes) | (-a | --all)] [--list] [...] -'git branch' [--track[=(direct|inherit)] | --no-track] [-f] [] +'git branch' [--track[=(direct|inherit)] | --no-track] [-f] + [--recurse-submodules] [] 'git branch' (--set-upstream-to= | -u ) [] 'git branch' --unset-upstream [] 'git branch' (-m | -M) [] @@ -235,6 +236,22 @@ how the `branch..remote` and `branch..merge` options are used. Do not set up "upstream" configuration, even if the branch.autoSetupMerge configuration variable is set. +--recurse-submodules:: + THIS OPTION IS EXPERIMENTAL! Causes the current command to + recurse into submodules if `submodule.propagateBranches` is + enabled. See `submodule.propagateBranches` in + linkgit:git-config[1]. Currently, only branch creation is + supported. ++ +When used in branch creation, a new branch will be created +in the superproject and all of the submodules in the superproject's +. In submodules, the branch will point to the submodule +commit in the superproject's but the branch's tracking +information will be set up based on the submodule's branches and remotes +e.g. `git branch --recurse-submodules topic origin/main` will create the +submodule branch "topic" that points to the submodule commit in the +superproject's "origin/main", but tracks the submodule's "origin/main". + --set-upstream:: As this option had confusing syntax, it is no longer supported. Please use `--track` or `--set-upstream-to` instead. diff --git a/advice.c b/advice.c index 1dfc91d176..e00d30254c 100644 --- a/advice.c +++ b/advice.c @@ -70,6 +70,7 @@ static struct { [ADVICE_STATUS_HINTS] = { "statusHints", 1 }, [ADVICE_STATUS_U_OPTION] = { "statusUoption", 1 }, [ADVICE_SUBMODULE_ALTERNATE_ERROR_STRATEGY_DIE] = { "submoduleAlternateErrorStrategyDie", 1 }, + [ADVICE_SUBMODULES_NOT_UPDATED] = { "submodulesNotUpdated", 1 }, [ADVICE_UPDATE_SPARSE_PATH] = { "updateSparsePath", 1 }, [ADVICE_WAITING_FOR_EDITOR] = { "waitingForEditor", 1 }, }; diff --git a/advice.h b/advice.h index 601265fd10..a7521d6087 100644 --- a/advice.h +++ b/advice.h @@ -44,6 +44,7 @@ struct string_list; ADVICE_STATUS_HINTS, ADVICE_STATUS_U_OPTION, ADVICE_SUBMODULE_ALTERNATE_ERROR_STRATEGY_DIE, + ADVICE_SUBMODULES_NOT_UPDATED, ADVICE_UPDATE_SPARSE_PATH, ADVICE_WAITING_FOR_EDITOR, ADVICE_SKIPPED_CHERRY_PICKS, diff --git a/branch.c b/branch.c index 02d46a69b8..70026b3c79 100644 --- a/branch.c +++ b/branch.c @@ -8,6 +8,8 @@ #include "sequencer.h" #include "commit.h" #include "worktree.h" +#include "submodule-config.h" +#include "run-command.h" struct tracking { struct refspec_item spec; @@ -483,6 +485,145 @@ void dwim_and_setup_tracking(struct repository *r, const char *new_ref, setup_tracking(new_ref, real_orig_ref, track, quiet); } +/** + * Creates a branch in a submodule by calling + * create_branches_recursively() in a child process. The child process + * is necessary because install_branch_config_multiple_remotes() (which + * is called by setup_tracking()) does not support writing configs to + * submodules. + */ +static int submodule_create_branch(struct repository *r, + const struct submodule *submodule, + const char *name, const char *start_oid, + const char *tracking_name, int force, + int reflog, int quiet, + enum branch_track track, int dry_run) +{ + int ret = 0; + struct child_process child = CHILD_PROCESS_INIT; + struct strbuf child_err = STRBUF_INIT; + struct strbuf out_buf = STRBUF_INIT; + char *out_prefix = xstrfmt("submodule '%s': ", submodule->name); + child.git_cmd = 1; + child.err = -1; + child.stdout_to_stderr = 1; + + prepare_other_repo_env(&child.env_array, r->gitdir); + /* + * submodule_create_branch() is indirectly invoked by "git + * branch", but we cannot invoke "git branch" in the child + * process. "git branch" accepts a branch name and start point, + * where the start point is assumed to provide both the OID + * (start_oid) and the branch to use for tracking + * (tracking_name). But when recursing through submodules, + * start_oid and tracking name need to be specified separately + * (see create_branches_recursively()). + */ + strvec_pushl(&child.args, "submodule--helper", "create-branch", NULL); + if (dry_run) + strvec_push(&child.args, "--dry-run"); + if (force) + strvec_push(&child.args, "--force"); + if (quiet) + strvec_push(&child.args, "--quiet"); + if (reflog) + strvec_push(&child.args, "--create-reflog"); + if (track == BRANCH_TRACK_ALWAYS || track == BRANCH_TRACK_EXPLICIT) + strvec_push(&child.args, "--track"); + + strvec_pushl(&child.args, name, start_oid, tracking_name, NULL); + + if ((ret = start_command(&child))) + return ret; + ret = finish_command(&child); + strbuf_read(&child_err, child.err, 0); + strbuf_add_lines(&out_buf, out_prefix, child_err.buf, child_err.len); + + if (ret) + fprintf(stderr, "%s", out_buf.buf); + else + printf("%s", out_buf.buf); + + strbuf_release(&child_err); + strbuf_release(&out_buf); + return ret; +} + +void create_branches_recursively(struct repository *r, const char *name, + const char *start_commitish, + const char *tracking_name, int force, + int reflog, int quiet, enum branch_track track, + int dry_run) +{ + int i = 0; + char *branch_point = NULL; + struct object_id super_oid; + struct submodule_entry_list submodule_entry_list; + + /* Perform dwim on start_commitish to get super_oid and branch_point. */ + dwim_branch_start(r, start_commitish, BRANCH_TRACK_NEVER, + &branch_point, &super_oid); + + /* + * If we were not given an explicit name to track, then assume we are at + * the top level and, just like the non-recursive case, the tracking + * name is the branch point. + */ + if (!tracking_name) + tracking_name = branch_point; + + submodules_of_tree(r, &super_oid, &submodule_entry_list); + /* + * Before creating any branches, first check that the branch can + * be created in every submodule. + */ + for (i = 0; i < submodule_entry_list.entry_nr; i++) { + if (submodule_entry_list.entries[i].repo == NULL) { + if (advice_enabled(ADVICE_SUBMODULES_NOT_UPDATED)) + advise(_("You may try updating the submodules using 'git checkout %s && git submodule update --init'"), + start_commitish); + die(_("submodule '%s': unable to find submodule"), + submodule_entry_list.entries[i].submodule->name); + } + + if (submodule_create_branch( + submodule_entry_list.entries[i].repo, + submodule_entry_list.entries[i].submodule, name, + oid_to_hex(&submodule_entry_list.entries[i] + .name_entry->oid), + tracking_name, force, reflog, quiet, track, 1)) + die(_("submodule '%s': cannot create branch '%s'"), + submodule_entry_list.entries[i].submodule->name, + name); + } + + create_branch(the_repository, name, start_commitish, force, 0, reflog, quiet, + BRANCH_TRACK_NEVER, dry_run); + if (dry_run) + return; + /* + * NEEDSWORK If tracking was set up in the superproject but not the + * submodule, users might expect "git branch --recurse-submodules" to + * fail or give a warning, but this is not yet implemented because it is + * tedious to determine whether or not tracking was set up in the + * superproject. + */ + setup_tracking(name, tracking_name, track, quiet); + + for (i = 0; i < submodule_entry_list.entry_nr; i++) { + if (submodule_create_branch( + submodule_entry_list.entries[i].repo, + submodule_entry_list.entries[i].submodule, name, + oid_to_hex(&submodule_entry_list.entries[i] + .name_entry->oid), + tracking_name, force, reflog, quiet, track, 0)) + die(_("submodule '%s': cannot create branch '%s'"), + submodule_entry_list.entries[i].submodule->name, + name); + repo_clear(submodule_entry_list.entries[i].repo); + } +} + void remove_merge_branch_state(struct repository *r) { unlink(git_path_merge_head(r)); diff --git a/branch.h b/branch.h index 509cfcc34e..04df2aa5b5 100644 --- a/branch.h +++ b/branch.h @@ -71,6 +71,35 @@ void create_branch(struct repository *r, int reflog, int quiet, enum branch_track track, int dry_run); +/* + * Creates a new branch in a repository and its submodules (and its + * submodules, recursively). The parameters are mostly analogous to + * those of create_branch() except for start_name, which is represented + * by two different parameters: + * + * - start_commitish is the commit-ish, in repository r, that determines + * which commits the branches will point to. The superproject branch + * will point to the commit of start_commitish and the submodule + * branches will point to the gitlink commit oids in start_commitish's + * tree. + * + * - tracking_name is the name of the ref, in repository r, that will be + * used to set up tracking information. This value is propagated to + * all submodules, which will evaluate the ref using their own ref + * stores. If NULL, this defaults to start_commitish. + * + * When this function is called on the superproject, start_commitish + * can be any user-provided ref and tracking_name can be NULL (similar + * to create_branches()). But when recursing through submodules, + * start_commitish is the plain gitlink commit oid. Since the oid cannot + * be used for tracking information, tracking_name is propagated and + * used for tracking instead. + */ +void create_branches_recursively(struct repository *r, const char *name, + const char *start_commitish, + const char *tracking_name, int force, + int reflog, int quiet, enum branch_track track, + int dry_run); /* * Check if 'name' can be a valid name for a branch; die otherwise. * Return 1 if the named branch already exists; return 0 otherwise. diff --git a/builtin/branch.c b/builtin/branch.c index 209b1cc442..d9a3ad53dd 100644 --- a/builtin/branch.c +++ b/builtin/branch.c @@ -27,7 +27,8 @@ static const char * const builtin_branch_usage[] = { N_("git branch [] [-r | -a] [--merged] [--no-merged]"), - N_("git branch [] [-l] [-f] []"), + N_("git branch [] [-f] [--recurse-submodules] []"), + N_("git branch [] [-l] [...]"), N_("git branch [] [-r] (-d | -D) ..."), N_("git branch [] (-m | -M) [] "), N_("git branch [] (-c | -C) [] "), @@ -38,6 +39,8 @@ static const char * const builtin_branch_usage[] = { static const char *head; static struct object_id head_oid; +static int recurse_submodules = 0; +static int submodule_propagate_branches = 0; static int branch_use_color = -1; static char branch_colors[][COLOR_MAXLEN] = { @@ -99,6 +102,15 @@ static int git_branch_config(const char *var, const char *value, void *cb) return config_error_nonbool(var); return color_parse(value, branch_colors[slot]); } + if (!strcmp(var, "submodule.recurse")) { + recurse_submodules = git_config_bool(var, value); + return 0; + } + if (!strcasecmp(var, "submodule.propagateBranches")) { + submodule_propagate_branches = git_config_bool(var, value); + return 0; + } + return git_color_default_config(var, value, cb); } @@ -622,7 +634,8 @@ int cmd_branch(int argc, const char **argv, const char *prefix) const char *new_upstream = NULL; int noncreate_actions = 0; /* possible options */ - int reflog = 0, quiet = 0, icase = 0, force = 0; + int reflog = 0, quiet = 0, icase = 0, force = 0, + recurse_submodules_explicit = 0; enum branch_track track; struct ref_filter filter; static struct ref_sorting *sorting; @@ -673,6 +686,7 @@ int cmd_branch(int argc, const char **argv, const char *prefix) OPT_CALLBACK(0, "points-at", &filter.points_at, N_("object"), N_("print only branches of the object"), parse_opt_object_name), OPT_BOOL('i', "ignore-case", &icase, N_("sorting and filtering are case insensitive")), + OPT_BOOL(0, "recurse-submodules", &recurse_submodules_explicit, N_("recurse through submodules")), OPT_STRING( 0 , "format", &format.format, N_("format"), N_("format to use for the output")), OPT_END(), }; @@ -715,6 +729,17 @@ int cmd_branch(int argc, const char **argv, const char *prefix) if (noncreate_actions > 1) usage_with_options(builtin_branch_usage, options); + if (recurse_submodules_explicit) { + if (!submodule_propagate_branches) + die(_("branch with --recurse-submodules can only be used if submodule.propagateBranches is enabled")); + if (noncreate_actions) + die(_("--recurse-submodules can only be used to create branches")); + } + + recurse_submodules = + (recurse_submodules || recurse_submodules_explicit) && + submodule_propagate_branches; + if (filter.abbrev == -1) filter.abbrev = DEFAULT_ABBREV; filter.ignore_case = icase; @@ -853,6 +878,9 @@ int cmd_branch(int argc, const char **argv, const char *prefix) git_config_set_multivar(buf.buf, NULL, NULL, CONFIG_FLAGS_MULTI_REPLACE); strbuf_release(&buf); } else if (!noncreate_actions && argc > 0 && argc <= 2) { + const char *branch_name = argv[0]; + const char *start_name = argc == 2 ? argv[1] : head; + if (filter.kind != FILTER_REFS_BRANCHES) die(_("The -a, and -r, options to 'git branch' do not take a branch name.\n" "Did you mean to use: -a|-r --list ?")); @@ -860,10 +888,14 @@ int cmd_branch(int argc, const char **argv, const char *prefix) if (track == BRANCH_TRACK_OVERRIDE) die(_("the '--set-upstream' option is no longer supported. Please use '--track' or '--set-upstream-to' instead.")); - create_branch(the_repository, - argv[0], (argc == 2) ? argv[1] : head, - force, 0, reflog, quiet, track, 0); - + if (recurse_submodules) { + create_branches_recursively(the_repository, branch_name, + start_name, NULL, force, + reflog, quiet, track, 0); + return 0; + } + create_branch(the_repository, branch_name, start_name, force, 0, + reflog, quiet, track, 0); } else usage_with_options(builtin_branch_usage, options); diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c index e630f0c730..44b6283c08 100644 --- a/builtin/submodule--helper.c +++ b/builtin/submodule--helper.c @@ -20,6 +20,7 @@ #include "diff.h" #include "object-store.h" #include "advice.h" +#include "branch.h" #define OPT_QUIET (1 << 0) #define OPT_CACHED (1 << 1) @@ -2983,6 +2984,42 @@ static int module_set_branch(int argc, const char **argv, const char *prefix) return !!ret; } +static int module_create_branch(int argc, const char **argv, const char *prefix) +{ + enum branch_track track; + int quiet = 0, force = 0, reflog = 0, dry_run = 0; + + struct option options[] = { + OPT__QUIET(&quiet, N_("print only error messages")), + OPT__FORCE(&force, N_("force creation"), 0), + OPT_BOOL(0, "create-reflog", &reflog, + N_("create the branch's reflog")), + OPT_SET_INT('t', "track", &track, + N_("set up tracking mode (see git-pull(1))"), + BRANCH_TRACK_EXPLICIT), + OPT__DRY_RUN(&dry_run, + N_("show whether the branch would be created")), + OPT_END() + }; + const char *const usage[] = { + N_("git submodule--helper create-branch [-f|--force] [--create-reflog] [-q|--quiet] [-t|--track] [-n|--dry-run] "), + NULL + }; + + git_config(git_default_config, NULL); + track = git_branch_track; + argc = parse_options(argc, argv, prefix, options, usage, 0); + + if (argc != 3) + usage_with_options(usage, options); + + if (!quiet && !dry_run) + printf_ln(_("creating branch '%s'"), argv[0]); + + create_branches_recursively(the_repository, argv[0], argv[1], argv[2], + force, reflog, quiet, track, dry_run); + return 0; +} struct add_data { const char *prefix; const char *branch; @@ -3389,6 +3426,7 @@ static struct cmd_struct commands[] = { {"config", module_config, 0}, {"set-url", module_set_url, 0}, {"set-branch", module_set_branch, 0}, + {"create-branch", module_create_branch, 0}, }; int cmd_submodule__helper(int argc, const char **argv, const char *prefix) diff --git a/submodule-config.c b/submodule-config.c index f95344028b..c9f54bc72d 100644 --- a/submodule-config.c +++ b/submodule-config.c @@ -7,6 +7,7 @@ #include "strbuf.h" #include "object-store.h" #include "parse-options.h" +#include "tree-walk.h" /* * submodule cache lookup structure @@ -726,6 +727,66 @@ const struct submodule *submodule_from_path(struct repository *r, return config_from(r->submodule_cache, treeish_name, path, lookup_path); } +/** + * Used internally by submodules_of_tree(). Recurses into 'treeish_name' + * and appends submodule entries to 'out'. The submodule_cache expects + * a root-level treeish_name and paths, so keep track of these values + * with 'root_tree' and 'prefix'. + */ +static void traverse_tree_submodules(struct repository *r, + const struct object_id *root_tree, + char *prefix, + const struct object_id *treeish_name, + struct submodule_entry_list *out) +{ + struct tree_desc tree; + struct submodule_tree_entry *st_entry; + struct name_entry *name_entry; + char *tree_path = NULL; + + name_entry = xmalloc(sizeof(*name_entry)); + + fill_tree_descriptor(r, &tree, treeish_name); + while (tree_entry(&tree, name_entry)) { + if (prefix) + tree_path = + mkpathdup("%s/%s", prefix, name_entry->path); + else + tree_path = xstrdup(name_entry->path); + + if (S_ISGITLINK(name_entry->mode) && + is_tree_submodule_active(r, root_tree, tree_path)) { + st_entry = xmalloc(sizeof(*st_entry)); + st_entry->name_entry = xmalloc(sizeof(*st_entry->name_entry)); + *st_entry->name_entry = *name_entry; + st_entry->submodule = + submodule_from_path(r, root_tree, tree_path); + st_entry->repo = xmalloc(sizeof(*st_entry->repo)); + if (repo_submodule_init(st_entry->repo, r, tree_path, + root_tree)) + FREE_AND_NULL(st_entry->repo); + + ALLOC_GROW(out->entries, out->entry_nr + 1, + out->entry_alloc); + out->entries[out->entry_nr++] = *st_entry; + } else if (S_ISDIR(name_entry->mode)) + traverse_tree_submodules(r, root_tree, tree_path, + &name_entry->oid, out); + free(tree_path); + } +} + +void submodules_of_tree(struct repository *r, + const struct object_id *treeish_name, + struct submodule_entry_list *out) +{ + CALLOC_ARRAY(out->entries, 0); + out->entry_nr = 0; + out->entry_alloc = 0; + + traverse_tree_submodules(r, treeish_name, NULL, treeish_name, out); +} + void submodule_free(struct repository *r) { if (r->submodule_cache) diff --git a/submodule-config.h b/submodule-config.h index 65875b94ea..fa229a8b97 100644 --- a/submodule-config.h +++ b/submodule-config.h @@ -6,6 +6,7 @@ #include "hashmap.h" #include "submodule.h" #include "strbuf.h" +#include "tree-walk.h" /** * The submodule config cache API allows to read submodule @@ -101,4 +102,37 @@ int check_submodule_name(const char *name); void fetch_config_from_gitmodules(int *max_children, int *recurse_submodules); void update_clone_config_from_gitmodules(int *max_jobs); +/* + * Submodule entry that contains relevant information about a + * submodule in a tree. + */ +struct submodule_tree_entry { + /* The submodule's tree entry. */ + struct name_entry *name_entry; + /* + * A struct repository corresponding to the submodule. May be + * NULL if the submodule has not been updated. + */ + struct repository *repo; + /* + * A struct submodule containing the submodule config in the + * tree's .gitmodules. + */ + const struct submodule *submodule; +}; + +struct submodule_entry_list { + struct submodule_tree_entry *entries; + int entry_nr; + int entry_alloc; +}; + +/** + * Given a treeish, return all submodules in the tree and its subtrees, + * but excluding nested submodules. Callers that require nested + * submodules are expected to recurse into the submodules themselves. + */ +void submodules_of_tree(struct repository *r, + const struct object_id *treeish_name, + struct submodule_entry_list *ret); #endif /* SUBMODULE_CONFIG_H */ diff --git a/submodule.c b/submodule.c index c689070524..5ace18a7d9 100644 --- a/submodule.c +++ b/submodule.c @@ -267,7 +267,9 @@ int option_parse_recurse_submodules_worktree_updater(const struct option *opt, * ie, the config looks like: "[submodule] active\n". * Since that is an invalid pathspec, we should inform the user. */ -int is_submodule_active(struct repository *repo, const char *path) +int is_tree_submodule_active(struct repository *repo, + const struct object_id *treeish_name, + const char *path) { int ret = 0; char *key = NULL; @@ -275,7 +277,7 @@ int is_submodule_active(struct repository *repo, const char *path) const struct string_list *sl; const struct submodule *module; - module = submodule_from_path(repo, null_oid(), path); + module = submodule_from_path(repo, treeish_name, path); /* early return if there isn't a path->module mapping */ if (!module) @@ -317,6 +319,11 @@ int is_submodule_active(struct repository *repo, const char *path) return ret; } +int is_submodule_active(struct repository *repo, const char *path) +{ + return is_tree_submodule_active(repo, null_oid(), path); +} + int is_submodule_populated_gently(const char *path, int *return_error_code) { int ret = 0; diff --git a/submodule.h b/submodule.h index 6bd2c99fd9..784ceffc0e 100644 --- a/submodule.h +++ b/submodule.h @@ -54,6 +54,9 @@ int git_default_submodule_config(const char *var, const char *value, void *cb); struct option; int option_parse_recurse_submodules_worktree_updater(const struct option *opt, const char *arg, int unset); +int is_tree_submodule_active(struct repository *repo, + const struct object_id *treeish_name, + const char *path); int is_submodule_active(struct repository *repo, const char *path); /* * Determine if a submodule has been populated at a given 'path' by checking if diff --git a/t/t3207-branch-submodule.sh b/t/t3207-branch-submodule.sh new file mode 100755 index 0000000000..6ef2733396 --- /dev/null +++ b/t/t3207-branch-submodule.sh @@ -0,0 +1,292 @@ +#!/bin/sh + +test_description='git branch submodule tests' + +GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME=main +export GIT_TEST_DEFAULT_INITIAL_BRANCH_NAME + +. ./test-lib.sh +. "$TEST_DIRECTORY"/lib-rebase.sh + +pwd=$(pwd) + +# Creates a clean test environment in "pwd" by copying the repo setup +# from test_dirs. +reset_test () { + rm -fr super && + rm -fr sub-sub-upstream && + rm -fr sub-upstream && + cp -r test_dirs/* . +} + +# Tests that the expected branch does not exist +test_no_branch () { + DIR=$1 && + BRANCH_NAME=$2 && + test_must_fail git -C "$DIR" rev-parse "$BRANCH_NAME" 2>err && + grep "ambiguous argument .$BRANCH_NAME." err +} + +test_expect_success 'setup superproject and submodule' ' + mkdir test_dirs && + ( + cd test_dirs && + git init super && + test_commit -C super foo && + git init sub-sub-upstream && + test_commit -C sub-sub-upstream foo && + git init sub-upstream && + # Submodule in a submodule + git -C sub-upstream submodule add "${pwd}/test_dirs/sub-sub-upstream" sub-sub && + git -C sub-upstream commit -m "add submodule" && + # Regular submodule + git -C super submodule add "${pwd}/test_dirs/sub-upstream" sub && + # Submodule in a subdirectory + git -C super submodule add "${pwd}/test_dirs/sub-sub-upstream" second/sub && + git -C super commit -m "add submodule" && + git -C super config submodule.propagateBranches true && + git -C super/sub submodule update --init + ) && + reset_test +' + +# Test the argument parsing +test_expect_success '--recurse-submodules should create branches' ' + test_when_finished "reset_test" && + ( + cd super && + git branch --recurse-submodules branch-a && + git rev-parse branch-a && + git -C sub rev-parse branch-a && + git -C sub/sub-sub rev-parse branch-a && + git -C second/sub rev-parse branch-a + ) +' + +test_expect_success '--recurse-submodules should die if submodule.propagateBranches is false' ' + test_when_finished "reset_test" && + ( + cd super && + echo "fatal: branch with --recurse-submodules can only be used if submodule.propagateBranches is enabled" >expected && + test_must_fail git -c submodule.propagateBranches=false branch --recurse-submodules branch-a 2>actual && + test_cmp expected actual + ) +' + +test_expect_success '--recurse-submodules should fail when not creating branches' ' + test_when_finished "reset_test" && + ( + cd super && + git branch --recurse-submodules branch-a && + echo "fatal: --recurse-submodules can only be used to create branches" >expected && + test_must_fail git branch --recurse-submodules -D branch-a 2>actual && + test_cmp expected actual && + # Assert that the branches were not deleted + git rev-parse branch-a && + git -C sub rev-parse branch-a + ) +' + +test_expect_success 'should respect submodule.recurse when creating branches' ' + test_when_finished "reset_test" && + ( + cd super && + git -c submodule.recurse=true branch branch-a && + git rev-parse branch-a && + git -C sub rev-parse branch-a + ) +' + +test_expect_success 'should ignore submodule.recurse when not creating branches' ' + test_when_finished "reset_test" && + ( + cd super && + git branch --recurse-submodules branch-a && + git -c submodule.recurse=true branch -D branch-a && + test_no_branch . branch-a && + git -C sub rev-parse branch-a + ) +' + +# Test branch creation behavior +test_expect_success 'should create branches based off commit id in superproject' ' + test_when_finished "reset_test" && + ( + cd super && + git branch --recurse-submodules branch-a && + git checkout --recurse-submodules branch-a && + git -C sub rev-parse HEAD >expected && + # Move the tip of sub:branch-a so that it no longer matches the commit in super:branch-a + git -C sub checkout branch-a && + test_commit -C sub bar && + # Create a new branch-b branch with start-point=branch-a + git branch --recurse-submodules branch-b branch-a && + git rev-parse branch-b && + git -C sub rev-parse branch-b >actual && + # Assert that the commit id of sub:second-branch matches super:branch-a and not sub:branch-a + test_cmp expected actual + ) +' + +test_expect_success 'should not create any branches if branch is not valid for all repos' ' + test_when_finished "reset_test" && + ( + cd super && + git -C sub branch branch-a && + test_must_fail git branch --recurse-submodules branch-a 2>actual && + test_no_branch . branch-a && + grep "submodule .sub.: fatal: A branch named .branch-a. already exists" actual + ) +' + +test_expect_success 'should create branches if branch exists and --force is given' ' + test_when_finished "reset_test" && + ( + cd super && + git -C sub rev-parse HEAD >expected && + test_commit -C sub baz && + # branch-a in sub now points to a newer commit. + git -C sub branch branch-a HEAD && + git -C sub rev-parse branch-a >actual-old-branch-a && + git branch --recurse-submodules --force branch-a && + git rev-parse branch-a && + git -C sub rev-parse branch-a >actual-new-branch-a && + test_cmp expected actual-new-branch-a && + # assert that branch --force actually moved the sub + # branch + ! test_cmp expected actual-old-branch-a + ) +' + +test_expect_success 'should create branch when submodule is not in HEAD:.gitmodules' ' + test_when_finished "reset_test" && + ( + cd super && + git branch branch-a && + git checkout -b branch-b && + git submodule add ../sub-upstream sub2 && + git -C sub2 submodule update --init && + # branch-b now has a committed submodule not in branch-a + git commit -m "add second submodule" && + git checkout branch-a && + git branch --recurse-submodules branch-c branch-b && + git checkout --recurse-submodules branch-c && + git -C sub2 rev-parse branch-c && + git -C sub2/sub-sub rev-parse branch-c + ) +' + +test_expect_success 'should not create branches in inactive submodules' ' + test_when_finished "reset_test" && + test_config -C super submodule.sub.active false && + ( + cd super && + git branch --recurse-submodules branch-a && + git rev-parse branch-a && + test_no_branch sub branch-a + ) +' + +test_expect_success 'should set up tracking of local branches with track=always' ' + test_when_finished "reset_test" && + ( + cd super && + git -c branch.autoSetupMerge=always branch --recurse-submodules branch-a main && + git -C sub rev-parse main && + test_cmp_config -C sub . branch.branch-a.remote && + test_cmp_config -C sub refs/heads/main branch.branch-a.merge + ) +' + +test_expect_success 'should set up tracking of local branches with explicit track' ' + test_when_finished "reset_test" && + ( + cd super && + git branch --track --recurse-submodules branch-a main && + git -C sub rev-parse main && + test_cmp_config -C sub . branch.branch-a.remote && + test_cmp_config -C sub refs/heads/main branch.branch-a.merge + ) +' + +test_expect_success 'should not set up unnecessary tracking of local branches' ' + test_when_finished "reset_test" && + ( + cd super && + git branch --recurse-submodules branch-a main && + git -C sub rev-parse main && + test_cmp_config -C sub "" --default "" branch.branch-a.remote && + test_cmp_config -C sub "" --default "" branch.branch-a.merge + ) +' + +reset_remote_test () { + rm -fr super-clone && + reset_test +} + +test_expect_success 'setup tests with remotes' ' + ( + cd test_dirs && + ( + cd super && + git branch branch-a && + git checkout -b branch-b && + git submodule add ../sub-upstream sub2 && + # branch-b now has a committed submodule not in branch-a + git commit -m "add second submodule" + ) && + git clone --branch main --recurse-submodules super super-clone && + git -C super-clone config submodule.propagateBranches true + ) && + reset_remote_test +' + +test_expect_success 'should get fatal error upon branch creation when submodule is not in .git/modules' ' + test_when_finished "reset_remote_test" && + ( + cd super-clone && + # This should succeed because super-clone has sub in .git/modules + git branch --recurse-submodules branch-a origin/branch-a && + # This should fail because super-clone does not have sub2 .git/modules + test_must_fail git branch --recurse-submodules branch-b origin/branch-b 2>actual && + grep "fatal: submodule .sub2.: unable to find submodule" actual && + test_no_branch . branch-b && + test_no_branch sub branch-b && + # User can fix themselves by initializing the submodule + git checkout origin/branch-b && + git submodule update --init --recursive && + git branch --recurse-submodules branch-b origin/branch-b + ) +' + +test_expect_success 'should set up tracking of remote-tracking branches' ' + test_when_finished "reset_remote_test" && + ( + cd super-clone && + git branch --recurse-submodules branch-a origin/branch-a && + test_cmp_config origin branch.branch-a.remote && + test_cmp_config refs/heads/branch-a branch.branch-a.merge && + # "origin/branch-a" does not exist for "sub", but it matches the refspec + # so tracking should be set up + test_cmp_config -C sub origin branch.branch-a.remote && + test_cmp_config -C sub refs/heads/branch-a branch.branch-a.merge && + test_cmp_config -C sub/sub-sub origin branch.branch-a.remote && + test_cmp_config -C sub/sub-sub refs/heads/branch-a branch.branch-a.merge + ) +' + +test_expect_success 'should not fail when unable to set up tracking in submodule' ' + test_when_finished "reset_remote_test" && + ( + cd super-clone && + git remote rename origin ex-origin && + git branch --recurse-submodules branch-a ex-origin/branch-a && + test_cmp_config ex-origin branch.branch-a.remote && + test_cmp_config refs/heads/branch-a branch.branch-a.merge && + test_cmp_config -C sub "" --default "" branch.branch-a.remote && + test_cmp_config -C sub "" --default "" branch.branch-a.merge + ) +' + +test_done -- cgit v1.3-5-g9baa From 5c11c0d52c398af65f19bcf65a46304552083214 Mon Sep 17 00:00:00 2001 From: Derrick Stolee Date: Mon, 7 Feb 2022 21:32:58 +0000 Subject: Documentation: add extensions.worktreeConfig details The extensions.worktreeConfig extension was added in 58b284a (worktree: add per-worktree config files, 2018-10-21) and was somewhat documented in Documentation/git-config.txt. However, the extensions.worktreeConfig value was not specified further in the list of possible config keys. The location of the config.worktree file is not specified, and there are some precautions that should be mentioned clearly, but are only mentioned in git-worktree.txt. Expand the documentation to help users discover the complexities of extensions.worktreeConfig by adding details and cross links in these locations (relative to Documentation/): - config/extensions.txt - git-config.txt - git-worktree.txt The updates focus on items such as * $GIT_DIR/config.worktree takes precedence over $GIT_COMMON_DIR/config. * The core.worktree and core.bare=true settings are incorrect to have in the common config file when extensions.worktreeConfig is enabled. * The sparse-checkout settings core.sparseCheckout[Cone] are recommended to be set in the worktree config. As documented in 11664196ac ("Revert "check_repository_format_gently(): refuse extensions for old repositories"", 2020-07-15), this extension must be considered regardless of the repository format version for historical reasons. A future change will update references to extensions.worktreeConfig within git-sparse-checkout.txt, but a behavior change is needed before making those updates. Helped-by: Elijah Newren Signed-off-by: Derrick Stolee Reviewed-by: Elijah Newren Signed-off-by: Junio C Hamano --- Documentation/config/extensions.txt | 31 +++++++++++++++++++++++++++++++ Documentation/git-config.txt | 8 ++++++-- Documentation/git-worktree.txt | 11 ++++++++--- 3 files changed, 45 insertions(+), 5 deletions(-) (limited to 'Documentation') diff --git a/Documentation/config/extensions.txt b/Documentation/config/extensions.txt index 4e23d73cdc..bccaec7a96 100644 --- a/Documentation/config/extensions.txt +++ b/Documentation/config/extensions.txt @@ -6,3 +6,34 @@ extensions.objectFormat:: Note that this setting should only be set by linkgit:git-init[1] or linkgit:git-clone[1]. Trying to change it after initialization will not work and will produce hard-to-diagnose issues. + +extensions.worktreeConfig:: + If enabled, then worktrees will load config settings from the + `$GIT_DIR/config.worktree` file in addition to the + `$GIT_COMMON_DIR/config` file. Note that `$GIT_COMMON_DIR` and + `$GIT_DIR` are the same for the main working tree, while other + working trees have `$GIT_DIR` equal to + `$GIT_COMMON_DIR/worktrees//`. The settings in the + `config.worktree` file will override settings from any other + config files. ++ +When enabling `extensions.worktreeConfig`, you must be careful to move +certain values from the common config file to the main working tree's +`config.worktree` file, if present: ++ +* `core.worktree` must be moved from `$GIT_COMMON_DIR/config` to + `$GIT_COMMON_DIR/config.worktree`. +* If `core.bare` is true, then it must be moved from `$GIT_COMMON_DIR/config` + to `$GIT_COMMON_DIR/config.worktree`. ++ +It may also be beneficial to adjust the locations of `core.sparseCheckout` +and `core.sparseCheckoutCone` depending on your desire for customizable +sparse-checkout settings for each worktree. By default, the `git +sparse-checkout` builtin enables `extensions.worktreeConfig`, assigns +these config values on a per-worktree basis, and uses the +`$GIT_DIR/info/sparse-checkout` file to specify the sparsity for each +worktree independently. See linkgit:git-sparse-checkout[1] for more +details. ++ +For historical reasons, `extensions.worktreeConfig` is respected +regardless of the `core.repositoryFormatVersion` setting. diff --git a/Documentation/git-config.txt b/Documentation/git-config.txt index 2285effb36..bdcfd94b64 100644 --- a/Documentation/git-config.txt +++ b/Documentation/git-config.txt @@ -141,9 +141,13 @@ from all available files. See also <>. --worktree:: - Similar to `--local` except that `.git/config.worktree` is + Similar to `--local` except that `$GIT_DIR/config.worktree` is read from or written to if `extensions.worktreeConfig` is - present. If not it's the same as `--local`. + enabled. If not it's the same as `--local`. Note that `$GIT_DIR` + is equal to `$GIT_COMMON_DIR` for the main working tree, but is of + the form `$GIT_DIR/worktrees//` for other working trees. See + linkgit:git-worktree[1] to learn how to enable + `extensions.worktreeConfig`. -f :: --file :: diff --git a/Documentation/git-worktree.txt b/Documentation/git-worktree.txt index 9e862fbcf7..b8d53c4830 100644 --- a/Documentation/git-worktree.txt +++ b/Documentation/git-worktree.txt @@ -286,8 +286,8 @@ CONFIGURATION FILE ------------------ By default, the repository `config` file is shared across all working trees. If the config variables `core.bare` or `core.worktree` are -already present in the config file, they will be applied to the main -working trees only. +present in the common config file and `extensions.worktreeConfig` is +disabled, then they will be applied to the main working tree only. In order to have configuration specific to working trees, you can turn on the `worktreeConfig` extension, e.g.: @@ -307,11 +307,16 @@ them to the `config.worktree` of the main working tree. You may also take this opportunity to review and move other configuration that you do not want to share to all working trees: - - `core.worktree` and `core.bare` should never be shared + - `core.worktree` should never be shared. + + - `core.bare` should not be shared if the value is `core.bare=true`. - `core.sparseCheckout` is recommended per working tree, unless you are sure you always use sparse checkout for all working trees. +See the documentation of `extensions.worktreeConfig` in +linkgit:git-config[1] for more details. + DETAILS ------- Each linked working tree has a private sub-directory in the repository's -- cgit v1.3-5-g9baa From 7316dc5f6f2c8297d32e47d5859933ffacb6c00e Mon Sep 17 00:00:00 2001 From: Derrick Stolee Date: Mon, 7 Feb 2022 21:33:01 +0000 Subject: sparse-checkout: set worktree-config correctly `git sparse-checkout set/init` enables worktree-specific configuration[*] by setting extensions.worktreeConfig=true, but neglects to perform the additional necessary bookkeeping of relocating `core.bare=true` and `core.worktree` from $GIT_COMMON_DIR/config to $GIT_COMMON_DIR/config.worktree, as documented in git-worktree.txt. As a result of this oversight, these settings, which are nonsensical for secondary worktrees, can cause Git commands to incorrectly consider a worktree bare (in the case of `core.bare`) or operate on the wrong worktree (in the case of `core.worktree`). Fix this problem by taking advantage of the recently-added init_worktree_config() which enables `extensions.worktreeConfig` and takes care of necessary bookkeeping. While at it, for backward-compatibility reasons, also stop upgrading the repository format to "1" since doing so is (unintentionally) not required to take advantage of `extensions.worktreeConfig`, as explained by 11664196ac ("Revert "check_repository_format_gently(): refuse extensions for old repositories"", 2020-07-15). [*] The main reason to use worktree-specific config for the sparse-checkout builtin was to avoid enabling sparse-checkout patterns in one and causing a loss of files in another. If a worktree does not have a sparse-checkout patterns file, then the sparse-checkout logic will not kick in on that worktree. Reported-by: Sean Allred Helped-by: Eric Sunshine Signed-off-by: Derrick Stolee Reviewed-by: Elijah Newren Signed-off-by: Junio C Hamano --- Documentation/git-sparse-checkout.txt | 16 ++++++++++++---- builtin/sparse-checkout.c | 28 +++++++++++++--------------- sparse-index.c | 10 +++------- t/t1091-sparse-checkout-builtin.sh | 4 ++-- 4 files changed, 30 insertions(+), 28 deletions(-) (limited to 'Documentation') diff --git a/Documentation/git-sparse-checkout.txt b/Documentation/git-sparse-checkout.txt index b81dbe0654..94dad137b9 100644 --- a/Documentation/git-sparse-checkout.txt +++ b/Documentation/git-sparse-checkout.txt @@ -31,13 +31,21 @@ COMMANDS Describe the patterns in the sparse-checkout file. 'set':: - Enable the necessary config settings - (extensions.worktreeConfig, core.sparseCheckout, - core.sparseCheckoutCone) if they are not already enabled, and - write a set of patterns to the sparse-checkout file from the + Enable the necessary sparse-checkout config settings + (`core.sparseCheckout`, `core.sparseCheckoutCone`, and + `index.sparse`) if they are not already set to the desired values, + and write a set of patterns to the sparse-checkout file from the list of arguments following the 'set' subcommand. Update the working directory to match the new patterns. + +To ensure that adjusting the sparse-checkout settings within a worktree +does not alter the sparse-checkout settings in other worktrees, the 'set' +subcommand will upgrade your repository config to use worktree-specific +config if not already present. The sparsity defined by the arguments to +the 'set' subcommand are stored in the worktree-specific sparse-checkout +file. See linkgit:git-worktree[1] and the documentation of +`extensions.worktreeConfig` in linkgit:git-config[1] for more details. ++ When the `--stdin` option is provided, the patterns are read from standard in as a newline-delimited list instead of from the arguments. + diff --git a/builtin/sparse-checkout.c b/builtin/sparse-checkout.c index 679c107036..314c8d61f8 100644 --- a/builtin/sparse-checkout.c +++ b/builtin/sparse-checkout.c @@ -15,6 +15,7 @@ #include "wt-status.h" #include "quote.h" #include "sparse-index.h" +#include "worktree.h" static const char *empty_base = ""; @@ -359,26 +360,23 @@ enum sparse_checkout_mode { static int set_config(enum sparse_checkout_mode mode) { - const char *config_path; - - if (upgrade_repository_format(1) < 0) - die(_("unable to upgrade repository format to enable worktreeConfig")); - if (git_config_set_gently("extensions.worktreeConfig", "true")) { - error(_("failed to set extensions.worktreeConfig setting")); + /* Update to use worktree config, if not already. */ + if (init_worktree_config(the_repository)) { + error(_("failed to initialize worktree config")); return 1; } - config_path = git_path("config.worktree"); - git_config_set_in_file_gently(config_path, - "core.sparseCheckout", - mode ? "true" : NULL); - - git_config_set_in_file_gently(config_path, - "core.sparseCheckoutCone", - mode == MODE_CONE_PATTERNS ? "true" : NULL); + if (repo_config_set_worktree_gently(the_repository, + "core.sparseCheckout", + mode ? "true" : "false") || + repo_config_set_worktree_gently(the_repository, + "core.sparseCheckoutCone", + mode == MODE_CONE_PATTERNS ? + "true" : "false")) + return 1; if (mode == MODE_NO_PATTERNS) - set_sparse_index_config(the_repository, 0); + return set_sparse_index_config(the_repository, 0); return 0; } diff --git a/sparse-index.c b/sparse-index.c index a1d505d50e..e93609999e 100644 --- a/sparse-index.c +++ b/sparse-index.c @@ -99,13 +99,9 @@ static int convert_to_sparse_rec(struct index_state *istate, int set_sparse_index_config(struct repository *repo, int enable) { - int res; - char *config_path = repo_git_path(repo, "config.worktree"); - res = git_config_set_in_file_gently(config_path, - "index.sparse", - enable ? "true" : NULL); - free(config_path); - + int res = repo_config_set_worktree_gently(repo, + "index.sparse", + enable ? "true" : "false"); prepare_repo_settings(repo); repo->settings.sparse_index = enable; return res; diff --git a/t/t1091-sparse-checkout-builtin.sh b/t/t1091-sparse-checkout-builtin.sh index 42776984fe..be6ea4ffe3 100755 --- a/t/t1091-sparse-checkout-builtin.sh +++ b/t/t1091-sparse-checkout-builtin.sh @@ -117,7 +117,7 @@ test_expect_success 'switching to cone mode with non-cone mode patterns' ' cd bad-patterns && git sparse-checkout init && git sparse-checkout add dir && - git config core.sparseCheckoutCone true && + git config --worktree core.sparseCheckoutCone true && test_must_fail git sparse-checkout add dir 2>err && grep "existing sparse-checkout patterns do not use cone mode" err ) @@ -256,7 +256,7 @@ test_expect_success 'sparse-index enabled and disabled' ' test_cmp expect actual && git -C repo config --list >config && - ! grep index.sparse config + test_cmp_config -C repo false index.sparse ) ' -- cgit v1.3-5-g9baa From f05da2b48b48a46db65fc768b3ffecaf996dd655 Mon Sep 17 00:00:00 2001 From: Josh Steadmon Date: Fri, 4 Feb 2022 21:00:49 -0800 Subject: clone, submodule: pass partial clone filters to submodules When cloning a repo with a --filter and with --recurse-submodules enabled, the partial clone filter only applies to the top-level repo. This can lead to unexpected bandwidth and disk usage for projects which include large submodules. For example, a user might wish to make a partial clone of Gerrit and would run: `git clone --recurse-submodules --filter=blob:5k https://gerrit.googlesource.com/gerrit`. However, only the superproject would be a partial clone; all the submodules would have all blobs downloaded regardless of their size. With this change, the same filter can also be applied to submodules, meaning the expected bandwidth and disk savings apply consistently. To avoid changing default behavior, add a new clone flag, `--also-filter-submodules`. When this is set along with `--filter` and `--recurse-submodules`, the filter spec is passed along to git-submodule and git-submodule--helper, such that submodule clones also have the filter applied. This applies the same filter to the superproject and all submodules. Users who need to customize the filter per-submodule would need to clone with `--no-recurse-submodules` and then manually initialize each submodule with the proper filter. Applying filters to submodules should be safe thanks to Jonathan Tan's recent work [1, 2, 3] eliminating the use of alternates as a method of accessing submodule objects, so any submodule object access now triggers a lazy fetch from the submodule's promisor remote if the accessed object is missing. This patch is a reworked version of [4], which was created prior to Jonathan Tan's work. [1]: 8721e2e (Merge branch 'jt/partial-clone-submodule-1', 2021-07-16) [2]: 11e5d0a (Merge branch 'jt/grep-wo-submodule-odb-as-alternate', 2021-09-20) [3]: 162a13b (Merge branch 'jt/no-abuse-alternate-odb-for-submodules', 2021-10-25) [4]: https://lore.kernel.org/git/52bf9d45b8e2b72ff32aa773f2415bf7b2b86da2.1563322192.git.steadmon@google.com/ Signed-off-by: Josh Steadmon Signed-off-by: Junio C Hamano --- Documentation/config/clone.txt | 5 +++++ Documentation/git-clone.txt | 7 ++++++- Documentation/git-submodule.txt | 6 +++++- builtin/clone.c | 36 +++++++++++++++++++++++++++++++-- builtin/submodule--helper.c | 30 +++++++++++++++++++++++++--- git-submodule.sh | 17 +++++++++++++++- t/t5617-clone-submodules-remote.sh | 41 ++++++++++++++++++++++++++++++++++++++ t/t7814-grep-recurse-submodules.sh | 41 ++++++++++++++++++++++++++++++++++++++ 8 files changed, 175 insertions(+), 8 deletions(-) (limited to 'Documentation') diff --git a/Documentation/config/clone.txt b/Documentation/config/clone.txt index 7bcfbd18a5..26f4fb137a 100644 --- a/Documentation/config/clone.txt +++ b/Documentation/config/clone.txt @@ -6,3 +6,8 @@ clone.defaultRemoteName:: clone.rejectShallow:: Reject to clone a repository if it is a shallow one, can be overridden by passing option `--reject-shallow` in command line. See linkgit:git-clone[1] + +clone.filterSubmodules:: + If a partial clone filter is provided (see `--filter` in + linkgit:git-rev-list[1]) and `--recurse-submodules` is used, also apply + the filter to submodules. diff --git a/Documentation/git-clone.txt b/Documentation/git-clone.txt index 984d194934..632bd1348e 100644 --- a/Documentation/git-clone.txt +++ b/Documentation/git-clone.txt @@ -16,7 +16,7 @@ SYNOPSIS [--depth ] [--[no-]single-branch] [--no-tags] [--recurse-submodules[=]] [--[no-]shallow-submodules] [--[no-]remote-submodules] [--jobs ] [--sparse] [--[no-]reject-shallow] - [--filter=] [--] + [--filter= [--also-filter-submodules]] [--] [] DESCRIPTION @@ -182,6 +182,11 @@ objects from the source repository into a pack in the cloned repository. at least ``. For more details on filter specifications, see the `--filter` option in linkgit:git-rev-list[1]. +--also-filter-submodules:: + Also apply the partial clone filter to any submodules in the repository. + Requires `--filter` and `--recurse-submodules`. This can be turned on by + default by setting the `clone.filterSubmodules` config option. + --mirror:: Set up a mirror of the source repository. This implies `--bare`. Compared to `--bare`, `--mirror` not only maps local branches of the diff --git a/Documentation/git-submodule.txt b/Documentation/git-submodule.txt index 7e5f995f77..4d3ab6b9f9 100644 --- a/Documentation/git-submodule.txt +++ b/Documentation/git-submodule.txt @@ -133,7 +133,7 @@ If you really want to remove a submodule from the repository and commit that use linkgit:git-rm[1] instead. See linkgit:gitsubmodules[7] for removal options. -update [--init] [--remote] [-N|--no-fetch] [--[no-]recommend-shallow] [-f|--force] [--checkout|--rebase|--merge] [--reference ] [--depth ] [--recursive] [--jobs ] [--[no-]single-branch] [--] [...]:: +update [--init] [--remote] [-N|--no-fetch] [--[no-]recommend-shallow] [-f|--force] [--checkout|--rebase|--merge] [--reference ] [--depth ] [--recursive] [--jobs ] [--[no-]single-branch] [--filter ] [--] [...]:: + -- Update the registered submodules to match what the superproject @@ -177,6 +177,10 @@ submodule with the `--init` option. If `--recursive` is specified, this command will recurse into the registered submodules, and update any nested submodules within. + +If `--filter ` is specified, the given partial clone filter will be +applied to the submodule. See linkgit:git-rev-list[1] for details on filter +specifications. -- set-branch (-b|--branch) [--] :: set-branch (-d|--default) [--] :: diff --git a/builtin/clone.c b/builtin/clone.c index 727e16e0ae..fb605e4c8d 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -71,6 +71,8 @@ static int option_dissociate; static int max_jobs = -1; static struct string_list option_recurse_submodules = STRING_LIST_INIT_NODUP; static struct list_objects_filter_options filter_options; +static int option_filter_submodules = -1; /* unspecified */ +static int config_filter_submodules = -1; /* unspecified */ static struct string_list server_options = STRING_LIST_INIT_NODUP; static int option_remote_submodules; @@ -150,6 +152,8 @@ static struct option builtin_clone_options[] = { OPT_SET_INT('6', "ipv6", &family, N_("use IPv6 addresses only"), TRANSPORT_FAMILY_IPV6), OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options), + OPT_BOOL(0, "also-filter-submodules", &option_filter_submodules, + N_("apply partial clone filters to submodules")), OPT_BOOL(0, "remote-submodules", &option_remote_submodules, N_("any cloned submodules will use their remote-tracking branch")), OPT_BOOL(0, "sparse", &option_sparse_checkout, @@ -650,7 +654,7 @@ static int git_sparse_checkout_init(const char *repo) return result; } -static int checkout(int submodule_progress) +static int checkout(int submodule_progress, int filter_submodules) { struct object_id oid; char *head; @@ -729,6 +733,10 @@ static int checkout(int submodule_progress) strvec_push(&args, "--no-fetch"); } + if (filter_submodules && filter_options.choice) + strvec_pushf(&args, "--filter=%s", + expand_list_objects_filter_spec(&filter_options)); + if (option_single_branch >= 0) strvec_push(&args, option_single_branch ? "--single-branch" : @@ -749,6 +757,8 @@ static int git_clone_config(const char *k, const char *v, void *cb) } if (!strcmp(k, "clone.rejectshallow")) config_reject_shallow = git_config_bool(k, v); + if (!strcmp(k, "clone.filtersubmodules")) + config_filter_submodules = git_config_bool(k, v); return git_default_config(k, v, cb); } @@ -871,6 +881,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix) struct remote *remote; int err = 0, complete_refs_before_fetch = 1; int submodule_progress; + int filter_submodules = 0; struct transport_ls_refs_options transport_ls_refs_options = TRANSPORT_LS_REFS_OPTIONS_INIT; @@ -1066,6 +1077,27 @@ int cmd_clone(int argc, const char **argv, const char *prefix) if (option_reject_shallow != -1) reject_shallow = option_reject_shallow; + /* + * If option_filter_submodules is specified from CLI option, + * ignore config_filter_submodules from git_clone_config. + */ + if (config_filter_submodules != -1) + filter_submodules = config_filter_submodules; + if (option_filter_submodules != -1) + filter_submodules = option_filter_submodules; + + /* + * Exit if the user seems to be doing something silly with submodule + * filter flags (but not with filter configs, as those should be + * set-and-forget). + */ + if (option_filter_submodules > 0 && !filter_options.choice) + die(_("the option '%s' requires '%s'"), + "--also-filter-submodules", "--filter"); + if (option_filter_submodules > 0 && !option_recurse_submodules.nr) + die(_("the option '%s' requires '%s'"), + "--also-filter-submodules", "--recurse-submodules"); + /* * apply the remote name provided by --origin only after this second * call to git_config, to ensure it overrides all config-based values. @@ -1299,7 +1331,7 @@ int cmd_clone(int argc, const char **argv, const char *prefix) } junk_mode = JUNK_LEAVE_REPO; - err = checkout(submodule_progress); + err = checkout(submodule_progress, filter_submodules); free(remote_name); strbuf_release(&reflog_msg); diff --git a/builtin/submodule--helper.c b/builtin/submodule--helper.c index c5d3fc3817..11552970f2 100644 --- a/builtin/submodule--helper.c +++ b/builtin/submodule--helper.c @@ -20,6 +20,7 @@ #include "diff.h" #include "object-store.h" #include "advice.h" +#include "list-objects-filter-options.h" #define OPT_QUIET (1 << 0) #define OPT_CACHED (1 << 1) @@ -1630,6 +1631,7 @@ struct module_clone_data { const char *name; const char *url; const char *depth; + struct list_objects_filter_options *filter_options; struct string_list reference; unsigned int quiet: 1; unsigned int progress: 1; @@ -1796,6 +1798,10 @@ static int clone_submodule(struct module_clone_data *clone_data) strvec_push(&cp.args, "--dissociate"); if (sm_gitdir && *sm_gitdir) strvec_pushl(&cp.args, "--separate-git-dir", sm_gitdir, NULL); + if (clone_data->filter_options && clone_data->filter_options->choice) + strvec_pushf(&cp.args, "--filter=%s", + expand_list_objects_filter_spec( + clone_data->filter_options)); if (clone_data->single_branch >= 0) strvec_push(&cp.args, clone_data->single_branch ? "--single-branch" : @@ -1852,6 +1858,7 @@ static int module_clone(int argc, const char **argv, const char *prefix) { int dissociate = 0, quiet = 0, progress = 0, require_init = 0; struct module_clone_data clone_data = MODULE_CLONE_DATA_INIT; + struct list_objects_filter_options filter_options; struct option module_clone_options[] = { OPT_STRING(0, "prefix", &clone_data.prefix, @@ -1881,17 +1888,19 @@ static int module_clone(int argc, const char **argv, const char *prefix) N_("disallow cloning into non-empty directory")), OPT_BOOL(0, "single-branch", &clone_data.single_branch, N_("clone only one branch, HEAD or --branch")), + OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options), OPT_END() }; const char *const git_submodule_helper_usage[] = { N_("git submodule--helper clone [--prefix=] [--quiet] " "[--reference ] [--name ] [--depth ] " - "[--single-branch] " + "[--single-branch] [--filter ]" "--url --path "), NULL }; + memset(&filter_options, 0, sizeof(filter_options)); argc = parse_options(argc, argv, prefix, module_clone_options, git_submodule_helper_usage, 0); @@ -1899,12 +1908,14 @@ static int module_clone(int argc, const char **argv, const char *prefix) clone_data.quiet = !!quiet; clone_data.progress = !!progress; clone_data.require_init = !!require_init; + clone_data.filter_options = &filter_options; if (argc || !clone_data.url || !clone_data.path || !*(clone_data.path)) usage_with_options(git_submodule_helper_usage, module_clone_options); clone_submodule(&clone_data); + list_objects_filter_release(&filter_options); return 0; } @@ -1994,6 +2005,7 @@ struct submodule_update_clone { const char *recursive_prefix; const char *prefix; int single_branch; + struct list_objects_filter_options *filter_options; /* to be consumed by git-submodule.sh */ struct update_clone_data *update_clone; @@ -2154,6 +2166,9 @@ static int prepare_to_clone_next_submodule(const struct cache_entry *ce, strvec_pushl(&child->args, "--prefix", suc->prefix, NULL); if (suc->recommend_shallow && sub->recommend_shallow == 1) strvec_push(&child->args, "--depth=1"); + if (suc->filter_options && suc->filter_options->choice) + strvec_pushf(&child->args, "--filter=%s", + expand_list_objects_filter_spec(suc->filter_options)); if (suc->require_init) strvec_push(&child->args, "--require-init"); strvec_pushl(&child->args, "--path", sub->path, NULL); @@ -2498,6 +2513,8 @@ static int update_clone(int argc, const char **argv, const char *prefix) const char *update = NULL; struct pathspec pathspec; struct submodule_update_clone suc = SUBMODULE_UPDATE_CLONE_INIT; + struct list_objects_filter_options filter_options; + int ret; struct option module_update_clone_options[] = { OPT_STRING(0, "prefix", &prefix, @@ -2528,6 +2545,7 @@ static int update_clone(int argc, const char **argv, const char *prefix) N_("disallow cloning into non-empty directory")), OPT_BOOL(0, "single-branch", &suc.single_branch, N_("clone only one branch, HEAD or --branch")), + OPT_PARSE_LIST_OBJECTS_FILTER(&filter_options), OPT_END() }; @@ -2540,20 +2558,26 @@ static int update_clone(int argc, const char **argv, const char *prefix) update_clone_config_from_gitmodules(&suc.max_jobs); git_config(git_update_clone_config, &suc.max_jobs); + memset(&filter_options, 0, sizeof(filter_options)); argc = parse_options(argc, argv, prefix, module_update_clone_options, git_submodule_helper_usage, 0); + suc.filter_options = &filter_options; if (update) if (parse_submodule_update_strategy(update, &suc.update) < 0) die(_("bad value for update parameter")); - if (module_list_compute(argc, argv, prefix, &pathspec, &suc.list) < 0) + if (module_list_compute(argc, argv, prefix, &pathspec, &suc.list) < 0) { + list_objects_filter_release(&filter_options); return 1; + } if (pathspec.nr) suc.warn_if_uninitialized = 1; - return update_submodules(&suc); + ret = update_submodules(&suc); + list_objects_filter_release(&filter_options); + return ret; } static int run_update_procedure(int argc, const char **argv, const char *prefix) diff --git a/git-submodule.sh b/git-submodule.sh index 652861aa66..87772ac891 100755 --- a/git-submodule.sh +++ b/git-submodule.sh @@ -10,7 +10,7 @@ USAGE="[--quiet] [--cached] or: $dashless [--quiet] status [--cached] [--recursive] [--] [...] or: $dashless [--quiet] init [--] [...] or: $dashless [--quiet] deinit [-f|--force] (--all| [--] ...) - or: $dashless [--quiet] update [--init] [--remote] [-N|--no-fetch] [-f|--force] [--checkout|--merge|--rebase] [--[no-]recommend-shallow] [--reference ] [--recursive] [--[no-]single-branch] [--] [...] + or: $dashless [--quiet] update [--init [--filter=]] [--remote] [-N|--no-fetch] [-f|--force] [--checkout|--merge|--rebase] [--[no-]recommend-shallow] [--reference ] [--recursive] [--[no-]single-branch] [--] [...] or: $dashless [--quiet] set-branch (--default|--branch ) [--] or: $dashless [--quiet] set-url [--] or: $dashless [--quiet] summary [--cached|--files] [--summary-limit ] [commit] [--] [...] @@ -49,6 +49,7 @@ dissociate= single_branch= jobs= recommend_shallow= +filter= die_if_unmatched () { @@ -347,6 +348,14 @@ cmd_update() --no-single-branch) single_branch="--no-single-branch" ;; + --filter) + case "$2" in '') usage ;; esac + filter="--filter=$2" + shift + ;; + --filter=*) + filter="$1" + ;; --) shift break @@ -361,6 +370,11 @@ cmd_update() shift done + if test -n "$filter" && test "$init" != "1" + then + usage + fi + if test -n "$init" then cmd_init "--" "$@" || return @@ -379,6 +393,7 @@ cmd_update() $single_branch \ $recommend_shallow \ $jobs \ + $filter \ -- \ "$@" || echo "#unmatched" $? } | { diff --git a/t/t5617-clone-submodules-remote.sh b/t/t5617-clone-submodules-remote.sh index e2dbb4eaba..ca8f80083a 100755 --- a/t/t5617-clone-submodules-remote.sh +++ b/t/t5617-clone-submodules-remote.sh @@ -28,6 +28,13 @@ test_expect_success 'setup' ' ) ' +# bare clone giving "srv.bare" for use as our server. +test_expect_success 'setup bare clone for server' ' + git clone --bare "file://$(pwd)/." srv.bare && + git -C srv.bare config --local uploadpack.allowfilter 1 && + git -C srv.bare config --local uploadpack.allowanysha1inwant 1 +' + test_expect_success 'clone with --no-remote-submodules' ' test_when_finished "rm -rf super_clone" && git clone --recurse-submodules --no-remote-submodules "file://$pwd/." super_clone && @@ -65,4 +72,38 @@ test_expect_success 'clone with --single-branch' ' ) ' +# do basic partial clone from "srv.bare" +# confirm partial clone was registered in the local config for super and sub. +test_expect_success 'clone with --filter' ' + git clone --recurse-submodules \ + --filter blob:none --also-filter-submodules \ + "file://$pwd/srv.bare" super_clone && + test_cmp_config -C super_clone true remote.origin.promisor && + test_cmp_config -C super_clone blob:none remote.origin.partialclonefilter && + test_cmp_config -C super_clone/sub true remote.origin.promisor && + test_cmp_config -C super_clone/sub blob:none remote.origin.partialclonefilter +' + +# check that clone.filterSubmodules works (--also-filter-submodules can be +# omitted) +test_expect_success 'filters applied with clone.filterSubmodules' ' + test_config_global clone.filterSubmodules true && + git clone --recurse-submodules --filter blob:none \ + "file://$pwd/srv.bare" super_clone2 && + test_cmp_config -C super_clone2 true remote.origin.promisor && + test_cmp_config -C super_clone2 blob:none remote.origin.partialclonefilter && + test_cmp_config -C super_clone2/sub true remote.origin.promisor && + test_cmp_config -C super_clone2/sub blob:none remote.origin.partialclonefilter +' + +test_expect_success '--no-also-filter-submodules overrides clone.filterSubmodules=true' ' + test_config_global clone.filterSubmodules true && + git clone --recurse-submodules --filter blob:none \ + --no-also-filter-submodules \ + "file://$pwd/srv.bare" super_clone3 && + test_cmp_config -C super_clone3 true remote.origin.promisor && + test_cmp_config -C super_clone3 blob:none remote.origin.partialclonefilter && + test_cmp_config -C super_clone3/sub false --default false remote.origin.promisor +' + test_done diff --git a/t/t7814-grep-recurse-submodules.sh b/t/t7814-grep-recurse-submodules.sh index 058e5d0c96..a4476dc492 100755 --- a/t/t7814-grep-recurse-submodules.sh +++ b/t/t7814-grep-recurse-submodules.sh @@ -544,4 +544,45 @@ test_expect_failure 'grep saves textconv cache in the appropriate repository' ' test_path_is_file "$sub_textconv_cache" ' +test_expect_success 'grep partially-cloned submodule' ' + # Set up clean superproject and submodule for partial cloning. + git init super && + git init super/sub && + ( + cd super && + test_commit --no-tag "Add file in superproject" \ + super-file "Some content for super-file" && + test_commit -C sub --no-tag "Add file in submodule" \ + sub-file "Some content for sub-file" && + git submodule add ./sub && + git commit -m "Add other as submodule sub" && + test_tick && + test_commit -C sub --no-tag --append "Update file in submodule" \ + sub-file "Some more content for sub-file" && + git add sub && + git commit -m "Update submodule" && + test_tick && + git config --local uploadpack.allowfilter 1 && + git config --local uploadpack.allowanysha1inwant 1 && + git -C sub config --local uploadpack.allowfilter 1 && + git -C sub config --local uploadpack.allowanysha1inwant 1 + ) && + # Clone the superproject & submodule, then make sure we can lazy-fetch submodule objects. + git clone --filter=blob:none --also-filter-submodules \ + --recurse-submodules "file://$(pwd)/super" partial && + ( + cd partial && + cat >expect <<-\EOF && + HEAD^:sub/sub-file:Some content for sub-file + HEAD^:super-file:Some content for super-file + EOF + + GIT_TRACE2_EVENT="$(pwd)/trace2.log" git grep -e content \ + --recurse-submodules HEAD^ >actual && + test_cmp expect actual && + # Verify that we actually fetched data from the promisor remote: + grep \"category\":\"promisor\",\"key\":\"fetch_count\",\"value\":\"1\" trace2.log + ) +' + test_done -- cgit v1.3-5-g9baa From 2df5387ed04159b188de65dff9654d4aae1062d2 Mon Sep 17 00:00:00 2001 From: Junio C Hamano Date: Wed, 9 Feb 2022 18:19:07 -0800 Subject: glossary: describe "worktree" We have description on "per worktree ref", but "worktree" is not described in the glossary. We do have "working tree", though. Casually put, a "working tree" is what your editor and compiler interacts with. "worktree" is a mechanism to allow one or more "working tree"s to be attached to a repository and used to check out different commits and branches independently, which includes not just a "working tree" but also repository metadata like HEAD, the index to support simultaneous use of them. Historically, we used these terms interchangeably but we have been trying to use "working tree" when we mean it, instead of "worktree". Most of the existing references to "working tree" in the glossary do refer primarily to the working tree portion, except for one that said refs like HEAD and refs/bisect/* are per "working tree", but it is more precise to say they are per "worktree". Signed-off-by: Junio C Hamano --- Documentation/glossary-content.txt | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-) (limited to 'Documentation') diff --git a/Documentation/glossary-content.txt b/Documentation/glossary-content.txt index c077971335..aa2f41f5e7 100644 --- a/Documentation/glossary-content.txt +++ b/Documentation/glossary-content.txt @@ -312,7 +312,7 @@ Pathspecs are used on the command line of "git ls-files", "git ls-tree", "git add", "git grep", "git diff", "git checkout", and many other commands to limit the scope of operations to some subset of the tree or -worktree. See the documentation of each command for whether +working tree. See the documentation of each command for whether paths are relative to the current directory or toplevel. The pathspec syntax is as follows: + @@ -446,7 +446,7 @@ exclude;; interface than the <>. [[def_per_worktree_ref]]per-worktree ref:: - Refs that are per-<>, rather than + Refs that are per-<>, rather than global. This is presently only <> and any refs that start with `refs/bisect/`, but might later include other unusual refs. @@ -669,3 +669,12 @@ The most notable example is `HEAD`. The tree of actual checked out files. The working tree normally contains the contents of the <> commit's tree, plus any local changes that you have made but not yet committed. + +[[def_worktree]]worktree:: + A repository can have zero (i.e. bare repository) or one or + more worktrees attached to it. One "worktree" consists of a + "working tree" and repository metadata, most of which are + shared among other worktrees of a single repository, and + some of which are maintained separately per worktree + (e.g. the index, HEAD and pseudorefs like MERGE_HEAD, + per-worktree refs and per-worktree configuration file). -- cgit v1.3-5-g9baa From 6a5678f2576dba579e35c6d86b02584abee0ac37 Mon Sep 17 00:00:00 2001 From: "brian m. carlson" Date: Mon, 14 Feb 2022 10:15:43 -0800 Subject: doc: clarify interaction between 'eol' and text=auto The `eol` takes effect on text files only when the index has the contents in LF line endings. Paths with contents in CRLF line endings in the index may become dirty unless text=auto. Signed-off-by: brian m. carlson Signed-off-by: Junio C Hamano --- Documentation/gitattributes.txt | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) (limited to 'Documentation') diff --git a/Documentation/gitattributes.txt b/Documentation/gitattributes.txt index 60984a4682..a71dad2674 100644 --- a/Documentation/gitattributes.txt +++ b/Documentation/gitattributes.txt @@ -161,11 +161,12 @@ unspecified. This attribute sets a specific line-ending style to be used in the working directory. This attribute has effect only if the `text` -attribute is set or unspecified, or if it is set to `auto` and the file -is detected as text. Note that setting this attribute on paths which -are in the index with CRLF line endings may make the paths to be -considered dirty. Adding the path to the index again will normalize the -line endings in the index. +attribute is set or unspecified, or if it is set to `auto`, the file is +detected as text, and it is stored with LF endings in the index. Note +that setting this attribute on paths which are in the index with CRLF +line endings may make the paths to be considered dirty unless +`text=auto` is set. Adding the path to the index again will normalize +the line endings in the index. Set to string value "crlf":: -- cgit v1.3-5-g9baa From e2ac9141e64e2cd3e690d1b5fc848949827c09b4 Mon Sep 17 00:00:00 2001 From: Junio C Hamano Date: Thu, 17 Feb 2022 16:24:23 -0800 Subject: The fifth batch Signed-off-by: Junio C Hamano --- Documentation/RelNotes/2.36.0.txt | 24 ++++++++++++++++++++++++ 1 file changed, 24 insertions(+) (limited to 'Documentation') diff --git a/Documentation/RelNotes/2.36.0.txt b/Documentation/RelNotes/2.36.0.txt index 3dfa5e409f..3e1261c584 100644 --- a/Documentation/RelNotes/2.36.0.txt +++ b/Documentation/RelNotes/2.36.0.txt @@ -25,6 +25,15 @@ UI, Workflows & Features * "git log --remerge-diff" shows the difference from mechanical merge result and the result that is actually recorded in a merge commit. + * "git log" and friends learned an option --exclude-first-parent-only + to propagate UNINTERESTING bit down only along the first-parent + chain, just like --first-parent option shows commits that lack the + UNINTERESTING bit only along the first-parent chain. + + * The command line completion script (in contrib/) learned to + complete all Git subcommands, including the ones that are normally + hidden, when GIT_COMPLETION_SHOW_ALL_COMMANDS is used. + Performance, Internal Implementation, Development Support etc. @@ -47,6 +56,9 @@ Performance, Internal Implementation, Development Support etc. all. Start the process of renaming it to "--annotate-stdin". (merge a2585719b3 jc/name-rev-stdin later to maint). + * "git update-index", "git checkout-index", and "git clean" are + taught to work better with the sparse checkout feature. + Fixes since v2.35 ----------------- @@ -140,6 +152,16 @@ Fixes since v2.35 * "git diff --diff-filter=aR" is now parsed correctly. (merge 75408ca949 js/diff-filter-negation-fix later to maint). + * When "git subtree" wants to create a merge, it used "git merge" and + let it be affected by end-user's "merge.ff" configuration, which + has been corrected. + (merge 9158a3564a tk/subtree-merge-not-ff-only later to maint). + + * Unlike "git apply", "git patch-id" did not handle patches with + hunks that has only 1 line in either preimage or postimage, which + has been corrected. + (merge 757e75c81e jz/patch-id-hunk-header-parsing-fix later to maint). + * Other code cleanup, docfix, build fix, etc. (merge cfc5cf428b jc/find-header later to maint). (merge 40e7cfdd46 jh/p4-fix-use-of-process-error-exception later to maint). @@ -157,3 +179,5 @@ Fixes since v2.35 (merge 45d0212a71 ll/doc-mktree-typofix later to maint). (merge e9b272e4c1 js/no-more-legacy-stash later to maint). (merge 6798b08e84 ab/do-not-hide-failures-in-git-dot-pm later to maint). + (merge 9325285df4 po/doc-check-ignore-markup-fix later to maint). + (merge cd26cd6c7c sy/modernize-t-lib-read-tree-m-3way later to maint). -- cgit v1.3-5-g9baa From e6ebfd0e8cbbd10878070c8a356b5ad1b3ca464e Mon Sep 17 00:00:00 2001 From: Junio C Hamano Date: Fri, 18 Feb 2022 09:53:31 -0800 Subject: The sixth batch Signed-off-by: Junio C Hamano --- Documentation/RelNotes/2.36.0.txt | 34 ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+) (limited to 'Documentation') diff --git a/Documentation/RelNotes/2.36.0.txt b/Documentation/RelNotes/2.36.0.txt index 3e1261c584..4e8309701b 100644 --- a/Documentation/RelNotes/2.36.0.txt +++ b/Documentation/RelNotes/2.36.0.txt @@ -34,6 +34,8 @@ UI, Workflows & Features complete all Git subcommands, including the ones that are normally hidden, when GIT_COMPLETION_SHOW_ALL_COMMANDS is used. + * "git branch" learned the "--recurse-submodules" option. + Performance, Internal Implementation, Development Support etc. @@ -59,6 +61,10 @@ Performance, Internal Implementation, Development Support etc. * "git update-index", "git checkout-index", and "git clean" are taught to work better with the sparse checkout feature. + * Use an internal call to reset_head() helper function instead of + spawning "git checkout" in "rebase", and update code paths that are + involved in the change. + Fixes since v2.35 ----------------- @@ -162,6 +168,32 @@ Fixes since v2.35 has been corrected. (merge 757e75c81e jz/patch-id-hunk-header-parsing-fix later to maint). + * "receive-pack" checks if it will do any ref updates (various + conditions could reject a push) before received objects are taken + out of the temporary directory used for quarantine purposes, so + that a push that is known-to-fail will not leave crufts that a + future "gc" needs to clean up. + (merge 5407764069 cb/clear-quarantine-early-on-all-ref-update-errors later to maint). + + * Because a deletion of ref would need to remove it from both the + loose ref store and the packed ref store, a delete-ref operation + that logically removes one ref may end up invoking ref-transaction + hook twice, which has been corrected. + (merge 2ed1b64ebd ps/avoid-unnecessary-hook-invocation-with-packed-refs later to maint). + + * When there is no object to write .bitmap file for, "git + multi-pack-index" triggered an error, instead of just skipping, + which has been corrected. + (merge eb57277ba3 tb/midx-no-bitmap-for-no-objects later to maint). + + * "git cmd -h" outside a repository should error out cleanly for many + commands, but instead it hit a BUG(), which has been corrected. + (merge 87ad07d735 js/short-help-outside-repo-fix later to maint). + + * "working tree" and "per-worktree ref" were in glossary, but + "worktree" itself wasn't, which has been corrected. + (merge 2df5387ed0 jc/glossary-worktree later to maint). + * Other code cleanup, docfix, build fix, etc. (merge cfc5cf428b jc/find-header later to maint). (merge 40e7cfdd46 jh/p4-fix-use-of-process-error-exception later to maint). @@ -181,3 +213,5 @@ Fixes since v2.35 (merge 6798b08e84 ab/do-not-hide-failures-in-git-dot-pm later to maint). (merge 9325285df4 po/doc-check-ignore-markup-fix later to maint). (merge cd26cd6c7c sy/modernize-t-lib-read-tree-m-3way later to maint). + (merge d17294a05e ab/hash-object-leakfix later to maint). + (merge b8403129d3 jd/t0015-modernize later to maint). -- cgit v1.3-5-g9baa From dab1b7905d0b295f1acef9785bb2b9cbb0fdec84 Mon Sep 17 00:00:00 2001 From: Junio C Hamano Date: Wed, 23 Feb 2022 16:58:13 -0800 Subject: The seventh batch Signed-off-by: Junio C Hamano --- Documentation/RelNotes/2.36.0.txt | 12 ++++++++++++ 1 file changed, 12 insertions(+) (limited to 'Documentation') diff --git a/Documentation/RelNotes/2.36.0.txt b/Documentation/RelNotes/2.36.0.txt index 4e8309701b..78e037b865 100644 --- a/Documentation/RelNotes/2.36.0.txt +++ b/Documentation/RelNotes/2.36.0.txt @@ -194,6 +194,17 @@ Fixes since v2.35 "worktree" itself wasn't, which has been corrected. (merge 2df5387ed0 jc/glossary-worktree later to maint). + * L10n support for a few error messages. + (merge 3d3c23b3a7 bs/forbid-i18n-of-protocol-token-in-fetch-pack later to maint). + + * Test modernization. + (merge d4fe066e4b sy/t0001-use-path-is-helper later to maint). + + * "git log --graph --graph" used to leak a graph structure, and there + was no way to countermand "--graph" that appear earlier on the + command line. A "--no-graph" option has been added and resource + leakage has been plugged. + * Other code cleanup, docfix, build fix, etc. (merge cfc5cf428b jc/find-header later to maint). (merge 40e7cfdd46 jh/p4-fix-use-of-process-error-exception later to maint). @@ -215,3 +226,4 @@ Fixes since v2.35 (merge cd26cd6c7c sy/modernize-t-lib-read-tree-m-3way later to maint). (merge d17294a05e ab/hash-object-leakfix later to maint). (merge b8403129d3 jd/t0015-modernize later to maint). + (merge 332acc248d ds/mailmap later to maint). -- cgit v1.3-5-g9baa From 715d08a9e51251ad8290b181b6ac3b9e1f9719d7 Mon Sep 17 00:00:00 2001 From: Junio C Hamano Date: Fri, 25 Feb 2022 15:47:20 -0800 Subject: The eighth batch Signed-off-by: Junio C Hamano --- Documentation/RelNotes/2.36.0.txt | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) (limited to 'Documentation') diff --git a/Documentation/RelNotes/2.36.0.txt b/Documentation/RelNotes/2.36.0.txt index 78e037b865..de1e11e25a 100644 --- a/Documentation/RelNotes/2.36.0.txt +++ b/Documentation/RelNotes/2.36.0.txt @@ -9,6 +9,10 @@ Backward compatibility warts * "git name-rev --stdin" has been deprecated and issues a warning when used; use "git name-rev --annotate-stdin" instead. + * "git clone --filter=... --recurse-submodules" only makes the + top-level a partial clone, while submodules are fully cloned. This + behaviour is changed to pass the same filter down to the submodules. + Note to those who build from the source @@ -65,6 +69,10 @@ Performance, Internal Implementation, Development Support etc. spawning "git checkout" in "rebase", and update code paths that are involved in the change. + * Messages "ort" merge backend prepares while dealing with conflicted + paths were unnecessarily confusing since it did not differentiate + inner merges and outer merges. + Fixes since v2.35 ----------------- @@ -205,6 +213,25 @@ Fixes since v2.35 command line. A "--no-graph" option has been added and resource leakage has been plugged. + * Error output given in response to an ambiguous object name has been + improved. + (merge 3a73c1dfaf ab/ambiguous-object-name later to maint). + + * "git sparse-checkout" wants to work with per-worktree configuration, + but did not work well in a worktree attached to a bare repository. + (merge 3ce1138272 ds/sparse-checkout-requires-per-worktree-config later to maint). + + * Setting core.untrackedCache to true failed to add the untracked + cache extension to the index. + + * Workaround we have for versions of PCRE2 before their version 10.36 + were in effect only for their versions newer than 10.36 by mistake, + which has been corrected. + (merge 97169fc361 rs/pcre-invalid-utf8-fix-fix later to maint). + + * Document Taylor as a new member of Git PLC at SFC. Welcome. + (merge e8d56ca863 tb/coc-plc-update later to maint). + * Other code cleanup, docfix, build fix, etc. (merge cfc5cf428b jc/find-header later to maint). (merge 40e7cfdd46 jh/p4-fix-use-of-process-error-exception later to maint). @@ -227,3 +254,5 @@ Fixes since v2.35 (merge d17294a05e ab/hash-object-leakfix later to maint). (merge b8403129d3 jd/t0015-modernize later to maint). (merge 332acc248d ds/mailmap later to maint). + (merge 04bf052eef ab/grep-patterntype later to maint). + (merge 6ee36364eb ab/diff-free-more later to maint). -- cgit v1.3-5-g9baa From 3248486920d0bfc5584747dc7af8414d05282191 Mon Sep 17 00:00:00 2001 From: Jeff Hostetler Date: Fri, 25 Mar 2022 18:02:48 +0000 Subject: fsmonitor: document builtin fsmonitor Document how `core.fsmonitor` can be set to a boolean to enable or disable the builtin FSMonitor. Update references to `core.fsmonitor` and `core.fsmonitorHookVersion` and pointers to `Watchman` to refer to it. Create `git-fsmonitor--daemon` manual page and describe its features. Signed-off-by: Jeff Hostetler Signed-off-by: Junio C Hamano --- Documentation/config/core.txt | 60 ++++++++++++++++++++------ Documentation/git-fsmonitor--daemon.txt | 75 +++++++++++++++++++++++++++++++++ Documentation/git-update-index.txt | 8 ++-- 3 files changed, 126 insertions(+), 17 deletions(-) create mode 100644 Documentation/git-fsmonitor--daemon.txt (limited to 'Documentation') diff --git a/Documentation/config/core.txt b/Documentation/config/core.txt index c04f62a54a..6303c36c7e 100644 --- a/Documentation/config/core.txt +++ b/Documentation/config/core.txt @@ -62,22 +62,54 @@ core.protectNTFS:: Defaults to `true` on Windows, and `false` elsewhere. core.fsmonitor:: - If set, the value of this variable is used as a command which - will identify all files that may have changed since the - requested date/time. This information is used to speed up git by - avoiding unnecessary processing of files that have not changed. - See the "fsmonitor-watchman" section of linkgit:githooks[5]. + If set to true, enable the built-in file system monitor + daemon for this working directory (linkgit:git-fsmonitor--daemon[1]). ++ +Like hook-based file system monitors, the built-in file system monitor +can speed up Git commands that need to refresh the Git index +(e.g. `git status`) in a working directory with many files. The +built-in monitor eliminates the need to install and maintain an +external third-party tool. ++ +The built-in file system monitor is currently available only on a +limited set of supported platforms. Currently, this includes Windows +and MacOS. ++ + Otherwise, this variable contains the pathname of the "fsmonitor" + hook command. ++ +This hook command is used to identify all files that may have changed +since the requested date/time. This information is used to speed up +git by avoiding unnecessary scanning of files that have not changed. ++ +See the "fsmonitor-watchman" section of linkgit:githooks[5]. ++ +Note that if you concurrently use multiple versions of Git, such +as one version on the command line and another version in an IDE +tool, that the definition of `core.fsmonitor` was extended to +allow boolean values in addition to hook pathnames. Git versions +2.35.1 and prior will not understand the boolean values and will +consider the "true" or "false" values as hook pathnames to be +invoked. Git versions 2.26 thru 2.35.1 default to hook protocol +V2 and will fall back to no fsmonitor (full scan). Git versions +prior to 2.26 default to hook protocol V1 and will silently +assume there were no changes to report (no scan), so status +commands may report incomplete results. For this reason, it is +best to upgrade all of your Git versions before using the built-in +file system monitor. core.fsmonitorHookVersion:: - Sets the version of hook that is to be used when calling fsmonitor. - There are currently versions 1 and 2. When this is not set, - version 2 will be tried first and if it fails then version 1 - will be tried. Version 1 uses a timestamp as input to determine - which files have changes since that time but some monitors - like watchman have race conditions when used with a timestamp. - Version 2 uses an opaque string so that the monitor can return - something that can be used to determine what files have changed - without race conditions. + Sets the protocol version to be used when invoking the + "fsmonitor" hook. ++ +There are currently versions 1 and 2. When this is not set, +version 2 will be tried first and if it fails then version 1 +will be tried. Version 1 uses a timestamp as input to determine +which files have changes since that time but some monitors +like Watchman have race conditions when used with a timestamp. +Version 2 uses an opaque string so that the monitor can return +something that can be used to determine what files have changed +without race conditions. core.trustctime:: If false, the ctime differences between the index and the diff --git a/Documentation/git-fsmonitor--daemon.txt b/Documentation/git-fsmonitor--daemon.txt new file mode 100644 index 0000000000..0fedf5a456 --- /dev/null +++ b/Documentation/git-fsmonitor--daemon.txt @@ -0,0 +1,75 @@ +git-fsmonitor--daemon(1) +======================== + +NAME +---- +git-fsmonitor--daemon - A Built-in File System Monitor + +SYNOPSIS +-------- +[verse] +'git fsmonitor--daemon' start +'git fsmonitor--daemon' run +'git fsmonitor--daemon' stop +'git fsmonitor--daemon' status + +DESCRIPTION +----------- + +A daemon to watch the working directory for file and directory +changes using platform-specific file system notification facilities. + +This daemon communicates directly with commands like `git status` +using the link:technical/api-simple-ipc.html[simple IPC] interface +instead of the slower linkgit:githooks[5] interface. + +This daemon is built into Git so that no third-party tools are +required. + +OPTIONS +------- + +start:: + Starts a daemon in the background. + +run:: + Runs a daemon in the foreground. + +stop:: + Stops the daemon running in the current working + directory, if present. + +status:: + Exits with zero status if a daemon is watching the + current working directory. + +REMARKS +------- + +This daemon is a long running process used to watch a single working +directory and maintain a list of the recently changed files and +directories. Performance of commands such as `git status` can be +increased if they just ask for a summary of changes to the working +directory and can avoid scanning the disk. + +When `core.fsmonitor` is set to `true` (see linkgit:git-config[1]) +commands, such as `git status`, will ask the daemon for changes and +automatically start it (if necessary). + +For more information see the "File System Monitor" section in +linkgit:git-update-index[1]. + +CAVEATS +------- + +The fsmonitor daemon does not currently know about submodules and does +not know to filter out file system events that happen within a +submodule. If fsmonitor daemon is watching a super repo and a file is +modified within the working directory of a submodule, it will report +the change (as happening against the super repo). However, the client +will properly ignore these extra events, so performance may be affected +but it will not cause an incorrect result. + +GIT +--- +Part of the linkgit:git[1] suite diff --git a/Documentation/git-update-index.txt b/Documentation/git-update-index.txt index 2853f168d9..53ea48a04e 100644 --- a/Documentation/git-update-index.txt +++ b/Documentation/git-update-index.txt @@ -498,7 +498,9 @@ FILE SYSTEM MONITOR This feature is intended to speed up git operations for repos that have large working directories. -It enables git to work together with a file system monitor (see the +It enables git to work together with a file system monitor (see +linkgit:git-fsmonitor--daemon[1] +and the "fsmonitor-watchman" section of linkgit:githooks[5]) that can inform it as to what files have been modified. This enables git to avoid having to lstat() every file to find modified files. @@ -509,8 +511,8 @@ looking for new files. If you want to enable (or disable) this feature, it is easier to use the `core.fsmonitor` configuration variable (see -linkgit:git-config[1]) than using the `--fsmonitor` option to -`git update-index` in each repository, especially if you want to do so +linkgit:git-config[1]) than using the `--fsmonitor` option to `git +update-index` in each repository, especially if you want to do so across all repositories you use, because you can set the configuration variable in your `$HOME/.gitconfig` just once and have it affect all repositories you touch. -- cgit v1.3-5-g9baa