diff options
| -rw-r--r-- | Documentation/fetch-options.adoc | 19 | ||||
| -rw-r--r-- | Documentation/git-clone.adoc | 25 | ||||
| -rw-r--r-- | Documentation/gitprotocol-v2.adoc | 16 | ||||
| -rw-r--r-- | builtin/clone.c | 2 | ||||
| -rw-r--r-- | builtin/fetch.c | 2 | ||||
| -rw-r--r-- | fetch-pack.c | 24 | ||||
| -rwxr-xr-x | t/t5710-promisor-remote-capability.sh | 60 | ||||
| -rw-r--r-- | transport.c | 1 |
8 files changed, 134 insertions, 15 deletions
diff --git a/Documentation/fetch-options.adoc b/Documentation/fetch-options.adoc index 1ef9807d00..a0cfb50d89 100644 --- a/Documentation/fetch-options.adoc +++ b/Documentation/fetch-options.adoc @@ -92,11 +92,20 @@ precedence over the `fetch.output` config option. Use the partial clone feature and request that the server sends a subset of reachable objects according to a given object filter. When using `--filter`, the supplied _<filter-spec>_ is used for - the partial fetch. For example, `--filter=blob:none` will filter - out all blobs (file contents) until needed by Git. Also, - `--filter=blob:limit=<size>` will filter out all blobs of size - at least _<size>_. For more details on filter specifications, see - the `--filter` option in linkgit:git-rev-list[1]. + the partial fetch. ++ +If `--filter=auto` is used, the filter specification is determined +automatically by combining the filter specifications advertised by +the server for the promisor remotes that the client accepts (see +linkgit:gitprotocol-v2[5] and the `promisor.acceptFromServer` +configuration option in linkgit:git-config[1]). ++ +For details on all other available filter specifications, see the +`--filter=<filter-spec>` option in linkgit:git-rev-list[1]. ++ +For example, `--filter=blob:none` will filter out all blobs (file +contents) until needed by Git. Also, `--filter=blob:limit=<size>` will +filter out all blobs of size at least _<size>_. ifndef::git-pull[] `--write-fetch-head`:: diff --git a/Documentation/git-clone.adoc b/Documentation/git-clone.adoc index 57cdfb7620..0db2d1e5f0 100644 --- a/Documentation/git-clone.adoc +++ b/Documentation/git-clone.adoc @@ -187,11 +187,26 @@ objects from the source repository into a pack in the cloned repository. Use the partial clone feature and request that the server sends a subset of reachable objects according to a given object filter. When using `--filter`, the supplied _<filter-spec>_ is used for - the partial clone filter. For example, `--filter=blob:none` will - filter out all blobs (file contents) until needed by Git. Also, - `--filter=blob:limit=<size>` will filter out all blobs of size - at least _<size>_. For more details on filter specifications, see - the `--filter` option in linkgit:git-rev-list[1]. + the partial clone filter. ++ +If `--filter=auto` is used the filter specification is determined +automatically through the 'promisor-remote' protocol (see +linkgit:gitprotocol-v2[5]) by combining the filter specifications +advertised by the server for the promisor remotes that the client +accepts (see the `promisor.acceptFromServer` configuration option in +linkgit:git-config[1]). This allows the server to suggest the optimal +filter for the available promisor remotes. ++ +As with other filter specifications, the "auto" value is persisted in +the configuration. This ensures that future fetches will continue to +adapt to the server's current recommendation. ++ +For details on all other available filter specifications, see the +`--filter=<filter-spec>` option in linkgit:git-rev-list[1]. ++ +For example, `--filter=blob:none` will filter out all blobs (file +contents) until needed by Git. Also, `--filter=blob:limit=<size>` will +filter out all blobs of size at least _<size>_. `--also-filter-submodules`:: Also apply the partial clone filter to any submodules in the repository. diff --git a/Documentation/gitprotocol-v2.adoc b/Documentation/gitprotocol-v2.adoc index d93dd279ea..f985cb4c47 100644 --- a/Documentation/gitprotocol-v2.adoc +++ b/Documentation/gitprotocol-v2.adoc @@ -812,10 +812,15 @@ MUST appear first in each pr-fields, in that order. After these mandatory fields, the server MAY advertise the following optional fields in any order: -`partialCloneFilter`:: The filter specification used by the remote. +`partialCloneFilter`:: The filter specification for the remote. It +corresponds to the "remote.<name>.partialCloneFilter" config setting. Clients can use this to determine if the remote's filtering strategy -is compatible with their needs (e.g., checking if both use "blob:none"). -It corresponds to the "remote.<name>.partialCloneFilter" config setting. +is compatible with their needs (e.g., checking if both use +"blob:none"). Additionally they can use this through the +`--filter=auto` option in linkgit:git-clone[1]. With that option, the +filter specification of the clone will be automatically computed by +combining the filter specifications of the promisor remotes the client +accepts. `token`:: An authentication token that clients can use when connecting to the remote. It corresponds to the "remote.<name>.token" @@ -828,8 +833,9 @@ future protocol extensions. The client can use information transmitted through these fields to decide if it accepts the advertised promisor remote. Also, the client -can be configured to store the values of these fields (see -"promisor.storeFields" in linkgit:git-config[1]). +can be configured to store the values of these fields or use them +to automatically configure the repository (see "promisor.storeFields" +in linkgit:git-config[1] and `--filter=auto` in linkgit:git-clone[1]). Field values MUST be urlencoded. diff --git a/builtin/clone.c b/builtin/clone.c index bb27472020..45d8fa0eed 100644 --- a/builtin/clone.c +++ b/builtin/clone.c @@ -1001,6 +1001,8 @@ int cmd_clone(int argc, NULL }; + filter_options.allow_auto_filter = 1; + packet_trace_identity("clone"); repo_config(the_repository, git_clone_config, NULL); diff --git a/builtin/fetch.c b/builtin/fetch.c index 8fbf3557ce..573c295241 100644 --- a/builtin/fetch.c +++ b/builtin/fetch.c @@ -2580,6 +2580,8 @@ int cmd_fetch(int argc, OPT_END() }; + filter_options.allow_auto_filter = 1; + packet_trace_identity("fetch"); /* Record the command line for the reflog */ diff --git a/fetch-pack.c b/fetch-pack.c index 40316c9a34..9f8f980516 100644 --- a/fetch-pack.c +++ b/fetch-pack.c @@ -35,6 +35,7 @@ #include "sigchain.h" #include "mergesort.h" #include "prio-queue.h" +#include "promisor-remote.h" static int transfer_unpack_limit = -1; static int fetch_unpack_limit = -1; @@ -1661,6 +1662,29 @@ static struct ref *do_fetch_pack_v2(struct fetch_pack_args *args, struct string_list packfile_uris = STRING_LIST_INIT_DUP; int i; struct strvec index_pack_args = STRVEC_INIT; + const char *promisor_remote_config; + + if (server_feature_v2("promisor-remote", &promisor_remote_config)) + promisor_remote_reply(promisor_remote_config, NULL); + + if (args->filter_options.choice == LOFC_AUTO) { + struct strbuf errbuf = STRBUF_INIT; + char *constructed_filter = promisor_remote_construct_filter(r); + + list_objects_filter_release(&args->filter_options); + /* Disallow 'auto' as a result of the resolution of this 'auto' filter below */ + args->filter_options.allow_auto_filter = 0; + + if (constructed_filter && + gently_parse_list_objects_filter(&args->filter_options, + constructed_filter, + &errbuf)) + die(_("couldn't resolve 'auto' filter '%s': %s"), + constructed_filter, errbuf.buf); + + free(constructed_filter); + strbuf_release(&errbuf); + } negotiator = &negotiator_alloc; if (args->refetch) diff --git a/t/t5710-promisor-remote-capability.sh b/t/t5710-promisor-remote-capability.sh index 6ef6431bd7..532e6f0fea 100755 --- a/t/t5710-promisor-remote-capability.sh +++ b/t/t5710-promisor-remote-capability.sh @@ -423,6 +423,66 @@ test_expect_success "clone with promisor.storeFields=partialCloneFilter" ' test_grep "'\''blob:limit=8k'\'' -> '\''blob:limit=7k'\''" err ' +test_expect_success "clone and fetch with --filter=auto" ' + git -C server config promisor.advertise true && + test_when_finished "rm -rf client trace" && + + git -C server config remote.lop.partialCloneFilter "blob:limit=9500" && + test_config -C server promisor.sendFields "partialCloneFilter" && + + GIT_TRACE_PACKET="$(pwd)/trace" GIT_NO_LAZY_FETCH=0 git clone \ + -c remote.lop.promisor=true \ + -c remote.lop.url="file://$(pwd)/lop" \ + -c promisor.acceptfromserver=All \ + --no-local --filter=auto server client 2>err && + + test_grep "filter blob:limit=9500" trace && + test_grep ! "filter auto" trace && + + # Verify "auto" is persisted in config + echo auto >expected && + git -C client config remote.origin.partialCloneFilter >actual && + test_cmp expected actual && + + # Check that the largest object is still missing on the server + check_missing_objects server 1 "$oid" && + + # Now change the filter on the server + git -C server config remote.lop.partialCloneFilter "blob:limit=5678" && + + # Get a new commit on the server to ensure "git fetch" actually runs fetch-pack + test_commit -C template new-commit && + git -C template push --all "$(pwd)/server" && + + # Perform a fetch WITH --filter=auto + rm -rf trace && + GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch --filter=auto && + + # Verify that the new filter was used + test_grep "filter blob:limit=5678" trace && + + # Check that the largest object is still missing on the server + check_missing_objects server 1 "$oid" && + + # Change the filter on the server again + git -C server config remote.lop.partialCloneFilter "blob:limit=5432" && + + # Get yet a new commit on the server to ensure fetch-pack runs + test_commit -C template yet-a-new-commit && + git -C template push --all "$(pwd)/server" && + + # Perform a fetch WITHOUT --filter=auto + # Relies on "auto" being persisted in the client config + rm -rf trace && + GIT_TRACE_PACKET="$(pwd)/trace" git -C client fetch && + + # Verify that the new filter was used + test_grep "filter blob:limit=5432" trace && + + # Check that the largest object is still missing on the server + check_missing_objects server 1 "$oid" +' + test_expect_success "clone with promisor.advertise set to 'true' but don't delete the client" ' git -C server config promisor.advertise true && diff --git a/transport.c b/transport.c index c7f06a7382..cde8d83a57 100644 --- a/transport.c +++ b/transport.c @@ -1219,6 +1219,7 @@ struct transport *transport_get(struct remote *remote, const char *url) */ struct git_transport_data *data = xcalloc(1, sizeof(*data)); list_objects_filter_init(&data->options.filter_options); + data->options.filter_options.allow_auto_filter = 1; ret->data = data; ret->vtable = &builtin_smart_vtable; ret->smart_options = &(data->options); |