aboutsummaryrefslogtreecommitdiffstats
AgeCommit message (Collapse)AuthorFilesLines
2021-05-07Merge branch 'jk/prune-with-bitmap-fix'Junio C Hamano4-15/+39
When the reachability bitmap is in effect, the "do not lose recently created objects and those that are reachable from them" safety to protect us from races were disabled by mistake, which has been corrected. * jk/prune-with-bitmap-fix: prune: save reachable-from-recent objects with bitmaps pack-bitmap: clean up include_check after use
2021-05-07Merge branch 'po/diff-patch-doc'Junio C Hamano1-1/+6
Doc update. * po/diff-patch-doc: doc: point to diff attribute in patch format docs
2021-05-07Merge branch 'hn/trace-reflog-expiry'Junio C Hamano1-3/+44
The reflog expiry machinery has been taught to emit trace events. * hn/trace-reflog-expiry: refs/debug: trace into reflog expiry too
2021-05-07Merge branch 'ab/pretty-date-format-tests'Junio C Hamano1-5/+4
Tweak a few tests for "log --format=..." that show timestamps in various formats. * ab/pretty-date-format-tests: pretty tests: give --date/format tests a better description pretty tests: simplify %aI/%cI date format test
2021-05-07Merge branch 'ps/config-env-option-with-separate-value'Junio C Hamano3-2/+23
"git --config-env var=val cmd" weren't accepted (only --config-env=var=val was). * ps/config-env-option-with-separate-value: git: support separate arg for `--config-env`'s value git.txt: fix synopsis of `--config-env` missing the equals sign
2021-05-07clean: remove unnecessary variableMatheus Tavares1-3/+1
The variable `matches` used to hold the return of a `dir_path_match()` call that was removed in 95c11ecc73 ("Fix error-prone fill_directory() API; make it only return matches", 2020-04-01). Now `matches` will always hold 0, which is the value it's initialized with; and the condition `matches != MATCHED_EXACTLY` will always evaluate to true. So let's remove this unnecessary variable. Interestingly, it seems that `matches != MATCHED_EXACTLY` was already unnecessary before 95c11ecc73. That's because `remove_directories` is always set to 1 when we have pathspecs; So, in the condition `!remove_directories && matches != MATCHED_EXACTLY`, we would either: - have pathspecs (or have been given `-d`) and ignore `matches` because `remove_directories` is 1; or - not have pathspecs (nor `-d`) and end up just checking that `0 != MATCHED_EXACTLY`, as `matches` would never get reassigned after its zero initialization (because there is no pathspec to match). Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-07mailinfo: stop parsing options manuallyĐoàn Trần Công Danh1-30/+45
In a later change, mailinfo will learn more options, let's switch to our robust parse_options framework before that step. Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-07mailinfo: load default metainfo_charset lazilyĐoàn Trần Công Danh1-9/+31
In a later change, we will use parse_option to parse mailinfo's options. In mailinfo, both "-u", "-n", and "--encoding" try to set the same field, with "-u" reset that field to some default value from configuration variable "i18n.commitEncoding". Let's delay the setting of that field until we finish processing all options. By doing that, "i18n.commitEncoding" can be parsed on demand. More importantly, it cleans the way for using parse_option. This change introduces some inconsistent brackets "{}" in "if/else if" construct, however, we will rewrite them in the next few changes. Signed-off-by: Đoàn Trần Công Danh <congdanhqx@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-07add: die if both --dry-run and --interactive are givenØystein Walle2-0/+6
The interactive machinery does not obey --dry-run. Die appropriately if both flags are passed. Signed-off-by: Øystein Walle <oystwa@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-06perl: use mock i18n functions under NO_GETTEXT=YÆvar Arnfjörð Bjarmason2-0/+13
Change the logic of the i18n functions I added in 5e9637c6297 (i18n: add infrastructure for translating Git with gettext, 2011-11-18) to use pass-through functions when NO_GETTEXT is defined. This speeds up the compilation time of commands that use this library when NO_GETTEXT=Y is in effect. Loading it and POSIX.pm is around 20ms on my machine, whereas it takes 2ms to just instantiate perl itself. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-06Makefile: regenerate *.pm on NO_PERL_CPAN_FALLBACKS changeÆvar Arnfjörð Bjarmason1-0/+1
Regenerate the *.pm files in perl/build/* if the NO_PERL_CPAN_FALLBACKS flag added to the *.pm files in 1aca69c0195 (perl Git::LoadCPAN: emit better errors under NO_PERL_CPAN_FALLBACKS, 2018-03-03) is changed. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-06Makefile: regenerate perl/build/* if GIT-PERL-DEFINES changesÆvar Arnfjörð Bjarmason1-1/+1
Change the logic to generate perl/build/* to regenerate those files if GIT-PERL-DEFINES changes. This ensures that e.g. changing localedir will result in correctly re-generated files. I don't think that ever worked. The brokenness pre-dates my 20d2a30f8ff (Makefile: replace perl/Makefile.PL with simple make rules, 2017-12-10). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-06Makefile: don't re-define PERL_DEFINESÆvar Arnfjörð Bjarmason1-3/+4
Since 07d90eadb50 (Makefile: add Perl runtime prefix support, 2018-04-10) we have been declaring PERL_DEFINES right after assigning to it, with the effect that the first PERL_DEFINES was ignored. That bug didn't matter in practice since the first line had all the same variables as the second, so we'd correctly re-generate everything. It just made for confusing reading. Let's remove that first assignment, and while we're at it split these across lines to make them more maintainable. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-06streaming.c: move {open,close,read} from vtable to "struct git_istream"Ævar Arnfjörð Bjarmason1-43/+29
Move the definition of the structure around the open/close/read functions introduced in 46bf043807c (streaming: a new API to read from the object store, 2011-05-11) to instead populate "close" and "read" members in the "struct git_istream". This gets us rid of an extra pointer deference, and I think makes more sense. The "close" and "read" functions are the primary interface to the stream itself. Let's also populate a "open" callback in the same struct. That's now used by open_istream() after istream_source() decides what "open" function should be used. This isn't needed to get rid of the "stream_vtbl" variables, but makes sense for consistency. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-06streaming.c: stop passing around "object_info *" to open()Ævar Arnfjörð Bjarmason1-22/+20
Change the streaming interface to stop passing around the "struct object_info" the open() functions. As seen in 7ef2d9a2604 (streaming: read non-delta incrementally from a pack, 2011-05-13) which introduced the "st->u.in_pack" assignments being changed here only the open_istream_pack_non_delta() path need these. So let's instead do this when preparing the selected callback in the istream_source() function. This might also allow the compiler to reduce the lifetime of the "oi" variable, as we've moved it from "git_istream()" to "istream_source()". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-06streaming.c: remove {open,close,read}_method_decl() macrosÆvar Arnfjörð Bjarmason1-25/+22
Remove the {open,close,read}_method_decl() macros added in 46bf043807c (streaming: a new API to read from the object store, 2011-05-11) in favor of inlining the definition of the arguments of these functions. Since we'll end up using them via the "{open,close,read}_istream_fn" types we don't gain anything in the way of compiler checking by using these macros, and as of preceding commits we no longer need to declare these argument lists twice. So declaring them at a distance just serves to make the code less readable. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-06streaming.c: remove enum/function/vtbl indirectionÆvar Arnfjörð Bjarmason1-25/+11
Remove the indirection of discovering a function pointer to use via an enum and virtual table. This refactors code added in 46bf043807c (streaming: a new API to read from the object store, 2011-05-11). We can instead simply return an "open_istream_fn" for use from the "istream_source()" selector function directly. This allows us to get rid of the "incore", "loose" and "pack_non_delta" enum variables. We'll return the functions instead. The "stream_error" variable in that enum can likewise go in favor of returning NULL, which is what the open_istream() was doing when it got that value anyway. We can thus remove the entire enum, and the "open_istream_tbl" virtual table that (indirectly) referenced it. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-06streaming.c: avoid forward declarationsÆvar Arnfjörð Bjarmason1-88/+83
Change code added in 46bf043807c (streaming: a new API to read from the object store, 2011-05-11) to avoid forward declarations of the functions it uses. We can instead move this code to the bottom of the file, and thus avoid the open_method_decl() calls. Aside from the addition of the "static helpers[...]" comment being added here, and the removal of the forward declarations this is a move-only change. The style of the added "static helpers[...]" comment isn't in line with our usual coding style, but is consistent with several other comments used in this file, so let's use that style consistently here. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-06sparse-index.c: remove set_index_sparse_config()Ævar Arnfjörð Bjarmason1-10/+1
Remove the set_index_sparse_config() function by folding it into set_sparse_index_config(), which was its only user. Since 122ba1f7b52 (sparse-checkout: toggle sparse index from builtin, 2021-03-30) the flow of this code hasn't made much sense, we'd get "enabled" in set_sparse_index_config(), proceed to call set_index_sparse_config() with it. There we'd call prepare_repo_settings() and set "repo->settings.sparse_index = 1", only to needlessly call prepare_repo_settings() again in set_sparse_index_config() (where it would early abort), and finally setting "repo->settings.sparse_index = enabled". Instead we can just call prepare_repo_settings() once, and set the variable to "enabled" in the first place. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-06git-p4: speed up search for branch parentJoachim Kuebart1-11/+10
For every new branch that git-p4 imports, it needs to find the commit where it branched off its parent branch. While p4 doesn't record this information explicitly, the first changelist on a branch is usually an identical copy of the parent branch. The method searchParent() tries to find a commit in the history of the given "parent" branch whose tree exactly matches the initial changelist of the new branch, "target". The code iterates through the parent commits and compares each of them to this initial changelist using diff-tree. Since we already know the tree object name we are looking for, spawning diff-tree for each commit is wasteful. Use the "--format" option of "rev-list" to find out the tree object name of each commit in the history, and find the tree whose name is exactly the same as the tree of the target commit to optimize this. This results in a considerable speed-up, at least on Windows. On one Windows machine with a fairly large repository of about 16000 commits in the parent branch, the current code takes over 7 minutes, while the new code only takes just over 10 seconds for the same changelist: Before: $ time git p4 sync Importing from/into multiple branches Depot paths: //depot Importing revision 31274 (100.0%) Updated branches: b1 real 7m41.458s user 0m0.000s sys 0m0.077s After: $ time git p4 sync Importing from/into multiple branches Depot paths: //depot Importing revision 31274 (100.0%) Updated branches: b1 real 0m10.235s user 0m0.000s sys 0m0.062s Signed-off-by: Joachim Kuebart <joachim.kuebart@gmail.com> Helped-by: Junio C Hamano <gitster@pobox.com> Helped-by: Luke Diamand <luke@diamand.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-06git-p4: ensure complex branches are cloned correctlyJoachim Kuebart1-0/+2
When importing a branch from p4, git-p4 searches the history of the parent branch for the branch point. The test for the complex branch structure ensures all files have the expected contents, but doesn't examine the branch structure. Check for the correct branch structure by making sure that the initial commit on each branch is empty. This ensures that the initial commit's parent is indeed the correct branch-off point. Signed-off-by: Joachim Kuebart <joachim.kuebart@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-05patience diff: remove unused variablePhillip Wood1-3/+0
Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-05patience diff: remove unnecessary string comparisonsPhillip Wood1-8/+3
xdl_prepare_env() calls xdl_classify_record() which arranges for the hashes of non-matching lines to be different so lines can be tested for equality by comparing just their hashes. This reduces the time taken to calculate the diff of v2.28.0 to v2.29.0 by ~3-4%. Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-05word diff: handle zero length matchesPhillip Wood2-3/+12
If find_word_boundaries() encounters a zero length match (which can be caused by matching a newline or using '*' instead of '+' in the regex) we stop splitting the input into words which generates an inaccurate diff. To fix this increment the start point when there is a zero length match and try a new match. This is safe as posix regular expressions always return the longest available match so a zero length match means there are no longer matches available from the current position. Commit bf82940dbf1 (color-words: enable REG_NEWLINE to help user, 2009-01-17) prevented matching newlines in negated character classes but it is still possible for the user to have an explicit newline match in the regex which could cause a zero length match. One could argue that having explicit newline matches or using '*' rather than '+' are user errors but it seems to be better to work round them than produce inaccurate diffs. Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-05ci: run test round with parallel-checkout enabledMatheus Tavares4-0/+22
We already have tests for the basic parallel-checkout operations. But this code can also run be executed by other commands, such as git-read-tree and git-sparse-checkout, which are currently not tested with multiple workers. To promote a wider test coverage without duplicating tests: 1. Add the GIT_TEST_CHECKOUT_WORKERS environment variable, to optionally force parallel-checkout execution during the whole test suite. 2. Set this variable (with a value of 2) in the second test round of our linux-gcc CI job. This round runs `make test` again with some optional GIT_TEST_* variables enabled, so there is no additional overhead in exercising the parallel-checkout code here. Note that tests checking out less than two parallel-eligible entries will fall back to the sequential mode. Nevertheless, it's still a good exercise for the parallel-checkout framework as the fallback codepath also writes the queued entries using the parallel-checkout functions (only without spawning any worker). Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-05parallel-checkout: add tests for basic operationsMatheus Tavares2-0/+271
Add tests to populate the working tree during clone and checkout using sequential and parallel mode, to confirm that they produce identical results. Also test basic checkout mechanics, such as checking for symlinks in the leading directories and the abidance to --force. Note: some helper functions are added to a common lib file which is only included by t2080 for now. But they will also be used by other parallel-checkout tests in the following patches. Co-authored-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-05parallel-checkout: add tests related to .gitattributesMatheus Tavares1-0/+194
Add tests to confirm that the `struct conv_attrs` data is correctly passed from the main process to the workers, and that they can properly convert the blobs before writing them to the working tree. Also check that parallel-ineligible entries, such as regular files that require external filters, are correctly smudge and written when parallel-checkout is enabled. Co-authored-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-05checkout-index: add parallel checkout supportMatheus Tavares1-9/+15
Allow checkout-index to use the parallel checkout framework, honoring the checkout.workers configuration. There are two code paths in checkout-index which call `checkout_entry()`, and thus, can make use of parallel checkout: `checkout_file()`, which is used to write paths explicitly given at the command line; and `checkout_all()`, which is used to write all paths in the index, when the `--all` option is given. In both operation modes, checkout-index doesn't abort immediately on a `checkout_entry()` failure. Instead, it tries to check out all remaining paths before exiting with a non-zero exit code. To keep this behavior when parallel checkout is being used, we must allow `run_parallel_checkout()` to try writing the queued entries before we exit, even if we already got an error code from a previous `checkout_entry()` call. However, `checkout_all()` doesn't return on errors, it calls `exit()` with code 128. We could make it call `run_parallel_checkout()` before exiting, but it makes the code easier to follow if we unify the exit path for both checkout-index modes at `cmd_checkout_index()`, and let this function take care of the interactions with the parallel checkout API. So let's do that. With this change, we also have to consider whether we want to keep using 128 as the error code for `git checkout-index --all`, while we use 1 for `git checkout-index <path>` (even when the actual error is the same). Since there is not much value in having code 128 only for `--all`, and there is no mention about it in the docs (so it's unlikely that changing it will break any existing script), let's make both modes exit with code 1 on `checkout_entry()` errors. Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-05t0028: extract encoding helpers to lib-encoding.shMatheus Tavares2-24/+26
The following patch will add tests outside t0028 which will also need to re-encode some strings. Extract the auxiliary encoding functions from t0028 to a common lib file so that they can be reused. Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-05parallel-checkout: add tests related to path collisionsMatheus Tavares3-2/+168
Add tests to confirm that path collisions are properly detected by checkout workers, both to avoid race conditions and to report colliding entries on clone. Co-authored-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-05builtin/checkout.c: complete parallel checkout supportMatheus Tavares1-4/+17
Pathspec-limited checkouts (like `git checkout *.txt`) are performed by a code path that doesn't yet support parallel checkout because it calls checkout_entry() directly, instead of unpack_trees(). Let's add parallel checkout support for this code path too. The transient cache entries allocated in checkout_merged() are now allocated in a mem_pool which is only discarded after parallel checkout finishes. This is done because the entries need to be valid when run_parallel_checkout() is called. Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-05make_transient_cache_entry(): optionally alloc from mem_poolMatheus Tavares6-13/+23
Allow make_transient_cache_entry() to optionally receive a mem_pool struct in which it should allocate the entry. This will be used in the following patch, to store some transient entries which should persist until parallel checkout finishes. Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-05t5601: mark protocol v2-only testJonathan Tan1-1/+1
A HTTP-clone test introduced in 4fe788b1b0 ("builtin/clone.c: add --reject-shallow option", 2021-04-01) only works in protocol v2, but is not marked as such. The aforementioned patch implements --reject-shallow for a variety of situations, but usage of a protocol that requires a remote helper is not one of them. (Such an implementation would require extending the remote helper protocol to support the passing of a "reject shallow" option, and then teaching it to both protocol-speaking ends.) For now, to make it pass when GIT_TEST_PROTOCOL_VERSION=0 is passed, add "-c protocol.version=2". A more complete solution would be either to augment the remote helper protocol to support this feature or to return a fatal error when using --reject-shallow with a protocol that uses a remote helper. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-05send-pack: support push negotiationJonathan Tan3-4/+99
Teach Git the push.negotiate config variable. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-05fetch: teach independent negotiation (no packfile)Jonathan Tan11-17/+300
Currently, the packfile negotiation step within a Git fetch cannot be done independent of sending the packfile, even though there is at least one application wherein this is useful. Therefore, make it possible for this negotiation step to be done independently. A subsequent commit will use this for one such application - push negotiation. This feature is for protocol v2 only. (An implementation for protocol v0 would require a separate implementation in the fetch, transport, and transport helper code.) In the protocol, the main hindrance towards independent negotiation is that the server can unilaterally decide to send the packfile. This is solved by a "wait-for-done" argument: the server will then wait for the client to say "done". In practice, the client will never say it; instead it will cease requests once it is satisfied. In the client, the main change lies in the transport and transport helper code. fetch_refs_via_pack() performs everything needed - protocol version and capability checks, and the negotiation itself. There are 2 code paths that do not go through fetch_refs_via_pack() that needed to be individually excluded: the bundle transport (excluded through requiring smart_options, which the bundle transport doesn't support) and transport helpers that do not support takeover. If or when we support independent negotiation for protocol v0, we will need to modify these 2 code paths to support it. But for now, report failure if independent negotiation is requested in these cases. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-04t6423: rename file within directory that other side renamedElijah Newren1-0/+58
Add a new testcase where one side of history renames: olddir/ -> newdir/ and the other side of history renames: olddir/a -> olddir/alpha When using merge.directoryRenames=true, it seems logical to expect the file to end up at newdir/alpha. Unfortunately, both merge-recursive and merge-ort currently see this as a rename/rename conflict: olddir/a -> newdir/a vs. olddir/a -> newdir/alpha Suggesting that there's some extra logic we probably want to add somewhere to allow this case to run without triggering a conflict. For now simply document this known issue. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-04work around zsh comment in __git_complete_worktree_pathsSardorbek Imomaliev1-2/+3
[PATCH]: contrib/completion/git-completion.bash, there is a construct where comment lines are placed between the command that is on the upstream of a pipe and the command that is on the downstream of a pipe in __git_complete_worktree_paths function. Unfortunately, this script is also used by Zsh completion, but Zsh mishandles this construct when "interactive_comments" option is not set (by default it is off on macOS), resulting in a breakage: $ git worktree remove [TAB] $ git worktree remove __git_complete_worktree_paths:7: command not found: # Move the comment, even though it explains what happens on the downstream of the pipe and logically belongs where it is right now, before the entire pipeline, to work around this problem. Signed-off-by: Sardorbek Imomaliev <sardorbek.imomaliev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-04trailer: add new .cmd config optionZheNing Hu3-19/+175
The `trailer.<token>.command` configuration variable specifies a command (run via the shell, so it does not have to be a single name or path to the command, but can be a shell script), and the first occurrence of substring $ARG is replaced with the value given to the `interpret-trailer` command for the token in a '--trailer <token>=<value>' argument. This has three downsides: * The use of $ARG in the mechanism misleads the users that the value is passed in the shell variable, and tempt them to use $ARG more than once, but that would not work, as the second and subsequent $ARG are not replaced. * Because $ARG is textually replaced without regard to the shell language syntax, even '$ARG' (inside a single-quote pair), which a user would expect to stay intact, would be replaced, and worse, if the value had an unmatched single quote (imagine a name like "O'Connor", substituted into NAME='$ARG' to make it NAME='O'Connor'), it would result in a broken command that is not syntactically correct (or worse). * The first occurrence of substring `$ARG` will be replaced with the empty string, in the command when the command is first called to add a trailer with the specified <token>. This is a bad design, the nature of automatic execution causes it to add a trailer that we don't expect. Introduce a new `trailer.<token>.cmd` configuration that takes higher precedence to deprecate and eventually remove `trailer.<token>.command`, which passes the value as an argument to the command. Instead of "$ARG", users can refer to the value as positional argument, $1, in their scripts. At the same time, in order to allow `git interpret-trailers` to better simulate the behavior of `git command -s`, 'trailer.<token>.cmd' will not automatically execute. Helped-by: Junio C Hamano <gitster@pobox.com> Helped-by: Christian Couder <christian.couder@gmail.com> Signed-off-by: ZheNing Hu <adlternative@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-04docs: correct descript of trailer.<token>.commandZheNing Hu1-16/+21
In the original documentation of `trailer.<token>.command`, some descriptions are easily misunderstood. So let's modify it to increase its readability. In addition, clarify that `$ARG` in command can only be replaced once. Signed-off-by: ZheNing Hu <adlternative@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-04docs: document symlink restrictions for dot-filesJeff King4-0/+25
We stopped allowing symlinks for .gitmodules files in 10ecfa7649 (verify_path: disallow symlinks in .gitmodules, 2018-05-04), and we stopped following symlinks for .gitattributes, .gitignore, and .mailmap in the commits from 204333b015 (Merge branch 'jk/open-dotgitx-with-nofollow', 2021-03-22). The reasons are discussed in detail there, but we never adjusted the documentation to let users know. This hasn't been a big deal since the point is that such setups were mildly broken and thought to be unusual anyway. But it certainly doesn't hurt to be clear and explicit about it. Suggested-by: Philip Oakley <philipoakley@iee.email> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-04fsck: warn about symlinked dotfiles we'll open with O_NOFOLLOWJeff King3-2/+48
In the commits merged in via 204333b015 (Merge branch 'jk/open-dotgitx-with-nofollow', 2021-03-22), we stopped following symbolic links for .gitattributes, .gitignore, and .mailmap files. Let's teach fsck to warn that these symlinks are not going to do anything. Note that this is just a warning, and won't block the objects via transfer.fsckObjects, since there are reported to be cases of this in the wild (and even once fixed, they will continue to exist in the commit history of those projects, but are not particularly dangerous). Note that we won't add these to the existing gitmodules block in the fsck code. The logic for gitmodules is a bit more complicated, as we also check the content of non-symlink instances we find. But for these new files, there is no content check; we're just looking at the name and mode of the tree entry (and we can avoid even the complicated name checks in the common case that the mode doesn't indicate a symlink). We can reuse the test helper function we defined for .gitmodules, though (it needs some slight adjustments for the fsck error code, and because we don't block these symlinks via verify_path()). Note that I didn't explicitly test the transfer.fsckObjects case here (nor does the existing .gitmodules test that it blocks a push). The translation of fsck severities to outcomes is covered in general in t5504. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-04t0060: test ntfs/hfs-obscured dotfilesJeff King6-13/+75
We have tests that cover various filesystem-specific spellings of ".gitmodules", because we need to reliably identify that path for some security checks. These are from dc2d9ba318 (is_{hfs,ntfs}_dotgitmodules: add tests, 2018-05-12), with the actual code coming from e7cb0b4455 (is_ntfs_dotgit: match other .git files, 2018-05-11) and 0fc333ba20 (is_hfs_dotgit: match other .git files, 2018-05-02). Those latter two commits also added similar matching functions for .gitattributes and .gitignore. These ended up not being used in the final series, and are currently dead code. But in preparation for them being used in some fsck checks, let's make sure they actually work by throwing a few basic tests at them. Likewise, let's cover .mailmap (which does need matching code added). I didn't bother with the whole battery of tests that we cover for .gitmodules. These functions are all based on the same generic matcher, so it's sufficient to test most of the corner cases just once. Note that the ntfs magic prefix names in the tests come from the algorithm described in e7cb0b4455 (and are different for each file). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-04t7450: test .gitmodules symlink matching against obscured namesJeff King1-38/+53
In t7450 we check that both verify_path() and fsck catch malformed .gitmodules entries in trees. However, we don't check that we catch filesystem-equivalent forms of these (e.g., ".GITMOD~1" on Windows). Our name-matching functions are exercised well in t0060, but there's nothing to test that we correctly call the matching functions from the actual fsck and verify_path() code. So instead of testing just .gitmodules, let's repeat our tests for a few basic cases. We don't need to be exhaustive here (t0060 handles that), but just make sure we hit one name of each type. Besides pushing the tests into a function that takes the path as a parameter, we'll need to do a few things: - adjust the directory name to accommodate the tests running multiple times - set core.protecthfs for index checks. Fsck always protects all types by default, but we want to be able to exercise the HFS routines on every system. Note that core.protectntfs is already the default these days, but it doesn't hurt to explicitly label our need for it. - we'll also take the filename ("gitmodules") as a parameter. All calls use the same name for now, but a future patch will extend this to handle other .gitfoo files. Note that our fake-content symlink destination is somewhat .gitmodules specific. But it isn't necessary for other files (which don't do a content check). And it happens to be a valid attribute and ignore file anyway. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-04t7450: test verify_path() handling of gitmodulesJeff King1-3/+16
Commit 10ecfa7649 (verify_path: disallow symlinks in .gitmodules, 2018-05-04) made it impossible to load a symlink .gitmodules file into the index. However, there are no tests of this behavior. Let's make sure this case is covered. We can easily reuse the test setup created by the matching b7b1fca175 (fsck: complain when .gitmodules is a symlink, 2018-05-04). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-03t7415: rename to expand scopeJeff King1-3/+10
This script has already expanded beyond its original intent of ".. in submodule names" to include other malicious submodule bits. Let's update the name and description to reflect that, as well as the fact that we'll soon be adding similar tests for other dotfiles (.gitattributes, etc). We'll also renumber it to move it out of the group of submodule-specific tests. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-03fsck_tree(): wrap some long linesJeff King1-12/+36
Many calls to report() in fsck_tree() are kept on a single line and are quite long. Most were pretty big to begin with, but have gotten even longer over the years as we've added more parameters. Let's accept the churn of wrapping them in order to conform to our usual line limits. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-03fsck_tree(): fix shadowed variableJeff King2-22/+25
Commit b2f2039c2b (fsck: accept an oid instead of a "struct tree" for fsck_tree(), 2019-10-18) introduced a new "oid" parameter to fsck_tree(), and we pass it to the report() function when we find problems. However, that is shadowed within the tree-walking loop by the existing "oid" variable which we use to store the oid of each tree entry. As a result, we may report the wrong oid for some problems we detect within the loop (the entry oid, instead of the tree oid). Our tests didn't catch this because they checked only that we found the expected fsck problem, not that it was attached to the correct object. Let's rename both variables in the function to avoid confusion. This makes the diff a little noisy (e.g., all of the report() calls outside the loop were already correct but need to be touched), but makes sure we catch all cases and will avoid similar confusion in the future. And we can update the test to be a bit more specific and catch this problem. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-03t7415: remove out-dated comment about translationJeff King1-3/+2
Since GETTEXT_POISON does not exist anymore, there is no point warning people about whether we should use test_i18ngrep. This is doubly confusing because the comment was describing why it was OK to use grep, but it got caught up in the mass conversion of 674ba34038 (fsck: mark strings for translation, 2018-11-10). Note there are other uses of test_i18ngrep in this script which are now obsolete; I'll save those for a mass-cleanup. My goal here was just to fix the confusing comment in code I'm about to refactor. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-03docs/format-patch: mention handling of mergesJeff King1-1/+9
Format-patch doesn't have a way to format merges in a way that can be applied by git-am (or any other tool), and so it just omits them. However, this may be a surprising implication for users who are not well versed in how the tool works. Let's add a note to the documentation making this more clear. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-05-03pack-objects: clamp negative depth to 0Jeff King2-0/+9
A negative delta depth makes no sense, and the code is not prepared to handle it. If passed "--depth=-1" on the command line, then this line from break_delta_chains(): cur->depth = (total_depth--) % (depth + 1); triggers a divide-by-zero. This is undefined behavior according to the C standard, but on POSIX systems results in SIGFPE killing the process. This is certainly one way to inform the use that the command was invalid, but it's a bit friendlier to just treat it as "don't allow any deltas", which we already do for --depth=0. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>