git/git.c, branch v2.10.3

Merge branch 'jk/common-main' into maint-2.10

2016-12-05T19:24:28Z

* jk/common-main: common-main: stop munging argv[0] path git-compat-util: move content inside ifdef/endif guards

common-main: stop munging argv[0] path

2016-11-29T19:01:48Z

Since 650c44925 (common-main: call git_extract_argv0_path(), 2016-07-01), the argv[0] that is seen in cmd_main() of individual programs is always the basename of the executable, as common-main strips off the full path. This can produce confusing results for git-daemon, which wants to re-exec itself. For instance, if the program was originally run as "/usr/lib/git/git-daemon", it will try just re-execing "git-daemon", which will find the first instance in $PATH. If git's exec-path has not been prepended to $PATH, we may find the git-daemon from a different version (or no git-daemon at all). Normally this isn't a problem. Git commands are run as "git daemon", the git wrapper puts the exec-path at the front of $PATH, and argv[0] is already "daemon" anyway. But running git-daemon via its full exec-path, while not really a recommended method, did work prior to 650c44925. Let's make it work again. The real goal of 650c44925 was not to munge argv[0], but to reliably set the argv0_path global. The only reason it munges at all is that one caller, the git.c wrapper, piggy-backed on that computation to find the command basename. Instead, let's leave argv[0] untouched in common-main, and have git.c do its own basename computation. While we're at it, let's drop the return value from git_extract_argv0_path(). It was only ever used in this one callsite, and its dual purposes is what led to this confusion in the first place. Note that by changing the interface, the compiler can confirm for us that there are no other callers storing the return value. But the compiler can't tell us whether any of the cmd_main() functions (besides git.c) were relying on the basename munging. However, we can observe that prior to 650c44925, no other cmd_main() functions did that munging, and no new cmd_main() functions have been introduced since then. So we can't be regressing any of those cases. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

cocci: refactor common patterns to use xstrdup_or_null()

2016-10-12T18:22:10Z

d64ea0f83b ("git-compat-util: add xstrdup_or_null helper", 2015-01-12) added a handy wrapper that allows us to get a duplicate of a string or NULL if the original is NULL, but a handful of codepath predate its introduction or just weren't aware of it. Signed-off-by: Junio C Hamano

common-main: call git_setup_gettext()

2016-07-01T22:09:10Z

This should be part of every program, as otherwise users do not get translated error messages. However, some external commands forgot to do so (e.g., git-credential-store). This fixes them, and eliminates the repeated code in programs that did remember to use it. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

common-main: call restore_sigpipe_to_default()

2016-07-01T22:09:10Z

This is another safety/sanity setup that should be in force everywhere, but which we only applied in git.c. This did catch most cases, since even external commands are typically run via "git ..." (and the restoration applies to sub-processes, too). But there were cases we missed, such as somebody calling git-upload-pack directly via ssh, or scripts which use dashed external commands directly. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

common-main: call sanitize_stdfds()

2016-07-01T22:09:10Z

This is setup that should be done in every program for safety, but we never got around to adding it everywhere (so builtins benefited from the call in git.c, but any external commands did not). Putting it in the common main() gives us this safety everywhere. Note that the case in daemon.c is a little funny. We wait until we know whether we want to daemonize, and then either: - call daemonize(), which will close stdio and reopen it to /dev/null under the hood - sanitize_stdfds(), to fix up any odd cases But that is way too late; the point of sanitizing is to give us reliable descriptors on 0/1/2, and we will already have executed code, possibly called die(), etc. The sanitizing should be the very first thing that happens. With this patch, git-daemon will sanitize first, and can remove the call in the non-daemonize case. It does mean that daemonize() may just end up closing the descriptors we opened, but that's not a big deal (it's not wrong to do so, nor is it really less optimal than the case where our parent process redirected us from /dev/null ahead of time). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

common-main: call git_extract_argv0_path()

2016-07-01T22:09:10Z

Every program which links against libgit.a must call this function, or risk hitting an assert() in system_path() that checks whether we have configured argv0_path (though only when RUNTIME_PREFIX is defined, so essentially only on Windows). Looking at the diff, you can see that putting it into the common main() saves us having to do it individually in each of the external commands. But what you can't see are the cases where we _should_ have been doing so, but weren't (e.g., git-credential-store, and all of the t/helper test programs). This has been an accident-waiting-to-happen for a long time, but wasn't triggered until recently because it involves one of those programs actually calling system_path(). That happened with git-credential-store in v2.8.0 with ae5f677 (lazily load core.sharedrepository, 2016-03-11). The program: - takes a lock file, which... - opens a tempfile, which... - calls adjust_shared_perm to fix permissions, which... - lazy-loads the config (as of ae5f677), which... - calls system_path() to find the location of /etc/gitconfig On systems with RUNTIME_PREFIX, this means credential-store reliably hits that assert() and cannot be used. We never noticed in the test suite, because we set GIT_CONFIG_NOSYSTEM there, which skips the system_path() lookup entirely. But if we were to tweak git_config() to find /etc/gitconfig even when we aren't going to open it, then the test suite shows multiple failures (for credential-store, and for some other test helpers). I didn't include that tweak here because it's way too specific to this particular call to be worth carrying around what is essentially dead code. The implementation is fairly straightforward, with one exception: there is exactly one caller (git.c) that actually cares about the result of the function, and not the side-effect of setting up argv0_path. We can accommodate that by simply replacing the value of argv[0] in the array we hand down to cmd_main(). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

add an extra level of indirection to main()

2016-07-01T22:09:10Z

There are certain startup tasks that we expect every git process to do. In some cases this is just to improve the quality of the program (e.g., setting up gettext()). In others it is a requirement for using certain functions in libgit.a (e.g., system_path() expects that you have called git_extract_argv0_path()). Most commands are builtins and are covered by the git.c version of main(). However, there are still a few external commands that use their own main(). Each of these has to remember to include the correct startup sequence, and we are not always consistent. Rather than just fix the inconsistencies, let's make this harder to get wrong by providing a common main() that can run this standard startup. We basically have two options to do this: - the compat/mingw.h file already does something like this by adding a #define that replaces the definition of main with a wrapper that calls mingw_startup(). The upside is that the code in each program doesn't need to be changed at all; it's rewritten on the fly by the preprocessor. The downside is that it may make debugging of the startup sequence a bit more confusing, as the preprocessor is quietly inserting new code. - the builtin functions are all of the form cmd_foo(), and git.c's main() calls them. This is much more explicit, which may make things more obvious to somebody reading the code. It's also more flexible (because of course we have to figure out _which_ cmd_foo() to call). The downside is that each of the builtins must define cmd_foo(), instead of just main(). This patch chooses the latter option, preferring the more explicit approach, even though it is more invasive. We introduce a new file common-main.c, with the "real" main. It expects to call cmd_main() from whatever other objects it is linked against. We link common-main.o against anything that links against libgit.a, since we know that such programs will need to do this setup. Note that common-main.o can't actually go inside libgit.a, as the linker would not pick up its main() function automatically (it has no callers). The rest of the patch is just adjusting all of the various external programs (mostly in t/helper) to use cmd_main(). I've provided a global declaration for cmd_main(), which means that all of the programs also need to match its signature. In particular, many functions need to switch to "const char **" instead of "char **" for argv. This effect ripples out to a few other variables and functions, as well. This makes the patch even more invasive, but the end result is much better. We should be treating argv strings as const anyway, and now all programs conform to the same signature (which also matches the way builtins are defined). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

setup: make startup_info available everywhere

2016-03-07T01:17:37Z

Commit a60645f (setup: remember whether repository was found, 2010-08-05) introduced the startup_info structure, which records some parts of the setup_git_directory() process (notably, whether we actually found a repository or not). One of the uses of this data is for functions to behave appropriately based on whether we are in a repo. But the startup_info struct is just a pointer to storage provided by the main program, and the only program that sets it up is the git.c wrapper. Thus builtins have access to startup_info, but externally linked programs do not. Worse, library code which is accessible from both has to be careful about accessing startup_info. This can be used to trigger a die("BUG") via get_sha1(): $ git fast-import <<-\EOF tag foo from HEAD:./whatever EOF fatal: BUG: startup_info struct is not initialized. Obviously that's fairly nonsensical input to feed to fast-import, but we should never hit a die("BUG"). And there may be other ways to trigger it if other non-builtins resolve sha1s. So let's point the storage for startup_info to a static variable in setup.c, making it available to all users of the library code. We _could_ turn startup_info into a regular extern struct, but doing so would mean tweaking all of the existing use sites. So let's leave the pointer indirection in place. We can, however, drop any checks for NULL, as they will always be false (and likewise, we can drop the test covering this case, which was a rather artificial situation using one of the test-* programs). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Merge branch 'jk/tighten-alloc'

2016-02-26T21:37:16Z

Update various codepaths to avoid manually-counted malloc(). * jk/tighten-alloc: (22 commits) ewah: convert to REALLOC_ARRAY, etc convert ewah/bitmap code to use xmalloc diff_populate_gitlink: use a strbuf transport_anonymize_url: use xstrfmt git-compat-util: drop mempcpy compat code sequencer: simplify memory allocation of get_message test-path-utils: fix normalize_path_copy output buffer size fetch-pack: simplify add_sought_entry fast-import: simplify allocation in start_packfile write_untracked_extension: use FLEX_ALLOC helper prepare_{git,shell}_cmd: use argv_array use st_add and st_mult for allocation size computation convert trivial cases to FLEX_ARRAY macros use xmallocz to avoid size arithmetic convert trivial cases to ALLOC_ARRAY convert manual allocations to argv_array argv-array: add detach function add helpers for allocating flex-array structs harden REALLOC_ARRAY and xcalloc against size_t overflow tree-diff: catch integer overflow in combine_diff_path allocation ...