git/ci, branch v2.24.3

Merge branch 'sg/ci-osx-gcc8-fix'

2019-10-24T04:34:03Z

CI build fix. * sg/ci-osx-gcc8-fix: ci: fix GCC install in the Travis CI GCC OSX job

ci: fix GCC install in the Travis CI GCC OSX job

2019-10-24T02:31:07Z

A few days ago Travis CI updated their existing OSX images, including the Homebrew database in the xcode10.1 OSX image that we use. Since then installing dependencies in the 'osx-gcc' job fails when it tries to link gcc@8: + brew link gcc@8 Error: No such keg: /usr/local/Cellar/gcc@8 GCC8 is still installed but not linked to '/usr/local' in the updated image, as it was before this update, but now we have to link it by running 'brew link gcc'. So let's do that then, and fall back to linking gcc@8 if it doesn't, just to be sure. Our builds on Azure Pipelines are unaffected by this issue. The OSX image over there doesn't contain the gcc@8 package, so we have to 'brew install' it, which already takes care of linking it to '/usr/local'. After that the 'brew link gcc' command added by this patch fails, but the ||-chained fallback 'brew link gcc@8' command succeeds with an "already linked" warning. Signed-off-by: SZEDER Gábor Signed-off-by: Junio C Hamano

ci(osx): use new location of the `perforce` cask

2019-10-23T02:46:41Z

The Azure Pipelines builds are failing for macOS due to a change in the location of the perforce cask. The command outputs the following error: + brew install caskroom/cask/perforce Error: caskroom/cask was moved. Tap homebrew/cask-cask instead. So let's try to call `brew cask install perforce` first (which is what that error message suggests, in a most round-about way). Prior to 672f51cb we used to install the 'perforce' package with 'brew install perforce' (note: no 'cask' in there). The justification for 672f51cb was that the command 'brew install perforce' simply stopped working, after Homebrew folks decided that it's better to move the 'perforce' package to a "cask". Their justification for this move was that 'brew install perforce' "can fail due to a checksum mismatch ...", and casks can be installed without checksum verification. And indeed, both 'brew cask install perforce' and 'brew install caskroom/cask/perforce' printed something along the lines of: ==> No checksum defined for Cask perforce, skipping verification It is unclear why 672f51cb used 'brew install caskroom/cask/perforce' instead of 'brew cask install perforce'. It appears (by running both commands on old Travis CI macOS images) that both commands worked all the same already back then. In any case, as the error message at the top of this commit message shows, 'brew install caskroom/cask/perforce' has stopped working recently, but 'brew cask install perforce' still does, so let's use that. CI servers are typically fresh virtual machines, but not always. To accommodate for that, let's try harder if `brew cask install perforce` fails, by specifically pulling the latest `master` of the `homebrew-cask` repository. This will still fail, of course, when `homebrew-cask` falls behind Perforce's release schedule. But once it is updated, we can now simply re-run the failed jobs and they will pick up that update. As for updating `homebrew-cask`: the beginnings of automating this in https://dev.azure.com/gitgitgadget/git/_build?definitionId=11&_a=summary will be finished once the next Perforce upgrade comes around. Helped-by: SZEDER Gábor Signed-off-by: Johannes Schindelin Signed-off-by: Derrick Stolee Signed-off-by: Junio C Hamano

Merge branch 'dl/honor-cflags-in-hdr-check'

2019-10-07T02:33:02Z

Dev support. * dl/honor-cflags-in-hdr-check: ci: run `hdr-check` as part of the `Static Analysis` job Makefile: emulate compile in $(HCO) target better pack-bitmap.h: remove magic number promisor-remote.h: include missing header apply.h: include missing header

Merge branch 'sg/travis-help-debug'

2019-10-07T02:33:01Z

Dev support update. * sg/travis-help-debug: travis-ci: do not skip successfully tested trees in debug mode

Merge branch 'bc/doc-use-docbook-5'

2019-10-06T03:25:16Z

Start using DocBook 5 (instead of DocBook 4.5) as Asciidoctor 2.0 no longer works with the older one. * bc/doc-use-docbook-5: Documentation: fix build with Asciidoctor 2

ci: run `hdr-check` as part of the `Static Analysis` job

2019-10-03T01:34:57Z

Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano

travis-ci: do not skip successfully tested trees in debug mode

2019-09-28T03:26:13Z

Travis CI offers shell access to its virtual machine environment running the build jobs, called "debug mode" [1]. After restarting a build job in debug mode and logging in, the first thing I usually do is to install dependencies, i.e. run './ci/install-dependencies.sh'. This works just fine when I restarted a failed build job in debug mode. However, after restarting a successful build job in debug mode our CI scripts get all clever, and exit without doing anything useful, claiming that "This commit's tree has already been built and tested successfully" [2]. Our CI scripts are right, and we do want to skip building and testing already known good trees in "regular" CI builds. In debug mode, however, this is a nuisiance, because one has to delete the cache (or at least the 'good-trees' file in the cache) to proceed. Let's update our CI scripts, in particular the common 'ci/lib.sh', to not skip previously successfully built and tested trees in debug mode, so all those scripts will do what there were supposed to do even when a successful build job was restarted in debug mode. [1] https://docs.travis-ci.com/user/running-build-in-debug-mode/ [2] 9cc2c76f5e (travis-ci: record and skip successfully built trees, 2017-12-31) Signed-off-by: SZEDER Gábor Signed-off-by: Junio C Hamano

Documentation: fix build with Asciidoctor 2

2019-09-16T19:20:39Z

Our documentation toolchain has traditionally been built around DocBook 4.5. This version of DocBook is the last DTD-based version of DocBook. In 2009, DocBook 5 was introduced using namespaces and its syntax is expressed in RELAX NG, which is more expressive and allows a wider variety of syntax forms. Asciidoctor, one of the alternatives for building our documentation, moved support for DocBook 4.5 out of core in its recent 2.0 release and now only supports DocBook 5 in the main release. The DocBoook 4.5 converter is still available as a separate component, but this is not available in most distro packages. This would not be a problem but for the fact that we use xmlto, which is still stuck in the DocBook 4.5 era. xmlto performs DTD validation as part of the build process. This is not problematic for DocBook 4.5, which has a valid DTD, but it clearly cannot work for DocBook 5, since no DTD can adequately express its full syntax. In addition, even if xmlto did support RELAX NG validation, that wouldn't be sufficient because it uses the libxml2-based xmllint to do so, which has known problems with validating interleaves in RELAX NG. Fortunately, there's an easy way forward: ask Asciidoctor to use its DocBook 5 backend and tell xmlto to skip validation. Asciidoctor has supported DocBook 5 since v0.1.4 in 2013 and xmlto has supported skipping validation for probably longer than that. We also need to teach xmlto how to use the namespaced DocBook XSLT stylesheets instead of the non-namespaced ones it usually uses. Normally these stylesheets are interchangeable, but the non-namespaced ones have a bug that causes them not to strip whitespace automatically from certain elements when namespaces are in use. This results in additional whitespace at the beginning of list elements, which is jarring and unsightly. We can do this by passing a custom stylesheet with the -x option that simply imports the namespaced stylesheets via a URL. Any system with support for XML catalogs will automatically look this URL up and reference a local copy instead without us having to know where this local copy is located. We know that anyone using xmlto will already have catalogs set up properly since the DocBook 4.5 DTD used during validation is also looked up via catalogs. All major Linux distributions distribute the necessary stylesheets and have built-in catalog support, and Homebrew does as well, albeit with a requirement to set an environment variable to enable catalog support. On the off chance that someone lacks support for catalogs, it is possible for xmlto (via xmllint) to download the stylesheets from the URLs in question, although this will likely perform poorly enough to attract attention. People still have the option of using the prebuilt documentation that we ship, so happily this should not be an impediment. Finally, we need to filter out some messages from other stylesheets that occur when invoking dblatex in the CI job. This tool strips namespaces much like the unnamespaced DocBook stylesheets and prints similar messages. If we permit these messages to be printed to standard error, our documentation CI job will fail because we check standard error for unexpected output. Due to dblatex's reliance on Python 2, we may need to revisit its use in the future, in which case this problem may go away, but this can be delayed until a future patch. The final message we filter is due to libxslt on modern Debian and Ubuntu. The patch which they use to implement reproducible ID generation also prints messages about the ID generation. While this doesn't affect our current CI images since they use Ubuntu 16.04 which lacks this patch, if we upgrade to Ubuntu 18.04 or a modern Debian, these messages will appear and, like the above messages, cause a CI failure. Signed-off-by: brian m. carlson Signed-off-by: Junio C Hamano

ci: restore running httpd tests

2019-09-06T17:06:41Z

Once upon a time GIT_TEST_HTTPD was a tristate variable and we exported 'GIT_TEST_HTTPD=YesPlease' in our CI scripts to make sure that we run the httpd tests in the Linux Clang and GCC build jobs, or error out if they can't be run for any reason [1]. Then 3b072c577b (tests: replace test_tristate with "git env--helper", 2019-06-21) came along, turned GIT_TEST_HTTPD into a bool, but forgot to update our CI scripts accordingly. So, since GIT_TEST_HTTPD is set explicitly, but its value is not one of the standardized true values, our CI jobs have been simply skipping the httpd tests in the last couple of weeks. Set 'GIT_TEST_HTTPD=true' to restore running httpd tests in our CI jobs. [1] a1157b76eb (travis-ci: set GIT_TEST_HTTPD in 'ci/lib-travisci.sh', 2017-12-12) Signed-off-by: SZEDER Gábor Signed-off-by: Junio C Hamano