git/progress.c, branch v2.24.0

Test the progress display

2019-09-17T16:39:16Z

'progress.c' has seen a few fixes recently [1], and, unfortunately, some of those fixes required further fixes [2]. It seems it's time to have a few tests focusing on the subtleties of the progress display. Add the 'test-tool progress' subcommand to help testing the progress display, reading instructions from standard input and turning them into calls to the display_progress() and display_throughput() functions with the given parameters. The progress display is, however, critically dependent on timing, because it's only updated once every second or, if the toal is known in advance, every 1%, and there is the throughput rate as well. These make the progress display far too undeterministic for testing as-is. To address this, add a few testing-specific variables and functions to 'progress.c', allowing the the new test helper to: - Disable the triggered-every-second SIGALRM and set the 'progress_update' flag explicitly based in the input instructions. This way the progress line will be updated deterministically when the test wants it to be updated. - Specify the time elapsed since start_progress() to make the throughput rate calculations deterministic. Add the new test script 't0500-progress-display.sh' to check a few simple cases with and without throughput, and that a shorter progress line properly covers up the previously displayed line in different situations. [1] See commits 545dc345eb (progress: break too long progress bar lines, 2019-04-12) and 9f1fd84e15 (progress: clear previous progress update dynamically, 2019-04-12). [2] 1aed1a5f25 (progress: avoid empty line when breaking the progress line, 2019-05-19) Signed-off-by: SZEDER Gábor Signed-off-by: Junio C Hamano

Revert "progress: use term_clear_line()"

2019-09-17T16:39:16Z

This reverts commit 5b12e3123b (progress: use term_clear_line(), 2019-06-24), because covering up the entire last line while refreshing the progress line caused unexpected problems during 'git clone/fetch/push': $ git clone ssh://localhost/home/szeder/src/tmp/linux.git/ Cloning into 'linux'... remote: remote: remote: remote: Enumerating objects: 999295 The length of the progress bar line can shorten when it includes throughput and the unit changes, or when its length exceeds the width of the terminal and is broken into two lines. In these cases the previously displayed longer progress line should be covered up, because otherwise the leftover characters from the previous progress line make the output look weird [1]. term_clear_line() makes this quite simple, as it covers up the entire last line either by using an ANSI control sequence or by printing a terminal width worth of space characters, depending on whether the terminal is smart or dumb. Unfortunately, when accessing a remote repository via any non-local protocol the remote 'git receive-pack/upload-pack' processes can't possibly have any idea about the local terminal (smart of dumb? how wide?) their progress will end up on. Consequently, they assume the worst, i.e. standard-width dumb terminal, and print 80 spaces to cover up the previously displayed progress line. The local 'git clone/fetch/push' processes then display the remote's progress, including these coverup spaces, with the 'remote: ' prefix, resulting in a total line length of 88 characters. If the local terminal is narrower than that, then the coverup gets line-wrapped, and after that the CR at the end doesn't return to the beginning of the progress line, but to the first column of its last line, resulting in those repeated 'remote: ' lines. By reverting 5b12e3123b (progress: use term_clear_line(), 2019-06-24) we won't cover up the entire last line, but go back to comparing the length of the current progress bar line with the previous one, and cover up as many characters as needed. [1] See commits 545dc345eb (progress: break too long progress bar lines, 2019-04-12) and 9f1fd84e15 (progress: clear previous progress update dynamically, 2019-04-12). Signed-off-by: SZEDER Gábor Signed-off-by: Junio C Hamano

Merge branch 'dr/progress-i18n'

2019-07-11T22:16:49Z

Progress messages have been made localizable. * dr/progress-i18n: l10n: localizable upload progress messages

l10n: localizable upload progress messages

2019-07-02T19:18:49Z

Currenly the data rate in throughput_string(...) method is output by simple strbuf_humanise_bytes(...) call and '/s' append. But for proper translation of such string the translator needs full context. Add strbuf_humanise_rate(...) method to properly print out localizable version of data rate ('3.5 MiB/s' etc) with full context. Strings with the units in strbuf_humanise_bytes(...) are marked for translation. Signed-off-by: Dimitriy Ryazantcev Signed-off-by: Junio C Hamano

progress: use term_clear_line()

2019-06-27T19:58:41Z

To make sure that the previously displayed progress line is completely covered up when the new line is shorter, commit 545dc345eb (progress: break too long progress bar lines, 2019-04-12) added a bunch of calculations to figure out how many characters it needs to overwrite with spaces. Use the just introduced term_clear_line() helper function to, well, clear the last line, making all these calculations unnecessary, and thus simplifying the code considerably. Three tests in 't5541-http-push-smart.sh' 'grep' for specific text shown in the progress lines at the beginning of the line, but now those lines begin either with the ANSI escape sequence or with the terminal width worth of space characters clearing the line. Relax the 'grep' patterns to match anywhere on the line. Note that only two of these three tests fail without relaxing their 'grep' pattern, but the third looks for the absence of the pattern, so it still succeeds, but without the adjustment would potentially hide future regressions. Note also that with this change we no longer need the length of the previously displayed progress line, so the strbuf added to 'struct progress' in d53ba841d4 (progress: assemble percentage and counters in a strbuf before printing, 2019-04-05) is not strictly necessary anymore. We still keep it, though, as it avoids allocating and releasing a strbuf each time the progress is updated. Signed-off-by: SZEDER Gábor Signed-off-by: Junio C Hamano

Merge branch 'sg/progress-off-by-one-fix'

2019-05-30T17:50:45Z

A brown-paper-bag bugfix to a change already in 'master'. * sg/progress-off-by-one-fix: progress: avoid empty line when breaking the progress line

progress: avoid empty line when breaking the progress line

2019-05-28T17:21:11Z

Since commit 545dc345eb (progress: break too long progress bar lines, 2019-04-12) when splitting a too long progress line, sometimes it looks as if a superfluous empty line were added between the title line and the counters. To make sure that the previously displayed progress line is completely covered up when writing the new, shorter title line, we calculate how many characters need to be overwritten with spaces. Alas, this calculation doesn't account for the newline character at the end of the new title line, and resulted in printing one more space than strictly necessary. This extra space character doesn't matter, if the length of the previous progress line was shorter than the width of the terminal. However, if the previous line matched the terminal width, then this extra space made the new line longer, effectively adding that empty line after the title line. Fix this off-by-one to avoid that spurious empty line. Signed-off-by: SZEDER Gábor Signed-off-by: Junio C Hamano

Merge branch 'jk/xmalloc'

2019-04-25T07:41:23Z

The code is updated to check the result of memory allocation before it is used in more places, by using xmalloc and/or xcalloc calls. * jk/xmalloc: progress: use xmalloc/xcalloc xdiff: use xmalloc/xrealloc xdiff: use git-compat-util test-prio-queue: use xmalloc

Merge branch 'sg/overlong-progress-fix'

2019-04-25T07:41:19Z

Updating the display with progress message has been cleaned up to deal better with overlong messages. * sg/overlong-progress-fix: progress: break too long progress bar lines progress: clear previous progress update dynamically progress: assemble percentage and counters in a strbuf before printing progress: make display_progress() return void

progress: break too long progress bar lines

2019-04-15T03:11:25Z

Some of the recently added progress indicators have quite long titles, which might be even longer when translated to some languages, and when they are shown while operating on bigger repositories, then the progress bar grows longer than the default 80 column terminal width. When the progress bar exceeds the width of the terminal it gets line-wrapped, and after that the CR at the end doesn't return to the beginning of the progress bar, but to the first column of its last line. Consequently, the first line of the previously shown progress bar is not overwritten by the next, and we end up with a bunch of truncated progress bar lines scrolling past: $ LANG=es_ES.UTF-8 git commit-graph write Encontrando commits para commit graph entre los objetos empaquetados: 2% (1599 Encontrando commits para commit graph entre los objetos empaquetados: 3% (1975 Encontrando commits para commit graph entre los objetos empaquetados: 4% (2633 Encontrando commits para commit graph entre los objetos empaquetados: 5% (3292 [...] Prevent this by breaking progress bars after the title once they exceed the width of the terminal, so the counter and optional percentage and throughput, i.e. all changing parts, are on the last line. Subsequent updates will from then on only refresh the changing parts, but not the title, and it will look like this: $ LANG=es_ES.UTF-8 ~/src/git/git commit-graph write Encontrando commits para commit graph entre los objetos empaquetados: 100% (6584502/6584502), listo. Calculando números de generación de commit graph: 100% (824705/824705), listo. Escribiendo commit graph en 4 pasos: 100% (3298820/3298820), listo. Note that the number of columns in the terminal is cached by term_columns(), so this might not kick in when it should when a terminal window is resized while the operation is running. Furthermore, this change won't help if the terminal is so narrow that the counters don't fit on one line, but I would put this in the "If it hurts, don't do it" box. Signed-off-by: SZEDER Gábor Signed-off-by: Junio C Hamano