git/read-cache.c, branch v2.31.7

Merge branch 'rs/calloc-array'

2021-03-19T22:25:38Z

CALLOC_ARRAY() macro replaces many uses of xcalloc(). * rs/calloc-array: cocci: allow xcalloc(1, size) use CALLOC_ARRAY git-compat-util.h: drop trailing semicolon from macro definition

Merge branch 'js/fsmonitor-unpack-fix'

2021-03-19T22:25:37Z

The data structure used by fsmonitor interface was not properly duplicated during an in-core merge, leading to use-after-free etc. * js/fsmonitor-unpack-fix: fsmonitor: do not forget to release the token in `discard_index()` fsmonitor: fix memory corruption in some corner cases

fsmonitor: do not forget to release the token in `discard_index()`

2021-03-17T19:19:28Z

In 56c6910028a (fsmonitor: change last update timestamp on the index_state to opaque token, 2020-01-07), we forgot to adjust `discard_index()` to release the "last-update" token: it is no longer a 64-bit number, but a free-form string that has been allocated. Signed-off-by: Johannes Schindelin Signed-off-by: Junio C Hamano

use CALLOC_ARRAY

2021-03-14T00:00:09Z

Add and apply a semantic patch for converting code that open-codes CALLOC_ARRAY to use it instead. It shortens the code and infers the element size automatically. Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano

Merge branch 'ns/raise-write-index-buffer-size'

2021-03-01T22:02:58Z

Raise the buffer size used when writing the index file out from (obviously too small) 8kB to (clearly sufficiently large) 128kB. * ns/raise-write-index-buffer-size: read-cache: make the index write buffer size 128K

Merge branch 'jh/fsmonitor-prework'

2021-03-01T22:02:56Z

Preliminary changes to fsmonitor integration. * jh/fsmonitor-prework: fsmonitor: refactor initialization of fsmonitor_last_update token fsmonitor: allow all entries for a folder to be invalidated fsmonitor: log FSMN token when reading and writing the index fsmonitor: log invocation of FSMonitor hook to trace2 read-cache: log the number of scanned files to trace2 read-cache: log the number of lstat calls to trace2 preload-index: log the number of lstat calls to trace2 p7519: add trace logging during perf test p7519: move watchman cleanup earlier in the test p7519: fix watchman watch-list test on Windows p7519: do not rely on "xargs -d" in test

read-cache: make the index write buffer size 128K

2021-02-24T21:40:30Z

Writing an index 8K at a time invokes the OS filesystem and caching code very frequently, introducing noticeable overhead while writing large indexes. When experimenting with different write buffer sizes on Windows writing the Windows OS repo index (260MB), most of the benefit came by bumping the index write buffer size to 64K. I picked 128K to ensure that we're past the knee of the curve. With this change, the time under do_write_index for an index with 3M files goes from ~1.02s to ~0.72s. Signed-off-by: Neeraj Singh Acked-by: Jeff Hostetler Signed-off-by: Junio C Hamano

read-cache: log the number of scanned files to trace2

2021-02-17T01:14:34Z

Report the number of files in the working directory that were read and their hashes verified in `refresh_index()`. FSMonitor improves the performance of commands like `git status` by avoiding scanning the disk for changed files. Let's measure this. Signed-off-by: Jeff Hostetler Reviewed-by: Taylor Blau Signed-off-by: Junio C Hamano

read-cache: log the number of lstat calls to trace2

2021-02-17T01:14:34Z

Report the total number of calls made to lstat() inside of refresh_index(). FSMonitor improves the performance of commands like `git status` by avoiding scanning the disk for changed files. This can be seen in `refresh_index()`. Let's measure this. Signed-off-by: Jeff Hostetler Reviewed-by: Taylor Blau Signed-off-by: Junio C Hamano

read-cache: try not to peek into `struct {lock_,temp}file`

2021-01-06T21:53:32Z

Similar to the previous commits, try to avoid peeking into the `struct lock_file`. We also have some `struct tempfile`s -- let's avoid looking into those as well. Note that `do_write_index()` takes a tempfile and that when we call it, we either have a tempfile which we can easily hand down, or we have a lock file, from which we need to somehow obtain the internal tempfile. So we need to leave that one instance of peeking-into. Nevertheless, this commit leaves us not relying on exactly how the path of the tempfile / lock file is stored internally. Signed-off-by: Martin Ågren Reviewed-by: Derrick Stolee Signed-off-by: Junio C Hamano