git/pack-write.c, branch v2.31.7

Merge branch 'jt/transfer-fsck-across-packs'

2021-03-01T22:02:57Z

The approach to "fsck" the incoming objects in "index-pack" is attractive for performance reasons (we have them already in core, inflated and ready to be inspected), but fundamentally cannot be applied fully when we receive more than one pack stream, as a tree object in one pack may refer to a blob object in another pack as ".gitmodules", when we want to inspect blobs that are used as ".gitmodules" file, for example. Teach "index-pack" to emit objects that must be inspected later and check them in the calling "fetch-pack" process. * jt/transfer-fsck-across-packs: fetch-pack: print and use dangling .gitmodules fetch-pack: with packfile URIs, use index-pack arg http-fetch: allow custom index-pack args http: allow custom index-pack args

fetch-pack: print and use dangling .gitmodules

2021-02-22T20:07:40Z

Teach index-pack to print dangling .gitmodules links after its "keep" or "pack" line instead of declaring an error, and teach fetch-pack to check such lines printed. This allows the tree side of the .gitmodules link to be in one packfile and the blob side to be in another without failing the fsck check, because it is now fetch-pack which checks such objects after all packfiles have been downloaded and indexed (and not index-pack on an individual packfile, as it is before this commit). Signed-off-by: Jonathan Tan Signed-off-by: Junio C Hamano

pack-write.c: prepare to write 'pack-*.rev' files

2021-01-26T02:32:43Z

This patch prepares for callers to be able to write reverse index files to disk. It adds the necessary machinery to write a format-compliant .rev file from within 'write_rev_file()', which is called from 'finish_tmp_packfile()'. Similar to the process by which the reverse index is computed in memory, these new paths also have to sort a list of objects by their offsets within a packfile. These new paths use a qsort() (as opposed to a radix sort), since our specialized radix sort requires a full revindex_entry struct per object, which is more memory than we need to allocate. The qsort is obviously slower, but the theoretical slowdown would require a repository with a large amount of objects, likely implying that the time spent in, say, pack-objects during a repack would dominate the overall runtime. Signed-off-by: Taylor Blau Signed-off-by: Junio C Hamano

pack-write: die on error in write_promisor_file()

2021-01-15T01:02:22Z

write_promisor_file() already uses xfopen(), so it would die if the file cannot be opened for writing. To be consistent with this behavior and not overlook issues, let's also die if there are errors when we are actually writing to the file. Suggested-by: Jeff King Suggested-by: Taylor Blau Signed-off-by: Christian Couder Signed-off-by: Junio C Hamano

fetch-pack: refactor writing promisor file

2021-01-13T00:01:07Z

Let's replace the 2 different pieces of code that write a promisor file in 'builtin/repack.c' and 'fetch-pack.c' with a new function called 'write_promisor_file()' in 'pack-write.c' and 'pack.h'. This might also help us in the future, if we want to put back the ref names and associated hashes that were in the promisor files we are repacking in 'builtin/repack.c' as suggested by a NEEDSWORK comment just above the code we are refactoring. Signed-off-by: Christian Couder Signed-off-by: Junio C Hamano

Merge branch 'rs/hashwrite-be64'

2020-11-25T23:24:52Z

Code simplification. * rs/hashwrite-be64: pack-write: use hashwrite_be64() midx: use hashwrite_be64() csum-file: add hashwrite_be64()

pack-write: use hashwrite_be64()

2020-11-12T17:40:10Z

Call hashwrite_be64() to write a 64-bit value instead of open-coding it using htonl() and hashwrite(). This shortens the code, gets rid of a buffer and several magic numbers, and makes the intent clearer. Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano

pack-write: use hashwrite_be32() instead of double-buffering array

2020-11-01T23:52:51Z

hashwrite() already buffers writes, so pass the fanout table entries individually via hashwrite_be32(), which also does the endianess conversion for us. This avoids a memory copy, shortens the code and reduces the number of magic numbers. Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano

pack-write: use hashwrite_be32() in write_idx_file()

2020-09-19T19:15:36Z

Call hashwrite_be32() instead of open-coding it. This shortens the code a bit and makes it easier to read. Signed-off-by: René Scharfe Signed-off-by: Junio C Hamano

Merge branch 'jb/doc-packfile-name' into master

2020-07-31T04:34:32Z

Doc update. * jb/doc-packfile-name: pack-write/docs: update regarding pack naming