git/pack-objects.c, branch v2.24.0

pack-objects: drop packlist index_pos optimization

2019-09-06T18:03:42Z

Once upon a time, the code to add an object to our packing list in pack-objects all lived in a single function. It computed the position within the hash table once, then used it to check if the object was already present, and if not, to add it. Later, in 2834bc27c1 (pack-objects: refactor the packing list, 2013-10-24), this was split into two functions: packlist_find() and packlist_alloc(). We ended up with an "index_pos" variable that gets passed through several functions to make it from one to the other. The resulting code is rather confusing to follow. The "index_pos" variable is sometimes undefined, if we don't yet have a hash table. This works out in practice because in that case packlist_alloc() won't use it at all, since it will have to create/grow the hash table. But it's hard to verify that, and it does cause gcc 9.2.1's -Wmaybe-uninitialized to complain when compiled with "-flto -O3" (rightfully, since we do pass the uninitialized value as a function parameter, even if nobody ends up using it). All of this is to save computing the hash index again when we're inserting into the hash table, which I found doesn't make a measurable difference in the program runtime (which is not surprising, since we're doing all kinds of other heavyweight things for each object). Let's just drop this index_pos variable entirely, simplifying the code (and pleasing the compiler). We might be better still refactoring this custom hash table to use one of our existing implementations (an oidmap, or a kh_oid_map). I stopped short of that here, but this would be the likely first step towards that anyway. Reported-by: Stephan Beyer Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

pack-objects: use object_id in packlist_alloc()

2019-09-06T18:03:39Z

The only caller of packlist_alloc() already has a "struct object_id", and we immediately copy the hash they pass us into our own object_id. Let's avoid the unnecessary round-trip to a raw sha1 pointer. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

hashmap: convert sha1hash() to oidhash()

2019-06-20T17:44:22Z

There are no callers left of sha1hash() that do not simply pass the "hash" member of a "struct object_id". Let's get rid of the outdated sha1-specific function and provide one that operates on the whole struct (even though the technique, taking the first few bytes of the hash, will remain the same). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

pack-objects: convert locate_object_entry_hash() to object_id

2019-06-20T17:03:32Z

There are no callers of locate_object_entry_hash() that aren't just passing us the "hash" member of a "struct object_id". Let's take the whole struct, which gets us closer to removing all raw sha1 variables. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

pack-objects: convert packlist_find() to use object_id

2019-06-20T16:54:58Z

We take a raw hash pointer, but most of our callers have a "struct object_id" already. Let's switch to taking the full struct, which will let us continue removing uses of raw sha1 buffers. There are two callers that do need special attention: - in rebuild_existing_bitmaps(), we need to switch to nth_packed_object_oid(). This incurs an extra hash copy over pointing straight to the mmap'd sha1, but it shouldn't be measurable compared to the rest of the operation. - in can_reuse_delta() we already spent the effort to copy the sha1 into a "struct object_id", but now we just have to do so a little earlier in the function (we can't easily convert that function's callers because they may be pointing at mmap'd REF_DELTA blocks). Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

pack-objects: drop unused parameter from oe_map_new_pack()

2019-02-14T23:26:15Z

Since 43fa44fa3b (pack-objects: move in_pack out of struct object_entry, 2018-04-14), we store the source pack for each object as a small index rather than as a pointer. When we see a new pack that has no allocated index, we fall back to generating an array of pointers by calling oe_map_new_pack(). Perhaps counter-intuitively, that function does not need to actually see our new index-less pack. It only allocates and populates the array with the existing packs, after which oe_set_in_pack() actually adds the new pack to the array. Let's drop the unused "struct packed_git" argument to oe_map_new_pack() to avoid confusion. Signed-off-by: Jeff King Signed-off-by: Junio C Hamano

Merge branch 'ph/pack-objects-mutex-fix'

2019-02-05T22:26:16Z

"git pack-objects" incorrectly used uninitialized mutex, which has been corrected. * ph/pack-objects-mutex-fix: pack-objects: merge read_lock and lock in packing_data struct pack-objects: move read mutex to packing_data struct

pack-objects: merge read_lock and lock in packing_data struct

2019-01-28T19:22:12Z

Rename the packing_data lock to obd_lock and upgrade it to a recursive mutex to make it suitable for current read_lock usages. Additionally remove the superfluous #ifndef NO_PTHREADS guard around mutex initialization in prepare_packing_data as the mutex functions themselves are already protected. Signed-off-by: Patrick Hogg Helped-by: Junio C Hamano Signed-off-by: Junio C Hamano

pack-objects: move read mutex to packing_data struct

2019-01-28T19:22:06Z

ac77d0c37 ("pack-objects: shrink size field in struct object_entry", 2018-04-14) added an extra usage of read_lock/read_unlock in the newly introduced oe_get_size_slow for thread safety in parallel calls to try_delta(). Unfortunately oe_get_size_slow is also used in serial code, some of which is called before the first invocation of ll_find_deltas. As such the read mutex is not guaranteed to be initialized. Resolve this by moving the read mutex to packing_data and initializing it in prepare_packing_data which is initialized in cmd_pack_objects. Signed-off-by: Patrick Hogg Reviewed-by: Duy Nguyen Signed-off-by: Junio C Hamano

pack-*.c: remove the_repository references

2018-11-12T05:50:06Z

Signed-off-by: Nguyễn Thái Ngọc Duy Signed-off-by: Junio C Hamano