diff options
Diffstat (limited to 'Documentation/technical')
| -rw-r--r-- | Documentation/technical/api-parse-options.adoc | 10 | ||||
| -rw-r--r-- | Documentation/technical/api-path-walk.adoc | 9 | ||||
| -rw-r--r-- | Documentation/technical/build-systems.adoc | 5 | ||||
| -rw-r--r-- | Documentation/technical/bundle-uri.adoc | 14 | ||||
| -rw-r--r-- | Documentation/technical/multi-pack-index.adoc | 82 | ||||
| -rw-r--r-- | Documentation/technical/sparse-checkout.adoc | 4 |
6 files changed, 99 insertions, 25 deletions
diff --git a/Documentation/technical/api-parse-options.adoc b/Documentation/technical/api-parse-options.adoc index 61fa6ee167..880eb94642 100644 --- a/Documentation/technical/api-parse-options.adoc +++ b/Documentation/technical/api-parse-options.adoc @@ -211,11 +211,13 @@ There are some macros to easily define options: Use of `--no-option` will clear the list of preceding values. `OPT_INTEGER(short, long, &int_var, description)`:: - Introduce an option with integer argument. - The integer is put into `int_var`. + Introduce an option with integer argument. The argument must be a + integer and may include a suffix of 'k', 'm' or 'g' to + scale the provided value by 1024, 1024^2 or 1024^3 respectively. + The scaled value is put into `int_var`. -`OPT_MAGNITUDE(short, long, &unsigned_long_var, description)`:: - Introduce an option with a size argument. The argument must be a +`OPT_UNSIGNED(short, long, &unsigned_long_var, description)`:: + Introduce an option with an unsigned integer argument. The argument must be a non-negative integer and may include a suffix of 'k', 'm' or 'g' to scale the provided value by 1024, 1024^2 or 1024^3 respectively. The scaled value is put into `unsigned_long_var`. diff --git a/Documentation/technical/api-path-walk.adoc b/Documentation/technical/api-path-walk.adoc index 3e089211fb..34c905eb9c 100644 --- a/Documentation/technical/api-path-walk.adoc +++ b/Documentation/technical/api-path-walk.adoc @@ -56,6 +56,14 @@ better off using the revision walk API instead. the revision walk so that the walk emits commits marked with the `UNINTERESTING` flag. +`edge_aggressive`:: + For performance reasons, usually only the boundary commits are + explored to find UNINTERESTING objects. However, in the case of + shallow clones it can be helpful to mark all trees and blobs + reachable from UNINTERESTING tip commits as UNINTERESTING. This + matches the behavior of `--objects-edge-aggressive` in the + revision API. + `pl`:: This pattern list pointer allows focusing the path-walk search to a set of patterns, only emitting paths that match the given @@ -69,4 +77,5 @@ Examples See example usages in: `t/helper/test-path-walk.c`, + `builtin/pack-objects.c`, `builtin/backfill.c` diff --git a/Documentation/technical/build-systems.adoc b/Documentation/technical/build-systems.adoc index d9dafb407c..3c5237b9fd 100644 --- a/Documentation/technical/build-systems.adoc +++ b/Documentation/technical/build-systems.adoc @@ -32,7 +32,10 @@ that generally have somebody running test pipelines against regularly: - OpenBSD The platforms which must be supported by the tool should be aligned with our -[platform support policy](platform-support.txt). +platform support policy (see platform-support.adoc). +// once we lose AsciiDoc compatibility, we can start writing the above as: +// xref:platform-support.adoc#platform-support-policy[platform support policy] +// or something like that, but until then.... === Auto-detection of supported features diff --git a/Documentation/technical/bundle-uri.adoc b/Documentation/technical/bundle-uri.adoc index 91d3a13e32..12283fa9ed 100644 --- a/Documentation/technical/bundle-uri.adoc +++ b/Documentation/technical/bundle-uri.adoc @@ -232,13 +232,13 @@ will interact with bundle URIs according to the following flow: are present in the client repository. If some are missing, then the client delays unbundling until other bundles have been unbundled, making those OIDs present. When all required OIDs are present, the - client unbundles that data using a refspec. The default refspec is - `+refs/heads/*:refs/bundles/*`, but this can be configured. These refs - are stored so that later `git fetch` negotiations can communicate each - bundled ref as a `have`, reducing the size of the fetch over the Git - protocol. To allow pruning refs from this ref namespace, Git may - introduce a numbered namespace (such as `refs/bundles/<i>/*`) such that - stale bundle refs can be deleted. + client unbundles that data using a refspec. The refspec used is + `+refs/*:refs/bundles/*`. These refs are stored so that later + `git fetch` negotiations can communicate each bundled ref as a `have`, + reducing the size of the fetch over the Git protocol. To allow pruning + refs from this ref namespace, Git may introduce a numbered namespace + (such as `refs/bundles/<i>/*`) such that stale bundle refs can be + deleted. 3. If the file is instead a bundle list, then the client inspects the `bundle.mode` to see if the list is of the `all` or `any` form. diff --git a/Documentation/technical/multi-pack-index.adoc b/Documentation/technical/multi-pack-index.adoc index cc063b30be..ffda70aa13 100644 --- a/Documentation/technical/multi-pack-index.adoc +++ b/Documentation/technical/multi-pack-index.adoc @@ -164,19 +164,81 @@ objects_nr($H2) + objects_nr($H1) + i (in the C implementation, this is often computed as `i + m->num_objects_in_base`). +=== Pseudo-pack order for incremental MIDXs + +The original implementation of multi-pack reachability bitmaps defined +the pseudo-pack order in linkgit:gitformat-pack[5] (see the section +titled "multi-pack-index reverse indexes") roughly as follows: + +____ +In short, a MIDX's pseudo-pack is the de-duplicated concatenation of +objects in packs stored by the MIDX, laid out in pack order, and the +packs arranged in MIDX order (with the preferred pack coming first). +____ + +In the incremental MIDX design, we extend this definition to include +objects from multiple layers of the MIDX chain. The pseudo-pack order +for incremental MIDXs is determined by concatenating the pseudo-pack +ordering for each layer of the MIDX chain in order. Formally two objects +`o1` and `o2` are compared as follows: + +1. If `o1` appears in an earlier layer of the MIDX chain than `o2`, then + `o1` sorts ahead of `o2`. + +2. Otherwise, if `o1` and `o2` appear in the same MIDX layer, and that + MIDX layer has no base, then if one of `pack(o1)` and `pack(o2)` is + preferred and the other is not, then the preferred one sorts ahead of + the non-preferred one. If there is a base layer (i.e. the MIDX layer + is not the first layer in the chain), then if `pack(o1)` appears + earlier in that MIDX layer's pack order, then `o1` sorts ahead of + `o2`. Likewise if `pack(o2)` appears earlier, then the opposite is + true. + +3. Otherwise, `o1` and `o2` appear in the same pack, and thus in the + same MIDX layer. Sort `o1` and `o2` by their offset within their + containing packfile. + +Note that the preferred pack is a property of the MIDX chain, not the +individual layers themselves. Fundamentally we could introduce a +per-layer preferred pack, but this is less relevant now that we can +perform multi-pack reuse across the set of packs in a MIDX. + +=== Reachability bitmaps and incremental MIDXs + +Each layer of an incremental MIDX chain may have its objects (and the +objects from any previous layer in the same MIDX chain) represented in +its own `*.bitmap` file. + +The structure of a `*.bitmap` file belonging to an incremental MIDX +chain is identical to that of a non-incremental MIDX bitmap, or a +classic single-pack bitmap. Since objects are added to the end of the +incremental MIDX's pseudo-pack order (see above), it is possible to +extend a bitmap when appending to the end of a MIDX chain. + +(Note: it is possible likewise to compress a contiguous sequence of MIDX +incremental layers, and their `*.bitmap` files into a single layer and +`*.bitmap`, but this is not yet implemented.) + +The object positions used are global within the pseudo-pack order, so +subsequent layers will have, for example, `m->num_objects_in_base` +number of `0` bits in each of their four type bitmaps. This follows from +the fact that we only write type bitmap entries for objects present in +the layer immediately corresponding to the bitmap). + +Note also that only the bitmap pertaining to the most recent layer in an +incremental MIDX chain is used to store reachability information about +the interesting and uninteresting objects in a reachability query. +Earlier bitmap layers are only used to look up commit and pseudo-merge +bitmaps from that layer, as well as the type-level bitmaps for objects +in that layer. + +To simplify the implementation, type-level bitmaps are iterated +simultaneously, and their results are OR'd together to avoid recursively +calling internal bitmap functions. + Future Work ----------- -- The multi-pack-index allows many packfiles, especially in a context - where repacking is expensive (such as a very large repo), or - unexpected maintenance time is unacceptable (such as a high-demand - build machine). However, the multi-pack-index needs to be rewritten - in full every time. We can extend the format to be incremental, so - writes are fast. By storing a small "tip" multi-pack-index that - points to large "base" MIDX files, we can keep writes fast while - still reducing the number of binary searches required for object - lookups. - - If the multi-pack-index is extended to store a "stable object order" (a function Order(hash) = integer that is constant for a given hash, even as the multi-pack-index is updated) then MIDX bitmaps could be diff --git a/Documentation/technical/sparse-checkout.adoc b/Documentation/technical/sparse-checkout.adoc index 67134bb768..0f750ef3e3 100644 --- a/Documentation/technical/sparse-checkout.adoc +++ b/Documentation/technical/sparse-checkout.adoc @@ -66,7 +66,7 @@ sparsity patterns: patterns from $GIT_DIR/info/sparse-checkout used to reasons: (1) users in cone mode specify directories rather than patterns (their directories are transformed into patterns, but users may think you are talking about non-cone mode if you use the - word "patterns"), and (b) the sparse specification might + word "patterns"), and (2) the sparse specification might transiently differ in the working tree or index from the sparsity patterns (see "Sparse specification vs. sparsity patterns"). @@ -356,8 +356,6 @@ understanding these differences can be beneficial. The behavior for these commands somewhat depends upon the merge strategy being used: * `ort` behaves as described above - * `recursive` tries to not vivify files unnecessarily, but does sometimes - vivify files without conflicts. * `octopus` and `resolve` will always vivify any file changed in the merge relative to the first parent, which is rather suboptimal. |