bug#6131: [PATCH]: fiemap support for efficient sparse file copy

Jim Meyering wrote:
> jeff.liu wrote:
>> Sorry for the delay.
>>
>> This is the new patch to isolate the stuff regarding to extents reading to a new module. and teach
>> cp(1) to make use of it.
>
> Jeff,
>
> I applied your patch to my rebased fiemap-copy branch.
> My first step was to run the usual
>
>   ./bootstrap && ./configure && make && make check
>
> "make check" failed on due to a double free in your new code:
> (x86_64, Fedora 13, ext4 working directory)
>
> To get details, I made this temporary modification:
Hi Jim,

I am sorry for the fault, it fixed at the patch below.
Would you please revie at your convenience?

Changes:
========
1. fix write_zeros() as Jim's comments, thanks for pointing this out.
2. remove char const *fname from struct extent_scan.
3. change the signature of open_extent_scan() from "void open_extent_scan(struct extent_scan
**scan)" to "void open_extent_scan(struct extent_scan *scan)"; the reason is I'd like to reduce once
memory allocation for the extent_scan variable, instead, using stack to save it.
4. remove close_extent_scan() from a function defined at extent-scan.c to extent-scan.h as a Macro
definination, but it does nothing for now, since initial extent scan defined at stack.
5. add a macro "free_extents_info()" defined at extent-scan.h to release the memory allocated to
extent info which should be called combine with get_extents_info(), it just one line, so IMHO,
define it as macro should be ok.

I have done the memory check via `valgrind`, no issue found.
make test against cp/sparse-fiemap failed at the extent compare stage, but the file content is
identical to each other by comparing those two files "j1/j2" manually.
Is it make sense if we verify them through diff(1) since the testing file is in small size?
or we have to merge the contig extents from the output of `filefrag', I admit I have not dig into
the filefrag-extent-compare at the moment, I need to recall the perl language syntax. :-P.

>From 50a3338db06442fa2d789fd65175172d140cc96e Mon Sep 17 00:00:00 2001
From: Jie Liu <jeff.liu@oracle.com>
Date: Wed, 29 Sep 2010 15:35:43 +0800
Subject: [PATCH 1/1] cp: add a new module for scanning extents

* src/extent-scan.c: Source code for scanning extents.
  Call open_extent_scan() to initialize extent scan.
  Call get_extents_info() to get a number of extents for each iteration.
* src/extent-scan.h: Header file of extent-scan.c.
  Wrap free_extent_info() as macro define to release the space allocated extent_info per extent scan.
  Wrap close_extent_scan() as macro define but do nothing at the moment.
* src/Makefile.am: Reference it and link it to copy_source.
* src/copy.c: Make use of the new module, replace fiemap_copy() with extent_copy().

Signed-off-by: Jie Liu <jeff.liu@oracle.com>

1 files changed, 68 insertions, 0 deletions

diff --git a/src/extent-scan.h b/src/extent-scan.h
new file mode 100644
index 000000000..07c2e5bc5
--- /dev/null
+++ b/src/extent-scan.h
@@ -0,0 +1,68 @@
+/* core functions for efficient reading sparse files
+   Copyright (C) 2010 Free Software Foundation, Inc.
+
+   This program is free software: you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation, either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+   Written by Jie Liu (jeff.liu@oracle.com).  */
+
+#ifndef EXTENT_SCAN_H
+# define EXTENT_SCAN_H
+
+/* Structure used to reserve information of each extent.  */
+struct extent_info
+{
+  /* Logical offset of an extent.  */
+  off_t ext_logical;
+
+  /* Extent length.  */
+  uint64_t ext_length;
+
+  /* Extent flags, use it for FIEMAP only, or set it to zero.  */
+  uint32_t ext_flags;
+};
+
+/* Structure used to reserve extent scan information per file.  */
+struct extent_scan
+{
+  /* File descriptor of extent scan run against.  */
+  int fd;
+
+  /* Next scan start offset.  */
+  off_t scan_start;
+
+  /* How many extent info returned for a scan.  */
+  uint32_t ei_count;
+
+  /* If true, fall back to a normal copy, either
+     set by the failure of ioctl(2) for FIEMAP or
+     lseek(2) with SEEK_DATA.  */
+  bool initial_scan_failed;
+
+  /* If ture, the total extent scan per file has been finished.  */
+  bool hit_last_extent;
+
+  /* Extent information.  */
+  struct extent_info *ext_info;
+};
+
+void
+open_extent_scan (int src_fd, struct extent_scan *scan);
+
+bool
+get_extents_info (struct extent_scan *scan);
+
+#define free_extents_info(ext_scan) free ((ext_scan)->ext_info)
+#define close_extent_scan(ext_scan) /* empty */
+
+#endif /* EXTENT_SCAN_H */