1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
|
==================================================
[SSD: it's a toss-up, when removing a million-file (1000 per dir) tree
on (sdd-ocz-summit-120G, F11, x86_64, lots of RAM)
$ z-mktree --root=z --depth=2 --b=1000;env time /p/bin/rm -rf z
0.50user 13.99system 0:17.83elapsed 81%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+10834minor)pagefaults 0swaps
$ z-mktree --root=z --depth=2 --b=1000;env time src/rm -rf z
0.52user 13.88system 0:17.41elapsed 82%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+8outputs (0major+10835minor)pagefaults 0swaps
==================================================
on (tmpfs, F11, x86_64, lots of RAM)
/t, with very deep, narrow tree, new rm is >20% faster:
$ z-mkdir 1000000
$ env time /p/bin/rm -rf $TMPDIR/z
6.25user 12.88system 0:19.22elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+70976minor)pagefaults 0swaps
$ z-mkdir 1000000
env time /bin/rm -rf $TMPDIR/z
$ env time /bin/rm -rf $TMPDIR/z
11.06user 14.32system 0:25.43elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
56inputs+0outputs (1major+79999minor)pagefaults 0swaps
*** slow*DOWN* of ~4% on a 1-1Kdir-1M-file (1k-files-per-dir) hierarchy
again, this is on tmpfs:
$ z-mktree --root=/t/z --depth=2 --b=1000;strace -c /p/bin/rm -rf /t/z
vv$ z-mktree --root=/t/z --depth=2 --b=1000;env time /cu/src/rm -rf /t/z
0.29user 2.75system 0:03.06elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+10834minor)pagefaults 0swaps
$ z-mktree --root=/t/z --depth=2 --b=1000;env time /p/p/coreutils-7.5/bin/rm -rf /t/z
0.14user 2.76system 0:02.92elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+184minor)pagefaults 0swaps
==================================================
[old rm]
iou$ z-mktree --root=/t/z --depth=2 --b=1000;strace -c /p/bin/rm -rf /t/z
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
90.66 0.086370 0 1001002 1 unlinkat
8.72 0.008310 1 10002 getdents64
0.22 0.000214 0 2001 openat
0.20 0.000188 0 2021 close
0.09 0.000089 0 6019 fstat64
0.08 0.000073 0 6003 fcntl64
0.03 0.000028 1 29 12 open
0.00 0.000000 0 4 read
0.00 0.000000 0 1 execve
0.00 0.000000 0 1000 lseek
0.00 0.000000 0 3 3 access
0.00 0.000000 0 3 brk
0.00 0.000000 0 1 ioctl
0.00 0.000000 0 2 munmap
0.00 0.000000 0 4 mprotect
0.00 0.000000 0 19 mmap2
0.00 0.000000 0 1 lstat64
0.00 0.000000 0 1 set_thread_area
0.00 0.000000 0 1 fstatat64
------ ----------- ----------- --------- --------- ----------------
100.00 0.095272 1028117 16 total
[new, fts-based rm]
iou$ z-mktree --root=/t/z --depth=2 --b=1000;strace -c ./rm -rf /t/z
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
95.88 0.122730 0 1001001 unlinkat
3.95 0.005058 1 7007 getdents64
0.10 0.000126 0 2022 close
0.05 0.000064 0 1001 openat
0.02 0.000020 0 1017 fstat64
0.00 0.000000 0 4 read
0.00 0.000000 0 29 12 open
0.00 0.000000 0 1 execve
0.00 0.000000 0 3 3 access
0.00 0.000000 0 510 brk
0.00 0.000000 0 1 ioctl
0.00 0.000000 0 2 munmap
0.00 0.000000 0 4 mprotect
0.00 0.000000 0 19 mmap2
0.00 0.000000 0 1 lstat64
0.00 0.000000 0 3003 fcntl64
0.00 0.000000 0 1 set_thread_area
0.00 0.000000 0 1001 fstatat64
------ ----------- ----------- --------- --------- ----------------
100.00 0.127998 1016627 15 total
iou$ export LC_ALL=C cu:198-rm-fts
iou$ z-mktree --root=/t/z --depth=2 --b=1000;strace -c ./rm -rf /t/z
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
93.49 0.110607 0 1001001 unlinkat
6.32 0.007475 1 7007 getdents64
0.10 0.000124 0 1001 fstatat64
0.06 0.000068 0 479 brk
0.02 0.000018 0 2008 close
0.01 0.000015 0 3003 fcntl64
0.00 0.000000 0 2 read
0.00 0.000000 0 3 open
0.00 0.000000 0 1 execve
0.00 0.000000 0 3 3 access
0.00 0.000000 0 1 ioctl
0.00 0.000000 0 1 munmap
0.00 0.000000 0 4 mprotect
0.00 0.000000 0 6 mmap2
0.00 0.000000 0 1 lstat64
0.00 0.000000 0 1003 fstat64
0.00 0.000000 0 1 set_thread_area
0.00 0.000000 0 1001 openat
------ ----------- ----------- --------- --------- ----------------
100.00 0.118307 1016526 3 total
iou$ z-mktree --root=/t/z --depth=2 --b=1000;strace -c /p/bin/rm -rf /t/z
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
92.87 0.107659 0 1001002 1 unlinkat
6.59 0.007641 1 10002 getdents64
0.22 0.000252 0 2001 openat
0.15 0.000179 0 6005 fstat64
0.10 0.000113 0 6003 fcntl64
0.05 0.000054 0 1000 lseek
0.02 0.000027 0 2007 close
0.00 0.000000 0 2 read
0.00 0.000000 0 3 open
0.00 0.000000 0 1 execve
0.00 0.000000 0 3 3 access
0.00 0.000000 0 3 brk
0.00 0.000000 0 1 ioctl
0.00 0.000000 0 1 munmap
0.00 0.000000 0 4 mprotect
0.00 0.000000 0 6 mmap2
0.00 0.000000 0 1 lstat64
0.00 0.000000 0 1 set_thread_area
0.00 0.000000 0 1 fstatat64
------ ----------- ----------- --------- --------- ----------------
100.00 0.115925 1028047 4 total
**********************************************************************
iou$ z-mktree --root=/t/z --depth=2 --b=1000;env time /p/bin/rm -rf /t/z
0.16user 3.35system 0:03.68elapsed 95%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+166minor)pagefaults 0swaps
iou$ z-mktree --root=/t/z --depth=2 --b=1000;env time /p/bin/rm -rf /t/z
0.16user 3.12system 0:03.34elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+166minor)pagefaults 0swaps
iou$ z-mktree --root=/t/z --depth=2 --b=1000;env time /p/bin/rm -rf /t/z
0.14user 3.12system 0:03.26elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+167minor)pagefaults 0swaps
iou$ z-mktree --root=/t/z --depth=2 --b=1000;env time /p/bin/rm -rf /t/z
0.12user 2.98system 0:03.17elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+167minor)pagefaults 0swaps
iou$ z-mktree --root=/t/z --depth=2 --b=1000;env time ./rm -rf /t/z
0.68user 3.08system 0:03.76elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+1566minor)pagefaults 0swaps
iou$ z-mktree --root=/t/z --depth=2 --b=1000;env time ./rm -rf /t/z
0.64user 3.12system 0:03.79elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+1565minor)pagefaults 0swaps
iou$ z-mktree --root=/t/z --depth=2 --b=1000;env time ./rm -rf /t/z
0.60user 3.09system 0:03.78elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+1566minor)pagefaults 0swaps
************************************************************************
************************************************************************
The above are relatively normal hierarchies.
Here is one that is extreme (/t is a tmpfs file system):
[shows that fts-based rm takes 20-25% more time than the old one]
iou$ mkdir .j && (cd .j && seq 1000000|xargs touch)
iou$ env time ./rm -rf /t/.j cu:198-rm-fts
0.10user 0.41system 0:00.59elapsed 87%CPU (0avgtext+0avgdata 0maxresident)k
256inputs+0outputs (1major+4459minor)pagefaults 0swaps
iou$ env time ./rm -rf /t/.j cu:198-rm-fts
0.82user 3.48system 0:04.35elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
256inputs+0outputs (1major+43130minor)pagefaults 0swaps
iou$ env time /p/bin/rm -rf /t/.j cu:198-rm-fts
0.14user 3.02system 0:03.25elapsed 97%CPU (0avgtext+0avgdata 0maxresident)k
256inputs+0outputs (1major+164minor)pagefaults 0swaps
iou$ env time strace -c /p/bin/rm -rf /t/.j cu:198-rm-fts
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
95.49 0.122568 0 1000002 1 unlinkat
4.51 0.005785 1 7797 getdents64
0.00 0.000000 0 2 read
0.00 0.000000 0 3 open
0.00 0.000000 0 7 close
0.00 0.000000 0 1 execve
0.00 0.000000 0 1 lseek
0.00 0.000000 0 3 3 access
0.00 0.000000 0 3 brk
0.00 0.000000 0 1 ioctl
0.00 0.000000 0 1 munmap
0.00 0.000000 0 4 mprotect
0.00 0.000000 0 6 mmap2
0.00 0.000000 0 1 lstat64
0.00 0.000000 0 5 fstat64
0.00 0.000000 0 3 fcntl64
0.00 0.000000 0 1 set_thread_area
0.00 0.000000 0 1 fstatfs64
0.00 0.000000 0 1 openat
0.00 0.000000 0 1 fstatat64
------ ----------- ----------- --------- --------- ----------------
100.00 0.128353 1007844 4 total
2.96user 21.57system 0:24.80elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
8inputs+0outputs (0major+406minor)pagefaults 0swaps
iou$ env time strace -c ./rm -rf /t/.j cu:198-rm-fts
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
83.46 0.160976 0 1000001 unlinkat
14.54 0.028036 21 1306 brk
2.01 0.003873 0 7795 getdents64
0.00 0.000000 0 2 read
0.00 0.000000 0 3 open
0.00 0.000000 0 8 close
0.00 0.000000 0 1 execve
0.00 0.000000 0 3 3 access
0.00 0.000000 0 1 ioctl
0.00 0.000000 0 1 munmap
0.00 0.000000 0 4 mprotect
0.00 0.000000 0 6 mmap2
0.00 0.000000 0 1 lstat64
0.00 0.000000 0 3 fstat64
0.00 0.000000 0 3 fcntl64
0.00 0.000000 0 1 set_thread_area
0.00 0.000000 0 1 fstatfs64
0.00 0.000000 0 1 openat
0.00 0.000000 0 1 fstatat64
------ ----------- ----------- --------- --------- ----------------
100.00 0.192885 1009142 3 total
3.86user 22.06system 0:26.30elapsed 98%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+43371minor)pagefaults 0swaps
======================================================
# On a faster system, the new fts-based rm still takes a 20% perf. hit,
# probably due to malloc (ab)use, just like above. Yep, sure looks like it.
# in /tmpfs, /t
mkdir .j && (cd .j && seq 1000000|xargs touch)
vv$ env time /p/p/coreutils-7.5/bin/rm -rf .j
0.14user 2.81system 0:02.98elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
264inputs+0outputs (1major+174minor)pagefaults 0swaps
vv$ cd /t; env time /cu/src/rm -rf .j
0.38user 3.17system 0:03.57elapsed 99%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+70489minor)pagefaults 0swaps
vv$ strace -c /p/p/coreutils-7.5/bin/rm -rf .j
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
66.41 0.081223 0 1000002 1 unlinkat
33.43 0.040887 42 978 getdents
0.16 0.000199 199 1 execve
0.00 0.000000 0 1 read
0.00 0.000000 0 3 open
0.00 0.000000 0 7 close
0.00 0.000000 0 6 fstat
0.00 0.000000 0 1 lstat
0.00 0.000000 0 1 lseek
0.00 0.000000 0 9 mmap
0.00 0.000000 0 3 mprotect
0.00 0.000000 0 1 munmap
0.00 0.000000 0 3 brk
0.00 0.000000 0 1 ioctl
0.00 0.000000 0 1 1 access
0.00 0.000000 0 3 fcntl
0.00 0.000000 0 1 fstatfs
0.00 0.000000 0 1 arch_prctl
0.00 0.000000 0 1 openat
0.00 0.000000 0 1 newfstatat
------ ----------- ----------- --------- --------- ----------------
100.00 0.122309 1001025 2 total
vv$ cd /t; strace -c /cu/src/rm -rf .j
% time seconds usecs/call calls errors syscall
------ ----------- ----------- --------- --------- ----------------
51.26 0.087638 0 1000001 unlinkat
26.49 0.045291 46 976 getdents
22.24 0.038024 18 2134 brk
0.00 0.000000 0 1 read
0.00 0.000000 0 3 open
0.00 0.000000 0 8 close
0.00 0.000000 0 4 fstat
0.00 0.000000 0 1 lstat
0.00 0.000000 0 9 mmap
0.00 0.000000 0 3 mprotect
0.00 0.000000 0 1 munmap
0.00 0.000000 0 1 ioctl
0.00 0.000000 0 1 1 access
0.00 0.000000 0 1 execve
0.00 0.000000 0 3 fcntl
0.00 0.000000 0 1 fstatfs
0.00 0.000000 0 1 arch_prctl
0.00 0.000000 0 1 openat
0.00 0.000000 0 1 newfstatat
------ ----------- ----------- --------- --------- ----------------
100.00 0.170953 1003151 1 total
|
