Some RPM performance stats

It’s awfully (or should I say thankfully) easy to forget how bad things were just a year ago when you live on the git-snapshot-of-the-day edge… I recently had to give some, hmm, palliative care to RPM 4.4.x and was just astounded by how slow it really was. To get an idea how things have progressed during the last year, below is a comparison of RPM 4.4.x, 4.6.x, 4.7.x and git HEAD performing a test install of 2776 packages from Fedora 10 DVD to an empty chroot on my T60. The time is best of three consecutive runs for each version, so the packages are hot in cache:

RPM 4.4.x

[root@localhost rpm-4.4.x]# time ./rpmi -ivh –test –root /home/test/ –nosignature ~pmatilai/tmp/f10-fixed-all.txt
Preparing…                ########################################### [100%]

real    3m5.113s
user    3m3.250s
sys    0m1.202s

RPM 4.6.X

[root@localhost rpm-4.6.x]# time ./rpm -ivh –test –root /home/test/ –nosignature ~pmatilai/tmp/f10-fixed-all.txt
Preparing…                ########################################### [100%]

real    0m22.203s
user    0m21.225s
sys    0m0.955s

RPM 4.7.x

[root@localhost rpm-4.7.x]# time ./rpm -ivh –test –root /home/test/ –nosignature ~pmatilai/tmp/f10-fixed-all.txt
Preparing…                ########################################### [100%]

real    0m21.674s
user    0m20.649s
sys    0m0.890s

RPM.org HEAD

[root@localhost rpm]# time ./rpm -ivh –test –root /home/test/ –nosignature ~pmatilai/tmp/f10-fixed-all.txt
Preparing…                ########################################### [100%]

real    0m7.552s
user    0m6.648s
sys    0m0.847s

We’ve come from 3min 5sec to ~7 seconds for the same operation, no wonder 4.4.x felt like a snail trashing about in a tar pit. It should be noted that the above relatively simple case of installing into an empty chroot doesn’t do justice to 4.7.x at all – to see the effects of the improvements there, one needs to look at upgrade/erasure cases and associated memory use. Lets try an erase of the same package set as above, with a small patch to print out memory use at end of “Preparing…” stage:

RPM 4.6.x

[root@localhost rpm-4.6.x]# time ./rpm -e –nosignature –test –root /home/test/ `rpm -qa –root /home/test/`
VmRSS:      156208 kB

real    5m13.749s
user    4m10.267s
sys    1m2.776s

RPM 4.7.x

[root@localhost rpm-4.7.x]# time ./rpm -e –nosignature –test –root /home/test/ `rpm -qa –root /home/test/`
VmRSS:       88220 kB

real    0m14.224s
user    0m12.242s
sys    0m1.928s

RPM.org HEAD

[root@localhost rpm]# time ./rpm -e –nosignature –test –root /home/test/ `rpm -qa –root /home/test/`
VmRSS:       88156 kB

real    0m14.837s
user    0m12.846s
sys    0m1.937s

From 5min 13s down to ~14 seconds for the erase case, with memory usage almost halved. In case you wonder why HEAD is a bit slower than 4.7.x: it’s because erasures are now properly ordered too, that has some overhead. I’m not bothering to compare 4.4.x here as it’d be practically the same as 4.6.x behavior, except being even worse due to crazy memory fragmentation which strikes in some corner cases. Also the above case doesn’t come even close to triggering the worst case behavior of 4.4.x and 4.6.x where it can easily eat gigabytes of memory, which can also mean it’s impossible to erase (not to mention upgrade) a package which you installed.

In case you’re curious where the speed advantages come from, it’s all about switching to hash tables where appropriate and in some cases improving existing hashing functions. Everything else is just minor details, which are starting to show up now that we’re talking about a few seconds instead of minutes. And to give credit where credit is due, the most performance work in rpm by far was done by Florian Festi, cheers :)

Leave a Reply

You must be logged in to post a comment.