metadata cache

[Anders F Björklund]

>> It would be possible to do a separate SQL index, that would map  
>> "pkgKey" into "file offset".
>
> Addding an index to the already existing xml also saves having to  
> duplicate all the information.
>
>> Just that ElementTree doesn't help much with this, so it would  
>> need a separate indexing run.
>
> So now it doesn't need to scan the entire file, but it can seek  
> directly to the element start...


Well, at least having an index cuts list time from
30 minutes to 30 seconds. On the other hand, that is
still ridiculously slow compared to the usual 3...
So I don't think the cache size/startup time matters ?

cache size (rpm-md)
  10M    /var/lib/smart/cache (trunk/info)
6.6M    /var/lib/smart/cache (metadata/pkgid)
5.6M    /var/lib/smart/cache (sqlite/pkgid)

nothing (cache load)
         2.91 real         2.39 user         0.50 sys
         2.54 real         2.08 user         0.44 sys
         2.47 real         2.01 user         0.41 sys

query (without info)
         3.56 real         2.80 user         0.58 sys
         3.17 real         2.47 user         0.51 sys
         3.08 real         2.40 user         0.46 sys

query --show-summary (needs info)
         3.81 real         2.97 user         0.60 sys
        27.33 real        23.48 user         3.23 sys
         4.61 real         3.35 user         1.11 sys

Pushing the code to the experimental branch, so you
can compare "trunk" and "metadata" and "sqlite" too.
* https://code.launchpad.net/~afb/smart/metadata
* https://code.launchpad.net/~afb/smart/sqlite

--anders


PS.
Loading xml takes 30 seconds, loading sql takes 3 minutes.
Can probably be improved, if needed (only done on update).