[PATCH] rpm_md Metadata vs. Mirroring

[Michael Jennings]

We recently discovered that "smart update" hitting mirrors might not
be a good idea.  Mirrors can (and do) lag behind the master site(s) to
varying degrees, and while pulling metadata which is a day or so old
(e.g., from a nightly mirror) isn't always a big deal, sometimes it's
highly undesireable (e.g., urgent security updates).

Since the goal of "smart update" is to get the most up-to-date
information available, it seems reasonable to get this directly from
the primary server rather than a mirror.

The attached patch does exactly that for rpm_md repositories.  (I
didn't want to touch the others because I can't test them.)  "smart
update" will always pull from the master, though mirror servers will
be consulted to download packages.  (And, as a consequence of the
mirror penalization structure, mirrors which are perpetually
out-of-date will incur higher penalties.)

I didn't make it a config option, but I can take a stab at doing so if
there's interest.  Feedback is encouraged. :-)

Michael

-- 
Michael Jennings (a.k.a. KainX)  http://www.kainx.org/  <mej at kainx.org>
Linux Server/Cluster Admin, LBL.gov       Author, Eterm (www.eterm.org)
-----------------------------------------------------------------------
 "Narrow is the road and too high a price to pay when loneliness is
  such a sanctuary.  Empty are the musings and wasted are the days
  when you say you were only waiting."
                                  -- Jars of Clay, "Famous Last Words"
-------------- next part --------------
diff -Nur -x '*.orig' -x '*.rej' smart-0.52/smart/channels/rpm_md.py mezzanine_patched_smart-0.52/smart/channels/rpm_md.py
--- smart-0.52/smart/channels/rpm_md.py	2007-08-07 09:56:15 -0700
+++ mezzanine_patched_smart-0.52/smart/channels/rpm_md.py	2008-06-11 18:22:55 -0700
@@ -59,6 +59,7 @@
         
         fetcher.reset()
         repomd = posixpath.join(self._baseurl, "repodata/repomd.xml")
+        fetcher.setUseMirror(0)
         item = fetcher.enqueue(repomd)
         fetcher.run(progress=progress)
 
@@ -103,6 +104,7 @@
                            "metadata for '%s'") % self
 
         fetcher.reset()
+        fetcher.setUseMirror(0)
         item = fetcher.enqueue(info["primary"]["url"],
                                md5=info["primary"].get("md5"),
                                uncomp_md5=info["primary"].get("uncomp_md5"),
@@ -116,7 +118,8 @@
                                  uncomp_sha=info["filelists"].get("uncomp_sha"),
                                  uncomp=True)
         fetcher.run(progress=progress)
- 
+        fetcher.setUseMirror(1)
+
         if item.getStatus() == SUCCEEDED and flitem.getStatus() == SUCCEEDED:
             localpath = item.getTargetPath()
             filelistspath = flitem.getTargetPath()
diff -Nur -x '*.orig' -x '*.rej' smart-0.52/smart/fetcher.py mezzanine_patched_smart-0.52/smart/fetcher.py
--- smart-0.52/smart/fetcher.py	2007-08-07 09:56:15 -0700
+++ mezzanine_patched_smart-0.52/smart/fetcher.py	2008-06-11 11:47:54 -0700
@@ -66,12 +66,14 @@
         self._activedownloads = 0
         self._activedownloadslock = thread.allocate_lock()
         self._maxactivedownloads = 0
+        self._usemirror = 1
         self.time = 0
         self._eta = 0
 
     def reset(self):
         self._items.clear()
         self._uncompressing = 0
+        self._usemirror = 1
 
     def cancel(self):
         self._cancel = True
@@ -105,6 +107,12 @@
     def getMirrorSystem(self):
         return self._mirrorsystem
 
+    def getUseMirror(self):
+        return self._usemirror
+
+    def setUseMirror(self, value):
+        self._usemirror = value
+
     def getCaching(self):
         return self._caching
 
@@ -166,7 +174,10 @@
     def enqueue(self, url, **info):
         if url in self._items:
             raise Error, _("%s is already in the queue") % url
-        mirror = self._mirrorsystem.get(url)
+        if self._usemirror:
+            mirror = self._mirrorsystem.get(url)
+        else:
+            mirror = self._mirrorsystem.getUnmirrored(url)
         item = FetchItem(self, url, mirror)
         self._items[url] = item
         if info:
diff -Nur -x '*.orig' -x '*.rej' smart-0.52/smart/mirror.py mezzanine_patched_smart-0.52/smart/mirror.py
--- smart-0.52/smart/mirror.py	2007-08-07 09:56:15 -0700
+++ mezzanine_patched_smart-0.52/smart/mirror.py	2008-06-11 11:39:18 -0700
@@ -81,6 +81,9 @@
             elements = [MirrorElement(self, "", "")]
         return MirrorItem(self, url, elements)
 
+    def getUnmirrored(self, url): 
+        return MirrorItem(self, url, [MirrorElement(self, "", "")])
+
     def getPenalities(self):
         self.updatePenality()
         return self._penality