My own contribution to apt download efficiency
|
| Author | Content |
|---|---|
| Bob_Robertson Oct 31, 2013 12:52 PM EDT |
A long time ago, I opened a feature request bug report for what “apt-fast” does. /etc/apt/sources.list can be populated with multiple archives, added by hand after using apt-spy to find the nearest/fastest archives. So long as “ftp.debian.org” is the last entry in the sources.list, it will be used only if the other archives haven’t updated to the latest package yet. Otherwise, apt will use the first entry in the list that has the package available. This also works with different archives, such as deb-multimedia.org, where packages are unique to the archive. Apt will launch as many downloads in parallel as their are unique packages available on different archives. So one will be pulling from the top of my sources.list entries, and one from deb-multimedia.org, and one from the google chrome archive. But if I’m running an update that wants to download, say, 30 packages, and all 30 have been replicated amongst the 5 main-stream Debian archive entries in my sources.list. Apt now will launch only one download at a time, pulling all 30 packages from the same archive, the first one on my list. Effectively, apt _ignores_ duplicate entries. What I wanted in my feature request was for apt to pull from all 5 “duplicate” entries at once. Not what aria is doing, pulling pieces of the same package from different places, just distribute the downloading of different packages from duplicate archive sites. So of those 30 packages in my example, the first 5 of those packages will be pulled in parallel, one from each of the 5 entries in sources.list. When one completes, the next package that is desired will be pulled from that archive, and so on until all 30 are retrieved. One of those 5 archives might provide only one big package, while another might provide a dozen smaller packages, but regardless the bottleneck becomes my own download speed and the load for file serving is distributed amongst the servers. This uses no new configuration what so ever. Apt already keeps records of what files are available in what archive in sources.list. Thanks for listening. |
Posting in this forum is limited to members of the group: [ForumMods, SITEADMINS, MEMBERS.]
Becoming a member of LXer is easy and free. Join Us!