Debtorrent / apt-p2p usage? (n.b. note in 1st post)

[confuseling]

I'd never heard of the thing, and am wondering how many others have?

http://packages.debian.org/sid/debtorrent

Seems like a great way to reduce strain on the servers, particularly for Testing / Sid / mixed users, who if they're like me (and I think they mostly are) probably update their systems more than strictly necessary.

The other thing I've considered doing is setting up a local network cache, so repeat downloads (often happening with a few virtual machines) can just bother my local network, rather than the servers. Something to think about...


[Note: I hadn't heard of apt-p2p either, and should have phrased the questions to include it. Please take them to mean either. They're basically the same thing - apt-p2p is newer and presumably preferred

http://packages.debian.org/wheezy/apt-p2p
]

[vv221]

I don't use it, instead I use apt-p2p, which goals are the same.
Anyway, a peer-to-peer system for APT is a very good idea!

[confuseling]

Ah, indeed... Thanks.

Seems to just be an improved version of debtorrent. So let's take the question to mean: Do you use either?

The poll is set up to allow you to change your answer.

--

Well, I've got it installed... Just not sure it's doing anything. It says it is reachable as a DHT 'node', if that's the word. But upload / download are stubbornly sitting at 0...

I guess time for some sitting and waiting...

[vv221]

Did you modify your sources.list entries to use apt-p2p ?
Installing it is not enough.

[confuseling]

Yes, localhost shows up when I do an 'aptitude update'. If I browse to localhost:9977 it says it's reachable, and at one point the 'Database' section was filling up. It looks like it *should* work, it just doesn't up/download anything.

There are a load of spurious errors in the logs (related to "The value 'testing' is invalid for APT::Default-Release as such a release is not available in the sources", which makes no sense because it is [probably relevant: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=638024 ] )

Does it matter which mirror you're pointed at? My limited understanding of a distributed hash table is that it should make no difference.

Anyway, I'll keep digging...

--

After some investigation, it seems not to work on Testing. I could understand it working less well, but I'm getting literally nothing, which is a shame. I suppose it's a critical mass thing, where it only starts to yield benefit if enough people are running it, and there's an associated cost (it slows things down) so most people aren't going to bother. Oh well... I'll keep fighting the good fight

[vv221]

I suppose connections to port 9977 are not blocked by your firewall ?

Sorry, I had not any problem with apt-p2p, so I don't know how to deal with them…

[confuseling]

I'm guessing not (although I admit, networking hurts my poor little brain).

My router is set to UPNP. The fact that localhost:9977 says it's reachable, combined with the fact that stable VMs work, leads me to believe that I've set it up right, and it just doesn't like testing. I'll keep playing around, see if I can figure something out...

Thanks for the tips anyhow

P.S. nobody voting?

[nadir]

P.S. nobody voting?

Ok, chef. Done (never heard of it).

[vv221]

Here is a HOWTO about how to install and configure apt-p2p :
https://www.isalo.org/wiki.debian-fr/in ... le=Apt-p2p

It's in French, but I think commands are quite understandable…
If you need a translation, I think I could go for it !

[confuseling]

Thanks for the link - the bits I couldn't piece together I google translated.

I take it you're on stable? What sort of figures are you getting for peer upload / download ratio?

Can anyone using Testing / Sid or mixed sources confirm that apt-p2p works for them?

[vv221]

I am on some kind of unstable/experimental mix, with pieces imported from here and there (stable and testing are in my sources.list too)…
Actually I have no idea about how to know apt-p2p statistics (upload, download, ratio, etc.).

[confuseling]

Go to http://localhost:9977/ in your web browser.

[vv221]

You are right : no download/upload from peers…
I will try setting MIN_DOWNLOAD_PEERS to 0, and see if I start downloading form peers.

[confuseling]

Have you set a default release?

Anything in the errors?

Code: Select all

grep exception /var/log/apt-p2p.log

produces page after page here...

[vv221]

Actually I have a 0.28% ratio for peer downloads…

I have set no default release.

No exceptions in my logs, but it is filled with timeouts !

[confuseling]

Actually I have a 0.28% ratio for peer downloads…
...

They will build statues in your honour.

On my stable VM I hit 5% on a good streak, and it's now down to 1.5%. Uploads stuck at 0%.

And on my mixed source real system it still hasn't left the starting gates. Methinks there's something wrong with it. Never mind, it's not like it slows things down enough or uses enough resources to matter significantly - I'll probably leave it on for now and see if it picks up eventually...

---

Aha... There's a port checking tool here;

http://www.canyouseeme.org/

To anyone reading, if you're on apt-p2p's default settings, it should see you on port 9977. It seems I did have a misconfigured router, evil little blighter crashes whenever I try pushing too many buttoons... Whether changing that has helped is another matter though.

[confuseling]

Well, I installed a Ubuntu 12.04 VM to see whether it worked in that, and it does (incidentally, Unity is as bad as everyone says it is... ). Getting approximately 20 - 50% peer downloads. So I'm guessing it is literally just a tipping point where the system starts to work. This bug report seems to indicate that it's near death due in part to lack of users...

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=653244

So to all readers - it's pretty low in system resource usage (you won't see it in top on a modern machine unless you're installing something), and doesn't appreciably slow updates or installs down if it can't download from peers. Consider setting it up on your boxes.

1) You need to adjust your sources.list - check the tutorial above if you can decipher French, or there's http://www.camrdale.org/apt-p2p/install/
2) You get better results with a fixed IP address, and by forwarding TCP and UDP to the port set in /etc/apt-p2p/apt-p2p.conf
3) It seems to work much better on Stable, I'm guessing because the DHT takes a while to discover where data is, and the high churn of Testing / Sid defeats it.

[vv221]

My first uploads !
4.15MiB ! 0.06%…

[confuseling]

Good work! Would probably have been more time-efficient to recite it over the phone in binary, but that's by the by

I'm trying to persuade a local network cache to play nicely with apt-p2p.

Had some luck - it seems that if you set apt-cacher-ng up normally, then install apt-p2p on the same machine, and add

OTHER_DIRS = /var/cache/apt-cacher-ng/

to /etc/apt-p2p/apt-p2p.conf, it'll index everything apt-cacher-ng downloads and add it to the DHT. It doesn't work the other way though - I can't get apt-cacher-ng to download through apt-p2p... Annoying.

On the plus side that's given me a peer upload % of 296102800.00. Dividing by zero for the win!

I wonder whether I could do this with squid or something. I doubt my poor little server could cope - it's getting rather long in the tooth...

[confuseling]

In answer to my own question, yes, you can set squid up for debs fairly easily, though I haven't tried to get it to download through apt-p2p yet. This is the gist of it:

http://itkia.com/using-squid-to-cache-a ... nd-ubuntu/

apt-cacher-ng kept complaining about hash sum mismatches. Might have been me, but it seems a fairly common problem with no clear solution I could find. Hoping to have better luck with squid.

--

Side point - check your settings for logrotate for apt-p2p. Its logs are ridiculously big, as in gigabytes.