The Debian Pool Directory

[WRPelfrey]

Hello, I'm new to this forum, I'm Visually, Hearing, and Speech Impaired Severely Disabled US Veteran,

Ok, my question. Is there a way to setup a "POOL" directory of all the ISO images instead of having to keep on swapping discs for some installs?

I have the hard drive space to store all of these files, as I know it would be a lot and huge. When I want to install something, I want apt-get / dpkg to look in this large "POOL" directory for packages, and, if it is not found, then go to the Internet to download the needed *.deb file(s). I don't want to have to play swap the disc in the DVD drive. Also, I don't have the money to burn all of the ISOs to discs. I know how to use jigdo to build the ISOs I need.

I hope I have explained this question properly.

Thanks for your help ahead of time!

WRPelfrey
https://WRPelfrey.com

[GarryRicketson]

This same question pops up from time to time, you do not need all of the
ISO images, nor all of the CD's to install Debian.
https://www.debian.org/CD/faq/#which-cd

Furthermore, in most cases it is not necessary to download all of the images for your architecture. The packages are sorted by popularity: The first CD/DVD/BD contains the installation system and the most popular packages.

====================

I have the hard drive space to store all of these files, as I know it would be a lot and huge. When I want to install something, I want apt-get / dpkg to look in this large "POOL" directory for packages,

Read the Debian documentation, that is what the /etc/apt/sources.list file is for, you edit it to point to where ever you want to install the package from.
So if you actually did have all the files, packages, etc on a hard drive some where, you include the url/location in your sources.list file.
https://wiki.debian.org/SourcesList
==================

To make your own repository, with all the packages, etc. is possible:
How could I create my own mirror of the Debian repositories
==============
https://www.debian.org/mirror/ftpmirror
=================
So to sum it all up, the answer to your question:

Ok, my question. Is there a way to setup a "POOL" directory of all the ISO images instead of having to keep on swapping discs for some installs?

Yes there is a way to do that, even though it is not necessary. All though you did not ask about how , etc,... the above links should help you figure that out
if you decide you must have your own private mirror or (pool directory).
======== edited =====
This is one thread, worded differently but essentially the same thing:
How to mirror Debian repository

[peter_irich]

I prefer to create own repository with reprepro. For this is need to create the repo directory with conf/distributions
and conf/updates files, see "man reprepro". Example it can be taken fro 1st disk/iso. Then command "reprepro update"
creates the full repository copy on your disk.

[GarryRicketson]

Thanks,
here is some links with more details about Reprepro
---------------------------------------------------------------------------------------------
https://packages.debian.org/stretch/reprepro
=====================================


Another method could be :
https://www.pcsuggest.com/how-to-use-an ... in-debian/

[debiman]

Ok, my question. Is there a way to setup a "POOL" directory of all the ISO images instead of having to keep on swapping discs for some installs?

I have the hard drive space to store all of these files, as I know it would be a lot and huge. When I want to install something, I want apt-get / dpkg to look in this large "POOL" directory for packages, and, if it is not found, then go to the Internet to download the needed *.deb file(s). I don't want to have to play swap the disc in the DVD drive. Also, I don't have the money to burn all of the ISOs to discs. I know how to use jigdo to build the ISOs I need.

this is possible (and i'm, sure instructions can be found on debian wiki, or on the forums here).

but please clarify:
do you have a particular reason to not simply install everything from the online repositories?
because that has been the usual way to do things for a long while now.

and welcome to the forums!

[WRPelfrey]

Ok, my question. Is there a way to setup a "POOL" directory of all the ISO images instead of having to keep on swapping discs for some installs?

I have the hard drive space to store all of these files, as I know it would be a lot and huge. When I want to install something, I want apt-get / dpkg to look in this large "POOL" directory for packages, and, if it is not found, then go to the Internet to download the needed *.deb file(s). I don't want to have to play swap the disc in the DVD drive. Also, I don't have the money to burn all of the ISOs to discs. I know how to use jigdo to build the ISOs I need.

this is possible (and i'm, sure instructions can be found on debian wiki, or on the forums here).

but please clarify:
do you have a particular reason to not simply install everything from the online repositories?
because that has been the usual way to do things for a long while now.

and welcome to the forums!

I'm not always in situations that an Internet connection is available.

[WRPelfrey]

I new these responses would be all over the map. Let's clarify some, are the ISO images a complete representation of the amd64, i386, and source POOLS?

If so, then I know how to extract the POOL directories from each disk. I ask these questions as some of the computers that I have taken care of are completely OFF-GRID. They all don't have a optical media drives. So, creating a recent IMAGE of the POOL directory is very beneficial. Also, downing loading the files just once for all computers saves Internet bandwidth and my time. I do believe that the powers to be, DEBIAN, would appreciate me downloading the POOL only once for all of these machines.

This also means that having the knowledge to tell "dpkg" and "apt-get" to use the STORED POOL(s) instead of looking for a non-existent Internet connection. This also means that pooling updates could be helpful also.

Sorry for any confusion, thanks for any help you can provide.

[GarryRicketson]

--- snip-- are the ISO images a complete representation of the amd64, i386, and source POOLS?

No, but there are some if's and but, etc . involved.
If you mean the ones listed here:https://cdimage.debian.org/debian-cd/cu ... 6/iso-dvd/
=========
And here:https://cdimage.debian.org/debian-cd/cu ... 4/iso-dvd/
No these do not contain everything they are not (a complete representation of the amd64, i386, and source POOLS)

And if I understand correctly , (and maybe I don't), but it looks like it says : ( I suggest reading the entire page in the above url, I am only quoting a small piece. )

from:https://cdimage.debian.org/debian-cd/cu ... 4/iso-dvd/
>> Only the first few images are available! Where are the rest?

We don't store/serve the full set of ISO images for all architectures, to reduce the amount of space taken up on the mirrors. You can use the jigdo tool to recreate the missing ISO images instead.

If you look at the jigdo list, https://cdimage.debian.org/debian-cd/cu ... jigdo-dvd/there are 14 dvd's. Compare with the
https://cdimage.debian.org/debian-cd/cu ... 4/iso-dvd/
Where there is only 3 Dvd's.....
I am not sure that even the jigdo 's 14 dvds, include all 55,000 packages that
are available in the Debian repositories, and pools. I guess you would just have to compare, them with the package lists.
https://packages.debian.org/stable/
===== and
https://packages.debian.org/stable/allpackages
============
There is lists of what each ISO image contains here:
https://cdimage.debian.org/debian-cd/cu ... /list-dvd/

I did some calculations, and looked at the contents of some of the images,
and it does appear the jigdo images, all 14 of them would have everything.
So I think they are a complete representation of the amd64, i386, and source POOLS.
(for just the Debian Stable version, )

=====
Additionally, https://www.debian.org/CD/http-ftp/
(again, just a small piece)

Official CD/DVD images of the stable release

To install Debian on a machine without an Internet connection, it's possible to use CD images (650 MB each) or DVD images (4.4 GB each). Download the first CD or DVD image file, write it using a CD/DVD recorder (or a USB stick on i386 and amd64 ports), and then reboot from that.

The first CD/DVD disk contains all the files necessary to install a standard Debian system.
To avoid needless downloads, please do not download other CD or DVD image files unless you know that you need packages on them.

WRPelfrey>>I do believe that the powers to be, DEBIAN, would appreciate me downloading the POOL only once for all of these machines.

Another important point:
From the same:

Note that some mirrors are not up to date — before downloading, check the version number of the images is the same as the one listed on this site! Additionally, note that many sites do not mirror the full set of images (especially the DVD images) due to its size.

Even though they are called "mirrors", not all of them are exactly the same,
Some other members here claim they are, but I have found for example, there are some archives available on some Mexican mirrors, that I could not find on some other mirrors, but that is kind of another topic, sorry.

===
I recommend the jigdo tool as mentioned above, read more about it in this link.
=========
https://www.debian.org/

Debian is a free operating system (OS) for your computer. An operating system is the set of basic programs and utilities that make your computer run.

Debian provides more than a pure OS: it comes with over 51000 packages, precompiled software bundled up in a nice format for easy installation on your machine. Read more...

I don't want to be telling you what is best, or what you should or shouldn't do,
but I think you would save your self a lot of trouble, and unnecessary bloat,complications, etc...
51000 packages is a lot, many are rather silly programs that most people do not need or have any use for, ok maybe not "silly", but "specialized", the point is , I think you would be better off looking at the lists of packages available, and take the time to to give serious thought about exactly which packages you think you need, and want, this may even include a few of the "specialized" packages, but certainly not all of them.

I ask these questions as some of the computers that I have taken care of are completely OFF-GRID. They all don't have a optical media drives. So, creating a recent IMAGE of the POOL directory is very beneficial.

Before going through all the trouble of downloading, 51000 packages,
many of which you will never use, I suggest just installing or trying to install
the first Debian DVD or CD image, and make sure those computers will even be able to run Debian, .... In simple words, first things first, install the basic Debian, if you can, if that works out, and it runs on your computers, then it might be worth considering using the jigdo method, or normal methods, to get
a collection of packages that you really want, and will use.
It is completely pointless to get 50,000 packages, when you do not even know if Debian will run on these computers yet.
Everyone has their own particular needs, and reasons, but I find it hard to grasp or believe any one would need 51000 packages, my system is quite complete with :

Code: Select all

Packages: 396 (pkg_info)  

and many of those I don't really use, some I tried, and didn't work as I had hoped, others were just a 1 time deal. But even if you find 1000 0r 2000 packages you think you want,.... that is so much simpler to work with, then just randomly downloading the entire 50,000 collection, (pool)
Yes, it does take some extra effort to actually read the package lists, and read about what each package does, but at least then you get packages that are of use to you, and if others are involved, ask them to do the same, then put together a list of what packages you all want, concentrate on downloading them , and put them on a storage device, or home brewed mirror (pool),...
Quoting part of:https://www.debian.org/CD/http-ftp/
again:

To avoid needless downloads, please do not download other CD or DVD image files unless you know that you need packages on them.

Any way, good luck with your project.

[WRPelfrey]

WOW! With my eyesight problems, digging through that last post... OUCH!

1. These computers already run debian, ONLY.
2. I am quite used to using "jigdo" on a Linux Internet connected box and I know how to "MAKE" the Windows version of it work when necessary, not an easy task on Windows.
3. I'm not the one using these computers and they always seem to need some strange program(s)/package(s) from most of the disks. One set of computer(s) are networked in an OFF-GRID situation, a very interesting private programming/gaming situation.

I asked for help with "REWIRING" dpkg and apt-get to recognize a POOL directory so that these computers can be hooked up to a large USB POOL drive from time to time for updates and upgrades. Please, I need answers to this question, well, while I also dig further for the answer all by myself.

Thanks for all the help!

[GarryRicketson]

You are welcome,

Ok, well sorry about my previous post. To long. Any way,
Please take a second look at my first reply :
http://forums.debian.net/viewtopic.php? ... 70#p679331

WRPelfrey »I have the hard drive space to store all of these files, as I know it would be a lot and huge. When I want to install something, I want apt-get / dpkg to look in this large "POOL" directory for packages,

GarryRicketson » Read the Debian documentation, that is what the /etc/apt/sources.list file is for, you edit it to point to where ever you want to install the package from.
So if you actually did have all the files, packages, etc on a hard drive some where, you include the url/location in your sources.list file.
https://wiki.debian.org/SourcesList

From the manual: You can link to it here:
https://manpages.debian.org/stretch/apt ... .5.en.html
Or if you do not want to go on line:

Code: Select all

 man sources.list 

DESCRIPTION
The source list /etc/apt/sources.list and the files contained in /etc/apt/sources.list.d/ are designed to support any number of active sources and a variety of source media. The files list one source per line (one-line style) or contain multiline stanzas defining one or more sources per stanza (deb822 style), with the most preferred source listed first (in case a single version is available from more than one source). The information available from the configured sources is acquired by apt-get update (or by an equivalent command from another APT front-end).

=====================

URI SPECIFICATION
The currently recognized URI types are:
file
The file scheme allows an arbitrary directory in the file system to be considered an archive. This is useful for NFS mounts and local mirrors or archives.

=============
There is more explanation, you or someone helping you needs to read
the entire manual to fully understand.

Code: Select all

EXAMPLES
Uses the archive stored locally (or NFS mounted) at /home/apt/debian for stable/main, stable/contrib, and stable/non-free.
deb file:/home/apt/debian stable main contrib non-free
Types: deb
URIs: file:/home/apt/debian
Suites: stable
Components: main contrib non-free

There are some other examples in the complete manual
Since you do not give specific details, as to where you actually have the "pool", it is impossible to give a specific example .

by WRPelfrey » I asked for help with "REWIRING" dpkg and apt-get to recognize a POOL directory

You do NOT REWIRE, dpkg or apt-get. You DO edit and modify your /etc/apt/sources.list file , so that it points apt-get to the pool.
There is some more about "dpkg", you should understand,
see : https://wiki.debian.org/dpkg

dpkg, a medium-level package manager for Debian

With dpkg --set-selections, you can set which packages are to be installed/removed etc, which gets done the next time you run apt-get dselect-upgrade.

Why "medium-level"?

Below it, there's dpkg-deb, which performs lower-level actions on .deb (binary) packages, such as reading the control information and directly extracting the contained files.

dpkg checks dependencies and will refuse to install a package whose dependencies aren't met, but it won't help you find and install those dependencies. You need a higher-level tool (e.g. aptitude, apt-get or dselect) for that.

If you have problems after modifying your sources.list file, please show us exactly what you have in it. And also explain specific details as to where the pool is located, for example, if the pool is on a usb device, or another computer, if it is a network of some sort. Then maybe someone can give a specific example of what the sources.list file should contain.
==== edit==
A important step to remember, when you edit the /etc/apt/sources.list
you MUST run,

Code: Select all

apt update

or

Code: Select all

apt-get update

for the changes to take effect.

Postby WRPelfrey » 2018-08-23 14:16

WOW! With my eyesight problems, digging through that last post... OUCH!
---snip---
Please, I need answers to this question, well, while I also dig further for the answer all by myself.

I can not be there, to read the manuals and documentation for you, I do sympathize, and understand the difficulty with bad eyesight. I also have very poor eye site, and it is getting worse, what I do is have my wife help me, and she reads it out loud, for me at times. How ever, since my hearing also is very bad, and I have to ask her to keep repeating what she reads, well, she loses patience, and I resort to using the biggest fonts I can and dig through the information by myself.

[GarryRicketson]

I am sorry, I forgot to mention, there should also be a file in /apt ,
it is named "apt.conf " See :
https://manpages.debian.org/stretch/apt ... .5.en.html
For details. Or "offline",

Code: Select all

 man apt.conf 

There are examples in:

In general the sample configuration file /usr/share/doc/apt/examples/configure-index.gz is a good guide for how it should look.

I suggest making a copy of your original , default "apt.conf", before making any changes in that.
Again, if you tell us some details about where your pool is, or will be located, then I think some one might be able to help you better then me. I am not very good at explaining things.
I have a old Debian 2 VM, and it does not have internet access, so similar to your situation, I have to down load the packages, and put them in a directory, then I point the sources.list file to that directory, it could also be a directory on another device or HD, the key being in making the apt program look there, and connect to that directory.
Apology if my explanations are not easy to read.

[debiman]

Ok, my question. Is there a way to setup a "POOL" directory of all the ISO images instead of having to keep on swapping discs for some installs?

I have the hard drive space to store all of these files, as I know it would be a lot and huge. When I want to install something, I want apt-get / dpkg to look in this large "POOL" directory for packages

if it is not found, then go to the Internet to download the needed *.deb file(s).

I have separated your original post into two quotes.
I believe the first part is well possible. on hard drive, no dvds.
It would take me some digging into debian resources (i usually just prepend web searches with the term "debian", navigating the debian universe's sites is too difficult for me) to give you a step-by-step, i hope you understand that i won't do all of that - but you seem capable yourself.

However, (still the first part) when you do this, your software pool is locked to a particular snapshot in time, and that's where the second part is tricky, because those other softwares would be the newest version, and compatibility is not guaranteed.
equally, i think it would be difficult to tell apt/dpkg to sometimes use the internet, sometimes not.