Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

[SOLVED] apt-get update fails on mounted iso of stretch DVD

Ask for help with issues regarding the Installations of the Debian O/S.
Message
Author
User avatar
gurfle
Posts: 367
Joined: 2009-06-04 02:08
Location: Seattle, Washington, US

[SOLVED] apt-get update fails on mounted iso of stretch DVD

#1 Post by gurfle »

With sources.list containing only this one line

Code: Select all

deb [arch=amd64] file:/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1 stretch main
and the identical official DVD iso file used for successful initial clean install from USB stick mounted with

Code: Select all

mount -o loop /media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1.iso /media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1
here is what happens:

Code: Select all

root@nickslaptop:/home/nick# apt-get update
Get:1 file:/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1 stretch InRelease
Ign:1 file:/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1 stretch InRelease
Get:2 file:/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1 stretch Release [17.8 kB]
Get:2 file:/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1 stretch Release [17.8 kB]
Get:3 file:/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1 stretch Release.gpg
Ign:3 file:/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1 stretch Release.gpg
Get:4 file:/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1 stretch/main amd64 Packages [888 kB]
Ign:4 file:/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1 stretch/main amd64 Packages
Get:4 file:/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1 stretch/main amd64 Packages [888 kB]
Ign:4 file:/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1 stretch/main amd64 Packages
Get:4 file:/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1 stretch/main amd64 Packages [3,766 kB]
Err:4 file:/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1 stretch/main amd64 Packages
  File not found - /media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages (2: No such file or directory)
Reading package lists... Done
N: Download is performed unsandboxed as root as file '/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1/dists/stretch/InRelease' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
W: The repository 'file:/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1 stretch Release' is not signed.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
E: Failed to fetch file:/media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages  File not found - /media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages (2: No such file or directory)
E: Some index files failed to download. They have been ignored, or old ones used instead.
root@nickslaptop:/home/nick# ls -al /media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages*
-r--r--r-- 1 root root 888470 Jun 17 06:12 /media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages.gz
root@nickslaptop:/home/nick# 
As the "ls -al" at the end shows, the official DVD contains the file "Packages.gz", not "Packages", but that can't really be the problem, as the exact same image file works when mounted from a USB stick.
Using "--allow-unauthenticated" option does not help either.
The exact same setup works for me in jessie. What has changed in stretch since jessie that needs to be done to get this to work?

Thanks very much in advance for any insights.



EDIT: Quick followup regarding behaviour in jessie:

sources.list:

Code: Select all

deb [arch=amd64] file:/media/nick/DebianRepositori/debian-8.0.0-amd64-DVD-1 jessie main contrib

Code: Select all

root@debaway:/etc/apt# apt-get update
Ign file: jessie InRelease
Ign file: jessie Release.gpg
Get:1 file: jessie Release [32.5 kB]
Reading package lists... Done 
root@debaway:/etc/apt# ls  -al /media/nick/DebianRepositori/debian-8.0.0-amd64-DVD-1/dists/jessie/main/binary-amd64/Packages*
-r--r--r-- 1 root root 975989 Apr 25  2015 /media/nick/DebianRepositori/debian-8.0.0-amd64-DVD-1/dists/jessie/main/binary-amd64/Packages.gz
root@debaway:/etc/apt# 
This is perfectly straight forward.
Why all the extra output activity and error in stretch under otherwise seemingly identical conditions?



EDIT #2: A bit more investigation reveals:

1) apt-get update on the jessie DVD iso files from stretch fail the same way they do on the stretch DVD files (while of course they do not fail in jessie).

2) apt-get update on the stretch DVD iso files from jessie works.

This indicates to me something has changed in the way "apt-get update" stretch handles locally mounted DVD iso-file repositories which is preventing me from accessing them.

Anybody know what I may be missing here?
Last edited by gurfle on 2017-07-05 17:03, edited 1 time in total.

luvr
Posts: 85
Joined: 2016-07-21 19:39
Location: Boom - The Home Town of Tomorrowland, Belgium

Re: apt-get update fails on mounted iso image of stretch DVD

#2 Post by luvr »

gurfle wrote:Anybody know what I may be missing here?
I seem to have read something about stricter security requirements, enforced by the APT system under the Debian stretch release. If I remember correctly, a repository needs an ‘InRelease’ file, in addition to the ‘Release’ file. I don’t believe that the disc images include such a file.
Last edited by luvr on 2017-07-03 19:37, edited 1 time in total.

User avatar
gurfle
Posts: 367
Joined: 2009-06-04 02:08
Location: Seattle, Washington, US

Re: apt-get update fails on mounted iso image of stretch DVD

#3 Post by gurfle »

Thanks for the insight, luvr.

Does this mean one must rebuild one's own iso image with the added "InRelease" file? This will be a pain. It is not clear to me yet what this file is, much less how to make one. Where can one find out how to go about the process of creating one's own copy of the repos on local media (besides the cumbersome DVDs)?

If what you say is true, it looks like it has become extremely difficult to put one's own copies of the repos on a local hard drive. :(

luvr
Posts: 85
Joined: 2016-07-21 19:39
Location: Boom - The Home Town of Tomorrowland, Belgium

Re: apt-get update fails on mounted iso image of stretch DVD

#4 Post by luvr »

gurfle wrote:It is not clear to me yet what this file is, much less how to make one.
Same here... I do not yet have any further info either.
If what you say is true, it looks like it has become extremely difficult to put one's own copies of the repos on a local hard drive. :(
I'm not entirely sure that my understanding is correct, but if it is, then copying an official repo should not be too problematic, provided that you include the new "InRelease" file.

User avatar
pylkko
Posts: 1802
Joined: 2014-11-06 19:02

Re: apt-get update fails on mounted iso image of stretch DVD

#5 Post by pylkko »

according to https://wiki.debian.org/SourcesList:
If you'd rather use your CD-ROM for installing packages or updating your system automatically with APT, you can put it in your /etc/apt/sources.list. To do so, you can use the apt-cdrom program like this:

Code: Select all

# apt-cdrom add

User avatar
gurfle
Posts: 367
Joined: 2009-06-04 02:08
Location: Seattle, Washington, US

Re: apt-get update fails on mounted iso image of stretch DVD

#6 Post by gurfle »

A partial, but still somewhat awkward solution is to symlink /media/cdrom to the mounted iso file:

Code: Select all

root@oldlaptop:/etc/apt# ls -al /media/cdrom
lrwxrwxrwx 1 root root 72 Jun 30 08:00 /media/cdrom -> /media/nick/Portable02-Black/DebianRepositories/debian-9.0.0-amd64-DVD-1
root@oldlaptop:/etc/apt# cat /etc/apt/sources.list
deb [arch=amd64 allow-insecure=true] cdrom:[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08]/ stretch main
root@oldlaptop:/etc/apt# apt-get update
Ign:1 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch InRelease
Ign:2 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch Release
Hit:3 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main amd64 Packages
Ign:4 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main all Packages
Hit:5 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main Translation-en
Ign:6 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main Translation-en_US
Ign:4 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main all Packages
Ign:3 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main amd64 Packages
Ign:6 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main Translation-en_US
Ign:5 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main Translation-en
Ign:4 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main all Packages
Hit:3 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main amd64 Packages
Ign:6 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main Translation-en_US
Ign:3 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main amd64 Packages
Hit:5 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main Translation-en
Ign:4 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main all Packages
Ign:5 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main Translation-en
Ign:6 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main Translation-en_US
Hit:3 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main amd64 Packages
Ign:4 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main all Packages
Ign:3 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main amd64 Packages
Hit:5 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main Translation-en
Ign:6 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main Translation-en_US
Ign:5 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main Translation-en
Ign:4 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main all Packages
Hit:3 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main amd64 Packages
Ign:6 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main Translation-en_US
Hit:5 cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch/main Translation-en
Reading package lists... Done        
W: The repository 'cdrom://[Debian GNU/Linux 9.0.0 _Stretch_ - Official amd64 DVD Binary-1 20170617-13:08] stretch Release' does not have a Release file.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.
root@oldlaptop:/etc/apt# 
At least this allows using all the original DVD iso images without rebuilding them with InRelease files.

However this procedure is cumbersome, as one must

1) Create the cdrom symlink each time a different DVD is needed by apt.
2) Run apt-get update.
3) Restore the cdrom symlink to /media/cdrom0 when done (assuming you still want your actual CD/DVD reader to be functional).

whereas the "file:" identifier in sources.list allows any number of DVD images to be accessible at the same time.

If this is the result of a "security" improvement, it is hard for me to see its purpose here: apt still has access the original DVD iso file images, just in a an extremely convoluted manner.


EDIT: This script reliably accomplishes the above three steps:

Code: Select all

RepoRoot=/media/nick/Portable02-Black
arch=amd64
if [ $# -lt 1 ]
then
  echo
  echo "Usage:  `basename $0` DVD-number [i386]"
  echo
  exit
fi
if [ x$2 != x ]
then
  arch=$2
fi
if [ ! -L /media/cdromo ]
then
  mv /media/cdrom /media/cdromo
  ln -s /media/cdrom_debian-9.0.0-$arch-DVD-$1 /media/cdrom
  mkdir -p $RepoRoot/DebianRepositories/debian-9.0.0-$arch-DVD-$1
  mount -o loop $RepoRoot/DebianRepositories/debian-9.0.0-$arch-DVD-$1.iso $RepoRoot/DebianRepositories/debian-9.0.0-$arch-DVD-$1
  echo Please wait here until done, then hit the Enter key to restore normal cdrom functionality
  read x
  umount $RepoRoot/DebianRepositories/debian-9.0.0-$arch-DVD-$1
  rm /media/cdrom
  mv /media/cdromo /media/cdrom
else
  echo
  echo Nothing done because /media/cdromo alread exists.
  echo Please fix before proceeding.
  echo
fi
If anybody knows how to make the "file:" identifier in sources.list work for apt in stretch, please post it here, since clearly this "solution" of pretending to be a cdrom is a horrible hack that is fraught with hazards :mrgreen:

luvr
Posts: 85
Joined: 2016-07-21 19:39
Location: Boom - The Home Town of Tomorrowland, Belgium

Re: apt-get update fails on mounted iso image of stretch DVD

#7 Post by luvr »

I haven't tried this, but couldn't you mount each of your DVD images on separate mount points, and use apt-cdrom on each of them?
For instance, let your mount points be, say, /mnt/u/debian1, /mnt/u/debian2, and so on.

You can then mount the DVD images on these mount points:

Code: Select all

mount -o loop,ro PATH_TO_DEBIAN_DVD_IMAGE_1 /mnt/u/debian1
mount -o loop,ro PATH_TO_DEBIAN_DVD_IMAGE_2 /mnt/u/debian2
... and so on. (There's probably a limit to the number of loop mounts supported by the system, so you may not be able to mount all DVD images simultaneously.)

You may have to add these mount points to your /etc/fstab before continuing, but I'm not really sure.

Then use the apt-cdrom command as follows, to add the DVD images to your sources.list:

Code: Select all

apt-cdrom --cdrom /mnt/u/debian1 add
apt-cdrom --cdrom /mnt/u/debian1 add
... and so on.

Disclaimer: I haven't tried this myself, so I cannot guarantee that it will work. If you try it, then I would very much appreciate it if you would want to post the results.
Last edited by luvr on 2017-07-03 04:48, edited 1 time in total.

User avatar
gurfle
Posts: 367
Joined: 2009-06-04 02:08
Location: Seattle, Washington, US

Re: apt-get update fails on mounted iso image of stretch DVD

#8 Post by gurfle »

Interesting luvr . . . Your suggestion does indeed allow "apt-cdrom" to read Package lists from the mounted iso files without linking to /media/cdrom (albeit still with the warning "Data from such a repository can't be authenticated and is therefore potentially dangerous to use.").

However, even then, "apt-get update" continues to require the mounted iso images to be linked to /media/cdrom. I therefore cannot see a way around using a script such as the one I posted. As far as I can tell, the "cdrom:" identifier makes apt-get only read from /media/cdrom, regardless of how apt-cdrom got the package information. Iso image files mounted anywhere else used to be accessible from apt-get with the "file:" identifier, but as of now, no one seems to know how to get that functionality to work in stretch, assuming it is even possible.

Any other ideas?

luvr
Posts: 85
Joined: 2016-07-21 19:39
Location: Boom - The Home Town of Tomorrowland, Belgium

Re: apt-get update fails on mounted iso image of stretch DVD

#9 Post by luvr »

gurfle wrote:However, even then, "apt-get update" continues to require the mounted iso images to be linked to /media/cdrom.
Bah... That’s just silly. Renders the “--cdrom” option on the “apt-cdrom” command pretty useless, in my humble opinion.
Iso image files mounted anywhere else used to be accessible from apt-get with the "file:" identifier, but as of now, no one seems to know how to get that functionality to work in stretch, assuming it is even possible.
I have just done a few experiments, and I got the “file:” URI to work. I can even suppress any warnings or errors about authentication issues. Here’s what I did:
  • Mount the DVD image files, with the “ro” (Read-Only) and “loop” options. To make sure that the images will get mounted automatically whenever the system is booted, add them to your ‘/etc/fstab’ file as follows:

    Code: Select all

    /PATH/TO/FIRST/DVD/IMAGE/FILE/debian-9.0.0-amd64-DVD-1.iso    /MOUNT/POINT/FOR/FIRST/DVD/IMAGE/FILE    udf,iso9660   ro,loop   0   0
    /PATH/TO/SECOND/DVD/IMAGE/FILE/debian-9.0.0-amd64-DVD-2.iso   /MOUNT/POINT/FOR/SECOND/DVD/IMAGE/FILE   udf,iso9660   ro,loop   0   0
    /PATH/TO/THIRD/DVD/IMAGE/FILE/debian-9.0.0-amd64-DVD-3.iso    /MOUNT/POINT/FOR/THIRD/DVD/IMAGE/FILE    udf,iso9660   ro,loop   0   0
    (... and so on, for all image files that you wish to mount ...)
  • Add the following lines to your ‘/etc/apt/sources.list’ file:

    Code: Select all

    deb [trusted=yes]   file:///MOUNT/POINT/FOR/FIRST/DVD/IMAGE/FILE/    stretch   main
    deb [trusted=yes]   file:///MOUNT/POINT/FOR/SECOND/DVD/IMAGE/FILE/   stretch   main
    deb [trusted=yes]   file:///MOUNT/POINT/FOR/THIRD/DVD/IMAGE/FILE/    stretch   main
    (... and so on, for all image files that you wish to use as a software source ...)
    Note that the “file:” prefix is followed by three forward slashes—two slashes follow the “file:” protocol specifier (just as with “http://” or any other protocol), and the third slash is the first character of the full path.
    Furthermore, there’s a terminating slash added to the path as well—to the best of my knowledge, this is traditionally required by APT.
  • Run the “apt update” or “apt-get update” command to update your local software indexes.
    No errors or warnings should be generated, and the image files should subsequently be available as software sources.
Last edited by luvr on 2017-07-03 16:35, edited 1 time in total.

User avatar
gurfle
Posts: 367
Joined: 2009-06-04 02:08
Location: Seattle, Washington, US

Re: apt-get update fails on mounted iso image of stretch DVD

#10 Post by gurfle »

This sounds real promising luvr! I will try it out as soon as I get a chance and report back.

On question, though:

Are you sure you did not accidentally copy fstab stuff into sources.list? It looks like "udf,iso9660 ro,loop 0 0" does not belong there, and instead "main" should be.

luvr
Posts: 85
Joined: 2016-07-21 19:39
Location: Boom - The Home Town of Tomorrowland, Belgium

Re: apt-get update fails on mounted iso image of stretch DVD

#11 Post by luvr »

gurfle wrote:Are you sure you did not accidentally copy fstab stuff into sources.list? It looks like "udf,iso9660 ro,loop 0 0" does not belong there, and instead "main" should be.
Oops! You’re right. Should actually be “stretch main”.
Thanks for catching this “slip of the keyboard”.
I have now corrected it.

User avatar
gurfle
Posts: 367
Joined: 2009-06-04 02:08
Location: Seattle, Washington, US

Re: apt-get update fails on mounted iso image of stretch DVD

#12 Post by gurfle »

Thanks, luvr, for posting how to get apt working for this in stretch.

My problem is then narrowed down to something being strange with the setup of my system (a clean, minimal instalation from DVD with mate as desktop), because here is what happens when I do as you suggest :

Code: Select all

root@oldlaptop:/home/nick# cat /etc/apt/sources.list
deb [arch=amd64 trusted=yes] file:///media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1/ stretch main
root@oldlaptop:/home/nick# 

Code: Select all

root@oldlaptop:/home/nick# tail -1 /etc/fstab
/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1.iso	/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1	udf,iso9660	ro,loop	0	0
root@oldlaptop:/home/nick# 

Code: Select all

oot@oldlaptop:/home/nick# mount /media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1
root@oldlaptop:/home/nick# 

Code: Select all

root@oldlaptop:/home/nick# apt-get update
Get:1 file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1 stretch InRelease
Ign:1 file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1 stretch InRelease
Get:2 file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1 stretch Release [17.8 kB]
Get:2 file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1 stretch Release [17.8 kB]
Get:3 file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1 stretch Release.gpg
Ign:3 file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1 stretch Release.gpg
Get:4 file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1 stretch/main amd64 Packages [888 kB]
Ign:4 file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1 stretch/main amd64 Packages
Get:4 file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1 stretch/main amd64 Packages [888 kB]
Ign:4 file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1 stretch/main amd64 Packages
Get:4 file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1 stretch/main amd64 Packages [3,766 kB]
Err:4 file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1 stretch/main amd64 Packages
  File not found - /media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages (2: No such file or directory)
Reading package lists... Done
N: Download is performed unsandboxed as root as file '/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1/dists/stretch/InRelease' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
E: Failed to fetch file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages  File not found - /media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages (2: No such file or directory)
E: Some index files failed to download. They have been ignored, or old ones used instead.
root@oldlaptop:/home/nick# 

Code: Select all

root@oldlaptop:/home/nick# ls -ald /media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1*
dr-xr-xr-x 1 root root       4096 Jun 17 06:41 /media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1
-rw-r--r-- 1 root root 3804725248 Jun 20 20:33 /media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1.iso
root@oldlaptop:/home/nick# 

Code: Select all

root@oldlaptop:/home/nick# ls -al /media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages*
-r--r--r-- 1 root root 888470 Jun 17 06:12 /media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages.gz
root@oldlaptop:/home/nick# 
Perhaps I have some more restrictive "security" features in place than you?

luvr
Posts: 85
Joined: 2016-07-21 19:39
Location: Boom - The Home Town of Tomorrowland, Belgium

Re: apt-get update fails on mounted iso image of stretch DVD

#13 Post by luvr »

gurfle wrote:Perhaps I have some more restrictive "security" features in place than you?
Hmmm... Strange: When I follow your post (just replacing “nick” with my own username, “luvr”), I get the same problem as you do.
I’ll try to recreate a working setup, and post the exact, working details.

luvr
Posts: 85
Joined: 2016-07-21 19:39
Location: Boom - The Home Town of Tomorrowland, Belgium

Re: apt-get update fails on mounted iso image of stretch DVD

#14 Post by luvr »

My copy of the ‘debian-9.0.0-amd64-DVD-1.iso’ file is in my personal ‘Downloads’ directory:

Code: Select all

$ ls -l /home/luvr/Downloads/debian-9.0.0-amd64-DVD-1.iso
-r--r--r-- 1 luvr luvr 3804708864 Jul  4 15:03 /home/luvr/Downloads/debian-9.0.0-amd64-DVD-1.iso
The mountpoint for the file is in the ‘/mnt’ directory:

Code: Select all

$ ls -ld /mnt/debian-9.0.0-amd64-DVD-1
dr-xr-xr-x 1 root root 4096 Jun 17 15:41 /mnt/debian-9.0.0-amd64-DVD-1
My ‘/etc/fstab’ includes an entry for the image file:

Code: Select all

$ tail --lines=1 /etc/fstab
/home/luvr/Downloads/debian-9.0.0-amd64-DVD-1.iso   /mnt/debian-9.0.0-amd64-DVD-1   udf,iso9660   ro,loop   0   0
My ‘/etc/apt/sources.list’ references the mounted image file:

Code: Select all

$ cat /etc/apt/sources.list 
deb   [trusted=yes]   file:///mnt/debian-9.0.0-amd64-DVD-1/   stretch   main
I mount the image file:

Code: Select all

# mount --verbose /mnt/debian-9.0.0-amd64-DVD-1
mount: /dev/loop0 mounted on /mnt/debian-9.0.0-amd64-DVD-1.
Finally, I run “apt-get update”:

Code: Select all

# apt-get update
Get:1 file:/mnt/debian-9.0.0-amd64-DVD-1 stretch InRelease
Ign:1 file:/mnt/debian-9.0.0-amd64-DVD-1 stretch InRelease
Get:2 file:/mnt/debian-9.0.0-amd64-DVD-1 stretch Release [17.8 kB]
Get:2 file:/mnt/debian-9.0.0-amd64-DVD-1 stretch Release [17.8 kB]
Get:3 file:/mnt/debian-9.0.0-amd64-DVD-1 stretch Release.gpg
Ign:3 file:/mnt/debian-9.0.0-amd64-DVD-1 stretch Release.gpg
Reading package lists... Done
This setup seems to work fine.

I have no idea why your original setup doesn’t work (even when I duplicate it on my system), and this one does.
Must have something to do with some kind of security issue, indeed, but I have no idea exactly what is wrong on that front.

User avatar
gurfle
Posts: 367
Joined: 2009-06-04 02:08
Location: Seattle, Washington, US

Re: apt-get update fails on mounted iso image of stretch DVD

#15 Post by gurfle »

Thanks again for your very thorough diagnostic trace, luvr!

This time following exactly what you did (just replacing “luvr” with “nick”, as you did in reverse) I finally get it to work now!

The only difference of substance to what I was doing before is that before the mount point was on a different partition :?

Since having the repos on a different partition seems like a very good and useful thing, especially for putting them on a portable hard drive, it would be nice to figure out how to get that to work, if at all possible. You say
luvr wrote:When I follow your post (just replacing “nick” with my own username, “luvr”), I get the same problem as you do.
That is very interesting, but more detail would help: For example, did you use a mount point on a different partition also? If not, then that is not the critical factor, but then can you post some more information on just what you did to get it to fail?

For my part, I checked to be sure that permissions and ownership of the two mount points (one on separate partition failing, one on main partition succeeding) were identical, so that can be ruled out. From the error messages, it looks nonetheless like some kind of permission issue, involving the "_apt" user, may be at the core of the problem. This is a feature not present in the version of apt that works for me in jessie, so perhaps something about the new "_apt" user needs to be tweaked?

Code: Select all

Err:4 file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1 stretch/main amd64 Packages
  File not found - /media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages (2: No such file or directory)
Reading package lists... Done
N: Download is performed unsandboxed as root as file '/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1/dists/stretch/InRelease' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
E: Failed to fetch file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages  File not found - /media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages (2: No such file or directory)

luvr
Posts: 85
Joined: 2016-07-21 19:39
Location: Boom - The Home Town of Tomorrowland, Belgium

Re: apt-get update fails on mounted iso image of stretch DVD

#16 Post by luvr »

gurfle wrote:For example, did you use a mount point on a different partition also? If not, then that is not the critical factor, but then can you post some more information on just what you did to get it to fail?
No, everything is one one partition in my case—this Debian system just has a “/” partition, and a swap partition.
From the error messages, it looks nonetheless like some kind of permission issue, involving the "_apt" user, may be at the core of the problem. This is a feature not present in the version of apt that works for me in jessie, so perhaps something about the new "_apt" user needs to be tweaked?
That’s my line of thinking, too. I cannot find much information about this “_apt” user, though, except that it was introduced for tightened security: it allows APT to drop its “root” privileges while downloading from the repositories. I’m puzzled, however, about which authorisations the user does or does not have, and where it gets them from.

Code: Select all

Err:4 file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1 stretch/main amd64 Packages
  File not found - /media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages (2: No such file or directory)
Reading package lists... Done
N: Download is performed unsandboxed as root as file '/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1/dists/stretch/InRelease' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
E: Failed to fetch file:/media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages  File not found - /media/nick/DebianRepositori/debian-9.0.0-amd64-DVD-1/dists/stretch/main/binary-amd64/Packages (2: No such file or directory)
I believe the complaints about the “missing” file are misleading—“missing” the point, so to speak, instead of the file. The problem, in my opinion, is not that the file does not exist, but rather that the “_apt” user does not have access to the directory path leading to the file.

I’m thinking of two further tests that could tell us which authorisation the “_apt” user is actually missing. Starting from the working configuration, we could:
  • Move the mountpoint to the location that it has in the failing configuration, leaving the image file at its working location. My gut feeling tells me that this should work. If I’m right, then the mountpoint is not the problem, but the location of the image file probably is.
  • Leave the mountpoint at its working location, moving the image file to its location in the failing configuration. This is, according to my gut feeling, more likely to fail.
It would be highly awkward if both of these options were successful, because then the “_apt” user would be authorised to access both the mountpoint and the image file, but not simultaneously. Now that would be strange...

Of course, just knowing which authorisation is missing, doesn’t explain why this happens, but at least it will narrow down the search field.

I’m afraid I will have to leave it at that for now. I probably won’t have the opportunity to pick it up again until the weekend.

User avatar
gurfle
Posts: 367
Joined: 2009-06-04 02:08
Location: Seattle, Washington, US

Re: apt-get update fails on mounted iso image of stretch DVD

#17 Post by gurfle »

luvr wrote:Starting from the working configuration, we could:
  • Move the mountpoint to the location that it has in the failing configuration, leaving the image file at its working location. My gut feeling tells me that this should work. If I’m right, then the mountpoint is not the problem, but the location of the image file probably is.
In fact I tried this, and it fails.
[*]Leave the mountpoint at its working location, moving the image file to its location in the failing configuration. This is, according to my gut feeling, more likely to fail.[/list]
This one works!

So it's the reverse of what you were guessing: Failure happens when the mount point is not on the root partition, but the location of the image file seems irrelevant.

However, these test really do not seem to reveal much, since it sounds like you got failure with everything in the same partition as root.

When you get a chance, can you post more details of how you got failure. Since yours did not involve a separate partition, knowing what you did will help narrow down the cause of failure even further:
When I follow your post (just replacing “nick” with my own username, “luvr”), I get the same problem as you do.
Does this mean you just created the exact same file and directory structure? In that case I would guess there is something peculiar with the choice of names or their position within the overall directory structure, since, with everything bing located in the same partition in your case, nothing else is different from the successful setups.

luvr
Posts: 85
Joined: 2016-07-21 19:39
Location: Boom - The Home Town of Tomorrowland, Belgium

Re: apt-get update fails on mounted iso image of stretch DVD

#18 Post by luvr »

gurfle wrote:So it's the reverse of what you were guessing
Hmmm... My gut feeling was wrong, then, obviously.
Does this mean you just created the exact same file and directory structure?
Exactly: my ‘fstab’ entry looked like this:

Code: Select all

/media/luvr/DebianRepositori/debian-9.0.0-amd64-DVD-1.iso   /media/luvr/DebianRepositori/debian-9.0.0-amd64-DVD-1   udf,iso9660   ro,loop   0   0
and my ‘sources.list’ entry was:

Code: Select all

deb [trusted=yes] file:///media/luvr/DebianRepositori/debian-9.0.0-amd64-DVD-1/ stretch main
(I did leave out the “arch=amd64” option, but whether or not I inserted it, didn’t make a difference.)
In that case I would guess there is something peculiar with the choice of names or their position within the overall directory structure, since, with everything bing located in the same partition in your case, nothing else is different from the successful setups.
Ah! While reading this remark, it suddenly hit me that we had been overlooking one glaringly obvious silly little detail: Both the mountpoint and the image file were in the ‘/media/USERNAME’ directory in the failing case. So, I realised that there must be something special about this location, and I decided to take a closer look:

Code: Select all

$ ls -ld /media/luvr
drwxr-x---+ 2 root root 4096 Jul  4 16:34 /media/luvr
At first sight, the owner of this directory—i.e., the ‘root’ user account—seems to have full access to it; its owning group—i.e., the ‘root’ group of users—has read and execute access; and everyone else is refused access to it. If that were the end of the story, then my own user account—i.e., ‘luvr’—would be unable to enter the location, since it does not belong to the ‘root’ group:

Code: Select all

$ id
uid=1000(luvr) gid=1000(luvr) groups=1000(luvr),24(cdrom),25(floppy),27(sudo),29(audio),30(dip),44(video),46(plugdev),50(staff),108(netdev),110(lpadmin),113(scanner),118(bluetooth)
But... there is more to the story: The permission bits are followed by a ‘+’ sign—meaning that the filesystem object (i.e., the directory, in this case) has extended Access Control List entries assigned to it:

Code: Select all

$ getfacl --tabular /media/luvr
getfacl: Removing leading '/' from absolute path names
# file: media/luvr
USER   root      rwx
user   luvr      r-x
GROUP  root      ---
mask             r-x
other            ---
So there you have it: The owning user (i.e., the uppercase ‘USER’ entry) is ‘root’ and has full access. My own user account, ‘luvr’, has read and execute access.
The owning group (i.e., the uppercase ‘GROUP’ entry) is ‘root’, and its members (besides the owning user) will be denied access.
The effective rights mask allows read and execute access (limiting access rights of group entries and of named users—does not apply to the owning user or to the ‘other’ entry, however).
Finally, all other users will be denied any access. And THAT explains why the ‘_apt’ user account runs into a brick wall when it attempts to access the location.

To conclude: The ‘/media/USERNAME’ directory is special, in that the owning ‘root’ user account has full access to it, and that the ‘USERNAME’ account has read and execute rights. All other users are denied any access to the location.

User avatar
gurfle
Posts: 367
Joined: 2009-06-04 02:08
Location: Seattle, Washington, US

Re: apt-get update fails on mounted iso image of stretch DVD

#19 Post by gurfle »

Wow! luvr, your explanation about these subtleties is really great 8)

Access with apt is now possible after doing:

Code: Select all

root@oldlaptop:/home/nick# setfacl -m _apt:rx /media/nick
root@oldlaptop:/home/nick# getfacl --tabular /media/nick
getfacl: Removing leading '/' from absolute path names
# file: media/nick
USER   root      rwx     
user   _apt      r-x     
user   nick      r-x     
GROUP  root      ---     
mask             r-x     
other            ---     

root@oldlaptop:/home/nick# 
Please let me know if this is not an appropriate way to go about accessing debian repository DVD iso files.

The thorough explanations of luvr, not only this last one leading up to the solution, but numerous others along the way of trouble shooting, combined with invaluable patience and persistence, have been extraordinary. I cannot offer enough thanks.

This thread is now marked as solved :P
Last edited by gurfle on 2017-07-28 17:46, edited 1 time in total.

luvr
Posts: 85
Joined: 2016-07-21 19:39
Location: Boom - The Home Town of Tomorrowland, Belgium

Re: apt-get update fails on mounted iso image of stretch DVD

#20 Post by luvr »

gurfle wrote:Please let me know if this is not an appropriate way to go about accessing debian repository DVD iso files.
I believe this is the right way to do it, if you have the ISO files on an external USB disk, and you want to use the image files without hassle after the USB disk gets automounted. I had begun to think about this issue, too, and I arrived at the same solution that you suggest.
I cannot offer enough thanks.
You're welcome! I found it very instructive to research the issue, and I learned a few details along the way myself.

Post Reply