Open-iSCSI service fails to mount and umount on boot

[drenriza]

Hi All
I am running Debian 12 and

Code: Select all

ii  open-iscsi                              2.1.8-1                              amd64        iSCSI initiator tools

Target is configured to automatic startup

Code: Select all

/etc/iscsi/nodes/iqn.2004-04.com....\:...\:iscsi.TARGET.56.../192.168.x.x\,3260\,1/default
node.startup = automatic

/etc/fstab is configured with the iSCSI drive

Code: Select all

UUID=89087... /mnt/mount.d ext4 _netdev 0 0

But I have two problems
1) When ever i shutdown my machine, the system starts to wait for ping requests?

Code: Select all

connection:0 ping timeout of 5 secs expired, recv timeout 5, last rx xxxxxxxx, last ping xxxxxxxx, now xxxxxxxxx
Job open-iscsi.service/stop running (1min 0s / 1min 30s)

2) When ever i boot my machine, the iSCSI target fails to get mounted and i manually (after boot) need to run

Code: Select all

sudo mount -a

Boot logs

Code: Select all

            Expecting device dev-disk-by\x2duuid...
[  OK  ] Started iscsid.service - iSCSI initiator daemon (iscsid).
            Starting open-iscsi.service - Login to default iSCSI targets...
[ TIME ] Timed out waiting for device dev-disk-by\x2duuid

I have followed this guide
https://wiki.debian.org/SAN/iSCSI/open-iscsi

Any suggestions as to why the system cant or wont mount and umount the iSCSI drive properly?
I have no problems when the drive is mounted manually.

I have tried to purge open-iscsi and targets, but this did not help.

Thank you in advance
Best regards

[drenriza]

Any and all suggestions as to what the problem can be are welcome.

[Aki]

Hello,

How did you choose the UUID value for the iSCSI initiator disk in the /etc/fstab ?

According to the wiki page you linked in the previous post:

You can then mount the new device, but under most situation, you will want to use LABEL= syntax in /etc/fstab (see e2label).

Code: Select all

    #device         mount point     FS      Options Backup  fsck
    LABEL=data1     /mnt/data1      ext3    _netdev 0       0

Why did you choose to use the UUID instead of the disk label ?

What is the status for open-iscsi.service and iscsid.service when mount fails after boot ? You can check with the following commands:

Code: Select all

sudo journalctl status open-iscsi.service 
sudo journalctl status iscsid.service 

Just a wild guess; perhaps you may also try adding the following options to the /etc/ftab (replace the UUID with the proper value) to ask systemd to start the open-iscsi.service before mounting the iSCSI device.

Code: Select all

UUID=89087... /mnt/mount.d ext4 _netdev,nofail,x-systemd.requires=open-iscsi.service    0   0

Hope this helps

[Aki]

Hello,

Have you got it sorted?

Do you also ask on the debian-user@lists.debian.org mailing list?

• Debian as iSCSI initiator on boot

[drenriza]

Hi Aki
Thanks for your response, please see my answers to your questions below.

Q1: Why did you choose to use the UUID instead of the disk label ?
Answer: I have tried both the UUID and disk label in /etc/fstab but the problem is the same as previously described when i shutdown and start my PC.

Q2: What is the status for open-iscsi.service and iscsid.service when mount fails after boot ?
Answer: sudo journalctl status open-iscsi.service

Code: Select all

open-iscsi.service - Login to default iSCSI targets
     Loaded: loaded (/lib/systemd/system/open-iscsi.service; enabled; preset: enabled)
     Active: active (exited) since Mon 2024-12-09 00:00:00CET; 1min 4s ago
     
Dec 09 00:00:00 PC1 systemd[1]: Starting open-iscsi.service - Login to default iSCSI targets...
Dec 09 00:00:00 PC1 iscsiadm[877]: Logging in to [iface: default, target: iqn.2004-04.com.ENDPOINT, portal: 192.168.x.x,3260]
Dec 09 00:00:00 PC1 iscsiadm[877]: Login to [iface: default, target: iqn.2004-04.com.ENDPOINT, portal: 192.168.x.x,3260] successful.
Dec 09 00:00:00 PC1 systemd[1]: Finished open-iscsi.service - Login to default iSCSI targets.

Q2: What is the status for open-iscsi.service and iscsid.service when mount fails after boot ?
Answer: sudo journalctl -u iscsid.service --since today

Code: Select all

Dec 09 00:01:45 PC1 iscsid[874]: connection-1:0 cannot make a connection to 192.168.x.x:3260 (-1,101) << Tries every 03 seconds until 00:02:42
Dec 09 00:02:42 PC1 iscsid[874]: connection-1:0 cannot make a connection to 192.168.x.x:3260 (-1,101)
Dec 09 00:02:45 PC1 iscsid[874]: connection-1:0 cannot make a connection to 192.168.x.x:3260 (-1,101)
Dec 09 00:02:48 PC1 iscsid[874]: connection-1:0 cannot make a connection to 192.168.x.x:3260 (-1,101)
Dec 09 00:02:51 PC1 iscsid[873]: Connection1:0 to [target: iqn.2004-04.com.ENDPOINT, portal: 192.168.x.x,3260] through [iface: default] is operational now

That the connection is 'operational now' is the last line.

Adjusting my /etc/fstab to also contain the following

Code: Select all

_netdev,nofail,x-systemd.requires=open-iscsi.service

Made no difference to the problem, and also crashed the open-iscsi.service with

Code: Select all

Failed to start open-iscsi.service - Login to default iSCSI targets.

Code: Select all

Dec 09 00:00:00 PC1 iscsid[874]: connection-1:0 cannot make a connection to 192.168.x.x:3260 (-1,101)
Dec 09 00:00:03 PC1 iscsid[874]: Connection-1:0 to [target: iqn.2004-04.com.ENDPOINT, portal: 192.168.x.x,3260] through [iface: default] is shutdown.

But if i manually run the commands after boot

Code: Select all

sudo systemctl restart open-iscsi.service
sudo mount -a

everything works as expected and the drive is mounted.

Thank you in advance
Best regards

[Aki]

Hello,

Q2: What is the status for open-iscsi.service and iscsid.service when mount fails after boot ?
Answer: sudo journalctl -u iscsid.service --since today

Code: Select all

Dec 09 00:01:45 PC1 iscsid[874]: connection-1:0 cannot make a connection to 192.168.x.x:3260 (-1,101) << Tries every 03 seconds until 00:02:42
Dec 09 00:02:42 PC1 iscsid[874]: connection-1:0 cannot make a connection to 192.168.x.x:3260 (-1,101)
Dec 09 00:02:45 PC1 iscsid[874]: connection-1:0 cannot make a connection to 192.168.x.x:3260 (-1,101)
Dec 09 00:02:48 PC1 iscsid[874]: connection-1:0 cannot make a connection to 192.168.x.x:3260 (-1,101)
Dec 09 00:02:51 PC1 iscsid[873]: Connection1:0 to [target: iqn.2004-04.com.ENDPOINT, portal: 192.168.x.x,3260] through [iface: default] is operational now

If I read correctly, the iscsid.service from the initiator (client) took six seconds to connect to the iscsi target (server).

There could be a race condition between:

• the service that make the iscsi disk available as device in the client through the iscsi initiator

• the service that mounts the partitions of the iscsi device in /etc/fstab

You might investigate this.

[drenriza]

Hi Aki
Thanks for your reply.

I am not sure how I would debug that and figure out what a solution would be, as far as I know the /etc/fstab entry is suppose to
eliminate any race conditions by waiting for
1) The network to come up
2) ISCSI login to happen
so the drive can be mounted.

During boot my client logs into the ISCSI target, but it just does not mount the drive.
I found someone having the exact same problem here as myself
https://github.com/open-iscsi/open-iscsi/issues/299

His solution was on his ISCSI target server, but the symptoms are the same and funny enough it also worked fine for him the first time and than it
started to fail afterwards.

It also worked as expected for me in the beginning, but than i changed IP address on my ISCSI target server,
and than ISCSI started giving me this problem after i added the new target, which was the same disk but under a new IP.

This seems like a unnecessary problem, and something to be desired from the implementation.
The Debian ISCSI guide that i followed (link in the beginning) also contains commands that are no longer available in Debian.

[Aki]

Perhaps you can add the x-systemd.device-timeout option to the iscsi disk configuration in /etc/fstab like this (to make systemd wait 30 seconds for the target to be available):

Code: Select all

_netdev,nofail,x-systemd.requires=open-iscsi.service,x-systemd.device-timeout=30s

[Aki]

Hello,

Have you made any progress ?