External USB drive is mounted as read only file system

[Germinx]

Hello.

My external USB drive is mounted as a read only file system and if I try to safely remove the device using nautilus the system hangs up.
e2fsck reports that the disk has erros but e2fsck can't fix it.
This drive was working fine until the last update and I don't know why this is happening now.
smartctl reports that the device lacks SMART capability so I can't use this for diagnose.

If I plug the drive in other machine that has not been updated yet e2fsck reports that the disk has errors but it can fix it.
The drive works fine and it can be safely removed. But when I plug the disk to the updated machine I have again the same errors.

I've read in the forum that recently others can't safely remove disks but I don't know if this will be related or not.

Drive information:

Code: Select all

#> fdisk -l
Disk /dev/sdc: 931.5 GiB, 1000204883968 bytes, 1953525164 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xa4e92792

Device     Boot     Start        End    Sectors   Size Id Type
/dev/sdc1            2048   20973567   20971520    10G 83 Linux
/dev/sdc2        20973568  125831167  104857600    50G 83 Linux
/dev/sdc3       125831168 1953523711 1827692544 871.5G 83 Linux

Drive information:

Code: Select all

># smartctl -i /dev/sdc3
smartctl 6.6 2016-05-31 r4324 [x86_64-linux-4.9.0-8-amd64] (local build)
Copyright (C) 2002-16, Bruce Allen, Christian Franke, www.smartmontools.org

=== START OF INFORMATION SECTION ===
Vendor:               TOSHIBA
Product:              External USB 3.0
Revision:             5438
Compliance:           SPC-4
User Capacity:        1,000,204,883,968 bytes [1.00 TB]
Logical block size:   512 bytes
Serial number:        *****
Device type:          disk
Local Time is:        Sun Dec  2 13:18:48 2018 CET
SMART support is:     Unavailable - device lacks SMART capability.

The system that is causing problems is Debian GNU/Linux 9 (stretch):

Code: Select all

Linux ocean 4.9.0-8-amd64 #1 SMP Debian 4.9.130-2 (2018-10-27) x86_64 GNU/Linux

e2fsck output:

Code: Select all

root@ocean:~# e2fsck /dev/sdc3
e2fsck 1.43.4 (31-Jan-2017)
TPART3: recovering journal
e2fsck: unable to set superblock flags on TPART3


TPART3: ********** WARNING: Filesystem still has errors **********

The system that is working fine is Debian GNU/Linux 9 (stretch) (NOT UPDATED YET):

Code: Select all

Linux sky 4.9.0-8-rt-amd64 #1 SMP PREEMPT RT Debian 4.9.110-3+deb9u4 (2018-08-21) x86_64 GNU/Linux

e2fsck output from the old machine:

Code: Select all

root@sky:~# e2fsck /dev/sdc3
e2fsck 1.43.4 (31-Jan-2017)
TPART3 contiene un sistema de ficheros con errores, se fuerza la revisión.
Paso 1: Verificando nodos-i, bloques y tamaños
Paso 2: Verificando la estructura de directorios
Paso 3: Revisando la conectividad de directorios
Paso 4: Revisando las cuentas de referencia
Paso 5: Revisando el resumen de información de grupos
TPART3: 302622/57122816 ficheros (0.4% no contiguos), 92100616/228461568 bloques

e2fsck says that the file file system has errors and e2fsck fix the errors. (I'm sorry but I can't install the English locales in that machine because I dont want to update that system.)

e2fsck output from the old machine running e2fsck again:

Code: Select all

root@sky:~# e2fsck /dev/sdc3
e2fsck 1.43.4 (31-Jan-2017)
TPART3: limpio, 302622/57122816 ficheros, 92100616/228461568 bloques

It says that it is clean but I'm getting errors if I plug the supposed clean drive to the updated machine.

I don't know what is happening here.
Any clue will be appreciated, thanks in advance and excuse me for my poor English I hope you can understand me.

[Head_on_a_Stick]

I can't install the English locales in that machine because I dont want to update that system

Prefix the commands with

Code: Select all

LC_ALL=C

to make them "speak" English

Do you get the same errors if you drop to a console with

Code: Select all

# systemctl isolate multi-user.target

then attempt to `mount` & `umount` the USB drive from the TTY?

[Germinx]

I can't install the English locales in that machine because I dont want to update that system

Prefix the commands with

Code: Select all

LC_ALL=C

to make them "speak" English

Do you get the same errors if you drop to a console with

Code: Select all

# systemctl isolate multi-user.target

then attempt to `mount` & `umount` the USB drive from the TTY?

Yes in "run level 3" mounting and unmounting from the TTY I have the same issue.
https://drive.google.com/file/d/1rc-Q0f ... sp=sharing

Is it possible that the disk is broken but for some reason the old machine can't detect it?

Here are some outputs from dmesg:

Code: Select all

[  141.080905] usb 2-8: new SuperSpeed USB device number 2 using xhci_hcd
[  141.103397] usb 2-8: New USB device found, idVendor=0480, idProduct=a202
[  141.103402] usb 2-8: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[  141.103407] usb 2-8: Product: External USB 3.0
[  141.103410] usb 2-8: Manufacturer: TOSHIBA
[  141.103414] usb 2-8: SerialNumber: ****
[  141.114877] usb-storage 2-8:1.0: USB Mass Storage device detected
[  141.115099] scsi host6: usb-storage 2-8:1.0
[  141.115146] usbcore: registered new interface driver usb-storage
[  141.115974] usbcore: registered new interface driver uas
[  143.988026] scsi 6:0:0:0: Direct-Access     TOSHIBA  External USB 3.0 5438 PQ: 0 ANSI: 6
[  143.989000] sd 6:0:0:0: Attached scsi generic sg2 type 0
[  143.990525] sd 6:0:0:0: [sdc] 1953525164 512-byte logical blocks: (1.00 TB/932 GiB)
[  143.990934] sd 6:0:0:0: [sdc] Write Protect is off
[  143.990943] sd 6:0:0:0: [sdc] Mode Sense: 23 00 00 00
[  143.991388] sd 6:0:0:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[  144.000622]  sdc: sdc1 sdc2 sdc3
[  144.012571] sd 6:0:0:0: [sdc] Attached SCSI disk
[  144.546277] EXT4-fs (sdc1): warning: mounting fs with errors, running e2fsck is recommended
[  144.553532] EXT4-fs (sdc1): mounted filesystem with ordered data mode. Opts: (null)
[  144.946763] EXT4-fs (sdc2): mounted filesystem with ordered data mode. Opts: (null)
[  145.124460] usb 2-8: reset SuperSpeed USB device number 2 using xhci_hcd
[  145.303118] EXT4-fs (sdc3): mounted filesystem with ordered data mode. Opts: (null)
[  150.867344] usb 2-8: reset SuperSpeed USB device number 2 using xhci_hcd
[  151.023796] usb 2-8: reset SuperSpeed USB device number 2 using xhci_hcd
[  151.308310] usb 2-8: reset SuperSpeed USB device number 2 using xhci_hcd
[  151.464689] usb 2-8: reset SuperSpeed USB device number 2 using xhci_hcd
[  151.620985] usb 2-8: reset SuperSpeed USB device number 2 using xhci_hcd
[  151.777338] usb 2-8: reset SuperSpeed USB device number 2 using xhci_hcd
[  151.933665] usb 2-8: reset SuperSpeed USB device number 2 using xhci_hcd
[  152.089984] usb 2-8: reset SuperSpeed USB device number 2 using xhci_hcd
[  152.113419] sd 6:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK
[  152.113426] sd 6:0:0:0: [sdc] tag#0 CDB: Write(10) 2a 00 3d c4 08 08 00 01 88 00
[  152.113431] blk_update_request: I/O error, dev sdc, sector 1036257288
[  152.113550] Aborting journal on device sdc3-8.

Code: Select all

[  912.111401] usb 2-8: reset SuperSpeed USB device number 5 using xhci_hcd
[  912.267573] usb 2-8: reset SuperSpeed USB device number 5 using xhci_hcd
[  912.423611] usb 2-8: reset SuperSpeed USB device number 5 using xhci_hcd
[  912.579667] usb 2-8: reset SuperSpeed USB device number 5 using xhci_hcd
[  912.971831] usb 2-8: reset SuperSpeed USB device number 5 using xhci_hcd
[  913.127883] usb 2-8: reset SuperSpeed USB device number 5 using xhci_hcd
[  913.283933] usb 2-8: reset SuperSpeed USB device number 5 using xhci_hcd
[  913.439994] usb 2-8: reset SuperSpeed USB device number 5 using xhci_hcd
[  913.600061] usb 2-8: reset SuperSpeed USB device number 5 using xhci_hcd
[  913.756201] usb 2-8: reset SuperSpeed USB device number 5 using xhci_hcd
[  913.779760] sd 6:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_OK
[  913.779767] sd 6:0:0:0: [sdc] tag#0 CDB: Write(10) 2a 00 07 80 08 08 00 01 b8 00
[  913.779772] blk_update_request: I/O error, dev sdc, sector 125831176
[  913.779780] buffer_io_error: 45 callbacks suppressed
[  913.779783] Buffer I/O error on dev sdc3, logical block 1, lost async page write
[  913.779796] Buffer I/O error on dev sdc3, logical block 2, lost async page write
[  913.779801] Buffer I/O error on dev sdc3, logical block 3, lost async page write
[  913.779806] Buffer I/O error on dev sdc3, logical block 4, lost async page write
[  913.779811] Buffer I/O error on dev sdc3, logical block 5, lost async page write
[  913.779832] Buffer I/O error on dev sdc3, logical block 6, lost async page write
[  913.779847] Buffer I/O error on dev sdc3, logical block 7, lost async page write
[  913.779852] Buffer I/O error on dev sdc3, logical block 8, lost async page write
[  913.779861] Buffer I/O error on dev sdc3, logical block 9, lost async page write
[  913.779867] Buffer I/O error on dev sdc3, logical block 10, lost async page write
[  913.900080] usb 2-8: reset SuperSpeed USB device number 5 using xhci_hcd
[  914.056227] usb 2-8: reset SuperSpeed USB device number 5 using xhci_hcd
[  914.212306] usb 2-8: reset SuperSpeed USB device number 5 using xhci_hcd
[  914.608481] usb 2-8: reset SuperSpeed USB device number 5 using xhci_hcd
[  914.779911] VFS: Dirty inode writeback failed for block device sdc3 (err=-5).

blk_update_request: I/O error, dev sdc, sector 1036257288

And thanks a lot for the locale tip!

[bw123]

...
This drive was working fine until the last update and I don't know why this is happening now.
...

good clue, what/how did you update? I don;t use googledrive because javascript disabled, is that link the output of the mount command? Do you get an error?

[Germinx]

I always use synaptic. All the packages are from stable and they are up to date and there are no broken packages or missed dependencies.

[Head_on_a_Stick]

the disk is broken

^ This.

You could double-check using a recent "live" ISO image, I suppose.

[Germinx]

I'll try a live and if that does not work I think that I'm going to buy a new one, copy all the data and try to format this.

Thanks a lot for your answers.

[llivv]

First thing I would do is backup sdc3 files on the old machine

If you choose to do the backup
run e2fsck on the old machine to make sure the filesystem is clean before making the backup.

don't update the old machine

and don't use the new machine until you are sure you have all the data you want from the new machine.
be patient
while we get a better handle on what you are telling us about your setup.

[Germinx]

Yes that will be my workflow.
I will run e2fsck and make the backup with the old machine. And then I'll try to format the disk to see if the error still appears.

I would like to apologize if I seemed impatient.
I was a bit nervous because this disk was my backup.

I really appreciate the work of the entire community that is around Debian.
I started using it 15 or 16 years ago and it has been the only system I have been using for many many years and this is the first time I have a problem.
I think it's a great statistic for an OS, one error in 16 years.. or even better.... zero errors... if the disk is broken

[llivv]

no worries - I know how frustrating it can be when the unexpected happens and how impossible it can be to find some pointers.
If you want to backup and than overwrite the misbehaving filesystem that is the easy way to proceed.

I just tried to help another user with a similar ext4 issue just recently.
I'll link to that thread so you can read some ext4 filesystem debugging techniques to try at your leisure if you like.
If they don't work at least you will have some new experience with e2fsck options for future reference

http://forums.debian.net/viewtopic.php?f=10&t=139106

edit: after looking at those 2 page again and rechecking the e2fsck e2undo mke2fs man pages
I'd suggest after the first backup
to make a second backup using

e2fsck

Code: Select all

/bin/umount /dev/sdc3

find a folder or partiton with enough free space to hold another backup file and run

Code: Select all

/sbin/e2fsck -z e2undo-sdc3-file /dev/sdc3 

I'd also suggest you read the e2fsck man page to to see how the -z option is explained there.

Another command you can run to find the superblock backup locations that may help later

Code: Select all

 /sbin/mke2fs -n /dev/sda3

again read the mke2fs man page to see how they explain when to use the -n option
[
I'll assume the defaults were used to make the ext4 filesystems originally
ie: mke2fs -t ext4 /dev/sdc3
or mkfs.ext4 /dev/sdc3
as I typically do
]

here is what the output looks like when I ran the command
take notes of the superblock backup locations after running the command.

Code: Select all

me@c10:/# /sbin/mke2fs -n /dev/sdb10
[mke2fs 1.44.4 (18-Aug-2018)
/dev/sdb10 contains a ext4 file system
last mounted on / on Sat Nov 24 18:51:03 2018
Proceed anyway? (y,N) y
Creating filesystem with 3905795 4k blocks and 977280 inodes
Filesystem UUID: abcdefgh-ijkl-mnop-qrstuvwxyzAB
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208

And if you have any questions about fixing the filesystem I'll try to keep my eye on this thread and (answer or try to help) if I can.

[Germinx]

One thing I still don't understand is why one machine says that the file system is clean and the other says it is not. Both should say the same as far as I understand. I don't really know what is going on.
So... Which computer should I use to make the backup? The one that says that all is fine or the other? One is not working properly and now I don't know which it is.
I'll read all the documentation carefully and I'll try to do my best, but I'll take a long time to give you an answer because I want to buy two new drives to make two different backups before proceeding to make any change.
When I have a solution I'll post it in the case if it serves as a help to someone else.

Thanks a lot for your help!!!

[llivv]

One thing I still don't understand is why one machine says that the file system is clean and the other says it is not. Both should say the same as far as I understand. I don't really know what is going on.
So... Which computer should I use to make the backup? The one that says that all is fine or the other? One is not working properly and now I don't know which it is.

I hope I didn't push you into extra work or extra expense for new disks.
Not enough space to make two backups. one is way better than none.

tar and bzip2 can be used together to make a backup.tar.bz archive that is much smaller than the full sized backup. I noticed sdc3 is your biggest partition.
Assumed you might have enough extra space - Sorry if I overstepped.

I'm assuming the old machine you can mount the USB disk on is the working machine
and the new machine you upgraded and can no longer mount the USB Disk on is the misbehaving box.
Does that make sense?
Not sure why the disk is misbehaving when plugged into the new machine.

I also believe that running e2fsck -z e2undo-sdc3
will take up less space than the actual size of the data it's backing up on /dev/sdc3
I'll run some more tests here to comfirm

[llivv]

I filled a 12Gb partition with 11Gb of misc file types and made backup using
e2fsck -z e2undo-sdc8
and
e2image -Q /dev/sdc8 sdc8.qcow2
the commands are taken almost word for word from the manuals

Code: Select all

@b12:~# /sbin/e2image -Q /dev/sdc8 sdc8.qcow2
e2image 1.43.4 (31-Jan-2017)

@b12:~# /usr/bin/du -h sdc8.qcow2
7.1M    sdc8.qcow2

@b12:~# /bin/bzip2 -z sdc8.qcow2

@b12:~$ /bin/ls
archives bin e2undo-sdc8 sdc8.qcow2.bz2

Code: Select all

@b12:~$ /bin/du -h sdc8.qcow2.bz2
196K    sdc8.qcow2.bz2

@b12:~$ /bin/du -h e2undo-sdc8
20K     e2undo-sdc8

@b12:~# /bin/mount /dev/sdc8 /mnt
[ 2970.149683] EXT4-fs (sdc8): mounted filesystem with ordered data mode. Opts: (null)

Code: Select all

@b12:~$ /bin/df -hlT
Filesystem     Type      Size  Used Avail Use% Mounted on
udev           devtmpfs  970M     0  970M   0% /dev
tmpfs          tmpfs     197M  624K  196M   1% /run
/dev/sdb12     xfs        15G  4.7G   11G  32% /
tmpfs          tmpfs     5.0M  4.0K  5.0M   1% /run/lock
tmpfs          tmpfs     393M     0  393M   0% /run/shm
/dev/sdc8      ext4       12G   11G  628M  95% /mnt

I'll test the restore / replay features later