Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230
dd fails for SD card
dd fails for SD card
I am trying to write a disk image to an SD card using
dd if=./whatever.img of=/dev/sdb
but it returns
dd: writing to '/dev/sdb': No space left on device
Yes, I did umount before calling dd.
The file is only 3.4G, while the card is 16G. In fact, df -h reports that the card has 15G available. The card does work in the sense that I can copy files to/from it after mounting it.
I think there's some issue with the way the SD card is formatted or partitioned, but I don't know enough to guess where the problem lies. Yesterday I called dd in exactly the same way on this card with a different .img file, and it worked fine. Yesterday's call to dd was the first time the SD card had been out of the package. The .img is to flash a Beaglebone Black, and the image flashed as it should once I put the card into the Beaglebone. Now I want to use a different .img file because I need lxqt, which this new .img provides. My latest attempt at a fix was to put the sd card in a Windows machine and reformat it. That did create what seems to be a single 15G partition which allows me to use the SD card as a mountable device, but the dd command still fails.
dd if=./whatever.img of=/dev/sdb
but it returns
dd: writing to '/dev/sdb': No space left on device
Yes, I did umount before calling dd.
The file is only 3.4G, while the card is 16G. In fact, df -h reports that the card has 15G available. The card does work in the sense that I can copy files to/from it after mounting it.
I think there's some issue with the way the SD card is formatted or partitioned, but I don't know enough to guess where the problem lies. Yesterday I called dd in exactly the same way on this card with a different .img file, and it worked fine. Yesterday's call to dd was the first time the SD card had been out of the package. The .img is to flash a Beaglebone Black, and the image flashed as it should once I put the card into the Beaglebone. Now I want to use a different .img file because I need lxqt, which this new .img provides. My latest attempt at a fix was to put the sd card in a Windows machine and reformat it. That did create what seems to be a single 15G partition which allows me to use the SD card as a mountable device, but the dd command still fails.
Re: dd fails for SD card
No. Formatting is never an issue with dd, and if the syntax you report using is the syntax you're actually using, then partitioning is irrelevant.rsfairman wrote:I am trying to write a disk image to an SD card using[snip]Code: Select all
dd if=./whatever.img of=/dev/sdb
I think there's some issue with the way the SD card is formatted or partitioned...
This is potentially interesting, but there are too many "yesterdays" floating around in your report. It would be helpful if you state explicitly:rsfairman wrote:Yesterday I called dd in exactly the same way on this card with a different .img file, and it worked fine.
1) which came first: the failure or the success? (If the success came first, a second test with that same file might prove worthwhile.)
2) the file that succeeded was at least as large as the file that failed, yes? (If not, retest with a big enough file.)
Edit/afterthought: a "big enough file" could come from /dev/zero, which would potentially save you a little time. (That is to say, the test is only trying to see if a particular area of the SD card is writable without error; it doesn't actually matter what you choose to write.)
Last edited by dasein on 2017-03-23 17:05, edited 1 time in total.
Re: dd fails for SD card
Fair enough on the "yesterdays," but it's pretty straightforward. Here's what I did, and which worked.
* Download "bone-debian-8.7-iot-armhf-2017-03-19-4gb.img.xz" and ran unxz on it. Exactly what this is is not relevant to the problem, but it's a disk image to be flashed to a Beaglebone Black (BBB). I'll call this old.img.
* Inserted a 16GB SD card (which I had taken out of the packaging moments before) into a debian jessie laptop, and made sure it was not mounted.
* Ran dd if=./old.img of=/dev/sdb
* Then mounted this SD card on the laptop so that I could manually edit a script which directs the contents of the card to be flashed to the BBB.
* Removed the SD card from laptop and inserted it in the BBB. Powered up BBB, waited maybe 10 minutes for the flash to complete, removed the SD card from BBB, and booted BBB. The new OS is now on the BBB.
So things worked as they should when I used a new SD card -- that is, success came first. The problem arose when I tried to reuse this card with a different disk image. What I did was
* Downloaded "bone-debian-8.7-lxqt-armhf-2017-03-19-4gb.img.xz" and ran unxz on it. Again, the exact contents is not relevant, but it has lxqt and should be similar to old.img otherwise. I'll call this new.img.
* Inserted the same SD card that had old.img on it into the laptop, and made sure it was not mounted.
* Ran dd if=./new.img of=/dev/sdb, but was told
dd: writing to '/dev/sdb': No space left on device
20481+0 records in
20490+0 records out
10495760 (10 MB) copied, 0.0670717 s, 156 MB/s
Actually, that's what it says now if I try the command, but it's said something similar all along. That is, the "no space left on device" message has remained the same. I can't say whether the values given after than have been the same.
* Now I'll be vague since I don't remember the details, but I tried various things with fdisk, parted and gparted, all to no avail.
* I put the SD card into a windows machine and reformatted with diskpart.
* Put the SD card back into linux machine and it mounts, showing 15G free. I tried copying files to/from the card and it works, although I don't know how much of the 15G claimed by df is really usable.
* Nevertheless, the dd command still fails with "No space left on device".
The two .img files are exactly the same size, but I think the new version (with lxqt) is considerably larger once it's unpacked and placed on the card. So I suppose it's possible that this was a bad card along along, and it was barely good enough to handle old.img, but not good enough for new.img. But I am finding that hard to believe since it's a new name brand card (Sandisk), and it's a 16G card, so it could be 75% crap and still get the job done.
Getting to Segfault's question, there *is* something odd here. If I mount the card and do a df, it says that there's 15G free. But, if I do fdisk /dev/sdb, and then 'p', it tells me
Disk /dev/sdb: 10 MiB, 10485760 bytes, 20480 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xca52207f
Device Boot Start End Sectors Size Id Type
/dev/sdb1 * 8192 6963199 6955008 3.3G 83 Linux
which seems odd.
I live in the boonies, and would rather not spend an hour in the car just to get a stupid SD card, but maybe it is a bad card, even though it is brand new.
* Download "bone-debian-8.7-iot-armhf-2017-03-19-4gb.img.xz" and ran unxz on it. Exactly what this is is not relevant to the problem, but it's a disk image to be flashed to a Beaglebone Black (BBB). I'll call this old.img.
* Inserted a 16GB SD card (which I had taken out of the packaging moments before) into a debian jessie laptop, and made sure it was not mounted.
* Ran dd if=./old.img of=/dev/sdb
* Then mounted this SD card on the laptop so that I could manually edit a script which directs the contents of the card to be flashed to the BBB.
* Removed the SD card from laptop and inserted it in the BBB. Powered up BBB, waited maybe 10 minutes for the flash to complete, removed the SD card from BBB, and booted BBB. The new OS is now on the BBB.
So things worked as they should when I used a new SD card -- that is, success came first. The problem arose when I tried to reuse this card with a different disk image. What I did was
* Downloaded "bone-debian-8.7-lxqt-armhf-2017-03-19-4gb.img.xz" and ran unxz on it. Again, the exact contents is not relevant, but it has lxqt and should be similar to old.img otherwise. I'll call this new.img.
* Inserted the same SD card that had old.img on it into the laptop, and made sure it was not mounted.
* Ran dd if=./new.img of=/dev/sdb, but was told
dd: writing to '/dev/sdb': No space left on device
20481+0 records in
20490+0 records out
10495760 (10 MB) copied, 0.0670717 s, 156 MB/s
Actually, that's what it says now if I try the command, but it's said something similar all along. That is, the "no space left on device" message has remained the same. I can't say whether the values given after than have been the same.
* Now I'll be vague since I don't remember the details, but I tried various things with fdisk, parted and gparted, all to no avail.
* I put the SD card into a windows machine and reformatted with diskpart.
* Put the SD card back into linux machine and it mounts, showing 15G free. I tried copying files to/from the card and it works, although I don't know how much of the 15G claimed by df is really usable.
* Nevertheless, the dd command still fails with "No space left on device".
The two .img files are exactly the same size, but I think the new version (with lxqt) is considerably larger once it's unpacked and placed on the card. So I suppose it's possible that this was a bad card along along, and it was barely good enough to handle old.img, but not good enough for new.img. But I am finding that hard to believe since it's a new name brand card (Sandisk), and it's a 16G card, so it could be 75% crap and still get the job done.
Getting to Segfault's question, there *is* something odd here. If I mount the card and do a df, it says that there's 15G free. But, if I do fdisk /dev/sdb, and then 'p', it tells me
Disk /dev/sdb: 10 MiB, 10485760 bytes, 20480 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xca52207f
Device Boot Start End Sectors Size Id Type
/dev/sdb1 * 8192 6963199 6955008 3.3G 83 Linux
which seems odd.
I live in the boonies, and would rather not spend an hour in the car just to get a stupid SD card, but maybe it is a bad card, even though it is brand new.
Re: dd fails for SD card
And when you repeated the test on "old.img"? (That was, after all, the whole point of me asking that question.)rsfairman wrote:...success came first
You can stop thinking about/mucking about with formatting/partitioning/etc. A dd to a device (as opposed to a partition or a file) clobbers the MBR, the filesystem, everything; the destination is basically just raw metal.rsfairman wrote:* I put the SD card into a windows machine and reformatted with diskpart.
Hm. Odd.rsfairman wrote:The two .img files are exactly the same size...
No manufacturing process is perfect.rsfairman wrote:I am finding that hard to believe since it's a new name brand card (Sandisk)
Not with a raw image write.rsfairman wrote:...and it's a 16G card, so it could be 75% crap and still get the job done.
Indeed. A 3.3GB partition on a 10MB device definitely qualifies as odd.rsfairman wrote:which seems odd.Code: Select all
Disk /dev/sdb: 10 MiB, 10485760 bytes, 20480 sectors [snip] Device Boot Start End Sectors Size Id Type /dev/sdb1 * 8192 6963199 6955008 3.3G 83 Linux
Unless you can reproduce your previous success with "old.img", this would seem to be the most likely explanation by far.rsfairman wrote:...maybe it is a bad card, even though it is brand new.
The only other thing I might be tempted to suggest is for you to attempt to zero-fill the SD card just to see what happens. (Use /dev/zero as the "if" for dd)
The next most likely explanation is user error. The dd syntax you've posted looks fine, but that doesn't mean that that's what you typed. If you have a spare thumbdrive sitting around, you could try to reproduce the error using a different medium.
- sunrat
- Administrator
- Posts: 6412
- Joined: 2006-08-29 09:12
- Location: Melbourne, Australia
- Has thanked: 116 times
- Been thanked: 462 times
Re: dd fails for SD card
I've had this problem before and still not sure what the solution was as I too tried various things to fix it (and it was long enough ago that I've forgotten ). I was subsequently able to correctly write the drive. dasein's suggestion to zero fill the card first is worth a try.
replace X with the actual drive letter. It's always best to sync as well, as it often takes a very long time to write a flash device and sync will make sure it's completely written. You know it's done when the command prompt returns.
Code: Select all
dd if=/dev/zero of=/dev/sdX ; sync
“ computer users can be divided into 2 categories:
Those who have lost data
...and those who have not lost data YET ” Remember to BACKUP!
Those who have lost data
...and those who have not lost data YET ” Remember to BACKUP!
Re: dd fails for SD card
This 10 MB is the memory of on-card microprocessor. I'd say this SD card is kaput. Sorry. But you can try and resuscitate it, maybe you get lucky. Try nuking the partition table, see if this bogus 3.3 GB goes away.
Re: dd fails for SD card
i'd have prefered 'sudo fdisk -l', because it shows the complete size of the hardware (sdb), not just the partition (sdb1).rsfairman wrote:Getting to Segfault's question, there *is* something odd here. If I mount the card and do a df, it says that there's 15G free. But, if I do fdisk /dev/sdb, and then 'p', it tells me
Disk /dev/sdb: 10 MiB, 10485760 bytes, 20480 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xca52207f
Device Boot Start End Sectors Size Id Type
/dev/sdb1 * 8192 6963199 6955008 3.3G 83 Linux
which seems odd.
anyhow, i sometimes get the message "action complted, but the kernel will continue to use the old partition table until you reboot" or sth like that.
what i'd do next:
- zero out the drive (not the partition, the whole drive, i.e. sdb in your case?) - the first few megs are sufficient
- reboot
- try again
i also remember that some sd card readers perform less than optimal under linux.
how do you connect the sd card?
Re: dd fails for SD card
This is weird...it works now. My guess is that rebooting fixed it somehow.
I started off trying to address questions people posted (without having rebooted yet). To address dasein's questions...
He wants to know what happens if I try putting old.img onto the sd card. That did work once, but not now:
I put in the sd card, and did
* df to see that the card had been mounted, so
* umount /dev/sdb1
* dd if=./old.img of=/dev/sdb tells me "No space left on device" just as it does with new.img.
He also suggesting trying (if I understood correctly)
dd if=/dev/zero of=/dev/sdb
and that gives exactly the same error. It writes 10MB, and no more.
And debiman wants to know what I'm using as the SD card reader. If he means the driver, then all I can say is that it's whatever is standard for debian 8.7. If he means the physical hardware, it's plugged into the USB port using an adapter. I've never cracked on of these adapters open, but I assume they're nothing but some traces to take the pads of the SD card to the USB receptacle.
At this point, I got out a brand new SD card, assuming that it would work without a hitch. But it didn't, and I got the same error message as with the old card. More out of frustration than for any other reason, I rebooted, and the old card now works.
Clearly, there was some hitch in the chain between dd and the USB port, and rebooting fixed it. As frustrating as this has been, I did learn something: Linux is not perfect. Needing to reboot Linux has always been the absolute last thing on my list, but now I know better.
Thanks,
Randall
I started off trying to address questions people posted (without having rebooted yet). To address dasein's questions...
He wants to know what happens if I try putting old.img onto the sd card. That did work once, but not now:
I put in the sd card, and did
* df to see that the card had been mounted, so
* umount /dev/sdb1
* dd if=./old.img of=/dev/sdb tells me "No space left on device" just as it does with new.img.
He also suggesting trying (if I understood correctly)
dd if=/dev/zero of=/dev/sdb
and that gives exactly the same error. It writes 10MB, and no more.
And debiman wants to know what I'm using as the SD card reader. If he means the driver, then all I can say is that it's whatever is standard for debian 8.7. If he means the physical hardware, it's plugged into the USB port using an adapter. I've never cracked on of these adapters open, but I assume they're nothing but some traces to take the pads of the SD card to the USB receptacle.
At this point, I got out a brand new SD card, assuming that it would work without a hitch. But it didn't, and I got the same error message as with the old card. More out of frustration than for any other reason, I rebooted, and the old card now works.
Clearly, there was some hitch in the chain between dd and the USB port, and rebooting fixed it. As frustrating as this has been, I did learn something: Linux is not perfect. Needing to reboot Linux has always been the absolute last thing on my list, but now I know better.
Thanks,
Randall
Re: dd fails for SD card
Unlikely, particularly since your narrative includes multiple reboots prior to the magic self-repair.rsfairman wrote:My guess is that rebooting fixed it somehow.
It's probably worth stress-testing the gadget prior to any actual use. Once again, I'm going to suggest zero-fill.
Re: dd fails for SD card
No, my narrative does not include multiple reboots. That's my point. It had been several days since the laptop on which I was running the dd command had been rebooted.
The original card works fine so far. I dd-ed the *.img to it, and flashed to the BBB with no issues. In fact, in the last couple of hours, I've dd-ed that SD card twice, and flashed both the no-lxqt version and the with-lxqt version to the BBB.
Of course, if I can't replicate the problem (and I honestly hope that I never do), then we won't know the cause for sure.
The original card works fine so far. I dd-ed the *.img to it, and flashed to the BBB with no issues. In fact, in the last couple of hours, I've dd-ed that SD card twice, and flashed both the no-lxqt version and the with-lxqt version to the BBB.
Of course, if I can't replicate the problem (and I honestly hope that I never do), then we won't know the cause for sure.
Re: dd fails for SD card
I just recreated the problem, and once again rebooting fixed it.
For reasons I do not want to dwell on, I needed to flash another image to the BBB (wifi issues with the blasted thing). So I attempted dd on the same card again, and got the same error. At this point, I thought that maybe there was something wrong with the adapter after all. So I pulled the card out of the adapter, put it back in, took the whole thing out of the laptop, etc., etc., numerous times, and each time I put it into the laptop, I got the same error from dd. Well, I thought it had to be the adapter, so I got a different one, and had the same behavior with the new adapter. I rebooted and the original adapter works fine.
I wasn't doing anything bizarre on this laptop between the time dd worked and the time it failed. I was running emacs, a couple of shells, and I ran the D compiler a few times. I may have run GDB too, but I'm not sure at what point I did that.
I don't really expect any kind of resolution here, but I think it's worth posting for the annals of weird and unexplained behavior.
For reasons I do not want to dwell on, I needed to flash another image to the BBB (wifi issues with the blasted thing). So I attempted dd on the same card again, and got the same error. At this point, I thought that maybe there was something wrong with the adapter after all. So I pulled the card out of the adapter, put it back in, took the whole thing out of the laptop, etc., etc., numerous times, and each time I put it into the laptop, I got the same error from dd. Well, I thought it had to be the adapter, so I got a different one, and had the same behavior with the new adapter. I rebooted and the original adapter works fine.
I wasn't doing anything bizarre on this laptop between the time dd worked and the time it failed. I was running emacs, a couple of shells, and I ran the D compiler a few times. I may have run GDB too, but I'm not sure at what point I did that.
I don't really expect any kind of resolution here, but I think it's worth posting for the annals of weird and unexplained behavior.
Re: dd fails for SD card
my bet is on this.rsfairman wrote:it's plugged into the USB port using an adapter.
it's a little more than just wires, there's some electronics in there and it might even need a driver.
and the cheap ones are, well, flaky.