Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Memory use disparity between two identical SD cards

Off-Topic discussions about science, technology, and non Debian specific topics.
Post Reply
Message
Author
LoopBack
Posts: 10
Joined: 2019-11-10 16:33

Memory use disparity between two identical SD cards

#1 Post by LoopBack »

I'm using Debian 10.70.
I have two 128GB SDXC cards.
I formatted both with GParted, to the default full size, as Ext2.
For card 1, GParted reported the first sector at 2048, the last sector at 243597311 and the total sectors as 243595264
For card 2, GParted reported the first sector at 2048 the last sector at 243630079 and the total sectors as 243628032
a difference in size of exactly 32768 bytes or 32KiB.

df reports the usable partition space available for card 2 as 4096 of 4KiB blocks larger than for card 1.

Then I used rsync to copy the same 50GiB of data to both of them: 347389 files & 16378 folders.
However:-
Dolphin reports that the memory used is 53,983,883,989 bytes for card 1 and 53,983,834,837 bytes for card 2; a difference of 49152 bytes or 48KiB.
Not much but I had expected that the memory used would be exactly the same for both cards.

I checked that the memory use attributed to the files was the same for both cards;
and so presumably the extra memory use arrises from differing ammounts used for the directory data;
but why is there a difference when the data copied to both cards is the same?

p.H
Global Moderator
Global Moderator
Posts: 3049
Joined: 2017-09-17 07:12
Has thanked: 5 times
Been thanked: 132 times

Re: Memory use disparity between two identical SD cards

#2 Post by p.H »

LoopBack wrote:For card 1, GParted reported the first sector at 2048, the last sector at 243597311 and the total sectors as 243595264
For card 2, GParted reported the first sector at 2048 the last sector at 243630079 and the total sectors as 243628032
a difference in size of exactly 32768 bytes or 32KiB.
More exactly, 32768 sectors of 512 bytes or 16 MiB.
LoopBack wrote:Dolphin reports that the memory used is 53,983,883,989 bytes for card 1 and 53,983,834,837 bytes for card 2; a difference of 49152 bytes or 48KiB.
(...)
and so presumably the extra memory use arrises from differing ammounts used for the directory data;
Or extra filesystem meta-data (inodes and so) due to the bigger filesystem size.

User avatar
craigevil
Posts: 5391
Joined: 2006-09-17 03:17
Location: heaven
Has thanked: 28 times
Been thanked: 39 times

Re: Memory use disparity between two identical SD cards

#3 Post by craigevil »

No two sd cards are exactly the same.
Raspberry PI 400 Distro: Raspberry Pi OS Base: Debian Sid Kernel: 5.15.69-v8+ aarch64 DE: MATE Ram 4GB
Debian - "If you can't apt install something, it isn't useful or doesn't exist"
My Giant Sources.list

LoopBack
Posts: 10
Joined: 2019-11-10 16:33

Re: Memory use disparity between two identical SD cards

#4 Post by LoopBack »

p.h wrote:More exactly, 32768 sectors of 512 bytes or 16 MiB.
Ooops, yes quite, Thanks p.h.
I guess so but I don't have the experience to investigate that further.
craigevil wrote:No two sd cards are exactly the same.
True but I think that the memory driver only sees the partition logically with no knowledge of the underlying physical device.

p.H
Global Moderator
Global Moderator
Posts: 3049
Joined: 2017-09-17 07:12
Has thanked: 5 times
Been thanked: 132 times

Re: Memory use disparity between two identical SD cards

#5 Post by p.H »

You can compare the two filesystems with

Code: Select all

dumpe2fs -h /dev/sdxy

LoopBack
Posts: 10
Joined: 2019-11-10 16:33

Re: Memory use disparity between two identical SD cards

#6 Post by LoopBack »

I have now checked all of the directory sizes for each partition using:-

Code: Select all

find /path-to-card  -type d  -exec  ls -ld --time-style=full-iso  '{}'  ';' 
and looked at the differences given for the bytes used for each directory [alone, not its contents].
{the --full-iso made it easier to pull out the size info for summing}.

Of the 16378 folders 110 had byte allocations that were different between each card.
The differences were either 1 or 2 blocks of 4KiB in size.
Card-1 had 56 that were bigger than card-2 even though it has the smaller partition size
and of these: 12 were 2 blocks larger.
For the larger partition in card-2: just 6 were 2 blocks larger.
p.h wrote:You can compare the two filesystems with

Code: Select all

dumpe2fs -h /dev/sdxy
For card-1 the outputs of that is:-

Code: Select all

Filesystem volume name:   SDS_EXT2
Last mounted on:          /media/user1/SDS_EXT2
Filesystem UUID:          ff8e063d-41f1-44a3-a0ea-b363fb721608
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      ext_attr resize_inode dir_index filetype sparse_super large_file
Filesystem flags:         signed_directory_hash 
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              7618560
Block count:              30449408
Reserved block count:     1522470
Free blocks:              15595804
Free inodes:              7215086
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1016
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
Filesystem created:       Sun Apr  4 12:33:02 2021
Last mount time:          Wed Apr  7 15:34:38 2021
Last write time:          Wed Apr  7 15:34:41 2021
Mount count:              1
Maximum mount count:      -1
Last checked:             Wed Apr  7 15:04:36 2021
Check interval:           0 (<none>)
Lifetime writes:          55 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:	          256
Required extra isize:     32
Desired extra isize:      32
Default directory hash:   half_md4
Directory Hash Seed:      1338f6f4-de6c-4b34-a6d3-107001ccbdcb
and for card-2 it is:-

Code: Select all

Filesystem volume name:   SDS2_EXT2
Last mounted on:          /media/user1/SDS2_EXT2
Filesystem UUID:          110a4cbb-8820-434d-a9ac-a61f226c1d34
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      ext_attr resize_inode dir_index filetype sparse_super large_file
Filesystem flags:         signed_directory_hash 
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              7618560
Block count:              30453504
Reserved block count:     1522675
Free blocks:              15599906
Free inodes:              7215086
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1016
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
Filesystem created:       Sun Apr  4 08:45:09 2021
Last mount time:          Wed Apr  7 15:34:37 2021
Last write time:          Wed Apr  7 15:34:42 2021
Mount count:              1
Maximum mount count:      -1
Last checked:             Wed Apr  7 15:07:57 2021
Check interval:           0 (<none>)
Lifetime writes:          55 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:	          256
Required extra isize:     32
Desired extra isize:      32
Default directory hash:   half_md4
Directory Hash Seed:      34d31303-32bb-4b72-b235-249d7a161b39

Post Reply