Memory use disparity between two identical SD cards

If it doesn't relate to Debian, but you still want to share it, please do it here

Memory use disparity between two identical SD cards

Postby LoopBack » 2021-04-05 15:13

I'm using Debian 10.70.
I have two 128GB SDXC cards.
I formatted both with GParted, to the default full size, as Ext2.
For card 1, GParted reported the first sector at 2048, the last sector at 243597311 and the total sectors as 243595264
For card 2, GParted reported the first sector at 2048 the last sector at 243630079 and the total sectors as 243628032
a difference in size of exactly 32768 bytes or 32KiB.

df reports the usable partition space available for card 2 as 4096 of 4KiB blocks larger than for card 1.

Then I used rsync to copy the same 50GiB of data to both of them: 347389 files & 16378 folders.
However:-
Dolphin reports that the memory used is 53,983,883,989 bytes for card 1 and 53,983,834,837 bytes for card 2; a difference of 49152 bytes or 48KiB.
Not much but I had expected that the memory used would be exactly the same for both cards.

I checked that the memory use attributed to the files was the same for both cards;
and so presumably the extra memory use arrises from differing ammounts used for the directory data;
but why is there a difference when the data copied to both cards is the same?
LoopBack
 
Posts: 9
Joined: 2019-11-10 16:33

Re: Memory use disparity between two identical SD cards

Postby p.H » 2021-04-05 16:40

LoopBack wrote:For card 1, GParted reported the first sector at 2048, the last sector at 243597311 and the total sectors as 243595264
For card 2, GParted reported the first sector at 2048 the last sector at 243630079 and the total sectors as 243628032
a difference in size of exactly 32768 bytes or 32KiB.

More exactly, 32768 sectors of 512 bytes or 16 MiB.

LoopBack wrote:Dolphin reports that the memory used is 53,983,883,989 bytes for card 1 and 53,983,834,837 bytes for card 2; a difference of 49152 bytes or 48KiB.
(...)
and so presumably the extra memory use arrises from differing ammounts used for the directory data;

Or extra filesystem meta-data (inodes and so) due to the bigger filesystem size.
p.H
 
Posts: 1739
Joined: 2017-09-17 07:12

Re: Memory use disparity between two identical SD cards

Postby craigevil » 2021-04-06 06:58

No two sd cards are exactly the same.
Raspberry PI 400 Distro: Raspberry Pi OS (Unstable) Kernel: 5.10.25-v8+ aarch64 DE: LXDE Ram 4GB
Debian - "If you can't apt-get something, it isn't useful or doesn't exist"
User avatar
craigevil
 
Posts: 5212
Joined: 2006-09-17 03:17
Location: Oz

Re: Memory use disparity between two identical SD cards

Postby LoopBack » 2021-04-06 07:41

p.h wrote:More exactly, 32768 sectors of 512 bytes or 16 MiB.

Ooops, yes quite, Thanks p.h.
I guess so but I don't have the experience to investigate that further.
craigevil wrote:No two sd cards are exactly the same.

True but I think that the memory driver only sees the partition logically with no knowledge of the underlying physical device.
LoopBack
 
Posts: 9
Joined: 2019-11-10 16:33

Re: Memory use disparity between two identical SD cards

Postby p.H » 2021-04-06 12:55

You can compare the two filesystems with
Code: Select all
dumpe2fs -h /dev/sdxy
p.H
 
Posts: 1739
Joined: 2017-09-17 07:12

Re: Memory use disparity between two identical SD cards

Postby LoopBack » 2021-04-07 14:09

I have now checked all of the directory sizes for each partition using:-

Code: Select all
find /path-to-card  -type d  -exec  ls -ld --time-style=full-iso  '{}'  ';'

and looked at the differences given for the bytes used for each directory [alone, not its contents].
{the --full-iso made it easier to pull out the size info for summing}.

Of the 16378 folders 110 had byte allocations that were different between each card.
The differences were either 1 or 2 blocks of 4KiB in size.
Card-1 had 56 that were bigger than card-2 even though it has the smaller partition size
and of these: 12 were 2 blocks larger.
For the larger partition in card-2: just 6 were 2 blocks larger.

p.h wrote:You can compare the two filesystems with
Code: Select all
dumpe2fs -h /dev/sdxy

For card-1 the outputs of that is:-

Code: Select all
Filesystem volume name:   SDS_EXT2
Last mounted on:          /media/user1/SDS_EXT2
Filesystem UUID:          ff8e063d-41f1-44a3-a0ea-b363fb721608
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      ext_attr resize_inode dir_index filetype sparse_super large_file
Filesystem flags:         signed_directory_hash
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              7618560
Block count:              30449408
Reserved block count:     1522470
Free blocks:              15595804
Free inodes:              7215086
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1016
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
Filesystem created:       Sun Apr  4 12:33:02 2021
Last mount time:          Wed Apr  7 15:34:38 2021
Last write time:          Wed Apr  7 15:34:41 2021
Mount count:              1
Maximum mount count:      -1
Last checked:             Wed Apr  7 15:04:36 2021
Check interval:           0 (<none>)
Lifetime writes:          55 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:             256
Required extra isize:     32
Desired extra isize:      32
Default directory hash:   half_md4
Directory Hash Seed:      1338f6f4-de6c-4b34-a6d3-107001ccbdcb


and for card-2 it is:-

Code: Select all
Filesystem volume name:   SDS2_EXT2
Last mounted on:          /media/user1/SDS2_EXT2
Filesystem UUID:          110a4cbb-8820-434d-a9ac-a61f226c1d34
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      ext_attr resize_inode dir_index filetype sparse_super large_file
Filesystem flags:         signed_directory_hash
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              7618560
Block count:              30453504
Reserved block count:     1522675
Free blocks:              15599906
Free inodes:              7215086
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1016
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         8192
Inode blocks per group:   512
Filesystem created:       Sun Apr  4 08:45:09 2021
Last mount time:          Wed Apr  7 15:34:37 2021
Last write time:          Wed Apr  7 15:34:42 2021
Mount count:              1
Maximum mount count:      -1
Last checked:             Wed Apr  7 15:07:57 2021
Check interval:           0 (<none>)
Lifetime writes:          55 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:             256
Required extra isize:     32
Desired extra isize:      32
Default directory hash:   half_md4
Directory Hash Seed:      34d31303-32bb-4b72-b235-249d7a161b39
LoopBack
 
Posts: 9
Joined: 2019-11-10 16:33


Return to Offtopic

Who is online

Users browsing this forum: No registered users and 14 guests

fashionable