Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

All algorithm on zram occupy same amount RAM?

Here you can discuss every aspect of Debian. Note: not for support requests!
Post Reply
Message
Author
lbuiyzxl
Posts: 66
Joined: 2017-05-11 08:35

All algorithm on zram occupy same amount RAM?

#1 Post by lbuiyzxl »

All algorithm on zram occupy same amount RAM?
Whether should I try report this as bug to Debian BTS?
Environment:

Code: Select all

$ uname -a
Linux debian 4.9.0-3-686-pae #1 SMP Debian 4.9.30-2+deb9u2 (2017-06-26) i686 GNU/Linux
Test result(all " mem_used_total" field is same?):

Code: Select all

$ source ./curscript 
deflate
[deflate]write
536870912 bytes (537 MB, 512 MiB) copied, 14.0555 s, 38.2 MB/s
[deflate]mm_stat
536870912  3276800 16777216        0 16777216        0        0
[deflate](compacted)mm_stat
536870912  3276800 16777216        0 16777216        0        0
[deflate]read
536870912 bytes (537 MB, 512 MiB) copied, 1.72796 s, 311 MB/s
lzo
[lzo]write
536870912 bytes (537 MB, 512 MiB) copied, 2.6149 s, 205 MB/s
[lzo]mm_stat
536870912  5767168 16777216        0 16777216        0        0
[lzo](compacted)mm_stat
536870912  5767168 16777216        0 16777216        0        0
[lzo]read
536870912 bytes (537 MB, 512 MiB) copied, 1.34421 s, 399 MB/s
lz4hc
[lz4hc]write
536870912 bytes (537 MB, 512 MiB) copied, 14.5352 s, 36.9 MB/s
[lz4hc]mm_stat
536870912  3670016 16777216        0 16777216        0        0
[lz4hc](compacted)mm_stat
536870912  3670016 16777216        0 16777216        0        0
[lz4hc]read
536870912 bytes (537 MB, 512 MiB) copied, 1.54674 s, 347 MB/s
lz4
[lz4]write
536870912 bytes (537 MB, 512 MiB) copied, 2.62061 s, 205 MB/s
[lz4]mm_stat
536870912  3670016 16777216        0 16777216        0        0
[lz4](compacted)mm_stat
536870912  3670016 16777216        0 16777216        0        0
[lz4]read
536870912 bytes (537 MB, 512 MiB) copied, 1.5425 s, 348 MB/s
$ 
./curscript:

Code: Select all

test ()
{
algorithm=$1
sudo modprobe -r zram
sudo modprobe zram num_devices=1
echo ${algorithm} | sudo tee /sys/block/zram0/comp_algorithm
echo 512M |sudo tee /sys/block/zram0/disksize >/dev/null
echo \[${algorithm}\]write
yes 12 | sudo dd of=/dev/zram0 |& tail -n 1
echo \[${algorithm}\]mm_stat
cat /sys/block/zram0/mm_stat
echo 1 | sudo tee /sys/block/zram0/compact >/dev/null
echo 1 | sudo tee /proc/sys/vm/compact_memory >/dev/null
sleep 1
echo \[${algorithm}\]\(compacted\)mm_stat
cat /sys/block/zram0/mm_stat
echo \[${algorithm}\]read
sudo dd of=/dev/null if=/dev/zram0 |& tail -n 1
}

test deflate
test lzo
test lz4hc
test lz4
I'm sorry for my bad English skills
I'm don't want to be a nuisance.

lbuiyzxl
Posts: 66
Joined: 2017-05-11 08:35

Re: All algorithm on zram occupy same amount RAM?

#2 Post by lbuiyzxl »

Maybe data in 1F is too rare than real data.
My retest result show lzo's (mem_used_total/orig_data_size) is better(smaller) than lz4.

Wed Aug 23 22:56:55 CST 2017{
Following test can't exact reproduce .
To make a reproducible test result for each algorithm,make enough memory pressure then backup your using swap device to a mirror file and use that mirror file to test your zram device for the compression result.
}
Environment:

Code: Select all

$ uname -a
Linux debian 4.9.0-3-686-pae #1 SMP Debian 4.9.30-2+deb9u2 (2017-06-26) i686 GNU/Linux
Retesting detail:

Code: Select all

$ source maxswaptest 
sudo /etc/init.d/zram stop
real	0m0.599s
sudo /etc/init.d/zram start deflate
real	0m0.111s
time ./a.out 2048 r  1 30
real	0m31.172s
[deflate]mm_stat
469659648 106177535 115027968        0 136069120     2776        8
sudo /etc/init.d/zram stop
real	0m42.506s
sudo /etc/init.d/zram start lzo
real	0m0.527s
time ./a.out 2048 r  1 30
real	0m22.937s
[lzo]mm_stat
684240896 204986478 217817088        0 237568000     3804      344
sudo /etc/init.d/zram stop
real	1m4.671s
sudo /etc/init.d/zram start lz4hc
real	0m0.364s
time ./a.out 2048 r  1 30
real	0m31.008s
[lz4hc]mm_stat
474968064 115749933 124841984        0 140357632     3621     1710
sudo /etc/init.d/zram stop
real	0m49.199s
sudo /etc/init.d/zram start lz4
real	0m0.244s
time ./a.out 2048 r  1 30
real	0m24.244s
[lz4]mm_stat
656371712 198882702 212914176        0 236359680     4013     2876
$ 
maxswaptest:

Code: Select all

test ()
{
echo "sudo /etc/init.d/zram stop"
(time sudo /etc/init.d/zram stop) |& grep '^real'
echo "sudo /etc/init.d/zram start $1"
(time sudo /etc/init.d/zram start $1) |& grep '^real'
echo 'time ./a.out 2048 r  1 30'
(time ./a.out 2048 r  1 30) |& grep '^real'
echo \[$1\]mm_stat
cat /sys/block/zram0/mm_stat
}
test deflate
test lzo
test lz4hc
test lz4
/etc/init.d/zram:

Code: Select all

#!/bin/sh
### BEGIN INIT INFO
# Provides:          zram
# Required-Start:    $local_fs
# Required-Stop:     $local_fs
# Default-Start:     S
# Default-Stop:      0 1 6
# Short-Description: Use compressed RAM as in-memory swap
# Description:       Use compressed RAM as in-memory swap
### END INIT INFO

# Author: Antonio Galea <antonio.galea@gmail.com>
# Thanks to Przemysław Tomczyk for suggesting swapoff parallelization
# Distributed under the GPL version 3 or above, see terms at
#      https://gnu.org/licenses/gpl-3.0.txt

FRACTION=200

MEMORY=`perl -ne'/^MemTotal:\s+(\d+)/ && print $1*1024;' </proc/meminfo`
#CPUS=`grep -c processor /proc/cpuinfo`
#swap_cache_pressure=29
#max_swap_priority=32767
#CPUS=$swap_cache_pressure
CPUS=1
SIZE=$(( MEMORY * FRACTION / 100 / CPUS ))
case "$1" in
  "start")
    echo 100 >/proc/sys/vm/swappiness
    echo 0 >/proc/sys/vm/page-cluster
    echo 0 >/proc/sys/vm/extfrag_threshold
    param=`modinfo zram|grep num_devices|cut -f2 -d:|tr -d ' '`
    modprobe zram $param=$CPUS
    for n in `seq $CPUS`; do
      i=$((n - 1))
      #echo 2 >/sys/block/zram$i/max_comp_streams
      echo $2 >/sys/block/zram$i/comp_algorithm
      echo $SIZE >/sys/block/zram$i/disksize
      mkswap /dev/zram$i
      #swapon /dev/zram$i -d -p $((max_swap_priority-i))
      swapon -p 10 /dev/zram$i
    done
    #echo 32767 >/proc/sys/vm/vfs_cache_pressure
    echo 1 >/proc/sys/vm/compact_memory
    #echo 0 >/proc/sys/vm/min_free_kbytes
    #echo 1 >/proc/sys/vm/lowmem_reserve_ratio
    #echo 0 >/proc/sys/vm/admin_reserve_kbytes
    ;;
  "stop")
    echo 60 >/proc/sys/vm/swappiness
    echo 3 >/proc/sys/vm/page-cluster
    echo 500 >/proc/sys/vm/extfrag_threshold
    #echo 100 >/proc/sys/vm/vfs_cache_pressure
    #echo 42980 >/proc/sys/vm/min_free_kbytes
    #echo 256 >/proc/sys/vm/lowmem_reserve_ratio
    #echo 8192 >/proc/sys/vm/admin_reserve_kbytes
    for n in `seq $CPUS`; do
      i=$((n - 1))
      swapoff /dev/zram$i && echo "disabled disk $n of $CPUS"
    done
    wait
    sleep .5
    modprobe -r zram
    ;;
  "compact")
    for n in `seq $CPUS`; do
      i=$((n - 1))
      echo 1 >/sys/block/zram$i/compact
    done
    echo 1 >/proc/sys/vm/compact_memory
    ;;
  *)
    echo "Usage: `basename $0` (start | stop | compact)"
    exit 1
    ;;
esac
a.out:

Code: Select all

//usage:
//indent ./a.c;gcc -O0 ./a.c
//./a.out max r/w repeat timeout

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <time.h>
int
main (int argc, char **argv)
{
  time_t const start_time = time (NULL);
  time_t timeout;
  int max;
  int repeat;
  if (argc == 5)
    {
      max = atoi (argv[1]);
      repeat = atoi (argv[3]);
      timeout = ((time_t) (atoi (argv[4])));
    }
  else
    return 1;

  unsigned char **block_array =
    calloc (sizeof (unsigned char *), (size_t) (max));

  size_t block_length = (size_t) (1024u * 1024u);

  unsigned char data[3];
  data[0] = 'a';
  data[1] = 'b';
  data[2] = 'c';

  unsigned i = 0u;
  //initialize block_array
  for (i = 0u; i < max; i++)
    {
      do
	{
	  if ((timeout > ((time_t) (0)))
	      && ((time (NULL) - start_time) > timeout))
	    {
	      printf ("%u:timeouted!\n", i);
	      return 0;
	    }
	  block_array[i] = malloc (block_length);
	  if (block_array[i] != NULL)
	    {
	      unsigned bi = 0u;
	      for (bi = 0u; bi < block_length; bi++)
		block_array[i][bi] = data[bi % ((unsigned) (sizeof (data)))];
	    }
	  else
	    {
	      printf ("%u error\n", i);
	    }
	}
      while (NULL == block_array[i]);
    }
  puts ("init ok");
  unsigned score = 0u;
  if (0 == repeat)
    goto show_score;
//do page read test
  if ('r' == argv[2][0])
    for (;;)
      {
	for (i = 0u; i < max; i++)
	  {
	    if ((timeout > ((time_t) (0)))
		&& ((time (NULL) - start_time) > timeout))
	      {
		puts ("timeouted!");
		goto show_score;
	      }

	    unsigned bi = 0u;
	    for (bi = 0u; bi < block_length; bi++)
	      {
		data[bi % ((unsigned) (sizeof (data)))] = block_array[i][bi];
	      }
	    score++;
	  }
	if (repeat > 0)
	  {
	    repeat--;
	    if (0 == repeat)
	      goto show_score;
	  }
      }
//do page write test
  else if ('w' == argv[2][0])
    for (;;)
      {
	for (i = 0u; i < max; i++)
	  {
	    if ((timeout > ((time_t) (0)))
		&& ((time (NULL) - start_time) > timeout))
	      {
		puts ("timeouted!");
		goto show_score;
	      }

	    unsigned bi = 0u;
	    for (bi = 0u; bi < block_length; bi++)
	      {
		block_array[i][bi] = data[bi % ((unsigned) (sizeof (data)))];
	      }
	    score++;
	  }
	if (repeat > 0)
	  {
	    repeat--;
	    if (0 == repeat)
	      goto show_score;
	  }
      }
show_score:
  printf ("score:%u\n", score);
  return 0;
}
I'm sorry for my bad English skills
I'm don't want to be a nuisance.

Post Reply