how to back up data and impact on the environment

If none of the more specific forums is the right place to ask

how to back up data and impact on the environment

Postby MagicPoulp » 2018-11-20 12:02

At my job, most people use Ubuntu. I use Debian stable. Because they are afraid to lose data or time, they buy new computers very often much before the previous computers would break. This has a strong environmental impact if all companies proceed similarly. For a company it is not in the culture to suddenly have a computer that breaks and require reinstallation. THey also buy extra laptops to make sure there is not secret data on their disk when they travel abroad.

This culture of no simple way to back up was spread by Microsoft Windows, probably because they wanted to sell more computers. It is well known that after some time, due to many corruptions and installations and startup scripts, Windows becomes super slow. Not because the hardware is deprecated, but because of Windows that has aged badly. This reminds the scandal with phones that were programmed to become slower after the warranty, to push selling more phones.

If we focus on Ubuntu and Debian, it is still not very clear how to efficiently backup. The state of Debian indirectly impacts Ubuntu and other Debian-based OSes and a large community of people. On a large scale, there is an impact on the environment. People often think their Ubuntu is slow to start or that the Unity GUI is laggy. This is often due to Software more than hardware, and due to graphical driver conflicts. It is very common on Linux to have graphics driver conflicts. Even when using the standard nouveau graphics driver, it is common to have laggy interfaces and tearing in web browsers. I have such tearing in my browser myself. It is related to the topic because lagginess makes people think their computer is broken or old.

There are 2 things to back up:
- the data, the files. This can be done manually using a cloud storage, a network, a USB stick.
- The system installation with all the programs definition, configuration, etc. This is not simple to backup. I tried a few times with dd and rsync to copies the partition image. I was not able to install it on a formated partition.
MagicPoulp
 
Posts: 49
Joined: 2018-11-05 21:30


Re: how to back up data and impact on the environment

Postby MagicPoulp » 2018-11-20 12:13

Here is a lazydog I made to recover from a copy of debian files. I basically copy all the files in /, and I setup grub. But I was not satisfied. There are always small details that change over times or that need more configuration.

Code: Select all
IMPORTANT: the last / in rsync is needed

--> Backup:
start:
thierry@small:~/Documents$ sudo su -
root@small:/root# mkdir /media/thierry/ZZZ
root@small:/root# mount /dev/sdc1 /media/thierry/ZZZ

mkdir /media/thierry/ZZZ/slash
time rsync -ahPHAXx --delete --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found} / /media/thierry/ZZZ/slash/ >/dev/null
or a simple version:
sudo rsync -ahPHAXx --delete --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found} / /media/thierry/ZZZ/slash/

----------------------------------------------------------------------------------------------------------------------
--> Install

--> prepare partition space
disable secure boot in BIOS
enable CSM and disalble the windows default
shrink windows partition using gparted on debian live

--> install the backup:
- start Debian on a live CD or USB stick
- shrink windows partition using gparted on debian live
- format the partition using gparted
need ext4 file system, and need 2 partitions, with 20 GB for swap

- mount the partition to install on
sudo fdisk -l
mount /dev/sdaX /destination

- mount the usb stick
USE USB 3.0 that is faster
sudo fdisk -l
mkdir /media/usb1
mount /dev/sdbX /media/usb1

- copy the data to the partition

mkdir /mnt/slash
mount /dev/sda7 /mnt/slash
sudo rsync -ahPHAXx /media/usb1/slash/ /mnt/slash/ >/dev/null
the progress can be shown using df -H

--> reload /etc/fstab

Change fstab on target system
the fstab only contains the linux partitions and the boot partition (efi)

As root, edit /mnt/slash/etc/fstab
For each partition (including swap), replace the first field with the new UUID or LABEL (it’s straightforward with LABELs):
UUID=the-long-uuid, or LABEL=yourlabel

two ways to get the UUIDs / LABELs:

ls -l /dev/disk/by-uuid/
or try blkid /dev/sdaX


--> install grub for uefi
if it does not work, try as follows:
https://wiki.debian.org/GrubEFIReinstall

REPLACE sdX by the DISK name like only an a, IT IS NOT the partition that Linux is installed on

for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt/slash$i; done
sudo chroot /mnt/slash
grub-mkconfig -o /boot/grub/grub.cfg
sudo grub-install --recheck /dev/sdX   <---------- replace the X
sudo update-grub


--> check the result of the grub installation
# lsblk -no NAME,UUID /dev/sdbX

where you substitute the desired partition for /dev/sdbX To list the UUIDs of partitions grub thinks it can boot, use grep:

# grep UUID= /boot/grub/grub.cfg


-------------------------
best link:
http://superuser.com/questions/709176/how-to-best-clone-a-running-system-to-a-new-harddisk-using-rsync

another tutorial but less pro
http://positon.org/clone-a-linux-system-install-to-another-computer

to check other things:
https://wiki.archlinux.org/index.php/full_system_backup_with_rsync

grub:
http://howtoubuntu.org/how-to-repair-restore-reinstall-grub-2-with-a-ubuntu-live-cd
MagicPoulp
 
Posts: 49
Joined: 2018-11-05 21:30

Re: how to back up data and impact on the environment

Postby MagicPoulp » 2018-11-20 12:17

arochester wrote:https://dflinux.frama.io/thebeginnershandbook/lcdd09-backup/#backing-up-with-deja-dup


Nice simple software for the large amount of people. Personally, I prefer to backup manually on USB stick. I always have small things I actively know I want to save. So I get a smaller bundle to save.

I think the main problem lies in backing up the system installation.
MagicPoulp
 
Posts: 49
Joined: 2018-11-05 21:30

Re: how to back up data and impact on the environment

Postby arochester » 2018-11-20 13:02

I think the main problem lies in backing up the system installation.


http://freesoftwaremagazine.com/article ... _computer/ ?
arochester
 
Posts: 1349
Joined: 2010-12-07 19:55

Re: how to back up data and impact on the environment

Postby debiman » 2018-11-20 18:39

i think it's a huge stretch to say people are buying new hardware all the time because they're too lazy to do backups (or because according to you there is no simple solution).
or maybe i simply didn't understand what you are saying.
i will grant you that the industry is pushing a mindset where the OS and the device are the same thing, and for most people a broken operating system or broken hardware is the same thing.
and that electronic devices are way too cheap these days.
the chinese are getting us by the balls, and one day they will come to the conclusion that their grip is now firm enough, and own us all.

btw, i use borg for backups.

MagicPoulp wrote:I use Debian stable.

*cough*
not according to your other posts.
User avatar
debiman
 
Posts: 3064
Joined: 2013-03-12 07:18

Re: how to back up data and impact on the environment

Postby stevepusser » 2018-11-20 22:06

*cough*
not according to your other posts.


The old "You f*** one goat, and what does everyone call you?!?" story, eh?
The MX Linux repositories: Backports galore! If we don't have something, just ask and we'll try--we like challenges. New packages: Flightgear 2018.2.2, 4.19.5 kernel, wine-staging 4.0~rc1, Pale Moon 28.2.2, Mesa 18.2.6, Midori 7.0
User avatar
stevepusser
 
Posts: 10278
Joined: 2009-10-06 05:53

Re: how to back up data and impact on the environment

Postby sunrat » 2018-11-20 23:26

That story had a sheep last I heard it.

Clonezilla works well for disk or partition backups. And they have restored perfectly the couple of times I have needed them.
“ computer users can be divided into 2 categories:
Those who have lost data
...and those who have not lost data YET ”
Remember to BACKUP!
User avatar
sunrat
 
Posts: 2495
Joined: 2006-08-29 09:12
Location: Melbourne, Australia

Re: how to back up data and impact on the environment

Postby MagicPoulp » 2018-11-21 08:53

debiman wrote:i think it's a huge stretch to say people are buying new hardware all the time because they're too lazy to do backups (or because according to you there is no simple solution).
or maybe i simply didn't understand what you are saying.


The policy in my company is that everyone gets a new computer every 2 or 3 years. I think this is wrong. One person should get a new computer when the previous computer stops working.'

There is a reason why there is such a policy. It takes a lot of time to set up a computer and install programs. Moreover, they are scared people will lose precious data. Many people do their work without having their work backed up daily.

In general, in most companies I have been to, people do not care much about the environment. They generalize the use of airplanes and laptops (toxic battery, wasted screen, short lifetime). Bosses have cars paid tax free by the companies. All employees get a new smartphone tax free quite often starting when new employed. People leave their computer without sleep mode all the time even during nights. They buy super powerful computers that they do not really need just because it is feels like sitting in a ferrari. And those guys are certified engineers, with strong background in the sciences, very high purchase potential, and so on. Still with an environmental IQ of 1 digit.
MagicPoulp
 
Posts: 49
Joined: 2018-11-05 21:30

Re: how to back up data and impact on the environment

Postby MagicPoulp » 2018-11-21 09:00

sunrat wrote:Clonezilla works well for disk or partition backups. And they have restored perfectly the couple of times I have needed them.


That looks like a nice program. Does the saved data take the same space as the partition that is backed up?

By using rsync, I had the advantage of not having to save the entire partition size.

One I tried this:
- start on debian live CV to have a working linux.
- use gparted to foramt my partition
- use dd to copy the binary of my disk taht was itself back up using the command dd.
- set up grub since partition configs have changed.
It did not work, because the grub install was too hard to configure. But it seemed really nice to have since one could reinstall a system very fast without a full manual installation.

So now I install debian, and then I copy files with rsync. It has the advantage of taking less space that a complete partition save.
MagicPoulp
 
Posts: 49
Joined: 2018-11-05 21:30

Re: how to back up data and impact on the environment

Postby sunrat » 2018-11-21 11:03

MagicPoulp wrote:
sunrat wrote:Clonezilla works well for disk or partition backups. And they have restored perfectly the couple of times I have needed them.


That looks like a nice program. Does the saved data take the same space as the partition that is backed up?


It only uses the space of the data. Unless you use the compression option and then it takes less.

I use Clonezilla to back up the system partition before doing anything potentially harmful like large software upgrades. For data backup of home, I use rsnapshot with a systemd timer. rsync is fine for that too, it's what rsnapshot uses as its backend.
“ computer users can be divided into 2 categories:
Those who have lost data
...and those who have not lost data YET ”
Remember to BACKUP!
User avatar
sunrat
 
Posts: 2495
Joined: 2006-08-29 09:12
Location: Melbourne, Australia

Re: how to back up data and impact on the environment

Postby llivv » 2018-11-21 14:02

MagicPoulp wrote:The policy in my company is that everyone gets a new computer every 2 or 3 years. I think this is wrong. One person should get a new computer when the previous computer stops working.'

There is a reason why there is such a policy. It takes a lot of time to set up a computer and install programs. Moreover, they are scared people will lose precious data. Many people do their work without having their work backed up daily.

In general, in most companies I have been to, people do not care much about the environment. They generalize the use of airplanes and laptops (toxic battery, wasted screen, short lifetime). Bosses have cars paid tax free by the companies. All employees get a new smartphone tax free quite often starting when new employed. People leave their computer without sleep mode all the time even during nights. They buy super powerful computers that they do not really need just because it is feels like sitting in a ferrari. And those guys are certified engineers, with strong background in the sciences, very high purchase potential, and so on. Still with an environmental IQ of 1 digit.

My comment regrading carbon footprint is that all of us are guilty of the waste our system produces since all of us benefit from the gross products. It wasn't too long ago when IIRC, there was a green movement of consciencious individuals who wanted to make changes to policy.
The futility of consciencious environmental policy has evolved into the only thing left that is green is $
produce more - make more $ - consume more - - - or be cast out. And it's a cold existence out there. much like the warming of the planet as the globle population grows and policy is ill equipped to balance human needs agaisnt human advances ....
Last edited by llivv on 2018-11-21 14:11, edited 1 time in total.
In memory of Ian Ashley Murdock (1973 - 2015) founder of the Debian project.
User avatar
llivv
 
Posts: 5709
Joined: 2007-02-14 18:10
Location: cold storage

Re: how to back up data and impact on the environment

Postby Segfault » 2018-11-21 14:06

There are 2 things to back up:
- the data, the files. This can be done manually using a cloud storage, a network, a USB stick.
- The system installation with all the programs definition, configuration, etc. This is not simple to backup. I tried a few times with dd and rsync to copies the partition image. I was not able to install it on a formated partition.

Your folks are using Linux as it was Windows.
Well, what about using good old UNIX way.
There is one server. It contains all data, files, everything. Server admin takes care of backing up the server.
All workstations are diskless GUI terminals. Think, how inexpensive this solution is. Any terminal can be used until it fails and nothing is ever lost. No expensive hard drives. Every user can use any terminal to log in and do their job.

Edit: With small SSD drives getting real cheap you could use a hybrid version of above, the OS is locally installed, but home directories are remote. Remember, this is not Windows, anything is possible.
Segfault
 
Posts: 814
Joined: 2005-09-24 12:24

Re: how to back up data and impact on the environment

Postby CwF » 2018-11-21 18:38

Segfault wrote:There is one server. It contains all data, files, everything....

...and analogous to the server/terminal model is hypervisor/vm. A little different in that the hypervisor isn't the server exactly, just the framework for the vm's. The vm's are the workers and 'terminals'. Task and functions are vm's. User data for me is not in any /home, but a network share. All that data is in a 'cow', a qcow2 file that can be backed up easily. Most vm's are on a common base, also a cow. Each layer to distinguish the particular vm is a layer, another cow. All runtimes cows, the top layers, are disposable. Every so often I upgrade, clean out cruft, and image the hypervisor. The vm's are an image already, they get caught in the hypervisor image, and induvidually.
MagicPoulp wrote:I was not able to install it on a formated partition.

You don't 'copy' it back to a formatted partition but dd it an empty one. For my images I use qemu-utilities, similar to dd.
CwF
 
Posts: 168
Joined: 2018-06-20 15:16

Re: how to back up data and impact on the environment

Postby debiman » 2018-11-22 06:17

Segfault wrote:Well, what about using good old UNIX way.
etc.

yay, back to the sixties!
seriously, what if they need to take their laptop on a business trip, or even home in the evening?
User avatar
debiman
 
Posts: 3064
Joined: 2013-03-12 07:18

Next

Return to General Questions

Who is online

Users browsing this forum: No registered users and 9 guests

fashionable