how to back up data and impact on the environment

[MagicPoulp]

At my job, most people use Ubuntu. I use Debian stable. Because they are afraid to lose data or time, they buy new computers very often much before the previous computers would break. This has a strong environmental impact if all companies proceed similarly. For a company it is not in the culture to suddenly have a computer that breaks and require reinstallation. THey also buy extra laptops to make sure there is not secret data on their disk when they travel abroad.

This culture of no simple way to back up was spread by Microsoft Windows, probably because they wanted to sell more computers. It is well known that after some time, due to many corruptions and installations and startup scripts, Windows becomes super slow. Not because the hardware is deprecated, but because of Windows that has aged badly. This reminds the scandal with phones that were programmed to become slower after the warranty, to push selling more phones.

If we focus on Ubuntu and Debian, it is still not very clear how to efficiently backup. The state of Debian indirectly impacts Ubuntu and other Debian-based OSes and a large community of people. On a large scale, there is an impact on the environment. People often think their Ubuntu is slow to start or that the Unity GUI is laggy. This is often due to Software more than hardware, and due to graphical driver conflicts. It is very common on Linux to have graphics driver conflicts. Even when using the standard nouveau graphics driver, it is common to have laggy interfaces and tearing in web browsers. I have such tearing in my browser myself. It is related to the topic because lagginess makes people think their computer is broken or old.

There are 2 things to back up:
- the data, the files. This can be done manually using a cloud storage, a network, a USB stick.
- The system installation with all the programs definition, configuration, etc. This is not simple to backup. I tried a few times with dd and rsync to copies the partition image. I was not able to install it on a formated partition.

[arochester]

https://dflinux.frama.io/thebeginnersha ... h-deja-dup

[MagicPoulp]

Here is a lazydog I made to recover from a copy of debian files. I basically copy all the files in /, and I setup grub. But I was not satisfied. There are always small details that change over times or that need more configuration.

Code: Select all

IMPORTANT: the last / in rsync is needed

--> Backup:
start:
thierry@small:~/Documents$ sudo su -
root@small:/root# mkdir /media/thierry/ZZZ
root@small:/root# mount /dev/sdc1 /media/thierry/ZZZ

mkdir /media/thierry/ZZZ/slash
time rsync -ahPHAXx --delete --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found} / /media/thierry/ZZZ/slash/ >/dev/null
or a simple version:
sudo rsync -ahPHAXx --delete --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found} / /media/thierry/ZZZ/slash/

----------------------------------------------------------------------------------------------------------------------
--> Install

--> prepare partition space
disable secure boot in BIOS
enable CSM and disalble the windows default
shrink windows partition using gparted on debian live

--> install the backup:
- start Debian on a live CD or USB stick
- shrink windows partition using gparted on debian live
- format the partition using gparted
need ext4 file system, and need 2 partitions, with 20 GB for swap

- mount the partition to install on
sudo fdisk -l
mount /dev/sdaX /destination

- mount the usb stick
USE USB 3.0 that is faster
sudo fdisk -l
mkdir /media/usb1
mount /dev/sdbX /media/usb1

- copy the data to the partition

mkdir /mnt/slash
mount /dev/sda7 /mnt/slash
sudo rsync -ahPHAXx /media/usb1/slash/ /mnt/slash/ >/dev/null
the progress can be shown using df -H

--> reload /etc/fstab

Change fstab on target system
the fstab only contains the linux partitions and the boot partition (efi)

As root, edit /mnt/slash/etc/fstab
For each partition (including swap), replace the first field with the new UUID or LABEL (it’s straightforward with LABELs):
UUID=the-long-uuid, or LABEL=yourlabel

two ways to get the UUIDs / LABELs:

ls -l /dev/disk/by-uuid/
or try blkid /dev/sdaX


--> install grub for uefi
if it does not work, try as follows:
https://wiki.debian.org/GrubEFIReinstall

REPLACE sdX by the DISK name like only an a, IT IS NOT the partition that Linux is installed on

for i in /dev /dev/pts /proc /sys /run; do sudo mount -B $i /mnt/slash$i; done
sudo chroot /mnt/slash
grub-mkconfig -o /boot/grub/grub.cfg
sudo grub-install --recheck /dev/sdX   <---------- replace the X
sudo update-grub


--> check the result of the grub installation
# lsblk -no NAME,UUID /dev/sdbX

where you substitute the desired partition for /dev/sdbX To list the UUIDs of partitions grub thinks it can boot, use grep:

# grep UUID= /boot/grub/grub.cfg


-------------------------
best link:
http://superuser.com/questions/709176/how-to-best-clone-a-running-system-to-a-new-harddisk-using-rsync

another tutorial but less pro
http://positon.org/clone-a-linux-system-install-to-another-computer

to check other things:
https://wiki.archlinux.org/index.php/full_system_backup_with_rsync

grub:
http://howtoubuntu.org/how-to-repair-restore-reinstall-grub-2-with-a-ubuntu-live-cd

[MagicPoulp]

https://dflinux.frama.io/thebeginnersha ... h-deja-dup

Nice simple software for the large amount of people. Personally, I prefer to backup manually on USB stick. I always have small things I actively know I want to save. So I get a smaller bundle to save.

I think the main problem lies in backing up the system installation.

[arochester]

I think the main problem lies in backing up the system installation.

http://freesoftwaremagazine.com/article ... _computer/ ?

[debiman]

i think it's a huge stretch to say people are buying new hardware all the time because they're too lazy to do backups (or because according to you there is no simple solution).
or maybe i simply didn't understand what you are saying.
i will grant you that the industry is pushing a mindset where the OS and the device are the same thing, and for most people a broken operating system or broken hardware is the same thing.
and that electronic devices are way too cheap these days.
the chinese are getting us by the balls, and one day they will come to the conclusion that their grip is now firm enough, and own us all.

btw, i use borg for backups.

I use Debian stable.

*cough*
not according to your other posts.

[stevepusser]

*cough*
not according to your other posts.

The old "You f*** one goat, and what does everyone call you?!?" story, eh?

[sunrat]

That story had a sheep last I heard it.

Clonezilla works well for disk or partition backups. And they have restored perfectly the couple of times I have needed them.

[MagicPoulp]

i think it's a huge stretch to say people are buying new hardware all the time because they're too lazy to do backups (or because according to you there is no simple solution).
or maybe i simply didn't understand what you are saying.

The policy in my company is that everyone gets a new computer every 2 or 3 years. I think this is wrong. One person should get a new computer when the previous computer stops working.'

There is a reason why there is such a policy. It takes a lot of time to set up a computer and install programs. Moreover, they are scared people will lose precious data. Many people do their work without having their work backed up daily.

In general, in most companies I have been to, people do not care much about the environment. They generalize the use of airplanes and laptops (toxic battery, wasted screen, short lifetime). Bosses have cars paid tax free by the companies. All employees get a new smartphone tax free quite often starting when new employed. People leave their computer without sleep mode all the time even during nights. They buy super powerful computers that they do not really need just because it is feels like sitting in a ferrari. And those guys are certified engineers, with strong background in the sciences, very high purchase potential, and so on. Still with an environmental IQ of 1 digit.

[MagicPoulp]

Clonezilla works well for disk or partition backups. And they have restored perfectly the couple of times I have needed them.

That looks like a nice program. Does the saved data take the same space as the partition that is backed up?

By using rsync, I had the advantage of not having to save the entire partition size.

One I tried this:
- start on debian live CV to have a working linux.
- use gparted to foramt my partition
- use dd to copy the binary of my disk taht was itself back up using the command dd.
- set up grub since partition configs have changed.
It did not work, because the grub install was too hard to configure. But it seemed really nice to have since one could reinstall a system very fast without a full manual installation.

So now I install debian, and then I copy files with rsync. It has the advantage of taking less space that a complete partition save.

[sunrat]

Clonezilla works well for disk or partition backups. And they have restored perfectly the couple of times I have needed them.

That looks like a nice program. Does the saved data take the same space as the partition that is backed up?

It only uses the space of the data. Unless you use the compression option and then it takes less.

I use Clonezilla to back up the system partition before doing anything potentially harmful like large software upgrades. For data backup of home, I use rsnapshot with a systemd timer. rsync is fine for that too, it's what rsnapshot uses as its backend.

[llivv]

The policy in my company is that everyone gets a new computer every 2 or 3 years. I think this is wrong. One person should get a new computer when the previous computer stops working.'

There is a reason why there is such a policy. It takes a lot of time to set up a computer and install programs. Moreover, they are scared people will lose precious data. Many people do their work without having their work backed up daily.

In general, in most companies I have been to, people do not care much about the environment. They generalize the use of airplanes and laptops (toxic battery, wasted screen, short lifetime). Bosses have cars paid tax free by the companies. All employees get a new smartphone tax free quite often starting when new employed. People leave their computer without sleep mode all the time even during nights. They buy super powerful computers that they do not really need just because it is feels like sitting in a ferrari. And those guys are certified engineers, with strong background in the sciences, very high purchase potential, and so on. Still with an environmental IQ of 1 digit.

My comment regrading carbon footprint is that all of us are guilty of the waste our system produces since all of us benefit from the gross products. It wasn't too long ago when IIRC, there was a green movement of consciencious individuals who wanted to make changes to policy.
The futility of consciencious environmental policy has evolved into the only thing left that is green is $
produce more - make more $ - consume more - - - or be cast out. And it's a cold existence out there. much like the warming of the planet as the globle population grows and policy is ill equipped to balance human needs agaisnt human advances ....

[Segfault]

There are 2 things to back up:
- the data, the files. This can be done manually using a cloud storage, a network, a USB stick.
- The system installation with all the programs definition, configuration, etc. This is not simple to backup. I tried a few times with dd and rsync to copies the partition image. I was not able to install it on a formated partition.

Your folks are using Linux as it was Windows.
Well, what about using good old UNIX way.
There is one server. It contains all data, files, everything. Server admin takes care of backing up the server.
All workstations are diskless GUI terminals. Think, how inexpensive this solution is. Any terminal can be used until it fails and nothing is ever lost. No expensive hard drives. Every user can use any terminal to log in and do their job.

Edit: With small SSD drives getting real cheap you could use a hybrid version of above, the OS is locally installed, but home directories are remote. Remember, this is not Windows, anything is possible.

[CwF]

There is one server. It contains all data, files, everything....

...and analogous to the server/terminal model is hypervisor/vm. A little different in that the hypervisor isn't the server exactly, just the framework for the vm's. The vm's are the workers and 'terminals'. Task and functions are vm's. User data for me is not in any /home, but a network share. All that data is in a 'cow', a qcow2 file that can be backed up easily. Most vm's are on a common base, also a cow. Each layer to distinguish the particular vm is a layer, another cow. All runtimes cows, the top layers, are disposable. Every so often I upgrade, clean out cruft, and image the hypervisor. The vm's are an image already, they get caught in the hypervisor image, and induvidually.

I was not able to install it on a formated partition.

You don't 'copy' it back to a formatted partition but dd it an empty one. For my images I use qemu-utilities, similar to dd.

[debiman]

Well, what about using good old UNIX way.
etc.

yay, back to the sixties!
seriously, what if they need to take their laptop on a business trip, or even home in the evening?

[debiman]

*cough*
not according to your other posts.

The old "You f*** one goat, and what does everyone call you?!?" story, eh?

oh i'm sure we will get more nickname-worthy quotes from MagicPoulp, the user who administers their system with a philosophical, nay, poetical attitude!

[debiman]

The policy in my company is that everyone gets a new computer every 2 or 3 years. I think this is wrong. One person should get a new computer when the previous computer stops working.'

There is a reason why there is such a policy. It takes a lot of time to set up a computer and install programs. Moreover, they are scared people will lose precious data. Many people do their work without having their work backed up daily.

In general, in most companies I have been to, people do not care much about the environment.

i understand.
unfortunately, the time/cost factor will always be the strongest argument.
fortunately, many companies are open to changing things if they prove to not be more expensive, or sometimes even cheaper. and many countries' laws support them in that.

btw, these cars, phones and computers don't get thrown away after 2 or 3 years - they get sold on.
enabling countless linux users to breathe new life into yesterdays corporate laptops, for many more years!

[Segfault]

Well, what about using good old UNIX way.
etc.

yay, back to the sixties!
seriously, what if they need to take their laptop on a business trip, or even home in the evening?

Laptop is different story indeed. I was talking about LAN workstations.

[CwF]

btw, these cars, phones and computers don't get thrown away after 2 or 3 years - they get sold on.
enabling countless linux users to breathe new life into yesterdays corporate laptops, for many more years!

Absolutely! The OP is looking at this in a typically uniformed way. Nerds everywhere benefit from dumpster diving, but that's not the whole story about 'waste'. Sometimes, efficiency improvements pay to throw away stuff. In large scale IT I can imagine electricity being a large budget line. In my world, it would be diesel fuel, and lost opportunity cost. One machine down in the wrong place at the wrong time could wipe out weeks of budget. I've seen new machines save $600/day in fuel cost, that's a free 'operator'. An old machine, that I could make work forever, could cost up to $100k/year to operate. A new $250k+ machine can lower that to $20-30k. I'm not the IT budget director, but I'd gamble the generalities are similar.

The machines I speak of make 'perfect pistes' in the most beautiful places in the world, perhaps for the most environmentaly sensitive hypocrites anywhere...

[debiman]

all that said, e-waste is a serious problem of ridiculous proportions.
it's most apparent with mobile phones & laptops (not the corporate models that have a high resale value, but the consumer crap, chromebooks and whathaveyou, that nobody wants to buy anymore when they start lagging after a year or two).
it's a grave we've been digging from 3 sides for a long time now:

• china and far-east producing & selling the components for a pittance, probably harming environment and underage workers in unimaginable ways along the way
• electronics companies pushig for new devices instead of repair or soft- or hardware upgrades (it's an old story, built-in obsoletion taken to the next level)
• consumers being effing sheeple as always

still don't see what it has to do with backups though.