Adblocking with /etc/hosts

Share your own howto's etc. Not for support questions!

Adblocking with /etc/hosts

Postby Head_on_a_Stick » 2016-07-24 20:48

/etc/hosts can be used to block unwanted sites system-wide, see https://en.wikipedia.org/wiki/Hosts_(file)

Configuration of this file can be tedious and time-consuming:
http://www.makeuseof.com/tag/modify-manage-hosts-file-linux/

Fortunately, there is a more efficient method available.
:)

Code: Select all
git clone http://git.r-36.net/hosts-gen && cd hosts-gen
sudo make install
sudo cp examples/gethostszero /bin
sudo chmod 775 /bin/gethostszero
sudo /bin/gethostszero
sudo hosts-gen

This will populate /etc/hosts with a list of the most egregious web abusing sites and will block them automatically.

To update the list, use:
Code: Select all
sudo gethostszero
sudo hosts-gen


With thanks to Christopher Lohmann for this solution.

To update the file automatically, a systemd timer could be used.
https://wiki.archlinux.org/index.php/Systemd/Timers

For this, create these two files:

/etc/systemd/system/hostszero.timer
Code: Select all
[Unit]
Description=Update /etc/hosts daily.

[Timer]
OnCalendar=daily
AccuracySec=1h
Persistent=true

[Install]
WantedBy=timers.target

/etc/systemd/system/hostszero.service
Code: Select all
[Unit]
Description=Update /etc/hosts

[Service]
Type=oneshot
ExecStart=/bin/gethostszero
ExecStart=/bin/hosts-gen

Then enable this with:
Code: Select all
sudo systemctl enable hostszero.timer

Disable it with:
Code: Select all
sudo systemctl disable hostszero.timer

Check the status with:
Code: Select all
systemctl status hostszero.timer

Which should look something like this:
Code: Select all
empty@TheLab:~$ systemctl status hostszero.timer
● hostszero.timer - Update /etc/hosts daily.
   Loaded: loaded (/etc/systemd/system/hostszero.timer; enabled)
   Active: active (waiting) since Sun 2016-07-24 18:00:27 BST; 2min 42s ago
empty@TheLab:~$

The original /etc/hosts file is available here:
http://someonewhocares.org/hosts/zero/hosts
ESTRAGON: We always find something, eh, Didi, to give us the impression we exist?
VLADIMIR (impatiently): Yes, yes, we're magicians.
User avatar
Head_on_a_Stick
 
Posts: 7893
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: Adblocking with /etc/hosts

Postby dasein » 2016-07-24 23:46

Integrating host-file-based blocking like this with a local DNS server like DNSMasq is a killer combination.

Not only does the combination make it easy to protect an arbitrarily large network from a single hosts file, it allows globally "suspect" domains (e.g., doubleclick) to be blocked with a single line.

Thanks for a fine HOWTO, HoaS.
User avatar
dasein
 
Posts: 7775
Joined: 2011-03-04 01:06
Location: Terra Incantationum

Re: Adblocking with /etc/hosts

Postby tomazzi » 2016-07-25 20:59

...and how this is different from using privoxy?
...or just a noscript?
Odi profanum vulgus
tomazzi
 
Posts: 730
Joined: 2013-08-02 21:33

Re: Adblocking with /etc/hosts

Postby rbern » 2016-07-26 04:21

tomazzi wrote:...and how this is different from using privoxy?
...or just a noscript?


Hi tomazzi,

I once had an idea for this, using another similar app called adsuck (https://github.com/conformal/adsuck), but it was never packaged for debian or ubuntu.

I think privoxy (still) does not support https, but apparently it will https://sourceforge.net/p/ijbswa/support-requests/1667/.

IMO, having noscript in every computer on a network uses more resources (RAM/CPU/bandwidth/browsing speed) than a single router computer filtering all those from the same computers in the network, depending on the size of the network.

Even in a single computer, this could save the RAM that is used by noscript and save all CPU cycles that noscript takes to filter all IP and addresses. More browsing speed with less CPU usage.

This seems interesting to me.
rbern
 
Posts: 8
Joined: 2011-07-16 11:18

Re: Adblocking with /etc/hosts

Postby edbarx » 2016-07-26 07:02

I created a .deb package for this. The procedure is simple.

The Method (howto):
  1. Download the sources
    Code: Select all
    git clone http://git.r-36.net/hosts-gen && cd hosts-gen

  2. Rename the directory to hosts-gen-0.1.0. This is required by dh_make.
  3. To Debianize the sources issue the next command while in hosts-gen-0.1.0
    Code: Select all
    dh_make --createorig

  4. cd to the debian subdirectory. Create a new file named hosts-gen.install using touch.
    Code: Select all
    cd debian
    touch hosts-gen.install

  5. Open hosts-gen.install in nano or another text editor. Add this line:
    Code: Select all
    examples/gethostszero /bin

  6. cd back to the parent directory hosts-gen-0.1.0
  7. Use dpkg-buildpackage to build a Debian package.
  8. Use dpkg -i package-name to install the package.

EDIT:
To check periodically for updates as instructed earlier in this thread, use the debian/hosts-gen.postinst file to add the required script.
Debian == { > 30, 000 packages }; Debian != systemd
The worst infection of all, is a false sense of security!
It is hard to get away from CLI tools.
User avatar
edbarx
 
Posts: 5394
Joined: 2007-07-18 06:19
Location: 35° 50 N, 14 º 35 E

Re: Adblocking with /etc/hosts

Postby Head_on_a_Stick » 2016-07-26 19:49

@tomazzi: I had never heard of privoxy :oops:

Thanks for posting, that looks very nice -- if I can throw together something cogent, I will post a HowTo for that as well :)

@edbarx: Brilliant, thanks!

I will see if I can package it for my OBS repository with the systemd unit files...
ESTRAGON: We always find something, eh, Didi, to give us the impression we exist?
VLADIMIR (impatiently): Yes, yes, we're magicians.
User avatar
Head_on_a_Stick
 
Posts: 7893
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: Adblocking with /etc/hosts

Postby edbarx » 2016-07-27 10:00

Post package installation script:
The debian/hosts-gen.postinst file can be used to populate the /etc/hosts file with the required entries. Just add the necessary commands.

Post package removal script:
The debian/hosts-gen.postrm file should be used to remove any remaining files generated by the package. It should also contain script to restore modified system files (/etc/hosts) to their original condition.
Debian == { > 30, 000 packages }; Debian != systemd
The worst infection of all, is a false sense of security!
It is hard to get away from CLI tools.
User avatar
edbarx
 
Posts: 5394
Joined: 2007-07-18 06:19
Location: 35° 50 N, 14 º 35 E

Re: Adblocking with /etc/hosts

Postby kevinthefixer » 2018-08-11 22:58

This is very similar to the adblockers used in Android, which I've had very good luck with. So I thought I'd give it a try. However, I didn't get very far as the git repository seems to have vanished. Anyone have an updated URL?
kevinthefixer
 
Posts: 72
Joined: 2018-05-05 22:30


Re: Adblocking with /etc/hosts

Postby Head_on_a_Stick » 2018-08-12 08:53

ESTRAGON: We always find something, eh, Didi, to give us the impression we exist?
VLADIMIR (impatiently): Yes, yes, we're magicians.
User avatar
Head_on_a_Stick
 
Posts: 7893
Joined: 2014-06-01 17:46
Location: /dev/chair

Re: Adblocking with /etc/hosts

Postby Bloom » 2018-08-12 12:31

My solution: first, make a copy of your hosts file.
Code: Select all
sudo cp /etc/hosts /etc/hosts.0

Then, run this script:
Code: Select all
sudo ./newhosts

And the contents of this newhosts script:
Code: Select all
#!/bin/bash
# get MVPS hosts file to block ad sites and merge it with our hosts file
wget -O - http://winhelp2002.mvps.org/hosts.txt | sed 's/[[:space:]]*#.*$//g;' |
   grep -v localhost | tr ' ' '\t' | tr -s '\t' | tr -d '\015' | sort -u >/tmp/hosts0
echo 0.0.0.0 safebrowsing.google.com >>/tmp/hosts0
echo 0.0.0.0 survival24x7.com >>/tmp/hosts0
echo 0.0.0.0 www.survival24x7.com >>/tmp/hosts0
echo 0.0.0.0 hanstrkr.com >>/tmp/hosts0
echo 0.0.0.0 www.hanstrkr.com >>/tmp/hosts0
echo Attention! This should be run as ROOT!
cat /etc/hosts.0 /tmp/hosts0|sudo dd of=/etc/hosts
rm -f /tmp/hosts0
User avatar
Bloom
 
Posts: 69
Joined: 2017-11-11 12:23

Re: Adblocking with /etc/hosts

Postby kevinthefixer » 2018-08-24 01:02

Head_on_a_Stick wrote:https://github.com/StevenBlack/hosts

Ran into dependency problems with this one (running Stretch). @Bloom's "newhosts" file worked a trick though. This should work on any system using /etc/hosts, I would think. Outlook mail loads in half the time, no noticeable difference in Gmail (no noticeable ads in Gmail either).
kevinthefixer
 
Posts: 72
Joined: 2018-05-05 22:30

Re: Adblocking with /etc/hosts

Postby bedtime » 2018-08-24 17:17

debiman wrote:i've been using https://github.com/gaenserich/hostsblock for years.

I installed this program, and it renamed my home directory to 'hostsblock'. Anyhow, long story short, I ended up losing a bunch of files whilst uninstalling and getting things back to normal—config files and scripts that took hours and hours to make. :(

Please be careful if you chose to install this app.
OS: Debian GNU/Linux (Buster—testing); Kernel: 4.16.0-2-amd64; Shell: sh; WM: i3
Machine: HP ProDesk 600 G3—i7-7700, 8GB RAM, 256 SSD, Intel HD Graphics 630

bedtime
 
Posts: 88
Joined: 2012-12-16 19:34

Re: Adblocking with /etc/hosts

Postby milomak » 2018-08-24 19:52

how is this better than say using ublock origin?
iMac - MacOS and Windows 10 (Bootcamp)/ Debian Sid (External SSD)
Laptop (64-bit) - Debian Sid, Win10,
Kodi Box - Debian Sid
milomak
 
Posts: 1758
Joined: 2009-06-09 22:20

Re: Adblocking with /etc/hosts

Postby dilberts_left_nut » 2018-08-24 20:58

milomak wrote:how is this better than say using ublock origin?

Because you deploy it on your dns relay and it covers every device on your network.
AdrianTM wrote:There's no hacker in my grandma...
User avatar
dilberts_left_nut
 
Posts: 4866
Joined: 2009-10-05 07:54
Location: enzed

Next

Return to Docs, Howtos, Tips & Tricks

Who is online

Users browsing this forum: No registered users and 1 guest

fashionable