Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

Block Ads & Fast Caching w/Privoxy +Squid

Share your HowTo, Documentation, Tips and Tricks. Not for support questions!.
Post Reply
Message
Author
debianized
Posts: 278
Joined: 2009-01-07 07:56

Block Ads & Fast Caching w/Privoxy +Squid

#1 Post by debianized »

This Howto will help you achieve the following:

I. Block web advertisements, as well as java annoyances and web bugs, for all browsers installed on your system by means of the Privoxy web filter, instead of using browser extensions such as Adblock Plus.

II. Setup up a web cache for use by all browsers on your system by means of the Squid3 web cache. This can give the appearance of almost instantaneous web browsing, and also allow you to turn off the browser cache for all installed browsers, since Squid3 can be used as a web cache for the entire system.

III. Allow you to run Squid3 in high anonymous mode, as an elite proxy, such that any websites you visit will not be able to detect you are even using a proxy.

The first step is to install privoxy, using the following command as root:

aptitude install privoxy

Open Iceweasel and check the AOL website to find an ad for later testing:

AOL.com

You should see an advertisement in the top right corner of the page, which will appear underneath the 'mail,' 'aim' and 'radio' icons:
Image

Now that we know where the advertisement is, let's point Iceweasel to the Privoxy proxy, to insure it is working and removing the ads from web pages. From the Iceweasel menu, select 'Edit,' then 'Preferences,' then 'Advanced' in the top right. You should now see four tabs labelled 'General,' 'Network,' 'Update' & 'Encryption.' Select the 'Network' tab. Select the first button labeled 'Settings.' (From here on out I will refer to these steps as Edit->Preferences->Advanced->Network-Settings)

Select 'Manual Proxy Configuration' and make the following changes:

For 'HTTP Proxy' type 127.0.0.1
For 'port' type 8118

Now go back to the AOL website and the ad should no longer display under the 'mail,' 'aim' and 'radio' icons on the right:

Image

Now we know for sure Privoxy is installed and working, we can move onto Squid3.

Before we install Squid3, we need to first check and insure that logrotate is installed, so that the Squid3 logs do not grow out of control. Open a terminal and type the following:

whereis logrotate

If the following appears after entering the above command,

/usr/sbin/logrotate /etc/logrotate.conf /etc/logrotate.d /usr/share/man/man8/logrotate.8.gz

it means logrotate is already installed and no further action is needed.

If the 'whereis logrotate' command instead returns the following.

logrotate:

then you need to install logrotate using the command 'aptitude install logrotate' .

Install Squid3 by issuing the following command as root:

aptitude install squid3

Let's point Icewseasel to the Squid3 proxy, to verify that Squid3 is working, by selecting Edit->Preferences->Advanced->Network-Settings, but changing 'port' to 3128.

The following link at browser spy checks whether you are surfing via a proxy or not:

http://browserspy.dk/proxy.php

The first two entries on that page should appear as follows:

Image

At this point, the Privoxy web filter is running on your machine, as well as the Squid3 caching proxy and we have verified that both are working correctly. The next step is to chain the two proxies together, such that one passes off requests to the other. Since Squid3 is the caching proxy and Privoxy is the web filter, we want to first send requests to Squid3 and have Squid3 pass the requests off to Privoxy. This way, ads will not be cached in Squid3, because Privoxy will filter them out before they ever reach the Squid3 cache. The connection will work like this:

Browser->Squid3->Privoxy->Internet

It becomes obvious we must set Squid3 to forward requests to Privoxy . This is done via editing the squid.conf file located at /etc/squid3/squid.conf. Open the file, with root privileges, in your favorite editor. Scroll to the very bottom of the squid.conf file and make the following entries:

cache_peer 127.0.0.1 parent 8118 7 no-digest no-query
shutdown_lifetime 0 seconds

The cache_peer entry above tells Squid3 to hand off requests to Privoxy on port 8118. The shutdown_lifetime entry shuts Squid3 down quickly, so that you are not tapping your fingers waiting for Squid3 to shutdown every time you reboot. The following entries anonymize Squid3 headers, turning Squid3 into an elite, or high anonymous proxy:

request_header_access From deny all
request_header_access Referer deny all
request_header_access Server deny all
request_header_access WWW-Authenticate deny all
request_header_access Link deny all
request_header_access Cache-Control deny all
reply_header_access From deny all
reply_header_access Referer deny all
reply_header_access Server deny all
reply_header_access User-Agent deny all
reply_header_access WWW-Authenticate deny all
reply_header_access Link deny all
via off
forwarded_for delete

Save the squid.conf file after adding the above entries to the end of the file, then restart the Squid3 proxy with the following command (as root):

squid3 -k reconfigure

The following two links should verify that Squid3 is now anonymous, with any trace of browsing via a proxy removed:

SamAir

ProxyLists ProxyJudge


At this point, all NECESSARY configuration of Squid3 is complete and we can turn our attention to Privoxy configuration. With root privileges, open the /etc/privoxy/config file in your favorite editor. Make sure the following variables are set. The enable-remote-toggle and enable-edit-actions variables will allow you to change the Privoxy configuration through a web browser:

enable-remote-toggle 1
enable-edit-actions 1
permit-access 127.0.0.1

Once you have made the above edits, save them, then restart privoxy, with root privileges, with the following commands:

/etc/init.d/privoxy stop
/etc/init.d/privoxy start

Now, you will be able to alter the Privoxy configuration through the web browser. However, there is one point you must ALWAYS remember here. Any time you wish to change the Privoxy config through the web editor, the browser MUST be pointed to the Privoxy proxy on port 8118. If your browser is pointed to the Squid3 proxy on port 3128, you will not be able to reach the Privoxy web editor. So open Iceweasel and select Edit->Preferences->Advanced->Network->Settings and make sure the 'port' under 'Manual proxy configuration' is set to 8118 (privoxy port) instead of 3128 (squid3 port).

Having done that, if you click on the following link:

1. http://config.privoxy.org/

2. Select "View & change the current configuration"

3. Select "Edit" for /etc/privoxy/match-all.action

4. Under 'Actions' Select 'Edit'

You will now be able to alter your Privoxy configuration via check boxes and text entries. The only change I make here is '+set-image-blocker Send a 1x1 transparent GIF' but feel free to tune it however you wish. After making whatever changes you wish to make here, scroll to the bottom of the page and click the 'Submit' button. Then go back to Edit->Preferences->Advanced->Network->Settings and make sure the 'port' under 'Manual proxy configuration' is set back to 3128, which is the Squid3 port. You can also turn off Iceweasel's cache at this point, since Squid3 is now operating as a web cache. Go to about:config, search for 'cache' and set the necessary cache entries to false or 0.

If you use the Chromium-browser, you must disable browser caching and point Chromium at a proxy, via the command line like this:

/usr/bin/chromium-browser -disk-cache-size=0 --proxy-server=127.0.0.1:3128

Or for incognito mode:

/usr/bin/chromium-browser -disk-cache-size=0 --incognito --proxy-server=127.0.0.1:3128

However, Chromium-browser is an unusual case, in that most web browsers will allow you to alter cache or proxy settings via the GUI.

CAVEATS:

1. Privoxy is an http only proxy, so does not filter https or ftp addresses
2. If you set the Privoxy web editor to alter java on web pages, be aware you might run into weirdness down the road. Perhaps a forum that uses java to log you in won't let you in, or a web page might not issue the pop-up window it normally does. If you are surfing via Privoxy and run into such an issue, go to Edit->Preferences>Advanced->Network->Settings and select "No proxy," to bypass the proxy temporarily. After you have finished on such a page, take the same route to turn the proxy back on.
3. Always remember, if you wish to use the Privoxy web editor, you MUST have the proxy port pointed to the Privoxy port, which is 8118. If it is pointed at the Squid3 port 3128 you will not be able to edit Privoxy via the web editor.
4. Presently, Privoxy is not starting at boot on my machine. Since I have the Privoxy log turned off, I can't yet diagnose if this issue is local to my machine or an issue with Squeeze, although I suspect it is a local issue.
5. These instructions were created from a new install of Squeeze, so may vary slightly from instructions for Lenny.
6. I make mistakes. If you see any in this document, please let me know so I can correct them and feel free to let me know if formatting or anything else can be improved.

AJones
Posts: 2
Joined: 2007-04-14 15:13

Re: Block Ads & Fast Caching w/Privoxy +Squid

#2 Post by AJones »

Hi There:

I tried your procedure. I find that Squid3 is not caching. The internet access is also slow. I am on Debian Squeeze.

Thanks,
AJ

Post Reply