Scheduled Maintenance: We are aware of an issue with Google, AOL, and Yahoo services as email providers which are blocking new registrations. We are trying to fix the issue and we have several internal and external support tickets in process to resolve the issue. Please see: viewtopic.php?t=158230

 

 

 

postfix/amavis not scanning email

Linux Kernel, Network, and Services configuration.
Post Reply
Message
Author
bithead
Posts: 48
Joined: 2015-10-21 02:54
Been thanked: 2 times

postfix/amavis not scanning email

#1 Post by bithead »

Hi all,

I'm running a postfix, amavis, spamassassin, dkimproxy, clamav system on Debian 8. Here are pertinent versions:
  • Linux mail 4.8.0-0.bpo.2-amd64 #1 SMP Debian 4.8.11-1~bpo8+1 (2016-12-14) x86_64 GNU/Linux
    amavisd-new 1:2.10.1-2~deb8u1
    clamav 0.99.2+dfsg-0+deb8u2
    dkimproxy 1.4.1-3
    postfix 2.11.3-1
    spamassassin 3.4.0-6
I've been through several configuration documents and tried many suggestions found within, but for the life of me I can't get inbound email to be passed off to amavis for spam & virus checking. dkim signatures are working, but that's it for the add-on services to postfix. I'm a the point where I feel like I'm throwing darts while blindfolded, hoping to get lucky and hit the right combination that makes it all work. It's worth noting that I can get spam and AV checking to occur if I telnet to port 10024 (amavis' listening port) and send mail from there. So the services are running, but incoming mail is not being routed to them.

Below I'll post the results of 'netstat -tap' followed by many configuration files. Please have a look and see if you can spot anything that might be helpful. I promise not to throw darts while you're looking. :)

Code: Select all

** netstat -tap

Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 *:imaps                 *:*                     LISTEN      1/init
tcp        0      0 *:57345                 *:*                     LISTEN      3213/rpc.statd
tcp        0      0 *:pop3s                 *:*                     LISTEN      23957/dovecot
tcp        0      0 mail.domain.com:10023   *:*                     LISTEN      9478/postgrey.pid -
tcp        0      0 mail.domain.com:10024   *:*                     LISTEN      9347/amavisd-new (m
tcp        0      0 mail.domain.com:10025   *:*                     LISTEN      9764/master
tcp        0      0 mail.domain.com:mysql   *:*                     LISTEN      4470/mysqld
tcp        0      0 mail.domain.com:10026   *:*                     LISTEN      4497/perl
tcp        0      0 *:submission            *:*                     LISTEN      9764/master
tcp        0      0 mail.domain.com:10027   *:*                     LISTEN      4534/perl
tcp        0      0 mail.domain.com:10028   *:*                     LISTEN      9764/master
tcp        0      0 mail.domain.com:10029   *:*                     LISTEN      9764/master
tcp        0      0 *:pop3                  *:*                     LISTEN      23957/dovecot
tcp        0      0 mail.domain.com:spamd   *:*                     LISTEN      9433/spamassassin.p
tcp        0      0 *:imap2                 *:*                     LISTEN      1/init
tcp        0      0 *:sunrpc                *:*                     LISTEN      3181/rpcbind
tcp        0      0 mail.domain.co:domain   *:*                     LISTEN      3862/named
tcp        0      0 mail.domain.co:domain   *:*                     LISTEN      3862/named
tcp        0      0 mail.domain.co:domain   *:*                     LISTEN      3862/named
tcp        0      0 *:ssh                   *:*                     LISTEN      3943/sshd
tcp        0      0 *:smtp                  *:*                     LISTEN      9764/master
tcp        0      0 mail.domain.com:953     *:*                     LISTEN      3865/lwresd

Code: Select all

** /etc/amavis/conf.d/15-content_filter_mode

use strict;
@bypass_virus_checks_maps = (
   \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);
@bypass_spam_checks_maps = (
   \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);
1;  # ensure a defined return

Code: Select all

** /etc/amavis/conf.d/20-debian_defaults

use strict;
$QUARANTINEDIR = "$MYHOME/virusmails";
$quarantine_subdir_levels = 1; # enable quarantine dir hashing
$log_recip_templ = undef;    # disable by-recipient level-0 log entries
$DO_SYSLOG = 1;              # log via syslogd (preferred)
$syslog_ident = 'amavis';    # syslog ident tag, prepended to all messages
$syslog_facility = 'mail';
$syslog_priority = 'debug';  # switch to info to drop debug output, etc
$enable_db = 1;              # enable use of BerkeleyDB/libdb (SNMP and nanny)
$enable_global_cache = 1;    # enable use of libdb-based cache if $enable_db=1
$inet_socket_port = 10024;   # default listening socket
$inet_socket_bind='127.0.0.1';  # added 170120
$sa_spam_subject_tag = '***SPAM*** ';
$sa_tag_level_deflt  = -999;  # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 6.31; # add 'spam detected' headers at that level
$sa_kill_level_deflt = 6.31; # triggers spam evasive actions
$sa_dsn_cutoff_level = 10;   # spam level beyond which a DSN is not sent
$sa_mail_body_size_limit = 200*1024; # don't waste time on SA if mail is larger
$sa_local_tests_only = 0;    # only tests which do not require internet access?
$MAXLEVELS = 14;
$MAXFILES = 1500;
$MIN_EXPANSION_QUOTA =      100*1024;  # bytes
$MAX_EXPANSION_QUOTA = 300*1024*1024;  # bytes
$final_virus_destiny      = D_DISCARD;  # (data not lost, see virus quarantine)
$final_banned_destiny     = D_BOUNCE;   # D_REJECT when front-end MTA
$final_spam_destiny       = D_BOUNCE;
$final_bad_header_destiny = D_PASS;     # False-positive prone (for spam)
$enable_dkim_verification = 1;
$virus_admin = "postmaster\@$mydomain"; # due to D_DISCARD default
$X_HEADER_LINE = "Debian $myproduct_name at $mydomain";
@viruses_that_fake_sender_maps = (new_RE(
  [qr'\bEICAR\b'i => 0],            # av test pattern name
  [qr/.*/ => 1],  # true for everything else
));
@keep_decoded_original_maps = (new_RE(
  qr'^MAIL-UNDECIPHERABLE$', # recheck full mail if it contains undecipherables
  qr'^(ASCII(?! cpio)|text|uuencoded|xxencoded|binhex)'i,
));
$banned_filename_re = new_RE(
  # block certain double extensions anywhere in the base name
  qr'\.[^./]*\.(exe|vbs|pif|scr|bat|cmd|com|cpl|dll)\.?$'i,
  qr'\{[0-9a-f]{8}(-[0-9a-f]{4}){3}-[0-9a-f]{12}\}?'i, # Windows Class ID CLSID, strict
  qr'^application/x-msdownload$'i,                  # block these MIME types
  qr'^application/x-msdos-program$'i,
  qr'^application/hta$'i,
  qr'.\.(exe|vbs|pif|scr|bat|cmd|com|cpl)$'i, # banned extension - basic
  qr'^\.(exe-ms)$',                       # banned file(1) types
);
@score_sender_maps = ({ # a by-recipient hash lookup table,
                        # results from all matching recipient tables are summed
  ## site-wide opinions about senders (the '.' matches any recipient)
  '.' => [  # the _first_ matching sender determines the score boost
   new_RE(  # regexp-type lookup table, just happens to be all soft-blacklist
    [qr'^(bulkmail|offers|cheapbenefits|earnmoney|foryou)@'i         => 5.0],
    [qr'^(greatcasino|investments|lose_weight_today|market\.alert)@'i=> 5.0],
    [qr'^(money2you|MyGreenCard|new\.tld\.registry|opt-out|opt-in)@'i=> 5.0],
    [qr'^(optin|saveonlsmoking2002k|specialoffer|specialoffers)@'i   => 5.0],
    [qr'^(stockalert|stopsnoring|wantsome|workathome|yesitsfree)@'i  => 5.0],
    [qr'^(your_friend|greatoffers)@'i                                => 5.0],
    [qr'^(inkjetplanet|marketopt|MakeMoney)\d*@'i                    => 5.0],
   ),
   { # a hash-type lookup table (associative array)
     'nobody@cert.org'                        => -3.0,
     'cert-advisory@us-cert.gov'              => -3.0,
     'owner-alert@iss.net'                    => -3.0,
     'slashdot@slashdot.org'                  => -3.0,
     'securityfocus.com'                      => -3.0,
     'ntbugtraq@listserv.ntbugtraq.com'       => -3.0,
     'security-alerts@linuxsecurity.com'      => -3.0,
     'mailman-announce-admin@python.org'      => -3.0,
     'amavis-user-admin@lists.sourceforge.net'=> -3.0,
     'amavis-user-bounces@lists.sourceforge.net' => -3.0,
     'spamassassin.apache.org'                => -3.0,
     'notification-return@lists.sophos.com'   => -3.0,
     'owner-postfix-users@postfix.org'        => -3.0,
     'owner-postfix-announce@postfix.org'     => -3.0,
     'owner-sendmail-announce@lists.sendmail.org'   => -3.0,
     'sendmail-announce-request@lists.sendmail.org' => -3.0,
     'donotreply@sendmail.org'                => -3.0,
     'ca+envelope@sendmail.org'               => -3.0,
     'noreply@freshmeat.net'                  => -3.0,
     'owner-technews@postel.acm.org'          => -3.0,
     'ietf-123-owner@loki.ietf.org'           => -3.0,
     'cvs-commits-list-admin@gnome.org'       => -3.0,
     'rt-users-admin@lists.fsck.com'          => -3.0,
     'clp-request@comp.nus.edu.sg'            => -3.0,
     'surveys-errors@lists.nua.ie'            => -3.0,
     'emailnews@genomeweb.com'                => -5.0,
     'yahoo-dev-null@yahoo-inc.com'           => -3.0,
     'returns.groups.yahoo.com'               => -3.0,
     'clusternews@linuxnetworx.com'           => -3.0,
     lc('lvs-users-admin@LinuxVirtualServer.org')    => -3.0,
     lc('owner-textbreakingnews@CNNIMAIL12.CNN.COM') => -5.0,
     # soft-blacklisting (positive score)
     'sender@example.net'                     =>  3.0,
     '.example.net'                           =>  1.0,
   },
  ],  # end of site-wide tables
});
1;  # ensure a defined return

Code: Select all

** /etc/amavis/conf.d/50-user

use strict;
@local_domains_acl = ( ".$mydomain" );
$QUARANTINEDIR = undef;
$sa_spam_subject_tag = '*****SPAM***** ';
$sa_spam_report_header = 1; # insert X-Spam-Report header field? default false
$sa_tag_level_deflt  = undef;  # add spam info headers if at, or above that level
$sa_tag2_level_deflt = 5.00;  # add 'spam detected' headers at that level
$sa_kill_level_deflt = 5.00;  # triggers spam evasive actions
$final_spam_destiny       = D_BOUNCE;
$sa_quarantine_cutoff_level = 999; # spam level beyond which quarantine is off
$spam_quarantine_to  = 'root@domain.com';
$log_level = 2;              # verbosity 0..5
1;  # ensure a defined return

Code: Select all

** /etc/clamav/freshclam.conf

NotifyClamd /etc/clamav/clamd.conf
DatabaseOwner clamav
UpdateLogFile /var/log/clamav/freshclam.log
LogVerbose false
LogSyslog false
LogFacility LOG_LOCAL6
LogFileMaxSize 0
LogTime true
Foreground false
Debug false
MaxAttempts 5
DatabaseDirectory /var/lib/clamav/
DNSDatabaseInfo current.cvd.clamav.net
AllowSupplementaryGroups false
ConnectTimeout 30
ReceiveTimeout 30
TestDatabases yes
ScriptedUpdates yes
CompressLocalDatabase no
Bytecode true
Checks 24
DatabaseMirror db.local.clamav.net
DatabaseMirror database.clamav.net

Code: Select all

** /etc/clamav/clamd.conf

LocalSocket /var/run/clamav/clamd.ctl
FixStaleSocket true
LocalSocketGroup clamav
LocalSocketMode 666
User clamav
AllowSupplementaryGroups true
ScanMail true
ScanArchive true
ArchiveBlockEncrypted false
MaxDirectoryRecursion 15
FollowDirectorySymlinks false
FollowFileSymlinks false
ReadTimeout 180
MaxThreads 12
MaxConnectionQueueLength 15
LogSyslog false
LogFacility LOG_LOCAL6
LogClean false
LogVerbose false
DatabaseDirectory /var/lib/clamav
SelfCheck 3600
Foreground false
Debug false
ScanPE true
ScanOLE2 true
ScanHTML true
DetectBrokenExecutables false
ExitOnOOM false
LeaveTemporaryFiles false
AlgorithmicDetection true
ScanELF true
IdleTimeout 30
PhishingSignatures true
PhishingScanURLs true
PhishingAlwaysBlockSSLMismatch false
PhishingAlwaysBlockCloak false
DetectPUA false
ScanPartialMessages false
HeuristicScanPrecedence false
StructuredDataDetection false
CommandReadTimeout 5
SendBufTimeout 200
MaxQueue 100
ExtendedDetectionInfo true
OLE2BlockMacros false
StreamMaxLength 25M
LogFile /var/log/clamav/clamav.log
LogTime true
LogFileUnlock false
LogFileMaxSize 0
Bytecode true
BytecodeSecurity TrustSigned
BytecodeTimeout 60000
OfficialDatabaseOnly false
CrossFilesystems true

Code: Select all

** /etc/dkimproxy/dkimproxy_in.conf

listen    127.0.0.1:10026
relay     127.0.0.1:10029

Code: Select all

** /etc/dkimproxy/dkimproxy_out.conf

listen    127.0.0.1:10027
relay     127.0.0.1:10028
domain    domain.com
signature dkim(c=relaxed)
signature domainkeys(c=nofws)
keyfile   /var/lib/dkimproxy/private.key
selector  selector1

Code: Select all

** /etc/postfix/main.cf

mailbox_size_limit = 0
message_size_limit = 30000000
queue_directory = /var/spool/postfix
command_directory = /usr/sbin
daemon_directory = /usr/lib/postfix
mail_owner = postfix
myhostname = mail.domain.com
myorigin = $mydomain
mydestination = $myhostname, localhost.$mydomain, localhost, $mydomain
virtual_alias_maps = hash:/etc/postfix/virtual
mynetworks = 192.168.1.0/8, 127.0.0.0/8
relay_domains =
virtual_alias_domains = domaintpe.com.tw
alias_maps = hash:/etc/postfix/aliases
alias_database = hash:/etc/postfix/aliases


content_filter=smtp-amavis:[127.0.0.1]:10024
smtp-amavis_destination_concurrency_limit = 20
smtpd_helo_required = yes
disable_vrfy_command = yes
smtpd_delay_reject = no
header_checks = regexp:/etc/postfix/header_checks.regexp
nested_header_checks =
smtpd_client_restrictions =
smtpd_helo_restrictions =
smtpd_sender_restrictions =
smtpd_recipient_restrictions =
        reject_unlisted_recipient,
        check_client_access    hash:/etc/postfix/GEN000_override,
        check_client_access  regexp:/etc/postfix/fqrdns.regexp,
        check_helo_access      hash:/etc/postfix/access,
        check_helo_access    regexp:/etc/postfix/helo_blacklist.regexp,
        check_sender_access    hash:/etc/postfix/blacklist,
        check_sender_access  regexp:/etc/postfix/sender_blacklist.regexp,
        check_sender_mx_access cidr:/etc/postfix/mx_access.txt,
        check_sender_access    hash:/etc/postfix/bdwl
        check_client_access    hash:/etc/postfix/broken_helos,
        reject_invalid_hostname,
        reject_non_fqdn_sender,
        reject_unknown_sender_domain,
        reject_unknown_recipient_domain,
        check_sender_access regexp:/etc/postfix/filter_10026_catchall,
        permit_mynetworks,
        reject_non_fqdn_hostname,
        reject_non_fqdn_recipient,
        reject_unauth_destination,
        check_recipient_access hash:/etc/postfix/restricted,
        reject_unknown_client,
        reject_unknown_hostname,
        reject_rbl_client zen.spamhaus.org,
        reject_rbl_client bl.spamcop.net,
smtpd_data_restrictions =
        reject_unauth_pipelining
debug_peer_level = 2
debugger_command =
         PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin
         xxgdb $daemon_directory/$process_name $process_id & sleep 5
sendmail_path = /usr/sbin/sendmail.postfix
newaliases_path = /usr/bin/newaliases.postfix
mailq_path = /usr/bin/mailq.postfix
setgid_group = postdrop
html_directory = no
manpage_directory = /usr/share/man
sample_directory = /usr/share/doc/postfix-2.1.5/samples
readme_directory = /usr/share/doc/postfix-2.1.5/README_FILES
smtpd_tls_cert_file=/etc/ssl/certs/ssl-cert-snakeoil.pem
smtpd_tls_key_file=/etc/ssl/private/ssl-cert-snakeoil.key
smtpd_use_tls=yes
smtpd_tls_session_cache_database = btree:${data_directory}/smtpd_scache
smtp_tls_session_cache_database = btree:${data_directory}/smtp_scache
inet_protocols = ipv4
receive_override_options = no_address_mappings

Code: Select all

** /etc/postfix/master.cf

smtp      inet  n       -       n       -       -       smtpd
pickup    fifo  n       -       n       60      1       pickup
        -o content_filter=dkimsign:127.0.0.1:10026
cleanup   unix  n       -       n       -       0       cleanup
qmgr      fifo  n       -       n       300     1       qmgr
rewrite   unix  -       -       n       -       -       trivial-rewrite
bounce    unix  -       -       n       -       0       bounce
defer     unix  -       -       n       -       0       bounce
trace     unix  -       -       n       -       0       bounce
verify    unix  -       -       n       -       1       verify
flush     unix  n       -       n       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
smtp      unix  -       -       n       -       -       smtp
relay     unix  -       -       n       -       -       smtp
showq     unix  n       -       n       -       -       showq
error     unix  -       -       n       -       -       error
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       n       -       -       lmtp
anvil     unix  -       -       n       -       1       anvil
maildrop  unix  -       n       n       -       -       pipe
  flags=DRhu user=vmail argv=/usr/local/bin/maildrop -d ${recipient}
old-cyrus unix  -       n       n       -       -       pipe
  flags=R user=cyrus argv=/usr/lib/cyrus-imapd/deliver -e -m ${extension} ${user}
cyrus     unix  -       n       n       -       -       pipe
  user=cyrus argv=/usr/lib/cyrus-imapd/deliver -e -r ${sender} -m ${extension} ${user}
uucp      unix  -       n       n       -       -       pipe
  flags=Fqhu user=uucp argv=uux -r -n -z -a$sender - $nexthop!rmail ($recipient)
ifmail    unix  -       n       n       -       -       pipe
  flags=F user=ftn argv=/usr/lib/ifmail/ifmail -r $nexthop ($recipient)
bsmtp     unix  -       n       n       -       -       pipe
  flags=Fq. user=foo argv=/usr/local/sbin/bsmtp -f $sender $nexthop $recipient
scache    unix  -       -       n       -       1       scache
discard   unix  -       -       n       -       -       discard
tlsmgr    unix  -       -       n       1000    1       tlsmgr
smtp-amavis unix -      -       n     -       2  smtp
    -o smtp_data_done_timeout=1200
    -o smtp_send_xforward_command=yes
    -o disable_dns_lookups=yes
    -o max_use=20
127.0.0.1:10025 inet n  -       n     -       -  smtpd
    -o content_filter=
    -o local_recipient_maps=
    -o relay_recipient_maps=
    -o smtpd_restriction_classes=
    -o smtpd_delay_reject=no
    -o smtpd_client_restrictions=permit_mynetworks,reject
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o mynetworks_style=host
    -o mynetworks=127.0.0.0/8
    -o strict_rfc821_envelopes=yes
    -o smtpd_error_sleep_time=0
    -o smtpd_soft_error_limit=1001
    -o smtpd_hard_error_limit=1000
    -o smtpd_client_connection_count_limit=0
    -o smtpd_client_connection_rate_limit=0
    -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks
retry     unix  -       -       n       -       -       error
proxywrite unix -       -       n       -       1       proxymap
submission  inet  n     -       n       -       -       smtpd
    -o smtpd_etrn_restrictions=reject
    -o smtpd_sasl_auth_enable=yes
    -o content_filter=dkimsign:[127.0.0.1]:10027
    -o receive_override_options=no_address_mappings
    -o smtpd_recipient_restrictions=permit_mynetworks,permit_sasl_authenticated,reject
    -o smtpd_client_restrictions=permit_mynetworks,reject
dkimsign    unix  -       -       n       -       10       smtp
    -o smtp_send_xforward_command=yes
    -o smtp_discard_ehlo_keywords=8bitmime,starttls
127.0.0.1:10028 inet  n  -      n       -       10      smtpd
    -o content_filter=
    -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
    -o smtpd_helo_restrictions=
    -o smtpd_client_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o mynetworks=127.0.0.0/8
    -o smtpd_authorized_xforward_hosts=127.0.0.0/8
127.0.0.1:10029 inet  n  -      n       -       10      smtpd
    -o content_filter=
    -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
    -o smtpd_helo_restrictions=
    -o smtpd_client_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o mynetworks=127.0.0.0/8
    -o smtpd_authorized_xforward_hosts=127.0.0.0/8

Code: Select all

** /etc/default/spamassassin

ENABLED=1
OPTIONS="--create-prefs --max-children 5 --helper-home-dir"
PIDFILE="/var/run/spamd.pid"
CRON=5

Code: Select all

** /etc/spamassassin/local.cf

report_safe 1
trusted_networks 192.168.1.0/24
score URIBL_BLACK 3.00
score URIBL_RHS_DOB 3.00
score ALL_TRUSTED -2.50
score BAYES_99 5.00
use_auto_whitelist 0
add_header all Report _REPORT_
bayes_file_mode 0777
use_pyzor 1
pyzor_path /usr/bin/pyzor
use_razor2 1
razor_config /etc/razor/razor-agent.conf
use_bayes 1
use_bayes_rules 1
bayes_auto_learn 1
Many thanks for any assistance!

kedaha
Posts: 3521
Joined: 2008-05-24 12:26
Has thanked: 33 times
Been thanked: 77 times

Re: postfix/amavis not scanning email

#2 Post by kedaha »

bithead wrote:Hi all,

I'm running a postfix, amavis, spamassassin, dkimproxy, clamav system on Debian 8.
Hi,
Setting up an email system is one of the most arduous tasks I've done and you're really on your own but I'd just like to comment that I too have "been through several configuration documents and tried many suggestions found within." It is a good idea —so others can help— to specify what howto, tutorial or other documentation you have used; the smallest imprecision can throw a spanner in the works.
In case it's of interest, I'd mention that I successfully set up my own email system using the guide for "jessie" at workaround.org and this is currently my working system; however, I wished to try other configurations including components like amavis and particularly postfixadmin, not documented there so I tested the setup detailed in this postfix-mail-server-setup-ubuntu-14-04/. Of course I used Debian, not Ubuntu and used postfixadmin from Debian's stable main repository in preference to the downloaded "latest tarball." I can report that I got the system up and running perfectly on a raspberry pi server set up to run Debian as per debian-jessie-on-rpi2/. My next step is to test this configuration for "stretch" with a view to using it when it gets released as stable.
DebianStable

Code: Select all

$ vrms

No non-free or contrib packages installed on debian!  rms would be proud.

dcihon
Posts: 279
Joined: 2012-09-23 16:18
Been thanked: 3 times

Re: postfix/amavis not scanning email

#3 Post by dcihon »


Post Reply