# Postfix + VirtualMailbox + Dovecot + Dspam problem



## cr4sh (Oct 29, 2013)

Hello. 

I try to build a mail system that allows to train SPAM/notSPAM via moving mails to/out SPAM folder. But *I* have a couple of problems with it.

Versions of FreeBSD, Dovecot, Dspam and Postfix.

```
FreeBSD mta 9.1-STABLE FreeBSD 9.1-STABLE
Dovecot: 2.2.6
Postfix: 2.10.1,1
Dspam: 3.9.0
```

First *I* show my configurations:
Postfix - main.cf

```
dovecot_destination_recipient_limit = 1
smtpd_helo_required = yes
smtpd_helo_restrictions = permit_mynetworks,
  permit_sasl_authenticated,
  reject_invalid_helo_hostname,
  reject_non_fqdn_helo_hostname
smtpd_reject_unlisted_sender = yes
smtpd_recipient_restrictions =
  permit_sasl_authenticated,
  reject_unknown_sender_domain,
  reject_non_fqdn_sender,
  reject_non_fqdn_hostname,
  reject_non_fqdn_recipient,
  reject_unknown_recipient_domain,
  permit_mynetworks,
  reject_non_fqdn_helo_hostname,
  reject_unauth_destination,
  reject_unauth_pipelining,
  reject_invalid_hostname,
  reject_rbl_client bl.spamcop.net
  check_policy_service inet:127.0.0.1:10023
smtpd_sender_login_maps =  pgsql:/usr/local/etc/postfix/pgsql_virtual_mismatch.cf
smtpd_sender_restrictions = reject_authenticated_sender_login_mismatch, permit_sasl_authenticated,  permit_mynetworks
smtpd_data_restrictions = reject_multi_recipient_bounce
virtual_mailbox_base = /mail
virtual_mailbox_maps = pgsql:/usr/local/etc/postfix/pgsql_virtual_mailbox_maps.cf
virtual_mailbox_domains = pgsql:/usr/local/etc/postfix/pgsql_virtual_mailbox_domains.cf
virtual_alias_maps = pgsql:/usr/local/etc/postfix/pgsql_virtual_alias_maps.cf
local_recipient_maps = $virtual_mailbox_maps
recipient_bcc_maps = pgsql:/usr/local/etc/postfix/pgsql_recipient_bcc_maps.cf
virtual_uid_maps = static:26
virtual_gid_maps = static:6
virtual_transport = dovecot
smtpd_delay_reject = yes
smtpd_helo_required = yes
broken_sasl_auth_clients = yes
smtpd_sasl_auth_enable = yes
smtpd_sasl_type = dovecot
smtpd_sasl_path = /var/spool/postfix/private/auth
smtpd_sasl_security_options = noanonymous
smtp_use_tls = yes
smtpd_use_tls = yes
smtp_tls_note_starttls_offer = yes
smtpd_tls_key_file = /etc/ssl/postfix/privatekey.pem
smtpd_tls_cert_file = /etc/ssl/postfix/cert.pem
smtpd_tls_CAfile = /etc/ssl/postfix/root.crt
smtpd_tls_loglevel = 1
smtpd_tls_received_header = yes
smtpd_tls_session_cache_timeout = 3600s
tls_random_source = dev:/dev/urandom
queue_directory = /var/spool/postfix
command_directory = /usr/local/sbin
daemon_directory = /usr/local/libexec/postfix
data_directory = /var/db/postfix
mail_owner = postfix
myhostname = mta.domain.pl
mydomain = domain.pl
mydestination = localhost.$mydomain, localhost
unknown_local_recipient_reject_code = 550
mynetworks_style = subnet
mynetworks = 192.168.1.0/24, 192.168.2.0/24, 192.168.4.0/24, 192.168.5.0/24, 192.168.7.0/24, 192.168.200.0/24, 10.10.3.0/24
relay_domains = pgsql:/usr/local/etc/postfix/pgsql_relay_domains.cf
recipient_delimiter = +
debugger_command =
         PATH=/bin:/usr/bin:/usr/local/bin:/usr/X11R6/bin
         ddd $daemon_directory/$process_name $process_id & sleep 5
sendmail_path = /usr/local/sbin/sendmail
newaliases_path = /usr/local/bin/newaliases
mailq_path = /usr/local/bin/mailq
setgid_group = maildrop
html_directory = /usr/local/share/doc/postfix
manpage_directory = /usr/local/man
sample_directory = /usr/local/etc/postfix
readme_directory = /usr/local/share/doc/postfix
message_size_limit=36700160
inet_protocols = ipv4
smtpd_milters = inet:192.168.1.252:8891
content_filter = scan:192.168.1.252:10025
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated      defer_unauth_destination
```

Postfix - master.cf

```
smtp      inet  n       -       n       -       -       smtpd
        -o content_filter=lmtp:unix:/var/run/dspam.sock

smtps     inet  n       -       n       -       -       smtpd
  -o syslog_name=postfix/smtps
  -o smtpd_tls_wrappermode=yes
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
  -o message_size_limit=36700160

pickup    fifo  n       -       n       60      1       pickup
cleanup   unix  n       -       n       -       0       cleanup
qmgr      fifo  n       -       n       300     1       qmgr
tlsmgr    unix  -       -       n       1000?   1       tlsmgr
rewrite   unix  -       -       n       -       -       trivial-rewrite
bounce    unix  -       -       n       -       0       bounce
defer     unix  -       -       n       -       0       bounce
trace     unix  -       -       n       -       0       bounce
verify    unix  -       -       n       -       1       verify
flush     unix  n       -       n       1000?   0       flush
proxymap  unix  -       -       n       -       -       proxymap
proxywrite unix -       -       n       -       1       proxymap
smtp      unix  -       -       n       -       -       smtp
relay     unix  -       -       n       -       -       smtp
showq     unix  n       -       n       -       -       showq
error     unix  -       -       n       -       -       error
retry     unix  -       -       n       -       -       error
discard   unix  -       -       n       -       -       discard
local     unix  -       n       n       -       -       local
virtual   unix  -       n       n       -       -       virtual
lmtp      unix  -       -       n       -       -       lmtp
anvil     unix  -       -       n       -       1       anvil
scache    unix  -       -       n       -       1       scache

dovecot    unix -        n       n       -       -       pipe
        flags=DRhu user=mailnull:mail argv=/usr/local/libexec/dovecot/deliver -f ${sender} -d ${user}@${nexthop} -m ${extension}

scan      unix  -       -       n       -       16      smtp
        -o smtp_send_xforward_command=yes
        -o smtp_enforce_tls=no

192.168.1.252:10026 inet  n -       n       -       16      smtpd
        -o content_filter=
        -o receive_override_options=no_unknown_recipient_checks,no_header_body_checks
        -o smtpd_helo_restrictions=
        -o smtpd_client_restrictions=
        -o smtpd_sender_restrictions=
        -o smtpd_recipient_restrictions=permit_mynetworks,reject
        -o mynetworks_style=host
        -o smtpd_authorized_xforward_hosts=192.168.1.252
```


----------



## cr4sh (Oct 29, 2013)

DSpam, dspam.conf:

```
Home /var/db/dspam
StorageDriver /usr/local/lib/dspam/libpgsql_drv.so
QuarantineAgent "/usr/libexec/mail.local"
DeliveryHost            192.168.1.252
DeliveryPort            10025
DeliveryIdent           localhost
DeliveryProto           SMTP
OnFail error
Trust root
Trust dspam
Trust apache
Trust mail
Trust mailnull 
Trust smmsp
Trust daemon
Debug *
DebugOpt process classify spam fp inoculation corpus
TrainingMode toe
TestConditionalTraining on
Feature whitelist
Algorithm graham burton
Tokenizer osb
PValue bcr
WebStats on
Preference "trainingMode=TOE"           # { TOE | TUM | TEFT | NOTRAIN } -> default:teft
Preference "spamAction=deliver" # { quarantine | tag | deliver } -> default:quarantine
Preference "spamSubject=[SPAM]"         # { string } -> default:[SPAM]
Preference "statisticalSedation=5"      # { 0 - 10 } -> default:0
Preference "enableBNR=on"               # { on | off } -> default:off
Preference "enableWhitelist=on"         # { on | off } -> default:on
Preference "signatureLocation=headers"  # { message | headers } -> default:message
Preference "tagSpam=off"                # { on | off }
Preference "tagNonspam=off"             # { on | off }
Preference "showFactors=off"            # { on | off } -> default:off
Preference "optIn=off"                  # { on | off }
Preference "optOut=off"                 # { on | off }
Preference "whitelistThreshold=10"      # { Integer } -> default:10
Preference "makeCorpus=off"             # { on | off } -> default:off
Preference "storeFragments=off"         # { on | off } -> default:off
Preference "localStore="                # { on | off } -> default:username
Preference "processorBias=on"           # { on | off } -> default:on
Preference "fallbackDomain=off"         # { on | off } -> default:off
Preference "trainPristine=off"          # { on | off } -> default:off
Preference "optOutClamAV=off"           # { on | off } -> default:off
Preference "ignoreRBLLookups=off"       # { on | off } -> default:off
Preference "RBLInoculate=off"           # { on | off } -> default:off
AllowOverride enableBNR
AllowOverride enableWhitelist
AllowOverride fallbackDomain
AllowOverride ignoreGroups
AllowOverride ignoreRBLLookups
AllowOverride localStore
AllowOverride makeCorpus
AllowOverride optIn
AllowOverride optOut
AllowOverride optOutClamAV
AllowOverride processorBias
AllowOverride RBLInoculate
AllowOverride showFactors
AllowOverride signatureLocation
AllowOverride spamAction
AllowOverride spamSubject
AllowOverride statisticalSedation
AllowOverride storeFragments
AllowOverride tagNonspam
AllowOverride tagSpam
AllowOverride trainPristine
AllowOverride trainingMode
AllowOverride whitelistThreshold
AllowOverride dailyQuarantineSummary
PgSQLServer             /tmp/
PgSQLUser               dspam
PgSQLPass               MWwDHj8uGoEa
PgSQLDb                 dspam
HashRecMax              98317
HashAutoExtend          on  
HashMaxExtents          0
HashExtentSize          49157
HashPctIncrease         10
HashMaxSeek             10
HashConnectionCache     10
IgnoreHeader DKIM-Signature
IgnoreHeader X-Bogosity
IgnoreHeader X-Spam-Checker-Version
IgnoreHeader X-Spam-Flag
IgnoreHeader X-Spam-Level
IgnoreHeader X-Spam-Status
IgnoreHeader X-GMX-Antispam
IgnoreHeader X-GMX-Antivirus
IgnoreHeader X-UI-Filterresults
Notifications   off
PurgeSignatures 14      # Stale signatures
PurgeNeutral    90      # Tokens with neutralish probabilities
PurgeUnused     90      # Unused tokens
PurgeHapaxes    30      # Tokens with less than 5 hits (hapaxes)
PurgeHits1S     15      # Tokens with only 1 spam hit
PurgeHits1I     15      # Tokens with only 1 innocent hit
 #      10.0.0.0/8      - Private IP addresses (RFC 1918)
LocalMX 127.0.0.1
SystemLog       on
UserLog         on
Opt out
ParseToHeaders on
ChangeModeOnParse on
ChangeUserOnParse full
ServerPID               /var/run/dspam.pid
ServerMode auto
ServerParameters        "--deliver=innocent -d %u"
ServerIdent             "localhost.localdomain"
ServerPID               /var/run/dspam.pid
ServerDomainSocketPath  "/var/run/dspam.sock"
ClientHost      /var/run/dspam.sock
ProcessorURLContext on
ProcessorBias on
StripRcptDomain off
```

Dovecot, dovecot.conf

```
auth_mechanisms = plain login
disable_plaintext_auth = no
login_greeting = Welcome to TLGPoland Mailserver.
log_path = /var/log/dovecot.log
debug_log_path = /var/log/dovecot-debug.log
mail_debug = yes
first_valid_uid = 26
last_valid_uid = 26
mail_uid = mailnull
first_valid_gid = 6
last_valid_gid = 6
mail_gid = mail
mail_location = maildir:/mail/%d/%n
ssl_ca =</etc/ssl/postfix/root.crt
ssl_cert =</etc/ssl/postfix/cert.pem
ssl_key =</etc/ssl/postfix/privatekey.pem
protocols = imap pop3 lmtp sieve
passdb {
    args = /usr/local/etc/dovecot/dovecot-sql.conf
    driver = sql
}
userdb {
    args = /usr/local/etc/dovecot/dovecot-sql.conf
    driver = sql
}
plugin {
    quota = maildir
    sieve_max_script_size = 512K
    sieve_dir = /mail/%d/%n/sieve/
    sieve = /mail/%d/%n/.dovecot.sieve
    mail_log_events = delete undelete expunge copy mailbox_delete mailbox_rename append flag_change
    mail_log_fields = uid box msgid size subject from
        antispam_backend = dspam
        antispam_dspam_binary = /usr/local/bin/dspam
        antispam_dspam_args = --client;--user;%Lu;--source=error;--deliver=spam,innocent;--signature=%%s
        antispam_dspam_spam = --class=spam
        antispam_dspam_notspam = --class=innocent
        antispam_spam = Spam
        antispam_signature = X-DSPAM-Signature
        antispam_signature_missing = error
        antispam_dspam_result_header = X-DSPAM-Result
}
service auth {
    unix_listener /var/spool/postfix/private/auth {
        group = postfix
        mode = 0660
        user = postfix
    }
}
service lmtp {
    unix_listener /var/spool/postfix/private/dovecot-lmtp {
        group = postfix
        mode = 0660
        user = postfix
    }
}
service imap-login {
    chroot = login
    client_limit = 256
    executable = /usr/local/libexec/dovecot/imap-login
    inet_listener imap {
        address = *
        port = 143
    }
    inet_listener imaps {
        address = *
        port = 993
    }
    process_limit = 128
    process_min_avail = 2
    service_count = 1
}
service imap {
    executable = /usr/local/libexec/dovecot/imap
}
service pop3-login {
    chroot = login
    client_limit = 256
    executable = /usr/local/libexec/dovecot/pop3-login
    inet_listener pop3 {
        address = *
        port = 110
    }
    inet_listener pop3s {
        address = *
        port = 995
    }
    process_limit = 128
    process_min_avail = 2
    service_count = 1
}
service pop3 {
    executable = /usr/local/libexec/dovecot/pop3
}
service managesieve-login {
    inet_listener sieve {
        port = 4190
    }
    inet_listener sieve_deprecated {
        port = 2000
    }
}
service managesieve {
}
protocol sieve {
    managesieve_max_line_length = 65536
    mail_max_userip_connections = 10
    managesieve_logout_format = bytes=%i/%o
    managesieve_implementation_string = Dovecot Pigeonhole
    #managesieve_sieve_capability =
    #managesieve_notify_capability =
    managesieve_max_compile_errors = 5
}
protocol imap {
    imap_client_workarounds = delay-newmail   tb-extra-mailbox-sep
    imap_max_line_length = 64 k
    mail_plugins = quota, antispam
}
protocol pop3 {
    pop3_client_workarounds = outlook-no-nuls oe-ns-eoh
    pop3_uidl_format = %08Xu%08Xv
}
protocol lmtp {
    mail_plugins = quota, sieve, antispam
}
protocol lda {
       mail_plugins = quota, sieve, antispam
}
```

Now *I* describe my problems.

 Incoming mails after ClamAV go to Postfix via SMTP again (twice OpenDKIM)
 After moving mails to/from SPAM folder *I* get clean mail from MAILER-DEAMON with body 

```
X-DSPAM-Reclassified: Spam
```

Postfix logs:

```
Oct 29 07:49:05 mx postfix/smtpd[53773]: connect from mx2.testserve.pl[10.23.23.2]
Oct 29 07:49:05 mx postfix/smtpd[53773]: Anonymous TLS connection established from mx2.testserve.pl[10.23.23.2]: TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)
Oct 29 07:49:06 mx postgrey[6613]: action=pass, reason=triplet found, client_name=mx2.testserve.pl, client_address=10.23.23.2, sender=testsender@testdomain.pl, recipient=recipient@mta.pl
Oct 29 07:49:06 mx postfix/smtpd[53773]: 15373189DD8: client=mx2.testserve.pl[10.23.23.2]
Oct 29 07:49:06 mx postfix/cleanup[53726]: 15373189DD8: message-id=<3b67d463455250e8479921f4ea6d64f1@testdomain.pl>
[B]Oct 29 07:49:06 mx opendkim[6579]: 15373189DD8: mx2.testserve.pl [10.23.23.2] not internal
Oct 29 07:49:06 mx opendkim[6579]: 15373189DD8: not authenticated
Oct 29 07:49:16 mx opendkim[6579]: 15373189DD8: message has signatures from testdomain.pl, testdomain.pl
Oct 29 07:49:16 mx opendkim[6579]: 15373189DD8: key retrieval failed (s=dkim, d=testdomain.pl): 'dkim._domainkey.testdomain.pl' query timed out
[/B]Oct 29 07:49:16 mx postfix/qmgr[42755]: 15373189DD8: from=<testsender@testdomain.pl>, size=2348, nrcpt=1 (queue active)
Oct 29 07:49:16 mx postfix/smtpd[53773]: disconnect from mx2.testserve.pl[10.23.23.2]
Oct 29 07:49:17 mx clamsmtpd: 100036: accepted connection from: 192.168.1.252
Oct 29 07:49:17 mx postfix/smtpd[53735]: connect from mta.domain.pl[192.168.1.252]
Oct 29 07:49:17 mx postfix/smtpd[53735]: 800DA189DDE: client=mta.domain.pl[192.168.1.252]
Oct 29 07:49:17 mx postfix/cleanup[53726]: 800DA189DDE: message-id=<3b67d463455250e8479921f4ea6d64f1@testdomain.pl>
[B]
Oct 29 07:49:17 mx opendkim[6579]: 800DA189DDE: mta.domain.pl [192.168.1.252] not internal
Oct 29 07:49:17 mx opendkim[6579]: 800DA189DDE: not authenticated
Oct 29 07:49:23 mx opendkim[6579]: 800DA189DDE: message has signatures from testdomain.pl, testdomain.pl
[/B]Oct 29 07:49:23 mx opendkim[6579]: 800DA189DDE: key retrieval failed (s=dkim, d=testdomain.pl): 'dkim._domainkey.testdomain.pl' query timed out
Oct 29 07:49:23 mx clamsmtpd: 100036: from=testsender@testdomain.pl, to=recipient@mta.pl, status=CLEAN
Oct 29 07:49:23 mx postfix/qmgr[42755]: 800DA189DDE: from=<testsender@testdomain.pl>, size=2751, nrcpt=1 (queue active)
Oct 29 07:49:23 mx postfix/pipe[53737]: 800DA189DDE: to=<recipient@mta.pl>, relay=dovecot, delay=5.9, delays=5.8/0/0/0.07, dsn=2.0.0, status=sent (delivered via dovecot service)
Oct 29 07:49:23 mx postfix/qmgr[42755]: 800DA189DDE: removed
Oct 29 07:49:23 mx postfix/smtpd[53735]: disconnect from mta.domain.pl[192.168.1.252]
Oct 29 07:49:23 mx postfix/lmtp[53781]: 15373189DD8: to=<recipient@mta.pl>, relay=mta.domain.pl[/var/run/dspam.sock], delay=17, delays=10/0/0/7.1, dsn=2.6.0, status=sent (250 2.6.0 <recipient@mta.pl> Message
accepted for delivery)
Oct 29 07:49:23 mx postfix/qmgr[42755]: 15373189DD8: removed
```

I'm trying to create a scenario:

```
Incoming mail > DSpam > OpenDKIM > ClamAV > Dovecot
```
But without multiple SMTP mail inserts. 
I will be very grateful for your help because I have no idea how to do it right.
Regards.


----------

