gpupdate /force
Regular Expression
Regular expression is a sequence of character(s) mainly used to find and replace patterns in a string or file.
So we can say that the task of searching and extracting is so common that Python has a very powerful library called regular expressions that handles many of these tasks quite elegantly.
| SYMBOL | USAGE |
|---|---|
| $ | Matches the end of the line |
| \s | Matches whitespace |
| \S | Matches any non-whitespace character |
| * | Repeats a character zero or more times |
| \S | Matches any non-whitespace character |
| *? | Repeats a character zero or more times (non-greedy) |
| + | Repeats a character one or more times |
| +? | Repeats a character one or more times (non-greedy) |
| [aeiou] | Matches a single character in the listed set |
| [^XYZ] | Matches a single character not in the listed set |
| [a-z0-9] | The set of characters can include a range |
| ( | Indicates where string extraction is to start |
| ) | Indicates where string extraction is to end |
How email works (MTA, MDA, MUA)
As simple as it is to use, email relies on a more complicated set of operating procedures than that of the Web. For most users, its operation is transparent, which means that it is not necessary to understand how email works in order to be able to use it.
However, the short introduction below has been provided to help you to understand its basic principles, give you an idea of how to best configure your email clients, and inform you about the underlying mechanisms of spam.
How Email Works
Email is based around the use of electronic mailboxes. When an email is sent, the message is routed from server to server, all the way to the recipient’s email server. More specifically, the message is sent to the mail server tasked with transporting emails (called the MTA, for Mail Transport Agent) to the recipient’s MTA. On the Internet, MTAs communicate with one another using the protocol SMTP, and so are logically called SMTP servers (or sometimes outgoing mail servers).
The recipient’s MTA then delivers the email to the incoming mail server (called the MDA, for Mail Delivery Agent), which stores the email as it waits for the user to accept it. There are two main protocols used for retrieving email on an MDA: POP3 (Post Office Protocol), the older of the two, which is used for retrieving email and, in certain cases, leaving a copy of it on the server; and IMAP (Internet Message Access Protocol), which is used for coordinating the status of emails (read, deleted, moved) across multiple email clients. With IMAP, a copy of every message is saved on the server, so that this synchronization task can be completed.
For this reason, incoming mail servers are called POP servers or IMAP servers, depending on which protocol is used:
To keep everyone from checking other users’ emails, MDA is protected by a user name called a login and by a password.
Retrieving mail is done using a software program called an MUA (Mail User Agent). When the MUA is a program installed on the user’s system, it is called an email client (such as Mozilla Thunderbird, Microsoft Outlook, Eudora Mail, Incredimail or Lotus Notes).
When it is a web interface used for interacting with the incoming mail server, it is called webmail.
Open Relay
By default, it is not necessary to authenticate oneself to send email, which means that it is very easy to falsify one’s own address when sending mail. For this reason, nearly all Internet service providers lock down their SMTP servers so that only their subscribers can use them, or more precisely, only machines whose IP address belongs to the ISP’s domain. This explains why users must modify the outgoing server settings in their email clients each time they move to a new home or business.
When an organization’s email server is improperly configured and allows third-party users on any network to send emails, this is called an open relay. Open relays are generally used by spammers, as using them hides the true origins of their messages. As a result, many ISPs keep an up-to-date blacklist of open relays to keep subscribers from receiving messages from such servers.
Enigma plugin (PGP encryption) Roundcube
You have to set up Enigma plugin :
cp /……../roundcubemail/plugins/enigma/config.inc.php.dist /……../roundcubemail/plugins/enigma/config.inc.php
<?php
// Enigma Plugin options
// --------------------
// A driver to use for PGP. Default: "gnupg".
$config['enigma_pgp_driver'] = 'gnupg';
// A driver to use for S/MIME. Default: "phpssl".
$config['enigma_smime_driver'] = 'phpssl';
// Enables logging of enigma operations (including Crypt_GPG debug info)
$config['enigma_debug'] = true;
// Keys directory for all users. Default 'enigma/home'.
// Must be writeable by PHP process
$config['enigma_pgp_homedir'] = enigma/home;
// Location of gpg binary. By default it will be auto-detected.
// This is also a way to force gpg2 use if there are both 1.x and 2.x on the system.
$config['enigma_pgp_binary'] = '/usr/bin/gpg';
// Location of gpg-agent binary. By default it will be auto-detected.
// It's used with GnuPG 2.x.
$config['enigma_pgp_agent'] = '';
// Location of gpgconf binary. By default it will be auto-detected.
// It's used with GnuPG >= 2.1.
$config['enigma_pgp_gpgconf'] = '';
// Enables signatures verification feature.
$config['enigma_signatures'] = true;
// Enables messages decryption feature.
$config['enigma_decryption'] = true;
// Enables messages encryption and signing feature.
$config['enigma_encryption'] = true;
// Enable signing all messages by default
$config['enigma_sign_all'] = false;
// Enable encrypting all messages by default
$config['enigma_encrypt_all'] = false;
// Enable attaching a public key to all messages by default
$config['enigma_attach_pubkey'] = false;
// Default for how long to store private key passwords (in minutes).
// When set to 0 passwords will be stored for the whole session.
$config['enigma_password_time'] = 0;
// With this option you can lock composing options
// of the plugin forcing the user to use configured settings.
// The array accepts: 'sign', 'encrypt', 'pubkey'.
//
// For example, to force your users to sign every email,
// you should set:
// - enigma_sign_all = true
// - enigma_options_lock = array('sign')
// - dont_override = array('enigma_sign_all')
$config['enigma_options_lock'] = array();
Installation and settings Enigma plugin
Preferences > Settings > Encryption you’ll have the possibility to enable/disable encryption-related features.
The keys are stored on the server.
Figure 1. Encryption preferences section.
Goto Settings > PGP Keys. There you can generate a new key pair or import keys.
Figure 2. Key generation form.
Figure 3. Key information frame.
Composing messages
Figure 4. Encryption options in compose.
Install Milter Manager on Centos 7
Install packages
Postfix is used as MTA because it’s installed by default.
Spamass-milter, clamav-milter and milter-greylist are used as milters. Milter packages registered in EPEL are used.
Register EPEL like the following:
% sudo yum install -y epel-release
Now, you install milters:
% sudo yum install -y spamass-milter-postfix clamav-scanner-systemd clamav-update clamav-milter clamav-milter-systemd milter-greylist
And you install RRDtool for generating graphs:
% sudo yum install -y rrdtool
Build and Install
milter manager can be installed by yum.
Register milter manager yum repository like the following:
% curl -s https://packagecloud.io/install/repositories/milter-manager/repos/script.rpm.sh | sudo bash
See also: <URL:https://packagecloud.io/milter-manager/repos/install>
Now, you install milter manager:
% sudo yum install -y milter-manager
Configuration
Here is a basic configuration policy.
milter-greylist should be applied only if S25R condition is matched to reduce needless delivery delay. But the configuration is automatically done by milter manager. You need to do nothing for it.
It’s difficult that milter manager runs on SELinux. Disable SELinux policy module for Postfix and Milter.
% sudo semodule -d postfix % sudo semodule -d milter
Configure spamass-milter
At first, you configure spamd.
spamd adds “[SPAM]” to spam mail’s subject by default. If you don’t like the behavior, edit /etc/mail/spamassassin/local.cf.
Before:
rewrite_header Subject [SPAM]
After:
# rewrite_header Subject [SPAM]
Add the following configuration to /etc/mail/spamassassin/local.cf. This configuration is for adding headers only if spam detected.
remove_header ham Status remove_header ham Level
Start spamd on startup:
% sudo systemctl enable spamassassin
Start spamd:
% sudo systemctl start spamassassin
Here are spamass-milter’s configuration items:
- Disable needless body change feature.
- Reject if score is larger than or equal to 15.
Change /etc/sysconfig/spamass-milter:
Before:
#EXTRA_FLAGS="-m -r 15"
After:
EXTRA_FLAGS="-m -r 15"
Start spamass-milter on startup:
% sudo systemctl enable spamass-milter
Start spamass-milter:
% sudo systemctl start spamass-milter
Configure clamav-milter
Update ClamAV virus database and start clamd.
Edit /etc/freshclam.conf like the following. It comments out “Example”, changes “NotifyClamd” value and uncomments other items.
Before:
Example #LogFacility LOG_MAIL #NotifyClamd /path/to/clamd.conf
After:
#Example LogFacility LOG_MAIL NotifyClamd /etc/clamd.d/scan.conf
Run freshclam by hand at the first time:
% sudo freshclam
Configure clamd.
Edit /etc/clamd.d/scan.conf like the following. It comments out “Example” and uncomments other items:
Before:
Example #LogFacility LOG_MAIL #LocalSocket /run/clamd.scan/clamd.sock
After:
#Example LogFacility LOG_MAIL LocalSocket /run/clamd.scan/clamd.sock
Start clamd on startup:
% sudo systemctl enable clamd@scan
Start clamd:
% sudo systemctl start clamd@scan
Configure clamav-milter.
Edit /etc/mail/clamav-milter.conf like the following. It comments out “Example”, change “ClamdSocket” value and uncomments other items:
Before:
Example #MilterSocket /run/clamav-milter/clamav-milter.socket #MilterSocketMode 660 #ClamdSocket tcp:scanner.mydomain:7357 #LogFacility LOG_MAIL
After:
#Example MilterSocket /run/clamav-milter/clamav-milter.socket MilterSocketMode 660 ClamdSocket unix:/run/clamd.scan/clamd.sock LogFacility LOG_MAIL
Add “clamilt” user to “clamscan” group to access clamd’s socket:
% sudo usermod -G clamscan -a clamilt
Start clamav-milter on startup:
% sudo systemctl enable clamav-milter
Start clamav-milter:
% sudo systemctl start clamav-milter
Configure milter-greylist
Change /etc/mail/greylist.conf for the following configurations:
- use the leading 24bits for IP address match to avoid Greylist adverse effect for sender uses some MTA case.
- decrease retransmit check time to 10 minutes from 30 minutes (default value) to avoid Greylist adverse effect.
- increase auto whitelist period to a week from 1 day (default value) to avoid Greylist adverse effect.
- don’t use Greylist when trusted domain passes SPF. (Trusted domains are configured in milter manager)
- use Greylist by default.
The configuration relaxes Greylist check to avoid Greylist adverse effect. It increases received spam mails but you should give priority to avoid false positive rather than false negative. You should not consider that you blocks all spam mails by Greylist. You can blocks spam mails that isn’t blocked by Greylist by other anti-spam technique such as SpamAssassin. milter manager helps constructing mail system that combines some anti-spam techniques.
Before:
socket "/run/milter-greylist/milter-greylist.sock" # ... racl whitelist default
After:
socket "/run/milter-greylist/milter-greylist.sock" 660
# ...
subnetmatch /24
greylist 10m
autowhite 1w
sm_macro "trusted_domain" "{trusted_domain}" "yes"
racl whitelist sm_macro "trusted_domain" spf pass
racl greylist sm_macro "trusted_domain" not spf pass
racl greylist default
Start milter-greylist on startup:
% sudo systemctl enable milter-greylist
Start milter-greylist:
% sudo systemctl start milter-greylist
Configure milter manager
Add “milter-manager” user to “clamilt” group to access clamav-milter’s socket:
% sudo usermod -G clamilt -a milter-manager
Add “milter-manager” user to “mail” group and “grmilter” group to access milter-greylist’s socket:
% sudo usermod -G mail -a milter-manager % sudo usermod -G grmilter -a milter-manager
Add “milter-manager” user to “postfix”” group to access spamass-milter’s socket:
% sudo usermod -G postfix -a milter-manager
milter manager detects milters that installed in system. You can confirm spamass-milter, clamav-milter and milter-greylist are detected:
% sudo /usr/sbin/milter-manager -u milter-manager -g milter-manager --show-config
The following output shows milters are detected:
...
define_milter("milter-greylist") do |milter|
milter.connection_spec = "unix:/run/milter-greylist/milter-greylist.sock"
...
milter.enabled = true
...
end
...
define_milter("clamav-milter") do |milter|
milter.connection_spec = "unix:/var/run/clamav-milter/clamav-milter.socket"
...
milter.enabled = true
...
end
...
define_milter("spamass-milter") do |milter|
milter.connection_spec = "unix:/run/spamass-milter/postfix/sock"
...
milter.enabled = true
...
end
...
You should confirm that milter’s name, socket path and “enabled = true”. If the values are unexpected, you need to change /etc/milter-manager/milter-manager.local.conf. See Configuration for details of milter-manager.local.conf.
But if we can, we want to use milter manager without editing miter-manager.local.conf. If you report your environment to the milter manager project, the milter manager project may improve detect method.
milter manager’s configuration is finished.
Start to milter manager on startup:
% sudo systemctl enable milter-manager
Start to milter manager:
% sudo systemctl start milter-manager
milter-test-server is usuful to confirm milter manager was ran:
% sudo -u milter-manager milter-test-server -s unix:/var/run/milter-manager/milter-manager.sock
Here is a sample success output:
status: accept elapsed-time: 0.128 seconds
If milter manager fails to run, the following message will be shown:
Failed to connect to unix:/var/run/milter-manager/milter-manager.sock
In this case, you can use log to solve the problem. milter manager is verbosily if –verbose option is specified. milter manager outputs logs to standard output if milter manager isn’t daemon process.
You can add the following configuration to /etc/sysconfig/milter-manager to output verbose log to standard output:
OPTION_ARGS="--verbose --no-daemon"
Restart milter manager:
% sudo systemctl restart milter-manager
Some logs are output if there is a problem. Running milter manager can be exitted by Ctrl+c.
OPTION_ARGS configuration in /etc/sysconfig/milter-manager should be commented out after the problem is solved to run milter manager as daemon process. And you should restart milter manager.
Configure Postfix
Enables Postfix:
% sudo systemctl enable postfix % sudo systemctl start postfix
Configure Postfix for milters. Append following lines to /etc/postfix/main.cf:
milter_protocol = 6
milter_default_action = accept
milter_mail_macros = {auth_author} {auth_type} {auth_authen}
For details for each lines.
Register milter manager to Postfix. It’s important that spamass-milter, clamav-milter and milter-greylist aren’t needed to be registered because they are used via milter manager.
Append following lines to /etc/postfix/main.cf:
smtpd_milters = unix:/var/run/milter-manager/milter-manager.sock
Reload Postfix’s configuration.
% sudo systemctl reload postfix
milter manager logs to syslog. If milter manager works well, some logs can be shown in /var/log/maillog. You need to send a test mail for confirming.
Conclusion
There are many configurations to work milter and Postfix together. They can be reduced by introducing milter manager.
Without milter manager, you need to specify sockets of spamass-milter, clamav-milter and milter-greylist to /etc/postfix/main.cf. With milter manager, you don’t need to specify sockets of them, just specify a socket of milter manager. They are detected automatically. You don’t need to take care some small mistakes like typo.
milter manager also detects which ‘/sbin/chkconfig -add’ is done or not. If you disable a milter, you use the following steps:
% sudo systemctl stop milter-greylist % sudo systemctl disable milter-greylist
You need to reload milter manager after you disable a milter.
% sudo systemctl reload milter-manager
milter manager detects a milter is disabled and doesn’t use it. You don’t need to change /etc/postfix/main.cf.
You can reduce maintainance cost by introducing milter manager if you use some milters on CentOS.
milter manager also provides tools to help operation. Installing them is optional but you can reduce operation cost too. If you also install them, you will go to Install to CentOS (optional) .
ISSUE:
after the installation i saw this error in the maillog file
warning: connect to Milter service unix:/run/milter-manager/milter-manager.sock: Permission denied
One solution is :
chown milter-manager:postfix /run/milter-manager/milter-manager.sock
but if you restart milter-manager you return back with the error because the permission over the file is reset.
or
vim /etc/sysconfig/milter-manager
#SOCKET_GROUP=”milter-manager”
SOCKET_GROUP=“postfix”