Posts

Posted on

How email works (MTA, MDA, MUA)

As simple as it is to use, email relies on a more complicated set of operating procedures than that of the Web. For most users, its operation is transparent, which means that it is not necessary to understand how email works in order to be able to use it.

However, the short introduction below has been provided to help you to understand its basic principles, give you an idea of how to best configure your email clients, and inform you about the underlying mechanisms of spam.

How Email Works

Email is based around the use of electronic mailboxes. When an email is sent, the message is routed from server to server, all the way to the recipient’s email server. More specifically, the message is sent to the mail server tasked with transporting emails (called the MTA, for Mail Transport Agent) to the recipient’s MTA. On the Internet, MTAs communicate with one another using the protocol SMTP, and so are logically called SMTP servers (or sometimes outgoing mail servers).

The recipient’s MTA then delivers the email to the incoming mail server (called the MDA, for Mail Delivery Agent), which stores the email as it waits for the user to accept it. There are two main protocols used for retrieving email on an MDA: POP3 (Post Office Protocol), the older of the two, which is used for retrieving email and, in certain cases, leaving a copy of it on the server; and IMAP (Internet Message Access Protocol), which is used for coordinating the status of emails (read, deleted, moved) across multiple email clients. With IMAP, a copy of every message is saved on the server, so that this synchronization task can be completed.

For this reason, incoming mail servers are called POP servers or IMAP servers, depending on which protocol is used:

Schematic diagram of MTA-MDA-MUA
To use a real-world analogy, MTAs act as the post office (the sorting area and mail carrier), which handle message transportation, while MDAs act as mailboxes, which store messages (as much as their volume will allow) until the recipients check the box. This means that it is not necessary for recipients to be connected in order for them to be sent email.

To keep everyone from checking other users’ emails, MDA is protected by a user name called a login and by a password.

Retrieving mail is done using a software program called an MUA (Mail User Agent). When the MUA is a program installed on the user’s system, it is called an email client (such as Mozilla Thunderbird, Microsoft Outlook, Eudora Mail, Incredimail or Lotus Notes).

When it is a web interface used for interacting with the incoming mail server, it is called webmail.

Open Relay

By default, it is not necessary to authenticate oneself to send email, which means that it is very easy to falsify one’s own address when sending mail. For this reason, nearly all Internet service providers lock down their SMTP servers so that only their subscribers can use them, or more precisely, only machines whose IP address belongs to the ISP’s domain. This explains why users must modify the outgoing server settings in their email clients each time they move to a new home or business.

When an organization’s email server is improperly configured and allows third-party users on any network to send emails, this is called an open relay. Open relays are generally used by spammers, as using them hides the true origins of their messages. As a result, many ISPs keep an up-to-date blacklist of open relays to keep subscribers from receiving messages from such servers.

Posted on

Enigma plugin (PGP encryption) Roundcube

You have to set up Enigma plugin :

cp /……../roundcubemail/plugins/enigma/config.inc.php.dist /……../roundcubemail/plugins/enigma/config.inc.php

<?php

// Enigma Plugin options
// --------------------

// A driver to use for PGP. Default: "gnupg".
$config['enigma_pgp_driver'] = 'gnupg';

// A driver to use for S/MIME. Default: "phpssl".
$config['enigma_smime_driver'] = 'phpssl';

// Enables logging of enigma operations (including Crypt_GPG debug info)
$config['enigma_debug'] = true;

// Keys directory for all users. Default 'enigma/home'.
// Must be writeable by PHP process
$config['enigma_pgp_homedir'] = enigma/home;

// Location of gpg binary. By default it will be auto-detected.
// This is also a way to force gpg2 use if there are both 1.x and 2.x on the system.
$config['enigma_pgp_binary'] = '/usr/bin/gpg';

// Location of gpg-agent binary. By default it will be auto-detected.
// It's used with GnuPG 2.x.
$config['enigma_pgp_agent'] = '';

// Location of gpgconf binary. By default it will be auto-detected.
// It's used with GnuPG >= 2.1.
$config['enigma_pgp_gpgconf'] = '';

// Enables signatures verification feature.
$config['enigma_signatures'] = true;

// Enables messages decryption feature.
$config['enigma_decryption'] = true;

// Enables messages encryption and signing feature.
$config['enigma_encryption'] = true;

// Enable signing all messages by default
$config['enigma_sign_all'] = false;

// Enable encrypting all messages by default
$config['enigma_encrypt_all'] = false;

// Enable attaching a public key to all messages by default
$config['enigma_attach_pubkey'] = false;

// Default for how long to store private key passwords (in minutes).
// When set to 0 passwords will be stored for the whole session.
$config['enigma_password_time'] = 0;

// With this option you can lock composing options
// of the plugin forcing the user to use configured settings.
// The array accepts: 'sign', 'encrypt', 'pubkey'.
//
// For example, to force your users to sign every email,
// you should set:
//     - enigma_sign_all     = true
//     - enigma_options_lock = array('sign')
//     - dont_override       = array('enigma_sign_all')
$config['enigma_options_lock'] = array();

 

Installation and settings Enigma plugin

Preferences > Settings > Encryption you’ll have the possibility to enable/disable encryption-related features.

The keys are stored on the server.

Figure 1. Encryption preferences section.
enigma_settings

Goto Settings > PGP Keys. There you can generate a new key pair or import keys.

Figure 2. Key generation form.
enigma_keygen

Figure 3. Key information frame.
enigma_keyinfo

Composing messages

Figure 4. Encryption options in compose.
enigma_compose

 

 

Posted on

Install Milter Manager on Centos 7

from here 

Install packages

Postfix is used as MTA because it’s installed by default.

Spamass-milter, clamav-milter and milter-greylist are used as milters. Milter packages registered in EPEL are used.

Register EPEL like the following:

% sudo yum install -y epel-release

Now, you install milters:

% sudo yum install -y spamass-milter-postfix clamav-scanner-systemd clamav-update clamav-milter clamav-milter-systemd milter-greylist

And you install RRDtool for generating graphs:

% sudo yum install -y rrdtool

Build and Install

milter manager can be installed by yum.

Register milter manager yum repository like the following:

% curl -s https://packagecloud.io/install/repositories/milter-manager/repos/script.rpm.sh | sudo bash

See also: <URL:https://packagecloud.io/milter-manager/repos/install>

Now, you install milter manager:

% sudo yum install -y milter-manager

Configuration

Here is a basic configuration policy.

milter-greylist should be applied only if S25R condition is matched to reduce needless delivery delay. But the configuration is automatically done by milter manager. You need to do nothing for it.

It’s difficult that milter manager runs on SELinux. Disable SELinux policy module for Postfix and Milter.

% sudo semodule -d postfix
% sudo semodule -d milter

Configure spamass-milter

At first, you configure spamd.

spamd adds “[SPAM]” to spam mail’s subject by default. If you don’t like the behavior, edit /etc/mail/spamassassin/local.cf.

Before:

rewrite_header Subject [SPAM]

After:

# rewrite_header Subject [SPAM]

Add the following configuration to /etc/mail/spamassassin/local.cf. This configuration is for adding headers only if spam detected.

remove_header ham Status
remove_header ham Level

Start spamd on startup:

% sudo systemctl enable spamassassin

Start spamd:

% sudo systemctl start spamassassin

Here are spamass-milter’s configuration items:

  • Disable needless body change feature.
  • Reject if score is larger than or equal to 15.

Change /etc/sysconfig/spamass-milter:

Before:

#EXTRA_FLAGS="-m -r 15"

After:

EXTRA_FLAGS="-m -r 15"

Start spamass-milter on startup:

% sudo systemctl enable spamass-milter

Start spamass-milter:

% sudo systemctl start spamass-milter

Configure clamav-milter

Update ClamAV virus database and start clamd.

Edit /etc/freshclam.conf like the following. It comments out “Example”, changes “NotifyClamd” value and uncomments other items.

Before:

Example
#LogFacility LOG_MAIL
#NotifyClamd /path/to/clamd.conf

After:

#Example
LogFacility LOG_MAIL
NotifyClamd /etc/clamd.d/scan.conf

Run freshclam by hand at the first time:

% sudo freshclam

Configure clamd.

Edit /etc/clamd.d/scan.conf like the following. It comments out “Example” and uncomments other items:

Before:

Example
#LogFacility LOG_MAIL
#LocalSocket /run/clamd.scan/clamd.sock

After:

#Example
LogFacility LOG_MAIL
LocalSocket /run/clamd.scan/clamd.sock

Start clamd on startup:

% sudo systemctl enable clamd@scan

Start clamd:

% sudo systemctl start clamd@scan

Configure clamav-milter.

Edit /etc/mail/clamav-milter.conf like the following. It comments out “Example”, change “ClamdSocket” value and uncomments other items:

Before:

Example
#MilterSocket /run/clamav-milter/clamav-milter.socket
#MilterSocketMode 660
#ClamdSocket tcp:scanner.mydomain:7357
#LogFacility LOG_MAIL

After:

#Example
MilterSocket /run/clamav-milter/clamav-milter.socket
MilterSocketMode 660 
ClamdSocket unix:/run/clamd.scan/clamd.sock 
LogFacility LOG_MAIL

Add “clamilt” user to “clamscan” group to access clamd’s socket:

% sudo usermod -G clamscan -a clamilt

Start clamav-milter on startup:

% sudo systemctl enable clamav-milter

Start clamav-milter:

% sudo systemctl start clamav-milter

Configure milter-greylist

Change /etc/mail/greylist.conf for the following configurations:

  • use the leading 24bits for IP address match to avoid Greylist adverse effect for sender uses some MTA case.
  • decrease retransmit check time to 10 minutes from 30 minutes (default value) to avoid Greylist adverse effect.
  • increase auto whitelist period to a week from 1 day (default value) to avoid Greylist adverse effect.
  • don’t use Greylist when trusted domain passes SPF. (Trusted domains are configured in milter manager)
  • use Greylist by default.

The configuration relaxes Greylist check to avoid Greylist adverse effect. It increases received spam mails but you should give priority to avoid false positive rather than false negative. You should not consider that you blocks all spam mails by Greylist. You can blocks spam mails that isn’t blocked by Greylist by other anti-spam technique such as SpamAssassin. milter manager helps constructing mail system that combines some anti-spam techniques.

Before:

socket "/run/milter-greylist/milter-greylist.sock"
# ...
racl whitelist default

After:

socket "/run/milter-greylist/milter-greylist.sock" 660
# ...
subnetmatch /24
greylist 10m
autowhite 1w
sm_macro "trusted_domain" "{trusted_domain}" "yes"
racl whitelist sm_macro "trusted_domain" spf pass
racl greylist sm_macro "trusted_domain" not spf pass
racl greylist default

Start milter-greylist on startup:

% sudo systemctl enable milter-greylist

Start milter-greylist:

% sudo systemctl start milter-greylist

Configure milter manager

Add “milter-manager” user to “clamilt” group to access clamav-milter’s socket:

% sudo usermod -G clamilt -a milter-manager

Add “milter-manager” user to “mail” group and “grmilter” group to access milter-greylist’s socket:

% sudo usermod -G mail -a milter-manager
% sudo usermod -G grmilter -a milter-manager

Add “milter-manager” user to “postfix”” group to access spamass-milter’s socket:

% sudo usermod -G postfix -a milter-manager

milter manager detects milters that installed in system. You can confirm spamass-milter, clamav-milter and milter-greylist are detected:

% sudo /usr/sbin/milter-manager -u milter-manager -g milter-manager --show-config

The following output shows milters are detected:

...
define_milter("milter-greylist") do |milter|
  milter.connection_spec = "unix:/run/milter-greylist/milter-greylist.sock"
  ...
  milter.enabled = true
  ...
end
...
define_milter("clamav-milter") do |milter|
  milter.connection_spec = "unix:/var/run/clamav-milter/clamav-milter.socket"
  ...
  milter.enabled = true
  ...
end
...
define_milter("spamass-milter") do |milter|
  milter.connection_spec = "unix:/run/spamass-milter/postfix/sock"
  ...
  milter.enabled = true
  ...
end
...

You should confirm that milter’s name, socket path and “enabled = true”. If the values are unexpected, you need to change /etc/milter-manager/milter-manager.local.conf. See Configuration for details of milter-manager.local.conf.

But if we can, we want to use milter manager without editing miter-manager.local.conf. If you report your environment to the milter manager project, the milter manager project may improve detect method.

milter manager’s configuration is finished.

Start to milter manager on startup:

% sudo systemctl enable milter-manager

Start to milter manager:

% sudo systemctl start milter-manager

milter-test-server is usuful to confirm milter manager was ran:

% sudo -u milter-manager milter-test-server -s unix:/var/run/milter-manager/milter-manager.sock

Here is a sample success output:

status: accept
elapsed-time: 0.128 seconds

If milter manager fails to run, the following message will be shown:

Failed to connect to unix:/var/run/milter-manager/milter-manager.sock

In this case, you can use log to solve the problem. milter manager is verbosily if –verbose option is specified. milter manager outputs logs to standard output if milter manager isn’t daemon process.

You can add the following configuration to /etc/sysconfig/milter-manager to output verbose log to standard output:

OPTION_ARGS="--verbose --no-daemon"

Restart milter manager:

% sudo systemctl restart milter-manager

Some logs are output if there is a problem. Running milter manager can be exitted by Ctrl+c.

OPTION_ARGS configuration in /etc/sysconfig/milter-manager should be commented out after the problem is solved to run milter manager as daemon process. And you should restart milter manager.

Configure Postfix

Enables Postfix:

% sudo systemctl enable postfix
% sudo systemctl start postfix

Configure Postfix for milters. Append following lines to /etc/postfix/main.cf:

milter_protocol = 6
milter_default_action = accept
milter_mail_macros = {auth_author} {auth_type} {auth_authen}

For details for each lines.

milter_protocol = 6 Use milter protocol version 6.
milter_default_action = accept MTA receives mails when MTA cannot access milter. Although there are problems between MTA and milter, MTA can deliver mails to clients. But until you recover milter, perhaps MTA receives spam mails and virus mails.

If you can recover the system quickly, you can specify ‘tempfail’ instead of ‘accept’. Default value is ‘tempfail’.
milter_mail_macros = {auth_author} {auth_type} {auth_authen} MTA gives information related SMTP Auth to milters. milter-greylist etc. uses it.

Register milter manager to Postfix. It’s important that spamass-milter, clamav-milter and milter-greylist aren’t needed to be registered because they are used via milter manager.

Append following lines to /etc/postfix/main.cf:

smtpd_milters = unix:/var/run/milter-manager/milter-manager.sock

Reload Postfix’s configuration.

% sudo systemctl reload postfix

milter manager logs to syslog. If milter manager works well, some logs can be shown in /var/log/maillog. You need to send a test mail for confirming.

Conclusion

There are many configurations to work milter and Postfix together. They can be reduced by introducing milter manager.

Without milter manager, you need to specify sockets of spamass-milter, clamav-milter and milter-greylist to /etc/postfix/main.cf. With milter manager, you don’t need to specify sockets of them, just specify a socket of milter manager. They are detected automatically. You don’t need to take care some small mistakes like typo.

milter manager also detects which ‘/sbin/chkconfig -add’ is done or not. If you disable a milter, you use the following steps:

% sudo systemctl stop milter-greylist
% sudo systemctl disable milter-greylist

You need to reload milter manager after you disable a milter.

% sudo systemctl reload milter-manager

milter manager detects a milter is disabled and doesn’t use it. You don’t need to change /etc/postfix/main.cf.

You can reduce maintainance cost by introducing milter manager if you use some milters on CentOS.

milter manager also provides tools to help operation. Installing them is optional but you can reduce operation cost too. If you also install them, you will go to Install to CentOS (optional) .

ISSUE:

after the installation i saw this error in the maillog file

warning: connect to Milter service unix:/run/milter-manager/milter-manager.sock: Permission denied

One solution is :

chown milter-manager:postfix /run/milter-manager/milter-manager.sock

but if you restart milter-manager you return back with the error because the permission over the file is reset.

or

vim /etc/sysconfig/milter-manager

#SOCKET_GROUP=”milter-manager”

SOCKET_GROUP=“postfix”

 

Posted on

Incorrect definition of table mysql.event: expected column ‘sql_mode’

[ERROR] Incorrect definition of table mysql.event: expected column ‘sql_mode’ at position 14

to have type set(‘REAL_AS_FLOAT’,’PIPES_AS_CONCAT’,’ANSI_QUOTES’,’IGNORE_SPACE’,’IGNORE_BAD_TABLE_OPTIONS’,’ONLY_FULL_GROUP_BY’,’NO_UNSIGNED_S

UBTRACTION’,’NO_DIR_IN_CREATE’,’POSTGRESQL’,’ORACLE’,’MSSQL’,’DB2′,’MAXDB’,’NO_KEY_OPTIONS’,’NO_TABLE_OPTIONS’,’NO_FIELD_OPTIONS’,’MYSQL323′,’M

YSQL40′,’ANSI’,’NO_AUTO_VALUE_ON_ZERO’,’NO_BACKSLASH_ESCAPES’,’STRICT_TRANS_TABLES’,’STRICT_ALL_TABLES’,’NO_ZERO_IN_DATE’,’NO_ZERO_DATE’,’INVAL

ID_DATES’,’ERROR_FOR_DIVISION_BY_ZERO’,’TRADITIONAL’,’NO_AUTO_CREATE_USER’,’HIGH_NOT_PRECEDENCE’,’NO_ENGINE_SUBSTITUTION’,’PAD_CHAR_TO_FULL_LEN

GTH’,’EMPTY_STRING_IS_NULL’,’SIMULTANEOUS_ASSIGNMENT’), found type set(‘REAL_AS_FLOAT’,’PIPES_AS_CONCAT’,’ANSI_QUOTES’,’IGNORE_SPACE’,’IGNORE_B

AD_TABLE_OPTIONS’,’ONLY_FULL_GROUP_BY’,’NO_UNSIGNED_SUBTRACTION’,’NO_DIR_IN_CREATE’,’POSTGRESQL’,’ORACLE’,’MSSQL’,’DB2′,’MAXDB’,’NO_KEY_OPTIONS

‘,’NO_TABLE_OPTIONS’,’NO_FIELD_OPTIONS’,’MYSQL323′,’MYSQL40′,’ANSI’,’NO_AUTO_VALU

Apr 26 04:22:08 mail mysqld: 2020-04-26  4:22:07 0 [ERROR] mysqld: Event Scheduler: An error occurred when initializing system tables. Disabling the Event Scheduler.

i solved copying from the log file how the column has to be set and run this in the mysql db:

ALTER TABLE `event` CHANGE `sql_mode` `sql_mode` SET(‘REAL_AS_FLOAT’,’PIPES_AS_CONCAT’,’ANSI_QUOTES’,’IGNORE_SPACE’,’IGNORE_BAD_TABLE_OPTIONS’,’ONLY_FULL_GROUP_BY’,’NO_UNSIGNED_SUBTRACTION’,’NO_DIR_IN_CREATE’,’POSTGRESQL’,’ORACLE’,’MSSQL’,’DB2′,’MAXDB’,’NO_KEY_OPTIONS’,’NO_TABLE_OPTIONS’,’NO_FIELD_OPTIONS’,’MYSQL323′,’MYSQL40′,’ANSI’,’NO_AUTO_VALUE_ON_ZERO’,’NO_BACKSLASH_ESCAPES’,’STRICT_TRANS_TABLES’,’STRICT_ALL_TABLES’,’NO_ZERO_IN_DATE’,’NO_ZERO_DATE’,’INVALID_DATES’,’ERROR_FOR_DIVISION_BY_ZERO’,’TRADITIONAL’,’NO_AUTO_CREATE_USER’,’HIGH_NOT_PRECEDENCE’,’NO_ENGINE_SUBSTITUTION’,’PAD_CHAR_TO_FULL_LENGTH’,’EMPTY_STRING_IS_NULL’,’SIMULTANEOUS_ASSIGNMENT’) CHARACTER SET utf8 COLLATE utf8_general_ci NOT NULL DEFAULT ”;