Category: Linux Command

Posted on

Apache command connection check

The netstat command has been deprecated and replaced by the ss command in most of the Linux distributions.

It reads various ‘/proc’ files to gather information. It would take more time when there are lots of connections to display.

1) Checking the number of concurrent Apache connections

Run following ss command to find the total number of concurrent connections to Apache:

# ss -ant | grep -E ':80|:443' | wc -l
500

Alternatively, you can get Apache concurrent connection using netstat command as shown below:

# netstat -ant | grep -E ':80|:443' | wc -l
430

2) Checking concurrent connections of Apache in detail

Run the below ss command to see detailed information of Apache connections instead of counting it.

It shows the active internet connections on the server on port 80 & 443:

# ss -ant | grep -E ':80|:443'

 LISTEN     0      128    10.10.6.160:80                       :                  
106.222.112.160:12650              
 TIME-WAIT  0      0      94.237.76.92:443                114.119.135.42:2366               
 TIME-WAIT  0      0      94.237.76.92:443                114.119.135.42:2406               
 TIME-WAIT  0      0      94.237.76.92:443                127.0.0.1:38400              
 ESTAB      0      0      127.0.0.1:38454              94.237.76.92:443                     
 ESTAB      0      0      94.237.76.92:443                117.249.205.234:64685              
 ESTAB      0      0      94.237.76.92:443                192.99.9.25:33132              
 ESTAB      0      0      94.237.76.92:443                66.249.71.82:49611              
 ESTAB      0      0      94.237.76.92:443                106.222.112.160:12648              
 TIME-WAIT  0      0      94.237.76.92:443                127.0.0.1:38412              
 ESTAB      0      0      127.0.0.1:38402              94.237.76.92:443                
 TIME-WAIT  0      0      94.237.76.92:443                157.46.105.172:45656              
 TIME-WAIT  0      0      94.237.76.92:443                127.0.0.1:38340              
 ESTAB      0      151496 94.237.76.92:443                106.222.112.160:12656              
 TIME-WAIT  0      0      94.237.76.92:443                127.0.0.1:38332              
 TIME-WAIT  0      0      94.237.76.92:443                127.0.0.1:38396              
 ESTAB      0      0      127.0.0.1:38460              94.237.76.92:443                
 TIME-WAIT  0      0      94.237.76.92:443                127.0.0.1:38374              
 ESTAB      0      0      94.237.76.92:80                 5.9.61.232:51082              
 ESTAB      0      0      94.237.76.92:443                60.8.123.152:64476              
 ESTAB      0      0      94.237.76.92:443                167.114.209.104:35758              
 ESTAB      0      0      94.237.76.92:80                 106.222.112.160:12643              
 ESTAB      0      0      94.237.76.92:443                167.114.158.215:53270                          
 ESTAB      0      0      94.237.76.92:443                66.249.71.147:56912              
 ESTAB      0      0      94.237.76.92:443                127.0.0.1:38454              
 ESTAB      0      0      94.237.76.92:443                127.0.0.1:38468                         
 ESTAB      0      0      94.237.76.92:443                127.0.0.1:38402              
 TIME-WAIT  0      0      94.237.76.92:443                127.0.0.1:38366

Check the same information using the netstat command as shown below:

# netstat -ant | grep -E ':80|:443'

3) Listing Apache connections sort by IP

To count the number of connections currently active in Apache from each IP address and to sort them, use the following command:

# ss -ant |grep -E ':80|:443'|grep ESTAB| awk '{print $5}' | cut -d":" -f1 | sort | uniq -c | sort -nr

       8 94.237.76.92
       8 127.0.0.1
       2 5.9.61.232
       2 106.222.112.160
       1 98.236.14.66
       1 66.249.72.22
       1 66.249.71.48
       1 192.99.9.25
       1 167.114.209.104
       1 167.114.158.215

Similarly, you can find the same information using netstat command as shown below:

# netstat -ant |grep -E ':80|:443'|grep ESTAB | awk '{print $5}' | cut -d":" -f1 | sort | uniq -c | sort -nr

      6 162.158.155.70
      5 127.0.0.1
      2 172.68.51.180
      2 172.68.215.98
      2 172.68.215.86
      2 172.68.215.77
      2 172.68.215.75
      2 172.68.215.113
      2 172.68.215.111
      2 172.68.215.109
      2 172.68.215.101
      2 172.68.215.100
      2 162.158.150.128
      2 162.158.150.120
      2 162.158.118.154
      2 141.101.96.253
      2 141.101.96.243
      2 141.101.76.234
      2 141.101.105.254
	  .
	  .

Bonus Tips: 1) Counting running Apache processes in Linux

ps command is used to display all running processes in Linux system. Use the following format, if you would like to count the running Apache processes in Linux:

# ps -auxw | grep httpd | grep -v grep | wc -l
12

1.a) Listing Apache processes with ps

Use the following command to see the running httpd processes in Linux:

# ps auxw | grep httpd | grep -v grep
nobody    7988  0.0  0.5 253280 23252 ?        S    14:32   0:00 /usr/sbin/httpd -k start
nobody    8050  0.0  0.6 253412 24276 ?        S    14:33   0:00 /usr/sbin/httpd -k start
nobody    8054  0.0  0.6 253280 23288 ?        S    14:33   0:00 /usr/sbin/httpd -k start
nobody    8158  0.0  0.6 253280 23296 ?        S    14:33   0:00 /usr/sbin/httpd -k start
nobody    8159  0.0  0.5 253280 23176 ?        S    14:33   0:00 /usr/sbin/httpd -k start
daygeek   8202  0.0  0.6 253416 23304 ?        S    14:34   0:00 /usr/sbin/httpd -k start
nobody    8203  0.0  0.5 253280 23052 ?        S    14:34   0:00 /usr/sbin/httpd -k start
nobody    8207  0.0  0.5 253280 23044 ?        S    14:34   0:00 /usr/sbin/httpd -k start
nobody    8213  0.0  0.6 253280 23300 ?        S    14:34   0:00 /usr/sbin/httpd -k start
nobody    8216  0.0  0.5 253280 23052 ?        S    14:34   0:00 /usr/sbin/httpd -k start
nobody    8218  0.0  0.6 253416 23304 ?        S    14:34   0:00 /usr/sbin/httpd -k start
nobody    8266  0.0  0.5 253148 23052 ?        S    14:35   0:00 /usr/sbin/httpd -k start
nobody    8267  0.0  0.5 253144 22800 ?        S    14:35   0:00 /usr/sbin/httpd -k start
nobody    8391  0.3  0.5 253144 22800 ?        S    14:35   0:00 /usr/sbin/httpd -k start
nobody    8393  0.5  0.5 253012 21776 ?        S    14:35   0:00 /usr/sbin/httpd -k start
nobody    8394  1.0  0.5 253144 22800 ?        S    14:35   0:00 /usr/sbin/httpd -k start
root     30500  0.0  0.0 227356  3584 ?        Ss   Jul25   2:33 /usr/sbin/httpd -k start

Let’s quickly look at the parameters

  • Serverlimit – Maximum number of Apache processes
  • StartServers – Number of processes to start when you start running Apache
  • MinSpareThreads/MaxSpareThreads – Number of threads to keep idle without being killed
  • ThreadsPerChild – Number of threads per process
  • MaxRequestWorkers – Number of concurrent connections to be supported. This is the main directive that you need to change to increase max connections in Apache
  • MaxConnectionsPerChild – Number of connections to be handled by each child before it is killed
Posted on

6 commands to check and list active SSH connections in Linux (connections in general)

1. Using ss command

ss is used to dump socket statistics. It allows showing information similar to netstat. It can display more TCP and state information than other tools. We will use grep function to only get the list of active SSH sessions on our local host

[root@node3 ~]# ss | grep -i ssh
tcp    ESTAB      0      0      10.0.2.32:ssh                  10.0.2.31:37802
tcp    ESTAB      0      64     10.0.2.32:ssh                  10.0.2.2:49966
tcp    ESTAB      0      0      10.0.2.32:ssh                  10.0.2.30:56088

From the above example we know that there are three hosts which are currently connected to our node3. We have active SSH connections from 10.0.2.31, 10.0.2.30 and 10.0.2.2

 

2. Using last command

last searches back through the file /var/log/wtmp (or the file designated by the -f flag) and displays a list of all users logged in (and out) since that file was created. Names of users and tty’s can be given, in which case last will show only those entries matching the arguments.

Using this command you can also get the information about the user using which the SSH connection was created between server and client. So below we know the connection from 10.0.2.31 is done using ‘deepak‘ user, while for other two hosts, ‘root‘ user was used for connecting to node3.

[root@node3 ~]# last -a | grep -i still
deepak   pts/1        Fri May 31 16:58   still logged in    10.0.2.31
root     pts/2        Fri May 31 16:50   still logged in    10.0.2.30
root     pts/0        Fri May 31 09:17   still logged in    10.0.2.2

Here I am grepping for a string “still” to get all the patterns with “still logged in“. So now we know we have three active SSH connections from 10.0.2.31, 10.0.2.30 and 10.0.2.2

 

3. Using who command

who is used to show who is logged on on your Linux host. This tool can also give this information

[root@node3 ~]# who
root     pts/0        2019-05-31 09:17 (10.0.2.2)
root     pts/1        2019-05-31 16:47 (10.0.2.31)
root     pts/2        2019-05-31 16:50 (10.0.2.30)

Using this command we also get similar information as from last command. Now you get the user details used for connecting to node3 from source host, also we have terminal information on which the session is still active.

We generally hear terminal as tty but here we see terminal is referenced as pts, but now:
What is the difference between tty and pts?
How to disable or enable individual tty terminal console in Linux?

 

4. Using w command

w displays information about the users currently on the machine, and their processes. This gives more information than who and last command and also serves our purpose to get the list of active SSH connections. Additionally it also gives us the information of the running process on those sessions.

Using w command you will also get the idle time details, i.e. for how long the session is idle. If the SSH session is idle for long period then it is a security breach and it is recommended that such idle SSH session must be killed, you can configure your Linux host to automatically kill such idle SSH session.

[root@node3 ~]# w
 17:01:41 up  7:44,  3 users,  load average: 0.00, 0.01, 0.05
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    10.0.2.2         09:17    9:41   0.31s  0.00s less -s
deepak   pts/1    10.0.2.31        16:58    3:06   0.03s  0.03s -bash
root     pts/2    10.0.2.30        16:50    5.00s  0.07s  0.02s w

 

5. Using netstat command

Similar to ss we have netstat command to show active ssh sessions. Actually we can also say that ss is the new version of netstat. Here we can see all the ESTABLISHED SSH sessions from remote hosts to our localhost node3. it is also possible that one or some of these active ssh connections are in hung state so you can configure your host to automatically disconnect or kill these hung or unresponsive ssh sessions in Linux.

[root@node3 ~]# netstat -tnpa | grep 'ESTABLISHED.*sshd'
tcp        0      0 10.0.2.32:22            10.0.2.31:37806         ESTABLISHED 10295/sshd: deepak
tcp        0      0 10.0.2.32:22            10.0.2.2:49966          ESTABLISHED 4329/sshd: root@pts
tcp        0      0 10.0.2.32:22            10.0.2.30:56088         ESTABLISHED 10125/sshd: root@pt

ALSO READ:

How to disconnect idle ssh session or keep idle ssh session active in Linux

 

6. Using ps command

Now to show active ssh sessions, ps command may not give you accurate results like other commands we discussed in this article but it can give you some more additional information i.e. PID of the SSHD process which are currently active and connected.

# ps auxwww | grep sshd: | grep -v grep
root      4329  0.0  0.1 154648  5512 ?        Ss   09:17   0:00 sshd: root@pts/0
root     10125  0.0  0.1 154648  5532 ?        Ss   16:50   0:00 sshd: root@pts/2
root     10295  0.0  0.1 154648  5480 ?        Ss   16:58   0:00 sshd: deepak [priv]
deepak   10301  0.0  0.0 156732  2964 ?        S    16:58   0:00 sshd: deepak@pts/1

 

Check ssh connection history

To get the ssh connection history you can always check your SSHD logs for more information on connected or disconnected SSH session. Now the sshd log file may vary from distribution to distribution. On my RHEL 7.4 my sshd logs are stored inside /var/log/sshd

Lastly I hope the steps from the article to check active SSH connections and ssh connection history in Linux was helpful. So, let me know your suggestions and feedback using the comment section.

Posted on

How to create and manage services in CentOS 7 with systemd

Systemd is a system and service manager and like most major Linux distributions the init deamon was replaced by systemd in CentOS 7. One of the main functions of systemd is to manage the services, devices, mount points, sockets and other entities in a Linux system. Each of these entity that are managed by systemd is called a unit. Each unit is defined by a unit file (configuration file) which is located in one of the following directories.
Directory Description
/usr/lib/systemd/system/ Unit files distributed with installed packages. Do not modify unit files in this location.
/run/systemd/system/ Unit files that are dynamically created at runtime. Changes in this directory are lost when rebooted.
/etc/systemd/system/ Unit files created by systemctl enable and custome unit files created by system administrators.

Any custom unit files that you create should be placed in the /etc/system/system/ directory. This directory takes precedence over other directories.

Unit files names are of the form

unit_name.unit_type

Unit_type can be one of the following:

Unit Type Description
device A device unit.
service A system service.
socket A socket for inter-process communication.
swap A swap file or device.
target A group of units.
timer A systemd timer.
snapshot A snapshot of systemd manager.
mount A mount point.
slice A group of unit that manage the system processes.
path A file or directory.
automount A automount point.
scope An externally created process.

Creating a new service (systemd unit)

To create a custom service to be managed by systemd, you create a unit file that defines the configuration of that service. To create a service named MyService for example, you create a file named MyService.service in /etc/systemd/system/

# vim /etc/systemd/system/MyService.service

The unit file of service consists of a set of directives that are organized in to three sections – Unit, Service and Install. Below is an example of a very simple unit file.

[Unit]
 Description=Service description

[Service]	
 ExecStart=path_to_executable

[Install]
 WantedBy=default.target

Once you have created the unit file with all the necessary configuration options, save the file and set the correct file permissions.

# chmod 664 /etc/systemd/system/MyService.service

The next step is to reload all unit files to make systemd know about the new service.

# systemctl daemon-reload

Finally to start the service, run

# systemctl start MyService.service

# systemctl enable MyService.service

to enable the service to start at boot

systemctl reboot

Reboot the host to verify whether the scripts are starting as expected during system boot.

[Unit] Section

The following are the main directives that you specify in the [Unit] section.

Description A short description of the unit.
Documentation A list of URIs pointing to the documentation for the unit.
Requires A list of units that must be started alongside the current unit. If the any these units fail to start then current unit will not be activated.
Wants Similar to the Requires directive but the difference is the current unit will be activated even if the depended units fail to start.
Before List of units that cannot be started before the current unit.
After The current unit can started only after the units listed here.
Conflicts List units that cannot be run concurrently with the current unit.

[Service] Section

Some of the common directives that you’ll see in service section are.

Type Defines the startup type of the unit which can be one of the values:

  • Simple: This is the default. The main process of the service is the process started with ExecStart.
  • Forking: The process started with ExecStart spawns a new child process which becomes the main process and the parent process is terminated when the startup is completed.
  • Onehot: Similar to simple but systemd waits for the process to exit before proceeding with other units.
  • Dbus: Similar to simple but systemd waits for the process to take a name on the dbus.
  • Notify: Similar to simple Systemd will wait for a notification from the process before continuing with other units.
  • Idle: Similar to simple but service will not run until all other jobs are finished.
ExecStart Specifies the command to the executed to start service.
ExecStartPre Specifies the command to be executed before the main process specified in the ExecStart is started.
ExecStartPost Specifies the command to be executed after the main process specified in the ExecStart has finished.
ExecStop Specifies the command to be executed when the service is stopped.
ExecReload Specifies the command to be executed when the service is restarted.
Restart Specifies when to restart the service automatically. Possible values are “always”, “on-success”, “on-failure”, “on-abnormal”, “on-abort”, or “on-watchdog”.

[Install] Section

The [install] section provides information required to enable or disable the units using the systemctl command. The common options are:

RequiredBy A list of units that requires unit. A symbolic link of this unit is created in the .requires directory of the listed unit.
WantedBy Specifies a list of targets under which the service should be started. A symbolic link of this unit is created in the .wants directory of the listed target.

Using systemctl to manage services

systemctl is the command line tool you can use to control and manage services in systemd. Let’s now take a look at the some of the important systemctl commands for service management.

Listing Service Units and Unit files

To list all the units that are loaded

# systemctl list-units

To list only units of type service

# systemctl list-units -t service

To list all installed unit files of type service

# systemctl list-unit-files -t service

To list all installed unit files of type service

# systemctl list-unit-files -t service

You can use the --state option to filter the output by the state of the unit. The following command lists all services that are enabled.

# systemctl list-unit-files --state enabled

Note the difference between list-units and list-unit-files is that list-unit will only show units that are loaded while list-unit-files shows all unit files that are installed on the system.

Start and Stop service

This is quite straightforward, start option to start a service and stop option to stop a service

# systemctl start service_name.service
# systemctl stop service_name.service

Restart and Reload services

The restart option will restart a service that is running. If the service is not running, it will be started.

# systemctl restart service_name.service

If you want to restart the service only if its running then use the try-restart option.

# systemctl try-restart service_name.service

The reload option will try to reload the service specific configuration of a unit if it is supported.

# systemctl reload service_name.service

Enable and Disable services

Units can be enabled or disabled using the enable or disable options of systemctl command. When a unit a enabled symbolic links are created in various locations as specified in the [install] section of the unit file. Disabling a unit will remove the symbolic links that wer created when the unit was enabled.

# systemctl enable service_name.service
# systemctl disable service_name.service

Reload Unit Files

Whenever you make any changes to the unit files you need to let systemd know by executing daemon-reload which reloads all unit files.

# systemctl daemon-reload

Modifying system services

The unit files that come with installed packages are stored in /usr/lib/systemd/system/. The unit files in this directory should not be modified directly as the changes will be lost when if you update the package. The recommended method is to first copy the unit file to /etc/systemd/system/ and make the changes in that location. The unit files in /etc/systemd/system/ takes precedence over unit files in /usr/lib/systemd/system/ so the original unit file will be overridden.

Posted on

Managing The Postfix Queue

In this article, I will go over the tools Postfix offers you to manage the mail queue. There are 6 actions you can perform on messages in the queue.

  • List messages.
  • Delete messages.
  • Hold messages.
  • Re-queue messages.
  • Display the contents of messages.
  • Flush the messages in the queue.

Listing Messages in the Postfix Queue

Postfix provides a tool called postqueue. The command provided with the -p switch will display an entry for each message in the queue. The output will include a column for the message ID, size, arrival time, send, and recipient addresses.

Messages that are currently in the active queue will display a asterisk to the right of the message ID.

If there is a message in the hold queue, it will display a exclamation point to the right of the message ID.

If a message is deferred, there will not be symbol / mark to the right of the message ID.

postqueue -p

-Queue ID- --Size-- ----Arrival Time---- -Sender/Recipient-------  
5866DAC07F3 362 Sat Oct 12 15:53:56 trankin@tullyr.com  
(connect to mail.tullyr.com[208.70.75.100]:25: Connection refused)  
<trankin@tullyr.com>

Deleting Messages in the Postfix Queue

To delete messages from the Postfix queue, you can use the postsuper command. The postsuper tool can be used to delete a single message, or all messages in the queue.

To delete a single message from the queue, provide the -d switch followed by the message ID.

postsuper -d 5866DAC07F3

When you want to delete all messages in the queue, you provide postsuper with the word ALL in all capitals instead of the message ID.

postsuper -d ALL

NOTE – Be careful when using the postsuper command with the -d ALL switch. This command will delete all messages in the queue immediately without prompting you for verification.

Holding Messages in the Postfix Queue

The Postfix hold queue provides a place for messages to be held indefinitely. If you move a message into the hold queue, it will not be delivered until you specifically remove it or move it back into the normal queue processing.

To place a message in the hold queue, you use the tool postsuper with the -h switch followed by message ID.

postsuper -h 5866DAC07F3

When you need to move a messae back into the normal queue for processing, you use the postsuper command with the -H switch followed by the message ID.

postsuper -H 5866DAC07F3

Re-queue Messages in the Postfix Queue

If you have messages that are incorrect based on a misconfigured Postfix installation, you may need to re-queue the messages via the postsuper command. Some example of bad messages could be problems such as incorrect transport type, incorrect rewriting of an address, or an incorrect next hop.

After fixing the Postfix configuration, you can re-queue a single message or all messages. To requeue a message, you will use the postsuper command with the -r switch. When this command is run, it will update the incorrect information based on the new configuration.

To re-queue a single message you pass the message ID as in the example below.

postsuper -r 5866DAC07F3

If you need to re-queue all messages, you can pass the capital word ALL.

postsuper -r ALL

Displaying the Contents of Messages in the Postfix Queue

If you need to view the contents of a message in the queue, Postfix provides the postcat tool for this. The postcat tool will display the contents of a file when provided with the -q switch followed by the message ID.

postcat -q 5866DAC07F3

Flushing Messages in the Postfix Queue

If you have messages in the queue that you would like to flush, you can use the postqueue command with the -f switch. Flushing the queue will cause Postfix to attempt to deliver all messages in the queue immediately.

postqueue -f

There are times when this is needed. However, it’s usually not a good idea to flush all of the messages in the queue. If you do this a lot, it can have an impact on the performance on your mail server. You should leave the queue management to the Postfix queue manager.

With Postfix, it’s possible to flush only those messages that are going to a specific domain. You can do this with the -s switch provided by postqueue. However, to do this the domain must be eligible for fast flush.

For a domain to be eligible for fast flush, it must be listed in the fast_flush_domains parameter. In Postfix, the default value for the fast_flush_domains includes all of the hosts that are listed in relay_domains.

To add the site to the fast_flush_domains you can append it to the line fast_flush_domains as shown below:

fast_flush_domains = $relay_domains tullyrankin.com

Now if you want to flush mail that is destined for the domain you added to the fast_flush_domains, you can use the postqueue command with the -s option as shown below:

postqueue -s tullyrankin.com