Linux services troubleshooting

If you cannot connect to your service..

When you start service, check that it says ok (most services say that when starting in CENTos). Rarely service might say ok and not start.

[root@clump ~]# service httpd start

Starting httpd: [ OK ]

[root@clump ~]#

If you try to connect to service and it’s not responding properly, first check with ps that process has started. Using ps with following switches and with the help of grep should show some results.

/etc/init.d/apache2 start in Ubuntu/Debian

ps xaufw | grep –i httpd

[root@clump ~]# ps xaufw | grep -i httpdroot 21453 0.0 0.2 3768 680 pts/2 S+ 08:33 0:00 \_ grep -i httpdroot 21438 0.0 0.9 8096 2516 ? Ss 08:30 0:00 /usr/sbin/httpdapache 21440 0.0 0.7 8228 1812 ? S 08:30 0:00 \_ /usr/sbin/httpdapache 21441 0.0 0.7 8228 1812 ? S 08:30 0:00 \_ /usr/sbin/httpdapache 21442 0.0 0.7 8228 1812 ? S 08:30 0:00 \_ /usr/sbin/httpd

If you can see your service, next thing to check is if network ports are open. They can be checked with netstat command.

If your using root –user, you can use command netstat –antpu Using same command with basic user rights gives output which doesn’t have PID/Program information.

Using netstat –antpu with basic user account:[tero@clump ~]$ netstat -antpu(No info could be read for "-p": geteuid()=537 but you should be root.)Active Internet connections (servers and established)Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program nametcp 0 0 127.0.0.1:3307 0.0.0.0:* LISTEN -tcp 0 0 ::1:3307 :::* LISTEN -tcp 0 0 :::80 :::* LISTEN -tcp 0 0 :::22 :::* LISTEN -tcp 0 148 ::ffff:193.166.135.104:22 ::ffff:195.148.208.119:1043 ESTABLISHED –

Using netstat –antpu with root -user account:[root@clump ~]# netstat -antpuActive Internet connections (servers and established)Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program nametcp 0 0 127.0.0.1:3307 0.0.0.0:* LISTEN 15980/0tcp 0 0 ::1:3307 :::* LISTEN 15980/0tcp 0 0 :::80 :::* LISTEN 21438/httpdtcp 0 0 :::22 :::* LISTEN 1485/sshdtcp 0 296 ::ffff:193.166.135.104:22 ::ffff:195.148.208.119:1043 ESTABLISHED 21401/2

If you can see your service in process list with ps and in netstatistics with netstat, then your service has started.

To avoid firewalls blocking access to your service, try accessing service by using localhost as hostname. Localhost is machines local network name and it can be used from within server to access things locally. You can’t use localhost as hostname to access server from other machines in network.

One of the ways trying to access your service is by using telnet. It doesn’t work with all programs but if it does you can see that your service is actually responding to incoming connections. Use telnet to debug by using following command: telnet localhost XX (where XX is port number of your application). Port number is easy to find with netstat.

[root@clump ~]# telnet localhost 80

Trying 127.0.0.1...

Connected to localhost.localdomain (127.0.0.1).

Escape character is '^]'.

In this example above httpd server (apache) responded to connection attempts. If we know that it responds we can check firewall / security settings next.

Iptables and SELinuxIptables is modern firewall used in our CENTos servers. If you haven’t added your application port to iptables list, firewall will in most cases block your access to service from network.

You can disable iptables firewall with command iptables -F

If your running your server outside lab enviroment, disabling firewall isn’t recommended.

If you left SELinux enabled during installion, you will find yourself facing some mysterious problems. SELinux does block most of suspicious activity and should therefore be disabled in our lab course. Command to disable SELinux when system is running is setenforce 0

When you want to disable SELinux permanently, find and edit file /etc/selinux/configThere are some good commented instructions how to do it inside that file.Just change line saying SELINUX=enforcing to SELINUX=disabled

Checking log filesCommonly all services write information to logs when software is started. Thisinformation for the most of the services can be found from /var/log –directory. Some services for example Apache does have it’s own logs written under /var/log/httpd –directory.

Use tail command to see last log entries. You can follow new incoming entriesautomatically by issuing command tail –f /var/log/logname &

If you still can’t solve your services problems with the instructions above, ask instructor!