configuring and debugging exim

Configuring and Debugging Exim

Stephen Bee

Tuesday, October 27, 2009

(I <3) Exim’s Guts


The Configuration File

Where is it?• /etc/exim.conf on Linux systems

• /usr/local/etc/exim/configure on FreeBSD systems

• exim -bV | grep Config (if all else fails)


The Configuration File

What’s in it?Partitioned into seven different sections:

• The ‘main section’

• ACL definitions

• Retry Rules

• Rewrite Rules

• Routers

• Transports

• Authenticators

Note: Other than the main section, all sections are headed with begin section_name


The main section

smtp_receive_timeout = 165sdaemon_smtp_ports = 25 : 465ignore_bounce_errors_after = 3dsystem_filter = /etc/cpanel_exim_system_filter

• Contains global settings and variables

• Always located at the top of exim.conf

Documentation for all available settings can be found at:http://exim.org/exim-html-current/doc/html/spec_html/ch14.html


http://exim.org/exim-html-current/doc/html/spec_html/ch14.html


List variables

hostlist bad_hosts = 192.168.99.123 : 192.168.87.243domainlist trusted_domains = foo.example : bar.exampleaddresslist spammers = [email protected] : [email protected] sysusers = foo : bar : root

• Contain domains, hosts, addesses, or local parts

• Colon separated, type based

Comprehensive documentation on list variables can be found at:http://exim.org/exim-html-current/doc/html/spec_html/ch10.html

Static Lists:

hostlist trustedmailhosts = lsearch;/etc/trustedmailhostsdomainlist local_domains = lsearch;/etc/localdomains

Dynamic Lists:


mailto:[email protected]






Routers and Directors

What are they?• The decision makers for how a message is handled

• Routers result in message delivery, directors do not

remote_delivery:driver = dnslookupdomains = ! +local_domainstransport = remote_smtp

Director

Router

fail_remote_domains: driver = redirect domains = ! +local_domains : ! localhost : ! localhost.localdomain allow_fail data = "remote deliveries are not permitted from this server"


Transports

remote_smtp: driver = smtp interface = 1.2.3.4

What are they?• The executioners of the actual message delivery

local_delivery: driver = appendfile file = /home/foo/mail/foo.example/joe/inbox


Example: Smart Hosts

ObjectiveRoute messages for a specific list of domains to a third party mail server.


Example: Smart HostsHow it’s done• Create the domain list file

touch /etc/smartdomainschown root:mail /etc/smartdomainschmod 0750 /etc/smartdomainsecho foo.example > /etc/smartdomains

• Add a named domainlist for that list file

domainlist smart_domains = lsearch;/etc/smartdomains

• Create a manualroute router (after democheck)router smarthost: driver = manualroute transport = remote_smtp route_list = +smart_domains 192.168.99.232


Example: Smart HostsMaking it more flexible.• Use a colon separated domain to host mapping

root@gibson [~]# cat /etc/smartdomains foo.example: 192.168.99.232bar.example: 192.168.99.254root@gibson [~]#

• Amend the router to use a key based lookup

smarthost: driver = manualroute transport = remote_smtp route_data = ${lookup{$domain}lsearch{/etc/smartdomains}}


ACLs

• Used for validation, scanning, whitelisting, etc.

• Only called during the SMTP reception process

• Conditions consist of the following

- An action (accept, deny, drop, defer)

- Criteria that if evaluated true, triggers the action

acl_connect: accept hosts = +trustedmailhosts deny

Comprehensive documentation on ACLs can be found at:http://exim.org/exim-html-current/doc/html/spec_html/ch40.html




Defining the ACL

• ACLs are executed based on the ACL selector they’ve been assigned.

All of the available ACL selection options are documented at:http://exim.org/exim-html-current/doc/html/spec_html/ch40.html#SECID189

acl_smtp_connect = acl_connect acl_smtp_data = check_message acl_smtp_mail = acl_mail acl_smtp_notquit = acl_notquit acl_smtp_rcpt = check_recipient




Whitelisting domains for spam

ObjectiveDisable only spamassassin scans for a list of host addresses.


Whitelisting domains for spam

How it’s done

Add this near the top of the check_message ACL

accept hosts = net-iplsearch;/etc/spamfreehosts

Create the domain list file

touch /etc/spamfreehostschown root:mail /etc/spamfreehostschmod 0750 /etc/spamfreehostsecho 1.2.3.4 > /etc/spamfreehosts


ALL? No, ALL!

ObjectiveDeny incoming connections from all hosts, except for our third party spam filtering service.


ALL? No, ALL!

How it’s done• Add the following to the top of the acl_connect ACL

accept hosts = :deny !hosts = @[] : net-iplsearch;/etc/trustedmailhosts message = This server does not handle mail directly


Half-time Q&A


Log Files


Exim’s Log Files

• /var/log/exim_mainlogLogs message arrival and delivery attempts

• /var/log/exim_rejectlogLogs delivery rejections based on policy (e.g. ACL)

• /var/log/exim_paniclogPrints a fresh copy of exim.conf to standard output

Tip: exim -bP log_file_path will display log file paths


Main log formatting

2009-09-30 12:23:40 1Mt2tw-0003vE-Ea <= [email protected] H=(cpanel.net) [127.0.0.1] P=esmtpa A=fixed_login:[email protected] S=745 id=0373931685581ab29f56199c78755f1a.squirrel@techdump.net2009-09-30 12:23:42 1Mt2tw-0003vE-Ea => [email protected] R=lookuphost T=remote_smtp H=gmail-smtp-in.l.google.com [209.85.211.67]2009-09-30 12:23:42 1Mt2tw-0003vE-Ea Completed2009-10-03 23:47:33 1MuEK5-0008S6-Io == [email protected] R=smarthost T=remote_smtp defer (111): Connection refused2009-09-30 18:33:00 1Mt8fH-0005xJ-Oe ** [email protected] R=fail_remote_domains: The mail server could not deliver mail to [email protected]. The account or domain may not exist, they may be blacklisted, or missing the proper dns entries.

Successful Message Delivery

• => indicates message arrival

• <= indicates successful message delivery

• == indicates message delivery has been deferred

• ** indicates that a delivery failure has occurred

Message Status Indicators
















Main log formatting

2009-09-30 12:23:42 1Mt2tw-0003vE-Ea => [email protected] R=lookuphost T=remote_smtp H=gmail-smtp-in.l.google.com [209.85.211.67]2009-10-03 23:47:33 1MuEK5-0008S6-Io == [email protected] R=smarthost T=remote_smtp defer (111): Connection refused

• R= indicates the assigned router

• T= indicates the assigned transport

Router and Transport Information






The reject log

2009-10-03 13:39:53 H=source.host.example.com [10.0.0.1] F=<[email protected]> rejected RCPT <[email protected]>: "JunkMail rejected - source.host.example.com [10.0.0.1] is in an RBL, see http://www.spamhaus.org/query/bl?ip=10.0.0.1"2009-10-03 17:07:20 H=75-170-234-130.desm.qwest.net (wergvan) [75.170.234.130] rejected MAIL <>: Access denied - Invalid HELO name (See RFC2821 4.1.1.1)2009-09-30 09:44:28 fixed_login authenticator failed for (example.com) [10.0.0.3]: 535 Incorrect authentication data (set_id=inna)2009-09-30 09:44:29 SMTP call from (cracker.example) [10.0.0.4] dropped: too many nonmail commands (last was "AUTH")

Examples

• Logs only policy-based rejections

• Makes it easier to differentiate rejections






http://www.spamhaus.org/query/bl?ip=10.0.0.1




Extracting log information

exigrep [-t<n>] [-I] [-l] [-v] <pattern> [<log file>]

• Written specifically for searching exim log files

• Returns all entries for matching messages

• Takes input via STDIN, or by specifying a file name

root@foo [~]# exigrep [email protected] /var/log/exim_mainlog2009-09-30 12:38:12 1Mt37t-000405-4r <= [email protected] H=(cpanel.net) [10.1.1.2] U=root P=esmtp S=14232009-09-30 12:38:12 1Mt37t-000405-4r => /home/foo/mail/ <[email protected]> R=central_filter T=address_directory2009-09-30 12:38:12 1Mt37t-000405-4r Completed






Testing Message Delivery


Launching an SMTP Session

root@gibson [~]# exim -bh 1.2.3.4**** SMTP testing session as if from host 1.2.3.4**** but without any ident (RFC 1413) callback.**** This is not for real!...TRUNCATED...>>> check condition = ${if eq {$interface_port}{25}{no}{yes}}>>> = yes>>> accept: condition test succeeded220-gibson.steve.cpanel.net ESMTP Exim 4.69 #1 Mon, 05 Oct 2009 10:22:59 -0400 220-We do not authorize the use of this system to transport unsolicited, 220 and/or bulk e-mail.

exim -bh <HOST-IP> • Launches a fake SMTP session from the provided IP• Provides a verbose amount of debugging output• No DNS lookups or callouts will occur


Launching an SMTP Session

root@gibson [~]# exim -bs220-gibson.steve.cpanel.net ESMTP Exim 4.69 #1 Mon, 05 Oct 2009 10:28:58 -0400 220-We do not authorize the use of this system to transport unsolicited, 220 and/or bulk e-mail.

root@gibson [~]#

exim -bs • Launches a full fledged local SMTP session

Useful option for -bs:• exim -oMa [host-ip] -bs

Allows you to forge the message’s host originroot@gibson [~]# exim -oMa 1.2.3.4 -bs220-gibson.steve.cpanel.net ESMTP Exim 4.69 #1 Mon, 05 Oct 2009 10:28:58 -0400


Testing Message Routing

root@gibson [~]# exim -bt [email protected]@cpanel.net router = lookuphost, transport = remote_smtp host mx1.cpanel.net [208.74.121.68] MX=0 host mx3.cpanel.net [208.74.121.69] MX=5 host mx2.cpanel.net [208.74.123.60] MX=10root@gibson [~]#

exim -bt [email-address] • Determines how exim would route a given address






Queue Management


Querying The Queue

exim -bp prints all queued messages to STDOUTroot@gibson [~]# exim -bp25m 2.9K 0t5C6f-0000c8-00 <[email protected]> [email protected] [email protected]

.......root@gibson [~]#

exim -bpc prints out total queued messagesroot@gibson [~]# exim -bpc88289129827root@gibson [~]#








Summarizing The Queue

• Meant to be used in a pipe from exim -bp

• -a flag causes statistics to sort by message age

• -c flag causes statistics to sort by message count

root@mx1 [~]# exim -bp | exiqsummCount Volume Oldest Newest Domain----- ------ ------ ------ ------ 1 2252 9m 9m foobar.example.com 1 1843 19h 19h mail3.local.example 1 1331 19h 19h mx.example.com 122 266KB 69h 3h cpanel.net--------------------------------------------------------------- 125 266KB 69h 9m TOTAL

exiqsumm [-a] [-c]


Queued Messages

• It’s frozen! A non-permanent error occurred (e.g. host down) during message delivery, and delivery has been deferred.

• Exim is in queue only modeThis occurs when the load average surpasses the value of queue_only_load in exim.conf

Why would a message be queued?


exiqgrep

root@gibson [~]# exiqgrep -f '[email protected]'20m 355 1MufOA-0003EF-Nh <[email protected]> [email protected]

root@gibson [~]#

exiqgrep [frsyozq] [expression]Uses regular expressions to search the mail queue

Useful application of:• exiqgrep -i -f ‘[email protected]’ | xargs exim -Mrm

Remove all messages with selected criteria










Processing Individual Messages

exim -M <MSG-ID> forces delivery of a message

• -Mrm removes the specified message from queue

• -Mvl displays a log of all previous delivery attempts

Useful options for -M

2009-10-05 04:41:44 Received from [email protected] U=root P=local-esmtp S=3552009-10-05 04:41:44 192.168.99.232 [192.168.99.232] Connection refused2009-10-05 04:41:44 [email protected] R=dumbhost T=remote_smtp defer (111): Connection refused






exinext

root@gibson [~]# exinext [email protected]: 192.168.99.232 [192.168.99.232/NULL] error 111: Connection refused first failed: 05-Oct-2009 04:41:44 last tried: 05-Oct-2009 04:41:44 next try at: 05-Oct-2009 04:56:44root@gibson [~]#

exinext <[email protected]> determines next scheduled delivery attempt






Processing the Queue

exim -q launches a queue runner process

• -qi only processes initial delivery attempts

• -qf forces delivery of all non-frozen messages

• -qff forces delivery of all messages, frozen or not

• -qfl forces delivery of locally destined messages

Useful options for -q


Processing the Queue

exim -S [email-address] • Processes queued messages matching the

provided address.• Partial e-mail addresses are permitted

exim -R [email-address]• Same as above, except based on the recipient(s)

rather than the message sender.

root@toothpick [~]# exim -v -S @kittens.comLOG: queue_run MAIN Start queue run: pid=1494 -S @kittens.comdelivering 1Mup8P-0000O2-2W (queue run pid 1494).....


The Debugger


Why use the debugger?

• You get all of the gory details on what’s going on behind the scenes with Exim.

• Can be used with any call to Exim


Enabling The Debugger

Passing -d to exim enables the debuggerroot@gibson [~]# exim -d -bsExim version 4.69 uid=0 gid=0 pid=15556 D=fbb95cfd.... TRUNCATED ....using ACL "acl_connect"processing "accept"accept: condition test succeededSMTP>> 220-foo.example.com ESMTP Exim 4.69 #1 Wed, 30 Sep 2009 12:51:08 -0500 220-We do not authorize the use of this system to transport unsolicited, 220 and/or bulk e-mail.smtp_setup_msg entered

Use the script command to store output to fileroot@gibson [~]# script -c 'exim -d -bs' debugger.output^CScript done, file is debugger.outputroot@gibson [~]#


Isolating The Output

The -d flag accepts a chain of modifiers, which enable/disable debugging on certain components of the delivery process.

• exim -d+allEnables for all components of delivery process

• exim -d-all+router+transportEnables only for router and transport logic

• exim -d-all+verifyEnables only sender verification logic

Examples:


That’s It!Q & A


configuring and debugging exim

Documents