McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Chapter 26

SMTPandFTP

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Email is used for sending a single message that includes text, voice, video, or graphics to one or more recipients.

Simple Mail Transfer Protocol (SMTP). Sending Mail

User creates mail that looks very similar to postal mail. Envelope: sender and receiver address Message:

Header: Define the sender, receiver, subject of the message, ... Body:contains the actual information to be read by the recipient.

Receiving mail Email system periodically checks the mailboxes. If a user has mail, it informs the user with a notice If the user is ready to read the mail, a list is displayed in which

each line contains a summary of the information about a particular message in the mailbox. Summary contains the sender mail address, the subject, and time the mail was sent or received.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26.1 Format of an email

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26.2 Email address

Mail handling system must use an addressing system with unique addresses.

Local part: Name of a special file, called the user mailbox, where all the mail received for a user is stored for retrieval by the user agent.

Domain Name: Organization usually selects one or more hosts to

receive and send email; they are sometimes called mail exchangers.

Domain name assigned to each mail exchanger either comes from the DNS database or is a logical name (e.g., the name of the organization).

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26.3 User agent

First component of an electronic mail system is the user agent (UA); Mail Reader.

Service provided by a user agent User agent is a software package (program) that

composes, reads, replies to, and forward messages. It also handles mailboxes. Composing messages:

Responsible for composing the email msg to be sent out.

Some of them provide a template and some provide with built-in editor; OR else user can type in an editor and paste into the user agent.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Reading messages User agent is used to read incoming messages. When a user invokes a user agent, it first checks the mail in the

incoming mailbox. A one-line summary [number, flag field indicating if the mail is new OR already read but not replied to OR read and replied to OR …, size of message, sender, subject if subject line in msg is not empty] of each received mail is shown.

Replying to messages After the reading a message, user can use the user agent to

reply to a message. User agent allows the user to reply to the original sender or to

reply to all recipients of the message. Reply contains the original message (for quick reference) and

new message. Forwarding messages: Can forward with or without extra

comments. Handling Mailboxes:

Inbox: Keeps all received emails until they are deleted by the user.

Outbox: Keeps all sent emails until the user deletes them.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

User Agent Types Command-Driven

Normally accepts a one-character command from the keyboard to perform its task.

mail, pine, elm, … GUI-Based

Contain GUI components that allow the user to interact with the software by using both the keyboard and the mouse.

Eudora, Microsoft’s Outlook, Netscape,… SMTP is a simple mail transfer protocol

Can send messages only in 7-bit ASCII format. Cannot be used to send binary files or video or audio. Cannot support few languages that are not supported

by ASCII

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26.4 MIME

Multipurpose Internet Mail Extensions (MIME) Supplementary protocol that allows non-ASCII data to

be sent through SMTP. Only an extension to SMTP. MIME transforms non-ASCII data at the sender side to

ASCII data and delivers them to client SMTP to sent through the Internet. Reverse is done at the receiver side.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26.5 MIME header

MIME defines five headers that can be added to original SMTP header:

MIME-Version Content-Type: Type of data used in the body; has subtypes

separated with slashes. Content-Transfer-Encoding Content-Id: Uniquely identifies the whole message in a

multiple-message environment. Content-Description: Whether the body is image, audio, or

video.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Content Types Text

7-bit ASCII; no transformation by MIME Multipart

Multiple, independent parts. Needs to define the boundary between each part, as

a parameter. Boundary is terminated using two hyphens.

Mixed: Present the parts in exact order as they are in message.

Parallel: Order of parts is unimportant. Digest: Same as mixed but default type/subtype is

message/RFC822 Alternative: Same message is repeated using different

formats.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Message: Body is itself a whole mail message, a part of a

mail message, or a pointer to a message. Subtypes:

RFC822: Used if the body is encapsulating another message (including header and the body).

Partial: Used if the original message has been fragmented into different mail messages and this message is one of the fragments.

Fragments must be reassembled at the destination by MIME.

Three parameters must be added: id, number, total Id: identifies the message and is present in all the

fragments. Number: sequence order of the fragment Total: number of fragments that comprise the original

message External-body.

Body does not contain the actual message but is only a reference (pointer) to the original message.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Image Original message is a stationary image, indicating that there

is no animation. Joint Photographic Experts Group (JPEG) Graphics Interchange Format (GIF)

Video Original message is a time-varying image (animation). Motion Picture Experts Group (MPEG) If the animated image contains sounds, it must be sent

separately using the audio content type. Audio

Original message is a sound. Only subtype is basic, which uses 8-KHz standard audio data.

Application Original message is a type of data not previously defined.

Octet-stream is used when the data must be interpreted as a sequence of 8-bit bytes (binary file).

PostScript is used when the data are in Adobe PostScript format.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Table 26.1Table 26.1 Data types and subtypes in MIMEData types and subtypes in MIME

Type Subtype Description

Text Plain Unformatted text; 7-bit ASCII

Multipart

Mixed Body contains ordered parts of different data types

Parallel Same as above, but no order

Digest Similar to mixed, but the default is message/RFC822

Alternative Parts are different versions of the same message

Message

RFC822 Body is an encapsulated message

Partial Body is a fragment of a bigger message

Ext. Body Body is a reference to another message

ImageJPEG Image is in JPEG

GIF Video is in GIF format

Video MPEG Video is in MPEG format

Audio Basic Single-channel encoding of voice at 8 KHz

ApplicationPostScript Adobe PostScript

Octet-Stream General binary data (8-bit bytes)

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Content-Transfer-Encoding Method to encode the messages into 0s and 1s for transport.

7bit: 7-bit ASCII encoding Length of line should not exceed 1000 characters

8bit: [NOT Recommended] 8-bit encoding; can send non-ASCII characters. Length of line should not exceed 1000 characters MIME does not do any encoding here SMTP must be able to transfer 8-bit non-ASCII characters.

Binary: [NOT Recommended] 8-bit encoding Non-ASCII characters can be sent, and the length of the line can exceed

1000 characters. MIME does not do any encoding here: SMTP must be able to transfer

binary data. Base64:

Solution for sending data made of bytes when the highest bit is not necessarily zero.

Base64 transforms this type of data to printable characters, which can then be sent as ASCII characters or any type of character set supported by underlying mail transfer mechanism.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Table 26.2 Table 26.2 Content-transfer encodingContent-transfer encoding

Category Description

Type ASCII characters and short lines

7bit Non-ASCII characters and short lines

8bit Non-ASCII characters with unlimited-length lines

Binary 6-bit blocks of data are encoded into 8-bit ASCII characters

Base64Non-ASCII characters are encoded as an equal sign followed by an ASCII code

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26. 6 Base64

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Table 26.3 Table 26.3 Base64 encoding tableBase64 encoding table

Value Code Value Code Value Code Value Code Value Code Value Code

0 A 11 L 22 W 33 h 44 s 55 3

1 B 12 M 23 X 34 i 45 t 56 4

2 C 13 N 24 Y 35 j 46 u 57 5

3 D 14 O 25 Z 36 k 47 v 58 6

4 E 15 P 26 a 37 l 48 w 59 7

5 F 16 Q 27 b 38 m 49 x 60 8

6 G 17 R 28 c 39 n 50 y 61 9

7 H 18 S 29 d 40 o 51 z 62 +

8 I 19 T 30 e 41 p 52 0 63 /

9 J 20 U 31 f 42 q 53 1

10 K 21 V 32 g 43 r 54 2

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26.7 Quoted-printable

Quoted-Printable Base64 is a redundant encoding scheme; that is, 24

bits becomes four characters and eventually is sent as 32 bits.

If the data consist mostly of ASCII characters with a small non-ASCII portion, we can use quoted-printable encoding.

If a character is: ASCII, it is sent as it is. Non-ASCII, sent as three characters. First character is

the equals sign (=). Next two characters are the hex of the byte.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26.8 MTA client and server

A mail transfer agent or MTA (also called a mail server, or a mail exchange server is a computer program or software agent which transfers electronic mail messages from one computer to another. The MTA works behind the scenes, while the user usually interacts with another program, the mail user agent (MUA), which contacts an MTA for actual delivery of the mail.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Typical MTAs:

Sendmail

Postfix

Exim

QMail

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26.9 Commands and responses

SMTP uses commands and responses to transfer messages between an MTA client and an MTA server.

Each command or reply is terminated by a two-character (carriage return and line feed) end-of-line token.

Commands Commands are sent from the client to the server. Consists of a keyword followed by zero and more arguments. SMTP defines 14 commands.

Responses Responses are sent from server to client. Response is a three-digit code that may be followed by

additional textual information.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Mail Transfer Phases Process of transferring a mail message occurs

in three phases Connection establishment

After a client has made a TCP connection to the well-known port 25, the SMTP server starts the connection phase.

Message transfer After connection establishment, a single message

between a sender and one or more recipients can be exchanged.

Connection termination After the message is transferred successfully, the client

terminates the connection.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Mail Delivery Delivery of email from sender to receiver consists

of three stages 1st stage

Email goes from user agent to local server Mail does not go directly to remote server because the

remote server may not be available at all times. Mail is stored in the local server until it can be sent.

2nd stage Email is relayed by local server, which now acts as the

SMTP client, to the remote server, which is the SMTP server.

Email is delivered to remote server and not to the remote user agent; Email is stored in mailbox of the user for later retrieval..

Email server runs the SMTP server program. 3rd stage

Remote user agent uses a mail access protocol such as POP3 or IMAP4 to access the mailbox and obtain the mail.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26.10 Email delivery

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Mail access protocols

SMTP is a push protocol; it pushes the message from the sender to the receiver even if the receiver does not want it.

3rd stage needs a pull protocol; the operation must start with the recipient.

Mail must stay in the mail server mailbox until the recipient retrieves it.

POP3 [Post Office Protocol] Client POP3 software is installed on the recipient

computer. Server POP3 software is installed on the mail server.

Client (user agent) opens a connection with the server on TCP port 110.

It then sends its user name and password to access the mailbox.

User can then list and receive mail messages, one by one.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

POP3 modes Delete mode: Mail is deleted from the mailbox after each

retrieval. Keep mode: Mail remains in the mailbox after retrieval.

IMAP4 (Internet Mail Access Protocol) POP3 assumes that each time a client accesses the server, the

whole mailbox will be cleared out (transferred). POP3 does not allow user to organize mail on the server; user

cannot have different folders on the server; POP3 does not allow the user to partially check the contents of the mail before downloading.

User can check the email header prior to downloading User can search the contents of email for a specific string of

characters prior to downloading User can partially download email. User can create, delete, or rename mailboxes on mail server User can create a hierarchy of mailboxes in a folder for email

storage. Web-Based Email: Hotmail or Yahoo

Mail transfer from client browser to mail server is via HTTP SMTP is used for mail transfer from sending mail server to

receiving mail server. Message from receiving mail server to browser is via HTTP

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26.11 POP3

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

File Transfer Protocol

Copying a file from one host to another Problems in FTP

Two systems may use different file name conventions

Two systems may have different ways to represent text and data.

Two systems may have different directory structures. FTP establishes two connections between

client and server. One connection is used for data transfer [Port 20] Other connection for control information (commands

and responses) [Port 21] Separation of commands and data transfer makes

FTP more efficient. Commands are very simple.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26.12 FTP

Client has three components User interface, Client control process, client data transfer process.

Server has two components Server control process, server data transfer process

Control connection is made between control processes; Control connection is maintained during entire interactive FTP session.

Data connection is made between data transfer processes; Data connection is opened & then closed for each file transferred.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26.13 Using the control connection

Control Connection Created in same way as other application programs. Connection remains open during the entire process. Service type used by the IP is minimize delay,

because this is an interactive connection between a user (human) and a server.

User types commands and expects to receive responses without significant delay.

Data connection Opened when data are ready to transfer. Closed

when it is not needed. Service type used by IP is maximize throughput.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26.14 Using the data connection

Communication over control connection Uses ASCII character set. Each command or response is only one short line, so

we need not worry about file format or file structure. Each line is terminated with a two-character

(carriage return and line feed) end-of-line token. Communication over data connection

Client must define the type to be transferred, the structure of the data, and the transmission mode.

Heterogeneity is solved by: file type, data structure, transmission mode.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

File type ASCII file:

By default for transferring text files. Sender transforms the file from its own representation

to ASCII characters Receiver transforms the ASCII characters to its own

representation. EBCDIC file

If one or both ends of connection use EBCDIC encoding (used in IBM computers) the file can be transferred using EBCDIC encoding.

Image file Default format for transferring binary files File is sent as continuous streams of bits without any

interpretation or encoding Used for compiled programs or images.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

File is encoded in ASCII or EBCDIC, another attribute must be added to define the printability of the file.

Nonprint Default format for transferring a text file. File contains no vertical specifications for printing. File cannot be printed without further processing

because there are no characters to be interpreted for vertical movement of the print head.

This format is used for files that will be stored and processed later.

TELNET File contains ASCII vertical characters such as CR

(carriage return), LF (line feed), NL (new line) and VT (vertical tab).

File is printable after transfer.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Data structure FTP can transfer a file across the data connection

using one of the following interpretations about structure of data:

File structure (default) File has no structure. Continuous stream of bytes

Record structure File is divided into records (or structs in C). Used only with text files.

Page structure File is divided into pages, with each page

having a page number and a page header. Pages can be stored or accessed randomly or

sequentially.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Transmission mode: FTP can transfer a file across the data connection by using one of following three transmission modes:

Stream mode Default mode Data is delivered from FTP to TCP as a continuous stream of

bytes; TCP is responsible for chopping data into segments of appropriate size.

End-of-file is closing of data connection by sender. If data is divided into records, each record will have 1-byte end-

of-record (EOR) character, and the end of the file will have a 1-byte end-of-file (EOF) character.

Block mode Data can be delivered from FTP to TCP in blocks. Block is preceded by 3-byte header. 1st byte is called the block descriptor; next 2 bytes defines the

size of block in bytes. Compressed mode

If file is big, data can be compressed. Commonly used compression method is run-length encoding. Consecutive appearance of a data unit are replaced by one

occurrence and number of repetitions. Usually, Blanks in text file and null characters in binary file are compressed.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26.15 File transfer

Means of File Transfer Retrieving a file: Copy file from server to client. Storing a file: copy file from client to server. List of directory or file names is to be sent from the

server to the client. Note that FTP treats a list of directory or file names as a file. It is sent over the data connection.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Example 1Example 1

Figure 26.16 (next slide) shows an example of how a file is stored.

1. The control connection is created, and several control commands and responses are exchanged.

2. Data are transferred record by record.

3. A few commands and responses are exchanged to close the connection.

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Figure 26.16 Example 1

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Table 26.4 Table 26.4 List of FTP commands in UNIXList of FTP commands in UNIX

Commands

!, $, account, append, ascii, bell, binary, bye, case, cd, cdup, close, cr, delete, debug, dir, discount, form, get, glob, hash, help, lcd, ls, macdef, mdelete, mdir, mget, mkdir, mls, mode, mput, nmap, ntrans, open, prompt, proxy, sendport, put, pwd, quit, quote, recv, remotehelp, rename, reset, rmdir, runique, send, status, struct, sunique, tenex, trace, type, user, verbose,?

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Example 2Example 2

We show some of the user interface commands that accomplish the same task as in Example 1. The user input is shown in boldface. As shown below, some of the commands are provided automatically by the interface. The user receives a prompt and provides only the arguments.

$ ftp challenger.atc.fhda.eduConnected to challenger.atc.fhda.edu220 Server readyName: forouzanPassword: xxxxxxxftp > ls /usr/user/report200 OK150 Opening ASCII mode......................226 transfer completeftp > close221 Goodbyeftp > quit

McGraw-Hill ©The McGraw-Hill Companies, Inc., 2004

Example 3Example 3

We show an example of using anonymous [public access OR without an account] FTP. We connect to internic.net, where we assume there are some public data available.

Sometimes, anonymous as user name and guest as password

$ ftp internic.netConnected to internic.net220 Server readyName: anonymous331 Guest login OK, send "guest" as passwordPassword: guestftp > pwd257 '/' is current directoryftp > ls200 OK150 Opening ASCII modebin...ftp > close221 Goodbyeftp > quit