1

System Programming

Abdul-Rahman Mahmood

http://alphapeeler.sourceforge.net

http://pk.linkedin.com/in/armahmood

abdulmahmood-sss twitter.com/alphapeeler

alphapeeler.sourceforge.net/pubkeys/pkey.htm

VC++, VB, ASP

About the instructor:Roles/Skills

ROLES # Y ORGANIZATIONS

Chief Technology Officer 4.2 Riysoft PVT LTD

Consultant / Head Crypto Div– MOD 3.0 Ministry of Defense , RikSof, SecureBytes, RT Japan, Pyntail, SSS

Proj Manager, Team Lead, OG-I 1.5 National Bank – Head Office (IT Group)

Proj Coord / Analyst / Tech Lead 2.5 Plexus (Global Partner - WSI International, Canada.)

QMR / Proj / Process / HR Manager 1.5 Softech Worldwide L.L.C. (A US based software house )

IT Manager (Computer Engineer) 1.5 Peritech Intl.– Nagoya, Japan

Sr. Software Engineer 3.5 Infinilogic Pvt. Ltd. (A UK based software house)

TECHNICAL SKILLS DETAILS

Languages & Tools C++, C#, ObjectiveC, PHP,ASP,VC,VB,COM,MTS,ATL,HTML,JScript,Qt

Mobile App Development Tools X-Code 3.2/4.0, Titanium. ADT, MS Silverlight 4, Expression Blend

Project Management Dot project, MS Project, Mind manager 6 pro, net office.

Software Architecture Rational rose, Design for databases, Erwin modeler, MS Visio 2003

Quality Assurance Testog, PVCS tracker, Requisite pro, Mantus BT

Configuration & Content Mgmt. Subversion, CVS, VSS, Git, SharePoint, Mambo Server, Wordpress, Durpal

Databases / RDBMS Mysql, Postgres, Oracle, MSSQL, MS Access, SQLite

Server Administration Win server, SUSE Linux, Apache / IIS, postfix, openxpki, samba, privoxy

Reporting Tools Crystal Reports 8.5, Microsoft SQL server reporting Services.

CERTIFICATIONS / WORKSHOPS

CERTIFICATIONS YEAR INSTITUTION DETAILS

QMR certification 2005 Softech worldwide L.L.C (USA) Quality Management Rep.

Quality Auditing 2005 Pakistan Institute of Quality Control ISO 9001:200

Brainbench CertificationsMS Visual C++, ASP, MS Visual Basic

2001 Brainbench Corporation, 14425 Penrose Place, Suite 150, Chantilly, VA 20151, U.S.A.

VC Transcript ID: 2574319VB Transcript ID: 2574319ASP Transcript ID: 2574319

Microsoft Certified System Engineer

1999 Microsoft Certified System Engineer, training at SSUET Network Admin, IIS , Tech. Support, Internetworking,

Microsoft Certified Professional

1998 Microsoft Corporation, Redmond, WA 98052-6399, U.S.A.

MS Win ServerMCP ID: 1270382

AutoCAD certification 1993 Computer Guide Institute Release 10

WORKSHOPS YEAR INSTITUTION DETAILS

Performance & Load Testing 2009 NUST – SEECS, Islamabad School of Electrical Engineering & Computer Science, NUST,17-1-2009

Network Security 2008 Networkers Society of Pak Sheraton Hotel, Karachi, 7-10-2008.

Microsoft PDC 2007 Microsoft – Pakistan SQL server 2008

AWARD / RECOGNITION / TRAININGS

AWARD/RECOGNITION YEAR INSTITUTION DETAILS URL

Revised Embedded OS in Operating Systems book by Dr. William

2011 Prentice Hall – USAISBN-10: 013230998XISBN-13: 9780132309981

Acknowledged by Dr. William Stallings in book preface (7th Edition).(Embedded OS : TinyOS & ECOS)

AlphaPeeler - packaged with Cryptography & Network Security book

2010 Prentice Hall – USADeveloped at SSUET, Karachi –Pakistan

Author of book Dr. William Stallings included this educational tool in his book of cryptography 5th

Edition.

NBP Excellence Award 2007 National Bank of Pakistan Awarded on meritorious achievements.

Certification for Professional Engineer

2000 Pakistan Engineering Council. Reg. # COMP/1343

SOFTEC 98 Award 1998 FAST – ICS, Lahore Implementation of DesignoCAD

PROCOM 98 Award 1998 FAST – ICS, Karachi Implementation of DesignoCAD

AlphaPeeler – Classical cryptography tool

1998-2011

Developed at SSUET - Karachi Most popular educational crypto toolhttp://alphapeeler.sourceforge.net

TEACHING/TRAINING YEAR ORGANIZATION DETAILS URL

Documentation, Linux 2010 Ministry of Defense- Islamabad Delivered training for employees

Decision Making, 360 Employee Evaluations

2008 The Shams Group (Shams Software Services) – USA.

Conducted internal training: Decision Making, 360° evaluations.

Req. Elicitation, RN, Coding Standards

2006 Plexus PVT. LTD. - Karachi SDLC, Development standards & secure coding techniques & tips

PHP, Linux, MySQL 2005 Softech Worldwide LLC - USA open-source technology trainings

Chapter One Introduction to Unix 5

System Programming

Introduction to Unix

Chapter One Introduction to Unix 6

Recommended Course Textbooks

Jerry Peek, Grace Todino, and John Strang , Learning the Unix Operating System Fifth Edition

Deborah S.Ray, Eric J.Ray, UNIX and Linux (Learn UNIX and Linux the quick and easy way) 4th Edition

J. Purcell, Linux Complete Command Reference, By RED HAT SOFTWARE, INC. 1st Edition

S. G. Kochan, P. Wood (2003)Unix Shell Programming, 3rd Edition, Sams, 460 p.

R. K. Michael (2003)Mastering UNIX Shell Scripting, Wiley, 680 p.

W. R. Stevens, S. A. Rago (2005)Advanced Programming in the Unix environment, 2nd Edition, Addison Wesley, 960 p.

K. A. Robbins, S. Robbins (1996)Practical Unix Programming, Prentice Hall, 658 p.

Chapter One Introduction to Unix 7

What is Unix?

A modern computer operating system

Operating system ―a program that acts as an intermediary

between a user of the computer and the computer hardware‖

Software that manages your computer’s resources (files, programs, disks, network, …)

Examples: Windows, MacOS, Solaris, BSD, Linux (e.g. OpenSUSE, Red Hat, Slackware)

Modern Stable, flexible, configurable, allows multiple

users and programs

What is Unix?

Unix is a multi-user, multi-taskingoperating system.

You can have many users logged into asystem simultaneously, each running manyprograms.

It's the kernel's job to keep each processand user separate and to regulate accessto system hardware, including cpu,memory, disk and other I/O devices.

Chapter One Introduction to Unix 8

Chapter One Introduction to Unix 9

Why Unix?

Used in many scientific and industrial settings

Huge number of free and well-written software programs

Open-source operating system (OS)

Excellent programming environment

Largely hardware-independent

Based on standards

Internet servers and services run on Unix Roughly 65% of the world’s web servers are

Linux/Unix machines running Apache

Brief History of Unix

First Version was created in Bell Labs in 1969.

Some of the Bell Labs programmers who hadworked on this project, Ken Thompson, DennisRitchie, Rudd Canaday, and Doug McIlroydesigned and implemented the first version of theUnix File System on a PDP-7 along with a fewutilities. It was given the name UNIX by BrianKernighan.

00:00:00 Hours, Jan 1, 1970 is time zero for UNIX.It is also called as epoch.

Chapter One Introduction to Unix 10

Chapter One Introduction to Unix 11

Brief History of Unix

Ken Thompson & Dennis Richie originally developed the earliest versions of Unix at Bell Labs for internal use in 1970s

Simple and elegant

Borrowed best ideas from other OSs

Meant for programmers and computer experts

Meant to run on ―mini computers‖

Chapter One Introduction to Unix 12

Early Unix History

Thompson also rewrote the operating system in high level language of his own design which he called B.

The B language lacked many features and Ritchie decided to design a successor to B which he called C.

They then rewrote Unix in the C programming language to aid in portability. Small portion written in assembly language

(kernel)

Remaining code written in C on top of the kernel

Chapter One Introduction to Unix 13

Unix History Multics 1965

First Edition 1971 (AT&T) (CACM 1974, 365-375)

1BSD 1977 (Berkeley)

Sixth Edition 1975 (AT&T)

4BSD 1980 (Berkeley)

SunOS 1985 (Sun)

System V 1985 (AT&T)

Tenth Edition 1989 (AT&T)

4.3BSD Net/2 1991 (Berkeley)

First Linux kernel 1992 (Linus)

Solaris 1993 (Sun)

FreeBSD-1.0 1993

NetBSD-1.0 1994

OpenBSD-2.0 1996

Max OS X 10.1 2001 (Apple)

Chapter One Introduction to Unix 14

Unix versions

Two main threads of development:

Berkeley software distribution (BSD) (http://www.bsd.org)

Unix System Laboratories (http://www.unix.org)

BSD

SunOS 4, Ultrix, BSDI, OS X, NetBSD, FreeBSD,

OpenBSD, Linux (GNU)

SYS V

System V (AT&T -> Novell -> SCO), Solaris

(SunOS 5), HP-UX (Hewlett-Packard), AIX

Chapter One Introduction to Unix 15

Brief History of Linux

Andrew Tanenbaum, a Dutch professor developed

MINIX to teach the inner workings of operating

systems to his students

In 1991 at the University of Helsinki, Linus Torvalds,

inspired by Richard Stallman’s GNU free software

project and the knowledge presented in Tanenbaum’s

operating system, created Linux, an open-source,

Unix-based operating system

Over the last decade, the effort of thousands of open-

source developers has resulted in the establishment of

Linux as a stable, functional operating system

Linux

Linux is a free Unix-type operating systemoriginally created by Linus Torvalds with theassistance of developers around the world.

It originated in 1991 as a personal project of LinusTorvalds, a Finnish graduate student.

The Kernel version 1.0 was released in 1994 andtoday the most recent stable version is 2.6.9

Developed under the GNU General Public License ,the source code for Linux is freely available toeveryone.

Chapter One Introduction to Unix 16

Chapter One Introduction to Unix 17

Layers in a Unix-based System

User Interface

Library interface

System interface calls User Mode

Kernel Mode

Hardware

(CPU, memory, disks, terminals, etc.)

Unix Operating System

(process management, memory management,

file system, I/O, etc.)

Standard Library

(open, close read, write, etc.)

Standard Utility Programs

(shell, editors, compilers, etc.)

Users

Chapter One Introduction to Unix 18

Unix Structure

The kernel is the core of the Unix operating system, controlling the system hardware and performing various low-level functions. Other parts of a Unix system (including user programs) call on the kernel to perform services for them.

The shell accepts user commands and is responsible for seeing that they are carried out.

Chapter One Introduction to Unix 19

Unix Structure (cont.)

Over four hundred utility programs or tools are supplied with the Unix system. These utilities (or commands) support a variety of tasks such as copying files, editing text, performing calculations, and developing software.

This course will introduce a limited number of these utilities or tools, focusing on those that aid in software development.

Chapter One Introduction to Unix 20

Getting started

Chapter One Introduction to Unix 21

The Unix Account

Logging in to a Unix machine requires an account on that system.

A user account is associated with login and password.

―login‖ is your user name (usually some variant of your real name)

Your password will not echo as you type

Remember good password practices

Chapter One Introduction to Unix 22

Logging into a UNIX system

init (Process ID 1 created by the kernel at bootstrap)

spawns getty for every terminal device

getty opens terminal device, sets file descriptors 0, 1, 2 to it, waits for a user name, usually sets some environment variable (TERM)

invokes login when user name entered

login reads password entry (getpwnam()), asks for user’s password (getpass()) and validates it; changes ownership of our terminal device, changes to our UID and changes to our home directory. Sets additional environment variables (HOME, SHEL, USER, LOGNAME, PATH)

invokes our login shell

Login shell (bash)

Logging into a UNIX system

bootstrap : The ROM routine used to load the OS is often known as

getty, short for "get teletype", is a Unix program running on a host computer that manages physical or virtual terminals (tty). When it detects a connection, it prompts for a username and runs the 'login' program to authenticate the user. getty is usually called by init.

One getty process serves one terminal. In some systems, for example Solaris, getty was replaced by ttymon.

Chapter One Introduction to Unix 23

Logging into a UNIX system

init (short for initialization) is a daemon process that is the ancestor of all other processes. Init is the first process started during booting, and is typically assigned PID number 1. It is started by the kernel using a hard-coded filename, and if the kernel is unable to start it, a kernel panic will result. Init continues running until the system is shut down.

Bash is a Unix shell written by Brian Fox for the GNU Project as a free softwarereplacement for the Bourne shell (sh).[3][4] Released in 1989,[5] it has been distributed widely as the shell for the GNU operating system and as the default shell on Linux and Mac OS X. It has been ported to Microsoft Windows and distributed with Cygwin and MinGW, to DOS by the DJGPP project, to Novell NetWare and to Android via various terminal emulation applications.

Chapter One Introduction to Unix 24

Bash

Chapter One Introduction to Unix 25

Logging into a UNIX system

Bash is a POSIX shell but with a number of extensions.

The Bourne shell (sh) was the default Unix shell of Unix Version 7. Most Unix-like systems continue to have /bin/sh—which will be the Bourne shell, or a symbolic link orhardlink to a compatible shell—even when other shells are used by most users.

Developed by Stephen Bourne at AT&T Bell Laboratories, it was a replacement for theThompson shell, whose executable file had the same name—sh. It was released in 1977 in the Version 7 Unix release distributed to colleges and universities. Although it is used as an interactive command interpreter, it was always intended as a scripting language and contains all the features that are commonly considered to produce structured programs.

Chapter One Introduction to Unix 26

Logging into a UNIX system

POSIX ( /ˈpɒzɪks/ POZ-iks), an acronym for "Portable Operating System Interface", is a family of standards specified by the IEEE for maintaining compatibility between operating systems. POSIX defines the application programming interface (API), along with command line shells and utility interfaces, for software compatibility with variants of Unix and other operating systemsChapter One Introduction to Unix 27

Chapter One Introduction to Unix 28

How Logins were processed

init (using /etc/ttys or /etc/inittab) -> forks and execs getty

programs on each terminal

getty gets user name -> execs login

login verifies password -> execs login shell

User uses login shell

Login methods: Using an X display

User logins in via getty/login, then runs startx

xdm -- reads username & passwd, starts X as that user

Somewhat like a startx without the login shell

Can start a "terminal" (or shell) window

Chapter One Introduction to Unix 29

Secure Login Tools

Terminal connection

PuTTy (on Windows)

MindTerm (Java applet)

Desktop connection

X-Win32

CDE, KDE, GNOME

WeirdX (Java application)

File transfer

WinSCP3

SmartFTP

Chapter One Introduction to Unix 30

What is a Shell?

Just a Unix program executed when you log in A command interpreter

provides the basic user interface to UNIX utilities

A programming language program consisting of shell commands is called a

shell script you can put commands in a file and execute it:

First, make the file executable (chmod u+x script−file) Lines starting with # are comments Make use of interpreter files (kernel feature!): the first line

of your script file must begin with a line:#!pathname optional−argumentswhere pathname is an absolute pathname (typically /bin/sh,

or /bin/bash) of the interpreter.

Chapter One Introduction to Unix 31

The Shell Prompt

After logging in, some information about the system will be displayed, followed by a shell prompt, where commands may be entered

$

%

#

username@hostname>

hostname %

Chapter One Introduction to Unix 32

The Shell

The shell is the program you use to send commands to the Unix system

Some commands are a single word

who

date

ls

Others use additional information

cat textfile

ls -l

Chapter One Introduction to Unix 33

Command Syntax

Commands must be entered exactly. If you make a mistake before entering, delete/backspace to fix it. Be careful!

command options argument(s)

Options modify a command’s execution

Arguments indicate upon what a command should act (often filenames)

Chapter One Introduction to Unix 34

Example Commands: ls (list)

ls –l

ls –a

ls –la

ls –a; ls –l

ls -F

ls –al textfile1

ls –al textfile1 textfile2

ls –al directory

-l long format, displaying Unix file types, permissions, number of hard links, owner, group, size, date, and filename

-F appends a character revealing the nature of a file, for example, * for an executable, or / for a directory. Regular files have no suffix.

-a lists all files in the given directory, including those whose names start with "." (which are hidden files in Unix). By default, these files are excluded from the list.

-R recursively lists subdirectories. The command ls -R / would therefore list all files.

-d shows information about a symbolic link or directory, rather than about the link's target or listing the contents of a directory.

-t sort the list of files by modification time.

-h print sizes in human readable format. (e.g., 1K, 234M, 2G, etc.)

Chapter One Introduction to Unix 35

Sample usage : ls

The following example demonstrates the output of the ls command given two different arguments:

$ pwd

/home/fred

$ ls -l

drwxr--r-- 1 fred editors 4096 drafts

-rw-r--r-- 1 fred editors 30405 edition-32

-r-xr-xr-x 1 fred fred 8460 edit

$ ls -F

drafts/

edition-32

edit*(x)

Chapter One Introduction to Unix 36

In this example, the user fred has a directory named drafts, a regular file called edition-32, and an executable named edit in his home directory. ls uses unix file permission notation to indicate which users or groups are allowed to access each file or directory.

drwxr--r-- 1 fred editors 4096 Mar 1 2007 drafts This means that the letters behind the file descriptor (d), which indicates a folder or 'directory', list three characters to indicate permissions for the current user (rwx), then the group to which the file belongs (r--), and the rights of others (r--).

'drafts' is a directory (d), the user has the right to read (r) write (w) and execute (x): rwx, group members have (r--), meaning read only, and others have (r--), meaning read only access.

Chapter One Introduction to Unix 37

Chapter One Introduction to Unix 38

Command Execution

The current shell (bash) executes built-in commands (echo, kill, pwd, …)

or shell scripts invoked by the . (dot) command: . shell-script

calls fork() to create a new shell process sub-shell (bash)

The sub-shell executes a shell script or calls exec() to execute a command or program terminates after script or command execution

During command execution, the parent either waits, or continues if

command is executed in the background

Chapter One Introduction to Unix 39

No Shell Prompt

If you don’t get a prompt

A program is probably running

If you see a special program prompt, try to quit the program (quit, bye, exit)

If you see nothing, you can

Stop the program with CTRL-Z (program will wait until started again)

Interrupt the program with CTRL-C (program will usually die)

Chapter One Introduction to Unix 40

Logging Out

Always log out when you are done

Use the exit command to log out of a

shell (sometimes logout or CTRL-D)

Note: if you are running in a windowing environment, logging out of the shell only ends that shell. You must also log out of the windowing, typically selecting an option from a menu.