screen scraping using “ibm personal communication” scraping ibm 5250.pdf · screen scraping...
TRANSCRIPT
www.tobuku.com
Version 1.0 - 1 -
SCREEN SCRAPING USING “IBM PERSONAL
COMMUNICATION”
Feb 2011 Level: √
By : Feri Djuandi Beginner Intermediate Expert
Platform : Visual C# 2010
INTRODUCTION
This article explains the programming technique used with IBM Personal Communications to
provide a way for users and programmers to access the iSeries host screen with a set of
functions that can be called from a custom application running on a workstation.
The Personal Communications is a terminal emulation program as shown in the following
example screen shots.
www.tobuku.com
Version 1.0 - 2 -
In a normal way, users are using this emulation program to access the IBM iSeries screens to
interact with the host. However it is possible to create a custom application with a nice
Windows graphical look rather than the original textual green screen. This leads us to a
programming technique so called screen scraping.
NOTE:
In relation to this context, IBM documentation uses the term of
“presentation space” rather than “terminal emulator”. In this article we
will use both terms interchangeably which have the same meanings.
Screen scraping is programming that translates between legacy application programs (written to
communicate with now generally obsolete input/output devices and user interfaces) and new
user interfaces so that the logic and data associated with the legacy programs can continue to
be used. Screen scraping is sometimes called advanced terminal emulation. A program that does
screen scraping must take the data coming from the legacy program that is formatted for the
screen of an older type of terminal such as an IBM 3270 or IBM 5250 and reformat it for a
Windows user or someone using a Web browser. The program must also reformat user input
from the newer user interfaces (such as a Windows graphical user interface or a Web browser)
so that the request can be handled by the legacy application as if it came from the user of the
older device and user interface.1
Data scraping is most often done to either (1) interface to a legacy system which has no other
mechanism which is compatible with current hardware, or (2) interface to a third-party system
which does not provide a more convenient Application Programming Interface (API). In the
second case, the operator of the third-party system may even see screen scraping as unwanted,
due to reasons such as increased system load, the loss of advertisement revenue, or the loss of
control of the information content.
Data scraping is generally considered an ad hoc, inelegant technique, often used only as a "last
resort" when no other mechanism is available.2
In spite of the pros and cons, the screen scraping is no doubt provides a simple and quick way to
bridge the modern Windows applications and the legacy system. The easiness becomes the
main reason why this technique is still used now. The screen scraping technique promotes the
reusability. Meaning the use of screen scrapper program does not require any modification or
enhancement of the original programs or screens in the host, so we can use the existing ones.
The development is only on the custom applications, while everything remains the same on the
host side. It significantly reduces the development cost because normally the development
efforts of COBOL or RPG programs are very expensive, while developing custom applications
(e.g. C++, VB) are typically less costly.
In the market today, there are choices for the programmers to select screen scraper software
with various features and platform supported. IBM Personal Communications is one of them.
With no intention promoting it as the best among the others, I see some reasons why we may
want to use this technique:
1 A definition from Whatis.com
2 Comments found in Wikipedia
www.tobuku.com
Version 1.0 - 3 -
� Free license. The IBM Personal Communications is a companion client application to the
IBM iSeries. It is a freeware program that does not obligate the user to pay for using it
(that’s what I read from resources in the Internet). This is a significant benefit. There are
some screen scraper programs in the market sold with different levels of price. Multiply
by the number of users in the company, and then you will end up with a surprising huge
amount. If we can have a free software, why should buy?
� Compatibility. This client program is developed by IBM to access its host machine. We
can be almost 100% sure of its compatibility. Surprisingly the IBM Personal
Communications works fine with a wide range of Windows programming language like
C++, VB6 up to .NET. This program was written many years ago which I believe having no
certain view of the technology development today, but I am amazed this program can
collaborate with Ms Visual C# 2010 nicely!
� Fast and stable. The IBM emulation program has been being used by many users for
quite long time until today through various version releases. It is proven stable and fast.
� Easy to develop. The API is quite easy to be used with the custom application. The
technical documentation and sample programs are available in the Internet in an easy to
understand fashion. Many chat rooms in the Web discussing the topic with assorted
technical cases.
EMULATOR API
There are numbers of API come with the IBM Personal Communications product where each
interface has a specific set of functions and may be used for different purposes. A programmer
only needs to choose an interface that best matches the functional requirements of the
application, although some applications may use more than one interface to achieve the desired
results. The programming interfaces are:
� Emulator High Level Language API (EHLLAPI)
This API provides functions to access the emulator such as characters on the host
screen. It also provides functions for sending keystrokes to the host, intercepting user-
entered keystrokes, querying the status of the host session, uploading and downloading
files, and other functions. This interface is often used for automated custom
applications which read host screens and enter keystrokes without direct user
intervention.
� Dynamic Data Exchange (DDE)
This interface is similar to the EHLLAPI interface in that it provides a programmable
means to read the host screen, send keystrokes, and perform related functions. It has
some additional functions for access to the emulator clipping rectangle, intercepting
mouse events, and adding/removing commands on the emulator menu bar.
� Personal Communications Session API (PCSAPI)
This interface is used to start, stop, and control emulator sessions and settings.
� Server-Requestor Programming Interface (SRPI)
www.tobuku.com
Version 1.0 - 4 -
This interface is used in cooperation with an IBM Enhanced Connectivity Facility (ECF)
application running on a host system. This API provides functions for writing
synchronous call-return interfaces to remote server programs.
� IBM Personal Communications Host Access Class Library (ECL)
It is a set of objects that allow application programmers and scripting language writers
to access host applications easily and quickly. Personal Communications supports three
different ECL layers (C++ objects, ActiveAutomation (OLE), and LotusScript Extension
(LSX)).
With no intention to disregard the other APIs, I choose EHLLAPI in this article for discussion
because I find it the most convenient to use with the custom applications than the others.
EHLLAPI PROGRAMMING
The model of the screen scrapper programming is shown in the following picture.
As seen in the picture, the custom application interacts with the host through the API callings.
The API will then communicates with the presentasion space to send the commands or
characters and receives back the response from the iSeries. The API does not directly access the
iSeries, instead still uses the presentation space in the middle. In this structure the custom
application may simulate the key strokes by sending the characters to the API functions as if the
user entering the keys through the terminal emulator. The host will not be aware whether the
interactions are done through the custom application or the emulator.
Requirement in the EHLLAPI programming:
� The IBM Personal Communication used in this article is version 5.7. I have no experience
in using the software other than the specified version, but I suppose there won’t be
many differences in the API architecture so you may follow the The API is installed
together with the emulation program so basically there is no additional installation or
plug in required for this purpose. Simply install Personal Communication, and that’s it. It
must be done either on the programmer’s computer (for development use) and on the
users’ computers (when the custom application is released for production).
Presentation
space
Custom
Applications API
Host
iSeries system
IBM PERSONAL
COMMUNICATION
www.tobuku.com
Version 1.0 - 5 -
� Any development tool can be used so long it supports the API DLL calls. The code
examples presented in this article are using Ms Visual C# 2010 Express. The reason why
I use this programming language is only for the convenience sake. You may use other
language if you personally are more familiar with it.
The EHLLAPI is implemented in a form of a DLL file. The file named PCSHLL32.DLL is located in
the ..\Program Files\IBM\Client Access\Emulator directory. Ensure this directory is referenced
in the PATH system variable so that the DLL file is accessible from any application working
directory. To verify whether the path is specified in the system variable, simply open the
command prompt and type the PATH command.
In the custom application, this API file must be referenced by declaring a command line
something like this:
...
[DllImport("PCSHLL32.dll")]
public static extern UInt32 hllapi(out UInt32 Func, StringBuilder Data,
out UInt32 Length, out UInt32 RetC);
...
Notice that inside the PCSHLL32.DLL file there is a function called hllapi. This is the one and the
only function that we use to perform the API call. All interactions to the iSeries system are done
through this function. What controls or differentiates the actions is the parameters sent to this
function. The calling of the hllapi function follows the syntax:
hllapi(out <function_number>, <data>, out <length>, out <ps_position> )
The hllapi function receives various combinations of parameter values that distinguish the
action to be performed. The hllapi function takes four parameter as follow:
1. Function number (integer). This number determines the action that the API will do for
the caller. For instance 1 to connect the presentation space, 2 to disconnect from the
presentation space, 3 to send key, etc. There is a command table in the Appendix that
lists the parameters for specified commands.
2. Data (string). The character or text that wishes to be sent to the presentation space or
read from.
3. Length (integer). The length of the corresponding character or text.
4. Position (integer). The position of the cursor in the presentation space.
NOTE:
The presentation space position is calculated using the formula:
Position = ( (row – 1) * screen_width) + col
row = vertical position of the cursor from the top edge, starts from 1 to
the maximum height of the screen.
col = horizontal position of the cursor from the left edge, starts from 1
to the maximum width of the screen.
Screen_width depends on the size chosen for the presentation space, but
typically the width is 80 characters.
www.tobuku.com
Version 1.0 - 6 -
The EHLLAPI allows a lot number of commands to be invoked by the caller. However among the
available commands there are some tasks most commonly used in the custom application:
� Connecting to the presentation space
� Disconnecting from the presentation space
� Reading text from the presentation space
� Sending keys
� Waiting the response from the host
� Getting the current cursor position
� Moving the cursor position
The next sections in this article will explain the commands in more detail.
www.tobuku.com
Version 1.0 - 7 -
Connecting to the Presentation Space Establishing the connection from a custom application to the host presentation space is the first
step that must be carried out before any other command can be run.
EHLLAPI function parameters:
Parameters Values
Function number 1
Data string The letter of the presentation space ID (PSID), i.e. A, B, C …
Length 1 for standard interface; 4 for enhanced interface
PS Position NA
Return value:
Return Description
0 The Connect Presentation Space function was successful; the host presentation space
is unlocked and ready for input.
1 An incorrect host presentation space ID was specified. The specified session either
does not exist or is a logical printer session. This return code could also mean that the
API Setting for DDE/EHLLAPI is not set on.
4 Successful connection was achieved, but the host presentation space is busy.
5 Successful connection was achieved, but the host presentation space is locked (input
inhibited).
9 A system error was encountered.
11 This resource is unavailable. The host presentation space is already being used by
another system function.
Example:
...
StringBuilder Data = new StringBuilder(4);
Data.Append(psid.ToString()); //the presentation space ID
UInt32 rc = 0; //NA
UInt32 f = 1;
UInt32 l = 4;
ret = hllapi(out f, Data, out l, out rc);
...
NOTE:
Before the custom application connects to the presentation space, be sure
that the IBM Personal Communications program is already running and
connected to the host.
Read the Appendix for the explanation of command methods to start the
emulation program
www.tobuku.com
Version 1.0 - 8 -
Disconnecting from the Presentation Space Terminating the connection from the host presentation space after the custom application
finishes.
EHLLAPI function parameters:
Parameters Values
Function number 2
Data string NA
Length NA
PS Position NA
Return value:
Return Description
0 The Disconnect Presentation Space function was successful.
1 The program was not currently connected to the host presentation space.
9 A system error was encountered.
Example:
...
StringBuilder Data = new StringBuilder(4);
Data.Append(sessId); //NA
UInt32 rc = 0; //NA
UInt32 f = 2;
UInt32 l = 4; //NA
ret = hllapi(out f, Data, out l, out rc);
...
www.tobuku.com
Version 1.0 - 9 -
Reading Text Reading the characters from a static text or field content at a certain position in the
presentation space.
EHLLAPI function parameters:
Parameters Values
Function number 8
Data string After the function called, this parameter contains the string read
from the presentation space.
Length Length of the target data string.
PS Position Position within the host presentation space of the first character.
Return value:
Return Description
0 The host presentation space contents were copied to the application program. The
target presentation space was active, and the keyboard was unlocked.
1 The program is not connected to a host session.
2 An error was made in specifying string length, or the sum of (Length - 1) + PS position is
greater than the size of the connected host presentation space.
4 The host presentation space contents were copied. The host presentation space was
waiting for host response.
5 The host presentation space was copied. The keyboard was locked.
7 The host presentation space position is not valid.
9 A system error was encountered.
Example:
...
StringBuilder Data = new StringBuilder(3000);
UInt32 rc = (UInt32)((rowPos - 1) * SCREEN_WIDTH + colPos);
UInt32 f=8;
UInt32 l = (UInt32)textLen;
ret = hllapi(out f, Data, out l, out rc);
txt = Data.ToString().Substring(0, (int)textLen);
...
www.tobuku.com
Version 1.0 - 10 -
Sending Keys Sending keystrokes to the current cursor position. This task can be two meanings: (1) sending a
command key (e.g. Enter, Tab, F3), or (2) sending text (e.g. entering a field)
EHLLAPI function parameters:
Parameters Values
Function number 3
Data string A string of keystrokes, maximum 255. Uppercase and lowercase
ASCII characters are represented literally. Function keys and shifted
function keys are represented by mnemonics. See Keyboard
Mnemonics in the Appendix.
Length Length of the source data string.
PS Position NA
Return value:
Return Description
0 The keystrokes were sent; status is normal.
1 The program is not connected to a host session.
2 An incorrect parameter was passed to EHLLAPI.
4 The host session was busy; all of the keystrokes could not be sent.
5 Input to the target session was inhibited or rejected; all of the keystrokes could not be
sent.
9 A system error was encountered.
Example:
...
StringBuilder Data = new StringBuilder(str.Length);
Data.Append(str);
UInt32 rc=0; //NA
UInt32 f=3;
UInt32 l=(UInt32) str.Length;
ret = hllapi(out f, Data, out l, out rc);
...
www.tobuku.com
Version 1.0 - 11 -
Waiting the Response
After a command is sent to the host, there may be a certain time for the host to process the
request until it can respond again for the subsequent command. During the period, the custom
application must wait for the host until it completes the work. The custom application should
not send another command or text to the host because they are potentially lost or abandoned
by the host.
EHLLAPI function parameters:
Parameters Values
Function number 4
Data string NA
Length NA
PS Position NA
Return value:
Return Description
0 The keyboard is unlocked and ready for input.
1 The application program is not connected to a valid session.
4 Timeout while still in XCLOCK (X []) or XSYSTEM.
5 The keyboard is locked.
9 A system error was encountered.
Example:
...
StringBuilder Data = new StringBuilder(0); //NA
UInt32 rc = 0; //NA
UInt32 f = 4;
UInt32 l = 0; //NA
ret = hllapi(out f, Data, out l, out rc); ...
www.tobuku.com
Version 1.0 - 12 -
Getting the Current Cursor Position To get the current position of the cursor in the presentation space.
EHLLAPI function parameters:
Parameters Values
Function number 7
Data string NA
Length NA
After the function called, it holds the host presentation space
position of the cursor
PS Position NA
Return value:
Return Description
0 The Query Cursor Location function was successful.
1 The program is not currently connected to a host session.
9 A system error was encountered.
Example:
...
StringBuilder Data = new StringBuilder(0); //NA
UInt32 rc = 0; //NA
UInt32 f = 7;
l = 0; //return position
ret = hllapi(out f, Data, out l, out rc);
...
www.tobuku.com
Version 1.0 - 13 -
Moving the Cursor Position To move the cursor to a new location in the presentation space.
EHLLAPI function parameters:
Parameters Values
Function number 40
Data string NA
Length NA
PS Position New cursor position in the presentation space
Return value:
Return Description
0 Cursor was successfully located at the specified position.
1 The program is not connected to a host session.
4 The session is busy.
7 A cursor location less than 1 or greater than the size of the connected host
presentation space was specified.
9 A system error was encountered.
Example:
...
StringBuilder Data = new StringBuilder(0); //NA
UInt32 rc = (UInt32)p; //the new position
UInt32 f = 40;
UInt32 l = 0; //NA
ret = hllapi(out f, Data, out l, out rc);
...
www.tobuku.com
Version 1.0 - 14 -
APPENDIX
EHLLAPI Functions Number
Action Function Number
Connect Presentation Space 1
Disconnect Presentation Space 2
Send Key 3
Wait 4
Copy Presentation Space 5
Search Presentation Space 6
Query Cursor Location 7
Copy Presentation Space to String 8
Set Session Parameters 9
Query Sessions 10
Reserve 11
Release 12
Copy OIA 13
Query Field Attribute 14
Copy String to Presentation Space 15
Pause 18
Query System 20
Reset System 21
Query Session Status 22
Start Host Notification 23
Query Host Update 24
Stop Host Notification 25
Search Field 30
Find Field Position 31
Find Field Length 32
Copy String to Field 33
Copy Field to String 34
Set Cursor 40
Start Close Intercept 41
www.tobuku.com
Version 1.0 - 15 -
Query Close Intercept 42
Stop Close Intercept 43
Start Keystroke Intercept 50
Get Key 51
Post Intercept Status 52
Stop Keystroke Intercept 53
Query Communication Event 81
Stop Communication Notification 82
Send File 90
Receive File 91
Cancel File Transfer 92
Convert Position or Convert RowCol 99
Connect Window Services 101
Disconnect Window Service 102
Query Window Coordinates 103
Window Status 104
Change Switch List LT Name 105
Change PS Window Name 106
Start Playing Macro 110
For the more detail specification of this function, please refer to the official document in the
Reference.
www.tobuku.com
Version 1.0 - 16 -
Keyboard Mnemonic Use these mnemonic codes to send the special key strokes to the presentation space.
Meaning Mnemonic Code
@ @@
Alt @A
Alt Cursor @$
AEhllapiFuncntion @A@Q
Backspace @<
Backtab (Left Tab) @B
Clear @C
Cmd (function) Key @A@Y
Cursor Down @V
Cursor Left @L
Cursor Right @Z
Cursor Select @A@J
Cursor Up @U
Delete @D
Dup @S@x
End @q
Enter @E
Erase EOF @F
Erase Input @A@F
Field Exit @A@E
Field Mark @S@y
Field - @A@-
Field + @A@+
Help @H
Hexadecimal @A@X
Home @0 ('@' followed by a zero)
Insert @I
Insert Toggle @A@I
Local Print @P
www.tobuku.com
Version 1.0 - 17 -
New Line @N
Page Up @u
Page Down @v
Print (PC) @A@t
Print Screen @A@T
Record Backspace @A@<
Reset @R
Shift @S
Sys Request @A@H
Tab (Right Tab) @T
Test @A@C
PA1 @x
PA2 @y
PA3 @z
PA4 @+
PA5 @%
PA6 @&
PA7 @' ('@' followed by a single quote)
PA8 @(
PA9 @)
PA10 @*
PF1/F1 @1
PF2/F2 @2
PF3/F3 @3
PF4/F4 @4
PF5/F5 @5
PF6/F6 @6
PF7/F7 @7
PF8/F8 @8
PF9/F9 @9
PF10/F10 @a
PF11/F11 @b
www.tobuku.com
Version 1.0 - 18 -
PF12/F12 @c
PF13/F13 @d
PF14/F14 @e
PF15/F15 @f
PF16/F16 @g
PF17/F17 @h
PF18/F18 @i
PF19/F19 @j
PF20/F20 @k
PF21/F21 @l
PF22/F22 @m
PF23/F23 @n
PF24/F24 @o
www.tobuku.com
Version 1.0 - 19 -
Starting the Personal Communications Program The IBM Personal Communications is the standard 5250 terminal emulator used in this article.
Before the custom application can perform the EHLLAPI function calls, the terminal emulation
program should be opened first because all the EHLLAPI function calls are done against the
program.
In a default installation, the Personal Communications program is located under the ..\Program
Files\IBM\Client Access\Emulator directory. The emulation program itself is called PCSWS.EXE.
If the terminal emulator is not started, then few lines of code need to be added within the
custom application to open the PCSWS program. Normally it can be done by a simple shell
command.
The Personal Communications program receives some parameters when it is launched. The
basic syntax to execute this program is:
PCSWS.EXE <WS_file>
The WS file is a workstation profile file that is created when the Personal Communications is run
in the first time. This workstation profile file contains the connection information like the system
name, workstation ID, etc as shown in the dialog window below. The screen should be familiar
to anyone who works with the iSeries quite often.
So when the PCSWS program is execute, it requires the WS file to connect to the host already
specified before. Besides the WS file name, there are other parameters that can be added to the
command when calling the PCSWS program.
� To suppress the IBM logo, add the parameter /Q to the command
Example:
www.tobuku.com
Version 1.0 - 20 -
..\PCSWS.EXE C:\AppData\TCPIP1.WS /Q
� To start the PCSWS program as an icon, not as a window, add the parameter /I to the
command.
Example:
..\PCSWS.EXE C:\AppData\TWINAX.WS /I
� To start a hidden PCSWS program, not as an icon or a window, add the parameter /H to
the command.
Example:
..\PCSWS.EXE C:\AppData\TWINAX.WS /H
www.tobuku.com
Version 1.0 - 21 -
REFERENCE
� Emulator Programming, Personal Communications for Windows, Version 5.9 (Document
Number SC31-8478-09). Tenth Edition, July 2006
http://publib.boulder.ibm.com/infocenter/pcomhelp/v5r9/topic/com.ibm.pcomm.doc/
books/html/emulator_programming.htm
� IBM i Access for Windows, sample programs are available to help develop IBM i Access
for Windows applications using Visual Basic
http://www-03.ibm.com/systems/i/software/access/windows/toolkit/vb.html
� Using EHLLAPI in C#, sample program by Luca Bertoncini
http://www.codeproject.com/KB/cs/all_ehllapi.aspx