agile web development with ewd - m/gateway · vista web development • browser-based gui for vista...
TRANSCRIPT
Agile Web Development with EWD
Rob TweedM/Gateway Developments Ltd
http://www.mgateway.comTwitter: @rtweed
Agile
• My interpretation:
– Faster
Agile
• My interpretation:
– Faster development
– Faster maintenance
– Faster performance
• Days and weeks, not months and years
Faster development
Agile
• My interpretation:
– Faster Development
• Days and weeks, not months and years
– Therefore cheaper
• $10k’s, not $M’s or $10M’s
Agile
• My interpretation:
– Faster maintenance
Everyone forgets Maintenance
Agile
• My interpretation:
– Faster performance
– than what?
Agile Web Development?
• Web Applications
– Applications that make use of a web browser to provide the client “front-end” interface
– Dynamically-generated pages of HTML (& Javascript, CSS)
– Controlled by user input and database content
• Amazon.com, eBay.com, etc
• Quest’s Care360
Benefits of Web Applications
• Nothing to install or upgrade on client
– Just requires a “standard” web browser
– Business logic is on a remote server
• Ubiquitous User Interface
– Browser + HTML
• Make use of Internet and its protocols to allow broad reach to users
• Now the de facto standard for applications
How do web applications work?
How do Web Applications Work?
Web Browser
Web Server
Simple “static” web page retrieval
SimpleFile Store
How do Web Applications Work?
Web Browser
Web Serverhttp
request(URL)
Simple “static” web page retrieval
SimpleFile Store
How do Web Applications Work?
Web Browser
Web Serverhttp
request(URL)
Simple “static” web page retrieval
SimpleFile Store
How do Web Applications Work?
Web Browser
Web Serverhttp
response(HTML)
Simple “static” web page retrieval
SimpleFile Store
How do Web Applications Work?
Web Browser
Web Server
Simple “static” web page retrieval
SimpleFile Store
disconnected
How do Web Applications Work?
Web Browser
Web Server
“Dynamic” web page retrieval
ScriptingEnvironment
Database
How do Web Applications Work?
Web Browser
Web Server
“Dynamic” web page retrieval
ScriptingEnvironment
Database
httprequest(URL)
How do Web Applications Work?
Web Browser
Web Server
“Dynamic” web page retrieval
Run query
Database
httprequest(URL)
How do Web Applications Work?
Web Browser
Web Server
“Dynamic” web page retrieval
ConstructPage (HTML)
Database
How do Web Applications Work?
Web Browser
Web Server
“Dynamic” web page retrieval
ConstructPage (HTML)
Database
httpresponse(HTML)
How do Web Applications Work?
Web Browser
Web Server
disconnected
“Dynamic” web page retrieval
ScriptingEnvironment
Database
Further Benefit of web applications
• Much more efficient use of network
• Much more efficient use of back-end
• Massively scalable
How do Web Applications Work?
Web Browser
Web Server
Mapping of incoming request toa corresponding query
Run query
Database
httprequest(URL)
Web Server doesn’t understand“dynamic” requests
And doesn’t remember you fromone request to the next
Forwarded by web serverdue to request structure
How do Web Applications Work?
Web Browser
Web Server
PersistentSession Data
SessionManagement
ApplicationDatabase
httprequest(URL)
State management must beprovided within the scripting
environment
How do Web Applications Work?
Web Browser
Web Server
PersistentSession Data
SessionManagement
ApplicationDatabase
Incomingrequest
How do Web Applications Work?
Web Browser
Web Server
PersistentSession Data
SessionManagement
ApplicationDatabase
Incomingrequest
?
?
?
How do Web Applications Work?
Web Browser
Web Server
PersistentSession Data
SessionManagement
ApplicationDatabase
Incomingrequest
?
?
?
• Original user back again?• Different valid user?• Someone pretending to be
the original user?
URLs are all you’ve got
Uniform Resource Locators
http://www.mgateway.com/mgw/mgw2.ewd?a=123&user=rob
transport method
target system domain name/IP Address
directory/path
“page” orResource name
name/value pairs
Web Application Frameworks
• Look after:
– Session management
– Security management
– State management
• Creating the illusion of a stateful environment on top of the stateless web architecture
Interface
Script
Mainstream Web Application Mainstream Web Application ArchitectureArchitecture
Browser
Web
ServerInternet/
Intranet
1. Request page
4. Send page
DBMSGateway SQL
2. Translate
request3. Build HTML
page
Interface
Script
egeg PHP ArchitecturePHP Architecture
Browser
Web
ServerInternet/
Intranet
1. Request page
4. Send page
DBMSGateway SQL
2. Translate
request3. Build HTML
page
Zend Engine
Interface
Script
.Net, Java (Tomcat), etc.Net, Java (Tomcat), etc
Browser
Web
ServerInternet/
Intranet
1. Request page
4. Send page
DBMSGateway SQL
2. Translate
request3. Build HTML
page
Web Middle Tier
Web-enabled VistA
• Modernise the UI
• Browser-based interface
• Get all those benefits of web applications
VistA Web Development
• Browser-based GUI for VistA
– Without any modification to VistA
• Desktop:
– As functional as CPRS without the downsides:• Nothing to install
• Nothing to update
• Mobile:
– As functional as Native Apps without the downsides:
• Cross-platform
• No App Store worries, delays or costs
How can it be done?
• Javascript frameworks
– Desktop
• ExtJS v4
• YUI, Dojo, jQuery etc
– Mobile
• Sencha Touch 2
• Dojo Mobile, jQTouch, jQuery Mobile etc
Javascript framework downsides
• Brutal learning curve
• Coding free-for-all
• Unlimited degrees of freedom
• Therefore maintenance nightmare
• Hardly a recipe for agile!
Interface
Script
Mainstream webMainstream web--enablement of enablement of VistAVistA
Browser
Web
ServerInternet/
Intranet
1. Request page
4. Send page
Vista
DatabaseGateway
Web Middle Tier
Java.Netetc
Result:
• Logic now in .Net or Java + Mumps– Skills required
– Maintenance
– Version changes
• Extra tier– Performance bottleneck
– Scalability bottleneck
– Extra complexity to maintain
– Extra cost
Result:
• Logic now in .Net or Java + Mumps– Skills required
– Maintenance
– Version changes
• Extra tier– Performance bottleneck
– Scalability bottleneck
– Extra complexity to maintain
– Extra cost
• Hardly a recipe for agile!
Agile Web Development?
Agile Web Development?
• EWD
– Web Application Development framework
– Designed to work natively with Cache and GT.M
• Ideal for legacy applications such as VistA– Not actually specifically created for VistA
Agile Web Development?
• EWD
– High performance, scalability
– High efficiency
Interface
Script
Mainstream webMainstream web--enablement of enablement of VistAVistA
Browser
Web
ServerInternet/
Intranet
1. Request page
4. Send page
Vista
DatabaseGateway
Web Middle Tier
Java.Netetc
VistA Web Enablement
• The mainstream forget or don’t realise:
– Mumps (Cache or GT.M) isn’t just a database
– Has its own integrated scripting language
– Mumps is capable of doing everything required for:
• Parsing HTTP requests
• Creating HTTP responses (HTML, XML, JSON ..)
• Session, state and security management
– Tailor-made for the web
EWD runtime Architecture
Browser
Browser
Browser
WebServer
(Apache)
m_apache
GT.M
http/https
xinetd – basedpersistent
connections
MG
WS
I
EWDRuntime
Yourscripts
Yourdata
EWDPage
routines
m_apache Architecture
Browser
Web Server
Web Server
Internet/
Intranet
interface
Apache
GT.M
Mumps
processesm_apache
GT.M Server
EWD runtime Architecture: WebLink
Browser
Browser
Browser
WebServer
(Apache/IIS)
WebLink Caché
http/https
WebLin
kin
terfa
ce
Caché
CS
P in
terfa
ce
Caché
Persistentconnections W
ebLin
kin
terfa
ce
EWDRuntime
Yourscripts
Yourdata
EWDWeblinkroutines
EWD runtime Architecture: CSP
Browser
Browser
Browser
WebServer
(Apache/IIS)
CSP Caché
http/https
CS
P in
terfa
ce
Caché
Persistentconnections
CS
P in
terfa
ce
Yourscripts
Yourdata
EWDCSP
pages
Handling stateless requests
Web Server
Apache
m_apache
1. Request arrives froma user
User
User
GT.M Server
User 1
User 2
Apache
m_apache
Web Server
User
User
GT.M Server
User 1
User 2
2. MGWSI establishesSocket and Session
Handling stateless requests
Apache
m_apache
Web Server
User
User
GT.M Server
User 1
User 2
Processing...
3. User request isprocessed on GT.M server
Handling stateless requests
Apache
m_apache
Web Server
User
User
GT.M Server
User 1
User 2
Processing...
4. Meanwhile a seconduser makes a request..
Handling stateless requests
Apache
m_apache
Web Server
User
User
GT.M Server
User 1
User 2
Processing...
5. 2nd sessionEstablished by
MGWSI
Processing...
Handling stateless requests
Web Server
User
User
GT.M Server
User 1
User 2
m_apache
Apache
6. User 2’s requestis completed.
Processing...
Write HTML
Handling stateless requests
m_apache
Apache
Web Server
User 3
User
Caché Server
User 1
User 2
Processing...
Waiting….
7. User 2’s processingstops. Connectionis maintained...
Handling stateless requests
Web Server
User 3
User
GT.M Server
User 1
User 2
Processing...
9. Session 2 isre-used for incomingRequest from User 3
Processing...
m_apache
Apache
Handling stateless requests
Web Server
User 3
User
GT.M Server
User 1
User 2
Waiting...
9. Session 1 has finishedHandling user 1's request
and is now free
Processing...
m_apache
Apache
Handling stateless requests
Web Server
User 3
User
GT.M Server
User 1
User 2
Processing...
9. 2nd request fromUser 2 is handled
by process 1
Processing...
m_apache
Apache
Handling stateless requests
• Each back-end GT.M process supports multiple browsers/users
– Perhaps 10:1 to 20:1 ratio of concurrent users to physical processes
– Highly scalable architecture
– Extremely high performance
• Unused back-end processes aren't shut down
• Process startup overhead only for the 1st request
• m_apache connects an incoming request to an existing waiting back-end process
Handling stateless requests
Beware: CSP
• When a back-end process finishes sending its response
• CSP doesn’t release it immediately to the free pool– Is available for requests from the previous user
– Isn’t available for any other requests for a period of time:
• Grace period
• Result: artificially inflates the number of processes necessary for handling HTTP traffic– Easy to run out of licenses
Agile Web Development?
• EWD
– High efficiency
• Its stateless architecture significantly reduces the resources needed to support large numbers of users
– Less memory
– Less CPU
– Far fewer processes (therefore licenses)
Compare using web apps
BrowserWeb Server
Web Server
Internet/
Intranet
interface
Apache
GT.M
Mumps
processesm_apache
GT.M Server
Browser
Browser
Browser
Browser
With current VistA usage
User
GT.M orCache
CPRS
Statefullconnections
Broker connections
Terminal
PersistentProcesses
EWD runtime Architecture
Browser
Browser
Browser
WebServer
(Apache)
m_apache
GT.M
http/https
xinetd – basedpersistent
connections
MG
WS
I
EWDRuntime
Yourscripts
Yourdata
EWDPage
routines
Architectural responsibilities
• Web Server looks after the HTTP protocol handling
• Processing the meaning of requests and creating the response is done in Mumps
Agile Web Development?
• EWD
– High performance, scalability
– Everything happens in the Mumps domain
– We know that GT.M & Cache can scale massively
– Web server does nothing other than passively pass through requests and responses
– No additional technology layers needed or wanted
J2EE architecture:
Agile Web Applications?
• EWD
– High performance, scalability
m_apache scaling
Browsers Apachehttp overTCP/IP
GT.M Server GT.M Server
Telnet overTCP/IP
Multiple servers
MGWSI
Components
Key:GT.M Server
m_apache scaling
Apache
GT.M Server GT.M Server
MGWSI
Components
Key:GT.M Server
Apache
Agile Web Development?
• EWD
– High reliability & security
– Looks after security & session management automatically behind the scenes
Reliability & Security
• Last 5 years:
– EWD supports the world's largest Internet-
facing Caché-based web application
• Quest Diagnostics Care360 EziOrder
– 30,000+ concurrent users
– 500,000 test requests/day
– 24 X 7 operation with no breakdowns or incidents
– No security breaches
Agile Web Development?
• EWD
– "If it can be automated, it is!“
– User extensible
• Custom tags
Agile development with EWD?
• Case Study, May 2012:
– 1 Clinician; 1 Developer
– Dr Tony Shannon
• Consultant in Emergency Medicine & Chief Clinical Information Officer, Leeds Teaching Hospitals, UK
• Fellowship in Informatics & MSc in IT Management
– Chris Casey
• Mumps, Cache, EWD developer
Project Objective
• Build a hospital clinical portal
– Based on clinical process patterns
– Clinically-led, user-centered design
– ie around the workflow and perspective of clinicians, and not:
• An IT technical developer's perspective
• A database expert's perspective
• Hospital management perspective
Agile focus
• 2-page high-level proposal
• Fast-paced development
• Rapid prototyping
– Build, review, change
• "storyboarding" as communication between clinician & developer
Open Source Focus
• GT.M
• ExtJS (GPL License)
• EWD + ExtJS v4 Tag Library
Clinical Focus
• Centered around key clinical processes– 5 key UI patterns
• 3 levels:– Strategic
• Overview across all patients in hospital– Different filters/stratification
– Tactical• Patients within a particular category
– Ward, department, specialty etc
– Operational• Individual patient
• Referral in, assessment, plan care, deliver care, referral out
Try it for yourself
• http://clinuip.wordpress.com/• http://ec2-184-73-102-27.compute-1.amazonaws.com/ewd/leeds/index.ewd
Total development time
• 10 days
This redefines agile!
• 1 Clinician + 1 Developer
• 10 days
• Entire Clinical Portal
– Including a database designed and built from scratch to support it!
• Orders of magnitude better than normal industry expectations
Implications
• Makes the timescales and costs associated with government procurements look ridiculous– Years of development
– Millions of dollars
• Total cost of Tony Shannon's project was less than $10,000
• 10 days
Not just sheer development speed
• Chris's application is highly maintainable– Clear and rigorous split between UI/Design
and back-end database programming
• Easily adaptable into mobile version:– EWD's Sencha Touch 2 tags are very similar
in use and syntax
– Able to share most of the back-end code
• Easily adaptable to use different database– Levels 1 & 2 grafted onto VistA in 3 days
How does EWD make this possible?
• Exploiting the best Javascript frameworks
– ExtJS, Sencha Touch etc
• But abstracting them into XML tags and integrating them with EWD Session
– Massively reduces learning curve
– Simplifies their use
– Standardises their use
• Maintainability!
Container Page & Fragments
• EWD breaks down complex UI into many small "fragments"
– Each is a discrete EWD "page"
• onBeforeRender method
• Markup or JSON
• Optional Javascript and/or "listeners"
– Fragments get injected into the container page
ExtJS tags
• Very high-level tags
– Describe a great deal of functionality in just one tag
• Data array
• Implied event (eg click a cell)
• Implied action (eg fetch another fragment)
Example
<ext4:container rootPath="/ext-4" onBeforeRender="getGridData^MyDemo" ><ext4:gridPanel title="Simpsons" frame="true" height="200" width="550“
sessionName="simpsons"><ext4:gridcolumn text="Name" width="100" sortable="false" hideable="false"
dataIndex="name" /><ext4:gridcolumn text="Email Address" width="150" dataIndex="email" /><ext4:gridcolumn text="Phone Number" flex="1" dataIndex="phone" />
</ext4:gridPanel></ext4:container>
EWD Page
Example
<ext4:container rootPath="/ext-4" onBeforeRender="getGridData^MyDemo" ><ext4:gridPanel title="Simpsons" frame="true" height="200" width="550“
sessionName="simpsons"><ext4:gridcolumn text="Name" width="100" sortable="false" hideable="false"
dataIndex="name" /><ext4:gridcolumn text="Email Address" width="150" dataIndex="email" /><ext4:gridcolumn text="Phone Number" flex="1" dataIndex="phone" />
</ext4:gridPanel></ext4:container>
OnBeforeRender Method:• Mumps function• Executed before page renders
ExamplegetGridData(sessid);n data;s data(1,"name")="Lisa"s data(1,"email")="[email protected]"s data(1,"phone")="555-111-1224"s data(2,"name")="Bart"s data(2,"email")="[email protected]"s data(2,"phone")="555-111-1234"s data(3,"name")="Homer"s data(3,"email")="[email protected]"s data(3,"phone")="555-111-1244"s data(4,"name")="Marge"s data(4,"email")="[email protected]"s data(4,"phone")="555-111-1245"d mergeArrayToSession^%zewdAPI(.data,"simpsons",sessid);QUIT ""
Example
“Mumps mindset” integration
• EWD’s integration and use of Mumps (GT.M & Cache):
– Designed by someone who understands:
• Mumps technology and mindset
• Web technologies
– Intuitive back-end coding
• Compare with Java, .Net and “mainstream” web frameworks
EWD: Design-Centric
• Design, not programming, is what makes great applications
• Clear separation of Design from Programming
• Reduce programming to trivial levels
Clear separation of presentation & programming
• EWD Session as middle ground
– Front-end decoupled from back-end
Middle Tier
• UI and Back-end are deliberately de-coupled, with "EWD Session" as middle ground
EWD“Back-end”
GT.M / Caché
db
EWDSession
Programmer
myVarPre-page
script
EWD Page
<?= #myVar ?>
Designer
setSessionValue^%zewdAPI
Page designer
EWD PageEWD
Session
Display Data
myVar<?= #myVar ?>
?
- UI Designer doesn't need back-end skills- UI can be initially designed with a simple, spoofed back-end
Programmer
EWD“Back-end”
GT.M / Caché
db
EWDSession
Fetch Data
myVarPre-page
script
setSessionValue^%zewdAPI
?
- Back-end developer doesn't need UI skills- Back-end can be changed without any impact on UI
Prototyping
• Back-end scripts initially hard-coded to generate test values for Session
• When design agreed, programmer can replace back-end scripts to use real queries, validation etc
– No change needed to front end
• Back-end logic can be completely replaced
– Just needs to maintain Session in the same way
– eg logic replaced to access VistA
Agile VistA Web Development
• Integration of EWD with VistA?
– What VistA logic to use?
– How to use that code?
Integrating EWD with VistA
• Zach Gonzales
– Oroville Hospital
Integrating EWD & VistA
• What RPCs to use?
• How to interface them with EWD?
• Agile:
– Don’t want everyone reinventing this wheel
RPC wrappering
• Creating EWD-aware APIs around core clinical RPCs
– Automatically map to EWD Session
– Just call the RPC
– Define UI to visualise the Session Data
EWD runtime Architecture
Browser
Browser
Browser
WebServer
(Apache)
m_apache
GT.M
xinetd – basedpersistent
connections
MG
WS
I
EWDRuntime
Yourscripts
Yourdata
EWDPage
routines
Accessing VistA Information
GT.M
MG
WS
I
EWDRuntime
Yourscripts
VistA
EWDPage
routines
Accessing VistA Information
GT.M
MG
WS
I
EWDRuntime
Yourscripts
VistA
EWDPage
routines
RPCs?
EWD-aware VistA APIs
RPC RPC
EWD Session
API, eg VitalsCreate / UsePredeterminedEWD Session data
Know which RPCsetc to call in VistA
RPC integration: the old waygetCurrentVitals.....n vitalList d FASTVIT^ORQQVI(.vitalList,DFN)n count s count=""f d q:'count. s count=$O(vitalsList(count)). q:'count. n vitalType s vitalType=$p($g(vitalsList(count)),U,2). n vitalDate s vitalDate=$$FMTE^XLFDT($p($g(vitalsList(count)),U,4)). n USMeasurement s USMeasurement=$p($g(vitalsList(count)),U,5). n metricMeasurement s metricMeasurement=$p($g(vitalsList(count)),U,6). n vitalMeasurement s vitalMeasurement=USMeasurement_" "_metricMeasurement. s vitals(count,"vitalMeas")=vitalType_": "_vitalMeasurement_" "_vitalDated deleteFromSession^%zewdAPI("vista_currVitals",sessid)d mergeArrayToSession^%zewdAPI(.vitals,"vista_currVitals",sessid)
….
RPC integration: the new way
getCurrentVitals(sessid)n oks ok=$$getCurrentVitals^RPCLibrary(sessid)QUIT ""
Scale of the Wrapering Task
• Approx 2,400 RPCs
Accessing VistA Information
GT.M
MG
WS
I
EWDRuntime
Yourscripts
VistA
EWDPage
routines
APIs
Agile VistA Web Development
Agile Web Development using EWD
• Faster development
• Faster maintenance
• Faster performance
No excuses for not using EWD
• Free, Open Source for GT.M
• Free for Caché
• Cross-compilation
• Applications work on both Caché & GT.M
• Easy cross development on mobile and desktop
EWD Simplicity
• No need for extra layers or additional technologies
• Fastest, most efficient, most scalable architecture you can use
– Nothing else will come close
Agile Development
• Orders of magnitude faster than anything else
• EWD redefines agility
Agile Web Development with EWD
Rob TweedM/Gateway Developments Ltd
http://www.mgateway.comTwitter: @rtweed