uPortal developers meeting Cornell University

August 7-8, 2003 [a Thursday and Friday]Attendees

Dan Ellentuck -- Columbia U.Alex Vigdor -- Columbia U.Stephen Barrett -- Cornell U.John Fereira -- Cornell U.Aaron Hamid -- Cornell U.Dave Koehler -- Cornell U. -- Thursday morning onlyRich Marisa -- Cornell U. -- Friday afternoon onlyMichael Oltz -- Cornell U.Peter Kharchenko -- IBS/UniconKen Weiner -- IBS/UniconKevin Gary -- Unicon/IBS, architect of AcademusJon Allen -- Instructional Media and Magic (IM&M)Justin Tilton -- IM&MJim Farmer -- JA-Sig/IM&M (project 'clerk' or 'administrator')Shoji Kajita-- Nagoya University, Associate Professor; and CEO of his own company which is the main distributor of WebCT in Japan.Mark Boyd -- SCTJan Nielsen -- SCTMike Zackrison-- SCTChuck Severance -- U Michigan -- Friday only

These notes were taken and edited by Michael Oltz (mdo1@cornell.edu) and corrected with input from the presenters and other attendees. Most of the links for companies, organizations, and standards are provided only once, usually at the first appearance of the name. A few are provided multiple times. If no link is given where you notice a word, go to the beginning of the document and search for it.

Decisions and action items are indicated by non-link underlines.

Thursday August 7

Ken -- introductions, agenda changes

The goal of this meeting is to nail down the completion of 2.2 -- one conversation will be how to complete what we are working on, another to discuss future features.

Michael Oltz, Cornell University 1 Revised 15 September 2003

JA-Sig board perspective, by Dave Koehler -- he is director of Business Information Systems at Cornell, and is on JA-Sig Steering Committee (often called the "JA-Sig board")

JA-Sig started in 1999 [java In Administration] to cooperate developing academic administration software in Java. Mostly to learn from each other, and reuse code. The JA-Sig Clearinghouse is to exchange miscellaneous free code and other things. The board intends to increase promotion and use of Clearinghouse. (a) Share more than just code, e.g. channels (b) get at it through uPortal. At the first JA-Sig meeting, it was asked, what should JA-Sig collaborate on? All the schools were looking at portals. So that's what they decided. After that started, a number of 'lurker' schools said, let the rest of the world use it too, not just for the schools that write it. JA-Sig went to Mellon Foundation for money to support broadening what kinds of schools could make use of it. They required that uPortal be open source.

The implementation schools are learning a lot from the project. Bernie Gleason (of Boston College) said that the total effort had a value of $3 million, but the results are given away free. Java and portals are not the only things that need coordination; so now JA-Sig has the reputation of not just talking but doing something. Colleges are asking JA-Sig to give guidance re other open source ideas, what to use, how to connect it together, choosing standards. For example JA-Sig is trying to help start up an effort of open source financials [in conjuction with National Association of College and University Business Officers NACUBO] including a vendor and several colleges; Mellon Foundation is not as interested in funding this one.

Dave mentioned 2 articles in Chronicle of Higher Education August 1, 2003 about open source in general, and about effects on higher education of Oracle's takeover of PeopleSoft.

He also mentioned a book "The Cathedral and the Bazaar" -- how open source model works and why it works. [The Cathedral and the Bazaar: Musings on Linux and Open Source by an Accidental Revolutionary revised edition, by Eric S. Raymond, O'Reilly, 2001]

The first production site: University of British Columbia (hereinafter "UBC") went live with uPortal in September, 2000. 30,000 students. Live only with freshmen at first.

Mellon funding is running out. Unicon has offered to continue to support Ken and Peter at least through the end of 2003.

Some colleges are comparing raw uPortal with Luminis (formerly called Campus Pipeline), [to decide] which to go with. JA-Sig is pleased that companies are embedding uPortal.

Michael Oltz, Cornell University 2 Revised 15 September 2003

[next JA-Sig conference is in Miami Beach this winter]

Jim question: Dave, what do you consider to be the impact of the Oracle/PeopleSoft merger?

Dave: If we don't bury our heads in the sand, the way we have chosen is not necessarily the way that will sustain value. Yale Princeton Columbia etc. will be in existence 50 to 100 years from now to support our environment, corporations may not or may be doing different kinds of things. Our handshake to others is an open hand, palm up -- we want it for free. Our cycle is the gestation period of an elephant, it takes a long time to decide what to do. Colleges tend to heavily customize canned products and that means we have to support it ourselves. In the long run, the Oracle/PeopleSoft will at least be a wakeup call to consider strategy. Nothing against the two companies as products, but what about total cost and fit to the problem. Dave's original take was it was a brilliant marketing ploy for Oracle, not that Oracle really wanted it. Now it has evolved into a conflict between the CEO's. They're presently waiting for the anti-trust ruling. But even if they do merge they would still be number 2 to SAP.

Carl Jacobsen (at University of Delaware, on JA-Sig Steering Committee) attended NACUBO conference.

Ken: What is the board's take on the portlet (JSR-168) spec. Does the board expect uPortal to be compliant?

Dave: I'm not the technical member of the board. We have not had a specific conversation about it. If you want board approval or consideration, write a one-page executive overview.

Ken: Will it be a problem if a lot of portlet-compliant containers come out and uPortal is not one of them?

Dave: We want open-standards, generally accepted things.

Jim: there is WSRP, Portlet, and Jetspeed's Velocity. Will a Sun Java portlet spec emerge as a strong business force? Dave, what is JA-Sig board's relation to Sun?

Dave: I don't think I can answer Ken's question. The developers have made good decisions in the past. Ken, please keep telling us what technical issues we should consider important and evaluate.

Sun has been our benevolent benefactor so far. We are more interested in their architects/luminaries than in marketing types. They have contributed something toward conferences. Sun can communicate with this kind of market without JA-

Michael Oltz, Cornell University 3 Revised 15 September 2003

sig, but not as well. Sun was concerned about the percentage of sites running on Sun boxes, but the issue is behind us.

Sun is not as eager to support the Clearinghouse as the other things. Dave thinks we have an influence on what Sun does.

Ken: What about Jim Farmer's role? What will happen when the funding runs out?

Dave: I don't know exactly how much is left in the pot. Jim is careful about spending. What Jim does is 'virtual glue' to other organizations. They are looking for some other source to continue this.

By the way, Columbia is trying to get a Mellon grant for CUCMS (Columbia University Content Management Suite, information and download available from the JA-Sig Clearinghouse).

Alex: That grant application is currently sort of in limbo.

Dave: Mellon (especially Ira Fuchs) is more interested in funding matching-fund grants now. The in-kind contribution of developers is not as important an influence on them now as it was.

Ken: is it that we don't want funding from them or that we can't?

Dave: We would like to get the money but the bar is now higher. They won't give money for 'evangelism' they want development to happen from the money and they want to see a plan for after-grant continuation of the results.

Jim: One bad thing is, if Mellon hadn't been so positive to begin with, the board might have taken another direction back then and there would not be quite this problem.

Kevin: What is the board's metric of success re uPortal?

Dave: a) It's a great product in and of itself.b) Continued exploration of the context of delivering products and services to the community. E.g. the content management project. SCT's learning management system. Connection to other standards groups and so on. A lot of serendipitous discovery is happening.

JA-Sig is now kind of an aggregating group for higher ed.

HEKATE is a new higher education organization whose focus is on web services and on using the web to facilitate education.

Michael Oltz, Cornell University 4 Revised 15 September 2003

So far you developers and uPortal in general have 'bet on the right horses'.

Jim: There will be questions based on the open source article. Jim summarizing what Dave said:

1) Uportal -- JA-Sig board expects project to continue more or less in the same form.2) Board now strongly supports open standards.3) Board has a broad application interest.4) Board satisfied with architectural decisions.5) Clearinghouse is important as a service to the community.

Common Solutions Group has not actually come out with any specific solutions.UPortal has. Dave congratulates the uPortal development group; the product continues to "wow".

Jim: What about corporate interest?

Dave: We don't want uPortal to fork into a corporate vs. a higher ed version. We don't want to lose control. We're wary. Companies expect high-class service.Potential problems: (a) if corporations invest bunches of money, the H.E. community won't get as much attention.(b) what if corporations hook in vertical-market standards or proprietary things, and we can't get the code? E.g. ".NET" ??? That's how the forking would happen. "They're bigger elephants than we're used to." Second session

Jim: Transitional comments: The reason for all the questions at the end of Dave's talk was, we wanted you to have authoritative answers; "When Dave talked to us, this is what he said" a) What is happening to the uPortal project? b) why are they doing other things?

Jim on CHEF: What is the "there" there? The CHEF project plans to implement their own 13 services there, which is more academically relevant than raw OKI. This will not be incorporated in the framework but will be available to uPortal community. [CHEF is software for group collaboration developed at U Michigan. Presentation on it on Friday, see later]

Ken on Portlets (JSR-168) and WSRP [the second session]

[JSR-168 is a specification from the Java Community Process:http://www.jcp.org/en/jsr/detail?id=168for an API for how to do what JA-Sig calls a "channel". WSRP is a spec from Oasis:

Michael Oltz, Cornell University 5 Revised 15 September 2003

http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=wsrpit defines how a client can aggregate application data from a remote server via web services. It would replace or parallel Ken's "remote channels" feature.] a) If we took the spec seriously, could we do it in uPortal 3.0?b) Would we have to cut features?

Jan Nielsen thinks there's a great advantage, "we can't not do it". All the other horses will overtake you if you don't do it. And we will be abandoned. SCT really wants this. He's not so comfortable with WSRP, not as much confidence. Jan expects the Java development community to run with JSR-168 "like a wild pack of dogs". [because it is from the Java community per se.]

Ken: WSRP is an enabler -- it allows things to communicate with each other once they have been written. [it is for remote channels] JSR-168 is a container specification really. Do we write a wrapper for 168 on top of uPortal? Or rewrite uPortal to 168 and make a wrapper for IChannel? Rewriting uPortal would take about a year.

Kevin: Also get involved in the standards process for 168 version 2.0.

Peter: A technical detail, the spec says that portlets must be loaded with the same class loader as the framework. This is problematic for our implementation of CAR files (channel archive files, like a jar).

Aaron: Did you say features would be cut?

Ken: Some of the features we've done may not make sense in 168.

Steve: What do people think is the timeframe of adoption and rollout?

Ken: As a first step write WSRP and try to write an adapter.

SCT says the industry will adopt JSR-168 rapidly.

Steve: We need an analysis of how much it needs to be altered to do it.

Ken: Gut feeling -- a lot.

Mark: -- It will be easier to rewrite uPortal as a portlet container than to write an adapter.

Ken: Adopting another JSR-168 portal framework in place of uPortal is technically one of our options.

Alex: What we're better at is value-added stuff, e.g. groups and permissions.Jan: It would take 4 people for 6 months [a deliberately casual estimate]

Michael Oltz, Cornell University 6 Revised 15 September 2003

Peter: The rendering is not going to be so difficult, it's the authentication, handling users, etc.

Ken: I am not so concerned about those issues. We already have mechanisms, just make the 168 API call that. I agree with their estimates. I compare it with what we did in going to 2.0 rewriting the internals. But it will be easier to assign work this time around. And there are compatibility tests available for 168. The difficult part will be morphing legacy code. Two person years. (4 people for 6 months).

Ken: If we don't implement JSR-168, we'd be making a one-off portal that isn't standards compliant.

Jan: Maybe JA-Sig should concentrate on integration rather than the framework.

Jim: In terms of adoption, Plumtree is first, Vignette is second, and uPortal is third largest enterprise portal in USA.

Steve: We should present several options to the board, and the first option is to get involved in the spec.

Mike Z: I suggested a year ago we get connected to the 168 group. But at the time the uPortal group was focused on remote channels/WSRP.

Jim: I recommended to the JA-Sig board some time ago that they not get involved in standards, because it takes a half- or full-time person and a lot of travel money. Costs about $150,000 a year. UPortal has an opportunity to be commodity implementation because of so many sites.

Jim's suggestion: (a) Ken and I will write up pros and cons of JSR-168, (b) go to developer list for comments, (c) go to our commercial partners and ask how important it is, (d) present the document to the JA-Sig board. He thinks that Gartner and their competitors are hesitant to make a call about 168.

Jim: SAP is now selling their product with MySQL db instead of Oracle. Oracle is losing market share.

Dan: In reference to Ken's motivation. I think uPortal got the extent of adoption that it did because it was trying to be a creditable implementation for higher education, not by trying to compete with anyone. I don't think we should see ourselves as competitors to IBM and Sun.

Mark Boyd: I hear such comparisons often because uPortal is being used so much. The competition is in the eye of the beholder.

Michael Oltz, Cornell University 7 Revised 15 September 2003

Kevin: I think uPortal's main competitor is Jetspeed. I think uPortal won by being advanced re XML, and by being by and for academia.

Steve: Doing things the higher ed way, other vendors have to be torqued in.

Mike Z: Within a year, anybody's shopping list for portals will have a checkmark for 168 compliance.

Kevin: The market will become 'buy a portal container', and 'buy portlets'.

Jim: Apache is really good about getting their products out. Except for Jetspeed.

Jim: If we do my suggestion, I don't think the board will discuss the technical aspects of it.

"Do we use the word 'portlet'?"

Ken: If we incrementally implement JSR-168, gradually, we would have hybrid channels and people would start calling them portlets. Ken does not want to go down that road.

Jan: We need to find out whether Jetspeed will do 168 and how soon.

Mark: Join efforts with Jetspeed????

Jim: Indiana is the only university that strongly supported Jetspeed. Michigan followed them. Now Michigan wants to go to uPortal. Possibly Indiana as well. I will continue to say "channel".

Ken: Until we get feedback from the board, steady as we go on development, don't start this yet.

Peter: Our options will be limited by what resources are available. Could we get more programmers?

Jim: Two options (a) business partners may contribute time, (b) go to Mellon's competitors.

Update on WSRP (Ken)

Ken: talked to people on WSRP team at Java One. Ken used Axis instead of Java XML reference implementation. He was able to generate from the WSDL (analogous to IDL for RPC). Now he has to write the client and the server part. He got started with it, got pulled away by other things, but he will resume this month (August). At first he thought just write a consumer, but now he thinks will also write a producer. It might not be perfectly compliant for 2.2 but it will

Michael Oltz, Cornell University 8 Revised 15 September 2003

basically work, and we can clean it up for the dot release. The way you use it will be just like remote channels. WSRP completely ignores user authentication. What we can do is if the channel uses CacheCredentials we can send the credentials along; this wouldn't necessarily be secure, but it's an option.

Steve: We (Cornell) are using authentication in the http header, so that takes it out of the standard for now, until the standard addresses the issue.

Ken: I don't know how we will do this for sites in general.

Jim: When would a channel developer use WSRP?

Ken: A developer wouldn't use it, the choice is up to the site admin: "where am I going to get my content from?"

Ken: Apache has a project Charon, which is based on code written at IBM Boeblingen. an WSRP compliance project.

Ken: One thing where WSRP is useful is you can load balance, move the cycles to another server. Or, share the information through a variety of servers.

Thursday afternoon session

CARs -- Mark Boyd

A CAR is a way to distribute a uPortal channel in a single file, similar to a JAR. All the class files, media, stylesheets, and other resources are put into the file, and it is loaded from uPortal.

Mark will evaluate rewriting in order to take the Portlet (JSR-168) spec (the requirement about class loaders) into consideration.

JSR168 indicates that all portlets should be loaded with the web container's class loader. Therefore he will see if the classpath can be modified to find jars in the cars directory to help segregate channel archives from third party libraries. Their names would also have to be changed from ".car" to ".jar" if we use the container's class loader. This would not permit dropping in new CARs without restarting the server. You have to restart the container to see new jars. He will evaluate gonig back to using the container's class loader. With this in mind new CARs would not be recognized until the container was stopped and restarted eliminating new feature item 4 below. This approach would also eliminate use of third party libraries included within a CAR. On the positive side item 2 would be supported automatically.

Michael Oltz, Cornell University 9 Revised 15 September 2003

New features1)* Documentation2)* Services and channel types deployable from CAR3)* Support of more browser accessible file types4) Automatic recognition of new* or updated CARs [see sentences above]5) Replacement of CarResourceWorker [this would still be feasible with jars, referring to e.g. loading images]6) Semi-automatic and automatic publication.7) Third party library content.* Available in 2.2

A separate class loader was used to provide class loading from a different directory beyond those defined by the servlet spec for webapps and to allow class containers to have the file extension of ".car".

Also related is the use of browser accessible content distributed as part of the channel in a CAR. Images can be served out of the CAR by use of the CarResourceWorker. This will be revisited to see how much of a performance gain can be had if such resources were extracted from the CAR and placed into a web server area. However, such resources would have to be identified in some way for the infrastructure to be able to pull them out. We have discussed various ways of doing this via either defining an internal structure on the CARs akin to that of the servlet's spec or by specifying files via a deployment descriptor. If the CarClassLoader goes away then an internal structure approach could not be used. Item 3 would then be solely dependent on the web server.

A problem: What if the name of categories and groups in a deployment descriptor does not match what's on the target portal instance? He will be looking at ways to allow the admins to resolve this issue including possibly allowing for partially published channels that could be viewed in the current channel manager and category and groups values could then be selected to finalize the publishing.Kevin: You could have a repository of CAR files somewhere and just point at that, not have to put on the specific server.

Mark: SCT uses CARs as their standard way of packaging channels. Since action to implement JSR168 support is pending directions from the jasig board we will keep the status quo for now but investigate those items not affected directly like semi/autopublishing and deployment descriptor defined web-visible files extraction.

Ken: If we are going to make October we should wrap up checkins by mid-September.

Michael Oltz, Cornell University 10 Revised 15 September 2003

Groups and Permissions -- Dan Ellentuck

Additions for 2.2

Groups:New filesystem group service org.jasig.portal.groups.filesystemLook at this file in the MAIN branch:website/implementors/services/filesystemGroupService_tutorial.htmlleaves are files which contain lists of principals or refer to other groups. The benefit to Dan is that it is so easy to use this model, a good introduction for novices to how groups work. After the file has been read in, the timestamp is checked to notice if it has changed. And it's good as a debugging tool.Mark: How might a Group Manager be improved to be able to handle such a need?Steve: I have a channel that lets authorized people to download lists of users, edit it, and upload it again.Alex: We are thinking of using CuCMS (Columbia University Content Management System) to manage the files.

Authorization:Some Columbia departments had their own sources of permission information and wanted to be able to connect it directly to the portal. Dan decided that theinformation requested to be imported was proprietary and all they would be doing is rewriting various rule engines. Instead, you would ask the remote server, doesthe user have this permission? And it would just say yes or no, possibly with anexpiration time.Model a Permission Request and Permission Response

Request captures a question about an Owner, Principal, Activity and Target.Response captures the answer at a specific point in time. May be cached, e.g. for certain Sources or Principals.

Authorization Source Implements simple request-response protocolLets portal use already-existing authorization service as a clientEach will have a factory and you define it in an XML file.

Portal Authorization Service has a Configuration with multiple Sources and a default combining Policy. This will have a factory and you define it in an XML file (this will be an enhancement of the existing authorization service).

Michael Oltz, Cornell University 11 Revised 15 September 2003

[The NISO (National Information Standards Organization) standard was mentioned as the basis for roles.]1

Jim: Examples -- library systems have a mechanism to find out roles (it happens to use SOAP).

Steve: The concept of authorization 'policy' is just beginning to be understood, a 'yes/no' answer is not sufficient.

Jim: When you get a channel that can depend on permissions more than "yes/no" -- well, roles are actually Group memberships.

Current uPortal Permission model

A Principal performs an Activity on a Target within the context provided by an Owner. Only the principal is multi-valued (multiple parts of each value) and gives a link into the groups system.

Could Activities and Targets also be groups? So you could assign permissions en masse? The reason to do this is to reduce the work for the administrator.

Aaron: Instead of making the uPortal authorization more powerful, why not put an API over it so it can be ripped out later and replaced? In which case, you would pass in an Object and get back an Object.

Motivations: (a) Requests came in over the last year for a native way to represent complex roles. (b) Dan read the OKI materials and saw their use of "types". Dan is suggesting this as a future idea. "It feels like there's very little support for it here."

Jim: Not necessarily.

Jim: As an aside www.osafoundation.org [Common Solutions Group + Mellon giving Mitch Kapor money to make PIMs for higher ed]

OASIS

1 The original NISO Circulation Interchange Protocol (NICP) documents defined standard roles for library users. The early drafts had a definition for each broad class of users. Subsequently “role” was changed to “User Privilege” and the definition was based on the type of library. The April 16, 2001 Twelfth Draft, Appendix A Enumerated Types specified Administration, Faculty, Graduate, Postdoctoral, Staff, and Undergraduate. Local definitions would be used. These were expected to align with classes of privileges. The subsequent ANSI/NISO Z39.83-2002 Circulation Interchange Part 1: Protocol (NCIP) published October 17, 2002, Appendix C Enumerated Data Types, listed only five for AgencyUserPrivilegeType: Faculty, Graduate, Postdoctoral, Staff, and Undergraduate. For the Public Scheme, the values are Adult, Child, Senior, Staff, and Young Adult (case sensitive as written here). [Footnote added by Jim Farmer].

Michael Oltz, Cornell University 12 Revised 15 September 2003

Extensible Access Control Markup Language ("XACML")

A subject performs an action on a resource. This is captured by a target.The logical unit of authorization is the Rule. Rules are aggregated into policies and policysets.java implementations:http://sunxacml.sourceforge.nethttp://www.jiffysoftware.comDocumentation:http://www.oasis-open.org/committees/tc_home.php?wg_abbrev=xacml

System roles:Policy administration point -- auth enginePolicy decision point -- rules enginePolicy enforcement point -- auth clientPolicy information point -- auth store

OKI

OKI Open Service Interface Definition ("OSID") for Authorization:An Agent performs a Function within a Qualifier. This is captured by an Authorization.

Agent may be individual or group

Qualifier is context and may be nestedthe Agent Function and Qualifier each contain a Type, a nested category meant to model an organization.

AuthorizationManager reads and writes Authorizations and answers authorization requests for a given Agent, Function and Qualifier.

One way to work with OKI is to write an external OKI adapter, and interface to the Authorization Service. Or, we could wrap the current AuthorizationService with OKI and therefore make it bidirectional.

Jim:1) Some people are skeptical because there's not much 'there' there.2) Mellon doesn't know what to do regarding OKI. However, many parties are saying that said parties plan to rush to do it.3) Michigan plans to implement OKI by October.

Kevin: Why are they reinventing the wheel, when are they going to actually do something, and it doesn't seem to be very e-learning oriented.

Michael Oltz, Cornell University 13 Revised 15 September 2003

Peter: it seems that MIT had some problems they wanted to solve on their own and wanted to do it in a general way.

Kevin: Why is CHEF [see session below] talking to OKI?

Jim: Mellon funded OKI, and Mellon is funding Michigan [their model is collaborative] Stanford's model is assessment. Mellon funded U Washington to do PubCookie but everybody else is using Yale's Central Authentication Service (CAS). He feels XACML seems to have the least momentum behind it of all the role representations. SAP has released all its interfaces, and PeopleSoft is coming along, so these don't seem to be such a big deal. [side note: Aggregated layouts relies heavily on groups and permissions and is not very fast]

Jim: It seems that the external authorization need is more of a channel concern rather than a framework concern. the local groups and permissions model was written more for the framework.

Internationalization: 2 vs 3 letter codes, Ken

How does the locale get represented? the default 2 letter codes or 3 letter? 2 letters leave out many countries and languages. is there a way to use the existing object and squeeze 3 letters in it?

Ken: make a custom one that can return a java.util.Locale object on request.

Jan: maybe we can use the variant field of the default Locale object to hold the 3-letter part. This is better because we can use the object with standard resource bundles.

Ken: but we can still do it with our custom class by letting it return a default Locale object whenever we need it. if we use 3-letter as a variant, what would we pick as the 2-letter base code?

Jan: The variant is a vendor-specific field. so we can choose whatever we like for the 2-letter code?

Jim: Let's write a letter to the Java community to recommend that they use 3-letter standards because libraries started to use it. Also write to WSRP and JSR-168. Also to the U.S. government Office of Management and Budget (OMB) to recommend that federal procurement require this. Drop the 3-letter requirement for uPortal for now and wait for the responses. jim writes, vetted by developers, vetted by the board, go out.

Michael Oltz, Cornell University 14 Revised 15 September 2003

The group agreed to move forward with 2 letter code support for uPortal 2.2. This allows us to use the standard java.util.Locale object throughout uPortal.

XLIFF: (by Justin)

[a sort of IDL for making files in N languages] you make a skeleton, then translate the text into other languages. Originally Justin and Peter were using the original English string as the translation unit identifier but that will not work in XLIFF 1.1 where it has to be a single token. Peter is writing a Swing tool that makes skeletons and allows you to fill them in.

Peter says the editor won't be ready for 2.2 but suggests that there is some preparatory work that might fit. Make a helper class that reads the list of locales then read [couldn't type fast enough]

There was some discussion of implementation details of this idea.

Friday, August 08, 2003

uPortal and CHEF by Chuck Severance

The Comprehensive collaborativE Framework (CHEF) initiative is a Jetspeed-based application development framework from the University of Michigan. The CHEF project would like their tools to work both with Jetspeed and with uPortal."We're kind of a bad portal." -- The personalization is very limited. We're very interested in building a lot of tools. it doesn't LOOK like Jetspeed.Course Management System -- Course tools"Work Tools" -- Research collaboration system

It's more like an aggregated layout rather than user personalizable.U Mich created "My UMich" based on Apple WebObjects. But a new manager at U. Michigan decided not to continue that project due to financial considerations.The portal engine is Jetspeed/Velocity/CHEF. "Teamlets" are what they call the 'channels'.

There are these things called "Services" -- various API's to do various common things like DB connections.

They have a concept like uPortal servants.

CHEF is more an application rather than a portal. They emphasize the tools behind it rather than the framework.

Jetspeed will be evolving based on WSRP and JSR-168.

Michael Oltz, Cornell University 15 Revised 15 September 2003

UMich wants to take advantage of various uportal thrusts:-- aggregated content-- XML at the core-- WSRP and JSR-168 implementations

Want to add these things to what JA-Sig is doing-- Notion of services-- Super-duper-channel-- CAR and then Super-CARCHEF implemented their services before OKI appeared.

OKI has a concept of 'API's' and 'Factories'; it's just interfaces.

Chuck considers a service to be the actual implementation, rather than the API.

What is a Service?-- Long-term lifecycle-- One instance-- Must be aware that multiple users can use service-- Can use memory resident information-- Pluggable interfaces

Each service has a 'cover' (a design pattern) for how to instantify it.

UMich uses Jakarta Turbine to find and load services.

Super-CAR file (he wants something like this)

Standardization for interchange of two types of packaged components-- User Interface package is one kind-- Service package which describes the broker for getting at the service.[Essentially he envisions the portal as being assembled from pluggable pieces.]He wants there to be an "UI" layer so that a channel could go either to a portal or run standalone.

CHEF 1.1 just released [July? 2003], there is still another year on the project.Secret Plan for Total World Domination:-- uPortal and CHEF steal best practice from each other

Ken: I don't think uPortal and CHEF are as similar as you mentioned

Ken: Is Jetspeed really moving toward 168?Chuck: Those Jetspeed people have been doing something in secret. it seems to be 168 related but they are not talking about it.

Michael Oltz, Cornell University 16 Revised 15 September 2003

Jim: Regarding services, when and how with respect to OKI?

Chuck: Someone was supposed to be working on OKI but they were pulled off to work on 1.1 but from now on we will have 1.0 FTE. it will probably take a year for their one person to get an OKI reference implementation done.

Jim: What is the priority of the 13 CHEF services you have? Would they be available to JA-Sig developers? How would you expect us to use them?

Chuck: They are done, and yes. For example the user directory service, he explained how to do it -- it would require use of Turbine.

Aaron: This seems to look like Web Services?

Chuck: No it looks like Turbine. Turbine is pretty much a service broker. [Ken thought it had a lot more in it.]

Aaron: Could you layer Turbine on a Web Service?

Chuck: Absolutely.

Jan: You seem to want a broker for {brokers such as CORBA or EJB's}. I (Jan) wouldn't find EJB's attractive in uPortal context.

Chuck: a) "My first desire is to create a market of people writing services."b) My second desire is to get uPortal to use our services.

Jan sees the lifecycle aspect of CHEF as key to the design and most useful to others.

"Chuck's OKI periodical table thing." A way to find and connect to all the modules of a web thing.

Dan: a) What do you expect to get out of other places using your stuff?b) What do you get out of being an OKI implementer?

Chuck: (a) Our provost is willing to invest large-scale in development at UMich only if we collaborate with others.(b) Michigan has received more than a million dollars in grants related to its CHEF effort.If something doesn't have OKI it's going to be harder to sell.OKI is becoming a checkoff. (A question that people ask before adopting something)

Jim: When will the point come that this can be used by for example UPortfolio?

Michael Oltz, Cornell University 17 Revised 15 September 2003

Chuck: Right away.

Jim: Jeff Merriman at OKI said in Italy "Web Services are dead".

Chuck: What he really means is Web Services are not the only thing on this planet. But he has to say it dramatically to get other people to think out of the box.

Chuck: The concept of a channel or a teamlet needs to be broadened.Ken: If Jetspeed is doing 168, do you think we should do the same thing, or combine efforts?

Chuck: Been thinking about that. Look at what the true value of uPortal and CHEF is. The aggregated layout aspect, which I don't think 168 handles well. Should uPortal be a competitor to Jetspeed? I (Chuck) will not write another 168. I won't invest the year of my 8 or 9 programmers to do 168.Ken: If we didn't do 168, would you still adopt uPortal anyway based on functionality?

Chuck: If Jetspeed had 168 but no aggregated content, and uportal had no 168 but it did have really cool aggregated layout, I would pick uPortal. What I really want is 168, aggregated layout, and CHEF tools working in it, with OKI. Should you merge? Well there are certain predispositions in Jetspeed that may make that painful. But if you jump into Jetspeed, don't lose the unique things that make uPortal what it is.

[Later in the morning, Chuck Severance arranged a conference call with one of his staff so we could ask questions.]

Glenn Golden, chief architect of CHEF, one of the Jetspeed committers, on a speakerphone

"Jetspeed 2" project is probably JSR-168 related.

Project Pluto is a Jakarta project that will be the reference implementation of JSR-168. Thomas Sheck(?)'s group [was chair of WSRP committee] at IBM Germany Stuttgart(?) is writing it and contributing it to Jakarta as soon as the voting process on JSR-168 is finished. It is a portlet container, not the other aspects of Jakarta. [there seems to be an email that leaked out]. Pluto would implement the API's and the lifecycles.

Chuck: uPortal group was thinking it would take 2 person years to do JSR-168. Will Jetspeed be able to put that much into it?

Michael Oltz, Cornell University 18 Revised 15 September 2003

Glenn: IBM has probably known for a year or more what 168 would look like. I think that the Pluto code will actually come out. I think Jetspeed will go in this direction.

Chuck: Will Jetspeed's Velocity survive?

Glenn: I think so, it doesn't overlap what JSR-168 specifies.[end of call]

Ken: Let's wait for Pluto and see if we can integrate it.[search for 'plutoproposal' on google] http://nagoya.apache.org/wiki/apachewiki.cgi?PlutoProposal

Internationalization, Shoji Kajita

Here is an information page. [See Appendix A].

To-do list:1. User interface changes2. Database changes3. Resource translations4. Code changes5. Documentation6. what next

1. UI Changesto let users select the language to be rendered in subscribing channels. Some of this will be in 2.2.

2. Database changesOne additional column in UP_LAYOUT_STRUCT table (LAYOUT)Eight new tablesUP_CHANNEL_MDATAUP_CHAN_TYPE_MDATAUP_LAYOUT_STRUCT_MDATAUP_GROUP_MDATA *UP_PERSON_DIR_MDATA *UP_USER_LAYOUT_MDATA *UP_USER_PROFILE_MDATA *UP_USER_MDATA **: added after meeting

3. Resource TranslationsIt was pointed out that there need to be more changes to the database.Metadata is also needed for UP_USER_LAYOUT(?) and for the UP_USER_PROFILE

Michael Oltz, Cornell University 19 Revised 15 September 2003

4. LocaleManager classOne instance of this class for each userThere is a locale choice associated with:(SEE ALSO DISCUSSION BELOW)

user profileuser layoutchannel definition [provided by developer, may be modified when published at a particular site]layout struct level [subscribe time]HTTP session parameterbrowser (i.e. ACCEPT_LANGUAGE)uPortal default locale from propertiesJVM default locale setting

(if a given listed setting is null, fall down the table to the next setting, etc.)

should there be a separate class for GUEST user? Peter pointed out that the logic would be different.

Ken: In properties, we specify all the languages that could possibly be chosen for the framework. But channels can specify they have other languages, so you can add another language. The user profile level will for the time being be the same as the properties.

At the user layout level there will be a Language Preference channel, to specify preferred languages in order, from the list provided in the properties.

A suggestion to be able to specify language at fragment level (tab, column), was rejected because there is not enough time to implement it.

Peter said that the languages that a particular channel can support is known by the channel developer and should be in the deployment descriptor (see the CAR concept). Provided at prepublication. when loaded into a particular site, the site admin should be able to modify it.

At the publication time, show the site admin what the site default is, and let them copy in to the specific channel's list if they want. Or they can leave it blank.At subscribe, if there's zero or one language the channel can display, then the user should not be given the ability to choose a language for that channel.

There will be a language preference connected per user also (there will be a UP_USER_MDATA table)

What is the priority of the locale, as looked at by channel when deciding how to render itself: This is the order in which the various locale settings will be put into the array that is sent to the locale resolver.

Michael Oltz, Cornell University 20 Revised 15 September 2003

subscribe time setting for a channel [could be null]publish time setting for a channel [could be null]user layout preferencebrowser specificationportal wide (.properties) settingJava locale default

for a guest user, they would have an empty locale preference, and use the same logic.The resolver will be an interface with a default implementation, chosen in portal.properties.

Alex volunteered to write the resolver part. or at least start it.

Ken: Who will make the database changes, they have to be done first? Nobody will volunteer to do it. It has to be done before October. There was a lengthy silence.

Peter: "I can be in charge of finding somebody to do it." (laughter) The problem isn't merely to do it, but to test it against many varieties of database. Peter will find someone.

Steve: We should abstract the database accesses.

Dan: Irrespective of how we do database access, we need some unit tests, which would make it easier to make changes.

Aaron: Could we use Hibernate? A generic persistence layer, database adapters etc. behind it.

Ken: Shoji will make the data.xml and tables.xml file changes and send them to Ken. And the developers will introduce one table at a time into the 2.2 codebase.

Shoji: A channel may want to know who specified a particular Locale in the input array.

Other people disagreed at first.

Dan: A particular channel may find it wants to override its locale setting.

Alex: A particular channel may want to override its resolver.

Ken: The need for such a thing will be rare, so there should be some way to do this but not in the regular channel interface.

Peter: in RuntimeData there is no reference back to the portal.

Michael Oltz, Cornell University 21 Revised 15 September 2003

Ken: What if we can choose a custom resolver at publish time?

A discussion ensued about how to address this concern. Some developers thought the array of Locales should not be in the RuntimeData or StaticData, as it's kind of a violation of the insulation of a channel from the framework.Ken: The channel should talk to the framework to find out this information. Why not use IPrivileged?Peter: No, the purpose of IPrivileged is for changing things about the framework, like the Preferences channel. People have been misusing it to access container information e.g. HTTPRequest which was not its original intent.Ken made a command decision to end the discussion, and proposed that there will be a utility method somewhere to be called to do the resolving. Others said that it would not work that way, it can't get at enough data to make the decision.[the break for lunch occurred at this time]

Ken: The official word for 2.2 is that channels do not know the source of the locale array entries. As a workaround use IPrivileged.

Ken: Shoji checked in a class called LocaleAwareXSLT but Ken is worried that it will give an impression that this is the enduring official way to do it. The XLIFF group may change it all around in the future. So should it be checked in? Perhaps we will build this functionality into the generic XSLT class. Result: we are going to keep this class but change the naming convention from what Shoji originally had. There would be a default file with no special extension.Shoji: Documentation -- who will write it?

"business logic" of Aggregated Layout -- Peter

He showed a diagram of how Aggregated Layout (A.L.) code processes the rendering, and explained it.

He explained how you describe pushed fragments in XML, then you run an Ant target to load it into the database. Michael (Ivanov) is working on ability to load pulled fragments this way also. There were some comments to improve the syntax of these files (most notably the groups should be referred to by their unique id, and not by their names).

restrictions on where a fragment can be in the user's layout, and whether and how it could be moved.

e.g. priority -- how close to front of tabs or folders it appears.depth -- how far from root of tree it is.parent -- restrictions on what the priority of the parent folder can be.

Michael Oltz, Cornell University 22 Revised 15 September 2003

Then you give a structure of folders and channels, and tell whether they are immutable, unremovable, and/or hidden.

Then Peter gave functional descriptions of what the most important methods do, such as AddNode

Dan: By 2.2, make a couple of examples.

Ken: In 2.2 Aggregated Layouts will be enabled by default.

Peter: IUserLayoutManager [and its default implementation] have been enhanced with several more methods to do A.L.

"Integrated Modes" -- the user interface part of Aggregated Layouts -- Justin

New "Integrated Modes" Structure and Theme

Modes:NormalPreferencesFragment Construction

This removes dependency on the "Preferences" channel;should make new structure development easier

The User Preferences as a separate channel is gone.

There are some request parameters that the UI uses to do the integrated-modes editing:delete node, show move targets, move node, show add targets, etc.

Justin showed what they look like in a URL line

There's no longer a header or footer channel; the lack of a header channel gives the site more control over what appears at the top.

The login channel appears as a wide narrow strip just under the tabs. Again this leaves more clear room for graphics etc. at the top.

If you resize fonts in the browser or change sizes of icons, the layout will adjust accordingly without the site having to twiddle with it.

Justin and Jon Allen put a lot of effort into making it work the same or at least work, on a wide variety of browsers, even Netscape 4.x; making this structure and theme work on NS 4 took about 80 percent of their entire time on the project.

Michael Oltz, Cornell University 23 Revised 15 September 2003

Should we use tables any more, or just use nested divs? Some people were still using NS 4 which does not support DIVs. But they degrade fairly quickly, don't work well.

The framework UI uses very little Javascript. Only the clock, the alert/confirms, and the pop for the window detach are Javascript.

The new User Preferences Mode puts icons, text boxes, etc. all over the place right in the actual layout, to allow the user to modify things.

Demonstration of how a channel will be added; the user chooses which tab they want to add a channel to, then choose which channel, then as a final step where on the tab they want to add it. Up to 2.1 we have had to choose the exact position first and then which channel to put there.

Jon Allen tried to demonstrate the new stuff but it crashed several times; someone back at the office had tinkered the demo portal the previous evening."If the demo crashes, that means it's good code -- it's fresh code." -- Chuck Severance

There is a demo site at http://www.immagic.com/uportal/

From the "im+m Portal Accounts" channel, choose "Aggregated Layouts/Integrated Modes"; that takes you to the 2.2 portal.

Steve: why does the login channel look different? it seems to be in the toolbar of the channel.

Justin: We didn't want people to think of the login channel as a channel. We hacked the stylesheet. We moved the login channel down into the tab area so that the institution could have a cleaner header.

Chuck: I think it is great that every part of even the framework is itself a channel.

Peter: [reactions to implementation details mentioned in passing]a) I think sending content to the theme stylesheet to get it displayed, is the wrong approach. b) And the theme stylesheet should not explicitly refer to the name of the login channel. The reason we put it in the header, is so they could be in the same place all the time, and would render the same way as any other channel. if you want to move the login channel somewhere else, you need a different example of something being in the header. c) It seems intuitive to me for the orange command line [during preferences editing] to be above the tabs.

Proposals for new features -- moderated by Ken

Michael Oltz, Cornell University 24 Revised 15 September 2003

a) Yale's Howard Gilbert's proposal for a new way to do error handling. The uPortal framework code as it is, needs work in this area.

He wanted to be here via speakerphone but there was no speakerphone.recommendations:Use try/finally to close constructsPortalException should not be thrown all the way to Tomcat. Should be captured first and have a more meaningful error message.

We called Susan Bramhall to ask some questions.

The ExceptionHelper is careful to log the error exactly once, and not multiple times as it goes up. The Helper is put into the PortalException class. She thinks it gets called as a result of creating the PortalException.There are many places in the code where a NullPointerException happens because the database can't be reached; it is far better to throw some other exception, not return null.

Ken: Howard [who was out of the office when we called] needs to show the actual new classes he wrote, and give a couple of examples, before we could make any decision. On August 13, 2003, a few days after the meeting, Howard posted a long message to the developer mailing list about his proposal. Subject "Re: Questions raised about exception handling"

Dan: Howard should also explain the difference between throwing different classes of exceptions, and using the I.D.'s and what is the advantage of the latter.

Susan: Maybe the Columbia people could come up to New Haven and we could have ...

Ken: An "error conference"

John Fereira: Someone at UBC had been talking about automatically contacting a specified person when an error occurs. But what about framework channels? It would be the portal administrator.

UBC was not present, but Ken read two proposals on their behalf. They want to be able to do an HTTPS post and then go to an HTTP page. So, they want to have a configuration parameter so that after a secure login (authentication) they could go to an HTTP page.

Keith at SCT has the SSL channels thing mentioned at the previous developers meeting partly done, should he check in what he has done, or not? He asked some questions and got no answers. HTTPS is a separate cookie space.

Michael Oltz, Cornell University 25 Revised 15 September 2003

Jan: "You will get popups in nested frames if you don't handle SSL and non-SSL content consistently."

Resolution: Ken will suggest on the email that Keith comment out the "Focus" ability and check in the rest of it. He sent this email on August 8.Ken: UBC wants to add the cababillity for image URLs to be re-written on the fly so they end up being loaded via a proxy. This way the image URLs and the portal page will both be served via https and the annoying browser pop up messages won't appear. We decided that we want to implement UBC's changes. Jan: Wanted to use log4j directly so we can use categories. That is each message is in a particular category.

Consensus was that we would not do that, but extend LogService with categories, so that we could drop in another implementation behind it when we drop support for 1.3 [see below]

Ken: When should we stop supporting 1.3?

Conclusion was, containers take a long time to switch to a newer major dot release. For example, WebSphere tends to be about a year behind. So Jan said we should wait until 1.5 becomes widely available.

Steve: The PreparedStatements implementation in RDBMServices is not complete enough; we need to make the PreparedStatements field public. [or, make a get for it] [Alex: or make it more complete]

Steve: He wants more published channel metadata. We put two more columns in UP_CHANNEL to note a primary and secondary contact person for each channel.

Kevin: Nick -- swapping out different versions of Xerces. He posted a message to the developer list on July 29 (Subject "Xerces dependency removal status") stating some of the issues he has with particular parsers etc. but he did get it working for Xerces anyhow. check it in?

Kevin: We would like to see an abstraction of the CharacterCache.

Kevin: Enhancements for a character (text only) content channel that does an end run around the transformations for efficiency.

Peter: Be sure to tell developers NOT to use it except as a last resort. It's mostly for legacy applications.

Jan: He wants a factory method for the ChannelManager so he can plug in a different implementation.

Michael Oltz, Cornell University 26 Revised 15 September 2003

Steve: He wants an implementationspecific method for the Stats classes.

Jim: Chuck has been trying to resolve the differences between OKI, JA-Sig, and Michigan.

Ken: Try out aggregated layouts and internationalization. Wrap up implementation by end of September.

Michael Oltz, Cornell University 27 Revised 15 September 2003

Appendix A

UportalI18nuPortal Internationalization

uPortal XSL Files and Database Translation

EditDictonary - Before starting editing the dictionary for each locale, readme first!

o TransKitjajp - Japanese (ja_JP) Dictionary o TransKitsvse - Swedish (sv_SE) Dictionary o TransKitengb - UK English (en_GB) Dictionary o TransKitdede - German (de_DE) Dictionary o TransKitptpt - Portuguese (pt_PT) Dictionary o TransKitplpl - Polish (pl_PL) Dictionary o TransKiteses - Spanish (es_ES) Dictionary o TransKitcaes - Catalan (ca_ES) Dictionary o TransKititit - Italian (it_IT) Dictionary

uPortal Locale : Three-letter-based Locale

Language codes in ISO 639-2 : three letter language codes Country codes in ISO 3166 : three letter country codes Examples:

o English in USA: eng_USA o English in United Kingdom of Great Britain & N. Ireland :

eng_GBR o Japanese in Japan: jpn_JPN o Swedish in Sweden: swe_SWE o German in Germany: deu_DEU (the language code "deu" is

terminology, not bibliographic. For more information, see the above ISO639-2 link.)

Internatinalization Strategy

Presentation Material : of uPortal Dev Meeting at Salt Like City (March, 2003)

Michael Oltz, Cornell University 28 Revised 15 September 2003

Developper Meeting Minutes - uPortal developers meeting on 6-7 March, 2003

Tips for Channel writers

Don't use getByte() that it assumes ISO-8859-1. Instead of it, use getByte("UTF-8") in the case that String object is encoded in UTF-8.

See javax.jdo.I18NHelper

Last Modified:2003-09-10 (Wed) 20:58:06References:[FrontPage] [EditDictonary]

Hiki 0.4.2a © 2002-2003 by TAKEUCHI Hitoshi.Powered by Ruby 1.6.7 and Amrita.Modified by Shoji Kajita.

Michael Oltz, Cornell University 29 Revised 15 September 2003