more than a robust and scalable ldap group store matthew ling ( matthew.ling@ucalgary )

Information Technologies 1

More Than a

Robust and ScalableLDAP Group Store

Matthew Ling ([email protected])

June, 2004

http://www.ucalgary.ca/


Agenda

Just-in-Time LDAP Group Store (JIT LDAP)

Comparison to PAGS

Institutional Group Implementation Issues

Identity Amalgamation



JIT LDAP

Just-in-Time LDAP Group Store



Motivation – Authorization (1)

Authorization:

Answers these question: (bottom-up)

Which groups does this person belong to?

Which groups does this sub-group belong to?

Does not answer these questions: (top-down)

Who belongs to this group?

Which sub-group belongs to this group?



Motivation – Authorization (2)

All LDAP Groups

Group A Group B Group C

Group/Subgroup Person

SG A.1 SG A.2 SG C.1 SG C.2 SG C.3

SG A.1.1

A B

SG C.2.1

AB C

SG B.1

C



Motivation - Size

Group has a large membership (1,000+)

Basic groups: faculty (~2,000 ), support staff (~3,000),

active student (20,000+), alumni (???)

Sub-groups (~1,000)



Motivation - Reusability

Applications share group information via LDAP

Portal and web applications share groups

LDAP attribute value is the name of a group

(e.g. which department a person belongs to?)



Motivation - Flexibility

Supports multiple LDAP attributes

classification: student, faculty, support staff etc

particulars: department, course, etc

Specify relationship among values of an LDAP attribute

flat values -> hierarchical relationship



Motivation - Manageability

LDAP attribute with 100+ values (e.g. course)

Automatic discovery of new values

not required: exhaustive group name list

Automatic derivation of relationship among values

not required: complete subgroup relationship map



JIT LDAP - Basics

uPortal compatibility: 2.1.x, 2.2

Minimalist and yet all-inclusive approach

(Minimalist) creates a user’s groups on-demand as (s)he signs on

(Minimalist) “learns” groups gradually in a need-to-know basis

(All-inclusive) A value of an LDAP attribute is a group

Group Manager/Permission Manager compatibility

static group/subgroup definition is available

Group name/membership definition delegated to external agents,

i.e., no “business knowledge”

Package: http://www.ucalgary.ca/~mling/uportal/jitldapgroup/



JIT LDAP – JIT or Not

All LDAP Groups




SG A.1.1

A B

SG C.2.1

AB C

SG B.1

C



JIT LDAP – non JIT

All LDAP Groups


Group and subgroup definition are invariant but person membership can change



SG A.1.1

A B

SG C.2.1

AB C

SG B.1

C

Load all group members together



JIT LDAP – JIT

All LDAP Groups

Group A Group C

SG A.1 SG C.2

SG A.1.1 SG C.2.1

A A

Group and subgroup are created as needed

B

SG A.2

B

Group B

SG B.1

CC

Group members are added incrementally Partial view of group/subgroups - (some may not be visible)

SG C.3 SG C.3 SG C.1 SG C.1



LDAP attribute name

LDAP filter: uid=${key}

<JitLDAPGroupStore>

<config>

<url>ldap://ldap.univerity:389/ou=people,o=some.university</url>

<bindDN>cn=manager,ou=people,o=some.univesity</bindDN>

<bindPassword>manager-password</bindPassword>

<keyfield>uid</keyfield>

<keyvalue>${key}</keyvalue>

<retention-time>180</retention-time>

</config>

<group attribute=“department” />

</JitLDAPGroupStore>

JIT LDAP - Configuration (1)




When a user signs on, a LDAP search is performed

All groups associated with the user are created (once)

automatically

A group is also created (once) by

Look up a person in the group via Group/Permission Manager

Name conflict amongst LDAP attributes:

prefixed by LDAP attribute name




Auto-create groups



JIT LDAP – Implications

Addition/Deletion of a group could become “fuzz”

Add a group:

(Implicit) A person has the group name

(Explicit) Include programmatically

Delete a group:

(Implicit) No person has the group name and regular flushing of cached

group entries

(Explicit) Exclude programmatically

No distributed locking or broadcasting during addition/deletion



JIT LDAP - Group Manager Integration

default super group “<attribute name>:”, e.g., “department:”

add the super group of an attribute to the group “Everyone”

all groups become subgroups of the default super group

pre-defined groups are defined as follows

<group attribute=“department” >

<subgroup-tree>

<subgroup name=“Department of Civil Engineering”/>

<subgroup name=“Department of Drama”/>

</subgroup-tree>

</group>



JIT LDAP - Subgroup Relationship

define a logical sub-group: the group name is not an LDAP attribute value

define subgroup hierarchy

<group attribute=“department” >

<subgroup-tree>

<subgroup name=“Faculty of Engineering”>

<subgroup name=“Department of Civil Engineering” />

</subgroup>

.........

</subgroup-tree>

</group>



JIT LDAP – Group Hierarchy

logical/real groups

default attribute super group



JIT LDAP – Design (1)

Portal Group Service Terminology

service name – the identifier of an external group store

group key : unique within group store

together with service name uniquely identify a group

typically, an abstract key (i.e. data-less, meaningless)

used by Group/Permission Manager

group name: a descriptive name

may not be unique

Group/Permission Manager GUI displays it in replace of the group key



JIT LDAP – Design (2)

JIT LDAP Design

group key = LDAP value prefixed with the attribute name

<LDAP attribute name> ‘:’ <LDAP value>

– uniqueness because of the prefix and the nature of data

– no abstract key management required

group name = group key

no group name management required



JIT LDAP - Permission Manager Implication

Permission manager: permission grant/deny by principal key

principal key = <service name>.<group key>

= <service name>.<prefixed LDAP value>

principal key is a now derivative of the name of the group

assumes that names of the group will not change over time

permission rules can be added by back-end processes before the

group is defined (because no abstract key assignment is needed)



JIT LDAP – Scalability and Performance

At most one LDAP search per user

LDAP search time is < 50 milliseconds

User signs on (search LDAP if the user’s groups were not cached)

Real user groups: Search LDAP for the user’s groups

Logical user groups: get all ancestor groups using internal group hierarchy

Memory requirement:

A single instance of the group hierarchy (per LDAP attribute)

A cache that maps all signed on users to their groups



JIT LDAP – Loose Ends

Key value: default node separator is dot, ‘.’

group key = an LDAP attribute value

A group key cannot contain dot ‘.’

JIT LDAP replaces all ‘.’ with ‘#’

Change node separator (available in uPortal 2.1.5 or above)

For example, “->”



JIT LDAP – Current Status

Using JIT LDAP since October 2003

Primary staff and faculty portal

Users (3600+): Most groups are derived from LDAP

primary classifications (with 1000+ members)

other categories (with 500+ members)

Permissions specified using JIT LDAP groups

500 sign-on’s per day (Sign on response time, 3 to 5 seconds)

Student portal in September 2004



JIT LDAP – What’s next? (1)

Better Group/Permission Manager Integration

No pre-definition of groups required

Find “Group of Persons”: searches data store instead of cache

Tricky in LDAP: a list of groups/subgroups may not exist

Permission Manager: (group key = group name)

Maybe specify the principal by using an open input field instead of

selection list?

Or, define a group name temporarily via group manager




Permission Manager Convenience

Union operation – multiple “GRANT” rules

Negation operation – “DENY” rules (kind of)

Supports group definition using set operators:

union: has one of the given group names

intersection: has all given group names

difference: has one but not the other group name

negation: has none of the given group names




Auto derivation of hierarchical relationship

Structured group names required, e.g., federated group names

English.101.L01.T02 triggers the creation of group hierarchies, like,

EnglishEnglish.101English.101.L01English.101.L01.T02

EnglishEnglish.101 (suppress the rest)

English101L01T02

(portal group name = leaf component of the portal group key)




Factor out JIT logic

Allows data store independent

Accepts user’s group names received during sign-on

JIT just caches the user’s group names

JIT re-queries sign-on server for group names if necessary

(Note: We have a SSO server that can provide a user’s group names at

sign-on time)



Comparison to PAGS

Comparison to PAGS



Comparison to PAGS (1)

Design Difference 1: Value as group key verses value as data

JIT LDAP: LDAP value is the key of the group

Value can be is AS IS

PAGS: LDAP value is a piece of personal data

Additional interpretation may be required




Design Difference 2: Abstract key verses verbose key

JIT LDAP: Use the name of the group as the key

No internal key management

Permission rules are group name dependent

PAGS: Use an abstract key

Abstract key management

Group names can change without affecting permission rules




Just-In-Time verses Invariant

JIT LDAP: New groups are created as needed

PAGS: Groups are defined (created) at start up

Group Definition

JIT LDAP: Explicit exclusion - include all unless specified

PAGS: Explicit inclusion – include only if specified




External verses Internal Group Membership Definition

JIT LDAP: No “business” logic

External agents set up group membership

Logical groups are there for (permission manager)

convenience only

PAGS: “business” logic defined via IPersonTester

Specifies group membership selection criteria (ad hoc)




Sub-group construction: bottom up verses top down

JIT LDAP: Bottom up

raw group loaded from LDAP

find all ancestors group from the internal hierarchy

PAGS: Top down

Recursive descends down hierarchy and tests



Institutional Group (IG)

Institutional Group Implementation Issues



Institutional Group (IG)

“Institutional” group characteristics:

Very large membership, lots of subgroups

Membership defined by complex business processes

Massive group definition/membership changes periodically

Examples:

courses taken by a student

active teaching assistant



IG - Implementation Issues

Our considerations and re-considerations

Just-in time or invariant

Group name space management

Group relationship

Automatic group membership – time sensitivity

Source of a person’s group information: pull or push model

Stateful or stateless group information



IG - Just-In-Time verses Invariant

Why Just-In-Time?

Performance – load as needed instead of all at group expansion

Availability – better suits 24x7 operation

Manageability

Delegation of administration/definition to external agents

Single authoritative definition source

Dynamic – group information can be pulled from or pushed by any

source



IG - Group Name Space (1)

How to resolve group name conflict:

the same name can be used to mean different things

Existing name conflict resolution methods:

Abstract key/Double naming: internal name in place of LDAP value

For example: PAGS

Name prefixing: prefix LDAP value with the LDAP attribute name

For example: JIT LDAP



IG - Group Name Space (2)

Better Solution: Structured/federated group name

Examples: DNS names, java class packages,

subsystem-A.application-B. ...

Our re-consideration: We are moving towards structured group names

Uses group names AS IS, no prefixing, no abstract key

Delegates the management of sub name space to applications

Maximizes reusability and sharing of group names amongst

applications



IG – Group Relationship (1)

How to define group relationship

Refer to a set of (related or unrelated) groups collectively during

authorization specifications

- Current method:

Each application defines its own, e.g,

JIT LDAP defines them as a hierarchical tree in its

configuration

These relationships are invisible among applications, not

reusable.




Simple methods to increase the reusability of sub-group

relationships:

Explicitly spell out in LDAP

Implicitly derived from the structured name



IG – Group Relationship(3)

Explicitly spell out: subgroup relationship is explicit, e.g,

LDAP values have these characteristics

a parent group always exists if one of its children group exists

e.g. ‘Faculty of Engineering’ exists if ‘Electrical Engineering’

exists

a child group may not exist even though its parent group exists

e.g ‘Faculty of Engineering’ can exist by itself



IG – Group Relationship(4)

Explicitly spell out: Pros/Cons:

(+) Relationship is available and shareable from LDAP

(+) Relationship can be ad hoc

(-) May duplicate a lot of data

(-) May be difficult to maintain data consistency



IG - Group Relationship (5)

Implicitly derived from structured group names

Examples:

English.101.L01.T02

EnglishEnglish.101English.101.L01English.101.L01.T02

W2004FREN101L01T02

W2004FREN101L01->T02




Implicitly derived sub-group relationship: Pros/Cons

(+) No duplicate data

(+) No name conflict

(-) tends to be hierarchical relationship only

(-) group names may become verbose, e.g., “a.b.c.d”

(+/-) Relationship is “visible” and shareable but each application

has to interpret the “same” way




Re-consideration:

Original plan:

Use portal internal sub-group definition

Revised plan

Use structured names and explicitly spell out sub-group

relationships



IG – Time Sensitivity (1)

When to add/drop group definition and membership massively

The role ceases but its function lingers

For example: Teaching assistant (TA)

Cease to be a TA at the end of term

Continue to perform some TA functions after the term ends,

e.g., submit student grades etc

HR may not tread the person as TA but the faculty continues to do so



IG – Time Sensitivity (2)

Individual relinquishes a role but (s)he needs access to data created

when (s)he was in that role

for example, payroll information for ex-TA

Not so good solution: print all such information

Some re-thinking about applications is required



IG – Group Information Source (1)

Original Plan: use LDAP

Open standard – well supported, use a “pull” model

Let participating applications have access to all person’s info

LDAP administrative password distribution and management

Re-consideration: a “push” model

a central server “pushes” out a person’s group information

no password distribution required

person’s information can be filtered



IG – Group Information Source (2)

Re-consideration:

Our SSO server is already providing the data

- Although proprietary, participating applications has to understand

the SSO server anyway.

- Derived (Logical) sub-groups can be defined and shared among

applications. (LDAP contains only “raw data”.)

How about Shibboleth? Is SAML sufficient?



IG – Stateful or stateless

Original Plan: pull model using LDAP

“Stateless” because each application retrieves its group info.

Group info. inconsistency across applications because of

application internal caches

Re-consideration: a “push” model

“Stateful”, applications received info. from the SSO server

Our SSO server can demarcate when a session begins and ends

Group info consistency across applications



Identity Amalgamation (IA)

Identity Amalgamation



Identity Amalgamation (IA)

One person owns multiple LDAP entries

one primary identity, multiple service account identities

multiple primary identities due to special/historical requirement, e.g.,

student ID, staff ID

Ideally, an identity should be converted to become a role.

In real life, this might not be possible.



A simple solution: linking identities (LDAP entries) using

a cross-reference (XRef) LDAP attribute

Look up roles of any LDAP entry that has the same XRef value

Merge resultant roles from the look up

Important:

Do not assume that a person must have an unique LDAP entry

IA – Linking LDAP entries



LDAP filter: xref=${key}

JIT LDAP sample solution

<config>

<url>ldap://ldap.univerity:389/ou=people,o=some.university</url>

<bindDN>cn=manager,ou=people,o=some.univesity</bindDN>

<bindPassword>manager-password</bindPassword>

<keyfield>xref</keyfield>

<keyvalue>${key}</keyvalue>

<retention-time>180</retention-time>

</config>

IA – JIT LDAP



IA – Implication

All services belonging to the person (not identity) are presented

A service has to determine which identity should be use if a person

has multiple identities.


End of Presentation


more than a robust and scalable ldap group store matthew ling ( matthew.ling@ucalgary )

Documents