making omelets without breaking eggs: adding enterprise features to condor jason stowe

67
Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Post on 18-Dec-2015

224 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Making Omelets Without Breaking Eggs:Adding Enterprise Features to Condor

Jason Stowe

Page 2: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

condor_q

1.0 Community

2.0 Condor += Hooks

3.0 Conversations with Miron

4.0 Configuration w/Pipes

5.0 Progress to date

Page 3: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Condor Community

Page 4: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe
Page 5: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

com·mu·ni·ty

Page 6: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

A group of people living … under the same government.

Page 7: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

A group of people having common interests: the scientific community

Page 8: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

… Sharing, participation, and fellowship.

Page 9: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe
Page 10: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Community = People

Page 11: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe
Page 12: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Prior work Condor for movies: 1000+ CPUs (Linux/OSX)

70+ TB storage Active Community member

Page 13: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Currently:Cycle Computing

Condor Advice, Software & Enterprise Support

Windows/Linux/OSX

Page 14: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Now work on bigger projects with people I sat with @ CondorWeek2005…

Page 15: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Robert Nordlund, HartfordLife Cutting-edge commercial user

Page 16: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

X000’s server VMs, X0000’s workstations

TBs storage,Vast IT resources

Page 17: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Commercial Users =sub-community

Page 18: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Share common interests… and resources

Page 19: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

- Windows (&Linux/Solaris)- Commercial DB (Oracle)- Large Storage Capacity- Network Infrastructure- Available Tools:

JMS, Web clusters, DNS

Page 20: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

How to Leverage Enterprise tools for Business Requirements -

Disaster Recovery/etc.

Page 21: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Bob Nordlund’s idea: Condor += Hooks

Page 22: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Configuration:Contribute hooks to Condor,

no workarounds (like extra daemons used in past)

Page 23: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

How do we get this done?

Page 24: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

How do we make an omelet,without breaking community eggs?

Page 25: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

A Conversation with Miron

Page 26: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Funding

Page 27: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Text Here

Condor TeamWisconsin

Gov’t & Other Grant Funding

Sources

Page 28: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Text Here

Condor TeamWisconsin

Gov’t & Other Grant Funding

Sources

Government Research

AcademicInstitutions

Science & Research

Page 29: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Condor TeamWisconsin

Gov’t & Other Grant Funding

Sources

Government Research

AcademicInstitutions

Industry

Science & Research

Page 30: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Couldn’t industry provide funding for a team of people?

Page 31: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Objectivity/Independence

Page 32: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Condor is Attractive:Open, Innovative, Community

Page 33: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Problem with Large Enterprise Funding Contracts

Page 34: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

= Dependence and Potential Conflict of Interest

Page 35: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe
Page 36: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Needs of the many must outweigh the needs of the few or the one.

Page 37: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

What is the optimal way from Miron/Wisconsin’s perspective?

Page 38: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Proposed Solution

Condor TeamWisconsin

3rd Party Integrator HartfordLife

Enterprise

Wisconsin Responsible for:• Architectural Decisions – Final approval for features• Code Review/Integration• Release Planning• Maintenance

Page 39: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Proposed Solution

Condor TeamWisconsin

3rd Party Integrator HartfordLife

Enterprise

Integrator Responsible for:• Wisconsin – Build & Test, architecture feedback,

development, code review, testing, and integration• Enterprise – requirements, scheduling, & support

Enterprise funds feature addition

Page 40: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Proposed Solution

Condor TeamWisconsin

3rd Party Integrator HartfordLife

Enterprise

Lowers overhead by dealing with

integrator not many companies

Scalable, Schedule

ResponsibilitySupport

Page 41: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

For the new feature:Wisconsin is Architect/Maintainer

Integrator is Developer/QA

Page 42: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Intellectual Property

Page 43: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Sharing, participation, and fellowship.

Page 44: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

One Unified Condor

Page 45: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

IP for Condor Code changes must be donated back to Condor

Page 46: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Plan the work. Work the plan.We got started…

Page 47: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

HartfordLife = EnterpriseCycle = Integrator

Wisconsin = Governing/Architect

Page 48: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Bob and I called someone we sat next to

@CondorWeek 2005

Page 49: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Ian Chesal, AlteraWritten Innovative Middleware,

Priority Scheduling and Quill (yay!)

Page 50: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Share common interests…

Page 51: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Configuration w/Pipes (CWP)

Page 52: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

CONDOR_CONFIG = cat file |

Page 53: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

CONDOR_CONFIG = default_fileLOCAL_CONDOR_CONFIG =

getconfig http://address/$(hostname) |

Page 54: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Current Status

Page 55: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

condor_status

vm1@Hartfordlife CLAIMED BUSYvm1@CycleComputing CLAIMED [email protected] CLAIMED BUSY

CWP Feature specification – DoneNMI Build & Test – Wrapping upCWP Implementation – In progressTesting & Code ReviewIntegration & Release

Page 56: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Condor_History

Page 57: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Community

Page 58: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Enterprise Community

Page 59: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Condor += Hooks

Page 60: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Model for feature addition

Page 61: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

if you are a Condor user

Page 62: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Need support with new functionality

(Condor += Hooks)

Page 63: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Participate in the Community

Page 64: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Talk with Miron.Talk with me.

Page 65: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

(Talk with everyone @CondorWeek 2006)

Page 66: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Help the community continue to work well for everyone.

Page 67: Making Omelets Without Breaking Eggs: Adding Enterprise Features to Condor Jason Stowe

Thank you. Questions?

[email protected]