data federation

16
Data Federation The process of bringing data together in a single virtual location for on-demand application access. www.opensciencegrid.org/cms THE DREAM, RIGHT?

Upload: sanam

Post on 12-Feb-2016

40 views

Category:

Documents


0 download

DESCRIPTION

Data Federation. The process of bringing data together in a single virtual location for on-demand application access. www.opensciencegrid.org/cms THE DREAM, RIGHT?. Where do the states keep their data?. Independent processing states keep membership data at home. Other states use IMS. - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Data Federation

Data Federation• The process of bringing data

together in a single virtual location for on-demand application access.

www.opensciencegrid.org/cms

THE DREAM, RIGHT?

Page 2: Data Federation

Where do the states keep their data?

• Independent processing states keep membership data at home.

• Other states use IMS.• But membership records are just the tip of

the iceberg…

Page 3: Data Federation

– Member benefits data– Voter records– Teacher certifications,

training– Salary data– Political contributions– Retirement plan data– Political activity– Event attendance

– Online advocacy– Mass emails– State RA – Local contracts and

bylaws– Survey data– Organizing new locals– ????

Where do the states keep their data?• Membership systems are not the system of record for:

Page 4: Data Federation

At OEA, our data is all over the place.

• IMS• Cognos• PacPulse• VAN• OEA Member Database (IMS extract)• MyOEA (mySEA feed)• Member Benefits• RegOnline• Grassroots.ohea.org (40k)• OEA RA • Ohio Department of Education• CapWiz• STRS• Locals

• School districts• OEA districts• Catalist• Political Parties• Secretary of the State• Pollsters• Live Call Centers• Meltwater• Exact Target• User’s drives in obscure docs• And more…..

Page 5: Data Federation

Why is this a problem?

Page 6: Data Federation

Users are lost when crossing systems.

Hi Jon, Wanted to check and see if you could match our most current [PAC] contributors list against members who are registered republican[Voter File].Thanks, Daniel A. Ramos UniServ Political Advocacy Consultant

Page 7: Data Federation

Some of the data is very private and should be protected. Who will see it?

Hi Jon The FCPE Task Force has met today. They have called Mary Suchy in to discuss the FCPE and the OEA RA. The FCPE Task Force would like for us to provide each district labels with their district materials. Can you please create a report in the raregistration website that includes the following: - OEA ID Number- Last 4 digits of SSN [ARGH!!!! NO, I CAN’T!]- Delegate Name- Alternate Name- Local Name- District Name (ie. NEOEA) not electoral unit- PacContributor? Thanks. Carol Taylor Price, AS/Conference Coordinator

Page 8: Data Federation

Data can come from anywhere in any format, size, or composition.

John-

I’ve got a question and maybe it’s more appropriate for membership but I know you to be a technical wiz w/ data etc.

I’m looking to do a mailing to our members who pay into STRS. In the past I get this list from membership and they run based on what info we have. It’s probably mostly accurate but may miss some members who may be in our data as something other than teachers but are paying into STRS. I also have a disk from STRS of all their contributing members but it’s 360K+ so only a fraction of those folks are our members.

Would it be possible and would it be advisable for us to run some type of member match from the STRS list and our membership data? If so, who could do that and how long would it take?

Any info you have would be helpful.

--Robert

Page 9: Data Federation

Does the user know what they want?

Jon,

See attached. [GOTV paid call list] Can you please match in the data from the Saturday calls into the main GOTV file I pulled. Also, any chance you can match in polling location name and address as separate fields into the spreadsheet.

Let me know when you think you can get these data sets finished.

Thanks, Daniel A. Ramos UniServ Political Advocacy Consultant

Page 10: Data Federation

Modify system of record data? (modify the source)

Jon,

Attached is the upload data. Please apply only to those that are marked in column ‘I’

For yes, maybe and already – please give the activist code EFOGOTV YesFor no – Please use EFOGOTVNo

There are three separate tabs of data on the excel.

Thanks,

Dan

Page 11: Data Federation

Who accesses what data in what systems?

Jon,

Had a good conversation with Hank Haynes today regarding FCPE. He had an interesting suggestion. Is there any way we can overlap the FCPE list with the ACEs list so we can see who is contributing to FCPE but not an ACE and vise versa and who is doing both.

Let me know if you need any assistance with this.

Also, we spoke about running some matches within the VAN to membership data from Mary’s department. The lists that she sends to the OEA – R members of prospective members and current members is missing some information that we may be able to match against the VAN. This include the HD’s, SD’s, School Districts and Counties.

Let me know if you want to discuss.

Thanks,

Dan

Page 12: Data Federation

Traditional Solution

• Data Warehousing– Extract data from outside sources– Transform the data to fit operational needs– Load it into an end target database (data mart)– Repeat

• Presentation Layer– User environment to access the data mart

Page 13: Data Federation

At NEA -> Cognos• For the states, its limited to the data NEA puts in the system.• Works very well for small group of OEA staff that use it.• Supposed to be the dream of open system where users can access

the data they need through informed querying of the data mart.• In reality, it’s a bunch of pre-templated reports for consumption.• Expensive… how many FTE to maintain? Realistic solution in the

states?• Inflexible… can be months to get new report. • Report creation rights limited.• Not the power tool that is sold by Cognos. (True of most

customers and products).

Page 14: Data Federation

Liberate Our Data• The more data we have in the states, the more powerful

and informed our decisions can be. This translates to effectiveness in accomplishing real-world tasks.

• Data enables, and in many cases drives, organizational activities.

• Data leads to informed decisions. Focus like a laser beam on targeted people instead of blanket communications to everyone.

• Data structures and tools must serve the tasks of the organizers, facilitators, bargainers, lobbyists, activists….using the language of the users.

Page 15: Data Federation

OEA’s datamart is a person

• Excel• SQL Server• VAN Matching• Catalist Extract (custom file from NEA) and

other cross-reference tables• I/O Tool• SQL Server Integration Services (replaced DTS)

Page 16: Data Federation

Discussion

• Is there a better way to do this in the states?• Any tools or suggestions?• What do your states do for requests like this?• What opportunities are we missing?