cti technical advisory committee (tac) orientation november 18, 2015

Credential Transparency Initiative (CTI)

Technical Advisory CommitteeOrientation Webinar

November 18, 2015

• Background & CTI Scope of Work• CTI Structure, Timeline and Pilot-site Partner

Approach• Technical Advisory Committee• Dublin Core Application Profile Steps

Topics

Background &Scope of CTI Work

The CTI scope of work for period beginning July 2015 – December 2017 is the responsibility of the project partners: George Washington University, Workcred, and SIU Center for Workforce Development .

1. Develop common metadata terms for describing key features for description of credentials and credentialing organizations and Quality Assurance (QA) bodies that accredit, endorse, or approve them.

2. Create a voluntary, web-based registry and test it with roughly 100 pilot-site partners.

3. Develop and test software applications that facilitate use of the registry by guidance counselors, colleges, employers, etc.

CTI Scope of Work

Work Already Done

During 2014 – 2015 a large group of stakeholders from across a range of organizations was convened by GWU and Workcred/ANSI.

Paper(s) were written and models by (stakeholder) teams of experts focusing on issues and solutions for improving credential transparency, including inventories of existing schemas, frameworks, and identification of initial properties and value vocabularies concepts for:

Credentials Trust and Quality Quality Assurance Competency Frameworks

Credential Registry

Pilot

Credential Directory App Pilot

Other Apps

Design Only

CredentialsQuality

AssuranceTrust and

Quality

Competency

Frameworks

Resources

Slide 6

Technical Advisory Committee & Pilot-site Credentialing Partners are Co-Engineering

Common Terminology/ Credential & Quality

Assurance Descriptors

Voluntary Registry for Sharing & App for Reviewing

and Comparing

Describing CredentialsExamples

Although we are looking at a wide variety of credentials, we are trying to build a concentration in the areas of manufacturing, healthcare and information technology. Apprenticeship credential Badge College diploma High school diploma Industry/professional or other

type of certification Occupational/professional

license

Postsecondary degree Postsecondary education

certificate provided by a higher education institution

Training-provider certificate other than one provided by a higher education institution

Approach - Pilot Partners Receive Services• Conversion of information in the credential and competency

domain to machine and human readable/comparable formats.• Credential information published to the Credential Registry.• Use of the Credential Directory App. • Personalized “Roadmap” document based on the Partner’s current

format of credential information and the steps that can be taken to enable to map that data to the CT description language.

How Pilot-Site Partners Participate • Provide access to technical and credential staff.• Provide access to credential information.• Participate with scheduled meetings throughout the project

period.• Provide ongoing feedback.• Participate in evaluation.

Credentialing Pilot-Site Partner Services

Pilot-site Partner’s Personalized Roadmap

The CTI staff will provide each pilot site a written document as a guide and roadmap to help you enhance their websites and web formats in the future:

Where the organization is today in terms of providing information and data about credentials that enables transparency and shared data.

How to transform or map their current data formats to enable ongoing updates to the credential registry and to optimize web formats based on RDF and the World Wide Web Consortiums (W3C) specifications for Linked Data.

Raise awareness and understanding of the vision for transparent credential and related assessment information.

Provide guidance to credentialing organizations on how to publish competency information through a wide variety of common formats.

Provide data in a machine-readable, structured format with the transition from proprietary to non-proprietary formats.

Provide data using standards recommended by the W3C.

Open and linked structured data.

Personalized Guide and Roadmap

Slide 10

Credential Registry

As an intermediate step, the plan is to utilize the Learning Registry (LR) as the Credential Registry. The LR is an existing open infrastructure developed in partnership with U.S. Departments of Education and Defense. It is actively used to publish educational resource metadata and paradata: Metadata about resources and their use are stored as JSON

documents. Each document is an “envelope” containing information about the metadata (the “payload”).

The metadata in a resource data description may be defined using any metadata standard. In the past, metadata has been stored in NSDL Dublin Core format, IEEE LOM format, and currently LRMI/JSON-LD.

It is schema agnostic where the payload is concerned. This means that leveraging the LR for storing metadata about credentials makes sense, because payloads aren’t tied to any one schema by the software.

APIs exist for storing, updating, and retrieving metadata. http://docs.learningregistry.org/en/latest/spec/Technical_Spec/index.html

http://docs.learningregistry.org/en/latest/spec/Resource_Data_Data_Model/index.html

http://docs.learningregistry.org/en/latest/spec/Services_and_APIs/index.html

http://docs.learningregistry.org/en/latest/spec/Technical_Spec/index.html

Learning Registry – Network of Nodes


The LR is not just a single machine, it is a network of nodes that supports replication of data between nodes, so if one node becomes unavailable, chances are very good that the data you’re after is available on another node in the network.

The primary LR node is hosted on Amazon cloud/web services.

A node can filter what comes into it via replication. For example, if an organization only interested in IT credentials sets up a node, they can configure a filter that only allows those credentials.

The LR supports eventual consistency rather than ACID consistency,

so if a node is down for a period of time, and then comes back up, other nodes will be able to push the missing data to the affected node. This makes the LR very resilient.


http://docs.learningregistry.org/en/latest/spec/Resource_Distribution_Network_Model/index.html

http://docs.learningregistry.org/en/latest/spec/Resource_Distribution_Network_Model/index.html

Credential Registry & ApplicationsRegistry& Apps

Description Design, Develop, & Pilot

Planning

Credential Registry – Open metadata repository utilized by the apps. X

Credential Directory App – Search for and compare credential information, including related competencies. X

Credentialing Value Transfer App – Review and analysis of "competency-based" resumes, transcripts, and portfolios based on transfer policies and recommendations to determine and communicate the potential and actual transfer of value between all types of credentials within the marketplace.

X

Competency Authority App – Aids colleges, certification organizations, and other stakeholders in developing more transparent and assessable competency statements based on employer requirements and leading competency frameworks.

X

Employer Signaling & Talent Pipeline Management App – Help employers more clearly “signal” or communicate their competency and credentialing requirements. X

CTI Structure, Timeline and Work That’s Underway

Credential Transparency Initiative Collaborative

Stakeholder Advisory

Council

Credential/ Credentialing Organization Descriptor

Subcommittee

Executive Committee

Evaluation Committee

Credential Transfer Value Subcommittee

Credential Directory

Application Subcommittee

Talent Pipeline Management and

Employer Signaling

Subcommittee

Competency Authoring

Application Subcommittee

Quality Assurance Descriptor

Subcommittee

Pilot-Site PartnersTechnical

Advisory Committee

CTI Technical Team (Project Staff)

CTI Technical Team works directly with the:• Technical Advisory

Committee• Technical

Subcommittees• Credential Registry

Pilot-Site Partners

CTI Program Team (Project Staff)

2015 201707 10 2016 04 07 10 2017 04 07 10

Begin Collaborative and Committees

Begin CR & CD Plan/ Design

Begin CR and CD

Development

Minimum Viable Product

Run Pilot and Evaluation of

CR and CD

Complete CA, TPM/ES, and

CVT Plans

Complete CR and CD Pilot

and Evaluation

Technical and Design Documents: CR and CD (Aug 15 – Dec 15)

Recruit Credentialing and QA Pilot Organizations

(Jul 1, 2015 – Dec 2015)

Continuously Run and Improve Functioning of CR and CD (Feb 2016 – Dec 2017)

Collect Data on Participation and Use for Evaluation (Jan 1, 2016 – Oct 2017)

Plans Developed for CA, TPM/ES, and CVT (Jul 1, 2016 – Nov 1, 2017)

Final Reports

Abbreviations:Competency Authoring (CA)Credential Directory (CD)Credential Registry (CR)Credential Value Transfer (CVT)Employer Signaling (ES)Minimal Viable Product (MVP)Quality Assurance (QA)Talent Pipeline Management (TPM)

Establish and Facilitate Collaborative (Executive Committee, Stakeholder group) and all Sub-Committees(Jul 1, 2015 – Dec 2017)

Descriptor and Evaluation Committee Documents

(Jul 1 – Dec 31, 2015)

Implement Evaluation Plan (Jan 2016 – Oct 2017)

CR & CD Development MVP(Nov 1, 2015 – July 2016)

Work with 50-100 Credentialing and QA Organizations(Jan 1, 2016 – Sept 2017)

Maintain Website (Oct 2015 – Dec 2017)Launch Website

(Jul 1 – Sept 30, 2015)

Begin CA, TPM/ESand CVT Planning

CTI Project Timelines

Slide 16

Pilot-Site Partner Questionnaire

Slide 17

The CTI Pilot Site Questionnaire is organized by three major categories of descriptors that address what is being declared to the public on each credential: Category 1: Declaring Name and Type of Credential and Credentialing

Organization—what is the name and type of credential that is being declared to the public by the credentialing organization.

Category 2: Declaring What Is Inside the Credential—what are the declared competencies, proficiency levels, assessments, evidence of attainment, and connection to competency frameworks (e.g., DQP, Beta Framework, O*NET).

Category 3: Declaring Other Key Characteristics and Relationships—what are the other key characteristics and relationships being declared for the credential and the credentialing organization such as scope of primary application and whether the credential is accredited, endorsed, recognized by an external quality assurance entity.


Slide 18

The questionnaire will have three major uses: 1. Explore CT-DL Definitions and Schema. This questionnaire will be used to

introduce and explain the preliminary Credential Transparency Description Language (CT-DL) definitions of properties and their value vocabularies; and to explore questions, problems, and issues with application of the CT-DL for different types of credentials and credentialing organizations.

2. Explore Availability and Location of Information. It also will be used to identify the current or future availability and location of the information for each attribute currently identified and whether this information will be made publicly available through an existing or future website and can be used in the pilot project.

3. Explore Data Formats and Access. Finally, the questionnaire will be used to explore the current data formats (e.g., Excel, PDF) for the publicly available information and how this data can be accessed and provided to CTI staff for the conversion into Linked Data formats and for the development of a customized roadmap for each pilot organization if pilot organizations decide to make this conversion in the future


Slide 19

The questionnaire will have three major uses: 1. Explore CTI-DL Definitions and Schema. This questionnaire will be used to

introduce and explain the definitions of descriptors and their coding schema and to explore questions, problems and issues with their application for different types of credentials and credentialing organizations.

2. Explore Availability and Location of Information. It also will be used to identify the current or future availability and location of the information for each descriptor and whether this information will be made publicly available through an existing or future website and can be used in the pilot project.

3. Explore Data Formats and Access. Finally, the questionnaire will be used to explore the current data formats (e.g., Excel, PDF) for the publicly available information and how this data can be accessed and provided to CTI staff for the conversion into Linked Data formats and for the development of a customized roadmap for each pilot organization if pilot organizations decide to make this conversion in the future

Questionnaire: Quality Assurance Questions

Slide 20

TAC Purpose & Goals

CTI Technical Advisory Committee (TAC)

Slide 22

The TAC provides input into development of the metadata infrastructure necessary to the creation of the Credential Registry and the project Apps.

Works with the technical team to promote collaboration and harmonization across initiatives that are developing data models, vocabularies, and schemas.

Credential Registry

(Credential

Domain)

Labor Market

Information Systems

Learning Systems

and Resources

Employer Human

Resource Information

Systems

Student Information

Systems

Transcript, Portfolio

and Resume Systems

TAC Input is Vital

Develop metadata infrastructure.

Provide input and feedback on design

and pilot of Credential Registry.


and pilot of Credential

Directory App.


of up to three additional apps.

TAC in Detail

TAC Position in CTI Technical Development

Credential/ Credentialing Organization Subcommitte

e

Executive Committee

Credential Transfer Value Application

Competency Authoring Application

Talent Pipeline Management and

Employer Signaling Application

Credential Directory Application

Quality Assurance

Subcommittee

Technical Advisory

Committee (TAC)

CTI Program Team (Project Staff)

CTI Technical Team (Project

Staff)Implementatio

n

Advise & Assist in

Metadata Design

Stakeholder Data

Analysis

CTI Metadata Design Objectives

• The metadata infrastructure developed must conform to the W3C specifications for semantic metadata including, but not limited to, Resource Description Framework (RDF), RDF Schema (RDFS), and Simple Knowledge Organization System (SKOS).

• The resulting metadata modeling must be syntax independent and support an array of serializations (e.g., Turtle, JSON-LD, RDFa, RDF/XML, and Microdata).

• The metadata must reuse existing, publicly available properties and classes and not reinvent unnecessarily.

• The metadata modeling must take into account the work of Schema.org; providing the means for the CTI work — or some portion of the work — to serve as the Schema.org extension for Credential description.

• The metadata model must be informed through transparent interactions with partnering and credentialing organizations and feedback from stakeholders.

https://schema.org/

Full & Thin Application Profiles

What’s in and out of scope?

The focus of the Domain Model is on description of Credentials, Credential Issuers, Quality Assurance Organizations and other entities necessary to express the range of credential description data and key relationships. Description of Credential Holders "earning" credentials and Credential Consumers "requesting" verification of credentials are out of the CTI project's scope.

Metadata Infrastructure

The TAC and Technical Team will develop common terminology (vocabulary) for describing key RDF properties for the following set of tentative entity classes or types:

1. Credentials2. Organization Types:

• Credentialing Organizations offering Credentials; and• Quality Assurance (QA) Organizations that recognize, endorse, or

accredit these Credentials and/or Credentialing Organizations3. Competency Frameworks defining knowledge, skills and habits of mind

in a field of endeavor4. Controlled Vocabularies (enumerations) to support interoperability

CTI Process: Application of DCAP

31

Information on the DCAP stages

Development Stages

32

Reliance on a Community Domain Model

Example Entity Attributes Informing a

Common CTI Description Language

Slide 33

CredentialTransparency and

Portability

Credential Name, Version, Type Competency Requirements Type and Scope of Primary

Application Labor Market Value Credential Transfer Value Education and Career Pathway

Connections Eligibility Requirements Education and Training

Opportunities Credential Holder Profile Occupational Regulation and

Licensing Maintaining Credentials Credential Removal Credential Costs

CredentialTrust and Quality

Competency Development and Validation

Assessment System Quality Assurance Authentication Version Management and Control

Quality AssuranceDescriptors

Purpose of the QA Body Scope of the QA Body Quality Criteria – Framework for

Evaluation Methodology to Assess Outcomes Validity and Reliability of the QA

Process Ensuring Impartiality of the

Process and Managing Conflicts of Interest

Complaint Process Appeal Process External Validation Publicly Available Information

Example Registry Diagram

Example Registry System

Where is TAC now in this process?

CTI Process Application of DCAP

Foundations from 1st Phase of

Project

Stakeholder Data Informing the CTI-DL

Emergent Domain Model

TAC Website, Tools and Liaison

Technical Planning Site http://www.credreg.net/

Google Technical Group https://groups.google.com/forum/#!forum/cti-technical-committee

Google Open Discussion Group https://groups.google.com/forum/#!forum/cti-open-discussion

Google Drive Documents for Technical Team by Invitation https://drive.google.com/drive/folders/0B1uSTaYpjhXJekc0dlpGdW9xZGc

CTI Github for Specifications Versioning

Technical Liaison:Jeanne Kitchens, Associate DirectorSIUC Center for Workforce [email protected]

http://www.credreg.net/

http://www.credreg.net/

https://groups.google.com/forum/#!forum/cti-technical-committee



https://groups.google.com/forum/#!forum/cti-open-discussion

https://groups.google.com/forum/#!forum/cti-open-discussion

https://drive.google.com/drive/folders/0B1uSTaYpjhXJekc0dlpGdW9xZGc



• Next meeting is December 1, 2015 with in-person participation option in Washington DC• CTI Collaborative Launch 8:30 am registration – 11:15 am • Committee Meetings 1:30 am – 2:00 pm

• TAC members in DC area can attend in-person, webinar will be used • Agenda

• DCAP • Registry and Directory design

• Ongoing communications via private TAC Google Group and review of documents via technical planning site and Google Drive.• Establish regular webinar/call schedule.

Next Steps

Appendix

Credential Directory AppStakeholders Can Use the App to:

• Identify credentials meeting user-defined criteria based on key features of credentials

• Compile customized lists and descriptions of credential based on search criteria

• Build customized, branded directories for different users based on their own criteria, without the need for continual, duplicative content development and updating

• Publish comparable information through public directories or through existing career and education guidance systems

Slide 43

Credential Directory App

Registry

Employer Signaling & Talent Pipeline Management App

Helps Employers:

• More clearly “signal” their competency and credentialing requirements

• Provide comparable information on competency and credentialing requirements for their jobs, and endorse or recognize credentials for these jobs

• Process and manage competency and credentialing information in job applicant tracking

• Helps employers source and manage relationships with credentialing organizations and related education and training providers in order to better manage their talent pipeline

Helps Credentialing Organizations:• In working with employers to validate the

required competencies in their credentialsSlide 44

Employer Signaling &

Talent Pipeline Management

App

Registry

Key Functions:

• Designed to support the easy review and analysis of competency-based resumes, transcripts, and portfolios

• Provides transfer value profiles of credentials based on new measures of declared and actual transfer value from multiple credentialing organizations

• Provides transfer value directories and comparisons based on search criteria

• Provides comparisons between recommended and actual transfer for recommendation services and credential transfer consortia

Slide 45

Credential Transfer Value

App

Registry

Credential Transfer Value App

Key Functions:

• Designed to aid colleges, certification organizations, and others in developing more transparent and assessable competency statements

• Provides authoring tools for writing and communicating competency statements, including grammar guides, feedback systems (e.g., rating “power of competency” statements), and competency frameworks and libraries

• Provides publishing tools for publishing competency statements on websites and registry

Slide 46

Competency Authoring App

Registry

Competency Authoring App