wg5 p02 proposal 2014 qualification of standard scripts
DESCRIPTION
WG5 P02 Proposal 2014 Qualification of Standard Scripts. Proposal through CSS 2014 http:// www.phusewiki.org/wiki/index.php?title=File:FDA_Scrips.ppt. Anyone should be able to submit a script, according to a check list Categorize scripts according to complexity - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: WG5 P02 Proposal 2014 Qualification of Standard Scripts](https://reader035.vdocument.in/reader035/viewer/2022062520/5681642a550346895dd5e932/html5/thumbnails/1.jpg)
#PhUSE
Standard Scripts Project 2
2014 - Proposal for Qualification of
Standard Scripts
![Page 2: WG5 P02 Proposal 2014 Qualification of Standard Scripts](https://reader035.vdocument.in/reader035/viewer/2022062520/5681642a550346895dd5e932/html5/thumbnails/2.jpg)
#PhUSE
Main Sections
Summary of prior proposal, 2013• hidden slides, to preserve history
Updated proposal, July 2014
![Page 3: WG5 P02 Proposal 2014 Qualification of Standard Scripts](https://reader035.vdocument.in/reader035/viewer/2022062520/5681642a550346895dd5e932/html5/thumbnails/3.jpg)
#PhUSE
Main Sections
Summary of prior proposalUpdated proposal– Objectives– Definitions: Qualification, States, Roles– Metadata and Test data– State Transitions
![Page 4: WG5 P02 Proposal 2014 Qualification of Standard Scripts](https://reader035.vdocument.in/reader035/viewer/2022062520/5681642a550346895dd5e932/html5/thumbnails/4.jpg)
#PhUSE
Proposal 2014 - Objectives• For End-users
– Clear overview of resources available, incl. purpose & state of each– Inspire confidence from first user experience– Ease-of-use:
• clear messaging from scripts• Reproducible results in users’ own environments• Consistency of scripts, learning first one makes remaining familiar
– Ease of converting users to contributors
• For Contributors & Standard Scripts Team– Ease of contribution: Modularize & standardize workflows & checklists– Modularize core components (e.g., FUTS for dependency checking?)– Automate testing, issue identification (e.g., concept similar to Spotfire/R compatibility)– Centralize & consolidate information & results
![Page 5: WG5 P02 Proposal 2014 Qualification of Standard Scripts](https://reader035.vdocument.in/reader035/viewer/2022062520/5681642a550346895dd5e932/html5/thumbnails/5.jpg)
#PhUSE
Qualification Proposalmeaningful terms in blue
http://www.phusewiki.org/wiki/index.php?title=File:WG5_P02_Proposal_-_2014.pptx
• Qualification Instructions – See template on PhUSE Wiki, link to file "Qualification of CSS scripts.xlsx"– Certification applies to new scripts and functionality– Confirmation applies to updates of existing script
• States:
Contributed, Develop, Review, Qualified• Roles
– Contributor: Anyone with appropriate skills & interests– Developer: A volunteer in CSS Working Group 5, familiar with our objectives**– Tester: A volunteer in CSS WG 05 who is familiar with our objectives**– Environment Tester: Anyone in community who is able to set up automatic test
replication in their work environment– Reviewer: Author of white papers, designers of script targets**
** suggests a quick-start onboarding page in CSS Phusewiki
![Page 6: WG5 P02 Proposal 2014 Qualification of Standard Scripts](https://reader035.vdocument.in/reader035/viewer/2022062520/5681642a550346895dd5e932/html5/thumbnails/6.jpg)
#PhUSE
Qualification Proposal• Metadata for scripts should indicate:
YML proposal– Whitepaper & output ID uniquely identify the target
<Script: Source, Title, Target>
– Programming language & version
<Language: name, version>
– Type of output:
<Script: Type>
– Study design:
<Script: StudyDesign>
– State of qualification <Stage>:
<Qualification: Stage>
– OS is not included, since should be indicated in OS compatibility report• Test Data requirements
– available as a program or script (text, not binary)– based on expected standards (SDTM, ADaM)– data requirements clearly & accurately specified for each script– include expected results from these data for defined tests/checks
![Page 7: WG5 P02 Proposal 2014 Qualification of Standard Scripts](https://reader035.vdocument.in/reader035/viewer/2022062520/5681642a550346895dd5e932/html5/thumbnails/7.jpg)
#PhUSE
Qualification Proposal• Transitions
"Contributed" is the original State of all scripts– to Develop, checklist includes
by Developer & Reviewer• R & D confer on suitability of contribution. Suitable starting point?
[ may require analysis details, specs, from contributor ]• D reviews components (checklist to be completed)• D works with Contributor to complete minimum components
[ including Test Data and Coverage of defined tests ]• D adds standard parameter, dependency checking• D writes Qualification of CSS scripts.xlsx, see PhUSE Wiki Qual Workbook
– to Review, checklist includes
by Tester• Review Qualification instructions, consider coverage of tests• Execute Qualification instructions• Work with Developer to complete execution successfully
![Page 8: WG5 P02 Proposal 2014 Qualification of Standard Scripts](https://reader035.vdocument.in/reader035/viewer/2022062520/5681642a550346895dd5e932/html5/thumbnails/8.jpg)
#PhUSE
Qualification Proposal• Transitions continued
– to Qualified by Tester & Environment Tester & Reviewer• T updates reference test outputs from certification/confirmation• E updates & executes local tests (posting PASS/FAIL results)• R confirms script output matches intention• R confirms qualification process covers important elements and
considerations. • R also provides user (rather than technical) feedback?• Achieve "Qualified" state when all tests in all test environments PASS (i.e.,
match outputs that T has certified and/or confirmed) and that R agrees that target is achieved
![Page 9: WG5 P02 Proposal 2014 Qualification of Standard Scripts](https://reader035.vdocument.in/reader035/viewer/2022062520/5681642a550346895dd5e932/html5/thumbnails/9.jpg)
#PhUSE
Qualification Proposal• Efforts Required
– Top priority• Finalize Qualification states, roles, workflow, checklists, and templates• Code criteria for "acceptable" (link to GPP, PhUSE)• Test definition, Test confirmation (also from White Paper team)• Test data
– Next priorities • Design test structure in google code• Develop scripts that will allow Environment Testing• Develop general components (e.g. parameter, dependency checking)• Identify Environment Testers based on
– Host environment– SAS or R version
• Identify opportunities to automate qualification. E.g.,– Environment Testers to post results back as machine readable– Script green-light/red-light qualification matrix on Phusewiki
![Page 10: WG5 P02 Proposal 2014 Qualification of Standard Scripts](https://reader035.vdocument.in/reader035/viewer/2022062520/5681642a550346895dd5e932/html5/thumbnails/10.jpg)
#PhUSE