quality initiative update
TRANSCRIPT
![Page 1: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/1.jpg)
![Page 2: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/2.jpg)
Patent Public Advisory Committee
Quarterly Meeting
Quality Initiative Update
May 5, 2016
![Page 3: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/3.jpg)
Quality Agenda
• Patent Quality Community Symposium
• Quality Metrics
• Re-evaluate AFCP 2.0 and Pre-appeal
• Topic Submission for Case Studies
• External Survey Results
3
![Page 4: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/4.jpg)
Patent Quality Community Symposium: Empowering Innovation Through Enhanced Quality
Richard Seidel
Acting Deputy Commissioner for Patent Administration
![Page 5: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/5.jpg)
Patent Quality Community
Symposium
• Wednesday, April 27, 2016
• Where
– USPTO Alexandria
– All four regional offices
– Webcast
• Participation: Over 2,2005
![Page 6: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/6.jpg)
Patent Quality Community
Symposium
Featured Presentations
• Updates on Enhanced Patent Quality
Initiative (EPQI)
• USPTO’s efforts to use Big Data
• Quality Metrics for FY2016
• Master Review Form Workshop (MRF)6
![Page 7: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/7.jpg)
Quality Metrics
Richard Seidel
Acting Deputy Commissioner for Patent Administration
![Page 8: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/8.jpg)
Quality Metrics Redefined
Final Disposition Compliance
In-Process Compliance
First Action (FAOM) Review
Search Review
Quality Index Reporting (QIR)
External Quality Survey
Internal Quality Survey
Composite Score
FY 2011 - FY 2015Product Indicators
Master Review FormCapturing both correctness and clarity of examiners’ final work
product using uniform criteria gathered in a single database
Process Indicators
Transactional QIRTracking the efficiency and consistency of our processes
(for example, to identify “churning”)
Perception Indicators
Survey ResultsContinuing to internally and externally poll perceptions of
patent quality
FY 2016
8
![Page 9: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/9.jpg)
Quality Metric Data Source:
Product Indicators
Correctness
Clarity
FY 2016 Key Product Metrics FY 2016
9
Product Indicators
Master Review FormCapturing both correctness and clarity of examiners’ final work
product using uniform criteria gathered in a single database
Process Indicators
Transactional QIRTracking the efficiency and consistency of our processes
(for example, to identify “churning”)
Perception Indicators
Survey ResultsContinuing to internally and externally poll
perceptions of patent quality
![Page 10: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/10.jpg)
Quality Metric Data Source:
Process Indicators
FY 2016 Key Process Indicators
FY 2016
10
Reopening Prevention
Consistency of
Decision-Making
Rework Reduction
Perception Indicators
Survey ResultsContinuing to internally and externally poll
perceptions of patent quality
Product Indicators
Master Review FormCapturing both correctness and clarity of examiners’ final work
product using uniform criteria gathered in a single database
Process Indicators
Transactional QIRTracking the efficiency and consistency of our processes
(for example, to identify “churning”)
![Page 11: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/11.jpg)
Quality Metric Data Source:
Perception Survey Results
FY 2016 Vital Perception Indicators
FY 2016
11
Root Cause Analysis
Validation/Verification
Product Indicators
Master Review FormCapturing both correctness and clarity of examiners’ final work
product using uniform criteria gathered in a single database
Process Indicators
Transactional QIRTracking the efficiency and consistency of our processes
(for example, to identify “churning”)
Perception Indicators
Survey ResultsContinuing to internally and externally poll
perceptions of patent quality
![Page 12: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/12.jpg)
Quality Metrics
12
• Federal Register Notice published on March 25, with comments due May 24– Requesting feedback on:
• Decision to replace Composite Quality Score with individual metrics
• How to objectively measure patent examination quality
• Standardized Master Review Form
Quality Metrics website: http://www.uspto.gov/patent/initiatives/quality-metrics
Contact Us: [email protected]
![Page 13: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/13.jpg)
Questions & Comments
Richard Seidel
Acting Deputy Commissioner for Patent Administration
(571) 272-2950
![Page 14: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/14.jpg)
Re-evaluate AFCP 2.0 and Pre-appeal
Remy Yucel
Assistant Deputy Commissioner for Patent Operations
![Page 15: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/15.jpg)
Re-evaluate AFCP 2.0 and Pre-appeal
• Tests how some of the best attributes of
the AFCP 2.0 and the Pre-Appeal pilots can
be combined to give both applicants and
examiners additional information
• Increased understanding of the issues will
lead to more accurate decisions on
subsequent courses of action15
![Page 16: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/16.jpg)
Re-evaluate AFCP 2.0 and Pre-appeal
• Features under consideration:
– Available within 2 months of final rejection
– Panel, including a neutral party
– Applicant participation to present arguments as in Pre-Appeal (5-page document) or claim amendments
– More information on panel decision (i.e. grounds of rejection withdrawn or maintained, claims rejected, allowed, additional brief comments)
16
![Page 17: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/17.jpg)
Questions & Comments
Remy Yucel
Assistant Deputy Commissioner for Patent Operations
(571) 272-0700
![Page 18: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/18.jpg)
Topic Submission for Case Studies
Anthony Caputa
Director, Office of Patent Quality Assurance (OPQA)
![Page 19: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/19.jpg)
Topic Submission for Case Studies:
Pilot Summary
• Federal Register Notice initiated the program on December 21, 2015
• Submissions were accepted through February 12, 2016
• USPTO invited stakeholders to submit patent quality-related topics for study
19
![Page 20: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/20.jpg)
What is a Case Study?
• Review of a single, quality-related issue
• Tailored to the selected issue
• Performed by USPTO
– Distinct from standard reviews completed by
the Office of Patent Quality Assurance (OPQA)
20
![Page 21: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/21.jpg)
Topic Submission for Case Studies:
Program Goals
• Use stakeholder experience to provide USPTO with a
wider range of topics to consider for a case study
• Use study results to better understand and enhance
quality of USPTO work products and processes to:
– Identify quality issues and examples of examination
best practices
– Reveal areas where further training may be needed
21
![Page 22: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/22.jpg)
Topics Selected for Case Study
1. Compliance of rejections with 35 U.S.C. 101 Official Guidance
2. Consistency of application of 35 U.S.C. 101 across Art Units/Technology Centers
3. Use of compact prosecution when making 35 U.S.C. 101 rejections
4. Correctness and clarity of motivation statements in 35 U.S.C. 103 rejections
5. Enforcement of 35 U.S.C. 112(a) written description in continuing applications
6. Consistent treatment of claims after May 2014 35 U.S.C. 112(f) training
22
![Page 23: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/23.jpg)
Topic Submission - Resources
• Topic Submission for Case Studies Webpage: http://www.uspto.gov/patent/initiatives/topic-submission-case-studies-pilot-program
• Enhanced Patent Quality Initiative Webpage: http://www.uspto.gov/patent/initiatives/enhanced-patent-quality-initiative
• Contact us at [email protected]
23
![Page 24: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/24.jpg)
Questions & Comments
Anthony Caputa
Director, Office of Patent Quality Assurance (OPQA)
(571) 272-0829
![Page 25: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/25.jpg)
External Quality Survey (EQS)
Martin Rater
Chief Statistician, Office of Patent Quality Assurance (OPQA)
![Page 26: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/26.jpg)
External Quality Survey (EQS)
• Conducted semi-annually since 2006– Most recent survey Q1 in FY16
• Surveys 3,000 frequent-filing customers with each survey
• Has been included in Patent Quality Composite (FY2011-15)
• Continues to be a vital quality indicator as we transition to new quality metrics in FY17
26
![Page 27: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/27.jpg)
Perception of Product:
Quality of Rejections Made
27
0%
10%
20%
30%
40%
50%
60%
70%
FY07Q1
FY07Q3
FY08Q3
FY09Q1
FY09Q3
FY10Q1
FY10Q3
FY11Q1
FY11Q3
FY12Q1
FY12Q3
FY13Q1
FY13Q3
FY14Q1
FY14Q3
FY15Q1
FY15Q3
FY16Q1
6e. 35 U.S.C. 112Rejections, Paragraph 2
6b. 35 U.S.C. 102Rejections
6d. 35 U.S.C. 112Rejections, Paragraph 1
6a. 35 U.S.C. 101Rejections
6c. 35 U.S.C. 103Rejections
Q6: Frequency of Technically, Legally, and Logically Sound Rejections
(Percent reporting “Most of the time” or “All of the time” of the time)
335 U.S.C. 112(b)
5 U.S.C. 102
35 U.S.C. 112(a)
35 U.S.C. 101
35 U.S.C. 103
Rejections under
![Page 28: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/28.jpg)
Perception of Product and Process:
Advancing Prosecution
28
Q10: In your experience, what have examiners done that has helped advance prosecution?
192
151
146
117
108
528
0 100 200 300 400 500 600
Interviews at all stages
Initiate contact with Applicant
Willing to discuss via phone/email
Well prepared for meetings
Well written responses
Collaborative, constructive, makes suggestions
![Page 29: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/29.jpg)
Perception of Product:
Consistency
29
Q9: In the past 3 months, have you experienced problems with the consistency of
examination quality from one examiner to another?
Large degree of
inconsistency
Small degree of
inconsistency
No inconsistency
0%
10%
20%
30%
40%
50%
60%
FY07
Q1
FY07
Q3
FY08
Q3
FY09
Q1
FY09
Q3
FY10
Q1
FY10
Q3
FY11
Q1
FY11
Q3
FY12
Q1
FY12
Q3
FY13
Q1
FY13
Q3
FY14
Q1
FY14
Q3
FY15
Q1
FY15
Q3
FY16
Q1
![Page 30: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/30.jpg)
Perception of Overall Quality
30
31%33%
30%27% 28%
32%
40%38%
42%45%
47%
51% 52% 51% 51% 50%47%
54%
21% 20%23%
25% 24%
18%
11%14% 14%
9% 9% 8% 9% 9% 8% 9%11%
9%
0%
10%
20%
30%
40%
50%
60%
FY07
Q1
FY07
Q3
FY08
Q3
FY09
Q1
FY09
Q3
FY10
Q1
FY10
Q3
FY11
Q1
FY11
Q3
FY12
Q1
FY12
Q3
FY13
Q1
FY13
Q3
FY14
Q1
FY14
Q3
FY15
Q1
FY15
Q3
FY16
Q1
Percent reporting
"good" or "excellent"
Percent reporting
"poor" or "very poor"
Q7: Percent Positive and Negative Ratings of Overall Examination Quality in Past 3 Months
![Page 31: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/31.jpg)
Link between Perception of
Consistency and Overall Quality
31
75%65%
27%
23%32%
53%
3% 3%
20%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
No Inconsistency Small Degee of
Inconsistency
Large Degree of
Inconsistency
Poor or Very Poor
Fair
Good or Excellent
Overall Quality
Perception
![Page 32: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/32.jpg)
Using EQS for Validating the
Quality Metric
32
• Utilize the External Quality Survey as a snapshot of
stakeholders’ perceptions
• Assure alignment of the quality data underlying our
metrics and our external stakeholders’ perceptions
• Exploit the flexibility of the Master Review Form to
capture data points that reflect patent quality
![Page 33: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/33.jpg)
Questions & Comments
Martin Rater
Chief Statistician, Office of Patent Quality Assurance (OPQA)[email protected]
(571) 272-5966
![Page 34: Quality Initiative Update](https://reader036.vdocument.in/reader036/viewer/2022062401/586765231a28ab68408b958d/html5/thumbnails/34.jpg)