image sharpness
DESCRIPTION
Image SharpnessTRANSCRIPT
UNITED STATES PATENT AND TRADEMARK OFFICE
APPLICATION NUMBER
FILING or 371 (c) DATE FIL FEE REC'D
12/908,161 10/20/2010
24739 CENTRAL COAST PATENT AGENCY, INC 3 HANGAR WAY SUITE D WATSONVILLE, CA 95076
682
Ul\TfE]) STI\TES ])EPA RTME'IT OF COMMERCE United States Patent and Trademark Office Adill",. COMMISSIO'JER FOR PATENTS
PO Box 1450 Alexandria, Virgmia 22313-1450 \VVi\V.uspto.gOY
ATTY.DOCKET.NO
P882 18 5 CONFIRMATION NO. 3367
FILING RECEIPT
111111111111111111111111]~!I]~~I~~I~~11~~~~Jj ~llllllllllllllllllllllllll
Date Mai led: 11/04/201 0
Receipt is acknowledged of this non-provisional patent application. The application will be taken up for examination in due course. Applicant will be notified as to the results of the examination. Any correspondence concerning the application must include the following identification information: the U.S. APPLICATION NUMBER, FILING DATE, NAME OF APPLICANT, and TITLE OF INVENTION. Fees transmitted by check or draft are subject to collection. Please verify the accuracy of the data presented on this receipt. If an error is noted on this Filing Receipt, please submit a written request for a Filing Receipt Correction. Please provide a copy of this Filing Receipt with the changes noted thereon. If you received a "Notice to File Missing Parts" for this application, please submit any corrections to this Filing Receipt with your reply to the Notice. When the USPTO processes the reply to the Notice, the USPTO will generate another Filing Receipt incorporating the requested corrections
Appl icant( s) Rodney Shaw, Aptos, CA;
Power of Attorney: The patent practitioners associated with Customer Number 24739
Domestic Priority data as claimed by applicant
Foreign Applications
Permission to Access - A proper Authorization to Permit Access to Application by Participating Offices (PTO/SB/39 or its equivalent) has been received by the USPTO.
If Required, Foreign Filing License Granted: 11/01/2010
The country code and number of your priority application, to be used for filing abroad under the Paris Convention, is US 12/908,161
Projected Publication Date: 04/26/2012
Non-Publication Request: No
Early Publication Request: No ** SMALL ENTITY **
page 1 of 3
Title
Sharpness in Digital Images
Preliminary Class
382
PROTECTING YOUR INVENTION OUTSIDE THE UNITED STATES
Since the rights granted by a U.S. patent extend only throughout the territory of the United States and have no effect in a foreign country, an inventor who wishes patent protection in another country must apply for a patent in a specific country or in regional patent offices. Applicants may wish to consider the filing of an international application under the Patent Cooperation Treaty (PCT). An international (PCT) application generally has the same effect as a regular national patent application in each PCT-member country. The PCT process simplifies the filing of patent applications on the same invention in member countries, but does not result in a grant of "an international patent" and does not eliminate the need of applicants to file additional documents and fees in countries where patent protection is desired.
Almost every country has its own patent law, and a person desiring a patent in a particular country must make an application for patent in that country in accordance with its particular laws. Since the laws of many countries differ in various respects from the patent law of the United States, applicants are advised to seek guidance from specific foreign countries to ensure that patent rights are not lost prematurely.
Applicants also are advised that in the case of inventions made in the United States, the Director of the USPTO must issue a license before applicants can apply for a patent in a foreign country. The filing of a U.S. patent application serves as a request for a foreign filing license. The application's filing receipt contains further information and guidance as to the status of applicant's license for foreign filing.
Applicants may wish to consult the USPTO booklet, "General Information Concerning Patents" (specifically, the section entitled "Treaties and Foreign Patents") for more information on timeframes and deadlines for filing foreign patent applications. The guide is available either by contacting the USPTO Contact Center at 800-786-9199, or it can be viewed on the USPTO website at http://www.uspto.gov/web/offices/pac/doc/general/index.html.
For information on preventing theft of your intellectual property (patents, trademarks and copyrights), you may wish to consult the U.S. Government website, http://www.stopfakes.gov. Part of a Department of Commerce initiative, this website includes self-help "toolkits" giving innovators guidance on how to protect intellectual property in specific countries such as China, Korea and Mexico. For questions regarding patent enforcement issues, applicants may call the U.S. Government hotline at 1-866-999-HAL T (1-866-999-4158).
GRANTED
LICENSE FOR FOREIGN FILING UNDER
Title 35, United States Code, Section 184
Title 37, Code of Federal Regulations, 5.11 & 5.15
The applicant has been granted a license under 35 U.S.C. 184, if the phrase "IF REQUIRED, FOREIGN FILING LICENSE GRANTED" followed by a date appears on this form. Such licenses are issued in all applications where the conditions for issuance of a license have been met, regardless of whether or not a license may be required as
page 2 of 3
set forth in 37 CFR 5.15. The scope and limitations of this license are set forth in 37 CFR 5.15(a) unless an earlier license has been issued under 37 CFR 5.15(b). The license is subject to revocation upon written notification. The date indicated is the effective date of the license, unless an earlier license of similar scope has been granted under 37 CFR 5.13 or 5.14.
This license is to be retained by the licensee and may be used at any time on or after the effective date thereof unless it is revoked. This license is automatically transferred to any related applications(s) filed under 37 CFR 1.53(d). This license is not retroactive.
The grant of a license does not in any way lessen the responsibility of a licensee for the security of the subject matter as imposed by any Government contract or the provisions of existing laws relating to espionage and the national security or the export of technical data. Licensees should apprise themselves of current regulations especially with respect to certain countries, of other agencies, particularly the Office of Defense Trade Controls, Department of State (with respect to Arms, Munitions and Implements of War (22 CFR 121-128)); the Bureau of Industry and Security, Department of Commerce (15 CFR parts 730-774); the Office of Foreign AssetsControl, Department of Treasury (31 CFR Parts 500+) and the Department of Energy.
NOT GRANTED
No license under 35 U.s.C. 184 has been granted at this time, if the phrase "IF REQUIRED, FOREIGN FILING LICENSE GRANTED" DOES NOT appear on this form. Applicant may still petition for a license under 37 CFR 5.12, if a license is desired before the expiration of 6 months from the filing date of the application. If 6 months has lapsed from the filing date of this application and the licensee has not received any indication of a secrecy order under 35 U.S.C. 181, the licensee may foreign file the application pursuant to 37 CFR 5.15(b).
page 3 of 3
To: From:
[email protected],officeactions@C ENTRALCOASTPA TENT. COM, plambuth@centralcoastpatent.< [email protected]
Cc: [email protected] Subject: Private PAIR Correspondence Notification for Customer Number 24739
Nov 04, 201005:49:26 AM
Dear PAIR Customer:
CENTRAL COAST PATENT AGENCY, INC 3 HANGAR WAY SUITE D WATSONVILLE, CA 95076 UNITED STATES
The following USPTO patent application(s) associated with your Customer Number, 24739 , have new outgoing correspondence. This correspondence is now available for viewing in Private PAIR.
The official date of notification of the outgoing correspondence will be indicated on the form PTOL-90 accompanying the correspondence.
Disclaimer: The list of documents shown below is provided as a courtesy and is not part of the official file wrapper. The content of the images shown in PAIR is the official record.
Application 12908161
Document Mailroom Date APP.FILE.REC 11/04/2010
Attorney Docket No. P882
To view your correspondence online or update your email addresses, please visit us anytime at https:llsportal.uspto.gov/secure/myportal/privatepair.
If you have any questions, please email the Electronic Business Center (EBC) at [email protected] with 'e-Office Action' on the subject line or call 1-866-217-9197 during the following hours:
Monday - Friday 6:00 a.m. to 12:00 a.m.
Thank you for prompt attention to this notice,
UNITED STATES PATENT AND TRADEMARK OFFICE
PATENT APPLICATION INFORMATION RETRIEVAL SYSTEM
PTO/SB/14 (07-07) Approved for use through 06/30/2010. OMB 0651-0032
U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE
Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it contains a valid OMB control number.
Attorney Docket Number P882 Application Data Sheet 37 CFR 1.76
Application Number
Title of Invention Sharpness in Digital Images
The application data sheet is part of the provisional or non provisional application for which it is being submitted. The following form contains the bibliographic data arranged in a format specified by the United States Patent and Trademark Office as outlined in 37 CFR 1.76. This document may be completed electronically and submitted to the Office in electronic format using the Electronic Filing System (EFS) or the document may be printed and included in a paper filed application.
Secrecy Order 37 CFR 5.2 D Portions or all of the application associated with this Application Data Sheet may fall under a Secrecy Order pursuant to
37 CFR 5.2 (Paper filers only. Applications that fall under Secrecy Order may not be filed electronically.)
Applicant Information: Applicant 1 I Remove I Applicant Authority (!)Inventor I OLegal Representative under 35 U.S.C. 117 I OParty of Interest under 35 U.S.C. 118
Prefix Given Name Middle Name Family Name Suffix
Rodney Shaw
Residence Information (Select One) (!) US Residency 0 Non US Residency o Active US Military Service
City Aptos State/Province I CA I Country of Residence i I US
Citizenship under 37 CFR 1.41(b) i US
Mailing Address of Applicant:
Address 1 281 Ventana Way
Address 2
City I Aptos I State/Province I CA
Postal Code 95003 I Countryi I US
All Inventors Must Be Listed - Additional Inventor Information blocks may be I Add I generated within this form by selecting the Add button.
Correspondence Information: Enter either Customer Number or complete the Correspondence Information section below. For further information see 37 CFR 1.33(a).
D An Address is being provided for the correspondence Information of this application.
Customer Number 24739
Email Address [email protected] I I Add Email I IRemove Emaill
Application Information:
Title of the Invention Sharpness in Digital Images
Attorney Docket Number P882 I Small Entity Status Claimed ~ Application Type Nonprovisional
Subject Matter Utility
Suggested Class (if any) I Sub Class (if any)1
Suggested Technology Center (if any)
Total Number of Drawing Sheets (if any) 9 I Suggested Figure for Publication (if any) I
EFS Web 2.2.2
PTO/SB/14 (07-07) Approved for use through 06/30/2010. OMB 0651-0032
U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it contains a valid OMB control number.
Attorney Docket Number P882 Application Data Sheet 37 CFR 1.76
Application Number
Title of Invention Sharpness in Digital Images
Publication Information: D Request Early Publication (Fee required at time of Request 37 CFR 1.219)
Request Not to Publish. I hereby request that the attached application not be published under 35 D U.S.C. 122(b) and certify that the invention disclosed in the attached application has not and will not be the
subject of an application filed in another country, or under a multilateral international agreement, that requires publication at eighteen months after filing.
Representative Information:
Representative information should be provided for all practitioners having a power of attorney in the application. Providing this information in the Application Data Sheet does not constitute a power of attorney in the application (see 37 CFR 1.32). Enter either Customer Number or complete the Representative Name section below. If both sections are completed the Customer Number will be used for the Representative Information during processing.
Please Select One: C!J Customer Number 1 0 US Patent Practitioner 10 Limited Recognition (37 CFR 11.9)
Customer Number 24739
Domestic Benefit/National Stage Information: This section allows for the applicant to either claim benefit under 35 U.S.C. 119(e), 120, 121, or 365(c) or indicate National Stage entry from a PCT application. Providing this information in the application data sheet constitutes the specific reference required by 35 U.S.C. 119(e) or 120, and 37 CFR 1.78(a)(2) or CFR 1.78(a)(4), and need not otherwise be made part of the specification.
Prior Application Status I Remove I Application Number Continuity Type Prior Application Number Filing Date (YYYY-MM-DD)
Additional Domestic Benefit/National Stage Data may be generated within this form I I Add by selecting the Add button.
Foreign Priority Information: This section allows for the applicant to claim benefit of foreign priority and to identify any prior foreign application for which priority is not claimed. Providing this information in the application data sheet constitutes the claim for priority as required by 35 U.S.C. 119(b) and 37 CFR 1.55(a).
I Remove I
Application Number Country i Parent Filing Date (YYYY-MM-DD) Priority Claimed
0 Yes C!J No
Additional Foreign Priority Data may be generated within this form by selecting the I I Add button. Add
Assignee Information: Providing this information in the application data sheet does not substitute for compliance with any requirement of part 3 of Title 37 of the CFR to have an assignment recorded in the Office.
Assignee 1 I Remove I
EFS Web 2.2.2
PTO/SB/14 (07-07) Approved for use through 06/30/2010. OMB 0651-0032
U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE Under the Paperwork Reduction Act of 1995, no persons are required to respond to a collection of information unless it contains a valid OMB control number.
Attorney Docket Number P882 Application Data Sheet 37 CFR 1.76
Application Number
Title of Invention Sharpness in Digital Images
If the Assignee is an Organization check here. D Prefix Given Name Middle Name Family Name Suffix
Mailing Address Information:
Address 1
Address 2
City State/Province
Country il Postal Code
Phone Number Fax Number
Email Address
Additional Assignee Data may be generated within this form by selecting the Add I I Add button.
Signature: A signature of the applicant or representative is required in accordance with 37 CFR 1.33 and 10.18. Please see 37 CFR 1.4(d) for the form of the signature.
Signature !Donald R. Boys! Date (YYYY-MM-DD) 2010-10-20
First Name Donald I Last Name I Boys Registration Number 35074
This collection of information is required by 37 CFR 1.76. The information is required to obtain or retain a benefit by the public which is to file (and by the USPTO to process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.14. This collection is estimated to take 23 minutes to complete, including gathering, preparing, and submitting the completed application data sheet form to the USPTO. Time will vary depending upon the individual case. Any comments on the amount of time you require to complete this form and!or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent and Trademark Office, U.S. Department of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO: Commissioner for Patents, P.O. Box 1450, Alexandria, VA 22313·1450.
EFS Web 2.2.2
Privacy Act Statement
The Privacy Act of 1974 (P .L. 93-579) requires that you be given certain information in connection with your submission of the attached form related to a patent application or patent. Accordingly, pursuant to the requirements of the Act, please be advised that: (1) the general authority for the collection of this information is 35 U.S.C. 2(b)(2); (2) furnishing of the information solicited is voluntary; and (3) the principal purpose for which the information is used by the U.S. Patent and Trademark Office is to process and/or examine your submission related to a patent application or patent. If you do not furnish the requested information, the U.S. Patent and Trademark Office may not be able to process and/or examine your submission, which may result in termination of proceedings or abandonment of the application or expiration of the patent.
The information provided by you in this form will be subject to the following routine uses:
1. The information on this form will be treated confidentially to the extent allowed under the Freedom of Information Act (5 U.S.C. 552) an the Privacy Act (5 U.S.C. 552a). Records from this system of records may be disclosed to the Department of Justice to determine whether the Freedom of Information Act requires disclosure of these records.
2. A record from this system of records may be disclosed, as a routine use, in the course of presenting evidence to a court, magistrate, or administrative tribunal, including disclosures to opposing counsel in the course of settlement negotiations.
3. A record in this system of records may be disclosed, as a routine use, to a Member of Congress submitting a request involving an individual, to whom the record pertains, when the individual has requested assistance from the Member with respect to the subject matter of the record.
4. A record in this system of records may be disclosed, as a routine use, to a contractor of the Agency having need for the information in order to perform a contract. Recipients of information shall be required to comply with the requirements of the Privacy Act of 1974, as amended, pursuant to 5 U.S.C. 552a(m).
5. A record related to an International Application filed under the Patent Cooperation Treaty in this system of records may be disclosed, a a routine use, to the International Bureau of the World Intellectual Property Organization, pursuant to the Patent Cooperation Treaty.
6. A record in this system of records may be disclosed, as a routine use, to another federal agency for purposes of National Security review (35 U.S.C. 181) and for review pursuant to the Atomic Energy Act (42 U.S.C. 218(c».
7. A record from this system of records may be disclosed, as a routine use, to the Administrator, General Services, or his/her designee, during an inspection of records conducted by GSA as part of that agency's responsibility to recommend improvements in records management practices and programs, under authority of 44 U.S.C. 2904 and 2906. Such disclosure shall be made in accordance with the GSA regulations governing inspection of records for this purpose, and any other relevant (i.e., GSA or Commerce) directive. Such disclosure shall not be used to make determinations about individuals.
8. A record from this system of records may be disclosed, as a routine use, to the public after either publication of the application pursuan to 35 U.S.C. 122(b) or issuance of a patent pursuant to 35 U.S.C. 151. Further, a record may be disclosed, subject to the limitations of 37 CFR 1.14, as a routine use, to the public if the record was filed in an application which became abandoned or in which the proceedings were terminated and which application is referenced by either a published application, an application open to public inspections or an issued patent.
9. A record from this system of records may be disclosed, as a routine use, to a Federal, State, or local law enforcement agency, if the USPTO becomes aware of a violation or potential violation of law or regulation.
EFS Web 2.2.2
DECLARATION AND POWER OF ATTORNEY FOR PATENT APPLICATION
ATTORNEY DOCKET NO.P882
As a below named inventor, I hereby declare that: My residence, post office address and citizenship are as stated below next to my name. I believe I am the original, first and sole inventor (if only one name is listed below) or an original, first and joint inventor (if plural names are listed below) of the subject matter which is claimed and for which a patent is sought on the invention entitled: Sharpness in Digital Images
the specification of which (check one) ~ is attached hereto. o was filed on: __ o Application Serial No. __ o and was amended on __ (If applicable)
I hereby state that I have reviewed and understand the contents of the above-identified specification, including the claims, as amended by any amendment referred to above. I acknowledge the duty to disclose information which is material to the examination ofthis application in accordance with Title 37, Code of Federal Regulations, s 1.56 (a). In the case that the present application is a continuation-in-part application, I further acknowledge the duty to disclose material information as defmed in 37 CFR s 1.56(a) which became available between the filing date of tile prior application and the filing date of the present application. I hereby claim foreign priority benefits under Title 35, United States Code sll9 of any foreign applications for patent or inventor's certificate listed below and have also identified below any foreign application for patent or inventor's certificate having a filing date before that ofthe application on which priority is claimed: Prior Foreign Application(s)
(Number) (Country) (Day/MonthIY ear Filed)
(Number) (Country) (Day/Month/Year Filed) I hereby claim the benefit under Title 35, United States Codes, 120 and 119 of any United States application(s) listed
below and, insofar as the subject matter of each of the claims of this application is not disclosed in the prior United States application in the manner provided by the first paragraph of Title 35, United States Code, 8112, I acknowledge the duty to disclose material information as dermed in Title 37, Code of Federal Regulations, 8156(a) which occurred betwecn the filing date of the prior application and the national or peT international filing date of this application.
(Application Serial No.): __ (Filing Date): __ (Statns): __ _ (Application Serial No.); __ (Filing Date): __ (Statns): __ _ (Application Serial No.): __ (Filing Date): __ (Statns): __ _ (Application Serial No.): __ (Filing Date): __ (Status): __ _ (Application Serial No.): __ (Filing Date): __ (Status): __ _
POWER OF ATTORNEY: As a named inventor, I hereby appoint:
[;gI Practitioners at customer number: 24739
OR
D Practitioners: Name: Registration number
to prosecute this application and transact all business in the Patent and Trademark Office connected therewith.
Please send all correspondence practitioners at:
~ The practitioners at the customer number indicated abovc
D Customer number: 24739
Page 2 DECLARATION AND POWER OF ATTORNEY FOR PATENT APPLICATION
ATTORNEY DOCKET NO. PSS2
Authorization To Permit Access To Application by Participating Offices
The above-signed hereby grants the USPTO authority to provide the European Patent Office (EPO), the Japan Patent Office (JPO), the Korean Intellectual Property Office (KIPO), the World Intellectual Property Office (WIPO), and any other intellectual property offices in which a foreign application claiming priority to the above-identified patent application is filed access to the above-identified patent application. See 37 CFR 1.14(c) and (h).
In accordance with 37 CFR 1.14(h)(3), access will be provided to a copy of the above-identified patent application with respect to: 1) the above-identified patent application-as-filed; 2) any foreign application to which the above-identified patent app1ication claims priority under 35 U.S.C. 119(a)-(d) if a copy of the foreign application that satisfies the certified copy requirement of 37 CFR 1.55 has been filed in the above-identified patent application; and 3) any U.S. application-as-filed from which benefit is sought in the above-identified patent application.
In accordance with 37 CFR 1.14(c), access may be provided to information concerning the date of filing the Authorization to Permit Access to Application by Participating Offices.
I hereby declare that all statements made herein of my own knowledge arc true and that all statements made on information and belief are believed to be tme; and further that these statements were made with the knowledge that willful false statements and the like so made are punishable by fine or imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that such willful false statements may jeopardize the validity ofthe application or any patent issued thereon.
I st inventor's signature: ____ -¥-_--'-'wt---"LO----=------,j'---L. __ -=--"""O::" ______ Dated: Oct J'I '"' Residence: 281 Ventana Way Aptos CA 95003 2. 0 10 Post Office Address: Same
Declaration and Power of Attorney- Page 2
~ ~ ~ ~
,........; Q 0
..... ~ ~
~
.~ ~
--u
\~ ~ ~ ~ ~
~ ~ ~ .~ I I I
~ M '--
Cl) 0 ~
...D ,..t:j \". 'e l ". .~ ~~ /
~ ~ ~
~ ~
I I
c!j
,~ \. ~ ~ ~
I I I
@ @ @ @ @ @
@ @ @ @ @ /
/ @ @ @
!
@ @ @ @ @ ( @ @ @
~)® @ @ @ \
\ \
\ \
('.l
@ @ @ @ @
i /
j /
0 ,......... ,......... ~ 0 ,.........
\ \)
@ @ @
@ @ @
@ @ /1 @ /
! /
@ @ I
\ @
\
@ @ @
@ @ @ / \
@ @ @ \
@ \ @ @ \ \ :
\ J \ \
@ @ @ \ \
\ ('.l 0 ,.........
) ,.........~.
o ,.........
,......... o ,.........
501 507 2048 2048
Original Image 1536 Final Image 1536
502 ~ Downsizing
Algorithm //-- 503 I
504 505 t Upsizing
Algorithm
Downsized Fi~D ~ Second Third "n"th
>- ----» Iteration 320 I Original ----» Iteration Iteration Iteration Chosen
480 506
Fig. 5
(601 602 603
Select an image to be enhanced for Sharpness
Downsize the image Apply a minimal- computation I--->'~I to a substantially convolution filter to image "0" I >'1
smaller resolution to produce an image "n" producing image "0"
(606 (605
Add the quotient for each pixel Divide the algebraic from step 605 to associated pixel ~ difference for each pixell~(--
value in image "0", to in step 604 by integer "n" produce an image "I".
607~ ~
Repeat process in step 606, incrementing the image no.
by 1 for each iteration, until the image no. is n-l.
(608
Present images "0" through "n" --~>,t>l1 in sequential order for a user
to select "best" image for sharpness
"" (604
Determine algebraic difference between
each pixel value of "n" with each pixel of "0"
"" ,.
~
( Upsize "best" image back to original larger resolution.
Fig. 6 610
(701 (702 (703
Select a plurality of Choose one of the Apply a minimal- computation digital images of ~ images as image "0" ~ convolution filter to "0" similar sharpness quality to be enhanced to produce an image "n"
for Sharpness
706~ ("705
Add the quotient for each pixel Divide the algebraic from step 605 to associated pixel <: difference for each pixel 141(--
value in image "0", to in step 604 by integer "n" produce an image "1".
\It ("707
Repeat process in step 606, incrementing the image no.
by 1 for each iteration, until the image no. is n-l.
(708
Present images "0" through "n" ---l)1Ii'I1 in sequential order for a user
to select "best" image for sharpness
..... ,.
'W' r 704
Determine algebraic difference between
each pixel value of "n" with same pixel of "0"
'" ,.
'W' (709
( Process each remaining image of I the plurality to produce just the .(--
Note the number of the image the user selects as best image
enhanced image of the same 710 number as in step 708
Fig. 7
M 1 1
.9
.8 .8
.7
.6 .6
.5 .5
.4 .4
.3 .3
.2 .2
.1 .1
o Local Pixel Value Average 255
Fig. 8
901 (902
Place filter and determine Determine algebraic new pixel value at first diffence between pixel position. original and new
pixel value
906\
Save the result of step 905 as new value for image "n"
\It ("907
Place filter at next position and repeat the process until all
positions are determined.
~
"
"" ,
(903
Select multiplier between o and I from graph using
segment preference
(905
Add result of step 904 back to original pixel 14:1(-
value
Fig. 9
"" --,
\It ,904
Multiply difference from step 902 by
multiplier from step 903
5
SHARPNESS IN DIGITAL IMAGES
CROSS-REFERENCE TO RELATED APPLICATIONS
N/A
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention is in the field of apparatus and techniques for enhancing the
10 apparent visual quality of images stored and presented digitally.
2. Description of Related Art
Many techniques exist for processing and filtering images displayed in a
computerized system as two-dimensional matrices of pixels, typically presented in a
15 rectangular matrix. A digital display presents pixels in color and brightness according to
values stored in memory for each pixel. For color, for example, there will be a separate
value in the RGB system for red (R), green (G) and blue (B). In an eight-bit computer
process, the values for each pixel for each attribute range from 0 to 255, which is 28. For
any of a wide variety of reasons, digital images may be less than optimal in image quality
20 as viewed by a person, and many commercial programs and techniques exist and are
available for altering the pixel values to improve the apparent quality of a digital image.
To improve the apparent quality of an image, an original image may be altered in one or
more of several attributes, such as brightness, contrast, color or what is known in the art
as sharpness. The present patent application is in the field of altering sharpness of digital
25 images to improve apparent quality to an observer.
To change apparent sharpness in a pixilated image requires changing individual
pixel values in relation to the values of surrounding (proximal) pixels. One technique
well known in the art for changing apparent sharpness is controlled application of what
are known in the art as convolution filters. References in the art to convolution filters
- 2 -
and their uses are numerous. For example RoboRealm at
http://www.roborealm.com/help/Convolution.php has a good description of convolution
filters and their uses.
Very generally, the way a convolution filter works is that a group of typically
5 adjacent pixels in the image to be enhanced is considered, the group having a central
pixel whose value is to be modified, dependent in some fashion on the value of the
adjacent pixels. The value of each pixel in the group is multiplied by a pre-determined
number, which theoretically may be different for each pixel in the group, the results are
added, the sum is divided by the number of pixels in the group (average), the determined
10 average value is divided by a number that is a function of the multipliers for each pixel,
and the final value is applied as a new value for the central pixel. The values for the
other pixels in the group are not changed. Next the filter is repositioned to have a
different central pixel, and asserted again just as above, to determine a new value for the
new center pixel, which may be a pixel adjacent to the pixel just previously altered. In
15 this way new values are determined for almost all pixels in the image Edge pixels may be
unchanged because of the geometry of the filter, and may be treated separately, such as
leaving with the original value, which in practice has little if any noticeable effect on the
enhanced image.
In the art of image enhancement, an important consideration is presenting
20 relatively small changes in an image to a person for determination of improvement,
because the optimum sharpness is a matter of opinion and viewing conditions. If changes
in an image presented to a person are quite large, it is difficult and time consuming to
select a preferred image. It is a good idea, therefore, to be able to present enhanced
images such that a newly enhanced image differs from an original or a previous
25 enhancement in what the present inventor chooses to call a "just appreciable visual
sharpness difference" (JA VSD).
An historic problem with convolution filters for enhancement of apparent
sharpness, is that the process is computation intensive, requiring in many cases
substantial computer power. Small filters are possible (minimum number of pixels
- 3 -
typically nine), and can be defined so the divisor for the last step is one, but the effect of
processing an image with such a minimal-computation filter is typically a very large and
unacceptable change in sharpness, far beyond what one might consider a JA VSD. To get
a small appreciable variation in sharpness, typically a JAVSD, larger filters with much
5 more computational power required have to be used. This is impractical for very large
images (many megapixels), or for cameras, iPods, cell phones, and other devices limited
in computational power.
What is critically needed in the art of image enhancement is a solution in which
minimal-computation filters may be used, and at the same time enhanced images may
10 still be presented to a user in just-appreciable visual differences. Also needed is a way to
process very large, high-pixel density (high resolution) with a minimum of computational
power, therefore in essentially real time. The present invention provides this much
needed solution.
15
BRIEF SUMMARY OF THE INVENTION
The inventor in the present case has considerable experience in image
enhancement technology, and has been less than satisfied with the time and computing
20 power necessary to enhance images visually, especially in the attribute of sharpness, as
known in the art. As a consequence, the inventor has developed a unique system and
process that accomplishes the desired end with a minimum in time and computing power.
In this invention, in one embodiment, a method for enhancing sharpness for a digital
image is provided, comprising the steps of ( a) in a display of a computerized appliance,
25 selecting an image to be enhanced in sharpness; (b) downsizing the selected image by a
standard downsizing algorithm executing on the computerized appliance to produce an
image 0 at resolution substantially less than resolution of the original image selected in
step (a); (c) applying a convolution filter to image 0 to produce an image n with enhanced
sharpness, where n is an integer; (d) subtracting pixel values for pixels of image n from
- 4 -
corresponding pixels for image 0, saving the differences; (e) dividing the differences in
step (d) by integer n, and saving the quotients; (f) adding the quotients from step (e) to
values for corresponding pixels in image 0 to produce an image 1, then to values of pixels
for image 1 to produce an image 2, and repeating until an image n-l is produced; (g)
5 presenting images 0 through n to a user for selection of a best image for sharpness; and
(h) upsizing the user-selected image by a standard upsizing algorithm back to the
resolution of the image selected in step (a).
In one embodiment the convolution filter is a 3 x 3 filter with multipliers of -1 at
all cells but the center cell. Also in one embodiment multiplier at the center cell is 9,
10 producing a divisor of 1 for application of the filter. In some embodiments n = 10 or
greater.
In another aspect of the invention a system for enhancing sharpness for a digital
image is provided, comprising a computerized appliance having a digital display and
executing software from a machine-readable medium, the software providing a
15 mechanism enabling a user to select an image to be enhanced, a downsizing algorithm
enabling the user to downsize the selected image to a resolution substantially less than the
than resolution of the original image selected, a convolution filter and functions for
applying the convolution filter to stored images to produce images enhanced for
sharpness, and an up sizing algorithm enabling the user to upsize an image to a higher
20 resolution. The user selects an image to be enhanced in sharpness, the image is
downsized to produce an image 0 at resolution substantially less than resolution of the
original image selected, the convolution filter is applied to image 0 to produce an image n
with enhanced sharpness, where n is an integer, the pixel values for pixels of image n are
subtracted from corresponding pixels for image 0, saving the differences, the differences
25 are divided by n, saving the quotients, the quotients are added back to the pixel values for
image 0 to produce an image 1, and the process is repeated adding the quotients to pixel
values of image 1 to produce an image 2, and so forth, until an image n-l is produced,
then images 0 through n are presented to the user for selection of a best image for
- 5 -
sharpness, then the selected image is upsized back to the resolution of the original image
selected to be enhanced in sharpness.
In one embodiment of the system the convolution filter is a 3 x 3 filter with
multipliers of -1 at all cells but the center cell. Also in one embodiment the multiplier at
5 the center cell is 9, producing a divisor of 1 for application of the filter. In some
embodiments n = 10 or greater.
In another aspect of the invention a method for enhancing sharpness for a digital
image is provided comprising the steps of ( a) in a display of a computerized appliance,
selecting an image to be enhanced in sharpness; (b) downsizing the selected image by a
10 downsizing algorithm executing on the computerized appliance to produce an image 0 at
resolution substantially less than resolution of the original image selected in step (a); (c)
applying a convolution filter to image 0 to produce an image n with enhanced sharpness,
where n is an integer; (d) subtracting pixel values for pixels of image n from
corresponding pixels for image 0, saving the differences; (e) dividing the differences in
15 step (d) by integer n, and saving the quotients; (f) adding the quotients from step (e) to
values for corresponding pixels in image 0 to produce an image 1; (g) displaying image 1
to a user and asking for approval; (h) in case of no approval at step (g), adding the
quotients from step (e) to the pixel values for image 1 to produce an image 2; (i)
repeating building new images by process of steps (g) and (h) until the user selects one as
20 best image; and (j) up sizing the user-selected image by an upsizing algorithm back to the
resolution of the image selected in step (a).
In one embodiment of this method the convolution filter is a 3 x 3 filter with
multipliers of -1 at all cells but the center cell. Also in one embodiment the multiplier at
the center cell is 9, producing a divisor of 1 for application of the filter. In some cases n
25 = 10 or greater.
In yet another aspect of the invention a system for enhancing sharpness for a
digital image is provided, comprising a computerized appliance executing software from
a machine-readable medium, the software providing a mechanism enabling a user to
select an image to be enhanced, a downsizing algorithm enabling the user to downsize
5
- 6 -
the selected image to a resolution substantially less than the than resolution of the original
image selected, a convolution filter and controls for applying the convolution filter to
stored images to produce images enhanced for sharpness, and an up sizing algorithm
enabling the user to upsize an image to a higher resolution.
The user selects an image to be enhanced in sharpness, the image is downsized to
produce an image ° at resolution substantially less than resolution of the original image
selected, the convolution filter is applied to image ° to produce an image n with enhanced
sharpness, where n is an integer, the pixel values for pixels of image n are subtracted
from corresponding pixels for image 0, saving the differences, the differences are divided
10 by n, saving the quotients, the quotients are added back to the pixel values for image ° to
produce an image 1, image 1 is displayed to the user to approve or not as a best image for
sharpness, in the case of no approval the saved quotients are added to the pixel values of
image 1 to produce an image 2, which is displayed to the user for approval, and the
process is repeated until the user selects an image as the best image for sharpness, then
15 the selected image is upsized back to the resolution of the original image selected to be
enhanced in sharpness.
In one embodiment the convolution filter is a 3 x 3 filter with multipliers of -1 at
all cells but the center cell. Also in one embodiment the multiplier at the center cell is 9,
producing a divisor of 1 for application of the filter. In some embodiment n = 10 or
20 greater.
In still another aspect of the invention a method for producing a sequence of
images enhanced for sharpness is provided, comprising the steps of ( a) selecting an image
to be enhanced as image 0, (b) applying a convolution filter to image ° to produce an
image n with enhanced sharpness, where n is an integer, (c) subtracting pixel values for
25 pixels of image n from corresponding pixels for image 0, saving the differences, (d)
dividing the differences in step (c) by integer n, and saving the quotients, and (e) adding
the quotients from step (d) to values for corresponding pixels in image ° to produce an
image 1, then to values of pixels for image 1 to produce an image 2, and repeating until
an image n-l is produced.
5
- 7 -
In one embodiment there is a further step for presenting the images as a sequence
of images to a user for selection of one of the images as a best image for sharpness.
BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS
Fig. 1 Fig. 1 is a representation of geometry and placement of a 3 x 3 convolution
filter 102 in the art.
Fig. 2 is an enlarged view of the filter of Fig. 1, with associated values, to
10 illustrate the computational procedure for asserting the filter at one position.
Fig. 3 is a representation of how the filter of Fig. 1 might be moved over an image
to produce heightened sharpness for the entire image.
Fig. 4 Fig. 4 illustrates a computerized appliance having Internet connection via a
wireless network that communicates with a station, thence through a gateway to the
15 Internet backbone, which represents all of the network interconnections in the Internet
network.
Fig. 5 illustrates a process for selectively enhancing sharpness of relatively large
images in a minimum amount of time, using devices of limited computational power.
Fig. 6 is a process flow diagram (flow chart) illustrating steps in a process
20 according to a preferred embodiment of the present invention.
Fig. 7 is a flow diagram illustrating a process for batch processing of similar
images in an embodiment of the present invention.
Fig. 8 is a diagram for use in preferential sharpening in different segments of an
image in an embodiment of the present invention.
25 Fig. 9 is a flow diagram depicting a process for preferential sharpening in
segments based on local pixel value averages indicating relative lightness or darkness in
the image in the local vicinity.
- 8 -
DETAILED DESCRIPTION OF THE INVENTION
Convolution filters are spatial filters. Spatial filtering is the filtering of an image
in the spatial domain. That is, the value of each pixel of the image is modified in
5 contextual relationship to values of neighboring pixels. Consider, for example, a digital
image of 320 rows and 480 columns, having a total of 1.536 x 105 pixels. The smallest
grouping of pixels which associates one pixel with all of its nearest neighbors is typically
a 3 x 3 matrix of nine pixels, in which a central pixel is seen surrounded by its eight
nearest neighbors.
10
15
Fig. 1 is a representation of geometry and placement of a 3 x 3 convolution filter
102 in the art, centered on a pixel 103 in the upper left comer of a portion of a pixilated
image 101. The rectangular geometry of filter 102 is seen to relate a central pixel 103 to
its eight surrounding closest neighbors. Only a small number of pixels at exaggerated
spacing distance is shown for image portion 101.
Fig. 2 is an enlarged view of the filter of Fig. 1, with associated values, to
illustrate the computational procedure for asserting the filter at one position. In this
representation each of the nine pixels associated in the 3 x 3 filter pattern has been given
a lower-case letter, a through i. Pixel e in this case is the center pixel 103 for which the
value will be changed in one assertion of the filter. Each of the nine positions in the
20 pattern has also been associated with a signed multiplier, which is +x for pixel e (103),
and -1 for each of the surrounding pixels. These numbers are multipliers in an algorithm
associated with the filter. Theoretically all of the multipliers may have unique values, but
for reasons of efficacy and by experience -1 is an appropriate choice for the eight
surrounding pixels. This multiplier is not the original pixel value at each position, but a
25 multiplier to be applied to the pixel value at that position each time the filter is asserted.
The procedure for the filter we are considering operates as follows:
(1) Determine the algebraic sum of the multipliers. This number is set aside as a divisor
for step (3) below. In the example of Fig. 2 this divisor is -8+x.
- 9 -
(2) Multiply each pixel value by the assigned multiplier and take the algebraic sum of the
results (-a-b-c-d+xe-f-g-h-i)
(3) Divide this result by the divisor determined in step (1) above.
(4) Replace the pixel value e by this new pixel value
5 (5) Move the filter to determine a new pixel value for another center pixel.
Fig. 3 is a representation of how the filter of Fig. 1 might be moved over an image
to produce heightened sharpness for the entire image. First the filter is applied at the
upper left comer of the image (A), over a center pixel that is the second pixel in the
second row. The filter procedure is applied for that center pixel, then the filter is moved
10 one pixel distance to the right (B), and applied to change the value for the third pixel in
the second row. This move and calculate procedure is repeated for all the pixels in the
second row, then the filter returns to the second column centered on the second pixel in
the third row (C). Then the filter is moved sequentially one pixel at a time through the
third row. This row-by-row procedure continues until all new pixel values are
15 determined that can be determined given the geometry of the filter. Now a new image is
stored that has enhanced sharpness compared to the original image.
The skilled artisan will recognize, of course, that the concept of a 3 x 3 filter with
multipliers assigned to each cell in the filter is just a convenient concept. What actually
happens is that a software routine, executing from a machine-readable medium coupled
20 to a computer appliance, consults a mapping of values in memory for an original image
that is to be modified by the algorithm, selects the appropriate values that are associated
with nine adjacent pixels, performs the steps of the algorithm, stores the new center pixel
value in memory for a new image, and then selects a new mapping of nine adjacent pixels
(moves the filter). If the pixel groups are selected in a manner that every pixel that may
25 be the center pixel of a 3 x 3 mapping, then new values will be determined a stored for all
pixels in the original image, except edge pixels.
Also a part of the algorithm is a step dictating that if a new value for a pixel is
determined to be zero or less than zero, zero is used; and if a new value is determined that
- 10 -
is greater than the maximum allowed (255 for an eight-bit computing machine), then 255
is used.
Given the procedure above for applying the convolution filter, it will be apparent
to the skilled person that in step (1), if one chooses 9 for the multiplier in the center cell
5 (x), than the divisor for step (3) is 1, and in effect step (3) may be skipped; a bonus in
computation efficiency.
It is rather well-known in the art that a 3 x 3 convolution filter with -1 as a
multiplier for the surrounding pixels is an appropriate choice to minimize computation
intensity, but to provide differences in sharpness in an altered image from an original that
10 is not too extreme, it is necessary to use a multiplier x that will require a division step
with a divisor greater than 1. Using 9 for x generates an amended image that has
dramatic enhanced sharpness. To produce an image enhanced for sharpness by a
JA VSD, it is necessary to use a much larger number for x. In practice it is seen that x
needs to be about 18, producing a divisor of 10, to produce a sharpness-enhanced image
15 at JAVSD. A typical user, however, will not be satisfied with viewing just a first
enhanced image. A user will typically want to see images enhanced step-by-step, until it
is apparent the image is too sharp. Then the user may back down to the just previous
image as the best choice.
To accomplish this in the art, assuming that x=18 produces a JAVSD, requires
20 that an enhanced image be produced by applying the filer with x= 18 (divisor 10) at all
positions that can be attained. Then a second enhanced image is produced with x= 17
(divisor 9), and so on (x = 16, 15, 14 ... ), until the user discovers the new image is too
sharp. This may require four or more image iterations with a new divisor greater than 1
for each. The skilled person will understand that the original image needs to be saved,
25 and each enhanced image produced from the original or from a previously enhanced
image also needs to be saved, and functionality needs to be provided for the user to select
anyone of the images as the preferred image for sharpness.
- 11 -
The process described above for producing and displaying enhanced images to a
viewer for selection to produce an image with preferred sharpness, is still quite
computationally intensive.
Fig. 4 illustrates a computerized appliance 401 having Internet connection via a
5 wireless network that communicates with a station 402, thence through a gateway 403 to
Internet backbone 405, which represents all of the network interconnections in the
Internet network. Two Internet-connected servers 406 and 407 are shown representing
all of the sites in the Internet network which may serve information and data to internet
connected appliances like appliance 401. Appliance 401 may be a cellular telephone, a
10 personal digital assistant, or any other Internet connectable appliance. In some cases
appliance 401 may be a laptop or desktop computer, or an iPad device. Internet
connection is represented in Fig. 4 as a means by which appliance 401 may receive
images, however, in some embodiments of the invention there may be no Internet
connection, and images may be loaded to the appliance by any other known data transfer
15 technique.
The skilled person will understand that appliance 401 will have a CPU and a
display, and will be capable of executing software 408 stored in local memory, without
this specification detailing the well-known components used in computerized appliances
for displaying images, and for executing software that may alter pixel values and display
20 altered images from original images stored in memory coupled to the device.
Fig. 5 illustrates a process for selectively enhancing sharpness of relatively large
images in a minimum amount of time, using devices of limited computational power. It
is well-known that the tendency in the art, due in part to the ever-descending cost of
memory and greater resolution in displays, is to images of higher resolution. Given the
25 descriptions above regarding the computational intensity of sharpness enhancement,
images of higher and higher resolution present an ever bigger problem in time and
computing power. There are, however, quite good downsizing algorithms available to
render high-resolution images at lower resolution. One site that deals with this issue is
http-.~L-:~:}Y~Jl1~p-l1Q.!Q.t}!!i~h~~::u~Q!nLl~11l:gg~Y1l:p-;;.i?;~,.htm .
- 12 -
So a first step in sharpness enhancement in an embodiment of the present
invention is to downsize the image desired to be enhanced. In Fig. 5 image 501 is an
image for which a user desires enhanced sharpness. This image is represented as 2048 x
1536, which is 3,145,728, or 3.146 mega-pixels. This particular size is used only for
5 exemplary purpose, and could be any image of high-resolution. A first step is
downsizing this image to a lower resolution, using a commercially available downsizing
algorithm.
In this example image 501 is downsized to 480 x 320, or 153,600 pixels, about
5% of the number of pixels in the larger image. The skilled person will recognize that the
10 visual quality of downsized image 502 will be essentially the same as the image 501, as
long as the display is presented without too much magnification. The smaller image will
be quite satisfactory for a user to make judgments as to the quality of sharpness. A very
big advantage is that application of a convolution filter to the smaller image will have to
deal with only one pixel in twenty of the larger image, and can operate either twenty
15 times quicker, or with far less computing power in the same time frame.
The next step in this unique process is running a minimal-computation
convolution filter over the smaller image in several steps to create a series of altered
images with just-appreciable visual difference from one image to the next, to create a
series of enhanced images 503, 504, 505 506. There is a unique difference in the way
20 this is done in this example than in the prior art. In this embodiment the 3 x 3 filter with
surrounding multipliers of -1 and x = 9 is used. this provides for the minimum
computation, because the divisor for step (3) on page 5 above will be 1, which allows us
to skip that step.
The result of the single pass of the minimal convolution filter, however is that the
25 sharpness change will be quite dramatic, as described previously above, beyond what
most users would select as a desirable improvement. In this embodiment this is handled
in a unique way. The pixel vales for the original image are saved as image 0, and the first
enhanced image is treated as image 10. Now our system takes, for each pixel, the
difference between the pixel value for image 10 and the pixel value for image 0, and
- 13 -
divides by ten. These differences are algebraically added back to image 0 to produce
image 1, an image with a just appreciable visual difference in sharpness from image O.
Adding the differences to the pixel values for image 1 produces image 2, an image with a
JAVSD from image 1, and a greater difference in sharpness from image O. The process is
5 repeated through image 9. Image 10 already exists as the result of applying the
convolution filter to image O.
We now have a series of ten images, each differing from its immediate neighbors
by JA VSD. A user may easily select the image that appears to be the best (in the eye of
the beholder) for sharpness.
10 It is not required that there be ten iterations. There may be five, or six, or four;
but there needs be several, so the user has a selection of several images from which to
choose. If the selection is too sparse, the best image to the user might well be between
two of the iterations presented. That is, one will appear to the user to be not sharp
enough, and an adjacent iteration will appear too sharp.
15 Once the "best" image is chosen by the user, it is needed to provide that result to
the larger resolution, which in this example is 2048 x 1536. In the prior art the process
would dictate that the filter be applied to the larger image. But in this embodiment of the
invention the best smaller image is simply upsized by a commercially-available algorithm
that has been determined to be appropriate. The result has been shown by the inventor to
20 be equal in quality to the prior art method of applying more computation-intensive filters
to the larger image, a process perhaps requiring orders of magnitude more power and
time.
Fig. 6 is a process flow diagram (flow chart) illustrating steps in a process
according to a preferred embodiment of the present invention, much as described above.
25 At step 601 an image is selected to be enhanced for sharpness. At step 602 the image is
downsized by applying a commercially-available downsizing algorithm, to a resolution
substantially less than the resolution of the original image selected in step 601, providing
an image 0 (502 in Fig. 5). At step 603 a convolution filter is applied to the reduced
resolution image to form a first image with enhanced sharpness (image 1 - 503 in Fig. 5).
- 14 -
At step 604 the algebraic difference in value between the pixel values for image 1 and the
original downsized image is determined. This algebraic difference is divided by an
integer n in step 605. At step 606 the quotient for each pixel is added to the pixel value
for image "0" to produce a second enhanced image 2. At step 607 the same quotient for
5 each pixel from step 605 is added to the pixels of image 2 to produce a third enhanced
image 3. Step 707 is repeated to add values to pixels of image 3, and so on, until an
image n-l is produced. At this point there are n enhanced images (0 to n), each
displaying a JA VSD with the one before. These may be displayed to a user, preferably in
order, and the user is invited to select the image judged to be the best for sharpness. This
10 is then upsized in the final act back to the original resolution of the larger image that was
first considered to be enhanced for sharpness. This final image may be saved.
The skilled person will recognize that the embodiments described herein may be
altered in several ways within the scope of the invention. The size of the "larger" image
is not a fixed value, for example, but can be anyone of a wide variety of resolutions. The
15 downsizing and up sizing algorithms are not fixed, but may be chosen from a variety of
readily-available and well-known algorithms. The size of the "smaller" image is not
fixed either, but may vary over a wide range. The smaller image is preferably
considerably smaller than the larger to effectively limit the number of pixels necessary to
recalculate in filter application. The number of iterations from image 0 to a final image,
20 each of which is produced by a single pass of the filter, is also not fixed, but is preferably
at least four, and more preferably eight or ten.
Another variation in the process might involve producing one alteration at a time,
and allowing the user to judge the new image before going on to a next. For example, the
system might present the first alteration to the user and wait for a signal to produce the
25 next, and then wait for a signal to produce another. The user may have access as well to
a "back" command, and to a command to compare the image with the original, so when
an image is presented that is slightly too sharp, the back command will revert to the just
previous image, and the user may then cause that image to be selected and upsized to the
original resolution. There are many similar possibilities.
- 15 -
Batch Processing
In another aspect of the present invention it may be desirable to do batch
processing, that is, to apply the process described above in different examples to a
5 plurality of digital images. A user may have, for example, a plurality of images of very
similar characteristics, such as a group of images captured by a digital camera in a
relatively short period of time, under similar circumstances of lighting, and without
changing settings on the camera, and displayed on the same monitor, perhaps a computer
display monitor.
10 Referring now to Fig. 6 and to the description of Fig. 6 above, and not considering
the downsizing or upsizing of an image, a process is described wherein a minimal
convolution filter is sequentially applied over the pixels of an image to be enhanced
(image 0), producing pixel values for an image "n". Sharpness is thus enhanced for
image "n", but the change (increase) in sharpness will typically be more than might be
15 desired. In this process the algebraic difference between the value for each original pixel
and the value for the pixel in the same position in the image for image "n" is determined,
and then the difference is divided by an integer. The integer may be theoretically any
integer, but the idea is to produce images between image 0 and image "n", in which there
is just an appreciable visual difference. So this integer value may be 1 0, for example, and
20 10 is set as "n".
The pixel value differences at each pixel position is divided by the integer, then
an image 1 is produced by adding to the pixel values at each pixel position for image 0
one-tenth of the difference between the pixel value for image 0 and the pixel value for the
same pixel in image 10. Similarly an image 2 is produced adding .2 times the difference
25 at each pixel position, an image 3 using .3 times the difference, and so on, producing
images 1 through 9 between image 0 and image 10, each successive image having a
sharpness increase of just an appreciable visual difference. These ten images are
displayed to a user, the user enabled to select the "best" image, that is, the one that seems
to have, for that user, the optimum quality of sharpness.
- 16 -
Assume now that this unique process is followed for one image of a plurality of
images of very similar characteristics by a user, and the user selects image 3 as the
optimum image sharpness. It may be assumed, then, that the image 3 for all of the
plurality of images will be, for this user, the image with the optimum sharpness. It will
5 not be necessary to produce image 1, 2, or 4 through 9 for any of the other images of the
plurality. Having selected the plurality of images and initiated a batch process, the batch
system in this embodiment will produce the ten images for the user, and enable selection
of the "best" image for the user, and then use the image selected (1st, 2nd, 3rd, etc.), to
produce a sharpness-enhanced image for all of the other images of the plurality.
10 Fig. 7 is a process flow diagram for the batch processing process described just
above, in which steps 702 through 708 repeat the process described with reference to Fig.
6 for one image. Step 701 is a first step for selecting the first image to be enhanced from
a plurality of similar images. Step 709 is for noting the number of the image chosen by
the user as the optimum image for sharpness, and step 710 repeats the process for every
15 other image of the plurality, but to produce only the image of the number selected. So it
is only necessary to produce all of the images for selection just once, then all of the other
enhanced images may be produced automatically.
Another example of batch processing in sharpness enhancement is in the area of
video technology. It is well-know that data streams for video are arranged to produce
20 successive frames in display, much in the manner of movie film presenting a rapidly
changing sequence of still images, each slightly altered from the previous. Typically all
of the frames in a video data stream will have very similar sharpness characteristics. If
one selects, then, just one frame, applies the process described above to the one frame,
and a user selects one of the candidate sharpness-enhanced images as the most
25 appropriate, then further processing may be truncated for all the other frames. Assume,
for example, that the process is applied to one frame, providing ten candidate images of
that frame, each with a JA VSD from the immediately preceding image, and the user
selects image four. One may safely assume that image four will be appropriate for all the
other frames of the video as well. and the original image for each frame may then be
5
- 17 -
processed to produce the fourth image, without producing all of the other candidate
Images.
Selective Segmentation in Image Sharpening
The processes described thus far in this specification apply the same process in
sharpness enhancement to every part of an image. The skilled artisan will understand
that in some cases a user may prefer to preferentially enhance sharpness in particular
segments of an image. In one instance, for example, a user may prefer to sharpen an
image preferentially in segments that are in shadow. In another instance a user may
10 prefer to sharpen an image preferentially in segments that are highlighted, that is in
brightness. In yet another instance a user may prefer to sharpen an image preferentially
in segments that are in midrange of brightness. Other similar preferences are possible.
Therefore, in another aspect of the present invention, a process is provided that
treats different segments of an image differently, according the local brightness
15 characteristics. Fig. 8 is a diagram that relates average local pixel value from 0 to 255
(assuming an eight-bit display system) for brightness to a vertical scale between 0 (at
origin) and 1. In Fig. 8 there is a straight line labeled S for use in a process to sharpen
preferentially in segments that are in shadow. A straight line labeled H is for use in a
process to sharpen preferentially in segments of an image highlighted. A curve M is for
20 use in a process to sharpen preferentially in midtone segments.
Assume for a first example that a user wants to sharpen preferentially in segments
that are in shadow. The process is very similar to that described above as conventional
art for producing an image n from an original image 0, in which a convolution filter is
applied sequentially to values for pixels of image 0 to produce image n. In this new and
25 non-conventional procedure, however, at each application of the filter, in addition to
producing a new pixel value for image n, the pixel values for each cell of the filter (nine
cells for a three by three filter) are added and divided by the number of the cells,
rendering a pixel value average for the cells in the vicinity of the center cell being altered
- 18 -
in value by the filter protocol. This average pixel value expresses the nature of the
segment in which the object pixel resides; that is, light, dark or midtone.
Now, in this preferential process for sharpening in shadow, the system utilizes the
graph of Fig. 8, and line S. Note that for a local pixel value average of zero (fully dark,
5 extreme shadow) the graph expresses 1, and for a local pixel value average of255 (fully
bright) the graph expresses zero, and for local averages in between, the value from the
graph is between 0 and 1 proportionally.
At every position for application of the filter the local average pixel value is
determined, and the new pixel value determined by the filter protocol is used to
10 determine the pixel value difference for that pixel position, and the difference is
multiplied by the value from the graph and added back for the new pixel value.
It will be apparent to the skilled person that this procedure, using the line S, will
preferentially sharpen in segments that a dark, and will sharpen less in segments that are
more light.
15 Ifit is desired to sharpen preferentially in areas that are light, then in the process
the line H is used from the graph of Fig. 8, and sharpness will be preferential for light
segments, and less for dark segments. If it is desired to sharpen in mid-tone segments,
the curve M will be used, so maximum effect will accrue for local averages near midtone
(halfway between 0 and 255), and there will be little effect near deep shadow or extreme
20 brightness, and the effect will demonstrate uniform differences, because the curves are
well-behaved.
It will be apparent to the skilled person that a graph may be created for just about
any segment enhancement. M may be inverted, for example, to sharpen preferentially in
both deep shadow or extreme brightness, but not at all at mid-tone. The skilled person
25 will also understand that the graph of Fig. 8 is used to illustrate the process, but that in
practice values will be picked from tables relating local pixel value averages to decimal
fractions between 0 and 1.
Fig. 9 is a flow diagram depicting the process just described for preferential
sharpening in segments based on local pixel values averages indicating relative lightness
- 19 -
or darkness in the image in the local vicinity. At step 901 the convolution filter is applied
for a first pixel in the original image "0". At step 902 the algebraic difference between
the original pixel value at the first position and the new pixel value is determined. At
step 903 a multiplier is selected according to the graph using the relationship for a
5 segment preference. At step 904 the difference from step 902 is multiplied by the
multiplier from step 903. At step 905 the result of step 904 is added back to the original
pixel value. At step 906 the result of step 905 is saved as the new pixel value for image
"n". At step 907 the filter is moved to a new position and the process is repeated until
new pixel values are determined for all of the pixel positions.
1 0 It will be apparent to the skilled person that there are many alterations that might
15
be made to embodiments described as examples herein, all within the scope of the
invention, which is limited only by the claims that follow.
5
10
15
- 20-
CLAIMS
1. A method for enhancing sharpness for a digital image, comprising the steps of:
(a) in a display of a computerized appliance, selecting an image to be enhanced in
sharpness;
(b) downsizing the selected image by a downsizing algorithm executing on the
computerized appliance to produce an image 0 at resolution substantially less than
resolution of the original image selected in step (a);
(c) applying a convolution filter to image 0 to produce an image n with enhanced
sharpness, where n is an integer;
(d) subtracting pixel values for pixels of image n from corresponding pixels for
image 0, saving the differences;
(e) dividing the differences in step (d) by integer n, and saving the quotients;
(f) adding the quotients from step (e) to values for corresponding pixels in image
o to produce an image 1, then to values of pixels for image 1 to produce an image 2, and
repeating until an image n-l is produced;
(g) presenting images 0 through n to a user for selection of a best image for
sharpness; and
20 (h) upsizing the user-selected image by an up sizing algorithm back to the
25
resolution of the image selected in step (a).
2. The method of claim 1 wherein the convolution filter is a 3 x 3 filter with multipliers
of -1 at all cells but the center cell.
3. The method of claim 2 wherein the multiplier at the center cell is 9, producing a
divisor of 1 for application of the filter.
4. The method of claim 1 wherein n = 10.
5
10
- 21 -
5. A system for enhancing sharpness for a digital image, comprising:
a computerized appliance having a digital display and executing software from a
machine-readable medium, the software providing:
a mechanism enabling a user to select an image to be enhanced;
a downsizing algorithm enabling the user to downsize the selected image to a
resolution substantially less than the than resolution of the original image selected;
a convolution filter and functions for applying the convolution filter to stored
images to produce images enhanced for sharpness; and
an upsizing algorithm enabling the user to upsize an image to a higher resolution;
wherein the user selects an image to be enhanced in sharpness, the image is
downsized to produce an image 0 at resolution substantially less than resolution of the
original image selected, the convolution filter is applied to image 0 to produce an image n
with enhanced sharpness, where n is an integer, the pixel values for pixels of image n are
15 subtracted from corresponding pixels for image 0, saving the differences, the differences
are divided by n, saving the quotients, the quotients are added back to the pixel values for
image 0 to produce an image 1, and the process is repeated adding the quotients to pixel
values of image 1 to produce an image 2, and so forth, until an image n-l is produced,
then images 0 through n are presented to the user for selection of a best image for
20 sharpness, then the selected image is upsized back to the resolution of the original image
selected to be enhanced in sharpness.
25
6. The system of claim 5 wherein the convolution filter is a 3 x 3 filter with multipliers
of -1 at all cells but the center cell.
7. The system of claim 6 wherein the multiplier at the center cell is 9, producing a
divisor of 1 for application of the filter.
8. The system of claim 1 wherein n = 10.
- 22-
9. A method for enhancing sharpness for a digital image, comprising the steps of:
(a) in a display of a computerized appliance, selecting an image to be enhanced in
sharpness;
5 (b) downsizing the selected image by a downsizing algorithm executing on the
computerized appliance to produce an image 0 at resolution substantially less than
resolution of the original image selected in step (a);
(c) applying a convolution filter to image 0 to produce an image n with enhanced
sharpness, where n is an integer;
10 (d) subtracting pixel values for pixels of image n from corresponding pixels for
image 0, saving the differences;
(e) dividing the differences in step (d) by integer n, and saving the quotients;
(f) adding the quotients from step (e) to values for corresponding pixels in image
o to produce an image 1;
15 (g) displaying image 1 to a user and asking for approval;
(h) in case of no approval at step (g), adding the quotients from step (e) to the
pixel values for image 1 to produce an image 2;
(i) repeating building new images by process of steps (g) and (h) until the user
selects one as best image; and
20 (j) upsizing the user-selected image by an up sizing algorithm back to the
25
resolution of the image selected in step (a).
10. The method of claim 9 wherein the convolution filter is a 3 x 3 filter with multipliers
of -1 at all cells but the center cell.
11. The method of claim 10 wherein the multiplier at the center cell is 9, producing a
divisor of 1 for application of the filter.
12. The method of claim 9 wherein n = 10.
5
10
- 23 -
13. A system for enhancing sharpness for a digital image, comprising:
a computerized appliance executing software from a machine-readable medium,
the software providing:
a mechanism enabling a user to select an image to be enhanced;
a downsizing algorithm enabling the user to downsize the selected image to a
resolution substantially less than the than resolution of the original image selected;
a convolution filter and controls for applying the convolution filter to stored
images to produce images enhanced for sharpness; and
an upsizing algorithm enabling the user to upsize an image to a higher resolution;
wherein the user selects an image to be enhanced in sharpness, the image is
downsized to produce an image 0 at resolution substantially less than resolution of the
original image selected, the convolution filter is applied to image 0 to produce an image n
with enhanced sharpness, where n is an integer, the pixel values for pixels of image n are
15 subtracted from corresponding pixels for image 0, saving the differences, the differences
are divided by n, saving the quotients, the quotients are added back to the pixel values for
image 0 to produce an image 1, image 1 is displayed to the user to approve or not as a
best image for sharpness, in the case of no approval the saved quotients are added to the
pixel values of image 1 to produce an image 2, which is displayed to the user for
20 approval, and the process is repeated until the user selects an image as the best image for
sharpness, then the selected image is upsized back to the resolution of the original image
selected to be enhanced in sharpness.
14. The system of claim 13 wherein the convolution filter is a 3 x 3 filter with multipliers
25 of -1 at all cells but the center cell.
15. The system of claim 14 wherein the multiplier at the center cell is 9, producing a
divisor of 1 for application of the filter.
5
- 24-
16. The system of claim 13 wherein n = 10.
17. A method for producing a sequence of images enhanced for sharpness, comprising
the steps of:
(a) selecting an image to be enhanced as image 0;
(b) applying a convolution filter to image 0 to produce an image n with enhanced
sharpness, where n is an integer;
(c) subtracting pixel values for pixels of image n from corresponding pixels for
image 0, saving the differences;
10 (d) dividing the differences in step (c) by integer n, and saving the quotients; and
(e) adding the quotients from step (d) to values for corresponding pixels in image
o to produce an image 1, then to values of pixels for image 1 to produce an image 2, and
repeating until an image n-l is produced.
15 18. The method of claim 17 comprising a further step for presenting the images as a
sequence of images to a user for selection of one of the images as a best image for
sharpness.
20
- 25 -
ABSTRACT OF THE DISCLOSURE
A method for enhancing sharpness for a digital image follows this sequence: (a)
in a display of a computerized appliance, selecting an image to be enhanced in
5 sharpness; (b) downsizing the selected image by a downsizing algorithm executing on the
computerized appliance to produce an image 0 at resolution substantially less than
resolution of the original image selected in step (a); (c) applying a convolution filter to
image 0 to produce an image n with enhanced sharpness, where n is an integer; (d)
subtracting pixel values for pixels of image n from corresponding pixels for image 0,
10 saving the differences; (e) dividing the differences in step (d) by integer n, and saving the
quotients; (f) adding the quotients from step (e) to values for corresponding pixels in
image 0 to produce an image 1, then to values of pixels for image 1 to produce an image
2, and repeating until an image n-l is produced; (g) presenting images 0 through n to a
user for selection of a best image for sharpness; and (h) upsizing the user-selected image
15 by an upsizing algorithm back to the resolution of the image selected in step (a).
Electronic Patent Application Fee Transmittal
Application Number:
Filing Date:
Title of Invention: Sharpness in Digital Images
First Named Inventor/Applicant Name: Rodney Shaw
Filer: Donald Rex Boys/Sheri Beasley
Attorney Docket Number: P882
Filed as Small Entity
Utility under 35 USC 111 (a) Filing Fees
Description Fee Code Quantity Amount Sub-Total in
USD($)
Basic Filing:
Utility filing Fee (Electronic filing) 4011 1 82 82
Utility Search Fee 2111 1 270 270
Utility Examination Fee 2311 1 110 110
Pages:
Claims:
Independent claims in excess of 3 2201 2 110 220
Miscellaneous-Filing:
Petition:
Description Fee Code Quantity Amount Sub-Total in
USD($)
Patent-Appeals-and-Interference:
Post-Allowance-and-Post-Issuance:
Extension-of-Time:
Miscellaneous:
Total in USD ($) 682
Electronic Acknowledgement Receipt
EFSID: 8661904
Application Number: 12908161
International Application Number:
Confirmation Number: 3367
Title of Invention: Sharpness in Digital Images
First Named Inventor/Applicant Name: Rodney Shaw
Customer Number: 24739
Filer: Donald Rex Boys/Sheri Beasley
Filer Authorized By: Donald Rex Boys
Attorney Docket Number: P882
Receipt Date: 20-0CT-2010
Filing Date:
TimeStamp: 12:59:04
Application Type: Utility under 35 USC 111 (a)
Payment information:
Submitted with Payment yes
Payment Type Deposit Account
Payment was successfully received in RAM $682
RAM confirmation Number 9417
Deposit Account 500534
Authorized User
File Listing:
Document I Document Description I
File Name I
File Size(Bytes)/ I Multi ,I Pages Number Message Digest Part /.zip (ifappl.)
1347662
1 Application Data Sheet US_ADSJorm_SB_14a.pdf no 4 c7971 e9fOfaOc83791fddOc026d40016164d
bSf1
Warnings:
Information:
63858
2 Oath or Declaration filed POA882.pdf no 2 Ob9fa 1621 fd40468c987f14e3de664f2fd62
eOO
Warnings:
Information:
317943
3 Drawings-only black and white line
Drawings.pdf no 9 drawings
8e 1 b2 6b8a803 6ac4d Sa 7 5 03 bfOa 75 21289f 01321
Warnings:
Information:
156533
4 P882PA.pdf yes 25 44e37a7971 ed4ccc69ae201 dgec3923aa61
07078
Multipart Description/PDF files in .zip description
Document Description Start End
Specification 1 19
Claims 20 24
Abstract 25 25
Warnings:
Information:
35898
5 Fee Worksheet (PTO-875) fee-info.pdf no 2 40b9743f03edfb668cbf343835eaed938a7d
cf16
Warnings:
Information:
Total Files Size (in bytes) 1921894
This Acknowledgement Receipt evidences receipt on the noted date by the USPTO of the indicated documents, characterized by the applicant, and including page counts, where applicable. It serves as evidence of receipt similar to a Post Card, as described in MPEP 503.
New Applications Under 35 U.S.c. 111 If a new application is being filed and the application includes the necessary components for a filing date (see 37 CFR 1.53(b)-(d) and MPEP 506), a Filing Receipt (37 CFR 1.54) will be issued in due course and the date shown on this Acknowledgement Receipt will establish the filing date of the application.
National Stage of an International Application under 35 U.S.c. 371 If a timely submission to enter the national stage of an international application is compliant with the conditions of 35 U.S.c. 371 and other applicable requirements a Form PCT/DO/EO/903 indicating acceptance of the application as a national stage submission under 35 U.S.c. 371 will be issued in addition to the Filing Receipt, in due course.
New International Application Filed with the USPTO as a Receiving Office If a new international application is being filed and the international application includes the necessary components for an international filing date (see PCT Article 11 and MPEP 1810), a Notification of the International Application Number and of the International Filing Date (Form PCT/RO/l 05) will be issued in due course, subject to prescriptions concerning national security, and the date shown on this Acknowledgement Receipt will establish the international filing date of the application.
PTO/SB/06 (12-04) Date: 10/20/10 Approved for use through 713112006. OMB 0651-0032
U.S. Patent and Trademark Office; U.S. DEPARTMENT OF COMMERCE Under the Paperwork Reduction Act of 1995 no persons are required to respond to a collection of information unless it displays a valid OMB control number
, PATENT APPLICATION FEE DETERMINATION RECORD Application or Docket Number
Substitute for Form PTO-875 . 12/908,161 APPLICATION AS FILED - PART I OTHER THAN
(Column 1) (Column 2) SMALL ENTITY OR SMALL ENTITY
FOR NUMBER FILED NUMBER EXTRA RATE ($) FEE ($) RATE ($) FEE ($)
BASIC FEE N/A N/A N/A 82 N/A (37 CFR 1.16(a), (b), or(c» SEARCH FEE N/A N/A N/A 270 N/A (37 CFR 1.16(k), (i), or (m» EXAMINATION FEE N/A N/A N/A 110 N/A (37 CFR 1.16(0), (P), or (q» TOTAL CLAIMS 18 X$26= X$52 (37 CFR 1.16(i)) minus 20 = OR INDEPENDENT CLAIMS 5 .. 2 X$110= 220 X$220= (37 CFR 1.16(h» minus 3 =
If the specification and drawings exceed 1 00 APPLICATION SIZE sheets of paper, the application size fee due is FEE $270 ($135 for small entity) for each additional
(37 CFR 1.16(5» 50 sheets or fraction thereof. See 35 U.S.C. 41 (a)(l)(G) and 37 CFR
MULTIPLE DEPENDENT CLAIM PRESENT (37 CFR 1.160» N/A N/A
* If the difference in column 1 is less than zero, enter "0" in column 2. TOTAL 682 TOTAL
APPLICATION AS AMENDED - PART II OTHER THAN
(Column 1) (Column 2) (Column 3) SMALL ENTITY OR SMALL ENTITY
CLAIMS HIGHEST ADDI- ADDI-REMAINING NUMBER PRESENT
c:( AFTER PREVIOUSLY EXTRA RATE ($) TIONAL RATE ($) TIONAL I- AMENDMENT PAID FOR FEE ($) FEE ($) Z w Total
* Minus ** = OR :I!! X X = C (37 CFR 1.16(i)) Z Independent
* Minus *** = X = X = W (37 CFR 1.16(h» OR :I!! c:( Application Size Fee (37 CFR 1.16(s»
FIRST PRESENTATION OF MULTIPLE DEPENDENT CLAIM (37 CFR 1.160)) N/A OR N/A
TOTAL TOTAL ADD'T FEE OR ADD'T FEE
(Column 1) (Column 2) (Column 3) OR
CLAIMS HIGHEST ADDI- ADDI-REMAINING NUMBER PRESENT
III AFTER PREVIOUSLY EXTRA RATE ($) TIONAl RATE ($) TIONAL I- AMENDMENT PAID FOR FEE ($) FEE ($) Z w Total OR :I!! * Minus ** = X = X = C (37 CFR 1.16(i)) Z Inaependent
* *** W Minus = X = X = :I!! (37 CFR 1.16(h)) OR c:( Application Size Fee (37 CFR 1.16(s»
FIRST. PRESENTATION OF MULTIPLE DEPENDENT CLAIM (37 CFR 1.16(j)) N/A OR N/A
TOTAL TOTAL ADD'T FEE OR ADD'T FEE
* If the entry in column 1 is less than the entry in column 2, write 'a' in column 3. *. If the 'Highest Number Previously Paid For" IN THIS SPACE is less than 20, enter '20'. ... If the 'Highest Number Previously Paid For" IN THIS SPACE is less than 3, enter '3' .
The 'Highest Number Previously Paid For" (Total or Independent) is the highest number found in the appropriate box in column 1.
ThiS collection of Information IS reqUIred by 37 CFR 1.16. The information IS reqUIred to obtain or retain a benefit by the public which IS to file (and by the USPTO to process) an application. Confidentiality is governed by 35 U.S.C. 122 and 37 CFR 1.14. This collection is estimated to take 12 minutes to complete, including gathering, preparing, and submitting the completed application form to the USPTO. Time will vary depending upon the individual case. Any comments on the amount of time you require to complete this form and/or suggestions for reducing this burden, should be sent to the Chief Information Officer, U.S. Patent and Trademark Office, U.S. Department of Commerce, P.O. Box 1450, Alexandria, VA 22313-1450. DO NOT SEND FEES OR COMPLETED FORMS TO THIS ADDRESS. SEND TO: Commissioner for Patents, P.O. Box 1450. Alexandria, VA 22313·1450.
If you need assistance in completing the form. call1·80()''PTO·9199 and select option 2.