CS 597

A typical weak paper

• Short and naïve introduction – demonstrates lack of background research and of expertise from the authors

• Methods / algorithms not very original – demonstrates lack of understanding of the state-of-the-art in the field

• Results show operation of system on one example case – lack of systematic study demonstrates laziness and greatly reduces belief that research described is generally applicable

• Short discussion limited to own research rather than putting work into perspective by comparing to previous studies – shows lack of knowledge of others’ work and reduces credibility

CS 597

A typical weak paper: summary

A weak paper is one where the authors describe work that is not very new, is not thoroughly validated, and is not properly placed in perspective with respect to previous work.

Mostly, it is weak because… the authors have been lazy and have not done proper background research.

CS 597

A typical strong paper

• Comprehensive expert introduction – demonstrates extensive background research, mastery of the whole field, understanding of the important issues, and clear positioning of the research as new

• Methods / algorithms are original – demonstrates good understanding of the state-of-the-art in the field, and expertise

• Results include thorough quantitative validation – the authors seriously stand behind their research and make efforts to prove how it is new / different / better

• Discussion puts work into perspective by comparing to previous studies – shows expertise and credibility by not being shy about comparing to other research

CS 597

A typical strong paper: summary

A strong paper is one where the authors demonstrate that they have complete expert understanding of the major open research issues in the field, and where they provide a convincing argument that they just professionally cleaned up one of those issues.

Mostly, it is strong because the authors know what they are doing!

And because they worked extremely hard on their research.

CS 597

Introduction; Originality

• Your paper needs to bring something new;

• But research is nowadays a very active field, and chances are that something very similar to your contribution has already been proposed 20 years ago.

• Again, don’t try to mask that fact, hoping that reviewers will be ignorant. On the contrary, use extensive literature research to frame your work comparatively to previous studies.

• It’s ok to write:• “XXX et al. (1972) have proposed […], an idea which we here

sought to further explore…”• “Our approach is related to that of YYY et al. (1995), …”• “We follow the hypothesis of ZZZ et al. (1938) and investigate …”

CS 597

Discussion

• Don’t try to hide weaknesses; readers and especially reviewers are not naïve and will explicitly ask you about those weaknesses if you don’t appropriately discuss them – in the end, you just waste time.

• It’s ok to have in your discussion statements like:

• “a clear limitation of our approach is that in its present from it cannot …”

• “our study has not addressed the problem of…”• “better methods have been proposed for [component X] of our

system …”

• Just be honest, fair, objective.

CS 597

Video compression for human viewers

• Many optimizations are possible and some are already implemented by modern video codecs:• Texture complexity• Spatial and temporal masking• Importance of motion• Human contrast sensitivity function• … [e.g., Osberger et al., 1997/1998/1999]

• But some remain which are more difficult to implement:• Importance of humans, faces, animals, …• Biological motion• Object size• Main actor vs. secondary actors• Man-made vs. natural objects• …

CS 597

Purpose of the study

To evaluate applicability of a biological model of visual attention to automatic prioritization of video

Systematic study of 64 variants of the modelDemonstration using MPEG-1 and MPEG-4Comparison with human eye movementsValidation using large collection of video clips

Key defining characteristic of the work: do not attempt to develop models for high-priority content (e.g., human faces); instead, develop model of the neurobiology of human visual processing.

CS 597Itti & Koch, Nature Reviews Neurosci 2001

CS 597

Comparison to eye movements

• 50 video clips, 0:06 to 1:30 each, over 25 minutes total• Outdoors day & night, parks, crowds, rooftop bar, etc• TV news, sports, commercials, talk shows, etc• Video games, test stimuli

• Task: “follow main actors and actions, try to understand overall what happens in each clip. We will ask you a question about main contents. Do not worry about details like specific text messages.”

• 8 subjects total, 4 to 6 subjects / clip (4.7 avg)• Normal corrected vision, ages 23-32, 3 females, 5 males, mixed

ethnicities

• ISCAN tracker• 240Hz recording, 9-point calibration every 5 clips• Affine + thin-plate-spline calibration transform [Bookstein, 1989]• 640x480 @ 60.27Hz doublescan, 33.185ms/movie frame

CS 597

CS 597

CS 597

Outdoors scene (easy)

CS 597

CS 597

Video game

CS 597

CS 597

TV, news (worst clip)

CS 597

CS 597

CS 597

50 clips

46,489frames

CS 597

50 clips

46,489frames

4.7 subjects/clip

Average blur at human eye =

1/2 x average blur overentire display

p<0.01 for every clip using continuous blur map, And for 48/50 clips using 3 discrete foveas

CS 597

50 clips

46,489frames

MPEG1: (average) Foveated compressed file size = 51% unfoveated compressed file size

MPEG4: (average) Foveated compressed file size = 46% unfoveated compressed file size

CS 597

Conclusions

• Bottom-up salience based on simple interacting center-surround operators may explain much more than we often expect.

• Video encoding priority predicted by simple salience model highly significantly correlates with human eye fixations, under our specific task.

• Significant compressed file size reductions were achieved by automatic foveation.

CS 597

The paper• June 30, 2003: Submitted to IEEE Transactions on Image Processing

• October 15, 2003: reviews received – 2 liked it a lot, one liked the idea but not the writing. Out of typical editorial verdicts:• Reject• Revise and resubmit / major revision required with re-review• Minor changes required, usually without re-review• Accept as is

ours is “revise and resubmit”

• October 22, 2003: revision submitted along with answers to reviews

• January 20, 2004: 2 reviewers liked the changes and recommend publication as is, one still wants a minor change. Paper status now:“accepted pending minor revision”

• January 22, 2004: second revision submitted with answers• January 24, 2004: paper officially accepted• January 28, 2004: final version of text, separate figures, separate

bibliography, author bio, etc sent to publisher.• June 24, 2004: official acceptance date (what happened Jan-June??)• October, 2004: publication

CS 597

The reviews: first a few points of general feedback

Reviewer 1 Comments:I. REVIEW

Please expand and give details in Section III.

A. Suitability of topic

1. Is the topic appropriate for publication in these transactions?

(X) Yes( ) Perhaps( ) No

2. Is the topic important to colleagues working in the field?

(X) Yes( ) Moderately So( ) No (explain: )

B. Content1. Is the paper technically sound?(X) Yes( ) No (why not? )

2. Is the coverage of the topic sufficiently comprehensive and balanced?

(X) Yes( ) Important information is missing or

superficially treated.( ) Treatment somewhat unbalanced,

but not seriously so.( ) Certain parts significantly

overstresses.

[etc etc…]

CS 597

The review: then a written commentary

III. DETAILED COMMENTS

Please state why you rated the paper as you did in Sections I and II. If you have indicated that revisions are required, please give the author specific guidance regarding those revisions, differentiating between optional and mandatory changes.

This is an interesting approach and it is quite well described. The methods are essentially sound.

The one concern I have with the approach is the nature of the instructions given to the subjects during the eye movement experiment, namely to 'not focus on small details.' This may not _guarantee_ the (favorable) outcome of the comparison with the model but it sure stacks the deck in favor of the model. This is, in fact, acknowledged by the author on p. 12 where he states that 'humans often fixated small details (...) instead of the most salient actor...despite the instructions given'. In other words, even though the subjects were explicitly instructed to behave like the model they did not always do so!

[etc etc…]

CS 597

Let’s write an answer

• Here we start with some general evaluation of the reviews:

• Key points to address here:• Reviewers always want more details, figures, validations,

tests, …• Editors always want a shorter paper overall

CS 597

CS 597

Is that a good answer?

• It does make a convincing argument• It is polite• It answers the question well• It is detailed enough

CS 597

Is that a good answer?

• NO – explaining to the reviewer why s/he misinterpreted your writing or what you intended to write or how s/he is missing some general knowledge in the field is not at all what we are after.

• If a reviewer makes a comment, no matter how irrelevant, besides the point, due to careless reading, etc. it may look to you, you must address it in the manuscript.

• Remember: If one out of 3 reviewers made that comment, one out of 3 readers are likely to make it.

• So, your answer should focus not on answering the question/comment, but on explaining how you have modified the paper such that this comment/question is addressed in the paper.

CS 597

-Explain how you modified the paper-Document exactly where you modified it so that reviewers

and editor can check it

CS 597

Note: you do not have to agree and comply with everyReviewer request, but you must demonstrate evidenceThat you have done something about every request.

CS 597

If the reviewer missed something that was explainedIn the paper, readers may miss it too. Always modifyThe paper to address that problem.

CS 597

CS 597

CS 597

CS 597

If you are going to not comply with a request, be sureTo back your decision up with some serious data!

CS 597

CS 597

CS 597

CS 597

CS 597

CS 597

CS 597

CS 597

Some of the revisions…

CS 597

CS 597

Revising the text is easy, but how about figures/data?

• When you submit a paper, be sure to make a full archive of:

• All the source code of all programs used• All the raw data• All the processing steps in perl/shell scripts• All the result analysis steps in perl/shell/matlab scripts• Etc• Etc

• And of course remember the first commandment:“thou shalt never manipulate data with your bare hands”

• Most reviewers will ask for revisions, no matter how good a paper is. You must be ready to address them!

CS 597

CS 597