![Page 1: Computer Vision Group An Interface for Art Historical Research · 2017-08-07 · Processing (IEEE), 2011 An Interface for Art Historical Research Dr. Sabine Lang sabine.lang@iwr.uni-heidelberg.de](https://reader034.vdocument.in/reader034/viewer/2022050216/5f61ff01904d5a2ff0148d06/html5/thumbnails/1.jpg)
An Interface for Art Historical Research
Prof. Dr. Björn Ommer (computer science)
Dr. Sabine Lang (art history)
Computer Vision Group
HCI / IWR, University of Heidelberg
Berliner Strasse 43
69120 Heidelberg, Germany
![Page 2: Computer Vision Group An Interface for Art Historical Research · 2017-08-07 · Processing (IEEE), 2011 An Interface for Art Historical Research Dr. Sabine Lang sabine.lang@iwr.uni-heidelberg.de](https://reader034.vdocument.in/reader034/viewer/2022050216/5f61ff01904d5a2ff0148d06/html5/thumbnails/2.jpg)
What do art historians do? ● study production processes
● find visual similarities
● formal analysis
● classification
● semantic analysis
● context study
● provenance research
An Interface for Art Historical Research Dr. Sabine Lang [email protected]
![Page 3: Computer Vision Group An Interface for Art Historical Research · 2017-08-07 · Processing (IEEE), 2011 An Interface for Art Historical Research Dr. Sabine Lang sabine.lang@iwr.uni-heidelberg.de](https://reader034.vdocument.in/reader034/viewer/2022050216/5f61ff01904d5a2ff0148d06/html5/thumbnails/3.jpg)
Working interdisciplinary in the
Computer Vision Group of
Heidelberg University
art history
computer vision
● interdisciplinary project
● embedded within both departments
(art history and computer science)
● close work environment > allows for direct feedback
● formulate research questions together
An Interface for Art Historical Research Dr. Sabine Lang [email protected]
![Page 4: Computer Vision Group An Interface for Art Historical Research · 2017-08-07 · Processing (IEEE), 2011 An Interface for Art Historical Research Dr. Sabine Lang sabine.lang@iwr.uni-heidelberg.de](https://reader034.vdocument.in/reader034/viewer/2022050216/5f61ff01904d5a2ff0148d06/html5/thumbnails/4.jpg)
Can computers propel the understanding
and reconstruction of drawing processes?
study production processes
find visual similarities
formal analysis
classification
semantic analysis
context study
provenance research
Central questions
(1) Are transformations due to technical peculiarities
or the artist’s interpretation?
(2) What statements can be made about the
reproduction process?
(a) original image
(c) degree of transformation
(b) 18th century reproduction
Computer Vision Group
International Conference on Image
Processing (IEEE), 2011
An Interface for Art Historical Research Dr. Sabine Lang [email protected]
![Page 5: Computer Vision Group An Interface for Art Historical Research · 2017-08-07 · Processing (IEEE), 2011 An Interface for Art Historical Research Dr. Sabine Lang sabine.lang@iwr.uni-heidelberg.de](https://reader034.vdocument.in/reader034/viewer/2022050216/5f61ff01904d5a2ff0148d06/html5/thumbnails/5.jpg)
(Source)
Codex Manesse (1305-1340) and
its reproductions from the 18th
century
identifying the temporal order identifying proportional modifications
An Interface for Art Historical Research Dr. Sabine Lang [email protected]
![Page 6: Computer Vision Group An Interface for Art Historical Research · 2017-08-07 · Processing (IEEE), 2011 An Interface for Art Historical Research Dr. Sabine Lang sabine.lang@iwr.uni-heidelberg.de](https://reader034.vdocument.in/reader034/viewer/2022050216/5f61ff01904d5a2ff0148d06/html5/thumbnails/6.jpg)
Can computers help with the analysis of
large image corpora, e.g. find gestures?
(Detail) Sachsenspiegel (c1220)
Heidelberg version
Central questions
(1) Which gestures can we identify?
(2) Do there exist varying types of
one gesture?study production processes
find visual similarities
formal analysis
classification
semantic analysis
context study
provenance research
Computer Vision Group
International Conference on Image
Processing (IEEE), 2011
An Interface for Art Historical Research Dr. Sabine Lang [email protected]
![Page 7: Computer Vision Group An Interface for Art Historical Research · 2017-08-07 · Processing (IEEE), 2011 An Interface for Art Historical Research Dr. Sabine Lang sabine.lang@iwr.uni-heidelberg.de](https://reader034.vdocument.in/reader034/viewer/2022050216/5f61ff01904d5a2ff0148d06/html5/thumbnails/7.jpg)
subset of representatives for
pointing gestures
contour responses
reference gesture
Relative position of a gesture
owner of gesture
reference object
owner of gesture
speaking
An Interface for Art Historical Research Dr. Sabine Lang [email protected]
![Page 8: Computer Vision Group An Interface for Art Historical Research · 2017-08-07 · Processing (IEEE), 2011 An Interface for Art Historical Research Dr. Sabine Lang sabine.lang@iwr.uni-heidelberg.de](https://reader034.vdocument.in/reader034/viewer/2022050216/5f61ff01904d5a2ff0148d06/html5/thumbnails/8.jpg)
Interface for art historical research
Interface; start screen, data collection Sachsenspiegel (c1220)
study production processes
find visual similarities
formal analysis
classification
semantic analysis
context study
provenance research
http://compvis10.iwr.uni-heidelberg.de/menu.php
An Interface for Art Historical Research Dr. Sabine Lang [email protected]
![Page 9: Computer Vision Group An Interface for Art Historical Research · 2017-08-07 · Processing (IEEE), 2011 An Interface for Art Historical Research Dr. Sabine Lang sabine.lang@iwr.uni-heidelberg.de](https://reader034.vdocument.in/reader034/viewer/2022050216/5f61ff01904d5a2ff0148d06/html5/thumbnails/9.jpg)
Interface
Data collection
Sachsenspiegel (c1220)
An Interface for Art Historical Research Dr. Sabine Lang [email protected]
![Page 10: Computer Vision Group An Interface for Art Historical Research · 2017-08-07 · Processing (IEEE), 2011 An Interface for Art Historical Research Dr. Sabine Lang sabine.lang@iwr.uni-heidelberg.de](https://reader034.vdocument.in/reader034/viewer/2022050216/5f61ff01904d5a2ff0148d06/html5/thumbnails/10.jpg)
Search example within the Sachsenspiegel (c1220)
Example shows two bounding boxes ( green)
Search results based on selected bounding boxes
An Interface for Art Historical Research Dr. Sabine Lang [email protected]
![Page 11: Computer Vision Group An Interface for Art Historical Research · 2017-08-07 · Processing (IEEE), 2011 An Interface for Art Historical Research Dr. Sabine Lang sabine.lang@iwr.uni-heidelberg.de](https://reader034.vdocument.in/reader034/viewer/2022050216/5f61ff01904d5a2ff0148d06/html5/thumbnails/11.jpg)
User feedback for retraining the model: positive results marked green, negatives red
An Interface for Art Historical Research Dr. Sabine Lang [email protected]
![Page 12: Computer Vision Group An Interface for Art Historical Research · 2017-08-07 · Processing (IEEE), 2011 An Interface for Art Historical Research Dr. Sabine Lang sabine.lang@iwr.uni-heidelberg.de](https://reader034.vdocument.in/reader034/viewer/2022050216/5f61ff01904d5a2ff0148d06/html5/thumbnails/12.jpg)
Algorithmic basis for art historicalprojects
Unsupervised Video Understanding by Reconciliation of
Posture Similarities (2017)
International Conference on Computer Vision (ICCV)
CliqueCNN: Deep Unsupervised Exemplar Learning (2016)
Conference on Advances in Neural Information Processing Systems
(NIPS)
An Interface for Art Historical Research Dr. Sabine Lang [email protected]
![Page 13: Computer Vision Group An Interface for Art Historical Research · 2017-08-07 · Processing (IEEE), 2011 An Interface for Art Historical Research Dr. Sabine Lang sabine.lang@iwr.uni-heidelberg.de](https://reader034.vdocument.in/reader034/viewer/2022050216/5f61ff01904d5a2ff0148d06/html5/thumbnails/13.jpg)
Future questions(1) What other tasks of art historical research
can be tackled and how?
(2) What is still missing regarding tools, abilities
of researchers and infrastructure?
(3) How does a collaboration between the two
disciplines change existing research
methods?
(4) How do we envision digital humanities to
look in the future?
study production processes
find visual similarities
formal analysis
classification
semantic analysis
context study
provenance research
An Interface for Art Historical Research Dr. Sabine Lang [email protected]
![Page 14: Computer Vision Group An Interface for Art Historical Research · 2017-08-07 · Processing (IEEE), 2011 An Interface for Art Historical Research Dr. Sabine Lang sabine.lang@iwr.uni-heidelberg.de](https://reader034.vdocument.in/reader034/viewer/2022050216/5f61ff01904d5a2ff0148d06/html5/thumbnails/14.jpg)
Further information & collaborators
https://hciweb.iwr.uni-heidelberg.de/compvis
https://hci.iwr.uni-heidelberg.de/publications-cv
Prof. Dr. Björn Ommer
(computer science)
Dr. Sabine Lang
(art history)
Dr. Peter Bell
(art history)
An Interface for Art Historical Research Dr. Sabine Lang [email protected]