A Greek Pottery Shape and School A Greek Pottery Shape and School Identification and Classification Identification and Classification

System System Using Image Retrieval TechniquesUsing Image Retrieval Techniques

Gulsebnem (Sheb) Bishop, Sung-Hyuk Cha, Charles Gulsebnem (Sheb) Bishop, Sung-Hyuk Cha, Charles TappertTappert

May 6th, 2005May 6th, 2005

School of Computer Science & Information SystemsWhite Plains, NY

We have successfully developed We have successfully developed an image-based an image-based

pottery shape and school pottery shape and school identification system for an identification system for an

unknown pottery or fragment unknown pottery or fragment to assist archaeologists in to assist archaeologists in identifying and recording identifying and recording

objects quickly and accurately.objects quickly and accurately.

Many uses to this system:Many uses to this system:1.1. The system can serve as an educational tool The system can serve as an educational tool

for novice archaeologists to identify and for novice archaeologists to identify and study artifacts or fragments quickly and study artifacts or fragments quickly and easily. easily.

2.2. It can serve as a valuable tool in excavations It can serve as a valuable tool in excavations for identification, classification and for identification, classification and reconstruction of fragments.reconstruction of fragments.

3.3. There are thousands of pottery fragments There are thousands of pottery fragments found every year in excavations, and they are found every year in excavations, and they are usually discarded without being recorded, yet usually discarded without being recorded, yet alone being classified. This system can alone being classified. This system can provide a quick, inexpensive and objective provide a quick, inexpensive and objective way of documenting and classifying these way of documenting and classifying these fragments. fragments.

4.4. It can assist in identification and analysis of It can assist in identification and analysis of pottery decorations.pottery decorations.

Our major task in this study is to identify Our major task in this study is to identify the shape and the school of a whole pot the shape and the school of a whole pot or a fragment at hand, by using shape or a fragment at hand, by using shape and color-based image retrieval and color-based image retrieval techniques. techniques.

Our system analyzes and compares Our system analyzes and compares extracted features to determine the top extracted features to determine the top five matching images and information five matching images and information related to these images and presents related to these images and presents them to the user for final decision. them to the user for final decision.

What makes this study unique is: What makes this study unique is: 1.1. Shape and color-based image Shape and color-based image

retrieval techniques will be used retrieval techniques will be used together for the first time. together for the first time.

2.2. Image retrieval from our Image retrieval from our database is not text based its image database is not text based its image based.based.

DATABASEDATABASETwo sections:Two sections:

1.1. Images of Pottery with Shape and School Images of Pottery with Shape and School InformationInformation

2.2. Information about the Extracted FeaturesInformation about the Extracted Features

Training Database200 Images

20 Distinct Shapes 4 Color Conventions

Alabastron Amphora Group

Crater Group

Lekythoi Group

Cups

Pyxis

Hydria-Kalpis Stamnos

Skyphos

Pelike

Oinochoi

Kyathos Kantharos

SchoolsSchools

Black Figure 630-530 BC

Red Figure530-470 BC

White Ground 550-330 BC

White Ground 460-420 BC

Pottery Identification and Pottery Identification and Retrieval System – PIRSRetrieval System – PIRS

1.1. We obtain a digital image of our object.We obtain a digital image of our object.2.2. This image goes through a segmentation This image goes through a segmentation

process.process.3.3. We then measure the regional properties We then measure the regional properties

of this segmented image.of this segmented image.

The regional properties measure object or The regional properties measure object or region properties in an image and returns region properties in an image and returns

them in a structure array. them in a structure array. 8 Regional Measurements 8 Regional Measurements

BoundingBoxBoundingBox MajorAxisLengthMajorAxisLength MinorAxisLengthMinorAxisLength EquivDiameterEquivDiameter EccentricityEccentricity OrientationOrientation SoliditySolidity ExtentExtent

3. Once the image is segmented and the features extracted 3. Once the image is segmented and the features extracted this information is compared to the information in our this information is compared to the information in our database. database.

4. The aim of the color and shape matching algorithm is to 4. The aim of the color and shape matching algorithm is to identify the top five matching pieces. identify the top five matching pieces.

5. After the user identifies the matching piece the system outputs information about that piece.

During the excavations archaeologists not During the excavations archaeologists not only find whole vases but they also find only find whole vases but they also find broken vases and single fragments. We broken vases and single fragments. We needed to find a solution to this problem needed to find a solution to this problem also. also.

Fragments belonging to the same pot go Fragments belonging to the same pot go through the same stage. through the same stage. 1. Obtain the image of the fragments. 1. Obtain the image of the fragments. 2. We put the fragments together 2. We put the fragments together through Jigsaw puzzle like algorithms. through Jigsaw puzzle like algorithms.

2. We segment the image.2. We segment the image.3. We extract the features.3. We extract the features.

4. Compare it to the information that we 4. Compare it to the information that we have in our database.have in our database.

5. Identifying the top five matches and 5. Identifying the top five matches and present it to the user. present it to the user.

Jigsaw puzzle problem has been thought of Jigsaw puzzle problem has been thought of as an important artificial intelligence as an important artificial intelligence

search problem. If one tries to solve the search problem. If one tries to solve the jigsaw puzzle problem based on shape the jigsaw puzzle problem based on shape the solution of the problem becomes harder. solution of the problem becomes harder. The patterns, colors or decorations on the The patterns, colors or decorations on the fragments help us tremendously locating fragments help us tremendously locating

the matching pieces. It reduces the search the matching pieces. It reduces the search space by utilizing this information. space by utilizing this information.

Single FragmentSingle FragmentThis last section makes sure that the single This last section makes sure that the single

fragments are recorded in the system. fragments are recorded in the system. If they have decorations on them or if the If they have decorations on them or if the

profile is clear they can be matched with profile is clear they can be matched with similar pieces. similar pieces.

Single fragments go through the same process. Single fragments go through the same process. 1.1. We obtain the image of the fragment. We obtain the image of the fragment. 2.2. We segment the image. We segment the image. 3.3. A template matching algorithm identifies A template matching algorithm identifies

the top five matches.the top five matches.

Training and TestingTraining and Testing Training Set: 200 ImagesTraining Set: 200 Images

Whole Pottery Testing Set: 400 ImagesWhole Pottery Testing Set: 400 ImagesFragments Testing Set: 400 ImagesFragments Testing Set: 400 Images

Attention given to 4 issues:Attention given to 4 issues:1.1. How accurately the system identifies the shapes How accurately the system identifies the shapes

of the whole vessels? of the whole vessels? 2.2. How accurately the system matches the How accurately the system matches the

fragments?fragments?3.3. How accurately the system identifies the single How accurately the system identifies the single

fragments?fragments?4.4. How accurately the system identifies the color How accurately the system identifies the color

conventions?conventions?

Queried Image Top five similar images retrieved

Queried Image Top five similar images retrieved

1.1. System detects the shapes of the selected images with System detects the shapes of the selected images with 99% accuracy.99% accuracy.

Queried Image Top five similar images retrieved

Queried Image Top five similar images retrieved

2. The system puts together the randomly cropped two dimensional images with high accuracy and matches it to the corresponding image with 98% accuracy. 3. When the system was tested with single fragments the accuracy rate depended on the area that we looked at. If it was an obvious and large enough area the accuracy rate was 99%. If the area was a less identifiable region the accuracy rate was 70%.

4. The color convention in both, whole 4. The color convention in both, whole and cropped images, was detected and cropped images, was detected with 98% accuracy. with 98% accuracy.

Queried Image Top five similar images retrieved

Even though our system yielded good Even though our system yielded good results there is plenty of future work results there is plenty of future work

to be done: to be done:

1. Working with less identifiable parts of 1. Working with less identifiable parts of the vases. the vases.

2. Working on the speed of the 2. Working on the speed of the identification process. identification process.

3. Extending the study to subtle shapes. 3. Extending the study to subtle shapes. 4. Working with real fragments.4. Working with real fragments.

REFERENCESREFERENCES

1. Kampel, M. & Sablatnig, R. Virtual Reconstruction of Broken and Unbroken 1. Kampel, M. & Sablatnig, R. Virtual Reconstruction of Broken and Unbroken Pottery. In Proceedings of the Fourth International Conference on 3-D Pottery. In Proceedings of the Fourth International Conference on 3-D Digital Imaging and Modeling, pp. 318-325 (2003).Digital Imaging and Modeling, pp. 318-325 (2003).

2. Lengyel, A. Computer Applications in Classical Archaeology. In Proceedings 2. Lengyel, A. Computer Applications in Classical Archaeology. In Proceedings of Computer Applications in Archaeology. pp. 56-62 (1975).of Computer Applications in Archaeology. pp. 56-62 (1975).

3. Main, P. The Storage Retrieval and Classification of Artefact Shapes. In 3. Main, P. The Storage Retrieval and Classification of Artefact Shapes. In Computer Application in Archaeology. pp. 39-48 (1978). Computer Application in Archaeology. pp. 39-48 (1978).

4. Hall, N. S. and Laflin, S. A Computer Aided Design Technique for Pottery 4. Hall, N. S. and Laflin, S. A Computer Aided Design Technique for Pottery Profiles. In Computer Applications in Archaeology. pp. 178-188 (1984). Profiles. In Computer Applications in Archaeology. pp. 178-188 (1984).

5. Lewis, P. H. and Goodson, K. J. Images, Databases and Edge Detection for 5. Lewis, P. H. and Goodson, K. J. Images, Databases and Edge Detection for Archaeological Object Drawings. Computer Applications and Quantitative Archaeological Object Drawings. Computer Applications and Quantitative Methods in Archaeology:149-153 (1990).Methods in Archaeology:149-153 (1990).

6. Durham, P., Lewis, P. H. and Shennan, S. J. Artefact Matching and retrieval 6. Durham, P., Lewis, P. H. and Shennan, S. J. Artefact Matching and retrieval Using the Generalised Hough Transform. In Proceedings of Proceedings of Using the Generalised Hough Transform. In Proceedings of Proceedings of Computer Applications in Archaeology. pp. 25-30 (1995).Computer Applications in Archaeology. pp. 25-30 (1995).

7. Sablatnig R. and Menard C. Computer based Acquisition of Archaeological 7. Sablatnig R. and Menard C. Computer based Acquisition of Archaeological Finds: The First Step towards Automatic Classification. In 3rd International Finds: The First Step towards Automatic Classification. In 3rd International Symposium on Computing and Archaeology. Vol. 1, pp. 429-446 (1996). Symposium on Computing and Archaeology. Vol. 1, pp. 429-446 (1996).

8. Kampel, M., Sablatnig, R. and Costa, E. 8. Kampel, M., Sablatnig, R. and Costa, E. Classification of Archaeological Fragments using Profile PrimitivesClassification of Archaeological Fragments using Profile Primitives. . In Computer Vision, Computer Graphics and Photogrammetry - a Common In Computer Vision, Computer Graphics and Photogrammetry - a Common Viewpoint, Proc. of the 25th Workshop of the Austrian Association for Viewpoint, Proc. of the 25th Workshop of the Austrian Association for Pattern Recognition (OEAGM). Vol. 147, pp. 151-158, Oldenburg, Wien, Pattern Recognition (OEAGM). Vol. 147, pp. 151-158, Oldenburg, Wien, München, 2001. München, 2001.

9. Kampel M. and Sablatnig R. 9. Kampel M. and Sablatnig R. 3D Puzzling of Archeological Fragments3D Puzzling of Archeological Fragments. In . In Proceedings of the 9th Computer Vision Winter Workshop, pp.31-40 (2004).Proceedings of the 9th Computer Vision Winter Workshop, pp.31-40 (2004).

10. Leitao, H. C. G. and Stolfi, J. Multiscale Method for Reassembly of Two-10. Leitao, H. C. G. and Stolfi, J. Multiscale Method for Reassembly of Two-Dimensional Fragmented Objects. In IEEE Trans. On Pattern Analysis and Dimensional Fragmented Objects. In IEEE Trans. On Pattern Analysis and Machine Intelligence, 24 (9), pp.1239-1251 (2002). Machine Intelligence, 24 (9), pp.1239-1251 (2002).

11. McBride, J. C. & Kimia, B. B. Archaeological Fragment Reconstruction Using 11. McBride, J. C. & Kimia, B. B. Archaeological Fragment Reconstruction Using Curve Matching. In Proceedings of the 2003 Conference on Computer Curve Matching. In Proceedings of the 2003 Conference on Computer Vision and Pattern Recognition Workshop, pp. 1-8 (2003).Vision and Pattern Recognition Workshop, pp. 1-8 (2003).

12. Kong, W. and Kimia, B. B. On Solving 2D & 3D Puzzles Using Curve 12. Kong, W. and Kimia, B. B. On Solving 2D & 3D Puzzles Using Curve Matching. In Proceedings of the 2001 IEEE Computer Society Conference Matching. In Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 583-591 (2001).on Computer Vision and Pattern Recognition, pp. 583-591 (2001).

13. Cha, S-H., Murirathnam, S. Comparing Color Images Using Angular 13. Cha, S-H., Murirathnam, S. Comparing Color Images Using Angular Histogram Measures. In Proceedings of 5th Joint Conference in Information Histogram Measures. In Proceedings of 5th Joint Conference in Information Sciences, vol. II, CVPRIP, p.139-142 (2000).Sciences, vol. II, CVPRIP, p.139-142 (2000).

14. H. S. Sawhney, H. S. and Hafner, J. L. Efficient Color Histogram Indexing. In 14. H. S. Sawhney, H. S. and Hafner, J. L. Efficient Color Histogram Indexing. In International Conference on Image Processing, vol. 1, pp. 66-70 (1994). International Conference on Image Processing, vol. 1, pp. 66-70 (1994).

15. Kampel, M. and Sablatnig, R. Color Classification of Archaeological 15. Kampel, M. and Sablatnig, R. Color Classification of Archaeological Fragments. In International Conference on Pattern Recognition (ICPR'00)-Fragments. In International Conference on Pattern Recognition (ICPR'00)-Volume 4, September (2000) pp. 4771.Volume 4, September (2000) pp. 4771.

16. Hart, E., Cha, S-H. and Tappert, C. Interactive Flag Identification Using 16. Hart, E., Cha, S-H. and Tappert, C. Interactive Flag Identification Using Image Retrieval Techniques. Pace University, SCIC Technical Report, Image Retrieval Techniques. Pace University, SCIC Technical Report, Number 203 (2004).Number 203 (2004).

17. Nagy, G. and Zou, J. Interactive Visual Pattern Recognition. In Proceedings 17. Nagy, G. and Zou, J. Interactive Visual Pattern Recognition. In Proceedings of the Internationalof the International