An Educational Tool for Basic Techniques in Beginner’s Pencil Drawing

Saeko Takagi†∗ Noriyuki Matsuda† Masato Soga† Hirokazu Taki†

Takashi Shima‡ Fujiichi Yoshimoto†

Faculty of Systems Engineering, Wakayama University† L.O.T Co., Ltd.‡

Abstract

A picture is one of important research subjects in orderto make our life spiritually rich. Most studies on pictures,however, only propose some substitute functions of actualdrawing or painting materials. There is no system thatevaluates pictures drawn by users and gives advice aboutthem. We propose a learning support system for basic tech-niques in beginner’s pencil drawing. The proposed systemreceives a subject (motif) data set and an image of user’ssketch and returns advice to the user. The system is com-posed of the four subsystems: feature extraction of motifs,feature extraction of sketches, error identification, and gen-eration and presentation of advice. We developed and ex-perimented a prototype system limited to treat a basic motifand some principal advice. As a result, the validity of theproposed system was confirmed.Keywords: computer aided instruction in sketching, imageprocessing of pencil drawings, evaluation of pictures

1. Introduction

Our life is becoming materially rich owing to recent de-velopments of various technologies. People are coming towant to making their life spiritually rich [13]. It is thereforeimportant to support various activities for making our lifespiritually rich.

Drawing or painting is one of the activities that can pro-vide us with mental satisfaction. There are many studies re-lated to generating artistic computer graphics. In the field ofnon-photorealistic rendering, including a historic work byHaeberli [6], a lot of drawing or painting algorithms wereproposed [4]. In particular, Kasao et al. extracted some fea-tures from several types of actual sketches and convertedtest images into these types of expressions [10]. However,it has not been considered that a system gives a user someevaluation or advice about pictures generated by him/her.In the design fields, some assistant systems were proposed.Although these systems aid users in making a certain design

∗Postal address: 930 Sakaedani, Wakayama city, Wakayama 640-8510,Japan, E-mail: saeko@sys.wakayama-u.ac.jp

such as posters [8, 12], they do not teach us how to drawwhat we see. As applications of image analysis, there weresome systems which gave users some appropriate examplesin order to refine their art works [11, 14]. These systemsevaluate user’s art works at color or composition based onsome rules or comparison with several professional paint-ings. However, a user’s work is not evaluated in relation towhat the user actually sees.

For six months, we had observed a culture school ofsketches for beginners. There are many errors in thesketches drawn by beginners, because they could not drawwhat they saw. In addition, although a teacher pointed outthe errors to them, common students were not able to cor-rect the errors by themselves. Most beginners need concreteadvice based on their errors.

In this paper, we propose an educational tool for basictechniques in beginner’s pencil drawing. The system aidsbeginners in learning a basic sketch using real pencils anddrawing paper. The system receives a subject (motif) dataset and an image of user’s sketch and returns advice to theuser. We consider that beginner’s sketches can be evalu-ated with comparison of the sketch and the portrayed ob-jects. Sketching is the essential basis of drawing and paint-ing. To observe an object correctly is a one of fundamentalskills in sketching and to point out errors in user’s drawingsis helpful in order to correct misunderstanding of objects.The purpose of the system is not only the support of self-educated beginners but also the auxiliary role of teachers inmass schooling.

The rest of this paper is organized as follows. In Sec-tion 2, the overview of the proposed system is described.In Section 3, the details of four subsystems and the pro-cess flow are explained. In Section 4, the developed pro-totype system is mentioned and some experiments of it aredescribed. Section 5 contains concluding remarks.

2. Overview of the proposed system

The purpose of beginner’s sketch study is to raise theability of portraying what he/she is seeing itself [3]. Then, ifa sketch is drawn based on this purpose, it can be evaluated

Figure 1. Overview of the proposed system.

with comparison of the sketch and the portrayed objects.The overview of the proposed system is shown in Fig-

ure 1. First, a learner inputs the motif composition datasuch as size of objects and distance between objects. A 3Dmodel of the motif is reconstructed and some features ofthe motif are extracted. Next, the learner takes a picture ofhis/her sketch with a digital camera. The photographed im-age is taken into the system. Some features of the sketch arealso extracted. Comparing the obtained features, the systemidentifies drawing errors based on a knowledge base of er-rors. Then, several kinds of advice are generated based ona knowledge base of advice, his/her learning history, and acurriculum. Finally, the advice is presented to the learner.

By the way, we considered which is better for learners,direct drawing on a computer with some devices such as atablet, or classic drawing with the actual pencil and draw-ing paper. These computer devices have not yet reproducedthe high expression and good feeling of the actual drawingmaterials. The eraser techniques are used extensively in theactual sketch. In the use of such devices, however, it is dif-ficult to draw with a eraser. There is a comparison reportof drawing to paper and drawing on a computer [2]. In thereport, using a computer for drawing, user’s attention wasturned to how to use a tool, and there was bad influence ondrawing activities. For these reasons, we adopt the classicand familiar drawing materials.

3. Data processing in the proposed system

3.1. Feature extraction of motifs

A 3D model of objects portrayed by a user is built fromthe composition data of the shape and the arrangement. Themotif composition data contains size of the objects, distancebetween objects, distance from the user to the objects, theuser’s eye level, etc.

From the composition data, a 3D model is generated andseveral images of the 3D model seen from the user’s view-

Figure 2. Example of a sketch.

Figure 3. Drawn areas.

point are rendered. Some feature parameters of the motifare calculated from these images. The feature parametersare used as a correct model in identification of errors.

3.2. Feature extraction of sketches

For the evaluation of sketches, it is necessary to extractsome useful features from sketches. In the feature extrac-tion subsystem for sketches, the following steps are per-formed. First, a rectangular region is cropped from the orig-inal photographed image that is regarded as the drawing pa-per. The pencil-drawn area is extracted from the croppedregion using the texture analysis based on the co-occurrencematrix and the gradient-based edge detection [1]. The tex-ture analysis is used to divide the cropped region into twoparts, temporary drawn areas and paper areas. The edge de-tection is used to extract the outlines of the actual drawnarea. Because a good result cannot be got by using only onemethod, we combine both of results from the two methodsfor the extraction of the actual drawn area. The temporarydrawn areas based on the texture analysis are calculated alittle extensively. In the temporary drawn areas, the areassurrounded by outlines the are regarded as the actual drawn

Figure 4. After classification.

areas. An example of a cropped sketch is shown in Figure2, and the actual drawn areas in the sketch is calculated asFigure 3.

In order to obtain single-pixel-wide center lines, the thin-ning algorithm [1] is performed to the drawn areas. The ob-tained lines are utilized as the skeleton data of the sketch forspecification of the each target part. The result of thinningis classified to branch points and segments which connectstwo branch point. Making use of the composition data ofthe motif and some geometric features, each specific partis described with these branch points and segments. Somenoises are reduced in this stage. An example after classifi-cation is shown in Figure 4, which is the result of processingthe image in Figure 3.

After specifying the target parts, some feature parame-ters are calculated by several heuristic methods. The reasonusing such methods is that a user’s sketch is frequently quitedifferent from the motif looked at from his/her viewpoint.

3.3. Error identification

From the feature parameters of a motif and a sketch ob-tained by the previous stages, some parameters for judgingerrors are calculated. The judging parameters are checkedwith the rules in a knowledge base of errors, then, someerrors are identified. Certain parameter groups about thedegree of errors are also stored in the knowledge base.

The error knowledge base and the parameter groups forjudging error are built on advice data sets obtained from theadvice in actual art classes for beginners. About each error,the kind of the error, the part containing the error, and thedegree of the error are identified. The results are passed tofollowing subsystem.

3.4. Generation and presentation of advice

Generated advice is presented to a user through four me-dia: text, voice, sample illustration, and 3D model.

Figure 5. Example of presenting text advice.

First, checking with the rules in a knowledge base of ad-vice, the advice by text (and voice) is generated based on thekind and degree of errors. The curriculum and each leaninghistory are also considered. The knowledge base of adviceis built on advice data sets obtained from the advice in ac-tual art classes for beginners and related books. The adviceon errors is first generated and if necessary, supplementaryadvice is also generated.

Since too much quantity of advice will spoil user’s willto sketch, the advice about all of the errors is not presentedto the user. Advice is selected based on the importance inpencil drawing, the frequency of use in actual art classes,each user’s learning history, etc. In addition, the presenta-tion order of advice is also controlled for making the ad-vice more effective. The selection and control are executedby certain algorithms established on analyzing the advicedata sets in actual art classes. The generated advice of textis displayed at an advice window and also read aloud withsynthetic voice.

Sample illustrations are used as assistance of text andvoice advice. They contain auxiliary lines, technical illus-trations about perspective, and correct sketches. Overlay-ing sample illustrations on a user’s sketch is considered toraise understanding of advice. Figure 5 shows an exampleof windows presenting text advice with overlaying auxiliarylines on a user’s sketch.

Advice by displaying a 3D model is helpful for users tounderstand the existence of errors in their own sketches[9].In the field of computer aided instruction, “error visualiza-tion” is used in order to support users’ awareness of the er-rors for learning from mistakes [7]. In this system, based onthe motif composition data, a basic 3D model is generated.Incorrect 3D models are generated by adding the result oferror identification to the basic 3D model. In usual meansin art classrooms, such as oral explanation and 2D illustra-tion, it is difficult to explain 3D contradiction in a sketch.On the other hand, the explanation becomes easy by errorvisualization with the incorrect 3D model, and users canalso understand errors more intuitively. Figure 6 shows an

Figure 6. Example of error visualization with3D model.

example of windows presenting 3D models.

4. Experiment of a prototype system

4.1. About handled advice in the prototype system

A prototype system was developed to verify validity ofthe proposed system.

For the prototype system, we chose a circle plate and abeer mug as a target motif. Straight-lines and ellipses arebasic factors in sketching. They are clearly contained inthis motif. In addition, ellipses at different eye levels aregood examples for training beginners to observe an objectcorrectly.

Generally, a sketch consists of two elements, which are aform and a torn [5]. Beginners start for catching a form.Therefore, as a first step, we treat a form. We establishseveral conditions of target sketches: a sketch is drawn withdeep lines and without any assisting lines or shade.

We selected the following eleven items of advice whosefrequency were high in a certain art class for beginners.

1. The viewpoint for the plate is too high.2. The margins are not same.3. The plate is big/small.4. The mug is thick/thin.5. The roundness of the plate is not enough.6. The end of the plate is sharp.7. The relations of ellipses in the mug are wrong.8. The width of the mug is not constant from top to bot-

tom.9. The rim of the plate bottom is high.

10. The mug is slanted.11. The plate is deep/shallow.

For these items of advice, the system calculates about fortyparameters of the motif, about fifty parameters of eachsketch, and about twenty parameters for judging errors.

In the error identification subsystem, the degree of eacherror was judged in a five-step evaluation. In the generationof advice, the number of errors contained in one advice pre-sentation is limited up to three. In addition, when the sketchhas few errors, the system praises the sketch. The controlof advice is based on the analysis of the coverage in the artclass.

4.2. Experiment of feature extraction of sketches

Unless the required parameters can be correctly ex-tracted from sketches, any effective evaluation and adviceabout the sketches cannot be returned to the users. Then,we investigated whether the required parameters could beextracted from actual sketches. The experimental objectswere the sketches which were drawn by three types of peo-ple: ten students in Wakayama University, sixteen peoplein a culture school held at a public hall, and twenty threefourth-grade schoolchildren. All of them had little knowl-edge and skill in drawing.

Since university students had been told in advance thatdrawing with clear and deep lines was desirable, in sketchesdrawn by the students, the feature extraction had been car-ried out almost correctly. In the culture school, some aux-iliary lines were left in many sketches and there were twosketches drawn with shade. After erasing these lines andshaded parts finely, the feature extraction of sketches wasmostly able to be carried out. Enough parameters were notcalculated in one sketch. The texture analysis was unsuc-cessful in this sketch, all of the drawn areas were not ex-tracted.

On the other hand, about the sketches drawn byschoolchildren, it was hard to extract the required param-eters. The main reason is failure in the classification of thetarget part, because the drawing line was not continuous inthe original sketch. In addition, the schoolchild’s sketchoften protruded from the drawing paper. The current proto-type system can not deal with this type of sketches.

4.3. Experiment and evaluation of the prototypesystem by beginners

Ten students used the prototype system. After learn-ing sketch with our system, they answered a questionnaireabout the feeling of use. The questionnaire was composedof six items about the operation of the system and eightitems about the advice given by the system.

Figure 7 (a) shows an example of motifs and (b) showsa typical example of sketches drawn by the students beforeusing the system. The prototype system gave the student thefollowing comments about this sketch:

(1) “The ellipses in the cross sections of the mug aredrawn from incorrect viewpoints.” This advice waspresented with an illustration as shown in Figure 8(a).

(a) Motif.

(b) Sketch.

Figure 7. Each example of actual motifs andsketches.

(2) “The composition is almost good, but it tends to be ina little right.”

(3) “Although the form of the plate is almost well drawn,the plate drawn by you looks as if your eye level is toohigh. It is seen more slender in your viewpoint.”

(4) “In order to draw an ellipse, by drawing an appropri-ate rectangle in this way, you can draw an ellipse moreeasily. a and b, and c and d are horizontally symmetri-cal. a and c, and b and d are almost vertically symmet-rical, too.” This advice was presented with an appro-priate rectangle and an ellipse on the sketch as shownin Figure 8(b).

Then, 3D models were displayed as shown in Figure 9 thatare the correct model based on the motif (left) and the in-correct model based on the sketch (right). After the set ofadvice was given by the system, the student modified hissketch. As shown in Figure 10, several points were im-proved from the sketch before using the system, which werethe size of the plate, the roundness of the plate, two ellipsesand the bottom curve of the mug, etc. The sketch after theadvice was resembling the actual motif more than the sketchbefore using the system.

(a) About the mug.

(b) About the plate.

Figure 8. Examples of advice windows withtext, voice, and illustration.

About operation of the system, most users answered thatmeasurement of the motif data was troublesome. Since theinput of the photographed images to the system took time,the complaint also came out mostly that the process of thesystem was not smooth. It is necessary to improve the inputinterface.

On the advice given by the system, about half of thestudents answered that the advice was easy to understand.Moreover, almost all the students answered that his/hersketch became better after correcting it according to theadvice. As mentioned above, it is considered that the ad-vice given by the system was effective. However, there wasalso a complaint that it was hard to understand how a sketchshould be corrected. It is found that the given advice wasnot enough for beginners.

4.4. Evaluation of advice from the viewpoint of anart teacher

We showed an art teacher the experimental result of uni-versity students’ sketches before and after advice. Conse-quently, it turned out that advice given by our system wasalmost suitable from the viewpoint of the art teacher. More-

Figure 9. Example of advice windows with 3Dmodel.

Figure 10. Example of sketches after advice.

over, comparing sketches before and after advice, it was es-timated that most sketches were improved. Moreover, thesystem could judge a minute error which was not noticedeasy by the art teacher. However, it was pointed out thatthe system gave a user unsuitable advice about one sketch.The reason is that the plate region and the mug region wererecognized contrary and the feature extraction failed, be-cause the plate was too big and the mug was too small inthe sketch.

5. Conclusion

We proposed a learning system which advise beginnerson a form in pencil drawing. The system is composed ofthe four subsystems: feature extraction of motifs, featureextraction of sketches, error identification, and generationand presentation of advice. A prototype system was de-veloped and experimented to about fifty sketches drawn byseveral kinds of beginners. The prototype system gave ef-fective advice about the sketches to the beginners and theycould modify their sketches based on the advice. The va-lidity of the proposed system was confirmed. However, itis necessary to raise the accuracy of the feature extraction.

Furthermore, we should deal with a tone. After dealing withboth of a form and a tone about a circle plate and a beermug, we will study other motifs such as fruits on a plate.

Acknowledgment

We thank Ryuzo Goda, Hideaki Kawanishi, NobuyukiKajimoto, and Yoriko Maruyama who contributed to de-velop the prototype system.

References

[1] K. R. Castleman. Digital Image Processing. Prentice Hall,1995.

[2] A. Chikawa, M. Iwata, and S. Tano. Effect on human cre-ative work by computer-supported drawing and sketching onpaper. In Proceedings of Human Interface 2000, pages 219–222. Human Interface Society, 2000.

[3] R. De Reyna. How to Draw What You See. Watson-GuptillPublications, 1996.

[4] B. Gooch and A. Gooch. Non-Photorealistic Rendering. AK Peters Ltd., 2001.

[5] A. L. Guptill. Rendering in Pencil. Watson-Guptill Publica-tions, 1979.

[6] P. E. Haeberli. Paint by numbers: Abstract image repre-sentations. Computer Graphics (Proceedings of ACM SIG-GRAPH 90), 24(4):207–214, 1990.

[7] T. Hirashima, T. Horiguchi, A. Kashihara, and J. Toyoda.Error-visualization by error-based simulation and its man-agement. International Journal of Artificial Intelligence inEducation, 9(1):17–31, 1998.

[8] J. Ichino and S. Tano. User interface for artistic patternscreating support. The Transactions of the Institute of Elec-tronics, Information and Communication Engineers, J82-D-II(10):1693–1709, 1999.

[9] N. Kajimoto, N. Matsuda, H. Taki, M. Soga, S. Takagi,F. Yoshimoto, and T. Shima. The proposal of the techniqueof error visualization for a learner’s pencil drawing. In Pro-ceedings of International Conference on Computers in Edu-cation 2002, pages 153–157. APC/AACE, 2002.

[10] A. Kasao and M. Nakajima. Characteristics extracted fromsketches and a drawing simulation. The Journal of TheInstitute of Image Information and Television Engineers,53(6):873–881, 1999.

[11] K. Nakakoji, B. N. Reeves, A. Aoki, H. Suzuki, andK. Mizushima. emmac: knowledge-based color critiquingsupport for novice multimedia authors. In Proceedings ofACM Multimedia ’95, pages 467–476. ACM, ACM Press,1995.

[12] T. Obata and M. Hagiwara. A color poster creating supportsystem on reflect kansei. Transactions of Information Pro-cessing Society of Japan, 41(3):701–710, 2000.

[13] Public Relations Office, Minister’s Secretariat, Cabinet Of-fice. Public Opinion Poll about the National Life (Investiga-tion in June, 2002). Government of Japan, 2002.

[14] S. Tanaka, J. Kurumizawa, S. Inokuchi, and Y. Iwadate.Composition analyzer: support tool for composition anal-ysis on painting masterpieces. Knowledge-based Systems,13(7-8):459–470, 2000.