Title: Video Game Business Management Economics:

Employing Industry-Level Data to Improve the Decision-Making Process

Intended Degree: Doctor of Philosophy (Knowledge Science)

Laboratory: Miyata Lab

Name: Eric Nelson Bailey (s1840008)

Research Content:

Current research into video game business management is sparse, and the creative nature of the products presents an

obstacle to quantitative research. Business management decisions are made by publishers and developers based on their

tacit knowledge of what has worked in the past according to their experience; however, left unverified there is a risk that

this tacit knowledge may not reflect rapidly changing market realities. Managerial economics is concerned with the

optimization of the business decision-making process given limited resources, and such a rational decision-making

process is required if publishers and developers want to ensure that the knowledge in their organizations best reflects the

reality of the wider industry. Judging the value of decisions by the effectiveness of their usage of limited resources opens

the tacit knowledge behind these decisions to rational evaluation. Through this research, I rely on this managerial

economics perspective and use knowledge discovery in database (KDD) techniques to answer the following questions:

MRQ: How can an economic perspective that views decision optimization in terms of making the best use of

limited organizational resources allow for the use of data from the wider industry to question assumptions and

improve video game business management decision processes?

SRQ1: What is the state of intellectual property exploitation and exploration strategies in innovation and business

management practices within leading video game organizations and how are those strategies changing?

SRQ2: What role do women play in making creative decisions within video game development organizations and

how does this compare to video game consumer demographics?

SRQ3: How do the product scope decisions made in video game project management reflect what provides value

to consumers?

This research proposes a learning process tailored to a creative industry such as video game development as an answer

to the MRQ. The proposed learning cycle expands on the traditional “double-loop” learning cycle by incorporating two

important steps: locating decisions based on tacit assumptions and framing questions to attempt to falsify those

assumptions, and then employing KDD techniques with data from the wider video game industry to validate the tacit

assumptions behind those decisions. KDD techniques were applied in answering the three subsidiary research questions

through quantitative research into the results of one strategic innovation management decision, one human resource or

“talent management” decision, and two project management product scope decisions in the video game industry. The

answers to these three research questions provide examples for how such techniques can quantify the results of business

decisions in a creative industry like video game development and open them to the rational examination employed in the

learning process proposed as an answer to the MRQ.

To answer SRQ1, I analyze a game catalog of 6,222 games from ten “third-party” console and PC game publishers

from 1980 to 2018 to determine the ratio between exploration and exploitation of intellectual property (IP) in hit “AAA”

producing video game console and PC publishers and discover that game publishers are growing more conservative, with

a greater reliance on exploitation strategies over time. The overall ratio of exploitation to exploration in the AAA

publishers sampled for 2018 was found to be 13%; however, franchise revenue volatility peaks within the first few games

and tapers, requiring the development of new franchises to bolster future growth before revenues from exploiting existing

IP falls too low to support new developments.

To answer SRQ2, I analyze 14,265 staff from 27 representative “AAA” best-selling games to determine gender ratios,

broken down by whether they are development or support roles, well-paying roles, or leadership roles. The results show

that, not only is overall gender diversity in video game development limited and not reflecting the demographics of the

target audience, but that diversity is even more limited for leadership and other roles with the power to make creative

decisions, which limits the potential for the products of these decisions to reach a wider audience.

To answer SRQ3, I analyze data for 31,964 entries on PC game service Steam for games and applications released

2005 to 2017. For the 725 entries sampled, only 14% of players on average had reached the end of the “single-player”,

“main-path” content. I then analyze data for 34,263 entries on Steam for games and applications released 2005 to 2017 to

determine when a functional scope decision, the decision to include a “multiplayer” mode in a video game, adds value

for players. The research finds that much of the content developed for video games goes unplayed and that the

implementation of a multiplayer mode is not always appropriate given how costly and complex it can be to develop.

Research Purpose:

“Cultural” or “creative” industries differ from other industries in that the most important “ingredients” of the product

that provide value to consumers are intangible, or “symbolic.” (Bilton, 2017) The creative industries are a valuable area

for knowledge science research because they are reliant on the experience-based tacit knowledge of their employees to

generate value. With the video game industry, which served as the focus of this research, the rapid pace of change soon

renders prior knowledge irrelevant. (Parkin, 2018) For this reason, creative industries, like the video game industry, can

provide insight into how concepts like tacit knowledge can influence decision making in the context of a rapidly changing

market “reality”, and provide a resource for discovering ways tacit knowledge and assumptions can be corrected or

verified by employing KDD approaches like data mining.

Tacit knowledge is important for the cultural and creative industries where creativity is the source of value; however,

this knowledge is limited by the experience of the holder and the holder’s intuition regarding those experiences. This

research contributes to knowledge management research in the creative industries by introducing a learning process that

adapts the traditional “double-loop learning” model (Argyris, 1977) to work in creative industries, such as the video game

industry, where tacitly held assumptions drive much of the value-creation process. It identifies decisions that could rely

on mistaken tacit assumptions and incorporates data from the wider industry into the validation process, rather than from

within the limits of the organization, with a focus on falsification of assumptions to avoid the confirmation biases that

make systematic learning processes difficult to implement in a creative industry. This research provides examples of the

knowledge science concepts of tacit knowledge, organizational learning, and KDD applied in practice with the validation

of decision-making assumptions using the video game industry case. Through four examples, the research quantifies

industry-wide data in three different management area in video game development: strategic innovation management,

human resource or “talent management”, and project product scope management. KDD techniques like these can be used

to quantify industry realities, and, when coupled with a falsification step, confront structural inertia and improve decision

processes throughout video game organizations and other cultural industry organizations, which have relied on the tacit

knowledge their employees have accrued through experience on a limited set of projects.

First, the quantification of IP exploration and exploitation trends in Chapter 5 served to quantify and confirm the prior

findings in the qualitative research by Tschang (2007) that the video game industry is growing more conservative as a

method for reducing the uncertainty and hedging against the growing risks of development. It did this by examining the

game catalogs of the top eight third-party publishers of console and PC games by revenue globally from 1980 through

2018.

Next, the quantification in Chapter 6 of women represented in positions of power or with creative voice within video

game development organizations confirmed the suspicions of Williams et al. (2009) that the content in video games

reflects the demographics of the developers, and discovered the same inequalities found by Smith et al. (2018) for

positions with creative power in the movie industry also exist in the video game industry. Chapter 6 also introduced a

novel visualization technique for discovering trends basic statistical analysis could miss in staff list gender representation.

This technique could be applied outside of the video game industry to any industry where its employees are credited in a

staff list scheme.

The quantification of product content scope usage patterns in PC video games in Chapter 7 greatly extends the limited

existing research into usage patterns focused on a subset of games by Bauckhage et al. (2012) and Weber et al. (2011)

through a novel approach that relies on KDD techniques and detecting video game “completion rates” by examining

achievement tags in a large body of video games. The quantification of the value of multiplayer functionality in Chapter

8 also presents how the same novel achievement analysis technique can be applied to extend the findings by Hullet et al.

(2011) that multiplayer functionality often goes unused.

Research Accomplishments:

Publications in Peer Reviewed Journals:

(1) Bailey, E., & Miyata, K. (2019). “Improving video game project scope decisions with data: An analysis of

achievements and game completion rates.” Entertainment computing. Vol. 31. p. 100299, doi:

10.1016/j.entcom.2019.100299

(2) Bailey, E., Miyata, K., & Yoshida, T. (2019). “Gender composition of teams and studios in game development.”

Games and culture. doi: 10.1177/1555412019868381

Publications under Review:

(3) Bailey, E., Miyata, K. (in review). “Exploration and exploitation in video game development: An analysis of the

shift in innovation trends in third-party console game publishers.”

Peer Reviewed Oral and Poster Presentations at International Academic Conferences:

(4) Bailey, E., & Miyata, K. (2018). “Estimating the value of multiplayer modes in video games: An analysis of sales,

ratings, and utilization rates.” 10th International Joint Conference on Knowledge Discovery, Knowledge

Engineering and Knowledge Management (IC3K) / Knowledge Discovery and Information Retrieval (KDIR) 2018.

Peer reviewed. Published in proceedings pages 152 – 166. Oral presentation on September 19th, 2017. Seville,

Spain. doi: 10.5220/0006898201550162

(5) Bailey, E., & Ito, Y. (2017). “Live streaming changes video game testing: Observing contextual player behavior

over video streaming services.” Annual Meeting of the Society for Social Studies of Science (4S). Peer reviewed.

No proceedings. Oral presentation on September 1st, 2017. Boston, USA.

(6) Bailey, E., & Ito, Y. (2017). “Livestreaming for user testing: Context-rich observation of game player behavior.”

International Conference on Management, Science and Engineering (ICMSE) 2017. Peer reviewed. August 19th,

2017. Published in proceedings pages 228 - 237. Oral presentation on August 19th, 2017. Nomi, Japan. doi:

10.1109/ICMSE.2017.8574431

(7) Bailey, E., & Miyata, K. (2017). “Analyzing video game completion achievements: Implications for game project

scope,” IFIP International Conference on Entertainment Computing (ICEC) 2017. Peer reviewed. Published in

Lecture Notes in Computer Science (LNCS) 10507, pages 469-472. Poster presentation on September 19th, 2017.

Tsukuba, Japan. doi: 10.1007/978-3-319-66715-7_65

Oral Presentations at International Industrial Conferences:

(8) Sato, N., & Bailey, E. (2017). GRAVITY DAZE 2 Geemu shinario seisaku: Mata wa watashi wa ika ni shite shinpai

suru no o yamete seigen o ai suru you ni natta ka. Computer Entertainment Developers Conference (CEDEC)

2017. Oral presentation by Sato, N. on September 1st, 2017. Yokohama, Japan.

References:

Argyris, C. 1977. Double Loop Learning in Organizations. Harvard Business Review, 55(5), 115-125.

Bauckhage, C., Kersting, K., Sifa, R., Thurau, C., Drachen, A. & Canossa, A. 2012. How players lose interest in playing

a game: An empirical study based on distributions of total playing times. 2012 IEEE Conference on

Computational Intelligence and Games (CIG), 139-146.

Bilton, C. 2017. The disappearing product: Marketing and markets in the creative industries, Edward Elgar Publishing.

Hullet, K., Nagappan, N., Schuh, E. & Hopson, J. 2011. Data analytics for game development. Proceedings of the 33rd

International Conference on Software Engineering, 2011, ICSE '11, 940-943.

Ikuine, F. 2006. Gemu sofuto no inobeeshon pataan: Kaihatsu seisansei no direnma [Innovation patterns in the video

game software industry: The development productivity dilemma]. Doctoral dissertation, University of Tokyo.

Parkin, S. 2018. The great video game exodus [Online]. Gamasutra. Available:

https://www.gamasutra.com/view/news/318588/The_great_video_game_exodus.php [Accessed August 8th,

2019].

Smith, S. L., Choueiti, M., Pieper, K., Case, A. & Choi, A. 2018. Inequality in 1,100 Popular Films: Examining Portrayals

of Gender, Race/Ethnicity, LGBT & Disability from 2007 to 2017. Annenberg Foundation.

Tschang, F. T. 2007. Balancing the tensions between rationalization and creativity in the video games industry.

Organization Science, 18(6), 989-1005.

Weber, B. G., John, M., Mateas, M. & Jhala, A. 2011. Modeling player retention in Madden NFL 11. Proceedings of the

Twenty-Third Conference on Innovative Applications of Artificial Intelligence.

Williams, D., Martins, N., Consalvo, M. & Ivory, J. D. 2009. The virtual census: Representations of gender, race and age

in video games. New Media & Society, 11, 815-834.

THE ABSTRACT FOR FORMAL HEARING

AStudyonUserRequirementsExtractionandCulturalAttributesforProductDesign

June 23, 2020

Graduate School of Advanced Science and Technology Japan Advanced Institute of Science and Technology.

Nagai Laboratory Knowledge Science

StudentID:S1620405Name:KIEUQueAnh

Supervisor:ProfessorYukariNagai

ResearchContentUser requirements play an important role in Product design. The requirements

can be used to help designers to determine the product's features for the design

process. Besides, it can be considered as the goal for designers in designing

products. Current approaches to obtaining user requirements usually use some

traditional methods such as surveys, questionnaires, and interviews. These

methods can gain some important information about products by interacting with

customers. However, the methods have some drawbacks as described as follows:

First, it is clear that obtaining the user's information in such a way is expensive.

Also, it is difficult to obtain a large number of users' information by using

traditional methods because we can only access a limited number of users.

Furthermore, sometimes we cannot get their real emotions. Therefore, the

obtained information could not reflect all aspects of what users need. Meanwhile,

the information about products on the web is available and growing rapidly. The

information comes from a very large number of users in different cultures. This

information is an essential knowledge for the new users when they would like to

purchase the product. Thus, it is important information for designers in designing

attractive products for users. This thesis presents a method for bridging user

requirements to designers in Product design. The motivation behind this method

is that user requirements are automatically collected by performing an opinion

mining method on a set of online product reviews, which is available on the web.

The user requirements are then provided to integrate with designers. In the thesis,

we investigate various ways to extract user opinions from customer's opinions for

designers in terms of designing products. The main contents of the thesis are

sketched as follows.

Chapter 3 investigates a study on verifying the contribution of top reviews and

helpful reviews on product design. We apply advanced machine learning models

to identify helpful reviews automatically and exploit the contribution of helpful

reviews on designing process. In the technical point of view, we compare various

machine learning models to select appropriate one for helpful reviews

identification task. We also evaluated our results with designers to understand

what is the perspective of designers with helpful reviews.

Chapter 4 proposes a user requirements extraction framework which can extract

aspects of products, summarizing reviews opinion corresponding with aspects

and classifying them into positive and negative categories. We propose a

supervised learning model and unsupervised model for extracting aspects

automatically. Experimental results on the standard data showed our method

attain high accuracy in comparison with strong baseline models. We proposed a

novel method using opinion summarization methods, along with the use of

keyword extraction to obtain user requirements. In addition to that, we propose

a deep learning model for sentiment classification and considering them as

reference information. Some case studies for evaluating the use of user

requirements with designers are carried out. The results of questionnaire by

interviewing with designers showed that the proposed framework is effective for

supporting designers in the early phase of product design.

Chapter 5 shows our third contribution by considering the cultural attributes and

their impact on product design in the early process of product design. We

investigate how cultural attributes can be able to change and effect on selecting

design concept. A case study with designers showed the contribution of cultural

attributes and user requirements in product design. The results showed that

cultural attributes and user requirements can help designers creating new

concepts for cultural-oriented product design.

Research Purpose This research aims at studying the bridge method of utilizing user feedback from

online customers for supporting designers in terms of designing products. The

research also focuses on answering the question of how cultural attributes can

contribute to the design process and design creativity.

More precisely, we aim at tackling the problem of utilizing user feedback to

support designers in terms of designing products. There are a lot of reviewers

available on the internet, but we would like to focus on top reviews and the

helpfulness of reviews for being considered in product design. For regular

reviews, we perform the uses of opinion summarization techniques to get user

requirements and form it into an appropriate format for designers to support them

in terms of designing products.

On the other important aspect, we aim at studying how cultural attributes can

combine with user requirements for supporting designers in the early phase of

product design. For this purpose, we would like to interview with designers from

various countries to understand the contribution of cultural attributes to product

design.

Research Accomplishment

○ Kieu Que Anh, Yukari Nagai, and Minh Nguyen, "Extracting Customer Reviews from Online Shopping and Its Perspective on Product Design”, Vietnamese Journal of Computer Science Vol. 6 No. 1 (2019) 43-56

◎Q.A.Kieu and Y. Nagai, ”Extracting User Requirements from online reviews for Product Design: A Supportive Framework for Designers, Journal of Intelligent & Fuzzy Systems, Vol. 37 no.6, pp. 7441-7451, 2019

◎ Kieu Que Anh, Yukari Nagai, Cultural-Oriented Product Design: A Case Study for Vietnamese Product, In Proceedings of KICSS 2019.

◎ Kieu Que Anh and Yukari Nagai,” Extracting User Requirements for Product Design: A

Supportive Framework for Designers, In Proceedings of Redo 2017, pp 594-760 ◎ Kieu Que Anh, Yukari Nagai, and Minh Nguyen, "Extracting Customer Reviews from

Online Shopping and Its Perspective on Product Design”, In Proceedings of KICSS 2017 conference, Nagoya, Japan

◎ Kieu Que Anh and Yukari Nagai, A Vietnamese culture framework for Product Development, JAIST World Conference: Advanced Design Creativity Track (JWC 2019)

1

公聴会⽤博⼠学位論⽂概要 論⽂題⽬： リスクマネジメントにおける知識創造と機械学習の統合アプローチ

―― 機械参加型（machine-in-the-loop）プロセスの提案 ―― 希望取得学位： 博⼠（知識科学） 研究室名： 内平研究室 学⽣番号： 1640416 ⽒名： 森 俊樹

第１部：研究内容

本格的な知識社会、すなわち「知識が中核の資源となり、知識労働者が中核の働き⼿となった社会」を迎えて、企業を取り巻く環境の変化はますます激しくなっている。こうした状況の中で企業が持続的競争優位を確⽴するためには、変化に対応して柔軟に適応していく能⼒の構築が必要であり、プロジェクト・リスクマネジメントの⼀層の強化が不可⽋である。しかしながら、リスクマネジメントのプロセスや⼿法は概ね標準化されているにも関わらず、その効果的な実践や定着化は必ずしも容易ではないという実態がある。従来、リスクマネジメントへのナレッジマネジメントの応⽤や⼈⼯知能（AI）・機械学習の応⽤などがそれぞれ個別に論じられてきたが、リスクマネジメントの本質的な課題に対して⼗分に踏み込めているとは⾔い難い。

本研究では、プロジェクト・リスクマネジメントの課題に関して実務者のインタビュー調査を実施し、帰納的テーマティック・アナリシス法（TA: Thematic Analysis）による分析を実施した。その結果、プロジェクト・リスクマネジメントの標準的なプロセスや技法などの“理想形”と、現場におけるリスクマネジメントの“実践”との間には⼤きなギャップがあり、それがリスクマネジメントの形骸化など、さまざまな実践上の難しさにつながっていることがわかった。このような実践上の難しさに対して、「限られた時間やコストやリソースの中で、不確実さを伴うさまざまな事象や状態に対して、トレードオフを含む意思決定を適切なタイミングで実施することの難しさ」を本質的な課題と仮定し、これに利害関係者の限定合理性に起因する取引コストの影響と⼈間の⼼理的特性に起因する認知バイアスの影響が作⽤して合理的な判断からの乖離が⽣じていると考察した。帰納的 TA の分析結果に対して、上記仮説との対応付けを⾏ったところ、リスクマネジメントの"難しさ"に関するコード全 41 件のうち、31 件（75.6%）は取引コストまたは認知バイアスの影響を受けていることが⽰され、上記仮説および理論的考察の妥当性を確認した。

プロジェクト・リスクマネジメントの本質的な課題に対応するには、⼈間の気づきや直観的な判断（システムⅠ）と合理的な思考（システムⅡ）を効果的に連携した意思決定⽀援の

2

枠組みが有効である。そこで、本研究では、⼈間と機械学習が協調してリスクに関する意思決定や知識創造を⾏う機械参加型（machine-in-the-loop）リスクマネジメントの枠組みを提案した。⼈間と機械学習は相補的な関係にあり、⼈間は、機械学習の予測結果を⽤いて⼈間がもつ先⼊観やバイアス（偏り）を排除し、合理的な意思決定につなげることができる⼀⽅、機械学習は、⼈間からの気づきやフィードバック（現実との⽭盾点や経験・感覚との違いなど）を得ることにより、データに不⾜している背景知識などを補完し、新たな変化や想定外の事象への対応⼒を⾼めることができる。機械参加型リスクマネジメントの適⽤により、（１）不確実さの低減による意思決定の合理化、（２）利害関係者間の合意形成による取引コストの低減、（３）システムⅠとシステムⅡの連携による認知バイアスの影響の緩和、が期待される。提案アプローチの実適⽤の効果についてインタビュー調査を実施し、演繹的テーマティック・アナリシス法（演繹的 TA）を⽤いて上記仮説を検証した。

機械参加型リスクマネジメントを実現するには、機械学習モデルが⼈間の意思決定プロセスに介⼊して、（１）新たな情報の提⽰による気づきの⽀援や（２）予測・推定結果に基づく新たな判断材料の提供、などを⾏う必要がある。そのためには、予測精度に優れ、かつ、解釈可能性も備えた機械学習モデルが求められる。しかしながら、⼀般に、機械学習モデルの予測精度と解釈可能性はトレードオフの関係にあった。そこで、本研究では、提案アプローチに適した機械学習モデルとして、ナイーブベイズ分類器を拡張した新たな機械学習モデル、SNB（superposed naive Bayes）を提案した。SNB は、ナイーブベイズの集団学習モデルを構築した後、線形近似で単⼀のナイーブベイズに逆変換することにより、予測精度と解釈可能性の両⽴を実現している。公開データセットを⽤いて SNB と他の主要な機械学習モデルのベンチマーク評価を実施し、提案⼿法（SNB）が、予測精度と解釈可能性の両⾯においてバランスのとれた予測結果を出⼒することを確認した。

第２部：研究の意義

プロジェクト・リスクマネジメントのプロセスや⼿法は概ね標準化されている（ISO 2009; PMI 2017）にも関わらず、その効果的な実践や定着化は必ずしも容易ではない（⽊野 2000; Bannerman 2008; Kutsch and Hall 2010）。そうした困難性への対応として、ナレッジマネジメントの適⽤（Massingham 2010; Alhawari et al. 2012; 内⽥ 2016）や⼈⼯知能（AI）・機械学習の適⽤（Takagi et al. 2005; Lee et al. 2009; Mendes et al. 2018）などが提案されている。しかしながら、これらの研究では、リスクマネジメントの本質的な課題に対して、⼗分に踏み込んだ分析・考察が⾏われているとは⾔い難い。そこで、本研究では、プロジェクト・リスクマネジメントの困難性に対して、取引コスト理論（Williamson 1981; 菊澤 2016）とプロスペクト理論（Kahneman 2011; 友野 2006）という新たな切り⼝から分析・考察した。その結果、不確実な状況下でのトレードオフを伴う意思決定の問題に加えて、取引コストや認知バイアスの影響が作⽤することにより合理的な判断からの偏りが⽣じて、リスク

3

マネジメントの形骸化などのさまざまな実践上の難しさにつながっていることを⽰した。プロジェクト・リスクマネジメントの本質的な課題に対して、取引コスト理論およびプロスペクト理論という新たな切り⼝から理論的考察を⾏っている点に本研究の新規性がある。

続いて、本研究では、上記プロジェクト・リスクマネジメントの本質的な課題への対応として、機械学習と知識創造の統合アプローチである機械参加型（machine-in-the-loop）リスクマネジメントの概念を⽰した。それは、⼈間と機械学習の相補性を最⼤限に利⽤し、⼈間の意思決定プロセスに機械学習が積極的に介在して気づきの⽀援や意思決定のための判断材料の提⽰を⾏うと共に、⼈間側からも⾏動結果や予測のギャップなどのフィードバックをもらうことにより、⼈間の知識更新と機械学習のモデル更新を同時並⾏的に⾏っていく仕組みである。ビジネスナレッジ側とデータ分析側がそれぞれ独⽴したサイクルをもつという意味では Wang and Wang（2008）とも似ているが、SECI モデルの各知識変換モードにおいて両者が密接に連携している点が異なっている。機械が⼈間の活動を⽀援するmachine-in-the-loop の概念⾃体は既に発表されている（Clark et al. 2018）が、その定義やアーキテクチャーについては、まだ⼗分に確⽴しているとは⾔い難い。プロジェクト・リスクマネジメントへの適⽤において本質的な課題を掘り下げた上で、machine-in-the-loop の具体的な枠組みや適⽤効果について踏み込んで論じている点に、本研究の理論的貢献があると考える。

さらに、本研究では、予測精度と解釈可能性を両⽴した新しい機械学習モデルを提案している。従来、さまざまな機械学習モデルが提案されてきたが、⼀般に、解釈可能性に優れた単純なモデル（例えば、決定⽊、ナイーブベイズ、ロジスティック回帰、など）は予測精度が低くなる⼀⽅、予測精度が⾼い複雑なモデル（例えば、SVM、ニューラルネットワーク、ランダムフォレスト、など）は解釈可能性が低下するというトレードオフ関係が存在した。しかしながら、machine-in-the-loop における⼈間と機械学習の協調においては、予測精度と解釈可能性をバランスよく両⽴することが望ましい。そこで、本研究では、新しい機械学習モデル SNB（superposed naive Bayes）を提案、ナイーブベイズのアンサンブル（集団学習）モデルを構築して予測精度を⾼めた上で（ステップ１）、その線形近似により単⼀のナイーブベイズに逆変換する（ステップ２）という２段階アプローチの採⽤により、予測精度と解釈可能性の両⽴を実現した。ここで、第１のステップは主に予測精度の向上に寄与し、第２のステップは主に解釈可能性の向上に寄与すると考えられる。実験によって提案⼿法（SNB）と主要な機械学習モデルを予測精度と解釈可能性の２つの観点から⽐較・評価し、提案⼿法（SNB）が、予測精度と解釈可能性の両⾯においてバランスの取れた予測結果を出⼒できることが確認できた。上記、新しい機械学習モデル（SNB）の提案、および、予測精度と解釈可能性のトレードオフ分析の⽅法において、本研究の新規性を主張する。

本研究は、プロジェクトマネジメント、ナレッジマネジメント、機械学習の３つの領域にまたがった研究であり、それぞれの領域おける研究の進化と結び付いて、今後、さらなる発展が期待できる。

4

第３部：研究業績

A. 学術誌掲載論⽂ A-1.（査読あり）※ 2018 Journal Impact Factor: 4.457

Mori, Toshiki, and Naoshi Uchihira, 2019, "Balancing the trade-off between accuracy and

interpretability in software defect prediction," Empirical Software Engineering, 24.2: 779-825.

A-2.（査読あり） 森俊樹・内平直志，2019,「プロジェクトとプログラムのリスクマネジメントにおける機械学習と知識創造の統合アプローチ」『国際 P2M 学会誌』，14.1: 415-435． B. 国際学会⼝頭発表論⽂ B-1.（査読あり） Mori, Toshiki, Shurei Tamura, and Shingo Kakui, 2013, "Incremental estimation of project failure risk

with Naive Bayes classifier," 7th ACM/IEEE International Symposium on Empirical Software

Engineering and Measurement, Baltimore: IEEE, 283-286.

B-2.（査読あり） Mori, Toshiki, 2015, “Superposed naive Bayes for accurate and interpretable prediction,” 14th IEEE

International Conference on Machine Learning and Applications (ICMLA), Miami: IEEE, 6 pages.

B-3.（査読あり）※ Journal First Paper

Mori, Toshiki, and Naoshi Uchihira, 2019, "Balancing the trade-off between accuracy and

interpretability in software defect prediction," 34th IEEE/ACM International Conference on

Automated Software Engineering (ASE), San Diego: IEEE/ACM.

C. 国内学会⼝頭発表論⽂ C-1.（簡易査読あり） 森俊樹・内平直志，2018，「リスクマネジメントにおける機械学習と知識創造の統合アプローチ」，プロジェクトマネジメント学会 2018 年度春季研究発表⼤会，東洋⼤学，6 pages. D. その他 D-1.（簡易査読あり） 森俊樹・覚井真吾・⽥村朱麗・藤巻昇，2013，「プロジェクト失敗リスク予測モデルの構築 」『プロジェクトマネジメント学会誌』，15.4: 3-8． D-2.（簡易査読あり） 森俊樹・覚井真吾・⽥村朱麗，2014，「プロジェクト失敗リスク予測モデル」『東芝レビュー』，69.1: 47-50．