document classification with neo4j

(graphs)-[:are]->(everywhere)

Document Classification with Neo4j

@kennybastani

Neo4j Developer Evangelist

Agenda

• Introduction to Neo4j

• Introduction to Graph-based Document Classification

• Graph-based Hierarchical Pattern Recognition

• Generating a Vector Space Model for Recommendations

• Graphify for Neo4j

• U.S. Presidential Speech Transcript Analysis

Introduction to Neo4j

The Property Graph Data Model

Graph Databases Book

Friend Of

Has Read

name: John age: 27

name: Sally age: 32

title: Graph Databasesauthors: Ian Robinson, Jim Webber

FRIEND_OFsince: 01/09/2013

HAS_READon: 2/03/2013rating: 5

HAS_READon: 02/09/2013rating: 4

FRIEND_OFsince: 01/09/2013

The Relational Table Model

Customers AccountsCustomer_Accounts

143 Alice

326 $100

725$63

981 $212

143 981

143 725

143 326

The Neo4j Browser

http://localhost:7474/

Neo4j Browser - finding help

Execute Cypher, Visualize

Introduction to Document Classification

Document Classification

Automatically assign a document to one or more classes

Documents may be classified according to their subjects or

according to other attributes

Automatically classify unlabeled documents to a set of

relevant classes using labeled training data

Example Use Cases for Document Classification

Sentiment Analysis for Movie Reviews

Scenario: A movie website allows users to submit reviews describing what they either liked or disliked about a particular movie.

Problem: The user reviews are unstructured text.

How do I automatically generate a score indicating whether the review was positive or negative?

Solution: Train a natural language parsing model on a dataset that has been labeled in previous reviews as either positive or negative.

Recommend Relevant Tags

Scenario: A Q/A website allows users to submit questions and receive answers from other users.

Problem: Users sometime do not know what tags to apply to their questions in order to increase discoverability for receiving answers.

Solution: Automatically recommend the most relevant tags for questions by classifying the text from training on previous questions.

Recommend Similar Articles

Scenario: A news website provides hundreds of new articles a day to users on a broad range of topics.

Problem: The site needs to increase user engagement and time spent on the site.

Solution: Train natural language parsing models for daily articles in order to provide recommendations for highly relevant articles at the bottom of each page.

How Automated Document Classification Works

X YDocument

Document

Label Label

Assign a set of labels that describes the document’s text

Supervised Learning

Step 1: Create a Training Dataset

State machines represent predicates that evaluate to 0 or 1 for a text match

Deep feature representations are selected and learned using an evolutionary algorithm

Step 2: Train a Natural Language Parsing Model

State machines map to classes of document labels that matched text during training

Deep Learning

= State Machine

Unlabeled Document

The natural language parsing model is used to classify other

unlabeled documents

XClass

YClass

ZClass

cos(θ)

Step 3: Classify Unlabeled Documents

Hierarchical Pattern Recognition (HPR)

What is Hierarchical Pattern Recognition (HPR)?

HPR is a graph-based deep learning algorithm I created that learns deep feature representations in linear time —

I created the algorithm to do graph-based traversals using a hierarchy of finite state machines (FSM).

Designed for scalable performance in P time:

Influences & Inspirations

Ray Kurzweil(Pattern Recognition Theory of

Jeff Hawkins(Hierarchical Temporal

Memory)

Hierarchical Pattern Recognition

How does feature extraction work?

Hierarchical Pattern Recognition

“Deep” feature representations are learned and associated with labels that are mapped to documents that the feature was discovered in.

The feature hierarchy is translated into a Vector Space Model for classification on feature vectors generated from unlabeled text.

HPR uses a probabilistic model in combination with an evolutionary algorithm to generate hierarchies of deep feature representations.

Graph-based feature learning

Learning new features from matches on training data

Cost Function for the Generations of Features

Reproduction occurs after a threshold of matches has been exceeded for a feature.

After replication the cost function is applied to increase that threshold every time the feature reproduces.

is the current threshold on the feature node.

is the minimum threshold, which I chose as 5 for new features.

Cost function:

Vector Space Model

Generating Feature Vectors

The natural language parsing model created during training can be turned into a global feature index.

This global feature index is a list of Neo4j internal IDs for every feature in the hierarchy.

Using that global feature index, a multi-dimensional vector space is created with a length equal to the number of features in the hierarchy.

Relevance Rankings

“Relevance rankings of documents in a keyword search can be calculated, using the assumptions of document similarities theory, by comparing the deviation of angles between each document vector and the original query vector where the query is represented as the same kind of vector as the documents.” - Wikipedia

Vector-based Cosine Similarity Measure

In practice, it is easier to calculate the cosine of the angle between the vectors, instead of the angle itself:

Cosine Similarity & Vector Space Model

Vector-based Cosine Similarity Measure

“The resulting similarity ranges from -1 meaning exactly opposite, to 1 meaning exactly the same, with 0 usually indicating independence, and in-between values indicating intermediate similarity or dissimilarity.”

via Wikipedia

Graphify for Neo4j

Graphify is a Neo4j unmanaged extension used for document and text classification using graph-based hierarchical pattern recognition.

https://github.com/kbastani/graphify

Example Project

Head over to the GitHub project page and clone it to your local machine.

Follow the directions listed in the README.md to install the extension.

Navigate to the /examples directory of the project.

examples/graphify-examples-author/src/java/org/neo4j/nlp/examples/author/main.java

U.S. Presidential Speech Transcript Analysis

Identify the Political Affiliation of a Presidential Speech

This example ingests a set of texts from presidential speeches with labels from the author of that speech in training phase. After building the training models, unlabeled presidential speeches are classified in the test phase.

The Presidents• Ronald Reagan

• labels: liberal, republican, ronald-reagan

• George H.W. Bush

• labels: conservative, republican, bush41

• Bill Clinton

• labels: liberal, democrat, bill-clinton

• George W. Bush

• labels: conservative, republican, bush43

• Barack Obama

• labels: liberal, democrat, barack-obama

Training

Each of the presidents in the example have 6 speeches to analyze.

4 of the speeches are used to build a natural language parsing model.

2 of the speeches are used to test the validity of that model.

Get Similar Labels/Classes

Ronald Reagan

Class Similarity

republican 0.7182046285385341

liberal 0.644281223102398

democrat 0.4854114595950056

conservative 0.4133639188595147

bill-clinton 0.4057969121945167

barack-obama 0.323947855372623

bush41 0.3222644898334092

bush43 0.3161309849153592

George H.W. Bush

Class Similarity

conservative 0.7032274806766954

republican 0.6047256274615608

liberal 0.4439742461594541

democrat 0.39114918238853674

bill-clinton 0.3234223107986785

ronald-reagan 0.3222644898334092

barack-obama 0.2929260544514002

bush43 0.29106733975087984

Bill Clinton

Class Similarity

democrat 0.8375678825642422

liberal 0.7847858060182163

republican 0.5561860529059708

conservative 0.45365774896422445

barack-obama 0.4507676679770066

ronald-reagan 0.4057969121945167

bush43 0.365042482383354

bush41 0.3234223107986785

George W. Bush

Class Similarity

conservative 0.820636570272315

republican 0.7056890956512284

liberal 0.5075788396061254

democrat 0.4505424322086937

bill-clinton 0.365042482383354

barack-obama 0.33801949243378965

ronald-reagan 0.3161309849153592

bush41 0.29106733975087984

Barack Obama

Class Similarity

democrat 0.7668017370739147

liberal 0.7184792203867296

republican 0.4847680475425114

bill-clinton 0.4507676679770066

conservative 0.4149264161292232

bush43 0.33801949243378965

ronald-reagan 0.323947855372623

bush41 0.2929260544514002

Get involved in the Neo4j community

http://stackoverflow.com/questions/tagged/neo4j

http://groups.google.com/group/neo4j

https://github.com/neo4j/neo4j/issues

http://neo4j.meetup.com/

(Thank You)

Get in touch

Twitter www.twitter.com/kennybastani

www.linkedin.com/in/kennybastani

GitHub www.github.com/kbastani

document classification with neo4j

Technology

grails and neo4j

neo4j and bioinformatics

big data,neo4j developer,neo4j graph database,casandra...

practical neo4j

neo4j in depth

neo4j graph storage

wide screen template - imas...2019/08/06 · document...

teleform: automated information capture - hp.com document...

wikipedia document classification

neo4j - graph database

graph database & neo4j

neo4j graphacademy: certificate of completion...neo4j...

recommendations with neo4j

graph databases & neo4j

document classification: protected - owasp

neo4j nosqllive

drupal and neo4j

neo4j manual stable

neo4j manual 1.7.2

naive bayes document classification