statistical analysis of gene expression data with oracle ... · pdf filestatistical analysis...

Statistical Analysis of Gene Expression Data With

Oracle & “R” (- data mining)

Patrick E. Hoffman Sc.D. Senior Principal Analytical

Consultant pat.hoffman@oracle.com

Agenda (Oracle & R Analysis)

ToolsLoading DataStatistical Analysis

– Most Important Genes MDL

– Correlations– t statistics, etc– R Visualizations

Predictive models?Other?

Analysis Tools

Tools For DB * PLSQL development– ODM (Oracle Data Miner)– TOAD (From Quest)– Jdeveloper (Free from Oracle) – does Java & plsql

development– Enterprise Manager Console ( Oracle client)

Managing the whole DB– SQLPlus (command line sql or plsql)

R Project– Open source clone of S-Plus

TextPad (editor)

The data

Affy gene expression (old AML/ALL)7129 genes72 patients (combined train/test)

Can be expanded to the new chips (50,000 genes)

Loading Data into Oracle DB?From flat file?

SQL*LoaderOracle Warehouse Builder

– Production (auto sql*ldr)– All types of data

Oracle Data Miner (ODM)– Quick & Easy

From R-Project tables

Screen Shots ODM’rto load csv file

Load Data From R to DB

Set up ODBC driver (in Windows)Download RODBC pkg (from cran)Load CSV fileSqlsave routine to store in DB

R code – send csv to db# you must install(download) the RODBC first# r script to load a csv file to Oracle db# install packages(RODBC to connect to a database) from CRAN (menu option)library(RODBC)

# use standard microsoft odbc connection# set up a dsn name to connect to correct service# setup the channel to database, ODBC must be set up to connect to Oracle DBchan1 <- odbcConnect(dsn="")odbcGetInfo(chan1) #make sure channel is ok

# load in a csv filefilename <- "D:\\clients\\affy\\gs_EXPRESSION.csv"csv <- read.csv(filename)

sqlDrop(chan1, "GS_EXPRESSION", errors = TRUE) #drop the old table# load the csv file to a table in the Oracle dbsqlSave(chan1, csv, tablename = "GS_EXPRESSION",

rownames = FALSE,colnames = T, fast = F) # fast did not work on 10g

7000 genes? 1000 column DB Limit?

Gone! Transactional FormatorNested Columns

Convert flat file table to one of these formats

Analysis

Pl/Sql Package affy– affy_to_trans Flat to Transactional– affy_ai Calculate MDL Attrib. Import.– trans_to_affy Convert top gene to flat form.– corr_genes Correlate genes to genes– corr_cases Correlate cases to cases– corr_target Correlate genes to target – T – statistic Calculate t-statistics– anova Stats for Multiple Classes

Convert gene expression tableto transactional format

Load data into table GS_EXPRESSIONUse Affy packageexec affy.affy_to_trans('GS_EXPRESSION');

OUTPUT - will be in the table AFFY_TRANS

Normal format

Transactional Format (72*700)

PLSQL Affy_to_Trans

ODM will give stats

And Histograms

For Transactional & Flat Tables

Histogram for One Gene

Attribute Importance Target is ALL or AML

This is Minimum Distance Length (MDL) algorithm

ODM GUI or plsql API

Top Genes by MDL (no normalization)

Classification & Clustering (ODM)

Transactional or Flat tablesSVM, Naïve Bayes, Adaptive BayesJava, Plsql, APIJava based GUIAdvanced K-means, Orthogonal ClusteringLift and ROC curves

10g Statistics & SQL AnalyticsRanking functions

– rank, dense_rank, cume_dist, percent_rank,ntile

Window Aggregate functions (moving and cumulative)

– Avg, sum, min, max, count, variance, stddev, first_value, last_value

LAG/LEAD functions– Direct inter-row reference using offsets

Reporting Aggregate functions– Sum, avg, min, max, variance, stddev, count,

ratio_to_report

Statistical Aggregates– Correlation, linear regression family, covariance

Linear regression– Fitting of an ordinary-least-squares regression

line to a set of number pairs. – Frequently combined with the COVAR_POP,

COVAR_SAMP, and CORR functions.

Descriptive Statistics– average, standard deviation, variance, min, max, median

(via percentile_count), mode, group-by & roll-up– DBMS_STAT_FUNCS: summarizes numerical columns

of a table and returns count, min, max, range, mean, stats_mode, variance, standard deviation, median,quantile values, +/- 3 sigma values, top/bottom 5 values

Correlations– Pearson’s correlation coefficients, Spearman's and

Kendall's (both nonparametric).

Cross Tabs– Enhanced with % statistics: chi squared, phi coefficient,

Cramer's V, contingency coefficient, Cohen's kappa

Hypothesis Testing– Student t-test , F-test, Binomial test, Wilcoxon Signed

Ranks test, Chi-square, Mann Whitney test, Kolmogorov-Smirnov test, One-way ANOVA

Distribution Fitting– Kolmogorov-Smirnov Test, Anderson-Darling Test, Chi-

Squared Test, Normal, Uniform, Weibull, Exponential

Pareto Analysis (documented)– 80:20 rule, cumulative results table

Analytical Functions with TX format

Correlating cases with casesCorrelating genes with genesCorrelating genes with TARGET (all/aml)T- statisticsANOVA

Correlating Cases-- put caseid’s to correlate in table aftran2 create view aftran2 as select distinct(caseid) from affy_trans;

create table pcor1 asselect a.caseid p1, b.caseid p2, corr(a.attr_value,

b.attr_value) corrfrom affy_trans a, affy_trans bwhere a.caseid < b.caseid and

a.attrib = b.attrib anda.caseid in (select * from aftran2)

group by a.caseid, b.caseidhaving corr(a.attr_value, b.attr_value) > .93 orcorr(a.attr_value, b.attr_value) < -.93;

199 cases correlated > .93

Correlating Genes

select a.attrib p1, b.attrib g2, corr(a.attr_value, b.attr_value) corr

from affy_trans a, affy_trans bwhere a.attrib < b.attrib and

a.seqnum = b.seqnum anda.attrib = 'X95735_at‘ --- zyxin

group by a.attrib, b.attribhaving corr(a.attr_value, b.attr_value) > .5 orcorr(a.attr_value, b.attr_value) < -.5;

208 genes correlate with zyxinacross 72 patients

Correlating Genes with TARGET

select a.attrib p1, b.attrib g2, corr(a.attr_value, b.attr_value) corr

from affy_trans a, affy_trans bwhere a.attrib < b.attrib and

a.seqnum = b.seqnum anda.attrib = ‘TARGET‘ --- AML/ALL

group by a.attrib, b.attribhaving corr(a.attr_value, b.attr_value) > .5 orcorr(a.attr_value, b.attr_value) < -.5;

57 genes correlate with TARGET

R Correlationslibrary(RODBC)chan1 <- odbcConnect(dsn="")odbcGetInfo(chan1) #make sure channel is ok# get gene expression data from db, but drop gene namegs1 <- sqlQuery(chan1, query = "create table gs1 as SELECT *

FROM GS_EXPRESSION")gs1 <- sqlQuery(chan1, query = "alter table gs1 drop column gene")gs <- sqlQuery(chan1, query = "SELECT * FROM GS1")# get correlation matrixcm <- cor(gs,use="pairwise.complete.obs")# Write the new table to a file.write.table( cm, file="D:\\clients\\affy\\gs_cm.csv", append = FALSE,

quote = FALSE, sep = ",", eol = "\n", na = "", dec = ".", row.names = T, col.names = T )

heatmap(cm, Rowv=NA, Colv=NA,symm=TRUE)

Case Correlation Matrix

Heatmap from R Correlation Matrix

Other Statistics

Corr_s - Spearman’s rho correlation coef. Corr_k - Kendall's tau-b correlation coef.stats_t_test_indep - equal variancestats_t_test_indepu - unequal variancestats_one_way_anova

T statistics for each gene (tx format)Create table t_stats asSELECT a.attrib, count(*) cnt,avg(a.attr_value) avg_atr,

avg(b.attr_value) avg_trg,STATS_T_TEST_INDEP(b.attr_value, a.attr_value,

'STATISTIC') t_observed,STATS_T_TEST_INDEP(b.attr_value, a.attr_value)*7130

two_sided_p_value FROM affy_trans A, affy_trans bWHERE a.attrib in (select * from aftran1) and

b.attrib = 'TARGET' and a.caseid = b.caseid

group by a.attrib;

T-statistic output table (Bonferroni correction)

F-distribution one way ANOVAdrop table anova;create table anova asSELECT a.attrib, count(*) cnt,avg(a.attr_value) avg_atr,

avg(b.attr_value) avg_trg,STATS_ONE_WAY_ANOVA(b.attr_value, a.attr_value,

'F_RATIO') f_ratio,STATS_ONE_WAY_ANOVA(b.attr_value, a.attr_value,

'SIG')*7130 p_valueFROM affy_trans A, affy_trans bWHERE a.attrib in (select * from aftran1) and b.attrib =

'TARGET' anda.caseid = b.caseid

group by a.attrib;

Top genes by ANOVA

R for Plotting and Visualization

#### get data and plot all variablesa1 <- sqlQuery(chan1, query = "select * from anova")plot(a1)

Scatterplot Matrix in R

Histograms of Expression Distribution#Generate Histograms of gene expression cases, with labels and cut offs

pam1 = 40 #Barspam2 = 5000 #Ceiling pam3 = -1000 #floorwx = 1000 #widthhx = 714 #floorN <- ncol(csv) # gene expression data is in csv N = 3 # do only 3 histogramsR <- nrow(csv)nom<- attr(csv, "names")par(mfrow = c(N-1,1))b <- 0:pam1c = pam1/(pam2 - pam3)b <- b/cb <- b + pam3for( num in 2:N) {

h1 <- csv[,num]h1[ h1>pam2] <- pam2h1[ h1<pam3] <- pam3h<- hist(h1,nclass=pam1,breaks=b, main = paste("Histogram of" , nom[num],"clamp

at",pam2,pam3), xlab = nom[num], col=5)}

3 columns of expression dist.

Histogram – no labelsN <- ncol(csv) # number of columnspam1 = 40 #Barspam2 = 5000 #Ceiling pam3 = -1000 #floorR <- nrow(csv)#make the break pointsb <- 0:pam1c = pam1/(pam2 - pam3)b <- b/cb <- b + pam3par(mfrow = c(N,1),mar=c(0,0,0,0)) # space on graphfor( num in 2:N) {

h1 <- csv[,num]h1[ h1>pam2] <- pam2h1[ h1<pam3] <- pam3h<- hist(h1,breaks=b, main = "", xlab = "",axes=F, col=5)

}

All 72 cases of expression dist.

More?

Many other applications of R BioconductorMany other applications of OracleOther code is available

Oracle Informatics Consulting

Life SciencesDM Workshop

A one day onsite technical session educating organizations on how to leverage one of their most valuable assets to provide insight in the operations of their business, the behavioral patterns of their customers and hidden relationships found deep within corporate data that can have direct impact to the bottom line.

Life SciencesDM Blueprint

A documented technical roadmap providing the organization with the strategy to integrate and deploy Life Sciences technology. This includes recommendations based on feedback from the Life Sciences workshop focusing on source data preparation, mining methodologies and supporting architecture.

Life SciencesDM Insight

A five day onsite engagement focused on providing a detailed analysis of the business problem, data preparation, model build and analysis and knowledge deployment extending the analysis of the Life Sciences workshop culminating with a technical roadmap with a strategy to integrate and deploy Life Sciences technology.

Life SciencesDMQuickstart

A thirty day engagement focused on taking a business problem and transforming into a Life Sciences solution. This includes transforming the business problem, preparing e data, creation of the mining model and knowledge deployment. Upon completion, results will be delivered mapped to the initial business problem.

Life SciencesDM Services

A series of custom services focused on delivering Life Sciences methodologies and solutions to provide insight in the operations of their business, the behavioral patterns of their customers and hidden relationships found deep within corporate data that can have direct impact to the bottom line.

Life Science Informatics experience

Gene expression analysisSequence Analysis (blast exon/intron prediction)Clinical/Medical data analysisQSAR/Cheminformatics – Isis,Molconz, Predictive ToxAnimal StudiesProtein analysis (arrays, Mass spec)Ontology's and Text Mining

Data Mining & Informatics Services

Contact Richard Solari

508-477-5765

630-561-9950

Richard.solari@oracle.com

Contact Patrick Hoffman

781-744-0783

617-755-6740

Pat.hoffman@oracle.com