Slide 1

StatisticalDesign& ModelsValidationIntroductionKnowledge InquiryHow we know, what we know andHow we know, we knowBouma Gary D. & G.B.J.Atkinson. (1995) A Handbook of Social Science Research. p.3DescriptionExplanationPredictionControlDevelopmentExplorationWhat, When Where, HowWhyPurpose of ResearchConfirmatory Factor Analysis & Path AnalysisInterestIdeaTheory? YY ?X YA B??A B C D E F G H IConceptualizationSpecify the meaning of the concepts and variables to be studied. OperationalizationHow will we actually measure the variables under study? Choice of Research MethodExperimental Research Survey Research Field Research Content Analysis Existing Data Research Comparative Research Evaluation Research Mixed DesignPopulation & SamplingWhom do we want to be able to draw conclusions about?Who will be observed for the purpose?ObservationCollecting data for analysis and interpretationData ProcessingTransforming the data collected into a form appropriate to manipulation and analysisAnalysisAnalyzing data and drawing conclusionsApplicationReporting results and assessing their implications.123546789Research Process & DesignInterestIdeaTheory? YY ?X YA B??A B C D E F G H IConceptualizationSpecify the meaning of the concepts and variables to be studied. OperationalizationHow will we actually measure the variables under study? Choice of Research MethodExperimental Research Survey Research Field Research Content Analysis Existing Data Research Comparative Research Evaluation Research Mixed DesignPopulation & SamplingWhom do we want to be able to draw conclusions about?Who will be observed for the purpose?ObservationCollecting data for analysis and interpretationData ProcessingTransforming the data collected into a form appropriate to manipulation and analysisAnalysisAnalyzing data and drawing conclusionsApplicationReporting results and assessing their implications.12579Research Process & DesignAnalysisDesignMeasurementDesignSamplingDesignResearchDesignData CollectingDesign3648InterestIdeaTheory? YY ?X YA B??A B C D E F G H IConceptualizationSpecify the meaning of the concepts and variables to be studied. OperationalizationHow will we actually measure the variables under study? Choice of Research MethodExperimental Research Survey Research Field Research Content Analysis Existing Data Research Comparative Research Evaluation Research Mixed DesignPopulation & SamplingWhom do we want to be able to draw conclusions about?Who will be observed for the purpose?ObservationCollecting data for analysis and interpretationData ProcessingTransforming the data collected into a form appropriate to manipulation and analysisAnalysisAnalyzing data and drawing conclusionsApplicationReporting results and assessing their implications.Research Process & DesignCross-sectional Study254721-3031-4041-50One-point of timeTrend Study2547255721-3021-3031-4031-4041-5041-50Same framework & instrumentsCohort Study2547255721-3021-3031-4031-4041-5041-50 51-60Same framework & instrumentsPanel Study2547255721-3021-3031-4031-4041-5041-50 51-60 Same individuals

Research Design: Time Dimensions8This presentation is only an overview of research. The only way to get better at research is to do it.Validity = Accuracy = Low BiasReliability = Precision = Low Variance

Probability DensityPrecisionReference valueAccuracyValueParameterStatisticsValidity and Reliability of Research FindingLow Validity = Low Accuracy = High BiasLow Reliability = Low Precision = High Variance

Probability DensityLow PrecisionReference valueLow AccuracyValueParameterStatisticsLow Validity and Low ReliabilityLow Validity = Low Accuracy = High BiasHigh Reliability = High Precision = Low Variance

Probability DensityPrecisionReference valueLow AccuracyValueParameterStatisticsLow Validity and High ReliabilityQuality of MeasurementA test with low validity because of low reliabilityA highly valid testA reliable test with low validity.Validity and Reliability of MeasurementNullHypothesis TestingNull Hypothesis TestingGoal: To determine if the independent variable has a statistically significant (real) effect on the dependent variable. That means, an effect that is UNLIKELY to be due to chance variations or sampling error.The null hypothesisResearchers make the initial assumption that the independent variable manipulation will have NO EFFECT on the dependent variable (will be null).

Under the null hypothesis, any observed difference between groups is assumed to be due to chance (random error) unless proven otherwise!

Inferential statistics are the tools used to resolve this question.

Inferential StatisticsTools for testing how likely it is that the results of a study are due to error or chance variation.

It is always possible that differences between groups & Relationship at the end of the study may have been due to sampling error, rather than being due to the independent variable.

Sampling error: the extent to which the groups were different at the start of the study.

Inferential StatisticsStatistical Significant: Type I error ()Type II error ()Power of test (1-)Confidence Interval (1-)Practical Significant: Effect size (2, 2)Sample DeterminationStatistical Model & AnalysisDescriptivestatisticsDescriptive StatisticsMeanStandard deviationVariance, CovarianceFrequency & Percentage & ratioPercentile, quartileMedian & modeRange, etc.Testing for Assumption of StatisticsKurtosisSkewnessNormal DistributionMultivariate NormalityMulticolinearityLinearityOutliersMean (Y)Mean (X1)Mean (X2)Mean (X3)Descriptive Statistics: How Importance?Measure of Central Tendency: Mean, Mode, MedianMeasure of Dispersion: Variance, Standard Deviation, Mean Deviation, RangeUnivariate: Variable, Variation & Variance2X12X22X32Y23YX1X2X3Descriptive Statistics:Mean Vectorvariance-covariance matrixBivariate: Variables, Variance & Covariance2X12X22X32Y2X12X22X32YBivariate: Variables, Variance & CovarianceCov (X1,Y)Cov (X1,X2)Cov (X1,X3)Cov (X2,X3)Cov (X2,Y)Cov (X3,Y)Cov (X1,Y)Cov (X1,X2)Cov (X1,X3)Cov (X2,X3)Cov (X2,Y)Cov (X3,Y)Statistical design&Conceptual Models10Y100100100010001000d1d2d1d2d3Observed variable(Nominal Scale)Observed variable(Interval Scale)1Latent variableCausal relationshipRelationshipd11Analysis UsingDependent Techniques10X1YOne-way ANOVA (Independent sample t-test)Statistical DesignYpostYpreOne-way ANOVA with repeated measured (Dependent sample t-test)Within-subjects Design??DifferentDifferentChange, Gain, DevelopmentBetween-subjects DesignDirect effectsDirect effectsBivariate Correlation Analysis (rxy)YXrxyYXZCov(x,y)rxyryzrxzCov(x,z)Cov(y,z)Cov(x,y)Standardized ScoreRaw ScoreStatistical DesignX1X2X3Y?Statistical DesignPartial & Part Correlation Analysis (Spurious or Indirect Causality)Direct effectsX1Y100100One-way ANOVA (F-test)YT2YT1One-way ANOVA with repeated measured Within-subjects DesignYT2????Between-subjects DesignStatistical DesignDirect effectsDirect effects10X1YTwo-way ANOVA (additive model) -- >No interaction effectsX2100100Main effect-X1Main effect-X2Between-subjects DesignStatistical DesignDirect effects10X1YTwo-way ANOVA (non-additive model) -- > Interaction effectsX2100100Main effectMain effectInteraction effectBetween-subjects DesignDirect effectsStatistical DesignY1010100100Multi-way ANOVA (the interactive structure)X1X2X3Between-subjects DesignStatistical DesignDirect effectsInteraction effectInteraction effectMain effectMain effectMain effectYOne-way Analysis of Covariance (ANCOVA) additive modelX1100100(Covariate)Z?Between-subjects DesignStatistical DesignY10100100Two-way ANCOVA (Interactive structure)ZX1X2(Covariate)Between-subjects DesignStatistical DesignDirect effectsMain effectInteraction effectMain effectInteraction effectMain effectX1X2X3YSimple Regression Analysis (SRA)Multiple Regression Analysis (MRA) (Convergent Causal structure)No Correlation(r = 0)Direct effectsy.x1 y.x2 y.x3XYy.xYXrxyStatistical DesignX1X2X3Multivariate Multiple Regression Analysis (MMR)(Convergent Causal structure two or several times)Y1Y2Direct effectsNo Correlation(r = 0)Statistical Design10X1X2X3Two-groups Discriminant Analysis (Discriminant structure)Binary Logistic Regression Analysis(Y)WWWDirect effectsNo Correlation(r = 0)Statistical DesignX1X2X3Multiple Discriminant Analysis(Discriminant Structure with more than two population groups)100100(Y)WWWDirect effectsNo Correlation(r = 0)Statistical DesignY11010100100Multivariate Analysis of Variance -- MANOVA(Interactive Structure two or several times)Y2X1X2X3Statistical DesignMain effectInteraction effectInteraction effectMain effectMain effectY110100100ZY2Multivariate Analysis of Covariance -- MANCOVA (Interactive Structure two or several times)X1X2(Covariate)Statistical DesignMain effectInteraction effectInteraction effectMain effectMain effectAnalysis UsingInterdependent TechniquesU1V1Canonical variates (Independent)Canonical variates (Dependent)U2V2RC1, 1X1X2X3X4Y1Y2Set of Independent variablesSet of Dependent variablesCanonical Function-1RC2, 2Canonical Loading2Canonical Loading2Simple CorrelationSimple CorrelationCanonical Correlation Analysis (CCA)Canonical weightCanonical WeightCanonical Function-2Statistical Model(Conceptualization)High

Low(Operationalization)Level of AbstractionConcept & ConstructVariablesIndicatorIndicatorIndicatorItemItemItemItemItemItemItemItemItemConceptual DefinitionTheoretical DefinitionReal DefinitionOperational Definition(How to measured?)Generalized ideaCommunicationReal worldHypothesis testingTime, Space, ContextTest-1Test-2Test-nPrinciple Component Analysis (PCA)231X1X2X3X4X5X6X7X8X9The Component Loading or the Structure/Pattern CoefficientFactor structure / Component / Dimensions / Unmeasured variablesMeasured variables (Observed) / Indicators / Items Statistical DesignMeasured variables (Observed) / Indicators / Items 231X1X2X3X4X5X6X7X8X9The Factor Loading or the Structure/Pattern CoefficientFactor structure / Component / Dimensions / Unmeasured variablesExploratory Factor Analysis (EFA) with Orthogonal RotationErrors or UniquenessStatistical ModelMeasured variables (Observed) / Indicators / Items 231X1X2X3X4X5X6X7X8X9The Factor Loading or the Structure/Pattern CoefficientFactor structure / Component / Dimensions / Unmeasured variablesExploratory Factor Analysis (EFA) with Oblique RotationErrors or UniquenessStatistical Model2,13,13,2Measurement Model:Construct X with 3 subdimensions or 3 factors231X1X2X3X4X5X6X7X8X92,13,13,22,11,13,14,25,26,27,38,39,3Statistical ModelMeasured variables (Observed) / Indicators / Items 231X1X2X3X4X5X6X7X8X9The Factor Loading or the Structure/Pattern CoefficientLatent Construct Unmeasured variablesErrors or UniquenessConfirmatory Factor Analysis (CFA)2,13,13,2Some Errors are correlatedSome Factors are correlated/ Some Factors are not correlated2,11,13,14,25,26,27,38,39,3Statistical Design123456789101112131415161718192021222324252627282930313233x1x2x3x4x5x6x7x8x9x10x11x12x13x14x15x16x17x18x19x20x21x22x23x24x25x26x27x28x29x30x31x32x33F-1F-2F-3F-4First-order Confirmatory Factor Analytic Model 2,13,24,33,14,24,1Statistical Design: First-order Factor Analysis123456789101112131415161718192021222324252627282930313233x1x2x3x4x5x6x7x8x9x10x11x12x13x14x15x16x17x18x19x20x21x22x23x24x25x26x27x28x29x30x31x32x33F-1F-2F-3F-4F-AF-BSecond-order Confirmatory Factor Analytic Model Statistical Design: Second-order Factor Analysis

First, Second-order Factor AnalysisFirst-order CFA and Second-order CFA M-1x1x2x3x4x5x6x7x8x9x10x11x12x13x14x15x16x17x18x19x20x21x22x23x24x25x26x27x28x29x30x31x32x33LV-1LV-2LV-3LV-4M-2Statistical Design: Multitraits-Multimethods Matrix

First-order CFA and Multitrait-Multimethod Matrix (MTMM)Analysis UsingDependent & Interdependent TechniquesSakesan Tongkhambanchong, Ph.D (Applied Behavioral Science Research)YX1X2X3Causal Modeling I: Path Analysis with Observed Variables(Assumption: Measurement error = 0)YX1X2X5X4Total Effect = Direct + Indirect Effects Total Effect = Direct + Indirect Effects X3Statistical Design21,12,13,12Y6,2Y4,2Y5,21X3,1X1,1X2,12X6,2X4,2X5,21Y3,1Y1,1Y2,1Causal Modeling II: Path Analysis with Latent Variables Linear Structural Equation Modeling (SEM)(Assumption: Measurement error > 0)4,21,15,26,32,13,14,25,26,21Total Effect = Direct + Indirect Effects Path Analysis + Confirmatory Factor Analysis Statistical Design