Modern Psychometrics

1. The history and evolution of psychometric testing Introduction What is psychometrics? Psychometrics in the 21st century History of assessment Chinese origins The ability to learn The nineteenth century Beginnings of psychometrics as a science Intelligence testing Eugenics and the dark decades Psychometric testing of ability The dark ages come to an end An abundance of abilities Tests of other psychological constructs Personality Integrity Interests Motivation Values Temperament Attitude Belief Summary 2. Constructing your own psychometric questionnaire The purpose of the questionnaire Making a blueprint Writing items Alternate-choice items Multiple-choice items Rating-scale items All questionnaires Knowledge-based questionnaires Person-based questionnaires Designing the questionnaire Piloting the questionnaire Item analysis Facility Discrimination Distractors Obtaining the reliability Cronbach’s alpha Split-half reliability Assessing validity Face validity Content validity Standardization 3. The Psychometric principles Reliability Test-retest reliability Parallel-forms reliability Split-half reliability Interrater reliability Internal consistency The standard error of measurement (SEM) Comparing test reliabilities Restriction of range Validity Face validity Content validity Predictive validity Concurrent validity Construct validity Differential validity Standardization Norm referencing Criterion referencing Equivalence Differential item functioning Measurement invariance Adverse impact Summary 4. Psychometric measurement True-score theory Identification of latent traits with factor analysis Spearman’s two-factor theory Vector algebra and factor rotation Moving into more dimensions Multidimensional scaling Application of factor analysis to test construction Eigenvalues Identifying the number of factors to extract using the Kaiser criterion Identifying the number of factors to extract using the Cattell scree test Other techniques for identifying the number of factors to extract Factor rotation Rotation to simple structure Orthogonal rotation Oblique rotation Limitations of the classical factor-analystic approach Criticisms of psychometric measurement theory The Platonic true score Psychological vs. physical true scores Functional assessment and competency testing Machine learning and the black box Summary 5. Item response theory and computer adaptive testing Introduction Item banks The Rasch model Assessment of educational standards The Birnbaum model The evolution of modern psychometrics Computer adaptive testing Item equating Polytomous IRT An intuitive graphical description of item tesponse theory Limitations of classical test theory A graphical Introduction to item response theory The logistic curve 3PL-model: difficulty parameter 3PL model: discrimination parameter 3PL model: guessing parameter The Fisher information function The test information function and its relationship to the standard error of measurement How to score an IRT test Principles of computer adaptive testing Summary of item response theory Confirmatory factor analysis 6. Personality theory Theories of personality Psychoanalytic theory Humanistic theory Social learning theory Behavioral genetics Type and trait theories Different approaches to personality assessment Self-report techniques and personality profiles Reports by others Online digital footprints Situational assessments Projective measures Observations of behavior Task performance methods Polygraph methods Repertory grids Sources and management of bias Self-report techniques and personality profiles Reports by others Online digital footprints Situational assessments Projective measures Observations of behavior Task performance methods Polygraph methods Repertory grids Informal methods of personality assessment State versus trait measures Ipsative scaling Spurious validity and the Barnum Effect Summary 7. Personality assessment in the workplace Prediction of successful employment outcomes Validation of personality questionnaires previously used in employment Historical antecedents to the five-factor model Stability of the five-factor model Cross-cultural aspects of the five-factor model Scale independence and the role of facets Challenges to scale construction for the five-factor model Impression management Acquiescence Response bias and factor structure Development of the five OBPI personality scales Assessing counterproductive behavior at work The impact of behaviorism Prepsychological theories of integrity Modern integrity testing Psychiatry and the medical model The dysfunctional tendencies The dark triad Assessing integrity at work The OBPI integrity scales Conclusion 8. Employing digital footprints in psychometrics Introduction Types of digital footprint Usage logs Language data Mobile sensors Images and audiovisual data Typical applications of digital footprints in psychometrics Replacing and complimenting traditional measures New contexts and new constructs Predicting future behavior Studying human behavior Supporting the development of traditional measures Advantages and challenges of employing digital footprints in psychometrics High ecological validity Greater detail and longitude Less control over the assessment environment Greater speed and unobtrusiveness Less privacy and control No anonymity Bias Enrichment of existing constructs Developing digital-footprint-based psychometric measures Collecting digital footprints How much data is needed? Preparing digital footprints for analysis Respondent-footprint matrix Data sparsity Reducing the dimensionality of the respondent-footprint matrix Singular value decomposition Latent Dirichlet allocation Building prediction models 9. Psychometrics in the era of the intelligent machine History of computerization in psychometrics Computerized statistics Computerized item banks Computerized item generation Automated advice and report systems The evolution of AI in psychometrics Expert systems Neural networks (machine learning) Parallel processing Predicting with statistics and machine learning Explainability Psychometrics in cyberspace What and where is cyberspace? The medium is the message Moral development in AI Kohlberg’s theory of moral development Do machines have morals? The laws of robotics Artificial general intelligence Conclusion