Second Language Research

Contents Preface 1. Introduction to Research 1.1 Research Methods 1.2 Different Types of Research 1.3 Ethical Practices 1.3.1 Informed Consent 1.3.1.1 Sufficient Information 1.3.1.2 The Informed Consent Document 1.3.1.3 Participant Comprehension in Informed Consent 1.3.1.4 The Language of Informed Consent 1.3.1.5 Voluntary Participation and Informed Consent 1.3.2 Child Second Language Learners 1.3.3 Issues Involved in Collecting Data Online 1.3.4 WEIRD Populations 1.3.5 Reporting and Anonymity 1.4 Institutional Review of Human Participants Research, Compliance, and Problem-Solving 1.4.1 Purpose of Reviews and IRB Responsibilities 1.4.2 Why Guidelines to Protect Human Subjects Were Developed 1.4.3 Development of Research Codes of Ethics 1.4.4 Preparing a Protocol for the IRB 1.4.5 General Use of Protocols in Research 1.5 Conclusion Points to Remember More to Do and More to Think About … Sample Responses: Time to Think & Time to Do Sample Responses: More to Do and More to Think About… 2. Research Reporting, Research Questions, and Replication 2.1 What Does a Research Report Look Like? 2.1.1 Quantitative Reporting 2.1.1.1 Title Page 2.1.1.2 Abstract 2.1.1.3 Introduction 2.1.1.4 Historical Overview 2.1.1.5 Major Players in this Research Area, Including Questions, Past Findings, and Controversies 2.1.1.6 General Goal of the Paper 2.1.1.7 Research Questions/Hypotheses 2.1.1.8 Methods Section 2.1.1.9 Participants 2.1.1.10 Materials 2.1.1.11 Procedures 2.1.1.12 Analysis 2.1.1.13 Results 2.1.1.14 Discussion/Conclusion 2.1.1.15 Notes 2.1.1.16 References 2.1.1.17 Appendices 2.1.2 Qualitative Reporting 2.1.3 Mixed-Method Reporting 2.2 Identifying Research Questions 2.2.1 Feasibility 2.2.2 Research Questions and Hypotheses 2.3 Replication 2.4 Conclusion Points to Remember More to Do and More to Think About … Sample Responses: Time to Think & Time to Do Sample Responses: More to Do and More to Think About… 3. Common Data Collection Measures 3.1 Introduction 3.2 The Significance of Data Collection Measures 3.3 Researching Formal Models of Language 3.3.1 Acceptability Judgments 3.3.1.1 Materials 3.3.1.2 Procedures 3.3.2 Gradient Judgments: Magnitude Estimation 3.3.3 Elicited Imitation 3.3.4 Interpretation Tasks 3.4 Psycholinguistics-Based Research 3.4.1 Sentence Interpretation 3.4.2 Reaction Time 3.4.3 Self-Paced Reading 3.4.4 Eye-Tracking 3.4.5 Neurolinguistic Elicitation 3.5 Interaction-Based Research 3.5.1 Picture Description Tasks 3.5.2 Spot the Difference 3.5.3 Jigsaw Tasks 3.5.4 Consensus Tasks 3.5.5 Consciousness-Raising Tasks 3.5.6 Computer-Mediated Research 3.6 Strategies and Cognitive Processes 3.6.1 Observations 3.6.2 Introspective Measures 3.6.2.1 Stimulated Recall 3.6.2.2 Think-Alouds or On-Line Tasks 3.6.2.3 Immediate Recalls 3.7 Sociolinguistic/Pragmatics-Based Research 3.7.1 Naturalistic Settings 3.7.2 Elicited Narratives 3.7.2.1 Silent Film 3.7.2.2 Film Strips with Minimal Sound 3.7.2.3 Picture Tasks 3.7.2.4 Narrative Inquiry 3.7.3 Discourse Completion Test (DCT) 3.7.4 Role Plays 3.7.5 Video Playback for Interpretation 3.8 Questionnaires and Surveys 3.9 Existing Databases 3.10 Pilot Testing 3.11 Conclusion Points to Remember More to Do and More to Think About … Notes Sample Responses: Time to Think & Time to Do Sample Responses: More to Do and More to Think About… 4. Coding 4.1 Preparing Data for Coding 4.1.1 Transcribing Oral Data 4.1.1.1 Transcription Conventions 4.1.1.2 Technology and Transcription 4.2 Data Coding 4.2.1 Scales of Measurement 4.2.2 Coding Nominal Data 4.2.3 Coding Ordinal Data 4.2.4 Coding Interval Data 4.3 Coding Systems 4.3.1 Common Coding Systems and Categories 4.3.1.1 T-Units 4.3.1.2 Suppliance in Obligatory Contexts (SOC) 4.3.1.3 Complexity, Accuracy, and Fluency (CAF) 4.3.1.4 Qualitative Coding Categories 4.3.2 Custom-Made Coding Systems 4.3.2.1 Question Formation 4.3.2.2 Corrective Feedback 4.3.2.3 Classroom Interaction 4.3.2.4 Second Language Writing Research 4.3.3 Coding Interpretive and Qualitative Data 4.4 Inter-Rater Reliability 4.4.1 Calculating Inter-Rater Reliability 4.4.1.1 Simple Percentage Agreement 4.4.1.2 Cohen's Kappa 4.4.1.3 Additional Measures of Reliability 4.4.1.4 Good Practice Guidelines for Inter-Rater Reliability 4.4.1.5 How Data Are Selected for Inter-Rater Reliability Tests 4.4.1.6 When to Carry out Coding Reliability Checks 4.5 The Mechanics of Coding 4.5.1 How Much to Code? 4.5.2 When to Make Coding Decisions 4.6 Software for Coding 4.7 Conclusion Points to Remember More to Do and More to Think About … Sample Responses: Time to Think & Time to Do Sample Responses: More to Do and More to Think About… 5. Research Variables, Validity, and Reliability 5.1 Introduction 5.2 Hypotheses 5.3 Variable Types 5.3.1 Independent and Dependent Variables 5.3.2 Moderator Variables 5.3.3 Intervening Variables 5.3.4 Control Variables 5.4 Operationalization 5.5 Validity 5.5.1 Content Validity 5.5.2 Face Validity 5.5.3 Construct Validity 5.5.4 Criterion-Related Validity 5.5.5 Predictive Validity 5.5.6 Internal Validity 5.5.6.1 Participant Characteristics 5.5.6.2 Language Background 5.5.6.3 Language Learning Experience 5.5.6.4 Proficiency Level 5.5.6.5 Participant Attrition 5.5.6.6 Participant Inattention and Attitude 5.5.6.7 Participant Maturation 5.5.6.8 Data Collection: Location and Collector 5.5.6.9 Instrumentation and Test Effect 5.5.6.10 Equivalence Between Pre- and Posttests 5.5.6.11 Giving the Goal of the Study Away 5.5.6.12 Instructions/Questions 5.5.7 External Validity 5.5.7.1 Sampling 5.5.7.2 Random Sampling 5.5.7.3 Nonrandom Sampling 5.5.7.4 Representativeness and Generalizability 5.5.7.5 Collecting Biodata Information 5.6 Reliability 5.6.1 Rater Reliability 5.6.2 Instrument Reliability 5.6.2.1 Test-Retest 5.6.2.2 Equivalence of Forms 5.6.2.3 Internal Consistency 5.7 Conclusion Points to Remember More to Do and More to Think About … Notes Sample Responses: Time to Think & Time to Do Sample Responses: More to Do and More to Think About… 6. Designing a Quantitative Study 6.1 Introduction 6.2 Research Materials 6.3 Intact Classes 6.4 Counterbalancing 6.5 Research Design Types 6.5.1 Correlational (Associational) Research 6.5.2 Experimental and Quasi-Experimental Research 6.5.2.1 Comparison Group Design 6.5.2.2 Control Group Design 6.5.3 Measuring the Effect of Treatment 6.5.3.1 Pretest/Posttest Design 6.5.3.2 Posttest Only Design 6.5.4 Repeated Measures Design 6.5.5 Factorial Design 6.5.6 Time-Series Design 6.5.7 One-Shot Designs 6.5.8 Research Syntheses and Meta-Analyses 6.6 Conclusion Points to Remember More to Do and More to Think About … Note Sample Responses: Time to Think & Time to Do Sample Responses: More to Do and More to Think About… 7. Qualitative and Interpretive Research 7.1 Defining Qualitative and Interpretive Research 7.2 Gathering Qualitative and Interpretive Data 7.2.1 Ethnographies 7.2.1.1 Advantages 7.2.1.2 Caveats 7.2.2 Case Studies 7.2.2.1 Advantages 7.2.2.2 Caveats 7.2.3 Interviews 7.2.3.1 Advantages 7.2.3.2 Caveats 7.2.4 Observations 7.2.4.1 Advantages 7.2.4.2 Caveats 7.2.5 Diaries/Journals 7.2.5.1 Advantages 7.2.5.2 Caveats 7.3 Analyzing Qualitative Data 7.3.1 Credibility, Transferability, Confirmability, and Dependability 7.3.2 Triangulation 7.3.3 The Role of Quantification in Qualitative and Interpretive Research 7.4 Conclusion Points to Remember More to Do and More to Think About … Note Sample Responses: Time to Think & Time to Do Sample Responses: More to Do and More to Think About… 8. Classroom Research 8.1 Classroom Research Contexts 8.2 Common Techniques for Data Collection in Classroom Research 8.2.1 Observations 8.2.1.1 Conducting Classroom Observations 8.2.1.2 Observation Procedures and Coding Schemes 8.2.1.3 Description of Observation Schemes 8.2.1.4 Caveats to Using or Modifying Existing Observation Schemes 8.3 Introspective Methods in Classroom Research 8.3.1 Uptake Sheets 8.3.2 Stimulated Recall 8.3.3 Diary Research in Classroom Contexts 8.4 Practical Considerations in Classroom Research 8.4.1 Logistical Issues to Consider When Carrying out Classroom Research 8.4.2 Logistics 8.4.2.1 Informed Consent 8.4.2.2 Debriefing Participants and Facilitators 8.4.2.3 Ensuring Confidentiality and Minimizing Disruption 8.4.2.4 Data Segmentation and Coding 8.4.2.5 Considering the Instructional Setting 8.4.2.6 Summary of Logistics 8.5 Purposes and Types of Research Conducted in Classroom Settings 8.5.1 The Relationship between Instruction and Learning in Second Language Classrooms 8.5.2 Action Research 8.5.2.1 Definitions 8.5.2.2 Theory and Background to Action Research 8.5.2.3 Action Research in Practice 8.5.3 Aptitude-Treatment Interaction 8.5.4 Getting Your Footing in the Literature 8.6 Conclusion Points to Remember More to Do and More to Think About … Sample Responses: Time to Think & Time to Do Sample Responses: More to Do and More to Think About… 9. Mixed Methods 9.1 Introduction 9.2 What Are Mixed Methods? 9.3 Why Use Mixed Methods? 9.4 Types of Mixed-Methods Studies 9.4.1 Concurrent Mixed-Methods 9.4.2. Sequential Mixed-Methods 9.5 Conducting a Mixed-Methods Study 9.5.1 Deciding Whether or Not to Conduct a Mixed-Methods Study 9.5.2 Planning Phase 9.6 Analyzing Data from a Mixed-Methods Study 9.7 Software for Analysis 9.8 Conclusion Points to Remember More to Do and More to Think About … Sample Responses: Time to Think & Time to Do Sample Responses: More to Do and More to Think About… 10. Concluding and Reporting Research 10.1 The Importance of Reporting Research 10.2 The Final Stages in Reporting Quantitative Research 10.2.1 The Discussion 10.2.2 Limitations, Future Research, and Conclusion Sections 10.3 The Final Stages in Reporting Qualitative Research 10.4 Reporting Mixed-Methods Research 10.5 Checklist for Completing Reports of Research 10.5.1 The Research Problem and Questions 10.5.2 The Research Hypotheses 10.5.3 The Audience 10.5.4 The Abstract 10.5.5 The Literature Review 10.5.6 The Design of the Study 10.5.7 Logistics 10.5.8 Participants 10.5.9 Data Gathering 10.5.10 Data Analysis 10.5.11 Conclusions 10.5.12 References 10.5.13 Footnotes, Endnotes, Figures, and Tables 10.5.13.1 Footnotes and Endnotes 10.5.13.2 Figures 10.5.13.3 Tables 10.5.14 Author’s Note/Acknowledgments 10.5.15 Post-Research Concerns 10.5.16 Final Touches and Formatting 10.6 Open Science 10.7 Conclusion Points to Remember More to Do and More to Think About … Sample Responses: Time to Think & Time to Do Sample Responses: More to Do and More to Think About… Glossary References Subject Index