D09: Advanced Quantitative Text Analysis

This course introduces students to the theory and practice of treating text as quantitative data for social scientific research. This course covers the full workflow of computational text analysis, from data collection and preprocessing to model selection, validation, and interpretation. Students learn core NLP concepts such as tokenization, part-of-speech tagging, embeddings, and document representations, followed by methods such as supervised and unsupervised methods including dictionaries, bag-of-words models, topic modeling, text classification, and clustering. The course follows with more sophisticated methods such as language models and neural networks. The emphasis is not only on how these tools work, but also when and why they should be used in substantive research.

Throughout the course, students engage with real-world data and implement methods in Python, developing practical coding skills alongside methodological understanding. Special attention is given to deep understanding of these methodologies, issues of measurement, construct validity, bias, and model evaluation, as well as the promises and limitations of newer approaches such as large language models. The final assignment is a short paper in which students apply one or more automated text analysis methods to a substantive or methodological research question of their choice.

By the end of the course, students will be able to critically assess text-as-data approaches, implement standard techniques, and integrate computational text analysis into broader social science research designs.

This one-week intensive course provides a comprehensive introduction to computational text analysis for social scientists. The goal is to equip participants with both conceptual understanding and practical skills to treat unstructured text as data for empirical research. The course combines short lectures, hands-on coding sessions, and group discussions of research applications. The emphasis is on methodological transparency, measurement validity, and the alignment between theoretical constructs and computational tools.

Participants learn how to curate, and preprocess textual data; how to choose appropriate representations of text; how to apply supervised and unsupervised learning methods; and how to evaluate and interpret their results. The course also highlights the use of large language models (LLMs) as tools for social science research, and discusses their strengths and limitations compared to traditional NLP techniques. All methods are demonstrated in Python, and participants work with real datasets throughout the week.

The course is suitable for graduate-level students and researchers who are new to text-as-data approaches but familiar with statistics. Prior programming experience (Python or R) is very strongly encouraged.

Day-by-Day Structure

Day 1: Foundations: From Text to Data

The first day begins by motivating text-as-data approaches in the social sciences: when are they useful, what kinds of questions can they answer, and what are their limitations? It covers the fundamentals of linguistic structure (tokens, types, n-grams, part-of-speech tags) and introduce the key idea that text must be transformed into representations before it can be analyzed statistically. Students learn basic preprocessing steps such as tokenization, normalization, stemming/lemmatization, etc., and discuss their implications for measurement. The hands-on session introduces participants to working with raw text files, building corpora, and preparing data for analysis.

Day 2: Representations: Dictionaries, Bag-of-Words, and Embeddings

Day 2 introduces the main ways to represent text numerically. It begins with dictionary methods and discuss how to construct, validate, and critique them. We then move to the bag-of-words model, term frequency–inverse document frequency (TF-IDF), and word co-occurrence matrices. The conceptual focus is on the trade-off between interpretability and expressiveness. In the afternoon, we introduce vector embeddings (from word2vec to modern contextual embeddings) explaining how they map semantic similarity to geometric space. Practical exercises include building document-feature matrices, exploring term statistics, and training simple embedding models.

Day 3: Supervised Methods: Text Classification and Prediction

Day 3 focuses on supervised learning, where the goal is to predict human-coded labels or outcomes from text. The course covers feature engineering, training-test splits, regularization, and performance metrics. Methods include logistic regression, SVMs, random forests, and gradient boosting. The practical session focuses on building and evaluating a classifier using real-world data, with attention to overfitting, cross-validation, and interpretability tools such as SHAP and LIME.

Day 4: Unsupervised Methods: Clustering and Topic Modeling

This day focuses on discovering latent structure in text without human labels. We introduce clustering algorithms (k-means, hierarchical clustering) and discuss how to evaluate cluster coherence and stability. We then cover topic modeling, including Latent Dirichlet Allocation (LDA), model assumptions, parameter choices, and common pitfalls. Participants learn how to choose the number of topics, interpret topic-word distributions, and validate topic solutions. Hands-on exercises include fitting LDA models, visualizing results, and critically examining model outputs.

Day 5: Advanced Topics: Language Models — from Markov chains to Transformers

The final day integrates the week’s lessons and connects them to broader research design considerations. We first discuss the concept of a “language model”, looking at large language models (GPT-type models) as feature extractors, labelers, and generators of synthetic text, along with their limitations, biases, and reproducibility challenges. Students work on mini-projects and receive guidance on designing their own text-as-data studies, from data collection to reporting standards. The course concludes with a wrap-up discussion and recommendations for further learning.

This description is subject to change at the discretion of the Instructor

2 Credits

For completion of all work before and during the course, as outlined by the Instructor, and 90% participation and attendance of the course.

2 Additional Credits

Course specific extra assignments. These can include submitting assignments before the course, daily assignments, and/or a final assignment to be completed after the course as decided by the Instructor.