D14: Statistical Analysis Basics in the Age of Artificial Intelligence & Large Language Models: Methods Workshop

Are you running advanced statistical models and feel like you have major holes in your knowledge? Are you a qualitative researcher who would like to develop foundations in quantitative research but has never gotten around to doing it on your own? Or are you curious about integrating AI into your teaching or analytical workflows? This workshop is tailored for PhD students and post-docs, designed for people who already know what research is. It will cover the basics, minimizing (though not entirely eliminating) math and focusing on the concepts necessary to build a strong basis in quantitative social science research. Artificial intelligence today allows us to demonstrate statistical concepts in novel ways, unlike the days of blackboard math or the steep learning curves of statistical software. AI can now program for us, produce datasets to demonstrate ideas, or show us what happens when we draw thousands of samples from a population in seconds, all without pior coding knowledge. We are going to take advantage of these technological developments to approach the basics of statistics (sampling, hypothesis testing, correlation, regression) from a different, more intuitive angle, helping people learn or solidify the much-needed statistical foundations they often struggle with. At the same time, we will learn enough AI and LLM basics to understand how and why LLMs work, how best to use them, and how to trust but verify their results in the world of analytics.

This course is aimed at multiple audiences. Primarily, it provides a solid, fast paced, intuitive introduction to statistical principles. If you feel you are missing stats foundations, or you never had it and would like to learn, but the thought of sitting in on an MA class sounds like the worst torture you can imagine putting yourself through, you are in the right place. 

But the course is taught with very different approach from what we are used to with blackboards or statistical software. Andrej Karpathy, AI education opinion leader already foresaw where coding, including statistical analysis, was going. He said “[t]he hottest new programming language is English”. With large language models, we do not need to learn coding. As long as we can describe what we want to achieve in plain English, AI will do it for us. If you want to deepen your understanding of artificial intelligence tools going beyond chatbots, we will certainly do this through our journey of learning/reinforcing statistical foundations.

Three clear implications if we want to approach quantitative education from this new paradigm. 

1. We need to focus on conceptual understanding of statistical analysis, with a special emphasis on language use. We must understand what we are doing and explain it in plain and clear English to be able to correctly and effectively get AIs to do the heavy lifting for us.

2. We need to understand how artificial intelligence tools work to better understand how to communicate with them, when and where to anticipate errors. We must learn how to verify results presented to us. 

3. Finally, we must understand the AI-based toolboxes available for us. Chatbots are good at analysis, but there are several alternatives to chose from and the tools available for analysis and coding do go beyond ChatGPT and their immediate competitors.

The course proceeds with this in mind covering both statistical understanding and AI basics. Statsitically, LLM-based tools allow us to move from the world of math into conceptual understandings how stats work. They allow us to play out complex scenarios, simulate what happens when we draw bigger samples or we test weaker or stronger relationships, all without the steep learning curves of programming and visualization. LLMs can create example datasets for us to demonstrate key statistical concepts. This course will cover sampling basics all the way to regression analysis. We will show sampling characteristics, central limit theorem through simulations the AI will provide for us. The course offers foundations for all other statsitical workshops one could take like panel and time series analysis, latent variable models, advanced or multilevel regression and causal inference.

We will truly start at the beginning, covering measures of central tendency (the mean), dispersion (variance) and understand what is at the core of probability theory that allows us to make statistical inferences. Then we go through hypothesis testing, inference, difference of means tests, nonparametric analysis, correlations, bivariate and multiple regression with a strong focus on the assumptions made, the utility and the limitations of these methods. If time allows, we will get into big data (such as quantitative text) analysis. We tie back these techniques to research design and finish with an overview of what else is out there in the world of statistics and what problems (inevitably discussed at some point during this workshop) will be solved by those more advanced methods.

No prior statistical or coding knowledge required. No readings. Bring your laptop (Windows, Mac, and Linux are all OK as long as you can install things on it freely) and please also bring an old-school paper notebook (and a pen or pencil) to be able to take notes by hand. Instructions on what to read, watch, install or sign up for will come after registration for the class is closed. (Today I would recommend having a Mistral account with Mistral API access, install OpenCode.ai and Mistral Vibe. But who knows what the best tools will be in 6 months.)

Learning goals: 

1. Understand the basics of inferential statistics including hypothesis testing, p-values, effect sizes, type I and type II errors, difference tests, correlations and (multiple) regression. Be versed in the assumptions fo regression models.
2. Understand how LLMs work and how to effectively use them for coding and statistical analysis.

Not a Member yet? Check out the long list of benefits here – Student Membership from only €33.00 and FREE for some locations!

1 Credit For completion of all pre-course work as decided by the Instructor. 2 Credits For completion of daily assignments, 90% participation and attendance of the course as decided by the Instructor. 2  Additional Credits 1 or 2 credits for a course specific final assignment, depending on the length of the assignment, to be completed after the course as decided by the Instructor.