PageSourceSearch

https://www.robertosazuwaness.com/causal-ai-book/

html robertosazuwaness.com collected 2026-10-03 23:20:13 UTC 38,523 bytes, 683 lines download raw bytes

1<!DOCTYPE html>
2<html lang="en"><head>
3  <meta charset="utf-8">
4  <meta http-equiv="X-UA-Compatible" content="IE=edge">
5  <meta name="viewport" content="width=device-width, initial-scale=1"><!-- Begin Jekyll SEO tag v2.8.0 -->
6<title>Causal AI Book | Robert Osazuwa Ness</title>
7<meta name="generator" content="Jekyll v3.10.0" />
8<meta property="og:title" content="Causal AI Book" />
9<meta property="og:locale" content="en_US" />
10<meta name="description" content="Applications of Statistical Machine Learning, Probabilistic Graphical Models, and Causal Inference" />
11<meta property="og:description" content="Applications of Statistical Machine Learning, Probabilistic Graphical Models, and Causal Inference" />
12<link rel="canonical" href="https://robertness.github.io/causal-ai-book/" />
13<meta property="og:url" content="https://robertness.github.io/causal-ai-book/" />
14<meta property="og:site_name" content="Robert Osazuwa Ness" />
15<meta property="og:type" content="website" />
16<meta name="twitter:card" content="summary" />
17<meta property="twitter:title" content="Causal AI Book" />
18<script type="application/ld+json">
19{"@context":"https://schema.org","@type":"WebPage","description":"Applications of Statistical Machine Learning, Probabilistic Graphical Models, and Causal Inference","headline":"Causal AI Book","url":"https://robertness.github.io/causal-ai-book/"}</script>
19
20<!-- End Jekyll SEO tag -->
21<link rel="stylesheet" href="/assets/main.css"><link type="application/atom+xml" rel="alternate" href="https://robertness.github.io/feed.xml" title="Robert Osazuwa Ness" /></head>
22<body><header class="site-header" role="banner">
23
24  <div class="wrapper"><a class="site-title" rel="author" href="/">Robert Osazuwa Ness</a><nav class="site-nav">
25        <input type="checkbox" id="nav-trigger" class="nav-trigger" />
26        <label for="nav-trigger">
27          <span class="menu-icon">
28            <svg viewBox="0 0 18 15" width="18px" height="15px">
29              <path d="M18,1.484c0,0.82-0.665,1.484-1.484,1.484H1.484C0.665,2.969,0,2.304,0,1.484l0,0C0,0.665,0.665,0,1.484,0 h15.032C17.335,0,18,0.665,18,1.484L18,1.484z M18,7.516C18,8.335,17.335,9,16.516,9H1.484C0.665,9,0,8.335,0,7.516l0,0 c0-0.82,0.665-1.484,1.484-1.484h15.032C17.335,6.031,18,6.696,18,7.516L18,7.516z M18,13.516C18,14.335,17.335,15,16.516,15H1.484 C0.665,15,0,14.335,0,13.516l0,0c0-0.82,0.665-1.483,1.484-1.483h15.032C17.335,12.031,18,12.695,18,13.516L18,13.516z"/>
30            </svg>
31          </span>
32        </label>
33
34        <div class="trigger"><a class="page-link" href="/">Home</a><a class="page-link" href="/causal-ai-book/">Causal AI Book</a><a class="page-link" href="/workshops.html">Workshops</a><a class="page-link" href="/blog/">Blog</a></div>
35      </nav></div>
36</header>
37<main class="page-content" aria-label="Content">
38      <div class="wrapper">
39        <article class="post">
40
41  <header class="post-header">
42    <h1 class="post-title">Causal AI Book</h1>
43  </header>
44
45  <div class="post-content">
46    <figure style="float: left; width: 300px; margin-right: 20px; margin-bottom: 10px;">
47  <img src="/assets/images/causal-ai-book-cover.jpg" alt="Causal AI Book Cover" width="300" />
48</figure>
49
50<p><em>Causal AI</em> is my <a href="https://www.manning.com/books/causal-ai">book on causality</a>. It is a practical, hands-on introduction to causal inference for modern machine-learning practitioners. Unlike traditional causal books that focus primarily on applied social science, public health, and econometrics, this book is designed for readers who work directly with deep learning, probabilistic modeling, and PyTorch-based pipelines. It bridges the gap between causal reasoning and the computational tools used in today’s AI systems.</p>
51
52<p>The book teaches readers how to build causal models, predict intervention outcomes, reason about counterfactuals, and combine causal structure with modern generative modeling. Each chapter includes fully worked PyTorch examples, showing how to integrate causal representations (e.g., structural causal models), and likelihood-based reasoning into the deep learning workflow. Readers walk away not only understanding causal concepts, but also knowing how to implement them in code, test causal hypotheses, and improve model reliability in real applications.</p>
53
54<p>By the end, you can use causality to design robust models, perform better experiments, build more interpretable systems, and leverage causal structure to enhance learning in multimodal generative models.</p>
55
56<p>You can buy the book here:</p>
57
58<ul>
59  <li><a href="https://www.amazon.com/Causal-AI-Robert-Ness/dp/1633439917">Amazon</a></li>
60  <li><a href="https://www.manning.com/books/causal-ai">Publisher</a></li>
61</ul>
62
63<p>If you’ve read the book, please consider leaving a book on Amazon.</p>
64
65<p>This page contains links to tutorials, notebooks, references, and errata.</p>
66
67<hr />
68
69<h2 id="chapter-1-introduction">Chapter 1: Introduction</h2>
70
71<h3 id="book-recommendations">Book recommendations</h3>
72
73<p>This book takes an opinionated approach to causality that focuses on graphs, probabilistic machine learning, Bayesian decision-making, and using deep learning tools such as PyTorch.</p>
74
75<p>For books with alternative perspectives that focus on econometrics, social science, and practical data science themes, check out:</p>
76
77<ul>
78  <li>
79    <p><a href="https://mixtape.scunning.com/">Causal Inference: The Mixtape</a></p>
80  </li>
81  <li>
82    <p><a href="https://www.packtpub.com/product/causal-inference-and-discovery-in-python/9781804612989">Causal Inference and Discovery in Python</a></p>
83  </li>
84  <li>
85    <p><a href="https://www.amazon.com/Causal-Inference-Statistics-Judea-Pearl/dp/1119186846/ref=sr_1_1?adgrpid=1332608660533906&amp;hvadid=83288112523943&amp;hvbmt=be&amp;hvdev=c&amp;hvlocphy=95641&amp;hvnetw=o&amp;hvqmt=e&amp;hvtargid=kwd-83288395424312%3Aloc-190&amp;hydadcr=3240_10731484&amp;keywords=causal+inference+in+statistics+a+primer&amp;qid=1693072626&amp;sr=8-1">Causal Inference in Statistics: A Primer</a></p>
86  </li>
87</ul>
88
89<h3 id="key-references-in-the-chapter">Key references in the chapter</h3>
90
91<ul>
92  <li>D’Amour, A., Heller, K., Moldovan, D., Adlam, B., Alipanahi, B., Beutel, A., Chen, C., Deaton, J., Eisenstein, J., Hoffman, M.D. and Hormozdiari, F., 2020. <strong>Underspecification presents challenges for credibility in modern machine learning.</strong> <em>arXiv preprint arXiv:2011.03395</em>.</li>
93</ul>
94
95<hr />
96
97<h2 id="chapter-2-primer-on-probability-modeling">Chapter 2: Primer on probability modeling</h2>
98
99<p>Our course on <a href="https://www.altdeep.ai/p/probml">probabilistic machine learning</a> covers in detail the elements of Bayesian and probabilistic inference covered in this chapter.</p>
100
101<ul>
102  <li><a href="https://github.com/altdeep/causalML/tree/master/book/chapter%202">Chapter 2 notebooks</a></li>
103</ul>
104
105<h3 id="book-recommendations-1">Book recommendations</h3>
106
107<ul>
108  <li>
109    <p>Murphy, K.P., 2022. <a href="https://probml.github.io/pml-book/book1.html">Probabilistic Machine Learning: An Introduction</a>. MIT Press.</p>
110  </li>
111  <li>
112    <p>Hsu, H.P., 1997. <a href="https://www.amazon.com/Schaums-Probability-Variables-Processes-Outlines-dp-1260453812/dp/1260453812/ref=dp_ob_title_bk">Schaum’s Outline of Theory and Problems of Probability, Random Variables, and Random Processes</a>. McGraw-Hill.</p>
113  </li>
114</ul>
115
116<hr />
117
118<h2 id="chapter-3-building-a-causal-graphical-model">Chapter 3: Building a causal graphical model</h2>
119
120<ul>
121  <li>
122    <p><a href="https://github.com/altdeep/causalML/tree/master/book/chapter%203">Chapter 3 notebooks</a></p>
123  </li>
124  <li>
125    <p>See additional code and causal modeling ideas in the <a href="https://github.com/altdeep/causalML/tree/master/projects">projects directory</a>.</p>
126  </li>
127</ul>
128
129<h3 id="causal-abstraction">Causal abstraction</h3>
130
131<ul>
132  <li>
133    <p>Beckers, S. and Halpern, J.Y., 2019, July. <strong>Abstracting causal models.</strong> In <em>Proceedings of the AAAI Conference on Artificial Intelligence</em> (Vol. 33, No. 01, pp. 2678–2685).</p>
134  </li>
135  <li>
136    <p>Beckers, S., Eberhardt, F. and Halpern, J.Y., 2020, August. <strong>Approximate causal abstractions.</strong> In <em>Uncertainty in Artificial Intelligence</em> (pp. 606–615). PMLR.</p>
137  </li>
138  <li>
139    <p>Rischel, E.F. and Weichwald, S., 2021, December. <strong>Compositional abstraction error and a category of causal models.</strong> In <em>Uncertainty in Artificial Intelligence</em> (pp. 1013–1023). PMLR.</p>
140  </li>
141</ul>
142
143<h3 id="independence-of-mechanism">Independence of mechanism</h3>
144
145<ul>
146  <li>
147    <p>Schölkopf, B., Janzing, D., Peters, J., Sgouritsa, E., Zhang, K. and Mooij, J., 2012. <strong>On causal and anticausal learning.</strong> <em>arXiv preprint arXiv:1206.6471</em>.</p>
148  </li>
149  <li>
150    <p>Rojas-Carulla, M., Schölkopf, B., Turner, R. and Peters, J., 2018. <strong>Invariant models for causal transfer learning.</strong> <em>Journal of Machine Learning Research</em>, 19(36), pp.1–34.</p>
151  </li>
152  <li>
153    <p>Besserve, M., Shajarisales, N., Schölkopf, B. and Janzing, D., 2018, March. <strong>Group invariance principles for causal generative models.</strong> In <em>International Conference on Artificial Intelligence and Statistics</em> (pp. 557–565). PMLR.</p>
154  </li>
155  <li>
156    <p>Parascandolo, G., Kilbertus, N., Rojas-Carulla, M. and Schölkopf, B., 2018, July. <strong>Learning independent causal mechanisms.</strong> In <em>International Conference on Machine Learning</em> (pp. 4036–4044). PMLR.</p>
157  </li>
158</ul>
159
160<h3 id="causal-data-fusion-and-transfer-learning">Causal data fusion and transfer learning</h3>
161
162<ul>
163  <li>
164    <p>Bareinboim, E. and Pearl, J., 2016. <strong>Causal inference and the data-fusion problem.</strong> <em>Proceedings of the National Academy of Sciences</em>, 113(27), pp.7345–7352.</p>
165  </li>
166  <li>
167    <p>Rojas-Carulla, M., Schölkopf, B., Turner, R. and Peters, J., 2018. <strong>Invariant models for causal transfer learning.</strong> <em>Journal of Machine Learning Research</em>, 19(36), pp.1–34.</p>
168  </li>
169  <li>
170    <p>Magliacane, S., van Ommen, T., Claassen, T., Bongers, S., Versteeg, P. and Mooij, J.M., 2017. <strong>Causal transfer learning.</strong> <em>arXiv preprint arXiv:1707.06422</em>.</p>
171  </li>
172</ul>
173
174<h3 id="causally-invariant-prediction">Causally invariant prediction</h3>
175
176<ul>
177  <li>
178    <p>Arjovsky, M., Bottou, L., Gulrajani, I. and Lopez-Paz, D., 2019. <strong>Invariant risk minimization.</strong> <em>arXiv preprint arXiv:1907.02893</em>.</p>
179  </li>
180  <li>
181    <p>Heinze-Deml, C., Peters, J. and Meinshausen, N., 2018. <strong>Invariant causal prediction for nonlinear models.</strong> <em>Journal of Causal Inference</em>, 6(2), p.20170016.</p>
182  </li>
183  <li>
184    <p>Rosenfeld, E., Ravikumar, P. and Risteski, A., 2020. <strong>The risks of invariant risk minimization.</strong> <em>arXiv preprint arXiv:2010.05761</em>.</p>
185  </li>
186  <li>
187    <p>Lu, C., Wu, Y., Hernández-Lobato, J.M. and Schölkopf, B., 2021. <strong>Nonlinear invariant risk minimization: A causal approach.</strong> <em>arXiv preprint arXiv:2102.12353</em>.</p>
188  </li>
189</ul>
190
191<hr />
192
193<h2 id="chapter-4-testing-your-causal-graph">Chapter 4: Testing your causal graph</h2>
194
195<ul>
196  <li><a href="https://github.com/altdeep/causalML/tree/master/book/chapter%204">Chapter 4 notebooks</a></li>
197</ul>
198
199<h3 id="tools-for-d-separation">Tools for d-separation</h3>
200
201<ul>
202  <li>
203    <p><em>NetworkX</em>’s <a href="https://networkx.org/documentation/stable/reference/algorithms/d_separation.html"><code class="language-plaintext highlighter-rouge">d_separation</code> algorithm</a>.</p>
204  </li>
205  <li>
206    <p><em>pgmpy</em>’s <a href="https://github.com/pgmpy/pgmpy/blob/467ada830ca3b9fff0acf8ce49e86e4de00e050e/pgmpy/base/DAG.py#L394"><code class="language-plaintext highlighter-rouge">get_independencies</code> method</a> enumerates all d-separations in a DAG.</p>
207  </li>
208  <li>
209    <p><a href="http://dagitty.net/">Dagitty.net</a> provides both an online application for building DAGs and evaluating d-separation. It also provides an R package.</p>
210  </li>
211  <li>
212    <p>The <a href="https://www.bnlearn.com/documentation/man/dsep.html"><code class="language-plaintext highlighter-rouge">dsep</code></a> function in the <em>bnlearn</em> R package evaluates simple d-separation statements as true or false.</p>
213  </li>
214  <li>
215    <p><a href="https://causalfusion.net/">Causalfusion</a> is an online app like Dagitty but with a valuable set of additional features. You need to apply for access.</p>
216  </li>
217</ul>
218
219<h3 id="background-on-statistical-hypothesis-testing">
219Background on statistical hypothesis testing</h3>
220
221<ul>
222  <li>
223    <p>See chapter 8 of <a href="https://www.amazon.com/Schaums-Probability-Variables-Processes-Outlines-dp-1260453812/dp/1260453812/ref=dp_ob_title_bk">Schaum’s Outline of Probability, Random Variables, and Random Processes</a> for a good introduction to statistical hypothesis testing.</p>
224  </li>
225  <li>
226    <p>Wikipedia has good descriptions of the <a href="https://en.wikipedia.org/wiki/Chi-squared_test">Chi-squared test</a>, <a href="https://en.wikipedia.org/wiki/G-test">G-test</a>, and <a href="https://en.wikipedia.org/wiki/Likelihood-ratio_test">likelihood-ratio test</a> for conditional independence.</p>
227  </li>
228  <li>
229    <p>Conditional independence tests <a href="https://pgmpy.org/_modules/pgmpy/estimators/CITests.html">implemented in pgmpy</a> and <a href="https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.power_divergence.html">SciPy</a>.</p>
230  </li>
231  <li>
232    <p>The PyWhy suite has a library for advanced statistical independence tests called <a href="https://github.com/py-why/pywhy-stats">pywhy-stats</a>.</p>
233  </li>
234</ul>
235
236<h3 id="tools-for-causal-discovery">Tools for causal discovery</h3>
237
238<ul>
239  <li>
240    <p>The PyWhy suite contains the popular <a href="https://github.com/py-why/causal-learn">causal-learn</a> library for causal discovery.</p>
241  </li>
242  <li>
243    <p>PyWhy also contains an experimental library called <a href="https://github.com/py-why/dodiscover">dodiscover</a>, which focuses on being a user-friendly interface for discovery.</p>
244  </li>
245  <li>
246    <p>The <a href="https://www.bnlearn.com/">bnlearn package</a> is an R package, and it has a corresponding <a href="https://pypi.org/project/bnlearn/">Python library</a>.</p>
247  </li>
248</ul>
249
250<h3 id="false-discovery-rate-and-causal-discovery">False discovery rate and causal discovery</h3>
251
252<ul>
253  <li>
254    <p>Wikipedia page on the <a href="https://en.wikipedia.org/wiki/Multiple_comparisons_problem">multiple comparisons problem</a> that occurs when doing repeated hypothesis testing. Standard statistical remedies are to do a <a href="https://en.wikipedia.org/wiki/Family-wise_error_rate">family-wise error rate correction</a> or calculate a <a href="https://en.wikipedia.org/wiki/False_discovery_rate">false discovery rate</a>.</p>
255  </li>
256  <li>
257    <p>Pena, J.M., 2008, March. <strong>Learning Gaussian graphical models of gene networks with false discovery rate control.</strong> In <em>European Conference on Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics</em> (pp. 165–176). Springer.</p>
258  </li>
259  <li>
260    <p>The bnlearn package implements the interleaved incremental association algorithm with FDR in the <a href="https://www.bnlearn.com/documentation/man/constraint.html"><code class="language-plaintext highlighter-rouge">iamb.fdr</code></a> function.</p>
261  </li>
262  <li>
263    <p>Gasse, M., Aussem, A. and Elghazel, H., 2014. <strong>A hybrid algorithm for Bayesian network structure learning with application to multi-label learning.</strong> <em>Expert Systems with Applications</em>, 41(15), pp.6755–6772.</p>
264  </li>
265</ul>
266
267<h3 id="go-deeper-on-functional-constraints-verma-constraints">Go deeper on functional constraints (Verma constraints)</h3>
268
269<ul>
270  <li>
271    <p>Tian, J. and Pearl, J., 2012. <strong>On the testable implications of causal models with hidden variables.</strong> <em>arXiv preprint arXiv:1301.0608</em>.</p>
272  </li>
273  <li>
274    <p>Bhattacharya, R. and Nabi, R., 2022, August. <strong>On testability of the front-door model via Verma constraints.</strong> In <em>Uncertainty in Artificial Intelligence</em> (pp. 202–212). PMLR.</p>
275  </li>
276</ul>
277
278<h3 id="more-on-causal-faithfulness">More on causal faithfulness</h3>
279
280<ul>
281  <li>
282    <p>Zhang, J. and Spirtes, P., 2016. <strong>The three faces of faithfulness.</strong> <em>Synthese</em>, 193, pp.1011–1027.</p>
283  </li>
284  <li>
285    <p>Ramsey, J., Zhang, J. and Spirtes, P.L., 2012. <strong>Adjacency-faithfulness and conservative causal inference.</strong> <em>arXiv preprint arXiv:1206.6843</em>.</p>
286  </li>
287</ul>
288
289<h3 id="selected-readings-on-causal-discovery">Selected readings on causal discovery</h3>
290
291<ul>
292  <li>
293    <p>Spirtes, P., 2001, January. <strong>An anytime algorithm for causal inference.</strong> In <em>International Workshop on Artificial Intelligence and Statistics</em> (pp. 278–285). PMLR.</p>
294  </li>
295  <li>
296    <p>Spirtes, P., Glymour, C. and Scheines, R., 2001. <em>
296Causation, Prediction, and Search.</em> MIT Press. (Introduces the PC algorithm, though one might enjoy <a href="https://www.youtube.com/watch?v=o2A61bJ0UCw&amp;ab_channel=BradyNeal-CausalInference">this intro</a> by Brady Neal.)</p>
297  </li>
298  <li>
299    <p>Chickering, D.M., 2002. <strong>Optimal structure identification with greedy search.</strong> <em>Journal of Machine Learning Research</em>, 3(Nov), pp.507–554.</p>
300  </li>
301  <li>
302    <p>Friedman, N. and Koller, D., 2003. <strong>Being Bayesian about network structure. A Bayesian approach to structure discovery in Bayesian networks.</strong> <em>Machine Learning</em>, 50, pp.95–125.</p>
303  </li>
304  <li>
305    <p>Heckerman, D., Meek, C. and Cooper, G., 2006. <strong>A Bayesian approach to causal discovery.</strong> In <em>Innovations in Machine Learning: Theory and Applications</em>, pp.1–28.</p>
306  </li>
307  <li>
308    <p>Meek, C., 2013. <strong>Causal inference and causal explanation with background knowledge.</strong> <em>arXiv preprint arXiv:1302.4972</em>.</p>
309  </li>
310  <li>
311    <p>Cooper, G.F. and Yoo, C., 2013. <strong>Causal discovery from a mixture of experimental and observational data.</strong> <em>arXiv preprint arXiv:1301.6686</em>.</p>
312  </li>
313  <li>
314    <p>Ogarrio, J.M., Spirtes, P. and Ramsey, J., 2016, August. <strong>A hybrid causal search algorithm for latent variable models.</strong> In <em>Conference on Probabilistic Graphical Models</em> (pp. 368–379). PMLR.</p>
315  </li>
316  <li>
317    <p>Ness, R.O., Sachs, K. and Vitek, O., 2016. <strong>From correlation to causality: statistical approaches to learning regulatory relationships in large-scale biomolecular investigations.</strong> <em>Journal of Proteome Research</em>, 15(3), pp.683–690.</p>
318  </li>
319  <li>
320    <p>Glymour, C., Zhang, K. and Spirtes, P., 2019. <strong>Review of causal discovery methods based on graphical models.</strong> <em>Frontiers in Genetics</em>, 10, p.524.</p>
321  </li>
322  <li>
323    <p>Zheng, Y., Huang, B., Chen, W., Ramsey, J., Gong, M., Cai, R., Shimizu, S., Spirtes, P. and Zhang, K., 2024. <strong>Causal-learn: Causal discovery in Python.</strong> <em>Journal of Machine Learning Research</em>, 25(60), pp.1–8.</p>
324  </li>
325</ul>
326
327<hr />
328
329<h2 id="chapter-5-building-causal-graphs-with-deep-probabilistic-machine-learning">Chapter 5: Building causal graphs with deep probabilistic machine learning</h2>
330
331<ul>
332  <li>
333    <p><a href="https://github.com/altdeep/causalML/tree/master/book/chapter%205">Chapter 5 notebooks</a></p>
334  </li>
335  <li>
336    <p><a href="https://bit.ly/3QbdSFv">Typeface MNIST data (Kaggle)</a></p>
337  </li>
338</ul>
339
340<h3 id="an-explanation-of-the-case-study-on-independence-of-mechanism-in-semi-supervised-learning">An explanation of the case study on independence of mechanism in semi-supervised learning</h3>
341
342<p>One way to understand this example is to use the combined perspectives of Bayesian reasoning and d-separation. In both the causal learning and anti-causal case, during training, we’re learning some set of parameters for a model of ( P(X, Y) ). Let’s decouple this set into (\theta_y) for the parameter set that parameterizes the causal Markov kernel for (Y) and (\theta_x) for the parameter set that parameterizes the causal Markov kernel for (X). Further, let’s take the Bayesian approach of treating (\theta_x) and (\theta_y) as random variables, and give them their own nodes in the DAG, as shown in the following figure.</p>
343
344<p><img src="https://uploads.teachablecdn.com/attachments/7hS6zIXdSt2Kt2Gs1yKX_Learning+copy.png" alt="DAG illustrating causal vs anti-causal learning. The DAG on the left depicts causal learning, with the feature causing the label. The DAG on the right depicts anti-causal learning, with the label causing the feature. Theta-x and theta-y are parameter oracles made explicit as nodes. Assume only X is observed. X is d-separated from theta-y in the causal learning case but not in the anti-causal case." /></p>
345
346<p>We’ll interpret these “Greek” nodes as oracles that set the parameters of the causal Markov kernels of (X) and (Y), and our job during unsupervised training is to infer these unknown parameters from (X) alone. We can see that in the causal learning case, (X) and (\theta_y) are d-separated and thus (X) is not informative of (\theta_y). But in the anti-causal learning case, (X) and (\theta_y) are d-connected, thus (X) is informative of (\theta_y).</p>
347
348<p>For more information, see:</p>
349
350<ul>
351  <li>Schölkopf, B., Janzing, D., Peters, J., Sgouritsa, E., Zhang, K. and Mooij, J., 2012. <a href="https://icml.cc/2012/papers/625.pdf">On causal and anticausal learning</a>. <em>arXiv preprint arXiv:1206.6471</em>.</li>
352</ul>
353
354<h3 id="vision-as-inverse-graphics">Vision as Inverse Graphics</h3>
355
356<ul>
357  <li>
358    <p><a href="https://ps.is.mpg.de/research_fields/inverse-graphics">Intro to Vision as Inverse Graphics</a> from Max Planck Institute for Intelligent Systems.</p>
359  </li>
360  <li>
361    <p>Romaszko, L., Williams, C.K., Moreno, P. and Kohli, P., 2017. <strong>Vision-as-inverse-graphics: Obtaining a rich 3D explanation of a scene from a single image.</strong> In <em>Proceedings of the IEEE International Conference on Computer Vision Workshops</em> (pp. 851–859).</p>
362  </li>
363</ul>
364
365<h3 id="causal-representation-learning-and-disentanglement">Causal representation learning and disentanglement</h3>
366
367<ul>
368  <li>
369    <p><a href="https://ei.is.mpg.de/research_projects/causal-representation-learning">Intro to Causal Representation Learning</a> from Max Planck Institute for Intelligent Systems.</p>
370  </li>
371  <li>
372    <p>Kumar, A., Sattigeri, P. and Balakrishnan, A., 2017. <strong>Variational inference of disentangled latent concepts from unlabeled observations.</strong> <em>arXiv preprint arXiv:1711.00848</em>.</p>
373  </li>
374  <li>
375    <p>Locatello, F., Bauer, S., Lucic, M., Raetsch, G., Gelly, S., Schölkopf, B. and Bachem, O., 2019, May. <strong>Challenging common assumptions in the unsupervised learning of disentangled representations.</strong> In <em>International Conference on Machine Learning</em> (pp. 4114–4124). PMLR.</p>
376  </li>
377  <li>
378    <p>Yang, M., Liu, F., Chen, Z., Shen, X., Hao, J. and Wang, J., 2020. <strong>CausalVAE: Structured causal disentanglement in variational autoencoder.</strong> <em>arXiv preprint arXiv:2004.08697</em>.</p>
379  </li>
380  <li>
381    <p>Wang, Y. and Jordan, M.I., 2021. <strong>Desiderata for representation learning: A causal perspective.</strong> <em>arXiv preprint arXiv:2109.03795</em>.</p>
382  </li>
383  <li>
384    <p>Ahuja, K., Hartford, J. and Bengio, Y., 2021. <strong>Properties from mechanisms: an equivariance perspective on identifiable representation learning.</strong> <em>arXiv preprint arXiv:2110.15796</em>.</p>
385  </li>
386  <li>
387    <p>Reddy, A.G. and Balasubramanian, V.N., 2022, June. <strong>On causally disentangled representations.</strong> In <em>Proceedings of the AAAI Conference on Artificial Intelligence</em> (Vol. 36, No. 7, pp. 8089–8097).</p>
388  </li>
389</ul>
390
391<h3 id="miscellaneous">Miscellaneous</h3>
392
393<ul>
394  <li>Ali Rahimi’s quote comparing machine learning to alchemy can be found at the <a href="https://youtu.be/ORHFOnaEzPc?t=719">11:59 mark in this video</a> of his NIPS 2017 Test-of-Time Award presentation.</li>
395</ul>
396
397<hr />
398
399<h2 id="chapter-6-structural-causal-models">Chapter 6: Structural Causal Models</h2>
400
401<ul>
402  <li>
403    <p><a href="https://github.com/altdeep/causalML/tree/master/book/chapter%206">Chapter 6 notebooks</a></p>
404  </li>
405  <li>
406    <p>Javaloy, A., Sánchez-Martín, P., &amp; Valera, I., 2023. <em>
406Causal normalizing flows: from theory to practice.</em> In: A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, &amp; S. Levine, eds. <em>Neural Information Processing Systems.</em> Curran Associates, Inc., pp. 58833–58864. doi: <a href="https://proceedings.neurips.cc/paper_files/paper/2023/hash/b8402301e7f06bdc97a31bfaa653dc32-Abstract-Conference.html">10.48550/arXiv.2306.05415</a>.</p>
407  </li>
408</ul>
409
410<hr />
411
412<h2 id="chapter-7-interventions-and-causal-effects">Chapter 7: Interventions and Causal Effects</h2>
413
414<ul>
415  <li><a href="https://github.com/altdeep/causalML/tree/master/book/chapter%207">Chapter 7 notebooks</a></li>
416</ul>
417
418<hr />
419
420<h2 id="chapter-8-counterfactuals-and-parallel-worlds">Chapter 8: Counterfactuals and Parallel Worlds</h2>
421
422<ul>
423  <li>
424    <p>Verma, S., Dickerson, J. and Hines, K., 2020. <strong>Counterfactual explanations for machine learning: A review.</strong> <em>arXiv preprint arXiv:2010.10596</em>.</p>
425  </li>
426  <li>
427    <p>Pearl, J., 2010. <strong>Brief report: On the consistency rule in causal inference: “axiom, definition, assumption, or theorem?”.</strong> <em>Epidemiology</em>, pp.872–875.</p>
428  </li>
429</ul>
430
431<h3 id="probabilities-of-causation">Probabilities of Causation</h3>
432
433<ul>
434  <li>
435    <p>Beckers, S., 2021. <strong>Causal sufficiency and actual causation.</strong> <em>Journal of Philosophical Logic</em>, 50(6), pp.1341–1374.</p>
436  </li>
437  <li>
438    <p>Knobe, J. and Shapiro, S., 2021. <strong>Proximate cause explained.</strong> <em>The University of Chicago Law Review</em>, 88(1), pp.165–236.</p>
439  </li>
440  <li>
441    <p>Pearl, J., 2022. <strong>Probabilities of causation: three counterfactual interpretations and their identification.</strong> In <em>Probabilistic and Causal Inference: The Works of Judea Pearl</em> (pp. 317–372).</p>
442  </li>
443</ul>
444
445<h3 id="uplift-modeling">Uplift Modeling</h3>
446
447<ul>
448  <li>
449    <p>Li, A. and Pearl, J., 2019, August. <strong>Unit selection based on counterfactual logic.</strong> In <em>Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence</em>.</p>
450  </li>
451  <li>
452    <p>Gutierrez, P. and Gérardy, J.Y., 2017, July. <strong>Causal inference and uplift modelling: A review of the literature.</strong> In <em>International Conference on Predictive Applications and APIs</em> (pp. 1–13). PMLR.</p>
453  </li>
454</ul>
455
456<hr />
457
458<h2 id="chapter-9-the-counterfactual-inference-algorithm">Chapter 9: The Counterfactual Inference Algorithm</h2>
459
460<ul>
461  <li>
462    <p><a href="https://github.com/altdeep/causalML/tree/master/book/chapter%209">Chapter 9 notebooks</a></p>
463  </li>
464  <li>
465    <p><a href="https://github.com/BasisResearch/chirho">ChiRho library</a> for causal inference with probabilistic models (extension of Pyro).</p>
466  </li>
467</ul>
468
469<h3 id="intractable-likelihood-methods-in-probabilistic-inference">Intractable likelihood methods in probabilistic inference</h3>
470
471<ul>
472  <li>
473    <p>Papamakarios, G., et al. <strong>Normalizing flows for probabilistic modeling and inference.</strong> <em>Journal of Machine Learning Research</em>, 22.1 (2021): 2617–2680.</p>
474  </li>
475  <li>
476    <p>Matsubara, T., et al. <strong>Robust generalised Bayesian inference for intractable likelihoods.</strong> <em>Journal of the Royal Statistical Society Series B: Statistical Methodology</em>, 84.3 (2022): 997–1022.</p>
477  </li>
478  <li>
479    <p>Ritchie, D., Horsfall, P. and Goodman, N.D., 2016. <strong>Deep amortized inference for probabilistic programs.</strong> <em>arXiv preprint arXiv:1610.05735</em>.</p>
480  </li>
481  <li>
482    <p>Murphy, K.P., 2023. <em>Probabilistic Machine Learning: Advanced Topics.</em> MIT Press.</p>
483  </li>
484</ul>
485
486<hr />
487
488<h2 id="chapter-10-causal-hierarchy-and-identification">Chapter 10: Causal Hierarchy and Identification</h2>
489
490<ul>
491  <li><a href="https://github.com/altdeep/causalML/blob/master/book/chapter%2010/Chapter_10_Identification_notebook.ipynb">Chapter 10 notebook</a></li>
492</ul>
493
494<h3 id="do-calculus-pearls-causal-hierarchy-and-identification-algorithms">Do-calculus, Pearl’s causal hierarchy and identification algorithms</h3>
495
496<ul>
497  <li>
498    <p>The <a href="https://github.com/Chrisejorge/Causal-Inference">Y0 repository for causal inference and identification</a>.</p>
499  </li>
500  <li>
501    <p>Bareinboim, E., Correa, J.D., Ibeling, D. and Icard, T., 2022. <strong>On Pearl’s hierarchy and the foundations of causal inference.</strong> In <em>Probabilistic and Causal Inference: The Works of Judea Pearl</em> (pp. 507–556).</p>
502  </li>
503  <li>
504    <p>Shpitser, I. and Pearl, J., 2006, July. <strong>Identification of joint interventional distributions in recursive semi-Markovian causal models.</strong> In <em>Proceedings of the National Conference on Artificial Intelligence</em>
504 (Vol. 21, No. 2, p. 1219).</p>
505  </li>
506  <li>
507    <p>Shpitser, I. and Pearl, J., 2008. <strong>Complete identification methods for the causal hierarchy.</strong> <em>Journal of Machine Learning Research</em>, 9, pp.1941–1979.</p>
508  </li>
509  <li>
510    <p>Huang, Y. and Valtorta, M., 2006. <strong>Pearl’s calculus of intervention is complete.</strong> <em>arXiv preprint arXiv:1206.6831</em>.</p>
511  </li>
512</ul>
513
514<h3 id="potential-outcomes-single-world-intervention-graphs-and-related-concepts">Potential outcomes, single world intervention graphs, and related concepts</h3>
515
516<ul>
517  <li>
518    <p>Malinsky, D., Shpitser, I. and Richardson, T., 2019, April. <strong>A potential outcomes calculus for identifying conditional path-specific effects.</strong> In <em>The 22nd International Conference on Artificial Intelligence and Statistics</em> (pp. 3080–3088). PMLR.</p>
519  </li>
520  <li>
521    <p>Shpitser, I., Richardson, T.S. and Robins, J.M., 2022. <strong>Multivariate counterfactual systems and causal graphical models.</strong> In <em>Probabilistic and Causal Inference: The Works of Judea Pearl</em> (pp. 813–852).</p>
522  </li>
523  <li>
524    <p>Robins, J.M. and Richardson, T.S., 2010. <strong>Alternative graphical causal models and the identification of direct effects.</strong> In <em>Causality and Psychopathology: Finding the Determinants of Disorders and Their Cures</em>, 84, pp.103–158.</p>
525  </li>
526  <li>
527    <p>Richardson, T.S. and Robins, J.M., 2013, July. <strong>Single world intervention graphs: a primer.</strong> In <em>Second UAI Workshop on Causal Structure Learning</em>, Bellevue, Washington.</p>
528  </li>
529  <li>
530    <p>Robins, J.M., vanderWeele, T.J. and Richardson, T.S., 2007. <strong>Contribution to discussion of “Causal Effects in the Presence of Non-compliance: a Latent Variable Interpretation” by A. Forcina.</strong> <em>Metron</em>, LXIV (3), pp. 288–298.</p>
531  </li>
532  <li>
533    <p>Geneletti, S. and Dawid, A.P., 2007. <strong>Defining and identifying the effect of treatment on the treated.</strong> Technical Report No. 3, Imperial College London, Department of Epidemiology and Public Health.</p>
534  </li>
535</ul>
536
537<h3 id="identification-of-effect-of-treatment-on-the-treated">Identification of effect of treatment on the treated</h3>
538
539<ul>
540  <li>Shpitser, I. and Tchetgen, E.T., 2016. <strong>Causal inference with a graphical hierarchy of interventions.</strong> <em>Annals of Statistics</em>, 44(6), p.2433.</li>
541</ul>
542
543<h3 id="partial-identification-bounds">Partial identification bounds</h3>
544
545<ul>
546  <li>
547    <p>Mueller, S. and Pearl, J., 2022. <strong>Personalized decision making — A conceptual introduction.</strong> <em>arXiv preprint arXiv:2208.09558</em>.</p>
548  </li>
549  <li>
550    <p>Li, A. and Pearl, J., 2022. <strong>Probabilities of causation with nonbinary treatment and effect.</strong> <em>arXiv preprint arXiv:2208.09568</em>.</p>
551  </li>
552  <li>
553    <p>Li, A. and Pearl, J., 2022, June. <strong>Unit selection with causal diagram.</strong> In <em>Proceedings of the AAAI Conference on Artificial Intelligence</em> (Vol. 36, No. 5, pp. 5765–5772).</p>
554  </li>
555</ul>
556
557<hr />
558
559<h2 id="chapter-11-building-a-causal-effect-estimation-workflow">Chapter 11: Building a Causal Effect Estimation Workflow</h2>
560
561<ul>
562  <li><a href="https://github.com/altdeep/causalML/tree/master/book/chapter%2011">Notebooks</a></li>
563</ul>
564
565<hr />
566
567<h2 id="chapter-12-primer-on-causality-in-decisions-bandits-and-reinforcement-learning">Chapter 12: Primer on Causality in Decisions, Bandits, and Reinforcement Learning</h2>
568
569<ul>
570  <li><a href="https://github.com/altdeep/causalML/tree/master/book/chapter%2011">Notebooks</a></li>
571</ul>
572
573<h3 id="causal-decision-theory-newcombs-paradox-and-introspectiondeliberation">Causal decision theory, Newcomb’s paradox, and introspection/deliberation</h3>
574
575<ul>
576  <li>
577    <p><a href="https://en.wikipedia.org/wiki/Causal_decision_theory">Causal decision theory</a> — Wikipedia.</p>
578  </li>
579  <li>
580    <p><a href="https://en.wikipedia.org/wiki/Newcomb%27s_paradox">Newcomb’s paradox</a> — Wikipedia.</p>
581  </li>
582  <li>
583    <p><a href="https://en.wikipedia.org/wiki/Agent_causation">Agent causation</a> — Wikipedia.</p>
584  </li>
585  <li>
586    <p>Stern, R., 2018. <strong>Diagnosing Newcomb’s problem with causal graphs.</strong> In <em>Newcomb’s Problem</em>, pp.201–220.</p>
587  </li>
588  <li>
589    <p>Skyrms, B., 1982. <strong>Causal decision theory.</strong> <em>The Journal of Philosophy</em>
589, 79(11), pp.695–711.</p>
590  </li>
591  <li>
592    <p>Lewis, D., 1981. <strong>Causal decision theory.</strong> <em>Australasian Journal of Philosophy</em>, 59(1), pp.5–30.</p>
593  </li>
594  <li>
595    <p>Joyce, J.M., 2002. <strong>Levi on causal decision theory and the possibility of predicting one’s own actions.</strong> <em>Philosophical Studies</em>, 110, pp.69–102.</p>
596  </li>
597  <li>
598    <p>Clarke, R., 1993. <strong>Toward a credible agent-causal account of free will.</strong> <em>Noûs</em>, 27(2), pp.191–203.</p>
599  </li>
600</ul>
601
602<h3 id="causal-bandits-and-reinforcement-learning-counterfactual-risk-minimization">Causal bandits and reinforcement learning, counterfactual risk minimization</h3>
603
604<ul>
605  <li>
606    <p>Bareinboim, E., Forney, A. and Pearl, J., 2015. <strong>Bandits with unobserved confounders: A causal approach.</strong> <em>Advances in Neural Information Processing Systems</em>, 28.</p>
607  </li>
608  <li>
609    <p>Buesing, L., Weber, T., Zwols, Y., Racaniere, S., Guez, A., Lespiau, J.B. and Heess, N., 2018. <strong>Woulda, coulda, shoulda: Counterfactually-guided policy search.</strong> <em>arXiv preprint arXiv:1811.06272</em>.</p>
610  </li>
611  <li>
612    <p>Swaminathan, A. and Joachims, T., 2015, June. <strong>Counterfactual risk minimization: Learning from logged bandit feedback.</strong> In <em>International Conference on Machine Learning</em> (pp. 814–823). PMLR.</p>
613  </li>
614  <li>
615    <p>Scott, S.L., 2010. <strong>A modern Bayesian look at the multi-armed bandit.</strong> <em>Applied Stochastic Models in Business and Industry</em>, 26(6), pp.639–658. (This paper doesn’t use causal reasoning, but its Bayesian framing of the bandit problem can extend to counterfactual reasoning by admitting counterfactual notation.)</p>
616  </li>
617</ul>
618
619<h3 id="causality-in-games-counterfactual-regret-minimization-in-multiagent-games">Causality in games, counterfactual regret minimization in multiagent games</h3>
620
621<ul>
622  <li>
623    <p>Zinkevich, M., Johanson, M., Bowling, M. and Piccione, C., 2007. <strong>Regret minimization in games with incomplete information.</strong> <em>Advances in Neural Information Processing Systems</em>, 20.</p>
624  </li>
625  <li>
626    <p>Brown, N., Lerer, A., Gross, S. and Sandholm, T., 2019, May. <strong>Deep counterfactual regret minimization.</strong> In <em>International Conference on Machine Learning</em> (pp. 793–802). PMLR.</p>
627  </li>
628  <li>
629    <p>Hammond, L., Fox, J., Everitt, T., Carey, R., Abate, A. and Wooldridge, M., 2023. <strong>Reasoning about causality in games.</strong> <em>Artificial Intelligence</em>, 320, p.103919.</p>
630  </li>
631  <li>
632    <p>Soto, M.G., Sucar, L.E. and Escalante, H.J., 2020. <strong>Causal games and causal Nash equilibrium.</strong> <em>Research in Computing Science</em>, 149, pp.123–133.</p>
633  </li>
634</ul>
635
636<hr />
637
638<h2 id="chapter-13-causality-and-large-language-models">Chapter 13: Causality and Large Language Models</h2>
639
640<ul>
641  <li><a href="https://github.com/altdeep/causalML/tree/master/book/chapter%2013">Chapter 13 notebooks</a></li>
642</ul>
643
644<h3 id="key-references-in-the-chapter-1">Key references in the chapter</h3>
645
646<ul>
647  <li>Kıcıman, E., Ness, R., Sharma, A. and Tan, C., 2023. <strong>Causal reasoning and large language models: Opening a new frontier for causality.</strong> <em>arXiv preprint arXiv:2305.00050</em>.</li>
648</ul>
649
650
651  </div>
652
653</article>
654
655      </div>
656    </main><footer class="site-footer h-card">
657  <data class="u-url" href="/"></data>
658
659  <div class="wrapper">
660
661    <h2 class="footer-heading">Robert Osazuwa Ness</h2>
662
663    <div class="footer-col-wrapper">
664      <div class="footer-col footer-col-1">
665        <ul class="contact-list">
666          <li class="p-name">Robert Osazuwa Ness</li><li><a class="u-email" href="mailto:robertness <at> gmail <dot> com">robertness <at> gmail <dot> com</a></li></ul>
667      </div>
668
669      <div class="footer-col footer-col-2"><ul class="social-media-list"><li><a href="https://github.com/robertness"><svg class="svg-icon"><use xlink:href="/assets/minima-social-icons.svg#github"></use></svg> <span class="username">robertness</span></a></li><li><a href="https://www.linkedin.com/in/osazuwa"><svg class="svg-icon"><use xlink:href="/assets/minima-social-icons.svg#linkedin"></use></svg> <span class="username">osazuwa</span></a></li></ul>
670</div>
671
672      <div class="footer-col footer-col-3">
673        <p>Applications of Statistical Machine Learning, Probabilistic Graphical Models, and Causal Inference
674</p>
675      </div>
676    </div>
677
678  </div>
679
680</footer>
681</body>
682
683</html>

Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.