1<!DOCTYPE html> 2<html> 3 <head> 4 <title>Wizard Zines - Comics</title> 5 <meta charset="utf-8"> 6 <meta http-equiv="X-UA-Compatible" content="IE=edge"> 7 <meta name="viewport" content="width=device-width, initial-scale=1, target-densityDpi=device-dpi" /> 8 <meta property="description" content=""> 9 <meta property="mobile-web-app-capable" content="yes"> 10 <meta property="apple-mobile-web-app-capable" content="yes"> 11 <meta property="apple-mobile-web-app-status-bar-style" content="black"> 12 <meta property="author" content=""> 13 14 15 <meta property="twitter:site" content="@b0rk"> 16 <meta property="twitter:creator" content="@b0rk"> 17 <meta property="og:image" content="/images/main-og-image.png"> 18 19 <meta property="og:site_name" content="wizard zines"> 20 <meta property="og:title" content="Wizard Zines - Comics"> 21 <meta property="og:url" content="https://wizardzines.com/comics/"> 22 <meta property="og:description" content=""> 23 24 <meta property="og:type" content="website" /> 25 26 <meta name="generator" content="Hugo 0.145.0"> 27 28 <link rel="shortcut icon" href="/images/favicon.ico" /> 29 <link rel="alternate" type="application/rss+xml" title="RSS" href="https://wizardzines.com/index.xml"> 30 31 <link href="//fonts.googleapis.com/css?family=Source+Sans+Pro:400,300|Montserrat:700" rel="stylesheet" type="text/css"> 32 <link href="https://fonts.googleapis.com/css2?family=Inter:wght@300;400;500;700;800&display=swap" rel="stylesheet"> 33 <link href="/css/style.css" rel="stylesheet"> 34 <link rel="me" href="https://social.jvns.ca/@b0rk"> 35 <link rel="me" href="https://social.jvns.ca/@b0rk_reruns"> 36 37
37<script async src="https://plausible.io/js/pa-voly1ZSJVJfUTomsqJDt-.js"></script>
37 38
38<script> 39 window.plausible=window.plausible||function(){(plausible.q=plausible.q||[]).push(arguments)},plausible.init=plausible.init||function(i){plausible.o=i||{}}; 40 plausible.init() 41 </script>
41 42
42<script> 43 addEventListener("DOMContentLoaded", (event) => { 44 var url = new URL(window.location.href); 45 var currentParams = new URLSearchParams(url.search); 46 var utmSource = currentParams.get('utm_source'); 47 48 if (document.referrer.includes('twitter.com') || document.referrer.includes("t.co")) { 49 utmSource = "twitter"; 50 } 51 var externalLinks = document.querySelectorAll('a'); 52 externalLinks.forEach(function(link) { 53 var href = link.getAttribute('href'); 54 if (href.includes("store.wizardzines.com")) { 55 var url = new URL(href); 56 url.searchParams.set('utm_source', utmSource); 57 link.setAttribute('href', url.href); 58 } 59 }); 60 }); 61 </script>
61 62 </head> 63 64 65<body> 66<span id="top"></span> 67<div class="sr-only"><a href="#content">Skip to Content</a></div> 68<header id="header" class="squiggle-bottom"> 69<div class="sr-only">Navigation:</div> 70<nav> 71 <h1> 72 <a href="/" class="star-link"> 73 <span class="star">â </span> 74 wizard zines 75 <span class="star">â </span> 76 </a> 77 </h1> 78 <p> 79 programming zines by Julia Evans 80 </p> 81 <a href="/" class="button">home</a> 82 <a href="https://store.wizardzines.com" class="button">store</a> 83 <a href="/comics" class="button">comics</a> 84 <a href="https://library.wizardzines.com" class="button">my library</a> 85</nav> 86</header> 87<span id="content"></span> 88 89 90 91
91<script src="/js/list.min.js"></script>
91 92
92<script src="/js/comic-search.js"></script>
92 93 <section class="squiggle-top squiggle-bottom comic" id="comics"> 94 <h1>Comics!</h1> 95 <p> 96 Here are (almost) all of the comics I've published! They're ordered 97 approximately by popularity, most popular first. 98 </p> 99 100 <div class="centered"> 101 <input placeholder="ð Search" 102 type="search" 103 id="search"> 104 <button id="search-btn">Search</button> 105 </div> 106 107 <div class="comic-pages list"> 108 109 <a href="/comics/bash-tricks/"> 110 <span class="title"> 111 bash tricks 112 </span> 113 <div class="keywords"></div> 114 <div class="transcript">### panel 1: ctrl + r 115search your history! 116I use this **constantly** to rerun commands 117 118### panel 2: magical braces 119```$ convert file.{jpg,png}``` 120expands to 121```$ convert file.jpg file.png``` 122 123`{1..5}` expands to 1 2 3 4 5 124(for i in {1..100}...) 125 126### panel 3: !! 127expands to the last command run 128`$ sudo !!` 129 130### panel 4: 131commands that start with a space don't go in your history. good if there's a password 132 133### panel 5: loops 134``` 135for i in *.png 136do 137convert $i $i.jpg 138done 139``` 140 141person: for loops: easy & useful! 142 143### panel 6: $ ( ) 144 145gives the output of a command 146 147``` $touch file- $ (date -1)``` 148create a file named file-2018-05-25 149 150### panel 7: more keyboard shortcuts 151ctrl + a beginning of line 152ctrl + e end of line 153ctrl + l clear the screen 154 155& lots more emacs shortcuts too!</div> 156 </a> 157 158 <a href="/comics/proc/"> 159 <span class="title"> 160 /proc 161 </span> 162 <div class="keywords"></div> 163 <div class="transcript">### panel 1: 164Every process on Linux has a PID (process ID) like 42. 165 166In /proc/42, there's a lot of VERY USEFUL information about process 42. 167 168### panel 2: /proc/PID/cmdline 169 170command line arguments the process was started with. 171 172### panel 3: /proc/PID/environ 173 174all of the process's environment variables 175 176### panel 4: /proc/PID lexe 177 178Symlink to the process's binary magic: works even if the binary has been deleted! 179 180### panel 5: /proc/PID/status 181 182Is the program running or asleep? How much memory is it using? And much more! 183 184### panel 6: /proc/PID/fd 185 186Directory with every file the process has open! 187 188Run ```$1s-1 /proc/42/fd``` to see the list of files for process 42. 189 190These symlinks are also magic & you can use them to recover deleted files 191 192### panel 7: /proc/PID/stack 193 194The kernel's current stack for the process. Useful if it's stuck in a system call. 195 196### panel 8: /proc/PID/maps 197 198List of process's memory maps. Shared libraries, heap, anonymous maps, etc. 199 200### panel 9: and more 201 202Look at ```man proc``` for more information! 203 204 205 206 207 208 209 210</div> 211 </a> 212 213 <a href="/comics/better-at-debugging/"> 214 <span class="title"> 215 how I got better at debugging 216 </span> 217 <div class="keywords"></div> 218 <div class="transcript">### Remember: the bug is happening for a logical reason. 219It's never magic. Really. Even when it makes no sense. 220 221### Be confident I can fix it 222before: maybe this is too hard 223now: well I've fixed a lot of hard bugs before 224 225### Talk to my coworkers 226person 1: ? 227person 2: ! 228 229### know my debugging toolkit 230before: I want to know $THING but I don't know how to find out 231now: I KNOW! I'll use tcpdump! 232 233### most importantly: I learned to like it 234before: oh no! a bug! 235now: I think I'm about to learn something 236(facial expression: determination)</div> 237 </a> 238 239 <a href="/comics/grep/"> 240 <span class="title"> 241 grep 242 </span> 243 <div class="keywords"></div> 244 <div class="transcript">### panel 1: 245grep lets you files for text search 246```$ grep bananas foo.txt``` 247 248Here are some of my favourite grep command line arguments! 249 250### panel 2: -E 251Use if you want regexps like ".+" to work. otherwise you need to use ".\+" 252 253### panel 3: -v 254invert match find : all lines that don't match 255 256### panel 4: -r 257recursive! Search all the files in a directory. 258 259### panel 5: -o 260only print the matching part of the line (not the whole line) 261 262### panel 6: -i 263case insensitive 264 265### panel 7: 266-A -B -C 267 268Show **c**ontext for your search 269```$grep -A 3 foo``` 270will show 3 lines of context **a**fter a match 271 272### panel 8: -l 273only show the **filenames** of the files that matched 274 275### panel 9: -F aka fgrep 276don't treat the match string as a regex 277eg ```$ grep -F...``` 278 279### panel 10: -a 280search binaries: treat binary data like it's text instead of ignoring it! 281 282### panel 11: grep alternatives 283ack ag ripgrep 284(better for searching code!)</div> 285 </a> 286 287 <a href="/comics/permissions/">
288 <span class="title"> 289 permissions 290 </span> 291 <div class="keywords"></div> 292 <div class="transcript">### panel 1: 293There are 3 things you can do to a file. 294 295**r**ead **w**rite e**x**ecute 296 297 298### panel 2: 299Is -1 file.txt shows you permissions. Here's how to interpret the output: 300rw- **bork** (user) can read & write 301rw- **staff** (group) can read & write 302r-- **ANYONE** can read 303 304### panel 3: File permissions are 12 bits 305 306First digit: setuid 307Second digit: setgid 308Third digit: sticky 309 310User 110 rwx 311Group 110 rwx 312all 100 rwx 313 314 315 316For files: 317r = can read 318W = can write 319X = can execute 320 321For directories, it's approximately: 322r = can list files 323W = can create files 324x = can cd into & access files 325 326### panel 4: 327110 in binary is 6 328 329so rw- = 110 = 6 330r-- = 100 = 4 331r-- = 100 = 4 332 333```chmod 644 file.txt|``` 334means change the permissions to: 335rw- r-- r-- 336 337Simple! 338 339### panel 5: 340setuid affects executables 341```$1s-1/bin/ping``` 342 343 344rw**s** r-x r-x root root 345the s means ping always runs as root 346 347```setgid``` does 3 different unrelated things for executables, directories, and regular files. 348 349person: unix! why?? 350unix: it's a long story 351 352 353 354 355</div> 356 </a> 357 358 <a href="/comics/wizard-programmer/"> 359 <span class="title"> 360 how to be a wizard programmer 361 </span> 362 <div class="keywords"></div> 363 <div class="transcript"></div> 364 </a> 365 366 <a href="/comics/more-bash-tricks/"> 367 <span class="title"> 368 more bash tricks 369 </span> 370 <div class="keywords"></div> 371 <div class="transcript">### `cd -` 372changes to the directory you were last in 373 374`pushd` & `popd` let you keep a stack 375 376### `ctrl + z` 377suspends (SIGTSTP) the running program 378 379### `fg` 380brings backgrounded/suspended program to the foreground 381 382### `bg` 383starts suspended program & backgrounds it (use after `ctrl + z`) 384 385### `shellcheck` 386(with hearts around it) 387 388shell script linter! helpes spot common mistakes 389 390### `<( )` 391(process substitution) 392 393treat process output like a file (no more temp files!) 394 395eg: 396`$diff <(ls) <(ls -a)` 397 398### `fc` 399("fix command") 400 401open the last command you ran in an editor 402 403then run the edited version 404 405### `type` 406tells you if something is a builtin, program, or alias 407 408try running type on 409- time 410- ping 411- pushd 412 413(they're all different types) 414</div> 415 </a> 416 417 <a href="/comics/ssh/"> 418 <span class="title"> 419 ssh 420 </span> 421 <div class="keywords"></div> 422 <div class="transcript">### ssh keys 423An ssh key is a secret key that lets you SSH to a machine 424 425person: hello! 426ssh: That's on my list of authorized keys! come in! 427 428### ssh-copy-id 429This script installs your SSH key on a host (over SSH) 430 431`$ ssh-copy-id user@host` 432(puts it in .ssh/authorized-keys etc) 433 434installing a SSH key is surprisingly finicky so this script is helpful! 435 436### port forwarding 437``` 438ssh [email protected] - Nfl 4393333:localhost:8888 440``` 4413333 = local port 4428888 = remote port 443 444Lets you view a remote server that's not on the internet in your browser. 445 446### just run 1 command 447`$ ssh user@host uname -a` 448runs the command `uname -a` & exits. 449 450### ssh-agent 451remembers your SSH key passphrase so you don't have to keep typing it 452 453### ~. 454<Enter> ~. closes the SSH , connection. Useful if it's hanging! 455 456### mosh 457ssh alternative: keeps the connection open if you disconnect + reconnect later 458 459### .ssh/config 460Lets you set, per host: 461- Username to use. 462- SSH key to use 463- an alias! 464 465so you can type `$ ssh ALIAS` instead of `ssh user@very longdomain.com`</div> 466 </a> 467 468 <a href="/comics/1-1s/"> 469 <span class="title"> 470 what to talk about in 1:1s 471 </span> 472 <div class="keywords">[manager]</div> 473 <div class="transcript">Each of these items is enclosed in a little thought bubble, with an image of a stick figure with short curly hair. The person is smiling in every illustration, except "what'
473s not going well". 474 475### what's been going well 476I LOVE this project! 477 478### what's not going well 479I got paged 10 times last week 480 481### team priorities 482how does my work fit in with company goals? 483 484### career planning 485I'd like to be promoted this year 486 487### ask for opportunities 488I want to work on a customer-facing project 489 490### ask for feedback 491do you have any concerns about how PROJECT is going? 492 493### brainstorm 494let's think about this problem! 495 496### give feedback 497the team seems really unfocused recently 498 499### ask for resources 500I think this training would really help me</div> 501 </a> 502 503 <a href="/comics/brackets-cheatsheet/"> 504 <span class="title"> 505 bash brackets cheat sheet 506 </span> 507 <div class="keywords">[bash]</div> 508 <div class="transcript">### shell scripts have a lot of brackets 509here's a cheat sheet to help you identify them all! we'll cover the details later. 510 511### (cd ~/music; pwd) 512`(...)` runs commands in a subshell. 513 514### VAR=$(cat file.txt) 515`$(COMMAND)` is equal to `COMMAND`'S stdout 516 517### { cd ~/music; pwd; } 518`(...)` groups commands. runs in the same process. 519 520### x=(1 2 3) 521`x=(...)` creates an array 522 523### x=$((2+2)) 524`$(())` does arithmetic 525 526### if [...] 527`/usr/bin/[` is a program that evaluates statements 528 529### <(COMMAND) 530"process substitution": an alternative to pipes 531 532### a{.png, .svg} 533this expands to `a.png a.svg` it's called "brace expansion" 534 535### if [[ ... ]] 536`[[` is bash syntax. it's more powerful than `[` 537 538### ${var//search/replace} 539see page 21 for more about `${...}`!</div> 540 </a> 541 542 <a href="/comics/box-model/"> 543 <span class="title"> 544 the box model 545 </span> 546 <div class="keywords"></div> 547 <div class="transcript">### every HTML element is in a box 548``` 549<div class="1"> 550<div class="2" /> 551<div class="3" /> 552</div> 553``` 554Illustration of a larger box, labelled 1. Nested inside it are two boxes. The one on top is labelled 2, and the one below 2 is labelled 3. 555 556 557### boxes have padding, borders, and a margin 558 559Illustration of a series of nested boxes. The middle box is empty. The area around the middle box is labelled "padding". The area around the padding is labelled "border". The area around the border is labelled "margin". 560 561### width & height don't include any of those 562 563The same illustration from the previous panel, but with two lines measuring the width and height of only the middle box, not the padding, border, or margin. 564 565### margins are allowed to overlap sometimes 566 567Illustration of two sets of nested boxes, similar to the diagrams above. One is on top of the other, and the area between the sets of boxes is shaded in green, showing that the bottom margin of the first set of boxes, and the top margin of the second set of boxes, overlap. 568 569the browser combines these top/bottom margins. 570 571look up "margin collapse" to learn more 572 573### `box-sizing: border-box;` includes border + padding in the width/height 574 575Illustration of a series of nested boxes with a middle box surrounded by padding, border, and margin. In this version, the lines measuring width and height extend all the way to the edge of the border (but don't include the margin surrounding the border.) 576 577### inline elements ignore other inline elements' vertical padding 578 579Illustration of two dotted line boxes stacked directly on top of one another. Each has the word "`span`" inside it. 580 581you can set vertical padding but the other span won't move</div> 582 </a> 583 584 <a href="/comics/memory-allocation/"> 585 <span class="title"> 586 memory allocation 587 </span> 588 <div class="keywords"></div> 589 <div class="transcript">### your program has memory 59010MB: program binary 5913MB: stack 592587 MB: heap 593 594the heap is what your allocator manages 595 596### Your memory allocator (malloc) is responsible for 2 things. 597THING 1: keep track of what memory is used/free. 598 599### THING 2: Ask the OS for more memory! 600malloc: oh no! I'm being asked for 40 MB and I don't have it. 601malloc: can I have 60 MB more? 602OS: here you go! 603 604### your memory allocator's interface 605- malloc(size_t size): allocate size bytes of memory & return a pointer to it. 606- free (void* pointer): mark the memory as unused (and maybe give back to the OS) 607- realloc(void pointer, size_t size): ask for more/less memory for pointer. 608- Calloc (size-t members, size_t size): allocate array + initialize to 0. 609 610### malloc tries to fill in for space memory when you ask 611your code: can I have 512 bytes of memory? 612malloc: YES! 613 614### malloc isn't magic! it's just a function!
615you can always: 616- use a different malloc library like jemalloc or tcmalloc (easy!) 617- implement your own malloc (harder)</div> 618 </a> 619 620 <a href="/comics/misc-commands/"> 621 <span class="title"> 622 misc commands 623 </span> 624 <div class="keywords"></div> 625 <div class="transcript">### `rlwrap` 626adds history & ctrl support to REPLs that don't already have them 627 628(`rl` stands for readline) 629 630`$ rlwrap python` 631 632### `watch` 633rerun a command every 2 second 634 635### `file` 636figures out what kind of file (png? pdf?) a file is 637 638### `pv` 639"pipe viewer", gives you stats on data going through a pipe 640 641### `cal` 642a tiny calendar (heart) 643 644### `ts` 645add a timestamp in front of every input line 646 647### `comm` 648find lines 2 sorted files have in common 649 650### `ncdu` 651figure out what's using all your disk space 652 653### `column` 654format input into columns 655 656### `diff` 657diff 2 files. Run with `-U 8` for context. 658 659### `xsel/xclip` 660copy/paste from system clipboard 661 662(`pbcopy`/`pbpaste` on Mac)</div> 663 </a> 664 665 <a href="/comics/sql-query-order/"> 666 <span class="title"> 667 SELECT queries start with FROM 668 </span> 669 <div class="keywords"></div> 670 <div class="transcript">Conceptually, every step (like "`WHERE`") of a query transforms its input, like this: 671 672cats 673 674owner: 1 name: daisy 675owner: 1 name: dragonsnap 676owner: 3 name: buttercup 677owner: 4 name: rose 678 679`WHERE owner = 1` 680 681owner: 1 name: daisy 682owner: 1 name: dragonsnap 683 684The query's steps don't happen in the order they're written: 685how the query' is written 686SELECT... 687FROM + JOIN 688WHERE ... 689GROUP BY ... 690HAVING ... 691ORDER BY... 692LIMIT... 693 694how you should think about it: 695FROM + JOIN 696â 697WHERE 698â 699GROUP BY 700â 701HAVING 702â 703SELECT 704â 705ORDER BY 706â 707LIMIT 708 709(In reality query execution is much more complicated than this. There are a lot of optimizations.)</div> 710 </a> 711 712 <a href="/comics/request-headers/"> 713 <span class="title"> 714 the most important HTTP request headers 715 </span> 716 <div class="keywords"></div> 717 <div class="transcript">These are the most important request headers: 718 719### Host 720The domain 721The only required header. 722`Host: examplecat.com` 723 724### User-Agent 725name + version of your browser and OS 726`User-Agent: curl 7.0.2` 727 728### Referer 729(yes, it's misspelled!) 730website that linked or included the resource 731`Referer: http://examplecat.com` 732 733### Authorization 734eg a password or API token 735base64 encoded user: password 736`Authorization: Basic YXZ` 737 738### Cookie 739send cookies the server sent you earlier 740keeps you logged in 741`Cookie: user=b0rk` 742 743### Range 744lets you continue downlats ("get bytes 100-200") 745`Range: bytes=100-200` 746 747### Cache-Control 748"max-age=60" 749means cached responses must be less than 60 seconds old 750 751### If-Modified-Since 752only send if resource was modified after this time 753`If-Modified-Since: Wed, 21 Oct...` 754 755### If-None-Match 756only send if the ETag doesn't match those listed 757`If-None-Match: "e7ddac"` 758 759### Accept 760MIME type you want the response to be 761`Accept: image/png` 762 763### Accept-Encoding 764set this to "gzip" and you'll probably get a compressed response" 765`Accept-Encoding: gzip` 766 767### Accept-Language 768set this to "fr-CA" and you might get a response in French 769`Accept-Languag: fr-CA` 770 771### Content-Type 772MIME type of request body, e.g. "application/json" 773 774### Content-Encoding 775will be "gzip" if the request body is gzipped 776 777### Connection 778"close" or "keep-alive". 779Whether to keep the TCP connection open</div> 780 </a> 781 782 <a href="/comics/curl/"> 783 <span class="title"> 784 curl 785 </span> 786 <div class="keywords"></div> 787 <div class="transcript">### `curl` 788 789smiling stick figure with short curly hair: it's my favourite way to make HTTP requests! 790 791great for testing APIs! 792 793`$ curl wizardzines.com` 794 795### `-H` 796is for for header 797 798good for POST requests to JSON APIs 799`-H` "content-Type:application/json" 800 801allow compressed response: 802`-H` "Accept-Encoding: gzip" 803 804### `-L` 805 806follow 3xx redirects 807 808### `--data` 809 810`--data '{"name": "julia"'` 811 812`--data @filename.json` 813 814(@ reads the data to send from a file) 815 816### `i` 817show response headers 818 819### `I` 820show ONLY response headers (makes a HEAD request) 821 822### `- X POST` 823send a POST request instead of a GET (`-X PUT` etc works too) 824 825### `- v` 826show request headers & more 827 828### `- k` 829insecure: don't verify SSL certificates 830 831### `--connect to ::IP` 832(or hostname) 833 834send request to IP instead 835 836use bfore changing DNS to a new IP 837
838### `copy as cURL` 839 840Have something in your browser you want to download from the command line? 841 842In Firefox/Chrome/Safari: 843 844Developer Tools 845-> Network tab 846-> right click on the request 847-> copy as curl 848(can have sensitive info in cookies!)</div> 849 </a> 850 851 <a href="/comics/bash-errors/"> 852 <span class="title"> 853 bash errors 854 </span> 855 <div class="keywords"></div> 856 <div class="transcript">### by default, bash will continue after errors 857bash, represented by a box with a smiley face: oh, was that an error? who cares, let's keep running!!! 858 859programmer, represented by a nonplussed stick figure with short curly hair: uh that is NOT what I wanted 860 861### `set -e` stops the script on errors 862``` 863set -e 864unzip fle.zip 865``` 866(typo! script stops here!) 867 868programmer, smiling: this makes your scripts WAY more predictable 869 870### by default, unset variables don't error 871`rm -r "$HOME/$SOMEPTH"` 872 873bash, happily: `$SOMEPTH` doesn't exist? no problem, i'll just use an empty string! 874 875programmer: OH NOOOO that means `rm -rf $HOME` 876 877### `set -u` stops the script on unset variables 878``` 879set-u 880rm -r "$HOME/$SOMEPTH" 881``` 882bash, concerned: I've never heard of `$SOMEPTH`! STOP EVERYTHING!!! 883 884### by default, a command failing doesn't fail the whole pipeline 885 886`curl yxqzq.ca | grep 'panda'` 887 888bash, pleased with itself: `curl` failed but `grep` succeeded so it's fine! success! 889 890### `set -o pipefail` makes the pipe fail if any command fails 891 892you can combine `set -e`, `set -u`, and `set -o pipefail` into one command I put at the top of all my scripts: 893 894`set -euo pipefail`</div> 895 </a> 896 897 <a href="/comics/xargs/"> 898 <span class="title"> 899 xargs 900 </span> 901 <div class="keywords"></div> 902 <div class="transcript">### xargs takes white space separated strings from stdin and converts them into command-line arguments 903``` 904$ echo "/home /tmp" 905| xargs ls 906``` 907will run `ls /home/tmp` 908 909### this is useful when you want to run the same command on a list of files! 910- delete (`xargs rm`) 911- combine (`xargs cat`) 912- search (`xargs grep`) 913- replace (`xargs sed`) 914 915### how to replace "foo" with "bar" in all .txt files: 916``` 917find. -name '*.txt' | 918xargs sed -i s/foo/bar/g 919``` 920 921### how to lint every Python file in your Git repo: 922``` 923git ls-files | grep pyl 924xargs pep8 925``` 926### if there are spaces in your filenames "my day.txt" xargs will think it's 2 files 927""my" and "day.txt" 928fix it like this: 929``` 930find -print0 | 931xargs -0 COMMAND 932``` 933### more useful xargs options 934`-n 1` (max-args): makes xargs run a separate process max-args for each input 935`-P` (capital P, max-procs): is the max number of parallel processes xargs will start</div> 936 </a> 937 938 <a href="/comics/awk/"> 939 <span class="title"> 940 awk 941 </span> 942 <div class="keywords"></div> 943 <div class="transcript">### panel 1: 944awk is a tiny programming language for manipulating columns of data 945 946person: I only know how to do 2 things with awk but it's still useful! 947 948### panel 2: basic awk program structure 949``` 950BEGIN {...} 951CONDITION (action} 952CONDITION (action} 953``` 954(do action on lines matching CONDITION) 955``` 956END {...} 957``` 958 959 960### panel 3: extract a column.of text with awk 961```awk -F, '{print $5}'``` 962the comma is the column separator 963the ' is a single quote! 964```{print $5}``` means print the 5th column 965 966person: this is 99% of what I do with awk 967 968### panel 4: 969SO MANY Unix commands print columns of text (ps! Is!) 970 971so being able to get the column you want with awk is GREAT 972 973### panel 5: awk program example 974sum the numbers in the 3rd column 975 976``` {s += $3};``` (action) 977``` END {print s}'``` (at the end, print the sum!) 978 979 980### panel 6: awk program example 981print every line over 80 characters 982```length($0) > 80``` 983"length" is the condition 984 985(there's an implicit ```{print}``` as the action)</div> 986 </a> 987 988 <a href="/comics/cors/">
989 <span class="title"> 990 CORS 991 </span> 992 <div class="keywords"></div> 993 <div class="transcript">cross-origin resource sharing 994 995Cross-origin requests are not allowed by default: (because of the same origin policy!) 996 997Javascript from clothes.com: POST request to api.clothes.com? 998 999Firefox (thought bubble): same origin flow chart 1000 1001Firefox: NOPE. api.clothes.com is a different origin from clothes.com 1002 1003If you run api.clothes.com, you can allow clothes.com to make requests to it using the `Access-Control-Allow-Origin` header. Here's what happens: 1004 1005javascript on clothes.com: `POST /buy_thing`\ 1006`Host: api.clothes.com` 1007 1008Firefox (thought bubble): That's cross-origin. I'm going to need to ask api.clothes.com if this request is allowed. 1009 1010Firefox: `OPTIONS /buy_thing`\ 1011`Host: api.clothes.com` ("hey, what requests are allowed?" preflight request) 1012 1013api.clothes.com: ```204 No Content`` ```Access-Control-Allow-Origin: clothes.com` 1014 1015Firefox (thought bubble): cool, the request is allowed! 1016 1017Firefox: `POST /buy_thing`\ 1018`Host: api.clothes.com`\ 1019`Referer: clothes.com/checkout` 1020 1021api.clothes.com: `200 OK`\ 1022`{"thing_bought": true}` 1023 1024This OPTIONS request is called a "preflight" request, and it only happens for some requests, like we described in the diagram on the same-origin policy page. Most GET requests will just be sent by the browser without a preflight request first, but POST requests that send JSON need a preflight.</div> 1025 </a> 1026 1027 <a href="/comics/css-variables/"> 1028 <span class="title"> 1029 CSS variables 1030 </span> 1031 <div class="keywords"></div> 1032 <div class="transcript">### duplication is annoying 1033 1034Illustration of a frowning stick figure with curly hair. 1035 1036person, thinking: ugh, I have `color: #f79` set in 27 places and now I need to change it in 27 places 1037 1038### define variables in any selector 1039 1040``` 1041body { 1042 --text-color: #f79; 1043body { 1044} 1045 1046``` 1047(applies to everything) 1048``` 1049#header { 1050--text-color: #c50; 1051} 1052``` 1053(applies to children of `#header`) 1054 1055### use variables with `var()` 1056``` 1057body { 1058 color: var(--text-color); 1059} 1060``` 1061(variables always start with `--`) 1062 1063### do math on them with `calc()` 1064``` 1065#sidebar { 1066 width: calc( 1067 var (--my-var) + 1em 1068 ); 1069} 1070``` 1071 1072### you can change a variable's value in Javascript 1073``` 1074let root = 1075 document.documentElement; 1076root.style.setProperty( 1077 '--text-color', 'black'); 1078``` 1079 1080 1081### changes to variables apply immediately 1082JS, represented by a box with a smiley face: set `--text-color` to red 1083 1084css renderer, also represented by a box with a smiley face: ok everything using it is red now! 1085 1086</div> 1087 </a> 1088 1089 <a href="/comics/containers-arent-magic/"> 1090 <span class="title"> 1091 containers aren't magic 1092 </span> 1093 <div class="keywords"></div> 1094 <div class="transcript">These 15 lines of bash will start a container running the fish shell. Try it! (download this script at bit.ly/containers-arent-magic) 1095 1096It only runs on Linux because these features are all Linux-only. 1097 1098`wget bit.ly/fish-container -O fish.tar` (# 1. download the image) 1099`mkdir container-root; cd container-root` 1100`tar -xf ../fish.tar` (# 2. unpack image into a directory) 1101`cgroup_id="cgroup_$(shuf -i 1000-2000 -n 1)"` (# 3. generate random cgroup name) 1102`cgcreate -g "cpu, cpuacct, memory: $cgroup_id"` (# 4. make a cgroup & set CPU/memory limits) 1103`cgset -r cpu. shares=512 "$cgroup_id"` 1104`cgset -r memory.limit_in_bytes=1000000000 \` 1105`"$cgroup_id"` 1106`cgexec -g "cpu, cpuacct, memory: $cgroup_id" \ ` (# 5. use the cgroup) 1107`unshare -fmuipn --mount-proc\` (# 6. make and use some namespaces) 1108` chroot "$PWD" \` (# 7. change root directory) 1109`/bin/sh -c "` 1110`/bin/mount -t proc proc /proc &&` (# 8. use the right /proc) 1111`hostname container-fun-times &&` (# 9. change the hostname) 1112`/usr/bin/fish"` (# 10. finally, start fish!)</div> 1113 </a> 1114 1115 <a href="/comics/virtual-memory/">
1116 <span class="title"> 1117 virtual memory 1118 </span> 1119 <div class="keywords"></div> 1120 <div class="transcript">### your computer has physical memory 1121memory 1122868 204-PIN SODIMM DDR3 CE 1123 1124### physical memory has addresses, like O-8GB 1125but when your program references an address like Ox 5c69a2a2, that's not a physical with memory address! It's a virtual address. 1126 1127 1128### every program has its own virtual address space 1129program 1: Ox 129520 â "puppies" 1130program 2: Ox 129520 â "bananas" 1131 1132 1133### Linux keeps a mapping, from virtual memory pages to physical memory pages called the page table 1134a "page" is a 4kb or chunk of memory (or sometimes bigger) 1135 1136PID -- virtual addr -- physical addr 11371971 -- Ox 20000 -- Ox 192000 11382310 -- Ox 20000 -- Ox 228000 11392310 -- Ox21000 -- Ox 9788000 1140 1141### when your program accesses a virtual address 1142CPU: I'm accessing Ox21000 1143MMU "memory management unit" (hardware): I'll look that up in the page table and then access the right physical address 1144 1145### every time you switch which process is running, Linux needs to switch the page table 1146Linux: here's the address of process 2950's page table 1147MMU: thanks, I'll use that now!</div> 1148 </a> 1149 1150 <a href="/comics/syscalls/"> 1151 <span class="title"> 1152 how to talk to your operating system 1153 </span> 1154 <div class="keywords"></div> 1155 <div class="transcript">## how to talk to your operating system 1156 1157### WRONG: 1158 1159weird guy: good morning madam would you care to open a file for me 1160operating system: what 1161 1162### RIGHT: 1163system calls! yay! (surrounded by hearts, smiley faces, and exclamation marks) 1164 1165happy stick figure with curly hair: 1166``` 1167open("/cool.txt") 1168connect(<my friend's computer>) 1169``` 1170your programs can: 1171- open 1172- read 1173- write 1174 1175files 1176 1177talk to other computers with networking: 1178- connect 1179- sendto 1180- recvfrom 1181 1182start other programs: 1183- execue 1184 1185AND MUCH MORE!!! 1186 1187(these are all system calls on Linux!)</div> 1188 </a> 1189 1190 <a href="/comics/cdn/"> 1191 <span class="title"> 1192 content delivery networks 1193 </span> 1194 <div class="keywords"></div> 1195 <div class="transcript">In 2004, if your website suddenly got popular, often the webserver wouldn't be able to handle all the requests. 1196 1197slashdot: 1198person 1: I want cat picture! 1199person 2: me too! 1200person 3: me 300,000! 1201 1202server, on fire: <no response> 1203web host: now you owe me $1000 for bandwidth 1204you: how will I pay for this? 1205 1206A CDN (content delivery network) can make your site faster and save you money by caching your site and handling most requests itself. 1207 120820 million requests for 1 cute cat picture -> CDN (many powerful computers) -> 1209just 1 request: hey send me that cat picture? 1210server: here you go! 1211 1212Today, there are many free or cheap CDN services available, which means if your site gets popular suddenly you can easily keep it running! 1213 1214This is great but caching can cause problems too! 1215 1216I updated my site yesterday but people are still seeing the old site! 1217(Cache-Control header) 1218 1219French users are seeing the English site?!? Why? 1220(Vary header) 1221 1222Next, we'll explain the HTTP headers your CDN or browser uses to decide how to do caching.</div> 1223 </a> 1224 1225 <a href="/comics/inline-vs-block/"> 1226 <span class="title"> 1227 inline vs block 1228 </span> 1229 <div class="keywords"></div> 1230 <div class="transcript">### HTML elements default to inline or block 1231example inline elements: `<a> <span> <strong> <i> <small> <abbr> <img> <q> <code>` 1232example block elements: `<p> <div> <ol> <ul><li> <h1> <h6> <blockquote> <pre>` 1233 1234### inline elements are laid out horizontally 1235text text text `<a>` text text 1236text text `<span>` text text 1237 1238### block elements are laid out vertically by default 1239`<div>` 1240`<p>` 1241to get a different layout, use `display: flex` or `display: grid` 1242 1243### inline elements ignore width & height* 1244Setting the width is impossible, but in some situations, you can use `line-height` to change the height 1245 1246`*` img is an exception to this: look up "replaced elements" for more 1247 1248### display can force an element to be inline or block 1249`display` determines 2 things: 12501. whether the element itself is `inline`, `block`, `inline-block`, etc 12512. how child elements are laid out (`grid`, `flex`, `table`, `default`, etc) 1252 1253### display: inline-block; 1254TRY ME! 1255`inline-block` makes a block element be laid out horizontally like an inline element 1256 1257inline text 1258more inline text 1259inline-block 1260inline text </div> 1261 </a> 1262 1263 <a href="/comics/ask-for-specific-feedback/">
1264 <span class="title"> 1265 ask for specific feedback 1266 </span> 1267 <div class="keywords">[manager]</div> 1268 <div class="transcript">I used to ask for feedback like this: 1269Illustration of two stick figures, both smiling. Person 1, the employee, has short curly hair, and person 2, the manager, doesn't have hair. 1270 1271person 1 (speech bubble): dо you have any feedback for me? 1272person 2 (speech bubble): not right now! 1273person 1 (thought bubble): is there something they're not telling me? 1274person 2 (thought bubble): what specifically does she want feed back on? 1275 1276 1277I've learned that I get WAY BETTER answers if I ask more specific questions! 1278- what do you think of this design? 1279- did I prioritize these things well? 1280- should I be doing more or less of X? 1281- do you have any concerns about PROJECT? 1282- was that email clear? 1283 1284 1285Bonus: asking specific questions forces me to actually think about which areas I might want to focus on.</div> 1286 </a> 1287 1288 <a href="/comics/ip/"> 1289 <span class="title"> 1290 ip 1291 </span> 1292 <div class="keywords"></div> 1293 <div class="transcript">### ip 1294(Linux only) 1295 1296lets you view + change network configuration. 1297 1298`ip OBJECT COMMAND` 1299 1300(`OBJECT` = addr, link neigh, etc) 1301(`COMMAND` = add, show, delete, etc) 1302 1303Here are some ways to use it! 1304 1305### ip addr list 1306shows ip addresses your devices. Look for something like this: 1307``` 13082: eth0: 1309 link/ether 3c:97... 1310 inet 192.168.1.170/24 1311``` 1312 1313### ip route list 1314displays the route table. 1315 1316`default via 192.168.1.1` (my router) 1317`169.240.0.0/16 dev docker` 1318`...` 1319 1320to see all route tables: 1321`ip route list table all` 1322 1323### change your MAC address 1324good for cafés with time limits (devil face emoji) 1325``` 1326$ ip link set wlan0 down 1327$ ip link set wlan0 address 13283ca9f4d1:00:32 1329$ ip link set wlan0 up 1330$ service network-manager 1331restart 1332``` 1333(or whatever you use) 1334 1335### `ip link` 1336network devices! (like eth0) 1337 1338### `ip neigh` 1339view/edit the ARP table 1340 1341### `ip xfrm` 1342is for IPsec 1343 1344### `ip route get IP` 1345what route will packets with $IP take? 1346 1347### `--color` 1348(the letters of "color" are in various rainbow colours) 1349pretty colourful output! 1350 1351### `-- brief` 1352show a summary 1353 1354 1355 1356</div> 1357 </a> 1358 1359 <a href="/comics/how-urls-work/"> 1360 <span class="title"> 1361 how URLs work 1362 </span> 1363 <div class="keywords"></div> 1364 <div class="transcript">`https://examplecat.com:443/cats?color=light%20gray#banana` 1365 1366- scheme (`https://`): Protocol to use for the request. Encrypted (`https`), insecure (`http`), or something else entirely (`ftp`). 1367- domain (`examplecat.com`): Where to send the request. For HTTP(s) requests, the Host header gets set to this (`Host: example.com`) 1368- port (`:443`): Defaults to 80 for HTTP and 443 for HTTPS. 1369- path (`/cats`): Path to ask the server for. The path and the query parameters are combined in the request, like: `GET /cats?color=light%20gray HTTP/1/1` 1370- query parameters (`color=light gray`): Query parameters are usually used to ask for a different version of a page ("I want a light gray cat!"). Example: 1371`hair-short&color=black&name=mr%20darcy`. Hair is the name, short is the value, separated by & 1372- URL (`encoding %20`): URLS aren't allowed to have certain special characters like spaces, @, etc. So to put them in a URL you need to percent encode them as % + hex representation of ASCII value. space is %20, % is %25, etc. 1373- fragment id (`#banana`): This isn't sent to the server at all. It's used either to jump to an HTML tag (`<a id="banana"..>`) or by Javascript on the page. 1374</div> 1375 </a> 1376 1377 <a href="/comics/namespaces/"> 1378 <span class="title"> 1379 namespaces 1380 </span> 1381 <div class="keywords">[containers]</div> 1382 <div class="transcript">### inside a container, things look different2` 1383Illustration of a smiling stick figure with curly hair. 1384 1385Person: I only see 4 processes in `ps aux`, that's weird... 1386 1387 1388### why things look different: namespaces 1389Illustration of a container, represented by a box with a smiley face 1390 1391Container: I'm in a different PID namespace so `ps aux` shows different processes! 1392 1393### every process has 7 namespaces 1394 1395``` 1396$ lsns -p 273 1397NS TYPE 13984026531835 cgroup 13994026531836 pid 14004026531837 user 14014026531838 uts 14024026531839 ipc 14034026531840 mnt 14044026532009 net 1405``` 1406-p is the PID 14074026532009 is the namespace ID 1408 1409you can also see a process's namespace with: 1410`$ ls -1 /proc/273/ns` 1411 1412### there's a default ("host" namespace) 1413Person: "outside a container" just means "using the default namespace" 1414 1415### processes can have any combination 1416Container: I'm using the host network namespace but my own mount container namespace!</div> 1417 </a> 1418 1419 <a href="/comics/units/">
1420 <span class="title"> 1421 CSS units 1422 </span> 1423 <div class="keywords"></div> 1424 <div class="transcript">### CSS has 2 kinds of units: absolute & relative 1425absolute: 1426- px 1427- pt 1428- pc 1429- in 1430- cm 1431- mm 1432 1433relative 1434- em 1435- rem 1436- vw 1437- vh 1438- % 1439 1440### `rem` 1441the root element's font size 1442 1443`1rem` is the same everywhere in the document. `rem` is a good unit for setting font sizes! 1444 1445### `em` 1446the parent element's font size 1447``` 1448.child { 1449 font-size: 1.5em; 1450} 1451``` 1452 1453Illustration of a box labelled "parent". Inside it is a box labelled, in larger text, "child". An arrow is pointing to the "child" text, labelled "font size is 1.5 x parent". 1454 1455 1456### O is the same in all units 1457``` 1458.btn { 1459 margin: 0; 1460} 1461``` 1462also, `0` is different from `none`. `border: 0` sets the border width and `border: none` sets the style 1463 1464### 1 inch = 96 px 1465on a screen, 1 CSS "inch" isn't really an inch, and 1 CSS "pixel" isn't really a screen pixel. look up "device pixel ratio" for more. 1466 1467### rem & em help with accessibility 1468``` 1469.modal { 1470 width: 20rem; 1471} 1472``` 1473this scales nicely if the user increases their browser's default font size</div> 1474 </a> 1475 1476 <a href="/comics/css-centering/"> 1477 <span class="title"> 1478 centering in CSS 1479 </span> 1480 <div class="keywords"></div> 1481 <div class="transcript">### center text with `text-align` 1482``` 1483h2 { 1484text-align: center; 1485} 1486``` 1487 1488### center block elements with `margin: auto` 1489 1490example HTML: 1491``` 1492<div class="parent"> 1493<div class="child"> 1494</div> 1495</div> 1496``` 1497 1498### `margin: auto` only centers horizontally 1499 1500``` 1501.child { 1502width: 400px; 1503margin: auto; 1504} 1505``` 1506 1507Illustration of a smaller box, labelled "child", inside a larger box. The child box is at the top of the larger (parent) box. An arrow pointing to the child box is labelled "not centered vertically!" 1508 1509### vertical centering is easy with flexbox or grid 1510 1511A spiky box labelled "TRY ME" 1512 1513here's how with grid: 1514``` 1515.parent { 1516display: grid; 1517place-items: center; 1518} 1519``` 1520 1521and with flexbox: 1522 1523``` 1524.parent { 1525display: flex; 1526} 1527.child { 1528margin: auto; 1529} 1530``` 1531 1532### it's ok to use a flexbox or grid just to center one thing 1533 1534Illustration of a smaller box nested inside a larger box. The larger box is labelled ".parent `(display: grid)`" and the smaller box is labelled ".child (centered!)"</div> 1535 </a> 1536 1537 <a href="/comics/less/"> 1538 <span class="title"> 1539 less 1540 </span> 1541 <div class="keywords"></div> 1542 <div class="transcript">### less is a pager 1543 1544that means it lets you view (not edit) text files. 1545 1546man uses your pager (usually `less`) to display man pages 1547 1548### many vim shortcuts work in less 1549- `/` search 1550- `n/N` next/prev match 1551- `j/k` down / up a line 1552- `m/'` mark/return to line 1553- `g`(`gg` in vim)/`G` beginning /end of file 1554 1555 1556### less -r 1557displays bash escape codes as colours 1558 1559try `ls --color | less -r` 1560 1561with `-r`: 1562- `a.txt` 1563- `a.txt.gz` (red, bold) 1564 1565without `-r` 1566- `a.txt` 1567ESCLOM ESC C01;31ma.txt.gz ESCCOM 1568(ugh) 1569 1570or piped in text 1571 1572### q 1573quit (smiley face) 1574 1575### v (lowercase) 1576edit file in your $EDITOR 1577 1578### arrow keys, Home / End, PgUp, Pg Dn work in less 1579 1580### F 1581press F to keep reading from the file as it's updated (like `tail -f`) 1582 1583press Ctrl+C to stop reading updates 1584 1585### + 1586`+` runs a command when less starts 1587 1588- `less +F` : follow updates 1589- `less +G`: start at end of file 1590- `less +20%`: start 20% into file 1591- `less +/foo`: search for 'foo' right away</div> 1592 </a> 1593 1594 <a href="/comics/netcat/"> 1595 <span class="title"> 1596 netcat 1597 </span> 1598 <div class="keywords"></div> 1599 <div class="transcript">### `nc` 1600lets you create TCP (or UDP) connections from the command line 1601 1602smiling stick figure, to box with smiley face: I hand wrote this HTTP request for you! 1603 1604### `nc - l PORT` 1605 1606start a server! this listens on `PORT` and prints everything received 1607 1608network connection -> `nc` -> `stdout` 1609 1610### `nc IP PORT` 1611 1612be a client! opens a TCP connection to `IP:PORT` (to send UDP use `-u`) 1613 1614### `make HTTP request by hand` 1615 1616``` 1617printf 'GET / HTTP/ 1.1\r\nHost:example.com\r\n\r\n' | nc example.com 80 1618``` 1619type in any weird HTTP request you want! (smiley face) 1620 1621### `send files` 1622 1623want to send a 100GB file to someone on the same wifi network? easy! 1624 1625``` 1626receiver: 1627nc-1 8000 > file 1628sender: 1629cat file | nc YOUR_IP 8080 1630``` 1631 1632happy stick figure with short curly hair: I love this trick It works even if you're disconnected from the internet!</div> 1633 </a> 1634 1635 <a href="/comics/having/">
1636 <span class="title"> 1637 HAVING 1638 </span> 1639 <div class="keywords"></div> 1640 <div class="transcript">person: every user has a different email right? 16411 query later... person, now sad: oh no 1642 1643This query uses `HAVING` to find all emails that are shared by more than one user: 1644 1645``` 1646SELECT email, COUNT(*) 1647FROM users 1648GROUP BY email 1649HAVING COUNT(*) > 1 1650``` 1651 1652users: 1653id 1, email [email protected] 1654id 2, email [email protected] 1655id 3, email [email protected] 1656 1657query output: 1658email [email protected], `COUNT`(*) 2 1659 1660`HAVING` is like `WHERE`, but with 1 difference: `HAVING` filters rows AFTER grouping and `WHERE` filters rows BEFORE grouping. 1661 1662Because of this, you can use aggregates (like `COUNT` (*)) in a `HAVING` clause but not with `WHERE`. 1663 1664Here's another `HAVING` example that finds months with more than $6.00 in income: 1665 1666``` 1667SELECT month 1668FROM sales 1669GROUP BY month 1670HAVING SUM(price) > 6 1671``` 1672sales: 1673month: Jan. item: catnip price: 5 1674month: Feb item: laser price: 8 1675month: March item: food price: 4 1676month: March item: food price: 3 1677 1678query output: 1679month: Feb 1680month: March</div> 1681 </a> 1682 1683 <a href="/comics/sed/"> 1684 <span class="title"> 1685 sed 1686 </span> 1687 <div class="keywords"></div> 1688 <div class="transcript">### sed is most often used for replacing text in a file 1689`$ sed s/cat/dog/g file.txt` 1690"cat"can be a regular expression 1691 1692### change a file in place. with -i 1693person: in GNU sed it's -i, in BSD sed, -i SUFFIX confuses me every time. 1694 1695### Some more sed incantations... 1696 1697### sed -n 12 p 1698print 12th line 1699-n suppresses output so only what you print with 'p' gets printed 1700 1701### sed 5 d 1702delete 5th line 1703 1704### sed /cat/d 1705delete lines matching /cat/ 1706 1707### sed -n 5,30 p 1708print lines 5-30 1709 1710### sed s+cat/+dog/+ 1711('+' can be any character) 1712Use + as a regex delimeter 1713person: way easier than escaping /s like s/cat\//dog\//! 1714 1715### sed -n s/cat/dog/p 1716only print changed lines. 1717 1718### sed G 1719double space a file (good for long error lines) 1720 1721### sed /cat la dog' 1722append 'dog' after lines containing 'cat' 1723 1724### sed 'i 17 panda" 1725insert "panda" on line 17</div> 1726 </a> 1727 1728 <a href="/comics/write-for-one-person/"> 1729 <span class="title"> 1730 write for one person 1731 </span> 1732 <div class="keywords"></div> 1733 <div class="transcript"></div> 1734 </a> 1735 1736 <a href="/comics/ngrep/"> 1737 <span class="title"> 1738 ngrep 1739 </span> 1740 <div class="keywords">[tcpdump]</div> 1741 <div class="transcript">### like grep for your network 1742(network is surrounded with glowy lines) 1743 1744`$ sudo ngrep GET` will find every plaintext HTTP GET request 1745 1746### ngrep syntax 1747``` 1748$ ngrep 1749 [options] 1750 [regular expression] 1751 [BPF filter] 1752``` 1753("regular expression" is what to search packets for) 1754"BPF filter" use the same format as tcpdump uses! 1755 1756### panel 3 1757 1758Illustration of a smiling stick figure with curly hair. 1759 1760person: I started using `ngrep` when I was intimidated by tcpdump and I found it easier (heart) 1761 1762### -d 1763is for device 1764 1765which network interface to use. same as tcpdump's `-i` (try `-d any`!) 1766ngrep 1767 1768### -W byline 1769prints line breaks as line breaks, not "\n". Nice when looking at HTTP requests 1770 1771### -I file.pcap -O file.pcap 1772read/write packets from/to a pcap file</div> 1773 </a> 1774 1775 <a href="/comics/disk-usage/"> 1776 <span class="title"> 1777 du & df 1778 </span> 1779 <div class="keywords"></div> 1780 <div class="transcript"></div> 1781 </a> 1782 1783 <a href="/comics/floating-point/"> 1784 <span class="title"> 1785 floating point 1786 </span> 1787 <div class="keywords"></div> 1788 <div class="transcript">### a double is 64 bits 1789``` 179010011011 10011011 10011011 1011011 179110011011 10011011 10011011 1011011 1792``` 1793(the first digit is the sign, the next 11 digits are the exponent, the rest is the fraction) 1794 1795± 2 ^ E - 1023 x 1.frac 1796 1797That means there are 2^64 doubles. 1798 1799The biggest one is about 2^1023 1800 1801### weird double arithmetic 1802 18032^52 + 0.2 = 2^52 (the next number after 2^52 is 2^52 + 1) 1804 1805### doubles get farther apart as they get bigger 1806betweeen 2^n and 2^n+1 there are always 2^52 doubles, evenly spaced 1807 1808that means that the next double after 2^60 is 2^60 =64 (2^60 / 2^52) 1809 1810### Javascript *only* has doubles (no integers!) 1811 1812> 2**53 18139007199254740992 1814 1815> 2**53+1 18169007199254740992 (same number! uh oh!) 1817 1818### panel 5 1819 1820person with short spiky hair, baffled: doubles are scary and their arithmetic is weird! 1821 1822person with short curly hair, calm and reassuring: they're very logical! just understand how they work and don't use integers over 2^53 in Javascript <3</div> 1823 </a> 1824 1825 <a href="/comics/cgroups/">
1826 <span class="title"> 1827 cgroups 1828 </span> 1829 <div class="keywords">[containers]</div> 1830 <div class="transcript">### processes can use a lot of memory 1831process 1: I want 10 GB of memory 1832process 2: me too! 1833Linux: guys, I only have 16 GB total 1834 1835### a cgroup is a group of processes 1836every process in a container is in the same cgroup 1837 1838### cgroups have memory/CPU limits 1839Linux: you three get 500 MB of RAM to share, okay? 1840 1841### use too much memory:| get OOM ("out of memory") killed 1842process: I want 1 GB of memory 1843Linux: NOPE your limit was 500 MB you die now! 1844process, dead: oh no 1845 1846### use too much CPU: get slowed down 1847process: I want to use ALL THE CPU! 1848Linux: you hit your quota for this 100ms period, you'll have to wait 1849 1850### cgroups track memory & CPU usage 1851Linux: that cgroup is using 112.3 MB of memory right now 1852 1853you can see it in `/sys/fs/cgroup`</div> 1854 </a> 1855 1856 <a href="/comics/shared-libraries/"> 1857 <span class="title"> 1858 shared libraries 1859 </span> 1860 <div class="keywords"></div> 1861 <div class="transcript">### panel 1: 1862Most programs on Linux use a bunch of C libraries. 1863 1864Some popular libraries: 1865 1866openssl (for SSL!) 1867sqlite (embedded db?) 1868zlib (gzip!) 1869lib pcre (regular expressions!) 1870libstdc++ (C++ standard library!) 1871 1872### panel 2: 1873There are 2 ways. to use any library: 18741. Link it into your binary 1875your code (big binary with lots of things!) | zlib | sqlite 1876 and 18772. Use separate shared libraries 1878your code 1879zlib 1880sqlite 1881(all different files) 1882 1883### panel 3: 1884Programs like this 1885your code | zlib | sqlite 1886are called "statically linked" 1887 1888programs like this 1889your code 1890zlib 1891sqlite 1892are called "dynamically linked" 1893 1894### panel 4: 1895person 1: how can I tell what shared libraries a Program is using? 1896person 2: Idd!! 1897 1898```$ Idd /usr/bin/curl 1899libz.so.1 => /lib/x86-64... 1900lib resolv.so.2 =>.... 1901libc.so.6 =>... 1902``` 1903+34 more 1904 1905### panel 5: 1906person 1: I got a "library not found" error when running my binary?! 1907 1908person 2: If you know where the library is, try setting the ```LD_LIBRARY_PATH``` environment variable 1909 1910dynamic linker: ```LO-LIBRARY_PATH``` tells me where to look! 1911 1912 1913### panel 6: Where the dynamic linker looks 19141. ```ODT. RPATH``` in your executable 19152. ```LD- LIBRARY_PATH``` 19163. ```DT- RUNPATH``` in executable 19174. ```/etc/ld.so.cache.``` 1918(run ```Idconfig -p``` to See contents) 19195. ```/lib, /usr/lib```</div> 1920 </a> 1921 1922 <a href="/comics/take-on-hard-projects/"> 1923 <span class="title"> 1924 take on hard projects 1925 </span> 1926 <div class="keywords"></div> 1927 <div class="transcript">To wrap up, let's talk about one last wizard skill: confidence. When there's a hard project, sometimes I think: 1928maybe someone better than me should work on this? 1929 1930and I imagine this magical human: 1931- codes really fast 1932- knows everything about every technology 1933- understands the business well 1934- great communicator 1935- has time for the project 1936- 20 years of experience 1937 1938 1939But in programming: 1940- we're changing the tech we use all the time. 1941- every project is different, and it's rarely obvious how to do it. 1942- there aren't many experts, and they certainly don't have time to do everything. 1943 1944So instead, we have me: 1945- learns fast 1946- works hard 1947- 6 years of experience 1948- good at debugging 1949 1950I figure "someone's gotta do this' write down a plan, and get started! A lot of the time, it turns out well. I learn something and feel a little more like a WIZARD.</div> 1951 </a> 1952 1953 <a href="/comics/anatomy-http-response/"> 1954 <span class="title"> 1955 anatomy of a HTTP response 1956 </span> 1957 <div class="keywords"></div> 1958 <div class="transcript">### HTTP responses have: 1959- a status code (200 OK! 404 not found!) 1960- headers 1961- a body (HTML, an image, JSON, etc) 1962 1963### Here's the HTTP response from `examplecat.com/cat.txt`: 1964``` 1965HTTP/1.1 200 OK 1966status 1967Accept-Ranges: bytes 1968Cache-Control: public, max-age=0 1969Content-Length: 33 1970Content-Type: text/plain; charset=UTF-8 Date: Mon, 09 Sep 2019 01:57:35 GMT 1971Etag: "ac5affa59f554a1440043537ae973790-ssl" 1972Strict-Transport-Security: max-age=31536000 1973Age: 0 1974Server: Netlify 1975[ASCII image of a cat, labelled "cat!" with a smiley face] 1976``` 1977The first line, `HTTP/1.1 200 OK` is the status code. "200" is the status. 1978 1979The lines from `Accept-Ranges` to `Server` are the headers. 1980 1981The cat picture is the body. 1982 1983### There are a few kinds of response headers: 1984 1985- when the resource was sent/modified: 1986``` 1987Date: Mon, 09 Sep 2019 01:57:35 GMT 1988Last-Modified: 3 Feb 2017 13:00:00 GMT 1989``` 1990 1991- about the response body: 1992``` 1993Content-Language: en-US 1994Content-Length: 33 1995Content-Type: text/plain; charset=UTF-8 1996Content-Encoding: gzip 1997``` 1998 1999- caching: 2000``` 2001ETag: "ac5affa..." 2002Vary: Accept-Encoding 2003Age: 255 2004Cache-Control: public, max-age=0 2005``` 2006 2007- security: (see page 25) 2008``` 2009X-Frame-Options: DENY 2010X-XSS-Protection: 1 2011Strict-Transport-Security: max-age=31536000 2012Content-Security-Policy: default-src https: 2013``` 2014 2015- and more: 2016``` 2017Connection: keep-alive 2018Accept-Ranges: bytes 2019Via: nginx 2020Set-Cookie: cat-darcy; HttpOnly; expires=27-Feb-2020 13:18:57 GMT; 2021```</div> 2022 </a> 2023 2024 <a href="/comics/anatomy-http-request/">
2025 <span class="title"> 2026 anatomy of a http request 2027 </span> 2028 <div class="keywords"></div> 2029 <div class="transcript">HTTP requests always have: 2030- a domain (like `examplecat.com`) 2031- a resource (like `/cat.png`) 2032- a method (`GET`, `POST`, or something else) 2033- headers (extra information for the server) 2034 2035There's an optional request body. `GET` requests usually don't have a body, and `POST` requests usually do. 2036 2037This is an HTTP 1.1 request for `examplecat.com/cat.png`. It's a `GET` request, which is what happens when you type a URL in your browser. It doesn't have a body. 2038``` 2039GET /cat.png HTTP/1.1 2040Host: examplecat.com 2041User-Agent: Mozilla... 2042Cookie: ..... 2043``` 2044`GET` = method (usually GET or POST) 2045`/cat.png` = resource being requested 2046`HTTP/1.1` = HTTP version 2047`examplecat.com` = domain being requested, header 2048`User-Agent: Mozilla`... = header 2049`Cookie: .....` = header 2050 2051Here's an example POST request with a JSON body: 2052``` 2053POST /add_cat HTTP/1.1 2054Host: examplecat.com 2055content type of body 2056Content-Type: application/json 2057Content-Length: 20 2058``` 2059{"name": "mr darcy"} 2060 2061`POST` = method 2062`Host: examplecat.com` = header 2063`Content-Type: application/json` = content type of body, header 2064`Content-Length: 20` = header 2065 2066`{"name": "mr darcy"}` = request body: the JSON we're the server sending to</div> 2067 </a> 2068 2069 <a href="/comics/why-containers/"> 2070 <span class="title"> 2071 why containers? 2072 </span> 2073 <div class="keywords"></div> 2074 <div class="transcript">### there's a lot of container hype 2075 2076Illustration of two stick figures with medium-length straight hair. One has hearts in their eyes and a speech bubble that says "containers" with hearts around it, the other one has a thought bubble that says "???" 2077 2078Here are 2 problems they solve... 2079 2080### problem: building software is annoying 2081``` 2082$ ./configure 2083$ make all 2084ERROR: you have version 2085 2.1.1 and you need 2086 at least 2.2.4 2087``` 2088 2089### solution: package all dependencies in a * container * 2090stick figure with medium length straight hair and bangs, thinking: I ran the container and the build worked RIGHT AWAY?? is that allowed?? 2091 2092Many CI systems use containers. 2093 2094### containers have their own filesystem 2095 2096This is the big reason containers are great. 2097 2098host OS, represented by a box with a smiley face: I'm running Ubuntu 19.04 2099 2100container, also represented by a box with a smiley face: I'm running an old CentOS distribution host OS from 2014! 2101 2102### problem: deploying software is annoying too 2103sad stick figure with short curly hair: ugh my website is broken because I used a Python 3.6 feature and the server only has Python 3.5 2104 2105### solution: deploy a container 2106server: I have the exact same version of everything as in development! no more silly errors! 2107 2108happy stick person with short curly hair: yay! I can get back to writing code!</div> 2109 </a> 2110 2111 <a href="/comics/dig/"> 2112 <span class="title"> 2113 dig 2114 </span> 2115 <div class="keywords"></div> 2116 <div class="transcript">### dig makes DNS queries! 2117``` 2118$ dig google.com 2119``` 2120answers have 5 parts: 2121- query: `google.com` 2122- TTL `22` 2123- clas: `IN` (for "internet", ignore this) 2124- record type: `A` 2125- record value: `172.217.13.110` 2126 2127### dig TYPE domain.com 2128this lets you choose which DNS record to query for! 2129 2130types to try: 2131- NS 2132- MX 2133- TXT 2134- CNAME 2135- A (default) 2136 2137### dig @ 8.8.8.8 domain 2138(google DNS server) 2139 2140dig@server lets you pick which DNS server to query. Useful when your system DNS is misbehaving :) 2141 2142### dig + trace domain 2143traces how the domain gets resolved, starting at the root nameservers. 2144 2145if you just updated DNS, dig + trace should show the new record. 2146 2147### dig -x 172.217.13.174 2148makes a reverse DNS query -find which domain resolves to an IP! Same as `dig ptr 172.217.13.174.in.addr.arpa` 2149 2150### dig +short domain 2151Usually dig pints lots of output! With +short it just prints the DNS record.</div> 2152 </a> 2153 2154 <a href="/comics/ps/">
2155 <span class="title"> 2156 ps 2157 </span> 2158 <div class="keywords"></div> 2159 <div class="transcript"></div> 2160 </a> 2161 2162 <a href="/comics/joins/"> 2163 <span class="title"> 2164 my rules for simple JOINs 2165 </span> 2166 <div class="keywords"></div> 2167 <div class="transcript">my rules for simple `JOIN`s 2168 2169Joins in SQL let you take 2 tables and combine them into one. 2170 2171On the left side of the page, there is an illustration of two small tables with the words "INNER JOIN" between them. One has columns labelled a, b, c, and d, the other has columns labelled x, y, and z. On the right side of the page, there is a big table with columns a, b, c, d, x, y, and z. 2172 2173Joins can get really complicated, so we'll start with the simplest way to join. Here are the rules I use for 90% of my joins: 2174 2175#### Rule I: only use LEFT JOIN and INNER JOIN 2176There are other kinds of joins (`RIGHT JOIN`, `CROSS JOIN`, `FULL OUTER JOIN`), but of the time I only use `LEFT JOIN` and `INNER JOIN`. 2177 2178#### Rule 2: refer to columns as `table_name.column_name` 2179You can leave out the table name if there's just one column with that name, but it can get confusing. 2180 2181#### Rule 3: Only include 1 condition in your join 2182Here's the syntax for a LEFT JOIN: 2183``` 2184table1 LEFT JOIN table2 ON <any boolean condition> 2185``` 2186I usually stick to a very simple condition, like this: 2187``` 2188table1 LEFT JOIN table2 2189 ON table1.some_column = table2.other_column 2190``` 2191#### Rule 4: One of the joined columns should have unique values 2192If neither of the columns is unique, you'll get strange results like this: 2193 2194owners_bad: 2195 2196| name | age | 2197|-------|-----| 2198| maher | 16 | 2199| maher | 32 | 2200| rishi | 21 | 2201 2202`INNER JOIN` 2203 2204cats_bad: 2205 2206| name | age | 2207|-------|------------| 2208| maher | daisy | 2209| maher | dragonsnap | 2210| rishi | buttercup | 2211 2212(these are "bad" versions of the "owners" and "cats" tables that don't `JOIN` well) 2213 2214``` 2215owners_bad INNER JOIN cats_bad 2216 ON owners_bad.name = cats_bad.owner 2217``` 2218 2219| name | name | age | 2220|-------|------------|-----| 2221| maher | daisy | 16 | 2222| maher | dragonsnap | 16 | 2223| maher | daisy | 32 | 2224| maher | dragonsnap | 32 | 2225| rishi | buttercup | 21 |</div> 2226 </a> 2227 2228 <a href="/comics/status-codes/"> 2229 <span class="title"> 2230 http status codes 2231 </span> 2232 <div class="keywords"></div> 2233 <div class="transcript">Every HTTP response has a status code. 2234browser, optimistically: `GET /cat.png` (request) 2235server, sadly: 404 not found (404 is the status code!) 2236 2237There are 50ish status codes but these are the most common ones in real life: 2238 22392xxs mean â Successâ 2240- 200 OK 2241 22423xx s aren't errors, just redirects to somewhere else 2243- 301 Moved Permanently 2244- 302 Found: temporary redirect 2245- 304 Not Modified: the client already has the latest version, "redirect" to that 2246 22474xx errors are generally the client's fault: it made some kind of invalid request 2248- 400 Bad Request 2249- 403 Forbidden: API key/OAuth/something needed 2250- 404 Not Found: we all know this one :) 2251- 429 Too Many Requests: you're being rate limited 2252 22535xx errors generally mean something's wrong with the server. 2254- 500 Internal Server Error: the server code has an error 2255- 503 Service Unavailable: could mean nginx (or whatever proxy) couldn't connect to the server 2256- 504 Gateway Timeout: the server was too slow to respond</div> 2257 </a> 2258 2259 <a href="/comics/sort-uniq/"> 2260 <span class="title"> 2261 sort & uniq 2262 </span> 2263 <div class="keywords"></div> 2264 <div class="transcript">### `sort` sorts its inputs 2265`$ sort name.txt` 2266 2267the default sort is alphabetical 2268 2269### `sort -n`: numeric sort 2270`sort order` (sad face): 2271- 12 2272- 15000 2273- 48 2274- 6020 2275- 96 2276 2277`sort -n` order (happy face): 2278- 12 2279- 48 2280- 96 2281- 6020 2282- 15000 2283 2284### `sort -h`: human sort 2285`sort -n` order (sad face): 2286- 15 G 2287- 30 M 2288- 45 K 2289- 200 G 2290 2291`sort -h` order (happy face): 2292- 45 K 2293- 30 M 2294- 15 G 2295- 200 G 2296 2297useful example: `du -sh * | sort -h` 2298 2299### `uniq` removes duplicates 2300before: 2301- a 2302- b 2303- b 2304- a 2305- c 2306- c 2307 2308after: 2309- a 2310- b 2311- a 2312- c 2313 2314(notice there are still 2 'a's! uniq only uniquifies adjacent matching lines 2315 2316### `sort` + `uniq` = (heart)
2317Pipe something to `sort | uniq` and you'll get a deduplicated list of lines! `sort -u` does the same thing. 2318 2319before `sort -u` (or `sort | uniq`): 2320- b 2321- a 2322- b 2323- a 2324 2325after: 2326- a 2327- b 2328 2329### `uniq -c` 2330counts each line it saw. 2331 2332Recipe: get the top 10 most common lines in a file: 2333``` 2334$ sort foo.txt 2335 | uniq -c 2336 | sort -n 2337 | tail -n 10 2338``` 2339 2340happy little stick figure with curly hair: I use this a lot!</div> 2341 </a> 2342 2343 <a href="/comics/indexes/"> 2344 <span class="title"> 2345 how indexes make your queries fast 2346 </span> 2347 <div class="keywords"></div> 2348 <div class="transcript">By default, if you run `SELECT * FROM cats WHERE name = 'mr darcy'` the database needs to look at every single row to find matches. 2349 2350database, sad: reading 30 GB of data from disk takes like 60 seconds by itself, you know! 2351(at 500 MB/s SSD speed) 2352 2353Indexes are a tree structure that makes it faster to find rows. Here's what an index on the 'name' column might look like. 2354 2355a-z 2356aaron to ahmed 2357aaron to abdullah 2358agnes to ahmed 2359molly to nasir 236060 children 2361waseem to zahra 2362 2363database indexes are b-trees and the nodes have lots of children (like 60) instead of just 2. 2364 2365log <sub>60</sub> (1,000,000,000) = 5.06 2366 2367 2368This means that if you have 1 billion names to look through, you'll only need to look at maybe nodes in the index to find the name you're looking for (5 is a lot less than 1 billion!!!). 2369 2370person 1: are you saying indexes can make my queries 1,000,000x faster? 2371person 2: yes! actually some queries. on large tables are basically impossible (or would take weeks) without using an index!</div> 2372 </a> 2373 2374 <a href="/comics/ss/"> 2375 <span class="title"> 2376 ss 2377 </span> 2378 <div class="keywords"></div> 2379 <div class="transcript">### panel 1 2380 2381two stick figures talking. the first one is bald and looks unhappy. the second one has short curly hair and is smiling. 2382 2383person 1: I can't start my server because it says something is using port 8080! 2384 2385person 2: 23861. Use ss ("socket statistics") to find the process ID using the port 23872. Kill the other process! 2388 2389 2390### * tuna, please! * 2391`$ ss -tunapl` 2392(the 'a' here doesn't do anything) 2393 2394This is my favourite way to use ss! It shows all the running servers. 2395 2396### -n 2397use numeric ports (80 not http) 2398 2399### -P 2400show PIDs using the socket 2401 2402### TONS of information 2403-i -m -o 2404(-i is in a spiky bubble, -m is in a cloud bubble, and -o is in a heart) 2405 2406### which sockets ss shows 2407listening or connections (non-listening/established)? 2408 2409default: connections 2410-1: listening 2411-a: both 2412 2413which protocols? 2414 2415default: all 2416-t: TCP 2417-u: UDP 2418-X: unix domain Sockets 2419 2420### netstat 2421netstat -tunapl and ss -tunap! do the same thing 2422 2423netstat is older and more complicated. If you're learning now I'd recommend ss!</div> 2424 </a> 2425 2426 <a href="/comics/nmap/"> 2427 <span class="title"> 2428 nmap 2429 </span> 2430 <div class="keywords"></div> 2431 <div class="transcript">### nmap lets you explore a network 2432which ports are open? 2433what hosts are up? 2434security people use it a lot! 2435 2436### find which hosts are up 2437`$ nmap-sn 192.168.1.0/24` 2438`168` is my home network 2439`-sn` means "ping scan". (not `-s-n` it's `-sn`) 2440 2441just finds hosts by pinging every one, doesn't port scan 2442 2443### aggressive scan 2444`nmap -v -A scanme.nmap.org` 2445`-A` = aggressive 2446port, server version, even OS 2447 2448### -Pn 2449skip doing a ping scan and assume every host is up. good if hosts block ping (lots do) 2450 2451### fast port scan 2452`$ nmap -SS-F 192.168.1.0/24` 2453 2454just sends a SYN packet to check if each port is open. 2455 2456I found out which ports my printer has open! 2457``` 245880 http 2459443 https 2460515 printer 2461631 ipp 24629100 jetdirect 2463``` 2464### -F 2465scan less ports: just the most common ones 2466 2467### -T4 or -T5 2468scan faster by timing out more quickly 2469 2470### â¡ check TLS version and ciphers â¡ 2471check if your server still supports old TLS versions 2472``` 2473$ nmap 2474--script ssl-enum-ciphers 2475-p 443 wizardzines.com 2476``` 2477list all scripts with: 2478`$ nmap --script-help '*'`</div> 2479 </a> 2480 2481 <a href="/comics/lsof/">
2482 <span class="title"> 2483 lsof 2484 </span> 2485 <div class="keywords"></div> 2486 <div class="transcript">### `lsof` 2487stands for list open files 2488 2489stick figure, distraught: somebody has that file open, WHO IS IT? 2490`lsof`, represented by a rectangle with a goofy face: I can tell you! 2491 2492### what `lsof` tells you 2493for each open file: 2494- `pid` 2495- file type (regular? directory? FIFO? socket?) 2496- file descriptor (FD column) 2497- user 2498- filename/socket address 2499 2500### `-p PID` 2501list the files `PID` has open 2502 2503### `lof /some /dir` 2504list just the open files in `some/dir` 2505 2506### `-i` 2507list open network sockets (sockets are files!) 2508 2509examples: 2510- `-i -n P` (`-n` & `-P` mean "don't resolve host names/ports" [also `-Pni`])" 2511- `-i : 8080` 2512- `-i TCP` 2513- `-i -s TCP:LISTEN` 2514 2515### find deleted files 2516`$lsof | grep deleted` 2517 2518will show you deleted files! 2519 2520You can recover open deleted files from 2521`proc/<pid>/fd/<fd>` 2522(`<pid>` is the process that opened the file) 2523 2524### `netstat` 2525another way to list open sockets on Linux is: 2526 2527`netstat -tunapl` 2528(tuna, please!) 2529 2530On Mac, `netstat` has different args.</div> 2531 </a> 2532 2533 <a href="/comics/man-pages/"> 2534 <span class="title"> 2535 man page sections 2536 </span> 2537 <div class="keywords"></div> 2538 <div class="transcript">man pages are split up into 8 sections 25391 2 3 4 5 6 7 8 2540 2541`$ man 2 read` 2542means "get me the man page for `read` from section 2". 2543 2544There's both 2545- a program called "read" 2546- and a system call called "read" 2547 2548So 2549 2550`$ man 1 read` 2551gives you a different man page from 2552`$ man 2 read` 2553 2554If you don't specify a section, man will look through all the sections & show the first one it finds. 2555 2556### man page sections 25571. programs 2558`$ man grep ` 2559`$ man ls` 2560 25612. system calls 2562`$ man sendfile 2563`$ man ptrace 2564 25653. C functions 2566`$man printf 2567`$ man fopen 2568 25694. devices 2570`$ man null` for /dev/null docs 2571 25725. file formats 2573`$ man sudoers` for `/etc/sudoers` 2574`$ man proc` files in `/proc`! 2575 25766. games 2577not super useful. `$man sl` is my favourite from that section 2578 25797. miscellaneous 2580explains concepts! 2581`$man 7 pipe` 2582`$ man 7 symlink` 2583 25848. sysadmin programs 2585`$ man apt` 2586`$ man chroot`</div> 2587 </a> 2588 2589 <a href="/comics/selectors/"> 2590 <span class="title"> 2591 CSS selectors 2592 </span> 2593 <div class="keywords"></div> 2594 <div class="transcript">### panel 1 2595 2596Illustration of a smiling stick figure with curly hair. 2597 2598person: now that we have the right attitude, let's move on to how CSS actually works! 2599 2600### div 2601matches `div` elements 2602`<div>` 2603 2604### #welcome 2605`#` matches elements by `id` 2606`<div id="welcome">` 2607 2608### .button 2609matches elements by `class` 2610`<a class="button">` 2611 2612### div .button 2613match every `.button` element that's a descendent of a `div` 2614 2615### div.button 2616match divs with class "`button`" 2617`<div class="button">` 2618 2619### div > .button 2620match every `.button` element that's a direct child of a `div` 2621 2622### .button, #welcome 2623matches both `button` and `#welcome` elements 2624 2625### a[href^="http"] 2626match `a` elements with a `href` attribute starting with `http` 2627 2628### a:hover 2629matches `a` elements that the cursor is hovering over 2630 2631### :checked 2632matches if a checkbox or radio button is checked 2633 2634### tr:nth-child(odd) 2635match every other child of a parent element</div> 2636 </a> 2637 2638 <a href="/comics/cookies/"> 2639 <span class="title"> 2640 cookies 2641 </span> 2642 <div class="keywords"></div> 2643 <div class="transcript">Cookies are a way for a server to store a little bit of information in your browser. 2644 2645They're set with the `Set-Cookie` response header, like this: 2646 2647### first request: server sets a cookie 2648 2649browser, represented by a box with a smiley face: `GET /my-cats` 2650 2651server, also represented by a box with a smiley face: 2652``` 2653200 OK 2654Set-Cookie: user = b0rk; HttpOnly 2655<response body> 2656``` 2657(`user` is the name, `b0rk` is the value. `HttpOnly` is the cookie options (expiry goes here)) 2658 2659### Every request after: browser sends the cookie back 2660browser: 2661``` 2662GET /my-cats 2663Cookie: user= b0rk 2664``` 2665server, thinking: oh, this is b0rk! I don't need to ask them who they are then! 2666 2667 2668Cookies are used by many websites to keep you logged in. Instead of `user=b0rk` they'
2668ll set a cookie like `sessionid=long-incomprehensible-id`. This is important because if they just set a simple cookie like `user=b0rk`, anyone could pretend to be b0rk by setting that cookie! 2669 2670Designing a secure login system with cookies is quite difficultâ to learn more about it, google "OWASP Session Management Cheat Sheet".</div> 2671 </a> 2672 2673 <a href="/comics/testing-checklist/"> 2674 <span class="title"> 2675 CSS testing checklist 2676 </span> 2677 <div class="keywords"></div> 2678 <div class="transcript">Finally, it's important to test your site with different browsers, screen sizes, and accessibility evaluation tools. 2679 2680### browsers 2681- Chrome 2682- Safari 2683- Firefox 2684- maybe others! 2685 2686### sizes 2687- small phone (300px wide) 2688- tablet (~700px) 2689- desktop (~1200px) 2690 2691### accessibility 2692- colour contrast 2693- text size 2694- keyboard navigation 2695- works with a screen reader 2696 2697### performance 2698- fake a slow/high latency network connection! 2699 2700Illustration of a smiling stick figure with curly hair. 2701 2702person: the most important thing is to know your users! Check your analytics: if 10% of your users are using IE, test your site on IE!</div> 2703 </a> 2704 2705 <a href="/comics/copy-on-write/"> 2706 <span class="title"> 2707 copy on write 2708 </span> 2709 <div class="keywords"></div> 2710 <div class="transcript">### On Linux, you start new processes using the fork() or clone() system call. 2711 2712calling fork creates a child process that's a copy of the caller 2713 2714### the cloned process has EXACTLY the same memory. 2715- same heap 2716- same stack 2717- same memory maps 2718 2719if the parent has 36B of memory, the child will too. 2720 2721### copying all that memory every time we fork would be slow and a waste of RAM 2722 2723often processes call `exec` right after `fork`, which means they don't use the parent process's memory basically at all! 2724 2725### so Linux lets them share physical RAM and only copies the memory when one of them tries to write 2726process: I'd like to change that memory 2727Linux: okay! I'll make you your own copy! 2728 2729### Linux does this by giving both the processes identical page tables. 2730(same RAM) 2731but it marks every page as read only. 2732 2733### when a process tries to write to a shared memory address: 27341. there's a page fault= 27352. Linux makes a copy of the page & updates the page table 27363. the process continues, blissfully ignorant 2737 2738process, happily: It's just like I have my own copy</div> 2739 </a> 2740 2741 <a href="/comics/oom-killer/"> 2742 <span class="title"> 2743 the OOM killer 2744 </span> 2745 <div class="keywords">[linux2]</div> 2746 <div class="transcript"></div> 2747 </a> 2748 2749 <a href="/comics/cpu-scheduling/"> 2750 <span class="title"> 2751 CPU scheduling 2752 </span> 2753 <div class="keywords">[linux2]</div> 2754 <div class="transcript"></div> 2755 </a> 2756 2757 <a href="/comics/http2/"> 2758 <span class="title"> 2759 HTTP/2 2760 </span> 2761 <div class="keywords"></div> 2762 <div class="transcript">HTTP/2 is a new version of HTTP. Here's what you need to know: 2763 2764### A lot isn't changing 2765All the methods, status codes, request/response bodies, and headers mean exactly the same thing in HTTP/2. 2766 2767before (HTTP/1.1): 2768``` 2769method: GET 2770path: /cat.gif 2771headers: 2772- Host: examplecat.com 2773- User-Agent: curl 2774``` 2775 2776after (HTTP/2): 2777``` 2778method: GET 2779path: /cat.gif 2780authority: examplecat.com 2781headers: 2782- User-Agent: curl 2783``` 2784 2785one change: 2786Host header => authority 2787 2788#### HTTP/2 is faster 2789Even though the data sent is the same, the way HTTP/2 sends it is different. The main differences are: 2790- It's a binary format (it's harder to ```tcpdump``` traffic and debug) 2791- Headers are compressed 2792- Multiple requests can be sent on the same connection at a time 2793 2794before (HTTP/1.1): 2795â request 1 2796response 1 â 2797â request 2 2798response 2 â 2799 2800after (HTTP/2): 2801â request 1 2802â request 2 2803response 2 â 2804response 1 â (out of order is ok) 2805(one TCP connection) 2806 2807All these changes together mean that HTTP/2 requests often take less time than the same HTTP/1.1 requests. 2808 2809### Sometimes you can switch to it easily 2810A lot of software (CDNs, nginx) let clients connect with HTTP/2 even if your server still only supports HTTP/1.1. 2811 28121. Firefox to CDN: HTTP/2 request 28132. CDN to your server: HTTP/1.1 request 28143. your server to CDN: HTTP/1.1 response 28154. CDN to Firefox: HTTP/2 response</div> 2816 </a> 2817 2818 <a href="/comics/tar/">
2819 <span class="title"> 2820 tar 2821 </span> 2822 <div class="keywords"></div> 2823 <div class="transcript">### panel 1 2824The tar file format combines many files into one file. 2825 2826a.txt 2827b.txt 2828dir/c.txt 2829 2830tar files aren't compressed by themselves. Usually you gzip them: .tar.gz or .tgz! 2831 2832### panel 2: 2833Usually when you use the 'tar' command, you'll run some incantation. 2834To unpack a tar.gz, use: 2835```tar -xzf file.tar.gz`` 2836 2837person 1: what's xzf? 2838person 2: let's learn! 2839 2840 2841### panel 3: -X is for extract 2842into the current directory by default (change with -C) 2843 2844### panel 4: -C is for create 2845makes a new tar file! 2846 2847### panel 5: -t is for list 2848lists the contents of a tar archive 2849 2850### panel 6: -f is for file 2851which tar file to create or unpack 2852 2853### panel 7: tar can compress / decompress 2854 2855-z gzip format (.gz) 2856-j bzip2 format (.bz2) 2857-J x2 format (.xz) 2858& more! see the man page 2859 2860### panel 8: putting it together 2861list contents of a .tar.bz2: 2862```$tar tvf file.tar.bz2 ``` 2863j = verbose 2864 2865create a .tar.gz: 2866```$ tar -c2f file.tar.gz dir/``` 2867dir/ = files to go in the archive</div> 2868 </a> 2869 2870 <a href="/comics/unix-domain-sockets/"> 2871 <span class="title"> 2872 unix domain sockets 2873 </span> 2874 <div class="keywords"></div> 2875 <div class="transcript">### unix domain sockets are files 2876 2877 2878``` 2879$ file mysock.sock 2880socket 2881``` 2882the file's permissions determine who can send data to the socket. 2883 2884### they let 2 programs on the same computer communicate 2885 2886 2887Docker uses Unix domain sockets, for example! 2888 2889process: GET/container (HTTP request) 2890Docker: Here you go! 2891 2892### There are 2 kind of unix domain sockets 2893 2894- `stream`: Like TCP! Lets you send a continuous stream of bytes 2895- `datagram`: Like UDP! Let you send discrete chunks of data 2896 2897### advantage 1 2898Lets you use file permission to restrict access to HTTP/database services! 2899 2900`chmod 600 secret.sock` 2901 2902This is why Docker uses a unix domain socket. (lock icon) 2903 2904evil process: run evil container 2905Linux, nonplussed: permission denied 2906 2907### advantage 2 2908UDP sockets aren't always reliable (even on the same compute). 2909 2910unix domain datagram sockets ARE reliable! And they won't reorder packets! 2911 2912Process: I can send data and I KNOW it'll arrive 2913 2914### advantage 3 2915You can send a file descriptor over a unix domain socket. Useful when handling untrusted input files. 2916 2917process: here's a file I downloaded from sketchy.com (putting it into video decoder, a sandboxed process)</div> 2918 </a> 2919 2920 <a href="/comics/iptables/"> 2921 <span class="title"> 2922 iptables 2923 </span> 2924 <div class="keywords"></div> 2925 <div class="transcript">### panel 1: 2926iptables lets you create rules to match network packets and accept/drop/modify them 2927 2928It's used for firewalls and NAT 2929 2930### tables have chains. chains have rules. 2931tables: 2932- `filter` 2933- `nat` 2934- `mangle` 2935- `raw` 2936- `security` 2937 2938chains: 2939- `INPUT` 2940- `FORWARD` 2941- `PREROUTING` 2942- etc 2943 2944rules: like `-s 10.0.0.0/8 -j DROP` 2945 2946### `iptables-save` 2947This prints out all iptables rules. You can restore them with `iptables-restore` but it's also the easiest way to view all rules. 2948 2949### `-j TARGET` 2950Every iptables rule has a target (what to do with matching packets). Options: 2951- `ACCEPT, DROP, RETURN` 2952- the name of an iptables chain 2953- an extension (man iptables.extensions) 2954Popular: `DNAT, LOG, MASQUERADE` 2955 2956### tables have different chains 2957filter: `INPUT, OUTPUT, FORWARD` 2958mangle: `INPUT, OUTPUT, FORWARD, PREROUTING, POSTROUTING` 2959nat: `OUTPUT, PREROUTING, POSTROUTING` 2960 2961It helps to know when packets get processed by a given table/chain (eg locally generated packets go through `FILTER` and `OUTPUT` 2962 2963### you can match lots of packet attributes 2964- `-s`: src ip 2965- `-d`: tcp/udp 2966- `-p`: dst ip 2967- `-i`: network interface 2968- `-m`: lots of things! (bpf rules! gcroups! ICMP type! cp! conntrack state! more! 2969 2970For more, run `$ man iptables-extensions`</div> 2971 </a> 2972 2973 <a href="/comics/tshark/"> 2974 <span class="title"> 2975 tshark 2976 </span> 2977 <div class="keywords">[tcpdump]</div> 2978 <div class="transcript">### Wireshark is an amazing graphical packet analysis tool 2979("Wireshark" has hearts around it) 2980 2981tshark is the command line version of Wireshark 2982 2983it can do 100x more things than tcpdump (heart) 2984 2985### `-Y` 2986filter which packets are captured 2987 2988``` 2989tshark -y 2990'http.request.method 2991== "GET" 2992``` 2993(uses Wireshark's SUPER POWERFUL filter language) 2994 2995### `-d` 2996is for "decode as" 2997 2998tells tshark what protocol to interpret a port as 2999 3000Example: 8888 is often HTTP! 3001 3002``` 3003$tshark 3004-d tcp.port==8888,http 3005``` 3006 3007### `-T FORMAT` 3008Output format. My favourites: 3009- json 3010- fields: csv/tsv 3011(for these above two you can specify which fields you want with `-e`) 3012- text: default summary 3013 3014### `-e` 3015Which fields to output. Ex: 3016``` 3017$tshark -T fields 3018-e http.request.method 3019-e http.request.uri 3020-e ip.dst 3021``` 3022(supports WAY more protocols than HTTP) 3023``` 3024GET /foo 92.183.216.34 3025POST /bar 10.23.38.132 3026``` 3027 3028### `-r file.pcap` 3029analyze packets from a file instead of the network 3030 3031### `-w` 3032(same as tcpdump) 3033 3034Write captured packets to a file. If `-w file.pcap` has permission issues, tr
3034y `tshark -w - > file.pcap`</div> 3035 </a> 3036 3037 <a href="/comics/explain/"> 3038 <span class="title"> 3039 EXPLAIN 3040 </span> 3041 <div class="keywords"></div> 3042 <div class="transcript">Sometimes queries run slowly, and `EXPLAIN` can tell you why! 3043 30442 ways you can use `EXPLAIN` in PostgreSQL: (other databases have different syntax for this) 3045 30461. Before running the query (`EXPLAIN SELECT... FROM ...`) 3047This calculates a query plan but doesn't run the query. 3048 3049I always run EXPLAIN on a query. before running it on my production database. I won't risk overloading the database with a slow query! 3050 30512. After running the query `(EXPLAIN ANALYZE SELECT ... FROM...)` 3052person 1: why is my query so slow? 3053person 2: `EXPLAIN ANALYZE` runs the query and analyzes why it was slow 3054 3055Here are the EXPLAIN ANALYZE results from PostgreSQL for the same query run on two tables of 1,000,000 rows: one table that has an index and one that doesn't 3056 3057`EXPLAIN ANALYZE SELECT * FROM users WHERE id = 1` 3058 3059unindexed table: 3060``` 3061Seq Scan on users 3062Filter: (id = 1) 3063Rows Removed by Filter: 999999 Planning time: 0.185 ms 3064Execution time: 179.412 ms 3065``` 3066"Seq Scan" means it's looking at each row (slow!) 3067 3068indexed table: 3069``` 3070Index Only Scan using users_id_idx on users 3071Index Cond: (id = 1) 3072Heap Fetches: 1 3073Planning time: (3.411 ms 3074Execution time: 0.088 ms 3075``` 3076the query runs 50 times faster with an index</div> 3077 </a> 3078 3079 <a href="/comics/cat/"> 3080 <span class="title"> 3081 cat 3082 </span> 3083 <div class="keywords"></div> 3084 <div class="transcript">### cat concatenates files 3085`$ cat myfile.txt` prints contents of myfile.txt| 3086 3087`$ cat *.txt` prints all .txt files put together! 3088 3089 3090### you can use cat as an EXTREMELY BASIC text editor: 30911. Run $ cat > file.txt 30922. type the contents (don't make mistakes (smiley face)) 30933. press ctrl+d to finish 3094 3095### cat -n 3096prints out the file with line numbers! 3097 30981. Once upon a midnight.. 30992. Over many a quaint. 31003. While I nodded, nearly 3101 3102### zcat 3103cats a gzipped file! 3104 3105Actually just a 1-line shell script that runs `gzip -cd`, but easier to remember. 3106 3107### tee 3108`tee file.txt` will write. its stdin to both stdout and file.txt 3109 3110`stdin` > `tee a.txt` > `stdout` and `a.txt` 3111 3112### how to redirect to a file owned by root 3113`$ sudo echo "hi">> x.txt` 3114 3115this will open x.txt as your user, not as root, so it fails! 3116 3117`$ echo "hi" I sudo tee -a x.txt` will open x.txt as root (smiley face)</div> 3118 </a> 3119 3120 <a href="/comics/find/"> 3121 <span class="title"> 3122 find 3123 </span> 3124 <div class="keywords"></div> 3125 <div class="transcript">### find searches a directory for files 3126`find /tmp -type d -print` 3127`tmp`: directory to search 3128`-type d`: which files 3129`-print`: action to do with the files 3130 3131There are my favourite find arguments! 3132 3133### -name/-iname 3134case insensitive 3135the filename! eg 3136`-name '*.txt'` 3137 3138### -path /-ipath 3139search the full path! 3140`-path /home/*/*.go` 3141 3142### -type [TYPE] 3143f: regular file 3144d: directory 31451: symlink 3146and more! 3147 3148### -maxdepth NUM 3149only descend NUM levels when searching a directory. 3150 3151### -Size O 3152find empty files! Useful to find files you created by accident 3153 3154### -exec COMMAND 3155action: run COMMAND on every file found 3156 3157### -print0 3158print null-separated filenames 3159Use with xargs -O! 3160 3161### -delete 3162action: delete all files found 3163 3164### locate 3165The locate command searches a database of every file on your system. 3166 3167good: faster than find 3168bad: can get out of date 3169 3170### $ sudo updatedb 3171 updates the database</div> 3172 </a> 3173 3174 <a href="/comics/take-breaks/"> 3175 <span class="title"> 3176 debugging is hard. take breaks. 3177 </span> 3178 <div class="keywords">[debugging]</div> 3179 <div class="transcript"></div> 3180 </a> 3181 3182 <a href="/comics/segfaults/"> 3183 <span class="title"> 3184 segmentation faults 3185 </span> 3186 <div class="keywords">[memory linux2]</div> 3187 <div class="transcript"></div> 3188 </a> 3189 3190 <a href="/comics/css-isnt-easy/">
3191 <span class="title"> 3192 CSS isn't easy 3193 </span> 3194 <div class="keywords"></div> 3195 <div class="transcript">### CSS seems simple at first 3196``` 3197h2 { 3198font-size: 22px; 3199} 3200``` 3201Illustration of a smiling stick figure with curly hair. 3202 3203person: ok this is easy! 3204 3205### and it is easy for simple tasks 3206image of a page with header and text underneath 3207a layout like this is simple to implement! 3208 3209### but website layout is not an easy problem 3210image of a page with a logo, header, text, sidebar, and multiple images 3211this needs to adjust to so many screen sizes! 3212 3213### the spec can be surprising 3214TRY ME! 3215CSS 2.1: setting `overflow: hidden;` on an inline-block element changes its vertical alignment 3216 3217Illustration of a stick figure with curly hair, looking worried. 3218 3219person: weird! 3220 3221### and all browsers have bugs 3222Safari: I don't support flexbox for `<summary>` elements 3223person: ok fine 3224 3225### accept that writing CSS is gonna take time 3226person: if I'm patient I can fix all the edge cases in my CSS and make my site look great everywhere!</div> 3227 </a> 3228 3229 <a href="/comics/page-faults/"> 3230 <span class="title"> 3231 page faults 3232 </span> 3233 <div class="keywords"></div> 3234 <div class="transcript">### every Linux process has a page table 3235 3236`*` page table `*` 3237 3238| virtual memory address | physical memory address | 3239|--------------------------|--------------------------| 3240| 0x19723000 | 0x1422000 | 3241| 0x19724000 | 0x1423000 | 3242| 0x1524000 | not in memory | 3243| 0x1844000 | 0x4a000 read only | 3244 3245### some pages are marked as either: 3246- read only 3247- not resident in memory 3248 3249when you try to access a page that's marked "not resident in memory" it triggers a ! page fault ! 3250 3251### what happens during a page fault? 3252- the MMU sends an interrupt 3253- your program stops running 3254- Linux kernel code to handle the page fault runs 3255 3256Linux, represented by a box with a smiley face: I'll fix the problem and let your program keep running 3257 3258### "not resident in memory" usually means the data is on disk! 3259 3260virtual memory: 3261Illustration of a bar that is about 60% filled in purple, labelled "in RAM". The remaining 40% is filled in orange and labelled "on disk" 3262 3263Having some virtual memory that is actually on disk is how `swap` and `nmap` work 3264 3265### how swap works 3266 32671. run out of RAM 3268Illustration where RAM bar is completely full, disk bar still has lots of room 3269 32702. Linux saves some RAM data to disk 3271Some of the RAM bar has now been moved over to the disk bar 3272 32733. mark those pages as "not resident in memory" in the page table 3274There are arrows between the RAM and disk bars, and the empty portion of the RAM bar is labelled "not resident" 3275 32764. When a program tries to access the memory, there's a ! page fault ! 3277 32785. Linux: time to move some data back to RAM! 3279Illustration of virtual memory and RAM, with arrows running back and forth between them 3280 32816. if this happens a lot, your program gets VERY SLOW 3282program, sadly: I'm always waiting for data to be moved in & out of RAM</div> 3283 </a> 3284 3285 <a href="/comics/network-protocols/"> 3286 <span class="title"> 3287 network protocols 3288 </span> 3289 <div class="keywords">[linux2]</div> 3290 <div class="transcript"></div> 3291 </a> 3292 3293 <a href="/comics/parameter-expansion/"> 3294 <span class="title"> 3295 bash parameter expansion 3296 </span> 3297 <div class="keywords"></div> 3298 <div class="transcript">## panel 1: `${...}` is really powerful 3299 3300person: "it can do a lot of string operations, my favourite is search/replace 3301 3302## panel 2: `${var}` 3303 3304same as `$var` 3305 3306## panel 3: `${&#35;var}` 3307 3308length of the string or array `var` 3309 3310example: 3311 3312``` 3313$ x=panda 3314$ echo ${#x} 33155 3316``` 3317 3318### panel 4: `${var/bear/panda}` 3319 3320search & replace. Example: 3321 3322``` 3323$ x="I'm a bearbear! 3324$ echo ${x/bear/panda} # replace 1 instance of 'bear' 3325I'm a pandabear!
3326$ echo ${x//bear/panda} # replace every instance of 'bear' 3327I'm a pandapanda! 3328``` 3329 3330### panel 5: `${var:-othervar}` 3331 3332use a default value if `var` is unset/null 3333 3334Example: 3335 3336``` 3337echo ${asdf:-some default value} 3338``` 3339 3340### panel 6: `${var:?some error}` 3341 3342prints "some error" and exits if `var` is null or unset 3343 3344### panel 7: `${var#pattern}` and `${var%pattern}` 3345 3346remove the prefix/suffix `pattern` from `var. 3347 3348Example: 3349 3350``` 3351$ x=motorcycle.svg 3352$ echo "${x%.svg}" 3353motorcycle 3354``` 3355 3356### panel 8: `${var:offset:length}` 3357 3358get a substring of `var`. Example: 3359 3360``` 3361$ x='panda bear time' 3362$ echo ${x:6:4} 3363time 3364``` 3365 3366### panel 9 3367 3368person: "there are LOTS more, look up 'bash parameter expansion'!" 3369 3370 3371 3372 3373</div> 3374 </a> 3375 3376 <a href="/comics/coalesce/"> 3377 <span class="title"> 3378 COALESCE 3379 </span> 3380 <div class="keywords"></div> 3381 <div class="transcript">`COALESCE` is a function that returns the first argument you give it that isn't `NULL` 3382``` 3383COALESCE(NULL, 1, 2) => 1 3384COALESCE(NULL, NULL, NULL) => NULL 3385COALESCE(4, NULL, 2) => 4 3386``` 3387#### 2 ways you might want to use `COALESCE` in practice: 3388 33891. Set a default value: 3390 3391In this table, a `NULL` discount means there's no discount, so we use `COALESCE` to set the default to O: 3392``` 3393SELECT name, 3394 price - COALESCE(discount, 0) as net_price 3395FROM products 3396``` 3397 3398products: 3399 3400| name | price | discount | 3401|----------|---------|------------| 3402| orange | 200 | NULL | 3403| apple | 100 | 23 | 3404| lemon | 150 | NULL | 3405 3406 3407query output: 3408 3409| name | net_price | 3410|----------|-------------| 3411| orange | 200 | 3412| apple | 77 | 3413| lemon | 150 | 3414 3415 3416 34172. Use data from 2 (or more!) different columns 3418 3419This query gets the best guess at a customer's state: 3420``` 3421SELECT customer, 3422 COALESCE(mailing_state, billing_state, ip_address_state) AS state 3423FROM addresses 3424``` 3425(Mailing address, most accurate. If not, try billing address. As a last resort, use their IP address) 3426 3427| customer | mailing_state | billing_state | ip_address_state | 3428|------------|-----------------|-----------------|--------------------| 3429| 1 | Bihar | Bihar | Bihar | 3430| 2 | NULL | Kerala | Kerala | 3431| 3 | NULL | NULL | Punjab | 3432| 4 | Gujarat | Punjab | Gujarat | 3433 3434 3435| state | 3436|------------| 3437| Bihar | 3438| Kerala | 3439| Punjab | 3440| Gujarat | 3441</div> 3442 </a> 3443 3444 <a href="/comics/single-quote/"> 3445 <span class="title"> 3446 single quote your strings 3447 </span> 3448 <div class="keywords"></div> 3449 <div class="transcript">In some SQL implementations (like PostgreSQL), if you double quote a string it'll interpret it as a column name: 3450 3451smiling stick figure with short curly hair: 3452``` 3453SELECT * FROM cats 3454WHERE name "ms piggy" ; 3455``` 3456 3457postgres: 3458``` 3459error: column "ms piggy" 3460does not exist 3461``` 3462 3463person, thinking: right, I need to use single quotes 3464 3465Here's a table explaining what different quotes mean in different SQL databases. "Identifier" means a column name or table name. 3466 3467Sometimes table names have special characters like spaces in them so it's useful to be able to quote them. 3468 3469| | single quotes ('miss piggy') | double quotes ("miss piggy") | backticks (`miss piggy`) | 3470|---------------|--------------------------------|----------------------------------|----------------------------| 3471| MySQL | string | string or identifier | identifier | 3472| PostgreSQL | string | identifier | invalid | 3473| SQLite | string | string or identifier | identifier | 3474| SQL server | string | string or identifier | invalid | 3475 3476person: I always use single quotes for strings in SQL queries! It keeps me (and others!) from getting confused.<
3476/div> 3477 </a> 3478 3479 <a href="/comics/mmap/"> 3480 <span class="title"> 3481 mmap 3482 </span> 3483 <div class="keywords"></div> 3484 <div class="transcript">### what's mmap for? 3485person 1: I want to work with a VERY LARGE FILE but it won't fit in memory 3486person 2: You could try mmap! 3487(mmap = "memory map") 3488 3489### load files lazily with mmap 3490When you mmap a file, it gets mapped into your program's memory. 34912 TB file: 2 TB of virtual memory 3492but nothing is ACTUALLY read into RAM until you try to access the memory. 3493(how it works: page faults!) 3494 3495### how to mmap in Python 3496``` 3497import mmap f= open("HUGE.txt") 3498mm= mmap.mmap (f. filenol), 0) 3499``` 3500(this won't read the file from disk! Finishes ~instantly.) 3501 3502`print (mm C-1000:7)` 3503this will read only the last 1000 bytes! 3504 3505### sharing big files with mmap 3506three processes: we all want to read the same file! 3507mmap: no problem! 3508 3509Even if 10 processes mmap a file, it will only. be read into memory once 3510 3511### dynamic linking uses mmap 3512program: I need to use libc.so.6 (standard library) 3513ld dynamic linker: you too eh? no problem. I always mmap, so that file is probably loaded into memory already. 3514 3515### anonymous memory maps 3516- not from a file (memory set to by default) 3517- with `MAP.SHARED`, you can use them to share memory with a subprocess!</div> 3518 </a> 3519 3520 <a href="/comics/processes/"> 3521 <span class="title"> 3522 processes 3523 </span> 3524 <div class="keywords"></div> 3525 <div class="transcript">## what's in a process? 3526 3527### PID 3528process: process #129 reporting for duty! 3529 3530### USER and GROUP 3531from offscreen: who are you running as? 3532process: julia! 3533 3534### ENVIRONMENT VARIABLES 3535like `PATH`! you can set them with 3536`$ env A=val./program` 3537 3538### SIGNAL HANDLERS 3539process 1: I ignore `SIGTERM`! 3540process 2: I shut down safely! 3541 3542### WORKING DIRECTORY 3543Relative paths (./blah) are relative to the working directory! `chdir` changes it. 3544 3545### PARENT PID 3546PID 129 -> PID 147 -> PID 1 (`init`) is everyone's ancestor 3547 3548### COMMAND LINE ARGUMENTS 3549see them in `/proc/PID/cmdline` 3550 3551### OPEN FILES 3552very open file has an offset. 3553 3554process: I've read 8000 bytes of that one 3555 3556### MEMORY 3557heap! stack! shared libraries! the program's binary! mmaped files! 3558 3559### THREADS 3560sometimes one 3561sometimes LOTS 3562 3563### CAPABILITIES 3564process 1: I have `CAP_PTRACE` 3565process 2: well I have `CAP_SYS_ADMIN` 3566 3567### NAMESPACES 3568process: I'm in the host network namespace 3569container process: I have my own namespace!</div> 3570 </a> 3571 3572 <a href="/comics/libc/"> 3573 <span class="title"> 3574 libc 3575 </span> 3576 <div class="keywords">[linux2]</div> 3577 <div class="transcript"></div> 3578 </a> 3579 3580 <a href="/comics/attitude-matters/"> 3581 <span class="title"> 3582 when debugging, your attitude matters 3583 </span> 3584 <div class="keywords">[debugging]</div> 3585 <div class="transcript"></div> 3586 </a> 3587 3588 <a href="/comics/overlay-filesystems/"> 3589 <span class="title"> 3590 overlay filesystems 3591 </span> 3592 <div class="keywords"></div> 3593 <div class="transcript">### how layers work: `mount -t overlay` 3594smiling stick figure with curly hair: can you combine these 37 layers into one filesystem? 3595 3596linux, represented by Tux the penguin: yes! Just run `mount ât overlay` with the right parameters! 3597 3598### `mount -t overlay` has 4 parameters 3599- `lowerdir`: list of read-only directories 3600- `upperdir`: directory where writes should go 3601- `workdir`: empty directory for internal use 3602- `target`: the merged result 3603 3604### `upperdir`: where all writes go 3605when you create, change, or delete a file, it's recorded in the upperdir. 3606 3607usually this starts out empty and is deleted when the container exits 3608 3609### lowerdir: the layers. read only. 3610smiling stick figure with curly hair: you can run `$ mount -t overlay` inside a container to gee all the lower dirs that were combined to create its filesystem! 3611 3612### here's an example! 3613``` 3614$ mount -t overlay overlay -o 3615 lowerdir=/lower , upperdir=/upper , workdir=/work /merged 3616$ ls /upper 3617cat.txt dog.txt 3618$ ls /lower 3619dog. txt bird.txt 3620$ ls /merged 3621cat.txt dog.txt bird.txt 3622``` 3623 3624the merged version of dog.txt is the one from the upper directory</div> 3625 </a> 3626 3627 <a href="/comics/shell/">
3628 <span class="title"> 3629 what's a shell? 3630 </span> 3631 <div class="keywords">[linux2]</div> 3632 <div class="transcript"></div> 3633 </a> 3634 3635 <a href="/comics/check-assumptions/"> 3636 <span class="title"> 3637 debugging tips: check your assumptions 3638 </span> 3639 <div class="keywords">[debugging]</div> 3640 <div class="transcript"></div> 3641 </a> 3642 3643 <a href="/comics/inodes/"> 3644 <span class="title"> 3645 inodes 3646 </span> 3647 <div class="keywords">[linux2]</div> 3648 <div class="transcript"></div> 3649 </a> 3650 3651 <a href="/comics/pipes/"> 3652 <span class="title"> 3653 pipes 3654 </span> 3655 <div class="keywords"></div> 3656 <div class="transcript">### panel 1 3657Sometimes you want to send the output of one process to the input of another 3658 3659``` 3660$ ls | wc -l 3661 53 3662``` 3663(53 files!) 3664 3665### a pipe is a pair of 2 magical file descriptors 3666Illustration of a tube where the left side is labelled "IN" and the right side is labelled "OUT". There is an arrow between them. "IN" is labelled "pipe input" and "OUT" is labelled "pipe output" 3667 3668`stdin` -> `ls` -> IN -> OUT -> `wc` -> `stdout` 3669 3670### panel 3 3671when `ls` does write (IN, "hi") `wc` can read it! 3672read (OUT) -> "hi" 3673 3674Pipes are one way -> You can't write to OUT. 3675 3676### Linux creates a buffer for each pipe 3677`ls` -> IN [Buffer: data waiting to be read] OUT -> `wc` 3678 3679If data gets written to the pipefaste than it's read, the buffer will fill up. 3680 3681When the buffer is full, writes to IN will block (wait) until the reader reads. This is normal & ok (smiley face). 3682 3683### what if your target process dies? 3684-> `ls` -> [dead IN face] [dead OUT face] -> [dead `wc` face] -> 3685 3686If `wc` dies, the pipe will close and `ls` will be sent `SIGPIPE`. By default, `SIGPIPE` terminates your process. 3687 3688### named pipes 3689`$ mkfifo my-pipe` 3690 3691This lets 2 unrelated pocesses communicate through a pipe 3692 3693process 1, wearing a hat: 3694``` 3695f=open(./my_pipe) 3696f.write("hi!\n") 3697``` 3698 3699process 2, with curly hair: 3700``` 3701f=open(./my_pipe) 3702f.readline() <- "hi!" 3703```</div> 3704 </a> 3705 3706 <a href="/comics/learn-one-thing/"> 3707 <span class="title"> 3708 learn one thing at a time 3709 </span> 3710 <div class="keywords">[debugging]</div> 3711 <div class="transcript"></div> 3712 </a> 3713 3714 <a href="/comics/ping/"> 3715 <span class="title"> 3716 ping 3717 </span> 3718 <div class="keywords"></div> 3719 <div class="transcript">### ping checks if you can reach a host and how long the host took to reply 3720`$ping health.gov.au` 3721 3722output: 3723`... time=253ms...` 3724 3725Australia is 17,000 km from me. at the speed of light it's still far! 3726 3727### ping works by sending an ICMP packet and waiting for a reply 3728ping: to: health.gov.au hello! 3729health.gov.au: I'm here! 3730 3731### myth: if a host doesn't reply to ping, that means it's down 3732Some hosts never respond to ICMP packets. This is why traceroute shows "..." sometimes. 3733 3734ping: hello! 3735host (thinking): not listening!! 3736 3737### traceroute tells you the path a packet takes to get to a destination 3738me â my ISP â NYC â Sacramento â Australia 3739 3740### example traceroute 3741 3742`$ traceroute health.gov.au` 3743`1: 192.168.1.1 3ms` â router 3744`2:...yul.ebox.ca 12 ms` â ISP 3745`...` 3746`8: NYC4. ALTER.NET 24 ms` â here the packet crossed the USA! from NYC- Sacramento! 3747`9: SAC1.ALTER.NET 97 ms` 3748`...` 3749`16: health.gov.au 253ms` â crossing the US takes time 3750 3751### mtr 3752like traceroute, but nicer output! try it! 3753 3754### last panel 3755look up how traceroute works (using TTLs!) it's simple + cool!</div> 3756 </a> 3757 3758 <a href="/comics/kill/"> 3759 <span class="title"> 3760 kill 3761 </span> 3762 <div class="keywords"></div> 3763 <div class="transcript"> 3764### kill doesn't just kill programs 3765you can send ANY signal to a program with kill! 3766 3767`$ kill -SIGNAL PID` 3768(name or number) 3769 3770### which signal kill sends 3771 3772name num 3773``` 3774kill => SIGTERM 15 3775kill -9 => SIGKILL 9 3776kill -KILL => SIGKILL 9 3777kill -HUP => SIGHUP 3778kill -STOP => SIGSTOP 3779``` 3780 3781### kill -l lists all signals. 37821. HUP 37832. INT 37843. QUIT 37854. ILL 37865. TRAP 37876. ABRT 37887. BUS 37898. FPE 37909. KILL 379110. USR1 379211. SEGV 379312. USR2 379413. PIPE 379514. ALRM 379615. TERM 379716. STKFLT 379817. CHLD 379918. CONT 380019. STOP 380120. TSTP 380221. TTIN 380322. TTOU 380423. URG 380524. XCPU 380625. XFS2 380726. VTALRM 380827. PROF 380928. WINCH 381024. POLL 381130. PWR 381231. SYS 3813 3814 3815### killall -SIGNAL NAME 3816signals all processes called NAME 3817for example: 3818 3819`$ killall firefox` 3820 3821useful flags: 3822-w: wait for all signaled processes to die 3823-i: ask before signalling 3824 3825### pgrep 3826prints PIDs of matching running programs 3827 3828pgrep fire matches firefox firebird 3829NOT bash firefox.sh 3830 3831To search the whole command line (eg bash firefox.sh), use `pgrep -f` 3832 3833### pkill 3834same as pgrep, but signals PIDS found. Example: 3835`$ pkill -f firefox` 3836 3837I use pkill more than killall these days.</div> 3838 </a> 3839 3840 <a href="/comics/terminals/">
3841 <span class="title"> 3842 terminals 3843 </span> 3844 <div class="keywords">[linux2]</div> 3845 <div class="transcript"></div> 3846 </a> 3847 3848 <a href="/comics/branch-pointer-commit/"> 3849 <span class="title"> 3850 a branch is a pointer to a commit 3851 </span> 3852 <div class="keywords">[git]</div> 3853 <div class="transcript">A branch in git is a pointer to a commit SHA 3854 3855master â 2e9fab 3856awesome-feature â 3bafea 3857fix-typo â 9a9a9a 3858 3859Here's some proof! In your favourite git repo, run this command: 3860 3861```$ cat .git/refs/heads/master``` 3862"master" is just a text file with the commit SHA master points at! 3863 3864Understanding what a branch is will make it WAY EASIER to fix your branches when they're broken: you just need to figure out how to get your branch to point at the right commit again! 3865 38663 main ways to change the commit a branch points to: 3867- ```git commit``` will point the branch at the new commit 3868- ```git pull``` will point the branch at the same commit as the remote branch 3869- ```git reset COMM T_SHA``` will point the branch at ```COMM T_SHA```</div> 3870 </a> 3871 3872 <a href="/comics/file-buffering/"> 3873 <span class="title"> 3874 file buffering 3875 </span> 3876 <div class="keywords"></div> 3877 <div class="transcript">### panel 1 3878 3879person with short curly hair, distressed and surrounded by question marks: I printed some text but it didn't appear on the screen. why?? 3880 3881cheerful person with long straight hair: time to learn about flushing! 3882 3883### On Linux, you write to files & terminals with the system call <3 `write` <3 3884 3885process, represented by a box with a smiley face: please write "I <3 cats" to file #1 (`stdout`) 3886 3887Linux, also represented by a box with a smiley face: okay! 3888 3889### I/O libraries don't always call `write` when you print 3890 3891`printf("I <3 cats");` 3892 3893`printf`: I'll wait for a newline before actually writing 3894 3895This is called buffering and it helps save on syscalls. 3896 3897### 3 kinds of buffering 3898(defaults vary by library) 3899 39001. None. this is the default for `stderr` 39012. Line buffering (write after newline). The default for terminals. 39023. "full" buffering (write in big chunks). The default for files and pipes. 3903 3904### flushing 3905(little picture of a toilet) 3906 3907To force your I/O library to write everything it has in its buffer right now, call `flush`~ 3908 3909`stdio`: I'll call `write` right away! 3910 3911### when it's useful to flush 3912- when writing an interactive prompt! 3913Python example: 3914`print ("password: ", flush=True)` 3915- when you're writing to a pipe/socket 3916 3917program: no seriously, actually write to that pipe please</div> 3918 </a> 3919 3920 <a href="/comics/hash-functions/"> 3921 <span class="title"> 3922 hash functions 3923 </span> 3924 <div class="keywords"></div> 3925 <div class="transcript"></div> 3926 </a> 3927 3928 <a href="/comics/openssl/"> 3929 <span class="title"> 3930 openssl 3931 </span> 3932 <div class="keywords"></div> 3933 <div class="transcript">### openssl is a tool for doing *SSL things* 3934(aka TLS) 3935- inspect certificates 3936- create CSRs 3937- sign certificates 3938 3939It uses the OpenSSL library (or LibreSSL) 3940 3941### inspect a certificate 3942 3943``` 3944$ openssl x509 -in 3945 FILE.crt -noout -text 3946``` 3947 3948This works for files ending in .crt or .pem! Try it out: 3949 3950you probably have certs in `usr/share/ca-certificates` 3951 3952### look at a website's certificate 3953``` 3954$openssl s_client 3955 -showcerts -connect 3956 google.com:443 3957``` 3958happy little stick figure with curly hair: pipe this to openssl x509 to parse! 3959 3960### panel 4 3961 3962certificate authority, represented by a box with a neutral expression: please upload a CSR 3963 3964person: a WHAT?! 3965 3966to get a SSL cert for your website, you need to make a file called a "certificate signing request." 3967 3968### make a CSR 3969``` 3970$ openssl req -new 3971 -sha256 -key FILE.key 3972 -out FILE.csr 3973``` 3974make a `FILE.key` with `$ openssl genrsa` 3975 3976### `md5/sha1/sha25b/sha512` 3977Not quite SSL but useful: 3978`$openSSL md5 FILE` 3979 3980computes the md5sum of FILE. Same for other digests 3981 3982`$ openssl LIST -digestcommands` shows all supported digests.</div> 3983 </a> 3984 3985 <a href="/comics/redirects/">
3986 <span class="title"> 3987 bash redirects 3988 </span> 3989 <div class="keywords"></div> 3990 <div class="transcript">### panel 1: unix programs have 1 input and 2 outputs 3991 3992When you run a command from a terminal, the input & outputs go 3993to/from the terminal by default. 3994 3995Picture of a program (represented by a box with a smiley face) with 1 arrow coming in and 2 arrows out. The arrows are numbered 0, 1, and 2, and there's a 3996comment: "each input/output has a number, its "file descriptor") 3997 3998**arrow 0 (coming into program): `<` redirects stdin** 3999 4000`wc < file.txt` and `cat file.txt | wc` both read `file.txt` to wc's stdin 4001 4002``` 4003wc < file.txt 4004cat file.txt 4005``` 4006 4007**arrow 1 (coming out of program): `>` redirects stdout** 4008 4009``` 4010cmd > file.txt 4011``` 4012 4013**arrow 2 (coming out of program): `2>` redirects stderr** 4014 4015``` 4016cmd 2> file.txt 4017``` 4018 4019### panel 2: `2>&1` redirects stderr to stdout 4020 4021``` 4022cmd > file.txt 2>&1 4023``` 4024 4025Illustration of cmd, represented by a box with a smiley face. There is one arrow, labelled "sdout(1)", leading to a box labelled "file.txt". There is a second arrow coming out of cmd, labelled "stderr(2)". Then, there's a squiggly third arrow, labelled "2>&1", that leads from "stderr(2)" to "file.txt". 4026 4027 4028### panel 3: `/dev/null` 4029 4030your operating system ignores all writes to `/dev/null` 4031 4032``` 4033cmd > /dev/null 4034``` 4035 4036picture of stdout going to a trash can (`/dev/null`) and stderr still going to the terminal 4037 4038### panel 2: sudo doesn't addect redirects 4039 4040 4041your bash shell opens a file to redirect to it, and it's running as you. So 4042 4043 4044``` 4045$ sudo echo x > /etc/xyz 4046``` 4047 won't work. do this instead: 4048 4049 4050 4051``` 4052$ sudo echo x | tee /etc/xyz 4053``` 4054</div> 4055 </a> 4056 4057 <a href="/comics/updating-dns/"> 4058 <span class="title"> 4059 why updating DNS is slow 4060 </span> 4061 <div class="keywords">[dns]</div> 4062 <div class="transcript"></div> 4063 </a> 4064 4065 <a href="/comics/shellcheck/"> 4066 <span class="title"> 4067 shellcheck 4068 </span> 4069 <div class="keywords"></div> 4070 <div class="transcript">### shellcheck finds problems with your shell scripts 4071`$ shellcheck my-script.sh` 4072shellcheck: oops, you can't use in an `if [ ... ]`! 4073 4074### it checks for hundreds of common shell scripting errors 4075shellcheck: hey, that's a bash- only feature but your script starts with `#!/bin/sh` 4076 4077### every shellcheck error has a number (like "SC2013") 4078and the shellcheck wiki has a page for every error with examples! I've learned a lot from the wiki.
4079 4080### it even tells you about misused commands 4081shellcheck: hey, it looks like you're not using `grep` correctly here 4082person: wow I'm not! thanks! 4083 4084### your text editor probably has a shellcheck plugin 4085shellcheck: I can check your shell scripts every time you save! 4086 4087### basically, you should probably use it 4088bash has too many weird edge cases for me to remember, I love that shellcheck can help me out!</div> 4089 </a> 4090 4091 <a href="/comics/understand-can-fix/"> 4092 <span class="title"> 4093 if you understand a bug, you can fix it 4094 </span> 4095 <div class="keywords">[debugging]</div> 4096 <div class="transcript"></div> 4097 </a> 4098 4099 <a href="/comics/sockets/"> 4100 <span class="title"> 4101 sockets 4102 </span> 4103 <div class="keywords"></div> 4104 <div class="transcript">### networking protocols are complicated 4105book: TCP/IP Illustrated, Volume 1, by Stevens (600 pages) 4106person: what if I just want to download a cat picture? 4107 4108### Unix systems have an API called the "socket API" that makes it easier to make network connections 4109Unix: you don't need to know how TCP works. I'll take care of it! 4110 4111### here's what getting a cat picture with the Socket API looks like: 41121. Create a socket: `fd= socket(AF_INET, SOCK-STREAM...)` 41132. Connect to an IP/port: `connect (fd, 12.13.14.15:80)` 41143. Make a request: `write (fd, "GET /cat.png HTTP/I.I...)` 41154. Read the response: `cat-picture= read (fd...)` 4116 4117### Every HTTP library uses sockets under the hood 4118`$curl awesome.com` 4119Python: `requests.get("yay.us")"` 4120(sockets) 4121 4122person: oh, cool, I could write an HTTP library too if I wanted`*`. Neat! 4123`*` SO MANY edge cases though! :) 4124 4125### AF_INET? What's that? 4126AF-INET means basically "internet socket": it lets you connect to other computers on the internet using their IP address. 4127 4128The main alternative is AF-UNIX ("unix domain socket") for connecting to programs on the same computer. 4129 4130### 3 kinds of internet (AF INET) sockets: 41311. `SOCK_STREAM` = TCP (curl uses this) 41322. `SOCK_DGRAM` = UDP (dig (DNS) uses this) 41333. `SOCK.RAW` = just let me send IP packets. I will implement my own protocol. (ping uses this)</div> 4134 </a> 4135 4136 <a href="/comics/file-descriptors/"> 4137 <span class="title"> 4138 file descriptors 4139 </span> 4140 <div class="keywords"></div> 4141 <div class="transcript">### Unix systems use integers to track open files 4142Process, represented by a box with a smiley face: Open `foo.txt` 4143kernel, also represented by a box with a smiley face: okay! that's 4144file #7 for you. 4145 4146these integers are called file descriptors 4147 4148 4149### `lsof` (list open files) will show you a process's open files 4150`$lsof -P 4242` 4151(4242 is the PID we're interested in) 4152 4153FD NAME 4154``` 41550 /dev/pts/tty1 41561 /dev/pts/tty1 41572 pipe: 29174 41583 /home/bork/awesome.txt 41595 /tmp/ 4160``` 4161(FD is for file descriptor) 4162 4163### file descriptors can refer to: 4164- files on disk 4165- pipes 4166- sockets (network connections) 4167- terminals (like `xterm`) 4168- devices (your speaker! `/dev/null`!) 4169- LOTS MORE (`event fd`, `inotify`, `signalfo`, `epoll`, etc.) 4170 4171little tiny smiling stick figure: not EVERYTHING on Unix is a file, but lots of things are 4172 4173 4174### When you read or write to a file/pipe/network connection you do that using a file descriptor 4175 4176person: connect to google.com 4177OS: ok! fd is 5! 4178person: write GET / HTTP/1.1) to fd #5 4179OS: done! 4180 4181### Let's see how some simple Python code works under the hood: 4182Python: 4183``` 4184f = open ("file.txt") 4185f. read lines() 4186``` 4187Behind the scenes: 4188Python program: open file.txt 4189OS: ok! fd is 4 4190Python program: read from file #4 4191OS: here are the contents! 4192 4193### (almost) every process has 3 standard FDs: 4194- `stdin`: 0 4195- `stdout`: 1 4196- `stderr`: 2 4197 4198 "read from stdin" 4199means 4200"read from the file descriptor O" 4201(could be a pipe or file or terminal)</div> 4202 </a> 4203 4204 <a href="/comics/response-headers/"> 4205 <span class="title"> 4206 HTTP response headers 4207 </span> 4208 <div class="keywords"></div> 4209 <div class="transcript">### Age 4210how many seconds response has been cached 4211```Age: 355``` 4212 4213### Date 4214when response was sent 4215```Date: Mon, 09 Sep 2019...``` 4216 4217### Last-Modified 4218when content was last modified (not always accurate) 4219 4220### ETag 4221Version of response body 4222```Etag: "ac5affa.."``` 4223
4224### Cache-Control 4225various caching 4226```settings Cache-Control: max-age=300``` 4227 4228### Vary 4229request headers that response will vary based on 4230 4231### Via 4232added by proxy servers 4233```Via: nginx``` 4234 4235### Expires 4236The response is stale and should be re-requested after this time. 4237 4238### Connection 4239"close" or "keep-alive" Whether to keep the TCP connection open 4240 4241### Set-Cookie 4242Sets a cookie. 4243```Set-Cookie: name=value; HttpOnly``` 4244 4245### Access-Control-* 4246Called CORS headers. These allow cross-origin requests. 4247 4248### Content-Type 4249MIME type of body 4250```Content-Type: text/plain``` 4251 4252### Content-Length 4253length of body in bytes 4254```Content-Length: 33``` 4255 4256### Content-Language 4257Language of body 4258```Content-Language: en-US``` 4259 4260### Content-Encoding 4261Whether body is compressed 4262```Content-Encoding: gzip``` 4263 4264### Location 4265URL to redirect to 4266```Location: /cat.png``` 4267 4268### Accept-Ranges 4269Whether Range request header is supported for this resource</div> 4270 </a> 4271 4272 <a href="/comics/tcpdump/"> 4273 <span class="title"> 4274 tcpdump 4275 </span> 4276 <div class="keywords">[bpf]</div> 4277 <div class="transcript">### tcpdump lets you view network packets being sent and received 4278 4279happy little stick figure: it's not the easiest to use but it's usually installed (heart) 4280 4281### `-n` 4282don't try to resolve IP addresses/ports to DNS/port names. makes it run faster 4283 4284### `i wlan0` 4285Which network interface to capture packets on 4286 4287person: I often use "`-i any`" to make sure I'm not missing any packets! 4288 4289### `-w file.pcap` 4290Write packets to a file for later analysis with tcpdump/tshark/wireshark/another tool 4291 4292pcap is short for "packet capture" 4293 4294### `-A` 4295print packet contents, not just headers. Nice if you want to quickly see what a few packets contain. 4296 4297### `-c 1000000` 4298Only capture a limited count of packets. 4299 4300person: I use it with `-w` so I don't accidentally fill up my disk!</div> 4301 </a> 4302 4303 <a href="/comics/head-tail/"> 4304 <span class="title"> 4305 head & tail 4306 </span> 4307 <div class="keywords"></div> 4308 <div class="transcript">### head 4309shows the first you 10 lines of a file. 4310 4311if you pipe a program's output to head, the program will stop after printing 10 lines (it gets sent SIG PIPE) 4312 4313### tail 4314 4315tail shows the last 10 lines! 4316 4317`tail -f FILE` will follow: 4318 4319print any new lines added to the end of FILE. Super useful for log files! 4320 4321 4322### -n NUM 4323-n NUM (either head or tail) will change the # lines shown 4324 4325NUM can also be negative. Example: `$ head -n 5 file.txt` will print all lines except the last 5 4326 4327 4328### -C NUM 4329show the first /last NUM bytes of the file 4330 4331`$ head -c 1k` will show the first 1024 bytes 4332 4333### tail --retry 4334keep trying to open file if it's inaccesible 4335 4336### tail --pid PID 4337stop when process PID stops running (with `-f`) 4338 4339### tail --follow-name 4340Usually `tail -f` will follow a file descriptor. 4341 4342`tail --follow-name FILENAME` 4343will keep following the same file name, even if the file descriptor changes 4344</div> 4345 </a> 4346 4347 <a href="/comics/mitmproxy/"> 4348 <span class="title"> 4349 mitmproxy 4350 </span> 4351 <div class="keywords"></div> 4352 <div class="transcript">### panel 1: 4353 4354phone: ??? 4355server: ??? 4356small sad stick figure, thinking: what is my phone saying about me? looks like it's encrypted 4357 4358### panel 2: 4359mitmproxy can proxy connections from your laptop or phone and let you see the contents. It even works with encrypted connections. 4360 4361An illustration showing a phone, a server (represented by a box with a smiley face), and mitmproxy between them. There are arrows going to and from mitmproxy and the phone and server. 4362 4363### how you use it 43641. install mitmproxy root CA on your laptop/phone 43652. run `mitmweb` (web UI version) on computer 43663. tell the program/phone to proxy through mitmproxy 4367 4368### how it works 4369phone: wizardzines.com certificate plz 4370mitmproxy: yes I am wizardzines.com 4371phone: sounds legit, this CA I trust [the fake mitmproxy CA you installed] says that certificate is valid 4372 4373### some apps pin a cert 4374makes mitmproxy not work, look up "trust killer" to get around that 4375 4376### script it in Python 4377modify requests/responses arbitrarily 4378 4379### other similar tools 4380(not all are free, though) 4381- charles proxy 4382- burp suite 4383- fiddler</div> 4384 </a> 4385 4386 <a href="/comics/oh-shit-wrong-branch/">
4387 <span class="title"> 4388 oh shit! I accidentally committed to the wrong branch! 4389 </span> 4390 <div class="keywords"></div> 4391 <div class="transcript">1. Check out the correct branch 4392 4393`git checkout correct-branch` 4394 4395`cherry-pick` makes a new commit with the same changes as *, but a different parent 4396 43972. Add the commit you wanted to it 4398 4399`git cherry-pick COMMIT_ID` 4400â use '`git log wrong-branch`' to find this 4401 44023. Delete the commit from the wrong branch. 4403``` 4404git checkout wrong-branch 4405git reset --hard HEAD^ 4406``` 4407be careful when running '`git reset --hard!`' always run '`git status`' first to make sure there aren't uncommitted changes and '`git stash`' to save them if there are</div> 4408 </a> 4409 4410 <a href="/comics/same-origin-policy/"> 4411 <span class="title"> 4412 the same-origin policy 4413 </span> 4414 <div class="keywords">[cors]</div> 4415 <div class="transcript"></div> 4416 </a> 4417 4418 <a href="/comics/async-functions/"> 4419 <span class="title"> 4420 async functions 4421 </span> 4422 <div class="keywords"></div> 4423 <div class="transcript"></div> 4424 </a> 4425 4426 <a href="/comics/socat/"> 4427 <span class="title"> 4428 socat 4429 </span> 4430 <div class="keywords"></div> 4431 <div class="transcript">### `socat` lets you proxy basically any 2 things 4432 4433Diagram of a star, a heart, and the word "`socat`". There are arrows going to and from `socat` to the heart and star. 4434 4435the basic syntax: 4436`socat THING1 THING2` 4437 4438### socat supports: 4439- tcp sockets 4440- unix domain sockets 4441- pipes 4442- SSL sockets 4443- files 4444- processes 4445- UDP sockets 4446- ... and MORE! 4447 4448### order doesn't matter 4449`socat THING1 THING2` 4450is the same as 4451`socat THING2 THING1` 4452 4453### expose a unix domain socket on port 1337 4454``` 4455socat TCP-LISTEN:1337 4456 UNIX-CONNECT:/path 4457``` 4458 4459### proxy from local HTTP port to remote server 4460``` 4461socat TCP-LISTEN:1337 4462 TCP:domain.com:80 4463``` 4464 4465### `-v` 4466write all transferred data to `stderr` 4467 4468happy little stick figure: useful for debugging! 4469</div> 4470 </a> 4471 4472 <a href="/comics/assembly/"> 4473 <span class="title"> 4474 assembly 4475 </span> 4476 <div class="keywords">[linux2]</div> 4477 <div class="transcript"></div> 4478 </a> 4479 4480 <a href="/comics/top/"> 4481 <span class="title"> 4482 top 4483 </span> 4484 <div class="keywords"></div> 4485 <div class="transcript">### top 4486a live-updating summary of the top users of your system's resources: 4487 4488sad little stick person: who's using all my memory 4489top: chrome, obv! 4490 4491let's explain some numbers in top! 4492 4493### load average 44943 numbers that roughly reflect demand for your CUs on the system in the last 1, 5, and 15 minutes. 4495 4496if it's higher than the # of CPUs you have, that's often bad! 4497 4498### memory 44994 numbers: total/free/used/cached 4500 4501One perhaps unexpected thing: total is NOT free+ used! 4502 4503total = free + used + cached (filesystem cache) 4504 4505### % CPU 4506confused stick figure with hair sticking out: 350%? what? 4507 4508this column is given as the % of a single core. If you have 4 cores, this can go up to 400%! 4509 4510### RES 4511this column is the "resident set size", aka how much RAM your process is using. 4512 4513SHR is how much of the RES is shared with other processes 4514 4515### htop 4516a prettier & more interactive version of top 4517 4518Illustration of a graph showing different users of system resources with graphs showing used and cached memory in different colours.</div> 4519 </a> 4520 4521 <a href="/comics/certificates/"> 4522 <span class="title"> 4523 TLS certificates 4524 </span> 4525 <div class="keywords"></div> 4526 <div class="transcript">To establish an HTTPS connection to examplecat.com, the client needs proof that the server is `examplecat.com`. 4527 4528Browser, represented by the Firefox logo: hey I want examplecat.com 4529 4530Server, represented by a box with a smiley face: here's proof that I'm examplecat.com. 4531 4532(the proof is called a certificate.) 4533 4534A TLS certificate has: 4535- a set of domains it's valid for (eg `examplecat.com`) 4536- a start and end date (example: July 1 2019 to Oct 1 2019) 4537- a secret private key that only the server has (this is the only secret part, the rest is public) 4538- a public key to use when encrypting 4539- a cryptographic signature from someone trusted 4540 4541A box that reads "wizardzines.com, Jul 1 - Oct 1 2019, <public key> with a logo that says Let's Encrypt Approved" 4542 4543The trusted entity that signs the certificate is called a Certificate Authority (CA) and they're responsible for only signing certificates for a domain for that domain's owner. 4544 4545smiling stick figure with short spiky hair: will you sign this certificate for examplecat.com? 4546 4547let's encrypt, represented by a box with a smiley face: lol no I checked `examplecat.com/.well-known/acme-challenge` and you don't own that domain. 4548 4549When your browser connects to `examplecat.com`, it validates the certificates using a list of trusted CAs installed on your computer. These CAS are called "root certificate authorities". 4550 4551browser, thinking: 45521. the examplecat.com server is signed by Let's Encrypt 45532. Let's Encrypt's cert is signed by IdenTrust 45543. IdenTrust is on my trusted list. 45554. This is okay!</div> 4556 </a> 4557 4558 <a href="/comics/capabilities/">
4559 <span class="title"> 4560 capabilities 4561 </span> 4562 <div class="keywords"></div> 4563 <div class="transcript">### we think of root as being all-powerful... 4564The following items are in spiky bubbles: 4565- edit any file 4566- change network config 4567- spy on any program's memory 4568 4569### ... but actually to do "root" things, a process needs the right â capabilitiesâ 4570Process, represented by a box with a smiley face: I want to modify the route table! 4571Linux, represented by a penguin: you need CAP_NET_ADMIN! 4572 4573### there are dozens of capabilities 4574Illustration of a smiling stick figure with curly hair. 4575 4576Person: `$ man capabilities` explains all of them but let's go over 2 important ones! 4577 4578### CAP_SYS_ADMIN 4579lets you do a LOT of things. avoid giving this if you can! 4580 4581### CAP_NET_ADMIN 4582allow changing network settings 4583 4584### by default containers have limited capabilities 4585Process: can I call process_vm_ready? 4586Linux: nope! you'd need CAP_SYS_PTRACE for that! 4587 4588### $ getpcaps PID 4589print capabilities that PID has 4590 4591### getcap / setcap 4592system calls: get and set capabilities!</div> 4593 </a> 4594 4595 <a href="/comics/security-headers/"> 4596 <span class="title"> 4597 HTTP security headers 4598 </span> 4599 <div class="keywords"></div> 4600 <div class="transcript">These are headers your server can set. They ask the browser to protect your users' data against attackers in different ways: 4601 4602### Content-Security-Policy (often called CSP) 4603Only allow CSS/Javascript from certain domains you choose to run on your website. Helps protect against cross-site-scripting (aka XSS) attacks. 4604 4605### Referrer-Policy 4606Control how much information is sent to other sites in the Referer header. Example: `Referrer-Policy: no-referrer`. 4607 4608(spelling is inconsistent with Referer header :( ) 4609 4610### Strict-Transport-Security (often called HSTS) 4611Require HTTPS. If you set this the client (browser) will never request a plain HTTP version of your site again. Be careful! You can't take it back! 4612 4613### Expect-CT 4614Certificate Transparency (CT) is a system that can help find malicious SSL certificates issued for your site. This header gives the browser a URL to use to report bad certificates to you. 4615 4616### X-XSS-Protection 4617Another way to protect against XSS attacks. Not supported by all browsers, `Content-Security-Polcy` is more powerful.</div> 4618 </a> 4619 4620 <a href="/comics/questions-to-ask/"> 4621 <span class="title"> 4622 questions to ask about your data 4623 </span> 4624 <div class="keywords"></div> 4625 <div class="transcript">It's really easy to make incorrect assumptions about the data in a table: 4626 4627stick figure with short curly hair, smiling: every hospital patient has a doctor right? 4628 4629same person, three hours later, sad: why is everyone from May 2013 missing a doctor?? 4630 4631### Some questions you might want to ask: 4632 4633- Does this column have `NULL` or `0` or empty string values? 4634 4635person, thinking: some patients have `NULL` names, that's good to know 4636 4637- How many different valves does this column have? 4638 4639person, thinking: huh there are 3000 extra doctors in the system who never worked at the hospital, I should filter them out 4640 4641- Are there duplicate values in this column? 4642 4643person, thinking: sometimes a doctor has 2 appointments at the same time, that shouldn't happen 4644 4645- Does the id column in table A always have a match in table B? 4646 4647person, thinking: why are there 213 doctor IDs with no match in the doctors table?! 4648 4649A lot of these can also be enforced by `NOT NULL` or `UNIQUE` or `FOREIGN KEY` constraints on your tables.</div> 4650 </a> 4651 4652 <a href="/comics/threads/"> 4653 <span class="title"> 4654 threads 4655 </span> 4656 <div class="keywords"></div> 4657 <div class="transcript">### Threads let a process do many different things at the same time 4658process: 4659thread 1: I'm calculating ten million digits of Ï! so fun! 4660thread 2: I'm finding a REALLY BIG prime number! 4661 4662### threads in the same process share memory 4663thread 1: I'll write some digits of to Ï O x 129420 in memory 4664thread 2: uh oh! that's where I was putting my prime numbers. 4665 4666### and they share code 4667calculate-pi 4668find-big-prime-number 4669 4670but each thread has its own stack and they can be run by different CPUs at the same time 4671CPU 1: Ï thread 4672CPU 2: primes thread 4673 4674### sharing memory can cause problems (race conditions!) 4675at the same time: 4676memory: 23 4677thread 1: I'm going to add 1 to that number! 4678thread 2: I'm going to add 1 to that number! 4679 4680RESULT: 24 4681WRONG. Should be 25! 4682 4683### why use threads instead of starting a new process? 4684a thread takes less time to create. 4685 4686sharing data between threads is very easy. But it's also easier to make mistakes with threads. 4687 4688thread 1: you weren't supposed to CHANGE that data!</div> 4689 </a> 4690 4691 <a href="/comics/main-ideas/">
4692 <span class="title"> 4693 highlight the main ideas 4694 </span> 4695 <div class="keywords"></div> 4696 <div class="transcript"></div> 4697 </a> 4698 4699 <a href="/comics/ethtool/"> 4700 <span class="title"> 4701 ethtool 4702 </span> 4703 <div class="keywords"></div> 4704 <div class="transcript">### ethtool is for people who need to manage physical networks 4705 4706server, represented by a sad rectangle: why no internet?? 4707smiling stick figure with short curly hair: oops! it would help if you ethernet cable was plugged in! 4708 4709### `ethtool eth0` 4710(name of network interface) 4711 4712this tells you: 4713- is it even connected? ("link detected") 4714- speed 4715- lots more 4716 4717### `--show-offload --offload` 4718your network card can do a lot for you! Like computing checksums. This is called "offloading". This lets you see/change configured offloads. 4719 4720### `--identify INTERFACE` 4721blink the light on the ethernet port. good if you have multiple ports! and cute. (heart) 4722 4723### `-S INTERFACE` 4724show statistics like bytes sent. works for wifi interfaces too. 4725 4726### `-s` 4727change speed/duplex/other settings of an interface. 4728 4729`$ ethtool - eth0 speed 100` 4730 4731### `-i INTERFACE` 4732show firmware info 4733 4734### `iw dev wlan0 link` 4735ethtool is mostly for Ethernet. 4736 4737To see the speed (and more) of a wireless connection, use iw. 4738 4739 4740 4741</div> 4742 </a> 4743 4744 <a href="/comics/kernel-features/"> 4745 <span class="title"> 4746 container kernel features 4747 </span> 4748 <div class="keywords"></div> 4749 <div class="transcript">### containers use these Linux kernel features 4750"container" doesn't have a clear definition, but Docker containers use all of these features. 4751 4752### pivot_root 4753set a process's root directory to a directory with the contents of the container image 4754 4755### cgroups 4756limit memory/CPU usage for a group of processes 4757 4758Linux, represented by a box with a smiley face: only 5OO MB of RAM for you! 4759 4760### namespaces 4761allow processes to have 4762their own: 4763- network 4764- PIDs 4765- hostname 4766- mounts 4767- users 4768- + more 4769 4770### capabilities 4771security: give specific permissions 4772 4773### seccomp-bpf 4774security: prevent dangerous system calls 4775 4776### overlay filesystems 4777this is what makes layers work! Sharing layers saves disk space & helps containers start faster</div> 4778 </a> 4779 4780 <a href="/comics/manager-job/"> 4781 <span class="title"> 4782 what's your manager's job? 4783 </span> 4784 <div class="keywords"></div> 4785 <div class="transcript">Understanding a little about your manager's job helps you work well with them! Some things your manager is responsible for: 4786 4787Each of these items is enclosed in a thought bubble with an illustration. 4788 4789### make sure the team is doing important projects 4790Illustration of a smiling stick figure (the manager). 4791 4792manager: X is a priority this quarter! 4793 4794### keep projects on track 4795Illustration of two smiling stick figures, one with medium length straight hair (the CEO) and another one with no hair (the manager). 4796 4797CEO: what's the status of x project? 4798manager: [needs to answer] 4799 4800### communicate with other teams 4801Illustration of two smiling stick figures, one with curly hair (person on other team) and another one with no hair (the manager). 4802 4803person on other team: we're doing x 4804manager: our teams should collaborate on that! 4805 4806### help team members grow 4807Illustration of a smiling stick figures with curly hair. 4808 4809person: I learned so much this year!</div> 4810 </a> 4811 4812 <a href="/comics/code-one-thing-at-a-time/"> 4813 <span class="title"> 4814 debugging tip: code one thing at a time 4815 </span> 4816 <div class="keywords">[debugging]</div> 4817 <div class="transcript"></div> 4818 </a> 4819 4820 <a href="/comics/css-specificity/"> 4821 <span class="title"> 4822 CSS specificity 4823 </span> 4824 <div class="keywords"></div> 4825 <div class="transcript">### different rules can set the same property 4826 4827which one gets chosen? 4828``` 4829a:visited { 4830 color: purple; 4831 font-size: 1.2em; 4832``` 4833``` 4834} 4835#start-link { 4836color: orange; 4837} 4838``` 4839 4840### CSS uses the "most specific" selector that matches an element 4841In our example, the browser will use `color: orange` because IDs (like `#start-link`) are more specific than pseudoclasses (like `:visited`) 4842 4843### TRY ME! CSS can mix properties from different rules 4844it'll use this font size: 4845``` 4846a:visited { 4847 color: purple; 4848 font-size: 1.2em; 4849``` 4850but use this color because `#start-link` is more specific: 4851``` 4852} 4853#start-link { 4854color: orange; 4855} 4856``` 4857### how CSS picks the "most specific" rule 4858 4859a selector with element names: 4860``` 4861body div span a { 4862 color:red; 4863} 4864``` 4865
4866loses to a selector with `.classes` or `:pseudoclasses`: 4867``` 4868.sidebar .link { 4869 color: orange; 4870} 4871``` 4872loses to a selector with an `#id`: 4873``` 4874#header a { 4875 color: purple; 4876} 4877``` 4878 4879loses to an inline style: 4880``` 4881style="color: green; 4882``` 4883 4884loses to an `!important` rule: 4885``` 4886"color: blue !important; 4887``` 4888 4889(`!important` is very hard to override, which makes life hard for your future self!)</div> 4890 </a> 4891 4892 <a href="/comics/tc/"> 4893 <span class="title"> 4894 tc 4895 </span> 4896 <div class="keywords"></div> 4897 <div class="transcript">### tc 4898is for "traffic control" 4899 4900humanoid traffic light, its hand held up as though directing traffic: packets! stop/slow down/go the other way! 4901 4902great for simulating network problems! 4903 4904### make your internet slow 4905``` 4906$ sudo tc qdisc add dev 4907 wlp3s0 root netem 4908 delay 500ms 4909``` 4910(delay packets by 500ms) 4911 4912and fast again: 4913``` 4914$ sudo tc qdisc del dev 4915 wlp3s0 root netem 4916``` 4917 4918### netem rules 4919netem ("network emulator") is a part of tc that lets you: 4920- drop 4921- duplicate 4922- delay 4923- corrupt 4924 4925packets. see the man page `$ man netem` 4926 4927### make your brother's internet slow 4928Have a Linux router? You can configure tc on it to make your brother's internet slower than yours. google "tc QoS" for a start. 4929 4930### show current tc settings 4931``` 4932$ tc qdisc show 4933$ tc class show dev DEV 4934$ tc filter show dev DEV 4935``` 4936 4937### panel 6 4938 4939smiling stick figure: `tc` can do 10 million more things! This is just the beginning!</div> 4940 </a> 4941 4942 <a href="/comics/containers-are-processes/"> 4943 <span class="title"> 4944 containers = processes 4945 </span> 4946 <div class="keywords"></div> 4947 <div class="transcript">### a container is a group of Linux processes 4948 4949Illustration of a smiling stick figure with curly hair. 4950 4951person: on a Mac, all your containers are actually running in a Linux virtual machine 4952 4953### panel 2 4954 4955person: I started 'top' in a container. Here's what that looks like in ps: 4956 4957- outside the container 4958``` 4959$ ps aux grep top 4960USER PID START COMMAND 4961root 23540 20:55 top 4962bork 23546 20:57 top 4963``` 4964- inside the container 4965``` 4966$ ps aux | grep top 4967USER PID START COMMAND 4968root 25 20:55 top 4969``` 4970(`root 23540 20:55 top` and `root 25 20:55 top` are the same process!) 4971 4972### container processes can do anything a normal process can... 4973 4974Illustration of a smiling stick figure with curly hair, and Linux, represented by its penguin mascot 4975 4976person: I want my container to do X Y Z W! 4977Linux: sure! your computer, your rules! 4978 4979### but usually they have restrictions 4980(there are drawings of locks on either side of the word "restrictions") 4981 4982Illustration of a container, represented by a box with a smiley face. Around it are arrows with the following labels: 4983- different PID namespace 4984- different root directory 4985- cgroup memory limit 4986- limited capabilities 4987- not allowed to run some system calls 4988 4989### the restrictions are enforced by the Linux kernel 4990Linux: NO, you can't have more memory! 4991person: on the next page we'll list all the kernel features that make this work!</div> 4992 </a> 4993 4994 <a href="/comics/bugs-are-normal/"> 4995 <span class="title"> 4996 writing code with bugs is normal 4997 </span> 4998 <div class="keywords">[debugging]</div> 4999 <div class="transcript"></div> 5000 </a> 5001 5002 <a href="/comics/null-surprises/"> 5003 <span class="title"> 5004 NULL surprises 5005 </span> 5006 <div class="keywords"></div> 5007 <div class="transcript">NULL isn't equal (or not equal!) to anything in SQL (x = NULL and x != NULL are never true for any x). This results in 2 behaviours that are surprising at first: 5008 5009### Surprise! x= NULL doesn't work 5010fish 5011name: NULL owner: bob 5012name: nemo owner: ahmed 5013 5014``` 5015SELECT * FROM fish 5016WHERE name = NULL 5017``` 5018 5019no results! 5020You need to use `x IS NULL` instead. 5021 5022works 5023name IS NULL 5024name IS NOT NULL 5025 5026doesn't work 5027name = NULL 5028name != NULL 5029 5030surprise! name != 'betty' doesn't match NULLs 5031 5032fish 5033name: NULL owner: bob 5034name: nemo owner: ahmed 5035 5036``` 5037SELECT FROM fish 5038WHERE name != 'betty' 5039``` 5040 5041name: NULL owner: bob 5042 5043To match NULLS as well, I'll often write something like `WHERE name = 'betty' OR name IS NULL` instead. 5044 5045 5046### more surprising truths 5047 5048More operations with NULL which might be surprising: 5049 50502 + NULL => NULL 5051NULL * 10 => NULL 5052CONCAT('hi', NULL) => NULL 5053NULL = NULL => NULL (NULL isn't even equal to itself!) 50542 = NULL => NULL
50552 != NULL => NULL</div> 5056 </a> 5057 5058 <a href="/comics/performance-reviews/"> 5059 <span class="title"> 5060 on surviving performance reviews 5061 </span> 5062 <div class="keywords">[manager]</div> 5063 <div class="transcript">Performance reviews can be really stressful. 5064 5065Illustration of two stick figures. One has no hair and is smiling, the other one has short curly hair and looks unhappy. 5066 5067person 1: here's the self assessment form to fill out! 5068person 2 (thought bubble): AWESOME PLAN: procrastinate for 2 weeks and then do it at the last minute in a panic! 5069 5070Here's what I've been doing for the last year or so, which has helped! About a month before performance review season comes around, I'll compile a HUGE DOCUMENT with: 5071- every project I did in the last year 5072- the project's goals & results 5073- cool graphs/metrics that show it was a success 5074- what my contributions to the project were 5075- people I've mentored (eg an intern!) 5076- project plans & documentation I've written 5077and send it to my manager. 5078 5079My manager's reaction: 5080Illustration of a smiling stick figure with no hair. 5081 5082THANK YOU! Having all this information makes it really easy for me to explain why your work is so great!</div> 5083 </a> 5084 5085 <a href="/comics/binary-search/"> 5086 <span class="title"> 5087 binary search 5088 </span> 5089 <div class="keywords"></div> 5090 <div class="transcript"></div> 5091 </a> 5092 5093 <a href="/comics/understand-the-bug/"> 5094 <span class="title"> 5095 understand the bug before trying to fix it 5096 </span> 5097 <div class="keywords">[debugging]</div> 5098 <div class="transcript"></div> 5099 </a> 5100 5101 <a href="/comics/css-grid-areas/"> 5102 <span class="title"> 5103 CSS grid areas 5104 </span> 5105 <div class="keywords"></div> 5106 <div class="transcript">### panel 1 5107 5108Illustration of a smiling stick figure with curly hair. 5109 5110person: CSS grid is a big topic, so I just want to show you one of my favourite grid features: areas! 5111 5112### let's say you want to build a layout 5113 5114Illustration of a long rectangle, labelled "header". Underneath it are two rectangles, side by side, labelled "sidebar" and "content" 5115 5116### `grid-template-areas` lets you define your layout in an almost visual way 5117 5118``` 5119grid-template-areas: 5120"header header" 5121"sidebar content" 5122``` 5123I think of it like this: 5124Illustration of a two rectangles side-by-side, both labelled "header". Underneath them are two rectangles, side by side, labelled "sidebar" and "content" 5125 5126### write your HTML 5127``` 5128<div class="grid"> 5129 <div class="top"></div> 5130 <div class="side"></div> 5131 <div class="main"></div> 5132</div> 5133``` 5134 5135### 2. define the areas 5136``` 5137.grid { 5138 display: grid; 5139 grid-template-columns: 200px 800px; 5140 grid-template-areas:"header header" 5141 "sidebar content"; 5142} 5143``` 5144 5145### 3. set grid-area 5146``` 5147.top {grid-area: header} 5148.side {grid-area: sidebar} 5149.main {grid-area: content} 5150``` 5151result: 5152Illustration of a long rectangle, labelled "`.top`". Underneath it are two rectangles, side by side, labelled "`.side`" and "`.main`"</div> 5153 </a> 5154 5155 <a href="/comics/caching-headers/"> 5156 <span class="title"> 5157 HTTP caching headers 5158 </span> 5159 <div class="keywords"></div> 5160 <div class="transcript">- ETag reponse header 5161- If-None-Match request header 5162- If-Modified-Since is similar to If-None-Match but with Last-Modified instead of ETag 5163 5164These 3 headers let the browser avoid downloading an unchanged file a second time. 5165 5166### initial request: 5167 5168browser, thinking: this page needs cats.css, let's request it! 5169browser: GET cats.css 5170server: 200 OK ETag:"ab23ef" (hash of the content) <the css file> 5171browser, thinking: OK, I'll save version ab23ef of cats.css in case I need it later 5172 5173### the next day: 5174 5175browser, thinking: cats.css! I've seen that file before. I'll ask if it's changed! 5176browser: GET cats.css If-None-Match:"ab23ef" (from the Etag) 5177server: 304 Not Modified 5178browser, thinking: yay I can use the old one, the page will load faster 5179 5180Vary: response header 5181 5182Sometimes the same URL can have multiple versions (Spanish, compressed or not, etc). Caches categorize the versions by request header like this: 5183 5184| Accept-Language | Accept-Encoding | content | 5185|--------------------|--------------------|----------------------------------------| 5186| en-US | - | hello | 5187| es-ES | - | hola | 5188| en-US | GZIP | f$xx99aef^.. (compressed gibberish) | 5189 5190The Vary header tells the cache which request headers should be the columns of this table. 5191 5192Cache-Control: request AND response header 5193 5194Used by both clients and servers to control caching behaviour. For example: 5195`Cache-Control: max-age=99999999999` from the server asks the CDN or browser 5196to cache the thing for a long time.</div> 5197 </a> 5198 5199 <a href="/comics/set-clear-expectations/">
5200 <span class="title"> 5201 set clear expectations 5202 </span> 5203 <div class="keywords">[manager]</div> 5204 <div class="transcript">I used to often get stressed out about whether the way I was prioritizing my work was reasonable. 5205 5206Illustration of a stick figure with short curly hair, looking uneasy. 5207 5208person: I'm spending a lot of time on X and no time on Y. I hope that's okay!!!! 5209 5210Everything got easier once I could just: 52111. come up for a plan for what to prioritize 52122. tell my manager the plan and ask if it sounds good 52133. trust them when they say yes 5214 5215Illustration of two stick figures talking. The employee has short curly hair, and the manager has no hair. 5216 5217employee: this quarter I'm planning to get BIG PROJECT done and spend time with my intern. I'm not planning to work on OTHER PROJECT at all. 5218manager: sounds good! Just do X too? 5219 5220Setting expectations is awesome because: 5221- I feel confident that my plans are reasonable 5222- my manager is aware of what I'm planning and can coordinate 5223 5224Everybody wins!!!</div> 5225 </a> 5226 5227 <a href="/comics/position-absolute/"> 5228 <span class="title"> 5229 position: absolute 5230 </span> 5231 <div class="keywords"></div> 5232 <div class="transcript">### `position: absolute;` doesn't mean absolutely positioned on the page... 5233``` 5234#star { 5235position: absolute; 5236top: 1em; 5237left: 1em; 5238} 5239``` 5240doesn't always place element at the top left of the page! 5241 5242### ... it's relative to the "containing block" 5243 5244the "containing block" is the closest ancestor with a `position` that isn't `static`, or the body if there isn't one. (`position: static` is the default) 5245 5246Illustration of a larger box, labelled "body", with a smaller box, labelled "`#star` nested inside it. The smaller box is off-centre within the larger box. The smaller box is labelled "this element has `position: relative` set" 5247 5248 5249### `top, bottom, left, right` will place an absolutely positioned element 5250``` 5251top: 50%; 5252bottom: 2em; 5253right: 30px; 5254left: -2in; 5255``` 5256"`left: -2in;`" is labelled "negative works too" 5257 5258Illustration of two overlapping boxes. The top of the smaller one is halfway down the height of the larger one. The gap between the tops of the two boxes is labelled "50%". The smaller one extends to the left of the larger one, representing "`left: -2in;`", and its right and bottom sides are nested inside the larger one, representing "`right: 30px;`" and "`bottom: 2em;`". 5259 5260 5261### left: 0; right: 0; != width: 100%; 5262`left: 0; right: 0;` 5263Illustration of two boxes. The smaller box is nested within the larger box. It is the same width as the larger box, and is aligned to the top of it. This illustration is labelled "left and right borders are both 0px away from containing block". 5264 5265`width: 100%;` 5266Illustration of two boxes. The smaller box is nested within the larger box, but its right edge extends past the right edge of the larger box. This illustration is labelled "width is the same as width of containing block". 5267 5268### absolutely positioned elements are taken out of the normal flow 5269 5270Illustration of two stick figures having a conversation. 5271 5272Person 1: will a parent element expand to fit an absolutely positioned 5273child? 5274Person 2: nope!</div> 5275 </a> 5276 5277 <a href="/comics/seccomp-bpf/"> 5278 <span class="title"> 5279 seccomp-bpf 5280 </span> 5281 <div class="keywords"></div> 5282 <div class="transcript">### all programs use system calls 5283program, represented by a box with a smiley face: read 2000 bytes from this file 5284 5285Linux, represented by a box with a smiley face:here you go! 5286 5287### rarely-used system calls can help an attacker 5288- `reboot` 5289- `request_key` 5290- `process_vm_ready` (read memory from another process) 5291 5292### seccomp-BPF lets you run a function before every system call 5293smiling stick figure with short curly hair: run this function before every syscall that process makes 5294 5295Linux, represented by a box with a smiley face: okay! 5296 5297### the function decides if that syscall is allowed 5298example function: 5299``` 5300if name in allowed_list { 5301 return true; 5302} 5303return false;
5304``` 5305`return false` means the syscall doesn't happen! 5306 5307### Docker blocks dozens of syscalls by default 5308Docker, represented by a box with a smiley face: most programs don't need those 5309system calls so I told Linux to block them for you! 5310 5311### 2 ways to block scary system calls 53121. limit the container's capabilities 53132. set a seccomp-bpf whitelist 5314 5315You should do both!</div> 5316 </a> 5317 5318 <a href="/comics/shell-arguments/"> 5319 <span class="title"> 5320 shell script arguments 5321 </span> 5322 <div class="keywords"></div> 5323 <div class="transcript">### panel 1: a script's arguments are in `$1`, `$2`, `$3`, etc 5324 5325``` 5326./script.sh panda banana 5327``` 5328 5329`$1` is `"panda"` and `$2` is `"banana"` 5330 5331 5332### panel 2: arguments are great for making simple scripts 5333 5334Here's a 1-line `svg2png` script that I use to convert SVGs to PNGs: 5335 5336``` 5337#!/bin/bash 5338inkscape "$1" -b white --export-png="$2" 5339``` 5340 5341I run it like this: 5342 5343``` 5344$ svg2png old.svg new.png 5345``` 5346 5347(arrow pointing to `"$2"`: "always quote your variables!") 5348 5349### panel 3: get all the arguments with `"${@}"` 5350 5351``` 5352ls --color "${@}" 5353``` 5354 5355 5356### panel 4: you can loop over arguments 5357 5358``` 5359for i in "${@}" 5360do 5361 echo "$i" 5362done 5363``` 5364 5365 5366### panel 5: 1 line shell scripts are great 5367 5368person: "I can write a tiny script so I don't have to remember a long command!" 5369 5370 5371</div> 5372 </a> 5373 5374 <a href="/comics/bash-if-statements/"> 5375 <span class="title"> 5376 bash if statements 5377 </span> 5378 <div class="keywords"></div> 5379 <div class="transcript">### the basic syntax 5380``` 5381if COMMAND then 5382 # do thing 5383else 5384 # do other thing 5385fi 5386``` 5387(you need a new line or ; before then) 5388 5389### `[` vs `[[` 5390there are 2 commands often used in if statements: `[` and `[[` 5391 5392`if [ -e file.txt ] ` 5393`/usr/bin/[` (aka `test`) is a program that returns 0 if the test you pass it succeeds 5394 5395`if [[ -e file.txt ]]` 5396`[[` is built into bash. It lets you do tests like `[[e x.txt && -e y.txt ]]` that wouldn't work with a command line tool 5397 5398 5399### `if COMMAND` 5400did `COMMAND` return 0? 5401 5402### if ! COMMAND 5403did `COMMAND` NOT return 0? 5404 5405### `if true` 5406`true` always returns 0 :) 5407 5408### `if [ -n "$var" ]` 5409is `$var` nonempty? 5410 5411### `if [ e file.txt ] ` 5412does `file.txt` exist? 5413 5414### combine with `&&` and `||` 5415`if [ -e file] && [ -e file2 ]` 5416 5417### `if [ -d somedir ]` 5418does `somedir` exist? 5419 5420### `if [ -x script.sh ] ` 5421is `script.sh` executable? 5422 5423### `man [` for more 5424you can do a lot!</div> 5425 </a> 5426 5427 <a href="/comics/window-functions-intro/"> 5428 <span class="title"> 5429 window functions 5430 </span> 5431 <div class="keywords"></div> 5432 <div class="transcript">Let's talk about an advanced SQL feature: window functions! 5433 5434Normally SQL expressions only (et you refer to information in a single row. 5435 5436`SELECT CONCAT(firstname, ' ', lastname) as full_name` 5437(`firstname` and `lastname` are 2 columns from the same row) 5438
5439person 1, bald and with a worried expression: can I refer to other rows though? Like subtract the value in the previous row? 5440 5441person 2, happy with curly hair: yes, with `*` window functions `*` 5442 5443Window functions are SQL expressions that let you reference values in other rows. The syntax (explained on the next page!) is: 5444 5445`[expression] OVER ([window definition])` 5446 5447Example: use `LAG()` to find how long since the last sale 5448 5449``` 5450SELECT item, 5451 day - LAG(day) OVER (ORDER BY day) 5452FROM sales 5453``` 5454 5455sales: 5456 5457| item | day | 5458|--------|-----| 5459| catnip | 2 | 5460| laser | 40 | 5461| tuna | 70 | 5462| tuna | 72 | 5463 5464 5465query output: 5466 5467| item | `day - LAG(day) OVER (ORDER BY day)` | 5468|--------|--------------------------------------| 5469| catnip | `NULL` (2 - `NULL`) | 5470| laser | 38 (40 - 2) | 5471| tuna | 30 (70 - 40) | 5472| tuna | 2 (72 - 2) | 5473 5474 5475They're part of `SELECT`, so they happen after `HAVING`: 5476 5477`FROM + JOIN` -> `WHERE` -> `GROUP BY` -> `HAVING` -> `SELECT` -> `ORDER BY` -> `LIMIT` 5478 5479(arrow pointing to `SELECT`) 5480happy little stick figure with curly hair: window functions are here!</div> 5481 </a> 5482 5483 <a href="/comics/why-same-origin-matters/"> 5484 <span class="title"> 5485 why the same-origin policy matters 5486 </span> 5487 <div class="keywords">[cors]</div> 5488 <div class="transcript"></div> 5489 </a> 5490 5491 <a href="/comics/change-one-thing-at-a-time/"> 5492 <span class="title"> 5493 debugging tip: change one thing at a time 5494 </span> 5495 <div class="keywords">[debugging]</div> 5496 <div class="transcript"></div> 5497 </a> 5498 5499 <a href="/comics/null-unknown-missing/"> 5500 <span class="title"> 5501 NULL: unknown or missing 5502 </span> 5503 <div class="keywords"></div> 5504 <div class="transcript">`NULL` is a special state in SQL. It's very commonly used as a placeholder for missing data ("we don't know her address!") 5505 5506What `NULL` means exactly depends on your data. For example, it's really important to know if `allergies IS NULL` means: 5507 5508- "no allergies" or 5509- "we don't know if she has allergies or not" 5510 5511`NULL` "should" mean "unknown" but it doesn't always. 5512 5513smiling stick figure with curly hair: it would be easier if `NULL` always meant the same thing but it really depends on your data! 5514 5515### where `NULL`s come from 5516 5517- There were already `NULL` values in the table 5518- The window function `LAG()` can return `NULL` 5519- You did a `LEFT JOIN` and some of the rows on the left didn't have a match for 5520 5521tiny pensive stick figure with curly hair: ooh, not very cat has an owner so sometimes the owner name is `NULL` 5522 5523### ways to handle NULL s * 5524 5525- Leave them in! 5526 5527smiling stick figure with curly hair: I'd rather see a `NULL` and know there's missing data than get misleading results 5528 5529- Filter them out! 5530`... WHERE first_name IS NOT NULL ...` 5531 5532- Use `COALESCE` or `CASE` to add a default value 5533 5534</div> 5535 </a> 5536 5537 <a href="/comics/css-isnt-design/"> 5538 <span class="title"> 5539 CSS isn't design 5540 </span> 5541 <div class="keywords"></div> 5542 <div class="transcript">### panel 1: web design is really hard 5543 5544Illustration of a stick figure with short curly hair, looking pensive. 5545 5546person (thinking): "wow, forms are way more complicated than I thought" 5547 5548### panel 2: writing CSS is also hard 5549 5550person (thinking): "ok, how exactly does flexbox work again?" 5551 5552### panel 3: remember that they're 2 different skills 5553 5554person (thinking): "hmm, I have NO IDEA what I want this site to look like, 5555maybe that's the problem and not CSS" 5556 5557### panel 4: CSS is easier when you have a good design 5558 5559Illustration of a box with smaller boxes arrayed inside it. 5560 5561person (thinking, and now smiling): "I can make it look like that!" 5562 5563### panel 5: usually you have to adjust the design 5564
5565person (thinking): "oh right, I didn't think about how that menu should look on desktop" 5566 5567### panel 6: sketching a design in advance can help! 5568 5569Illustration of a box with text reading "title", and a grid of smaller boxes underneath. 5570 5571even a simple sketch can help you think! 5572</div> 5573 </a> 5574 5575 <a href="/comics/group-by/"> 5576 <span class="title"> 5577 GROUP BY 5578 </span> 5579 <div class="keywords"></div> 5580 <div class="transcript">`GROUP BY` combines multiple rows into one row. Here's how it works for this table & query: 5581 5582``` 5583SELECT item, (COUNT(*), MAX(price) 5584FROM sales 5585GROUP BY item 5586``` 5587(COUNT(*), MAX(price) aggregates 5588 5589sales: 5590 5591| item | price | 5592|--------|-------| 5593| catnip | 5 | 5594| laser | 8 | 5595| tuna | 4 | 5596| tuna | 3 | 5597 5598 query output: 5599 5600| item | count | price | 5601|--------|-------|-------| 5602| catnip | 1 | 5 | 5603| laser | 1 | 8 | 5604| tuna | 2 | 4 | 5605 5606 56071. Split the table into groups for each value that you grouped by: 5608 5609item='catnip' 5610 5611| item | price | 5612|--------|------------| 5613| catnip | 5 | 5614 5615 5616item='laser' 5617 5618| item | price | 5619|-------|------------| 5620| laser | 8 | 5621 5622item='tuna' 5623 5624| item | price | 5625|------|------------| 5626| tuna | 4 | 5627| tuna | 3 | 5628 56292. Calculate the aggregates from the query for each group: 5630 5631| item | price | 5632|--------|------------| 5633| catnip | 5 | 5634 5635``` 5636COUNT(*)=1 5637MAX(price)=5 5638``` 5639 5640| item | price | 5641|-------|------------| 5642| laser | 8 | 5643 5644``` 5645COUNT(*)=1 5646MAX(price)=8 5647``` 5648 5649| item | price | 5650|------|------------| 5651| tuna | 4 | 5652| tuna | 3 | 5653 5654``` 5655COUNT(*)=2 5656MAX(price)=4 5657``` 5658 56593. Create a result set with 1 row for each group 5660 5661| item | count | price | 5662|--------|-------|-------| 5663| catnip | 1 | 5 | 5664| laser | 1 | 8 | 5665| tuna | 2 | 4 | 5666</div> 5667 </a> 5668 5669 <a href="/comics/whats-a-header/"> 5670 <span class="title"> 5671 what's a header? 5672 </span> 5673 <div class="keywords"></div> 5674 <div class="transcript">Every HTTP request and respnse has headers. Headers are a way for the browser or server to send extra information! 5675 5676Firefox: `Accept-Encoding:gzip` 5677 5678This means "I understand compressed responses" 5679 5680### Headers have a name and a value. 5681 5682`Accept-Encoding` is the name, `gzip` is the value. 5683 5684### Header names aren't case sensitive: 5685 5686`aCcEpT-eNcOdIng: gzip` is totally valid. 5687 5688### There are a few different kinds of headers: 5689 5690- Describe the body: 5691``` 5692Content-Type: image/png 5693Content-Encoding: gzip 5694Content-Length: 12345 5695Content-Language: es-ES 5696``` 5697- Ask for a specific kind of response 5698``` 5699Accept: image/png 5700Range: bytes=l-10 5701Accept-Encoding: gzip 5702Accept-Language: es-ES 5703``` 5704(Every Accept-header has a corresponding Content-header) 5705 5706- Manage caches: 5707``` 5708ETag: "abc123" 5709If-None-Match: "abc123" 5710Vary: Accept-Encoding 5711If-Modified-Since: 3 Aug 2019 13:00:00 GMT 5712Last-Modified: 3 Feb 2018 11:00:00 GMT 5713Expires: 27 Sep 2019 13:07:49 GMT 5714Cache-Control: public, max-age=300 5715``` 5716 5717- Say where the request comes from: 5718``` 5719User-Agent: curl 5720Referer: https://examplecat.com 5721``` 5722 5723- Cookies: 5724``` 5725Set-Cookie: name=julia; HttpOnly (server -> client) 5726Cookie: name=julia (client -> server) 5727``` 5728 5729and more!</div> 5730 </a> 5731 5732 <a href="/comics/debugger-features/"> 5733 <span class="title"> 5734 amazing debugging features 5735 </span> 5736 <div class="keywords">[debugging]</div> 5737 <div class="transcript"></div> 5738 </a> 5739 5740 <a href="/comics/understand-error-messages/"> 5741 <span class="title"> 5742 understand your error messages 5743 </span> 5744 <div class="keywords">[debugging]</div> 5745 <div class="transcript"></div> 5746 </a> 5747 5748 <a href="/comics/request-methods-1/"> 5749 <span class="title"> 5750 HTTP request methods 1 5751 </span> 5752 <div class="keywords"></div> 5753 <div class="transcript">Every HTTP request has a method. It's the first thing in the first line: 5754 5755`GET /cat.png HTTP/1.1` 5756 5757`GET` means it's a `GET` request 5758 5759There are 9 methods in the HTTP standard. 80% of the time you'll only use 2 (`GET` and `POST`). 5760 5761### `GET` 5762When you type an URL into your browser, that's a `GET` request. 5763 5764examplecat.com/cat.png 5765 5766client, represented by a box with a smiley face: 5767``` 5768GET /cat.png 5769Host: examplecat.com 5770``` 5771 5772server, also represented by a box with a smiley face: 5773``` 5774200 OK 5775Content-Type: image/png 5776 5777<the cat picture> 5778``` 5779 5780### `POST` 5781When you hit submit on a form, that's (usually) a `POST` request. 5782 5783client: 5784``` 5785POST /add_cat 5786Content-Type: application/json 5787 5788{"name": "mr darcy"} 5789``` 5790(`POST` requests usually have a request body) 5791 5792server: 5793``` 5794200 OK 5795Content-Type: text/html 5796 5797<after sign up page> 5798``` 5799 5800The big difference between `GET` and `POST` is that `GET`s are never supposed to change anything on the server. 5801 5802### `HEAD` 5803 5804Returns the same result as GET, but without the response body. 5805
5806client: 5807``` 5808HEAD /cat.png 5809``` 5810 5811server: 5812``` 5813200 OK 5814Content-Type: image/png 5815``` 5816(no image, just headers)</div> 5817 </a> 5818 5819 <a href="/comics/signals/"> 5820 <span class="title"> 5821 signals 5822 </span> 5823 <div class="keywords"></div> 5824 <div class="transcript">### If you've ever used kill you've used signals 5825person, angrily: DIE!!! 5826process, sad: okay 5827 5828### the Linux kernel sends processes signals in lots of situations 5829- your child terminated 5830- the timer you set expired 5831- that pipe is closed 5832- illegal instruction 5833- segmentation fault 5834 5835### you can send signals yourself with the kill system call or command 5836 5837``` 5838SIGINT Ctrl-C 5839SIGTERM kill 5840SIGKILL kill -9 5841SIGHUP kill -HUP 5842``` 5843(various levels of "die") 5844`SIGHUP` is often interpreted as "reload config", e.g. by nginx. 5845 5846### Every signal has a default action, which is one of: 5847- ignore 5848- kill process 5849- kill process AND make core dump file 5850- stop process 5851- resume process 5852 5853### Your program can set Custom handlers for almost any signal 5854person: `SIGTERM` (terminate) 5855process: okay! I'll (clean up and then exit! 5856 5857exceptions: 5858`SIGSTOP` & `SIGKILL` can't be ignored 5859
5860dead program: got `SIGKILL`ed 5861 5862### Signals can be hard to handle correctly since they can happen at ANY time 5863process: handling a signal 5864person: SURPRISE! another signal!</div> 5865 </a> 5866 5867 <a href="/comics/pid-namespaces/"> 5868 <span class="title"> 5869 PID namespaces 5870 </span> 5871 <div class="keywords"></div> 5872 <div class="transcript">### the same process has different PIDs in different namespaces 5873PID in host / PID in container 587423512 / 1 (PID 1 is special) 587523513 / 4 587623518 / 12 5877 5878### PID namespaces are in a tree 5879 5880Diagram showing "host PID namespace (the root)" with three arrows coming down from it, each pointing to a label that says "child". 5881 5882Often the tree is just 1 level deep (every child is a container) 5883 5884### you can see processes in child PID namespaces 5885Illustration of a host, represented by a box with heart eyes and a big smile. 5886 5887host: aw! look at all those containers running! 5888 5889### if PID 1 exits, everyone gets killed 5890Illustration of PID 1, represented by a box with a smiley face, and Linux, represented by its penguin mascot. 5891 5892PID 1: ok I'm done! 5893Linux: I'm kill -q'ing everyone else in this PID namespace IMMEDIATELY 5894 5895### Killing PID 1 accidentally would be bad 5896Illustration of a container process, represented by a box with a smiley face, and Linux, represented by its penguin mascot. 5897 5898container process: `kill 1` 5899Linux: do you WANT everyone to die? I'm not gonna let you do that 5900 5901### rules for signaling PID 1 5902- from same container: only works if the process has set a signal handler 5903- from the host: only SIGKILL and SIGSTOP are ok, or if there's a signal handler</div> 5904 </a> 5905 5906 <a href="/comics/bugs-teach-you/"> 5907 <span class="title"> 5908 let your bugs teach you 5909 </span> 5910 <div class="keywords">[debugging]</div> 5911 <div class="transcript"></div> 5912 </a> 5913 5914 <a href="/comics/http-apis/"> 5915 <span class="title"> 5916 using HTTP APIs 5917 </span> 5918 <div class="keywords"></div> 5919 <div class="transcript">Lots of services (Twitter! Twilio! Google!) let you use them by sending them HTTP requests. If an HTTP API doesn't come with a client library, don't be scared! You can just make the HTTP requests yourself. Here's what you need to remember: 5920 5921### Set the right `ContentâType` header 5922Often you'll be sending a POST request with a body, and that means you need a `ContentâType` header that matches the body. 5923 5924The 2 main options are: 5925- `application/json` (JSON!) 5926- `application/x-www-form-urlencoded` (same as what an HTML form does) 5927 5928If you don't set the `ContentâType`, your request won't work. 5929 5930Smiling stick figure with short curly hair: a common error is to try to send POST data as one content type (like JSON) when it's actually another (like application/x-www-form-urlencoded) 5931 5932### Identify yourself 5933Most HTTP APIs require a secret API key so they know who you are. 5934Here's how that looks for the Twilio API: 5935``` 5936curl 5937https://api.twilio.com/2010-04-01/Accounts/ACCOUNT_ID/Messages.json 5938 -H "Content-Type: application/json" 5939 -u CCOUNT_ID:AUTH_TOKEN 5940 -d '{ 5941 "from": "+15141234567", 5942 "to": "+15141234567", 5943 "body": "a text message" 5944}' 5945``` 5946(this sends a POST request) 5947 5948`u ACCOUNT_ID:AUTHO_TOKEN` sends the username/password in the Authorization </div> 5949 </a> 5950 5951 <a href="/comics/conntrack/"> 5952 <span class="title"> 5953 conntrack 5954 </span> 5955 <div class="keywords"></div> 5956 <div class="transcript">### conntrack 5957not a command line tool: 5958 5959it's a Linux kernal system for tracking TCP/UDP connections. 5960 5961It's a kernel module called `nf-conntrack` 5962 5963### conntrack is used for: 5964- NAT (in a router!) 5965- firewalls (eg only allow outbound connections) 5966 5967You control it with iptables rules. 5968 5969### conntrack has a table of every connection 5970Each entry contains: 5971- src + dest IP 5972- src + dest ports 5973- the connection state (eg `TIME_WAIT`) 5974 5975### how to enable conntrack 5976enable: 5977`$ sudo modprobe nf_conntrack` 5978 5979check if it's enabled: 5980`$lsmod | grep conntrack` 5981 5982change table size with the sysct | 5983`net.netfilter.nf_conntrack_max` 5984 5985### if the conntrack table gets full, no new connections can start 5986smiling rectangle: hello? 5987(SYN packet gets dropped) 5988sad rectangle: silence 5989 5990### moral: be careful about enabling conntrack! 5991sad stick person with curly hair: why are connections mysteriously failing? 5992happy stick figure with medium length straight hair: maybe the conntrack table is full!</div> 5993 </a> 5994 5995 <a href="/comics/ways-to-count/">
5996 <span class="title"> 5997 SQL: ways to count 5998 </span> 5999 <div class="keywords"></div> 6000 <div class="transcript"></div> 6001 </a> 6002 6003 <a href="/comics/bash-functions/"> 6004 <span class="title"> 6005 bash functions 6006 </span> 6007 <div class="keywords"></div> 6008 <div class="transcript">### panel 1: defining functions is easy 6009 6010 6011``` 6012say_hello() { 6013 echo "hello!" 6014} 6015``` 6016 6017and so is calling them: 6018 6019``` 6020say_hello 6021``` 6022 6023(no parentheses when calling a function! 6024 6025### panel 2: functions have exit codes 6026 6027``` 6028failing_function () { 6029 return 1 6030} 6031``` 6032 6033`0` is a success, everything else is a failure. A program's exit codes work the same way -- 0 is success, everything else is failure. 6034 6035 6036### panel 3: you can't return a string 6037 6038you can only return an exit code from 0 to 255 6039 6040 6041### panel 4: arguments are `$1`, `$2`, `$3`, etc 6042 6043``` 6044say_hello() { 6045 echo "Hello, $1!" 6046} 6047say_hello "Ahmed" 6048``` 6049 6050the above code prints `Hello, Ahmed!`. Again, `say_hello "Ahmed"`, not `say_hello("Ahmed")` 6051 6052### panel 5: The `local` keyword declares local variables 6053 6054``` 6055say_hello() { 6056 local x 6057 x=$(date) # this is a local variable 6058 y=$(date) # this is a global variable 6059} 6060``` 6061 6062### panel 6: `local x=VALUE` suppresses errors 6063 6064this line never fails, even if `asdf` doesn't exist: 6065 6066``` 6067local x=$(asdf) 6068``` 6069 6070but this will fail (as you would expect) -- if you have `set -e` set, it'll stop the program 6071 6072``` 6073local x 6074x=$(asdf) # this line will fail 6075``` 6076 6077person: "I really have NO IDEA why it's like this, bash is weird sometimes" 6078</div> 6079 </a> 6080 6081 <a href="/comics/trap/"> 6082 <span class="title"> 6083 trap 6084 </span> 6085 <div class="keywords"></div> 6086 <div class="transcript">### when your script exits, sometimes you need to clean up 6087nonplussed stick figure with short curly hair: oops, the script I created a bunch of temp files I want to delete 6088 6089### `trap` sets up callbacks 6090`trap COMMAND EVENT` 6091 6092COMMAND: what command to run 6093EVENT: when to run the command 6094 6095### bash runs COMMAND when EVENT happens 6096`trap "echo 'hi!!!'" INT` 6097 6098OS, represented by a box with a smiley face: <sends `SIGINT` signal> 6099bash, also represented by a box with a smiley face: ok, time to print out `hi!!!!` 6100 6101### events you can trap 6102- unix signals (`INT`, `TERM`, etc) 6103- the script exiting (`EXIT`) 6104- every line of code (`DEBUG`) 6105- function returns (`RETURN`) 6106 6107### example: kill all background processes when Ctrl+C is pressed 6108`trap "kill $(jobs -p)" INT` 6109 6110when you press `CTRL+C`, the OS sends the script a `SIGINT` signal 6111 6112### example: cleanup files when the script exits 6113``` 6114function cleanup() { 6115 rm -rf $TEMPDIR 6116 rm $TEMPFILE 6117} 6118trap cleanup EXIT 6119``` 6120EXIT is a fake "signal" that triggers on exit</div> 6121 </a> 6122 6123 <a href="/comics/container-networking/"> 6124 <span class="title"> 6125 container networking 6126 </span> 6127 <div class="keywords"></div> 6128 <div class="transcript"></div> 6129 </a> 6130 6131 <a href="/comics/figure-out-great-at/"> 6132 <span class="title"> 6133 figure out what your manager is great at 6134 </span> 6135 <div class="keywords">[manager]</div> 6136 <div class="transcript">Different managers are good at different things! I've worked with managers who are amazing at: 6137 6138Each of these items is enclosed in a thought bubble. 6139 6140- product design 6141- helping people resolve conflicts 6142- understanding the business 6143- building remote teams 6144- prioritizing ruthlessly 6145- running meetings 6146- solving tricky technical problems 6147- organizational politics 6148 6149Not every manager is good at every single thing, and that's okay! I like to figure out what my manager is awesome at and lean on them for those things. (heart) 6150 6151Also, strengths change over time! If they're not good at something today, maybe check back in a year & see if that's changed.</div> 6152 </a> 6153 6154 <a href="/comics/container-registries/"> 6155 <span class="title"> 6156 container registries 6157 </span> 6158 <div class="keywords"></div> 6159 <div class="transcript">### sharing container images is useful 6160smiling stick figure with curly hair: I made an image you can use to run Redis with
6160just one command! 6161 6162smiling bald stick figure: yay! 6163 6164### a registry iS a server that serves images 6165 6166images have an ID, like "leff92" 6167 6168and sometimes a tag, like "18.04" or "latest" 6169 6170### registries let you download just the layers you need 6171client, represented by a box with a smiley face: I already have the Ubuntu base image, I just need `0fe223` 6172 6173registry, represented by a smaller box with a smiley face: here's `0fe223`! 6174 6175### there are public container registries... 6176person: I'm going to use the latest official public Redis image to test my code! 6177 6178### ... and private registries 6179developer at COMPANY, represented by a smiling bald stick figure: every time we build our web service, we upload a new image to our private registry 6180 6181### be careful where your container images come from 6182smiling stick figure: I'll just run this image from RANDOM_PERSON 6183 61842 months later: oh no! RANDOM_PERSON is mining bitcoin on my server 6185</div> 6186 </a> 6187 6188 <a href="/comics/dns-server-types/"> 6189 <span class="title"> 6190 the 4 types of DNS servers 6191 </span> 6192 <div class="keywords">[dns]</div> 6193 <div class="transcript"></div> 6194 </a> 6195 6196 <a href="/comics/subqueries/"> 6197 <span class="title"> 6198 subqueries 6199 </span> 6200 <div class="keywords"></div> 6201 <div class="transcript">Some questions can't be answered with one Simple SQL query. 6202 6203For example, this query finds owners who have named their dogs popular names: ("boring" owners :)) 6204 6205dogs: 6206 6207| owner | name | 6208|----------|-----------| 6209| ken | darcy | 6210| bob | darcy | 6211| bob | lassie | 6212| ahmed | darcy | 6213| sara | floof | 6214| sara | lassie | 6215 6216``` 6217SELECT owner 6218FROM dogs 6219WHERE name in SELECT name 6220 FROM dogs 6221 GROUP BY name 6222 HAVING count(*) > 2) 6223``` 6224the subquery evaluates to ('darcy') 6225 6226query output: 6227 6228| owner | 6229|----------| 6230| ken | 6231| bob | 6232| ahmed | 6233 6234 6235### common table expressions 6236 6237"Common table expressions" (or CTEs) let you name a query so people reading it can understand what it's for. 6238 6239Here's the query above rewritten using a CTE: 6240``` 6241WITH popular_dog_names AS ( 6242 SELECT name 6243 FROM dogs 6244 GROUP BY name 6245 HAVING count(*) > 2 6246) 6247SELECT owner 6248FROM dogs INNER JOIN popular_dog_names 6249 ON dogs.name = popular_dog_names.name 6250``` 6251### Where you can use a. subquery/CTE 6252 6253#### in a `FROM` 6254``` 6255SELECT ..... 6256FROM (<subquery or CTE>) 6257GROUP BY ..... 6258``` 6259 6260#### in a `WHERE` 6261``` 6262SELECT ... 6263WHERE name IN (<subquery>) 6264``` 6265</div> 6266 </a> 6267 6268 <a href="/comics/stacking-contexts/"> 6269 <span class="title"> 6270 stacking contexts 6271 </span> 6272 <div class="keywords"></div> 6273 <div class="transcript">### a z-index can push an element up/down... 6274``` 6275.first { 6276 z-index: 3; 6277} 6278. second { 6279 z-index: 0; 6280} 6281``` 6282Illustration of two boxes. The one labelled "`.first`" is layered over top of the other one. 6283 6284### TRY ME: but a higher z-index doesn't always put an element on top 6285 6286Illustration of a box labelled "`z-index: 0`". On top of that is a box labelled "`z-index: 10`". Another box is on top of that one. Layered over top of all of these is a box labelled "`z-index: 2`". 6287 6288`z-index: 2` is on top! why? 6289 6290 6291### every element is in a stacking context 6292 6293The same illustration as the previous panel, but a label pointing to both the "`z-index: 10`" and "`z-index: 2`" boxes says, "these 2 elements are in different 6294stacking contexts" 6295 6296### a stacking context is like a Photoshop layer 6297 6298Illustration of two boxes, each with three smiley faces and an "ok" button in it, one layered on top of the other. These are labelled "two 'layers'". 6299 6300by default, an element's children share its stacking context 6301 6302### setting z-index creates a stacking context 6303``` 6304#modal { 6305 z-index: 5; 6306 position: absolute; 6307} 6308``` 6309this is a common way to create a stacking context 6310 6311### stacking contexts are confusing 6312You can do a lot without understanding them at all. But if `z-index` ever isn't working the way you expect, that's the day to learn about stacking contexts (smiley face)</div> 6313 </a> 6314 6315 <a href="/comics/where/">
6316 <span class="title"> 6317 WHERE 6318 </span> 6319 <div class="keywords"></div> 6320 <div class="transcript">`WHERE` filters the table you start with. For example, let's break down this query that finds all owners with cats named "daisy" 6321 6322``` 6323SELECT owner 6324FROM cats 6325WHERE name = 'daisy' 6326``` 6327`FROM cats` is pulling from a database with tables of cats and people. 6328 6329| owner | name | 6330|-------|------------| 6331| 1 | daisy | 6332| 1 | dragonsnap | 6333| 3 | buttercup | 6334| 4 | rose | 6335 6336`WHERE name = 'daisy' ` 6337 6338| owner | name | 6339|-------|------------| 6340| 1 | daisy | 6341 6342`SELECT owner` 6343 6344| owner | 6345|-------| 6346| 1 | 6347 6348 6349## What you can put in a WHERE : 6350 6351### `expr LIKE `...` 6352Check if a string contains a substring! 6353`WHERE name LIKE '%darcy%'` 6354 6355(% is a wildcard, like * in your shell) 6356 6357### `exprIN (...)` 6358Check if an expression is in a fist of values 6359`WHERE name IN ('bella', 'simba')` 6360 6361### `=, !=, <, >=` 6362these work the way you'd guess, except when `NULL` is involved. 6363 6364`WHERE revenue - costs >=0` 6365 6366### `expr IS NULL`, `expr- IS NOT NULL` 6367more about NULL on pages 15-17 6368 6369= NULL (crossed out) 6370IS NULL (circled) 6371 6372### `AND, OR, NOT` 6373 6374You can `AND` together as many conditions as you want 6375 6376tiny little illustration of a smiling stick figure with curly hair: If I'm using lots of ANDs, like to write them like this: 6377``` 6378(....) 6379AND (....) 6380AND (....) 6381``` 6382(put all the ORs in the parentheses) 6383</div> 6384 </a> 6385 6386 <a href="/comics/padding-margin/"> 6387 <span class="title"> 6388 padding syntax 6389 </span> 6390 <div class="keywords"></div> 6391 <div class="transcript">### there are 4 ways to set padding 6392`padding: 1em;` 6393(all sides) 6394 6395`padding: 1em 2em;` 6396(first value is vertical, second is horizontal) 6397 6398`padding: 1em 2em 3em;` 6399(first value is top, second is horizontal, third is bottom) 6400 6401`padding: 1em 2em 3em 4em;` 6402(first value is top, second is right, third is bottom, fourth is left) 6403 6404### tricks to remember the order 64051. trouble 6406top 6407right 6408left 6409bottom 6410 64112. it's clockwise 6412 6413### you can also set padding on just 1 side 6414``` 6415padding-top: 1em; 6416padding-right: 10px; 6417padding-bottom: 3em; 6418padding-left: 4em; 6419``` 6420 6421### TRY ME: differences between padding & margin 6422- padding is "inside" an element: the background color covers the padding, you can click padding to click an element, etc. Margin is "outside". 6423- you can center with margin: auto, but not with padding 6424- margins can be negative, padding can't 6425 6426### margin syntax is the same as padding 6427`border-width` also uses the same order: 6428top, right, bottom, left</div> 6429 </a> 6430 6431 <a href="/comics/cname/"> 6432 <span class="title"> 6433 CNAME records 6434 </span> 6435 <div class="keywords">[dns]</div> 6436 <div class="transcript">### there are 2 ways to set up DNS for a website 64371. set an A record with an IP 6438`www.cats.com A 1.2.3.4` 6439 64402. set a CNAME record with a domain name 6441`www.cats.com CNAME cats.github.io` 6442 6443 6444### CNAME records redirect every DNS record, not just the IP 6445I like to use them whenever possible so that if my web host's IP changes, I don't need to change anything! 6446 6447### what actually happens during a CNAME redirect 6448Illustration of a conversation between a resolver, represented by a box with a smiley face holding a magnifying glass, and an authoritative nameserver, represented by a box with a smiley face wearing a crown. 6449 6450resolver: what's the A record for `www.cats.com`? 6451authoritative nameserver: `www.cats.com CNAME cats.github.io` 6452resolver (thinking): okay, I'll look up the A record for `cats.github.io`! 6453 6454### rules for when you can use CNAME records 64551. you can only set CNAME records on subdomains (like `www.example.com`), not root domains (like `example.com`) 64562. if you have a CNAME record for a subdomain, that subdomain can't have any other records 6457 6458(technically you can ignore these rules, but it can cause problems, the RFCs say you shouldn't, and many DNS providers enforce these rules) 6459 6460### some DNS providers have workarounds to support CNAME for root domains 6461Look up "CNAME flattening" or "ANAME" to learn more.</div> 6462 </a> 6463 6464 <a href="/comics/dns-record-types/">
6465 <span class="title"> 6466 DNS record types 6467 </span> 6468 <div class="keywords">[dns]</div> 6469 <div class="transcript"></div> 6470 </a> 6471 6472 <a href="/comics/talk-about-problems/"> 6473 <span class="title"> 6474 talk about problems early 6475 </span> 6476 <div class="keywords">[manager]</div> 6477 <div class="transcript">### Every so often I'll start with a small problem 6478Illustration of a stick figure with short curly hair, looking nonplussed. 6479 6480employee: hmm this isn't great 6481 6482### and forget to talk about it until I'm REALLY MAD 6483Illustration of a stick figure with short curly hair, looking very upset, and another stick figure, the manager, who has medium length straight hair, and looks confused, with question marks over their head. 6484 6485employee: THIS IS TERRIBLE 6486manager, thinking: whoa where did that come from? 6487 6488### It's way better to bring up a problem early and figure it out before it turns into a big deal! 6489Illustration of a stick figure with short curly hair, looking nonplussed, and their manager, a stick figure with medium length straight hair, who is smiling. 6490 6491employee: I got paged 15 times this week, can we talk about how to improve this? 6492manager: yes let's work on that!</div> 6493 </a> 6494 6495 <a href="/comics/how-to-read-error/"> 6496 <span class="title"> 6497 how to read an error message 6498 </span> 6499 <div class="keywords">[debugging]</div> 6500 <div class="transcript"></div> 6501 </a> 6502 6503 <a href="/comics/why-dns/"> 6504 <span class="title"> 6505 why we need DNS 6506 </span> 6507 <div class="keywords">[dns]</div> 6508 <div class="transcript"></div> 6509 </a> 6510 6511 <a href="/comics/flexbox-basics/"> 6512 <span class="title"> 6513 flexbox basics 6514 </span> 6515 <div class="keywords"></div> 6516 <div class="transcript">### display: flex; 6517set on a parent element to lay out its children with a flexbox layout. 6518 6519by default, it sets `flex-direction: row;` 6520 6521 6522### flex-direction: row; 6523Illustration of three boxes, one with a star, one with a heart, and one with a starburst. They are side-by-side in a single row. 6524 6525by default, children are laid out in a single row. 6526 6527the other option is `flex-direction: column` 6528 6529### flex-wrap: wrap; 6530Illustration of three boxes, one with a star, one with a heart, and one with a starburst. The star and heart boxes are side-by-side, then an arrow winds around to the starburst box, which is underneath the other two, aligned to the left. 6531 6532will wrap instead of shrinking everything to fit on one line 6533 6534### justify-content: center; 6535Illustration of three boxes, one with a star, one with a heart, and one with a starburst. The star and heart boxes are side-by-side. The starburst box is centred underneath them. 6536 6537horizontally center (or vertically if you've set `flex-direction: column`) 6538 6539### align-items: center; 6540Illustration of three boxes, one with a star, one with a heart, and one with a starburst. The boxes are different heights, and are placed side-by-side in a single row, centred horizontally. 6541 6542vertically center (or horizontally if you've set `flex-direction: column`) 6543 6544### you can nest flexboxes 6545 6546A box labelled `display: flex`. Inside it are two smaller boxes, side-by-side. Each is also labelled `display: flex`. One of the smaller boxes has three boxes side-by-side in it. The other smaller box has three boxes stacked on top of one another, inside it.</div> 6547 </a> 6548 6549 <a href="/comics/probably-seen-bug-before/"> 6550 <span class="title"> 6551 debugging tip: you've probably seen this bug before 6552 </span> 6553 <div class="keywords">[debugging]</div> 6554 <div class="transcript"></div> 6555 </a> 6556 6557 <a href="/comics/write-a-test/"> 6558 <span class="title"> 6559 debug by writing a test 6560 </span> 6561 <div class="keywords">[debugging]</div> 6562 <div class="transcript"></div> 6563 </a> 6564 6565 <a href="/comics/subdomains/"> 6566 <span class="title"> 6567 subdomains 6568 </span> 6569 <div class="keywords">[dns]</div> 6570 <div class="transcript">
6570### to make a subdomain, you just have to set a DNS record! 6571 6572To set up cats.yourdomain.com, create a DNS record like this in your authoritative nameservers: 6573 6574cats.yourdomain.com A 1.2.3.4 6575 6576yourdomain is the name 6577A is the record type 65781.2.3.4 is the value 6579 6580 6581### there are 2 ways a nameserver can handle subdomains 65821. Store their DNS records itself 6583 6584nameserver, represented by a box with a smiley face wearing a crown: here's the IP for cats.yourdomain.com! 6585 65862. Redirect to another authoritative nameserver 6587 6588(this happens if you set an NS record for the subdomain, it's called "delegation") 6589 6590nameserver: ask this other DNS server instead! 6591 6592### you can create multiple levels of subdomains 6593For example, you can make: 6594a.b.c.d.e.f.g.example.com 6595 6596up to 127 levels is allowed! 6597 6598### www is a common subdomain 6599Usually www.yourdomain.com and yourdomain.com point to the exact same IP address. 6600 6601If you wanted to confuse people, you could make them totally different websites! 6602 6603### panel 5 6604Illustration of a smiling stick figure with curly hair. 6605 6606person: I love using subdomains for my projects (like dns-lookup.jvns.ca) because they're free, I can give a subdomain a different IP, and it keeps projects separate. 6607</div> 6608 </a> 6609 6610 <a href="/comics/tld/"> 6611 <span class="title"> 6612 top-level domains 6613 </span> 6614 <div class="keywords">[dns]</div> 6615 <div class="transcript"></div> 6616 </a> 6617 6618 <a href="/comics/intermittent-bugs/"> 6619 <span class="title"> 6620 how to handle intermittent bugs 6621 </span> 6622 <div class="keywords">[debugging]</div> 6623 <div class="transcript"></div> 6624 </a> 6625 6626 <a href="/comics/registrar/"> 6627 <span class="title"> 6628 picking a domain registrar 6629 </span> 6630 <div class="keywords">[dns]</div> 6631 <div class="transcript"></div> 6632 </a> 6633 6634 <a href="/comics/authoritative-nameservers/"> 6635 <span class="title"> 6636 authoritative nameservers 6637 </span> 6638 <div class="keywords">[dns]</div> 6639 <div class="transcript">### your domain has dns records 6640 6641`example.com A 1.2.3.4 300` 6642`example.com` is the name 6643`A` is the type of record 6644`1.2.3.4` is the value 6645`300` is the TTL 6646 6647### these records are cached on lots of servers 6648 6649server 8.8.8.8, thinking: I was told example.com's IP is `1.2.3.4` 6650 6651but when that cache expires... 6652 6653### the only source of truth is your authoritative nameserver 6654 6655your nameserver: I have all example.com's DNS records! 6656 6657### how to get a domain's authoritative nameserver: ask its TLD namesever 6658 6659person: who's the authority for example.com? 6660.com nameserver: `b.iana-servers.net` 6661 6662### here's how to look up example.com's nameserver 6663 6664Run this: 6665``` 6666$ dig ns example.com 6667g.gtld-servers.net 6668``` 6669`g.gtld-servers.net` is one of the .com nameservers 6670 6671### you can update your nameserver on your registrar's website 6672 6673person: hey I want to use a different nameserver 6674registrar: I'll tell the TLD nameservers!</div> 6675 </a> 6676 6677 <a href="/comics/make-code-easy-to-debug/"> 6678 <span class="title"> 6679 make your code easy to debug 6680 </span> 6681 <div class="keywords">[debugging]</div> 6682 <div class="transcript"></div> 6683 </a> 6684 6685 <a href="/comics/work-with-manager-promoted/"> 6686 <span class="title"> 6687 work with your manager to get promoted 6688 </span> 6689 <div class="keywords">[manager]</div> 6690 <div class="transcript">Where I work, my manager wants people on the team to get promoted. If people are being promoted, it (hopefully) means that they're growing & getting more awesome at their jobs, which makes the team's manager look good! 6691 6692Illustration of a smiling stick figure with short curly hair. 6693 6694person, thinking: huh, maybe promotions are just a normal thing we can have a conversation about? 6695 6696Some ways to start conversations: 6697- can we walk through the expectations for the next level to make sure I understand them? 6698- what areas do you think I should focus on? 6699- if I accomplished X Y Z, do you think that would be enough to get promoted? 6700 6701 6702If this is something you care about, keep checking in periodically! The person who cares the most about your career is you â¡â¡</div> 6703 </a> 6704 6705 <a href="/comics/conversations-constructive/">
6706 <span class="title"> 6707 keep conversations mostly constructive 6708 </span> 6709 <div class="keywords"></div> 6710 <div class="transcript">I've had periods with some managers where, every time we talk, we're talking about SOME problem: 6711 6712Two illustrations of the same stick figure with curly hair, looking unhappy. 6713 6714me: why did y happen? 6715me: X has been a problem for a year and it's STILL not fixed 6716 6717These days, I try to bring up problems that I'm interested in fixing and bring ideas for solutions when I can. Often we just talk about our work: 6718 6719Each item is illustrated with a smiling stick figure with curly hair saying them. 6720 6721- here's an idea I had... 6722- my intern is doing awesome work! 6723- did you see that great thing this other team did? 6724- here's an interesting bug from this past week... 6725- I thought of an onboarding project for the new person! 6726 6727Sometimes venting can be useful too, though! If there's a problem, it's often helpful to bring it up even if I don't have a solution.</div> 6728 </a> 6729 6730 <a href="/comics/mental-model/"> 6731 <span class="title"> 6732 debugging tip: build your mental model 6733 </span> 6734 <div class="keywords">[debugging]</div> 6735 <div class="transcript"></div> 6736 </a> 6737 6738 <a href="/comics/sha-same-code/"> 6739 <span class="title"> 6740 a SHA always refers to the same code 6741 </span> 6742 <div class="keywords"></div> 6743 <div class="transcript">Let's start with some fundamentals! If you understand the basics about how git works, it's WAY easier to fix mistakes. So let's explain what a git commit is! 6744 6745Every git commit has an id like 3f29abcd233fa, also called a SHA ("Secure Hash Algorithm"). 6746 6747A SHA refers to both: 6748- the changes that were made in that commit (see them with ```git show```) 6749- a snapshot of the code after that commit was made 6750 6751No matter how many weird things you do with git, checking out a SHA will always give you the exact same code. It's like saving your game so that you can go back if you die You can check out a commit like this: 6752```git checkout 3f29abk``` 6753 6754SHAS are long but you can just use the first 6 chars 6755 6756This makes it way easier to recover from mistakes! 6757 6758person at 10 am: ok, let's commit, that's a2992b 6759person at 11 am: I really screwed up this file, let's go back to the version from a2992b</div> 6760 </a> 6761 6762 <a href="/comics/request-methods-2/"> 6763 <span class="title"> 6764 HTTP request methods 2 6765 </span> 6766 <div class="keywords"></div> 6767 <div class="transcript">### OPTIONS 6768 6769`OPTIONS` is mostly used for `CORS` requests. The `CORS` page has more about that. 6770It also tells you which methods are available. 6771 6772### DELETE 6773 6774Used in many APIs (like the Stripe API) to delete resources. 6775 6776box with a smiley face 1: `DELETE /v1/customers/cus_12345` 6777à¼("delete this customer please!") 6778 6779box with a smiley face 2: `200 OK` 6780("deleted!") 6781 6782### PUT 6783 6784Used in some APIs (like the S3 API) to create or update resources. `PUT /cat/1234` lets you `GET /cat/1234` later. 6785 6786### PATCH 6787Used in some APIs for partial updates to a resource ("just change this 1 field"). 6788 6789### TRACE 6790I've never seen a server that supports this, you probably don't need to know about it. 6791 6792### CONNECT 6793Different from all the others: instead of making a request to a server directly, it asks for a proxy to open a connection. 6794 6795If you set the `HTTPS_PROXY` environment variable to a proxy server, many HTTP libraries will use this protocol to proxy your requests. 6796 6797client, represented by a box with a smiley face: 6798`CONNECT test.com` 6799`$AFO XXRTZ` 6800(encrypted request) 6801 6802proxy, also represented by a box with a smiley face, thinking: ok, I'll open a connection to test.com. 6803 6804proxy: `$AFO XXRTZ` 6805 6806test.com, represented by a box with a smiley face: [is here]</div> 6807 </a> 6808 6809 <a href="/comics/build-support-system/"> 6810 <span class="title"> 6811 build the support system you need 6812 </span> 6813 <div class="keywords"></div> 6814 <div class="transcript">The flip side of "figure out what things they're great at" is that there are always going to be things your manager I can't help you with. When that happens, there are a few choices: 68151. Get mad that they can't help 68162. Resign yourself to not getting help with those things 68173. Find help elsewhere!!! 6818 6819Lara Hogan (her blog is GREAT) has an amazing blog post called "When your manager isn't supporting you, build a Voltron" about building a crew of people with lots of different skills who you can ask for help! S
6819ome of her tips: 6820- figure out what you need help with before asking. Use their time well!` 6821- focus on problem solving, not venting 6822 6823Illustration of a big cool robot with wings, holding a big sword. Various parts of its body are labelled with the points below. 6824 6825A Voltron is a robot built out of several other robots 6826- works in a different field 6827- awesome at communication 6828- more experience than me 6829 6830bit.ly/managervoltronbingo has a useful bingo card!</div> 6831 </a> 6832 6833 <a href="/comics/receiving-email/"> 6834 <span class="title"> 6835 receiving email at your domain 6836 </span> 6837 <div class="keywords">[dns]</div> 6838 <div class="transcript"></div> 6839 </a> 6840 6841 <a href="/comics/domain-privacy/"> 6842 <span class="title"> 6843 domain privacy 6844 </span> 6845 <div class="keywords">[dns]</div> 6846 <div class="transcript"></div> 6847 </a> 6848 6849 <a href="/comics/check-assumptions-2/"> 6850 <span class="title"> 6851 debugging tip: more assumptions to check 6852 </span> 6853 <div class="keywords">[debugging]</div> 6854 <div class="transcript"></div> 6855 </a> 6856 6857 <a href="/comics/get-specific-about-bug/"> 6858 <span class="title"> 6859 debugging tip: get specific about what the bug is 6860 </span> 6861 <div class="keywords">[debugging]</div> 6862 <div class="transcript"></div> 6863 </a> 6864 6865 <a href="/comics/how-to-give-good-feedback/"> 6866 <span class="title"> 6867 how to give good feedback 6868 </span> 6869 <div class="keywords"></div> 6870 <div class="transcript"></div> 6871 </a> 6872 6873 <a href="/comics/directories-and-symlinks/"> 6874 <span class="title"> 6875 directories and symlinks 6876 </span> 6877 <div class="keywords"></div> 6878 <div class="transcript"></div> 6879 </a> 6880 6881 <a href="/comics/ipv6/"> 6882 <span class="title"> 6883 ipv6 6884 </span> 6885 <div class="keywords"></div> 6886 <div class="transcript"></div> 6887 </a> 6888 6889 <a href="/comics/whats-a-mac-address/"> 6890 <span class="title"> 6891 what's a mac address? 6892 </span> 6893 <div class="keywords"></div> 6894 <div class="transcript"></div> 6895 </a> 6896 6897 <a href="/comics/inter-process-communication/"> 6898 <span class="title"> 6899 inter-process communication 6900 </span> 6901 <div class="keywords"></div> 6902 <div class="transcript"></div> 6903 </a> 6904 6905 <a href="/comics/2fa/"> 6906 <span class="title"> 6907 2fa 6908 </span> 6909 <div class="keywords"></div> 6910 <div class="transcript">person 1: I have a really secure email password! 6911person 2: that's awesome! but you know, if a hacker got my password, they STILL can't get into my email :) 6912person 1: what? how? 6913 6914### There are 3 common ways to use 2FA: 6915 6916#### SMS (okay!) 6917person: I'd like to login 6918email: I've sent you an SMS with a code. Enter the code to finish logging in 6919 6920Problems: 6921- Your phone # can get stolen (this happens in real life!) 6922- Sometimes SMS doesn't arrive 6923 6924#### google authenticator app, aka TOTP (very good!) 6925person: I'd like to login 6926phone: 12345 6927email: enter the code from that app on your phone! 6928 6929Problem: These codes can still be phished 6930 6931#### security key, aka U2F (the easiest to use! the most secure!) 6932person: I'd like to login 6933tap yubikey - done! 6934These work AWESOME for gmail! You just plug it into a USB port! 6935 6936Problems: 6937- you have to buy it 6938- not every website has support 6939</div> 6940 </a> 6941 6942 <a href="/comics/user-space-vs-kernel-space/"> 6943 <span class="title"> 6944 user space vs kernel space 6945 </span> 6946 <div class="keywords"></div> 6947 <div class="transcript"></div> 6948 </a> 6949 6950 <a href="/comics/the-senior-engineer/"> 6951 <span class="title"> 6952 the senior engineer 6953 </span> 6954 <div class="keywords"></div> 6955 <div class="transcript"></div> 6956 </a> 6957 6958 <a href="/comics/ways-i-want-my-team-to-be/">
6959 <span class="title"> 6960 ways i want my team to be 6961 </span> 6962 <div class="keywords"></div> 6963 <div class="transcript"></div> 6964 </a> 6965 6966 <a href="/comics/tcp/"> 6967 <span class="title"> 6968 tcp 6969 </span> 6970 <div class="keywords"></div> 6971 <div class="transcript"></div> 6972 </a> 6973 6974 <a href="/comics/page-table/"> 6975 <span class="title"> 6976 page table 6977 </span> 6978 <div class="keywords"></div> 6979 <div class="transcript"></div> 6980 </a> 6981 6982 <a href="/comics/having-productive-conversations-when-i-disagree/"> 6983 <span class="title"> 6984 having productive conversations when i disagree 6985 </span> 6986 <div class="keywords"></div> 6987 <div class="transcript"></div> 6988 </a> 6989 6990 <a href="/comics/what-does-an-operating-system-do/"> 6991 <span class="title"> 6992 what does an operating system do? 6993 </span> 6994 <div class="keywords"></div> 6995 <div class="transcript"></div> 6996 </a> 6997 6998 <a href="/comics/the-stack/"> 6999 <span class="title"> 7000 the stack 7001 </span> 7002 <div class="keywords"></div> 7003 <div class="transcript"></div> 7004 </a> 7005 7006 <a href="/comics/no-feigning-surprise/"> 7007 <span class="title"> 7008 no feigning surprise 7009 </span> 7010 <div class="keywords"></div> 7011 <div class="transcript"></div> 7012 </a> 7013 7014 <a href="/comics/how-to-talk-to-your-operating-system/"> 7015 <span class="title"> 7016 how to talk to your operating system 7017 </span> 7018 <div class="keywords"></div> 7019 <div class="transcript"></div> 7020 </a> 7021 7022 <a href="/comics/anatomy-of-a-packet/"> 7023 <span class="title"> 7024 anatomy of a packet 7025 </span> 7026 <div class="keywords"></div> 7027 <div class="transcript">when you get a webpage, like Facebok, it comes into your computer in many small packets. 7028 7029Let's see what those look like! 7030 7031Packets are split into a few sections (or "headers") 7032 7033### ethernet/wifi 7034`82:53:ac:99:2f:33` (MAC address) 7035 7036"physical layer": this gets changed constantly as your packet moves between computers 7037 7038### IP ("internet protocol") 7039`FROM: 172.96.2.3 TO: 123.9.2.32` 7040 7041in charge of getting your packet to the right server (like an address on an envelope) 7042 7043### TCP (or UDP) 7044`sequence number: 877392` (counts bytes sent so far) 7045`checksum: 8847` (detect corrupted data) 7046`from: port 9979 to: port 80` 7047 7048in charge of preventing data corruption and helping you retry lost packets. video streaming uses UDP instead. UDP does not try to be reliable. 7049 7050### HTTP (or whatever) 7051``` 7052GET / HTTP 1.1 7053Host: google.com 7054Accept-Language:en-US 7055``` 7056the actual data you're trying to send!</div> 7057 </a> 7058 7059 <a href="/comics/networking-concepts/"> 7060 <span class="title"> 7061 networking concepts 7062 </span> 7063 <div class="keywords"></div> 7064 <div class="transcript"></div> 7065 </a> 7066 7067 <a href="/comics/network-address-translation/"> 7068 <span class="title"> 7069 network address translation 7070 </span> 7071 <div class="keywords"></div> 7072 <div class="transcript"></div> 7073 </a> 7074 7075 <a href="/comics/mutexes/"> 7076 <span class="title"> 7077 mutexes 7078 </span> 7079 <div class="keywords"></div> 7080 <div class="transcript"></div> 7081 </a> 7082 7083 <a href="/comics/man-pages-are-awesome/"> 7084 <span class="title"> 7085 man pages are awesome 7086 </span> 7087 <div class="keywords"></div> 7088 <div class="transcript"></div> 7089 </a> 7090 7091 <a href="/comics/rr/"> 7092 <span class="title"> 7093 rr 7094 </span> 7095 <div class="keywords"></div> 7096 <div class="transcript"></div> 7097 </a> 7098 7099 <a href="/comics/bpf-filters/"> 7100 <span class="title"> 7101 bpf filters 7102 </span> 7103 <div class="keywords"></div> 7104 <div class="transcript"></div> 7105 </a> 7106 7107 <a href="/comics/bash-tips/">
7108 <span class="title"> 7109 bash tips 7110 </span> 7111 <div class="keywords"></div> 7112 <div class="transcript"></div> 7113 </a> 7114 7115 <a href="/comics/the-cap-theorem/"> 7116 <span class="title"> 7117 the cap theorem 7118 </span> 7119 <div class="keywords"></div> 7120 <div class="transcript"></div> 7121 </a> 7122 7123 <a href="/comics/acid/"> 7124 <span class="title"> 7125 acid 7126 </span> 7127 <div class="keywords"></div> 7128 <div class="transcript">## what's acid? 7129 7130notes from Martin Kleppman's *amazing* "Designing Data-Intensive Applications" book. 7131 7132ACID is about safety guarantees for database transactions. 7133 7134### Atomicity 7135NOT about concurrent writes, that's "isolation" 7136 7137application: do these 5 writes 7138atomic DB: omg there was an error in the middle, rolling them all back! 7139 7140### Consistency 7141super overloaded term. This sense of "consistency" is actually an application property not a DB property. 7142 7143not linearizability 7144not as in "eventual consistency" 7145 7146About preserving application invariants like "every sale gets an invoice" 7147 7148### Isolation 7149 7150app 1: I'm selling a watch 7151app 2: I'm selling the same watch 7152 7153Isolation is about preventing rare conditions like this. 7154 7155Some isolation levels: 7156- serializability 7157- snapshot isolation 7158- read committed 7159 7160### Durability 7161 7162Durable DB: I committed your writes 7163app: phew my data won't get lost even if the DB crashes/there's a hardware failure 7164 7165Perfect durability doesn't exist. 7166 7167Can involve: 7168- write-ahead log (usually) 7169- replication</div> 7170 </a> 7171 7172 <a href="/comics/the-filesystem-cache/"> 7173 <span class="title"> 7174 the filesystem cache 7175 </span> 7176 <div class="keywords"></div> 7177 <div class="transcript"></div> 7178 </a> 7179 7180 <a href="/comics/computers-are-fast/"> 7181 <span class="title"> 7182 computers are fast 7183 </span> 7184 <div class="keywords"></div> 7185 <div class="transcript"></div> 7186 </a> 7187 7188 <a href="/comics/blogging-principles/"> 7189 <span class="title"> 7190 blogging principles 7191 </span> 7192 <div class="keywords"></div> 7193 <div class="transcript"></div> 7194 </a> 7195 7196 <a href="/comics/how-does-dns-work/"> 7197 <span class="title"> 7198 how does dns work 7199 </span> 7200 <div class="keywords"></div> 7201 <div class="transcript"></div> 7202 </a> 7203 7204 <a href="/comics/directories-and-symlinks-1/"> 7205 <span class="title"> 7206 directories and symlinks 7207 </span> 7208 <div class="keywords"></div> 7209 <div class="transcript"></div> 7210 </a> 7211 7212 <a href="/comics/linux-tracing-systems/"> 7213 <span class="title"> 7214 linux tracing systems 7215 </span> 7216 <div class="keywords"></div> 7217 <div class="transcript"></div> 7218 </a> 7219 7220 <a href="/comics/read-the-source/"> 7221 <span class="title"> 7222 read the source code 7223 </span> 7224 <div class="keywords"></div> 7225 <div class="transcript"></div> 7226 </a> 7227 7228 <a href="/comics/getting-started-with-ftrace/"> 7229 <span class="title"> 7230 getting started with ftrace 7231 </span> 7232 <div class="keywords"></div> 7233 <div class="transcript"></div> 7234 </a> 7235 7236 <a href="/comics/vim-sessions/"> 7237 <span class="title"> 7238 vim sessions 7239 </span> 7240 <div class="keywords"></div> 7241 <div class="transcript"></div> 7242 </a> 7243 7244 <a href="/comics/whats-slow-on-a-computer/"> 7245 <span class="title"> 7246 what's slow on a computer 7247 </span> 7248 <div class="keywords"></div> 7249 <div class="transcript"></div> 7250 </a> 7251 7252 <a href="/comics/learning-to-design-software/"> 7253 <span class="title">
7254 learning to design software 7255 </span> 7256 <div class="keywords"></div> 7257 <div class="transcript"></div> 7258 </a> 7259 7260 <a href="/comics/ways-to-build-expertise/"> 7261 <span class="title"> 7262 ways to build expertise 7263 </span> 7264 <div class="keywords"></div> 7265 <div class="transcript"></div> 7266 </a> 7267 7268 <a href="/comics/tips-for-reading-code/"> 7269 <span class="title"> 7270 tips for reading code 7271 </span> 7272 <div class="keywords"></div> 7273 <div class="transcript"></div> 7274 </a> 7275 7276 <a href="/comics/scenes-from-design-docs/"> 7277 <span class="title"> 7278 scenes from design docs 7279 </span> 7280 <div class="keywords"></div> 7281 <div class="transcript"></div> 7282 </a> 7283 7284 <a href="/comics/love-your-bugs/"> 7285 <span class="title"> 7286 love your bugs 7287 </span> 7288 <div class="keywords"></div> 7289 <div class="transcript">(thanks to Allison Kaptur for teaching me this attitude! she has a great talk called "Love Your Bugs.) 7290 7291Debugging is a great way to learn. First, the harsh reality of bugs in your code is a good way to reveal problems with your mental model. 7292 7293program: error: too many open files 7294person: I can't just open as? many files as I want?. Interesting! 7295 7296Fixing bugs is a good way to learn to write also more reliable code! 7297 7298person, thinking: hmm, I should put in error handling here in case that data base query times out. 7299 7300Also, you get to solve a mystery and get immediate feedback about whether you were right or not. 7301 7302person 1: that's weird... 7303person 1: oh goodness, that's a lot of errors 7304person 1: I have an idea! 7305person 1: [coding a fix] 7306person 1: it works now! 7307person 2: great work! 7308 7309Nobody writes great code without writing + fixing lots of bugs. So let's talk about debugging skills a bit!</div> 7310 </a> 7311 7312 <a href="/comics/lets-build-expertise/"> 7313 <span class="title"> 7314 let's build expertise! 7315 </span> 7316 <div class="keywords"></div> 7317 <div class="transcript"></div> 7318 </a> 7319 7320 <a href="/comics/learning-at-work/"> 7321 <span class="title"> 7322 learning at work 7323 </span> 7324 <div class="keywords"></div> 7325 <div class="transcript"></div> 7326 </a> 7327 7328 <a href="/comics/start-learning-today/"> 7329 <span class="title"> 7330 it's not too late to start learning 7331 </span> 7332 <div class="keywords"></div> 7333 <div class="transcript"></div> 7334 </a> 7335 7336 <a href="/comics/invest-in-understanding/"> 7337 <span class="title"> 7338 invest in understanding 7339 </span> 7340 <div class="keywords"></div> 7341 <div class="transcript"></div> 7342 </a> 7343 7344 <a href="/comics/good-questions-2/"> 7345 <span class="title"> 7346 asking good questions (part 2) 7347 </span> 7348 <div class="keywords"></div> 7349 <div class="transcript"></div> 7350 </a> 7351 7352 <a href="/comics/good-questions-1/"> 7353 <span class="title"> 7354 asking good questions 7355 </span> 7356 <div class="keywords"></div> 7357 <div class="transcript">One of my favorite tools for learning is asking questions of all the awesome people I know! 7358 7359what's a good question? 7360 7361### good questions: 7362- are easy for the person to answer 7363- get you the information you're looking for 7364 7365### Here are some strategies for asking them: 7366- state what you know 7367person 1: so, I know when the database gets a lot of writes, the hard drive can't keep up. 7368person 2: that's right! I don't think that was) our problem, though. Look at this... 7369 7370This helps because: 7371- I'm forced to think about what I know 7372- I'm less likely to get answers that are too basic or too advanced 7373 7374Guessing the answer: 7375- makes me think! 7376- helps my coworker see what kind of answer I'm looking for 7377 7378guess what the answer might be 7379person 1: Do we have 5 load balancers because we get a lot of HTTP requests? 7380person 2: actually, we just want to be sure it's ok if one goes down.</div> 7381 </a> 7382 7383 <a href="/comics/building-confidence-in-kubernetes/">
7384 <span class="title"> 7385 building confidence in kubernetes 7386 </span> 7387 <div class="keywords"></div> 7388 <div class="transcript"></div> 7389 </a> 7390 7391 <a href="/comics/understand-their-goals/"> 7392 <span class="title"> 7393 understand your manager's goals 7394 </span> 7395 <div class="keywords">[manager]</div> 7396 <div class="transcript">Illustration of two stick figures having a conversation. The manager is smiling and has straight shoulder length hair. The employee looks confused and has short curly hair. 7397 7398manager: can you get metrics on X's speed? 7399me: why? That won't help us get the code done! 7400 7401They might be asking for metrics because: 7402- they're hearing complaints about X being slow (that you might not be hearing!) 7403- without metrics, it's hard for them to have an informed conversation about those complaints (& defend you if X is actually fast!) 7404 7405Having regular conversations about their priorities for the team is SO USEFUL and means that I'm surprised less often. (illustration of two smiley faces) 7406 7407Illustration of the same two stick figures as above, but now they're both smiling. 7408 7409manager: performance / speed is getting more important recently! 7410me: good to know, should I work on speeding up X?</div> 7411 </a> 7412 7413 <a href="/comics/remember-only-human/"> 7414 <span class="title"> 7415 remember your manager's only human 7416 </span> 7417 <div class="keywords">[manager]</div> 7418 <div class="transcript">Sometimes I fall into a trap where I think my manager should be able to solve EVERY problem on the team and if they're not then they're not doing their job. (the word "every" is surrounded by glowing lines for emphasis) 7419 7420It's helpful for me to remember that at any given time they're probably dealing with a lot! 7421 7422Illustration of a smiling stick figure, representing the manager, surrounded by spiky bubbles containing the following items. 7423 7424- hire 2 people 7425- coordinate with other teams 7426- make sure the intern gets an offer on time (illustration of a clock) 7427- write 10 performance reviews 7428- finalize plans for next quarter 7429- make sure we have an onboarding plan for the new person 7430- interview new manager candidate 7431- a team member is unhappy, figure out what's going on 7432- ... personal life (smiley face) 7433 7434I try to be somewhat aware of what my manager is dealing with & help out when I can. 7435 7436Illustration of two smiling stick figures, one with curly hair representing the employee, and one with medium length straight hair, representing the manager. 7437 7438employee: Here's a project I think could be a good fit for the new person! 7439manager: good idea, thanks!</div> 7440 </a> 7441 7442 <a href="/comics/emotional-labour/"> 7443 <span class="title"> 7444 on emotional labour 7445 </span> 7446 <div class="keywords">[manager]</div> 7447 <div class="transcript">"Emotional labour" is the idea that dealing with feelings-related problems is work. 7448 7449Illustration of two stick figures having a conversation. The employee has short curly hair and looks angry. The manager is smiling and has no hair. 7450 7451I'm angry that my contributions on that project weren't recognized... 7452manager: [understanding face, doing work] 7453 7454Emotional labour is part of what managers are paid to do. But!! Managers aren't therapists. 7455 7456Illustration of a smiling stick figure, crossed out in red. 7457 7458manager: tell me about your father... 7459 7460not good 1:1 material (smiley face) 7461 7462When I'm upset about something, I try to be clear about why and ideally explain what I think a reasonable resolution would be. 7463 7464employee: can we just make sure it features in my next performance review? 7465manager: yes definitely!</div> 7466 </a> 7467 7468 <a href="/comics/how-to-work-well-with-manager/"> 7469 <span class="title"> 7470 how to work well with your manager 7471 </span> 7472 <div class="keywords">[manager]</div> 7473 <div class="transcript">Most of the rest of this zine is about 7474 7475COMMUNICATION 7476 7477(The word "communication" is surrounded by hearts, smiley faces, stars, and exclamation marks) 7478 7479Basically your manager'
7479s job is to make sure that your team is getting work done that will help the business. 7480 7481This is awesome because it means that if you just communicate with them well, then you can mostly focus on programming!!! 7482 7483(the word "awesome" is surrounded by glowing lines and hearts) 7484 7485Communicating well can help you: 7486- get awesome opportunities 7487- solve problems 7488- build trust 7489- understand priorities 7490- get promoted 7491- get feedback 7492 7493(each of the above items is in a spikey bubble) 7494 7495To start, let's talk about 1:1s (which hopefully your manager schedules regularly).</div> 7496 </a> 7497 7498 <a href="/comics/new-manager/"> 7499 <span class="title"> 7500 getting a new manager 7501 </span> 7502 <div class="keywords">[manager]</div> 7503 <div class="transcript">Being assigned a new manager is a little scary. Not all of my managers have been great! 7504 7505Illustration of a stick figure with short curly hair, looking uncertain. 7506 7507person: OH NO what if my new manager is hard to work with ?!?! 7508 7509But! More than once I've started out thinking, 7510 7511Illustration of a stick figure with short curly hair, looking scared. 7512 7513person: who is this person they seem suspicious 7514 7515and ended up, a year later, at 7516 7517Illustration of a stick figure with short curly hair, smiling. 7518 7519person: wow they have helped me and the team so much, this is AMAZING 7520 7521so I try to assume that's where we'll end up. 7522 7523Some things I've found helpful: 7524- write a document explaining my past work to them 7525- ask them about any concerns directly - often they have great answers! 7526- pay close attention to what they do well 7527- tell them when they do something great</div> 7528 </a> 7529 7530 <a href="/comics/whats-a-branch/"> 7531 <span class="title"> 7532 what's a branch? 7533 </span> 7534 <div class="keywords"></div> 7535 <div class="transcript">You can think about a Git branch in 3 different ways: 7536 7537### 1. just the commits that "branch" off 7538this is how I usually think about branches: 7539 7540`armadillo` branches off `main` 7541 7542Illustration of a vertical black line, labelled "main". Coming of off it is a red line, labelled "armadillo" The armadillo line has two dots on it. 7543 7544The two dots are labelled "I think of the armadillo branch as these 2 commits" 7545 7546#### How this shows up in git: 7547Git DOESN'T KNOW that `armadillo` is branched off of `main`: for all it knows, main could be branched off of `armadillo`! You need to tell it when you merge or rebase, for example: 7548``` 7549git checkout main 7550git merge armadillo 7551``` 7552 7553### 2. every previous commit 7554Even though git doesn't treat the `main` branch in any special way, I think of `main` differently from other branches. 7555 7556Illustration of a vertical red line, labelled "main", which has four dots along it. Coming of off it is a black line, labelled "armadillo". The red dots on the red line are labelled "I think of my main branch as these 4 commits" 7557 7558#### How this shows up in git: 7559It's what `git log BRANCHNAME` shows you! How `git log main` works: 7560 7561Illustration of a vertical line with four dots along it. The dot at the top is labelled `main` (start here). The lines between the dots are labelled "parent". 7562 7563### 3. just the commit at the end 7564This is how branches are actually implemented in git. 7565 7566Illustration of a vertical black line, labelled "main", which has four dots along it. Coming of off it is a red line, labelled "armadillo". The final dot along "armadillo" is labelled "the latest commit on the branch" 7567 7568#### How this shows up in git: 7569It's how branches are stored internally: a branch is fundamentally a name for a 7570commit ID. 7571 7572`.git/refs/heads/main` (branch name) 7573`a276f62` (ID of the latest commit on the branch)</div> 7574 </a> 7575 7576 <a href="/comics/the-current-branch-head/"> 7577 <span class="title"> 7578 the current branch: HEAD 7579 </span> 7580 <div class="keywords"></div> 7581 <div class="transcript">### HEAD is a tiny file containing the name of the current branch 7582 7583Diagram of three boxes in a row, joined by lines. One has a heart, one has a star, and one has a squiggle. The final one, with the squiggle, is labelled "`main`". 7584 7585`HEAD` = `main` 7586`main` = [squiggle] 7587 7588### when you commit, git updates the current branch to point at the new commit 7589 7590Diagram of three boxes in a row, joined by lines. One has a heart, one has a star, and one has a squiggle. The final one, with the squiggle, is labelled "`main`". 7591 7592`HEAD` = `main` 7593`main` = [squiggle] 7594 7595Diagram of four boxes in a row, joined by lines. One has a heart, one has a star, one has a squiggle, and one has a spiral. The final one, with the spiral, is labelled "`main`". 7596 7597`HEAD` = `main` 7598`main` = [spiral] 7599 7600### SO MANY things in git use the current branch 7601 7602* `git commit` moves it forward 7603* `git merge` merges into it 7604* `git rebase` copies commits from it 7605* `git push` and `git pull` sync it with a remote 7606 7607### many git disasters are caused by accidentally running a command while on the wrong branch 7608 7609Illustration of a sad stick figure 7610 7611person: `git commit` 7612person, thinking: UGH I didn't mean to do that on `main` 7613 7614### I keep my current branch in my shell prompt 7615 7616`~/work/homepage (main) $` 7617 7618to me it's as important as knowing what directory I'm in 7619 7620### panel 6 7621 7622Illustration of a smiling stick figure with curly hair. 7623 7624person: I think `HEAD` is a weird name for the current branch (why not `CURRENT` or something?) but we're stuck with it</div> 7625 </a> 7626 7627 <a href="/comics/rules-for-rebasing/">
7628 <span class="title"> 7629 rules for rebasing 7630 </span> 7631 <div class="keywords"></div> 7632 <div class="transcript">### don't rebase a million tiny commits 7633 7634you can end up having to fix the same merge conflict 25 times and it's a nightmare. 7635 7636instead, do it in 2 steps: 7637 76381. squash into 1 commit with `git rebase -i` 76392. `git rebase main` 7640 7641### don't force push to a shared branch 7642 7643 7644it's totally ok if it's your own branch that nobody else will ever have to git pull from, but if other people are using it, it makes things weird 7645 7646### don't do more than one thing in a `git rebase -i` 7647 7648you can 7649 7650* combine commits 7651* reorder commits 7652* edit commits 7653 7654but don't do all of them at once! It's too confusing! 7655 7656### don't rebase other people's commits 7657 7658I only modify my own commits 7659 7660### stop a rebase if it's going badly 7661 7662it's MUCH easier to run `git rebase --abort` and bail out than to have to undo it later. It'll take you back to where you were before the rebase. 7663 7664### you never have to rebase 7665 7666the only reason to rebase is to tidy up your git history, if you're not comfortable rebasing then just don't do it! 7667 7668You can merge or `git commit --amend` instead</div> 7669 </a> 7670 7671 <a href="/comics/remote-branch-caching/"> 7672 <span class="title"> 7673 remote branch caching 7674 </span> 7675 <div class="keywords"></div> 7676 <div class="transcript">### the "up to date" in `git status` is misleading 7677 7678``` 7679$ git status 7680Your branch is up to date with origin/main 7681``` 7682 7683this does NOT mean that you're up to date with the remote main branch. But why not??? 7684
7685### some old version control systems only worked if you were online 7686 7687Illustration of a sad stick figure with short curly hair. 7688 7689person (thinking): my internet went out, guess I can't work 7690 7691### git works offline 7692 7693Illustration of a smiling stick figure with short straight hair. 7694 7695git developer (thinking): I want to be able to code on a train with no internet 7696 7697git developer (thinking): NOTHING in git will use the internet except `git pull`, `git push`, and `git fetch` 7698 7699### this makes `git status` weird 7700 7701git developer (thinking): we need to tell people if their branch is up to date... with NO INTERNET??? how? 7702 7703### solution: CACHING 7704 7705Every remote branch has a local cache named like `origin/mybranch` (`origin` is the remote name, `mybranch` is the branch name) 7706 7707Git doesn't call it a cache though, it calls it a "remote tracking branch" 7708 7709local branch: `mybranch` 7710 7711cache: `origin/mybranch` (only updated on `git pull`, `git push`, `git fetch`) 7712 7713remote branch: `origin mybranch` (`git push origin mybranch` updates this) 7714 7715(git has no easy way to see when `origin/mybranch` was last updated)</div> 7716 </a> 7717 7718 <a href="/comics/orphan-commits/"> 7719 <span class="title"> 7720 orphan commits 7721 </span> 7722 <div class="keywords"></div> 7723 <div class="transcript">### commits in git are usually saved forever 7724 7725Except! Orphaned commits are deleted periodically. 7726 7727Illustration of a little garbage can. 7728 7729Commits are orphaned when you: 7730 7731- `git commit --amend` 7732- `git rebase` 7733- delete a branch that hasn't been merged 7734 7735### what is an orphaned commit? 7736 7737it's a commit that isn't in the history of any branch 7738 7739they're almost totally invisible, since Git will usually only show you commits on branches 7740 7741### orphan #1: `git commit --amend` 7742 7743before: 7744An illustration for a box that says `parent`, with a line to a second box that says `fix color buug` (typo!). The second box is labelled `main` branch. 7745 7746after: 7747The same diagram as above, but there is now a second line coming out of the `parent` box, going to a third box that says `fix color bug`. The `fix color buug` box is now labelled "now it's an orphan!" and the `fix color bug` box is labelled "`main` branch". 7748 7749### orphan #2: `git rebase` 7750 7751before: 7752A box with two branches coming out of it. The top one is labelled "`main` branch". The second branch has two boxes, one with a heart, and one with a star. This branch is labelled "`feature` branch". 7753 7754after: 7755A box with two branches coming out of it. The top branch consists of three boxes, one blank, one with a heart, and one with a star. The blank box is labelled "`main` branch", and the box with the star is labelled "`feature` branch". The second branch consists of two boxes, one with a heart, and one with a star. This branch is labelled "now these two are orphans!" 7756 7757### orphan #3: `deleting unmerged branch` 7758 7759before: 7760A box with two branches coming out of it. The first branch consists of one blank box, labelled "`main` branch". The second branch consists of two boxes, one with a heart, and one with a star. This branch is labelled "`feature` branch". 7761 7762after deleting `feature`: 7763The same diagram as above, except that the second branch is now labelled "now these two are orphans!" 7764 7765### how to find orphan commits 7766 7767the only way to find them is with `git reflog` (or by memorizing their commit ID somehow)</div> 7768 </a> 7769 7770 <a href="/comics/oh-shit-undo-5-commits-ago/"> 7771 <span class="title"> 7772 oh shit! I want to undo something from 5 commits ago! 7773 </span> 7774 <div class="keywords"></div> 7775 <div class="transcript">If you made a mistake but want to keep all of the commits since then, `git revert` is your friend! 7776 7777`git revert` will create a reverse patch for the changes in a commit and add it as a new commit. 7778 77791. Find the commit SHA for the commit you want to undo. 7780 77812. Run: 7782`git revert SHA` 7783 77843. Enter a commit message for the revert commit. 7785 7786Now all of the changes you made in that commit are undone! 7787
7788person: this is super useful if you push a bad commit to a shared repository and need to undo it!</div> 7789 </a> 7790 7791 <a href="/comics/oh-shit-split-commit/"> 7792 <span class="title"> 7793 oh shit! I want to split my commit into 2 commits! 7794 </span> 7795 <div class="keywords"></div> 7796 <div class="transcript">1. Stash any uncommitted changes (so they don't get mixed up with the changes from the commit): 7797`git stash` 7798 77992. Undo your most recent commit: 7800`git reset HEAD^` 7801(safe: this points your branch at the parent commit but doesn't change any files) 7802 78033. Use `git add` to pick and choose which files you want to commit and make your new commits! 7804 78054. Get your uncommitted changes back: 7806`git stash pop` 7807 7808person: you can use `git add -p` if you want to commit some changes to a file but not others!</div> 7809 </a> 7810 7811 <a href="/comics/oh-shit-diff-nothing-happened/"> 7812 <span class="title"> 7813 oh shit! I tried to run a diff but nothing happened! 7814 </span> 7815 <div class="keywords"></div> 7816 <div class="transcript">Suppose you've edited 2 files 7817``` 7818$ git status 7819On branch main 7820Changes to be committed: 7821 7822modified: staged.txt 7823Changes not staged for commit: 7824 7825modified: unstaged.txt 7826``` 7827 7828("`modified: staged.txt`" are staged changes added with `git add`. `modified: unstaged.txt` are unstaged changes.) 7829 7830Here are the 3 ways git can show you a diff for these changes: 7831- `git diff`: unstaged changes 7832- `git diff --staged`: staged changes 7833- `git diff HEAD`: staged+unstaged changes 7834 7835A couple more diff tricks: 7836- `git diff --stat` gives you a summary of which files were changed & number of added/deleted lines 7837- `git diff --check` checks for merge conflict markers & whitespace errors</div> 7838 </a> 7839 7840 <a href="/comics/oh-shit-rebase-conflicts/"> 7841 <span class="title"> 7842 oh shit! I started rebasing and now I have 1000000 conflicts to fix! 7843 </span> 7844 <div class="keywords"></div> 7845 <div class="transcript">This can happen when you're rebasing many commits at once. 7846 78471. Escape the rebase of doom: 7848`git rebase --abort` 7849 78502. Find the commit where your branch diverged from main: 7851`git merge-base main my-branch` 7852 78533. Squash all the commits in your branch together: 7854`git rebase -i $SHA_YOU_FOUND` 7855 78564. Rebase on main: 7857`git rebase main` 7858 7859person: alternatively, if you have 2 branches with many conflicting commits, you can just merge!</div> 7860 </a> 7861 7862 <a href="/comics/oh-shit-change-message/"> 7863 <span class="title"> 7864 oh shit! I need to change the message on my last commit! 7865 </span> 7866 <div class="keywords"></div> 7867 <div class="transcript">No problem! Just run: 7868 7869`git commit --amend` 7870 7871Then edit the commit message & save! 7872 7873`git commit --amend` will replace the old commit with a new commit with a new SHA, so you can always go back to the old version if you really need to. 7874 7875Person: if you run `git commit` but change tour mind, you can always abort by deleting the commit message & saving + quitting. Or quit without saving!</div> 7876 </a> 7877 7878 <a href="/comics/oh-shit-merge-conflict/"> 7879 <span class="title"> 7880 oh shit! I have a merge conflict! 7881 </span> 7882 <div class="keywords"></div> 7883 <div class="transcript">Suppose you had `main` checked out and ran `git merge feature-branch`. 7884 7885If that causes a merge conflict, you'll see something like this in the files with conflicts: 7886 7887``` 7888<<<<<<< HEAD 7889if x == 0: 7890 return false 7891======= 7892``` 7893(this is the code from `main`) 7894``` 7895if y == 6: 7896 return true 7897elif x ==0: 7898 return false 7899feature-branch 7900>>> d34367 7901``` 7902(this is the code from `feature-branch`) 7903 7904### To resolve the conflict: 7905 79061. Edit the files to fix the conflict 79072. `git add` the fixed files 79083. `git diff` --check: check for more conflicts. 79094. `git commit` when you're done. (or `git rebase --continue` if you're rebasing!) 7910 7911Smiling stick figure with medium length straight hair: You can use a GUI to visually resolve conflicts with `git mergetool`. Meld (meldmerge.org) is a great choice!</div> 7912 </a> 7913 7914 <a href="/comics/oh-shit-time-machine/">
7915 <span class="title"> 7916 oh shit! I did something terribly wrong, does git have a magic time machine? 7917 </span> 7918 <div class="keywords"></div> 7919 <div class="transcript">Yes! It's called git reflog and it logs every single thing you do with git so that you can always go back. 7920 7921Suppose you ran these git commands: 7922 7923``` 7924git checkout my-cool-branch (1) 7925git commit -am "add cool feature" (2) 7926git rebase master (3) 7927``` 7928 7929Here's what git reflog's output would look like. 7930It shows the most recent actions first: 7931 7932```245fc8d HEAD @{2} rebase -i (start):``` (3) checkout master 7933```b623930 HEAD @{3} commit:``` (2) add cool feature 7934```01d7933 HEAD @{4} checkout:``` (1) moving from master to my-cool-branch 7935 7936 7937If you really regret that rebase and want to go back, here's how: 7938 7939```git reset --hard b623930``` 7940```git reset --hard HEAD@{3} ``` 79412 ways to refer to that commit before the rebase 7942</div> 7943 </a> 7944 7945 <a href="/comics/oh-shit-edit-commit/"> 7946 <span class="title"> 7947 oh shit! I committed but I want to make one small change! 7948 </span> 7949 <div class="keywords"></div> 7950 <div class="transcript">1. Make your change 79512. Add your files with git add 79523. Run: 7953`git commit --amend --no-edit` 7954 7955person: this usually happens to me when I forget to run tests/linters before committing! 7956 7957You can also add a new commit and use `git rebase -i` to squash them but this is about a million times faster.</div> 7958 </a> 7959 7960 <a href="/comics/oh-shit-committed-file/"> 7961 <span class="title"> 7962 oh shit! I committed a file that should be ignored! 7963 </span> 7964 <div class="keywords"></div> 7965 <div class="transcript">Did you accidentally commit a 1.5GB file along with the files you actually wanted to commit? We've all done it. 7966 79671. Remove the file from Git's index: 7968`git rm --cached FILENAME` 7969This is safe: it won't delete the file 7970 79712. Amend your last commit: 7972`git commit --amend` 7973 79743. (optional) Edit your `.gitignore` so it doesn't happen again 7975 7976person: now your coworkers won't be stuck downloading a HUGE git commit</div> 7977 </a> 7978 7979 <a href="/comics/merge-commits/"> 7980 <span class="title"> 7981 merge commits 7982 </span> 7983 <div class="keywords"></div> 7984 <div class="transcript">### merging 2 diverged branches creates a commit 7985 7986`git merge mybranch` 7987 7988Diagram of two boxes in a row, one with a heart, and one with a star. From the star, it branches out into a branch with a hash symbol, labelled `main`. The other branch coming off of the star has a box with a spiral followed by a box with a spiky symbol. The two branches converge in a box with a diamond symbol, labelled "merge commit!". 7989 7990merge commits have a few surprising gotchas! 7991 7992### gotcha: merging isn't symmetric 7993 7994normal: 7995``` 7996git checkout main 7997git merge mybranch 7998``` 7999weird: 8000``` 8001git checkout mybranch 8002git merge main 8003``` 8004these two result in the same code, but the merge commit's parents have a different order 8005 8006This comes up when you use `HEAD^`: it refers to the first parent, and usually you want that to be the commit from the main branch 8007 8008### gotcha: you can keep coding during a merge 8009 8010If you forget you're doing a merge, it's easy to accidentally keep writing code and add a bunch of unrelated changes into the merge commit. 8011 8012I use my prompt to remind me. 8013 8014### gotcha: git show doesn't tell you what the merge commit did 8015 8016It'll often just show the merge commit as "empty" even if the merge did something important (like discard changes from one side). 8017 8018Illustration of a tiny sad stick person with curly hair 8019 8020person: why 8021 8022### tip: see what a merge did with `git show --remerge-diff` 8023 8024`git show --remerge-diff COMMIT_ID` 8025 8026will re-merge the parents and show you the difference between the original merge and what's actually in the merge commit</div> 8027 </a> 8028 8029 <a href="/comics/meet-the-merge/"> 8030 <span class="title"> 8031 meet the merge 8032 </span> 8033 <div class="keywords"></div> 8034 <div class="transcript">### merging is a huge thing in git 8035 8036But the terminology around merging is a bit confusing: 8037 8038- `git merge` 8039 isn't the only way to combine branches: you can also use `git rebase`! 8040 8041- merge conflicts (surrounded by sad faces) can happen if you do any of these: 8042 - `git merge` 8043 - `git rebase` 8044 - `git cherry-pick` 8045 - `git revert` 8046 - `git stash pop ` 8047 8048- merge commits are only created by `git merge` 8049 8050Illustration of two stick figures talking, one is bald and looks unhappy, the other has curly hair and is smiling. 8051 8052person 1: ... and what the heck is "fast forward"? 8053 8054person 2: let's talk about it! 8055 8056### there are 3 situations when combining branches 8057 80581. easy: no divergence ("fast-forward") 8059Diagram of a box with a heart in it, labelled "main". Branching off it in a horizontal line, are three boxes with a star, a hash symbol, and a squiggle. The squiggle box is labelled "panda". 8060 8061git merge moves the main branch forward to where the panda branch is, like this: 8062Same diagram as above, except now the squiggle box is labelled "main" as well as "panda". 8063 80642. harder: diverged branches, no conflicts 8065Diagram of two boxes in a horizontal line, one with a heart, and one with a star. Branching off of the star box are two boxes, one with a hash symbol and one with a spiral. These two boxes are labelled "
8065editing different code". 8066 8067you have to decide whether to merge or rebase, but it'll succeed 8068 80693. hardest: diverged branches with merge conflicts 8070 The same diagram as above, except now the two final boxes are labelled "editing the same code", and there is a sad stick figure standing beside it. 8071 8072 8073you have to decide whether to merge or rebase, AND fix a merge conflict 8074 8075### git merge checks for these 3 situations in order 8076 80771. is this the "easy" situation? 8078- if no, run the merge 8079- if yes, fast forward! 8080 80812. run the merge. Is there a merge conflict? 8082- if yes, tell you to manually resolve the conflict 8083- if no, done! 8084 80853. tell you to manually resolve the conflict 8086 8087### `git pull` needs to combine branches too 8088 8089`git pull` will ONLY fast forward (easy mode) by default. If it can't, it'll ask you to specify if you want to rebase or merge. 8090 8091`git pull --rebase` runs `git rebase` 8092 8093`git pull --no-rebase` runs `git merge `</div> 8094 </a> 8095 8096 <a href="/comics/meet-the-commit/"> 8097 <span class="title"> 8098 meet the commit 8099 </span> 8100 <div class="keywords"></div> 8101 <div class="transcript">### commits never change 8102once you've made a commit, it's set in stone: 8103- the files in it never change 8104- its diff never changes 8105- its history never changes 8106- the message/author never change 8107 8108### commit hashes 8109commits never change because their ID is calculated from their contents. 8110 8111Illustration of a box labelled `sha1 hash`. 8112 8113Going into the box are: 8114- every file 8115- parent(s) 8116- message 8117- author 8118- timestamp 8119 8120Coming out of the box is an arrow labelled `3530a42`. 8121 8122### you can think of commits as a pile of diffs 8123 8124Illustration of a stack of boxes connected with lines. Each box is labelled "diff", except for the bottom one, which is labelled "START". The top box has an arrow pointing to it that says "current". 8125 8126if you combine all the diffs together, you'll get the current state of the project! 8127 8128(not how Git works, but a VERY useful way to think about commits!) 8129 8130### you can also think of commits as a pile of snapshots 8131 8132Illustration of a stack of boxes connected with lines. Each box is labelled "snapshot", except for the bottom one, which is labelled "START". The top box has an arrow pointing to it that says "current". 8133 8134this is how Git is implemented! 8135 8136confused bald stick figure: is git saving a NEW copy, EVERY TIME?? 8137happy stick figure with curly hair: not quite! it has some tricks! (on the next page) 8138 8139 8140### diffs are calculated from snapshots 8141 8142Illustration of two boxes, one on top of the other, connected with lines. Both boxes are labelled "snapshot". 8143 8144the diff is the difference between a commit and its parent 8145 8146happy stick figure with curly hair: hey what's the diff for `353ea42`? 8147 8148git, represented by a box with a smiley face: let me calculate that REALLY FAST! 8149 8150### things git can do with a commit 8151- get the files in the commit (like `git checkout`) 8152- calculate the diff from its parent (like `git show`) 8153- merge if with arother commit (like `git merge`) 8154- look at its parents, grandparents, etc (like `git log`) 8155</div> 8156 </a> 8157 8158 <a href="/comics/meet-the-branch/"> 8159 <span class="title"> 8160 meet the branch 8161 </span> 8162 <div class="keywords"></div> 8163 <div class="transcript">### theoretically you could use git without branches 8164 8165You could keep track of your commit IDs manually: 8166 8167Illustration of a smiling stick figure with medium-length straight hair. 8168 8169person: hmm, what was I working on? oh yes, `a38b997`! 8170 8171But most people use branches. 8172 8173### every branch has 3 things 8174 8175- a name (like `main`) 8176- a latest commit (like `2e9ffc`) 8177- a reflog of how that branch has evolved over time (page 26) 8178 8179Branches also sometimes have a corresponding remote branch which they "track" 8180 8181### branches are core to how git stores your work 8182 8183If your commits are "lost" (not on a branch) (page 13): 8184- (sad face) git's garbage collection will eventually delete them 8185- (sad face) they'll become incredibly difficult to find 8186 8187### the only difference between the main branch and any other branch is how you treat them 8188 8189For example: it's common to never commit to main directly, and instead commit to other branches which you merge into main when you're done. 8190 8191### all changes to a branch are recorded in its reflog 8192 8193The reflog records every rebase, amended commit, pull, merge, reset, commit, etc. You can look at the reflog like this: 8194 8195`git reflog BRANCHNAME` 8196 8197reflog stands for "reference log" (not re-flog ) (smiley face) 8198 8199### git will let you do literally anything with a branch 8200 8201- when you push/pull a branch, the local branch name doesn't have to match the remote branch name
8202- you can remove commits from a branch with `git reset` 8203 8204Git often won't protect you from messing up your branch!</div> 8205 </a> 8206 8207 <a href="/comics/losing-your-work/"> 8208 <span class="title"> 8209 losing your work 8210 </span> 8211 <div class="keywords"></div> 8212 <div class="transcript">### people are always saying: 8213 8214Illustration of two stick figures talking. One is bald and smiling, the second has long curly hair and is frowning. 8215 8216person 1: don't worry! it's impossible to lose your work in git! 8217 8218person 2 (thinking): my lost work says otherwise 8219 8220but some parts of git are MUCH safer than others 8221 8222### commits on a branch / tag 8223 8224(lock icon) never change 8225 8226Illustration of a smiling stick figure with curly hair. Their speech bubble is surrounded by hearts and stars. 8227 8228person: you can ALWAYS use the commit ID to get your work back! 8229 8230### orphan commits 8231 8232(lock icon) never change, except... 8233 8234they'll eventually get deleted by git's garbage collection 8235 8236(usually not for a few months though) 8237 8238### branches and `HEAD` 8239 8240(unlocked lock icon) change ALL THE TIME 8241 8242(clock going backwards icon) BUT there's a history of all the changes in the reflog 8243 8244Tiny cute illustration of a smiling stick figure with curly hair. 8245 8246person: the reflog is NOT easy to use but at least it's there 8247 8248### staging area 8249 8250(unlocked lock icon) changes ALL THE TIME 8251 8252(crossed out clock going backwards icon) no history 8253 8254(sad face) just gotta be careful 8255 8256### the stash 8257 8258(crossed out clock going backwards icon) `git stash pop` deletes entries forever 8259 8260... but you can technically get them back by using `git fsck` to search EVERY SINGLE COMMIT</div> 8261 </a> 8262 8263 <a href="/comics/explore-a-commit/"> 8264 <span class="title"> 8265 let's explore a commit 8266 </span> 8267 <div class="keywords"></div> 8268 <div class="transcript">### panel 1: you can see for yourself how git is storing your files! 8269 8270You just need one command: `git cat-file -p` 8271 8272First, get a commit ID. You can get one from `git log` 8273 8274### panel 2: read the commit 8275 8276``` 8277 8278$ git cat-file -p 3530a4 8279tree 22b920 8280parent 56cfdc 8281author Julia <[email protected]> 1697682215 -0500 8282committer Julia <[email protected]> 1697682215 -0500 8283``` 8284 8285### panel 3: read the directory 8286 8287``` 8288$ git cat-file -p 22b920 8289100644 blob 4fffb2 .gitignore 8290100644 blob e351d9 404.html 8291100644 blob cab416 Cargo.toml 8292100644 blob fe442d hello.html 8293040000 tree 9de29f src 8294``` 8295 8296### panel 4: read a file 8297 8298``` 8299 8300$ git cat-file -p fe442d 8301<!DOCTYPE html> 8302<html lang="en"> 8303 <body> 8304 <h1>Hello!</h1> 8305 </body> 8306</html> 8307``` 8308 8309 8310### panel 5: and we're done! 8311 8312`fe442d` is the sha1 hash of the contents of the file. It's called a "blob id". 8313 8314this is how git keeps things efficient: it only needs to make a new copy when the file changes</div> 8315 </a> 8316 8317 <a href="/comics/inside-git/"> 8318 <span class="title"> 8319 inside .git 8320 </span> 8321 <div class="keywords"></div> 8322 <div class="transcript">### `HEAD` 8323`HEAD` is a tiny file that just contains the name of your current branch 8324 8325`.git/HEAD` 8326`ref: refs/heads/main` 8327 8328`HEAD` can also be a commit ID, that's called "detached `HEAD` state" 8329 8330### branches 8331a branch is stored as a tiny file that just contains 1 commit ID. It's stored in a folder called `refs/heads`. 8332 8333`7622629` - (actually 40 characters) 8334 8335tags are in `refs/tags`, the stash is in `refs/stash` 8336 8337### commit 8338a commit is a small file containing its parent(s), message, tree, and author 8339 8340`.git/objects/7622629` 8341``` 8342tree c4e6559 8343parent 037ab87 8344author Julia <[email protected]> 1697682215 8345committer Julia <[email protected]> 1697682215 8346commit message goes here 8347``` 8348these are compressed, the best way to see objects is with `git cat-file -p HASH` 8349 8350### trees 8351trees are small files with directory listings. The files in it are called "blobs" 8352 8353`.git/objects/c4e6559` 8354``` 8355100644 blob e351d93 404.html 8356100644 blob cab4165 hello.py 8357040000 tree 9de29f7 lib 8358``` 8359the permissions here LOOK like unix permissions, but they're actually super restricted, only 644 and 755 are allowed 8360 8361### blobs 8362blobs are the files that contain your actual code 8363 8364`.git/objects/cab4165` 8365`print("hello world!!!!")` 8366 8367### reflog 8368the reflog stores the history of every branch, tag, and `HEAD` 8369 8370`.git/logs/refs/heads/main` 8371``` 83722028ee0 c1f9a4c 8373Julia Evans <[email protected]> 83741683751582 8375commit: no ligatures in code 8376``` 8377each line of the reflog has: 8378- before/after commit IDs 8379- user + - timestamp 8380- log message 8381 8382### remote-tracking branches 8383remote-tracking branches store the most recently seen commit ID for a remote branch 8384 8385`.git/refs/remotes/origin/main` 8386`a9bbcae` 8387 8388when git status says "you're up to date with `origin/main`", it's just looking at this 8389 8390### .git/config 8391.git/config is a config file for the repository. it's where you configure your remotes 8392 8393`.git/config` 8394``` 8395[remote "origin"] 8396url = [email protected]: jvns/int-exposed 8397fetch = +refs/heads/*: refs/remotes/origin/* 8398[branch "main"] 8399remote = origin 8400merge refs/heads/main 8401``` 8402git has and local global settings, the local settings are here and the global ones are in `~/.gitconfig` 8403 8404### hooks 8405hooks are optional scripts that you can set up to run (eg before a commit) to do anything you want 8406
8407`.git/hooks/pre-commit` 8408``` 8409#!/bin/bash 8410any-commands-you-want 8411``` 8412 8413### the staging area 8414the staging area stores files when you're preparing to commit 8415 8416`.git/index` 8417`(binary file)`</div> 8418 </a> 8419 8420 <a href="/comics/git-head/"> 8421 <span class="title"> 8422 HEAD is the commit you have checked out 8423 </span> 8424 <div class="keywords">[git]</div> 8425 <div class="transcript">In git you always have some commit checked out. `HEAD` is a pointer to that commit and you'll see `HEAD` used a lot in this zine. Like a branch, `HEAD` is just a text file. Run `cat .git/HEAD` or `git status` to see the current `HEAD`. 8426 8427Examples of how to use HEAD: 8428- show the diff for the current commit: 8429`git show HEAD` 8430 8431- UNDO UNDO UNDO UNDO: reset branch to 16 commits ago 8432`git reset --hard HEAD~16` 8433(`HEAD~16` means 16 commits ago) 8434 8435- show what's changed since 6 commits ago: 8436`git diff HEAD~6` 8437 8438- squash a bunch of commits together 8439`git rebase -i HEAD~8` 8440(this opens an editor, use "fixup" to squash commits together)</div> 8441 </a> 8442 8443 <a href="/comics/head-and-heads/"> 8444 <span class="title"> 8445 HEAD and heads 8446 </span> 8447 <div class="keywords"></div> 8448 <div class="transcript">### panel 1: 8449 8450have you ever seen refs/heads/main or HEAD and wondered what they mean? 8451 8452here's the deal: 8453 8454* `head` = branch 8455* `HEAD` = current branch 8456 8457(yes, these are TERRIBLE names) 8458 8459 8460### panel 2: a head in git is a branch 8461 8462nobody really uses the term "head" for a branch except the official git docs though 8463 8464### panel 3: HEAD is the current branch 8465 8466for example HEAD could be set to main 8467 8468it's stored in .git/HEAD 8469 8470Unless you don't have a current branch... 8471 8472### panel 4: `HEAD` can be a commit ID instead of a branch 8473 8474 8475This means you have no current branch. Git calls this a "detached head state" (another terrible name!) 8476 8477(silly picture of a stick figure whose head has fallen off) 8478 8479fixing this is easy though: 8480 8481`git checkout BRANCHNAME` 8482 8483### panel 5: the current branch matters for these commands 8484 8485``` 8486git commit 8487git rebase 8488git merge 8489git cherry-pick 8490``` 8491 8492these 4 will work if you have no current branch but will create commits that you have no easy way to refer to 8493 8494``` 8495git pull 8496git push 8497``` 8498 8499these don't work at all if you're not a branch</div> 8500 </a> 8501 8502 <a href="/comics/git-mistakes-cant-fix/"> 8503 <span class="title"> 8504 git mistakes you can't fix 8505 </span> 8506 <div class="keywords"></div> 8507 <div class="transcript">Most mistakes you make with git can be fixed. If you've ever committed your code, you can get it back. That's what the rest of this zine is about! 8508 8509Here are the dangerous git commands: the ones that throw away uncommitted work. 8510 8511- `git reset --hard COMMIT` 85121. Throws away uncommitted changes 85132. Points current branch at `COMMIT` 8514 8515Very useful, but be careful to commit first if you don't want to lose your changes 8516 8517- `git clean` 8518 8519Deletes files that aren't tracked by Git. 8520 8521- `git checkout BRANCH FILE` (or directory) 8522 8523Replaces FILE with the version from `BRANCH`. 8524Will overwrite uncommitted changes.</div> 8525 </a> 8526 8527 <a href="/comics/git-discussion-bingo/"> 8528 <span class="title"> 8529 git discussion bingo 8530 </span> 8531 <div class="keywords"></div> 8532 <div class="transcript">A grid of boxes, like a bingo card, with the following text in them: 8533 8534- WTF is detached HEAD state 8535- just use magit 8536- subversion was so much worse 8537- rewriting history is bad 8538- I just do not care how git works 8539- I hate git 8540- git is a directed acyclic graph 8541- I only know 5 commands 8542- just spend 15 minutes learning git's internals 8543- content addressed storage 8544- git's design is so elegant 8545- you have to understand the linux kernel dev workflow 8546- a branch is just a pointer to a commit 8547- something about "porcelain" 8548- subversion was better 8549- I've used git for 10 years and I have no idea how it works 8550- mercurial is better 8551- git is not github 8552- the CLI is badly designed 8553- merge sucks, only use rebase 8554- something about Linus Torvalds 8555- commits are immutable snapshots 8556- you should just read Pro Git 8557- rebase sucks, only use merge 8558- I just delete my git repo if I mess it up</div> 8559 </a> 8560 8561 <a href="/comics/git-branches-the-rules/">
8562 <span class="title"> 8563 git branches: the rules 8564 </span> 8565 <div class="keywords"></div> 8566 <div class="transcript">### branches have very few rules 8567 8568git lets you move branches forwards, backwards, or sideways if you want 8569 8570Illustration of three circles in a vertical line, with an additional branch extending out of the middle circle. The top circle is labelled "`main`". The middle circle is labelled "You could move `main` here. The circle in its own branch is labelled "or here." 8571 8572### all changes to a branch are recorded in its reflog 8573 8574You can look at the reflog like this: 8575 8576`git reflog BRANCHNAME` 8577 8578reflog stands for "reference log" 8579 8580### when you delete a branch, its reflog is deleted 8581 8582Illustration of a sad stick figure with short curly hair, talking to a box with a smiley face representing git. 8583 8584person: what if I wanted to look at the history of that branch to recover something? 8585 8586git: too bad! 8587 8588### git will eventually delete any commit that isn't on a branch/tag/etc 8589 8590Illustration of four circles in a vertical line. The top one is labelled "`main`". There is a branch coming off of the second-from-bottom circle, and it is labelled "will be deleted by garbage collection after ~90 days unless you put it on a branch." 8591 8592### git `branch -d` won't let you delete unmerged branches 8593 8594Illustration of three circles in a vertical line. The top one is labelled "`main`". There is a branch coming off of the bottom circle, labelled "my branch (not merged)" 8595 8596to delete an unmerged branch, you need to force it with `-D` 8597 8598### rules git doesn't have about branches 8599 8600- when you push/pull a branch, the name doesn't have to match 8601- the main branch doesn't have any special protections in git itself (though tools like GitHub can protect it)</div> 8602 </a> 8603 8604 <a href="/comics/fixing-diverged-branches/"> 8605 <span class="title"> 8606 fixing diverged remotes 8607 </span> 8608 <div class="keywords"></div> 8609 <div class="transcript">### ways to reconcile two diverged branches 8610 8611Illustration of a sequence of boxes joined with lines. The first box is a star, the second box is a heart, and then it branches out into two boxes, one with a hash symbol and one with a squiggle. Hash symbol box is labelled âlocal mainâ and squiggle box is labelled âremote mainâ 8612 8613- combine the changes from both with (1) rebase or (2) merge! 8614- throw out your local changes (3) after breaking your local branch! 8615- throw out the remote changes (4) to get rid of something you accidentally pushed (be REAL careful with this one) 8616 8617### reasons to throw away changes 8618 8619- Iâll throw away local changes if I accidentally committed to `main` instead of a new branch 8620- Iâll throw away remote changes if I want to amend a commit after pushing it, and Iâm the only one working on that branch 8621 8622### 1. rebase 8623``` 8624git pull --rebase 8625git push 8626``` 8627Illustration of four boxes (star, heart, squiggle, hash) in a straight line, labelled âlocal mainâ and âremote mainâ 8628 8629Many people like to configure `git config pull.rebase true` to make this the default when they run `git pull` 8630 8631### 2. merge 8632``` 8633git pull --no-rebase 8634git push 8635``` 8636Illustration of two boxes (star and heart) that then diverge into two branches (hash and squiggle) then reconvene into a fifth box, with a diamond in it, labelled âlocal `main`â and âremote `main`â 8637 8638### 3. throw away local changes 8639``` 8640git switch -c newbranch 8641git switch main 8642git reset --hard origin/main 8643``` 8644(the first line is labelled âoptional: save your changes on `main` to `newbranch` so theyâre not orphaned) 8645 8646Illustration of two boxes (star and heart) that then diverge into two branches (hash and squiggle), which are labelled ânew branchâ and âlocal `main`" and "remote `main`â respectively. 8647 8648### 4. throw away remote changes (DANGER!) 8649 8650`git push --force` 8651 8652Illustration of two boxes (star and heart) that then diverge into two branches one with a hash symbol, labelled âlocal `main`, remote `main`â, and one with a squiggle, whose box is a dotted line, and thatâs labelled âorphanâ. 8653 8654I ONLY do this if there'
8654s nobody else working on the branch.</div> 8655 </a> 8656 8657 <a href="/comics/every-git-jargon/"> 8658 <span class="title"> 8659 every git jargon 8660 </span> 8661 <div class="keywords"></div> 8662 <div class="transcript">### config 8663``` 8664.git/config 8665hook 8666.gitconfig 8667alias 8668global 8669local 8670``` 8671 8672### history 8673``` 8674log 8675blame 8676bisect 8677diff 8678``` 8679 8680### commit 8681``` 8682commit 8683checkout 8684tree-ish 8685show 8686patch 8687apply 8688remotes 8689restore 8690``` 8691 8692### staging area 8693``` 8694index 8695staged 8696cached 8697grep 8698add 8699status 8700staging area 8701``` 8702 8703### branches 8704``` 8705HEAD 8706refs/heads/main 8707detached HEAD state 8708head 8709HEAD^, HEAD~, HEAD^^ 8710reference 8711symbolic reference 8712reset 8713tag 8714main 8715master 8716reflog 8717.. 8718... 8719``` 8720 8721### other features 8722``` 8723stash 8724worktree 8725subtree 8726submodule 8727revert 8728``` 8729 8730### merging 8731``` 8732merge 8733conflict 8734rebase 8735interactive rebase 8736fast forward merge 8737cherry-pick 8738squash 8739ours/theirs 8740``` 8741 8742### remotes 8743``` 8744upstream 8745downstream 8746push 8747pull 8748fetch 8749clone 8750fork 8751remote 8752refspec 8753origin 8754``` 8755</div> 8756 </a> 8757 8758 <a href="/comics/every-git-command/"> 8759 <span class="title"> 8760 every git command I use 8761 </span> 8762 <div class="keywords"></div> 8763 <div class="transcript">getting started: git init, git clone 8764 8765move between branches: git branch, git checkout, git switch 8766 8767restore old files: git checkout, git restore 8768 8769preparing to commit: git status, git add, git mv, git rm, git diff, git reset 8770 8771combining branches: git merge, git rebase, git cherry-pick 8772 8773working with others: git pull, git push, git fetch, git remote 8774 8775making commits: git commit 8776 8777configuring git: git config, git remote 8778 8779code archaeology: git blame, git log FILENAME, git log -S SEARCh, git show, git diff 8780 8781trash changes: git stash, git checkout ., git reset --hard, git rebase -i 8782 8783git troubleshooting: git log BRANCH, git status, git diff, git reflog 8784 8785editing history: git rebase -i, git reset --hard</div> 8786 </a> 8787 8788 <a href="/comics/commit-parent/"> 8789 <span class="title"> 8790 every commit has a parent 8791 </span> 8792 <div class="keywords">[git]</div> 8793 <div class="transcript">Every commit (except the first one!) has a parent commit! You can think of your git history as looking like this: 8794 8795current commit - c6045c - `HEAD` - "make cats blue" 8796parent - 304db6 - `HEAD^` - "add cats" 8797grandparent - a92eab - `HEAD^^` - "fix typo" 8798b29aff - "initial commit" 8799 8800`HEAD` always refers to the current commit you have checked out, and `HEAD^` is its parent. So if you want to go look at the code from the previous commit, you can run 8801 8802`git checkout HEAD^` 8803 8804commits don't always have 1 parent. Merge commits actually have 2 parents! 8805 8806`git log` shows you all the ancestors of the current commit, all the way back to the initial commit</div> 8807 </a> 8808 8809 <a href="/comics/diverged-branches/"> 8810 <span class="title"> 8811 diverged branches 8812 </span> 8813 <div class="keywords"></div> 8814 <div class="transcript">### when pushing/pulling, the hardest problems are caused by diverged branches 8815 8816sad error messages: 8817``` 8818! [rejected] 8819main -&gt; main 8820``` 8821(non `fast-forward`) 8822 8823`fatal: Not possible to fast-forward, aborting` 8824 8825`fatal: Need to specify how to reconcile divergent branches.` 8826 8827### what are diverged branches 8828 8829it looks like this: 8830 8831Diagram with two blank boxes, followed by a box with a heart in it, that then branches out into two branches, one with a hash symbol in it, labelled "local main", and one with a squiggle in it, labelled "remote main". 8832 8833### there are 4 possibilities with a remote branch 8834 88351. up to date (with a heart) 8836Illustration of three boxes in a row, labelled both "local" and "remote" 8837 88382. need to pull 8839Illustration of four boxes in a row. The second box in the sequence is labelled "local", the fourth branch is labelled "remote". 8840 88413. need to push 8842Illustration of four boxes in a row. The second box in the sequence is labelled "remote", the fourth branch is labelled "local". 8843 88444. diverged (need to decide how to solve it) (sad face) 8845Illustration of two boxes in a row, that then branches out into two branches. One of the branches has one box, labelled "remote", and the other branch has two boxes, labelled "local". 8846 8847### how to tell your branches have diverged: `git status` 8848 88491. `$ git fetch` (get the latest remote state first) 88502. `$ git status` 8851Your branch and '`origin/main`'
8851 have diverged, and have 1 and 1 different commits each, respectively. 8852 8853(use "`git pull`" to merge the remote branch into yours) 8854 8855(diverged is highlighted) 8856 8857### fix diverged branches before making more commits 8858 8859First illustration: two boxes in a row, then branches out into two branches, each with one box. It's labelled "not so bad to resolve..." 8860 8861Second illustration: two boxes in a row, then branches out into two branches, but each branch has a whole bunch of boxes. 8862 8863Illustration of a sad stick figure with curly hair. 8864 8865person: oh no 8866 8867### there's no one solution 8868 8869Illustration of a smiling stick figure with curly hair. 8870 8871person: on the next page we'll talk about some options!</div> 8872 </a> 8873 8874 <a href="/comics/detached-head-state/"> 8875 <span class="title"> 8876 detached HEAD state 8877 </span> 8878 <div class="keywords"></div> 8879 <div class="transcript">### how git knows what your current branch is: .git/HEAD 8880 8881`.git/HEAD` is a file where git stores either: 88821. a branch name: the current branch 88832. a commit ID: this means you don't have a current branch. git calls this "detached HEAD state" 8884 8885### by itself, .git/HEAD being a commit ID is okay 8886 8887Illustration of a smiling stick figure with short curly hair. 8888 8889person: it's a great way to look at an old version of your code! 8890 8891I don't do it often, but it's super useful! 8892 8893git does it internally during a rebase! 8894 8895### the only problem is that new commits you make can get "lost" (page 13) 8896 8897Illustration of five dots in a vertical stack, connected by lines. The top dot is labelled "main" and the bottom dot is labelled "HEAD". There is a dotted line branching off from "HEAD". The dot at the end of the dotted line is labelled "new commit will go here. danger! it won't be on any branch!" 8898 8899### ways you can end up in detached HEAD state 8900 8901You will end up in detached HEAD state if you checkout: 8902 8903- a tag 8904 `$ git checkout v1.3` 8905- a remote-tracking branch 8906 `$ git checkout origin/main` 8907- a commit ID 8908 `$ git checkout a3ffab9` 8909 8910### if you accidentally create commits in detached HEAD state, it's SUPER easy to avoid losing them 8911 8912just create a new branch! 8913 8914`git checkout -b oops` 8915 8916(you can also create a branch with `git switch -c` if you prefer) 8917 8918### git has a little language for referring to commits 8919 8920- the current commit: `HEAD` 8921- the previous commit: `HEAD^` 8922- 3 commits ago: `HEAD^^^` 8923- 3 commits ago: `HEAD~3` 8924 8925The full documentation is at: 8926`man gitrevisions` 8927</div> 8928 </a> 8929 8930 <a href="/comics/combining-diverged-branches/"> 8931 <span class="title"> 8932 combining diverged branches 8933 </span> 8934 <div class="keywords"></div> 8935 <div class="transcript">### there are 3 options for combining branches 8936 8937- merge 8938- rebase 8939- squash 8940 8941for example, letâs say weâre combining these 2 branches: 8942 8943Diagram: A box with a heart. To its right is a box with a star. From here, it branches out into branch 1, which consists of one box with a hash symbol, and branch 2, which consists of a branch with a spiral, followed by a branch with a squiggle. 8944 8945### panel 2: 8946 8947git rebase 8948 8949Diagram: A box with a heart. To its right is a box with a star. From here, it branches out into branch 1, which consists of a box with a hash symbol, followed by a branch with a spiral, then a box with a squiggle. Branch 2 consists of a box with a spiral, followed by a box with a squiggle. Branch 2 is made up of dotted lines and labelled âlostâ. 8950 8951git merge 8952 8953Diagram: A box with a heart. To its right is a box with a star. From here, it branches out into branch 1, which consists of a box with a hash symbol. Branch 2 consists of a box with a spiral, followed by a box with a squiggle. Branches 1 and 2 both lead into a new box, with a diamond. 8954 8955git merge --squash 8956 8957Diagram: A box with a heart. To its right is a box with a star. From here, it branches out into branch 1, which consists of a box with a hash symbol, followed by a new box containing both a squiggle and a spiral. Branch 2 consists of a box with a spiral, followed by a box with a squiggle. Branch 2 has a box with a spiral, followed by a branch with a squiggle. 8958 8959### all 3 methods result in the EXACT SAME FILES 8960 8961some differences are: 8962- the diff git shows you for the final commit 8963- the commit ids 8964- the specific flavour of suffering the method causes 8965 8966### rebase 8967 8968pro: you can keep your git history simple: 8969 8970Diagram: a git history that is just a series of boxes in a straight line. 8971 8972pain: 8973- harder to learn [sad face] 8974- harder to undo [sad face] 8975- easier to mess up [sad face] 8976 8977(I love rebase though!) 8978### merge 8979 8980pro: if you mess something up, the original commits are still in your branchâs history 8981 8982pain: when I look at histories like this I feel dread [sad face] 8983 8984Diagram: a complicated git history with a number of different branches. 8985### squash 8986 8987pro: have 20 messy commits? nobody needs to know! 8988 8989And itâs pretty simple to use. 8990 8991pain: âugh, someone squashed their 3000-line branch into 1 commitâ [sad face] 8992</div> 8993 </a> 8994 8995 <a href="/comics/combining-branches/">
8996 <span class="title"> 8997 combining branches 8998 </span> 8999 <div class="keywords"></div> 9000 <div class="transcript">### there are 3 options for combining branches 9001 9002* `merge` 9003* `rebase` 9004* `squash` 9005 9006for example, let's say we're combining these 2 branches: 9007 9008Diagram: A box with a heart. To its right is a box with a star. From here, it branches out into branch 1, which consists of one box with a hash symbol, and branch 2, which consists of a branch with a spiral, followed by a branch with a squiggle. 9009 9010### panel 2: 9011 90121. `git rebase` 9013Diagram: A box with a heart. To its right is a box with a star. From here, it branches out into branch 1, which consists of a box with a hash symbol, followed by a branch with a spiral, then a box with a squiggle. Branch 2 consists of a box with a spiral, followed by a box with a squiggle. Branch 2 is made up of dotted lines and labelled "orphan". 9014 90152. `git merge` 9016Diagram: A box with a heart. To its right is a box with a star. From here, it branches out into branch 1, which consists of a box with a hash symbol. Branch 2 consists of a box with a spiral, followed by a box with a squiggle. Branches 1 and 2 both lead into a new box, with a diamond. 9017 90183. `git merge --squash` 9019Diagram: A box with a heart. To its right is a box with a star. From here, it branches out into branch 1, which consists of a box with a hash symbol, followed by a new box containing both a squiggle and a spiral. Branch 2 consists of a box with a spiral, followed by a box with a squiggle. Branch 2 is made up of dotted lines and labelled "orphan". 9020 9021### all 3 methods result in the EXACT SAME FILES 9022 9023some differences are: 9024 9025* the diff git shows you for the final commit 9026* the specific flavour of suffering the method causes 9027 9028### merge 9029 9030pro: if you mess something up, the original commits are still in your branch's history 9031 9032pain: when I look at histories like this I feel dread 9033 9034Diagram: a complicated git history with a number of different branches. 9035 9036### rebase 9037 9038pro: you can keep your git history simple: 9039 9040Diagram: a git history that is just a series of boxes in a straight line. 9041 9042pain: 9043- harder to learn [sad face] 9044- harder to undo [sad face] 9045- easier to mess up [sad face] 9046 9047(I love rebase though!) 9048 9049### squash 9050 9051pro: have 20 messy commits? nobody needs to know! 9052And it's pretty simple to use. 9053 9054pain: "ugh, someone squashed their 3000-line branch into 1 commit"</div> 9055 </a> 9056 9057 <a href="/comics/branches-have-no-rules/"> 9058 <span class="title"> 9059 branches have no rules 9060 </span> 9061 <div class="keywords"></div> 9062 <div class="transcript">### you might expect git to enforce some rules about branches 9063 9064some rules you might imagine: 9065 9066* you can't remove commits from a branch, only add them
9067* the `main` branch has to stay more less in sync with `origin/main` 9068 9069But there are no rules. 9070 9071git character with demon hat: want to do something horrible to your branch? no problem! 9072 9073### there are literally no rules 9074 9075commands that you can use to do weird stuff to a branch: 9076 9077* `git reset` 9078* `git rebase` 9079 9080### instead of rules, we have conventions 9081 9082for example: 9083 9084* run `git pull` often to keep your `main` up to date 9085* if you're working with a big team, don't commit to `main` directly 9086 9087Illustration of the git demon talking to a nonplussed stick figure with curly hair. 9088 9089git demon: you've just gotta be really careful to not do the wrong thing and not mess up your branch 9090 9091person: um... thanks? 9092 9093### our only saviour: the reflog 9094 9095`git reflog BRANCHNAME` 9096 9097will show you the history of every change to the branch, so you can always undo 9098 9099the reflog is a VERY unfriendly UI, but it's always there.</div> 9100 </a> 9101 9102 <a href="/comics/tcp-how-to-reliably-get-a-cat/"> 9103 <span class="title"> 9104 TCP: how to reliably get a cat 9105 </span> 9106 <div class="keywords"></div> 9107 <div class="transcript">Step 3 in our plan is "open a TCP connection!" Let's learn what this "TCP" thing even is 9108 9109### When you send a packet sometimes it gets lost 9110 9111jvns.ca server â Cat packets â lightning bolt 9112laptop: nope never got it 9113 9114### TCP lets you send a stream of data reliably, even if packets get lost or sent in the wrong order. 9115 9116four butterflies, labelled TCP C, TCP D, TCP D (duplicates), TCP A, and TCP B 9117 9118laptop: it says "abcd"! 9119 9120### how does TCP work, you ask? WELL! 9121 9122### how to know what order the packets should gо in:
9123Every packet says what range of bytes it has. 9124 9125Like this: 9126once upon a ti â bytes 0-13 9127agical oysterbytes â 30-42 9128me there was a m â bytes 14-29 9129 9130Then the client can assemble all the pieces into: 9131"once upon a time there was a magical oyster" 9132 9133The position of the first byte (0,14,30 in our example) is called the "sequence number" 9134 9135### how to deal with lost packets: 9136When you get TCP data, you have to acknowledge it (ACK): 9137 9138jvns.ca server: here is part of a cat picture! that should be 28832 bytes so far! 9139jvns.ca server (thinking): yay 9140 9141laptop: ACK! I have received all 28832 bytes 9142 9143If the server doesn't get an acknowledgement, it will retry sending the data.</div> 9144 </a> 9145 9146 <a href="/comics/misc-networking-tools/"> 9147 <span class="title"> 9148 miscellaneous networking tools 9149 </span> 9150 <div class="keywords"></div> 9151 <div class="transcript">### stunnel 9152make a SSL proxy for an insecure server 9153 9154### hping3 9155make any TCP packet 9156 9157### wget 9158download files 9159 9160### aria2c 9161a fancier wget 9162 9163### rsync 9164sync files over SSH or locally 9165 9166### lsof 9167what ports are being used? 9168 9169### httpie 9170like curl but friendlier 9171 9172### iftop/nethogs/ntop/iptraf/nload 9173see what's using bandwidth 9174 9175### whois 9176is this domain registered? 9177 9178### ipcalc 9179easily see what 13.21.2.3/25 means 9180 9181### python3 -m http.server 9182serve files from a directory 9183 9184### nftables 9185new version of iptables 9186 9187### zenmap 9188GUI for nmap 9189 9190### p0f 9191identify OS of hosts connecting to you 9192 9193### openVPN, wireguard 9194VPNs 9195 9196### tcpflow 9197capture and assemble TCP streams 9198 9199### sysctl 9200configure Linux kernel's network stack 9201 9202### ab/iperf 9203benchmarking tools 9204 9205### links 9206a browser in your terminal 9207 9208### telnet 9209can help debug text network protocols</div> 9210 </a> 9211 9212 <a href="/comics/every-linux-networking-tool-i-know/"> 9213 <span class="title"> 9214 every Linux networking tool I know 9215 </span> 9216 <div class="keywords"></div> 9217 <div class="transcript">### ping 9218"are these computers even connected?" 9219 9220### curl 9221make any HTTP request you want 9222 9223### httpie 9224like curl but easier ("http get") 9225 9226### wget 9227download files 9228 9229### tc 9230on a linux router, slow down your brother's internet (and much more) 9231 9232### dig/nslookup 9233what's the IP for that domain? (DNS query) 9234 9235### whois 9236is this domain registered? 9237 9238### ssh 9239secure shell ð 9240 9241### scp 9242copy files over a SSH connection 9243 9244### rsync 9245copy only changed files (works over SSH) 9246 9247### ngrep 9248grep for your network 9249 9250### tcpdump 9251"show me all packets on Port 80!" 9252 9253### wireshark 9254look at those packets in a GUI 9255 9256### tshark 9257command line super powerful packet analysis 9258 9259### tcpflow 9260capture & assemble TCP streams 9261 9262### ifconfig 9263"what's my IP address?" 9264 9265### route 9266view & change the route table 9267 9268### ip 9269replaces ifconfig, route, and more! 9270 9271### arp 9272see your ARP table 9273 9274### mitmproxy 9275spy on SSL connections your programs are making 9276 9277### nmap 9278in ur network scanning ur ports 9279 9280### zenmap 9281GUI for nmap 9282 9283### p0f 9284identify OS of hosts connecting to you 9285 9286### openvpn 9287a VPN 9288 9289### wireguard 9290a newer VPN 9291 9292### nc 9293netcat! make TCP connections manually 9294 9295### socat 9296proxy a TCP socket to a unix domain socket + LOTS MORE 9297 9298### telnet 9299like SSH but insecure 9300 9301### ftp/sftp 9302copy files. sftp does it over SSH. 9303 9304### netstat/ss/sof/fuser 9305"what ports are servers using?" 9306 9307### iptables 9308set up firewalls and NAT! 9309 9310### nftables 9311new version of iptables 9312 9313### hping3 9314construct any TCP packet you want 9315 9316### traceroute/mtr 9317what servers are on the way to that server? 9318 9319### tcptraceroute 9320Use top packets instead of icmp to traceroute 9321 9322### ethtool 9323manage physical Ethernet connections + network cards. 9324 9325### iw/iwconfig 9326manage wireless network settings (see speed/frequency!) 9327 9328### sysctl 9329configure Linux kernel's network stack 9330 9331### openssl 9332do literally anything with SSL certificates. 9333 9334### stunnel 9335make a SSL proxy server for an insecure server 9336 9337### iptraf/nethogs/iftop/ntop 9338see what's using bandwidth 9339 9340### ab/nload/perf 9341benchmarking tools 9342 9343### python 3 -m http.server 9344serve files from a directory 9345 9346### ipcalc 9347easily see what 13.21.2.3/25 means 9348 9349### nsenter 9350enter a container process's network namespace</div> 9351 </a> 9352 9353 <a href="/comics/bpf/">
9354 <span class="title"> 9355 BPF cheat sheet 9356 </span> 9357 <div class="keywords">[tcpdump]</div> 9358 <div class="transcript"></div> 9359 </a> 9360 9361 <a href="/comics/how-kubernetes-can-break-networking/"> 9362 <span class="title"> 9363 how kubernetes can break - networking 9364 </span> 9365 <div class="keywords"></div> 9366 <div class="transcript"></div> 9367 </a> 9368 9369 <a href="/comics/how-kubernetes-can-break-etcd/"> 9370 <span class="title"> 9371 how kubernetes can break - etcd 9372 </span> 9373 <div class="keywords"></div> 9374 <div class="transcript"></div> 9375 </a> 9376 9377 <a href="/comics/why-the-same-origin-policy-matters/"> 9378 <span class="title"> 9379 why the same origin policy matters 9380 </span> 9381 <div class="keywords"></div> 9382 <div class="transcript">Browsers work hard to make sure that `evil.com` can't make requests to `other-website.com`. But `evil.com` can request `other-website.com` from its own server. So what's the big deal? 9383 9384Here are 2 reasons it's important to prevent Javascript code from making arbitrary requests from your browser: 9385 9386### Reason 1: cookies 9387Browsers often send your cookies with HTTP requests. You don't want `evil.com` to be able to make requests using your login cookies. They'd be logged in as you! 9388 9389evil.com Javascript: Send a GET request to mail.google.com with their current login cookies. 9390 9391browser: I'll do it, but you can't see the response unless the server says it's okay. 9392 9393(the browser will actually do it!) 9394 9395### Reason 2 : network access 9396You might be on a private network (for example your company's corporate network) that `evil.com` doesn't have access to, but your computer does. 9397 9398evil.com Javascript: POST request to secrets.corp.company.com/send_money please. 9399 9400browser: No! Same origin policy! I'm not even going to make that request without checking first.</div> 9401 </a> 9402 9403 <a href="/comics/https/"> 9404 <span class="title"> 9405 HTTPS 9406 </span> 9407 <div class="keywords"></div> 9408 <div class="transcript">HTTPS: HTTP + secure 9409 9410Here's what your browser does when it asks for 9411`https://examplecat.com/cat.png:` 9412 94131. Negotiate an encryption key (AES symmetric key) to use for this connection to examplecat.com. The browser and server will use the same key to encrypt/decrypt content. 9414 9415Simplified version of how picking the encryption key works: 9416 9417browser, represented by the Firefox logo: hey I want examplecat.com 9418server, represented by a box with a smiley face: here's proof that I'm examplecat.com 9419browser, thinking: story checks out! 9420browser: key exchange 9421server: key exchange 9422browser and server, thinking: we're going to use A$29FXY2.... as the encryption key 9423 9424This protocol for secure communication is called TLS (previously SSL) and you can use it on any TCP connection 9425 94262. Write an HTTP request 9427``` 9428GET /cat.png HTTP/1.1 9429Host: examplecat.com 9430User-Agent: Mozilla/... 9431``` 9432 94333. Encrypt the HTTP request With AES & send if to examplecat.com 9434 9435browser: $Af9bbca^~gggBF 9436server, thinking: ah, I see, 9437``` 9438GET /cat.png HTTP/1.1 9439Host: examplecat.com 9440... 9441``` 9442 94434. Receive encrypted HTTP response 9444 9445server: BXF^56â¡gxx... 9446browser: nice, that means 9447``` 9448200 OK 9449Content-Type: image/png 9450... 9451```</div> 9452 </a> 9453 9454 <a href="/comics/http-redirects/"> 9455 <span class="title"> 9456 HTTP redirects 9457 </span> 9458 <div class="keywords"></div> 9459 <div class="transcript">Sometimes you type a URL into your browser: 9460`examplecat.com/dog.png` 9461 9462but end up at a slightly different URL: 9463`examplecat.com/cat.png` 9464ooh, where did the cat come from? I didn't type that! 9465 9466### Here's what's going on behind the scenes: 9467 9468browser: 9469``` 9470GET /dog.png 9471HTTP/1.1 Host: examplecat.com 9472``` 9473server: 9474``` 9475301 Moved Permanently 9476Location: /cat.png 9477``` 9478browser: okay, I'll try `/cat.png` instead 9479 9480browser: 9481``` 9482GET /cat.png HTTP/1.1 9483Host: examplecat.com 9484``` 9485server: 9486``` 9487200 OK 9488<rest of website here> 9489``` 9490The Location header tells the browser what new URL to use. 9491 9492The new URL doesn't have to be on the same domain: examplecat.com/panda can redirect to pandas.com. Setting up redirects is a great thing to do if you move your site to a new domain! 9493 9494### ! Warning ! 9495`301 Moved Permanently` redirects are PERMANENT: after a browser sees one once, it'll always use `examplecat.com/cat.png` when someone types `examplecat.com/dog.png` forever. You can't take it back and decide to not to redirect. If you're not sure you want to redirect your site for eternity, use `302 Found` to redirect instead.</div> 9496 </a> 9497 9498 <a href="/comics/http-exercises/">
9499 <span class="title"> 9500 HTTP exercises 9501 </span> 9502 <div class="keywords"></div> 9503 <div class="transcript">Making HTTP requests with curl to real internet websites and trying different headers is my favourite way to play around with HTTP & learn. 9504 9505### curl tips: 9506- `-i` shows the response headers 9507- `-I`shows the response headers (by sending a HEAD request) 9508- `-H` adds a request header 9509 9510Try the Range header: 9511`curl -i https://examplecat.com/cat.txt -H "Range: bytes=8-17"` 9512 9513Request (and print out!) a compressed response: 9514``` 9515curl -i https://examplecat.com 9516 -H "Accept-Encoding: gzip" -- output - 9517``` 9518 9519Get a webpage in Spanish: 9520`curl -i https://twitter.com -H "Accept-Language: es-ES` 9521 9522Get redirected to another URL: 9523(hint: look at the `Location` header!) 9524`curl -i http://examplecat.com` 9525 9526Guess what content delivery network Github iS using: 9527(hint: it's in a header starting with xâ) 9528`curl -I https://github.githubassets.com` 9529 9530Find out when example.com was last updated 9531(hint: `LastâModified`) 9532`curl -I example.com` 9533 9534Get a 404 not found: 9535`curl -i examplecat.com/bananas`</div> 9536 </a> 9537 9538 <a href="/comics/opening-file/"> 9539 <span class="title"> 9540 opening a file 9541 </span> 9542 <div class="keywords">[linux2]</div> 9543 <div class="transcript"></div> 9544 </a> 9545 9546 <a href="/comics/surprise/"> 9547 <span class="title"> 9548 writing tip: say something surprising 9549 </span> 9550 <div class="keywords"></div> 9551 <div class="transcript"></div> 9552 </a> 9553 9554 <a href="/comics/writing-ask-questions/"> 9555 <span class="title"> 9556 writing tip: ask good questions 9557 </span> 9558 <div class="keywords"></div> 9559 <div class="transcript"></div> 9560 </a> 9561 9562 <a href="/comics/scenes-from-kubernetes/"> 9563 <span class="title"> 9564 scenes from kubernetes 9565 </span> 9566 <div class="keywords"></div> 9567 <div class="transcript"></div> 9568 </a> 9569 9570 <a href="/comics/kubernetes-components/"> 9571 <span class="title"> 9572 kubernetes components 9573 </span> 9574 <div class="keywords"></div> 9575 <div class="transcript"></div> 9576 </a> 9577 9578 <a href="/comics/command-line-arguments/"> 9579 <span class="title"> 9580 command line arguments 9581 </span> 9582 <div class="keywords">[linux2]</div> 9583 <div class="transcript"> 9584### every process has command line arguments 9585`$ ls 1 /usr/bin` 9586(`ls`, `-l`, and `usr` are arguments!) 9587 9588### they're passed to the program as an array 9589example from Python: 9590``` 9591import sys 9592print(sys.argv) 9593``` 9594`['test.py', 'file.txt' ]` 9595 9596### arguments can be any sequence of bytes 9597`$ python program.py â¥` 9598(emoji are totally allowed!) 9599 9600### the first argument is the executable's name 9601``` 9602[ 9603'ls' 9604'-1', 9605'/usr/bin/' 9606] 9607``` 9608(`ls` is the executable name) 9609 9610### the total length of the arguments is limited 9611you can find the limits on your system with `xargs -show-limits` 9612It's usually ~2MB 9613 9614### you can decide how you parse arguments 9615- `-flag`: single dash! 9616- `--flag`: 2 dashes! 9617- `â¥â¥flag`: weird emoji scheme that will be very annoying to use!</div> 9618 </a> 9619 9620 <a href="/comics/file-locking/"> 9621 <span class="title"> 9622 file locking 9623 </span> 9624 <div class="keywords"></div> 9625 <div class="transcript"></div> 9626 </a> 9627 9628 <a href="/comics/terminal-escape-codes/"> 9629 <span class="title"> 9630 terminal escape codes let you change colour 9631 </span> 9632 <div class="keywords"></div> 9633 <div class="transcript"></div> 9634 </a> 9635 9636 <a href="/comics/imsi-catchers-fake-cellphone-towers/"> 9637 <span class="title"> 9638 IMSI catchers (fake cellphone towers) 9639 </span> 9640 <div class="keywords"></div> 9641 <div class="transcript"></div> 9642 </a> 9643 9644 <a href="/comics/clock-gettime/"> 9645 <span class="title"> 9646 clock_gettime 9647 </span> 9648 <div class="keywords">[linux2]</div> 9649 <div class="transcript">### programs can be slow for a lot of reasons 9650 9651Illustration of two programs, each represented by a box with a smiley face. 9652 9653program 1: I'm waiting for a database query, you? 9654program 2: I'm using SO MUCH CPU! 9655 9656### it'
9656s not obvious when a program is using CPU 9657Illustration of a stick figure with curly hair, looking unhappy. 9658 9659person: my webserver took 6 seconds to respond to that request! why? 9660 9661### panel 3 9662person: how can I tell how much CPU time was used in this part of my code? 9663 9664### clock-gettime 9665clock-gettime is a system call. It can tell you how much CPU time your process/thread used since it started. 9666 9667### how to track CPU time 96681. run clock-gettime 96692. do the thing (eg handle a HTTP request) 96703. run clock-gettime 96714. subtract! 9672 9673### this trick works when You have 1 HTTP request per thread at a time 9674Illustration of Ruby and node.js, each represented by a box with a smiley face. 9675 9676Ruby: I can use clock-gettime 9677node.js: doesn't work for me, I have an event loop!</div> 9678 </a> 9679 9680 <a href="/comics/ways-to-count-rows/"> 9681 <span class="title"> 9682 ways to count rows 9683 </span> 9684 <div class="keywords"></div> 9685 <div class="transcript">Here are three ways to count rows: 9686 96871. `COUNT(*)`: count all rows 9688 9689This counts every row, regardless of the values in the row. Often used with a `GROUP BY` to get common values, like in this "most popular names" query: 9690``` 9691SELECT first_name, COUNT(*) 9692FROM people 9693GROUP BY first_name 9694ORDER BY COUNT(*) DESC 9695LIMIT 50 9696``` 96972. `COUNT(DISTINCT column)`: get the number of distinct values 9698 9699Really useful when a column has duplicate values. For example, this query finds out how many species every plant genus has: 9700``` 9701SELECT genus, COUNT (DISTINCT species) 9702FROM plants 9703GROUP BY 1 9704ORDER BY 2 9705ORDER BY DESC 9706``` 9707"`GROUP BY 1`" means group by the first expression in the `SELECT`" 9708 97093. `SUM(CASE WHEN expression THEN 1 ELSE 0 END)` 9710This trick using `SUM` and `CASE` lets you count how cats vs dogs vs other animals each owner has: 9711``` 9712SELECT owner 9713, SUM(CASE WHEN type = 'dog' then 1 else 0 end) AS num_dogs 9714, SUM(CASE WHEN type = 'cat' then 1 else 0 end) AS num_cats 9715, SUM(CASE WHEN type NOT IN ('dog', 'cat') then 1 else 0 end) 9716end) AS num_other 9717FROM pets GROUP BY owner 9718``` 9719 9720pets: 9721 9722| owner | type | 9723|------------|-------------| 9724| 1 | dog | 9725| 1 | cat | 9726| 2 | dog | 9727| 2 | parakeet | 9728 9729 9730| owner | num_dogs | num_cats | num_other | 9731|----------|-------------|-------------|--------------| 9732| 1 | 1 | 1 | 0 | 9733| 2 | 1 | 0 | 1 |</div> 9734 </a> 9735 9736 <a href="/comics/example-join/"> 9737 <span class="title"> 9738 SQL example: LEFT JOIN + GROUP BY 9739 </span> 9740 <div class="keywords"></div> 9741 <div class="transcript">## example: LEFT JOIN + GROUP BY 9742 9743This query counts how many items every client bought 9744(including clients who didn't buy anything): 9745 9746``` 9747SELECT name, COUNT (item) AS items_bought 9748FROM owners LEFT JOIN sales 9749 ON owners.id = sales.client 9750GROUP BY name 9751ORDER BY items_bought DESC 9752``` 9753 9754`FROM owners LEFT JOIN sales...` 9755 9756 9757#### owners 9758 9759| id | name | 9760|----|---------| 9761| 1 | maher | 9762| 2 | rishi | 9763| 3 | chandra | 9764 9765 9766#### sales 9767 9768| item | client | 9769|--------|--------| 9770| catnip | 1 | 9771| laser | 1 | 9772| tuna | 1 | 9773| tuna | 2 | 9774 9775 9776`ON owners.id=sales.client` 9777 9778| id | name | item | 9779|----|---------|--------| 9780| 1 | maher | catnip | 9781| 1 | maher | laser | 9782| 1 | maher | tuna | 9783| 2 | rishi | tuna | 9784| 3 | chandra | NULL | 9785 9786 9787`GROUP BY name` 9788 9789(same chart as previous, except the "maher" rows are circled, as are the "rishi" and "chandra" rows) 9790 9791 9792`SELECT name, COUNT(item) AS items.bought` 9793 9794| name | items_bought | 9795|---------|--------------| 9796| rishi | 1 | 9797| chandra | 0 | 9798| maher | 3 | 9799 9800(`COUNT(item)` doesn't count `NULL`s) 9801 9802 9803`ORDER BY items_bought DESC` 9804 9805| name | items_bought | 9806|---------|--------------| 9807| maher | 3 | 9808| rishi | 1 | 9809| chandra | 0 |</div> 9810 </a> 9811 9812 <a href="/comics/example-baby-feedings/">
9813 <span class="title"> 9814 SQL example: get the time between baby feedings 9815 </span> 9816 <div class="keywords"></div> 9817 <div class="transcript">This query finds the time since a baby's last feeding/diaper change. 9818 9819``` 9820SELECT event, hour, 9821 hour - LAG(hour) OVER(PARTITION BY event ORDER BY hour ASC) 9822 AS time_since_last 9823FROM baby_log 9824WHERE event in ('feeding', 'diaper') 9825ORDER BY hour ASC 9826``` 9827 98281. `FROM baby_log` 9829 9830| event | hour | 9831|------------|------| 9832| feeding | 1 | 9833| cough | 1 | 9834| diaper | 3 | 9835| feeding | 4 | 9836| diaper | 5 | 9837| diaper | 5 | 9838| feeding | 7 | 9839| cough | 7 | 9840 9841 98422. `WHERE event IN ('diaper', 'feeding') 9843 9844| event | hour | 9845|------------|------| 9846| feeding | 1 | 9847| diaper | 3 | 9848| feeding | 4 | 9849| diaper | 5 | 9850| diaper | 5 | 9851| feeding | 7 | 9852 9853 98543. OVER (PARTITION BY event ORDER BY hour ASC) 9855(this `ORDER BY` only affects the windows, not the query output) 9856 9857(There's a diagram of the table from step 2. There are arrows pointing to two smaller tables that break out only the lines where the event is "feeding", and only the lines where the event is "diaper", respectively) 9858 9859 98604. `SELECT` type, hour, hour-LAG(hour) 9861 9862| event | hour | time_since_last | 9863|------------|------|------------------------------------------------------------| 9864| feeding | 1 | `NULL` (`LAG()` is `NULL` for the first row in the window) | 9865| feeding | 4 | 3 | 9866| feeding | 7 | 3 | 9867| diaper | 3 | `NULL` | 9868| diaper | 5 | 2 | 9869| diaper | 5 | 0 | 9870 9871 98725. `ORDER BY hour ASC` 9873 9874| event | hour | time_since_last | 9875|------------|------|------------------| 9876| feeding | 1 | `NULL` | 9877| diaper | 3 | `NULL` | 9878| feeding | 4 | 3 | 9879| diaper | 5 | 2 | 9880| diaper | 5 | 0 | 9881| feeding | 7 | 3 |</div> 9882 </a> 9883 9884 <a href="/comics/select/"> 9885 <span class="title"> 9886 SELECT 9887 </span> 9888 <div class="keywords"></div> 9889 <div class="transcript">SELECT is where you pick the final columns that appear in the table the query outputs. Here's the syntax: 9890 9891``` 9892SELECT expression_l [AS alias], 9893 expression_2 [AS alias2], 9894FROM ... 9895``` 9896Some useful things YOU can do in a SELECT : 9897 9898- Combine many columns with SQL expressions 9899A few examples: 9900``` 9901CONCAT (first_name, ' ', last_name) 9902DATE_TRUNC('month', created) 9903``` 9904 9905(This is PostgreSQL syntax for rounding date, other SQL dialects have different syntax) 9906 9907- Alias an expression with AS 9908 9909`first_name || ' ' || last_name AS full_name` is a mouthful! If you alias an expression with AS, you can use the alias elsewhere in the query to refer to that expression. 9910 9911(|| is a concatenation operation) 9912 9913``` 9914SELECT first_name || ' ' || last_name AS full_name 9915FROM people 9916ORDER BY full_name desc 9917``` 9918(`full_name` refers to `first_name || ' ' || last_name`) 9919 9920 9921- Select columns with SELECT * 9922 9923When I'm starting to figure out a query, I'll often write something like 9924``` 9925SELECT * FROM some _ table LIMIT 10 9926``` 9927just to quickly see what the columns in the table look like. 9928</div> 9929 </a> 9930 9931 <a href="/comics/over-assigns-window/"> 9932 <span class="title"> 9933 OVER() assigns every row a window 9934 </span> 9935 <div class="keywords"></div> 9936 <div class="transcript">A "window" is a set of rows: 9937 9938| name | class | grade | 9939|---------|---------|---------| 9940| juan | 1 | 93 | 9941| lucia | 1 | 98 | 9942 9943(a window!) 9944 9945A window can be as big as the whole table (an empty `OVER()` is the whole table!) or as small as just one row. 9946 9947`OVER()` is confusing at first, so here's an example! Let's run this query that ranks students in each class by grade: 9948 9949``` 9950SELECT name, class, grade, 9951 ROW_NUMBER() OVER (PARTITION BY class 9952 ORDER BY grade DESC) 9953 AS rank_in_class 9954FROM grades 9955``` 9956 9957Step 1: Assign every row a window. `OVER (PARTITION BY class)` means that there are 2 windows: one each for class 1 and 2. 9958 9959grades: 9960 9961| name | class | grade | 9962|---------|---------|---------| 9963| juan | 1 | 93 | 9964| lucia | 1 | 98 | 9965| raph | 2 | 88 | 9966| chen | 2 | 90 | 9967 9968(Beside this table there is an illustration of two smaller popped-out tables showing the first two rows, and the second two rows respectively) 9969 9970Step 2: Run the function. We need to run `ROW_NUMBER()` to find each row's rank in its window: 9971 9972query output: 9973 9974| name | class | grade | rank_in_class | 9975|---------|---------|---------|---------------| 9976| juan | 1 | 93 | 2 | 9977| lucia | 1 | 98 | 1 | 9978| raph | 2 | 88 | 2 | 9979| chen | 2 | 90 | 1 |</div> 9980 </a> 9981 9982 <a href="/comics/order-by-limit/">
9983 <span class="title"> 9984 ORDER BY and LIMIT 9985 </span> 9986 <div class="keywords"></div> 9987 <div class="transcript">### `ORDER BY` and `LIMIT` 9988 9989`ORDER BY` and `LIMIT` happen at the end and affect the final output of the query. 9990 9991`ORDER BY` lets you sort by anything you want! 9992 9993The syntax is: 9994`ORDER BY` [expression] `ASC` or `DESC` 9995 9996(`ASC` stands for ascending) 9997 9998For example, this query sorts cats by the length of their name (shortest first): 9999 10000``` 10001SELECT * FROM cats 10002ORDER BY LENGTH(name) ASC 10003``` 10004 10005cats: 10006 10007| owner | name | 10008|-------|------------| 10009| 1 | daisy | 10010| 1 | dragonsnap | 10011| 3 | buttercup | 10012| 4 | rose | 10013 10014 10015results of query: 10016 10017| owner | name | 10018|-------|------------| 10019| 4 | rose | 10020| 1 | daisy | 10021| 3 | buttercup | 10022| 1 | dragonsnap | 10023 10024 10025`LIMIT` lets you limit the number of rows output. 10026 10027The syntax is: 10028 10029### `LIMIT` [integer] 10030 10031For example, this is the same as the previous query, but it limits to only the 2 cats with the shortest names: 10032 10033``` 10034SELECT * FROM cats 10035ORDER BY LENGTH(name) ASC 10036LIMIT 2 10037``` 10038 10039cats: 10040 10041| owner | name | 10042|-------|------------| 10043| 1 | daisy | 10044| 1 | dragonsnap | 10045| 3 | buttercup | 10046| 4 | rose | 10047 10048 10049results of query: 10050 10051| owner | name | 10052|-------|------------| 10053| 4 | rose | 10054| 1 | daisy | 10055</div> 10056 </a> 10057 10058 <a href="/comics/inner-left-join/"> 10059 <span class="title"> 10060 INNER JOIN and LEFT JOIN 10061 </span> 10062 <div class="keywords"></div> 10063 <div class="transcript"></div> 10064 </a> 10065 10066 <a href="/comics/getting-started-select/"> 10067 <span class="title"> 10068 getting started with SELECT 10069 </span> 10070 <div class="keywords"></div> 10071 <div class="transcript"> 10072A SQL database contains a bunch of tables 10073 10074sales: 10075 10076| client | item | 10077|-------|--------| 10078| x | x | 10079| x | x | 10080| x | x | 10081 10082clients: 10083 10084| id | name | 10085|-------|--------| 10086| x | x | 10087| x | x | 10088| x | x | 10089 10090cats: 10091 10092| owner | name | 10093|-------|--------| 10094| x | x | 10095| x | x | 10096| x | x | 10097 10098 10099Every SELECT query takes data from those tables and outputs table of results. 10100 10101### cats: 10102 10103| owner | name | 10104|-------|------------| 10105| 1 | daisy | 10106| 1 | dragonsnap | 10107| 3 | buttercup | 10108| 4 | rose | 10109 10110### query: 10111 10112``` 10113SELECT * 10114FROM cats 10115WHERE owner = 1 10116``` 10117 10118### query output 10119 10120| owner | name | 10121|-------|------------| 10122| 1 | daisy | 10123| 1 | dragonsnap | 10124 10125### A few basic facts to start out: 10126- SELECT queries have to be written in the order: 10127`SELECT ... FROM ... WHERE ... GROUP BY ... HAVING ... ORDER BY ... LIMIT` 10128- SQL isn't case sensitive: `select * from table` is fine too. 10129 10130This zine will use ALL CAPS for SQL keywords like `FROM`. 10131 10132smiling stick figure with curly hair: there are other kinds of queries like `INSERT/ UPDATE / DELETE` but this zine is just about `SELECT`</div> 10133 </a> 10134 10135 <a href="/comics/explain-your-slow-queries/"> 10136 <span class="title"> 10137 EXPLAIN your slow queries 10138 </span> 10139 <div class="keywords"></div> 10140 <div class="transcript">Sometimes queries run slowly, and EXPLAIN can you why! 10141 101422 ways you can use EXPLAIN in PostgreSQL: 10143(other databases have different syntax for this) 10144 101451. Before running the query (`EXPLAIN SELECT ... FROM ...`) 10146 10147This calculates a query plan but doesn't run the query. 10148 10149Smiling stick figure with long straight hair: I _always_ run `EXPLAIN` on a query before running my production database. I won't risk overloading the database with a slow query! 10150 101512. After running the query (`EXPLAIN ANALYZE SELECT ... FROM`) 10152 10153Smiling bald stick figure: why is my query so slow? 10154 10155Smiling stick figure with short curly hair: `EXPLAIN ANALYZE` runs the query and analyzes why it was slow! 10156 10157Here are the `EXPLAIN ANALYZE` results from PostgreSQL for the same query run on 2 tables of rows: one table that has an index and one that doesn't 10158
10159`EXPLAIN ANALYZE SELECT * FROM users WHERE id = 1` 10160 10161unindexed table: 10162``` 10163Seq Scan on users 10164Filter: (id = 1) 10165Rows Removed by Filter: 999999 10166Planning time: 0.185 ms 10167Execution time: 179.412 ms 10168``` 10169 10170(`Seq Scan` means it's looking at each row (slow!)) 10171 10172indexed table: 10173``` 10174Index Only Scan using 10175 users_id_idx on users 10176Index Cond: (id = 1) 10177Heap Fetches: 1 10178Planning time: 3.411 ms 10179Execution time: 0.088 ms 10180``` 10181(the query runs 50 times faster with an index)</div> 10182 </a> 10183 10184 <a href="/comics/case/"> 10185 <span class="title"> 10186 CASE 10187 </span> 10188 <div class="keywords"></div> 10189 <div class="transcript">Often I want to categorize by something that isn't a column: 10190 10191person 1: I want to count children/adults/teenagers but there's no column for that! 10192 10193person 2: no problem! jut categorize people based on age! 10194 10195`CASE` is how to write an `if` statement in SQL. Here's the syntax: 10196``` 10197CASE 10198 WHEN <condition> THEN <result> 10199 WHEN <other-condition> THEN <result> 10200 ... 10201 ELSE <result> 10202END 10203``` 10204 10205## example: 10206 10207Here's how to categorize people into age ranges! 10208``` 10209SELECT first_name, age, CASE 10210 WHEN age < 13 THEN 'child' 10211 WHEN age < 20 THEN 'teenager' 10212 ELSE 'adult' END AS age_range 10213FROM people 10214``` 10215(returns first `THEN` where the condition matches) 10216 10217people: 10218 10219| first_name | age | 10220|---------------|--------| 10221| ahmed | 5 | 10222| marle | 17 | 10223| akira | 60 | 10224| pablo | 15 | 10225 10226 10227result: 10228 10229| first_name | age | age_range | 10230|---------------|--------|--------------| 10231| ahmed | 5 | child | 10232| marle | 17 | teenager | 10233| akira | 60 | adult | 10234| pablo | 15 | teenager |</div> 10235 </a> 10236 10237 <a href="/comics/user-namespaces/"> 10238 <span class="title"> 10239 user namespaces 10240 </span> 10241 <div class="keywords"></div> 10242 <div class="transcript">### user namespaces are a security feature... 10243 10244smiling bald stick figure: I'd like root in the container to be totally unprivileged 10245 10246smiling stick figure with curly hair: you want a user namespace! 10247 10248 10249### but not all container runtimes use them 10250same user! 10251 10252(two arrows point to two smiley faces, one labelled "root in container", the other labelled "root on host") 10253 10254### "root" doesn't always have admin access 10255 10256Container process, represented by a box with a smiley face: I'm root so I can do ANYTHING right? 10257 10258Tux: actually you have limited capabilities so mostly you can just access files owned by root! 10259 10260### in a user namespace, UIDs are mapped to host UIDs 10261process: I'm running as UID O 10262 10263Linux: Oh, that's mapped to 12345 10264 10265The mapping is in `/proc/self/uid_map` 10266 10267### unmapped users show up as "nobody" 10268 10269`$ unshare --user bash` (create user namespace) 10270`$ ls -l /usr/bin` 10271`.. nobody nogroup apropos` 10272` nobody nogroup apt` 10273 10274(these are "actually" owned by root but we didn't map any user) 10275 10276### how to find out if you have a separate user namespace 10277compare the results of 10278`$ ls /proc/PID/ns` 10279 10280between a container process and a host process.</div> 10281 </a> 10282 10283 <a href="/comics/pivot-root/"> 10284 <span class="title"> 10285 pivot-root 10286 </span> 10287 <div class="keywords"></div> 10288 <div class="transcript">### a container image is a tarball of a filesystem 10289(or several tarballs: 1 per layer) 10290 10291pensive stick figure with short curly hair: if someone sends me a tarball of their filesystem, how do I use that though? 10292 10293### chroot: change a process's root directory 10294If you chroot to /fake/root when it opens the file /usr/bin/redis it'll get /fake/root/usr/bin/redis instead. 10295 10296You can "run" a container just by using chroot, like this: 10297``` 10298$ mkdir redis; cd redis 10299$ tar -xzf redis. tar 10300$ chroot $PWD /usr/bin/redis 10301# done ! redis is running! 10302``` 10303 10304### programs can break out of a chroot 10305#### chroot: 10306Illustration of a box labelled "whole filesystem". Inside it is another box labelled "redis container directory". 10307 10308All these files are still there! A root process can access them if it wants. 10309 10310#### pivot_root 10311 10312Illustration of a box labelled "redis container directory". 10313 10314You can unmount the old filesystem so it's impossible to access it. 10315 10316Containers use pivot_root instead of chroot. 10317 10318### to have a "container" you need more than pivot_root 10319pivot_root alone won't let you: 10320- set CPU/memory limits 10321- hide other running processes 10322- use the same port as another process 10323- restrict dangerous system calls</div> 10324 </a> 10325 10326 <a href="/comics/network-namespaces/">
10327 <span class="title"> 10328 network namespaces 10329 </span> 10330 <div class="keywords"></div> 10331 <div class="transcript">### network namespaces are kinda confusing 10332Illustration of an unhappy-looking stick figure with curly hair. 10333 10334person: what does it MEAN for a process to have its own network?? 10335 10336### namespaces usually have 2 interfaces 10337(+ sometimes more) 10338- the loopback interface (127.0.0.1/8, for connections inside the namespace) 10339- another interface (for connections from outside) 10340 10341### every server listens on a port and network interface(s) 10342`0.0.0.0:8080` means "port 8080 on every network interface in my namespace" 10343 10344### 127.0.0.1 stays inside your namespace 10345Illustration of a server, represented by a box with a smiley face, and a smiling stick figure with curly hair. 10346 10347server, thinking: I'm listening on 127.0.0.1 10348person: that's fine but nobody outside your network server namespace will be able 10349to make requests to you! 10350 10351### your physical network card is in the host network namespace 10352 10353Illustration of a rectangular box drawn with a dotted line. Inside it are: 10354- the label "host network namespace" 10355- 192.168.1.149, with an arrow pointing to it reading "requests from other computers" 10356- network card 10357 10358### other namespaces are connected to the host namespace with a bridge 10359 10360Illustration of a rectangular box drawn with a dotted line. Inside it are: 10361- the label "host network namespace" 10362- three boxes, each labelled "container"</div> 10363 </a> 10364 10365 <a href="/comics/layers-containers/"> 10366 <span class="title"> 10367 layers (containers) 10368 </span> 10369 <div class="keywords"></div> 10370 <div class="transcript">### different images have similar files 10371Rails container image and Django container image, each represented by a box with a smiley face: we both use Ubuntu 18.0! 10372 10373### reusing layers saves disk space 10374Rails image 10375- Rails app 10376- ubuntu:18.04 10377 10378Django image 10379- Django app 10380- ubuntu:18.04 10381 10382Both have the exact same files on disk for ubuntu:18.04. 10383 10384### a layer is a directory 10385``` 10386$ ls 8891378eb* 10387bin/ home/ mnt/ run/ tmp/ 10388boot/ lib/ opt sbin/ usr/ 10389dev/ lib64/ proc/ srv/ var/ 10390etc/ media/ root/ sys/ 10391``` 10392`etc` are files in an ubuntu:18.04 layer 10393 10394### every layer has an ID 10395usually the ID is a sha256 hash of the layer's contents 10396 10397example: `8e99fae2..` 10398 10399### if a file is in 2 layers, you'll see the version from the top layer 10400 10401Two rectangular boxes on top of one another, each labelled `/code/file.py`. The one on top is the version you'll see in the merged image. 10402 10403### by default, writes go to a temporary layer 10404 10405Illustration of a rectangle labelled "temp layer", with a bunch of other smaller rectangles stacked underneath it. The temp layer is labelled "thse files might be deleted after the container exits." 10406 10407To keep your changes, write to a directory that's mounted from outside the container</div> 10408 </a> 10409 10410 <a href="/comics/layers/"> 10411 <span class="title"> 10412 layers 10413 </span> 10414 <div class="keywords"></div> 10415 <div class="transcript">### different images have similar files 10416Rails container image and Django container image: we both use Ubuntu 18.04! 10417 10418### reusing layers saves disk space 10419Rails image: 10420Rails app 10421ubuntu:18.04 10422 10423Django image: 10424Django app 10425ubuntu:18.04 10426 10427exact same files on disk! 10428 10429### a layer is a directory 10430``` 10431$ ls 8891378eb* 10432bin/home/mnt/run/tmp/ 10433boot/lib/ opt sbin/ usr/ dev/lib64/proc/srv/var/ 10434etc/media/ root/sys/ 10435``` 10436files in an ubuntu:18.04 layer 10437 10438### every layer has an ID 10439usually the ID is a sha256 hash of the layer's contents 10440example: `8e99fae2..` 10441 10442### if a file is in 2 layers, you'll see the version from the top layer 10443`/code/file.py` (this is the version you'll see in the merged image) 10444`/code/file.py` 10445 10446### by default, writes go to a temporary layer 10447temp layer (these files might be deleted after the container exits) 10448 10449To keep your changes, write to a directory that's mounted from outside the container</div> 10450 </a> 10451 10452 <a href="/comics/how-to-namespace/">
10453 <span class="title"> 10454 how to make a namespace 10455 </span> 10456 <div class="keywords"></div> 10457 <div class="transcript">### processes use their parent's namespaces by default 10458 10459parent, represented by a box with a smiley face: I'm in the host network namespace! 10460 10461child, represented by a smaller box with a smiley face (created with 'clone' syscall): me too! 10462 10463### but you can switch namespaces at any time 10464box with a smiley face: I'm starting a container so it needs its own namespaces 10465 10466### command line tools 10467- `$ unshare --net COMMAND`: run COMMAND in a new network namespace 10468- `$ sudo lsns`: list all namespaces 10469- `$ nsenter -t PID --all COMMAND`: run COMMAND in the same namespaces as PID 10470 10471### namespace system calls 10472- clone: make a new process 10473- unshare: make + use a namespace 10474- setns: use an existing namespace 10475 10476### *clone* lets you create new namespaces for a child process 10477parent: `clone (... CLONE_NEWNET)` 10478 10479child: I have my own network namespace! 10480 10481 10482### each namespace type has a man page 10483``` 10484$ man network_namespaces 10485... 10486A physical network device 10487can live in exactly one 10488network namespace. 10489```</div> 10490 </a> 10491 10492 <a href="/comics/big-idea-every-dependency/"> 10493 <span class="title"> 10494 containers: the big idea: include EVERY dependency 10495 </span> 10496 <div class="keywords"></div> 10497 <div class="transcript">### containers package EVERY dependency together 10498 10499smiling stick figure with short curly hair: to make sure this program will run on 10500your laptop, I'm going to send you every single file you need 10501 10502### a container image iS a tarball of a filesystem 10503Here's what's in a typical Rails app's container: 10504- your app's code 10505- libc + other system libraries 10506- Ubuntu base OS 10507- Ruby interpreter 10508- Ruby gems 10509 10510### how images are built 105110. start with a base OS 105121. install program + dependencies 105132. configure it how you want 105143. make a tarball of the WHOLE FILESYSTEM 10515 10516tiny stick figure with short curly hair: this is what `docker build` does! 10517 10518### running an image 105191. download the tarball 105202. unpack it into a directory 105213. run a program and pretend that directory is its whole filesystem 10522 10523### images let you "install" programs realty easily 10524person, thinking: I can set up a Postgres test database in like 5 seconds! wow!</div> 10525 </a> 10526 10527 <a href="/comics/container-ip-addresses/"> 10528 <span class="title"> 10529 container IP addresses 10530 </span> 10531 <div class="keywords"></div> 10532 <div class="transcript">### containers often get their own IP address 10533wordpress container 1: I'm running WordPress at `172.17.2.3:8080`! 10534 10535wordpress container 2: I'm using 172.17.0.49:8080! 10536 10537 10538### containers use private IP addresses 10539 10540These are reserved for private networks (RFC 1918). 10541 10542This is because they're not directly on the public internet. 10543 10544### for a packet to get to the right place, it needs a route 10545packet `172.16.2.3`: hi! I'm here! 10546 10547router, represented by a box with a nonplussed expression: I don't have any entry 10548matching `172.16.2.3` in my route table, sorry! 10549 10550### inside the same computer, you'll have the right routes 10551 10552same computer: 10553``` 10554$ curl 172.16.2.3:8080 10555<html>.... 10556``` 10557different computer: 10558``` 10559$ curl 172.16.2.3:8080 10560.... no reply .... 10561``` 10562 10563### distributing the right routes is complicated 10564 10565box with a smiley face: a new container started, 10.2.73.4 should go to X computer now 10566 10567route table, also represented by a box with a smiley face, thinking: wow these things change a lot 10568 10569### cloud providers have systems to make container IPs work 10570 10571In AWS, this is called an "elastic network interface" route table</div> 10572 </a> 10573 10574 <a href="/comics/config-options/"> 10575 <span class="title"> 10576 container configuration options 10577 </span> 10578 <div class="keywords"></div> 10579 <div class="transcript">### panel 1 10580Illustration of a smiling stick figure with curly hair. 10581 10582person: here are the 6 most important things you can configure when starting a container! 10583 10584### map a port to the host 10585Illustration of two boxes drawn with dotted lines. One is labelled "host", the other is labelled "container". The "host" box says "port 1234", and the "container box" is labelled "port 8080". There is a double-ended arrow pointing back and forth between the two ports. 10586 10587### mount directories from the host 10588Illustration of two boxes drawn with dotted lines. One is labelled "host", the other is labelled "container". The "host" box says "`~/code/blah`", and the "container box" says "`/src`". There is a double-ended arrow pointing back and forth between the two boxes. 10589 10590### set capabilities 10591 10592### add seccomp-bpf filters 10593 10594### set memory and CPU limits 10595person: only 200 MB RAM for you 10596 10597### use the host network namespace 10598Usually the default is to use a new network namespace!</div> 10599 </a> 10600 10601 <a href="/comics/css-inspector/">
10602 <span class="title"> 10603 the CSS inspector 10604 </span> 10605 <div class="keywords"></div> 10606 <div class="transcript">### all major browsers have a CSS inspector 10607usually you can get to it by right clicking on an element and then "inspect element, but sometimes there are extra step 10608 10609### see overridden properties 10610`button {` 10611 `display: inline-block;` 10612 `color: var(--orange);` (this line in strikethrough) 10613`}` 10614 10615### edit CSS properties 10616``` 10617element { 10618{ 10619``` 10620(lets you change this element's properties) 10621``` 10622button { 10623 display: inline-block; 10624 border: 1px solid black; 10625} 10626``` 10627(this lets you change the border of every `<button>`!) 10628 10629### see computed styles 10630 10631person, represented by a smiling stick figure: here's a website with 12000 lines of CSS, what `font-size` does this link have? 10632browser, represented by a box with a smiley face: 12px, because of `x.css` line 436 10633 10634 10635### look at margin & padding 10636 10637Box Model 10638 10639Illustration of a small box labelled 1261 x 26. On the outside of that box is the word "padding". Surrounding the padding is the border. Surrounding the border is the margin. 10640 10641### and LOTS more 10642different browsers have different tools! 10643 10644For example, Firefox has special tools for debugging grid/flexbox.</div> 10645 </a> 10646 10647 <a href="/comics/media-queries/"> 10648 <span class="title"> 10649 media queries 10650 </span> 10651 <div class="keywords"></div> 10652 <div class="transcript">### media queries let you use different CSS in different situations 10653``` 10654@media print { 10655 #footer { 10656 display: none; 10657} 10658``` 10659(`print` is the media query, and the rest is the CSS to apply) 10660 10661### max-width & min-width 10662``` 10663@media (max-width: 500px) { 10664 // CSS for small screens 10665} 10666@media (min-width: 950px) { 10667 // CSS for large screens 10668} 10669``` 10670 10671### print and screen 10672`screen` is for computer/ mobile screens 10673 10674`print` is used when printing a webpage 10675 10676there are more: `tv`, `tty`, `speech`, `braille`, etc 10677 10678### accessibility queries 10679you can sometimes find out a user's preferences with media queries 10680 10681examples: 10682`prefers-reduced-motion: reduce` 10683`prefers-color-scheme: dark` 10684 10685 10686### you can combine media queries 10687it's very common to write something like this: 10688``` 10689@media screen and 10690 (max-width: 1024px) 10691``` 10692 10693### the viewport meta tag 10694`<meta name="viewport" content="width=device-width, initial-scale=1">` 10695 10696Your site will look bad on mobile if you don't add a tag like this to the `<head>` in your HTML. Look it up to learn more!</div> 10697 </a> 10698 10699 <a href="/comics/css-hiding/"> 10700 <span class="title"> 10701 hiding elements with CSS 10702 </span> 10703 <div class="keywords"></div> 10704 <div class="transcript">### there are many ways to make an element disappear 10705Illustration of a smiling stick figure with curly hair. 10706 10707person: which one to use depends: do you want the empty space it left to be filled? 10708 10709### TRY ME: display: none; 10710other elements will move to fill the empty space 10711 10712Illustration of three boxes side-by-side, with a heart, x, and star, respectively. When the "x" box is set to `display: none;`, the heart and star boxes will now be side-by-side. 10713 10714### visibility: hidden; 10715the empty space will stay empty 10716 10717Illustration of three boxes side-by-side, with a heart, x, and star, respectively. When the "x" box is set to `visibility: hidden;`, the heart and star boxes will have a gap between them the size of the "x" box. 10718 10719### opacity: 0; 10720like `visibility: hidden`, but you can still click on the element & it'll still be visible to screen readers. Usually `visibility: hidden` is better. 10721 10722### how to slowly fade out 10723``` 10724#fade:hover { 10725 transition: all 1s ease; 10726 visibility: hidden; 10727 opacity: 0; 10728} 10729``` 10730set the opacity just so that the transition works 10731 10732### TRY ME: z-index 10733z-index sets the order of overlapping positioned elements 10734 10735Illustration of two boxes, a smaller one with an "x" in it, that is overlapped over a larger empty box. There is an arrow pointing to a second illustration where the boxes are stacked in the opposite order, so that the small box is underneath of the large box.</div> 10736 </a> 10737 10738 <a href="/comics/css-transitions/">
10739 <span class="title"> 10740 CSS transitions 10741 </span> 10742 <div class="keywords"></div> 10743 <div class="transcript">### an element's computed style can change 10744 107452 ways this can happen: 10746 107471. pseudo-classes 10748(like `:hover`) 10749 107502. Javascript code 10751`el.classList.add('x')` 10752 10753### new styles change the element instantly... 10754``` 10755a:hover { 10756 color: red; 10757} 10758``` 10759the element will turn red right away 10760 10761### unless you set the transition property 10762 10763``` 10764a { 10765 color: blue; 10766 transition: all 2s; 10767} 10768a:hover { 10769 color: red; 10770} 10771``` 10772("`all 2s`" = will fade from blue to red over 2s) 10773 10774 10775### transition has 3 parts 10776`transition: color 1s ease;` 10777 10778`color`: which CSS properties to animate 10779`1s`: duration 10780`ease`: timing function 10781 10782### not all property changes can be animated.... 10783`list-style-type: square;` 10784 10785CSS renderer, represented by a box with a smiley face: I don't know how to animate that, sorry! 10786 10787### ...but there are dozens of properties that can 10788if it's a number or color, it can probably be animated! 10789``` 10790font-size: 14px; 10791rotate: 90deg; 10792width: 20em; 10793``` 10794</div> 10795 </a> 10796 10797 <a href="/comics/css-specifications/"> 10798 <span class="title"> 10799 css specifications 10800 </span> 10801 <div class="keywords"></div> 10802 <div class="transcript">### CSS has specifications 10803 10804CSS 2.1, represented by an image of a document with many lines of text: hello, this is how max-width works in excruciating detail 10805 10806### there used to be just one specification 10807 10808Illustration of a smiling stick figure with curly hair. 10809 10810person: it's called "CSS 2" and I still like to reference it to learn the basics 10811 10812### today, every CSS feature has its own specification 10813you can find them all at https://www.w3.org/TR/CSS/ 10814 10815there are dozens of specs, for example: colors, flexbox, and transforms 10816 10817### major browsers usually obey the spec 10818 10819but sometimes they have bugs 10820 10821Illustration of a happy little caterpillar-type bug. 10822 10823browser, represented by a box with a smiley face: oops, I didn't quite implement that right... 10824 10825### levels 10826CSS versions are called "levels". 10827 10828new levels only add new features. They don't change the behaviour of existing CSS code 10829 10830### new features take time to implement 10831https://caniuse.com 10832(The URL is surrounded by little hearts and stars) 10833 10834can tell you which browser versions support a CSS feature</div> 10835 </a> 10836 10837 <a href="/comics/css-isnt-arbitrary/"> 10838 <span class="title"> 10839 CSS isn't arbitrary 10840 </span> 10841 <div class="keywords"></div> 10842 <div class="transcript"></div> 10843 </a> 10844 10845 <a href="/comics/css-borders/"> 10846 <span class="title"> 10847 CSS borders 10848 </span> 10849 <div class="keywords"></div> 10850 <div class="transcript">### `border` has 3 components 10851`border: 2px solid black;` 10852 10853is the same as 10854``` 10855border-width: 2px; 10856border-style: solid; 10857border-color: black; 10858``` 10859 10860### `border-style` options 10861- `solid` 10862- `dotted` 10863- `dashed` 10864- `double` 10865 10866(each word is surrounded by the border it describes) 10867 10868+ lots more (`inset`, `groove`, etc) 10869 10870 10871### `border-{side}` 10872you can set each side's border separately: 10873``` 10874aborder-bottom: 10875 2px solid black; 10876``` 10877 10878### `border-radius` 10879border-radius lets you have rounded corners 10880 10881`border-radius: 10px;` 10882 10883`border-radius: 50%;` will make a square into a circle! 10884 10885 10886### box-shadow 10887lets you add a shadow to any element 10888 10889`box-shadow: 5px 5px 8px black;` 10890the first "5px" is the x offset, the second "5px" is the x offset, "8px" is the blur radius, and "black" is the color. 10891 10892### outline 10893`outline` is like `border`, but it doesn't change an element's size when you add it 10894 10895outlines on `:hover/: active` help with accessibility: with keyboard navigation, you need an outline to see what's focused</div> 10896 </a> 10897 10898 <a href="/comics/backwards-compatibility/"> 10899 <span class="title"> 10900 CSS backwards compatibility 10901 </span> 10902 <div class="keywords"></div> 10903 <div class="transcript">### browsers support old HTML + CSS forever 10904 10905Illustration of a smiling stick figure with long hair, talking to a browser from 2020, represented by a box with a smiley face. 10906 10907person: I wrote this CSS in 1998 109082020 browser: still works great! 10909 10910### this makes CSS hard to write... 10911 10912Illustration of two stick figures talking 10913
10914person 1: why are CSS units so weird? 10915person 2, with grey hair: let me tell you a story from 20 years ago... 10916 10917### but it means it's worth the investment 10918 10919Illustration of a smiling stick figure with long hair, talking to a browser, represented by a box with a smiley face. 10920 10921person: I spent DAYS getting this CSS to work 10922browser: I'll make sure it keeps working forever! 10923 10924### if you don't follow the 10925standards, you're not guaranteed backwards compatibility my site broke! 10926(oh yeah, Firefox dropped support for that experiment 10927 10928### your CSS doesn't have to support browsers from 1998 10929 10930Illustration of a smiling stick figure with short curly hair. 10931 10932person: just test that your CSS works on the browsers that your users are using! 10933 10934### newer features are often easier to use 10935what people expect from a website has changed a LOT since 1998. Newer CSS features make responsive design easy 10936 10937 10938 10939 10940</div> 10941 </a> 10942 10943 <a href="/comics/default-stylesheets/"> 10944 <span class="title"> 10945 browser default stylesheets 10946 </span> 10947 <div class="keywords"></div> 10948 <div class="transcript">### every browser has a default stylesheet (aka "user agent stylesheet") 10949 10950a small sample from the Firefox default stylesheet: 10951``` 10952h1 { 10953 font-size: 2em; 10954 font-weight: bold; 10955} 10956``` 10957 10958### different browsers have different defaults 10959 10960Illustration of a smiling stick figure with curly hair. 10961 10962person: buttons & forms have some of the biggest differences 10963 10964### you can read the default stylesheet 10965Firefox's default stylesheets are at: 10966`resource://gre-resources/` 10967 10968### every property also has a default "initial value" 10969the initial value (defined in the spec) is what's used if no stylesheet has set anything. For example, `background-color`'s initial value is `transparent` 10970 10971 10972### a CSS property can be set in 5 ways 10973(listed from lowest priority to highest priority) 10974 109751. the initial value 109762. the browser's default stylesheet 109773. the website's stylesheets and user stylesheets 109784. inline styles set with HTML/JS</div> 10979 </a> 10980 10981 <a href="/comics/why-i-love-bash/"> 10982 <span class="title"> 10983 why I love bash 10984 </span> 10985 <div class="keywords"></div> 10986 <div class="transcript">### it's SO easy to get started 10987 10988Here's how: 10989 109901. Make a file called `hello.sh` and put some commands in it, like 10991 `ls /tmp` 109922. Run it with `bash hello.sh` 10993 10994### pipes & redirects are super easy 10995managing pipes in other languages is annoying. in bash, it's just: 10996`cmd1 | cmd2` 10997 10998### batch file operations are easy 10999smiling stick figure with curly hair: let's convert every .png to a .jpg 11000bash, with hearts in its eyes: I was born for this 11001 11002### it's surprisingly good at concurrency 11003smiling stick figure with curly hair: let's start 12 programs in parallel & wait for them all to finish 11004bash: yep no problem! 11005 11006### it doesn't change 11007bash is weird and old, but the basics of how it works haven't changed in 30 years. If you learn it now, it'll be the same in 10 years. 11008 11009### bash is GREAT for some tasks 11010But it's also EXTREMELY BAD at a lot of things. 11011I don't use bash if I need: 11012* unit tests 11013* math (bash barely has numbers!) 11014* easy-to-read code âº</div> 11015 </a> 11016 11017 <a href="/comics/subshells/"> 11018 <span class="title"> 11019 subshells 11020 </span> 11021 <div class="keywords"></div> 11022 <div class="transcript">### a subshell is a child shell process 11023bash, represented by a box with a smiley face: hey, can you run this bash code for me? 11024other bash process: sure thing! 11025 11026### some ways to create a subshell 110271. put code in parentheses `(...)` 11028`(cd $DIR; ls)` 11029(runs in subshell) 11030 110312. put code in `$(...)` 11032`var=$(cat file.txt)` 11033(runs in subshell) 11034 110353. pipe/redirect to a code block 11036`cat x.txt while read line...` 11037(piping to a loop makes the loop run in a subshell) 11038 110394. and lots more 11040for example, process substitution `<()` creates a subshell 11041 11042### `cd` in a subshell doesn't `cd` in the parent shell 11043``` 11044( 11045cd subdir/ 11046mv x.txt y.txt 11047) 11048``` 11049I like to do this so I don't have to remember to `cd` back at the end! 11050 11051### setting a variable in a subshell doesn't update it in the main shell 11052``` 11053var=3 11054(var=2) 11055echo $var 11056``` 11057(this prints 3, not 2) 11058 11059### it's easy to create a subshell and not notice 11060`x=$(some_function)` 11061 11062sad stick person: I changed directories in some_function, why I didn't it work? 11063happy stick person: it's running in a subshell!</div> 11064 </a> 11065 11066 <a href="/comics/compatibility/">
11067 <span class="title"> 11068 POSIX compatibility 11069 </span> 11070 <div class="keywords"></div> 11071 <div class="transcript">### there are lots of Unix shells 11072- dash 11073- bash 11074- sh 11075- zsh 11076- fish 11077- csh 11078- tcsh 11079- ksh 11080 11081you can find out your user's default shell by running: 11082`$ echo $SHELL` 11083 11084### POSIX is a standard that defines how Unix shells should work 11085 11086sh, dash, bash, zsh, and ksh, all represented by little boxes with smiley faces: if your script sticks to POSIX, we'll all run it the same way! (mostly [smiley face]) 11087 11088fish: I don't care about POSIX 11089 11090### some shells have extra features 11091bash, zsh, and ksh: we have extra features that aren't in POSIX 11092sh and dash: we keep it simple & just do what POSIX says 11093 11094### on most systems, /bin/sh only supports POSIX features 11095smiling stick figure with short curly hair: if your script has `#!/bin/sh` at the top, don't use bash-only features in it! 11096 11097### some people write all their scripts to follow POSIX 11098smiling stick figure with straight chin length hair: I only use POSIX features 11099smiling stick figure with short curly hair, labelled "me": I use lots of bash-only features! 11100 11101### this zine is about bash scripting 11102smiling stick figure with short curly hair: most things in this zine will work in any shell, but some won't! page 15 lists some non-POSIX features</div> 11103 </a> 11104 11105 <a href="/comics/bashisms/"> 11106 <span class="title"> 11107 non-POSIX features 11108 </span> 11109 <div class="keywords"></div> 11110 <div class="transcript">### some bash features aren't in the POSIX spec 11111 11112 11113Illustration of a smiling stick figure with curly hair. 11114 11115Person: here are some examples! These won't work in POSIX shells like `dash` and `sh`. 11116 11117### arrays 11118POSIX shells only have one array: `$@` for arguments 11119 11120### [[ $DIR=/home/*]] 11121POSIX alternative: match strings with `grep` 11122 11123### [[ ... ]] 11124POSIX alternative: 11125`[ ... ]` 11126 11127### diff <(./cmd1) <./cmd2) 11128this is called "process substitution", you can use named pipes instead 11129 11130### the local Keyword 11131in POSIX shells, all variables are global 11132 11133### for ((i=0; i <3; i++)) 11134`sh` only has for `x` in ... loops, not C-style loops 11135 11136### a. {png, svg} 11137you'll have to type 11138`a.png a.svg` 11139 11140### {1..5} 11141POSIX alternative: 11142`$(seq 1 5)` 11143 11144### $'\n' 11145POSIX alternative: 11146`$(printf "\n")` 11147 11148### ${var//search/replace} 11149POSIX alternative: 11150pipe to `sed`</div> 11151 </a> 11152 11153 <a href="/comics/environment-variables/"> 11154 <span class="title"> 11155 environment variables 11156 </span> 11157 <div class="keywords">[bash, shell]</div> 11158 <div class="transcript">### panel 1: every process has environment variables 11159 11160how to see any process's environment variables on Linux: 11161 11162``` 11163cat /proc/$PID/environ | tr '\0' '\n' 11164``` 11165 11166### panel 2: shell scripts have 2 kinds of variables 11167 111681. environment variables 111692. shell variables 11170 11171unlike in most languages, in shell you access both of these in the exact same way: `$VARIABLE` 11172 11173 11174### panel 3: export sets environment variables 11175 11176``` 11177export ANIMAL=panda 11178``` 11179 11180`export ANIMAL=panda` means that every child process will have `ANIMAL` set to `panda` 11181 11182### panel 4: child processes inherit environment variables 11183 11184this is wy the variables set in your `.bash_profile` work in all programs you start from the terminal. They're all child processes of your bash shell! 11185 11186
11187### panel 5: shell variables aren't inherited 11188 11189``` 11190var=panda 11191``` 11192 11193in this example, `$var` only gets set in this process, not in child processes 11194 11195### panel 6: you can set environment variables when starting a program 11196 11197Illustration of a smiling stick figure with curly hair, talking to env, represented by a box with a smiley face. 11198 11199Person: `env VAR=panda ./myprogram` 11200env: OK! I'll set `VAR` to `panda` and then start `./myprogram`</div> 11201 </a> 11202 11203 <a href="/comics/variables/"> 11204 <span class="title"> 11205 bash variables 11206 </span> 11207 <div class="keywords"></div> 11208 <div class="transcript">### how to set a variable 11209- `var=value` right (no spaces!) 11210- `var = value` wrong 11211 11212`var = value` will try to run the program var with the arguments "`=`" and "`value`" 11213 11214### how to use a variable: "$var" 11215``` 11216filename=blah.txt 11217echo "$filename" 11218``` 11219they're case sensitive. environment variables are traditionally all-caps, like `$HOME` 11220 11221### there are no numbers, only strings 11222``` 11223a=2 11224a="2" 11225``` 11226both of these are the string "2" 11227 11228technically bash can do arithmetic, but I avoid it 11229 11230### always use quotes around variables 11231`$filename="swan 1.txt"` 11232 11233 11234`$ cat $filename` (wrong) 11235 11236bash: ok, I'll run `cat swan 1.txt` 112372 files! oh no! we didn't mean that! 11238cat: Um `swan` and `1.txt` don't exist... 11239 11240$ cat "$filename" (right!) 11241 11242bash: ok, I'll run `cat "swan 1.txt"` 11243cat '"swan 1.txt"`! that's a file! yay! 11244 11245### ${varname} 11246To add a suffix to a variable like "2", you have to use `${varname}`. Here's why: 11247 11248`$ zoo=panda` 11249`$ echo "$zoo2"` prints `""`, `zoo2` isn't a variable 11250`$ echo "${zoo}2"` this prints "`panda2`" like we wanted</div> 11251 </a> 11252 11253 <a href="/comics/bash-quotes/"> 11254 <span class="title"> 11255 bash quotes 11256 </span> 11257 <div class="keywords"></div> 11258 <div class="transcript">### bash has 3 kinds of quotes 11259 11260#### 'single quotes' 11261``` 11262$ echo '$HOME\n' 11263$HOME \n 11264``` 11265#### "double quotes" 11266``` 11267$ echo "$HOME \n" 11268/home/bork\n 11269``` 11270(only double quotes expand variables) 11271#### $' 11272``` 11273$ echo $'$HOME\n' 11274$HOME 11275``` 11276(invisible newline) 11277(only `$'` expands escape sequences like `\n` or `\'`) 11278 11279### you can quote multiline strings 11280``` 11281$ MESSAGE="Usage: 11282 11283here's an explanation of how to use this script!" 11284``` 11285 11286### here documents 11287 11288heredocs are a way to write string containing quotes: 11289 11290expands variables: 11291``` 11292$ cat <<PANDA 11293he said: 11294"that's $5" 11295PANDA 11296``` 11297 11298doesn't expand: 11299``` 11300$ cat <<'PANDA' 11301he said: 11302"that's $5" 11303PANDA 11304``` 11305 11306### a trick to escape any string: `!:q` 11307make bash do it for you! 11308 11309``` 11310$ # He said "that's $5" 11311$ !:q 11312'/# He said "that'\'s $5"' 11313``` 11314(3 strings squished together) 11315 11316 11317### escaping and 11318here are a few ways to get a ' or ": 11319``` 11320\' and \" 11321" ' " and ' " ' 11322$'\'' 11323"\"" 11324``` 11325person: `'\''` doesn't work!</div> 11326 </a> 11327 11328 <a href="/comics/bash-pipes/"> 11329 <span class="title"> 11330 bash pipes 11331 </span> 11332 <div class="keywords"></div> 11333 <div class="transcript">### sometimes you want to send the output of one process to the input of another 11334``` 11335$ ls | wc -1 1133653 11337``` 11338(53 files!) 11339 11340### a pipe is a pair of 2 magical file descriptors 11341ls -> stdout -> IN -> pipe -> OUT -> stdin -> wx 11342 11343(IN and OUT are file descriptors) 11344 11345### panel 3 11346when ls does 11347`write(IN, "hi")` 11348 11349wc can read it! 11350`read(OUT)-> "hi"` 11351 11352Pipes are one way. You can't write to OUT 11353 11354### the OS creates a buffer for each pipe 11355IN -> data waiting to be read -> OUT 11356 11357when the buffer gets full: 11358 11359process, represented with by a box with a smiley face: `write(IN, "..."` 11360OS, represented by a box with a nonplussed face: it's full! I'm going to pause you until there's room again, 11361 11362### named pipes 11363you can create a file that acts like a pipe with `mkfifo` 11364``` 11365$ mkfifo mypipe 11366$ ls > mypipe & 11367$ wc < mypipe 11368``` 11369(this does the same thing as `ls | wc`) 11370 11371### you can use pipes in other languages! 11372only shell has the syntax `process1 | process2` but you can create pipes in basically any language!</div> 11373 </a> 11374 11375 <a href="/comics/bash-input/">
11376 <span class="title"> 11377 bash input 11378 </span> 11379 <div class="keywords"></div> 11380 <div class="transcript">### read -r var reads stdin into a variable 11381``` 11382$ read -r greeting 11383 hello there! 11384``` 11385(type here and press enter) 11386``` 11387+ $ echo "$greeting" 11388hello there! 11389``` 11390### you can also read into multiple variables 11391``` 11392$ read -r name1 name2 11393ahmed fatima 11394$ echo "$name2" 11395fatima 11396``` 11397 11398### by default, read strips whitespace 11399`" a b c " -> "a b c"` 11400 11401it uses the `IFS` ("Input Field Separator") variable to decide what to strip 11402 11403### set `IFS=''` to avoid stripping whitespace empty string 11404 11405`$ IFS=''`(empty string) `read -r greeting` 11406``` 11407 hi there! 11408$ echo "$greeting" 11409 hi there! 11410``` 11411 11412the spaces are still there! 11413 11414### more `IFS` uses: loop over every line of a file 11415by default, for loops will loop over every word of a file (not every line). Set `IFS=''` to loop over every line instead! 11416 11417(don't forget to unset IFS when you're done!) 11418``` 11419IFS='' 11420for line in $(cat file.txt) 11421do 11422 echo $line 11423done 11424```</div> 11425 </a> 11426 11427 <a href="/comics/bash-globs/"> 11428 <span class="title"> 11429 bash globs 11430 </span> 11431 <div class="keywords"></div> 11432 <div class="transcript">### globs are a way to match strings 11433beware: the `*` and the `?` in a glob are different than `*` and `?` in a regular expression!!! 11434 11435bear* 11436 11437matches -> bear â 11438matches -> bearable â 11439doesn't match -> bugbear x 11440 11441### bash expands globs to match filenames 11442smiling stick figure with short curly hair: `cat *.txt` 11443 11444bash, represented by a box with a smiley face, thinking: let's find all the `.txt` files in this directory... 11445 11446bash: `exec(["cat", "sun.txt" "planet.txt"])` 11447 11448cat, also represented by a box with a smiley face, thinking: `sun.txt` and `planet.txt`, got it 11449 11450(cat doesn't know that you wrote `cat *.txt`) 11451 11452 11453### there are just 3 special characters 11454`*` matches 0+ characters 11455`?` matches 1 character 11456`[abc]` matches `a` or `b` or `c` 11457 11458person: I usually just use * in my globs 11459 11460### use quotes to pass a literal '*' to a command 11461`$ egrep 'b.*' file.txt` 11462 11463the regexp 'b.*' needs to be quoted so that bash won't translate it into a list of files with b. at the start 11464 11465### filenames starting with a dot don't match 11466unless the glob starts with a dot, like `.bash*` 11467person: `ls *.txt` 11468bash: there's `.bees.txt`, but I'm not going to include that</div> 11469 </a> 11470 11471 <a href="/comics/bash-for-loops/"> 11472 <span class="title"> 11473 bash for loops 11474 </span> 11475 <div class="keywords"></div> 11476 <div class="transcript">### for loop syntax 11477``` 11478for i in panda swan 11479do 11480 echo "$i" 11481done 11482``` 11483 11484### the semicolons are weird 11485usually in bash you can always replace a newline with a semicolon. But not with for loops! 11486 11487`for i in a b; do ...; done` 11488 11489you need semicolons before do and done but it's a syntax error to put one after do 11490 11491### looping over files is easy 11492``` 11493for i in *.png 11494do 11495 convert "$i" "${i/png/jpg}" 11496done 11497``` 11498this converts all png files to jpgs! 11499 11500### for loops loop over words, not lines 11501`for word in $(cat file.txt)` 11502 11503loops over every word in the file, NOT every line (see page 18 for how to change this!) 11504 11505### while loop syntax 11506``` 11507while COMMAND 11508do 11509 ... 11510done 11511``` 11512like an if statement, runs COMMAND and checks if it returns 0 (success) 11513 11514### how to loop over a range of numbers 115153 ways: 11516``` 11517for i in $(seq 1 5) 11518for i in {1..5} 11519for ((i=1; i<6; i++) 11520``` 11521the second two only work in bash, not sh</div> 11522 </a> 11523 11524 <a href="/comics/bash-debugging/"> 11525 <span class="title"> 11526 bash debugging 11527 </span> 11528 <div class="keywords"></div> 11529 <div class="transcript">### our hero: `set -x` 11530 11531`set -x` prints out every line of a script as it executes, with all the variables 11532expanded! 11533 11534`#!/bin/bash 11535set -x` 11536(I usually put `set -x` at the top) 11537 11538### or `bash -X` 11539`$ bash -x script.sh` does the same thing as putting `set -x` at the top of `script.sh` 11540 11541### you can stop before every line 11542`trap read DEBUG`\ 11543the `DEBUG` "signal" is triggered before every line of code 11544 11545### a fancy step debugger trick 11546put this at the start of your script to confirm every line before it runs: 11547 11548`trap '(read -p "\[$BASH_SOURCE: $LINENO] $BASH_COMMAND")' DEBUG` 11549 11550- `read -p` prints a message, press enter to continue 11551- `$BASH_SOURCE` is the script filename 11552- `$LINENO` is the line number 11553- `$BASH_COMMAND` is the next command that will run 11554 11555### how to print better error messages 11556this die function: 11557`die() { echo $1 >&2; exit 1; }` lets you exit the program and print a message if a command fails, like this: 11558`some_command || die "oh no!"`</div> 11559 </a> 11560 11561 <a href="/comics/builtins/">
11562 <span class="title"> 11563 bash builtins 11564 </span> 11565 <div class="keywords"></div> 11566 <div class="transcript">### most bash commands are programs 11567 11568You can run `which` to find out which binary is being used for a program: 11569``` 11570$ which ls 11571/bin/ls 11572```` 11573 11574### but some commands are functions inside the bash program 11575smiling stick figure with short curly hair: `$ echo hi` 11576bash, represented by a box with a smiley face: ooh, echo? I'll call my builtin function that does that! 11577 11578### type tells you if a command is a builtin 11579``` 11580$ type grep 11581grep is /bin/grep 11582$ type echo 11583echo is a builtin 11584$ type cd 11585cd is a builtin 11586``` 11587 11588### examples of builtins 11589- `declare` 11590- `type` 11591- `source` 11592- `alias` 11593- `read` 11594- `cd` 11595- `printf` 11596- `echo` 11597 11598 11599### a useful builtin: `alias` 11600`alias` lets you set up shorthand commands, like: 11601`alias gc="git commit"` 11602 11603`~/.bashrc` runs when bash starts, put aliases there! 11604 11605### a useful builtin: `source` 11606`bash script.sh` runs `script.sh` in a subprocess, so you can't use its variables / functions. 11607 11608`source script.sh` is like pasting the contents of `script.sh`</div> 11609 </a> 11610 11611 <a href="/comics/background-processes/"> 11612 <span class="title"> 11613 background processes 11614 </span> 11615 <div class="keywords"></div> 11616 <div class="transcript">### scripts can run many processes in parallel 11617``` 11618python -m http.server & 11619curl localhost:8080 11620``` 11621& starts python in the "background", so it keeps running while `curl` runs 11622 11623### wait waits for all background processes to finish 11624``` 11625command1 & 11626command2 & 11627wait 11628``` 11629this waits for both `command1` and `command2` to finish 11630 11631### concurrency is easy* in bash 11632in other languages: 11633smiling stick figure with short curly hair, thinking: threads? how do I do that again? 11634 11635in bash: 11636``` 11637thing1 & 11638thing2 & 11639wait 11640``` 11641`*` (if you keep it very simple) 11642 11643### background processes sometimes exit when you close your terminal 11644you can keep them running with `nohup` or by using `tmux/screen`. 11645 11646`$ nohup ./command &` 11647 11648### panel 5: 11649person: `jobs`, `fg`, `bg`, and `disown` let you juggle many processes in the same terminal, but I almost always just use multiple terminals instead 11650 11651### panel 6: 11652- `jobs`: list shell's background processes 11653- `disown`: like nohup, but after process has started 11654- `fg and bg`: move process to foreground/background</div> 11655 </a> 11656 11657 <a href="/comics/dns-queries/"> 11658 <span class="title"> 11659 DNS queries 11660 </span> 11661 <div class="keywords"></div> 11662 <div class="transcript"></div> 11663 </a> 11664 11665 <a href="/comics/dns-queries-arent-harmless/"> 11666 <span class="title"> 11667 DNS queries aren't harmless 11668 </span> 11669 <div class="keywords"></div> 11670 <div class="transcript"></div> 11671 </a> 11672 11673 <a href="/comics/dig-output/"> 11674 <span class="title"> 11675 how to read dig output 11676 </span> 11677 <div class="keywords"></div> 11678 <div class="transcript"></div> 11679 </a> 11680 11681 <a href="/comics/glue-records/"> 11682 <span class="title"> 11683 glue records 11684 </span> 11685 <div class="keywords"></div> 11686 <div class="transcript"></div> 11687 </a> 11688 11689 <a href="/comics/airports-lie-dns/"> 11690 <span class="title"> 11691 how airports lie to you with DNS 11692 </span> 11693 <div class="keywords"></div> 11694 <div class="transcript"></div> 11695 </a> 11696 11697 <a href="/comics/dns-distributed/"> 11698 <span class="title"> 11699 DNS is distributed 11700 </span> 11701 <div class="keywords"></div> 11702 <div class="transcript"></div> 11703 </a> 11704 11705 <a href="/comics/spf-dkim/"> 11706 <span class="title"> 11707 SPF & DKIM records 11708 </span> 11709 <div class="keywords"></div> 11710 <div class="transcript"></div> 11711 </a> 11712 11713 <a href="/comics/how-dns-breaks/"> 11714 <span class="title"> 11715 things that can break your DNS 11716 </span> 11717 <div class="keywords"></div> 11718 <div class="transcript"></div> 11719 </a> 11720 11721 <a href="/comics/caching-levels/">
11722 <span class="title"> 11723 DNS cache levels 11724 </span> 11725 <div class="keywords"></div> 11726 <div class="transcript"></div> 11727 </a> 11728 11729 <a href="/comics/why-dns-updates-are-slow/"> 11730 <span class="title"> 11731 why DNS updates are slow: caching 11732 </span> 11733 <div class="keywords"></div> 11734 <div class="transcript">### You might have heard that DNS updates need time to "propagate". 11735What's actually happening is that there are old cached records which need to expire. 11736 11737### DNS records are cached in many places 11738- browser caches 11739- DNS resolver caches 11740- operating system caches 11741 11742google.com, represented by a box with a smiley face: my DNS records are cached on billions of devices! 11743 11744### let's see what happens when you update an IP 11745bananas.com Aâ¾ 11746300 [changed to] 60 117471.2.3.4 [changed to] 5.6.7.8 11748 11749beware: even if you change the TTL to 60s, you still have to wait 300 seconds for the old record to expire 11750 11751### 30 seconds later... 11752(you go to bananas.com in your browser) 11753 11754Illustration of a resolver, represented by a box with a smiley face holding a magnifying glass, and a browser, represented by the Firefox logo of a fox wrapped around a globe 11755 11756browser: hey what's the IP for bananas.com? 11757resolver, thinking: let's check my cache for bananas.com... found it!! 11758resolver: it's 1.2.3.4! 11759 11760### 400 seconds later... 11761(you refresh the page again) 11762browser: hey what's the IP for bananas.com? 11763resolver, thinking: The TTL (300s) is up, better ask for a new IP... 11764resolver: it's 5.6.7.8! 11765 11766### 12 hours later... 11767(you check 1.2.3.4's logs to make sure all the traffic has moved over) 11768 11769Illustration of a stick figure with curly hair looking confused, and a rogue DNS resolver, which looks like the other resolvers except that it is wearing a burglar mask. 11770 11771person: that's weird, the old server is still getting a few requests... 11772rogue DNS resolver: I don't care about your TTL! I just cache everything for 24 hours! 11773 11774the culprit: a rogue DNS resolver</div> 11775 </a> 11776 11777 <a href="/comics/txt-records/"> 11778 <span class="title"> 11779 TXT records & more 11780 </span> 11781 <div class="keywords"></div> 11782 <div class="transcript">### TXT records can contain literally anything 11783 11784``` 11785examplecat.com TXT 11786"hello! I'm an example cat!" 11787``` 11788(though they're usually ASCII) 11789 11790### they're often used to verify that you own your domain 11791google, represented by a box with a smiley face: put "banana stand panda" in a TXT record to prove you) own this domain! 11792 11793### reasons to verify your domain 11794- to issue SSL certificates with Let's Encrypt 11795- to use Single Sign On (SSO) for a service 11796- to get access to Google/ Facebook's data about your domain (eg search data) 11797 11798### they're also used for email security (SPF/DKIM/DMARC) 11799Illustration of two smiling stick figures talking. 11800 11801person 1: should we create a DNS record type for SPF? 11802person 2: nah let's just put it all in TXT records! 11803 11804(not a historically accurate summary of the design process for SPF records) 11805 11806### TXT records can contain many strings 11807Each string is at most 256 characters, and clients will concatenate them together. 11808 11809You'll see this in DKIM records, because they're usually more than 256 characters. 11810 11811### some other record types 11812CAA: restrict who can issue certificates for your domain 11813PTR: reverse DNS map IP addresses to domain names (look these up with `dig -x`) 11814SRV: holds both an IP address and a port number</div> 11815 </a> 11816 11817 <a href="/comics/root-nameservers/"> 11818 <span class="title"> 11819 the root nameservers 11820 </span> 11821 <div class="keywords"></div> 11822 <div class="transcript">### every DNS resolver starts with a root nameserver 11823Illustration of a conversation between a resolver, represented by a box with a smiley face holding a magnifying glass, and a root nameserver, represented by a box with a smiley face wearing a stack of crowns. 11824 11825resolver: what's the IP for example.com? 11826root nameserver: You should ask a `.com` nameserver! They're at `a.gtld-servers.net, b....` 11827 11828### root nameserver IP addresses almost never change 11829`a.root-servers.net`'s IP (`198.41.0.4`) hasn't changed since 1993. DECADES ago! 11830 11831### there are thousands of physical root nameservers, but only 13 IP addresses 11832Each IP refers to multiple physical servers, you'll get the one closest to you. (this is called "anycast") 11833 11834There's a map at https://root-servers.org 11835 11836### if they didn't exist, resolvers wouldn'
11836t know where to start 11837resolver, distressed: I need an IP address of an initial server to query, and I can't use DNS to get that IP! 11838 11839### every resolver has the root IPs hardcoded in its source code 11840example: https://wzrd.page/bind 11841 11842You can query one like this: 11843`dig @198.41.0.4 example.com` 11844 11845All the IPs will give you the exact same results, there are just lots of them for redundancy. 11846 11847Here they are! 11848``` 11849a.root-servers.net 198.41.0.4 11850b.root-servers.net 199.9.14.201 11851c.root-servers.net 192.33.4.12 11852d.root-servers.net 199.7.91.13 11853e.root-servers.net 192.203.230.10 11854f.root-servers.net 192.5.5.241 11855g.root-servers.net 192.112.36.4 11856h.root-servers.net 198.97.190.53 11857i.root-servers.net 192.36.148.17 11858j.root-servers.net 192.58.128.30 11859k.root-servers.net 193.0.14.129 118601.root-servers.net 199.7.83.42 11861m.root-servers.net 202.12.27.33 11862```</div> 11863 </a> 11864 11865 <a href="/comics/dns-hierarchy/"> 11866 <span class="title"> 11867 the DNS hierarchy 11868 </span> 11869 <div class="keywords"></div> 11870 <div class="transcript">### there are 3 main levels of authoritative DNS servers 11871root (wearing 3 crowns): I'm in charge of EVERYTHING 11872.com nameserver (wearing 2 crowns): I'm in charge of all domains ending in `.com` 11873example.com nameserver (wearing 1 crown): I'm in charge of all domains ending in `example.com` 11874 11875### the root nameserver delegates 11876what's the IP for example.com? 11877root: I am not concerned with petty details like that. Here's the address of the .com nameserver. 11878 11879### the .com nameserver also delegates 11880what's the IP for example.com? 11881.com nameserver: I am not concerned with petty details like that either. Here's the address of the example.com nameserver 11882 11883### the example.com nameserver actually answers your questions 11884what's the IP for example.com? 11885example.com nameserver: 93.184.216.34! 11886 11887### this design lets DNS be decentralized 11888example: for my domain `jvns.ca` 11889 11890root (ICANN controls this!) 11891delegates to 11892.ca nameserver (Canada controls this!) 11893delegates to 11894jvns.ca nameserver (I control this!)</div> 11895 </a> 11896 11897 <a href="/comics/tcp-dns/"> 11898 <span class="title"> 11899 TCP DNS 11900 </span> 11901 <div class="keywords"></div> 11902 <div class="transcript">### If you manage servers, sometimes DNS just breaks for no obvious reason 11903Illustration of a smiling stick figure with curly hair. 11904 11905person: TCP DNS is an uncommon but VERY annoying cause of DNS problems! Let's learn about it! 11906 11907### DNS queries can use either UDP or TCP 11908A UDP DNS response has to be less than 4096 bytes. UDP is the default. 11909 11910TCP can send an unlimited amount of data. It's only used when UDP wouldn't work. 11911 11912### large DNS responses automatically use TCP 11913speech bubble 1: here's a UDP DNS query! 11914speech bubble 2: sorry, my response is too big to fit in a UDP packet! get the rest with TCP! 11915 11916### what's in a giant DNS response? 11917person: I've seen responses with hundreds of internal server IP addresses (for example when using Consul) 11918 11919### how not supporting TCP DNS can ruin your day 119201. your server is happily making UDP DNS queries 119212. one day, the responses get bigger and switch to TCP 119223. oh no! the queries fail! 11923 11924### 2 reasons TCP DNS might not work 119251. some DNS libraries (like musl's getaddrinfo) don't support TCP. This is why DNS sometimes breaks in Alpine Linux. 119262. it could be blocked by your firewall. You should open both UDP port 53 and TCP port 53.</div> 11927 </a> 11928 11929 <a href="/comics/search-domains/"> 11930 <span class="title"> 11931 search domains 11932 </span> 11933 <div class="keywords"></div> 11934 <div class="transcript">### panel 1: 11935In an internal network (like in a company or school), sometimes you can connect to a machine by just typing its name, like this: 11936 11937`$ ping labcomputer-23` 11938 11939Let's talk about how that works! 11940 11941### many DNS lookup functions support "local" domain names 11942browser, represented by a box with a smiley face: where's lab23? 11943function, represented by a rectangle with squiggly lines: where's lab23.degrassi.ca? 11944arrow pointing to resolver (server) represented by a box with a smiley face holding a magnifying glass 11945 11946(the function appends a base domain `degrassi.ca` to the end) 11947 11948### the base domain is called a "search domain" 11949On Linux, search domains are configured in `/etc/resolv.conf` 11950 11951Example: 11952`search degrassi.ca` 11953this tells `getaddrinfo` to turn `lab23` into `lab23.degrassi.ca` 11954 11955### getaddrinfo doesn't always use search domains
11956It uses an option called ndots to decide. 11957``` 11958search degrassi.ca 11959options ndots:5 11960``` 11961this means "only use search domains if the domain name contains less than 5 dots" 11962 11963### search domains can make DNS queries slower 11964browser: where's `jvns.ca`? 11965getaddrinfo, represented by a rectangle with squiggly lines: okay, first I'll try `jvns.ca.degrassi.ca` 11966 11967this is silly but it can happen! 11968 11969 11970### avoid search domains by putting a "." at the end 11971Use `http://jvns.ca.` instead of `http://jvns.ca` 11972 11973Illustration of a smiling stick figure with curly hair. 11974 11975person: "local" domain names like this mostly exist inside of big institutions like universities</div> 11976 </a> 11977 11978 <a href="/comics/resolver-vs-authoritative/"> 11979 <span class="title"> 11980 resolvers vs authoritative nameservers 11981 </span> 11982 <div class="keywords"></div> 11983 <div class="transcript">### panel 1 11984 11985One reason DNS is confusing is that the DNS server you query (a resolver) is different from the DNS server where the records are stored (a network of authoritative nameservers. 11986 11987Beside "resolver" there is an illustration of a smiling little box holding a magnifying glass, and beside "authoritative nameserver" there is an illustration of a smiling little box with a crown. 11988 11989### anytime your browser makes a DNS query, it's asking a resolver 11990Illustration of a conversation between a browser, represented by the Firefox logo of a fox wrapped around a globe, and a resolver, represented by a smiling little box holding a magnifying glass 11991 11992browser: what's the IP for `example.com`? 11993resolver: I'll find out for you! 11994 11995### anytime you update a domain's DNS records, you're updating an authoritative nameserver 11996Illustration between a smiling stick figure with curly hair, and an authoritative nameserver, represented by a pink box with a smiley face wearing a crown. 11997 11998person: set the IP for example.com to 1.2.3.4 11999authoritative nameserver: got it! Next time someone asks, that's what I'll tell them. 12000 12001### how a resolver handles queries 120021. check its cache, or (if that fails) 120032. find the right authoritative nameserver and ask it 12004 12005### how an authoritative nameserver handles queries 120061. check its database for a match 120072. that's it, there's no step 2. It's the authority! (illustration of a crown) 12008 12009### the terminology is really confusing 12010Other names for resolvers: 12011- recursive resolver 12012- DNS recursor 12013- public DNS server 12014- recursive nameserver 12015- DNS resolution service 12016- caching-only nameserver 12017 12018Types of authoritative nameservers: 12019- root nameserver 12020- TLD nameserver (like `.com` or `.ca`) 12021</div> 12022 </a> 12023 12024 <a href="/comics/resolvers-can-lie/"> 12025 <span class="title"> 12026 resolvers can lie 12027 </span> 12028 <div class="keywords"></div> 12029 <div class="transcript">### When a resolver gets a DNS query, it has 2 options: 12030 12031Illustration of a resolver, represented by a box with a smiley face holding a magnifying glass. 12032 12033resolver: I could tell you what the authoritative nameservers, said... or I could LIE! 12034 12035### block ads / malware 12036 12037Illustration of conversation between a resolver and a a browser, represented by the Firefox logo of a fox wrapped around a globe 12038 12039browser: what's the IP for doubleclick.net? 12040(ad domain, definitely exists) 12041resolver: that domain doesn't exist 12042 12043PiHole blocks ads this way. 12044 12045### reason to lie: to show you ads (rude!) 12046browser: what's the IP for zzz.jvns.ca? 12047(doesn't exist) 12048resolver: here's an IP that will show you ads! 12049 12050This is called "DNS hijacking". 12051 12052### reason to "lie": internal domain names 12053browser: what's the IP for corp.examplecat.com? 12054(doesn't exist on the public internet) 12055corporate resolver: here's an internal IP address! 12056 12057### reason to lie: airport DNS resolvers sometimes lie 12058browser: what's the IP for google.com? 12059airport resolver: you didn't log in yet so I will lie! here is our login page's IP! 12060 12061### how does your computer know which resolver to use? 12062When you connect to a network, the router tells your computer which search domain and resolver to use (using DHCP). 12063 12064Illustration of a router, represented by a box with antennae and a smiley face 12065 12066router: `192.168.1.1 search domain: lan`</div> 12067 </a> 12068 12069 <a href="/comics/ns-records/">
12070 <span class="title"> 12071 NS records 12072 </span> 12073 <div class="keywords"></div> 12074 <div class="transcript">### What's actually happening when the root nameserver redirects to the .com nameserver, on page 6? 12075 12076Illustration of a resolver, represented by a box with a smiley face holding a magnifying glass, and a root nameserver, represented by a pink box with a smiley face, wearing a stack of three crowns 12077 12078resolver: what's the IP for example.com? 12079root nameserver: I am not concerned with petty details like that. Here's the address of the .com nameserver 12080(this is an NS record) 12081 12082### The root nameserver can return two kinds of DNS records: 12083NS records: (in the Authority section) 12084``` 12085com. 172800 NS a.gtld-servers.net 12086com. 172800 NS b.gtld-servers.net 12087``` 12088com. is the name 12089172800 is the TTL 12090NS is the type 12091b.gtld-servers.net is the value 12092 12093glue records: (in the Additional section) 12094``` 12095a.gtld-servers.net 86400 A 192.5.6.30 12096b.gtld-servers.net 86400 A 192.33.14.30 12097``` 12098a.gtld-servers.net is the name 1209986400 is the TTL 12100A is the type 12101192.33.14.30 is the value 12102 12103 12104### The NS record gives you the domain name of the server to talk to next, but not its IP address. 12105resolver: But I need the IP for `a.gtld-servers.net` to communicate with it! 12106is there a glue record? 12107 12108### 2 ways the resolver gets the IP address 121091. If it sees a glue record for a.gtld-servers.net, the resolver will use that IP 121102. otherwise, it'll start a whole separate DNS lookup for a.gtld-servers.net 12111 12112 12113### glue records help resolvers avoid infinite loops 12114without a glue record for `a.gtld-servers.net`: disaster! 12115 12116resolver: what's the IP for `a.gtld-servers.net`? 12117root nameserver: You should ask `a.gtld-servers.net` 12118 12119### terminology note 12120NS records are DNS records with type "NS". 12121 12122Also, an "A record" means "record with type A", "MX record" means "record with type MX", etc. 12123 12124(confusingly, this is not true for glue records, glue records have type A or AAAA. It's weird, I know.)</div> 12125 </a> 12126 12127 <a href="/comics/negative-caching/"> 12128 <span class="title"> 12129 negative caching 12130 </span> 12131 <div class="keywords"></div> 12132 <div class="transcript">### Here's a problem I've had many times 12133 12134Illustration of a stick figure with curly hair and a distressed expression. 12135Person's thought bubble: I set up my new domain, everything looks good, but it's not working?!?! 12136 12137### I finally learned last year that my problem was "negative caching" 12138Same person, now smiling: now I never have this problem anymore! 12139 12140### resolvers cache negative results 12141Illustration of a resolver, represented by a box with a smiley face holding a magnifying glass, and an authoritative nameserver, represented by a box with a smiley face wearing a crown. 12142 12143resolver: what's the IP for `bees.jvns.ca`? 12144authoritative nameserver: I don't have any records for that! 12145resolver (thought bubble) `caching: no A records for bees. jvns.ca` 12146 12147### the TTL for caching negative results comes from the SOA record 12148`example.com. 3600 IN SOA ns.icann.org. noc.dns.icann.org. 2021120741 7200 3600 1209600 3600` 12149 12150it's the smaller of the first number and the last number (in this case 3600 seconds) 12151 12152### what you need to know about SOA records 121531. they control the negative caching TTL 121542. you can't change them (unless you run your own authoritative nameserver) 121553. how to find yours: `dig SOA yourdomain.com` 12156 12157### how to avoid this problem 12158Just make sure not to visit your domain before creating its DNS record! 12159 12160That's it! (if you really want more details, see RFC 2308)</div> 12161 </a> 12162 12163 <a href="/comics/mx-records/"> 12164 <span class="title"> 12165 MX records 12166 </span> 12167 <div class="keywords"></div> 12168 <div class="transcript">### there are two important problems in email 12169From: Kermit @frog.com 12170To: [email protected] 12171 121721. Make sure the message gets to the right recipient. This is what MX records are for. 121732. Make sure the sender didn't lie about their From: address. This is what SPF, DKIM, and DMARC records are for. 12174 12175SPF/DKIM/DMARC are very complicated but we'
12175ll give a tiny incomplete summary. 12176 12177### MX records tell you the mail server for a domain 12178``` 12179$ dig +short MX gmail.com 121805 gmail-smtp-in.l.google.com. 12181``` 121825 is the priority 12183google.com is the server's domain name 12184 12185### copy and paste your MX records 12186Illustration of a smiling stick figure with curly hair. 12187 12188person: you're probably using an email service like Fastmail/Gmail, so just copy the records they tell you to use 12189 12190### tiny guide to SPF/DKIM/DMARC records 12191 12192SPF: list of allowed sender IP addresses 12193Example: `v=spf1 ip4:2.3.4.5 -all` 12194 12195DKIM: sender's public key 12196Example: `v=DKIM1; k=rsa; p=MIGFMA0GCSqGSI.......` 12197 12198DMARC: what to do about SPF/DKIM failures 12199Example: `v=DMARC1; p=reject; rua=mailto:[email protected]`</div> 12200 </a> 12201 12202 <a href="/comics/life-of-a-dns-query/"> 12203 <span class="title"> 12204 life of a DNS query 12205 </span> 12206 <div class="keywords"></div> 12207 <div class="transcript">### 1 12208An illustration of a smiling stick figure with curly hair, talking to a browser, represented by the Firefox logo of a fox wrapped around a globe. 12209 12210person: I want to go to https://example.com 12211browser: hmm, I don't have an IP address for example.com cached. I'll ask a resolver! 12212 12213### 2 12214An illustration of a browser talking to a resolver, represented by a box with a smiley face holding a magnifying glass. 12215 12216browser: what's the IP for example.com? 12217resolver: hmm, I'll look in my cache... 12218 12219### 3 12220⤠DNS cache ⤠12221archive.org: 207.241.224.2 12222jvns.ca: 172.64.80.1 12223 12224resolver: nope, I don't have it cached, I need to ask the authoritative nameservers! I have the root nameserver IPs hardcoded. 12225 12226note: we're pretending the resolver has no .com domains cached. Normally it would use its cache to skip step 4. 12227 12228### 4 12229An illustration of a browser talking to a root nameserver, represented by a box with a smiley face wearing three crowns. 12230 12231resolver: What's the IP for example.com? 12232root nameserver: ask a .com nameserver! It's at a.gtld-servers.net 12233 12234â com NS a.gtld-servers.net. 12235ca NS a.ca-servers.net. 12236horse NS a.nic.horse. 12237 12238(NS stands for "nameserver") 12239 12240 12241### 5 12242An illustration of a browser talking to a .com nameserver, represented by a box with a smiley face wearing two crowns. 12243 12244resolver: what's the IP for example.com? 12245.com nameserver: ask an example.com. nameserver! It's at a.iana-servers.net 12246 12247list of DNS records: 12248neopets.com, NS, ns-42.awsdns-05.com. 12249â example.com, NS, a.iana-servers.net. 12250 12251### 6 12252An illustration of a browser talking to an example.com nameserver, represented by a box with a smiley face wearing one crown. 12253 12254resolver: what's the IP for example.com? 12255example.com nameserver: it's 93.184.216.34! 12256resolver: great, I'll tell the browser! 12257 12258â example.com, A, 93.184.216.34 12259</div> 12260 </a> 12261 12262 <a href="/comics/lets-meet-dig/"> 12263 <span class="title"> 12264 let's meet dig 12265 </span> 12266 <div class="keywords"></div> 12267 <div class="transcript">### dig is my favourite tool for investigating DNS issues 12268 12269I find its default output unnecessarily confusing, but it's the only standard tool I know that will give you all the details. 12270 12271### tiny guide to dig's full output 12272``` 12273$ dig example.com 12274; <<>> DiG 9.16.24 <<>> +all example.com 12275;; global options: +cmd 12276;; Got answer: 12277;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 27580 1227818 12279;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1 12280 12281;; OPT PSEUDOSECTION: 12282; EDNS: version: 0, flags:; udp: 1232 12283;; QUESTION SECTION: 12284; example.com. IN A 12285 12286;; ANSWER SECTION: 12287example.com. 86400 IN A 93.184.216.34 12288;; Query time: 0 msec 12289;; SERVER: 127.0.0.1#53(127.0.0.1) 12290;; WHEN: Wed Jan 26 11:32:03 EST 2022 12291;; MSG SIZE rcvd: 56 12292 12293``` 12294`NOERROR` is the response code 12295 12296`example.com. 86400 IN A 93.184.216.34` is the answer to our DNS query. The "." at the end means that example.com isn't a subdomain of some other domain (like it'
12296s not example.com.degrassi.ca). This might seem obvious, but DNS tools like to be unambiguous. 12297 12298### panel 3: 12299Illustration of a smiling stick figure with curly hair. 12300 12301person: `$ dig +noall +answer` means "Just show me the answer section of the DNS response." It's a lot less to look at! 12302 12303### panel 4: 12304 12305`$ dig +noall +answer example.com` 12306 12307`example.com. 86400 IN A 93.184.216.34` 12308 12309example.com is the name 1231086400 is the TTL 12311IN is the class 12312A is the record type 1231393.184.216.34 is the content 12314 12315just the answer! so much less overwhelming!</div> 12316 </a> 12317 12318 <a href="/comics/getaddrinfo/"> 12319 <span class="title"> 12320 getaddrinfo 12321 </span> 12322 <div class="keywords"></div> 12323 <div class="transcript">### panel 1: 12324One weird thing about DNS is that different programs on a single computer can get different results for the same domain name. 12325 12326Let's talk about why! 12327 12328Illustration of a program, represented by a box with a smiley face, and a resolver (server), represented by a box with a smiley face holding a magnifying glass. Between them is a function, represented by a rectangle with squiggly lines on it. There are arrows going back and forth between the function and both the program and the resolver (server). 12329 12330The function is the problem. 12331 12332### reason 1: many (but not all!!) programs use the function getaddrinfo for DNS lookups... 12333ping, represented by a box with a smiley face: I use getaddrinfo! 12334dig, also represented by a box with a smiley face: I don't! 12335 12336So if you see an error message like "`getaddrinfo: nodename or servname not provided...`", that's a DNS error. 12337 12338### and not using getaddrinfo might give a different result 12339- the program might not use `/etc/hosts` (dig doesn't) 12340- the program might use a different DNS resolver (some browsers do this) 12341 12342### reason 2: there are many different versions of `getaddrinfo`... 12343- the one in `glibc` 12344- the one in `musl libc` 12345- the one in Mac OS 12346 12347And of course, they all behave slightly differently :) 12348 12349### you can have multiple getaddrinfos on your computer at the same time 12350For example on a Mac, there's your system `getaddrinfo`, but you might also be running a container that's using `musl`. 12351 12352### glibc and musl getaddrinfo are configured with `/etc/resolv.conf` 12353IP of resolver to use 12354``` 12355# Generated by NetworkManager 12356nameserver 192.168.1.1 12357nameserver fd13: d987:748a::1 12358``` 12359On a Mac, `/etc/resolv.conf` exists, but it's not used by the system `getaddrinfo`.</div> 12360 </a> 12361 12362 <a href="/comics/dns-packet/"> 12363 <span class="title"> 12364 everything in a DNS packet 12365 </span> 12366 <div class="keywords"></div> 12367 <div class="transcript"> 12368I literally mean everything, I copied this verbatim from a real DNS request using Wireshark. (DNS packets are binary but we're showing a human-readable representation here) 12369 12370### Let's look at the actual data being sent during a DNS query: 12371 12372Illustration of a browser, represented by the Firefox logo of a fox wrapped around a globe, talking to a resolver, represented by a box with a smiley face holding a magnifying glass. 12373 12374browser: what's the IP for example.com? 12375resolver: 93.184.216.34! 12376 12377 12378### request 12379`Query ID: 0x05a8` 12380(randomly generated) 12381`Flags: 0x1000` 12382(these flags just mean "this is a request") 12383`Questions: 1` 12384`Answer records: 0` 12385`Authority records: 0` 12386`Additional records: 0` 12387`Question:` 12388`Name: example.com` 12389`Type: A 12390(A is for IPv4 address. other types: MX, CNAME, AAAA, etc) 12391`Class: IN` 12392(IN stands for "INternet") 12393 12394### response 12395`Query ID: 0x05a8` 12396(matches request ID) 12397 12398`Flags: 0x8580` 12399the response code is encoded in the last 4 bits of these flags. The 3 main response codes are: 12400- NOERROR (success!) 12401- NXDOMAIN (doesn't exist!) 12402- SERVFAIL (error!) 12403``` 12404Questions: 1 12405Answer records: 1 12406Authority records: 0 12407Additional records: 0 12408``` 12409(copied from request) 12410``` 12411Question: 12412Name: example.com 12413``` 12414(domain names aren't case sensitive) 12415``` 12416Type: A 12417Class: IN 12418Answer records: 12419Name: example.com 12420Type: A 12421Class: IN 12422TTL: 86400 12423Content: 93.184.216.34 12424``` 12425(the IP we asked for) 12426``` 12427Authority records: 12428(empty) 12429Additional records: 12430(empty) 12431``` 12432page 12 ("NS records") talks more about these 2 sections 12433 12434Illustration of a smiling stick figure with curly hair. 12435 12436Person: I'm always surprised by how little is actually in a DNS packet!</div> 12437 </a> 12438 12439 <a href="/comics/cast-of-characters/">
12440 <span class="title"> 12441 DNS: cast of characters 12442 </span> 12443 <div class="keywords"></div> 12444 <div class="transcript">Let's meet the cast and see how they communicate with each other! 12445 12446browser: where's example.com? (function call) 1244793.184.216.34! 12448â 12449function: where's example.com? (DNS query) 1245093.184.216.34! 12451â 12452resolver: where's example.com? (DNS query) 1245393.184.216.34! 12454â 12455authoritative nameservers 12456 12457### browser 12458Your browser uses DNS to look up IP addresses every time it visits a domain, like example.com. 12459The browser has a DNS cache. 12460 12461### function 12462Your operating system provides a function to do DNS lookups. On Linux and Mac it's getaddrinfo. 12463Your operating system also might have a DNS cache. 12464 12465### resolver 12466The function sends requests to a server called a resolver which knows how to find the authoritative nameservers. 12467The resolver has a DNS cache. 12468 12469### authoritative nameservers 12470The authoritative nameservers are the servers where the DNS records are actually stored. 12471They're wearing crowns because they're In Charge.</div> 12472 </a> 12473 12474 <a href="/comics/dns-records/"> 12475 <span class="title"> 12476 DNS records 12477 </span> 12478 <div class="keywords"></div> 12479 <div class="transcript">### When you make DNS changes for your domain, you're editing a DNS record 12480ТÑÑе: A 12481Name (subdomain): paw 12482Use @ for root 12483IPv4 address: 1.2.3.4 12484TTL: 1 min 12485 12486Here's what the same record looks like with dig 12487(we'll explain dig on page 18) 12488 12489``` 12490$ dig +noall +answer paw.examplecat.com 12491paw.examplecat.com. 60 IN A 1.2.3.4 12492``` 12493 12494### DNS records have 5 parts 12495- name (eg `tail.examplecat.com`) 12496- type (eg `CNAME`) 12497- value (eg `tail.jvns.ca`) 12498- TTL (eg `60`) 12499- class (eg `IN`) 12500 12501different record types have different kinds of values: `A` records have an IP address, and `CNAME` records have a domain name. 12502 12503### name 12504`paw.examplecat.com` 12505 12506When you create a record, you'll usually write just the subdomain (like `paw`). 12507 12508When you query for a record, you'll get the whole domain name (like `paw.examplecat.com`). 12509 12510### TTL 12511`60` 12512"time to live". How long to cache the record for, in seconds. 12513 12514### class 12515`IN` 12516"IN" stands for "INternet". You can ignore it, it's always the same. 12517 12518### record type 12519`A` 12520"A" stands for "IPv4 Address". 12521 12522### value 12523`1.2.3.4` 12524the IP address we asked for!</div> 12525 </a> 12526 12527 <a href="/comics/dig-command-line-arguments/"> 12528 <span class="title"> 12529 dig command line arguments 12530 </span> 12531 <div class="keywords"></div> 12532 <div class="transcript">illustrtion of a laptop. its keyboard just says QWERTY. 12533 12534### the basics: dig @SERVER TYPE DOMAIN 12535(SERVER and TYPE are both optional) 12536 12537Examples: 12538``` 12539dig example.com 12540dig @8.8.8.8 NS example.com 12541dig TXT example.com 12542dig @8.8.8.8 example.com 12543``` 12544default type: A 12545default server: from `/etc/resolv.conf` 12546(on Linux) 12547 12548### tip: put +noall +answer in your ~/.digrc 12549This makes your output more readable by default, and you can always go back to the full output with `dig +all`. 12550 12551### dig +noall 12552Hide all output. 12553Useless by itself, but `dig +noall +authority` will just show you the "Authority" section of the response. 12554 12555### dig +short DOMAIN 12556Only show the record content. 12557 12558`$ dig +short example.com 93.184.216.34` 12559 12560### dig +trace DOMAIN 12561Traces how the domain gets resolved, starting at the root nameservers. 12562 12563This avoids all the caches, which is useful to make sure you set your record correctly.</div> 12564 </a> 12565 12566 <a href="/comics/tiny-dns-resolver/"> 12567 <span class="title"> 12568 a tiny DNS resolver 12569 </span> 12570 <div class="keywords"></div> 12571 <div class="transcript">On page 5 (life of a DNS query), we saw how resolvers work. This code does the same thing, but it actually works. 12572 12573``` 12574def resolve(domain): 12575# Start at a root nameserver 12576nameserver = "198.41.0.4" 12577# A "real" resolver would check its cache here 12578while True: 12579reply = query(domain, nameserver) 12580ip = get_answer(reply) 12581if ip: 12582# Best case: we get an answer to our query and we're done return ip 12583nameserver_ip = get_glue(reply) 12584if nameserver_ip: 12585# Second best: we get the IP address* of the nameserver to ask next 12586nameserver = nameserver_ip 12587else: 12588# Otherwise: we get the domain name* of the nameserver to ask next 12589nameserver_domain = get_nameserver(reply) nameserver = resolve(nameserver_domain) 12590* Actual DNS resolvers are more complicated than this, but this is the core algorithm. 12591``` 12592Smiling stick figure with curly hair: You can find the whole program at https://github.com/jvns/tiny-resolver 12593</div> 12594 </a> 12595 12596 <a href="/comics/a-and-aaaa-records/">
12597 <span class="title"> 12598 A & AAAA records 12599 </span> 12600 <div class="keywords"></div> 12601 <div class="transcript">### there are two kinds of IP addresses: IPv4 and IPv6 12602 12603Every website needs an IPv4 address. 12604 12605IPv6 addresses are optional. 12606 12607### panel 2: 12608 12609A stands for IPv4 Address 12610 12611Example: `93.184.216.34` 12612 12613AAAA stands for IPv6 AAAAddress (joke, but kinda true) 12614 12615Example: `2606:2800:220:1:248:1893:25c8:1946` 12616 12617it's called AAAA (4 As) because IPv6 addresses have 4x as many bytes 12618 12619### in theory, the Internet is moving from IPv4 to IPv6 12620This is because there are only 4 billion IPv4 addresses (the internet has grown a LOT since the 1980s when IPv4 was designed!) 12621 12622### happy eyeballs* 12623If your domain has both an A and an AAAA record, clients will use an algorithm called "happy eyeballs" to decide whether IPv4 or IPv6 will be faster. 12624 12625`*` yes that is the real name 12626 12627### using IPv6 isn't always easy 12628- not all web hosts give you an IPv6 address 12629- lots of ISPs don't support IPv6 (mine doesn't!) 12630 12631### IP addresses have owners 12632You can find any IP's owner by looking up its ASN ("Autonomous System Number"). 12633 12634(except local IPs like `192.168.x.x`, `127.x.X.X`, `10.x.x.x`, `172.16.x.x`)</div> 12635 </a> 12636 12637 <a href="/comics/write-a-tiny-program/"> 12638 <span class="title"> 12639 write a tiny program 12640 </span> 12641 <div class="keywords"></div> 12642 <div class="transcript">Does your bug involve a library you don't understand? 12643 12644Illustration of an unhappy stick figure with curly hair. 12645person (thinking): UGH, `requests` is NOT working how I expected it to! 12646 12647I like to convert my code using that library into a tiny standalone program which has the same bug: 12648 12649Illustration of two programs, one represented by a big messy scribble, the second represented by three tidy lines. 12650giant buggy program => 20 lines of buggy code 12651 12652I find this makes it WAY EASIER to experiment and ask for help. And if it turns out that library actually has a bug, you can use your tiny program to report it.</div> 12653 </a> 12654 12655 <a href="/comics/write-a-message-asking-for-help/"> 12656 <span class="title"> 12657 write a message asking for help 12658 </span> 12659 <div class="keywords"></div> 12660 <div class="transcript">When I'm REALLY stuck, I'll write an email to a friend: 12661- "Here's what I'm trying to do..." 12662- "I did X and I expected Y to happen, but instead..." 12663- "Could this be because....?" 12664- "This seems impossible because..." 12665- "I've tried A, B, and C to fix it, but...." 12666 12667This helps me organize my thoughts, and often by the time I finish writing, I've magically fixed the problem on my own! 12668 12669It has to be a specific person, so that the imaginary version of them in my mind will say useful things :)</div> 12670 </a> 12671 12672 <a href="/comics/write-a-failing-test/"> 12673 <span class="title"> 12674 write a failing test 12675 </span> 12676 <div class="keywords"></div> 12677 <div class="transcript">If your program already has tests, adding a failing test is a great way to work on your bug! 12678 12679Illustration of a smiling stick figure with curly hair. 12680 12681person (thinking): this function should return X, but it's returning Y 12682 12683- it forces you to pinpoint what exactly the bug is 12684- it's easy to tell when you've fixed it (the test passes!) 12685- you can keep the test to make sure the bug doesn't come back</div> 12686 </a> 12687 12688 <a href="/comics/why-bugs-feel-impossible/"> 12689 <span class="title"> 12690 why some bugs feel "impossible" 12691 </span> 12692 <div class="keywords">[debugging]</div> 12693 <div class="transcript"></div> 12694 </a> 12695 12696 <a href="/comics/use-a-debugger/"> 12697 <span class="title"> 12698 use a debugger 12699 </span> 12700 <div class="keywords"></div> 12701 <div class="transcript">A debugger is a tool for stepping through your code line by line and looking at variables. But not all debuggers are equal! Some languages' debuggers have more features than others. Your debugger might let you: 12702- jump into a REPL to poke around (see page 25) 12703- watch a location in memory and stop the program any time it's modified 12704- "record replay" debuggers let you record your entire program's execution and time travel 12705 12706Illustration of a smiling stick figure with curly hair. 12707 12708person (thinking): I love record/replay debuggers because they make hard-to-reproduce bugs easier: I just have to reproduce the bug once</div> 12709 </a> 12710 12711 <a href="/comics/types-of-debugging-tools/">
12712 <span class="title"> 12713 types of debugging tools 12714 </span> 12715 <div class="keywords"></div> 12716 <div class="transcript">Here are some tools I've found useful: 12717- debuggers! (most languages have one!) 12718- profilers: `perf, pprof, py-spy` 12719- tracers: `strace, ltrace, ftrace, BPF tools` 12720- network spy tools: `tcpdump, wireshark, ngrep, mitmproxy` 12721- web automation tools: `selenium, playwright` 12722- load testers: `ab, wrk` 12723- test frameworks: `pytest, RSpec` 12724- linters/static analysis tools: `black, eslint, pyright` 12725- data formatting tools: `xd, hexdump, jq, graphviz` 12726- dynamic analysis tools: `valgrind, asan, tsan, ubsan` 12727- fuzzers/property testing: `hypothesis, quickcheck, Go's fuzzer` 12728 12729(I've never used those last two but lots of people say they're helpful.)</div> 12730 </a> 12731 12732 <a href="/comics/try-out-a-new-tool/"> 12733 <span class="title"> 12734 try out a new tool 12735 </span> 12736 <div class="keywords"></div> 12737 <div class="transcript">There are TONS of great debugging tools (listed on the next page!), but often they have a steep learning curve. 12738 12739Some tips to get started: 12740- get someone more experienced to show you an example of how they'd use the tool. (this is SO helpful!!!) 12741- try it out when investigating a low stakes bug, so it's no big deal if it doesn't work out. 12742- take notes with examples of the options you used, so you can refer to them next time.</div> 12743 </a> 12744 12745 <a href="/comics/track-your-progress/"> 12746 <span class="title"> 12747 track your progress 12748 </span> 12749 <div class="keywords">[debugging]</div> 12750 <div class="transcript"></div> 12751 </a> 12752 12753 <a href="/comics/timebox-your-investigation/"> 12754 <span class="title"> 12755 timebox your investigation 12756 </span> 12757 <div class="keywords"></div> 12758 <div class="transcript">Sometimes I need to trick myself into getting started: 12759 12760Illustrations of a stick figure with short curly hair. 12761 12762person (thinking, looking unhappy): "UGH, I do NOT want to look at this CSS bug!!!!" 12763 12764Giving myself a time limit really helps: 12765 12766Illustration of an alarm clock 12767person (thinking, now smiling): "Okay, I'll just see what I can figure out in 20 minutes..." 12768 12769You can't always solve it in 15 minutes, but this works surprisingly often! 12770 12771... 15 minutes later ... 12772 12773person (thinking, happy): "all fixed! That wasn't so hard!"</div> 12774 </a> 12775 12776 <a href="/comics/tidy-up-your-code/"> 12777 <span class="title"> 12778 tidy up your code 12779 </span> 12780 <div class="keywords"></div> 12781 <div class="transcript">Messy code is harder to debug. 12782 12783Illustration of a smiling stick figure with curly hair. 12784person (thinking): "this function is 100 lines??? who named these variables?!?!" (annotation: it was me) 12785 12786Doing a tiny bit of refactoring can make things easier, like: 12787- rename variables or functions 12788- format it with a code formatter (`go fmt`, `black`, etc.) 12789- add comments 12790- delete old/untrue comments 12791 12792Don't go overboard with the refactoring though: making too many changes can easily introduce new bugs.</div> 12793 </a> 12794 12795 <a href="/comics/tell-a-friend-what-you-learned/"> 12796 <span class="title"> 12797 tell a friend what you learned 12798 </span> 12799 <div class="keywords"></div> 12800 <div class="transcript">I love to celebrate squashing a bug by telling a friend: 12801 12802Illustration of a smiling stick figure with curly hair. 12803 12804person: hey marie, did you know about this weird thing that can happen with CSS flexbox? 12805 12806Some possible outcomes of this: 12807- they've seen that bug too, and teach me something else! 12808- they learn something new! 12809- they ask questions I hadn't thought of 12810- they tell me about a website/tool I didn't know about 12811- it helps solidify my knowledge!</div> 12812 </a> 12813 12814 <a href="/comics/take-a-break/"> 12815 <span class="title"> 12816 take a break 12817 </span> 12818 <div class="keywords"></div> 12819 <div class="transcript">Illustration of a steaming hot beverage. 12820 12821Investigating a tricky bug requires a LOT of focus. 12822 12823Illustration of a sad stick figure with long straight hair. 12824 12825person (thinking): "ugh, nothing is working..." 12826 12827(annotations on person): googling the same error message for the 7th time. very frustrated 12828 12829Instead, try one of these magical debugging techniques (even a 5 minute break can really help!): 12830- ride your bike! 12831- go to bed! 12832- get a coffee! 12833- have a shower! 12834- eat lunch! 12835 12836Illustration of the same person, now happily riding their bike.</div> 12837 </a> 12838 12839 <a href="/comics/sprinkle-assertions-everywhere/">
12840 <span class="title"> 12841 sprinkle assertions everywhere 12842 </span> 12843 <div class="keywords"></div> 12844 <div class="transcript">Some languages have an `assert` keyword that you can use to crash the program if a condition fails. Assertions let you: 12845- come up with something that should ALWAYS be true 12846- immediately crash the program if it isn't this variable is undefined!!! 12847 12848Illustration of a program, represented by a box with an unhappy face. 12849program (thinking): "this variable is undefined!!! STOP EVERYTHING!" 12850 12851This is a great way to force yourself to think about what's ALWAYS true in your program, and check if you're right. 12852 12853Illustration of a smiling stick figure with curly hair. 12854person (thinking): "the radius can never be 0, right? or can it?"</div> 12855 </a> 12856 12857 <a href="/comics/shorten-your-feedback-loop/"> 12858 <span class="title"> 12859 shorten your feedback loop 12860 </span> 12861 <div class="keywords"></div> 12862 <div class="transcript">when you're investigating a bug, you'll need to run the buggy code a million times. 12863 12864Illustration of a stick figure, holding their hands to their face in despair. 12865 12866person (thinking): ugh, i need to type all this information into the form to trigger the bug again??? this is literally the 30th time :( :( 12867 12868ways to speed it up: 12869- use a browser automation tool to fill in forms / click buttons for you! 12870- write a unit test! 12871- autorun your code every time you save! 12872</div> 12873 </a> 12874 12875 <a href="/comics/share-debugging-stories/"> 12876 <span class="title"> 12877 share your debugging stories 12878 </span> 12879 <div class="keywords">[debugging]</div> 12880 <div class="transcript"></div> 12881 </a> 12882 12883 <a href="/comics/rule-things-out/"> 12884 <span class="title"> 12885 rule things out 12886 </span> 12887 <div class="keywords"></div> 12888 <div class="transcript">Once I have a list of suspects, I can think about how to eliminate them. 12889 12890Illustration of a pensive stick figure with curly hair. 12891 12892person (thinking): "I'm really confused, but I can at least check if the server returned the right HTTP response here.." 12893 12894Illustration of a box that says "client", and a box that says "server", with arrows going back and forth between them. Both boxes are labelled "suspicious". 12895 12896person (thinking): "that response looks good! the server isn't the problem!" 12897 12898Illustration of a box that says "client", and a box that says "server", with arrows going back and forth between them. The client box is labelled "suspicious", with exclamation marks and question marks surrounding it, but the "server" box is labelled "ok", with a check mark and smiley faces. 12899 12900note: here we're assuming that was the only request being made. Otherwise this wouldn't be a safe conclusion :)</div> 12901 </a> 12902 12903 <a href="/comics/retrace-the-codes-steps/"> 12904 <span class="title"> 12905 retrace the code's steps 12906 </span> 12907 <div class="keywords"></div> 12908 <div class="transcript">Here's a classic (but still very effective!) way to get started: 12909 129101. find the line of code where the error happened 129112. trace backwards to investigate what could have caused that error. keep asking "why?" 12912 12913example: 12914- There's an error on line 58... 12915- that's because this variable has the wrong value... 12916- the value is set by calling this function... 12917- that function is making an HTTP request to the API... 12918- the API response doesn't have the format I expected! Why is that? 12919 12920In the corner of the page, there is an illustration of a goofy-looking bug with a long neck and curly antennae saying "Chase me!"</div> 12921 </a> 12922 12923 <a href="/comics/reread-the-error-message/"> 12924 <span class="title"> 12925 reread the error message 12926 </span> 12927 <div class="keywords"></div> 12928 <div class="transcript">After I've read the error message, I sometimes run into one of these 3 problems: 12929 12930Each person is represented by a stick figure with curly hair. 12931 12932### 1. misreading the message 12933 12934person (thinking) ok, it says the error is in file X 12935 12936spoiler: it actually said file Y 12937 12938### 2. disregarding what the message is saying 12939 12940person (thinking): well, the message says X, but that's impossible... 12941 12942spoiler: it was possible 12943 12944### 3. not actually reading it 12945 12946person (thinking): ok, I read it... 12947 12948spoiler: she did not read it</div> 12949 </a> 12950 12951 <a href="/comics/reproduce-the-bug/">
12952 <span class="title"> 12953 reproduce the bug 12954 </span> 12955 <div class="keywords"></div> 12956 <div class="transcript">My favourite way to get information about buggy code is to run the buggy code and experiment on it. (Add print statements! Make a tiny change!) 12957 12958If the bug is happening on your computer every time you run your program: hooray! You've reproduced the bug! 12959 12960An illustration of a smiling stick figure with curly hair. 12961person (thinking): "ok, time to debug! I've got my print statements ready to go!" 12962 12963But if you can't make the bug happen, you're left guessing. 12964 12965An illustration of a sad stick figure with curly hair. 12966person (thinking): "what was variable X set to when the bug happened? guess there's NO WAY TO KNOW" 12967 12968cute illustration of a bug: the next page has tips!</div> 12969 </a> 12970 12971 <a href="/comics/reduce-randomness/"> 12972 <span class="title"> 12973 reduce randomness 12974 </span> 12975 <div class="keywords"></div> 12976 <div class="transcript">It's much easier to debug when your program does the exact same thing every time you run it. 12977 12978Illustration of a sad stick figure with curly hair. 12979 12980person (thinking): "the bug only happens 10% of the time, it's SO HARD to figure out if my change fixed it or not." 12981 12982There are a bunch of tools for controlling your program's inputs to reduce randomness, for example: 12983- many random number generators let you set the seed so you get the same results every time. 12984- `faketime` fakes the current time. 12985- libraries like ruby's `vcr` can record http requests. 12986- record/replay debuggers like `rr` record everything.</div> 12987 </a> 12988 12989 <a href="/comics/read-the-librarys-code/"> 12990 <span class="title"> 12991 read the library's code 12992 </span> 12993 <div class="keywords"></div> 12994 <div class="transcript">Lots of code isn't documented. But when there are no docs, there's always the source code! It sounds intimidating at first, but a quick search of the code sometimes gets me my answer really quickly. 12995 12996Tips for exploring an unfamiliar library's code: 12997 12998- search the tests! Tests are a GREAT source of examples. 12999- git clone it locally to make it easier to navigate. 13000- search for your error message and trace back. 13001- if it's a Python/JS/Ruby library, sometimes I'll edit the library's code on my computer to add print statements (just remember to take them out after!)</div> 13002 </a> 13003 13004 <a href="/comics/read-the-error-message/"> 13005 <span class="title"> 13006 read the error message 13007 </span> 13008 <div class="keywords"></div> 13009 <div class="transcript">Error messages are a goldmine of information, but they can be very annoying to read: 13010 13011(image of an error message, represented by a stack of squiggly lines, with 2 notes pointing to it): 13012- giant 50 line stack trace full of impenetrable jargon, often seems totally unrelated to your bug 13013- can even be misleading, like "permission denied" sometimes means "doesn't exist" 13014 13015Tricks to extract information from giant error messages: 13016- If there are many different error messages, start with the first one. Fixing it will often fix the rest. 13017- If the end of a long error message isn't helpful, try looking at the beginning (scroll up!) 13018- On the command line, pipe it to `less` so that you can scroll/search it 13019```(./my_program 2>&1 | less)``` 13020 13021Note: if you don't include `2>&1`, `less` won't show you the error messages (just the output)</div> 13022 </a> 13023 13024 <a href="/comics/read-the-docs/"> 13025 <span class="title"> 13026 read the docs 13027 </span> 13028 <div class="keywords"></div> 13029 <div class="transcript">There are many ways to read the docs! 13030 13031- the surgical strike: Search for a specific function, find an example on the page, copy it and leave. (this is often me :)) 13032- the question quest: You have a specific question and you'll keep skimming different pages until you find the answer. 13033- the IDE integration: Set up your editor or IDE so that you can instantly jump to a function's documentation. 13034- the rigorous read: Get a cup of coffee and read all of the docs cover to cover, like a book. 13035</div> 13036 </a> 13037 13038 <a href="/comics/preserve-the-crime-scene/">
13039 <span class="title"> 13040 preserve the crime scene 13041 </span> 13042 <div class="keywords"></div> 13043 <div class="transcript">One of the easiest ways to start is to save a copy of the buggy code and its inputs/outputs: 13044 13045An illustration of stick figure wearing a top hat. Beside them is a bug in a mason jar. 13046person (thinking): "don't touch anything! we need to preserve evidence!" 13047 13048Depending on the situation, you might want to: 13049- make a git commit of the buggy code! (on a branch, just for you) 13050- save the input that triggered the bug 13051- save logs/screenshots to analyze later</div> 13052 </a> 13053 13054 <a href="/comics/one-thing-at-a-time/"> 13055 <span class="title"> 13056 one thing at a time 13057 </span> 13058 <div class="keywords"></div> 13059 <div class="transcript">It's tempting to try lots of fixes at once to save time: 13060 13061Illustration of a smiling stick figure with curly hair. 13062dream: I'm going to add Z, and replace X with Y, and improve C-- that'll definitely fix it! 13063 13064Illustration of the same stick figure, now sad. 13065reality: ... now there's a new problem AND it's still broken 13066 13067If I found I've done this by accident, I'll: 13068- undo all my changes (`git stash!`) 13069- make a list of things to investigate, one at a time</div> 13070 </a> 13071 13072 <a href="/comics/make-sure-your-code-is-running/"> 13073 <span class="title"> 13074 make sure your code is running 13075 </span> 13076 <div class="keywords"></div> 13077 <div class="transcript">Illustration of an unhappy stick figure with curly hair. 13078 13079person (thinking): NOTHING I try is helping, this is IMPOSSIBLE 13080 13081person (thinking): wait... nothing I try is changing anything.... is my code even being run???? 13082 13083If my changes have no effect at all, often it means I've made a silly mistake (like forgetting to restart the app) and my changes aren't being run! 13084 13085I like to check that my code is being run by printing something out (like `print("asdf"`). Or, if that's not possible, I'll introduce an error so that it crashes.</div> 13086 </a> 13087 13088 <a href="/comics/minimal-reproduction/"> 13089 <span class="title"> 13090 make a minimal reproduction 13091 </span> 13092 <div class="keywords">[debugging]</div> 13093 <div class="transcript"></div> 13094 </a> 13095 13096 <a href="/comics/look-at-recent-changes/"> 13097 <span class="title"> 13098 look at recent changes 13099 </span> 13100 <div class="keywords"></div> 13101 <div class="transcript">Often when something is broken, it's because of a recent change. Usually I look at recent changes manually, but git bisect is an amazing tool for finding exactly which git commit caused the problem. 13102 13103We don't have space for a full `git bisect` tutorial here, but here's how you start using it: 13104 13105``` 13106git bisect start 13107git bisect bad HEAD 13108git bisect good 1fe9dc 13109``` 13110(1fe9dc is the ID of a commit that doesn't have the bug) 13111 13112Then you can either tag buggy commits manually or run a script that does it automatically.</div> 13113 </a> 13114 13115 <a href="/comics/list-what-youve-learned/"> 13116 <span class="title"> 13117 list what you've learned 13118 </span> 13119 <div class="keywords">[debugging]</div> 13120 <div class="transcript"></div> 13121 </a> 13122 13123 <a href="/comics/learn-one-small-thing/"> 13124 <span class="title"> 13125 learn one small thing 13126 </span> 13127 <div class="keywords"></div> 13128 <div class="transcript">Bugs are a GREAT way to discover things on the edge of your knowledge. 13129 13130Illustrations of a stick figure with curly hair. 13131 13132person (thinking, looking worried): "hmm, part of the problem here is that I don't understand how position: absolute works..." 13133 13134Finding one small thing I don't understand and learning it is really useful (and pretty fun!) 13135 13136person (thinking, now smiling): "now I understand position: absolute! cool!"</div> 13137 </a> 13138 13139 <a href="/comics/spy-tools/"> 13140 <span class="title"> 13141 know your spy tools 13142 </span> 13143 <div class="keywords">[debugging]</div> 13144 <div class="transcript"></div> 13145 </a> 13146 13147 <a href="/comics/keep-a-log-book/">
13148 <span class="title"> 13149 keep a log book 13150 </span> 13151 <div class="keywords"></div> 13152 <div class="transcript">I don't usually write things down. But 2 hours into debugging, I get really confused: 13153 13154Illustration of a frazzled-looking stick figure with curly hair. 13155 13156person (thinking): wait, what did that error message I saw 2 hours ago say again exactly?? 13157 13158person (thinking): did I already try this??? 13159 13160Keeping a document with notes makes it WAY easier to stay on track. It might contain: 13161- specific inputs I tried 13162- error messages I saw 13163- stack overflow URLs 13164 13165The log makes it easier to ask for help later if needed!</div> 13166 </a> 13167 13168 <a href="/comics/jump-into-a-repl/"> 13169 <span class="title"> 13170 jump into a REPL 13171 </span> 13172 <div class="keywords"></div> 13173 <div class="transcript">In dynamic languages (like Python / Ruby / JS), you can use a debugger to jump into an interactive console (aka "REPL") at any point in your code. 13174 13175Here's how to do it in Python 3: 13176 131771. edit your code 13178 13179`my_var = call_some_function() breakpoint()` 13180add "`breakpoint()`"! 13181 13182 131832. rerun your code (refresh the page, whatever) 131843. play around in the REPL! You can call any function you want / try out fixes! 13185 13186How to do it in other languages: 13187- Ruby: 13188`binding.pry` 13189- Python (before 3.7): 13190`import pdb; pdb.set_trace()` 13191- Javascript: 13192`debugger;` 13193</div> 13194 </a> 13195 13196 <a href="/comics/investigate-the-bug-together/"> 13197 <span class="title"> 13198 investigate the bug together 13199 </span> 13200 <div class="keywords"></div> 13201 <div class="transcript">I find investigating a bug with someone else SO MUCH more fun than doing it alone. 13202 13203Illustrations of two smiling stick figures, one with short curly hair, and one with longer straight hair. 13204 13205Debugging together lets you: 13206 13207- Teach each other new tools! 13208 13209person 1: I wish we could find out x, but that's impossible... 13210person 2: Let's use my favourite tool, strace!!!!!! 13211 13212- Learn new concepts! 13213 13214person 2: What is this CORS thing?!?! 13215person 1: Oh, I can explain that! 13216 13217- Keep each other on track 13218 13219person 2: Maybe the problem is Y? 13220person 1: We already ruled that out! Right, I forgot!</div> 13221 </a> 13222 13223 <a href="/comics/inspect-unreproducible-bugs/"> 13224 <span class="title"> 13225 inspect unreproducible bugs 13226 </span> 13227 <div class="keywords"></div> 13228 <div class="transcript">When you can't reproduce a bug locally, it's tempting to just try random fixes and pray. Resist the temptation! 13229 13230Some ways to get information: 13231- try to reproduce the environment where it happened 13232- ask for screenshots / screen recordings 13233- add more logging, deploy your code, and repeat until you understand what caused the bug 13234- read the code VERY VERY carefully (incredibly boring but it actually does work sometimes) 13235- do your experimentation somewhere where you can reproduce the bug (on a staging server? on someone else's computer?)</div> 13236 </a> 13237 13238 <a href="/comics/identify-one-small-question/"> 13239 <span class="title"> 13240 identify one small question 13241 </span> 13242 <div class="keywords"></div> 13243 <div class="transcript">Debugging can feel huge and impossible. But all you have to do to make progress is: 132441. come up with ONE QUESTION about the bug. 132452. make sure the question is small enough that you can investigate it in ~20 minutes 132463. figure out the answer to that question 13247 13248Illustration of a smiling stick figure with curly hair, surrounded by other question marks, which are crossed out. 13249 13250person (thinking): hmm, this database all these query is slow... well, can I find out if the query is using an index? 13251 13252ignore other questions for now! one at a time!</div> 13253 </a> 13254 13255 <a href="/comics/guesses-are-often-wrong/"> 13256 <span class="title"> 13257 guesses are often wrong 13258 </span> 13259 <div class="keywords">[debugging]</div> 13260 <div class="transcript"></div> 13261 </a> 13262 13263 <a href="/comics/find-the-type-of-bug/">
13264 <span class="title"> 13265 find the type of bug 13266 </span> 13267 <div class="keywords"></div> 13268 <div class="transcript">If the bug is totally new to you, find out if there's a name people use for that type of bug! 13269 13270Illustration of two stick figures. Person 1 has curly hair and looks worried, Person 2 has straight hair and is smiling. 13271 13272person 1: "this bug is happening intermittently, it's so weird." 13273person 2: "that sounds like it might be a race condition..." 13274person 1 (thinking): "oh, what's a race condition?" 13275 13276examples: 13277- `terminated by signal SIGSEGV (address boundary error)` 13278segmentation fault 13279- `flexbox: div doesn't fit in other div (CSS)` 13280item overflowing container 13281- `nodename nor servname provided, or not known` 13282DNS lookup failure 13283- `RecursionError: maximum recursion depth exceeded` 13284stack overflow 13285</div> 13286 </a> 13287 13288 <a href="/comics/find-related-bugs/"> 13289 <span class="title"> 13290 find related bugs 13291 </span> 13292 <div class="keywords"></div> 13293 <div class="transcript">Illustration of two adorable bugs. They are holding hands and their antennae are intertwined. 13294 13295When you're done fixing a bug, glance around to see if there are any obvious places in your code that have the same bug. 13296 13297Illustration of a smiling stick figure with short curly hair. 13298 13299person (thinking): "I was calling function X wrong, I'll check if we're calling that function wrong anywhere else!" 13300 13301person (thinking): "wow, my assumption about how Y worked was TOTALLY wrong, I should go back and fix some things..."</div> 13302 </a> 13303 13304 <a href="/comics/find-a-version-that-works/"> 13305 <span class="title"> 13306 find a version that works 13307 </span> 13308 <div class="keywords"></div> 13309 <div class="transcript">If I have a bug with how I'm using a library, I like to: 13310 13311- find a code example in the documentation 13312- make sure it works 13313- slowly change it to be more like my broken code 13314- test if it's still working after every single tiny change 13315 13316Illustration showing a bunch of points with arrows between them. Each point has a check mark beside it, until one that is labelled "Oh THAT'S what broke it!!!" 13317 13318This puts me back on solid ground: with every change I make that DOESN'T cause the bug to come back, I know that change wasn't the problem.</div> 13319 </a> 13320 13321 <a href="/comics/find-a-new-source-of-info/"> 13322 <span class="title"> 13323 find a new source of info 13324 </span> 13325 <div class="keywords"></div> 13326 <div class="transcript">We all know to look at the official documentation. Here are some less obvious places to look for answers: 13327 13328- the project's Discord, Slack, IRC channel, or mailing list 13329- code search (search all of GitHub for how other people are using that library!) 13330- GitHub issues (did someone else have the same problem?) 13331- release notes (is the bug fixed in the new version?) 13332- a book chapter (you might have a book on this topic!) 13333- blog posts (sometimes there's an amazing explanation on the 2nd page of Google results) 13334</div> 13335 </a> 13336 13337 <a href="/comics/explain-the-bug-out-loud/"> 13338 <span class="title"> 13339 explain the bug out loud 13340 </span> 13341 <div class="keywords"></div> 13342 <div class="transcript">Explaining what's going wrong out loud is magic. 13343 13344Illustrations of two stick figures. One has curly hair, and one has short straight hair and is wearing a big t-shirt with a picture of a rubber duck. 13345 13346person (looking sad): "so, when I do X thing, I'm getting an error, and it doesn't make any sense because I already checked that A and B are working...." 13347other person: huh... 13348 13349person (now smiling, with an exclamation mark above their head): "OH I SEE WHAT I DID WRONG" 13350other person (also smiling): "happy to help!" 13351 13352People call this "rubber ducking" because the other person might as well be a rubber duck.</div> 13353 </a> 13354 13355 <a href="/comics/draw-a-diagram/">
13356 <span class="title"> 13357 draw a diagram 13358 </span> 13359 <div class="keywords"></div> 13360 <div class="transcript">Some ideas: 13361 13362### network diagram 13363An illustration of a network, with a cylinder labelled DB, and boxes labelled "factory", "handler", "obj", "model 1", and "model 2", with arrows amongst them showing their relationships. 13364 13365### flowchart 13366A flowchart with boxes "set flag", "run cmd", "if failed, retry", and "return result", with arrows amongst them illustrating a process. 13367 13368### state diagram 13369A diagram with boxes labelled "inventory page", "cart page", and "checkout page", with arrows amongst them labelled "cart icon", "continue shopping", "checkout", and "cancel". 13370 13371### or anything else (like a data structure!) 13372A box labelled "on | off | on | off". The first "off" is labelled "[1, 1, 1, 0, 0, 1, 1, 1, 0", and the second "off" is labelled "5 seconds".</div> 13373 </a> 13374 13375 <a href="/comics/document-your-quest/"> 13376 <span class="title"> 13377 document your quest 13378 </span> 13379 <div class="keywords"></div> 13380 <div class="transcript">For very tricky bugs, writing up an explanation of what went wrong and how you figured it out is an amazing way to share knowledge and make sure you really understand it. 13381 13382Ways I've done this in the past: 13383- complain about it in the internal chat! (so people can search for it!) 13384- write a quick explanation in the commit message 13385- write a fun blog post telling my tale of woe! 13386- for really important work bugs, write a 5-page document with graphs explaining all the weird stuff I learned along the way 13387</div> 13388 </a> 13389 13390 <a href="/comics/do-the-annoying-thing/"> 13391 <span class="title"> 13392 do the annoying thing 13393 </span> 13394 <div class="keywords"></div> 13395 <div class="transcript">Illustrations of an unhappy-looking stick figure with short curly hair. 13396 13397Sometimes when I'm debugging, there are things I'll refuse to try because they take too long. 13398 13399person (thinking): ugh, that part of the code is so confusing, I don't want to look at it... 13400 13401But as I become more and more desperate, eventually I'll give in and do the annoying thing. Often it helps! 13402 13403person (thinking): FINE, I'll look at that code... oh, yeah, here's the bug.</div> 13404 </a> 13405 13406 <a href="/comics/do-a-victory-lap/"> 13407 <span class="title"> 13408 do a victory lap 13409 </span> 13410 <div class="keywords"></div> 13411 <div class="transcript">Once you've solved it, don't forget to celebrate! Take a break! Feel smart! 13412 13413Illustration of a smiling stick figure with curly hair. 13414 13415person (thinking): "i did it, i did it, i'm amazing" (now is not the time for humility) 13416 13417The best part of understanding a bug is that it makes it SO MUCH easier for you to solve similar future bugs. 13418 13419Illustration of a smiling stick figure with curly hair, and another figure with short spiky hair. 13420 13421person (thinking): I've seen something like this before, maybe the problem is X? 13422colleague: (annotation, saying that they're awestruck at your brilliance)</div> 13423 </a> 13424 13425 <a href="/comics/delete-the-buggy-code/"> 13426 <span class="title"> 13427 delete the buggy code 13428 </span> 13429 <div class="keywords"></div> 13430 <div class="transcript">Sometimes the buggy code is not worth salvaging and should be deleted entirely. Reasons you might do this: 13431 13432Illustration of an uneasy-looking stick figure with curly hair. 13433- it uses a confusing library / tool 13434 13435person (thinking): this library isn't working, I'm going to switch to Y instead 13436 13437Illustration of the same person, now smiling. 13438- you have a better idea for how to implement it 13439 13440person (thinking): I bet I could avoid all these problems if I took X approach instead...</div> 13441 </a> 13442 13443 <a href="/comics/track-changes/">
13444 <span class="title"> 13445 debugging tip: track what you changed 13446 </span> 13447 <div class="keywords">[debugging]</div> 13448 <div class="transcript"></div> 13449 </a> 13450 13451 <a href="/comics/slow-down/"> 13452 <span class="title"> 13453 debugging tip: slow down 13454 </span> 13455 <div class="keywords">[debugging]</div> 13456 <div class="transcript"></div> 13457 </a> 13458 13459 <a href="/comics/ask-questions/"> 13460 <span class="title"> 13461 debugging tip: ask lots of questions 13462 </span> 13463 <div class="keywords">[debugging]</div> 13464 <div class="transcript"></div> 13465 </a> 13466 13467 <a href="/comics/comment-out-code/"> 13468 <span class="title"> 13469 comment out code 13470 </span> 13471 <div class="keywords"></div> 13472 <div class="transcript">Commenting out code is an amazing way to quickly do experiments and figure out which part of your code is to blame. You can: 13473 13474- comment out a function call and replace it with a hardcoded value, to check if the function call is broken 13475- if the error message doesn't give you a line number, comment out huge chunks of the program until the problem goes away 13476- comment out some code and rewrite it to see if the new version is better</div> 13477 </a> 13478 13479 <a href="/comics/colours-graphs-and-sounds/"> 13480 <span class="title"> 13481 colours, graphs, and sounds 13482 </span> 13483 <div class="keywords"></div> 13484 <div class="transcript">Instead of printing text, your program can tell you about its state by generating a picture! Or playing sounds at key moments! 13485 13486Some ways your programs can generate pictures or sounds: 13487- add colours to your log lines (every letter of 'colours' is a different colour) 13488- add red outlines around every HTML element! ("red" and "outlines" have a red outline around them) 13489- Haskell has an option to beep at the start of every major garbage collection (there's a bell icon after "beep") 13490- draw a chart of events over time (chart icon) 13491- use graphviz to generate a diagram of your program's internal state (there's a picture of a little graph diagram with a -> b, a -> c)</div> 13492 </a> 13493 13494 <a href="/comics/brainstorm-some-suspects/"> 13495 <span class="title"> 13496 brainstorm some suspects 13497 </span> 13498 <div class="keywords"></div> 13499 <div class="transcript">brainstorming every possible cause I can think of helps me not get stuck on the 1 or 2 most obvious possibilities. 13500 13501In a box representing a sheet of paper: 13502- could I be using the wrong version of this library? 13503- am I passing the wrong argument to function X? 13504- is something wrong with the server? 13505- is the entire internet broken??? 13506 13507(there are two notes on the side pointing at the above text) 13508- sometimes I find it easier to think clearly when writing by hand on paper. 13509- no filter! even ridiculous ideas! 13510</div> 13511 </a> 13512 13513 <a href="/comics/ask-debugging-questions/"> 13514 <span class="title"> 13515 ask lots of questions 13516 </span> 13517 <div class="keywords">[debugging]</div> 13518 <div class="transcript"></div> 13519 </a> 13520 13521 <a href="/comics/analyze-the-logs/"> 13522 <span class="title"> 13523 analyze the logs 13524 </span> 13525 <div class="keywords"></div> 13526 <div class="transcript">If you can't reproduce a bug, sometimes you need to comb through the logs for clues. Some tips: 13527 13528- filter out irrelevant lines (for example with grep -v) 13529- find 1 failed request and search for that request's ID to get all the logs for that request 13530- build a timeline: copy and paste log lines (and your interpretations!) into a document 13531- if you see a suspicious log line, search to make sure it doesn't also happen during normal operation 13532- if there's a cascade of errors, find the first error that started the problems</div> 13533 </a> 13534 13535 <a href="/comics/add-pretty-printing/"> 13536 <span class="title"> 13537 add pretty printing 13538 </span> 13539 <div class="keywords"></div> 13540 <div class="transcript">Sometimes you print out an object, and it just prints the class name and reference ID, like this: 13541 13542`MyObject<#18238120323>` 13543 13544Illustration of a frowning stick figure with curly hair. 13545 13546person (thinking): "ugh, thanks, very helpful... " 13547 13548Implementing a custom string representation for a class you're often printing out can save a LOT of time. The name of the method you need to implement is: 13549- Python: `.__str__ ` 13550- Ruby: `.to_s` 13551- JavaScript: `.toString` 13552- Java: `.toString` 13553- Go: `String()` 13554 13555Also, pretty-printing libraries (like `pprint` in Python or `awesome_print` in Ruby) are great for printing out arrays/hashmaps.</div> 13556 </a> 13557 13558 <a href="/comics/add-lots-of-print-statements/">
13559 <span class="title"> 13560 add lots of print statements 13561 </span> 13562 <div class="keywords"></div> 13563 <div class="transcript">I love to add print statements that print out 1, 2, 3, 4, 5... 13564 13565An illustration of a printer printing out lines of text. 13566 13567``` 13568console.log(1) 13569console.log(2) 13570console.log(3) 13571``` 13572 13573Using descriptive strings is smarter, but I usually use numbers or "wtf???" 13574 13575 13576This helps me construct a timeline of which parts of my code ran and in what order: 13577 13578Illustration of timeline of code, with some arrows pointing at it numbered 1, 3, 2. Between 1 and 3, it says "everything is okay". Between 3 and 2 it says "the cause", with a picture of a bug, and after 2, it says "the error message" with a picture of a page of text. 13579 13580Often I'll discover something surprising, like "wait, 3, never got printed??? Why not???".</div> 13581 </a> 13582 13583 <a href="/comics/add-a-comment/"> 13584 <span class="title"> 13585 add a comment 13586 </span> 13587 <div class="keywords"></div> 13588 <div class="transcript">Some bug fixes are a little counterintuitive. Otherwise you would have written the code that way in the first place! You might think: 13589 13590Illustration of a smiling stick figure with curly hair. 13591 13592person (thinking): "I'll remember why I added this code, I spent 5 hours this is a debugging it! 13593 13594this is a trap!!!!! 13595 13596Adding a comment can help future you (or your coworkers!) avoid accidentally reviving a bug later. 13597 13598person (thinking): ooh, I could simplify this code! 13599 13600Illustration of a dancing bug, singing "I'm back!"</div> 13601 </a> 13602 13603 <a href="/comics/a-debugging-manifesto/"> 13604 <span class="title"> 13605 a debugging manifesto 13606 </span> 13607 <div class="keywords"></div> 13608 <div class="transcript">### 1. inspect, don't squash 13609Try to fix the bug (crossed out, bad) 13610Understand what happened (checkmarks, smiley faces) 13611 13612### 2. Being stuck is temporary. 13613person (thinking): I WILL NEVER FIGURE THIS OUT 13614... 20 minutes later... 13615person (thinking): Wait, I haven't tried X... 13616 13617### 3. Trust nobody and nothing 13618person (thinking): This library can't be buggy... 13619person (thinking): Or CAN IT??? 13620(slowly growing horror) 13621off to the side, a bug looks on, with a sneaky expression 13622 13623### 4. It's probably your code 13624person (thinking): I KNOW my code is right 13625... 2 hours later ... 13626person (thinking): Ugh, my code WAS the problem?!!? 13627 13628### 5. don't go it alone 13629person 1: "WHAT IS HAPPENING?!?" 13630person 2: "What if we try X?" 13631 13632### 6. There's always a reason. 13633A computer, illustrated by a box with a smiley face, surrounded by ones and zeros: Computers are always logical, even when it doesn't feel that way. 13634 13635### 7. Build your toolkit 13636person (thinking, holding a box labelled TOOLZ): "wow, the CSS inspector makes debugging SO much easier" 13637 13638### 8. It can be an adventure. 13639person: "You wouldn't BELIEVE the weird bug I found!" 13640adorable weird bug, standing beside them: hi!</div> 13641 </a> 13642 13643 <a href="/comics/float-gaps/"> 13644 <span class="title"> 13645 the gaps between floats 13646 </span> 13647 <div class="keywords"></div> 13648 <div class="transcript">## title: the gaps between floats 13649 13650## panel 1: floating point numbers have to fit into 32 or 64 bits 13651 13652This means there are only 2^64 64-bit floats, the same way there are only 2^64 64-bit integers 13653 13654## panel 2: this means floating point numbers have to be spread out 13655 13656you can imagine them all spaced out on a number line, like this: (picture of a bunch of lines, with small gaps between them. The gaps are smaller on the left and bigger on the right) 13657 13658## panel 3: the gaps start small. 13659 13660the next 64-bit float after 1.0 is 1 point (lots of 0s) 2 13661 13662the gap between these two floats is 0 point (lots of 0s) 2, or 2^-52 13663 13664gaps are always a power of 2 13665 13666## panel 4: the gaps get bigger as the numbers get bigger 13667 13668the next 64-bit float after 1000000000000000000 is that number plus 16384. 13669 13670so the gap is 16384, or 2^14! 13671 13672## panel 5: the gaps make calculations inaccurate 13673 13674when you do math on floating point numbers, often you have to round the result to the nearest float 13675 13676usually this doesnât make a big difference, but small mistakes can add up 13677 13678## panel 6: this inaccuracy is inevitable 13679 13680if you want math to be fast, you have to store the numbers in a fixed number of bits, like 64 bits. So youâre always going to have accuracy issues. 13681 13682</div> 13683 </a> 13684 13685 <a href="/comics/signed-integers/">
13686 <span class="title"> 13687 signed vs unsigned integers 13688 </span> 13689 <div class="keywords"></div> 13690 <div class="transcript">## signed vs unsigned integers 13691 13692## there are 2 ways to interpret every integer 13693 13694unsigned: 13695- always 0 or more 13696- example: 8 bit unsigned ints are `0` to `255` 13697 13698signed: 13699- half positive, half negative 13700- example: 8 bit signed ints. are `-128` to `127` 13701 13702## negative integers are represented in a counterintuitive way 13703You might think that this is -5: `10000101` 13704 13705(1 is the sign bit, and 101 in binary is 5) 13706 13707But actually this is -5: `11111011` 13708 13709this looks weird, but we'll explain why! 13710 13711## integer addition wraps around 13712 13713for example, for 8-bit integers `255 + 1 = 0` 13714 13715for 16-bit integers, `65535 + 1 = 0` 13716 13717by "addition", we mean "what the x86 `add` instruction does" 13718 13719## panel: 13720 13721but if `255 + 1 = 0`, you could also say `255 = -1` 13722 13723## examples of bytes and their signed/unsigned ints 13724 13725 13726| byte | unsigned | signed | 13727|----------|----------|--------| 13728| `00000000` | 0 | 0 | 13729| `01111111` | 127 | 127 | 13730| `01111111` | 128 | -128 | 13731| `10000001` | 129 | -129 | 13732| `11111011` | 251 | -5 | 13733| `11111111` | 255 | -1 | 13734 13735subtract 256 from unsigned numbers to get the signed numbers 13736 13737## this way of handling signed integers is called "two's complement" 13738 13739It's popular because you can use the same circuits to add signed and unsigned integers. 13740 13741`5 + 255` has exactly the same result as `5 + (-1)`: they're both 4! 13742 13743</div> 13744 </a> 13745 13746 <a href="/comics/science-floating-point/"> 13747 <span class="title"> 13748 science <3 floating point 13749 </span> 13750 <div class="keywords"></div> 13751 <div class="transcript">## science <3 floating point 13752 13753## floating point was invented to do scientific computation 13754- weather simulations! 13755- earthquake modeling! 13756- orbital mechanics! 13757 13758## scientists don't need unlimited precision... 13759 13760we only know an electron's mass to 9 decimal places anyway... 13761 137629 decimal places is already VERY precise! 13763 13764## but they do need TINY numbers and GIANT numbers 13765mass of hydrogen atom: 13766 13767`1.6735575 * 10^-24` grams 13768 13769distance to Andromeda galaxy: 13770 13771`2.4 * 10^22` meters 13772 13773## floating point is inspired by scientific notation 13774`1.6735575 x 10^-24` 13775 13776The idea in floating point is to store a number by splitting it into: 13777 13778- the exponent (like `-24`) 13779- the multiplier (like `1.6735575`) 13780- and its sign (+ or -) 13781 13782## floating point isn't just used for science though 13783For example, Javascript's number type is floating point. Before it added `BigInt` in 2021, Javascript didn't have integers at all! 13784 13785Similarly, numbers in JSON are often interpreted as floating point numbers. 13786 13787 13788## panel: 13789people usually explain floating point as "it's scientific notation, but in binary!" That's true, but I've never found it intuitive so we're going to explain it a different way.</div> 13790 </a> 13791 13792 <a href="/comics/nan/"> 13793 <span class="title"> 13794 NaN and infinity 13795 </span> 13796 <div class="keywords"></div> 13797 <div class="transcript">## NaN and infinity 13798 13799## NaN stands for "not a number" 13800It means the result of the calculation is undefined. 13801 13802`0/0 = NaN` 13803 13804`sqrt(-1) = NaN` 13805 13806`log(-1) = NaN` 13807 13808## infinity 13809"Infinity" just means "this number is too big for floating point to handle." There are two infinities: one positive, one negative. 13810 13811`2.0**1024 = inf` 13812 13813(`2.0**1024` means `2^1024`) 13814 13815`-1/0 = -inf` 13816 13817`inf 10 = inf` 13818 13819`inf - inf = NaN` 13820 13821 13822## NaNs spread 13823As soon as one NaN gets in, it gets everywhere 13824 13825`NaN * 5 = NaN` 13826 13827`NaN + 2 = NaN` 13828 13829## NaN != NaN 13830 13831NaN isn't equal to anything (including itself) 13832 13833## NaN and infinity: the bits 13834A floating point value is `NaN` or `infinity` if the bits in the exponent are all 1. For example, this is a `NaN`: 13835 13836`01111111 11110001 00000000 00000000 00000000 00000000 00000000 00000000` 13837 13838It's `infinity` if the offset bits are all 0, otherwise it's `NaN`. 13839 13840There are 2^52 values like this: 2 of them are `±infinity` and the other 2^52-2 are `NaN`. 13841 13842We usually treat `NaN` like a single value though. 13843 13844## a note on byte order 13845All of the floating point examples in this zine use a big endian byte order, be
13845cause it's easier to read. But most computers use a little endian byte order. 13846 13847You can see this in action at `https://memory-spy.wizardzines.com`</div> 13848 </a> 13849 13850 <a href="/comics/bytes-not-bits/"> 13851 <span class="title"> 13852 meet the byte 13853 </span> 13854 <div class="keywords"></div> 13855 <div class="transcript">## meet the byte 13856 13857## You might have heard that a computer's memory is a series of bits (Os and 1s)... 13858 13859`010100110101010110110111` 13860 13861but you only access them in groups of 8 bits - a byte! 13862 13863`01010011 1010101 10110111` 13864 13865 13866## 2 ways to think about a byte 138671. 8 bits 13868 138692. an integer from 0 to 255 13870 13871`00000000` = `0` 13872 13873`00000001` (8 bits!) = `1` (integer!) 13874 13875`00000010` = `2` 13876 13877`01011001` = `89` 13878 13879 13880## you can't just access 1 bit 13881Every byte in your computer's memory has an address. 13882 13883If you want to fetch 1 bit, you need to fetch the whole byte at that address and then extract the bit. 13884 13885 13886## some things that are 1 byte 13887- the boolean `true` (in C) `00000001` 13888- the ASCII character F `01000110` 13889- the red part of the colour `#FF00FF` `11111111` 13890 13891## most things are more than one byte 13892- integers and floats are Usually 4 bytes or 8 bytes 13893- strings are LOTS of bytes (for example, in UTF-8 a heart emoji is 3 bytes) 13894 13895## bytes weren't always 8 bits 13896In the past, people experimented with lots of different byte sizes (2, 3, 4, 5, 6, 8, and 10 bits!) 13897 13898But now we've standardized on 8 bits pretty much everywhere. 13899</div> 13900 </a> 13901 13902 <a href="/comics/little-big-endian/"> 13903 <span class="title"> 13904 little vs big endian 13905 </span> 13906 <div class="keywords"></div> 13907 <div class="transcript">## little endian / big endian 13908 13909## we write dates in two main orders 139101. 2023-03-17 ("big endian") 139112. 17-03-2023 ("little endian") 139123. 03-17-2023 ("american") 13913 13914"big endian" means that the big unit (the year) is at the start ("big end first") 13915 13916## similarly: computers order bytes in 2 ways 13917Here are 2 ways your computer might represent the integer 271: 139181. big endian: `00000001 00001111` 139192. little endian: `00001111 00000001` 13920 13921How this corresponds to 271: 13922 13923`00000001 00001111` is 271 in binary 13924 13925## When you send integers on a computer network, they have to be big endian. Here's how that works: 13926 13927Computer A has the 16-bit integer "271" in its memory: `00001111 00000001` 13928 13929Computer A flips the bytes and sends it as big endian: `00000001 00001111` 13930 13931Computer B receives the big endian integer 13932 13933Computer B flips the bytes and stores it in memory as little endian: `00001111 00000001` 13934 13935## a little history 13936Before 1980, computers ordered their bytes in different ways. 13937 13938In 1980, the Internet started being standardized, causing a huge fight over which byte order to use on the Internet. 13939 13940The terms "big/little endian" come from that fight: they were coined in an article called "On Holy Wars and a Plea For Peace" which compares the byte order fight to the Big/Little Endians in Gulliver's Travels. 13941 13942Big endian won that fight, so most Internet protocols (IPv4, TCP, UDP, etc.) are big endian. 13943 13944But almost all modern computers are little endian. Some machines, like the Xbox 360, are big endian though.</div> 13945 </a> 13946 13947 <a href="/comics/integers/"> 13948 <span class="title"> 13949 integers 13950 </span> 13951 <div class="keywords"></div> 13952 <div class="transcript">## integers 13953 13954## panel 1: 13955To decode bytes as integers, we need to know 3 things: 139561. the integer's size (8 bit, 16 bit, 32 bit, or 64 bit) 139572. is it little or big endian? 139583. is it signed or unsigned? 13959 13960## panel 2: 13961how signed integers work is the hardest part) to understand (I only learned how it works a couple months ago!). Just knowing that unsigned and signed integers are different will take you a long way. 13962 13963## 2 bytes, 3 interpretations 13964`254 | 0 ` 13965 13966We could interpret these 2 bytes as: 139671. `254` (little endian) 139682. `65024` (big endian, unsigned) 139693. `-512` (big endian, signed) 13970 13971## how you decode bytes depends on the context 13972- in a program's memory, the type of the variable tells you the integer's size and if it's signed/unsigned 13973- your CPU determines if integers are big or little endian (you don't have a choice) 13974- for a binary network protocol (like DNS), the specification (for DNS, that's RFC 1035) will tell you how to decode the bytes 13975 13976## examples of types 13977- in Rust, an `i64` is a signed 64-bit integer 13978- in Go, a `uint32` is an unsigned 32-bit integer 13979- in C, a `short` is usually a signed 16-bit integer, depending on the platform</div> 13980 </a> 13981 13982 <a href="/comics/integer-overflow/">
13983 <span class="title"> 13984 integer overflow 13985 </span> 13986 <div class="keywords"></div> 13987 <div class="transcript">## integer overflow 13988 13989### integers have a limited amount of space 13990 13991The 4 usual sizes for integers are 8 bits, 16 bits, 32 bits, and 64 bits 13992 13993### the biggest 8-bit unsigned integer is 255 13994 13995... so what happens if you do 255 + 1? 13996going above/below the limits is called overflow 13997 13998the result wraps around to the other side 13999 14000 255 + 1 = 0 14001 255 + 3 = 2 14002 200 * 2 = 144 14003 0 - 2 = 254 14004 14005### maximum numbers for different sizes 14006 14007bits: unsigned signed 14008 140098: 127 255 14010 1401116: 32767 65535 14012 1401332: 2 billion ~4 billion 14014 1401564: ~9 quintillion ~18 quintillion 14016### overflows often don't throw errors 14017 14018computer (thinking): "255 + 1? that number is 8 bits, so the answer is 0! that's what you wanted right?" 14019 14020This can cause VERY tricky bugs 14021### some languages where integer overflow happens 14022 14023 Java/Kotlin 14024 C/C++ 14025 Rust 14026 Swift 14027 C# 14028 SQL 14029 R 14030 Go 14031 Dart 14032 Python (only in numpy) 14033 14034Some throw errors on overflow, some don't, for some it depends on various factors. Look up how it works in your language! 14035</div> 14036 </a> 14037 14038 <a href="/comics/printing-floats/"> 14039 <span class="title"> 14040 how floats are printed 14041 </span> 14042 <div class="keywords"></div> 14043 <div class="transcript">## how floats are printed 14044 14045## computers lie when they print out floats 14046(by rounding) 14047 14048For example `0.12` isn't `0.12`, it's actually (roughly): 14049 14050`0.119999999999999995559` 14051 14052is my computer LYING to me??? about NUMBERS? 14053 14054 14055## the string -> float translation 14056If your program says: 14057 14058`x = 0.12` 14059 14060your interpreter / compiler needs to translate "`0.12`" into the float `0.119999999999999995559`. Most languages will use the `strtod` ("string to double") function from libc to do that translation. 14061 14062## the float -> string translation 14063This is where the rounding comes in. Computers round to make the numbers shorter and easier to read. 14064`1.19999999999999995559` 14065 14066 ⪠1.2 14067 14068## float -> string translation is actually super complicated 14069Every floating point number needs a unique string representation. 14070 14071There are a bunch of academic papers about how to do this well, search "Printing floating point numbers accurately" to read more about it. 14072 14073## some examples of printing floats 14074`1.19900000000000006573` 14075 14076âª`1.199` 14077 14078`1.19999999000000001637` 14079 14080âª`1.19999999` 14081 14082`1.19999999999998996358` 14083 14084⪠`1.9999999999999` 14085 14086`1.19999999999999995559` 14087 14088âª`1.2` 14089 14090## you can also print floats in base 16 or base 2 14091 14092For example, 0.1 as a 32-bit float is: 14093 14094base 16: `0x1.99999ap-4` (`p-4` is the base 16. version of `e-4`) 14095 14096base 2: `1.10011001100110011001101p-100` 14097 14098The base 2/base 16 representations are not rounded, but they're rarely used.</div> 14099 </a> 14100 14101 <a href="/comics/bit-tricks/"> 14102 <span class="title"> 14103 how bitwise operations are used 14104 </span> 14105 <div class="keywords"></div> 14106 <div class="transcript">### Binary formats often pack information into bytes very tightly to save space. 14107 14108For example, here are 2 bytes from a real TCP packet: 14109 14110`10000000 00010000` 14111 14112The first "`1000`" is the offset (4 bits) 14113The following "`000`" is reserved (3 bits) 14114The remaining "`00010000`" are the flags (9 bits) 14115 14116Here's how `&`, `|`, `<<`, `>>` can be used to pack/unpack data into bytes. 14117 14118### bit masking 14119Let's say we have the 2 bytes from the previous panel, and we want to extract just the flags part. Here's how to do it with `&` (bitwise and): 14120 14121The idea is that you put a mask "on top" of the bytes to erase bits: 14122 14123`X: 10000000 00010000` (number) 14124`0x01FF: 000000001 1111111` (bit mask) 14125`x & 0x01FF: 000000001 0010000` (how they combine) 14126 14127`000000001`: these 7 bits all get set to 0 14128`0010000`: these 4 bits stay the same 14129 14130### check/set bit flags 14131(see page 16 for more) 14132 14133set a bit flag with or: 14134``` 14135x = x | 0b010000; 14136``` 14137 14138check a bit flag with and: 14139``` 14140if ((x & 0b010000) != 0) { 1414100001000 X 14142} 14143``` 14144(this example is in C) 14145 14146### unpack/pack bits 14147Now let's talk about the offset from the first panel. We can't do calculations in it with the packed form, so we need to unpack it. 14148 14149You can unpack with >>: 14150``` 1415110000000 -> 00001000 14152X -> X >> 4 14153``` 14154 14155and pack with <<: 14156``` 141570001000 -> 10000000 14158X -> X << 4 14159``` 141601000 in binary is 8, which in this case is the TCP offset value.</div> 14161 </a> 14162 14163 <a href="/comics/hexadecimal/">
14164 <span class="title"> 14165 hexadecimal 14166 </span> 14167 <div class="keywords"></div> 14168 <div class="transcript">## panel 1: let's talk about how to write binary data 14169 14170one way: binary 14171 14172`01111111 11111111 11111111`\ 14173it's easy to see the bits... 14174 14175`1010110110101001010`\ 14176but it's hard to read a lot of them 14177 14178another way: base 10\ 14179`83888607`\ 14180but I have NO IDEA how many bits that is 14181 14182## panel 2: now the best way to write binary data: base 16! 14183 14184It's short AND maps well to bits!\ 14185`7fffff`\ 14186Every hexadecimal digit represents 4 bits. So 1 byte (8 bits) is always 2 hexadecimal digits. 14187 14188## panel 3: there are 16 hex digits: `0 â f` 14189 14190``` 14191| hex | decimal | binary | 14192| 0 | 0 | 0000 | 14193| 1 | 1 | 0001 | 14194| 2 | 2 | 0010 | 14195| 3 | 3 | 0011 | 14196| 4 | 4 | 0100 | 14197| 5 | 5 | 0101 | 14198| 6 | 6 | 0110 | 14199| 7 | 7 | 0111 | 14200| 8 | 8 | 1000 | 14201| 9 | 9 | 1001 | 14202| a | 10 | 1010 | 14203| b | 11 | 1011 | 14204| c | 12 | 1100 | 14205| d | 13 | 1101 | 14206| e | 14 | 1110 | 14207| f | 15 | 1111 | 14208``` 14209 14210## panel 4: 0x means it's hex 14211 14212In many languages, the 0x prefix lets you write numbers in hexadecimal.\ 14213For example, in C:\ 142140x20 == 32 (base 16)\ 142150b10100 20 (base 2)\ 14216061 == 49 (base 8)\ 14217be careful: the 0 prefix meaning "base 8" can really trip you up! 14218 14219## panel 5: things hexadecimal is used for 14220 14221color codes! (e.g. `#FF00FF`)\ 14222memory addresses!\ 14223hashes! (like git commit IDs)\ 14224displaying binary data! (like with `hexdump`) 14225</div> 14226 </a> 14227 14228 <a href="/comics/the-bits/"> 14229 <span class="title"> 14230 floating point: the bits 14231 </span> 14232 <div class="keywords"></div> 14233 <div class="transcript">### panel 1: 14234Floats need to fit into 64 bits. But how do we actually convert a number like 10.87 into 64 bits? 14235 14236First, we split the number into 3 parts: the sign, a power of 2 and an offset 14237 14238(The usual term is "significand", but I find that term calling it "offset") 14239 14240`10.87 = + (8 + 2.87) `
14241 14242(8 is the biggest power of 2 that's less than 10.87) 14243 14244Next, we encode the sign, power of 2, and offset into bits! 14245 14246### encoding the sign (1 bit) 14247 `+ is 0` 14248 14249 `- is 1` 14250 14251### floating point encoding is defined in the IEEE 754 standard 14252 14253since it's standardized, it works the same way on every computer! 14254 14255it was originally defined in 1985 14256 14257 14258### encoding the exponent (11 bits, 2^-1023 to 2^1023) 14259 14260`8` 14261 14262â `2^3 = 8` 14263 14264`3` 14265 14266â add 1023 (this makes sure that the result is positive) 14267 14268`1026` 14269 14270â write it in binary, in 11 bits 14271 14272`10000000010` 14273 14274 14275### encoding the offset (52 bits) 14276 14277`2.87` 14278 14279â divide by the gap size, 2^-49 14280 14281in this case (2^exponent-52) 14282 14283`1615666366319165.3 ` 14284 14285â round 14286 14287`1615666366319165` 14288 14289â write it in binary, 52 bits 14290 14291`01011011110101110000101000 ` 14292 14293`11110101110000101000111101` 14294 14295### And here's `10.87`! 14296 14297`01000000 00100101 10111101 01110000 10100011 11010111 0001010 00111101` 14298 14299</div> 14300 </a> 14301 14302 <a href="/comics/floating-point-representation/"> 14303 <span class="title"> 14304 floating point representation 14305 </span> 14306 <div class="keywords"></div> 14307 <div class="transcript">### the (64-bit) floating point number line 14308 14309Floating point numbers aren't evenly distributed. Instead, they're organized into windows: [0.25, 0.5], [0.5, 1], [1,2], [2,4], [4,8], [8,16], all the way up to [2^1023, 2^1024]. 14310 14311Every window has 252 floats in it. 14312 14313The windows [-2, -1], [-1, -1/2], [-1/2, -1/4], [-1/4, 0], [0, 1/4], [1/4, 1/2], [1/2, 1], and [1, 2], each have 2^52 numbers. [2, 4] has 2^52 numbers. [4, 8] has 2^52 numbers. 14314 14315Illustration of a horizontal line, with the windows plotted out on it, showing that each window doubles in size as it moves away from zero. 14316 14317### the windows go from REALLY small to REALLY big 14318 14319The window closest to 0 is [2^-1023, 2^-1022] 14320 14321This is TINY: a hydrogen atom weighs about 2^-76 grams. 14322 14323The biggest window is [2^1023, 2^1024]. 14324 14325This is HUUUGE: the farthest galaxy we know about is about 2^90 meters away. 14326 14327### the gaps between floats double with every window 14328 14329window: [1, 2] gap: 2^-52 14330window: [2, 4] gap: 2^-51 14331window: [4, 8] gap: 2^-50 14332window: [8, 16] gap: 2^-49 14333 14334### why does `10000000000000000.0 + 1 = 10000000000000000.0`? 14335- In the window [2^n, 2^n+1], the gap between floats is 2^n-52 14336- `10000000000000000.0` is in the window [2^53, 2^54], where the gap is 2^1 (or 2) 14337- So the next float after `10000000000000000.0` is `10000000000000002.0`</div> 14338 </a> 14339 14340 <a href="/comics/floating-point-math/"> 14341 <span class="title"> 14342 floating point math 14343 </span> 14344 <div class="keywords"></div> 14345 <div class="transcript">## floating point math 14346 14347let's deconstruct `0.1 + 0.2` 14348 143491. O The closest 64-bit float to 0.1 is (roughly) `0.1000000000000000055511151231` 14350 143512. For 0.2, it's (roughly) `0.2000000000000000111022302462` 14352 143533. `0.1000000000000000055511151231 + 0.2000000000000000111022302462 = 0.3000000000000000166533453693` 14354 143554. Inconveniently, `0.3000000000000000166533453693` is exactly in between 2 floating point numbers: `0.2999999999999999888977` and `0.30000000000000004440892` 14356 143575. How do we pick the answer? `0.30000000000000004440892` has an even offset, so we round to that one 14358 14359## losing a little precision is okay 14360`0.1 0.2 0.30000000000000004` is usually no big deal. Do you REALLY need your answer to be accurate to 16 decimal places? Probably not! 14361 14362## the more numbers you add, the more precision you lose 14363This Go code: 14364 14365`var meters float32 = 0.0 ` 14366 14367`for i = 0; i < 100000000; i++ { meters += 0.01` 14368 14369`} fmt.Println(meters)` 14370 14371prints out `262144`, not `1000000` because `262144.0+ 0.1 = 262144.0` 14372 14373## adding a number to a MUCH smaller number is bad 14374For example: 14375 143762 xx 53 + 1.0 = 2 xx 53 14377 143781.0 + 2 xx -57 = 1.0 14379 14380(try it!) 14381 14382## Use scientific computing libraries if you can 14383There are special algorithms for adding up lots of small floating numbers without losing accuracy! 14384 14385For example `numpy` implements them.</div> 14386 </a> 14387
14388 <a href="/comics/floating-point-weird/"> 14389 <span class="title"> 14390 floating point is weird 14391 </span> 14392 <div class="keywords"></div> 14393 <div class="transcript">## floating point is weird 14394 14395## floating point 10.0 is not the same as the integer 10 1439610 (64-bit integer): `0x000000000000000a` 14397 1439810.0 (64-bit float): `0x4024000000000000` 14399 14400(what's this 4024 doing???) 14401 14402## computer integers work almost exactly the way you'd expect 14403 14404`1 + 2 - 3 = 0` 14405 14406but floating point numbers don't: 14407 14408` (0.1 + 0.2) - 0.3 = 0.0000000000000000555` 14409 14410## checking for float equality is dangerous 14411`if x == 0.3`: bad! 14412 14413`(0.1 + 0.2)` is not equal to `0.3`! 14414 14415Instead, check if x is very close to 0.3, something like this: 14416 14417`if abs(x 0.3) 0.0000001:` 14418 14419## in floating point, very large integers get rounded 14420For example: `10000000000000001.0 == 10000000000000000.0` 14421 14422(16 zeros) 14423 14424(try comparing those 2 numbers in your favourite language! they're the same!) 14425 14426 14427## (x + y) + z is not the same as x + (y + z) 14428For example: `(9007199254740992.0+ 1.0) 1.0 = 9007199254740991.0` 14429 14430(the math term for this problem is "floating point addition isn't associative") 14431 14432## some intuition for precision 1443332-bit floats have about 8 digits of precision 14434 1443564-bit floats have about 16 digits of precision</div> 14436 </a> 14437 14438 <a href="/comics/floating-point-alternatives/"> 14439 <span class="title"> 14440 floating point alternatives 14441 </span> 14442 <div class="keywords"></div> 14443 <div class="transcript">## more floating point alternatives 14444 14445## there are many alternative ways to represent numbers 14446These are all implemented in software (not hardware) so they're a lot slower, and different languages have different libraries. 14447 14448## alternative 1: decimal floating point 14449This is like regular floating point, but in base 10 instead of base 2. It's also standardized in IEEE 754. 14450 14451Examples: Python's `decimal` module or Java's `BigDecimal` 14452 14453## alternative 2: fractions 14454This lets you do exact calculations with fractions (1/10 + 2/10 = 3/10) 14455 14456Examples: Python's fractions module in the standard library, Lisps have first-class support 14457 14458## alternative 3: symbolic computation 14459For example, `sqrt(2)` instead of `1.414`. 14460 14461You'll see this in computer algebra systems like Mathematica, Maple, or sympy. 14462 14463## alternative 4: interval arithmetic 14464The idea is to store every number as a range so that you can precisely track your error bars. 14465 14466Probably the least mainstream of these alternatives. 14467 14468## alternative 5: binary-coded decimal 14469This is how floating point numbers (and integers) were stored on IBM computers in the 60s, and you can still occasionally see it today in old formats like ISO 8583 for financial transactions.</div> 14470 </a> 14471 14472 <a href="/comics/fixed-point/"> 14473 <span class="title"> 14474 fixed point 14475 </span> 14476 <div class="keywords"></div> 14477 <div class="transcript">## fixed point 14478 14479## just because you see 0.23, doesn't mean it's floating point 14480 14481For example, in this RGBA color: `rgba(211, 7, 23, 0.23)` 14482 14483`0.23` isn't a float at all, it's the 8-bit integer `59`. Let's see how that works! 14484 14485## fixed point numbers are integers 14486 14487You interpret them as the integer divided by some fixed number (like 255 or 10000) 14488 14489For example, that opacity should be divided by 255 14490 14491`59 / 255 = 0.23ish` 14492 14493## things fixed point is often used for 14494 14495money: `$1.23 => 123` 14496 14497time: `0.1 seconds => 100000 microseconds` 14498 14499opacity: `0.23 => 59` 14500 14501## fixed point is the most common alternative to floating point 14502 14503It's very simple and it's pretty easy to implement! 14504 14505## implementing fixed point is easy 14506 14507(especially if you only need to add and subtract) 14508 14509You just need: 14510- an integer 14511- some code to display it (by dividing by 255 or something) 14512 14513## fixed point can help avoid accuracy issues 14514 14515If you try to represent the current Unix epoch in nanoseconds as a 64-bit float, you'll lose accuracy. 14516 14517But if it's a 64-bit integer, it'll be fine.</div> 14518 </a> 14519 14520 <a href="/comics/bytes/">
14521 <span class="title"> 14522 bytes 14523 </span> 14524 <div class="keywords"></div> 14525 <div class="transcript"></div> 14526 </a> 14527 14528 <a href="/comics/bitwise-operations/"> 14529 <span class="title"> 14530 bitwise operations 14531 </span> 14532 <div class="keywords"></div> 14533 <div class="transcript">### bitwise operations operate one bit at a time 14534 14535The results can be surprising when you write them in base 10: 14536`8 & 3 = 0` 14537 14538but in binary it makes more sense: 14539``` 1454000001000 (8) 14541& 00000011 (3) 14542= 00000000 14543``` 14544### & 14545Bitwise and: the result is 1 if BOTH bits are 1 14546``` 145471 & 1 = 1 145481 & 0 = 0 145490 & 0 = 0 1455011 & 10 = 10 14551``` 14552 14553### | 14554Bitwise or: the result is 1 if EITHER bit is a 1 14555``` 145561 | 1 = 1 145571 | 0 = 1 145580 | 0 = 0 1455911 | 10 = 11 14560``` 14561 14562### ^ 14563Bitwise xor: the result is 1 if EXACTLY ONE bit is a 1 14564``` 145651 ^ 1 = 0 145661 ^ 0 = 1 145670 ^ 0 = 0 1456811 ^ 10 = 01 14569``` 14570 14571### ~ 14572Bitwise not: FLIP all the bits 14573``` 14574~0 = 1 14575~1 = 0 14576~10 = 01 14577``` 14578 14579### << 14580Left shift: add 0s to the end 14581 14582`1110 <<< 3 = 1110000` 14583 14584`<< n` is the same as multiplying by 2^n 14585 14586### >> 14587Right shift: chop bits off the end 1458801100001 >> 2 = 00011000 14589 14590`>> n` is the same as dividing by 2^n 14591 14592### there are actually two right shifts 14593 14594unsigned right shift 14595``` 14596253 >> 1 = 126 1459711111101 -> 01111110 14598``` 14599always pad on the left with a 0 14600 14601signed right shift 14602``` 14603-3 >> 1 = 2 1460411111101 -> 11111110 14605``` 14606if the number is negative, pad on the left with 1 instead of a 0 14607 14608In some languages, unsigned right shift is >>>. In other languages, both right shifts are >> and the integer's type determines which is used.</div> 14609 </a> 14610 14611 <a href="/comics/bit-flags/"> 14612 <span class="title"> 14613 bit flags 14614 </span> 14615 <div class="keywords"></div> 14616 <div class="transcript">## bit flags 14617 14618## bit flags are a clever way to store lots of information in one integer 14619 14620If you have many options which are true or false, you can encode them all into an integer, with 1 bit for each option. 32 bits 32 options! 14621 14622For example, some of the bit flags the open function in C uses: 14623- nofollow 14624- append 14625- truncate 14626- create 14627- write only 14628- read write 14629 14630(this is on Linux) 14631 14632 14633## where you'll see bit flags 14634In libc, the open, socket, and mmap functions use bit flags to pass options. 14635 14636The TCP and UDP protocol headers both have a flags field which has bit flags. 14637 14638## bit flags are used a lot in C code 14639 14640Here's some C code that opens a new file: 14641`fd = open("file.txt", O_RDWR | O_CREAT, 0666);` 14642 14643`O_RDWR` is: `00000010` 14644 14645`O_CREAT` is: `01000000` 14646 14647`O_RDWR | O_CREAT` is: `01000010` 14648 14649You can check if a bit flag is set in C like this: 14650 14651`if (flags & O_RDWR) { ... }` 14652 14653## fun example: tic tac toe! 14654Here's a way to encode the state of a tic tac toe game in 18 bits: 14655 14656x positions: 14657 14658`100` 14659 14660`010` 14661 14662`010` 14663 14664O positions: 14665 14666`010` 14667 14668`001` 14669 14670`100`</div> 14671 </a> 14672 14673 <a href="/comics/big-integers/"> 14674 <span class="title"> 14675 big integers 14676 </span> 14677 <div class="keywords"></div> 14678 <div class="transcript">## big integers 14679 14680## integers don't have to overflow 14681Instead, integers can expand to use more space as they get bigger. Integers that expand are called "big integers". 14682 14683big integer: I'm going to use ONE THOUSAND bytes of space! 14684 14685## big integer math is slower 14686It's slower because it's implemented in software, not hardware. 14687 14688So a big integer addition is actually turned into lots of smaller additions. 14689 14690## how big integers are represented (in Go, as of 2023) 14691You can think of this array of 64-bit integers as being the number written in base 2^64 14692 14693## some languages only have big integers 14694Python 3 and Ruby: we'd rather have slower math and no weird overflow problems! 14695 14696This works because people don't do a lot of math in Ruby/Python (except with numpy, which doesn't use big integers). 14697 14698## some languages offer big integers as an option 14699Go, Javascript, Java, and lots more. 14700Each language has its own big integer implementation. 14701 14702## when are big integers useful? 14703- they're used in cryptography (e.g. for large key sizes) 14704- for math on really big integers</div> 14705 </a> 14706 14707 <a href="/comics/bases/">
14708 <span class="title"> 14709 bases 14710 </span> 14711 <div class="keywords"></div> 14712 <div class="transcript">### We usually write numbers in base 10, but you can write numbers in any base. 14713 14714Let's write the number 103 in 3 different bases: 14715 14716base 10: `103` 14717(powers of 10) 14718``` 147191 x 100 = 100 147200 x 10 = 0 147213 x 1 = 3 14722= 103 14723``` 14724 14725base 2: `1100111` 14726(powers of 2) 14727``` 147281 x 64 = 64 147291 x 32 = 32 147300 x 16 = 16 147310 x 8 = 8 147321 x 4 = 4 147331 x 2 = 2 147341 x 1 = 1 1473564 + 32 + 16 + 8 + 4 + 2 + 1 = 103 14736``` 14737 14738base 3: `67` 14739(powers of 16) 14740``` 147416 x 16 = 96 147427 x 1 = 7 1474396 + 7 = 103 14744``` 14745 14746### base 2, 10, and 16 are the main bases we use on computers 14747- base 2 is called binary 14748- base 10 is called decimal 14749- base 16 is called hexadecimal 14750 14751### how to convert from base 10 to base 2 14752Let's convert 19! We'll start on the right and move left. 147531. Divide by 2: 19/2 = 9 remainder 1 147542. Write the remainder (1) below, and 9 on the left 147553. Repeat 14756 14757answer: 10011! 14758 14759person: but in real life I'd just ask a computer 14760</div> 14761 </a> 14762 14763 <a href="/comics/ascii/"> 14764 <span class="title"> 14765 ASCII 14766 </span> 14767 <div class="keywords"></div> 14768 <div class="transcript">## panel 1: a string is an array of bytes 14769 14770ASCII is the simplest string encoding: 1 character = 1 byte. Let's see how it works! 14771 14772(We usually use UTF-8, which is WAY more complicated) 14773 14774## panel 2: every printable ASCII character 14775 14776``` 14777!"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ\ 14778[]^_`abcdefghijklmnopqrstuvwxyz{|}~ 14779``` 14780 14781There are no accents because it's an English encoding: the "A" in ASCII is for "American". 14782 14783## panel 3: there are 128 ASCII characters 14784 14785Only the bytes 0 to 127 are defined. 14786 14787It's very limited: you can really see why we need more powerful encodings like UTF-8! 14788 14789## panel 4: how bytes map to characters 14790 14791Here's a partial list, look up "ASCII table" for the full list. Bytes (in base 10) are on the left, characters are on the right. 14792 1479333 is !, 34 is " 14794 1479548 is 0, 49 is 1 14796 1479764 is A, 65 is B 14798 1479997 is a, 98 is b 14800 14801## panel 5: a trick to translate from lowercase to uppercase 14802 14803In ASCII, the lowercase letters are 32 more than the uppercase letters. So you can just subtract 32!</div> 14804 </a> 14805 14806 <a href="/comics/many-meanings/"> 14807 <span class="title"> 14808 8 bytes, many meanings 14809 </span> 14810 <div class="keywords"></div> 14811 <div class="transcript">## 8 bytes, many meanings 14812 14813The same bytes can mean many things. Here are 8 bytes and a bunch of things they could potentially mean 14814 14815a picture of 8 bytes: the ASCII characters for 'computer' 14816 14817some things they could mean: 14818 14819* 8 8-bit integers 14820* 4 unsigned 16-bit integers 14821* a 64-bit pointer 14822* 2 IPv4 addresses 14823* x86 machine code 14824* 2 32-bit floating point numbers 14825* 1 64-bit floating point number 14826* 2 RGBA colours 14827 14828person: "don't worry if you don't understand all this right now! We'll explain. 14829 14830note on x86 machine code: this code is nonsense, but search "ascii shellcode" for x86 code which is valid ASCII.</div> 14831 </a> 14832 14833 <a href="/comics/32-bit-small/"> 14834 <span class="title"> 14835 32 bits is small 14836 </span> 14837 <div class="keywords"></div> 14838 <div class="transcript">## panel 1: using 32-bit integers is dangerous 14839 14840Let's see some examples of how it can go wrong and why it's almost always better to use 64-bit integers instead! 14841 14842(32-bit floats are bad too, for similar reasons) 14843 14844## panel 2: 32 bit integers are at most 4 billion 14845 14846unsigned 32-bit ints go from 0 to 4,294,967,295 (4 billion) 14847 14848signed 32-bit ints go from 14849-2,147,483,648 to 2,147,483,647 14850 14851## panel 3: times "4 billion" wasn't enough 14852 14853**Database primary keys**: 4 billion records really isn't that much. 14854 14855**IPv4 addresses**: turns out we want more than 4 billion computers on the internet. Oops. 14856 14857**Registers**: in the 90s, registers were 32 bits. 4 billion bytes of RAM is 4GB. We need more than that. 14858 14859**Unix timestamp**s: 2 billion seconds after Jan 1, 1970 is Jan 19, 2038. That's going to be an exciting day. (look up "2038 problem"!) 14860 14861## panel 4: 64 bits is usually big enough 14862 14863For example, 2^64 seconds after Jan 1, 1970 is over 100 billion years in the future: well after the death of the sun. 14864 14865So a 64-bit timestamp is definitely enough space. 14866 14867## panel 5: be wary of using 32-bit integers by accident 14868 14869Systems that were designed in the 90s often have a 32-bit integer as the default. 14870 14871For example, in MySQL an INTEGER is 32 bits.</div> 14872 </a> 14873 14874 <a href="/comics/path/">
14875 <span class="title"> 14876 PATH 14877 </span> 14878 <div class="keywords"></div> 14879 <div class="transcript"> 14880### PATH is how your shell knows where to find programs 14881It's a list of directories that your shell searches in order. 14882 14883smiling stick figure: `$ python3` 14884 14885shell, represented by a nautilus shell: 14886`PATH=/bin:/home/bork/bin:/usr/bin` 14887(directories are separated by colons) 14888 14889shell: 148901. `/bin/python3`? nope, doesn't exist 148912. `/home/bork/bin/python3`? nope, doesn't exist 148923. `usr/bin/python3`? there it is!!! run that! 14893 14894 14895### how to add a program to your `PATH` 14896 148971. find the directory the program is in 148982. update `PATH` in your config with that directory 148993. restart your shell 14900 14901for WAY TOO MUCH info about how to do this, see `https://wzrd.page/path` 14902 14903 14904### ...but which directory was the program installed in? 14905 14906remember how you installed it: 14907 14908little stick figure with curly hair, thinking: hmm, I used the Rust installer, where does that install things? 14909 14910... or do a brute force search: 14911`find / -name python3 | grep bin` 14912 14913(usually I put a `2>/dev/nu11` too) 14914 14915### `PATH` ordering drama 14916 14917little stick figure with curly hair, thinking: ugh, no, don't run THAT `python3`, run the other one! 14918 14919You can prioritize a directory by adding it to the *beginning* of your `PATH` 14920 14921 14922### gotcha: not everything uses your `PATH` 14923cron jobs usually have a very basic `PATH`, maybe just `/bin` and `/usr/bin` 14924 14925In a cron job I'll use the absolute path 14926`/home/bork/bin/someprogram`</div> 14927 </a> 14928 14929 <a href="/comics/some-people-who-make-programming-easier/"> 14930 <span class="title"> 14931 some people who make programming easier 14932 </span> 14933 <div class="keywords"></div> 14934 <div class="transcript">### the loud newbie 14935newbie: wait, HOW does X work?? 14936other person, thinking: I'm so glad they asked, I was wondering that too... 14937 14938### the grumpy old timer 14939new person: X is so cool! 14940grumpy old timer: it is! let me tell! you about some ways it can break though.... 14941 14942### the bug chronicler 14943that bug was so gnarly, I'm going to write an EXTREMELY CLEAR description of what happened so we I can all learn from it 14944 14945### the documentarian 14946person 1: here's how you do X... 14947documentarian: I'll put those instructions in our wiki! 14948 14949### the "today I learned..." 14950I just learned this cool new tool... 14951check out this weird bug! 14952 14953### the "I've read the entire internet" 14954person: how does X work? 14955TAB GIRL: ah, I read about that recently... here's a link from my 200 browser tabs 14956 14957### the tool builder 14958everyone keeps getting confused by X! I'm going to fix it with CODE. 14959 14960### the question answerer 14961person 1: hey can you explain how X works? 14962question answerer: I would LOVE to 14963 14964### blank final panel 14965?</div> 14966 </a> 14967 14968 <a href="/comics/strace-command-line-flags-i-love/"> 14969 <span class="title"> 14970 strace command line flags I love 14971 </span> 14972 <div class="keywords"></div> 14973 <div class="transcript">### -e 14974overwhelmed by all the system calls. you don't understand? Try 14975 14976`strace -e open` 14977 14978and it'll just show you opens. much simpler! 14979 14980### -f is for follow 14981Does your program start subprocesses! lots do! 14982 14983Use `-f` to see what those are doing too. Or just always use `-f`! That's what I do. 14984 14985### -p is for PID 14986"OH NO I STARTED THE PROGRAM 6 HOURS AGO AND NOW I WANT TO STRACE IT" 14987 14988Do not worry! Just find your process's PID (like 747) and 14989`strace -p 747` 14990 14991(tip: if the process runs as root you'll need to be root, too because SECURITY) 14992 14993 14994### -s is for strings!! 14995Sometimes I'm looking at the output of a recvfrom and it's like: 14996 14997recvfrom (6, "And then the monster...") 14998and OH NO THE SUSPENSE. 14999 15000`strace -s 800` will show you the first 800 characters of each string. I use it all the time! 15001 15002### -o is for output! 15003 15004Let's get real. No matter what, strace prints too much damn output. Use 15005`strace -o too_much_stuff.txt` and sort through it later. 15006 15007### -y 15008Have no idea which file the file descriptor "3" refers to? `-y` is a flag in newer versions of strace, and it'll show you filenames instead of just numbers! 15009 15010### Putting it all together: 15011 15012Want to spy on an ssh session? 15013 15014`strace -f -o ssh.txt ssh juliabox.com`
15015 15016Want to see what files a Dropbox sync process is opening? (with PID: 230) 15017 15018`strace -f -p230 -e open`</div> 15019 </a> 15020 15021 <a href="/comics/your-domains-authoritative-nameservers/"> 15022 <span class="title"> 15023 your domain's authoritative nameservers 15024 </span> 15025 <div class="keywords"></div> 15026 <div class="transcript">### when you register a domain, your registrar runs your authoritative nameservers by default 15027your registrar, represented by a box with a smiley face wearing a crown: I'm taking care of your DNS! 15028 15029You can change your nameservers in your registrar's control panel. 15030 15031 15032### LOTS of services can be your authoritative nameserver 15033your registrar: I can manage your DNS records! 15034AWS, also represented by a box with a smiley face wearing a crown: me too! 15035shopify, also also represented by a box with a smiley face wearing a crown: me three! 15036 15037Nonplussed stick figure with curly hair: ok chill I only need one of you to do it 15038 15039### how to find your domain's nameservers 15040``` 15041$ dig +short NS neopets.com 15042ns-42.awsdns-05.com. 15043ns-1191.awsdns-20.org. 15044``` 15045`neopets.com` is using AWS's nameservers right now 15046 15047### how to change your nameservers 150481. Copy your DNS records to the new nameservers (use dig to check that it worked) 2. On your registrar's website, update your nameservers 150493. Wait 48 hours 150504. Delete the old DNS records (to save your future self confusion) 15051 15052### why changing your nameservers is slow 15053registrar: here's the new nameserver for example.com! 15054.com nameserver, represented by a box with a smiley face, wearing a stack of three crowns: ok great, I've saved this record: `example.com NS newns.com 172800` 15055 15056updates are VERY SLOW because this TTL is 2 days 15057 15058### what can go wrong if you don't delete the old records 15059 15060Illustration of a nonplussed stick figure with curly hair. 15061 15062person: I'll go to $OLD_NAMESERVER to change my DNS records! 15063person: WHY doesn't it WORK?!?!? 15064person: oh right, I changed this domain's nameservers last year, oops!</div> 15065 </a> 15066 15067 <a href="/comics/git-cheat-sheet/"> 15068 <span class="title"> 15069 git cheat sheet 15070 </span> 15071 <div class="keywords"></div> 15072 <div class="transcript">Illustration of a smiling stick figure with short curly hair. 15073 15074Person: git has 17 million options but this is how I use it! 15075 15076### getting started 15077#### start a new repo: 15078`git init` 15079 15080#### clone an existing repo: 15081`git clone $URL` 15082 15083### know where you are 15084`git status` 15085 15086### prepare to commit 15087#### add untracked file: 15088(or unstaged changes) `git add $FILE` 15089 15090#### add ALL untracked files and unstaged changes: 15091`git add` 15092 15093#### choose which parts of a file to stage: 15094`git add -p` 15095 15096#### delete or move file: 15097``` 15098git rm $FILE 15099git mv $OLD $NEW 15100``` 15101 15102#### tell git to forget about a file without deleting it: 15103`git rmcached $FILE` 15104 15105#### unstage everything: 15106`git reset HEAD` 15107 15108### make commits 15109#### make a commit: 15110(and open a text editor to write the message) 15111`git commit` 15112 15113#### make a commit: 15114`git commit -m 'message'` 15115 15116#### commit all unstaged changes: 15117`git commit -am 'message'` 15118 15119### move between branches 15120#### switch branches: 15121`git switch $NAM`E OR `git checkout $NAME` 15122 15123#### create a branch: 15124`git switch -c $NAME` OR `git checkout -b $NAME` 15125 15126#### list branches: 15127`git branch` 15128 15129#### delete a branch 15130`git branch -d $NAME` 15131 15132#### force delete a branch: 15133`git branch -D $NAME` 15134 15135#### list branches by most recently committed to: 15136``` 15137git branch 15138--sort--committerdate 15139``` 15140### look at a branch's history 15141#### log the branch 15142`git log main` 15143 15144#### show how two branches relate to each other: 15145`git log-graph a b` 15146 15147#### one line log: 15148`git log-oneline` 15149 15150### code archaeology 15151 15152#### show who last changed each line of a file: 15153`git blame $FILENAME` 15154 15155#### show every commit that modified a file: 15156`git log $FILENAME` 15157 15158#### find every commit that added or removed some text: 15159`git log S banana` 15160 15161### diff commits 15162#### show diff between a commit and its parent: 15163`git show $COMMIT_ID` 15164 15165#### show diff between a merge commit and its merged parents: 15166`git show --remerge-diff $COMMIT_ID` 15167 15168#### diff two commits: 15169`git diff $COMMIT_ID $COMMIT_ID` 15170 15171#### just show diff for one file:
15172`git diff $COMMIT_ID $FILENAME` 15173 15174#### show a summary of a diff: 15175`git diff $COMMIT_ID --stat git show $COMMIT_ID --stat` 15176 15177### diff staged/unstaged changes 15178#### diff all staged and unstaged changes: 15179`git diff HEAD` 15180 15181#### diff just staged changes: 15182`git diff --staged` 15183 15184#### diff just unstaged changes: 15185`git diff` 15186 15187### configure git 15188#### set a config option: 15189`git config user.name 'Julia'` 15190 15191#### see all possible config options: 15192`man git-config` 15193 15194#### set option globally: 15195`git config --global ...` 15196 15197#### add an alias: 15198`git config alias.st status` 15199 15200### important git files 15201#### local git config: 15202`.git/config` 15203 15204#### global git config: 15205`~/.gitconfig` 15206 15207#### list of files to ignore: 15208`.gitignore` 15209 15210### combine diverged branches 15211#### how the branches look before: 15212Diagram of two boxes in a row, connected by lines. The first one has a heart, the second one has a star. Branching off from the star, there is one branch with a box with a hashtag symbol, labelled "main". The second branch consists of a box with a spiral and a box with a squiggle. The second branch is labelled "banana". 15213 15214#### combine with rebase: 15215``` 15216git switch banana 15217git rebase main 15218``` 15219 15220Diagram of two boxes in a row, connected by lines. The first one has a heart, the second one has a star. Branching off from the star, there is one branch with a box with a hashtag symbol, labelled "main". The box with the spiral and the box with the squiggle have been added on after the box with the hashtag. The box with the squiggle is labelled "banana". The second branch, with the box with a spiral and the box with a squiggle, are drawn with dotted lines and labelled "lost". 15221 15222#### combine with merge: 15223``` 15224git switch main 15225git merge banana 15226git commit 15227``` 15228 15229This diagram is like the "before" diagram, except now the two branches converge into a new box, with a diamond in it, labelled "main". 15230 15231#### combine with squash merge: 15232``` 15233git switch main 15234git merge git commit 15235squash banana 15236``` 15237 15238This diagram is like the "before" diagram, except now, in the first of the two branches, after the hashtag symbol, there is a new box with both a spiral and a squiggle in it, labelled "main". 15239 15240### bring a branch up to date with another branch 15241(aka "fast-forward merge") 15242main 15243banana 15244---0-0 15245``` 15246git switch main 15247git merge banana 15248``` 15249banana 15250---0-2 main 15251 15252### copy one commit onto another branch 15253before: 15254-K 15255â main 15256+banana 15257git cherry-pick $COMMIT_ID 15258after: 15259K 15260main 15261© 15262-banana 15263 15264### add a remote 15265`git remote add $NAME $URL` 15266 15267### push your changes 15268#### push the main branch to the remote origin: 15269`git push origin main` 15270 15271#### push a branch to the remote origin that you've never pushed before: 15272`git push u origin $NAME` 15273 15274#### push the current branch to its remote "tracking branch": 15275`git push` 15276 15277#### force push: 15278`git push --force-with-lease` 15279 15280#### push tags: 15281`git push --tags` 15282 15283### pull changes 15284#### fetch changes: 15285(but don't change any of your local branches) 15286`git fetch origin main` 15287 15288#### fetch changes and then merge them into your current branch: 15289`git pull origin main` OR `git pull` 15290 15291#### fetch changes and then rebase your current branch: 15292`git pull --rebase` 15293 15294#### fetch all branches: 15295`git fetch --all` 15296 15297### ways to refer to a commit 15298every time we say $COMMIT_ID, you can use any of these: 15299* a branch (`main`) 15300* a tag (`v0.1`) 15301* a commit ID (`3e887ab`) 15302* a remote branch (`origin/main`) 15303* current commit (`HEAD`) 15304* 3 commits ago (`HEAD^^^`) 15305* 3 commits ago (`HEAD~3`)</div> 15306 </a> 15307 15308 <a href="/comics/the-floating-point-number-line/"> 15309 <span class="title"> 15310 the floating point number line 15311 </span> 15312 <div class="keywords"></div> 15313 <div class="transcript">### the (64-bit) floating point number line 15314Floating point numbers aren't evenly distributed. Instead, they're organized into windows: [0.25, 0.5], [0.5, 1], [1,2], [2,4], [4,8], [8,16], all the way up to [2^1023, 2^1024]. 15315 15316Every window has 2^52 floats in it. 15317- between -2 and -1 15318- between -1 and - 1/2 15319- between - 1/2 and - 1/4 15320- between - 1/4 and 0 15321- between 0 and 1/4 15322- between 1/4 and 1/2 15323- between 1/2 and 1 15324- between 1 and 2 15325 15326### the windows go from REALLY small to REALLY big 15327The window closest to 0 is [2-1023 2-1022]. 15328 15329This is TINY: a hydrogen atom weighs about 2^-76 grams. 15330 15331The biggest window is [2^1023, 2^1024]. 15332 15333This is HUUUGE: the farthest galaxy we know about is about 2^90 meters away. 15334 15335### the gaps between floats double with every window 15336 15337- window: [1, 2] gap: 2^-52 15338- window: [2, 4] gap: 2^-51 15339- window: [4, 8] gap: 2^-50 15340- window: [8, 16] gap: 2^-49 15341 15342### why does `10000000000000000.0 + 1 = 10000000000000000.0?` 15343- In the window [2^n, 2^n+1], the gap between floats is 2^n-52 15344- `10000000000000000.0` is in the window [2^53, 2^54], where the gap is 2^1 (or 2) 15345- So the next float after `10000000000000000.0` is `10000000000000002.0`</div> 15346 </a> 15347 15348 <a href="/comics/scenes-from-distributed-systems/">
15349 <span class="title"> 15350 scenes from distributed systems 15351 </span> 15352 <div class="keywords"></div> 15353 <div class="transcript"></div> 15354 </a> 15355 15356 <a href="/comics/git-references/"> 15357 <span class="title"> 15358 git references 15359 </span> 15360 <div class="keywords"></div> 15361 <div class="transcript">### git often uses the term "reference" in error messages 15362``` 15363$ git switch asdf 15364fatal: invalid reference: asdf 15365 15366$ git push 15367To github.com:jvns/int-exposed 15368 ! [rejected] main -> main 15369error: failed to push some refs to 'github.com:jvns/int-exposed' 15370``` 15371 15372"ref" and "reference" mean the same thing 15373 15374Illustration of a tiny worried-looking stick person with a thought bubble reading "!" 15375 15376### "reference" often just means "branch" 15377 15378in those two error messages, you can replace "reference" with "branch" 15379 15380in my experience, it's: 1538196% "branch" 153823% "tag" 153833% "HEAD" 153840.01% something else 15385 15386### it's an umbrella term 15387 15388Illustration of git, represented by a box with a smiley face 15389 15390git, thinking: "well, I COULD check if the thing we failed to push is a branch or tag or what, and customize the error message based on that...." 15391git, thinking: "seems complicated, let's just print out "reference"" 15392 15393sad person: "why?" 15394 15395### reference: the definition 15396 15397References are files: either `.git/HEAD` or files in `.git/refs`. There are 5 main types. 15398 15399Here's a list of every type of git reference that I have ever used: 15400 15401- HEAD: `.git/HEAD` 15402- branches: `.git/refs/heads/BRANCH` 15403- tags: `.git/refs/tags/TAG` 15404- remote-tracking branches: `.git/refs/remotes/REMOTE/BRANCH` 15405- stash: `.git/refs/stash` 15406 15407all of these files contain a commit ID, but the way that commit ID is used depends on what type of reference it is 15408 15409(examples of more obscure references are `.git/FETCH_HEAD` and `.git/refs/notes/...` but I've never needed to think about those and your repository probably doesn't even have notes) 15410 15411### git's garbage collection starts with references 15412the algorithm is: 154131. find all references, and every commit in every reference's reflog 154142. find every commit in the history of any of those commits 154153. delete every commit that wasn't found</div> 15416 </a> 15417 15418 <a href="/comics/knowing-where-you-are-in-git/"> 15419 <span class="title"> 15420 knowing where you are in git 15421 </span> 15422 <div class="keywords"></div> 15423 <div class="transcript">### many git disasters are caused by accidentally running a command while on the wrong branch... 15424 15425Illustration of a stick figure with a neutral expression. 15426 15427person: `git commit` 15428 15429person, thinking: UGH I didn't mean to do that on `main` 15430 15431### ... or by forgetting you're in the middle of a multistep operation 15432 15433smiling stick figure with curly hair: la la la just writing code 15434 15435same person, now distressed and surrounded by exclamation marks: OMG I FORGOT I WAS IN THE MIDDLE OF A MERGE CONFLICT 15436 15437### I always keep track of 2 things 15438 154391. am I on a branch, or am I in detached `HEAD` state? 154402. am I in the middle of some kind of multistep operation? (`rebase`, `merge`, `bisect`, etc) 15441 15442### I keep my current branch in my shell prompt 15443 15444`~/work/homepage (main) $` 15445 15446to me it's as important as knowing what directory I'm in 15447 15448git comes with a script to do this in bash/zsh called `git-prompt.sh` 15449 15450### decoder ring for the default git shell prompt 15451 15452`(main)` 15453 15454on a branch, everything is normal 15455 15456`((2e832b3...))` 15457`((v1.0.13))` 15458 15459the double brackets (( )) mean `detached HEAD state`. this prompt can only happen if you explicitly `git checkout` a commit/tag/remote-tracking branch 15460 15461`(main|CHERRY-PICK)` 15462`(main|REBASE 1/1)` 15463`(main|MERGING)` 15464`(main|BISECTING)` 15465 15466in the middle of a cherry-pick/rebase/merge/bisect</div> 15467 </a> 15468 15469 <a href="/comics/learning-on-my-own/"> 15470 <span class="title"> 15471 learning on my own 15472 </span> 15473 <div class="keywords"></div> 15474 <div class="transcript"></div> 15475 </a> 15476 15477 <a href="/comics/submodules/">
15478 <span class="title"> 15479 submodules 15480 </span> 15481 <div class="keywords"></div> 15482 <div class="transcript">### panel 1 15483 15484Illustration of a smiling stick figure with curly hair. 15485 15486person: I find submodules confusing and I avoid them if possible, but here's what I've learned from other people's writing on submodules 15487 15488(especially Dmitry Mazin's great "Demystifying Git Submodules" post) 15489 15490### submodules let you store another git repository as a subdirectory 15491``` 15492git submodule add 15493 https://github.com/jvns/myrepo 15494 ./myrepo 15495``` 15496(`jvns` is the remote, `myrepo` is the local path) 15497 15498Git will store the commit ID and URL of the submodule 15499 15500### gotcha: cloning a repository doesn't download its submodules 15501 15502To get the submodules, you can run this after cloning the repository: 15503 15504`git submodule update --init` 15505 15506### gotcha: git pull and git checkout don't update submodules gotcha: git pull and git checkout don't update submodules 15507 15508To actually update them, you have to run: 15509 15510`git submodule update` 15511 15512every single time you switch branches or pull 15513 15514### gotcha: git submodule update puts the submodule in detached HEAD state 15515 15516might not be a big deal if you're only using the submodule in a read-only way, but seems like it could get weird if you're editing it 15517 15518### some submodule config options 15519 15520automatically update submodules after a pull/checkout: 15521 15522`submodule.recurse true` 15523 15524show which commits were added/removed in `git diff/git status`: 15525``` 15526status.submoduleSummary true 15527diff.submodule log 15528```</div> 15529 </a> 15530 15531 <a href="/comics/merge-conflict-tips/"> 15532 <span class="title"> 15533 merge conflict tips 15534 </span> 15535 <div class="keywords"></div> 15536 <div class="transcript">### use `diff3` or `zdiff3` to see the original version of the code 15537 15538`git config --global merge.conflictstyle diff3` 15539 15540This will add an extra section in the middle of your merge conflicts 15541 15542### if you get confused, merge (or cherry-pick) 1 commit at a time 15543 15544This can make the conflicts smaller and easier to resolve! 15545 15546`git-imerge` is a tool to make this easier, though I haven't tried it 15547 15548### use rerere to remember how you resolved a conflict during a rebase 15549 15550`git config --global rerere.enabled true` 15551 15552This means you won't have to resolve the exact same conflict over and over again 15553 15554### `git checkout --ours/theirs` can take all changes from one side 15555 15556For example 15557 15558`git checkout --ours file.txt` 15559 15560will take the version of file.txt from the "ours" side of the merge 15561 15562(though upsettingly the meaning of "ours" and "theirs" depends on whether you merged or rebased) 15563 15564### if you can't tell which code comes from which branch, looking on the web can help 15565 15566Illustration of an uncertain-looking stick figure with short curly hair. 15567 15568person (thinking): I'll just go to GitLab and see what `file.txt` looks like on the main branch 15569 15570### `git merge-tree` can check for merge conflicts without actually merging the branches 15571``` 15572$ git merge-tree --write-tree 15573 main mybranch 15574... 15575Auto-merging file.py 15576CONFLICT (content): 15577Merge conflict in file.py 15578```</div> 15579 </a> 15580 15581 <a href="/comics/interactive-rebase/"> 15582 <span class="title"> 15583 interactive rebase 15584 </span> 15585 <div class="keywords"></div> 15586 <div class="transcript">### git rebase -i lets you garden your commits 15587 15588I use it like this: 155891. make commits chaotically, `git commit -am 'wip'` 155902. clean up with `git rebase -i` before sending them off for code review 15591 15592### interactive rebase's UI is a text file 15593 15594when you run `git rebase -i main`, it'll open a text editor with something like this in it: 15595``` 15596pick 399990 add some padding 15597pick fb59d8 french translation 15598pick 617b19 sort titles 15599pick 31b81f hashchange 15600``` 15601### deleting commits 15602 15603You can delete a commit just by deleting that line in the text editor! 15604 15605(same as previous panel but the "french translation" line is scribbled out) 15606 15607### combine commits with fixup 15608 15609Here's how to combine all 4 commits into 1 commit: 15610(`f` stands for `fixup`) 15611``` 15612pick 399990 add some padding 15613f fb59d8 french translation 15614f 617b19 sort titles 15615f 31b81f hashchange 15616``` 15617### check that the tests pass with exec 15618 15619You can run make test on every intermediate commit to make sure your tests pass like this: 15620``` 15621git rebase -i --exec 15622"make test" main 15623``` 15624(you can also use this to format every commit's code!) 15625 15626### some other tips 15627 15628* `reword` lets you edit a commit message 15629* If something goes very wrong, I try to run `git rebase --abort` ASAP, because undoing rebases is annoying</div> 15630 </a> 15631 15632 <a href="/comics/git-worktree/">
15633 <span class="title"> 15634 git worktree 15635 </span> 15636 <div class="keywords"></div> 15637 <div class="transcript">### git worktree lets you have 2 branches checked out at the same time 15638 15639Illustration of a smiling stick figure with curly hair, and a git worktree, represented by a box with a smiley face 15640 15641person: ugh, I want to take a look at this other branch, but I have all these uncommitted changes... 15642git worktree: i can help! 15643 15644### creating a worktree 15645 15646You can check out a branch into a new directory like this: 15647 15648`git worktree add ~/my/repo mybranch` 15649 15650(`my` is the directory, `mybranch` is the branch) 15651 15652Then you can run any normal git commands in the new directory: 15653``` 15654$ cd ~/my/repo 15655$ git pull 15656``` 15657### two worktrees cant have the same branch checked out 15658 15659Here's what happens if you try: 15660``` 15661$ git checkout main 15662fatal: main is already checked out at /home/bork/work/homepage 15663``` 15664### it's way faster (and uses less space!) than cloning the repository again 15665 15666Because worktrees share a .git directory, it just needs to check out the files from the branch you want to use! 15667 15668### other worktree commands 15669 15670List all worktrees: 15671 15672`$ git worktree list` 15673 15674Delete a worktree: 15675 15676`$ git worktree remove ~/my/repo` 15677 15678### sometimes I use worktrees to keep my .git directory and its checkout separate 15679 15680this lets me put the checkout in Dropbox but not the .git directory: 15681``` 15682$ git clone --bare [email protected]:jvns/myrepo 15683$ cd myrepo.git 15684$ git worktree add ~/Dropbox/myrepo main 15685``` 15686(`Dropbox` is the directory, `main` is the branch)</div> 15687 </a> 15688 15689 <a href="/comics/git-add-p/"> 15690 <span class="title"> 15691 git add -p 15692 </span> 15693 <div class="keywords"></div> 15694 <div class="transcript">### `git add -p` lets you stage some changes and not others 15695 15696I use this if I want to commit my real changes, but not the random debugging code I added. 15697 15698(this is one of the tasks GUIs and IDEs are best at, but I always use `git add -p` anyway) 15699 15700### what the interface looks like 15701``` 15702--- a/package.json 15703+++ b/package.json 15704@@ -1,7 +1,7 @@ 15705 "name": "homepage", 15706- "version": "1.0.0", 15707+ "version": "1.0.1", 15708 "devDependencies": { 15709- "dart-sass": "^1.25.0" 15710+ "dart-sass": "^1.26.0", 15711(1/1) Stage this hunk 15712[y,n,q,a,d, s,e,?]? 15713``` 15714package.json is the filename 15715lines 4-9 are the diff 15716`[y,n,q,a,d, s,e,?]` is your choice 15717### y(es)/n(o)/q(uit) 15718 15719y means "stage this change" 15720 15721n means "don't" 15722 15723q quits, keeping what you did so far. pretty straightforward. 15724 15725### how to check your work 15726 15727`git diff --cached` 15728 15729will show your staged changes 15730 15731 15732### s: split into two parts 15733s will split a diff into smaller diffs you can say y or n to individually, like this: 15734``` 15735+++ b/package.json 15736@@ -1,7 +1,7 @@ 15737- "version": "1.0.0", 15738+ "version": "1.0.1", 15739 "devDependencies": { 15740``` 15741BUT! This only works if there's a newline between the two parts. 15742 15743### how to split a diff if there's no newline 15744You can use the e ("edit") option to edit the diff manually: 15745- to remove a - line, replace "-" with a space 15746- to remove a + line, delete the whole line 15747 15748version 1: 15749``` 15750"name": "homepage", 15751- "version": "1.0.0", 15752- "devDependencies": { "version": "1.0.1", 15753+ "devDependenciezzz' 15754``` 15755 15756version 2: 15757``` 15758"name": "homepage", 15759- "version": "1.0.0", 15760+ "version": "1.0.1", 15761[space] "devDependencies": 15762[space] 15763``` 15764 15765(or you can just say 'n' and edit your code! that's what I do!)</div> 15766 </a> 15767 15768 <a href="/comics/folder-gotchas/"> 15769 <span class="title"> 15770 folder gotchas 15771 </span> 15772 <div class="keywords"></div> 15773 <div class="transcript">## panel 1: `ls ..` and `cd ..` refer to different folders if you `cd` to a symlinked folder 15774 15775`~/Dropbox -> ~/Library/CloudStorage/Dropbox` 15776 15777 15778``` 15779cd ~ 15780cd Dropbox 15781ls .. 15782cd .. 15783``` 15784 15785 15786* `ls ..` lists `~/Library/CloudStorage` 15787* `cd ..` moves to `~` 15788 15789 15790this is because `ls` is a program and `cd` is run by the shell. 15791 15792The shell handles `..` differently from other programs. 15793 15794 15795## panel 2: `ls ~/Dropbox` will list the contents of the folder 15796 15797this is annoying if you just want to look at its permissions, or where it links to 15798 15799 15800to fix this: 15801 15802``` 15803ls -d ~/Dropbox 15804``` 15805 15806## panel 3: deleting a folder and recreating it with the exact same name makes everything weird 15807 15808 15809everything you do in the folder will fail with weird errors like: 15810 15811``` 15812$ touch newfile 15813touch: newfile: no such file 15814or directory 15815``` 15816 15817how to fix it: 15818 15819``` 15820cd . 15821``` 15822 15823 15824## panel 4: on Mac OS, these are not the same: `cp -R a/ b` and `cp -R a b` 15825 15826 15827* `cp -R a/ b` merges the contents of `a` into `b` 15828* `cp -R a b` copies the whole folder into `b/a` 15829 15830 15831 15832## panel 4: tip: `cd -` switches to the folder you were previously in 15833 15834 15835## panel 5: notes on `mv file.txt dest` 15836 15837 15838* if `dest` is a file: renames `file.txt` 15839* if `dest` is a folder: moves `file.txt` to that folder 15840</div> 15841 </a> 15842 15843 <a href="/comics/oh-shit-committed-to-main/">
15844 <span class="title"> 15845 oh shit! I committed something to main that should have been on a brand new branch! 15846 </span> 15847 <div class="keywords"></div> 15848 <div class="transcript">1. Make sure you have main checked out: 15849`git checkout main` 15850 158512. Create the new branch: 15852`git branch my-new-branch` 15853 158543. Remove the unwanted commit from main: 15855``` 15856git status 15857git reset --hard HEAD~ 15858``` 15859(careful!) 15860 158614. Check out the new branch! 15862`git checkout my-new-branch` 15863 15864 15865Smiling stick figure with medium length straight hair: `git branch` and `git checkout -b` both create a new branch. The difference is `git checkout -b` also checks out the branch</div> 15866 </a> 15867 15868 <a href="/comics/whats-http/"> 15869 <span class="title"> 15870 what's HTTP? 15871 </span> 15872 <div class="keywords"></div> 15873 <div class="transcript">HTTP is the protocol (Hypertext Transfer Protocol) that's used when you visit any website in your browser. 15874 15875Firefox, to server: HTTP request - cat picture please 15876 15877server, to Firefox: HTTP response - cat.gif 15878 15879The exciting thing about HTTP is that even though it's used for literally every website, HTTP requests and responses are easy to look at and understand: 15880 15881server: here's an HTTP response! 15882 15883person: that response has the wrong Content-type header, that's why the website isn't working! 15884 15885Example of what an HTTP request and response might look like: 15886 15887### request 15888 15889request line: 15890`GET / HTTP/I .1 .1` 15891 15892headers: 15893``` 15894Host: examplecat.com 15895User-Agent: curl 15896Accept: */* 15897``` 15898### response 15899 15900status: 15901`HTTP/I .1 .1 200 0K` 15902 15903headers: 15904``` 15905Cache-Control: max-age=604800 15906Content-Type: text/ html 15907Etag: "1541025663+ident" 15908Server: ECS (nyb/1D0B) 15909Vary: Accept-Encoding 15910X-Cache: HIT 15911Content-Length: 1270 15912``` 15913 15914body: 15915``` 15916<!doctype html> 15917<title>Examp1e Cat</title> 15918... 15919``` 15920 15921that text is a lot to understand, so let's get started learning what of it means!</div> 15922 </a> 15923 15924 <a href="/comics/types-of-terminal-programs/"> 15925 <span class="title"> 15926 types of terminal programs 15927 </span> 15928 <div class="keywords"></div> 15929 <div class="transcript">### knowing what type of program you're in really helps 15930 15931stick person with curly hair, thinking: why doesn't `Ctrl+C` quit?? Oh, I'm in a REPL, I should use `CTRL+D` instead. 15932 15933 15934### 1. REPLs 15935(`sqlite`, `ipython`, `bash`) 15936 15937- you can probably use basic `readline` shortcuts to edit text 15938- `Ctrl+D` usually quits 15939- REPL stands for Read code, Evaluate it, Print the output, Loop (repeat) 15940 15941### 2. full screen programs 15942(`top`, `ncdu`) 15943 15944- `q` might quit 15945- `?` might open the help 15946- gotcha: if mouse reporting is on, you can't select text without pressing `Shift` 15947 15948### 3. noninteractive programs 15949(`grep`, `find`) 15950 15951- `Ctrl+C` usually quits 15952- gotcha: you can get "stuck" waiting for input on stdin if you forget to specify an input (like if you run `cat` by itself) 15953 15954### programs that play by their own rules 15955`vim` doesn't act like any other program 15956 15957usually I avoid these unless (like with `vim`) I've made a special effort to learn them. 15958 15959### `CTRL+C` doesn't always quit 15960 15961REPLs and full-screen programs often use `CTRL+C` to man "stop the current operation" instead of "quit the program"</div> 15962 </a> 15963 15964 <a href="/comics/the-terminal-cast-of-characters/"> 15965 <span class="title"> 15966 the terminal: cast of characters 15967 </span> 15968 <div class="keywords"></div> 15969 <div class="transcript">The "terminal" is actually a bunch of components that work together. 15970Let's imagine that you're running `python3 blah.py`. 15971 15972Illustration of a flow chart. It begins with a smiling stick figure with curly hair, labelled "you". Arrows leading away from "you" are labelled "keyboard shortcuts", "type", and "click". 15973 15974The arrows lead to a little character with a winky cursor face, labelled "terminal emulator: xterm, iTerm, GNOME terminal". 15975 15976In the middle of the diagram are two boxes, PTMX, and TTY. Between the two of them is the OS terminal driver ("Linux, Mac OS"). 15977 15978There are arrows labelled "bytes" going between the terminal emulator and the OS terminal driver, and between the OS terminal driver and the programs on the far right. 15979 15980The programs ("cat, vim, top, bash") are represented by two box with smiley faces, labelled "shell" and "python". 15981 15982### the terminal emulator 15983your terminal emulator is a translator: 15984 15985- it translates all your typing/clicks into bytes 15986- and it takes all the bytes the program sends and displays them on the screen 15987 15988### the terminal driver 15989the terminal driver is part of your operating system. It's in charge of sending signals to Python when you press `Ctrl+C`, and some other more obscure things. 15990 15991(more on page 24) 15992 15993### the shell 15994the shell is a special program which you use to start all other programs 15995 15996The shell doesn't do much after a program has started. Programs get a copy of the shell's current directory, environment variables, and input/outputs, etc and then they're on their own.</div> 15997 </a> 15998 15999 <a href="/comics/escape-codes/">
16000 <span class="title"> 16001 terminal escape codes 16002 </span> 16003 <div class="keywords"></div> 16004 <div class="transcript">### a program's input and outputs are streams of bytes 16005 16006everything you type goes into standard input (almost) 16007 16008Illustration of a program, represented by a big box with a smiley face. There is an "in" arrow going into it, and "1 out" and "2 ERR" arrows coming out of it. 16009 16010all the output you see comes from either standard output or standard error 16011 16012your terminal emulator can only communicate with programs by reading/writing bytes 16013 16014### some inputs/outputs are text and some are special instructions 16015 16016in: mouse position, ctrl+left arrow 16017out: make text green, make cursor invisible 16018 16019### these special instructions are called "escape codes" 16020 16021they're called "escape codes" because they all start with the ESC character 16022 16023five ways people print out `ESC`: 16024- `\033` 16025- `^[` 16026- `ESC` 16027- `\e` 16028- `\x1b` 16029 16030### example: how colours get set 16031 16032program: â`^[[31m`â 16033 16034Terminal emulator, represented by a box wth little arms and legs and a cute cursor winking face: âok, I'll make text red from now on!!â 16035 16036(there are also codes for bold, underline, background colour) 16037 16038### programs can easily "break" your terminal by printing escape codes 16039 16040program, represented by a frowning rectangle: âoops I made your cursor disappearâ 16041 16042It's easy to fix though, run `reset` to print a special escape code that resets everything</div> 16043 </a> 16044 16045 <a href="/comics/stdin-stdout-stderr/"> 16046 <span class="title"> 16047 stdin, stdout, stderr 16048 </span> 16049 <div class="keywords"></div> 16050 <div class="transcript">### all terminal programs have 1 input and 2 outputs 16051 16052they're numbered: stdin is "0", stdout is "1", stderr is "2". 16053 16054Illustration of a program, represented by a box with a smiley face. There is an arrow labelled "0 IN" going into it, and arrows labelled "1 OUT" and "2 ERR" coming out of it. 16055 16056(the numbers are called "file descriptors") 16057 16058### 3 things you can set the inputs/outputs to 16059 160601. the TTY (so output is displayed in your terminal emulator) 160612. a file 160623. a pipe (to write output to the input of another program) 16063 16064### your shell is in charge of setting up stdin/stdout/stderr 16065 16066tiny smiling tick figure: âpython3 script.py > out.txtâ 16067 16068shell: âok, I'll set stdout to out.txt for that programâ 16069 16070### when you redirect, the shell opens the file *before* the program starts 16071 16072`sudo echo blah > file.txt` 16073 16074shell, thinking: first i'll open `file.txt`... THEN I'll run `sudo echo blah` 16075 16076this is why `file.txt` isn't opened as root! 16077 16078### on 2>&1 16079 160802>&1 redirects stderr to stdout 16081 16082The same illustration from the first panel, but with the addition of an arrow coming out from "2 ERR" and going into "1 OUT". 16083 16084you could also do `echo "oops" 1>&2`if you want to write a message to stderr in a script. 16085 16086### gotcha: programs often buffer stdout but not stderr 16087 16088when a program writes text to stdout, it'll often 16089 160901. check if stdout is a TTY (using the `isatty` function) 160912. if not, "buffer" the writes until there's `1KB` of data to write, for performance reasons 16092 16093(this is the default in libc)</div> 16094 </a> 16095 16096 <a href="/comics/shell-history/"> 16097 <span class="title"> 16098 shell history 16099 </span> 16100 <div class="keywords"></div> 16101 <div class="transcript">### your shell has a history of the commands you ran 16102 16103some ways to access history: 16104 16105* press the up arrow 16106* run `history` 16107* search it with `Ctrl-R` (in `bash/zsh`) 16108* use `!33` to rerun lin 35 from `history` (bash/zsh) 16109 16110### how long does your shell store history for? 16111 16112(sad face) in bash, the default is 16113500 commands (not enough!) 16114 16115(happy face) in fish, the default is 16116256,000 commands 16117 16118if you're using bash, you might want to set `HISTSIZE` and `HISTFILESIZE` to stor more history 16119 16120in zsh, it's `HISTSIZE` and `SAVEHIST` 16121 16122### when does your shell save history? 16123 16124by default, bash and zsh only save history to a file when you exit the shell 16125 16126fish saves the history continuously 16127 16128### where is history stored? 16129 16130bash: `.bash_history` 16131 16132zsh: run `echo $HISTFILE` 16133 16134fish: mine is in `~/.local/share/fish/fish_history` 16135 16136smiling stick figure with curly hair (thinking): âsometimes I copy over my shell history when setting up a new computer!â 16137 16138### `history` doesn't include everything 16139 16140usually it includes: 16141- the contents of the history file when the shell *started* 16142- the commands you ran in this shell session 16143 16144if I want to use the history from another terminal tab, I'll open a new tab 16145 16146### a useful history tool: atuin 16147 16148atuin lets you: 16149 16150* save unlimited history 16151* search history more easily 16152* save commands as soon as you run them
16153* sync your history (optionally) 16154</div> 16155 </a> 16156 16157 <a href="/comics/quitting-in-the-terminal/"> 16158 <span class="title"> 16159 quitting in the terminal 16160 </span> 16161 <div class="keywords"></div> 16162 <div class="transcript">### quitting a terminal program isn't always easy 16163 16164Illustration of a stick figure with short curly hair. They look distressed and have an exclamation mark above their head. 16165 16166person (thinking): "I pressed `Ctrl-C` 17 times and NOTHING HAPPENED" 16167 16168### ways to quit 16169 16170- `Ctrl-C` - the default 16171- `Ctrl-D` - if you're at a prompt in a `REPL >>>` 16172- `q` - if it's a full screen program 16173- `Ctrl-\` - sometimes works if `Ctrl-C` doesn't 16174- `kill -9` - the last resort 16175 16176### how `Ctrl-D` works 16177 16178programs that read input will usually have some code like this: 16179``` 16180text = read_line() 16181if (text == EOF) { 16182 exit() 16183} 16184``` 16185`Ctrl-D` is how you send an EOF to the program ("I'm done!") 16186 16187important: `Ctrl-D` ONLY works if you press it on an empty line 16188 16189### how `Ctrl-C` works * 16190 16191`*` unless your program is in "raw mode", we'll talk about that later 16192 16193person, smiling: "`ctrl-C`" 16194 16195terminal emulator, represented by a box with a dollar sign: "ok, C is the 3rd letter of the alphabet, I'll write 3 to the tty" 16196 16197OS terminal driver, represented by a box labelled "OS": ah, a 3, that means I should send the `SIGINT` signal to the current program 16198 16199program, represented by a box with a smiley face: ooh, a `SIGINT`, I will [shutdown gracefully, immediately exit, ignore it, stop a subtask, etc] 16200 16201`*` unless your program is in "raw mode", we'll talk about that later 16202 16203### some programs have weird quitting incantations 16204 16205for example every text editor (vim, nano, emacs, etc) has its own completely unique way to quit</div> 16206 </a> 16207 16208 <a href="/comics/path-tips/"> 16209 <span class="title"> 16210 PATH tips 16211 </span> 16212 <div class="keywords"></div> 16213 <div class="transcript">### add a directory to your PATH 16214 16215at the end: 16216 16217`export PATH=$PATH:/my/dir` 16218 16219at the beginning: 16220 16221`export PATH=/my/dir/:$PATH` 16222 16223in fish: 16224 16225`set -e PATH $PATH /my/dir` 16226 16227(illustration of a little fish with a heart-shaped tail) 16228 16229### you shell's config file 16230 16231bash: `.bashrc or .bash_profile` (exactly which one is a bit of a rabbit hole sadly) 16232 16233zsh: `~/.zshrc` 16234 16235fish: `~/.config/fish/config.fish` 16236 16237(illustration of a little fish with a heart-shaped tail) 16238 16239### show what your shell is actually going to do when you run the program 16240 16241`type python3` 16242 16243instead of running what's in `PATH`, sometimes it'll run a builtin or alias or cached entry 16244 16245### show the first match on your PATH for a program 16246 16247`which python3` 16248 16249(but in zsh `which` acts like `type`) 16250 16251### show ALL matches on your PATH for a program, in order 16252 16253`which -a python3` 16254 16255### look at your PATH 16256 16257`echo $PATH` 16258 16259### show each entry on its own line 16260 16261`echo $PATH | tr ':' '\n'` 16262 16263### clear the PATH cache (bash/zsh) 16264 16265`hash -r` 16266 16267why you might need to do this: bash and zsh cache `PATH` lookups, so sometimes updating your `PATH` doesn't work properly</div> 16268 </a> 16269 16270 <a href="/comics/path-shell/"> 16271 <span class="title"> 16272 PATH and finding programs 16273 </span> 16274 <div class="keywords"></div> 16275 <div class="transcript">### PATH is how your shell knows where to find programs 16276 16277Illustration of a smiling stick figure with curly hair, and shell, represented by a box with a smiley face. 16278 16279person: run `python3` 16280 16281PATH is 16282``` 16283/bin 16284/home/bork/bin 16285/usr/bin 16286``` 16287shell, thinking: 16288 16289`/bin/python3`? nope, doesn't exist 16290`/home/bork/bin/python3`? nope, doesn't exist 16291`/usr/bin/python3`? there it is!!! I'll run that! 16292 16293### how to add a program to your PATH 16294 162951. find the folder the programs is in 16296 162972. update your shell config to add it to your `PATH` 16298 162993. restart your shell, for example by opening a new terminal tab 16300 16301### ...but how do you find the folder 16302 16303* think about how you installed it 16304 16305person (thinking): hmm, I used the Rust installer, where does that install things? 16306 16307* a brute force search 16308 16309`find / -name python3 | grep bin` 16310 16311### `PATH` ordering drama 16312 16313person (thinking): ugh, no, don't run THAT `python3`, run the other one! 16314 16315You can prioritize a folder by adding it to the beginning of your `PATH` 16316 16317### gotcha: not everything uses your shell's `PATH` 16318 16319cron jobs usually have a very basic `PATH`, maybe just `/bin` and `/usr/bin` 16320 16321In a cron job I'll use the absolute path, like: 16322 16323`/home/bork/bin/someprogram`</div> 16324 </a> 16325 16326 <a href="/comics/more-filename-tips/">
16327 <span class="title"> 16328 more filename tips (in the shell) 16329 </span> 16330 <div class="keywords"></div> 16331 <div class="transcript">### handle filenames starting with a dash with `--` or `./` 16332 16333`mv -- -file.txt dest` 16334 16335`mv ./-file.txt dest` 16336 16337(otherwise `mv` thinks `-file.txt` is an invalid option) 16338 16339### match all filenames ending in `.png` 16340 16341`rm *.png` 16342 16343(`*.png` is called a "glob" and it's handled by the shell so you can use it with any program!) 16344 16345### match `.png` files in any subdirectory 16346 16347`rm **/*.png` 16348 16349(only works in zsh/fish) 16350 16351### match filenames starting with a dot 16352 16353ls .* 16354 16355(these aren't included in * by default) 16356 16357### * gotcha: regular expressions 16358 16359if you want to pass a regexp with a * to `grep`: 16360 16361`grep '\s*test' file.txt` 16362 16363you need to quote it otherwise it will be treated as a glob 16364 16365### you can drag files from your GUI file manager to escape the filename 16366 16367This only works if your terminal emulator supports it. 16368 16369### GNU ls will quote filenames with spaces in the name 16370``` 16371$ ls 16372"julia's file.txt" 16373``` 16374(properly quoted!) 16375 16376you can check if you have this feature by running: 16377 16378`ls --quoting-style=shell`</div> 16379 </a> 16380 16381 <a href="/comics/meet-the-program/"> 16382 <span class="title"> 16383 meet the program 16384 </span> 16385 <div class="keywords"></div> 16386 <div class="transcript">Terminal programs have a lot of hidden rules and conventions (like some programs don't quit when you press `Ctrl+C`!), and knowing them makes your life WAY easier. 16387 16388Here are some of the questions I ask myself when starting a program! 16389 16390- is it a REPL? (`Ctrl+D` probably quits) (I can probably use basic `readline` shortcuts) -> is it using canonical mode? (arrow keys don't work when entering text, try `rlwrap`) 16391 16392- am I actually in another program it started? (A pager like `less` or a text editor like `vim`) 16393 16394- is it noninteractive? (quit with `Ctrl+C`) -> is it stuck because it's waiting for input on stdin and I forgot to pipe to it? 16395 16396- is it a shell? -> is this a minimal shell like `dash`? (if so, can I use `bash` instead?) 16397 16398- is it full screen? (`q` might quit) -> mouse reporting: is it on? -> (selecting text doesn't work, but you might be able to click to navigate) 16399 16400- am I in some kind of special environment? -> (a server? tmux/screen? 16401a container? a virtualenv? what environment variables are set?)</div> 16402 </a> 16403 16404 <a href="/comics/line-editing/"> 16405 <span class="title"> 16406 line editing 16407 </span> 16408 <div class="keywords"></div> 16409 <div class="transcript">### editing text you typed in seems so basic: 16410 16411`>>> print("helo")` 16412 16413oops, forgot an l! 16414 16415but there's actually no standard system 16416 16417 16418### programs need to implement even the most basic things 16419 16420Illustration of a little smiling stick figure with curly hair. 16421 16422person: "left arrow" 16423 16424program, represented by a box with a smiley face: "ok I will move the cursor to the left" 16425 16426often programs will use the readline library for this 16427 16428### option 1: NOTHING 16429 16430 16431person (angry): "even the ARROW KEYS don't work???" 16432 16433program (blissfully content): arrow keys? what's that? 16434 16435 16436* Only `Ctrl-W` `Ctrl-U` and backspace work 16437* Examples: `cat`, `nc`, `git` 16438* You're probably in this situation if you press the left arrow key and it prints `^[[D` 16439* You can often add readline shortcuts with `rlwrap`, like this: 16440 16441$ rlwrap nc 16442 16443### option 2: READLINE 16444 16445 16446person (neutral): "it's a little awkward but at least I can use those weird keyboard shortcuts from emacs!" 16447 16448 16449* LOTS of keyboard shortcuts: `Ctrl-A` `Ctrl-E` , arrow keys, many more 16450* You can use `Ctrl-R` to look at history 16451* Examples: `bash`, `irb`, `psql` 16452* If you press `Ctrl-R` and you see "reverse-i-search" , you're probably using readline 16453* Configurable with the `~/.inputrc` config file 16454 16455 16456### option 3: CUSTOM 16457 16458person (smiling): "wow, I can type a multiline command without it being a total disaster?? amazing!" 16459 16460* The keyboard shortcuts are probably influenced by readline 16461* Examples: `fish`, `zsh`, `ipython` 16462* usually you only see custom implementations in bigger projects 16463</div> 16464 </a> 16465 16466 <a href="/comics/filename-tips-in-the-shell/">
16467 <span class="title"> 16468 filename tips (in the shell) 16469 </span> 16470 <div class="keywords"></div> 16471 <div class="transcript">### your shell can help you type weird filenames 16472 16473person: "ugh how do I escape that filename again?" 16474shell: "I can handle it! Just use `Tab`!" 16475 16476### cycle through matching filenames 16477 16478`rm f<Tab><Tab><Tab><Tab>` 16479 16480(doesn't work in bash unless you configure it) 16481 16482### configure bash to cycle through matching filenames 16483 16484Add this to your `~/.inputrc:` 16485``` 16486set show-all-if-ambiguous on 16487set menu-complete-display-prefix on 16488TAB: menu-complete 16489``` 16490### tab complete from the middle of a filename 16491 16492`ls *thing*<Tab>` 16493 16494(or in fish just `ls thing<Tab>`) 16495 16496### tab completion can go wrong 16497 16498programs can change how tab completion works with plugins called "completions" 16499 16500this is usually GREAT (`git add <Tab>` only completes modified files!) but sometimes it's buggy 16501 16502### quote filenames with spaces 16503 16504`cat "Julia Evans.txt"` 16505 16506(if you don't do this you get weird "file not found" errors for `Julia` and `Evans.txt`) 16507 16508### tab completion works inside quoted strings 16509 16510`cat "File N<Tab`</div> 16511 </a> 16512 16513 <a href="/comics/every-core-unix-program-i-use/"> 16514 <span class="title"> 16515 every core unix program I use 16516 </span> 16517 <div class="keywords"></div> 16518 <div class="transcript">### basic file stuff 16519- `touch` - create file 16520- `mkdir` - create directory 16521- `cp` - copy 16522- `mv` - move 16523- `rm` - delete 16524- `ln` - create symlink 16525- `ls` - list directory 16526 16527### how big is it 16528- `wc` - word count 16529- `du` - file size in bytes 16530- `df` - filesystem usage 16531 16532### slice & dice files 16533- `sed` - replace regex 16534- `tr` - replace character 16535- `grep` - search file 16536- `cut` - get column 16537- `awk` - get column (+ more) 16538- `sort` - sort lines 16539- `uniq` - unique lines 16540- `head` - first 10 lines 16541- `tail` - last 10 lines 16542 16543### filesystems 16544- `mount` - mount a filesystem 16545- `umount` - unmount 16546- `dd` - copy data to a disk 16547 16548### manage processes 16549- `ps` - list processes 16550- `lsof` - list open files 16551- `kill` - send a signal 16552- `pkill` - fancy `kill` 16553- `top` - who's using CPU? 16554- `uptime` - time since reboot 16555 16556### permissions 16557- `chown` - change owner 16558- `chmod` - change permissions 16559 16560### time stuff 16561- `time` - measure runtime 16562- `date` - current time 16563- `sleep` - wait X seconds 16564- `cal` - cute calendar 16565 16566### useful with pipes 16567- `less` - scroll text 16568- `cat` - print file contents 16569- `tee` - stdin -> file + stdout 16570- `xargs` - run cmd for each line 16571- `find` - find files by name 16572 16573### compression 16574- `tar` - make/extract tar files 16575- `gzip` - compress with gzip 16576- `gunzip` - decompress 16577- `zip` - compress with zip 16578- `unzip` - decompress 16579 16580### & more 16581- `which` - find cmd on PATH 16582- `man` - read man page 16583- `crontab` - edit crontab 16584- `md5sum` - calculate md5sum 16585- `diff` - diff files</div> 16586 </a> 16587 16588 <a href="/comics/copy-and-paste-in-the-terminal/"> 16589 <span class="title"> 16590 copy and paste in the terminal 16591 </span> 16592 <div class="keywords"></div> 16593 <div class="transcript">### multiline 16594 16595It's SO scary when you paste a bunch of commands by accident and then it runs them all. 16596 16597fish, zsh, and newer bash versions protect you from this: you have to press `Enter` before running the thing you pasted. This is called "bracketed paste" 16598 16599### problem: copying with the mouse can go wrong 16600 16601- copying 400 lines of text by dragging is nobody's idea of a good time 16602- sometimes extra whitespace that you didn't want gets added at the end of lines 16603 16604### panel 3 16605 16606smiling stick figure with short curly hair: âcopying a LOT of text is way easier if you don't use the mouse! Here are 2 tricks for copying without the mouse. " 16607 16608### copy trick 1: pbcopy 16609 16610macOS comes with two programs that can copy from stdin / paste to stdout, like this: 16611 16612`cat main.go | pbcopy` 16613 16614They're SO useful and on Linux I like to write my own versions of `pbcopy`/`pbpaste` using `xsel` or `xclip` 16615 16616 16617### pbcopy over SSH 16618 16619you can even implement pbcopy over SSH (yes really!) with this bas
16619h one-liner. 16620 16621 16622 16623It uses an escape code called "OSC 52". 16624``` 16625printf "\033]52;c;%s\007" 16626 "$(base64 | tr -d '\n')" 16627``` 16628### copy trick 2: syncing the vim clipboard 16629 16630I use vim as a terminal text editor, and I find it's WAY easier if I sync my system clipboard with the vim clipboard like this: 16631 16632`set clipboard=unnamed` 16633 16634`tmux` can also copy to your system clipboard.</div> 16635 </a> 16636 16637 <a href="/comics/canonical-mode/"> 16638 <span class="title"> 16639 canonical mode 16640 </span> 16641 <div class="keywords"></div> 16642 <div class="transcript">### panel 1 16643 16644We said earlier that every program has to implement text editing (on page 21) 16645 16646This is not 100% true! The TTY driver technically has a very limited text editing system called "canonical mode" that hasn't changed since the 80s 16647 16648 16649### what using canonical mode feels like 16650 16651stressed-out stick figure with curly hair, surrounded by question marks: âI pressed an arrow key and it just printed out `^[[D`???â 16652 16653terminal driver, represented by a square smiley face: âwhat's an arrow key?" 16654 16655### how canonical mode works 16656 166571. you type in text 16658`(helloo<Backspace><Enter>)` 166592. the TTY driver lets you edit the text until you press `<Enter>` 166603. the TTY driver sends the line of text to the program 16661 16662### canonical mode is incredibly limited 16663 16664The only ways it lets you edit text are: 16665- backspace 16666- `CTRL+W` (delete word) 16667- `CTRL+U` (delete line) 16668 16669The good thing is those 3 things almost always work. 16670 16671### Interactive programs almost never use canonical mode... 16672 16673bash, represented by a box with a smiley face, thinking: I want my users to be able to use their arrow keys! this isn't the 80s! 16674 16675You can try out canonical mode by running `cat` and typing. 16676 16677### instead, programs receive bytes as soon as you type them 16678 16679bash, thinking: okay, `[[D`, that means "left arrow", I'll tell the terminal emulator to move the cursor... 16680 16681(usually by using a library like `readline`) 16682</div> 16683 </a> 16684 16685 <a href="/comics/stty/"> 16686 <span class="title"> 16687 stty 16688 </span> 16689 <div class="keywords"></div> 16690 <div class="transcript">### your TTY driver has configuration 16691you can see how it's configured by running: 16692 16693`stty -a` 16694 16695for example it print out the current window size! 16696 16697### `Ctrl+S` 16698by default, pressing `Ctrl+S` 16699wi(( freeze your terminal 16700(and `Ctrl+Q` wi(( unfreeze) 16701I have never wanted this in 16702my life, you can turn it off 16703with `stty -ixon` 16704(fish turns it off by default) 16705 16706###fun fact: changing `Ctrl+C` 16707technically you can use `stty` to set a different keyboard shortcut for `Ctrl+C`, (ike "`u`" 16708 16709`stty intr u` 16710 16711this is extremely chaotic and I can't imagine a reason that I would ever do this though 16712 16713### programs have to configure the TTY driver to get friendly features 16714developer: I want arrow keys to work in my program! 16715 16716other person: better tell the TTY driver to turn off canonical mode! 16717(more on the next page) 16718 16719### the TTY driver's settings are called "termios settings" 16720for the gnarly details: 16721 16722`man termios` 16723 16724but if you're writing a terminal program libraries like `readline` or `ncurses` will handle setting up the TTY driver 16725 16726### panel 6 16727 16728smiling stick figure with short curly hair: 16729I've only needed to use `stty` once in the last 20 years and I mostly don't understand its output but I think it's a fun view into terminal internals! 16730</div> 16731 </a> 16732 16733 <a href="/comics/meet-the-tty-driver/"> 16734 <span class="title"> 16735 meet the TTY driver 16736 </span> 16737 <div class="keywords"></div> 16738 <div class="transcript">### the TTY driver is the most obscure part of the system 16739You almost never need to think about it, but when I've wanted to do something weird (like put a terminal in a web browser) understanding the driver is SO USEFUL 16740 16741### when you start your terminal emulator, it asks the OS to create a "pseudoterminal pair" which is a pair of two files 16742terminal emulator <-> TTY <-> TTY driver <-> TTY <-> program 16743 16744### a "TTY" is the program's side of the pair 16745programs use it to: 16746- communicate with the terminal emulator by reading/writing bytes 16747- configure the TTY driver (more on the next page!) 16748 16749Run `tty` fo see the current TTY! 16750 16751### the TTY driver is why `Ctrl+C` does the same thing relatively consistently 16752program: 16753you press `Ctrl+C`, I send a signal! 16754well, unless the program tells me it wants the raw bytes! 16755 16756### some things the TTY driver is in charge of 16757(you might think "these are unrelated" and you'd be right) 16758 16759- storing the terminal window's size 16760- sending a `SIGHUP` signal when you close your terminal 16761- a basic mode for entering text called "canonical mode" 16762- pausing the output and confusing you when you press `Ctrl+S` 16763- tracking which process is in the "foreground" and sending what you type there</div> 16764 </a> 16765 16766 <a href="/comics/keyboard-shortcuts/">
16767 <span class="title"> 16768 keyboard shortcuts 16769 </span> 16770 <div class="keywords"></div> 16771 <div class="transcript"> 16772### editing text ([almost] always works) 16773- `backspace` 16774- `Ctrl + W`: delete previous word 16775- `Ctrl + U`: delete line 16776 16777(except in text editors) 16778 16779 16780### quitting 16781- `Ctrl + C`: quit (`SIGINT`) 16782- `Ctrl + Z`: stop process (`SIGTSTP`) (resume with `fg` or `bg` or kill (with `kill`) 16783- `Ctrl + D`: quit (in a REPL) (more on page 20) 16784- `q`: quit (in some full screen programs) 16785- `Enter`: exit frozen SSH session 16786 16787or the nuclear option: 16788``` 16789$ ps aux | grep THING 16790bork 7213 ... THING 16791$ kill -9 7213 16792``` 16793 16794### editing text 16795(these often work in a readline-like situation) 16796 16797- `arrow keys` 16798- `Ctrl + A` or `Home`: beginning of line 16799- `Ctrl + E` or `End`: end of line 16800 16801- `Ctrl + arrow keys`: left/right a word 16802or sometimes `Alt + arrow keys` 16803or `Option + arrow keys` 16804or `Alt+b / Alt+f` 16805- `Ctrl + K`: delete line forward 16806- `Ctrl + Y`: paste (from `Ctrl+K` or `Ctrl+U`) 16807- `Ctrl + H`: might work if `Backspace` doesn't 16808 16809also many shells have a "vi mode" if that's your jam 16810 16811 16812### other useful stuff 16813- Ctrl + L: clear screen 16814- Ctrl + R: search history 16815- Ctrl + Q: unfreeze screen (that you froze with `Ctrl+S`, more on page 25) 16816 16817### copy and paste 16818in your terminal emulator, it's usually: 16819`Ctrl + Shift + C/V` 16820or `Cmd + C/V` 16821 16822### mouse stuff that might work 16823- `Option + click` 16824- or `Alt + click`: place cursor 16825- `scroll wheel`: scroll</div> 16826 </a> 16827 16828 <a href="/comics/editing-text-in-a-repl/"> 16829 <span class="title"> 16830 editing text in a REPL 16831 </span> 16832 <div class="keywords"></div> 16833 <div class="transcript">### editing text in a REPL doesn't always work 16834little stick figure, horrified: when I press my ARROW KEYS it just prints out `[[D`??? what? 16835 16836### this is because every program has fo implement text editing itself 16837 16838terminal program author, represented by a nonplussed looking stick figure with medium length straight hair, thinking: 16839but I just want arrow keys to work?? shouldn't that be automatic? 16840 16841unix, represented by a box with a smiley face: NOPE you gotta do it 16842 16843### you do gef a few things automatically 16844- backspace (occasionally backspace won't work and you have to use 16845`Ctrl+H` instead) 16846- `Ctrl+W` (delete word) 16847- `Ctrl+U` (delete line) 16848 16849(see page 25 for what "automatically" means) 16850 16851 16852### REPLs mostly have the same keyboard shortcuts 16853there's a very popular library called "readline", and everyone either uses it or imitates how it works 16854 16855for example `Ctrl+A` ("go to beginning of line") comes from readline 16856 16857 16858### `rlwrap` adds readline keyboard shortcuts 16859for example on my machine the dash shell doesn't use readline but you can make it better by running: 16860 16861`rlwrap dash` 16862 16863 16864### built in programs on Mac don't use `readline` 16865(for example `sq1ite3`) 16866 16867this is probably because `readline` is GPL licensed 16868 16869They use `libedit` which is worse. I like to install a sqlite version with `readline` support and use that instead. 16870</div> 16871 </a> 16872 16873 <a href="/comics/term/"> 16874 <span class="title"> 16875 TERM 16876 </span> 16877 <div class="keywords"></div> 16878 <div class="transcript"> 16879### different terminal emulators use different escape codes 16880 16881terminal emulator 1: if you print out `ESC[2J` I'll clear the screen! 16882 16883terminal emulator 2: for me it's `ESC[HESC[J`! 16884 16885### your system has a database called "terminfo" with escape codes in it 16886how if plays out when you press `Ctrl+L` to clear the screen: 16887 16888program, with a little heart over it, thinking: 16889ah, she wants fo clear the screen! I'll look up how to do that in the terminfo database... 16890 16891(on my machine, the database is in `/usr/share/terminfo`) 16892 16893program: `ESC[HESC[J` 16894 16895terminal emulator, thinking: ok, clearing the screen! 16896 16897 16898### how programs know what terminal you're using: `TERM` 16899your terminal emulator sets the `TERM` environment variable when it starts 16900 16901fun fact: terminal emulators often say they're "`xterm-256color`" even if they're not 16902 16903### this can break when SSHing into an old system with a new terminal emulator 16904(in a VERY annoying way) 16905 16906happy little stick fiture: I am using ghostty 16907 16908program, with a little heart over it: NOPE never heard of it 16909 16910 16911### some ways to fix `TERM` 16912- install the terminfo file for your terminal emulator on the system 16913- use a different terminal emulator 16914- just set `TERM=xterm-256color`, it'll often sort of work</div> 16915 </a> 16916 16917 <a href="/comics/the-mouse/">
16918 <span class="title"> 16919 the mouse 16920 </span> 16921 <div class="keywords"></div> 16922 <div class="transcript">### when you click in the terminal, it can either be handled by 16923 16924 16925your terminal emulator 16926(represented by a box with a winky cursor face and little arms and legs) 16927(good if you want to copy text) 16928 16929or 16930 16931the program 16932(represented by a box with a smiley face) 16933(lots of programs have mouse support!) 16934 16935 16936### programs can tell the terminal emulator to let them handle the mouse 16937program: if there's a mouse click, send me escape codes to tell me where it was! 16938 16939terminal emulator: okay! I'll disable all my usual mouse functions like "selecting text"! 16940 16941this is called "mouse reporting" 16942 16943### some programs that have mouse support 16944 16945- tmux: resize a pane! right-click for a menu! 16946- htop: click to sort columns! 16947- micro: text editor with good mouse support 16948- vim: click on the tab bar! 16949- and lots more! (`lazygit`, `mc`, `zellij`, `btop`...) 16950 16951 16952### how to force the terminal emulator to handle the mouse: 16953 16954unhappy stick figure with short curly hair, thinking: 16955ugh no I don't want to focus that pane, I want to COPY SOME TEXT!!! 16956 16957 16958* could be something else too, it 16959depends on your terminal emulator 16960 16961 16962### the scroll wheel 16963In some programs (like `less`) the scroll wheel does the same thing as pressing up/down arrow keys really fast 16964 16965terminal emulator: UP UP UP UP UP UP UP UP UP UP UP UP 16966 16967in other programs (like `lazygit`) it uses "mouse reporting" to report where your mouse was when you scrolled 16968 16969 16970### other mouse features your terminal emulator might have 16971 16972- `Shift+click` (or something) to open a link in a browser 16973- `Alt+click` (or maybe `Option`) to move the cursor when editing a command in your shell</div> 16974 </a> 16975 16976 <a href="/comics/colours/"> 16977 <span class="title"> 16978 colours in the terminal 16979 </span> 16980 <div class="keywords"></div> 16981 <div class="transcript">### your terminal emulator has 16 configurable colours 16982| | normal | bright | 16983|-----------|-----------|-----------| 16984| black | 0 | 0 | 16985| red | 1 | 1 | 16986| green | 2 | 2 | 16987| yellow | 3 | 3 | 16988| blue | 4 | 4 | 16989| purple | 5 | 5 | 16990| cyan | 6 | 6 | 16991| white | 7 | 7 | 16992 16993### these are called "ANSI colours" 16994you can configure them in your terminal emulator's settings 16995 16996OR 16997 16998run a script that prints escape codes to magically set up your colours 16999`https://wzrd.page/scripts` 17000(my favourite way!) 17001 17002 17003### programs can use ANSI colours by printing an escape code 17004`echo -e "\033[34m blue text"` 17005 17006`3` means "normal fg colour" 17007`4` means "blue" 17008 17009### the default ANSI colours often have bad contrast 17010`ls --color` often displays directories in ANSI "blue" which can look like this: 17011 17012[bar of illegibly dark text against a dark background, which says "can you read this?"] 17013 17014ANSI "yellow" on white also often has bad contrast 17015 17016### "minimum contrast" 17017Picking ANSI colours which always have good contrast is impossible. 17018 17019the only real solution is to use a terminal emulator which has a "minimum contrast" feature 17020(like iTerm or kitty) which will fix contrast issues 17021 17022### usually if a program is writing to a pipe, it'll disable colours 17023`$ grep blah file.txt | less` 17024 17025grep, represented by a box with a smiley face: 17026better turn off colours so that I don't accidentally show 17027someone `^[[34ntext here]</div> 17028 </a> 17029 17030 <a href="/comics/meet-the-terminal-emulator/"> 17031 <span class="title"> 17032 meet the terminal emulator 17033 </span> 17034 <div class="keywords"></div> 17035 <div class="transcript">### your terminal emulator has two main jobs 17036 170371. turn your actions (typing & clicking) into bytes and send them 170382. receive bytes and display them visually 17039 17040Illustration of a terminal emulator, with a winking cursor face, and a program, represented by a box with a smiley face. The program has a heart above it, and there are arrows going back and forth between them labelled "bytes" 17041 17042 17043### a little bit of history 17044it's called an "emulator" because in the 80s a "terminal" was a separate machine from the computer 17045 17046Illustration of a bulky old monitor, with a keyboard attached with a spiral cord, and a wire running to a panl of buttons and displays, labelld "mainframe". There are arrows going back and forth between them labelled "bytes" 17047 17048We still use the same 80s protocol! 17049 17050 17051### what are these "bytes"? 17052the bytes are either: 17053- text (like `cat blah.txt`) 17054- escape codes (for example to tell the terminal what colour to display the text in) 17055- control characters example `Ctrl+C` is the byte `3`) 17056 17057 17058### it's in charge of copy and paste 17059your terminal emulator lets you select text and copy/paste it (usually with `Ctrl+Shift+
17059C` (Linux) or `Cmd+C` (Mac)) 17060 17061(copy & paste tips on page 18!) 17062 17063 17064 17065### it manages colours and fonts! 17066some terminal emulators come with a big theme library of different colourschemes! 17067 17068if yours doesn't, this site has colourschemes for many terminal emulators: 17069`iterm2colorschemes.com` 17070 17071 17072### fun fact: how `Ctrl-X` gets translated to bytes 17073``` 17074Ctrl-A => 1 17075Ctrl-B => 2 17076... 17077Ctrl-Z => 26 17078``` 17079 17080`Ctrl` is the only modifier key I trust in the terminal, all of the others can work differently depending on the situation 17081</div> 17082 </a> 17083 17084 <a href="/comics/redirects-1/"> 17085 <span class="title"> 17086 redirects 17087 </span> 17088 <div class="keywords"></div> 17089 <div class="transcript">### redirect to a file: 17090`cmd > file.txt` 17091terminal emulator into program, program out to file.txt, error out to terminal emulator 17092 17093### append to a file: 17094`cmd >> file.txt` 17095terminal emulator into program, program out to file.txt (append mode), error out to terminal emulator 17096 17097### send a file to stdin: 17098`cmd < file.txt` 17099file.txt into program, program out and err to terminal emulator 17100 17101### redirect stderr to a file: 17102`cmd 2 > file.txt` 17103terminal emulator into program, program out to nowhere, err out to file.txt 17104 17105### redirect stdout AND stderr: 17106`cmd > file.txt 2>&1` 17107terminal emulator into program, out and err to file.txt 17108 17109### pipe stdout: 17110`cmd1 | cmd2` 17111terminal emulator into program 1, 1 out to program 2 via pipe, 2 out to command line, program 2 out 1 and 2 to terminal emulator 17112 17113### pipe stdout AND stderr: 17114`cmd1 2>&1 | cmd2` 17115terminal emulator into program 1, 1 and 2 out to program 2 via pipe, program 2 out 1 and 2 to terminal emulator 17116 17117### three gotchas 171181. `cmd file.txt > file.txt` 17119will delete the contents of `file.txt` 17120some people use `set -o noclobber` (in bash/zsh) to avoid this 17121 17122but I just have "never read from redirect to the same file" seared into my memory. 17123 171242. `sudo echo blah > /root/file.txt` 17125doesn't write to `/root/file.txt` as root. Instead, do: 17126 17127`echo blah | sudo tee /root/file.txt` 17128or 17129`sudo sh -c 'echo blah > /root/file.txt'` 17130 171313. `cmd 2>&1 > file.txt` 17132doesn't write both stdout and stderr to `file.txt`. Instead, do: 17133`cmd > file.txt 2>&1` 17134 17135 17136### `cat` vs `<` 17137I almost always prefer to do: 17138 17139`cat file.txt | cmd` 17140 17141instead of 17142 17143`cmd < file.txt` 17144 17145it works fine & it feels better to me 17146 17147using `cat` can be slower if if's a GIANT file though 17148 17149### `&>` and `&|` 17150some shells support `&>` and `&|` to redirect/pipe both stdout and stderr 17151 17152(also some shells use `|&` instead of `&|`) 17153</div> 17154 </a> 17155 17156 <a href="/comics/job-control/"> 17157 <span class="title"> 17158 job control 17159 </span> 17160 <div class="keywords"></div> 17161 <div class="transcript">### your shell lets you run many programs ("jobs") in the same terminal tab 17162programs can either be: 17163- foreground 17164- background 17165- stopped (which is more like "paused") 17166 17167### `&` runs a program in the background 17168for example I like to convert 100 files in parallel like this: 17169``` 17170for i in `seq 1 100` 17171do 17172 convert $i.png $i.jpg & 17173done 17174``` 17175 17176### `jobs` lists backgrounded & stopped jobs 17177 17178``` 17179$ jobs 17180[1] Running python blah.py & 17181[2] Stopped vim 17182``` 17183 17184use the numbers to bring them to the foreground or background (like `fg %2`), kill them (kill `%2`), or disown them 17185 17186### when you close a terminal tab all jobs are killed with a `SIGHUP` signal 17187you can stop this with `disown` or by starting the program with `nohup`: 17188`disown %1` (job number goes here) 17189`nohup my_program &` 17190 17191### a trick to kill programs if `Ctrl+C` doesn't work 17192 171931. press `Ctrl+Z` to stop the program 171942. run `kill %1` to kill it (or `kill -9 %1` if you're feeling extra murderous) 17195 17196 17197### a little flowchart 17198Three boxes, labelled "running in foreground", "stopped", and "running in background" 17199 17200`Ctrl+Z` goes from "running in foreground" to "stopped" 17201`fg` goes from "stopped" to "running in foreground" 17202`fg` goes from "running in background" to "running in foreground" 17203`bg` goes from "stopped" to "running in background" 17204</div> 17205 </a> 17206 17207 <a href="/comics/meet-the-shell/">
17208 <span class="title"> 17209 meet the shell 17210 </span> 17211 <div class="keywords"></div> 17212 <div class="transcript">### the shell starts programs 17213when you run a program in the terminal, you're actually asking your shell to start it for you 17214 17215it turns out that starting programs is a surprisingly complicated job! 17216 17217### the 3 most popular shells 17218there are LOTS of shells but 95% of people use 17219- bash (default on Linux) 17220- zsh (default on Mac (in 2025)) 17221- fish (aims to be more user friendly) 17222 17223### fish: the friendly interactive shell 17224 17225ASCII illustration of a fish 17226 17227I love how fish has friendly defaults that I can use without configuring it 17228 17229this is (mostly) not a fish propaganda zine though 17230 17231Little illustration of a smiling stick figure holding up a sign that says "fish 4eva", labelled "me" 17232 17233 17234### bash and zsh are both "POSIX shells" 17235this means they follow a standard for how Unix shells should behave, but there are differences 17236 17237I'll mention when something varies between shells! 17238 17239 17240### where to find your shell's config file 17241 17242bash: 17243`~/.bashrc` or `~/.bash_profile` (which one is a rabbit hole, huge flow chart at `wzrd.page/bashrc`) 17244 17245zsh: 17246`~/.zshrc` 17247 17248fish: 17249`~/.config/fish/config.fish` 17250 17251 17252### `.bashrc` vs `.bash_profile` 17253here's an trick to figure out whether bash is using `.bashrc` or `.bash_ profile` (or both!) 17254 17255Add: 17256``` 17257echo "this is .bashrc" 17258echo "this is . bash _ profile" 17259``` 17260to each file, open a new terminal tab, and see what it prints out!</div> 17261 </a> 17262 17263 <a href="/comics/terminal-redirects/"> 17264 <span class="title"> 17265 terminal redirects 17266 </span> 17267 <div class="keywords"></div> 17268 <div class="transcript">### redirect to a file: 17269`cmd > file.txt` 17270terminal emulator into program, program out to file.txt, error out to terminal emulator 17271 17272### append to a file: 17273`cmd >> file.txt` 17274terminal emulator into program, program out to file.txt (append mode), error out to terminal emulator 17275 17276### send a file to stdin: 17277`cmd < file.txt` 17278file.txt into program, program out and err to terminal emulator 17279 17280### redirect stderr to a file: 17281`cmd 2 > file.txt` 17282terminal emulator into program, program out to nowhere, err out to file.txt 17283 17284### redirect stdout AND stderr: 17285`cmd > file.txt 2>&1` 17286terminal emulator into program, out and err to file.txt 17287 17288### pipe stdout: 17289`cmd1 | cmd2` 17290terminal emulator into program 1, 1 out to program 2 via pipe, 2 out to command line, program 2 out 1 and 2 to terminal emulator 17291 17292### pipe stdout AND stderr: 17293`cmd1 2>&1 | cmd2` 17294terminal emulator into program 1, 1 and 2 out to program 2 via pipe, program 2 out 1 and 2 to terminal emulator 17295 17296### three gotchas 172971. `cmd file.txt > file.txt` 17298will delete the contents of `file.txt` 17299some people use `set -o noclobber` (in bash/zsh) to avoid this 17300 17301but I just have "never read from redirect to the same file" seared into my memory. 17302 173032. `sudo echo blah > /root/file.txt` 17304doesn't write to `/root/file.txt` as root. Instead, do: 17305 17306`echo blah | sudo tee /root/file.txt` 17307or 17308`sudo sh -c 'echo blah > /root/file.txt'` 17309 173103. `cmd 2>&1 > file.txt` 17311doesn't write both stdout and stderr to `file.txt`. Instead, do: 17312`cmd > file.txt 2>&1` 17313 17314 17315### `cat` vs `<` 17316I almost always prefer to do: 17317 17318`cat file.txt | cmd` 17319 17320instead of 17321 17322`cmd < file.txt` 17323 17324it works fine & it feels better to me 17325 17326using `cat` can be slower if if's a GIANT file though 17327 17328### `&>` and `&|` 17329some shells support `&>` and `&|` to redirect/pipe both stdout and stderr 17330 17331(also some shells use `|&` instead of `&|`) 17332</div> 17333 </a> 17334 17335 <a href="/comics/terminal-less/"> 17336 <span class="title"> 17337 less 17338 </span> 17339 <div class="keywords"></div> 17340 <div class="transcript">### many programs use `less` without telling you 17341`less` lets you through text, so programs will use `less` by default any time they want to display a lot of text 17342 17343`git`, represented by a box with a smiley face: I want to display a huge diff... I'll show it in `less`! 17344 17345`man`, also represented by a box with a smiley face: I need to display a man page... I'll use `less`! 17346 17347it's `less` because it's an improved version of `more` 17348
17349### how to know you're in `less` 17350if it's suddenly full screen and there's this little colon in the bottom left, it might be `less` 17351 17352 17353### a few `less` tips 17354- quit: `q` 17355- help: h 17356- scroll: arrow keys/spacebar/mouse wheel 17357- search: `/banana ENTER` 17358- next/prev match: `n/N` 17359- go to start/end: `g/G` 17360 17361also piping to `less -R` will interpret escape codes like colours 17362 17363 17364### how to tell a program not to use `less` 17365you can set the `PAGER` environment variable to something else to 17366programs to use that instead 17367 17368I've never had any reason to set `PAGER` though 17369 17370### programs will also drop you into `vim` sometimes 17371the default text editor is `vim`. If you don't like `vim` you can set the EDITOR environment variable 17372 17373`export EDITOR=micro` 17374(your favourite editor here) 17375</div> 17376 </a> 17377 17378 <a href="/comics/inside-the-commit/"> 17379 <span class="title"> 17380 inside the commit 17381 </span> 17382 <div class="keywords"></div> 17383 <div class="transcript">### you can see for yourself how git is storing your files! 17384 17385You just need one command: `git cat-file -p` 17386 17387First, get a commit ID. You can get one from `git log` 17388 17389### 1. read the commit 17390``` 17391git cat-file -p 3530a4 17392tree 22b920 17393parent 56cfdc 17394author Julia 1697682215 -0500 17395committer Julia 1697682215 -0500 17396 17397commit message goes here 17398``` 17399`22b920` is the directory ID 17400 17401I just use `git cat-file` for fun and learning, never to get things done 17402 17403### 2. read the directory 17404``` 17405$ git cat-file -p 22b920 17406100644 blob 4fffb2 .gitignore 17407100644 blob e351d9 404.html 17408100644 blob cab416 Cargo.toml 17409100644 blob fe442d hello.html 17410040000 tree 9de29f src 17411``` 17412(`fe442d` is a file ID) 17413(IDs are actually 40 characters) 17414 17415### 3. read a file 17416 17417``` 17418$ git cat-file -p fe442d 17419<!DOCTYPE html> 17420<html lang="en" 17421<body> 17422<h1>Hello!</h1> 17423</body> 17424</html></p> 17425``` 17426### 4. and we're done! 17427 17428`fe442d` is the sha1 hash of the contents of the file. It's called a "blob id". Commit and tree IDs re hashes too. 17429 17430Using a hash to identify each file is how git avoids duplication: if the file's contents don't change, the hash won't change, so git doesn't need to store a new version! 17431</div> 17432 </a> 17433 17434 <a href="/comics/lost-commits/"> 17435 <span class="title"> 17436 lost commits 17437 </span> 17438 <div class="keywords"></div> 17439 <div class="transcript"> 17440### commits in git are usually saved forever 17441 17442But even if git still has your commits, they're not always easy to find. 17443 17444Some ways commits get "lost": 17445- `git commit --amend` 17446- `git rebase` 17447- deleting an unmerged branch 17448- `git stash drop` 17449 17450### the three levels of losing commits 17451- annoying: the commit isn't in the history of any branch/tag, but it's relatively easy to find 17452- nightmare: you need to search every single commit to find it 17453- disaster: it's been deleted 17454 17455### how commits can get lost: git commit --amend 17456before: 17457Diagram of two boxes side by side, labelled "main branch". The one on the left is labelled "`parent`". The one on the right is labelled "`fix color buug`" (typo!). 17458 17459after: 17460The same diagram as above, but the initial two boxes are now labelled "Now it's "lost"!". Also branching off of "`parent`" is a third box, labelled "`fix color bug`". That branch is now labelled "main branch". 17461 17462### how commits can get lost: git rebase 17463before: 17464Two boxes side-by-side, connected by a line. These are labelled "`main branch`". Also branching off of the leftmost box are two further boxes, one labelled with a heart, and one with a star. These are labelled "`feature branch`". 17465 17466after: 17467An initial box with two lines of boxes coming off of it. The topmost line of boxes is a blank box, followed by a heart, then a star. The blank box is labelled "`main branch`". The heart and star boxes are labelled "`feature branch`". The lower line of boxes have a heart and a star and are highlighted in red and labelled "now these two are "lost!". 17468
17469### how commits can get lost: git stash drop 17470before: 17471Three boxes in a horizontal row. The left two boxes are blank. The middle box is labelled "`main branch`". The rightmost box has a star, and is labelled "`stashed commit`". 17472 17473after: 17474The same diagram as above, but now the rightmost box is labelled "now it's "lost"!". 17475 17476stash is the only way I've seen the "nightmare" situation happen. 17477 17478### you can find lost commits 17479 17480I find it very comforting to know that git keeps my lost commits around. How to find them: 17481- annoying: use the reflog (page 26) 17482- nightmare: use `git fsck` 17483- disaster: impossible (but this has never happened to me) 17484</div> 17485 </a> 17486 17487 <a href="/comics/meet-the-remote/"> 17488 <span class="title"> 17489 meet the remote 17490 </span> 17491 <div class="keywords"></div> 17492 <div class="transcript">### any repository you're pushing to / pulling from is called a "remote" 17493 17494remotes can be: 17495- hosted by GitHub/GitLab/etc. 17496- on your own server 17497- just a folder on your computer 17498 17499### git push syntax 17500 17501(same for git pull) 17502 17503`git push origin main` 17504 17505"`origin`" is the remote name, "`main`" is the remote branch. 17506 17507the default name for a remote is origin but you can name it anything 17508 17509### tip! 17510I like to configure `push.autoSetupRemote true` to automatically set up tracking the first time I push a new branch 17511 17512 17513### remotes are where the drama happens 17514 17515Smiling stick figure with short curly hair: I spent 3 hours working on `cats.py` 17516 17517person: git pull 17518 17519git, represented by a box with a smiley face: fun fact! your coworker totally rewrote that file! 17520 17521### example: I use 2 remotes when contributing to open source projects 17522 17523Diagram of a box labelled "local repo". Local repo has an arrow labelled "push to here", pointing to a box labelled "My personal GitHub fork". That box has an arrow labelled "pull request", pointing to a box labelled "main project repo name: "origin"". That box has an arrow labelled "pull from here", pointing back to the "local repo" box. 17524 17525### remotes are configured in `.git/config` 17526 17527every remote has a name and URL 17528``` 17529[remote "origin"]
17530url = [email protected]:jvns/myrepo 17531branch ["main"] 17532remote = origin 17533merge = refs/heads/main 17534``` 17535"`origin`" is the name, "`[email protected]:jvns/myrepo`" is the URL. 17536 17537this sets up "tracking" between local main remote main on origin so that git knows what to push to when you run git push or git pull 17538 17539### protocols 17540 17541Git has 3 main protocols for remotes. The protocol is embedded in the URL. 17542- HTTP (I use this if I only want to pull) 17543 `https://github.com/jvns/myrepo` 17544- SSH (I use this if I need to push) 17545 `[email protected]:jvns/myrepo` 17546- local 17547 `file:///home/bork/myrepo` 17548</div> 17549 </a> 17550 17551 <a href="/comics/the-reflog/"> 17552 <span class="title"> 17553 the reflog 17554 </span> 17555 <div class="keywords"></div> 17556 <div class="transcript">### a reflog is a log of commit IDs 17557 17558I use the reflog to find "lost" commits: it contains every commit ID that the branch/tag/HEAD has ever pointed to. 17559 17560### some differences between `git log main` and `git reflog main` 17561- reflog entries older than 90 days might get deleted by `git gc` 17562- the reflog can show you where your branch was before a rebase. `git log` can't 17563- the reflog isn't shared between repositories. `git log` is. 17564- if I'm looking at the reflog, I'm having a bad day 17565 17566### which reflog to use? 17567The main two I use are: 17568 17569- `git reflog` 17570- every single commit you've ever had checked out 17571- has everything but very noisy 17572- it's the reflog for `HEAD ` 17573 17574- `git reflog BRANCH` 17575 - just the history for that branch, might be less noisy 17576 17577### how to use the reflog 175781. run git reflog 175792. sadly stare at output until you find a log message that looks right 175803. look at the commit 17581``` 17582 git show $COMMIT_ID 17583 git log $COMMIT_ID 17584``` 175854. repeat until you find the thing 175865. use something like 17587`git reset --hard $COMMIT_ID` or `git branch $NAME $COMMIT_ID` to put the commit on a branch 17588 17589### the reflog kind of sucks 17590- (sad face) if you delete a branch, git deletes its reflog 17591- (sad face) if you drop a stash entry, you can't use the reflog to get it back 17592- (sad face) reflog entries don't correspond exactly to git commands you ran 17593 17594But it's the best we have. 17595 17596### `git fsck`: the last resort 17597 17598If a commit isn't in the reflog (for example if you "lost" it with `git stash drop`), there's still hope! 17599 17600You can use `git fsck` to list every commit ID that's unreferenced. 17601 17602I've never done this though: I try to avoid getting into this situation. 17603</div> 17604 </a> 17605 17606 <a href="/comics/reset/"> 17607 <span class="title"> 17608 reset 17609 </span> 17610 <div class="keywords"></div> 17611 <div class="transcript">### git has no undo 17612there's no 17613- unadd 17614- uncommit 17615- unmerge 17616- unrebase 17617 17618instead, git has a single dangerous command for undoing: 17619`git reset` 17620 17621### most git commands move the current branch forwards 17622- `git commit` 17623Illustration of three boxes in a row, connected by lines. There is an arrow pointing from the second box to the third box. 17624- `git merge` 17625Illustration of two boxes in a row, connected by lines. From the second box, two lines diverge to two other boxes, and from those two, lines converge back into a final box. There is an arrow pointing from one of the diverged boxes into the final merged box. 17626- `git pull` 17627Illustration of five boxes in a row, connected by lines. There is an arrow pointing from the second box to the fifth box. 17628 17629(though rebase is a sideways move) 17630 17631### git reset can move the current branch anywhere 17632- backwards! 17633- forwards! 17634- "sideways"! 17635 17636Illustration of five boxes, connected with lines into two branches, with arrows pointing in all directions amongst them. 17637 17638this makes it possible to undo, but you can also really mess up your branch 17639 17640### how git reset works 17641 17642`git reset HEAD^` 176431. finds the commit ID corresponding to HEAD^ (for example a2b3c4) 176442. forces your current branch to point to a2b3c4 176453. unstages all changes 17646 17647### `--hard`: the danger option 17648 17649`git reset $COMMIT_ID` 17650Keeps all the files in your working directory exactly the same. 17651 17652`git reset --hard $COMMIT_ID` 17653Throws away all your uncommitted changes. Useful but dangerous. 17654 17655### problems `reset` can cause 17656 17657- (sad face) it's easy to "lose" commits, especially if you move a branch backwards 17658- (sad face) if you use `--hard`, you can permanently lose your uncommitted changes</div> 17659 </a> 17660 17661 <a href="/comics/the-staging-area/">
17662 <span class="title"> 17663 the staging area 17664 </span> 17665 <div class="keywords"></div> 17666 <div class="transcript">### git has a 2-stage commit process 17667 176681. tell git what you want to stage (`git add`, `git rm`, `git mv`, etc.) 176692. make the commit with git commit 17670 17671Diagram showing two boxes, labelled "untracked files" and "unstaged changes". They converge into a box labelled "stage" via `git add`. They then flow into a box labelled "committed", which has a heart and smiley face beside it, via `git commit`. 17672 17673### git uses 3 terms interchangeably for the staging area 17674 176751. staged (like `--staged`) 176762. cache (like `--cached`) 176773. index (like `--keep-index`) 17678 17679it's total chaos but they're all the same thing 17680 17681tiny illustration of a sad stick figure with curly hair: why 17682 17683### tip: you can use `git add -p` to commit only certain parts of a file 17684 17685person: I only want to commit my actual changes, not all the random debugging code I put in 17686 17687### gotcha: `git diff` only shows unstaged changes 17688 17689You can use: 17690- `git diff HEAD` to see ALL changes you haven't committed yet 17691- `git diff --cached` to see staged changes 17692 17693### gotcha: `git commit -a` doesn't automatically add new files 17694 17695person: I CONSTANTLY forget to add new files and then get confused about why they didn't get committed</div> 17696 </a> 17697 17698 <a href="/comics/the-diff-algorithm/"> 17699 <span class="title"> 17700 the diff algorithm 17701 </span> 17702 <div class="keywords"></div> 17703 <div class="transcript">### git is CONSTANTLY showing you diffs 17704 17705smiling stick figure with short curly hair: `git show COMMIT_ID` 17706 17707git, represented by a box with a smiley face: here's the diff! 17708 17709and it makes it seem like git thinks in terms of diffs 17710 17711### have you ever noticed your git diffs don't make sense? 17712 17713git: `deleted...` 17714`added...` 17715 17716person: but I didn't DELETE that file, I MOVED it 17717 17718### in git, moving a file is the same as deleting the old one and adding the new one 17719``` 17720git mv old.py new.py 17721``` 17722is the same as 17723``` 17724cp old.py new.py 17725git rm old.py 17726git add new.py 17727``` 17728### git is just guessing about your intentions 17729 17730person: 17731``` 17732git mv old.py new.py 17733git commit 17734``` 17735git: well the OLD version has `old.py` and the NEW version has `new.py` and they have the same contents... so I guess you moved it 17736 17737### diff is an algorithm 17738 17739the algorithm: 17740- takes 2 versions of the code 17741- compares them 17742- tries to summarize it in a human readable way 17743 17744(but it doesn't always do a great job) 17745 17746### git has many diff algorithms 17747 17748person: I've been trying out `histogram` because I don't like how the default algorithm displays the diff when I rearrange code 17749 17750how to try it out: 17751`git diff --histogram` 17752</div> 17753 </a> 17754 17755 <a href="/comics/diverged-branches-1/"> 17756 <span class="title"> 17757 diverged remote branches 17758 </span> 17759 <div class="keywords"></div> 17760 <div class="transcript">### when pushing/pulling, the hardest problems are caused by diverged branches 17761``` 17762! [rejected] 17763main -> main 17764(non fast-forward 17765``` 17766`fatal: Not possible to fastforward, aborting` 17767 17768`fatal: Need to specify how to reconcile divergent branches.` 17769 17770(each of these three messages is in a spiky bubble, and they are all surrounded by numerous sad faces.) 17771 17772### what are diverged branches? 17773 17774both sides have commits that the other doesn't, like this: 17775 17776An illustration of two boxes in a row, connected by a line. The first one has a star, the second has a heart. Branching out from the heart are a box with a hash symbol, labelled "`local main`", and a box with a squiggle, labelled "`remote main`". 17777 17778I like to fix my diverged branches before making more commits. 17779 17780### there are 4 possibilities with a remote branch 17781 177821. up to date 17783Illustration of three boxes in a row, connected by lines. The final box is labelled both "local" and "remote". 17784 177852. need to pull 17786Illustration of four boxes in a row, connected by lines. The second box is labelled "local" and the fourth one is labelled "remote". 17787 177883. need to push 17789Illustration of four boxes in a row, connected by lines. The second box is labelled "remote" and the fourth one is labelled "local". 17790 177914. DIVERGED (need to decide how to solve it) 17792Illustration of two boxes in a row, connected by lines. Diverging from the second box are two branches. One has one box in it and is labelled "remote". The other one has two boxes and is labelled "local". 17793 17794Illustration of a smiling stick figure with short curly hair. 17795 17796person: when I have a diverged branch, I usually just run `git pull --rebase` and move on. On the next page we'll talk about some other options though! 17797 17798### how to tell if your branches have diverged: git status 17799`$ git fetch` (get the latest remote state first) 17800 17801`$ git status` 17802Your branch and '`origin/main`'
17802 have diverged, and have 1 and 1 different commits each, respectively. 17803(use "`git pull`" to merge the remote branch into yours) 17804 17805### git fetch and git pull 17806 17807`git fetch` just fetches the latest commits from the remote branch. 17808 17809`git pull origin main` has 2 parts: 17810- `run git fetch origin main` 17811- `run git merge origin/main` (or sometimes rebase) 17812 17813(More about how to tell `git pull` to merge/rebase on page 16!) 17814</div> 17815 </a> 17816 17817 <a href="/comics/unix-permissions/"> 17818 <span class="title"> 17819 unix permissions 17820 </span> 17821 <div class="keywords"></div> 17822 <div class="transcript">### There are 3 things you can do to a file 17823 17824**r**ead **w**rite e**x**ecute 17825 17826### `ls -l file.txt` shows you permissions. 17827Here's how to intepret the output: 17828 17829`rw- rw- r-- bork staff` 17830 17831the first `rw-` means bork (user) can read & write 17832the second `rw-` means staff (group) can read & write 17833`r--` means ANYONE can read 17834 17835 17836### File permissions are 12 bits 17837 17838The first digit is `setuid`, the second digit is `setgid`, the third digit is `sticky` 17839`110` (user) `110` (group) `100` (all) 17840 17841For files: 17842r = can read 17843w = can write 17844x = can execute 17845 17846For directories, it's approximately: 17847r = can list files 17848w = can create files 17849x = can cd into & access files 17850 17851### 110 in binary is 6 17852So `rw-` 17853= 110 17854= 6 17855 17856`r--` 17857= 100 17858= 4 17859 17860`r-- ` 17861= 100 17862= 4 17863 17864`chmod 644 file.txt` means change the permissions to `rw- r-- r--`: simple! 17865 17866### setuid affects executables 17867`$ls -l /bin/ping` 17868`rws r-x r-x root root` 17869 17870(the s means ping always runs as oot) 17871 17872`setgid` does 3 different unrelated things for executables, directories, and regular files. 17873 17874person: unix why! 17875unix, cheerfully: it's a long story</div> 17876 </a> 17877 17878 <a href="/comics/disk-usage-1/"> 17879 <span class="title"> 17880 disk usage 17881 </span> 17882 <div class="keywords"></div> 17883 <div class="transcript">### `du` 17884tells you how much disk space files/directories take up 17885 17886`-s`: summary: total size of all files in a directory 17887`-h`: human readable sizes 17888 17889### `df` 17890tells you how much free space each partition has. `-h` for human-readable sizes 17891 17892| `Filesystem` | `Size` | `Used` | `Avail` | `Use%` | `Mounted on` | 17893|---------------|---------|---------|----------|---------|---------------| 17894| `/dev/sda3` | `18G` | `G` | `2.5G` | `86%` | `/` | 17895| `udev` | `483M` | `4.0K` | `483M` | `1%` | `/dev` | 17896| `tmpf`s | `99M` | `1.4M` | `97M` | `2%` | `/run` | 17897| `/dev/sda4` | `167G` | `157G` | `9.9G` | `95%` | `/home` | 17898 17899### `df -i` 17900instead of % disk fee, report how many inodes are used/fee on each partition 17901 17902happy little stick person: running out of inodes is VERY ANNOYING. You can't create new files! 17903 17904### `ncdu` 17905see what's using disk space in an interactive way 17906 17907| 17.5 GiB | [#####] | /music | 17908|---------- |--------- |---------- | 17909| 3.2 GiB | [## ] | /photos | 17910| 5.7 MiB | [ ] | /code | 17911| 2.0 MiB | [ ] | file.pdf | 17912 17913### `iostat` 17914get statistics about disk reads/writes 17915 17916`# iostat 5` (interval to report at)` 17917 17918| Device: | kB_read/s | kB_wrtn/s | 17919|--------- |----------- |----------- | 17920| sda | 2190.21 | 652.87 | 17921| sdb | 6.00 | 0.00 |</div> 17922 </a> 17923 17924 <a href="/comics/ps-1/"> 17925 <span class="title"> 17926 ps 17927 </span> 17928 <div class="keywords"></div> 17929 <div class="transcript">### `ps` 17930 17931`ps` shows which processes are running 17932 17933I usually run `ps` like this: 17934`$ ps aux` 17935 17936u means include username column 17937a+x together show all processes 17938 17939(`ps -ef` works too) 17940 17941### `w` 17942is for wide. `ps auxwww` will show all the command like args for each process 17943 17944### `e` 17945is for environment. `ps auxe` will show the environment vars! 17946 17947### wchan 17948you can choose which columns to show with `ps` (`ps -eo...`) 17949 17950One cool column is '`wchan`', which tells you the name of the kernel function if the process is sleeping. 17951 17952try it: 17953`$ ps -eo user,pid,wchan,cmd` 17954 17955### process state 17956Here's what the letters in `ps`'s STATE column mean: 17957 17958- `R`: running 17959- `S/D`: asleep 17960- `Z`: zombie 17961- `l`: multithreaded 17962- `+`: in the foreground 17963 17964### `f` 17965is for "forest" :) `ps auxf` will show you an ASCII art process tree! 17966 17967`pstree` can display a process tree, too. 17968 17969### `ps` has 3 different sets of command line arguments 17970(broken heart) 17971 179721. UNIX (1 dash) 179732. BSD (no dash) 179743. GNU (2 dashes) 17975 17976you can write monstrosities like: 17977`$ ps f -f` 17978 17979`f` is "forest" (BSD) 17980`-f` is "full format" (UNIX)</div> 17981 </a> 17982 17983 <a href="/comics/tc-1/">
17984 <span class="title"> 17985 tc 17986 </span> 17987 <div class="keywords"></div> 17988 <div class="transcript">### `tc` 17989is for "traffic control" 17990 17991humanoid traffic light, hand raised as if directing traffic: packets! stop/slow down/go the other way! 17992 17993### make your internet slow 17994``` 17995$ sudo tc qdisc add dev 17996 wlp3s0 root netem 17997 delay 500ms 17998``` 17999(delay packets by 500 ms) 18000 18001and fast again 18002``` 18003$ sudo tc qdisc del dev 18004 wlp3s0 root netem 18005``` 18006 18007### `netem` rules 18008`netem` ("network emulator") is a part of `tc` that lets you: 18009- drop 18010- duplicate 18011- delay 18012- corrupt 18013 18014packets. See the man page: 18015 18016`$ man netem` 18017 18018### make your brother's internet slow 18019Have a Linux router? You can configure `tc` on it to make your brother's internet slower than yours. 18020 18021google: "tc QoS" for a start. 18022 18023### show current `tc` settings 18024``` 18025$ tc qdisc show 18026$ tc class show dev DEV 18027$ tc filter show dev DEV 18028``` 18029### panel 6 18030 18031smiling stick figure with short curly hair: `tc` can do 10 million more things! this is just the beginning!</div> 18032 </a> 18033 18034 <a href="/comics/ip-1/"> 18035 <span class="title"> 18036 ip 18037 </span> 18038 <div class="keywords"></div> 18039 <div class="transcript">### `ip` 18040(Linux only) 18041lets you view + change network configuration 18042 18043`$ ip OBJECT COMMAND` 18044 18045(`OBJECT`: addr, link, neigh, etc. `COMMAND`: add, show, delete, etc.) 18046 18047### `ip addr list` 18048shows ip addresses of your devices. Look for something like this: 18049 18050``` 180512: eth0: 18052 link/ether 3c:97... 18053 inet 192.168.170/24 18054``` 18055 18056### `ip route list` 18057displays the route table. 18058 18059``` 18060default via 192.168.1.1 18061169.240.0.0/16 dev docker0 18062``` 18063(`192.168.1.1` is my router) 18064 18065to see all route tables: 18066`$ ip route list table all` 18067 18068### change your MAC address 18069good for cafes with time limits (little devil face) 18070 18071``` 18072$ ip link set wlan0 down 18073$ ip link set eth0 address 18074 3c:a9:f4:d1:00:32 18075$ ip link set wlan0 up 18076$ service network-manager 18077 restart 18078``` 18079(or whatever you use) 18080 18081### `ip link` 18082network devices! (like `eth0`) 18083 18084### `ip neigh` 18085view/edit the ARP table 18086 18087### `ip xfrm` 18088is for IPsec 18089 18090### `ip route get IP` 18091what route will packets with `$IP` take? 18092 18093### `--color` 18094pretty colourful output! 18095 18096### `--brief` 18097show a summary 18098</div> 18099 </a> 18100 18101 <a href="/comics/linux-system-calls/"> 18102 <span class="title"> 18103 linux system calls 18104 </span> 18105 <div class="keywords"></div> 18106 <div class="transcript">### The Linux kernel has code to do a lot of things 18107- read from a hard drive 18108- make network connections 18109- create new processes 18110- kill process 18111- change file permissions 18112- keyboard drivers 18113 18114### Your program doesn't know how to do those things 18115program, blithely: TCP? dude I have no idea how that works. 18116 18117program: NO, I do not know how the ext4 filesystem is implemented. I just want to read some files! 18118 18119### Programs ask Linux to do work using system calls 18120 18121program: please write to this file 18122(switch to running kernel code) 18123 18124Linux: done! I wrote 1097 bytes! 18125(program resumes) 18126 18127### Every program uses system calls 18128Python program: I use the 'open' syscall to open files 18129Java program: me too! 18130C program: me three! 18131 18132### And every system call has a number (e.g. chmod is 390 on x86.64) 18133so what's actually going on when you change a file's permissions is: 18134 18135program: run syscall #90 with these arguments 18136Linux: ok! 18137 18138### You can see which system calls a program is using with strace 18139`$ strace ls /tmp` 18140 18141will show you evey system call 'ls' uses! it's really fun! 18142 18143warning: strace has high overhead so don't run it on your production database 18144</div> 18145 </a> 18146 18147 <a href="/comics/how-diffie-hellman-key-exchange-works/"> 18148 <span class="title"> 18149 how diffie hellman key exchange works 18150 </span> 18151 <div class="keywords"></div> 18152 <div class="transcript">### diffie hellman key exchange 18153 18154is a system for establishing a secret key in the open. Illustration of two stick figures sending messages to each other 18155 18156person 1: ... 18157person 2: ... 18158 18159everyone can read these messages, but nobody knows their secret key! 18160 18161### diffie hellman key exchange requires a ~~~ magic function ~~~ 18162 18163f(s, a) = sâa. 18164 18165You put 2 numbers in to f and get a result (we'll call it sâa). There are two rules this function has to follow: 18166 181671. It's commutative: (sâa)âb is always the same as (sâb)âa 181682. It's hard to undo: if you know sâa and s, you can't easily "divide" to figure out what "a" was 18169 18170### Finding a magic function that works this way requires a lot of math... 18171 18172two examples: 18173 181741. elliptic curve multiplication (where sâx means "
18174add the point s to itself x times") 181752. modular arithmetic (where sâx = s^x mod q). 18176 18177But you don't need to understand the math to get the basic idea. 18178 18179### how diffie hellman works 18180 181811. Choose s (some cryptographers choose this and tell everyone "hey this is what we're using guys") 181822. Each person picks a random number. Left person picks a, right person picks b. 181833. "Multiply" s by the number and send it. 18184left person sends: sâa 18185right person sends: sâb 18186Nobody can figure out a and b because of Rule 2! 181874. "Multiply" the number the other person sent. 18188left person calculates: (sb)âa 18189right person calculates: (sâa)âb 18190These two numbers are the same because of Rule 1! 181915. We're done! (sâb)âa is the secret key!</div> 18192 </a> 18193 18194 <a href="/comics/test/"> 18195 <span class="title"> 18196 test 18197 </span> 18198 <div class="keywords"></div> 18199 <div class="transcript"></div> 18200 </a> 18201 18202 </div> 18203 </section> 18204 18205 18206<footer class="squiggle-top"> 18207 <div style="grid-area: links"> 18208 <h3> 18209 <a href="https://wizardzines.com">Wizard Zines</a> 18210 </h3> 18211 <ul> 18212 <li><a href="https://store.wizardzines.com/pages/faq">FAQ</a></li> 18213 <li><a href="https://wizardzines.com/newsletter">Newsletter</a></li> 18214 <li><a href="https://jvns.ca">Julia's blog</a></li> 18215 <li><a href="https://wizardzines.com/#free-zines">Free zines</a></li> 18216 <li><a href="https://wizardzines.com/#experiments">Experiments</a></li> 18217 <li><a href="https://wizardzines.com/comics">Comics</a></li> 18218 <li><a href="https://wizardzines.com/#posters">Posters</a></li> 18219 </ul> 18220 </div> 18221 18222 <div style="grid-area: links2"> 18223 <h3> 18224 <a href="https://store.wizardzines.com/">Store</a> 18225 </h3> 18226 <ul> 18227 <li><a href="https://wizardzines.com/print">Print directions</a></li> 18228 <li><a href="https://wizardzines.com/errors">Zine errors</a></li> 18229 <li><a href="https://wizardzines.com/education">Educators</a></li> 18230 <li><a href="https://store.wizardzines.com/pages/shipping-policy">Shipping 18231 policy</a></li> 18232 <li><a href="https://store.wizardzines.com/pages/privacy-policy">Privacy 18233 policy</a></li> 18234 <li><a href="https://wizardzines.com/accessibility">Accessibility</a> 18235 </li> 18236 </ul> 18237 </div> 18238 <div class="social-icons" style="grid-area: socials"> 18239 <a href="https://social.jvns.ca/@b0rk" 18240 aria-label="Mastodon" 18241 target="_blank"> 18242 <i class="fab fa-mastodon"></i> 18243 </a> 18244 <a href="https://bsky.app/profile/b0rk.jvns.ca" 18245 title="Bluesky" 18246 aria-label="Bluesky" 18247 class="bluesky" 18248 target="_blank"> 18249 <i class="fab fa-bluesky"></i> 18250 </a> 18251 <a href="https://www.instagram.com/wizardzines" 18252 title="Instagram" 18253 aria-label="Instagram" 18254 class="instagram" 18255 target="_blank"> 18256 <i class="fab fa-instagram"></i> 18257 </a> 18258 <a href="https://www.linkedin.com/company/wizardzines" 18259 title="LinkedIn" 18260 aria-label="LinkedIn" 18261 class="linkedin" 18262 target="_blank"> 18263 <i class="fab fa-linkedin"></i> 18264 </a> 18265 </div> 18266 18267 18268 <form style="grid-area: newsletter" 18269 class="subscribe" 18270 action="https://app.kit.com/forms/5339543/subscriptions" 18271 method="post" 18272 data-sv-form="5339543" 18273 data-uid="7f81cdc5df" 18274 data-format="inline" 18275 data-version="5" 18276 data-options="{"settings":{"after_subscribe":{"action":"message","success_message":"Success! Now check your email to confirm your subscription.","redirect_url":""},"analytics":{"google":null,"fathom":null,"facebook":null,"segment":null,"pinterest":null,"sparkloop":null,"googletagmanager":null},"modal":{"trigger":"timer","scroll_percentage":null,"timer":5,"devices":"all","show_once_every":15},"powered_by":{"show":true,"url":"
18276https://kit.com/features/forms?utm_campaign=poweredby&utm_content=form&utm_medium=referral&utm_source=dynamic"},"recaptcha":{"enabled":false},"return_visitor":{"action":"show","custom_content":""},"slide_in":{"display_in":"bottom_right","trigger":"timer","scroll_percentage":null,"timer":5,"devices":"all","show_once_every":15},"sticky_bar":{"display_in":"top","trigger":"timer","scroll_percentage":null,"timer":5,"devices":"all","show_once_every":15}},"version":"5"}" 18277 min-width="400 500 600 700 800 900"> 18278 <h3>Newsletter</h3> 18279 <div data-style="minimal"> 18280 <div data-element="fields" 18281 data-stacked="true" 18282 class="seva-fields formkit-fields"> 18283 <div class="formkit-field"><input class="formkit-input" 18284 name="email_address" 18285 style="color: rgb(0, 0, 0); border-color: rgb(0, 0, 0); border-radius: 10px; font-weight: 400;" 18286 aria-label="your email address" 18287 placeholder="your email address" 18288 required="" 18289 type="email"></div> 18290 <div class="formkit-field"> 18291 <div role="button" tabindex="0"> 18292 <fieldset data-group="checkboxes" 18293 class="formkit-9815" 18294 type="Custom" 18295 order="1" 18296 save_as="Tag" 18297 group="field"> 18298 <div class="formkit-checkboxes" 18299 data-element="tags-checkboxes" 18300 data-group="checkbox"><input class="formkit-checkbox" 18301 id="tag-660783703-3963798" 18302 checked 18303 type="checkbox" 18304 name="tags[]" 18305 value="3963798"><label for="tag-660783703-3963798">new 18306 zine announcements</label></div> 18307 <div class="formkit-checkboxes" 18308 data-element="tags-checkboxes" 18309 data-group="checkbox"><input class="formkit-checkbox" 18310 id="tag-660783703-3963799" 18311 type="checkbox" 18312 name="tags[]" 18313 value="3963799"><label for="tag-660783703-3963799">new 18314 comics as they're posted</label></div> 18315 <div class="formkit-checkboxes" 18316 data-element="tags-checkboxes" 18317 data-group="checkbox"><input class="formkit-checkbox" 18318 id="tag-660783703-3963801" 18319 type="checkbox" 18320 name="tags[]" 18321 value="3963801"><label for="tag-660783703-3963801">a 18322 comic from the archives every saturday</label> 18323 </div> 18324 </fieldset> 18325 </div> 18326 </div><button data-element="submit" 18327 class="formkit-submit formkit-submit"> 18328 subscribe! 18329 </button> 18330 </div> 18331 </div> 18332 </form> 18333</footer> 18334 18335</body> 18336 18337</html>
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.