Poor man’s linear algebra textbook

Posted on March 2, 2011

I keep learning new bits of linear algebra all the time, but I’m always hurting for a useful reference.  I probably should get a good book (which?), but in the meantime I’m collecting several nice online sources that ML researchers seem to often recommend: The Matrix Cookbook, plus a few more tutorial/introductory pieces, aimed at an intermediate-ish level.

Main reference:

Tutorials/introductions:

After studying for this last stats/ML midterm, I’ve now printed them out and stuck them in a binder.  A poor man’s linear algebra textbook.

I’d love to learn of more or different stuff out there.  (There are always the appendixes of linear algebra reviews in Hastie et al. ESL and Boyd+Vandenberghe CvxOpt, but I’ve always found them a little too small for usefulness+understanding.)

Update May 2015: tweaked creditation for the CS229/CMU/Kolter review, fixed some dead links.

This entry was posted in Uncategorized. Bookmark the permalink.

9 Responses to Poor man’s linear algebra textbook

  1. Joanna says:
    March 2, 2011 at 11:12 pm

    I *loved* EE263, which has a course textbook (not a real book, but rather the sort of thing that is printed at Kinko’s). But perhaps you have equivalent or more advanced material in whatever else you have from Boyd.

  2. brendano says:
    March 2, 2011 at 11:21 pm

    You mean this? http://www.stanford.edu/class/ee263/support.html

    I like “crimes against matrices.”

    This definitely looks like more of the right sort of thing than his convex optimization book/course…

  3. Fred Mailhot says:
    March 3, 2011 at 2:09 am

    Sam Roweis (RIP) had some useful stuff:

    http://www.cs.nyu.edu/~roweis/notes.html

  4. Dean Eckles says:
    March 3, 2011 at 4:57 am

    I liked Trefethen and Bau’s Numerical Linear Algebra. I used it for Math 104 at Stanford, though we only covered part of the book. Doesn’t have everything, but most of what seems to matter for statistics. I like the treatment of the SVD and QR factorization.

    The first 5 lectures are here for sampling: http://www.amath.washington.edu/courses/584-autumn-2007/handouts/

  5. Patrick says:
    March 3, 2011 at 5:05 pm

    I’ve gotten a lot of mileage out of Golub and Van Loan (mostly the perturbation results). I’ve heard good things about Horn and Johnson.

  6. Bob Carpenter says:
    March 3, 2011 at 10:52 pm

    You could do a lot worse than Gilbert Strang’s classic text. His lectures from 1999 are online, too:

    http://ocw.mit.edu/courses/mathematics/18-06-linear-algebra-spring-2010/video-lectures/

    I have a friend who taught from Trefethen and Bau III last semester. He thought the book was a mixed blessing, being equally full of insight and typos.

  7. brendano says:
    March 3, 2011 at 10:59 pm

    Strang has multiple votes.

  8. Dean Eckles says:
    March 4, 2011 at 1:03 am

    I’ve got Strang’s text also. I find it a bit too quirky and conversational to be ideal for reference. But it does have a broader range than Trefethen & Bau.

    Would be interested to know what some of the typos in T&B are… I don’t doubt there are, but would like to fix them in my copy. I can’t find any errata online.