I keep learning new bits of linear algebra all the time, but I’m always hurting for a useful reference. I probably should get a good book (which?), but in the meantime I’m collecting several nice online sources that ML researchers seem to often recommend: The Matrix Cookbook, plus a few more tutorial/introductory pieces, aimed at an intermediate-ish level.
- The Matrix Cookbook – 71 pages of identities and such. This seems to be really popular.
- Zico Kolter’s linear algebra review and reference [link#2]- it seems to introduce all the essentials and has very nice visual intutions for some things. (May 2015 update:) There’s now a nice video course too to go with it. (26 pages)
- Minka’s Old and New Matrix Algebra Useful for Statistics – has a great part on how to do derivatives. (19 pages)
- MacKay’s The Humble Gaussian – OK, not really pure linear algebra anymore, but quite enlightening. (12 pages)
After studying for this last stats/ML midterm, I’ve now printed them out and stuck them in a binder. A poor man’s linear algebra textbook.
I’d love to learn of more or different stuff out there. (There are always the appendixes of linear algebra reviews in Hastie et al. ESL and Boyd+Vandenberghe CvxOpt, but I’ve always found them a little too small for usefulness+understanding.)
Update May 2015: tweaked creditation for the CS229/CMU/Kolter review, fixed some dead links.