Youtube playlist for the course:
https://www.youtube.com/playlist?list=PLdPQZLMHRjDK8ZbLIcq1Q...
Materials on Github:
I took this course 3 years ago. I found it fast-moving, and it focused a lot more on applications than fundamentals, which meant it was more wide than it was deep. This didn't turn out so well when I decided to study ML later and needed stronger linear algebra fundamentals, but it was a fun course. There were a couple interesting course projects, one of which was using linear algebra to balance a (simulated) 2D robot.
No one, and let me repeat that, no one "gets" linear algebra, differential equations, or frequency domain on the first pass. It takes years to absorb and multiple passes.
See:
Bruner / Spiral Curriculum.
Ebbinghaus / Spacing effect
Hattie / Deep-surface-transfer learning
Chunking ("How People Learn" has a good copy on this)
Etc.
The way you do this is you take a course, and then you take more courses. After a few years, it all connects and makes sense. The first course, I find, is often best short, simplified, and applied. Once you get through that, you can go deeper.
Different angles are nice too. For linear algebra:
- Quantum computing
- Statistics and probability
- Machine learning
- Control theory
- Image processing
- Abstract algebra / groups / etc.
- Computer graphics
All come to mind.
On a mile-high level, this course seems ideal for a first pass. On a detailed level, I'm confused by some licensing issues.
Not with the way it is taught. But if the course structure is changed slightly to have reinforcement of early concepts woven through the course, people learn much better.
At least that was my experience when I taught it. See https://bentilly.blogspot.com/2009/09/teaching-linear-algebr... for more detail on my experience.
Where do you feel the gaps were for what you needed for ML? Downthread, Jesse Grizzle notes they've added some stuff in 2023 (it's on Github I think?) to support an ML class.
What would you recommend for building a strong linear algebra foundation?
Also a big fan of Strang. "Linear algebra and its applications" has problem sets with solutions for odd number questions.
Would highly recommend https://mathacademy.com/courses/linear-algebra or https://mathacademy.com/courses/mathematics-for-machine-lear...
I originally spent time working through practice problems from one of Strang's books, now really appreciate how systematic math academy is in assessing, building a custom curriculum, then doing spaced repetition.
i don't really care how many people i respect liked it, i have to be honest, i hated strang's "linear algebra and its applications."
there's a strang text on computational science that was much more my speed (less of the baby talk and repetitive manual arithmetic exercises) and i think that some of the revisions that came later (+ "learning with data") were better.
i did not find doing endless exercises of gaussian elimination or qr factorization by hand on small matrices to be all that enlightening.
this michigan course looks awesome!
I took 18.085 (applied linear algebra) as a grad student at MIT. The best taught math course I've ever taken. Strang is a fantastic teacher.
After working with math academy, any form of video learning seems so inefficient. I think people lose a lot of time watching these videos thinking that they are learning without applying anything by themselves.
UMich has a couple other linear algebra courses that might be better for that: MATH 214, MATH 217 are the numbers if I remember correctly. 217 is known for having a high workload and greater rigor, but some say it's worth it even for non-Math majors.
In terms of books, I would say Linear Algebra Done Right. The book requires some background to understand efficient. But, once you have some background, it is very good for having a systematic and rigorous understanding of Linear Algebra theory
LADR is the SICP of linear algebra.
If you can handle it, fabulous. If not, you're really in deep doo-doo. There did not seem to be a half-way to me. Astounding exercises, and also some are astoundingly hard.
Anything by Gilbert Strang
Man this would have been nice when I was in school.
For some reason linear algebra still isn't part of standard Mechanical Engineering course load (Calc 1, 2, 3, DiffEq) which made life extremely difficult in some of the later classes. I remember spending weeks brute forcing a lot of things that would have been trivial with a little bit of matrix math.
I took a superficially similar class as a 400 level elective but it assumed everyone already knew linear algebra going in, and it was a disaster.
> For some reason linear algebra still isn't part of standard Mechanical Engineering course load (Calc 1, 2, 3, DiffEq)
Wow. In my undergrad all engineering majors had to take linear algebra (calc 3 was optional for computer engineering).
Maybe some schools do but it's baffling to me its not a universal requirement. It'd be dramatically more useful than Calc3 for most engineers.
Michigan doesn't seem to require it as the College of Engineering core classes or as part of the BSME (checked because they're who this course is through):
https://me.engin.umich.edu/academics/undergrad/handbook/bach...
And my alma mater has a very similar progression.
First year linear algebra is a requirement in Canada for national accreditation of any engineering program.
Linear Algebra was a first year subject for me as well. I studied Engineering in Australia.
The Robotics concentration requires Linear (Matrix) Algebra.
And some upper level courses have a prerequisite.
But indeed it does seem a avoidable for many MechE majors.
Funny thread about UM engineering students avoiding taking UM math classes. https://www.reddit.com/r/uofm/comments/15w18gv/reminder_you_...
This would have helped me get an actual CS degree 25 years ago instead of CS-lite (networking & server admin).
It's not that I can't do calculus, I took it in high school, and then again in my first go-round in CS. It's that I hate calculus. Not the subject itself, just the grinding away at problem sets.
I did a refresher in pre-calc, calc I, calc II & discrete mathematics during COVID at the local community college (was planning to finish the few credits I need for an actual CS BS) & I started calc III twice (but dropped both times). I even got a 4.0 on my first calc III exam (and this was an in-person class, so no online shenanigans).
I just have some kind of weird aversion to 3 dimensional calculus. I have convinced myself that I'm simply not smart enough to actually do the work. I understand it, I just get clammy with it.
Truth be told, maths are my kryptonite. Despite working with numbers all day every day for 30+ years, and writing a lot of software over the years (and not just CRUD, but games of all things), I am absolutely ashamed that I just can't seem to grok math with any rigor.
I have all the Stewart textbooks on my shelf, many textbooks from libgen (ones I've seen recommended on HN from people who went to much better universities than I attended), and I even work through problems a few hours per week. I just can't seem to make that leap from a guy who's "good with numbers" (from a layperson's perspective) to a guy who's good at math.
Maybe I need to break open one of my physics textbooks and actually use the calculus in an applied context and that will break whatever mental barrier I have (I've even watched all of the 3 blue 1 brown videos, countless youtube lectures, etc).
A few hours per week simply isn't enough, the best success I've had studying was 6 hours a day, resting for 3 (leisure activity), "working" for 3 (class, commute, chores, related reading) and sleeping 12 hours.
In books like Stewart, staring at a theorem until you can write it's proof should trivialize most problems in the book.
If a method for solving a particular problem is too difficult for you maybe consider researching and/or inventing some new method to solve these problem. People created these methods in the first place because earlier methods were too tricky
Or just focus on work that doesn't require hundreds of hours to gain proficiency. As long as you have time every day to stop, think, and come up with an idea that solves a problem you won't become intellectually unfit.
There are deep concepts behind multi variable cal. But if you just want to pass the course, memorizing problem shapes through practice will get you through calc 3.
Do all the homework problems check the answer in the back of the book. You’ll make it.
Maybe stop torturing yourself for a bit. Sounds miserable.
We had the basic Linear Algebra in high school
Currently watching:
https://ocw.mit.edu/courses/6-042j-mathematics-for-computer-...
and that assumption seems to be there as well, so very glad of the posting of the Youtube links elsethread.
Same... I didn't have to take Linear Algebra in ChemE undergrad. DiffEq had a little bit of LA... and ChemE had few classes where bits of pieces of LA where introduced and applied.
Graduate school definitely made up for lost time... LA was very front and center in the applied math courses.
Yes back in 2005 when I first went to undergrad as a mech engineering major, linear algebra was not a requirement. Our mechanics professors were highly irritated by this.
I don't think this has changed much (but absolutely should). I've watched in real time as Micron representatives reject mechanical engineers and prefer résumés from industrial engineers for design roles due to their superior grasp on linear algebra and statistics. I'm paraphrasing but "it's easier to teach an IE how to do FEA than it is to teach a mechanical engineer DOE and Weibull analysis".
What do these acronyms mean?—IE, FEA, DOE?
IE == Industrial Engineering: broad, but generally "Engineering of Systems" instead of a physical product. Laying out factories, setting up supply chains, etc. It's morphed a little bit from the original field so the name isn't super accurate.
FEA == Finite Element Analysis: advanced method of predicting the strength of a product via numerical simulation.
DOE == Design of Experiments: evaluation of how the outputs of a system change as you vary the inputs. At a high level, you build model of the system, then vary all the inputs through their entire range and to build a response surface of the output.
Yeah, stats is the other major deficiency in the course load. I think one course is required but its basically high school level "check out these normal distributions, 68-95-99.7, here's a Z-score, see you later".
Thankfully the companies I've worked for have done a really good job with advanced stats training.
Why can't professors set LA as a prereq for their courses?
Or use it in their courses and earn students that they need to learn it so succeed?
For folks interested in 101 on linear algebra - I highly recommend book "Linear Algebra: Theory, Intuition, Code" by Mike X Cohen.
After trying a couple of courses and books, I liked it the most because it gives a pretty deep overview of the concepts, alongside the numpy and matlab code, which I found refreshing.
It's has good amount of proofs and has sections designed to build your intuition, which I really appreciated.
Chapter 13 of the textbook was added in January 2022. It covers separating hyperplanes, signed distance to a hyperplane, Max-margin Classifiers, a remark on Soft Margin Classifiers, and the Orthogonal Projection Operator. The additional material was added to support EECS 445, Machine Learning at Michigan.
This is (one of?) the authors of the course, for what it's worth. Welcome to HN! Pelt him with questions, everybody. :)
Chapter 13 of the textbook was added in January 2022. It covers separating hyperplanes, signed distance to a hyperplane, Max-margin Classifiers, a remark on Soft Margin Classifiers, and the Orthogonal Projection Operator. The material was added to support EECS 445, Machine Learning at Michigan.
I like Lay, it's one of the few math books anyone can read cover to cover, prove every statement and solving every problem, with no experience. It's like the Thomas' calculus equivalent linear algebra. If you do the work you'll get an easy A and will have built a great foundation for further engineering or theoretical study.
What's the best online credential for doing linear algebra? I like to do some self-studying but also, I'd like some form of "evidence" that I actually know my stuff and don't have t constantly explain that I do
I can't personally vouch for the program as I have not attended, but the University of Illinois offers quite a few mathematics courses online geared toward high school students, distance learners, and those preparing for grad school.
It is self-paced, so may not be what you're looking for, and it is expensive ($1250 if you have a BS already), but I seriously considered going this route before deciding to save big $$ and attend the local community college (which was actually a decent decision).
Program link: https://netmath.illinois.edu/
They offer 2 linear algebra courses, Math 257, which is Linear Algebra with Computer Applications (likely the "easy" applied version) and Math 416, Abstract Linear Algebra. Some of these Netmath courses do not have online lectures, but the Abstract LA course has video lectures from 2016.
From their site: "Math 416 is a rigorous, abstract treatment of linear algebra. Topics to be covered include vector spaces, linear transformations, eigenvalues and eigenvectors, diagonalizability, and inner product spaces. The course concludes with a brief introduction to the theory of canonical forms for matrices and linear transformations."
When I was investigating what to do in order to solidify my math credentials (still a work in progress), I knew UofI was a good school, and figured credit in one of their courses (online or not) would not be a terrible investment. At a bare minimum it wouldn't be belittled or untrusted like other online certificates might.
Plus the credit should transfer anywhere, if that's important.
> Math 416 is a rigorous, abstract treatment of linear algebra. Topics to be covered include vector spaces, linear transformations, eigenvalues and eigenvectors, diagonalizability, and inner product spaces. The course concludes with a brief introduction to the theory of canonical forms for matrices and linear transformations
Just be warned that this is literally the graduate level linear algebra course taken by mathematics majors. If you are looking for applications, this might not be it. On the other hand, if you are looking for a deep understanding of the fundamentals - I would say you found it.
Who are you trying to prove this to?
Probably trying to transition from a SWE role to a ML one
MATH 214 (intro to Linear) was the least enjoyable class during my undergraduate at Umich. This seems like a better intro.
For engineering we had to pick either multivariate calculus or linear algebra for more upper level math courses. I picked multivariate, and I'll say it was also my least enjoyable there. I look back wondering what would have gone different if I picked linear algebra instead, but who knows, maybe I'd have just as blech of an experience with that. Lot of great classes in the EECS department though.
> Lot of great classes in the EECS department though.
Couldn’t agree more, Jack! Great times during 482… tranquil compared to the 470 slog that started immediately after every night :)
I totally remember 482 (Operating Systems for those reading) being really interesting. Story I remember is one of the final projects and dealing with locks in C++ world where I'd get close to full solution, but some errors from the locks, then I'd make a change and suddenly those previous failing tests passed but new ones failed. I didn't realize that could happen.
Great times. And I really liked how we did it all in C++ (other than computer vision 442 that was in matlab) rather than Python which some places do. Having that lower level understanding of languages in school makes understanding code so much easier, and something I didn't have to learn on my own.
MATH 217 was one of my favorites! Ive heard that the math department can be a little unenthusiastic about the non-major courses, but overall it’s a really welcoming place in my experience.
I love Linear Algebra. I took it in college almost 20 years ago and I still use it everyday. The higher level maths almost broke me academically. And it was a course in LA that really kept my head in the game. Even now, when I'm talking to students I try and encourage them to take the class if it's available.
For me LA was spread across several courses (I was at Michigan in engineering too), and I never got enough internalization of when it was useful from these, sadly. It definitely seemed more useful that a lot of the higher level maths, like you imply.