Collecting the best books mentioned on hacker news, reddit and other places
Coding the Matrix: Linear Algebra through Applications to Computer Science 
A hands on introduction to both Python and Linear Algebra using real world cases (ex. you are given a high res image, make a low res version to put on your website so that it could load more quickly).
"Coding the Matrix" : https://www.amazon.com/Coding-Matrix-Algebra-Applications-Co...
The texts that I hate least (LA and I have a long and rocky relationship):
- Coding the Matrix, Klein https://www.amazon.com/Coding-Matrix-Algebra-Applications-Co... This has a strong emphasis on LA's utility in CS, and includes concepts outside traditional LA that enrich the narrative.
- Intro to Linear Algebra, Strang https://www.amazon.com/Introduction-Linear-Algebra-Gilbert-S... Strang approaches LA from a practical less-theoretical angle, which makes it very sensible if you're an engineer but may not be as suitable if you're a mathematician.
- Linear Algebra, A Modern Intro, Poole https://www.amazon.com/Linear-Algebra-Introduction-Available... This is a solid text that has worked out most of its bugs over the editions.
- Linear Algebra and its Applications, Lay https://www.amazon.com/Linear-Algebra-Applications-Updated-C... Like Poole, this is also a solid and long running text.
The books by Klein and Strang also benefit from free videos of those courses that are available from Coursera/BrownU and MIT OCW. Klein's is also available on the Kindle.
Starts on Page 2 at the bottom and moves up. Half of this course was taught on Coursera in 2013.
Book on Amazon, Kindle version $3: https://www.amazon.com/Coding-Matrix-Algebra-Applications-Co...
While I did take courses in probability, linear algebra, and lots of calculus, until recently, I forgot most of the probability and all of the linear algebra I learned in school. As for calculus, I only remembered how to take basic derivatives. In any case, I've been spending the past month brushing up on my linear algebra and probability, and it's been a struggle, but now that I'm motivated and under no time pressure to relearn the material, I find it way more fascinating than I did in college. In fact, I skipped tons of my linear algebra classes because I thought the subject was dry and dull. I also rushed through my probability and stats homework just so I could get a good grade on them. I think if you're motivated, and you can do basic math, you should be able to educate yourself in calculus, probability, and linear algebra. It'll be a struggle, but with motivation, you'll be able to pick up the concepts.
for probability and stats: https://www.amazon.com/Introduction-Probability-2nd-Dimitri-...
for linear algebra: https://www.amazon.com/Coding-Matrix-Algebra-Applications-Co...
this was my college calculus textbook: https://www.amazon.com/Calculus-7th-James-Stewart/dp/0538497.... I can't comment if it was good or not because by college, I had taken calculus twice so it was all a refresher
best of luck! You sound educated enough (yes, I'm judging from the couple sentences you wrote) that I think you won't have any problems acquiring math knowledge with persistence.
My favorite LA books are Linear Algebra by Friedberg/Insel which is a combination of Axler style book with more computation oriented one (Terry Tao has a set of lectures based off this book). Another one I like is Modern Intro To LA by Henry Ricardo which implicitly introduced me to Replacement theorem which is really overlooked in a ton of LA books. Again, this book's a rigorous mixture of both theory and computation done very well. If you've never seen higher level math before, there's Linear Algebra: Gateway to Mathematics by Robert Messer. It has tons of commentary about elementary set theory and proof techniques along the way. Whenever someone mentions Axler's book, someone else brings up Treil's book. But there's a third one in the same league/group which is Linear Algebra: An Introduction to Abstract Mathematics by Robert Valenza. Other favorites are Coding the Matrix by Philip Klein for Python aficionados and Linear Algebra Through Geometry by Banchoff/Wermer for those who like geometry.
If you are way beyond all this, you can still pick up new things from Advanced Linear Algebra by Steven Roman.