Theoretical neuroscience and machine learning are both research areas that draw from a wide variety of fields (biology/maths/physics/engineering/computer science/psychology to name a few), and people join the field with many academic backgrounds. This set of materials is my ongoing list of useful background (and occasionally foreground) resources. It is very much an ongoing piece of work, if you have any comments or suggestions please email dorrellwec "at" gmail "dot" com. They are grouped by rough subject area.

There are many good free online courses in machine learning, I list three ones that either I took and enjoyed, or look good. They cover the same material, so one is enough. The usual set of pre-requisites is good basic knowledge of probability/statistics, calculus, and linear algebra. If you feel you do not have that, then I would recommend learning those first. They all have coding components, since machine learning is about using computers, so again, if you haven't learnt to code yet I would recommend doing that first.

- Harvard's CS181: I took this course and really enjoyed it, good for people with the maths background. The webpage freely hosts the 2021 version of the course, complete with lectute videos, slide decks, ipad notes, and a full (and fairly high quality) textbook of the course!
- Andrew Ng's Coursera Course: This is a pretty famous course taken by loads of people with very high reviews. It's almost certainly excellent, though I can't vouch for it myself.
- Oxford Machine Learning Course: A course I've been told is good, complete with slides, videos, practicals and problem sets. It's a little old (2015), but I can't find a more up to date version of the course freely available.

- Information Theory, Inference, and Learning Algorithm's: a fabulous and incredibly insightful textbook by David MacKay (also with free pdf). Full of novel ideas and still surprisingly relevant for machine learning despite being nearly 20 years old. It won't fill you in on the cutting edge ML, but it will make you understand it better once you're there.
- 3Blue1Brown Neural Network Videos: A youtube series explaining neural networks, it's really great, like all of the videos on this channel.
- Pattern Recognition and Machine Learning: Christopher Bishop's classic text for learning machine learning, it does an excellent job of introducing all the material and there's a free pdf at the link. It's less insightful than MacKay, but I used it as a reference while taking one of the courses listed above, in that role it excelled.
- Sutton and Barto's RL Textbook is the standard to learn reinforcement learning from, and for good reason. It is an excellent well presented book, and the pdf is free online, what more could you want?

Both machine learning and theoretical neurscience are kind of fields of applied maths, and require good maths skills. In my opinion, the pre-requisites are a working knowledge of linear algebra, probability & statistics, and calculus/differential equations. All of these are covered in thousands of courses, I list below a couple of particularly good or comprehensive options:

- MIT opencourseware is a website that hosts loads of the courses taught at MIT, often complete with notes and videos of lectures. You can probably
find all sorts there, some examples:
- 18.01,18.02,18.03: improve your calculus
- 18.05: statistics

- The youtube channel 3Blue1Brown has great content on calculus, linear algebra, fourier series, complex numbers, linear algebra, differential equations and more!
- I learnt linear Algebra from Gilbert Strang's famous book, I really recommend it.

Beyond these basics that everyone should definitely know, I've used a few good resources to extend my maths knowledge when needed, and would recommend them to others:

- Group Theory: Visual Group Theory by Nathan Carter. It starts very (I mean very) basic but by the end it has covered some serious material in a very intuitive way. I accessed it through my university library, so have no online pdf.
- Lie Theory: Naive Lie Theory by John Stillwell was a nice introduction. I didn't use an online pdf.
- Topology: Dexter Chua has notes for loads of the maths courses taught at Cambridge. The courses are, as far as I can tell, thorough, and his notes are good. I studied the basic topology course and enjoyed it.
- Complex Analysis: Visual Complex Analysis by Tristan Needham looks excellent. I admit, I got distracted before my attempt to read this book took off (covid suddenly forced me to uproot), but it looks excellent, and receives rave reviews.
- Bayesian Non-parametrcs:Peter Orbanz's notes are supremely clear.

There's a special set of resources that aim to train maths for neuroscientists. I don't know how good any of these are personally, but they've been recommended to me, may they prove useful to you:

- Mathematical Tools for Neural and Cognitive Science: a course at NYU by Eero Simoncelli, all lectures on youtube.
- Mathematical Tools for Neuroscience: a similar Harvard course that doesn't assume too much and has a good-looking syllabus.
- Neuromatch Academy: is a free online comp neuro summer school that I've heard is excellent. I think you can either access the resources through the link or attend next time they run it!

I've been asked how to learn coding before, and I don't quite know what to answer. I, like most of the people I've met, basically learnt by trying, failing, and googling the answer on stack overflow. I recommend this route to others. Come up with some fun projects, preferably with a friend, and try coding them up. A quick prime number generator, implement an RL agent that learns to navigate a gird world, make a simple game, that kind of thing. And I'd recommend learning python, the coding language most used in science. That said, there are plenty of online resources that might help, I list a couple:

- CS50: Harvard run an intro to computer science that is free and online, probably a good start. I like the approach they take because it focuses on concepts rather than syntax.
- I have a friend that read "Learn Python the Hard Way" and said it was very good.

I've found it both very easy and very hard to learn neuroscience, having spent my undergrad as a physicist. To some extent there's no big theories like relativity that are universally assumed knowledge for astronmers, so there's less of a checklist of things you need to learn before you can even have a useful conversation. And I've found that if you have a specific project or question at hand then you can usually learn the requisite material from a few papers. That said, there are some core biology and neuroscience ideas that you should know, they are shocking really, and everyone should know them for culture. And once you try and start forming some cohesive understanding of neuroscience, like I was used to doing in physics, you're somewhat stumped by the fact that it appears not to exist.

In terms of basic biology, I'm talking the basics of life: DNA, cells, respiration, if you don't know this stuff you should, it's mind-boggling. I learnt it in lectures and was boggled. I suspect an author like Bill Bryson would do a good job (he has recently released a book on the human body, and the short history of nearly everything I think covers biochemistry). Beyond that I'm not sure.

Then for neuroscience I again am unsure. There's some basics you should know, about how neurons work, how the basics of information transmission could operate within them.

And then you get beyond the basics that we definitely know, and you want to understand how the brain works. Well then I'm stumped, the best I can advise is Kandel's Principles of Neural Science which is full of information and readable.

One thing I will say is that I DO NOT recommend Dayan & Abbot's theoretical neuroscience textbook. I didn't enjoy it, and I suspect it is only popular because it is the only such textbook in existence.

Other people have written similar lists to me, I recommend refering to them too: