“Learning From Data” by Yaser Abu-Mostafa (Caltech) on edX.org

edx-logo-headerTo deepen my knowledge about Machine Learning I decided last year to attend “Learning From Data” on edX. This online course was designed by Yaser Abu-Mostafa – a renowned expert on the subject and professor of Electrical Engineering and Computer Science at California Institute of Technology (Caltech). I can say without the slightest hesitation that this course was a wonderful intellectual experience. Prof. Abu-Mostafa conceived the course so skilfully that it was as much a joy to attend, as it was challenging. And this finding couldn’t be further from a naturalness, especially given that the syllabus took a path through quite theoretical terrain.

A pedagogical masterpiece

LFDedxThe course was comprised of two weekly video lectures – each about one and a half hour in length. Every lecture started out with a quick revision and ended with a QnA. The weekly course material founded the basis of the assignments which could not have been better crafted. Most of the exercises expected from the student to implement a certain algorithm – f.x. Linear Regression, Perceptrion, Support Vector Machine – in any language and have it evaluate a specific problem which would lead to a figure. For example, counting the number of on average necessary iterations to reduce the in-sample-error to a specified threshold. ┬áThis figure would have to be chosen from several offered options. I was very much impressed with the precision of the wording of the exercises which were often far from trivial. I invested easily up to ten hours on solving a full weekly assignment. Different from most courses I attended and do attend was that you could only answer every question once. Quite strict, but in my opinion pedagogically very efficient because it forces you to think twice as thorough. In my opinion the exercise design is absolutely essential for the value of any course – no matter of MOOC or offline at a university – the effort that was put into the lectures needs to be doubled for the exercises in my opinion.

Extraordinarily valuable was the discussion board. Regarding quality of posts, questions, answers, comments and ideas so far the highest level I experienced on a MOOC course.

The only disappointment in the end was that – because “the system isn’t ready yet” – no graded certificates were issued. Only an ungraded certificate depending on exceeding a threshold score to ensure one of the quintessential principles of ML … “no free lunch”. A strict grading culture is the backbone of any serious learning experience because especially MOOCs depend on enabling the student to assess her/his progress objectively and prevent the course from turning into essentially nothing else than infotainment.

“No free lunch”

The “no free lunch” sound bite originated from the fundamental theory of VC-dimensions which says that in terms of generalization, i.e. diminishing the infamous out-of-sample error, cannot be gained by data snooping and afterwards adjusting the algorithm without paying for it – data is the currency.