ByteRot: Rise of the Scientific Programmer

http://byterot.blogspot.nl/2015/01/future-of-programming-rise-of-the-scientific-developer-bigdata-datascience-machine-learning-and-fall-of-the-craftsman.html

Python is a very popular language in the academic space. It is easy to pick up works everywhere and has some functional aspects making it terse. But that is not all: it sits on top of a huge wealth of scientific libraries and it can talk to Java and C as well. Industry innovations have started to come straight from the Universities. From the early 2000s where the academia seemed completely irrelevant to now where it leads the innovation.PySpark has come fully from the heart of Berkeley's University. Many of the contributors to Hadoop code and its wide ecosystem are in the academia.

We are now in need of people who can scientifically argue about algorithms and data (is coding anything but code+data?) and most of them could implement an algorithm given the paper or mathematical notation. And guess what, this is the trend for jobs with "Machine Learning":

Trend of jobs containing "Machine Learning" - Source: ITJobsWatch

And this is really not just Hadoop. According to the source above Machine learning jobs have had 41% rise from 2013 to 2014 while hadoop jobs had only 16%.

This Deep Learning thing is real. It is already here. All those existing algorithms need to be polished and integrated with the new concepts and some will be just replaced. If you can give interactions of a person with a site to a deep network, it can predict with a high confidence whether they are gonna buy, leave or indecisive. It can find patterns in diseases that we as humans cannot. This is what we were waiting for (and we were afraid of?). Machine intelligence is here.

'Knowing and Doing' blog

Excellent Computer Science blog: http://www.cs.uni.edu/~wallingf/blog

Interesting articles include:

Thinking in Types, and Good Design

Students who become accustomed to programming in languages like Python and Ruby often become accustomed to using untyped lists, arrays, hashes, and tuples as their go-to collections. They are oh, so, handy, often the quickest route to a program that works on the small examples at hand. But those very handy data structures promote sloppy design, or at least enable it; they make it easy not to see very basic objects living in the code.
Who needs a Game class when a Python list or Ruby array works out of the box? I'll tell you: you do, as soon as you try to almost anything else in your program. Otherwise, you begin working around the generality of the list or array, writing code to handle special cases really aren't special cases at all. They are simply unbundled objects running wild in the program.
Good design is good design. Most of the features of a good design transcend any particular programming style or language.

Code Duplication as a Hint to Think Differently

When defining a program to process an inductively-defined data type, the structure of the program should follow the structure of the data.

This guideline helps many programmers begin to write recursive programs in a functional style, rather than an imperative style.

Take Small Steps

If a CS major learns only one habit of professional practice in four years, it should be:

Take small steps.

A corollary:

If things aren't working, take smaller steps.

I once heard Kent Beck say something similar, in the context of TDD and XP. When my colleague Mark Jacobson works with students who are struggling, he uses a similar mantra: Solve a simpler problem. As Dr. Nick notes, students and professionals alike should scale the step size according to their level of knowledge or their confidence about the problem.

Software Design is a Bet on a Particular Future

This truth is expressed nicely by Reginald Braithwaite:

Software design is the act of making bets about the future. A well-designed program is a bet on what will change in the future, and what will not change. And a well-designed program communicates the nature of that bet by being relatively flexible about things that the designers think are most likely to change, and being relatively inflexible about the things the designers think are least likely to change.

That's what refactoring is all about, of course. Sometimes, a particular guess turns out to be wrong. We have the wrong factors, the wrong components, for adding a new feature. So we change the shape of the code -- we factor it into different components -- to reflect our new best understanding of the future. Then we move on.
Sometimes, though, there are forces that make more desirable a relatively monolithic piece of code (or, as Braithwaite points out, a system decomposed into relatively less flexible components). In these cases, we need to defactor, to use Braithwaite's term: we recombine some or all of the parts to create a new design.
Predicting the future is hard, even for experienced programmers. One of the goals of agile design is to not think too far ahead, because that means committing to a future too far removed from what we already know to be true about our program.