2015: Thomas W. Dinsmore: Predictions for Big Analytics

2015: Predictions for Big Analytics

Apache Spark usage will explode.
Analytics in the cloud will take off.
Python will continue to gain on R as the preferred open source analytics platform.
H2O will continue to win respect and customers in the Big Analytics market.
SAS customers will continue to seek alternatives.




Bad 2014 predictions!

(2) “Co-location” will be the latest buzzword.
Few people use the word “co-location”, but thanks to YARN, vendors like SAS and Skytree are now able to honestly position their products as running “inside” Hadoop. YARN has changed the landscape for analytics in Hadoop, so that products that interface through MapReduce are obsolete.
(3) Graph engines will be hot.
Graph engines did not take off in 2014. Development on Apache Giraph has flatlined, and open source GraphLab is quiet as well. Apache Spark’s GraphX is the only graph engine for Hadoop under active development; the Spark team recently promoted GraphX from Alpha to production. However, with just 10 out of 132 contributors working on GraphX in Release 1.2, the graph engine is relatively quiet compared to the SQL, Machine Learning and Streaming modules.
(4) R approaches parity with SAS in the commercial job market.
As of early 2014, when Bob Muenchin last updated his job market statistics, SAS led R in job postings, but R was closing the gap rapidly.
Linda Burtch of Burtch Works is the nation’s leading executive recruiter for quants and data scientists. I asked Linda what analytic languages hiring managers seek when they hire quants. “My clients are still more frequently asking for SAS, although many more are now asking for either SAS or R,” she says. “I also recommend to my clients who ask specifically for SAS skills to be open to those using R, and many will agree after the suggestion. ”