Big data has become so big, it’s spread beyond the tech world. When 163-year-old publication New York Times hired a chief data scientist earlier this year, it became clear that even non-technical organizations were hopping on the big data train. To successfully predict what their customers want or how they might behave, companies that know how to mine big data — also know as companies who hire good data scientists — have the advantage.
To do their jobs effectively, data scientists must do a whole lotta dirty data work. The New York Times calls it “data janitor work.” In a recent article, NYT reported that data scientists spend from 50 percent to 80 percent of their time laboriously collecting and prepping data before it can be extracted into digestible insights.
“Data wrangling is a huge — and surprisingly so — part of the job,” Monica Rogati, VP for…
View original post 77 more words