A Big Data TV interview with Director of Data Sciences Ofer Mendelevitch from Hortonworks. Ofer talks Big Data, Yahoo!, and the origins of Hadoop.
Dave Feinleib: Dave Feinleib here with another episode of Big Data TV. I’m speaking with Ofer Mendelevitch. Ofer great to have you on the show.
Ofer: Thanks for having me.
Dave: We’re here at Big Data date night at the Microsoft campus in Mountain View, California. Ofer tell us a little about Hortonworks. What the company does, and for those who don’t know, what Hadoop is.
Ofer: Hortonworks is one of the providers of Hadoop. Hadoop is an open source platform for data management and large scale computation that scales horizontally, and can process big data, lots of big data, relatively inexpensively and easily.
Dave: A little about the history of the company. You started out at Yahoo, and you sort of knew this technology really early on. Talk a little about that.
Ofer: What happened was, in about 2005 what is now known as Hadoop started in Yahoo. Yahoo hired the first person who started this project and built a team around Hadoop. Essentially moving it over into an Apache open source project. Then Yahoo was really a big user, and still is one of the largest users, of Hadoop, and they started investing in it more and more, progressing the technology. Hortonworks spun out of Yahoo. Around 2011 took the core team of developers in the Hadoop group and built essentially a company around it. Hortonworks today we provide HDP, which is Hortonworks Data Platform. We are completely one hundred percent open source. Everything we do goes back into open source. Our business model really is about support for Hadoop. For enterprises we really do that really, really well.
Dave: Great. You have a partnership with Microsoft where you work very closely with them on Azure. Talk a little about that.
Ofer: We have a very strong partnership with Microsoft. The partnership is really an engineered partnership, meaning that a big team of engineers on both sides worked really, really well to make sure Hadoop can run in the Windows environment, can be managed from the Windows platforms of management systems, management, etc. It’s a way to help Hadoop as well as Azure run on these platforms.
Dave: What are some of the key challenges you’re seeing? You’re out there talking to enterprise customers. What are some of the big challenges they have, and how are you helping them overcome them?
Ofer: One of the things I’m seeing really strongly is having had the experience of Yahoo using Hadoop really early on in a web company, I think there’s a lot of value that gets generated for an enterprise to use that. I think Hadoop was not really ready until maybe a year ago or so to be used in an enterprise environment. Now it’s almost like all the value enterprise can take advantage of. The community solved a lot of the problems that enterprises needed to have. Like security snapshots, a lot of different management tools and things like that. It’s almost ready for enterprise, and they can really leverage big data using this platform.
Dave: That’s great. Security snapshots, what are some of the other key insights or innovations that you’ve seen that enterprises really have to have to adopt this.
Ofer: A lot of it is the management tools. Hortonworks has launched Ambari which is an open source project. Part of the Apache, Hadoop eco-system to help take care of the dev-ops, the people who manage the cluster. You’ve got to take care of the developers. There’s a lot of tools that we keep innovating on — Pig and Hive too. There’s Hadoop 2.0 which is coming out soon. A lot of these tools are really what the enterprises need. In addition, security was done last year, and it’s pretty much solved.
Dave: Ofer, thanks so much for being on the show.