On paper, Spark and Tez have a lot in common: both possess in-memory capabilities, can run on top of Hadoop YARN and support all data types from any data sources. So, what’s the difference?
Welcome to Xplenty's Blog
All things data
People talk a lot about Hadoop, and we like to keep up to date with the latest gossip by reading Hadoop blogs. If you'd also like to jump into the conversation and read the best Hadoop posts out there, here are our favorite Hadoop blogs for 2014.
Apache Spark is setting the world of Big Data on fire. With a promise of speeds up to 100 times faster than Hadoop MapReduce and comfortable APIs, some think this could be the end of Hadoop MapReduce. Or is it?
Big Data is mostly famous, well, for being big. So if you have anything under a petabyte, why would you even think about using Apache Hadoop? But you should.
Following our post about Hadoop security for the enterprise, or the lack thereof, one of the ways to make Hadoop more secure is by installing an additional platform. Five major Hadoop security projects are currently available: Apache Knox Gateway, Apache Sentry, Apache Argus, Apache Accumulo and Project Rhino. Let’s see what they provide.
While Hadoop has proved its power for scalable storage and processing of Big Data, it may not be enterprise-ready when it comes to security. Hortonworks, Cloudera and MapR address this problem by providing Enterprise Hadoop distributions, and there are several Hadoop security projects, such as Apache Argus and Knox. But what does Hadoop provide right out of the box?
Hadoop YARN may be the gun that hangs on the wall in the first act and kills MapReduce in the last—Google Trends clearly shows that interest in Hadoop is still on the rise, but Apache Spark is closing in fast.
These days, there is a renaissance of Hadoop-based Big Data projects: Impala, Spark, Storm, Flink and HBase as well as several SQL-on-Hadoop tools. Most of these projects are still in their infancy though, if not in the Apache Incubator, so they’re mostly used by early adopters and none of them has become an industry standard. Yet.
YARN's promise was big and exciting, but did it deliver? Now that we’ve spent a year with the baby elephant, we're happy to announce YARN Week: a three post series about our YARN thoughts and experiences.