Facebook open sources realtime big data search with Presto

In June 2013, Facebook first talked about a new query engine called Presto in their conference:

At a conference for developers at Facebook headquarters on Thursday, engineers working for the social networking giant revealed that it’s using a new homemade query engine called Presto to do fast interactive analysis on its already enormous 250-petabyte-and-growing data warehouse.

…

“Historically, our data scientists and analysts have relied on Hive for data analysis,” Traverso said. “The problem with Hive is it’s designed for batch processing. We have other tools that are faster than Hive, but they’re either too limited in functionality or too simple to operate against our huge data warehouse. Over the past few months, we’ve been working on Presto to basically fill this gap.”

Facebook has now gone one step ahead in the race for real-time big data search engines and released Presto as open source.

Now, Facebook wants other data-driven organizations to use, and it hopes, refine Presto. The company has posted the software’s source code and is encouraging contributions from other parties. The software is already being tested by a number of other large Internet services, namely AirBnB and Dropbox.

To learn more about Presto and to download and try Presto, check out http://prestodb.io

Reach1to1 also provides consulting services to implement big data solutions using such new technologies like Presto, Hive, Hadoop and other big data technologies. To read more about our services please contact us.