Hadapt and Hadoop - dummies

By Dirk deRoos

Late in the year 2010, Hadapt was formed as a start-up by two Yale University students and an assistant professor of computer science. Professor Daniel Abadi and Kamil Bajda-Pawlikowski, a PhD student from Yale’s computer science department, had been working on the research project HadoopDB.

After this paper was published, Justin Borgman, a student from the Yale School of Management, became interested in the work. He would later team up with Professor Abadi and Kamil Bajda-Pawlikowski to form Hadapt.

The Hadapt strategy is to join Apache Hadoop with a Shared-Nothing MPP database to create an adaptive analytics platform. This approach provides a standard SQL interface on Hadoop and enables analytics across unstructured, semistructured, and structured data on the same cluster.

Like Apache Hive and other technologies, Hadapt provides a familiar JDBC/ODBC interface for submitting SQL or MapReduce jobs to the cluster. Hadapt provides a cost-based query optimizer, which can decide between a combination of MapReduce jobs and MPP database jobs to fulfill a query, or the job can be handled by the MPP database for fast interactive response.

By joining an Apache Hadoop cluster with an MPP database cluster to create a hybrid system, Hadapt solves the query response time and partial SQL support (via HiveQL) found in Apache Hive.