Sqoop Connectors and Drivers - dummies

Sqoop Connectors and Drivers

By Dirk deRoos

Sqoop connectors generally go hand in hand with a JDBC driver. Sqoop does not bundle the JDBC drivers because they are usually proprietary and licensed by the RDBMS or DW vendor. So there are three possible scenarios for Sqoop, depending on the type of data management system (RDBMS, DW, or NoSQL) you are trying to interact with. Take a look at each one:

  • Your data management system is supported by one of the bundled Sqoop connectors. In this case, you need to acquire the JDBC driver from your data management system provider and install the .jar file associated with it in your $SQOOP_HOME/lib directory. ($SQOOP_HOME is an environment variable that refers to the directory pathname on your system where you install Apache Sqoop.)

  • Sqoop does not include a connector for your database management system. That means you need to download one from a third-party vendor, along with a JDBC driver if the connector requires one. (Couchbase and Teradata both do, for example.)

  • Your database management system does not provide a Sqoop connector but a JDBC driver is available. In this case, you leverage Sqoop’s generic JDBC connector and download and install your vendor’s JDBC driver.