java - Is there a way to parallel insertion of records in hive tables using jdbc?

Question

I want to use JDBC to insert 1 million records in Hive table.

1. The Dataset is very huge. It has 2400 columns.
2. At the moment I am using simple insert statements with 1000 records in values section.
3. This is because I can not use addBatch() as it didnt work in hive driver. And also it creates a map reduce job for each insert statement which takes time to complete.
4. I am not able to add more records in values as the query string length becomes problem.
5. I tried to run multiple insert statements(having 1000 records in values of each) using threadpool in ExecutorService. But it didn't work as it seems like it still runs in sequence.
6. Is there any other way to achieve parallelism in this case?

question from:https://stackoverflow.com/questions/66048934/is-there-a-way-to-parallel-insertion-of-records-in-hive-tables-using-jdbc

Answer 1

Waitting for answers