I am using the Spark Cassandra connector.
It take 5-6 minutes for fetch data from Cassandra table. In Spark I have seen many tasks and Executor in log. The reason might be that Spark divided the process in many tasks!
Below is my code example :
public static void main(String[] args) {
SparkConf conf = new SparkConf(true).setMaster("local[4]")
.setAppName("App_Name")
.set("spark.cassandra.connection.host", "127.0.0.1");
JavaSparkContext sc = new JavaSparkContext(conf);
JavaRDD<Demo_Bean> empRDD = javaFunctions(sc).cassandraTable("dev",
"demo");
System.out.println("Row Count"+empRDD.count());
}
See Question&Answers more detail:
os 与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…