开源软件名称(OpenSource Name):opensourceteams/flink-maven-scala开源软件地址(OpenSource Url):https://github.com/opensourceteams/flink-maven-scala开源编程语言(OpenSource Language):Scala 99.8%开源软件介绍(OpenSource Introduction):技术交流
源码Flink源码分析视频集
中文文档
Flink 本地运行交互Shell
benv.fromElements(1,2,3).map(i => i * i ).print
1
4
9
运行 jar 到 Flink 集群 flink run -c com.opensourceteams.module.bigdata.flink.example.stream.worldcount.nc.SocketWindowWordCount ./flink-maven-scala-2-0.0.1.jar
创建flink java 项目mvn archetype:generate \
-DarchetypeGroupId=org.apache.flink \
-DarchetypeArtifactId=flink-quickstart-java \
-DarchetypeVersion=1.7.1
-DgroupId=com.opensourceteams \
-DartifactId=flink-maven-java \
-Dversion=0.0.1 \
-Dpackage=com.opensourceteams.module.bigdata.flink \
-DinteractiveMode=false
创建flink scala项目mvn archetype:generate \
-DarchetypeGroupId=org.apache.flink \
-DarchetypeArtifactId=flink-quickstart-scala \
-DarchetypeVersion=1.7.1
-DgroupId=com.opensourceteams \
-DartifactId=flink-maven-scala-2 \
-Dversion=0.0.1 \
-Dpackage=com.opensourceteams.module.bigdata.flink \
-DinteractiveMode=false
查看jar中文件列表jar tvf test.jar
maven 运行某个类mvn exec:java -Dexec.mainClass=wikiedits.WikipediaAnalysis
scala 隐式转换,需要用到的包import org.apache.flink.api.scala._
import org.apache.flink.table.api.scala._
执行计划图
//执行计划
//println(env.getExecutionPlan)
//StreamGraph
//println(env.getStreamGraph.getStreamingPlanAsJSON) Execute Plan{"nodes":[{"id":1,"type":"Source: Socket Stream","pact":"Data Source","contents":"Source: Socket Stream","parallelism":1},{"id":2,"type":"Flat Map","pact":"Operator","contents":"Flat Map","parallelism":1,"predecessors":[{"id":1,"ship_strategy":"FORWARD","side":"second"}]},{"id":3,"type":"Map","pact":"Operator","contents":"Map","parallelism":1,"predecessors":[{"id":2,"ship_strategy":"FORWARD","side":"second"}]},{"id":5,"type":"Window(TumblingProcessingTimeWindows(3000), ProcessingTimeTrigger, SumAggregator, PassThroughWindowFunction)","pact":"Operator","contents":"Window(TumblingProcessingTimeWindows(3000), ProcessingTimeTrigger, SumAggregator, PassThroughWindowFunction)","parallelism":1,"predecessors":[{"id":3,"ship_strategy":"HASH","side":"second"}]},{"id":6,"type":"Sink: Print to Std. Out","pact":"Data Sink","contents":"Sink: Print to Std. Out","parallelism":1,"predecessors":[{"id":5,"ship_strategy":"FORWARD","side":"second"}]}]}
StreamGraph Plan{"nodes":[{"id":1,"type":"Source: Socket Stream","pact":"Data Source","contents":"Source: Socket Stream","parallelism":1},{"id":2,"type":"Flat Map","pact":"Operator","contents":"Flat Map","parallelism":1,"predecessors":[{"id":1,"ship_strategy":"FORWARD","side":"second"}]},{"id":3,"type":"Map","pact":"Operator","contents":"Map","parallelism":1,"predecessors":[{"id":2,"ship_strategy":"FORWARD","side":"second"}]},{"id":5,"type":"Window(TumblingProcessingTimeWindows(3000), ProcessingTimeTrigger, SumAggregator, PassThroughWindowFunction)","pact":"Operator","contents":"Window(TumblingProcessingTimeWindows(3000), ProcessingTimeTrigger, SumAggregator, PassThroughWindowFunction)","parallelism":1,"predecessors":[{"id":3,"ship_strategy":"HASH","side":"second"}]},{"id":6,"type":"Sink: Print to Std. Out","pact":"Data Sink","contents":"Sink: Print to Std. Out","parallelism":1,"predecessors":[{"id":5,"ship_strategy":"FORWARD","side":"second"}]}]}
Flink 环境,配置
example
Flink1.7.2 DataStream 源码分析(流处理)
Flink1.7.2 Dataset 源码分析(批处理)
Flink1.7.2 时序图
Flink 1.7.2 Error 收集
|
2023-10-27
2022-08-15
2022-08-17
2022-09-23
2022-08-13
请发表评论