说明
几乎所有Flink 函数类都有其Rich 版本
例如:
MapFunction的富函数:RichMapFunctionFlatMapFunction的富函数:RichFlatMapFunctionFilterFunction的富函数:RichFilterFunctionReduceFunction的富函数:RichReduceFunction
应用场景
需要在函数处理之前,连接数据库(只连接一次),执行结束之后还要关闭数据库
获取得运行时相关的上下文信息
功能
可以获取 运行环境的上下文,并拥有一些生命周期方法,所以可以实现更复杂的功能
生命周期方法有:
open()方法:初始化方法,当算子(如 map 或者filter)被调用之前,open()会被调用,只会执行一次。如:连接数据库close()方法:是生命周期中的 最后一个调用的方法,做一些清理工作。如:关闭数据库
getRuntimeContext()
父类的 getRuntimeContext() 方法:提供了 RuntimeContext 的一些信息,例如:
- 函数执行的并行度
- 任务的名字
- 以及state 状态
案例
package filter;
import org.apache.flink.api.common.functions.RichMapFunction;
import org.apache.flink.configuration.Configuration;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
public class TestFilter2 {
public static void main(String[] args) throws Exception {
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setParallelism(2);
DataStreamSource<Long> dss = env.fromSequence(1, 10);
SingleOutputStreamOperator<Long> filter = dss.map(new MyMap());
filter.print();
env.execute();
}
public static class MyMap extends RichMapFunction<Long, Long> {
@Override
public void open(Configuration parameters) throws Exception {
String taskName = getRuntimeContext().getTaskName();
int index = getRuntimeContext().getIndexOfThisSubtask();
System.out.println("open()-----subtask:"+index+"--taskName:["+taskName+"]");
}
@Override
public void close() throws Exception {
String taskName = getRuntimeContext().getTaskName();
int index = getRuntimeContext().getIndexOfThisSubtask();
System.out.println("close()-----subtask:"+index+"--taskName["+taskName+"]");
}
@Override
public Long map(Long data) throws Exception {
String taskName = getRuntimeContext().getTaskName();
int index = getRuntimeContext().getIndexOfThisSubtask();
System.out.println("map()-----subtask:"+index+"--taskName["+taskName+"]");
return data;
}
}
}
执行结果:
open()-----subtask:1--taskName:[Source: Sequence Source -> Map -> Sink: Print to Std. Out]
open()-----subtask:0--taskName:[Source: Sequence Source -> Map -> Sink: Print to Std. Out]
map()-----subtask:1--taskName[Source: Sequence Source -> Map -> Sink: Print to Std. Out]
2> 1
map()-----subtask:1--taskName[Source: Sequence Source -> Map -> Sink: Print to Std. Out]
map()-----subtask:0--taskName[Source: Sequence Source -> Map -> Sink: Print to Std. Out]
2> 2
1> 6
map()-----subtask:1--taskName[Source: Sequence Source -> Map -> Sink: Print to Std. Out]
map()-----subtask:0--taskName[Source: Sequence Source -> Map -> Sink: Print to Std. Out]
2> 3
1> 7
map()-----subtask:1--taskName[Source: Sequence Source -> Map -> Sink: Print to Std. Out]
2> 4
map()-----subtask:1--taskName[Source: Sequence Source -> Map -> Sink: Print to Std. Out]
map()-----subtask:0--taskName[Source: Sequence Source -> Map -> Sink: Print to Std. Out]
2> 5
1> 8
map()-----subtask:0--taskName[Source: Sequence Source -> Map -> Sink: Print to Std. Out]
1> 9
map()-----subtask:0--taskName[Source: Sequence Source -> Map -> Sink: Print to Std. Out]
1> 10
close()-----subtask:1--taskName[Source: Sequence Source -> Map -> Sink: Print to Std. Out]
close()-----subtask:0--taskName[Source: Sequence Source -> Map -> Sink: Print to Std. Out]
根据执行结果可知:
- 2个并行度,所以 2个任务
- 每个任务 先执行1次
open() - 有多少个数据,就执行多少次
map()方法 - 最后才执行
close()方法
参考
https://blog.csdn.net/Yuan_CSDF/article/details/117375805
https://blog.csdn.net/qq_33982605/article/details/106286917