说明
当下游算子 与 上游算子 并行度 一样时,默认的数据传递方式是 forward
上下游并行度相同时,一对一发送,将元素转发到本地运行的下游操作的分区器
方法声明
filter、keyBy 后可以调用
forward()
案例
package chongfenqu;
import org.apache.flink.api.common.functions.RichMapFunction;
import org.apache.flink.api.common.serialization.SimpleStringSchema;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.configuration.Configuration;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer;
import org.apache.kafka.clients.consumer.ConsumerConfig;
import java.util.Properties;
public class F10forward {
public static void main(String[] args) throws Exception {
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
// 定义kafka相关配置项
Properties pro = new Properties();
pro.setProperty(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,"hadoop1:9092,hadoop2:9092,hadoop3:9092");
pro.setProperty(ConsumerConfig.GROUP_ID_CONFIG,"consumer-group");
pro.setProperty(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG,"latest");
// 创建FlinkKafka消费者,需要指定主题、反序列化、kafka配置
FlinkKafkaConsumer<String> fkc = new FlinkKafkaConsumer<>("kafka2flink", new SimpleStringSchema(), pro);
DataStreamSource<String> dss = env.addSource(fkc);
dss.print("dss:");
DataStream<Tuple2<Integer,String>> mapDS = dss.map(new RichMapFunction<String, Tuple2<Integer,String>>() {
@Override
public Tuple2<Integer, String> map(String s) throws Exception {
return Tuple2.of(getRuntimeContext().getIndexOfThisSubtask(),s);
}
});
mapDS.print("mapDS::");
System.out.println(env.getExecutionPlan());
env.execute();
}
}
生成计划可视图
访问:https://flink.apache.org/visualizer/
将上面打印的执行计划生成图,由于没有设置并行度,所以并行度等于CPU线程数,即:都是 8,所以都是 forward,如下:
参考
https://blog.51cto.com/u_9928699/5670559

