bucketingsink专题

Flink BucketingSink with Custom AvroParquetWriter create empty file

问题: I have created a writer for BucketingSink. The sink and writer works without error but when it comes to the writer writing avro genericrecord to parquet, the file was created from in-progress,

Flink BucketingSink 源码分析

0x1 摘要 BucketingSink类提供了非常完美的功能支持数据落HDFS,在实际业务中不建议自己去实现,直接采用此类可以避免一些坑。注:此文基于Flink 1.6.3 版本源码。 0x2 BucketingSink 类结构分析 我们关注RichSinkFunction、CheckpointedFunction、CheckpointListener三个父类 0x3 先看