@@ -17,21 +17,22 @@ the application state.
17
17
18
18
#### Background
19
19
20
- Every object handed over between the operators of the application or stored in the application state is serialized using
21
- Flink serialization mechanism.
20
+ Every object handed over between operators or stored in application state is serialized using
21
+ Flink serialization mechanism. Flink is able to efficiently serialize most of Java simple types, and POJOs where fields are basic types.
22
22
23
- Flink is able to efficiently serialize most of Java simple types, and POJOs where fields are basic types.
24
-
25
- When Flink cannot natively serialize an object, it falls back to using [ Kryo] ( https://github.com/EsotericSoftware/kryo ) .
26
- Unfortunately, Kryo serialization is less efficient and has a considerable impact on performance. In particular on CPU
23
+ When Flink cannot natively serialize an object, it falls back to [ Kryo] ( https://github.com/EsotericSoftware/kryo ) .
24
+ Unfortunately, Kryo serialization is less efficient and has a considerable impact on performance, in particular on CPU
27
25
utilization.
28
26
29
- One important case where Flink cannot natively serialize a field is with Collections. Due to Java type erasure, Flink
30
- cannot discover the type of the elements of the collection, and falls back to Kryo to serialize it.
27
+ One important case where Flink cannot natively serialize a field is with Collections. Due to Java type erasure, at runtime Flink
28
+ cannot discover the type collections' elements, forcing it to fall back to the less efficient Kryo serialization.
29
+
30
+ You can easily prevent using Kryo for collections, adding a ` TypeInfo ` to the class and defining a custom TypeInfoFactory that
31
+ describe the content of the collection.
31
32
32
33
#### Defining a TypeInfo
33
34
34
- To prevent this, you can explicitly define the type of the collection elements using the ` @TypeInfo ` annotation and
35
+ To prevent this, you can explicitly define the collection'a elements type using the ` @TypeInfo ` annotation and
35
36
defining a custom ` TypeInfoFactory ` .
36
37
This is demonstrated in these two record classes:
37
38
[ ` VehicleEvent ` ] ( src/main/java/com/amazonaws/services/msf/domain/VehicleEvent.java )
@@ -40,9 +41,12 @@ and [`AgggregateVehicleEvent`](src/main/java/com/amazonaws/services/msf/domain/A
40
41
``` java
41
42
public class VehicleEvent {
42
43
// ...
44
+
43
45
@TypeInfo (SensorDataTypeInfoFactory . class)
44
46
private Map<String , Long > sensorData = new HashMap<> ();
47
+
45
48
// ...
49
+
46
50
public static class SensorDataTypeInfoFactory extends TypeInfoFactory<Map<String , Long > > {
47
51
@Override
48
52
public TypeInformation<Map<String , Long > > createTypeInfo (Type t , Map<String , TypeInformation<?> > genericParameters ) {
@@ -52,8 +56,7 @@ public class VehicleEvent {
52
56
}
53
57
```
54
58
55
- For more details about Flink serialization, see [ Data Types & Serialization] ( https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/datastream/fault-tolerance/serialization/types_serialization/#data-types--serialization )
56
- in Flink documentation.
59
+ For more details about Flink serialization, see [ Data Types & Serialization] ( https://nightlies.apache.org/flink/flink-docs-release-1.20/docs/dev/datastream/fault-tolerance/serialization/types_serialization/#data-types--serialization ) in Flink documentation.
57
60
58
61
#### Testing serialization
59
62
0 commit comments