Using kafka-handler 4.0 in HDP 3.1.4 - hive

Question

We are using HDP 3.1.4 and want to use kafka-handler for Kafka and Hive integration.
Lots of our topics in Kafka are serialized with Avro using Confluent Schema Registry and the default kafka-handler shipped with HDP 3.1.4 does not contain new features for Avro messages serialized with Confluent that contains Magic Byte. in new versions of kafka-handler we have some properties such as 'avro.serde.type'='skip' and 'avro.serde.skip.bytes'='5' to handle Magic Byte.
So we wanted to replace kafka-handler with our own compiled kafka-handler 4.0.
we simply started with replacing kafka-handler 4.0 with default HDP kafka-handler but we faced this error :
MetaException(message:java.lang.NoSuchFieldError AVRO_SERDE_TYPE)
AVRO_SERDE_TYPE is added in hive-serde 4.0 and we investigate it is because kafka-handler has dependency to kafka-exec and kafka-exec contains our old hive-serde that shipped with HDP.
We should replace hive-serde too. so we continue with shading hive-serde in kafka-handler:
<relocation>
<pattern>org.apache.hadoop.hive.serde2</pattern>
<shadedPattern>org.shaded.apache.hadoop.hive.serde2_shaded</shadedPattern>
</relocation>
but we face this exception:
SQL Error [40000] [42000]: Error while compiling
statement: FAILED: ClassCastException class org.apache.hadoop.hive.kafka.KafkaSerDe
so how we should handle this issue ?