I am using libjitsi for G.711 encode/decode. For PCMU it is fine, however, for PCMA, there is only encoder available. Why is the decoder not supported?
http://bluejimp.com/jitsi/libjitsi/javadoc/org/jitsi/impl/neomedia/codec/audio/alaw/package-summary.html
Related
I am in the process of performing a full 8-bit quantization of ssd_mobilenet_v2_320x320_coco17 using Object detection API. For this I run the following notebook.
In my case my final hardware requires that the output tensors are TFLite_Detection_PostProcess (they actually can be float32). However, when performing post-training quantization, with the Object detection API, the final tflite output tensors are StatefulPartitonedCall, which are not accepted by the application running on the mcu. So somehow I need to convert to TFLite_Detection_PostProcess.
Some searching reveales that some are using converter = tf.lite.TFLiteConverter.from_frozen_graph(... output_array=output_array...) for convertion where they use
output_arrays = ['TFLite_Detection_PostProcess',
'TFLite_Detection_PostProcess:1',
'TFLite_Detection_PostProcess:2',
'TFLite_Detection_PostProcess:3']
In my case the notebook is using converter = tf.lite.TFLiteConverter.from_saved_model(...) which does not accept an parameter such as output_array.
So, can I quantize using tf.lite.TFLiteConverter.from_saved_model and define my output tensors to TFLite_Detection_PostProcess?
See also TF Object detection API issue tracker TF2 Detection API Models on mobile TFLite_Detection_PostProcess #10130
I'am trying to convert a tf.keras model based on mobilenetv2 with transpose convolution using latest tf-nighlty. Here is the conversion code
#saved_model_dir='/content/ksaved' # tried from saved model also
#converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
converter = tf.lite.TFLiteConverter.from_keras_model(reshape_model)
converter.experimental_new_converter=True
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
converter.representative_dataset = representative_dataset_gen
tflite_quant_modell = converter.convert()
open("/content/model_quant.tflite", "wb").write(tflite_quant_modell)
The conversion was successful(in google colab); but it has quantize and dequantize operators at the ends(as seen using netron). All operators seems to be supported. Representative data set images are float32 in generator and the model has a 4 channel float32 input by default. It looks like we need a UINT8 input and output inside model for coral TPU. How can we properly carry out this conversion?
Ref:-
Full integer quantization of weights and activations
How to quantize inputs and outputs of optimized tflite model
Coral Edge TPU Compiler cannot convert tflite model: Model not quantized
I tried with 'tf.compat.v1.lite.TFLiteConverter.from_keras_model_file' instead of v2 version.I got error: "Quantization not yet supported for op: TRANSPOSE_CONV" while trying to quantize the model in latest tf 1.15 (using representative dataset) and "Internal compiler error. Aborting! " from coral tpu compiler using tf2.0 quantized tflite
Tflite model # https://github.com/tensorflow/tensorflow/issues/31368
It seems to work until the last constitutional block (1x7x7x160)
The compiler error(Aborting) does not give any information regarding the potential cause and all types of convolutional layers seems to be supported as per coral docs.
Coral doc: https://coral.ai/docs/edgetpu/models-intro/#quantization
Here is a dummy model example of quantizing a keras model. Notice I'm using strict tf1.15 for the example, because tf2.0 deprecated:
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
with the from_keras_model api. I think the most confusing thing about this is that you can still call it but nothing happens. This means that model will still take in float inputs. I notice that you are using tf2.0, because from_keras_model is a tf2.0 api. Coral still suggest using tf1.15 for converting model for now. I suggest downgrading tensorflow or maybe even just use this (while keeping tf2.0, it may or may not work):
tf.compat.v1.lite.TFLiteConverter.from_keras_model_file
More on it here.
I always make sure not to use the experimental converter:
converter.experimental_new_converter = False
Kears/TensorFlow layer UpSampling2D() cannot be deployed to TensorRT (known behavior).
I am trying to find a solution by replacing the layer UpSampling2D() by other Keras layer with parallel behaviour.
Theoretically Conv2DTranspose() should do the work, by setting specific weights and fixing the weights of the layers in training.
I am looking for some help on how to do that.
I did a test run by replacing all the UpSampling 2D() with Conv2DTranspose() in my model and then converted it to UFF. (I only trained the model for 1 epoch to save time).
The converter then complained about DataFormatVecPermute instead.
Converting conv2d_transpose_1/conv2d_transpose-0-VecPermuteNHWCToNCHW-LayoutOptimizer as custom op: DataFormatVecPermute
Warning: No conversion function registered for layer: DataFormatVecPermute yet.
And the parser in C++ couldn't parse this model successfully either.
I looked for several samples on the web to build a stacked autoencoder for data denoising but I don't seem to understand a fundamental part of the encoder part:
https://blog.keras.io/building-autoencoders-in-keras.html
Following the examples I built the autoencoder like that:
inputs = Input(shape=(timesteps, 50))
encoded1 = Dense(30, activation="relu")(inputs)
encoded2 = Dense(15, activation="relu")(encoded1)
encoded3 = Dense(5, activation="relu")(encoded2)
decoded1 = Dense(15, activation="relu")(encoded3)
decoded2 = Dense(30, activation="relu")(decoded1)
decoded = Dense(50, activation="sigmoid")(decoded2)
autoencoder = Model(inputs=inputs, outputs=decoded)
encoder = Model(inputs, encoded3)
autoencoder.compile(loss='mse', optimizer='rmsprop')
autoencoder.fit(trainX,
trainX,
epochs=epochs,
batch_size=512,
callbacks=callbacks,
validation_data=(trainX, trainX))
On the examples there is mostly a model with the encoder and a seperate model with the decoder. I always see that only the decoder model get's trained. The encoder is not trained. But for my usecase I only need the encoder model to denoise the data. Why does the encoder need no training?
Your interpretation about encoder-decoder is wrong. Encoder encodes your input data into some high dimensional representation which is abstract but it's very powerful if you want use that as features for further prediction. To make sure encoded output is as close to your actual input, you have decoder which decodes your encoded high-dimensional input back to original input. During training, both encoder and decoder are involved i.e. the weights of the encoder layers and decoder layers both are updated. If the encoder is not trained how it's going to learn the encoding mechanism. During inference, you use only the encoder module as you want to encode the input.
I'm playing with Tensorflow seq2seq model and I'm wondering how I can feed a trained seq2seq decoder with an arbitrary initial decoder memory (during the training, this initial decoder memory is an output of the encoder).
I figured that I need to use feed_dict and TF forces me to feed input sequence because a placeholder is defined for it, same for decoder input.
But if I do so, I can't actually force the initial decoder memory to be what I want it to be because it's defined from the decoder input. I still tried to do it but it ignores my third line.
for t in range(seq_length):
feed_dict[enc_inp[t]] = X[t] #encoder input
for t in range(seq_length):
feed_dict[dec_inp[t]] = Y[t] #decoder input
for t in range(seq_length):
feed_dict[dec_memory[t]] = np.random.rand(memory_dim) #value I want to feed as initial memory of decode
Do you have any clue on how to do what I want ? ie feeding the model variables states even though they aren't designed to be fed by placeholders.
Thanks
Shouldn't you be feeding random numbers, but rather 0's? That's a suggestion in the karpathy lesson on rnns: Recurrent Neural Networks, Image Captioning