I was trying out Intel Low Precision Optimization Tool in Linux. Initially I have created one environment named lpot_environment and installed tensorflow using the below command:
conda create -n lpot_environment python=3.7
pip install intel-tensorflow==2.3.0
Then I cloned the github repository as :
git clone https://github.com/intel/neural-compressor.git
I have the frozen model downloaded.
As an initial step toward intel lpot, I would like to run with a dummy dataloader. But I failed to find one.
It would be great if anybody send a code sample using dummy dataset in intel lpot.
You can use the below code to quantize a TensorFlow checkpoint and run with a dummy dataloader.
quantizer = Quantization('./conf.yaml')
dataset = quantizer.dataset('dummy_v2', \
input_shape=(100, 100, 3), label_shape=(1, ))
quantizer.model = common.Model('./model/public/rfcn-resnet101-coco-tf/rfcn_resnet101_coco_2018_01_28/')
quantizer.calib_dataloader = common.DataLoader(dataset)
quantized_model = quantizer()
Run quantization will create a dummy dataloader.
Related
I have this script where I want to get the callbacks to a separate CSV file in sagemaker custom script docker container. But when I try to run in local mode, it fails giving the following error. I have a hyper-parameter tuning job(HPO) to run and this keeps giving me errors. I need to get this local mode run correctly before doing the HPO.
In the notebook I use the following code.
from sagemaker.tensorflow import TensorFlow
tf_estimator = TensorFlow(entry_point='lstm_model.py',
role=role,
code_location=custom_code_upload_location,
output_path=model_artifact_location+'/',
train_instance_count=1,
train_instance_type='local',
framework_version='1.12',
py_version='py3',
script_mode=True,
hyperparameters={'epochs': 1},
base_job_name='hpo-lstm-local-test'
)
tf_estimator.fit({'training': training_input_path, 'validation': validation_input_path})
In my lstm_model.py script the following code is used.
lgdir = os.path.join(model_dir, 'callbacks_log.csv')
csv_logger = CSVLogger(lgdir, append=True)
regressor.fit(x_train, y_train, batch_size=batch_size,
validation_data=(x_val, y_val),
epochs=epochs,
verbose=2,
callbacks=[csv_logger]
)
I tried creating a file before hand like shown below using tensorflow backend. But it doesn't create a file. ( K : tensorflow Backend, tf: tensorflow )
filename = tf.Variable(lgdir , tf.string)
content = tf.Variable("", tf.string)
sess = K.get_session()
tf.io.write_file(filename, content)
I can't use any other packages like pandas to create the file as the TensorFlow docker container in SageMaker for custom scripts doesn't provide them. They give only a limited amount of packages.
Is there a way I can write the csv file to the S3 bucket location, before the fit method try to write the callback. Or is that the solution to the problem? I am not sure.
If you can even suggest other suggestions to get callbacks, I would even accept that answer. But it should be worth the effort.
This docker image is really narrowing the scope.
Well for starters, you can always make your own docker image using the Tensorflow image as a base. I work in Tensorflow 2.0 so this will be slightly different for you but here is an example of my image pattern:
# Downloads the TensorFlow library used to run the Python script
FROM tensorflow/tensorflow:2.0.0a0 # you would use the equivalent for your TF version
# Contains the common functionality necessary to create a container compatible with Amazon SageMaker
RUN pip install sagemaker-containers -q
# Wandb allows us to customize and centralize logging while maintaining open-source agility
RUN pip install wandb -q # here you would install pandas
# Copies the training code inside the container to the design pattern created by the Tensorflow estimator
# here you could copy over a callbacks csv
COPY mnist-2.py /opt/ml/code/mnist-2.py
COPY callbacks.py /opt/ml/code/callbacks.py
COPY wandb_setup.sh /opt/ml/code/wandb_setup.sh
# Set the login script as the entry point
ENV SAGEMAKER_PROGRAM wandb_setup.sh # here you would instead launch lstm_model.py
I believe you are looking for a pattern similar to this, but I prefer to log all of my model data using Weights and Biases. They're a little out of data on their SageMaker integration but I'm actually in the midst of writing an updated tutorial for them. It should certainly be finished this month and include logging and comparing runs from hyperparameter tuning jobs
Looking for advice about a debugger for a TF2 model.
I would like to train the following transformer model
https://github.com/tensorflow/models/blob/master/official/transformer/v2
on my own data.
My problem is that I have difficulty figuring out shape of data returned by the _parse_example function from the data_pipeline.py file. To start with Pydev doesn't brake inside _parse_example function, neither does PyCharm, which appears to be using Pydev internally. The options offered by TensorBoard 2.0.0 seem to be applicable to TF1, not TF2:
sess = tf.Session()
sess = tf_debug.TensorBoardDebugWrapperSession(sess, "localhost:6064")
sess.run(my_fetches)
hook = tf_debug.TensorBoardDebugHook("localhost:6064")
my_estimator.fit(x=x_data, y=y_data, steps=1000, monitors=[hook])
keras.backend.set_session(
tf_debug.TensorBoardDebugWrapperSession(tf.Session(), "localhost:6064"))
model.fit(...)
So what tool can I use to see tensor's data and its shape? Option 2 from the above list seems to make sense, except for the fact that I don't see a call for my_estimator.fit from transformer's implementation for TF2.
My environment was created in Ubuntu 18.04 using anaconda:
conda create -n mytest tensorflow-gpu.
I use Eclipse with PyDev plugin.
Thanks.
I followed the docs Building an optimized serving binary then Testing the development environment, and I got
Inference error rate: 10.4%
Is it expected that a fresh install of a release build of TensorFlow Serving would give a 10% error rate on the provided example model?
My environment:
AWS EC2
OS: Amazon Linux AMI release 2018.03
Instance Type: r5.large
Steps to reproduce:
# download tensorflow serving code
git clone https://github.com/tensorflow/serving
cd serving
# build optimized serving binary
docker build --pull -t $USER/tensorflow-serving-devel -f tensorflow_serving/tools/docker/Dockerfile.devel .
# run & open shell for generated docker image
docker run -it -p 8600:8600 ec2-user/tensorflow-serving-devel:latest
# train the mnist model
python tensorflow_serving/example/mnist_saved_model.py /tmp/mnist_model
# serve the model
tensorflow_model_server --port=8500 --model_name=mnist --model_base_path=/tmp/mnist_model/ &
# test the client
python tensorflow_serving/example/mnist_client.py --num_tests=1000 --server=localhost:8500
The example mnist_saved_model.py as part of tensorflow_serving's example is more focused on speed to create a model and a simple example on how to save a model then accuracy.
In https://www.tensorflow.org/tfx/serving/serving_advanced, it shows that when the above code is trained with 100 iterations it has a 13.1% error rate, and when trained with 2000 iterations it has a 9.5% error rate.
The default if --training_iteration is not specified is 1000, so your 10.4 error rate is inline with these results.
You will find this mnist model provides better accuracy (and takes much longer to train): https://github.com/tensorflow/models/tree/master/official/mnist
This model will work with the slight changes to the mnist_client.py example.
try this:
train the mnist model
git clone https://github.com/tensorflow/models
export PYTHONPATH="$PYTHONPATH:$PWD/models"
pushd models/official/mnist/
python mnist.py --export_dir /tmp/mnist_model
serve the model
tensorflow_model_server --port=8500 --model_name=mnist --model_base_path=/tmp/mnist_model/ &
switch back the original directory
popd
make the follow changes to mnist_client.py to work with the new model
diff --git a/tensorflow_serving/example/mnist_client.py b/tensorflow_serving/example/mnist_client.py
index 592969d..85ef0bf 100644
--- a/tensorflow_serving/example/mnist_client.py
+++ b/tensorflow_serving/example/mnist_client.py
## -112,8 +112,8 ## def _create_rpc_callback(label, result_counter):
sys.stdout.write('.')
sys.stdout.flush()
response = numpy.array(
- result_future.result().outputs['scores'].float_val)
- prediction = numpy.argmax(response)
+ result_future.result().outputs['classes'].int64_val)
+ prediction = response[0]
if label != prediction:
result_counter.inc_error()
result_counter.inc_done()
## -143,9 +143,9 ## def do_inference(hostport, work_dir, concurrency, num_tests):
for _ in range(num_tests):
request = predict_pb2.PredictRequest()
request.model_spec.name = 'mnist'
- request.model_spec.signature_name = 'predict_images'
+ request.model_spec.signature_name = 'classify'
image, label = test_data_set.next_batch(1)
- request.inputs['images'].CopyFrom(
+ request.inputs['image'].CopyFrom(
tf.contrib.util.make_tensor_proto(image[0], shape=[1, image[0].size]))
result_counter.throttle()
result_future = stub.Predict.future(request, 5.0) # 5 seconds
test the client
python tensorflow_serving/example/mnist_client.py --num_tests=1000 --server=localhost:8500
Inference error rate: 0.8%
Is Tensorflow Serving an example expected to have a 10% error rate?
Yes, this particular example is expected to have a 10% error rate as the accuracy of this model on training and testing data is almost the same (around 90%) i.e., this is a very basic neural network as shown here.
If you want a good prediction accuracy, you might have to use resnet_client.py or you can actually add more layers and tune the hyper-parameters to get higher prediction accuracy or lesser inference error rate.
A tutorial on how to use the resent model to serve is given here. This should give you a much lesser inference error rate.
I wanted to use the tf.contrib.distribute.MirroredStrategy() on my Multi GPU System but it doesn't use the GPUs for the training (see the output below). Also I am running tensorflow-gpu 1.12.
I did try to specify the GPUs directly in the MirroredStrategy, but the same problem appeared.
model = models.Model(inputs=input, outputs=y_output)
optimizer = tf.train.AdamOptimizer(LEARNING_RATE)
model.compile(loss=lossFunc, optimizer=optimizer)
NUM_GPUS = 2
strategy = tf.contrib.distribute.MirroredStrategy(num_gpus=NUM_GPUS)
config = tf.estimator.RunConfig(train_distribute=strategy)
estimator = tf.keras.estimator.model_to_estimator(model,
config=config)
These are the results I am getting:
INFO:tensorflow:Device is available but not used by distribute strategy: /device:CPU:0
INFO:tensorflow:Device is available but not used by distribute strategy: /device:GPU:0
INFO:tensorflow:Device is available but not used by distribute strategy: /device:GPU:1
WARNING:tensorflow:Not all devices in DistributionStrategy are visible to TensorFlow session.
The expected result would be obviously to run the training on a Multi GPU system. Are those known issues?
I've been facing a similar issue with MirroredStrategy failing on tensorflow 1.13.1 with 2x RTX2080 running an Estimator.
The failure seems to be in the NCCL all_reduce method (error message - no OpKernel registered for NCCL AllReduce).
I got it to run by changing from NCCL to hierarchical_copy, which meant using the contrib cross_device_ops methods as follows:
Failed command:
mirrored_strategy = tf.distribute.MirroredStrategy(devices=["/gpu:0","/gpu:1"])
Successful command:
mirrored_strategy = tf.distribute.MirroredStrategy(devices=["/gpu:0","/gpu:1"],
cross_device_ops=tf.contrib.distribute.AllReduceCrossDeviceOps(
all_reduce_alg="hierarchical_copy")
)
In TensorFlow new version, AllReduceCrossDeviceOps isn't exist. You may use distribute.HierarchicalCopyAllReduce() instead:
mirrored_strategy = tf.distribute.MirroredStrategy(devices= ["/gpu:0","/gpu:1"],cross_device_ops=tf.distribute.HierarchicalCopyAllReduce())
In my current project I'm using machine learning on the Raspberry Pi for sensor fusion. Since I heard about the release of Tensorflow Lite I'm really interested to deploy and use it to run Lite models on the platform.
On the Tensorflow website are hints for Android and iOS, but I couldn't find any hints about any other platforms. Is there a (WIP) installation/compile guide out to bring TF Lite to the Raspi?
TIA
#all, if you are still in the trials to make tensorflow lite running on Raspberry Pi 3, my "pull-request" may be useful. Please look at https://github.com/tensorflow/tensorflow/pull/24194.
Following the steps, 2 apps (label_image and camera) can be running on Raspberry Pi 3.
Best,
--Jim
There is a very small section on Raspberry PI in the TFLite docs at https://www.tensorflow.org/mobile/tflite/devguide#raspberry_pi. That section links to this GitHub doc with instructions for building TFLite on Raspberry PI - tensorflow/rpi.md.
There is no official demo app yet, but the first location says one is planned. It will probably be shared at that same location when ready (that is where the Android and iOS demo apps are described).
You can install TensorFlow PIP on Raspberry pi with "pip install tensorflow" however, if you want only TFLite you can build a smaller pip that has only the tflite interpreter (you can then do conversion on another big machine).
Info on how to do it is here:
https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/pip_package
Then, you can use it. Here is an example of how you might use it!
import tflite_runtime as tflr
interpreter = tflr.lite.Interpreter(model_path="mobilenet_float.tflite")
interpreter.allocate()
input = interpreter.get_input_details()[0]
output = interpreter.get_input_details()[0]
cap = cv2.VideoCapture(0) # open 0th web camera
while 1:
ret, frame = cap.read()
frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
frame = cv2.resize(frame, input.shape[2],input.shape[1])
frame = np.reshape(im, input.shape).astype(np.float32)/128.0-1.0
interpreter.set_tensor(input["index"], frame)
interpreter.invoke()
labels = interpreter.get_tensor(output["index"])
top_label_index = np.argmax(labels, axis=-1)
Hope this helps.
I would suggest next links:
The lightest way is by using TensorFlowLite interpreter only. You can find more information following this link: Install just the TensorFlow Lite interpreter
You have to remember, if you use interpreter only you have to follow a little bit different logic.
# Initiate the interpreter
interpreter = tf.lite.Interpreter(PATH_TO_SAVED_TFLITE_MODEL)
# Allocate memory for tensors
interpreter.allocate_tensors()
# Get input and output tensors
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
# Add a batch dimension if needed (data_tensor - your data input)
input_data = tf.extend.dims(data_tensor, axis=0)
# Predict
interpreter.set_tensor(input_details[0]['index'], data_tensor)
interpreter.invoke()
# Obtain results
predictions = interpreter.get_tensor(output_details[0]['index'])
Build from source for the Raspberry Pi
Install TensorFlow with pip