How do I register "custom" Op (actually, from syntaxnet) with tensorflow serving? - tensorflow

Question

I'm trying to serve a model exported from syntaxnet but the parser_ops are not available. The library file with the ops is found (out-of-tree) at:
../models/syntaxnet/bazel-out/local-opt/bin/syntaxnet/parser_ops.so
I'm currently hacking the mnist_inference example, (because I don't know how to build anything out-of-tree with bazel), and the command I'm running is:
./bazel-out/local-opt/bin/tensorflow_serving/example/mnist_inference --port=9000 /tmp/model/00000001
And the error I'm getting is:
F tensorflow_serving/example/mnist_inference.cc:208] Check failed: ::tensorflow::Status::OK() == (bundle_factory->CreateSessionBundle(bundle_path, &bundle)) (OK vs. Not found: Op type not registered 'FeatureSize')
And FeatureSize is definitely defined in the parser_ops.so, I just don't know how to load it.

Answer 1

I'm not too familiar with TF (I work on Bazel) but it looks like you need to add parser_ops as a dependency of mnist_inference.
There is a right way to do this and a wrong (easier) way.
The Right Way
Basically you add syntaxnet as a dependency of the example you're building. Unfortunately, the syntax net project and the tensorflow serving project import tensorflow itself under different names, so you have to do some mangling of the serving WORKSPACE file to get this working.
Add the following to the tensorflow_serving WORKSPACE file:
local_repository(
name = "syntaxnet",
path = "/path/to/your/checkout/of/models/syntaxnet",
)
This allows you to refer to the targets in syntaxnet from the tensorflow project (by prefixing them with "#syntaxnet"). Unfortunately, as mentioned above, you also have to get all of syntaxnet's external dependencies into the WORKSPACE file, which is annoying. You can test out if it's working with bazel build #syntaxnet//syntaxnet:parser_ops_cc.
Once you've done that, then add the cc_library #syntaxnet//syntaxnet:parser_ops_cc (parser_ops.so is a cc_binary, which can't be used as a dependency) to mnist_inference's deps:
deps = [
"#syntaxnet//syntaxnet:parser_ops_cc",
"#grpc//:grpc++",
...
Note that this still won't quite work: parser_ops_cc is a private target in syntaxnet (so it can't be depended on from outside its package) but you could add an attribute to it like visibility = ["//visibility:public"] if you're just trying things out:
cc_library(
name = "parser_ops_cc",
srcs = ["ops/parser_ops.cc"],
visibility = ["//visibility:public"]
...
The Wrong Way
You have a .so, which you can add a src file for your binary. Add the directory it's in as a new_local_repository() and add it to srcs in the BUILD file.
WORKSPACE file:
new_local_repository(
name = "hacky_syntaxnet",
path = "/path/to/syntaxnet/bazel-out/local-opt/bin/syntaxnet",
build_file_content = """
exports_files(glob(["*"])) # Make all of the files available.
""",
)
BUILD file:
srcs = [
"mnist_inference.cc",
"#hacky_syntaxnet//:parser_ops.so"
],