FlatBuffer supports random access. But I couldn't find any example/tutorial on how to do it in Java.
I looked into this article on how facebook uses FlatBuffer: https://code.facebook.com/posts/872547912839369/improving-facebook-s-performance-on-android-with-flatbuffers/
They are using some FlatBufferHelper class, which I couldn't find anywhere.
"random access" means that in FlatBuffers you can take a buffer of serialized data, and using the accessor functions, access any field of any object inside of it in any order.
To learn how to use it, you want to go here https://google.github.io/flatbuffers/flatbuffers_guide_tutorial.html and select "Java". The article you linked is about FlatBuffers at Facebook, and they may have their own (non-public) code.
Related
I need to know the equivalent C++ function for https://www.tensorflow.org/api_docs/python/tf/train/import_meta_graph
Can anyone please help with it?
There is no direct equivalent of tf.train.import_meta_graph() in C++, because its main role is to set up Python data structures that are useful in interpreting the low-level GraphDef that the MetaGraph contains.
There are a couple of related functions that might be useful, however:
In the C API, TF_LoadSessionFromSavedModel() will import a "SavedModel" file, which contains one or more meta graphs, and instantiate a session using it. You can create a saved model in Python using the tf.saved_model.builder.SavedModelBuilder class. See the SavedModel documentation for more details.
In the C++ API, the tensorflow::LoadSavedModel() function serves a similar role.
I want to use several encodings in the presentation layer to encode a object/structure in the application layeri independenty from encoding scheme (such as binary, XML, etc) and programming language (Java, Javascript, PHP, C).
An example would be to transfer an object from a producer to a consumer in a byte stream. The Java client would encode it using something like this:
Object var = new Dog();
output.writeObject(var);
The server would share the Dog class definitions and could regenerate the object doing something like this:
Object var = input.readObject();
assertTrue(var instanceof Dog); // passes
It is important to note that producer and consumer would not share the type of var, and, therefore, the consumer would not need the type to decode var. They only would share data type definitions, if ever:
public interface Pojo {}
public class Dog implements Pojo { int i; String s; } // Generated by framework from a spec
What I found:
Java Serialization: It is language dependent. Cannot be used with for example javascript.
Protobuf library: It is limited to a specific binary format. It is not possible to support additional binary formats. Need name of class ("class" of message).
XStream, Simple, etc. They are rather limited to text/XML and require name of the class.
ASN.1: The standards are there and could be used with OBJECT IDENTIFIER and type definitions but they lack on documentation and tutorials.
I prefer 4th option because, among others, it is a standard. Is there any active project that support such requirements (specially something based on ASN.1)? Any usage example? Does the project include codecs (DER, BER, XER, etc.) that can be selected at runtime?
Thanks
You can find several open source and commercial implementation of tools for ASN.1. These usually include:
a compiler for the schema, which will generate code in your desired programming language
a runtime library which is used together with the generated code for encoding and decoding
ASN.1 is mainly used with the standardized communication protocols for telecom industry, so the commercial tools have very good support for the ASN.1 standard and various encoding rules.
Here are some starter tutorials and even free e-books:
http://www.oss.com/asn1/resources/asn1-made-simple/introduction.html
http://www.oss.com/asn1/resources/reference/asn1-reference-card.html
http://www.oss.com/asn1/resources/books-whitepapers-pubs/asn1-books.html
I know that the OSS ASN.1 commercial tools (http://www.oss.com/asn1/products/asn1-products.html) will support switching the encoding rules at runtime.
To add to bosonix's answer, there's also Objective System's tools at http://www.obj-sys.com/. The documentation from both OSS and Objective Systems includes many example uses.
ASN.1 is pretty much perfect for what you're looking for. I know of no other serialisation system that does this quite so thoroughly.
As well as a whole array of different binary encodings (ranging from the comprehensively tagged BER all the way down to the very packed-together PER), it does XML and now also JSON encodings too. These are well standardised by the ITU, so it is in theory fully inter operable between tool vendors, programming languages, OSes, etc.
There are other significant benefits to ASN.1. The schema language lets you define constraints on the value of message fields, or the sizes of arrays. These then get checked for you by the generated code. This is far more complete than many other serialisations. For instance, Google Protocol Buffers doesn't let you do this, meaning that you have to check the range of message fields (where applicable) in hand written code. That's tedious, error prone, and hard to maintain.
The only other ones that do this are XSD and JSON schemas. However with those you're at the mercy of the varying quality of tools used to turn those into source code - I've not yet seen any decent ones for JSON schemas. I'm not aware of whether or not Microsoft's xsd.exe honours such constraints either.
I ingested large binary into MarkLogic using the content ingestion framework, leaving the binary files on the file system, and I used the transformation to extract metadata from the images into properties. When I search on this content using the search API it does not return facets. I believe that this happens because the fragment returned contains the pointer to the image on the file system and not the properties document. Is there any way around this? I'd like to created faceted navigation base upon the properties.
If you take a look at the Search Developer's Guide for 5.0, section 2.2.6 talks about the fragment scope option that is new in 5.0, I think that will handle your case. There's an example in there showing how to create a facet on the last-modified property using a local fragment scope, and it sounds like that pattern might be what you're looking for.
If the search API doesn't handle this use-case, you could always call cts:element-values and cts:frequency yourself. You can still use search:parse and search:resolve to provide query parsing and basic search results.
http://docs.marklogic.com/5.0doc/docapp.xqy#search.xqy?start=1&cat=all&query=cts%3Aelement-values&button=search
I'd like to store some additional information along with a document, but I can't use bundles or packages, and I cannot store it inside the document itself.
The application is a text editor, and I'd like it to store code folding and bookmark locations with the document, but obviously this cannot be embedded into the code directly, and I don't want to alter the code with ugly comments.
Can I use NSDocument to store information in the resource fork of a document? If so, how can I do this? Should I directly write to <filename>/..namedfork/rsrc or is there an API available?
First, don't use the resource fork. It's virtually deprecated. Instead, use extended attributes. They can be set programmatically at the BSD level via setxattr and getxattr. Extended attributes are used in many places... for example, in the latest OS X, the resource fork itself is implemented as a special type of extended attributes.
For example, the Cocoa text system automatically adds an extended attribute to a file to specify the encoding.
I thought NSFileManager and NSFileWrapper supported extended attributes since Snow Leopard, but I can't find any documentation :p You can always use the BSD level functions, though.
Does the state need to move with the file if it's copied to another computer? If not, you could do a lot worse than emulating the way Bare Bones handles document state with BBEdit. They store state for all documents in ~/Library/Preferences/com.barebones.bbedit.PreferenceData/Document State.plist.
The resource fork documentation is here. But it contains plenty of suggestions to not use the resource fork.
I have a class on my web site for reading and writing resource forks, which I have never got around to moving to my GitHub repository because, as Yuji points out, they are not really used any more.
I was going to say alias files and web Internet location file are the only places they are used, but I used and tested it on Mac OS X v10.7 (Lion), and they are not even used there any more; they may still be used for custom icons. I didn't test for that exclusively. I will have to see how that affect my NDAlias class on 10.7.
ndresourcefork
I would like to know some libraries in objective-C for xml parsing. I think it is a very common need, but I found limited resources for handling this task:
Google Code projects: TouchCode (TouchXML)
NSXMLParser
What is your best solution to work with XML in objective-C language? Please advice.
What is the solution that you have used for your product?
NSXMLParser is a stream-oriented class; you set it up and get delegate callbacks when it detects something. Usually this is not what you want to do, but can be much faster and lower memory.
TouchXML will parse the XML itself using libxml, and create an object tree for the entire XML structure. This allows you to easily access the contents of the XML tree, using manual traversal methods or basic XPaths (more sophisticated XPath support is planned).
It serves a narrow purpose, but if your goal is to parse untidy HTML, you might want to try a static library I started called TagScraper. It doesn't handle many/most XML/HTML entities correctly, but it could be pretty easily patched to. URL: http://github.com/searls/TagScraper
Its value is that it provides a simple XPath mechanism that hides the tidying/querying/assembling for you, and then it provides the parsed elements & attributes in a tree-like data structure of Tag.h nodes.