I read and re-read the doc, but I still don't understand the difference between, table_regions and table_areas
For me this two-parameter makes the same thing..., but the doc specifies then table_regions work on approximate region.
I think the doc can be more specific about what means approximate region and what is the difference with table_areas
I hope someone can explain to me clearly the difference between these two functions
I suggested the introduction of this feature (https://github.com/atlanhq/camelot/issues/209).
Using table_areas, you can specify the actual coordinates of the table boundary.
Camelot finds a table which have exactly these coordinates.
Instead, using table_regions, you can specify the region of the page where Camelot search for tables. Camelot finds all the tables which reside inside this region.
Feel free to ask for clarification.
Related
Is there a way to compare the similarity between sentences in sql? I have large dataset and I need to identify instances where there are similar words in a two or more setences.
How do I tell SQL to only return the values below?
From what I have googled, there may be a way to do this using a Full-Text Search and Semantic Search, but I have been able to find an article that addresses what I am trying to achieve.
Could someone in the group, provide me example or point to an article that could help me? Better yet, is what I am trying to do even achievable in SQL.
No, there is not.
Part of the problem is that "similarity" is a complex setup and this requires a program to analyze the sentence POSSIBLY with months of programming. You give pretty simplistic examples - grats. Even that is not as easy as you think. What about "the small boy wear red t-shirt" - would small boy be a difference or not?
This requires a LOT of work, and a LOT of definition, or a LOT of training of possibly a multi layer neural network.
SQL generally is awful at string manipulation - the best you get is SOUNDEX and that just compares 4 letters of the first word (RTFM, it is actually QUITE interesting how it works, but it makes it absolutely unsuitable for anything like comparing sentences.
So, no - this is simply way outside the scope of anything in SQL, you will have to download the data and use an out of SQL approach (which is also a LOT more fit for this type of work).
You can obviously work around that with simplistic SQL such as #ASH was suggesting - but this is not looking for "similar sentences" but working around specific markers that ARE SPECIFIC FOR YOUR DATA SET. THis is overfitting and bypassing answering the question you have asked.
You can try SOUNDEX function. Google SOUNDEX and then understand if this works for your case. The query is:
SELECT *
FROM your_table
WHERE SOUNDEX(Sentence) = SOUNDEX(Sentence);
I was reading some interesting questions about the topic "Can we make a program that, given a particular sequence, produces the next terms", like this one, and I really like the detailed answer of this one. I understand that the answer is "That's impossible without more restrictions", and that given some restrictions (polynomials, rational function or boolean map) we know some good algorithms, as the second answer I linked explains.
Now, a natural question is how much can we solve, trying our best even if we can't always solve it, to answer the original, general question. What I usually do when facing a hard sequence is trying to see if it's in OEIS, and if it seems to be there, seeing if there is any formula or algorithm to produce it in there. You can download a small version of OEIS with the first terms of each sequence, and you can make queries to find formulas or maple algorithms for a particular sequence. My question is, do you think it's feasible to download a small version of OEIS that includes, with the first terms, a little algorithm to produce it?
The natural problem here is that I haven't seen any link to download the entire database of OEIS with all the details, which maybe deserves its own question. Even if we had this, you need to read the formulas/algorithms (that can be written in different languages, from what I've seen) and interpret them correctly. But I thought maybe someone here knows how to solve this, in any case thanks in advance.
You could, as you note, download the sequences and their A-numbers from the link mentioned here: https://oeis.org/wiki/Welcome#Compressed_Versions
After searching that and finding one sequence (or a small number of sequences) of interest, you could scrape the respective page(s) for formulas. There are specific fields for Maple and Mathematica, which may be helpful, and otherwise, an entry in the PROGRAM field should include identifying information when it is not one of the standard languages with its own field in the database. See: http://oeis.org/wiki/Style_Sheet
Unofficially, but with the interests of the OEIS in mind, I would not recommend trying to download or scrape the OEIS in its entirety. Whether it's one person, or a whole host of people, we would certainly recommend using the compressed version of the database to identify sequences of interest by A-number first, then pulling their entire entry by scraping the site or querying the OEIS using methods that you have already mentioned: Programmatic access to On-Line Encyclopedia of Integer Sequences
If this sounds laborious, perhaps an alternative is the Wolfram Cloud, which actives this through other means. For example, you can navigate to the cloud (you may have to register just to get access) at: https://www.wolframcloud.com/
Typing in something like FindSequenceFunction[{1, 2, 3, 5, 17, 305, 34865}] will give you a formula, if Wolfram/Mathematica can find one. The documentation for FindSequenceFunction can be found here: https://reference.wolfram.com/language/ref/FindSequenceFunction.html
Wolfram/Mathematica can also invoke the OEIS using packages like the one described here: https://mathematica.stackexchange.com/questions/40/is-it-possible-to-invoke-the-oeis-from-mathematica
I am going to attend the Instance Matching of OAEI, now I need to make my results to Alignment Format. In order to achieve it, I have learned official tutorials.(link:http://alignapi.gforge.inria.fr/tutorial/tutorial1/index.html).
But there are many differences between the method taught and the method I want. In other words, I can't understand the API.
This is my situation:
I have 2 rdf file(person11.rdf and person12.rdf respectively.data link is http://oaei.ontologymatching.org/2010/im/index.html, the PR dataset), each file has information of many person. I want to find the coreferent entities, the results must be printed in Alignment Format. I find the results by using SPARQL, but I don't know how to print it in Alignment Format.
So, I have three questions:
First, if I want to generate a Alignment Format file, is the method taught the only way?
Second, can you give me your method(code better) to generate the Alignment Format file? Maybe I am wrong from the beginning, can you give me some suggestions?
Third, if you attended OAEI or know something about Instance Matching, can you give me some advice? I want to find the coreferent entities.
Thank you!
First question: I guess that the "mentioned method" is the one in tutorial1. It is not the appropriate one since you have to write a program to output the alignment format and this is a command line interface tutorial. In this case, you'd better look at http://alignapi.gforge.inria.fr/tutorial/tutorial2/index.html
Then, there are basically two ways to do:
The advised one (for several reasons and for participating to OAEI) is to follow these tutorials, to create an empty alignment in it, to create the correspondences from the results of your SPARQL query and to render it. Everything is covered by the tutorials but the part concerning your SPARQL queries. This assumes that you are programming in Java.
The non-advised solution (primarily non advised because you will have to debug your own renderer), is to write, in any programming language that you want a program that output the format (which corresponds to what you cite).
Think about it: how would you expect that the Alignment API knows the results of your SPARQL query? If you come up with a nice solution, contact the API developers, they may integrate it and others could benefit.
Second question: I cannot do better than what is above.
Third question: too general. Read the OAEI results (http://oaei.ontologymatching.org) and look at the code of others.
Good luck!
For a contract work, I need to digitalize a lot of old, scanned-graphic-only plenary debate protocol PDFs from the Federal Parliament of Germany.
The problem is that most of these files have a two-column format:
Sample Protocol http://sert.homedns.org/img/btp12001.png
I would love to read your answer to my following questions:
How I can split the two columns before feeding them into OCR?
Which commercial, open-source OCR software or framework, do you recommend and why?
Please note that any tool, programming-language, framework etc. is all fine. Don't hesitate recommend esoteric products, libraries if you think they are cut for the jub ^__^!!
UPDATE: These documents are already scanned by the parliament o_O: sample (same as the image above) and there are lots of them and I want to deliver on the contract ASAP so I can't go fetch print copies of the same documents, cut and scan them myself. There are just too many of them.
Best Regards,
Cetin Sert
Cut the pages down the middle before you scan.
It depends what OCR software you are using. A few years ago I did some work with an OCR API, I cant quite remember the name but I think there's lots of alternatives. Anyway this API allowed me to define regions on the page to OCR, If you always know roughly where the columns are you could use an SDK to map out parts of the page.
I use Omnipage 17 for such things. It has an batchmode too, where you can put the documents in an folder, where they was grabed, and put the result into another.
It autorecognit the layout, include columns, or you can set the default layout to columns.
You can set many options how the output should look like.
But try a demo, if it goes correct. I have at the moment problems with ligaturs in some of my documents. So words like "fliegen" comes out as "fl iegen" so you must spell them.
Take a look at http://www.wisetrend.com/wisetrend_ocr_cloud.shtml (an online, REST API for OCR). It is based on the powerful ABBYY OCR engine. You can get a free account and try it with a few of your images to see if it handles the 2-column format (it should be able to do it). Also, there are a bunch of settings you can play with (see API documentation) - you may have to tweak some of them before it will work with 2 columns. Finally, as a solution of last resort, if the 2-column split is always in the same place, you can first create a program that splits the input image into two images (shouldn't be very difficult to write this using some standard image processing library), and then feed the resulting images to the OCR process.
Can someone explain what a SpanQuery is, and what are typical use cases for it?
The documentation is very laconic, and keeps mentioning the concept of "span", which I'm not quite sure I get.
Spans provide a proximity search feature to Lucene. They are used to find multiple terms near each other, without requiring the terms to appear in a specified order. You can specify the terms that you want to find, and how close they must be. You can combine these span queries with each other or with other types of Lucene queries.
Found this all about the SpanQuery
A span query is a query that returns infomation about where in a document each match took place. You use the getSpans() method to get the locations.
The following deck of slides (unfortunately in Powerpoint) contain an example: http://www.cnlp.org/apachecon2005/AdvancedLucene.ppt
The javadocs you linked to are for a class in the " org.apache.lucene.search.spans " package. if you had clicked on the "package" link on those javadocs you would have been taken to...
https://lucene.apache.org/core/4_10_0/core/org/apache/lucene/search/spans/package-summary.html
...where the concept of Spans and what a Span is are explained in depth.