Are there any really good testing frameworks for solr ? I heard about Randomized Continuous Testing . Does anyone know how to use Randomized Continuous Testing for solr ?
This question is far too broad. There are many test frameworks you can use for Solr (.. almost all of them), and you can use either a live backend or the Embeddable Solr Server for integration testing.
If you watch to the end of the presentation, Dawid gives several examples of how to do randomized testing with Solr and Lucene, including linking to LUCENE-3492 which tracks the issue in Lucene and Solr. There is a Wiki page that contains information about how to run the internal Solr tests.
Related
Currently we use soap-ui/manual posting of xmls etc for load testing our spring boot webapplications.
Looking for any free load testing tools that others have been using and want to recommend?
Thanks for your help!
The specific load testing or in general testing tool is the one you will "like" more by lots of personal/company needs. There are plenty of them, here is the short list of them I used:
Blazemeter
Gatling
JMeter
I presonally spent most of load testing time in Gatling, first of all it is using scala development language and quite easy to include in your Java project via maven/gradle, secondaly apart from other benefits it have direct JDBC connection possibilty which let's you to have your test data directly in database. Lots of other pros. But one more time it is strongly my opinion and my preferences. E.g. if you are big fan of XML you will most probably like JMetter, in general Blazemter is kind of a next level of JMeter.
I usually code the test using my current favorite language, Python in recent times. For example in python this would be some code around the requests library, and possibly some multi-process code- nothing to heavy or complex.
I find it more flexible to code myself, on average having more control over the load (there are pros and cons to using a prebuilt tool in this sense) and it usually integrates better with other code in my automation suite.
But the answer is somewhat context dependent, is there someone with the knowledge and resource to develop a tool ? do you have to document the results or make them comparable with other systems?
I am going to implement Lucene search into my project and I want to make a best start.
So I consider between 3 versions of Lucene (Java/C#.Net/C++) which is the best version upon these criterias :
1.performance
2.easy to implement
3.plenty of documents ?
Assume the system is Window server, and I ask it for a long-term use.
Thanks
I would say Java. Lucene was initially developed in Java and I would think there are bigger community, more documentation and bigger deployments using Java.
Granted, Windows is not usually considered as primary platform for deploying Java services but it still would work with flying colors. Many people using Windows for Java development and even deployment so I don't expect any major issues.
Unless you've got a specific feature you need, I would look at best being:
a) Whatever platform you are developing the program in -- there are lots of advantages to not having to switch tools/contexts/platforms to muck around with the search internals.
b) Whatever platform your ops guys want to deal with -- I know lots of windows ops guys hate dealing with java as it is a strange foreign language. For example.
c) All of the above being equal, Java is the real flagship lucene project that everyone else is keeping up with with and that has the most tools & resources. It is the way to go if you don't have any reason not to use java. Solr is another advantage here -- you can pretty easily use a pre-wrapped fully functional lucene http server.
In any case, keep in mind that at least theoretically any lucene index written on one platform is readable by others so you don't necessarily have to fully commit to a single platform.
We are developing a web application using Spring MVC, Spring and Hibernate.
We need to add efficient free text search capabilities to our applications. For this we are thinking of using either Hibernate Search (it uses Lucene under the hood) or directly lucene.
What is the best option for us as we are already using hibernate in our application? What are the pros and cons of one over the other?
Thanks.
You said it yourself - you'll be using Lucene one way or the other.
The raw Lucene API isn't very easy to use. It's much more low-level than Hibernate Search. if you're already using Hibernate, then it's a no-brainer - use Hibernate Search to implement your text search functionality.
disclaimer: I'm one of the developers of Hibernate Search.
The goal of the project is not to compete with Lucene nor Solr, but to facilitate as much as possible integration with Hibernate applications, to avoid having to maintain the two worlds in sync and duplicate all mapping and CRUD operations.
While we provide some common helpers and a nice encapsulation, Hibernate Search can also hand you over a direct reference to the Lucene API, so in case you find yourself needing to use the "raw" Lucene API you will never be stuck. Also for writing to the index Hibernate Search provides a common pattern which will solve most of known requirements, but in case you have very non-standard requirements you can get full control of the written Documents.
Solr is a good alternative, but as it is a separate server you have to interact with it via REST APIs which is quite different, with it's pros and cons. Having a second service to manage is not always wanted, and of course the remote invocations will never be as efficient as direct references to Lucene and to all it's internal filters and caches.
Not all functionality of Lucene can be exposed via a remote API, and if you need to do some "low level" operation, if this is not implemented in Solr you won't be able to do it (without patching Solr). Still Solr is very cute, especially when you want to share the index with other non-Java applications, and so we might add a Solr backend for Hibernate Search to eventually keep a Solr server in synch (especially if there's interest for it, and possibly some help).
Finally, the Lucene API is really hard core stuff. We spend a lot of effort to make the best use of it to provide top performance while exposing a stable API to people using Hibernate Search, basically until now all releases have been backwards compatible to provide a "drop-in" performance boost to use latest greatest tricks from Lucene - which actually changes API quite often; these changes are always exciting, but be prepared to maintain that in your application if you don't use a proper abstraction.
The other way of using Lucene is to get the middlman API which is known as SOLR. SOLR will connect to Lucene and perfom HTTP calls for search. Please note that you will need to build and Parse the XML what Solr consumes. All the functionality of Lucene is exponse via SOLR and should be really helpful.
I’m evaluating full text search methods for Rails 3 ATM. Does anyone here have a recommendation? Seems to me as if most of the known methods (Sunspot, Sphinx, Ferret, Xapian) aren’t yet ready for Rails 3. Is that so? At the moment I’ve got plenty of resources left on the machine were I’d like to deploy my app but nevertheless, I’d like to keep the idle load for the search engine as low as possible. I’m planning to use PostgreSQL if that’s of any relevance here.
After some reading I’m almost sure that I’d like to use Sunspot or Xapian. But if there’s any other (and better) solution please tell me :-) Especially regarding Sunspot I’m not sure if it was clever to have a complete Tomcat running in addition to my Rails app. Anyone has experience with this constellation?
Thanks in advance,
Ulf
If you are using PostgreSQL you can get an awful lot out of its built-in text search capabilities before you need to reach for external libraries. I've been using tsearch queries for years with excellent results.
PostgreSQL full text search analyses word proximity to calculate Relevance & ranking and offers useful features like highlighting of search results.
It is also aware of language specific normalisation rules, for example it knows to ignore the s and es pluralization suffixes in English; so searches for 'country' will also bring back highlighted results for 'countries', much the same way that Google does.
I'm not suggesting that you shouldn't use the libraries that you've mentioned, but it is worth investigating the database to see if will already fulfil the majority, if not all of your requirements.
You can use sunspot with Rails3, no problem. We have done so successfully using the sunspot/sunspot_rails gems (1.2.rc4). And it's not too much of a hassle to run Solr within a Tomcat server.
For fulltext-search features you should use a search engine.
For example you could use the Lucene Library with jRuby.
If you like to stay with standard Ruby (cRuby) you coud use Solr.
For rails there are also some Solr plugins:
For example starting with http://wiki.apache.org/solr/SolRuby could be a good idea.
Sunspot is Rails3 ready, we're using it on a few Rails3 apps already. I've had a lot of success with Solr and Sunspot. So much that we're starting a blog series on it
I'm looking into some example UI on top of Solr that show of the functionality available in a demo, like e.g. drill down faceted search. I found Blacklight, which looks intensively interesting. Is there any other software that is worth researching or is Blacklight definitive the way to go? Thanks.
Have you looked at using the Velocity templating built into Solr? You can find more about "Solritas" here: http://wiki.apache.org/solr/Solritas
I am about to put together a demo Solr site for a presentation, and am going down the Solritas route. You get faceting, clustering, and more! And no extra server to run.