Use Nutch with newest Elasticsearch - apache

Based on this question I have managed to successfully integrate Nutch and Elasticsearch, albeit by downgrading my Elasticsearch version. How can I modify the Nutch source code to accommodate the latest version of Elasticsearch (0.90.2+)? I have tried modifying the Ivy dependency to this version of Elasticsearch and also modified Nutch's Elasticsearch compatibility code so it would build properly, but I end up with an error as Nutch times out waiting for a response from Elasticsearch; the two are unable to communicate.

I think I found the solution. You need to modify all references to the version number, both in ivy/ivy.xml and pom.xml (which is the file I forgot to change). Changing both 0.19.4s to 0.90.2s should do the trick. Also, you need to change item.failed() in src/java/org/apache/nutch/indexer/elastic/ElasticWriter.java to item.isFailed() to match the newer Elasticsearch refactoring.

Related

Can I use old apache solr indexed data after upgrading apache version?

I have updated the apache2 version form apache2.2.14 to apache2.4.7 and also apache-solr package form 1.4.x to 4.x.
Before upgradation, I have indexed all the content.
After upgradation, in apache configuration it showing 0% indexed.
is there any way to use old indexing?
Solr / Lucene only upgrades older index formats in smaller increments, so you'll have to at least stop by a 3.x release on the way to be able to use the 1.4 index formats.
I'd also recommend going from 4 to 5 as well, since you're already doing the upgrade now, and will be stuck in the past again if you don't do the 5.x upgrade as well (6.0 was just released).
My suggestion is to optimize for easy reindexing, and do that. You'll run into the same issue later, or after doing any major changes to your schema.
The index format is backward compatible between two consecutive major
Solr versions. So a Solr 3.x index is compatible with a Solr 4.x
index. However if you have a Solr 1.x index and want to upgrade to
Solr 4.x then you would need to first upgrade to Solr 3.x first.
upgrade between major Solr Versions
So you'll have to go via 3.x (as Mats says below) or reindex or use the IndexUpgrader tool on your index.
http://lucene.apache.org/core/4_0_0/core/org/apache/lucene/index/IndexUpgrader.html

Facing problems in Clearcase 8 plugin for Eclipse Luna for config spec update

I have been using Eclipse Helios/Luna with plugin installation of clearcase plugin 8.0.1.x version and have imported the base code.
Recently, when trying to change the config-spec to update the build files, the process is running for hours continuously even without a single file.
Previously, we had Clearcase version as 7.0.x version and the corresponding plugin installed on Eclipse Helios without facing any issues.
But, have been facing issues during update of files after the upgrade of clearcase server to 8.0.x and with the corresponding client plugin in eclipse.
Also, tried manual update from repository for the whole folder. Still no luck.
But, if I know manually update the file one by one individually, it is going fine.
As the number of files in the project is more, it would not be efficient for us.
Can someone provide some alternatives to resolve this problem?
As a workaround, we have been creating new views everytime there is an update to the config-spec.
One workaround would be to switch to dynamic views, supported in the latest 8.x version of ClearTeam/ClearCase. No reload needed with those kind of views.
The other would be to test with a minimal config spec, and load rules making sure you are loading only as few files as possible (jsut to check that those files do update when you change the config spec).
It can also depends on the exact version of your ClearCase installation.
There are some patches for hanging issue (like "PM48668 Problem: The IBM Rational ClearTeam Explorer may hang intermittently when many views are started.")

Pig versions and UDF

I am using pig version as 0.12,But for creating UDFs i am using the jar file of Pig 0.9 version.
I simply downloaded the jar file for Pig 0.9 version and added that in my eclipse classpath.
All the UDFs that I created using Pig 0.9 version API works fine.
But I would like to know the impact on that.
Is there any problem that I will face in future
The issues that you will face is API inconsistencies as time goes by. Some of the core APIs are relatively stable. Heck, most. But the longer you use an old Pig API the higher the chance you'll get an issue running in the cluster.
Something else to think about is are you overriding your Pig version in the cluster. For example, say you have an uber-jar with the pig scripts in it. If that JAR contains Pig v.09, you'll actually use that version rather than .12. By not migrating, you might be pulling in the wrong version of Pig.

Update OSGi Bundle in Apache Geronimo

I am an absolute novice with Apache Geronimo, so my forgive my obvious ignorance.
Apache Geronimo ships with older versions of the HTTPCore and HTTPClient libraries. The current version of OpenID requires at least v4.1 of both HTTPCore and HTTPClient. I have found the OSGi Bundle page in the Geronimo Console, and see how to use this to install a new bundle. I also see that existing bundles have an "update" option, although this does not seem to update either of these bundles to the latest version.
Is there an easy magical way to just update these? If I need to download bundles for the appropriate version, where, exactly, would I go to download them? I've located and downloaded JARs for these two libraries, and have included them in my project and been able to package it with Maven, but this doesn't (apparently) deploy the updated versions into Geronimo (i.e., it loads the old versions and I get runtime errors).
If I had bundles for the later version, the UI is pretty obvious as to how to install them - will that replace the existing version, or will I then have both versions installed? Will this present a runtime issue? If so, how do I remove the exiting bundle (I see no way via the UI to accomplish this).
Any assistance is greatly appreciated.
David Mullin

Trying to transfer older version of mediawiki to new server

So, in college I had a Debian server which used to host a wiki, with mediawiki version 1.9. This server stopped working, and all I have now is its HD. I want to transfer this wiki to a new server, which also runs Debian, but I can't do that with Debian's current stable version of mediawiki, 1.15, because it is not possible to transfer a wiki to another version of mediawii.
So, my idea is to install mediawiki 1.9 on the new server, and then move the wiki. But I am having problems with installing it.
When I go to http://my_hostname/config/index.php, to configure the new wiki, so that I can transfer the other one, I get the following message, and nothing happens:
Checking environment...
Please include all of the lines below when reporting installation problems.
PHP 5.3.3-7+squeeze8 installed"
I really don't have a clue on what is wrong. ANY help would be greatly appreciated!
Like you commented yourself, it would certainly be worth a try to using your existing configuration. The configuration is typically some site preferences and database configuration, so make sure dat your database is setup in the same way as before.
Regarding your configuration problems, maybe php filters away the errors. You can check this by searching for error_reporting in your php.ini.
Just follow the instructions at http://www.mediawiki.org/wiki/Manual:Upgrading. Probably no point in trying to install at MediaWiki 1.9.
You may want to do a clean install of MediaWiki 1.18.1 first just to make sure your system is ready, all dependencies met, etc.