I have 2 server nodes running collocated with spark worker. I am using shared ignite RDD to save my dataframe. My code works fine when I work with only one server node stared, if I start both server nodes code fails with
Grid is in invalid state to perform this operation. It either not started yet or has already being or have stopped [gridName=null, state=STOPPING]
DiscoverySpi is configured as below
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<!--
Ignite provides several options for automatic discovery that can be used
instead os static IP based discovery. For information on all options refer
to our documentation: http://apacheignite.readme.io/docs/cluster-config
-->
<!-- Uncomment static IP finder to enable static-based discovery of initial nodes. -->
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<!--<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder">-->
<property name="shared" value="true"/>
<property name="addresses">
<list>
<!-- In distributed environment, replace with actual host IP address. -->
<value>v-in-spark-01:47500..47509</value>
<value>v-in-spark-02:47500..47509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
I know this exception generally means ignite instanace either not started or stopped and operation tried with same, but I don't think this is the case for reasons that with single server node it works fine and also I am not explicitly closing ignite instance in my program.
Also in my code flow I do perform operations in transaction which works, so it is like
create cache1 : works fine
Create cache2 : works fine
put value in cache1 ; works fine
igniteRDD.saveValues on cache2 : This step failes with above mentioned exception.
USE this link for complete error trace
caused by part is pasted below here also
Caused by: java.lang.IllegalStateException: Grid is in invalid state to perform this operation. It either not started yet or has already being or have stopped [gridName=null, state=STOPPING]
at org.apache.ignite.internal.GridKernalGatewayImpl.illegalState(GridKernalGatewayImpl.java:190)
at org.apache.ignite.internal.GridKernalGatewayImpl.readLock(GridKernalGatewayImpl.java:90)
at org.apache.ignite.internal.IgniteKernal.guard(IgniteKernal.java:3151)
at org.apache.ignite.internal.IgniteKernal.getOrCreateCache(IgniteKernal.java:2739)
at org.apache.ignite.spark.impl.IgniteAbstractRDD.ensureCache(IgniteAbstractRDD.scala:39)
at org.apache.ignite.spark.IgniteRDD$$anonfun$saveValues$1.apply(IgniteRDD.scala:164)
at org.apache.ignite.spark.IgniteRDD$$anonfun$saveValues$1.apply(IgniteRDD.scala:161)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:883)
at org.apache.spark.rdd.RDD$$anonfun$foreachPartition$1$$anonfun$apply$28.apply(RDD.scala:883)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1897)
at org.apache.spark.SparkContext$$anonfun$runJob$5.apply(SparkContext.scala:1897)
at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:70)
at org.apache.spark.scheduler.Task.run(Task.scala:85)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:274)
... 3 more</pre>
It looks like the node embedded in the executor process is stopped for some reason while you are still trying to run the job. To my knowledge the only way for this to happen is to stop the executor process. Can this be the case? Is there anything in the log except the trace?
I am working on Gemfire and Spring data caching. I have successfully startup a local cache server from Spring. But I could not connect to a remote cache server with following configuration. But I can connect remote server using Gfsh--> connect --locator = remoter IP[10334]
<gfe:client-cache id="client-cache" pool-name="my-pool">
</gfe:client-cache>
<gfe:pool id="my-pool" subscription-enabled="true">
<gfe:locator host="remote ip" port="10334" />
</gfe:pool>
<gfe:client-region id="Customer" name="Customer" cache-ref="client-cache">
<gfe:cache-listener>
<bean class="com.demo.util.LoggingCacheListener" />
</gfe:cache-listener>
</gfe:client-region>
<bean id="cacheManager"
class="org.springframework.data.gemfire.support.GemfireCacheManager">
<property name="regions">
<set>
<ref bean="Customer" />
</set>
</property>
</bean>
The issue log is "Unable to prefill pool to minimum because:
com.gemstone.gemfire.cache.client.NoAvailableLocatorsException: Unable to connect to any locators in the list [remoeserver:10334]"
After I started another locator and server on my desktop, Spring can connect the cluster. But it said the region did not exist when Spring #Cachable is fired. The error log is Request processing failed; nested exception is "Region named /Customer/Customer was not found during get request". The region name should be /Customer.
A client region is merely a proxy to a master region (e.g., partitioned or replicated region) configured on a cache server. The server must also be configured use the same locator address.
I have connection string, but it isn't working. Why?
Error:
A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: SQL Network Interfaces, error: 26 - Error Locating Server/Instance Specified)
Connection string:
<add name="CompanyEventConnectionString"
connectionString="database=db;server=PETLITSKY-R\SQLEXP;user id=sas; password=P#ssword1; MultipleActiveResultSets=True;" />
In the SQL Server Configuration Manager->SQL Server Network Configuration->Protocols, check that named pipes and tcp/ip are enabled.
If that doesn't work then it may be your firewall blocking it, try this link: http://msdn.microsoft.com/en-us/library/cc646023.aspx
This may be useful too: http://support.microsoft.com/kb/287932
<add name="ConString" connectionString="Data Source=.;InitialCatalog=ServerMaintenance;User Id=sa;Password=something;" />
Is there a way to browse the content of an H2 or an HSQLDB in-memory database for viewing? For example, during a debugging session with Hibernate in order to check when the flush is executed; or to make sure the script that instantiates the DB gives the expected result.
Does it exist an addon or a library that you can embed with your code in order to allow this?
Please, mention which one you're talking about (H2 or HSQLDB) in case you have an answer specific to one of them.
You can run H2 web server within your application that will access the same in-memory database. You can also access the H2 running in server mode using any generic JDBC client like SquirrelSQL.
UPDATE:
Server webServer = Server.createWebServer("-web,-webAllowOthers,true,-webPort,8082").start();
Server server = Server.createTcpServer("-tcp,-tcpAllowOthers,true,-tcpPort,9092").start();
Now you can connect to your database via jdbc:h2:mem:foo_db URL within the same process or browse the foo_db database using localhost:8082. Remember to close both servers. See also: H2 database in memory mode cannot be accessed by Console.
You can also use Spring:
<bean id="h2Server" class="org.h2.tools.Server" factory-method="createTcpServer" init-method="start" destroy-method="stop" depends-on="h2WebServer">
<constructor-arg value="-tcp,-tcpAllowOthers,true,-tcpPort,9092"/>
</bean>
<bean id="h2WebServer" class="org.h2.tools.Server" factory-method="createWebServer" init-method="start" destroy-method="stop">
<constructor-arg value="-web,-webAllowOthers,true,-webPort,8082"/>
</bean>
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" destroy-method="close" depends-on="h2Server">
<property name="driverClass" value="org.h2.Driver"/>
<property name="jdbcUrl" value="jdbc:h2:mem:foo_db"/>
</bean>
BTW you should only depend on assertions and not on manual peeking the database contents. Use this only for troubleshooting.
N.B. if you use Spring test framework you won't see changes made by a running transaction and this transaction will be rolled back immediately after the test.
For H2, you can start a web server within your code during a debugging session if you have a database connection object. You could add this line to your code, or as a 'watch expression' (dynamically):
org.h2.tools.Server.startWebServer(conn);
The server tool will start a web browser locally that allows you to access the database.
In H2, what works for me is:
I code, starting the server like:
server = Server.createTcpServer().start();
That starts the server on localhost port 9092.
Then, in code, establish a DB connection on the following JDBC URL:
jdbc:h2:tcp://localhost:9092/mem:test;DB_CLOSE_DELAY=-1;MODE=MySQL
While debugging, as a client to inspect the DB I use the one provided by H2, which is good enough, to launch it you just need to launch the following java main separately
org.h2.tools.Console
This will start a web server with an app on 8082, launch a browser on localhost:8082
And then you can enter the previous URL to see the DB
With HSQLDB, you have several built-in options.
There are two GUI database managers and a command line interface to the database. The classes for these are:
org.hsqldb.util.DatabaseManager
org.hsqldb.util.DatabaseManagerSwing
org.hsqldb.cmdline.SqlTool
You can start one of the above from your application and access the in-memory databases.
An example with JBoss is given here:
http://docs.jboss.org/jbpm/v3.2/userguide/html/ch07s03.html
You can also start a server with your application, pointing it to an in-memory database.
org.hsqldb.Server
For HSQLDB, The following worked for me:
DatabaseManager.threadedDBM();
And this brought up the GUI with my tables and data once I pointed it to the right named in-mem database.
It is basically the equivalent of newing up a DatabaseManager (the non Swing variety), which prompts for connection details, and is set to --noexit)
I also tried the Swing version, but it only had a main, and I was unsure of the arguments to pass. If anyone knows, please post here.
Just because I searched for hours for the right database name: The name of the database is the name of your datasource. So try with URL jdbc:hsqldb:mem:dataSource if you have a data source bean with id=dataSource. If this does not work, try testdb which is the default.
You can expose it as a JMX feature, startable via JConsole:
#ManagedResource
#Named
public class DbManager {
#ManagedOperation(description = "Start HSQL DatabaseManagerSwing.")
public void dbManager() {
String[] args = {"--url", "jdbc:hsqldb:mem:embeddedDataSource", "--noexit"};
DatabaseManagerSwing.main(args);
}
}
XML context:
<context:component-scan base-package="your.package.root" scoped-proxy="targetClass"/>
<context:annotation-config />
<context:mbean-server />
<context:mbean-export />
This is a Play 2 controller to initialize the H2 TCP and Web servers:
package controllers;
import org.h2.tools.Server;
import play.mvc.Controller;
import play.mvc.Result;
import java.sql.SQLException;
/**
* Play 2 controller to initialize H2 TCP Server and H2 Web Console Server.
*
* Once it's initialized, you can connect with a JDBC client with
* the URL `jdbc:h2:tcp://127.0.1.1:9092/mem:DBNAME`,
* or can be accessed with the web console at `http://localhost:8082`,
* and the URL JDBC `jdbc:h2:mem:DBNAME`.
*
* #author Mariano Ruiz <mrsarm#gmail.com>
*/
public class H2ServerController extends Controller {
private static Server h2Server = null;
private static Server h2WebServer = null;
public static synchronized Result debugH2() throws SQLException {
if (h2Server == null) {
h2Server = Server.createTcpServer("-tcp", "-tcpAllowOthers", "-tcpPort", "9092");
h2Server.start();
h2WebServer = Server.createWebServer("-web","-webAllowOthers","-webPort","8082");
h2WebServer.start();
return ok("H2 TCP/Web servers initialized");
} else {
return ok("H2 TCP/Web servers already initialized");
}
}
}
I've a problem with H2 version 1.4.190 remote connection to inMemory (as well as in file) with Connection is broken: "unexpected status 16843008" until do not downgrade to 1.3.176. See Grails accessing H2 TCP server hangs
This is more a comment to previous Thomas Mueller's post rather than an answer, but haven't got enough reputation for it. Another way of getting the connection if you are Spring JDBC Template is using the following:
jdbcTemplate.getDataSource().getConnection();
So on debug mode if you add to the "Expressions" view in Eclipse it will open the browser showing you the H2 Console:
org.h2.tools.Server.startWebServer(jdbcTemplate.getDataSource().getConnection());
Eclipse Expressions View
H2 Console
I don't know why is it working fine at yours machines, but I had to spend a day in order to get it is working.
The server works with Intellij Idea U via url "jdbc:h2:tcp://localhost:9092/~/default".
"localhost:8082" in the browser alse works fine.
I added this into the mvc-dispatcher-servlet.xml
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close" depends-on="h2Server">
<property name="driverClassName" value="org.h2.Driver"/>
<property name="url" value="jdbc:h2:tcp://localhost:9092/~/default"/>
<property name="username" value="sa"/>
<property name="password" value=""/>
</bean>
<bean id="h2Server" class="org.h2.tools.Server" factory-method="createTcpServer" init-method="start" destroy-method="stop" depends-on="h2WebServer">
<constructor-arg>
<array>
<value>-tcp</value>
<value>-tcpAllowOthers</value>
<value>-tcpPort</value>
<value>9092</value>
</array>
</constructor-arg>
</bean>
<bean id="h2WebServer" class="org.h2.tools.Server" factory-method="createWebServer" init-method="start" destroy-method="stop">
<constructor-arg>
<array>
<value>-web</value>
<value>-webAllowOthers</value>
<value>-webPort</value>
<value>8082</value>
</array>
</constructor-arg>
</bean>
What about comfortably viewing (and also editing) the content over ODBC & MS-Access, Excel?
Softwareversions::
H2 Version:1.4.196
Win 10 Postgres ODBC Driver Version: psqlodbc_09_03_0210
For Win7 ODBC Client: win7_psqlodbc_09_00_0101-x64.msi
H2 Server:
/*
For JDBC Clients to connect:
jdbc:h2:tcp://localhost:9092/trader;CIPHER=AES;IFEXISTS=TRUE;MVCC=true;LOCK_TIMEOUT=60000;CACHE_SIZE=131072;CACHE_TYPE=TQ
*/
public class DBStarter {
public static final String BASEDIR = "/C:/Trader/db/";
public static final String DB_URL = BASEDIR + "trader;CIPHER=AES;IFEXISTS=TRUE;MVCC=true;LOCK_TIMEOUT=10000;CACHE_SIZE=131072;CACHE_TYPE=TQ";
static void startServer() throws SQLException {
Server tcpServer = Server.createTcpServer(
"-tcpPort", "9092",
"-tcpAllowOthers",
"-ifExists",
// "-trace",
"-baseDir", BASEDIR
);
tcpServer.start();
System.out.println("H2 JDBC Server started: " + tcpServer.getStatus());
Server pgServer = Server.createPgServer(
"-pgPort", "10022",
"-pgAllowOthers",
"-key", "traderdb", DB_URL
);
pgServer.start();
System.out.println("H2 ODBC PGServer started: " + pgServer.getStatus());
}
}
Windows10 ODBC Datasource Configuration which can be used by any ODBC client:
In Databse field the name given in '-key' parameter has to be used.
I have JRE 1.6 and am using the following hibernate.cfg.xml file. I am always getting "Cannot open connection" and "The port number 1433/DB is not valid."
<hibernate-configuration>
<session-factory>
<property name="hibernate.connection.driver_class">com.microsoft.sqlserver.jdbc.SQLServerDriver</property>
<property name="hibernate.connection.url">jdbc:sqlserver://IP/DB</property>
<property name="hibernate.connection.username"></property>
<property name="hibernate.connection.password"></property>
<property name="hibernate.connection.pool_size">10</property>
<property name="show_sql">true</property>
<property name="dialect">org.hibernate.dialect.SQLServerDialect</property>
<property name="hibernate.hbm2ddl.auto">update</property>
</session-factory>
</hibernate-configuration>
From the official documentation:
Building the Connection URL
The general form of the connection URL is
jdbc:sqlserver://[serverName[\instanceName][:portNumber]][;property=value[;property=value]]
where:
jdbc:sqlserver:// (Required) is known as the sub-protocol and is constant.
serverName (Optional) is the address of the server to connect to. This could be a DNS or IP address, or it could be localhost or 127.0.0.1 for the local computer. If not specified in the connection URL, the server name must be specified in the properties collection.
instanceName (Optional) is the instance to connect to on serverName. If not specified, a connection to the default instance is made.
portNumber (Optional) is the port to connect to on serverName. The default is 1433. If you are using the default, you do not have to specify the port, nor its preceding ':', in the URL.
property (Optional) is one or more option connection properties. For more information, see Setting the Connection Properties. Any property from the list can be specified. Properties can only be delimited by using the semicolon (';'), and they cannot be duplicated.
So use the following instead:
jdbc:sqlserver://IP;databaseName=DB
You are connecting to host/instance. It should be a backslash: host\instance. Are you mixing up the concepts of instance and databases?
host\instance;databaseName=DB