I am trying to connect to BigQuery for migrating data and I am currently facing some issues in connection.
2 approaches which I am currently focusing on
Database Connector with passing JDBC connection string
<db:config name="Database_Config" doc:name="Database Config" doc:id="07671c43-86bc-4768-b914-c71058120615" >
<db:generic-connection url="jdbc:bigquery://https://www.googleapis.com/bigquery/v2:443;ProjectId=<project-id>;OAuthType=0;OAuthPvtKeyPath=<path-to-key-file>;OAuthServiceAcctEmail=<email>" driverClassName="com.simba.googlebigquery.jdbc42.Driver"/>
</db:config>
The above results in an error when testing connection
ToolingException{message='Got status code: 500 when trying to
resolve a Mule Runtime operation. Reason: 'Server Error.
{"errorType":null,"errorMessage":null,"errorDetail":null,"additionalProperties":{servlet=org.glassfish.jersey.servlet.ServletContainer-7bf5036,
message=Request failed.,
url=/mule/tooling/applications/4cfe61fc-a304-4ee5-8b52-36ed6e4cfd66/components/Database_Config/connection,
status=500}}'' , rootCauseMessage='null' , rootCauseType='null'
, rootCauseStackTrace='[]'} at
org.mule.tooling.client.api.exception.ToolingException$Builder.build(ToolingException.java:141)
at
org.mule.tooling.agent.rest.client.RestAgentToolingService.handleToolingAgentHandlerException(RestAgentToolingService.java:911)
at
org.mule.tooling.agent.rest.client.RestAgentToolingService.lambda$serviceExceptionOrToolingException$43(RestAgentToolingService.java:859)
at
org.mule.tooling.agent.rest.client.RestAgentToolingService.serviceExceptionOr(RestAgentToolingService.java:873)
BigQuery Connector provided by MuleSoft
<bigquery:config name="BigQuery__Configuration" doc:name="BigQuery Configuration" doc:id="3b8287ed-f3e5-4721-bd9e-96950347cf3a" >
<bigquery:jwt-connection privateKeyId="<private-key-id>" privateKey="<private-key>" issuer="<email>" projectId="<project-id>" />
</bigquery:config>
The above results in an error
org.mule.runtime.api.connection.ConnectionException: Could not
create connection Caused by:
org.mule.runtime.api.exception.MuleRuntimeException: Invalid PKCS8
data.
I made sure the credentials passed are appropriate and we also checked with the customer regarding appropriate privileges to use this but still did not find a way to solve this.
References
mulesoft-bigquery-jdbc
I am using ignite native and using atomicity as TRANSACTIONAL_SNAPSHOT when I am trying the load the old storage which was configured with amoticity TRNASACTIONAL it is giving the Unknown page type issue after deleting the .dat file but if I am using new storage it is working fine. Can anybody help me?
org.h2.jdbc.JdbcSQLException: General error: "java.lang.IllegalStateException: Unknown page type: 10009 pageId: 0002ffff00000006"; SQL statement:
CREATE TABLE "DFM"."ANSWER_TYPE_ENUM" (_KEY VARCHAR INVISIBLE NOT NULL,_VAL OTHER INVISIBLE,"ID" VARCHAR,"ENUM_VALUE" VARCHAR) engine "org.apache.ignite.internal.processors.query.h2.H2TableEngine" [50000-197]
I've never seen errors like these, but I would say that TRANSACTIONAL_SNAPSHOT is experimental and should be avoided for now.
I'm getting this error when trying to run a select after connecting to Hive.
Is this a bad jar file?
org.jkiss.dbeaver.model.impl.jdbc.JDBCException: SQL Error: Method not supported
at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCConnectionImpl.prepareStatement(JDBCConnectionImpl.java:170)
at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCConnectionImpl.prepareStatement(JDBCConnectionImpl.java:1)
at org.jkiss.dbeaver.model.DBUtils.createStatement(DBUtils.java:985)
at org.jkiss.dbeaver.model.DBUtils.prepareStatement(DBUtils.java:963)
at org.jkiss.dbeaver.runtime.sql.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:313)
at org.jkiss.dbeaver.runtime.sql.SQLQueryJob.extractData(SQLQueryJob.java:633)
at org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsProvider.readData(SQLEditor.java:1169)
at org.jkiss.dbeaver.ui.controls.resultset.ResultSetDataPumpJob.run(ResultSetDataPumpJob.java:132)
at org.jkiss.dbeaver.runtime.AbstractJob.run(AbstractJob.java:91)
at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54)
Caused by: java.sql.SQLException: Method not supported
at org.apache.hadoop.hive.jdbc.HiveConnection.createStatement(HiveConnection.java:229)
at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCConnectionImpl.createStatement(JDBCConnectionImpl.java:350)
at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCConnectionImpl.prepareStatement(JDBCConnectionImpl.java:138)
... 9 more
There is a calls in hive jdbc jar called org.apache.hive.jdbc.HiveResultSetMetaData . This class contains a method isWritable which is not supported by hive yet. This is the reason why you get the error "Method not supported".
Take the source code of this class and update the above method. Then generate the class and replaced it in the jar. This worked for me.
I'm using Play framework 2.x with SQL driver: com.microsoft.sqlserver.jdbc.SQLServerDriver
I'm trying to run a simple query:
SELECT [org].[name] FROM [ref].[organisations_bak] AS org
but I get the following error:
play.api.Application$$anon$1: Execution exception[[SQLServerException: Invalid object name 'ref.organisations_bak'.]]
at play.api.Application$class.handleError(Application.scala:293) ~[play_2.10-2.2.2.jar:2.2.2]
at play.api.DefaultApplication.handleError(Application.scala:399) [play_2.10-2.2.2.jar:2.2.2]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$12$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:165) [play_2.10-2.2.2.jar:2.2.2]
at play.core.server.netty.PlayDefaultUpstreamHandler$$anonfun$12$$anonfun$apply$1.applyOrElse(PlayDefaultUpstreamHandler.scala:162) [play_2.10-2.2.2.jar:2.2.2]
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33) [scala-library-2.10.3.jar:na]
at scala.util.Failure$$anonfun$recover$1.apply(Try.scala:185) [scala-library-2.10.3.jar:na]
Caused by: com.microsoft.sqlserver.jdbc.SQLServerException: Invalid object name 'ref.organisations_bak'.
at com.microsoft.sqlserver.jdbc.SQLServerException.makeFromDatabaseError(SQLServerException.java:216) ~[sqljdbc4-4.0.2206.100.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerStatement.getNextResult(SQLServerStatement.java:1515) ~[sqljdbc4-4.0.2206.100.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerStatement.doExecuteStatement(SQLServerStatement.java:792) ~[sqljdbc4-4.0.2206.100.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerStatement$StmtExecCmd.doExecute(SQLServerStatement.java:689) ~[sqljdbc4-4.0.2206.100.jar:na]
at com.microsoft.sqlserver.jdbc.TDSCommand.execute(IOBuffer.java:5696) ~[sqljdbc4-4.0.2206.100.jar:na]
at com.microsoft.sqlserver.jdbc.SQLServerConnection.executeCommand(SQLServerConnection.java:1715) ~[sqljdbc4-4.0.2206.100.jar:na]
I need to use the schema reference in my queries but I cant even get a simple query like this to work on my play app, simple queries without schema references work fine
SELECT name FROM organisations_bak
My Scala code looks like this:
import java.sql.ResultSet
import play.api.db.DB
DB.withConnection {
conn =>
val res = conn.createStatement.execute("SELECT [org].[name] FROM [ref].[organisations_bak] AS org")
}
Any help would be appreciated.
Thanks
In my case the issue was different user permissions. The server I used for development has been setup in some weird way that my user didn't have the permission to access the [ref] schema.
Just as a test I switched over to an AWS RDS SQL Server instance with the default DBA (owner) user settings and everything worked.
This means that the library and the code works it's my server that's at fault, but that's another issue.
try using
SELECT [org].[name] FROM [ref].[dbo].[organisations_bak] AS org
I got the following exception when i deployed the war on WAS 8.5
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'identityAssociationCleanupTask' defined in URL [wsjar:file:/C:/Program%20Files/IBM/Worklight/WorklightServer/worklight-jee-library.jar!/conf/core.xml]: Invocation of init method failed; nested exception is org.springframework.orm.jpa.JpaSystemException: "WRKSCHM.CLUSTER_SYNC" is an undefined name.. SQLCODE=-204, SQLSTATE=42704, DRIVER=3.57.82 {prepstmnt -234009374 SELECT t0.HOSTVMID, t0.ID, t0.UPDATETIMESTAMP, t0.VERSION FROM WRKSCHM.CLUSTER_SYNC t0 WHERE t0.ID = ? optimize for 1 row [params=(String) identityAssociationCleanupTask]} [code=-204, state=42704]SQLCA OUTPUT[Errp=SQLNQ1FC, Errd=-2145779603, 0, 0, 0, -10, 0]
"WRKSCHM.CLUSTER_SYNC" is an undefined name.. SQLCODE=-204, SQLSTATE=42704, DRIVER=3.57.82
An error occurred during implicit system action type "2". Information returned for the error includes SQLCODE "-204", SQLSTATE "42704" and message tokens "WRKSCHM.CLUSTER_SYNC".. SQLCODE=-727, SQLSTATE=56098, DRIVER=3.57.82
An error occurred during implicit system action type "2". Information returned for the error includes SQLCODE "-204", SQLSTATE "42704" and message tokens "WRKSCHM.CLUSTER_SYNC".. SQLCODE=-727, SQLSTATE=56098, DRIVER=3.57.82; nested exception is <openjpa-1.2.2-r422266:898935 nonfatal general error> org.apache.openjpa.persistence.PersistenceException: "WRKSCHM.CLUSTER_SYNC" is an undefined name.. SQLCODE=-204, SQLSTATE=42704, DRIVER=3.57.82 {prepstmnt -234009374 SELECT t0.HOSTVMID, t0.ID, t0.UPDATETIMESTAMP, t0.VERSION FROM WRKSCHM.CLUSTER_SYNC t0 WHERE t0.ID = ? optimize for 1 row [params=(String) identityAssociationCleanupTask]} [code=-204, state=42704]SQLCA OUTPUT[Errp=SQLNQ1FC, Errd=-2145779603, 0, 0, 0, -10, 0]
"WRKSCHM.CLUSTER_SYNC" is an undefined name.. SQLCODE=-204, SQLSTATE=42704, DRIVER=3.57.82
An error occurred during implicit system action type "2". Information returned for the error includes SQLCODE "-204", SQLSTATE "42704" and message tokens "WRKSCHM.CLUSTER_SYNC".. SQLCODE=-727, SQLSTATE=56098, DRIVER=3.57.82
An error occurred during implicit system action type "2". Information returned for the error includes SQLCODE "-204", SQLSTATE "42704" and message tokens "WRKSCHM.CLUSTER_SYNC".. SQLCODE=-727, SQLSTATE=56098, DRIVER=3.57.82
But I am able to deploy the same on the embedded server of eclipse.
And iam not able to see any internal tables under APPCNTR database.
Iam using worklight 6.0 and i installed using installation manager 1.6.3.
Iam using DB2 10.1,I have manually created APPCNTR,WRKLGHT,WLREPORT databases and set the schemas for WRKLGHT as WRKSCHM and WLREPORT as WLRESCHM.
I followed this http://pic.dhe.ibm.com/infocenter/wrklight/v6r0m0/index.jsp?topic=%2Fcom.ibm.worklight.help.doc%2Fdeploy%2Fc_deploy_custom_war_file_to_app_server.html, for Configuring WebSphere Application Server for DB2 manually.
followed http://pic.dhe.ibm.com/infocenter/wrklight/v6r0m0/index.jsp?topic=%2Fcom.ibm.worklight.help.doc%2Fdeploy%2Fc_deploy_custom_war_file_to_app_server.html this to Configuring WebSphere Application Server manually and deploying .
Please help me if I have missed any configuration.
You have not provided details about your environment (Worklight version, WAS profile, database) so it is difficult to pinpoint exactly what the problem could be; my answer is based on certain assumptions, such as that you are using DB2.
Looking closely to the error:
WRKSCHM is the schema name that Worklight Server is trying to access through JPA
CLUSTER_SYNC is a table in the 'WRKLGHT' database
This error can be caused by either the table not being properly created (DB2 setup process is incomplete), or by a schema name mismatch (WAS datasource setup incorrect).
Please verify the following on your environment:
DB2 Setup
Make sure that your DB2 is properly configured for Worklight; verify that the necessary databases are created ('WRKLGHT' for Worklight Server, 'WLREPORT' for the Worklight Reports feature and 'APPCNTR' for Worklight Application Center). The following documentation might be useful to you:
Creating the DB2 databases
Setting up the 'WRKLGHT' and 'WLREPORT' databases
Setting up the 'APPCNTR' database
DB2 Schema Configuration
Make sure that your WAS configuration is properly defining the appropriate schema name that you used in the steps above; the schema name is passed to the JDBC driver using the currentSchema property. The following documentation might be helpful as well:
Configuring DB2 on WAS Liberty profile
Configuring DB2 on WAS Standard profile
DB2 Cheat Sheet (how to list the current DB2 schema names)
DB2 schema qualifiers
I hope this will help you get past this problem.