Connecting to a database after creating it with Kotlin Exposed - kotlin

I'm using jetbrains' exposed library to create and populate a database.
The database does not exist, and I am creating it. However I could not find a simple way to connect to the SQL engine, create a database and connect to that database without multiple connections.
That sounds a little clunky. Is there a better way to do it maybe?
Here is a small example :
var db = Database.connect("jdbc:mysql://localhost:3308", driver = "com.mysql.jdbc.Driver", user = "root", password = "aRootPassword")
transaction(db) { SchemaUtils.createDatabase("imdb") }
// avoid reconnect?
db = Database.connect("jdbc:mysql://localhost:3308/imdb", driver = "com.mysql.jdbc.Driver", user = "root", password = "aRootPassword")
transaction(db) { SchemaUtils.create (TitleRatings) }

You need a connection pool, e.g. HikariCP. It pools database connections and reuses them. This gives you a huge performance boost compared to individually opened connections.
I usually wrap it in a simple class like this:
import com.zaxxer.hikari.HikariConfig
import com.zaxxer.hikari.HikariDataSource
import javax.sql.DataSource
public object DB {
var db: DataSource = connect();
public fun connect(): DataSource {
val config = HikariConfig()
config.jdbcUrl = "jdbc:mysql://localhost:3308"
config.username = "com.mysql.jdbc.Driver"
config.password = "aRootPassword"
config.driverClassName = "com.mysql.jdbc.Driver"
return HikariDataSource(config)
}
}
My transactions then look like this one:
transaction(Database.connect(DB.db)) {
SchemaUtils.createDatabase("imdb")
}

Related

How to properly create and connect to database with Kotlin Exposed?

I did check the similar question here, but it didn't work as expected for me, I still need to reconnect to the created database and I'm not sure how or even if I can avoid that.
Here is my code:
hikari.properties:
jdbcUrl=jdbc:mariadb://localhost:3306/
driverClassName=org.mariadb.jdbc.Driver
username=root
dataSource.databaseName=DBNAME //this doesn't seem to do much, I'm getting the same behavior with or without it
fun initDB() {
val config = HikariConfig("/hikari.properties")
val ds = HikariDataSource(config)
transaction(connect(ds)) {
SchemaUtils.createDatabase("DBNAME")
}
config.jdbcUrl = "jdbc:mariadb://localhost:3306/DBNAME"
//ds.jdbcUrl = "jdbc:mariadb://localhost:3306/DBNAME" //THIS WILL NOT WORK
val ds2 = HikariDataSource(config)
transaction(connect(ds2)) {
SchemaUtils.create( Tables... )
}
}
The reason I make a new datasource, is because otherwise I'll get this error:
java.lang.IllegalStateException: The configuration of the pool is sealed once started. Use HikariConfigMXBean for runtime changes. HikariConfigMXBean doesn't seem allow jdbcUrl changes.
There must be a more elegant way to do this, right?
It is not a good practice. Creating and filling db it is two separate processes. DB creates once by the DBA, then you just connect and use it. Your approach has huge security violation. User from the datasource must have create db privilege.
But if you want proceed with your current approach, first you should create a db without using datasource, then create datasource and connect to the db. Something like this
import com.zaxxer.hikari.HikariConfig
import com.zaxxer.hikari.HikariDataSource
import org.jetbrains.exposed.sql.Database
import org.jetbrains.exposed.sql.SchemaUtils
import org.jetbrains.exposed.sql.Table
import org.jetbrains.exposed.sql.transactions.transaction
fun main(args: Array<String>) {
//LOAD proerties
val config = HikariConfig("/hikari.properties")
//Hikari properties content
// jdbcUrl=jdbc:mysql://localhost:3306/hikari
// driverClassName=com.mysql.cj.jdbc.Driver
// username=root
// password=<pwd here>
//Here replace dbname from jdbc url to empty
transaction(Database.connect(url = config.jdbcUrl.replace("hikari", ""),
driver = config.driverClassName,
user = config.username,
password = config.password)) {
SchemaUtils.createDatabase("hikari")
}
val ds = HikariDataSource(config)
transaction(Database.connect(ds)) {
SchemaUtils.create(Users)
}
}
object Users : Table() {
val id = varchar("id", 10) // Column<String>
val name = varchar("name", length = 50) // Column<String>
override val primaryKey = PrimaryKey(id, name = "PK_User_ID")
}

How to import raven db with versioning bundle

I have a database with versioning bundle turned on. I make an export and then try to import the exported dump to newly created db. I get the exception "Modifying a historical revisionis not allowed". I found this question and answer from Ayende, that its by design. But how do I import data to an empty database if the versioning bundle was turned on and there are revisions in it?
For now I did the following thing: I make a new database without a versioning bundle, but with replication bundle in it. Import to that db(and it works), but I have a lot of duplicates if I perform search.
After that I create another new database, with replication and versioning bundle turned on. And I replicate from the db with duplicates this db. And it works, but it seems its a lot of things todo.
Am I doing the right thing? Is there an easier way to get your data from the dump?
I've had the same issue. The ONLY way I could get the import to work was to do the following:
Create DB with Versioning enabled (They recommend that DBs be created with any bundles that may be used).
Disable the versioning bundle. I've done this by editing the database settings and removing the bundle from the "Raven/ActiveBundles" setting.
Import your database.
Enable the versioning bundle. Just add the "Versioning" bundle back to the "Raven/ActiveBundles" setting.
If anyone has a better idea, I'd love to hear it. :)
The new version of RavenDb 3.0 give the permission to import with the Bundle already activated. So all the trick of disabling and enabling is not of any use anymore. But you might have new issues like Index not in synch with your revision or having the "versioning" not displayed in the Settings=>Versioning window.
To fix your versioning if they don't show in the versioning window but that they are visible in the System Documents, you need to add the Id individually or run some kind of code.
Option parameter for import since RavenDb 3.0.3745:
--disable-versioning-during-import
Fix of Versioning not containing an Id :
private void UpdateVersioning(string destinationRavenDbServer, string databaseName)
{
using (var documentStore = new DocumentStore { Url = destinationRavenDbServer, DefaultDatabase = databaseName })
{
documentStore.Initialize();
using (var session = documentStore.OpenSession())
{
var versioningInfoList = session.Advanced.LoadStartingWith<RavenJObject>("Raven/Versioning/", pageSize: 1024 );
foreach (var versioningInfo in versioningInfoList)
{
if (!versioningInfo.ContainsKey("Id"))
{
var fullInternalId = session.Advanced.GetDocumentId(versioningInfo);
var idSplitted = fullInternalId.Split('/');
var newId = idSplitted[idSplitted.Length - 1];
versioningInfo.Add("Id", newId);
}
}
session.SaveChanges();
}
}
}
Fix for re-indexing all the database:
private void ResetAllIndexes(string destinationRavenDbServer, string databaseName)
{
using (var documentStore = new DocumentStore { Url = destinationRavenDbServer, DefaultDatabase = databaseName })
{
documentStore.Initialize();
var indexes = documentStore.DatabaseCommands.GetIndexNames(0, 1024); // Update the 1024 first index, but you get the idea
foreach (var indexName in indexes)
{
documentStore.DatabaseCommands.ResetIndex(indexName);
}
}
}

One-Way sync of data without changing schema of source database

I Have a database that we want to partially sync data out of into another database (on Azure).
I have been looking at Sync Framework 2.1 and believe it can solve the problem, however i cannot figure it out from the online documentation.
We have the restraint that we cannot change the schema of the database however we are on SQL 2008 R2 which means that we can use track changes.
I am looking for some advise on how this might be achieved.
currently i have a SyncOrchestrator
var orch = new SyncOrchestrator
{
LocalProvider = new SampleServerSyncProvider(),
RemoteProvider = new SampleClientSymcProvider(),
Direction = SyncDirectionOrder.Upload
};
and then a sync provider
public class SampleServerSyncProvider : DbServerSyncProvider
{
private String SQLLocalConnection = "valid connection string";
public SampleServerSyncProvider()
{
SqlConnection serverConn = new SqlConnection(SQLLocalConnection);
Connection = serverConn;
Connection.Open();
var cmTableSyncAdapter = new SqlSyncAdapterBuilder
{
Connection = serverConn,
ChangeTrackingType = ChangeTrackingType.SqlServerChangeTracking,
SyncDirection = SyncDirection.Bidirectional,
TableName = "my table"
};
SyncAdapters.Add(cmTableSyncAdapter.ToSyncAdapter());
}
}
Currently i am getting an error that talks about initializing the connection. But I cannot find an initialize method on any of the objects
System.InvalidOperationException : Cannot create a SyncAdapter for table 'My table' by using
SqlSyncAdapterBuilder because the connection to the server has not yet
been initialized. Initialize the Connection property of the
SqlSyncAdapterBuilder before you call any of the SqlSyncAdapterBuilder
methods
SQL Change Tracking is only supported on the older offline providers (SqlClientSyncProvider/DbServerSyncProvider/SyncAgent). The newer providers you're trying to use (SqlSyncProvider/SyncOrchestrator) requires a custom change tracking. You cannot mix and match the database sync providers.
have you looked at using SSIS instead?

RavenDB IsOperationAllowedOnDocument not supported in Embedded Mode

RavenDB throws InvalidOperationException when IsOperationAllowedOnDocument is called using embedded mode.
I can see in the IsOperationAllowedOnDocument implementation a clause checking for calls in embedded mode.
namespace Raven.Client.Authorization
{
public static class AuthorizationClientExtensions
{
public static OperationAllowedResult[] IsOperationAllowedOnDocument(this ISyncAdvancedSessionOperation session, string userId, string operation, params string[] documentIds)
{
var serverClient = session.DatabaseCommands as ServerClient;
if (serverClient == null)
throw new InvalidOperationException("Cannot get whatever operation is allowed on document in embedded mode.");
Is there a workaround for this other than not using embedded mode?
Thanks for your time.
I encountered the same situation while writing some unit tests. The solution James provided worked; however, it resulted in having one code path for the unit test and another path for the production code, which defeated the purpose of the unit test. We were able to create a second document store and connect it to the first document store which allowed us to then access the authorization extension methods successfully. While this solution would probably not be good for production code (because creating Document Stores is expensive) it works nicely for unit tests. Here is a code sample:
using (var documentStore = new EmbeddableDocumentStore
{ RunInMemory = true,
UseEmbeddedHttpServer = true,
Configuration = {Port = EmbeddedModePort} })
{
documentStore.Initialize();
var url = documentStore.Configuration.ServerUrl;
using (var docStoreHttp = new DocumentStore {Url = url})
{
docStoreHttp.Initialize();
using (var session = docStoreHttp.OpenSession())
{
// now you can run code like:
// session.GetAuthorizationFor(),
// session.SetAuthorizationFor(),
// session.Advanced.IsOperationAllowedOnDocument(),
// etc...
}
}
}
There are couple of other items that should be mentioned:
The first document store needs to be run with the UseEmbeddedHttpServer set to true so that the second one can access it.
I created a constant for the Port so it would be used consistently and ensure use of a non reserved port.
I encountered this as well. Looking at the source, there's no way to do that operation as written. Not sure if there's some intrinsic reason why since I could easily replicate the functionality in my app by making a http request directly for the same info:
HttpClient http = new HttpClient();
http.BaseAddress = new Uri("http://localhost:8080");
var url = new StringBuilder("/authorization/IsAllowed/")
.Append(Uri.EscapeUriString(userid))
.Append("?operation=")
.Append(Uri.EscapeUriString(operation)
.Append("&id=").Append(Uri.EscapeUriString(entityid));
http.GetStringAsync(url.ToString()).ContinueWith((response) =>
{
var results = _session.Advanced.DocumentStore.Conventions.CreateSerializer()
.Deserialize<OperationAllowedResult[]>(
new RavenJTokenReader(RavenJToken.Parse(response.Result)));
}).Wait();

SQL connection in Grails

In my application there are a few places where I want to use pure SQL. When I hard code the connection details in Sql.newInstance it works fine. For obvious reasons I would prefer to not hard code the connection details.
When I use the dataSource variable it comes up as null. My code in the controller is:
import groovy.sql.Sql
def dataSource
def sql = Sql.newInstance(dataSource)
sql.executeInsert("insert into....")
My code in the datasource config file is:
environments {
development {
dataSource {
dbCreate = "create-drop"
url = "jdbc:postgresql:mydev"
username = "xxx"
password = "xxx"
As you can see I'm using PostgreSQL. I've also tried it using the default grails database with the same results.
Any ideas would be appreciated.
You need to define dataSource outside of your controller action. Otherwise spring cannot do the required dependency injection for you:
class YouController {
def dataSource
def yourAction() {
def sql = new Sql(dataSource)
[..]
}
}