Liquibase table name prefix - liquibase

I'm currently developing a prototype for a Spring based plugin system. The idea is that plugins can use JPA entities and a liquibase changelog to maintain the database structure. In order to separate the tables created by plugins from the tables of the core system the plugins should be forced to use a prefix for table name.
For JPA/Hibernate that can be easily archived by using a naming strategy. But I've found no way to archive that for the liquibase changeset.
For example the plugin defines a changelog like follows
<changeSet id="2015-03-17-00-01" author="foo">
<createTable tableName="fooentity">
<column name="id" type="INT">
<constraints primaryKey="true" nullable="false" />
</column>
<column name="name" type="VARCHAR(100)">
<constraints nullable="false" />
</column>
</createTable>
</changeSet>
The table should be created with name "plugin_fooentity". The plugin itself should not know anything about the prefix since the prefix is given by the plugin/core system.
Would be great if someone can give me a hint for a possible solution.

Maybe you can use modifySql for this?
You would have to copy this to all changesets that you define but it should be possible.
It has a subtag called regExpReplace which you could use to define a general term like create table (\w*?) .* and replace this with create table plugin_$1.

For me it worked like this:
Using the modifySql statement
I created 2 properties:
<property name="table.prefix" value="TBL_"/>
<property name="schema.name" value="PUBLIC"/>
Then added the following statements:
<modifySql>
<regExpReplace replace="CREATE\ TABLE\ ${schema.name}.([\w]*)\ (.*)" with="CREATE TABLE ${schema.name}.${table.prefix}$1 $2"/>
</modifySql>
<modifySql>
<regExpReplace replace="CREATE\ UNIQUE\ INDEX\ ${schema.name}.([\w]*)\ ON\ PUBLIC.([\w]*)\((.*)\)" with="CREATE UNIQUE INDEX ${schema.name}.$1 ON ${schema.name}.${table.prefix}$2($3)"/>
</modifySql>
<modifySql>
<regExpReplace replace="CREATE\ INDEX\ ${schema.name}.([\w]*)\ ON\ ${schema.name}.([\w]*)\((.*)\)" with="CREATE INDEX ${schema.name}.$1 ON ${schema.name}.${table.prefix}$2($3)"/>
</modifySql>

Related

Liquibase changeset changeLogPropertyDefined is not working

In liquibase I want to execute a particular change set based on the 'context' property value.
In this case I have passed -Dcontext=local (I've checked this values is getting picked properly) through command line and tried to check that property within my changeset by using changeLogPropertyDefined. But It's not working..
Please find below my changeset
<changeSet id="1" author="dm">
<preConditions onFail="MARK_RAN" onSqlOutput="TEST">
<changeLogPropertyDefined property="context" value="local"/>
</preConditions>
<createTable tableName="accountold">
<column autoIncrement="true" name="id" type="BIGINT">
<constraints nullable="false" primaryKey="true"/>
</column>
<column name="version" type="BIGINT">
<constraints nullable="false"/>
</column>
</createTable>
</changeSet>
You should add the "link" to the property in the top of your changeset file to reference value is passed from jvm args:
<property name="context" value="${context}"/>
Then you could use your property in precondition and anywhere you want in this changeset.
P.S. Maybe it's will be better to give different name for jvm arg and lb property to avoid ambiguity of interpretation.

Handle sqlplus substitution variables (&&vars) in Liquibase

my project is trying to migrate to liquibase but the lack of support for bind variables is making this difficult.
During our deployment we have sql scripts containing sqlplus substitution variables, like for example.
-- load_seed.sql ---
insert into <table>
values('&&host', '&&port', '&&user');
The value of these variables is different per environment, therefore we define profiles like these.
<DEV_profile.sql>
DEFINE host='dev.company.org'
DEFINE port=4008
..
<UAT_profile.sql>
DEFINE host='uat.company.org'
...
and the we run the deployment like this:
./deploy.ksh DEV
---- deploy.ksh ---
sqlplus <<END
<connection>
#$1_profile
#load_seed
The correct profile is picked up at execution time and the variables replaced.
Could you please suggest how to handle a case like this with Liquibase?
The equivalent functionality in Liquibase is provided by changelog parameters.
In your changelog, you define parameters, which are basically key-value pairs, and liquibase decides which value to use based on the value of a context or a label or a dbms.
When you want to apply the changeset to a given environment, you specify the context or label on the command line or in the liquibase.properties. Liquibase can determine the dbms based on the connection URL.
Here's an example that is somewhat similar to what you describe:
<databaseChangeLog
xmlns="http://www.liquibase.org/xml/ns/dbchangelog"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:ext="http://www.liquibase.org/xml/ns/dbchangelog-ext"
xsi:schemaLocation="http://www.liquibase.org/xml/ns/dbchangelog http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-3.6.xsd
http://www.liquibase.org/xml/ns/dbchangelog-ext http://www.liquibase.org/xml/ns/dbchangelog/dbchangelog-ext.xsd">
<property name="host" value="dev.company.org" context="DEV"/>
<property name="port" value="4008" context="DEV"/>
<property name="user" value="DEV_USER" context="DEV"/>
<property name="host" value="uat.company.org" context="UAT"/>
<property name="port" value="4321" context="UAT"/>
<property name="user" value="UAT_USER" context="UAT"/>
<changeSet id="1" author="joe">
<insert tableName="someTableName">
<column name="host" type="varchar(255)" value="${host}"/>
<column name="port" type="varchar(8)" value="${port}"/>
<column name="user" type="varchar(255)" value="${user}"/>
</insert>
</changeSet>
</databaseChangeLog>
https://docs.liquibase.com/concepts/basic/changelog-property-substitution.html
does not support sql changelog property substitution. you would have to migrate to (xml, yaml, json)

Liquibase loadData as string, not CLOB resource

The Problem
I recently upgraded Liquibase to 3.6.2 from 3.4.2.
Loading seed data from a CSV into text fields now results in a CLOB resource error. Before it would simply insert the text as a value.
The Setup
I'm using Liquibase to manage migrations of my data.
I have a table with an code and description column. description is of type TEXT.
<changeSet author="" id="create-table-degrees">
<createTable tableName="degrees">
<column name="code"
type="varchar(2)">
<constraints primaryKey="true"/>
</column>
<column name="description"
type="text">
<constraints unique="true"/>
</column>
</createTable>
<rollback>
<dropTable tableName="degrees"/>
</rollback>
</changeSet>
I have seed data in a CSV:
code,description
"D1","MASTERS"
"D2","DOCTORATE"
I load it using loadData:
<changeSet author="" id="seed-degrees">
<loadData file="seeds/degrees.csv"
tableName="degrees" />
</changeSet>
The Error
Unexpected error running Liquibase: CLOB resource not found: MASTERS
The Question
Is there a way to keep Liquibase from interpreting seed values as file paths instead of strings, or do I need to manually define the column types as String in loadData.
e.g. I would like to avoid having to modify the old changeSet to:
<changeSet author="" id="seed-degrees">
<loadData file="seeds/degrees.csv"
tableName="roles">
<column name="description" type="string" />
</loadData>
</changeSet>
The workaround listed in CORE-3287: Anver S December 3, 2018, 3:07 PM
While adding an explicit column type definition as defined in original
stackoverflow post
<column name="description" type="string" />
does the trick - for me it effectively requires to update already
applied changesets which ideally I'd try to avoid.

Liquibase: How to set the default value of a date column to be "now" in UTC format?

How do you set the default value of a date column to be "now" in UTC format? I think the answer involves the defaultValueComputed attribute on the column element.
The documentation states:
defaultValueComputed A value that is returned from a function or
procedure call. This attribute will contain the function to call.
What langauge is the function referred to supposed to be written in? Java? Is the function supposed to be the database vendor -specific date function I want to use? Is there any more documentation I can read on this topic?
Maybe this topic in the liquibase forum will help?
I think defaultValueComputed will take a database specific function to express "now". In mySQL it would be CURRENT_TIMESTAMP so it could look like this:
<createTable tableName="D_UserSession">
<column name="ts" type="TIMESTAMP" defaultValueComputed="CURRENT_TIMESTAMP"/>
</createTable>
(Copied from the forum post.)
This should be:
<property name="now" value="now()" dbms="mysql,h2"/>
<property name="now" value="current_timestamp" dbms="postgresql"/>
<property name="now" value="sysdate" dbms="oracle"/>
<property name="now" value="getdate()" dbms="mssql"/>
<changeSet author="me" id="sample_usage_demo">
<addColumn schemaName= "dbo" tableName="demo_table" >
<column name="demo_column" type="datetime" defaultValueDate="${now}">
<constraints nullable="false" />
</column>
</addColumn>
</changeSet>
In MySQL, to use a DATETIME column with fractions of second like DATETIME(6) (microseconds precision), use default value of NOW(6) (caution: CURRENT_TIMESTAMP(6) for some reason produces an error with me using liquibase 3.5.3):
<column name="created_at" type="DATETIME(6)" defaultValueComputed="NOW(6)" >
<constraints nullable="false" />
</column>
Note that the value will be stored internally in UTC, but read using the server's timezone settings (##global.time_zone, ##session.time_zone).
This works with SQlite:
<column name="last_updated_at" type="TIMESTAMP" defaultValueComputed="CURRENT_TIMESTAMP">
<constraints nullable="false"/>
</column>
Adding '$now' didn't work for me. I am using SQlite as the DB.
This worked for me:
<property name="now" value="UNIX_TIMESTAMP()" dbms="mysql"/>
<column name="ts" type="timestamp" valueDate="${now}"/>
I found it thanks to this answer: https://stackoverflow.com/a/9100388/3107952
You should probably use timestamp with time zone as that will keep the timestamps in UTC as opposed to local server time, which might be problem if you have a multi-region setup.
You can readmore about the timezone part on this Stack Overflow post.
databaseChangeLog:
- changeSet:
id: 007
author: joe
changes:
- addColumn:
tableName: my_table
columns:
- column:
name: created_at
type: timestamp with time zone
- addDefaultValue:
columnName: created_at
defaultValueComputed: now()
tableName: my_table
Due to the fact, that the requested timezone UTC is not mentioned in all of the answers here, I'd like to state another set of solutions for different database vendors.
Especially, the current answers here does not state the correct solution for Oracle.
<changeSet logicalFilePath="my_changeset.xml"
id="1"
author="me"
dbms="mariadb,h2">
<addColumn tableName="MY_TABLE">
<column name="MY_ZONED_DATE_TIME_COLUMN"
type="timestamp(6)"
defaultValueComputed="now()">
<constraints nullable="false"/>
</column>
</addColumn>
</changeSet>
<changeSet logicalFilePath="my_changeset.xml"
id="1"
author="me"
dbms="postgresql">
<addColumn tableName="MY_TABLE">
<column name="MY_ZONED_DATE_TIME_COLUMN"
type="timestamp(6)"
defaultValueComputed="timezone('UTC', now())">
<constraints nullable="false"/>
</column>
</addColumn>
</changeSet>
<changeSet logicalFilePath="my_changeset.xml"
id="1"
author="me"
dbms="oracle">
<addColumn tableName="MY_TABLE">
<column name="MY_ZONED_DATE_TIME_COLUMN"
type="timestamp(6)"
defaultValueComputed="sys_extract_utc(systimestamp)">
<constraints nullable="false"/>
</column>
</addColumn>
</changeSet>
(One may use properties, like showed in Dominika's answer, but we had some really bad experiences with properties in liquibase.)
Summary:
now() is fine for MariaDB, MySql and H2v
now() is not completely fine for H2, I just got a correct result starting the h2 database with UTC like jdbc:h2:mem:./my_database;TIME ZONE=UTC (h2database in version 2.x needed). Then now() is working for sure.
I'm not sure about current_timestamp on PostgreSQL, but timezone('UTC', now()) works. :)
In Oracle, sysdate (mentioned in a few other answers here) is not enough, see as well "How to get UTC value for SYSDATE on Oracle", but sys_extract_utc(systimestamp) does the trick.
I used function the database vendor.
For Oracle it`s a sysdate:
<column name="create_date" type="DATETIME" valueDate="sysdate" defaultValueComputed="sysdate" />
As liquibase is common changelog for any database, to make it generic you should not depend on any specific database like oracle, postegres, mysql instead it should be generic enough to work for any/every database.
Below is how it should be implemented :
<column name="time" type="${type.datetime}" defaultValueComputed="${column.datetime.defaultValue}"/>
This should work for all databases, for oracle, it inserts SYSTIMESTAMP as DATA_DEFAULT.

NHibernate - Trying to get it to use a SQL Server Row version

As per the StackOverflow question 'NHibernate and sql timestamp columns as version', I use the following mapping:
<version name="RowNumber" generated="always" unsaved-value="null" type="BinaryBlob">
<column name="RowNumber" not-null="false" sql-type="timestamp" />
</version>
<property name="CreateDate" column="CreateDate" type="DateTime" update="false" insert="false" />
(Other properties after this last).
But when I run my ASP.MVC app I get:
[Path]\Clients.hbm.xml(7,90): XML validation error: The element 'urn:nhibernate-mapping-2.2:version' cannot contain child element 'urn:nhibernate-mapping-2.2:column' because the parent element's content model is empty.
But as far as I can see 2.2 is the latest version of the mapping, so how can anyone put a column element inside the version element?
Sorry if this is really basic,
In case anyone else has this problem:
It works as Ayende Rahien specifies in this blog on NHibernate - but only (AFAIK) on version 2.1.n; I was using 2.0.n. I also think you need the object's field/property to be byte[], not System.Linq.Binary as that type has no default constructor (but I am not sure about this - I seemed to have to do this)
Example (excuse the names):
<version name="RowKludge" type="BinaryBlob" generated="always" unsaved-value="null" >
<column name="RowNumber"
not-null="false"
sql-type="timestamp"/>
</version>
A SQL server 'timestamp' is not your regular timestamp, hence the requirement that the type should be a binary blob.
Note that if you do migrate you will need to change the NHibernate configuration in Web/App config - most tutorials currently available seem to be for v.2.0 (or earlier) - you need an uptodate reference or tutorial for 2.1
A quick look in the documentation reveals that your mapping is not correct. It should be something like this:
<version name="RowNumber" column="RowNumber"
generated="always" unsaved-value="null"
type="Timestamp" />
Best Regards,
Oliver Hanappi