I would like to programmatically generate a value (id) for a column during an insert using a Liquibase changelog (XML). For example:
<changeSet ...>
<preConditions ...>
<insert tableName="my_table">
<column name="my_id" value="<call-code-to-generate-id>" />
...
</insert>
I need to do this programmatically because the id needs a character prefix (determined by code) followed by a sequence number (legacy code and lots of other components in the system use it). The changelog needs to support both Oracle and SQL Server, and sequence numbers are generated differently (the code currently handles this).
I have looked at ChangeLogParser and SqlGenerator but I'm not seeing an easy way to do this. I was thinking it might be possible to parse the XML looking for 'value='' and replace the value with a generated id, but I'm not sure the effect this would have on Liquibase to determine if the changelog has been updated or not.
Does anyone know if this is possible and if so, how to do this?
I'm using Liquibase 3.5.3 and Java 8.
I got this working using a subclass of the Liquibase class InsertDataChange. I put the class in the package liquibase.sqlgenerator.ext so that Liquibase automatically registers it.
package liquibase.sqlgenerator.ext;
import liquibase.change.ColumnConfig;
import liquibase.change.core.InsertDataChange;
import liquibase.database.Database;
import liquibase.statement.SqlStatement;
public class MyInsertDataChange extends InsertDataChange {
#Override
public SqlStatement[] generateStatements(final Database database) {
for (final ColumnConfig column : getColumns()) {
final String tableName = getTableName();
final String name = column.getName();
final String value = column.getValue();
if (columnNeedsId(tableName, name, value)) {
column.setValue(generateId(tableName, name, value));
}
}
return super.generateStatements(database);
}
private boolean columnNeedsId(final String tableName, final String name, final String value) {
// Details omitted
return true;
}
private String generateId(final String tableName, final String name, final String value) {
// Details omitted
final String sequenceName = "whatever";
final long sequenceNumber = 123L;
return sequenceName + sequenceNumber;
}
}
Related
I am using Apache Ignite as the back-end data store in a SpringBoot Application.
I have a requirement where I need to get all the entities whose name matches one of the names from a set of names.
Hence i am trying to get it implemented using a #Query configuration and a method named findAllByName(Iterable<String> names)as below:
Here on the Query, I am trying to use the 'IN' clause and want to pass an array of names as an input to the 'IN' clause.
#RepositoryConfig(cacheName = "Category")
public interface CategoryRepository extends IgniteRepository<Category, Long>
{
List<Category> findByName(String name);
#Query("SELECT * FROM Category WHERE name IN ( ? )")
Iterable<Category> findAllByName(Iterable<String> names); // this method always returns empty list .
}
In this the method findAllByName always returns empty list, even when ignite has Categories for which the name field matches the data passed in the query.
I am unable to figure out if there is a problem with the Syntax or the query of the method signature or the parameters.
Please try using String[] names instead for supplying parameters.
UPDATE: I have just checked the source, and we don't have tests for such scenario. It means that you're on uncharted territory even if it is somehow possible to get to work.
Otherwise looks unsupported currently.
I know your question is more specific to Spring Data Ignite feature. However, as an alternate, you can achieve it using the SqlQuery abstraction of Ignite.
You will form your query like this. I have pasted the sample below with custom sql function inSet that you will write. Also, the below tells how this is used in your sql.
IgniteCache<String, MyRecord> cache = this.ignite
.cache(this.environment.getProperty(Cache.CACHE_NAME));
String sql = "from “my-ignite-cache”.MyRecord WHERE
MyRecord.city=? AND inSet(?, MyRecord.flight)"
SqlQuery<String, MyRecord> sqlQuery = new SqlQuery<>(MyRecord.class,
sql);
sqlQuery.setArgs(MyCity, [Flight1, Flight2 ] );
QueryCursor<Entry<String, MyRecord>> resultCursor = cache.query(sqlQuery);
You can iterate the result cursor to do something meaningful from the extracted data.
resultCursor.forEach(e -> {
MyRecord record = e.getValue();
// do something with result
});
Below is the Ignite Custom Sql function which is used in the above Query - this will help in replicating the IN clause feature.
#QuerySqlFunction
public static boolean inSet(List<String> filterParamArgIds, String id) {
return filterParamArgIds.contains(id);
}
And finally, as a reference MyRecord referred above can be defined something like this.
public class MyRecord implements Serializable {
#QuerySqlField(name = "city", index = true)
private String city;
#QuerySqlField(name = "flight", index = true)
private String flight;
}
I'm using UUID's as PK in my tables. They're stored in a BINARY(16) MySQL column. I find that they're being mapped to string type in YII. The CRUD code I generate breaks down though, because these binary column types are being HTML encoded in the views. Example:
<?php echo
CHtml::link(CHtml::encode($data->usr_uuid), /* This is my binary uuid field */
array('view', 'id'=>$data->usr_uuid)); ?>
To work around this problem, I overrode afterFind() and beforeSave() in my model where I convert the values to/from hex using bin2hex and hex2bin respectively. See this for more details.
This takes care of the view problems.
However, now the search on PK when accessing a url of the form:
http://myhost.com/mysite/user/ec12ef8ebf90460487abd77b3f534404
results in User::loadModel($id) being called which in turn calls:
User::model()->findByPk($id);
This doesn't work since the SQL is being generated (on account of it being mapped to php string type) is
select ... where usr_uuid='EC12EF8EBF90460487ABD77B3F534404'
What would have worked is if I could, for these uuid fields change the condition to:
select ... where usr_uuid=unhex('EC12EF8EBF90460487ABD77B3F534404')
I was wondering how I take care of this problem cleanly. I see one possiblity - extend CMysqlColumnSchema and override the necessary methods to special case and handle binary(16) columns as uuid type.
This doesn't seem neat as there's no support for uuid natively either in php (where it is treated as string) or in mysql (where I have it as binary(16) column).
Does anyone have any recommendation?
If you plan using it within your own code then I'd create my own base AR class:
class ActiveRecord extends CActiveRecord
{
// ...
public function findByUUID($uuid)
{
return $this->find('usr_uuid=unhex(:uuid)', array('uuid' => $uuid));
}
}
If it's about using generated code etc. then customizing schema a bit may be a good idea.
I used the following method to make working with uuid (binary(16)) columns using Yii/MySQL possible and efficient. I mention efficient, because I could have just made the column a CHAR(32) or (36) with dashes, but that would really chuck efficient out of the window.
I extended CActiveRecord and added a virtual attribute uuid to it. Also overloaded two of the base class methods getPrimaryKey and setPrimaryKey. With these changes most of Yii is happy.
class UUIDActiveRecord extends CActiveRecord
{
public function getUuid()
{
$pkColumn = $this->primaryKeyColumn;
return UUIDUtils::bin2hex($this->$pkColumn);
}
public function setUuid($value)
{
$pkColumn = $this->primaryKeyColumn;
$this->$pkColumn = UUIDUtils::hex2bin($value);
}
public function getPrimaryKey()
{
return $this->uuid;
}
public function setPrimaryKey($value)
{
$this->uuid = $value;
}
abstract public function getPrimaryKeyColumn();
}
Now I get/set UUID using this virtual attribute:
$model->uuid = generateUUID(); // return UUID as 32 char string without the dashes (-)
The last bit, is about how I search. That is accomplished using:
$criteria = new CDbCriteria();
$criteria->addCondition('bkt_user = unhex(:value)');
$criteria->params = array(':value'=>Yii::app()->user->getId()); //Yii::app()->user->getId() returns id as hex string
$buckets = Bucket::model()->findAll($criteria);
A final note though, parameter logging i.e. the following line in main.php:
'db'=>array(
...
'enableParamLogging' => true,
);
Still doesn't work, as once again, Yii will try to html encode binary data (not a good idea). I haven't found a workaround for it so I have disabled it in my config file.
I want to populate selectOneMenu component with values extracted from database by sql query.
Query returns only store names which I want to enter as values to selectOneMenu
I get java.lang.IllegalArgumentException with stack starting with :
java.lang.IllegalArgumentException at com.sun.faces.renderkit.SelectItemsIterator.initializeItems(SelectItemsIterator.java:216)
at com.sun.faces.renderkit.SelectItemsIterator.hasNext(SelectItemsIterator.java:135)
at com.sun.faces.renderkit.html_basic.MenuRenderer.renderOptions(MenuRenderer.java:762)
This is my xhtml code (This is the only use of selectItems):
<h:selectOneMenu id="storeName" value="#{shoplist.store}">
<f:selectItems value="#{buyHistory.stores}" />
</h:selectOneMenu>
This is query from buyHistory bean:
public ResultSet getStores() throws SQLException {
...
PreparedStatement getStores = connection.prepareStatement(
"SELECT distinct STORE_NAME "
+ "FROM BuyingHistory ORDER BY STORE_NAME");
CachedRowSet rowSet = new com.sun.rowset.CachedRowSetImpl();
rowSet.populate(getStores.executeQuery());
return rowSet;
}
What am I doing wrong? Should I convert somehow from resultSet to SelectItem array/list?
Should I convert somehow from resultSet to SelectItem array/list?
Yes, that's one of the solutions. See also our h:selectOneMenu wiki page. The IllegalArgumentException will be thrown when the value is not an instance of SelectItem, or an array, or Iterable or Map.
Ultimately, your JSF backing beans should be completely free of java(x).sql dependencies. I.e. you should have no single line of import java(x).sql.Something; in your JSF code. Otherwise, it's bad design anyway (tight-coupling). Learn how to create proper DAO classes.
Why do you think that JSF would know how to transform a ResultSet from the persistence layer? JSF is a presentation layer framework :)
Yes you need to convert it to a SelectItem-List like this:
private List<SelectItem> transformToSelectItems(ResultSet resultSet) {
List<SelectItem> selectItems = new ArrayList<SelectItem>();
while(resultSet.next()) {
String storeName = resultSet.getString("STORE_NAME");
SelectItem item = new SelectItem(storeName, storeName);
selectItems.add(item);
}
return selectItems;
}
Be sure to notice BalusC's answer. This is just an example of how to construct a dynamic SelectItem-List. But you should definetely not have a ResultSet in your JSF-ManagedBeans.
I'm using p6spy to log the sql statements generated by my program. The format for the outputted spy.log file looks like this:
current time|execution time|category|statement SQL String|effective SQL string
I'm just wondering if anyone knows if there's a way to alter the spy.properties file and have only the last column, the effective SQL string, output to the spy.log file? I've looked through the properties file but haven't found anything that seems to support this.
Thanks!
In spy.properties there is a property called logMessageFormat that you can set to a custom implementation of MessageFormattingStrategy. This works for any type of logger (i.e. file, slf4j etc.).
E.g.
logMessageFormat=my.custom.PrettySqlFormat
An example using Hibernate's pretty-printing SQL formatter:
package my.custom;
import org.hibernate.jdbc.util.BasicFormatterImpl;
import org.hibernate.jdbc.util.Formatter;
import com.p6spy.engine.spy.appender.MessageFormattingStrategy;
public class PrettySqlFormat implements MessageFormattingStrategy {
private final Formatter formatter = new BasicFormatterImpl();
#Override
public String formatMessage(int connectionId, String now, long elapsed, String category, String prepared, String sql) {
return formatter.format(sql);
}
}
There is no such option provided to achieve it via configuration only yet. I think you have 2 options here:
fill a new bug/feature request report (which could bring benefit to others using p6spy as well) on: https://github.com/p6spy/p6spy/issues?state=open or
provide custom implementation.
For the later option, I believe you could achieve it via your own class (depending on the logger you use, let's assume you use Log4jLogger).
Well, if you check relevant part of the Log4jLogger github as well as sourceforge version, your implementation should be rather straightforward:
spy.properties:
appender=com.EffectiveSQLLog4jLogger
Implementation itself could look like this:
package com;
import com.p6spy.engine.logging.appender.Log4jLogger;
public class EffectiveSQLLog4jLogger extends Log4jLogger {
public void logText(String text) {
super.logText(getEffectiveSQL(text));
}
private String getEffectiveSQL(String text) {
if (null == text) {
return null;
}
final int idx = text.lastIndexOf("|");
// non-perfect detection of the exception logged case
if (-1 == idx) {
return text;
}
return text.substring(idx + 1); // not sure about + 1, but check and see :)
}
}
Please note the implementation should cover github (new project home, no version released yet) as well as sourceforge (original project home, released 1.3 version).
Please note: I didn't test the proposal myself, but it could be a good starting point and from the code review itself I'd say it could work.
I agree with #boberj, we are used to having logs with Hibernate formatter, but don't forget about batching, that's why I suggest to use:
import com.p6spy.engine.spy.appender.MessageFormattingStrategy;
import org.hibernate.engine.jdbc.internal.BasicFormatterImpl;
import org.hibernate.engine.jdbc.internal.Formatter;
/**
* Created by Igor Dmitriev on 1/3/16
*/
public class HibernateSqlFormatter implements MessageFormattingStrategy {
private final Formatter formatter = new BasicFormatterImpl();
#Override
public String formatMessage(int connectionId, String now, long elapsed, String category, String prepared, String sql) {
if (sql.isEmpty()) {
return "";
}
String template = "Hibernate: %s %s {elapsed: %sms}";
String batch = "batch".equals(category) ? ((elapsed == 0) ? "add batch" : "execute batch") : "";
return String.format(template, batch, formatter.format(sql), elapsed);
}
}
In p6Spy 3.9 this can be achieved quite simply. In spy.properties set
customLogMessageFormat=%(effectiveSql)
You can patch com.p6spy.engine.spy.appender.SingleLineFormat.java
removing the prepared element and any reference to P6Util like so:
package com.p6spy.engine.spy.appender;
public class SingleLineFormat implements MessageFormattingStrategy {
#Override
public String formatMessage(final int connectionId, final String now, final long elapsed, final String category, final String prepared, final String sql) {
return now + "|" + elapsed + "|" + category + "|connection " + connectionId + "|" + sql;
}
}
Then compile just the file
javac com.p6spy.engine.spy.appender.SingleLineFormat.java
And replace the existing class file in p6spy.jar with the new one.
I am trying to update an embedded entity and JPA seems to generate the wrong SQL.
I have a Company entity with an embedded Logo entity
#Entity
public class Company {
private Long id;
#Embedded
private Logo logo;
// Omitted other fields, getters, setters, etc
}
#Embeddable
public class Logo {
private String fileName;
private String fileExtension;
private String imageDataType;
// Omitted getters and setters
}
In my DAO method I am trying to update the embedded logo like this:
#Override
public void setLogo(Logo logo, Long companyId) {
String q = "update Company c SET c.logo = :logo where c.id = :companyId";
Query query = entityManager.createQuery(q);
query.setParameter("companyId", companyId);
query.setParameter("logo", logo);
query.executeUpdate();
}
JPA (Hibernate actually) generates the following SQL.
update px_company set file_extension, file_name, file_type=(?, ?, ?) where id=?
Hibernate seems to understand it must update the three embedded logo fields, but it generates invalid SQL for it. The generated SQL results in an error.
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ' file_name, file_type=('jpg', '7679075394', 0) where id=1' at line 1
Any idea how I should update the embedded entity?
A bit old but just had the same issue - you should fully resolve properties of embedded classes in JPQL:
update Company c
SET c.logo.fileName = :fileName
,c.logo.fileExtension = :fileExtension
,c.logo.imageDataType= :imageDataType
where c.id = :companyId