Unable to bulk insert using NHibernate - nhibernate

I've tried adding bulk inserting to my application, but the Batcher is still NonBatchingBatcher with a BatchSize of 1.
This is using C#3, NH3RC1 and MySql 5.1
I've added this to my SessionFactory
<property name="adonet.batch_size">100</property>
And my code goes pretty much like this
var session = SessionManager.GetStatelessSession(type);
var tx = session.BeginTransaction();
session.Insert(instance);
I'm using HILO identity generation for the instances in question, but not for all instances on the database. The SessionFactory.OpenStatelessSession doesn't take a type, so it can't really know it can do batching on this type, or...?
After some digging into NHibernate, I found something in SettingsFactory.CreateBatcherFactory that might give some additional info
// It defaults to the NonBatchingBatcher
System.Type tBatcher = typeof (NonBatchingBatcherFactory);
// Environment.BatchStrategy == "adonet.factory_class", but I haven't
// defined this in my config file
string batcherClass = PropertiesHelper.GetString(Environment.BatchStrategy, properties, null);
if (string.IsNullOrEmpty(batcherClass))
{
if (batchSize > 0)
{
// MySqlDriver doesn't implement IEmbeddedBatcherFactoryProvider,
// so it still uses NonBatchingFactory
IEmbeddedBatcherFactoryProvider ebfp = connectionProvider.Driver as IEmbeddedBatcherFactoryProvider;
Could my configuration be wrong?
<hibernate-configuration xmlns="urn:nhibernate-configuration-2.2" >
<session-factory name="my application name">
<property name="adonet.batch_size">100</property>
<property name="connection.driver_class">NHibernate.Driver.MySqlDataDriver</property>
<property name="connection.connection_string">my connection string
</property>
<property name="dialect">NHibernate.Dialect.MySQL5Dialect</property>
<property name="proxyfactory.factory_class">NHibernate.ByteCode.Castle.ProxyFactoryFactory, NHibernate.ByteCode.Castle</property>
<!-- To avoid "The column 'Reserved Word' does not belong to the table : ReservedWords" -->
<property name="hbm2ddl.keywords">none</property>
</session-factory>
</hibernate-configuration>

I know this question is a year old, but there is a NuGet package that adds MySQL batching functionality to NHibernate. The reason that it's not baked directly into NHibernate is that the functionality required a reference to the MySQL.Data assembly, and the dev team didn't want the dependency.

IIRC, batching is currently supported for Oracle and SqlServer only.
As almost any other aspect of NH, this is extensible, so you can write your own IBatcher/IBatcherFactory and inject them via configuration.
Sidenote: current version of NH is 3.0 GA.

Really old question but...let's be completely correct
Another reason for batching not working can be use of stateless session (as in your case). Stateless session does not support batching. From documentation:
The insert(), update() and delete() operations defined by the
StatelessSession interface are considered to be direct database
row-level operations, which result in immediate execution of a SQL
INSERT, UPDATE or DELETE respectively. Thus, they have very different
semantics to the Save(), SaveOrUpdate() and Delete() operations
defined by the ISession interface.

Related

using Nhibernate lazy proxys with a webservice

Recently i had some performance problems in a SOAP webservice I wrote a while ago. I noticed I had a lot of queries going on and my hbm.xml mappings where full of lazy=false statements. I upgraded to NHibernate 3.0 and removed the lazy = false stuff and everything was a LOT faster....but now i am getting the following error:
System.InvalidOperationException: There was an error generating the XML document. ---> System.InvalidOperationException: The type UserProxy was not expected. Use the XmlInclude or SoapInclude attribute to specify types that are not known statically.
User is a class of which i removed the lazy=false property from the class tag like this:
<class name="User" table="Users" >
<id name="DatabaseID" unsaved-value="0" column="ID" type="integer" >
<generator class="native"/>
</id>
<property name="IsExpert"/>
.....more stuff here....
</class>
My webservice has a method like this (simplified a little..in real-life i use a repository-like pattern between the service and nhibernate):
[WebMethod]
public User GetUser(int userid)
{
session = GetCurrentSession();
return session.Load<User>(userid);
}
The webservice expects to serialize a user and NHibernate gives me a UserProxy (which is not a user exactly). How should I overcome this?
Don't return entities from the web method. Use a DTO.
Webservices cannot serialise proxies - session.Load(userId) will return a proxy. You should user session.Get(userId) .
I think the answers saying you should use DTOs are not helpful, there is a time and place for DTOs and sometimes you may just want to return the entity.
If the User has child proxy properties, I have a class for handling this situation. Basically it loops through all properties (using reflection, and recursively going through child objects and collections) and uses the NHibernate.IsInitialized to check whether the property is a proxy or the genuine article. If it is a proxy then it sets it to null, thus making it possible for WCF to serialise it.

EclipseLink very slow on inserting data

I'm using the latest EclipseLink version with MySQL 5.5 (table type InnoDB). I'm inserting about 30900 records (which could be also more) at a time.
The problem is, that the insert performance is pretty poor: it takes about 22 seconds to insert all records (compared with JDBC: 7 seconds). I've read that using batch writing should help - but doesn't!?
#Entity
public class TestRecord {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
public Long id;
public int test;
}
The code to insert the records:
factory = Persistence.createEntityManagerFactory("xx_test");
EntityManager em = factory.createEntityManager();
em.getTransaction().begin();
for(int i = 0; i < 30900; i++) {
TestRecord record = new TestRecord();
record.test = 21;
em.persist(record);
}
em.getTransaction().commit();
em.close();
And finally my EclipseLink configuration:
<persistence-unit name="xx_test" transaction-type="RESOURCE_LOCAL">
<class>com.test.TestRecord</class>
<properties>
<property name="javax.persistence.jdbc.driver" value="com.mysql.jdbc.Driver" />
<property name="javax.persistence.jdbc.url" value="jdbc:mysql://localhost:3306/xx_test" />
<property name="javax.persistence.jdbc.user" value="root" />
<property name="javax.persistence.jdbc.password" value="test" />
<property name="eclipselink.jdbc.batch-writing" value="JDBC" />
<property name="eclipselink.jdbc.cache-statements" value="true"/>
<property name="eclipselink.ddl-generation.output-mode" value="both" />
<property name="eclipselink.ddl-generation" value="drop-and-create-tables" />
<property name="eclipselink.logging.level" value="INFO" />
</properties>
</persistence-unit>
What I'm doing wrong? I've tried several setting, but nothing seems the help.
Thanks in advance for helping me! :)
-Stefan
Another thing is to add ?rewriteBatchedStatements=true to the data URL used by the connector.
This caused executing about 120300 inserts down to about 30s which was about 60s before.
JDBC Batch writing improves performance drastically; please try it
Eg: property name="eclipselink.jdbc.batch-writing" value="JDBC"
#GeneratedValue(strategy = GenerationType.IDENTITY)
Switch to TABLE sequencing, IDENTITY is never recommended and a major performance issue.
See,
http://java-persistence-performance.blogspot.com/2011/06/how-to-improve-jpa-performance-by-1825.html
I seem to remember that MySQL may not support batch writing without some database config as well, there was another post on this, I forget the url but you could probably search for it.
Probably the biggest difference besides the mapping conversion is the caching. By default EclipseLink is placing each of the entities into the persistence context (EntityManager) and then during the finalization of the commit it needs to add them all to the cache.
One thing to try for now is:
measure how long an em.flush() call takes after the loop but before the commit. Then if you want you could call em.clear() after the flush so that the newly inserted entities are not merged into the cache.
Doug

NHibernate setting access="field.camelcase-underscore" fails in version 3

I have a solution that was created with NHib 1.2 which we're upgrading to NHib 3.0.
Our hbm file has the following property:
<property name="ContentId" column="ContentId" access="field.camelcase-underscore" />
The class doesn't have a ContentId property. This was working fine in NHib 1.2 but now we're getting getting the following exception:
Could not compile the mapping document: XXXX.Core.Domain.Video.hbm.xml ---> NHibernate.MappingException: Problem trying to set property type by reflection ---> NHibernate.MappingException: class Core.Domain.Video, Core, Version=1.0.0.29283, Culture=neutral, PublicKeyToken=null not found while looking for property: ContentId ---> NHibernate.PropertyNotFoundException: Could not find the property 'ContentId', associated to the field '_contentId', in class 'Core.Domain.Video'.
Why would this stop working? Is it still supported in NHib 3?
We have many many properties like this that we might need to add.
NHibernate greatly improved its error messaging and diagnostics in NH2.X and again in NH3.X. You are telling NHibernate that you have a property and you want to map it via field access to a field named by _camelCase convention. You don't have a property named ContentId and NHibernate is letting you know that you lied to it. :)
Try updating your mapping to:
<property name="_contentId" column="ContentId" access="field" />
You will need to update any HQL or Criteria queries to use _contentId rather than ContentId. Another option would be to add a private ContentId property.
I'd like to provide information which helped me answer this question:
http://groups.google.com/group/nhusers/browse_thread/thread/e078734a221c3c0c/ec8b873b385d4426?lnk=gst&q=field+camelcase+underscore#ec8b873b385d4426
In this link Fabio explains the same problem you are having like this:
This mapping
<property name="PositiveValue" access="field.camelcase-underscore" />
mean: For my property named "PositiveValue" you (NH) have to access to
the field; to discover which is the associated field you (NH) have to
use the strategy "camelcase-underscore".
If there is no property you can't use the accessor with a specific
strategy.
Which struck me as a little odd because it meant adding dummy, unused properties, just to make the nhibernate3 compiler happy. The underlying functionality is the same.

Coldfusion ORM 9.0.1 - Error while resolving relationship

I got this example from the adobe coldfusion documentation, some of the names are changed but everything else is the same, unless I am just so frustrated that I have missed a letter.
user.cfc:
/**
*#persistent
*/
component
{
property name="id" fieldtype="id" generator="native";
property name="userName" type="string" length="100";
property name="Credential" fieldtype="one-to-one" cfc="model.user.credentials";
}
credentials.cfc:
/**
*#persistent
*/
component
{
property name="id" fieldtype="id" generator="foreign" params="{property='userinfo'}";
property name="userinfo" fieldtype="one-to-one" cfc="model.user.user" constrained="true";
property name="passwordHash" type="string";
}
no matter how I word it, after searching many sites, I still get a error of:
Error while resolving the relationship Credential in cfc user. Check the column mapping for this property.
I have checked that both cfcs are accessible by coldfusion by removing the one-to-one properties and the tables have been created successfully.
I am using SQL Server 2008 with Coldfusion 9.0.1 under Apache 2.2 web server.
I am new to ORM and Hibernate but have successfully created different types of relationships and will confess to a less then expert level of coldfusion.
Thanks, this is really bothering me as this came directly from the coldfusion documentation.
Do you have a mapping for model?
If not, add one, or you could try:
property name="Credential" fieldtype="one-to-one" cfc="credentials";

Connect to ESRI Shape File (DBase *.dbf file) from NHibernate

I've been trying to work out how to connect to an ESRI shape file (which I think is a DBase table file) through NHibernate but haven't had any luck with anything I've tried.
Currently, my config's looking like this:
<property name="connection.provider">NHibernate.Connection.DriverConnectionProvider</property>
<!--<property name="dialect">NHibernate.Dialect.GenericDialect</property>
<property name="connection.driver_class">NHibernate.Driver.OdbcDriver</property>
<property name="connection.connection_string">Database=A303.dbf;protocol=TCPIP</property>-->
<property name="connection.driver_class">NHibernate.Driver.OdbcDriver</property>
<!--<property name="connection.connection_string">driver={IBM DB2 ODBC DRIVER};Database=a303.dbf;protocol=TCPIP</property>-->
<property name="connection.connection_string">Provider=VFPOLEDB.1; Data Source=C:\projects\rm4\Sandbox\bin\Debug\A303.dbf;Extended Properties=dBase III</property>
<property name="dialect">NHibernate.Dialect.DB2Dialect</property>
<property name="use_outer_join">true</property>
<property name="proxyfactory.factory_class">NHibernate.ByteCode.Castle.ProxyFactoryFactory, NHibernate.ByteCode.Castle</property>
<property name="show_sql">true</property>
I've left the commented out bits in so you can see what values I've been trying. No matter what I try, I get the error message:
ERROR [IM002] [Microsoft][ODBC Driver Manager] Data source name not found and no default driver specified
I've gone through most of the connection string I've found online and in some answers to questions on here was getting to the 'clutching at straws' phase where I'm just putting anything in so thought I'd better ask for help.
I'm not even sure if it's possible to connect to this type of file from NHibernate but, if it is, does anyone know what should be in the config?
A Shapefile (.shp) is not a dbf, per se. It actually is a collection of files, one of which is a DBF, but the shapefile itself that stores the geometry is a different format altogether.
There is a whitepaper on the ESRI website (www.esri.com)
I would try a different NHibernate driver. Here is a list of NHibernate drivers from the documentation.
Judging from the provider name in your connection string, I would try NHibernate.Driver.OleDbDriver.
Failing this, I would eliminate NHibernate from the mix and see if you can connect using the standard .NET data classes, such as System.Data.Odbc.OdbcConnection and System.Data.OleDb.OleDbConnection. If you cannot connect at this level, then the problem is not NHibernate.