How can I use ORMLite with objects that contain ArrayLists of other objects? - arraylist

I have a Android java project with quite complex datatypes including ArrayLists of other complex datatypes. I’m trying to store my data in a database using ORMLite (4.42).
I have major problems converting between ArrayLists and ForeignCollection, has anyone manage to do this?
I don’t want a continuous sync with the database, instead I have a save method that saves the entire structure and a read method that restores the entire method. After an object is restored from the database, the object is sent via interfaces where the receiver expects a ArrayLists and not orm lite ForeignCollections.
My approach is the following. For each ArrayList I have a complementary ForeignCollection. Before storing an object to the database, the content of the ArrayList is copied to the ForeignCollection, and when read back, vice versa.
When I read out a data structure from the database, I want the ensure that the entire structure is restored since later on, the object will be sent to users that are not aware about orm lite package, thus I set eager = true.
My complex datatype looks like this:
#DatabaseTable(tableName = "accounts")
public class Account {
#DatabaseField(generatedId = true)
private int id;
#ForeignCollectionField (eager = true, maxEagerLevel = 2)
private ForeignCollection<Order> ordersFoC;
private ArrayList<Order> ordersArL;
The Problem
Before storing the Account to the database, I tried to do the following. My hope was to copy the objects in the ArrayList to the ForeignCollection and the update the database with the Account object. The object myAccount has the same id as an object in the database so I expect an update of that object.
ordersFoC.addAll(orderArL);
myDao.createOrUpdate(myAccount);
No luck. The addAll method does directly write the orders to the database so I end up with duplicates of all orders since “addAll” adds and does not update any existing records.
So I tried to first remove all the orders from the database, but again with no luck:
ordersFoC.clear(); // Also tried: ordersFoC.removeAll(ordersFoC);
ordersFoC.addAll(orderArL);
myDao.createOrUpdate(myAccount);
The objects are not removed from the database. I read at groups.google.com (by Philip Fu) that this is a “known” problem with ormlite, that eager ForeignCollection does not play well with removeAll() or clear().
Is there a pattern that I have missed that allows me to work with objects that includes ArrayLists of complex types using the ormlite framework?

I've got my code working with ArrayLists ok. I declared my fields like this:
#ForeignCollectionField(eager = true, maxEagerForeignCollectionLevel = 99)
private Collection<PersonalNameStructure> personalnamestructures = new ArrayList<>();
and the getter as
public List<PersonalNameStructure> getPersonalNameStructures() {
return new ArrayList<>(personalnamestructures) ;
}
so it creates the ArrayList on the fly rather than trying to maintain it.
Obviously you may realise this already, but you also need to be careful around writing the objects as the eager setting affects reading foreign collections, it doesn't do anything for foreign fields or when writing a record.

Related

Using the DoctrineObjectConstructor, how are new entities created?

I am attempting to use JMSSerializerBundle to consume JSON into Doctrine entities. I need to both create new entities where they do not already exist in the database, and update existing entities when they do already exist. I am using the DoctrineObjectConstructor included in the JMSSerializer package to help with this. When I consume JSON which contains a property designated as an identifier, such as:
{
"id": 1,
"some_other_attribute": "stuff"
}
by attempting to deserialize it, JMSSerializer causes warnings and eventually dies with an exception for attempting to utilize reflection to set properties on a null value. The warnings all look like this:
PHP Warning: ReflectionProperty::setValue() expects parameter 1 to be object, null given in /Users/cdonadeo/Repos/Ubertester/vendor/jms/serializer/src/JMS/Serializer/GenericDeserializationVisitor.php on line 176
If I manually insert an entity with ID 1 in my database and make another attempt then I receive no errors and everything appears to be working correctly, but I'm now short half my functionality. I looked at the code for the DoctrineObjectConstructor class, and at the top is a comment:
/**
* Doctrine object constructor for new (or existing) objects during deserialization.
*/
But I don't see how it could possibly create a new a new entity because after the construct() function has done all of its checks, at the end it calls:
$object = $objectManager->find($metadata->name, $identifierList);
And since the identifier does not exist in the database the result is null which is ultimately what gets returned from the function. This explains why inserting a row in the database with the appropriate ID makes things work: find() now returns a proper Entity object, which is what the rest of the library expects.
Am I using the library wrong or is it broken? I forked the Git repo and made an edit, and trying it out everything seems to work more or less the way I expected. That edit does have some drawbacks that make me wonder if I'm not just making this more difficult than it has to be. The biggest issue I see is that it will cause persisted and unpersisted entities to be mixed together with no way to tell which ones are which, but I don't know if that's even a big deal.
For Doctrine entities use configuration:
jms_serializer:
object_constructors:
doctrine:
fallback_strategy: "fallback" # possible values ("null" | "exception" | "fallback")
see configuration reference https://jmsyst.com/bundles/JMSSerializerBundle/master/configuration

Does CF ORM have an Active Record type Update()?

Currently I am working partly with cfwheels and its Active Record ORM (which is great), and partly raw cfml with its Hibernate ORM (which is also great).
Both work well for applicable situations, but the thing I do miss most when using CF ORM is the model.update() method that is available in cfwheels, where you can just pass a form struct to the method, and it will map up the struct elements with the model properties and update the records.. really good for updating and maintaining large tables. In CF ORM, it seems the only way to to update a record is to set each column individually, then do a save. Is this the case?
Does cf9 ORM have an Active Record type update() (or equivalent) method which can just receive a struct with values to update and update the object without having to specify each one?
For example, instead of current:
member = entityLoadByPK('member',arguments.id);
member.setName(arguments.name);
member.setEmail(arguments.email);
is there a way to do something like this in CF ORM?
member = entityLoadByPK('member',arguments.id);
member.update(arguments);
Many thanks in advance
In my apps I usually create two helper functions for models which handle the task:
/*
* Get properties as key-value structure
* #limit Limit output to listed properties
*/
public struct function getMemento(string limit = "") {
local.props = {};
for (local.key in variables) {
if (isSimpleValue(variables[local.key]) AND (arguments.limit EQ "" OR ListFind(arguments.limit, local.key))) {
local.props[local.key] = variables[local.key];
}
}
return local.props;
}
/*
* Populate the model with given properties collection
* #props Properties collection
*/
public void function setMemento(required struct props) {
for (local.key in arguments.props) {
variables[local.key] = arguments.props[local.key];
}
}
For better security of setMemento it is possible to check existence of local.key in variables scope, but this will skip nullable properties.
So you can make myObject.setMemento(dataAsStruct); and then save it.
There's not a method exactly like the one you want, but EntityNew() does take an optional struct as a second argument, which will set the object's properties, although depending on how your code currently works, it may be clunky to use this method and I don;t know whether it'll have any bearing on whether a create/update is executed when you flush the ORM session.
If your ORM entities inherit form a master CFC, then you could add a method there. Alternatively, you could write one as a function and mix it into your objects.
I'm sure you're aware, but that update() feature can be a source of security problems (known as the mass assignment problem) if used with unsanitized user input (such as the raw FORM scope).

Ensuring inserts after a call to a custom NHibernate IIdentifierGenerator

The setup
Some of the "old old old" tables of our database use an exotic primary key generation scheme [1] and I'm trying to overlay this part of the database with NHibernate. This generation scheme is mostly hidden away in a stored procedure called, say, 'ShootMeInTheFace.GetNextSeededId'.
I have written an IIdentifierGenerator that calls this stored proc:
public class LegacyIdentityGenerator : IIdentifierGenerator, IConfigurable
{
// ... snip ...
public object Generate(ISessionImplementor session, object obj)
{
var connection = session.Connection;
using (var command = connection.CreateCommand())
{
SqlParameter param;
session.ConnectionManager.Transaction.Enlist(command);
command.CommandText = "ShootMeInTheFace.GetNextSeededId";
command.CommandType = CommandType.StoredProcedure;
param = command.CreateParameter() as SqlParameter;
param.Direction = ParameterDirection.Input;
param.ParameterName = "#sTableName";
param.SqlDbType = SqlDbType.VarChar;
param.Value = this.table;
command.Parameters.Add(param);
// ... snip ...
command.ExecuteNonQuery();
// ... snip ...
return ((IDataParameter)command
.Parameters["#sTrimmedNewId"]).Value as string);
}
}
The problem
I can map this in the XML mapping files and it works great, BUT....
It doesn't work when NHibernate tries to batch inserts, such as in a cascade, or when the session is not Flush()ed after every call to Save() on a transient entity that depends on this generator.
That's because NHibernate seems to be doing something like
for (each thing that I need to save)
{
[generate its id]
[add it to the batch]
}
[execute the sql in one big batch]
This doesn't work because, since the generator is asking the database every time, NHibernate just ends up getting the same ID generated multiple times, since it hasn't actually saved anything yet.
The other NHibernate generators like IncrementGenerator seem to get around this by asking the database for the seed value once and then incrementing the value in memory during subsequent calls in the same session. I would rather not do this in my implementation if I have to, since all of the code that I need is sitting in the database already, just waiting for me to call it correctly.
Is there a way to make NHibernate actually issue the INSERT after each call to generating an ID for entities of a certain type? Fiddling with the batch size settings don't seem to help.
Do you have any suggestions/other workarounds besides re-implementing the generation code in memory or bolting on some triggers to the legacy database? I guess I could always treat these as "assigned" generators and try to hide that fact somehow within the guts of the domain model....
Thanks for any advice.
The update: 2 months later
It was suggested in the answers below that I use an IPreInsertEventListener to implement this functionality. While this sounds reasonable, there were a few problems with this.
The first problem was that setting the id of an entity to the AssignedGenerator and then not actually assigning anything in code (since I was expecting my new IPreInsertEventListener implementation to do the work) resulted in an exception being thrown by the AssignedGenerator, since its Generate() method essentially does nothing but check to make sure that the id is not null, throwing an exception otherwise. This is worked around easily enough by creating my own IIdentifierGenerator that is like AssignedGenerator without the exception.
The second problem was that returning null from my new IIdentifierGenerator (the one I wrote to overcome the problems with the AssignedGenerator resulted in the innards of NHibernate throwing an exception, complaining that a null id was generated. Okay, fine, I changed my IIdentifierGenerator to return a sentinel string value, say, "NOT-REALLY-THE-REAL-ID", knowing that my IPreInsertEventListener would replace it with the correct value.
The third problem, and the ultimate deal-breaker, was that IPreInsertEventListener runs so late in the process that you need to update both the actual entity object as well as an array of state values that NHibernate uses. Typically this is not a problem and you can just follow Ayende's example. But there are three issues with the id field relating to the IPreInsertEventListeners:
The property is not in the #event.State array but instead in its own Id property.
The Id property does not have a public set accessor.
Updating only the entity but not the Id property results in the "NOT-REALLY-THE-REAL-ID" sentinel value being passed through to the database since the IPreInsertEventListener was unable to insert in the right places.
So my choice at this point was to use reflection to get at that NHibernate property, or to really sit down and say "look, the tool just wasn't meant to be used this way."
So I went back to my original IIdentifierGenreator and made it work for lazy flushes: it got the high value from the database on the first call, and then I re-implemented that ID generation function in C# for subsequent calls, modeling this after the Increment generator:
private string lastGenerated;
public object Generate(ISessionImplementor session, object obj)
{
string identity;
if (this.lastGenerated == null)
{
identity = GetTheValueFromTheDatabase();
}
else
{
identity = GenerateTheNextValueInCode();
}
this.lastGenerated = identity;
return identity;
}
This seems to work fine for a while, but like the increment generator, we might as well call it the TimeBombGenerator. If there are multiple worker processes executing this code in non-serializable transactions, or if there are multiple entities mapped to the same database table (it's an old database, it happened), then we will get multiple instances of this generator with the same lastGenerated seed value, resulting in duplicate identities.
##$##$#.
My solution at this point was to make the generator cache a dictionary of WeakReferences to ISessions and their lastGenerated values. This way, the lastGenerated is effectively local to the lifetime of a particular ISession, not the lifetime of the IIdentifierGenerator, and because I'm holding WeakReferences and culling them out at the beginning of each Generate() call, this won't explode in memory consumption. And since each ISession is going to hit the database table on its first call, we'll get the necessary row locks (assuming we're in a transaction) we need to prevent duplicate identities from happening (and if they do, such as from a phantom row, only the ISession needs to be thrown away, not the entire process).
It is ugly, but more feasible than changing the primary key scheme of a 10-year-old database. FWIW.
[1] If you care to know about the ID generation, you take a substring(len - 2) of all of the values currently in the PK column, cast them to integers and find the max, add one to that number, add all of that number's digits, and append the sum of those digits as a checksum. (If the database has one row containing "1000001", then we would get max 10000, +1 equals 10001, checksum is 02, resulting new PK is "1000102". Don't ask me why.
A potential workaround is to generate and assign the ID in an event listener rather than using an IIdentifierGenerator implementation. The listener should implement IPreInsertEventListener and assign the ID in OnPreInsert.
Why dont you just make private string lastGenerated; static?

Linq To SQL DAL and lookup data source

I am learning linq to sql and I am trying to setup a lookup combo box in my web page. I was using a linq data source but I then moved my linqtosql code to it's own class library for my DAL dll (took out of app_code folder). So, I am converting the page code to be able to still have lookups driven now by a biz object.
Here's what I have done in my biz layer...
Public Class KosherTypes
Public Shared Function GetKosherTypes() As List(Of KosherTypeLookup)
Dim db As New DBDataContext
Dim types = From kt In db.KosherTypes _
Where kt.IsDeleted = False _
Order By kt.Name _
Select New KosherTypeLookup With {.Name = kt.Name, .ID = kt.KosherTypeID}
Return types.ToList
End Function
End Class
I then setup an object data source and mapped it to this class.
I have a few questions as when I was doing internet searches I didn't find anyone that seemed to be doing this and yet I know lookup tables / combo boxes are common...
Did I totally just miss something and there are better way(s) to do this?
I went with returning a list but I could have returned an IEnumerable or IQueryable. In my reading it seemed IQueryable had more functionality for linq to sql but since I am returning a simple two column list then I only need to return an IEnumerable or List. I went with a List since it's strongly typed. Did I make a good decision? AKA - Should I just have returned and IEnumerable or perhaps gone with IQueryable?
Thanks!
I'll answer in reverse order:
I wouldn't use IQueryable outside of your repos / DAL for the simple reason that since execution is deferred, you lose control of what exactly is executed (i.e., an aribtrary function could be assigned as a delegate for WHERE), making maintenance and testing much harder. I don't see an issue with you returning an IEnumberable(Of KosherTypeLookup) though.
If the lookup is a static lookup that never or rarely changes, you might want to look at a way to cache the lookup after the first use, to avoid having to hit the db every time that box is called. It really depends on your expected load, though. As always, performance/load testing will highlight where you need to optimize.

vb.net object persisted in database

How can I go about storing a vb.net user defined object in a sql database. I am not trying to replicate the properties with columns. I mean something along the lines of converting or encoding my object to a byte array and then storing that in a field in the db. Like when you store an instance of an object in session, but I need the info to persist past the current session.
#Orion Edwards
It's not a matter of stances. It's because one day, you will change your code. Then you will try de-serialize the old object, and YOUR PROGRAM WILL CRASH.
My Program will not "CRASH", it will throw an exception. Lucky for me .net has a whole set of classes dedicated for such an occasion. At which time I will refresh my stale data and put it back in the db. That is the point of this one field (or stance, as the case may be).
You can use serialization - it allows you to store your object at least in 3 forms: binary (suitable for BLOBs), XML (take advantage of MSSQL's XML data type) or just plain text (store in varchar or text column)
Before you head down this road towards your own eventual insanity, you should take a look at this (or one day repeat it):
http://thedailywtf.com/Articles/The-Mythical-Business-Layer.aspx
Persisting objects in a database is not a good idea. It kills all the good things that a database is designed to do.
You could use the BinaryFormatter class to serialize your object to a binary format, then save the resulting string in your database.
The XmlSerializer or the DataContractSerializer in .net 3.x will do the job for you.
#aku, lomaxx and bdukes - your solutions are what I was looking for.
#1800 INFORMATION - while i appreciate your stance on the matter, this is a special case of data that I get from a webservice that gets refreshed only about once a month. I dont need the data persisted in db form because thats what the webservice is for. Below is the code I finally got to work.
Serialize
#'res is my object to serialize
Dim xml_serializer As System.Xml.Serialization.XmlSerializer
Dim string_writer As New System.IO.StringWriter()
xml_serializer = New System.Xml.Serialization.XmlSerializer(res.GetType)
xml_serializer.Serialize(string_writer, res)
Deserialize
#'string_writer and xml_serializer from above
Dim serialization As String = string_writer.ToString
Dim string_reader As System.IO.StringReader
string_reader = New System.IO.StringReader(serialization)
Dim res2 As testsedie.EligibilityResponse
res2 = xml_serializer.Deserialize(string_reader)
What you want to do is called "Serializing" your object, and .Net has a few different ways to go about it. One is the XmlSerializer class in the System.Xml.Serialization namespace.
Another is in the System.Runtime.Serialization namespace. This has support for a SOAP formatter, a binary formatter, and a base class you can inherit from that all implement a common interface.
For what you are talking about, the BinaryFormatter suggested earlier will probably have the best performance.
I'm backing #1800 Information on this one.
Serializing objects for long-term storage is never a good idea
while i appreciate your stance on the matter, this is a special case of data that I get from a webservice that gets refreshed only about once a month.
It's not a matter of stances. It's because one day, you will change your code. Then you will try de-serialize the old object, and YOUR PROGRAM WILL CRASH.
If it crashes (or throws an exception) all you are left with is a bunch of binary data to try and sift through to recreate your objects.
If you are only persisting binary why not just save straight to disk. You also might want to look at using something like xml as, as has been mentioned, if you alter your object definition you may not be able to unserialise it without some hard work.