I have this DDL
CREATE TABLE user
(
id bigint NOT NULL PRIMARY KEY,
name string NOT NULL
);
org.jooq.codegen.KotlinGenerator generates such a class:
data class Users(
var id: Long? = null,
var name: String? = null,
): Serializable
I expect non-nullable fields to be non-nullable, like:
data class Users(
var id: Long,
var name: String,
): Serializable
I only use these settings:
generate.apply {
isRecords = true
isDaos = true
isPojosAsKotlinDataClasses = true
isSpringAnnotations = true
isPojos = true
}
Jooq ver. 3.15.10
How do I configure the generator so that the fields are non-nullable?
As of jOOQ 3.17, there are pending feature requests to add some additional convenience at the price of correctness to generated classes:
#10212 Add option to handle nullability in KotlinGenerator
#12934 Null-safe views of columns known to be non-null
The main reason why this hasn't been implemented yet is the many ways such non-nullability promises can break in the event of using operators like LEFT JOIN, UNION and many others, in case of which an expression that appears to be non-null will produce null values nonetheless.
While #10212 is very hard to get right, #12934 might be a compromise to be implemented soon, given that it affects only generated data class types, which can be used on a 1:1 basis by users who understand the tradeoffs.
Related
I'm reviewing Spark SQL for a project and I see all the pieces of the API I need (SQL Parser, Dataset, Encoder, LogicalPlan, etc) however I'm having difficulty tying them together the way I'd like.
Essentially I want the following functionality:
var ddl = parseDDL(RAW_DDL);
var query = parseQuery("SELECT c1, c2 FROM t1 WHERE c2='value'", ddl);
var selectFields = query.getSelectFields();
for(var field: selectFields) {
var name = field.getName();
var type = field.getType(); // <~~~ want this in terms of `t1` from ddl
...
}
The type information for the select list in terms of the DDL is what I'm after.
Ideally I'd like a soup-to-nuts example with Spark SQL, if possible.
UPDATE
To clarify let's say I have an SQL schema file with several CREATE TABLE statements:
File: com/abc/MovieDb.sql
CREATE TABLE Movie (
Title varchar(255),
Year integer,
etc.
);
CREATE TABLE Actor (
FirstName varchar(255),
LastName varchar(255),
etc.
);
etc.
I want to use Spark SQL to parse a number of arbitrary SQL SELECT statements against this schema. Importantly, I want to get type information about the select list of each query in terms of the Movie, Actor, etc. tables and columns in the schema. For example:
SELECT Title, Year FROM Movie WHERE Year > 1990
I want to parse this query against the schema and get the type information for the select list. Again the queries are arbitrary, however the schema is stable so something like:
var parser = createParser(schema);
var query = parser.parseQuery(arbitraryQuery);
var selectedFields = query.getSelectedFields();
for (var field: selectedFields) {
var name = field.getName();
var type = field.getType();
}
Most important is the field.getType() call.
I assumed this would be an easy Yes or No type question, but perhaps my use-case is off the beaten path. Time to dive into it myself...
to get columns information here is what can be done
suppose you have dataframe with columns A,B,C,D in it
val inputDf= Seq(("foo","Bar",0,0.0)).toDf("a","b","c","d")
val newDf = inputDf.select("a","c")
val columnInfo= newDf.dtypes // should give you something like (("a","StringType"),("c","IntegarType"))
again this is not tested code but generally this is how you can get the column names and their types.
When I define a domain object like:
class MusicPlayed {
String user
Date date = new Date()
String mood
static mapping = {
id name: 'played_id'
version false
}
}
I get a postgres sequence automatically defined like:
CREATE SEQUENCE seq_music_played
INCREMENT 1
MINVALUE 1
MAXVALUE 9223372036854775807
START 1
CACHE 1;
That's great -- but I'd love to have this become the default value for my id field. In other words, I'd like to have the table defined with:
played_id bigint DEFAULT nextval('seq_music_played'::regclass) NOT NULL,
... but this doesn't happen. So when my client code requires manual SQL invocation, I'm stuck pulling new values form the sequence instead of just relying on auto-population.
Is there any way to cause this table to be created "the way I want," or do I need to forgo gorm's table-creation magic and just create the tables myself with a db-creation script that runs at install-time?
Note My question is similar to How to set up an insert to a grails created file with next sequence number?, but I'm specifically looking for a solution that doesn't pollute my client code.
This works for me :-)
static mapping = {
id generator: 'native', params: [sequence: 'my_seq'], defaultValue: "nextval('my_seq')"
}
Generating something like:
create table author (
id int8 default nextval('nsl_global_seq') not null,...
for postgresql.
I would use:
static mapping = {
id generator: 'native', params:[sequence:'your_seq']
}
Additionally, i would update the DEFAULT-Value of the id-column via
ALTER TABLE your_table ALTER COLUMN id SET DEFAULT nextval('your_seq');
This is extremely useful for manual INSERTs
UPDATE - use liquibase for the default-column-problem:
changeSet(author:'Bosh', id:'your_table_seq_defaults', failOnError: true) {
sql ("ALTER TABLE your_table ALTER COLUMN id SET DEFAULT nextval('your_seq')")
}
I tend to create my tables directly in PostgreSQL and then map them in grails.
I took the best idea to the sequences-generated-IDs from here:
http://blog.wolfman.com/articles/2009/11/11/using-postgresql-with-grails
Give it a try and then smile at your former problems :)
rawi
You can define it in Config.groovy
grails.gorm.default.mapping = {
id generator: 'sequence'
}
I have a to transfer RDF data from a triplestore to property tables. Example of a triple store and a property table is given below.
triplestore
Subject Property Object
Sub1 prop1 hello
Sub2 prop1 hello1
Sub3 prop1 hello2
Sub1 prop2 world
Sub2 prop2 world1
Sub3 prop2 world2
Sub4 prop3 random
Property Table 1
Subject prop1 prop2
Sub1 hello world
Sub2 hello1 world1
Sub3 hello2 world2
Property Table 2
Subject prop3
Sub4 random
This is a very simplified version of the dataset I am using. There are around a million records in the triplestore table. More than one property tables have to be created depending upon the various groupings of the various properties and objects. I have identified and created the various property tables? The properties that make a property table are chosen in such a way that a subject is fully contained by a single property table?
The problem that I am facing is the insertion of data from the triplestore to the property tables. Is there a way that data for a particular subject can be inserted into a row of property table in a single insert statement.If it cant be done in a single query what is the most efficient way to do so.
I am using python to create a dump of SQL queries which I latter run on a postgres server.
This is easy if you have a known, fixed set of properties. If you do not have a known set of fixed properties you have to generate dynamic SQL, either from your app, from PL/PgSQL or using the crosstab function from the tablefunc extension.
For fixed property sets you can self-join:
http://sqlfiddle.com/#!12/391b7/6
SELECT p1."Subject", p1."Object" AS "prop1", p2."Object" AS "prop2"
FROM triplestore p1
INNER JOIN triplestore p2 ON (p1."Subject" = p2."Subject")
WHERE p1."Property" = 'prop1'
AND p2."Property" = 'prop2'
ORDER BY p1."Subject";
SELECT p1."Subject", p1."Object" AS "prop1"
FROM triplestore p1
WHERE p1."Property" = 'prop3'
ORDER BY p1."Subject";
To turn these into INSERTs simply use INSERT ... SELECT eg:
INSERT INTO "Property Table 1"
SELECT p1."Subject", p1."Object" AS "prop1"
FROM triplestore p1
WHERE p1."Property" = 'prop3'
ORDER BY p1."Subject";
Generally speaking what you try to do smells a bit of EAV (Entity Attribute Value) Pattern - which is widely considered an antipattern. In addition i think i don't really understand what you are trying to achieve therefor sorry if my answer doesn't suit your needs
If your problem is storing data of previously unknown format under a certain key - in your example this seem to be subject - i would suggest using the postgres contrib hstore extension. Using this would allow you to create a table like
create table foo (
id serial not null primary key,
subject character varying not null,
properties hstore
);
in which the properties field is essentially what Ruby for instance calls a "Hash". You can insert key/value pairs into this store (from your above example for instance 'prop1=>hello' and select it with equivalent syntax.
Inserting is fairly straight forward:
insert into foo (subject, properties) values ('Sub1', 'prop1=>Hello'::hstore);
Advantage over using other methods is that hstore supports btree, gin and gist indexes (all of which under certain preconditions). In your case - doing mostly direct matches searching for a certain value in a property even btree works since it support the equality operator for hstore.
I currently have an issue when updating a component collection on an entity. It was originally mapped as a bag but that was causing all entries to be deleted and reinserted each time. Changing it to a set has fixed that problem but introduced a new one.
The component type is called Tracking it has a composite key of UserID and ItemID and two properties which are nullable dates. When one of these is created DateRead set to the current time, it is then replaced later with an entry with the new date on.
The underlying SQL NHibernate generates has a where clause which checks that all properties match.
The issue is, the other date DateAcknowledged is often null, and the generated SQL seems to have a syntax error, to do a null check its doing this: = NULL rather than: IS NULL, as shown:
DELETE FROM TrackingTable
WHERE ItemId = 'a68f6dea-1c00-42e2-bc40-9fcf01121bd8' /* #p0 */
AND UserId = 'c8aa41a4-e4c2-4347-ae6e-b48738a53b47' /* #p1 */
AND DateRead = '2012-01-26T12:56:46.00' /* #p2 */
AND DateAcknowledged = NULL /* #p3 */
The thing is, the two dates should not be needed at all to determine what to delete. Simply having the where check item ID and user ID would do.
Here is the mapping code where I define the set:
Set(x => x.Trackings,
mapper =>
{
mapper.Key(k => k.Column("ItemId"));
mapper.Table("Tracking");
mapper.Access(Accessor.NoSetter);
},
collectionMapping => collectionMapping.Component(TrackingMap.Mapping()));
And here is the mapping for the component:
public class TrackingMap
{
public static Action<IComponentElementMapper<Tracking>> Mapping()
{
return c =>
{
c.ManyToOne(x => x.User, a => { a.Column("UserId"); a.ForeignKey("UserId"); });
c.Property(x => x.DateRead);
c.Property(x => x.DateAcknowledged, a => a.NotNullable(false));
};
}
}
Is there a way to tell NHibernate to use the keys only on the where clause or for it to compare nulls in the correct way?
This is covered by section 7.2. Collections of dependent objects which notes that this is not supported:
Please note that a composite element mapping doesn't support null-able properties if you're using a <set>. NHibernate has to use each columns value to identify a record when deleting objects (there is no separate primary key column in the composite element table), which is not possible with null values. You have to either use only not-null properties in a composite-element or choose a <list>, <map>, <bag> or <idbag>.
Please see this answer for a good explanation of how to delete NHibernate entities using criteria. This should allow you to use only ItemId and UserId when determining the items to delete and safely ignore the date comparisons.
I have some tables that represent various types. They usually just consist of an ID (int), and a name. Ideally I would have this in an enum. Is there a way to map a table like this onto an enum?
EDIT: How would I handle it if there were extra fields other than an ID and Name?
If it's just an id and a name, I generally do this:
public enum FootScent : int
{
Unknown = 0,
Mild = 1,
Sweaty =2,
SteppedInSomething = 3
}
and then on the LINQ entity property:
[Column("foot_scent_id", DbType = "Int NOT NULL")]
public FootScent Scent { get; set; }
For lookup tables with columns other than "id" and "name" that are needed, I usually just make a normal LINQ entity for them, though depending on your implementation it would probably be worth caching these to minimize trips to the DB.