Migrating Bugzilla to non fresh JIRA duplicate key - migration

I have a JIRA environment which already has some information and i'm trying to merge all the bugzilla bugs into JIRA.
I'm trying to use the importer form JIRA "BugzillaImportBean.java‎"
But it's failing when it tries to insert into the OS_CURRENTSTEP table because of a unique Key violation, essentially the ID already exists in JIRA in that table.
So it crashes at
final GenericValue issue = createIssue(resultSet, getProductName(resultSet, true), componentName);
Error importing data from Bugzilla: com.atlassian.jira.exception.CreateException: Could not create new current step for #259350: root cause: while inserting: [GenericEntity:OSCurrentStep][id,357430][startDate,2010-07-23 05:32:14.414][status,Open][owner,][finishDate,null][actionId,0][stepId,1][dueDate,null][entryId,259350] (SQL Exception while executing the following:INSERT INTO OS_CURRENTSTEP (ID, ENTRY_ID, STEP_ID, ACTION_ID, OWNER, START_DATE, DUE_DATE, FINISH_DATE, STATUS, CALLER) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) (Duplicate entry '357430' for key 1))
What is the best way of fixing this?
Bugzilla Database Schema: http://tldp.org/LDP/bugzilla/Bugzilla-Guide/dbschema.html
Jira Database Schema: http://confluence.atlassian.com/display/JIRA/Database+Schema
http://confluence.atlassian.com/display/JIRA/Modifying+the+Bugzilla+Importer
CREATE TABLE `OS_CURRENTSTEP` (
`ID` decimal(18,0) NOT NULL,
`ENTRY_ID` decimal(18,0) default NULL,
`STEP_ID` decimal(9,0) default NULL,
`ACTION_ID` decimal(9,0) default NULL,
`OWNER` varchar(60) default NULL,
`START_DATE` datetime default NULL,
`DUE_DATE` datetime default NULL,
`FINISH_DATE` datetime default NULL,
`STATUS` varchar(60) default NULL,
`CALLER` varchar(60) default NULL,
PRIMARY KEY (`ID`),
KEY `wf_entryid` (`ENTRY_ID`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;

The problem could be a duplicate sequence value. Check the SEQUENCE_VALUE_ITEM table, look for a row such as "OSCurrentStep" (if this is not the name, the mapping of tables to entity names is in WEB-INF/classes/entitydefs/entitymodel.xml)
select * from SEQUENCE_VALUE_ITEM where SEQ_NAME='OSCurrentStep'
Check what is the maximal used value:
select MAX(ID) from OS_CURRENTSTEP
Set SEQ_ID bigger than the maximal used value, rounding up to a multiple of 10.
(Described in http://confluence.atlassian.com/display/JIRA/Database+Schema # SEQUENCE_VALUE_ITEM)
The failed duplicate key '357430' is a multiple of 10, which suggests this is the reason
An easier but less likely solution: are you trying to import the same issue a second time?
If so, then "click the 'Import only new issues' checkbox in the importer" as described here: http://confluence.atlassian.com/display/JIRA/Importing+Data+from+Bugzilla
(You will notice that the failed statement is inside this condition: if (!onlyNewIssues || !previouslyImportedKeys.containsKey...)

Looks like the Bugzilla Importer has got confused about Status and Workflow steps. I can't remember if it tries to create new workflow steps on the fly? That importer is a right dog's breakfast, which is why I wrote my own product to do imports into JIRA. I'm doing another one tomorrow in fact.
Anyway, one way to narrow down the problem is to import a subset of issues. Perhaps you don't have the mapping from Bugzilla states (customized?) to JIRA statuses complete?
There's more info about the guts of this at http://confluence.atlassian.com/display/JIRA/Issue+status+and+workflow
~Matt

Related

current_timestamp() vs CURRENT_TIMESTAMP for default value on TIMESTAMP column (mariadb:latest/mysql, laravel, sequel pro)

What is the difference between CURRENT_TIMESTAMP and current_timestamp() ?
I'm using laravel, and in the laravel migration file for my Tasks table I have this:
$table->timestamp('created_at')->useCurrent();
$table->timestamp('updated_at')->useCurrent()->useCurrentOnUpdate();
For my database container, I am using a local docker setup with:
mariadb:latest (seems to be bringing up version 10.8.3-MariaDB-1:10.8.3+maria~jammy (mariadb.org binary distribution) )
and the weird thing is..
I have Sequel Pro open, and when trying to manually insert a record ( for testing purposes ) through the sequel pro interface and it is failing with the following error:
Incorrect datetime value: 'current_timestamp()' for column .. created_at..
Notice when I click to add a new row, the defaults are 'current_timestamp()'
If I manually change these defaults to 'CURRENT_TIMESTAMP' instead of 'current_timestamp()' it seems to work:
The function/call or lower case version of CURRENT_TIMESTAMP does not work...
If I add a new row programatically / with laravel:
$newTask = new Task();
$newTask->title = 'testing';
$newTask->save();
the row is inserted properly ( with the current timestamp values.. ):
Where is this problem at?
Laravel side/configuration?
Could it be the 'mariadb:latest' bringing up a bug?
Could it be a sequel pro bug ?
This is the create table definition btw:
CREATE TABLE `tasks` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`title` varchar(255) COLLATE utf8mb4_unicode_ci NOT NULL,
`description` text COLLATE utf8mb4_unicode_ci DEFAULT NULL,
`created_at` timestamp NOT NULL DEFAULT current_timestamp(),
`updated_at` timestamp NOT NULL DEFAULT current_timestamp() ON UPDATE current_timestamp(),
PRIMARY KEY (`id`)
) ENGINE=InnoDB AUTO_INCREMENT=10 DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_unicode_ci;
While developing this question, I have decided to download MySql Workbench and try inserting the values there through the MySql Workbench interface,
and it seems to work, and it seems that it is because MySql Workbench simply runs INSERT queries:
INSERT INTO `tasks_pabloserver_db`.`tasks` (`title`) VALUES ('teeest');
INSERT INTO `tasks_pabloserver_db`.`tasks` (`title`) VALUES ('test444');
which work and insert the proper default/timestamp values.
I looked at the table structure in MySql Workbench and the default value is 'current_timestamp()' and not 'CURRENT_TIMESTAMP', and it still works so it cannot be the database version I guess.
So then I tried to run these same INSERT statements in Sequel Pro and it also worked properly, so my conclusion is that Sequel Pro interface has a bug and that is to blame.

Oracle Application Express - 'ORA-01843: not a valid month error' in one table only

I am creating a database in Oracle Application Express and am having a problem inserting a date into one of the tables.
INSERT INTO VIEWING( VIEWING_ID, VIEWING_DATE, TIME, PROPERTY_ID, AGENT_ID)
VALUES('3', '12-07-2015' ,'10:00','1', '101');
I've tried every combination of date format, and trying to force the date to my correct format
to_date('12-07-2015','MM-DD-YYYY')
But nothing is working
CREATE TABLE Viewing (
Viewing_ID number(10) NOT NULL,
Viewing_Date date NOT NULL,
Time timestamp(7) NOT NULL,
Property_ID number(10) NOT NULL,
Agent_ID number(10) NOT NULL,
PRIMARY KEY (Viewing_ID));
ALTER TABLE Viewing ADD CONSTRAINT FK_Viewing_Agent_ID FOREIGN KEY (Agent_ID) REFERENCES Agent (Agent_ID);
ALTER TABLE Viewing ADD CONSTRAINT FK_Viewing_Property_ID FOREIGN KEY (Property_ID) REFERENCES Property (Property_ID);
Every Resource I have found suggests it is most likely a parsing or syntax error but so far nothing has helped.
I have a second table in the schema that I can insert dates into without a problem, the only difference on this table is that the date is required (I have tried making it nullable to test and I still get the same error)
I should point out that Im am completely new to Oracle and this is part of a study project. If I had I choice I would be using SQL Server! But Ive been at this for hours and think its time to admit defeat!
Thanks
It is due to TIME column, not VIEWING_DATE. This worked:
INSERT INTO VIEWING( VIEWING_ID, VIEWING_DATE, TIME, PROPERTY_ID, AGENT_ID)
VALUES(4, date '2015-12-07' , timestamp '2015-12-07 10:00:00',1, 101);

Configure Grails 3.0.9 for Oracle 12c with identity PK?

I am trying to map my domain objects to use the new Oracle 12c identity type primary keys, AKA auto-increment in some other systems.
Hibernate 4 does not have Oracle12cDialect, it only has Oracle10gDialect.
Oracle10gDialect has a method called supportsIdentityColumns() whitch is hard coded to return false, so mapping my GORM domain object with generator:"identity" results in an error saying that the Oracle10gDialect does not support identity generator.
I cannot use the GORM select generator because I do not have a secondary unique key and I cannot use a Hibernate generated key because then Hibernate and other (external) inserts into the tables would generate overlapping keys.
Example of Existing Oracle 12c DDL:
create table person (
id number(10,0) generated by default as identity,
version number(10,0) not null,
home_address_id number(10,0),
name varchar(255) not null,
primary key (id)
);
GORM Object:
class Person {
String name
Address homeAddress
static mapping = {
id column: 'person_key', generator: 'identity'
}
static constraints = {
homeAddress nullable: true
}
}
In Memory DB Result (Works Perfect):
Hibernate: create table person (person_key bigint generated by default as identity, version bigint not null, home_address_id bigint, name varchar(255) not null, primary key (person_key))
Hibernate: alter table person add constraint FK_bemy93e8a8i6nknj4n21m6fub foreign key (home_address_id) references address
Hibernate: insert into person (person_key, version, home_address_id, name) values (null, ?, ?, ?)
Oracle DB Result (Broken):
org.hibernate.MappingException: org.hibernate.dialect.Oracle10gDialect does not support identity key generation
How do I get Grails 3.0.9 to work with the above Oracle table definition?
Hibernate 4 can not be configured to use Oracle 12c identity key generation.
Creating a custom Oracle12cDialect did not allow us to use identity key generation. It requires additional support in Hibernate 4 that is not there.
What does work is sticking with the Oracle10gDialect and using generator: 'sequence-identity' and then naming the sequence like this:
static mapping = {
id column: 'person_key', generator: 'sequence-identity', params:[sequence:'person_seq']
}
This virtually achieves the same result other than creating the tables with the identity key word in the DDL. Even if we had been able to get the identity keyword in the table definition, Oracle would simply have created its own sequence in the background to use every time a record was inserted. Using sequence-identity rather than sequence, also avoids the double DB call to insert a new row. With identity-sequence the insert DML is a single call like this:
insert into person (person_key, version, home_address_id, name) values (person_seq.nextval, ?, ?, ?)
With generator: 'sequence' new record inserts become two DB calls like this:
select person_seq.nextval from dual;
insert into person (person_key, version, home_address_id, name) values (?, ?, ?, ?)
So the only downside that I see for 'identity-sequence' over 'identity' is simply that Oracle will not automatically keep track of which sequence to use for which table and automatically use it when no key value is provided in the insert statement. But even that could probably be handled by a before insert trigger, at which point you might be almost exactly where you would be anyway if Hibernate 4 had supported generator: identity.
Hibernate 5 does have an Oracle 12c Dialect, specifically adding "identity" support: org.hibernate.dialect.Oracle12cDialect. So either use Hibernate 5, or write a custom 12c-based Dialect for Hibernate 4.

Create Table Syntax Error - MS Access, SQL view

I keep getting CREATE TABLE Syntax Error, but I don't see the error! What is causing the error?
My SQL:
CREATE TABLE my_employee
(
employee_id INTEGER PRIMARY KEY NOT NULL,
first_name VARCHAR(25) NOT NULL,
last_name VARCHAR(30) NOT NULL,
address VARCHAR(10) NOT NULL,
birthdate DATE,
salary NUMERIC(8,2) DEFAULT 15000,
marital_status CHAR(1)
);
Since your DDL statement includes DEFAULT, you must execute it with ADO. I loaded your statement into a string variable and executed it from Access 2007 like this:
CurrentProject.Connection.Execute strSql
The salary field is decimal with precision 8, scale 2, and default 15000.
DEFAULT is one of the Access SQL features added with Jet 4.0. Those features are not available for a statement executed from DAO. If you are using Access' query designer to create and execute the statement, you're using DAO. Same if you were using CurrentDb.Execute. But CurrentProject.Connection is an ADO object, so it can .Execute Jet 4.0 features.
Note NOT NULL is not necessary after PRIMARY KEY since PRIMARY KEY implies NOT NULL. However PRIMARY KEY NOT NULL does not trigger an error. The statement works as you originally wrote it as long as you execute it from ADO.
Well I was having the same problem with Ms Access 2007, but I solved it later on.
It is because actually some features are disabled by default for security reasons over there.
When it shows you syntax error, you can see the message at menu bar somewhere or at the bottom: Some Features Are Disabled For Security Reasons....
Click on the message then proceed to enable further features.
As HansUp said: "default" in DDL doesn't work here. As an alternative you can create the table without the default first and add the default via the TableDef afterwards:
CurrentDb().Execute "create table my_employee ..."
CurrentDb().TableDefs("my_employee").Fields("salary").DefaultValue = 15000
Your problem is in your PRIMARY KEY declaration
CREATE TABLE my_employee
(
employee_id INTEGER NOT NULL,
first_name VARCHAR(25) NOT NULL,
last_name VARCHAR(30) NOT NULL,
address VARCHAR(10) NOT NULL,
birthdate DATE,
salary NUMERIC(8,2) DEFAULT 15000,
marital_status CHAR(1),
PRIMARY KEY (employee_id)
);

How do you setup Post Revisions/History Tracking with ORM?

I am trying to figure out how to setup a revisions system for posts and other content. I figured that would mean it would need to work with a basic belongs_to/has_one/has_many/has_many_though ORM (any good ORM should support this).
I was thinking a that I could have some tables like (with matching models)
[[POST]] (has_many (text) through (revisions)
id
title
[[Revisions]] (belongs_to posts/text)
id
post_id
text_id
date
[[TEXT]]
id
body
user_id
Where I could join THROUGH the revisions table to get the latest TEXT body. But I'm kind of foggy on how it will all work. Has anyone setup something like this?
Basically, I need to be able to load an article and request the latest content entry.
// Get the post row
$post = new Model_Post($id);
// Get the latest revision (JOIN through revisions to TEXT) and print that body.
$post->text->body;
Having the ability to shuffle back in time to previous revisions and removing revisions would also be a big help.
At any rate, these are just ideas of how I think that some kind of history tracking would work. I'm open to any form of tracking I just want to know what the best-practice is.
:EDIT:
It seems that moving forward, two tables seems to make the most sense. Since I plan to store two copies of text this will also help to save space. The first table posts will store the data of the current revision for fast reads without any joins. The posts body will be the value of the matching revision's text field - but processed through markdown/bbcode/tidy/etc. This will allow me to retain the original text (for the next edit) without having to store that text twice in one revision row (or having to re-parse it each time I display it).
So fetching will be be ORM friendly. Then for creates/updates I will have to handle revisions separately and then just update the post object with the new current revision values.
CREATE TABLE IF NOT EXISTS `posts` (
`id` int(10) unsigned NOT NULL AUTO_INCREMENT,
`published` tinyint(1) unsigned DEFAULT NULL,
`allow_comments` tinyint(1) unsigned DEFAULT NULL,
`user_id` int(11) NOT NULL,
`title` varchar(100) NOT NULL,
`body` text NOT NULL,
`created` datetime NOT NULL,
PRIMARY KEY (`id`),
KEY `user_id` (`user_id`),
KEY `published` (`published`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ;
CREATE TABLE IF NOT EXISTS `postsrevisions` (
`id` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`post_id` int(10) unsigned NOT NULL,
`user_id` int(10) unsigned NOT NULL,
`is_current` tinyint(1) unsigned DEFAULT NULL,
`date` datetime NOT NULL,
`title` varchar(100) NOT NULL,
`text` text NOT NULL,
`image` varchar(200) NOT NULL,
PRIMARY KEY (`id`),
KEY `post_id` (`post_id`),
KEY `user_id` (`user_id`),
KEY `is_current` (`is_current`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8 ;
Your Revisions table as you have shown it models a many-to-many relationship between Posts and Text. This is probably not what you want, unless a given row in Text may provide the content for multiple rows in Posts. This is not how most CMS architectures work.
You certainly don't need three tables. I have no idea why you think this is needed for 3NF. The point of 3NF is that an attribute should not depend on a non-key attribute, it doesn't say you should split into multiple tables needlessly.
So you might only need a one-to-many relationship between two tables: Posts and Revisions. That is, for each post, there can be multiple revisions, but a given revision applies to only one post. Others have suggested two alternatives for finding the current post:
A flag column in Revisions to note the current revision. Changing the current revision is as simple as changing the flag to true in the desired revision and to false to the formerly current revision.
A foreign key in Posts to the revision that is current for the given post. This is even simpler, because you can change the current revision in one update instead of two. But circular foreign key references can cause problems vis-a-vis backup & restore, cascading updates, etc.
You could even implement the revision system using a single table:
CREATE TABLE PostRevisions (
post_revision_id SERIAL PRIMARY KEY,
post_id INT NOT NULL,
is_current TINYINT NULL,
date DATE,
title VARCHAR(80) NOT NULL,
text TEXT NOT NULL,
UNIQUE KEY (post_id, is_current)
);
I'm not sure it's duplication to store the title with each revision, because the title could be revised as much as the text, couldn't it?
The column is_current should be either 1 or NULL. A unique constraint doesn't count NULLs, so you can have only one row where is_current is 1 and an unlimited number of rows where it's NULL.
This does require updating two rows to make a revision current, but you gain some simplicity by reducing the model to a single table. This is a great advantage when you're using an ORM.
You can create a view to simplify the common case of querying current posts:
CREATE VIEW Posts AS SELECT * FROM PostRevisions WHERE is_current = 1;
update: Re your updated question: I agree that proper relational design would encourage two tables so that you could make a few attributes of a Post invariant for all that post's revisions. But most ORM tools assume an entity exists in a single table, and ORM's are clumsy at joining rows from multiple tables to constitute a given entity. So I would say if using an ORM is a priority, you should store the posts and revisions in a single table. Sacrifice a little bit of relational correctness to support the assumptions of the ORM paradigm.
Another suggestion is to consider Dimensional Modeling. This is a school of database design to support OLAP and data warehousing. It uses denormalization judiciously, so you can usually organize data in a Star Schema. The main entity (the "Fact Table") is represented by a single table, so this would be a win for an ORM-centric application design.
You'd probably be better off in this case to put a CurrentTextID on your Post table to avoid having to figure out which revision is current (an alternative would be a flag on Revision, but I think a CurrentTextID on the post will give you easier queries).
With the CurrentTextID on the Post, your ORM should place a single property (CurrentText) on your Post class which would allow you to access the current text with essentially the statement you provided.
Your ORM should also give you some way to load the Revisions based on the Post; If you want more details about that then you should include information about which ORM you are using and how you have it configured.
I think two tables would suffice here. A post table and it's revisions. If you're not worried about duplicating data, a single table (de-normalized) could also work.
For anyone interested, here is how wordpress handles revisions using a single MySQL posts table.
CREATE TABLE IF NOT EXISTS `wp_posts` (
`ID` bigint(20) unsigned NOT NULL AUTO_INCREMENT,
`post_author` bigint(20) unsigned NOT NULL DEFAULT '0',
`post_date` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`post_date_gmt` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`post_content` longtext NOT NULL,
`post_title` text NOT NULL,
`post_excerpt` text NOT NULL,
`post_status` varchar(20) NOT NULL DEFAULT 'publish',
`comment_status` varchar(20) NOT NULL DEFAULT 'open',
`ping_status` varchar(20) NOT NULL DEFAULT 'open',
`post_password` varchar(20) NOT NULL DEFAULT '',
`post_name` varchar(200) NOT NULL DEFAULT '',
`to_ping` text NOT NULL,
`pinged` text NOT NULL,
`post_modified` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`post_modified_gmt` datetime NOT NULL DEFAULT '0000-00-00 00:00:00',
`post_content_filtered` text NOT NULL,
`post_parent` bigint(20) unsigned NOT NULL DEFAULT '0',
`guid` varchar(255) NOT NULL DEFAULT '',
`menu_order` int(11) NOT NULL DEFAULT '0',
`post_type` varchar(20) NOT NULL DEFAULT 'post',
`post_mime_type` varchar(100) NOT NULL DEFAULT '',
`comment_count` bigint(20) NOT NULL DEFAULT '0',
PRIMARY KEY (`ID`),
KEY `post_name` (`post_name`),
KEY `type_status_date` (`post_type`,`post_status`,`post_date`,`ID`),
KEY `post_parent` (`post_parent`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 ;