I got a question regarding id's inside an event sourcing architecture.
Since it's the goal to have a rebuildable state using the eventstore, the data projection should have a deterministic generation of primary keys for the current state data.
e.g.: when i create and then delete an entry afterwards, i save the id of the deleted item into the event store entry. Now when I want to rebuild my state from the Eventstore, the created item event should generate the same id into the projection - to be able to run the deleted event properly.
Does this mean i cannot use random UUID's or am i missing something?
When you start a rebuild of your read-model, then you could reset the "counter" that the automatic primary key generation value is based up-on. So that each time you do a rebuild, the primary-key always starts with for example 1.
How that is done depends on the database used.
Alternatively, you could just set the primary key manually using a counter in your code "somehow".
Related
I'm creating an application with Java Spring and Oracle DB.
In the app, I want to generate a primary key value that is unique as well as ordered and without gaps: 1,2,3,4,5 instead of 1,2,5,7,8,9.
I've at one point used max(id) + 1 to get the maximum value of the id, and the id of the next/current transaction. However I know it isn't perfect in the case of concurrency with multiple users or multiple sessions.
I've tried using sequences, but even with the ORDER tag it could still create gaps with the possibility of a failed transaction.
REATE SEQUENCE num_seq
START WITH 1
INCREMENT BY 1
ORDER NOCACHE NOCYCLE;
I need there to be gapless values as a requirement, however I'm unsure how it's possible in the case of multiple users/multiple sessions.
Don't do it.
The goal of primary keys is not to be displayed on the UI or to be exposed to the external world, but only to provide a unique identifier of the row.
In simple words, a primary key doesn't need to be sexy or good looking. It's an internal identifier.
If you are considering the idea of having serial identifier, that means you probably want to display it somewhere or you want to expose it to the external world. If that's the case, then create a secondary column (also unique) that serves this "public relations" goal. It can be automatically generated, or updated at leisure without affecting the integrity of the database.
It can also be generated by a secondary process that runs in a deferred way (e.g. every 10 minutes) that finds all the "unassigned" new rows, and gives them the new number. This has the advantage that is not vulnerable to concurrency.
Background:
I am trying to solve one simple problem. I have a database with two tables, one stores text (this is something like articles), and the other stores the category to which this text belongs. Users can make changes to the text, and I need to save who and when made the changes, also when saving changes, the user writes a comment on his changes, which I also save.
As I have done now:
I added another table to which I save everything related to changes, who made the changes and when, as well as a comment on the changes, and the ID of the text to which the changes apply.
What is the problem:
Deleting the text also needs to be recorded in history, but since in the records with history there is a foreign key with a check, then I have to delete the entire history that is associated with the text, so as not to get an error.
What I have tried else:
I tried to add an attribute to the table with the text "Deleted", and the row is not physically deleted, but the "Deleted" = 1 flag is simply set, and at the same time I can save the history, and even the moment of deletion. But there is another problem, the table with the text has an attribute "Name", which must be unique, and if the record is not physically deleted, then when I try to insert a new record with the value "Name", which already exists, I get a uniqueness error, although the old record with such a name is considered remote.
Question:
What are the approaches to solving the problem, in which it is possible to save the history of changes in another table, even after deleting records from the main table, and at the same time keep the uniqueness of some attributes of the main table and maintain data integrity.
I would be grateful for any options and hints.
A good practice is to use a unique identifier such as a UUID as the primary key for your primary record (ie. your text record). That way, you can safely soft delete the primary record and any associated metadata can be kept without fear of collisions in the future.
If you need to enforce uniqueness of certain attributes (such as the Name you mentioned) you can create a secondary index (non-clustered index in SQL terminology) on that column in the table and then, when performing the soft delete you can set the Name to NULL and record the old Name value in some other column. For SQL Server (since 2008), in order to allow multiple NULL values in a unique index you need to created what they call a filtered index where you explicitly say you want to ignore NULL values.
In other words, you schema would consist of something like this:
a UUID as primary key for the text record
change metadata would have a foreign key relation to text record via the UUID
a Name column with a non-clustered UNIQUE index
a DeletedName column that will store the Name when record is deleted
a Deleted bit column that can be NULL for non-deleted records and set to 1 for deleted
When you do a soft-delete, you would execute an atomic transaction that would:
set the DeletedName = Name
set Name = NULL (so as not to break the UNIQUE index)
mark record as deleted by setting Deleted = 1
There are other ways too but this one seems to be easily achievable based on what you already have.
In my opinion, you can do it in one of two ways:
Using the tables corresponding to the main table, which includes the action field, and using the delete , insert , update trigger of main tables for filling.
ArticlesTable(Id,Name) -> AuditArticlesTable(Id,Name,Action,User,ModifiedDate)
You can use the Filtered unique index (https://learn.microsoft.com/en-us/sql/relational-databases/indexes/create-filtered-indexes?view=sql-server-ver15) on the “Name” field to solving your issue on adding same name when exists another instance as deleted record
In my design, I have many tables which use FKs. The issue is because certain records will be deleted and re-added at various points of time as they are linked to specific project files, the references will be always be inaccurate if I rely on the traditional auto-incrementing ID (because each time they are re-added they will be given a new ID).
I previously asked a question (Sqlite - composite PK with two auto-incrementing values) as to whether I can create a composite auto-incrementing ID however it appears to not be possible as answered by the question I was linked.
The only automatic value I can think of that'll always be unique and never repeated is a full date value, down to the second - however the idea of using a date for the tables' IDs feels like bad design. So, if I instead place a full date field in every table and use these as the FK reference, am I looking at any potential issues down the line? And am I correct in thinking it would be more efficient to store it as integer rather than a text value?
Thanks for the help
Update
To clarify, I am not looking asking in regards to Primary Keys. The PK will be standard auto-incrementing ID. I am asking in regards to basing hundreds of FKs on dates.
Thank you for the replies below, the difficulty i'm having is I can't find a similar model to learn from. The end result is i'd like the application to use project files (like Word has their docx files) to import data into the database. Once a new project is loaded, the previous project's records are cleared but their data is preserved in the project file (the application's custom file format / a txt file) so they can be added once again. The FKs will all be project-based so they will only be referencing records that exist at the time in the database. For example, as it's a world-building application, let's say a user adds a subject type that would be relevant to any project (e.g. mathematics), due to the form it's entered on in the application, the record is given a_type number of 1, meaning it’s something that persists regardless of the project loaded. Another subject type however may be Demonology which only applies to the specific project loaded (e.g. a fantasy world). A school_subject junction table needs both of these in the same table to reference as the FK. So let’s say Demonology is the second record in the subjects type table, it has an auto-increment value of 2 - thus the junction table records 2 as it’s FK value. The issue is, before this project is re-opened again, the user may have added 10 more subject types that are universal and persist, thus next time the project’s subject type records and school_subject records are added back, Demonology is now given the ID of 11. However, the school_subject junction table is re-recreated with the same record having 2 as its value. This is why I’d like a FK which will always remain the same. I don’t want all projects to be present in the database, because I want users to be able to backup and duplicate individual projects as well know that even if the application is deleted, they can re-download and re-open their project files.
This is a bit long for a comment.
Something seems wrong with your design. When you delete a row in a table, there should be no foreign key references to that key. The entity is gone. Does not exist (as far as the database is concerned). Under most circumstances, you will get an error if you try to delete a row in one table where another row refers to that row using a foreign key reference.
When you insert a row into a table, the database becomes aware of that entity. There should not be references to it.
Hence, you have an unusual situation. It sounds like you have primary keys that represent something in the real world -- such as a social security number or vehicle identification number. If that is the case, you might want this id to be the primary key of the table.
Another option is soft deletion. Once one of these rows is inserted in the table, it cannot be deleted. However, you can set a flag that says that it is deleted. Then, foreign key references can stay to the "soft" deleted row.
So as apart of of a project I am doing I have been restricted on the editing of the Database from which I am building this application. I can not directly make changes to the Database from the SQL Command Prompt, all changes and new entries have to be made through Apex. Just clarifying this.
So I have a table TAXIUSER that has a primary key of TAXIUSERID. The primary key format for data currently in the Database is as follows:
UID0001
UID0002
and so on. Currently the user has to manually enter a new primary key each time they add an entry to the table. How through the Apex application solely (I can't use SQL Workshop either!) can I achieve something that would allow the user to not have to manually enter the primary key and preferably have it increment still in the above format.
EDIT: Sorry I should clarify I am using an Form on a Table with Report and version 11g.
You can do this by creating a read only item with a default value (or a computation) of
SELECT
'UID'||LPAD(MAX(SUBSTR(taxiuserid,4) + 1),4,0)
FROM
taxiuser;
Note that your prerequisites (no access to db) are forcing this to be a flawed approach:
Errors out with primary key constraint violation if 2 or more users are creating records at the same time. You could minimize the risk by calculating the new value on submit but then the user will not see the value when they're creating the user
Only works for the first 9999 users
Edit: If you use the computation after submit method, you will need to add condition to allow edits to made to existing entries. Otherwise the computation will attempt to change the primary key of that entry causing an error. To do this simply add a condition of Value in Expression 1 is Null, then specify the column from which the data is being entered into the table, like as follows:
P8_TAXIUSERID
This essentially means the computation will only ever attempt a value to TAXIUSERID when P8_TAXIUSERID (the column from which your are entering the data) contains a null value.
Right now I have all my mappings as hbm.xml. I want to switch dynamically the type of Id generator for certain entities from 'identity' to 'assigned' at runtime (application start).
This is because I need to support importing data from previous system and keep existing ids.
Is this possible? How?
The generator is part of the mappings, so you need to change the mappings before creating the session factory.
This is easy to do with Fluent or ConfORM. It's possible to change XML mappings before feeding them to the configuration, but it's cumbersome.
Just check for a configuration flag (that you'll change when starting the app), and call the appropriate generator.
It's not clear why you would need to keep existing id's. I think you should not be needing to keep existing id's. Maybe you need to keep alternate id's instead?
If the previous system has it's own database, then you:
1) Need another mapping for the other table in the other database
2) Copy the data to your existing database (with key identity)
Which means you will need new id's anyway.
Example: Suppose you want to copy a table of 'airlines' and the previous system uses the 'airline-code' as the primary key. You could use an integer as primary key in your new database and the airlinecode as your alternate key.