This is more of a request for a quick explanation of the sequences used to generate ID from the Spring Batch tables that store Job and Step information.
I've ran the below sequences in DB2 for Spring Boot + Batch application:
CREATE SEQUENCE AR_REPORT_ETL_STEP_EXECUTION_SEQ AS BIGINT MAXVALUE 9223372036854775807 NO CYCLE;
CREATE SEQUENCE AR_REPORT_ETL_JOB_EXECUTION_SEQ AS BIGINT MAXVALUE 9223372036854775807 NO CYCLE;
CREATE SEQUENCE AR_REPORT_ETL_JOB_SEQ AS BIGINT MAXVALUE 9223372036854775807 NO CYCLE;
When the Spring Batch job is running, each ID field is being incremented by 20 on each new record. Though this isn't a major issue, it's still slightly confusing as to why.
I had removed the sequences and added them again with INCREMENT BY 1. This is now incremented every second record by 1 and the other record by 20.
Any tips or explanation would be a great learning opportunity.
For performance reasons, Db2 for Linux Unix Windows by default will preallocate 20 numbers of a sequence and keep them in memory for faster access.
If you don't want that caching behaviour and can tolerate the overhead of allocating without caching, then you can use the NO CACHE option when defining the sequence. But be aware that without caching, Db2 must do synchronous transaction-log-write for each number to be allocated from the sequence, which is usually undesirable in high frequency insert situations.
Remember to explicitly activate the database (i.e do not depend on auto-activation), as unused pre-allocated cached sequence numbers get discarded when the database deactivates.
Example no cache syntax:
CREATE SEQUENCE AR_REPORT_ETL_STEP_EXECUTION_SEQ
AS BIGINT MAXVALUE 9223372036854775807 NO CACHE NO CYCLE;
You can read more details in the documentation.
Related
I am using SQL Server's Sequence to generate running number for my document number on my "doc_no" column. A quick check revealed my table's Identity column number to skip is due to server restart and one way to prevent that is to disable the cache (on my local machine). What about sequence? Is it the same way for Sequence? Is there any way to see the current sequence number in the Sequence itself?
CREATE SEQUENCE [dbo].[PB_SEQ_DOCNO]
AS [int]
START WITH 1
INCREMENT BY 1
MINVALUE -2147483648
MAXVALUE 2147483647
CACHE
GO
The CACHE keyword (without an explicit cache size) uses the default server cache size - which is probably 1000 (just like for the IDENTITY columns). So SQL Server in the background caches the next 1000 sequence numbers. If your server crashes or unexpectedly restarts, those cached sequence numbers are "lost" (as in your case here, obviously).
You can use a smaller cache size (by specifying e.g. CACHE 50) or you can even turn this off altogether (NO CACHE) - benefit is not/less gaps, drawbacks is slower performance when dishing out the sequence numbers.
I have a sequence having a huge difference of value around 2000 while using Merge..I initially had a cache of 20 while defining my sequence definition..later I redefined my sequence to
CREATE SEQUENCE "DMODS"."ORDER_SEQ"
MINVALUE 1
MAXVALUE 9999999999
INCREMENT 1
START WITH 1
NOCACHE
ORDER
NOCYCLE
I noticed that the value of the sequence is now varying by 2000 on an average for inserts/updates happening on the same day..can you please throw some light on this?
to quote Tom Kyte (https://asktom.oracle.com/pls/asktom/f?p=100:11:0::::P11_QUESTION_ID:530735152441)
Sequences will never generate a gap free sequence of numbers.
If someone calls seq.nextval and doesn't insert the value somewhere (or does insert it
and later rolls back their transaction) that number is GONE.
Sequences are database objects and may be aged out of the shared pool like everything
else -- it doesn't take a database shutdown to produce a gap.
You should never count on a sequence generating anything even close to a gap free
sequence of numbers. They are a high speed, extremely scalable multi-user way to
generate surrogate keys for a table.
basically, it's not a problem, they do what they do. If you need a continuous sequence of numbers for your primary key you need to generate it yourself (max+1)
Dave
First I run all these in java with jdbc driver...
Here I define a table:
create table HistoryCCP(
ID NUMBER(6) NOT NULL,
SCRIPT VARCHAR2(1000) NOT NULL
)
Here I define a sequence:
CREATE SEQUENCE SYSTEM.HistoryId
MINVALUE 1
MAXVALUE 1000000
INCREMENT BY 1
START WITH 1
NOORDER
NOCYCLE
Now I insert to table by using this here:
insert into HistoryCCP
values (SYSTEM.HistoryId.nextval ,'HELLOOOO ')
Whenever I close the program and run it again and try to insert, it increments it by ten!
And when I defined sequence like this:
CREATE SEQUENCE SYSTEM.HistoryId
MINVALUE 1
MAXVALUE 1000000
INCREMENT BY 1
START WITH 1
CACHE 100 -- added cache parameter
NOORDER
NOCYCLE
It increase it by 100!
Do you know why it behaves like this and how to increment it by 1?
Never rely on sequences for gap free numbering.
The cache value is the number of sequence values that are held in memory by the database server to avoid the need to keep updating it's internal $SEQ table with the most recently used value. If you reduce the cache value then you increase the rate at which the $SEQ table has to be modified, which slows the system.
Cached values can be aged out, and are lost on system restart, and values are not reused if a transaction gets rolled back.
The presence of gaps should not be a problem for you -- if it is then you'll need to use something other than a sequence to generate the numbers, and doing so will serialise inserts to that table.
Try to use NOCACHE options for sequence.
http://docs.oracle.com/cd/B28359_01/server.111/b28310/views002.htm
NOCACHE would work, but also would be a bad idea under for many reasons, and a total nonsense if you plan to bring your application on a Oracle RAC.
Oracle Sequences are for (internal) unique ID, not for strictly progressive number imposed by requirements. As example, let's say that using a sequence for generating the classical "protocol number" is a common flaw of many financial accounting software: looks easy when beginning but when the project grows it kills you.
I want to update the cache size of an existing sequence and i want to describe a sequence in oracle like table . how to do it ?
and what are all the drawbacks of increasing the cache value of an sequence
Alter sequence seq_name cache 20;
See the docs.
To get the ddl you may use the dbms_metadata package, wich can be used for any object:
select dbms_metadata.get_ddl('SEQUENCE','SEQ_NAME') from dual;
Increasing the cache size is useful when you have massive fetches from sequence. Increasing it has no drawback considering the fact that you use them.
But if you generate 1 milion values at a time and you use only 10, maybe is not a good ideea, because 999990 values are lost. Next session will generate another 1000000 values.
I think the engine works to generate them and allocate values for your session.
For example in my opinion, a cache 10 times less than you normally use in a session is ok.
UPDATE: Adding David Aldridge's comment:
The usefullness of a large cache is really related to the rate at
which it is used in general, so not just for large selects but for
systems with many session all using one value at a time. As
background, the performance problem with a small cache is caused by
the need for the SEQ$ system table to be modified when the cache is
exhausted. It's a small operation but obviously you don't want to be
doing it 100 times a second.
So, increasing the cache you'll have fewer concurent sessions on the same resource.
I have created a table in APEX that has a PK that is incremented by a SQL sequence:
CREATE SEQUENCE seq_increment
MINVALUE 1
START WITH 880
INCREMENT BY 1
CACHE 10
This seems to work perfectly. The issue is that sometimes, usually when I get on in the morning and run a process to enter a new row, it skips a bunch of numbers. I only care because these numbers are being used as the ID# of documents in my company and losing/skipping blocks of numbers is not going to be acceptable when this tool goes live.
It does seem to jump to the next '10' number. i.e. yesterday my last test assigned 883 and this morning it assigned 890 as the next number. Looking at my code for creation of the sequence I notice that I have set it up to cache 10 values so that it will process quicker. Is it possible that this cache is getting dumped over night and that it is pulling 890 because it had 880-889 in cache and it was dumped?
Are there other potential causes and solutions?
Sequences will not and can not generate gap-free values. So you'd expect that numbers will occasionally be skipped. That's perfectly normal when you're using sequences.
As you've surmised, the most likely scenario is that the sequence cache is aging out of the shared pool overnight when the APEX application isn't being used. You can reduce the frequency of gaps by declaring your sequence NOCACHE but that will decrease performance and it will not eliminate gaps it will just make them less frequent.
Oracle sequences are never guaranteed to be contiguous. If you need an absolutely contiguous set of values, you'll need to implement a custom solution.
Odds are that CACHE 10 is why you're losing numbers in this case. The cache value is how many sequence values are stored in memory for future use. Rebooting will clear the cache and cause 10 new values to be retrieved. Similarly, if the sequence is not used for long enough, the current set of values may be flushed out of the shared pool, also causing a new set of values to be retrieved.
This is clearly not the case in your instance, but sequence numbers can also be lost due to rollbacks. A rolled back transaction involving one or more sequences discards the sequence value(s).
Some sequence numbers have been aged out of one of the in-memory structures (shared pool I think?). This is expected behaviour for sequences. The only guarantee that you have is that they are unique. If you need to present gap-free sequences you'll have to do this at reporting time using e.g. rownum pseudo-column. It is made this way deliberately otherwise you would have to serialise all inserts i.e. lock table. And even that wouldn't work properly if an insert was rolled back!