I am quite new to Redis before posting this question i tried couple of option in Redis such as HASH, SET, MULTI SET to create table like below but i could not make it
I need the below table to be inserted into Redis and insertedtime column should be with Latest timestamp as first row for every Processnumber
Processnumber | insertedtime
------------------+--------------------------
370905502434097 | 2017-05-11 07:11:10+0000
370905502434097 | 2017-05-11 07:09:09+0000
370905502434097 | 2017-05-11 07:06:55+0000
370905502434097 | 2017-05-11 07:03:55+0000
4024007195422711 | 2017-05-11 07:11:10+0000
4024007195422711 | 2017-05-11 07:09:09+0000
4024007195422711 | 2017-05-11 07:06:55+0000
4024007195422711 | 2017-05-11 07:03:55+0000
5591980872085425 | 2017-05-11 07:11:10+0000
5591980872085425 | 2017-05-11 07:09:09+0000
5591980872085425 | 2017-05-11 07:06:55+0000
5591980872085425 | 2017-05-11 07:03:55+0000
5591980872085425 | 2017-05-11 06:46:35+0000
Any Help or Suggestion is Highly Appreciated
Thanks in Advance!!!...
With Redis you first need to know what type of queries you'll be running on the data in order to know how to "insert" it.
Looking at your table, it appears like the process number is a unique identifier and the two main questions that are askable are:
Given a processnumber, give me back a timestamp
Given a datestamp range, give me back the processnumbers in it
For this type of operations, a Sorted Set would do nicely. Read more about the ZADD and its friends.
Related
I have three relational postgres tables (timescale hypertables) and need to get my data into a CSV file, but I am struggling to get it to the format I want. I am using django as frameworks, but I need to solve this with raw SQL.
Imagine I have 2 tables: drinks and consumption_data.
The drinks table looks like this:
name | fieldx | fieldy
---------+--------+------
test-0 | |
test-1 | |
test-2 | |
The consumption_data table looks like this:
time | drink_id | consumption
------------------------+-------------+------------------------------
2018-12-15 00:00:00+00 | 2 | 123
2018-12-15 00:01:00+00 | 2 | 122
2018-12-15 00:02:00+00 | 2 | 125
My target table should join these two tables and give me all consumption data with the drink names back.
time | test-0 | test-1 | test-2
------------------------+-------------+---------+-------
2018-12-15 00:00:00+00 | 123 | 123 | 22
2018-12-15 00:01:00+00 | 334 | 122 | 32
2018-12-15 00:02:00+00 | 204 | 125 | 24
I do have all the drink-ids and all the names, but those are hundreds or thousands.
I tried this by first querying the consumption data for a single drink and renaming the column: SELECT time, drink_id, "consumption" AS test-0 FROM heatflowweb_timeseriestestperformance WHERE drink_id = 1;
Result:
time | test-0 |
------------------------+-------------+
2018-12-15 00:00:00+00 | 123 |
2018-12-15 00:01:00+00 | 334 |
2018-12-15 00:02:00+00 | 204 |
But now, I would have to add hundreds of columns and I am not sure how to do this. With UNION? But I don't want to write thousands of union statements...
Maybe there is an easier way to achieve what I want? I am not an SQL expert, so what I need could be super easy to achieve or also impossible... Thanks in advance for any help, really appreciated.
I am looking for a recommended solution to solve problems like the one below.
Preferably with PostgreSQL, but any other SQL based solution would help.
So there is a table like this:
day | object_id | property_01 | ...
2022-01-24 | object_01 | A | ...
2022-01-23 | object_01 | A | ...
2022-01-22 | object_01 | A | ...
2022-01-21 | object_01 | B | ...
2022-01-20 | object_01 | B | ...
2022-01-19 | object_01 | A | ...
2022-01-18 | object_01 | A | ...
2022-01-17 | object_01 | B | ...
2022-01-16 | object_01 | A | ...
The base table is a daily "backup", so normally there are no gaps in the line of days.
There will be always a row since the birth of an object till it is deleted from the main DB.
What I need is to tell is that if an object is in "A" state at the day of query, how long is this object in it.
In the example the right ansver is since 2022-01-22, so a simple MIN() does not give the right ansver.
Also I do not need to tell all of the intervals in the "A" state, just how long the current one keeps.
What I tried:
I. Select all the object_id's that are in "A" state at a given date.
II: Select all the rows based on the object_id's.
Now I am trying to make a recursive query, but if there is some better solution, it would be a great help.
Thanks a lot!
I’m interested to know if someone here has ever come across a situation where the source is not always unique when dealing with snapshots in DBT.
I have a data lake where data arrives on an append only basis. Every time the source is updated, a new recorded is created on the respective table in the data lake.
By the time the DBT solution is ran, my source could have more than 1 row with the unique id as the data has changed more than once since the last run.
Ideally, I’d like to update the respective dbt_valid_to columns from the snapshot table with the earliest updated_at record from the source and subsequently add the new records to the snapshot table making the latest updated_at record the current one.
I know how to achieve this using window functions but not sure how to handle such situation with dbt.
I wonder if anybody has faced this same issue before.
Snapshot Table
| **id** | **some_attribute** | **valid_from** | **valid_to** |
| 123 | ABCD | 2021-01-01 00:00:00 | 2021-06-30 00:00:00 |
| 123 | ZABC | 2021-06-30 00:00:00 | null |
Source Table
|**id**|**some_attribute**| **updated_at** |
| 123 | ABCD | 2021-01-01 00:00:00 |-> already been loaded to snapshot
| 123 | ZABC | 2021-06-30 00:00:00 |-> already been loaded to snapshot
-------------------------------------------
| 123 | ZZAB | 2021-11-21 00:10:00 |
| 123 | FXAB | 2021-11-21 15:11:00 |
Snapshot Desired Result
| **id** | **some_attribute** | **valid_from** | **valid_to** |
| 123 | ABCD | 2021-01-01 00:00:00 | 2021-06-30 00:00:00 |
| 123 | ZABC | 2021-06-30 00:00:00 | 2021-11-21 00:10:00 |
| 123 | ZZAB | 2021-11-21 00:10:00 | 2021-11-21 15:11:00 |
| 123 | FXAB | 2021-11-21 15:11:00 | null |
Standard snapshots operate under the assumption that the source table we are snapshotting are being changed without storing history. This is opposed to the behaviour we have here (basically the source table we are snapshotting is nothing more than an append only log of events) - which means that we may get away with simply using a boring old incremental model to achieve the same SCD2 outcome that snapshots give us.
I have some sample code here where I did just that that may be of some help https://gist.github.com/jeremyyeo/3a23f3fbcb72f10a17fc4d31b8a47854
I agree it would be very convenient if dbt snapshots had a strategy that could involve deduplication, but it isn’t supported today.
The easiest work around would be a stage view downstream of the source that has the window function you describe. Then you snapshot that view.
However, I do see potential for a new snapshot strategy that handles append only sources. Perhaps you’d like to peruse the dbt Snapshot docs and strategies source code on existing strategies to see if you’d like to make a new one!
Is it possible with cl_gui_alv_grid to make two columns with the same header?
Suppose I want to display data like this :
| Tuesday | Wednesday | Thursday |
|---------------|---------------|---------------|
| Po | Delivery | Po | Delivery | Po | Delivery |
|----|----------|----|----------|----|----------|
| 7 | 245.00 | 4 | 309.00 | 12 | 774.00 |
| 4 | 105.00 | 2 | 88.00 | 3 | 160.00 |
| 10 | 760.00 | 5 | 291.00 | 20 | 1836.00 |
...
For this I think about two solutions, but I don't know if it possible.
First solution : Make two levels of field catalog, in the first one three columns, and in the second 6 columns.
Second : Make field catalog with 3 columns, and concatenate two values under each column.
Thanks.
There is a strange workaround on a german site, which deals with inheriting from alv_grid in order to override some crucial methods, to allow it, to merge cells, the source is a well known and appreciated german abap page, but, as it says, it is in german. Let us hope, any translator engine can translate this for You in a proper way, but as it looks like, this could be a step in the right direction.... but as it seems, You should fix all columns for that ( or at least those with merged cells ).
Please refer to this and tell me, if it helped:
Merge cells of alv-grid
I've got problem with SQL.
Here is my code:
SELECT Miss.Name, Miss.Surname, Master.Name, Master.Surname,
COUNT(Date.Id_date) AS [Dates_together]
FROM Miss, Master, Date
WHERE Date.Id_miss = Miss.Id_miss AND Date.Id_master = Master.Id_master
GROUP BY Miss.Name, Miss.Surname, Master.Name, Master.Surname
ORDER BY [Dates_together] DESC
and I've got the result:
Dorothy | Mills | James | Jackson | 28
Dorothy | Mills | Kayne | West | 28
Emily | Walters | James | Jackson | 13
Emily | Walters | Tom | Marvel | 12
Sunny | Sunday | Kayne | West | 9
and I really do not know what to change to have a result like this:
Dorothy | Mills | James | Jackson | 28
Emily | Walters | Tom | Marvel | 12
Sunny | Sunday | Kayne | West | 9
Because I don't want to to have duplicated names of master or miss in a result... :(
Can anyone help me?
It looks like your result set is correct, as you are getting the appropriate distinct combinations.
The "duplicates" are accurate, because you are querying the combinations of the Miss and Master records, not the Miss and Master records themselves. For instance, in your second result set, it doesn't capture the fact that Dorothy Mills dated Kayne West 28 times.
You don't mention which database you're working with, but if I have this correctly you're trying to determine how many times a given couple have been on a date?
I think you need to ask your self what happens if you have two people, of either sex, that share the same combination of christian and surname...
Start off with :
Select idMaster, idMiss, count(*) as datecount from [Date] group by idMaster, idDate
From there, you need to simply need to add their names to the results...
Should get you started on the right track...