We have a slowly changing dimension ETL package, which reads data from Task table to update DimTask table. The thing is that, we added a new column 'Category' into Task table, and want DimTask to slowly change on it (that is, once the value of 'Category' of one TaskID is changed in Task table, we want to add a new row in DimTask table to record this new value with new start and end date).
So we inert 'Category' into both Task and DimTask table, then we added the 'Category' in advanced editor of the ETL package, as well as the OLE DB Source and Insert Destination. The Error here is that, the advanced editor says 'There must be at least one column of Fixed, Changing, or Historical type on the input of a Slowly Changing Dimension transform.'
We are not sure why this appears, does this mean we have to use the Slowly changing dimension Wizard to go through the process (like choose primary key, which columns are historical) all over again each time we want to update the slowly changing dimension?
Is there any way we could only add this new column? Because we have hundreds of other columns in the table and it would costs lots of time to go through the Wizard again.
Thanks a lot for your help!
Oh we found that the column type of 'Input columns' under slowly changing dimension's advanced editor's 'Input and Output Properties' would be automatically deleted when adding a new column. Once we fill the Column types (especially the Key type), the ETL starts to work.
enter image description here
Related
I have imported data table to my cytoscape map for the following continuous mapping accorging to values. After some time, I have imported another data table and then deleted the previous one. The original data are not present in either the node or edge table but I still can see the names of the original columns when selecting column for continuous mapping i. g. for size or colour. Moreover, this warning appears: "The current table does not have the selected column. Please select another column." Do you know how to solve this so that I no longer see the names when these columns are not even in the table?
I would be grateful for any ideas.
I have tried to delete the data table and import the data again but it didn't help. I have also tried to clone current network and import data to the new map, but the old names are still present in the column selection.
Do you know how to solve this so that I no longer see the names when these columns are not even in the table?
I would be grateful for any ideas.
A couple points and then a couple suggestions:
The style is going to "remember" the selected column name, even if you delete it since it doesn't know what to change it to. It will add a warning icon mentioning that the column is missing. You have to choose a new column (or reload the missing column) to address that issue. [I think you know this already, but just stating for completeness :)]
The pulldown list of column names should be updated when you delete table columns. This does indeed sound like a bug.
Are you running the latest Cytoscape 3.9.1?
Have you tried selecting another style and then returning back to this style? That might "refresh" the column name list.
Have you tried saving/restoring the session?
I have a dataview grid bound to a datasource at run time. The datasource is filled from an access database via a DataAdapter. The data fills and displays correctly, and updates to existing rows seem to work OK but I have two problems:
When I type something in a new row and then press return or switch to a different row, I want the DataAdapter to add that row then and there to the database so I can retrieve the Autonumber index of the new record from Access and use that to add an associated record in a different table (Entries, a many to many linking table). This isn't happening. In the RowLeave event I have adapter.Update(dsSentences) and then I check for the new row, but the RowCount doesn't reflect its presence even though the newly added data is visible in the grid, and the adapter.Update doesn't seem to have triggered the Insert query that I specified in the DataAdapter. So nothing is added.
(edit: OK, so the new row has not yet been added when this event is fired. Which event should I then use to commit the data and retrieve the Autonumber primary key for my new record? I've tried UserAddedRow but that one fires before you've entered any data into the new row.)
THe second problem is that I need to update the data independently and then have the grid reflect those changes. How do I do that? Is there some call that will force the grid to get the updated data from the DataAdapter via the Dataset? Any help would be much appreciated. I'm almost ready to dtop the whole idea of binding data and do it all through code, Data binfing is supposed to save time but I'm finding it labyrinthine and unpredictable.
FWIW here's the query I'm using to fill the grid:
PARAMETERS nIdCollection Long;
SELECT tblSentences.IdSentence, tblSentences.SentenceText, tblSentences.SentenceParsed, Not IsNull([tblSentences]![SentenceParsed]) AS HasParsed, Entries.IdEntry
FROM tblSentences INNER JOIN Entries ON tblSentences.IdSentence = Entries.IdSentence
WHERE (((Entries.IdCollection)=[nIdCollection]))
ORDER BY Entries.SortValue;
As you can see, it requires a record in Entries. After I've entered a new record in tblSentences, before there are any entries the IdEntry will be null assuming it shows up at all. That's why I need to intercept directly after the Insert, add the record to Entries and requery to keep everything in order. You could do it all in an SQL stored procedure but I have to use Access.
Edit: After a lot of googling I've come to the conclusion that what I'm trying to do = add a record to a table through an additional INSERT query apart from the one handled by the DataAdapter, every time a new row is added - simply can't be done if you are using data binding. I am going to have to delete all my code and start from scratch populating the grid through code (unbound). I think it's the only way to do what I want. I will leave this here as a warning to anyone else not to make my mistake of trying to use Data binding when your data is coming from more than one table. Bad mistake.
A have a volunteer timesheet data entry system which allows the volunteers to enter the times they have spent on various activities. I used the VB.net Designer to create the system (OK, I know now that that was not a good move!) so please don't ask me to show my code, most of it is generated by the Designer. My problem is this:
Each new record is assigned a negative number as a primary key when it is entered which is the way a dgv works with Access Automumber keys. I am executing the following statements in the RowValidating event when the row is valid.
a_dgv.EndEdit()
a_dgv.CommitEdit(DataGridViewDataErrorContexts.Commit)
Me.TimeSheets2BindingSource.EndEdit()
Me.TableAdapterManager.UpdateAll(Me.MembershipDataSet)
This code does not update the primary key value on the dgv although it does so in the Access table. If a user then attempts to delete or alter a record he has earlier added in the same session the update fails with a concurrency error. The only answer if have found to this problem is to refill the whole table. This is obviously not a desirable solution. Does anyone have a proven tested one?
I should probably mention that my table has two databound comboboxes
I was under the impression that a datagrid that is the result of a dataset from say Access does not show the PK values as -1, -2, -3.
If you created the disconnected dataset (or datatable) in code from a fill (pull data from Access), then each row normally does not show the PK.
However, regardless of the above, assuming you entered 5 rows, and now need to see the PK values?
You will during data entry in the grid should see this:
In above, I have added two rows. Your save code is somewhat like this:
tblHotels = DataGridView1.DataSource
rstDataReader.Update(tblHotels)
tblHotels.AcceptChanges()
That will send the data back to SQL server (or Access), and the autonumber PK 'ids are then generated. However, such changes are NOT pulled back into the dataset/datatable. In other words, the PK id's are generated in the database, but UNLESS you re-pull the data, you are not going to see the PK values.
You WILL have to re-pull the data. However, you can keep the current position of the grid, and re-fill the data like this:
rstDataReader.Update(tblHotels)
tblHotels.AcceptChanges()
Dim MyTop As Integer = DataGridView1.FirstDisplayedScrollingRowIndex
tblHotels.Clear()
rstDataReader.Fill(tblHotels)
DataGridView1.FirstDisplayedScrollingRowIndex = MyTop
And then you should see this:
The other way would be to send + update each row as you edit data, and then pull the PK, but obvious then you not be able to update the all your grid changes with a SAVE button, and thus of course no un-do ability.
I find the above that re-positions the top of the grid does not flicker. On the other hand, I suppose this could/would depend on how large the data set is (but then again, loading up a grid with too many rows is less then ideal).
So, as far as I can tell, you have to re-pull the dataset/datatable to get the new generated PK id's, or you have to save + pull for each row you edit. For a gridview with even several 100 rows, I don't see any flicker with the above code.
I have a table input and I need to add the calculation to it i.e. add a new column. I have tried:
to do the calculation and then, feed back. Obviously, it stuck the new data to the old data.
to do the calculation and then feed back but truncate the table. As the process got stuck at some point, I assume what happens is that I was truncating the table while the data was still getting extracted from it.
to use stream lookup and then, feed back. Of course, it also stuck the data on the top of the existing data.
to use stream lookup where I pull the data from the table input, do the calculation, at the same time, pull the data from the same table and do a lookup based on the unique combination of date and id. And use the 'Update' step.
As it is has been running for a while, I am positive it is not the option but I exhausted my options.
It's seems that you need to update the table where your data came from with this new field. Use the Update step with fields A and B as keys.
actully once you connect the hope, result of 1st step is automatically carried forward to the next step. so let's say you have table input step and then you add calculator where you are creating 3rd column. after writing logic right click on calculator step and click on preview you will get the result with all 3 columns
I'd say your issue is not ONLY in Pentaho implementation, there are somethings you can do before reaching Data Staging in Pentaho.
'Workin Hard' is correct when he says you shouldn't use the same table, but instead leave the input untouched, and just upload / insert the new values into a new table, doesn't have to be a new table EVERYTIME, but instead of truncating the original, you truncate the staging table (output table).
How many 'new columns' will you need ? Will every iteration of this run create a new column in the output ? Or you will always have a 'C' Column which is always A+B or some other calculation ? I'm sorry but this isn't clear. If the case is the later, you don't need Pentaho for transformations, Updating 'C' Column with a math or function considering A+B, this can be done directly in most relational DBMS with a simple UPDATE clause. Yes, it can be done in Pentaho, but you're putting a lot of overhead and processing time.
I've made a dataset using the dataset designer, and I'm trying to add a column to reflect changes made to the database (added a column, nothing fancy). Is there a way to 'refresh' the dataset schema from the datasource without deleting my adapter (and all the methods and queries I've created)?
I know its been a while since you posted but as I was having the same problem and figured out how to do this I reckoned I'll post the solution that worked for me.
Right click on the dataset object you want to update (on the strip at the bottom of your viewpane)
Select "Edit in Dataset Designer"
in the dataset designer, right click on the header of the table you want to add a column to
select configure... this will bring up the sql statement that is used to draw values into the dataset for this table
Edit the sql to include the column you want to include in your dataset's table and click finish i.e. in the select statement, include your columns name in the list
close the dataset designer then go to any controls (in my case its a datagridview), click on the tasks arrow (top right hand corner next to the handle) and select add column
select the newly created column from the list of databound columns and click "add"
select "edit columns" from the task menu
move the column to the correct position (it will always be placed as the last column in your grid and you may not want it to be the last column)
voila, I know its hardly snappy but it beats the hell out of deleting the dataset and then fixing up all the coding errors that come up... also after doing it a few times it'll be like second nature (I hope)
regards
p.s. am working in VS2010
Had to just delete the adapter and the table. It's rather annoying but I guess there really isn't a way around it. Maybe in VS2010 or later versions of .net.