We use post-deployment scripts to maintain history of the data in the tables. I am trying to add a new column to the existing table through post deployment script. I have written a post-deployment script to add the new column and one more post deployment script to add data into the newly inserted column.I am trying to publish my database then I can see my alter table script before adding data but it throws an error 'Invalid column name 'NewlyAddedColumn'' My question can we alter the schema using post deployment scripts? I tried using Commit command after altering the table in the post deployment but still encountered the same error message. I am running the post-deployment script to add new column before accessing it to insert data.Could some one help me with this issue.
Yes you can. Right-click on the post deployment script, go to the properties and set Build = none. But why do you want to add column in the post script? Why don't you want to add it to the project?
Related
I have a need to create a SQL database and a table and Insert data into the table from another SQL database . And also to use this newly created database as a oledb source in another dataflow in the same SSIS package. The table and database name are fixed.
I tried using script task to create database and tables. But when I have to insert data , I am not able to give database name in the connection manager as the database is created only in runtime.
I have tried setting ValidExternalMetaData to false, but that doesnt seems to help as well.
Any idea or suggestions on how to accomplish this will be of great help. Thanks
I think you just need two things to make this work:
While developing the package, the database and table will need to exist.
Set DelayValidation to true on the connection manager and dataflow tasks in order to avoid failures with connection tests before they are created.
use a variable to hold the new table name create and populate the using the variable then use the variable name in the source object.
We need to pull data from Salesforce into Snowflake ODS Layer. For this we are using IICS.
As we need to replicate the Salesforce table into snowflake (as it is) we are trying to use the option of "Dynamic Schema Change Handling" in IICS where we are setting it to "Alter and Apply DDL changes". But, my job is trying to Alter the table and create all the fields from source table even if they are present in the target table.
All we want to do is "if source is adding a new column, we want to get that added automatically to the target and if source drops a column we should drop the same in target table".
Can some one please help, if there is an option to achieve this?
Thank you in advance!
Regards,
Edu
I'm loading data from a named external stage (S3) by using COPY INTO, and this S3 bucket keep all old files.
Here's what I want:
When a new file comes in, truncate the table and load the new file only, if there's no new file coming in, just keep the old data without truncation.
I understand that I can put option like FORCE = False to avoid loading old files again, but how do I only truncate the table when there's new file coming in?
I would likely do this a bit differently, since there isn't a way to truncate/delete records in the target table from the COPY command. This will be a multi-step process, but can be automated via Snowflake:
Create a transient table. For sake of description, I'll just call this STG_TABLE. You will also maintain your existing target table called TABLE.
Modify your COPY command to load to STG_TABLE.
Create a STREAM called STR_STG_TABLE over STG_TABLE.
Create a TASK called TSK_TABLE with the following statement
This statement will execute only if your COPY command actually loaded any new data.
CREATE OR REPLACE TASK TSG_TABLE
WAREHOUSE = warehouse_name
WHEN SYSTEM$STREAM_HAS_DATA('STR_STG_TABLE')
AS
INSERT OVERWRITE INTO TABLE (fields)
SELECT fields FROM STR_STG_TABLE;
The other benefit of using this method is that your transient table will have the full history of your files, which can be nice for debugging issues.
If we have added one new columns in a particular table and when we run the release pipeline in azure devops we need to execute an update query to update default value to the newly added column.
How can we do that?
How its possible using sql query file?
anybody can help??
Just use the SQL Server database deploy task
https://learn.microsoft.com/en-us/azure/devops/pipelines/targets/azure-sqldb?view=azure-devops&tabs=yaml
make sure your SQL script includes a check for if the column already exists or the job will fail
I have a custom table in MODX database set up and working, thanks to this article:
http://bobsguides.com/custom-db-tables.html
and now I need to add new column to this existing table. How can I do this the "MODX way"? Or do I have to create the component from scratch again?
You can manually add the new column to the database, then update your xml schema and map files to include the new column metadata. If you have a build script you could simply run it again after amending the schema to regenerate the map files.
I could be more specific if you paste in your existing schema and description of the column you want to add.
I believe MigxDB plugin (part of migx plugin) sets up a utility under manager page to just do that.
Install migx as instructed (you need to do an extra step to set it up so read the instruction)
load your modified schema in midx-package manager and do 'parse schema' and then 'add field'.
Make sure you have package name and pre-fix specified when loading your schema. modx forum has a dedicated section for migx if you need further clarification.