I have several tabular cubes (.bim files) where I am trying to do some documentation.
I would like to find a way to automatically export for a given cube:
Column name (cube level)
Column name (source level)
Table name (cube level)
Table name (source level)
Measure with corresponding dax code.
Is there a way I can get all these values with a single export, rather than building it myself by constantly copy pasting?
I did it with installing SQLite 3 driver, setting it to point to metadata.sqlitedb from the instancename\OLAP\Data\DBName folder.
Then I have imported column, measure, etc. into the model as structured data source.
After refresh I can query all the data I need for documentation with DAX, can be saved manually with Dax Studio to excel or CSV, or automatically with Powershell SQLServer module to CSV.
Complicated first, but then, you have a self-documentating structure inside the model that can be refreshed when changed.
Example selfdoc
OR, if You are not afraid of some programming in C#,
create some classes for model/table/etc in JSON and read the bim file which is JSON with JsonConvert.DeserializeObject(content);
Related
I am creating an SSIS package which gets data dynamically, creates table in destination and inserts data there.
But I don't want to define meta data in script component so that when I select something from different source and different table then it simply insert that data into destination table.
So please suggest which component I should use.
to get every thing dynamically without knowing columns.
Dealing with tables only not with excel or another type.
Here is the snap shot of script component
You cannot do this in a dataflow at all. Dataflows have static meta data that must be defined at design time.
I recommend that you use BiML, which dynamically generates packages based on meta data at run time.
Otherwise you can do the entire thing (Source and Destination) in a single script Task (not dataflow component).
I am developing a migration tool and using Talend ETL tool (Free edition).
Challenges faced:-
is it possible to create a Talend job that uses dynamic schema every time it runs i.e. no hard-coded mappings in tMap component.
I want user to give a input CSV/Excel file and the job should create mappings on the basis of that input file. Is it possible in talend?
Any other free source ETL tool can also be helpful, or any sample job.
Yes, this can be done in Talend but if you do not wish to use a tMap then your table and file must match exactly. The way we have implemented it is for stage tables which are all datatype of varchar. This works when you are loading raw data into a stage table, and your validation is done after the load, prior to loading the stage data into a data warehouse.
Here is a summary of our method:
the filenames contain the table name so the process starts with a tFileList and parsing out the table name from the file name.
using tMSSQLColumnList obtain each column name, type, and length for the table (one way is to store it as an inline table in tFixedFlowInput)
run this thru a tSetDynamicSchema to produce your dynamic for that table
use a file input reference the dynamic schema.
load that into a MSSQLOutput again referencing the dynamic schema.
One more note on data types. It may work with data types than varchar, but our stage tables only have varchar and datetime. We had issues with datetime, so we filtered out those column types with a tMap.
Keep in mind, this is a summary to point you in the right direction, not a precise tutorial. But with this info in your hands, it can save you many hours of work while building your solution.
I have Multiple CSV files in Folder
Example :
Member.CSv
Leader.CSv
I need to load them in to Data base tables .
I have worked on it using ForEachLoop Container ,Data FlowTask, Excel Source and OLEDB Destination
we can do if by using Expressions and Precedence Constraints but how can I do using Script task if I have more than 10 files ..I got Stuck with this one
We have a similar issue, our solution is a mixture of the suggestions above.
We have a number of files types sent from our client on a daily basis.
These have a specific filename pattern (e.g. SalesTransaction20160218.csv, Product20160218.csv)
Each of these file types have a staging "landing" table of the structure you expect
We then have a .net script task that takes the filename pattern and loads that data into a landing table.
There are also various checks that are done within the csv parser - matching number of columns, some basic data validation, before loading into the landing table
We are not good enough .net programmers to be able to dynamically parse an unknown file structure, create SQL table and then load the data in. I expect it is feasible, after all, that is what the SSIS Import/Export Wizard does (with some manual intervention)
As an alternative to this (the process is quite delicate), we are experimenting with a HDFS data landing area, then it allows us to use analytic tools like R to parse the data within HDFS. After that utilising PIG to load the data into SQL.
I have a bunch of different xml souces (over 100), and I am trying to pipe them all into an OLE DB via SSIS. I started out trying to use the Union All function, but since not all of the data that goes into each row of the database is of the same type, some of the values are entered in as "ignore" by SSIS, and come out as null. Each xml source has multiple outputs as well, so I am trying to avoid manually pointing each node of each xml source at a different db source, and then combining them. I know that you can change the input type of the data in the xml source under the advanced options, but there doesn't appear to be a way to default everything to one data type (string would work for me). Is there a way to use the union all feature to work for me, or a better way to do this? Thanks in advance.
Does your server have access to folder that contains xml files?
If it does, using SSIS for that is not nesessary. Just use OPENROWSET
Also you can make stored procedure that selects data from all xml sources and union all results. This SP you can use in your SSIS package.
Sorry if the question isn't phrased very well but I'm new to SSAS and don't know the correct terms.
I have changed the name of a table and its columns. I am using said table as a dimension for my cube, so now the cube won't process. Presumably I need to make updates in the analysis server to reflect changes to the source database?
I have no idea where to start - any help gratefully received.
Thanks
Phil
Before going into the details of how to amend the cube, have you considered creating a view with the same name as the old table which maps the new column names to the old?
The cube processing process should pick this up transparently.
EDIT
There are quite a lot of variations on how to amend SSAS - it depends on your local set-up.
If your cube definition is held in source control (which it should ideally be), you need to check the cube definition out and amend it from there.
If your definition exists only on the server you need to open it from the server:
Open the Business Intelligence
Development Studio (BIDS) -
typically on the Windows start menu
under Programs > Microsoft SQL
Server 2005.
Go to File > Open > Analysis Services Database
Select your server/database and click OK.
Once you have the project open in BIDS, you can amend the Data Source View to switch to the new table.
These instructions are based on the principle that it's going to be easier to alias the new table to look like the old in the DSV, since this means fewer changes within the cube definition.
Open the Data Source View from the Solution Explorer - there should be only one.
Locate the table you need to change in the DSV
Right-click on the table and select Replace Table > With New Named Query
Replace the existing query with a query from the new table with the new columns aliased with the new names:
SELECT ~new column name~ AS ~old column name~
FROM ~new_table~
Once the new query has been set, deploy the changes:
If you use source control, check in and deploy the project to the target server.
If you opened the cube definition from the server, select File > Save All
Finally, re-process the cube.