search for Open source data governance platform The requirements include basic data aggregation, data standardization, thematic library, data catalog, super search, BI tools and other functions. Please recommend a few open source projects that I can combine and deliver.
i spent a lot of time to do it,but it is so difficulty, some can help me?
Related
Power BI can connect to various data sources and run SELECT queries.
Is it possible to run also other queries (INSERT INTO, UPDATE...)?
Now I need it for a postgresql database, but could use also for others in the future.
No, you can't run directly INSERT/UPDATE queries from Power BI. This isn't the idea of the tool. If you find you need it, then probably there is a major flaw in your design, or you are not using the right tool for this job. But there are few ways to workaround this (again, I'm not saying that you SHOULD do it). Usually this is done in a combination with custom written Power App, embedded in your report in Power Apps visual. The idea is that the app will write to the database, and will refresh your report after that (if needed).
You can start here and I will recommend you to look at this in-depth session - Writing back data to PowerBI from your reports.
The answer is No if I am very straight forward. PBI is a analysis platform for data. There are probably some advance way to do that but, this is not logical or good idea to think about manipulating data from report or from any BI tools. You can search answers from different blog where the same questions asked. For more details, you can check below links-
help link 1
help link 2
I want to have an access port for non-tech savvy individuals in which they could make reports of their own without needing to know SQL what-so-ever.
It would be best if I could create custom fields of myself, and then just let the users in the access port pick and choose whichever they like with a custom date range.
I've explored the options Google Data Studio offers, but it looks to me like it mostly puts an emphasis on data visualization.
In addition, my attempts to make custom queries with it were not successful, since the platform is rigid in terms of deciding which field is a metric and which is a dimension (and it does so inaccurately). This makes it hard to query reports as you normally would using BigQuery, which doesn't have these somewhat arbitrary limitations.
Perhaps I've misunderstood something about the platform due to my limited experience with it, but it looks like Data Studio isn't going to fit the bill for me.
EDIT: In addition, the platform should have a way of exporting said reports as CSV files, a feature that Data Studio doesn't have as far as I know.
It would be great to receive suggestions for a different platform which would better fit my needs, or even suggestions on how to make better use of Data Studio.
Have you looked at using a tool like redash (https://redash.io)? Assuming your GA360 data is in BigQuery you can connect redash to BQ. Then you can author queries and visualize.
You can also use the Google Could SDK to connect to BQ and run custom queries to generate new tables in BQ based on the GA360 session data. Then use redash, or any tool, to report/visualize.
Within an Informatica repository, how I might I generate a report to show, from a high level, how every field in every target was derived? For example I'd like to show:
Whether a field was "simply" passed through from a source
Whether a field was computed and/or transformed and/or aggregated, etc., based on one or more source fields and perhaps external factors
I know this is a vague question, but this is a question that I myself have been asked to answer.
Might a query of Informatica's tables yield this kind of information? I'm new both to Informatica and to this specific repository.
Thanks in advance!
Informatica provides repository database views for you to retrieve information from the repository. Please see the Informatica PowerCenter Repository Guide for more detailed information about the views. It may be possible to get that info what you are looking for from the views, but I think is not easy to get this kind of info. Repository views gives you e.g. general information about the source and target tables.
It will be difficult to achieve using the Repository. Look for Data Lineage available in PowerCenter Metadata Manager. If you don't have this available and would need to analyze Source to Target dependencies for a single workflow, you can try a tool I've created here: xmlAnalyzer It's a simple, online tool to chceck all mappings within an XML file and list all Source-Target dependencies. You can find out for example how many Source fields have impact on a given Target field. It will not show however what are the transformations along the way.
You will find following tool from informatica marketplace useful for this requriement.
https://community.informatica.com/solutions/xml_reporter_for_informatica
Team Foundation Server version-control of Web Applications
I feel let down :( ...
Checking-in and out .mdf files, branching and merge all work well in TFVC, however there is no data conflict resolution (conflict resolution for code is great!), only a choice is offered between whole source or target files.
I am not suggesting that data-tables are displayed side-by-side, because scripts in DAC/ 'database projects' can be compared for changes in table structure and data differences compared by stored procedures.
Some method must be commonly used, but apparently that MS expect the code and data versioning to be managed iteratively/ separately? (I would like to avoid trialling additional proprietary software like Red-Gate Source-Control.)
FULL version control is required as new feature branches will change the DB, but core/testing data needs to be retained.
So PLEASE!! Help me with pointers to straight-forward FULL version-control practices for web applications that include sql versioning with intelligent merge and roll-back capabilities.
Many thanks!
You could try Red Gate Deployment Manager to manage your deployments, which also comes with a free community edition. However, although not strictly mandatory we would recommend that you do this in conjunction with SQL Source Control, which would allow you to specify static data tables to put in version control. Although this is third party software, the database objects are saved as plain text .sql files, and not a proprietary format.
I have done a project for data migration and transformation on a clients database and I need to show them what exactly I have done and what the columns map to etc.
I cant find a tool to do the documentation for me ? If i have to do this manually it will take me days if not weeks.
The only documentation tool I am aware of is Pragmatic Work's BI Documenter. It builds out Windows help files (.chm) or HTML extracts. It's a clever product.
I was looking for a similar product 4ish years back and that was the only COTS product I had found. There was a dead project on... Codeplex I think. Otherwise, you can have fun enumerating through the objects in the package with the API. If you go that route, you can get some free rep by answering this question. Extracting SQL statements from a SSIS/DTSX package