I am looking at ways to bring an Excel Spreadsheet, which uses lots of Visual Basic Access code - onto the Power Platform. The purpose of it to do Forecasting and Optimisation algotithms.
What is the best way to re-create this code?
I am thinking that we move all the data from the spreadsheet into Dataverse. It seems the only option to run the code is to put it into an Azure Function?
Is there a way to do this another way, through AI Builder or Power Automate? Could this be ran through Power Fx - or is Power Fx limited to basic instructions?
It seems limited in the way we can run code through Power Platform... any help or advice is appreciated.
Todd
I dont think a 1:1 copy paste action is possible (because of that VB access code).
Although Dataverse is a relational database it supports most common database functionalities. And if it is simple data (and not too much of it) you could alternatively use SharePoint lists. This is often simpler, and less expensive in licenses.
Also Power Automate can be used for automatically updating data in DV or Lists.
So if you manage to get you data into on of these two, the "Power Platform way" would be to expose it using Power BI.
It can be a overhvelming undertaking if you are not familliar with PP at all. And I would advise you to see if it is possible to slice it into smaller chuncks and try it out one small bit a a time.
Related
Power BI can connect to various data sources and run SELECT queries.
Is it possible to run also other queries (INSERT INTO, UPDATE...)?
Now I need it for a postgresql database, but could use also for others in the future.
No, you can't run directly INSERT/UPDATE queries from Power BI. This isn't the idea of the tool. If you find you need it, then probably there is a major flaw in your design, or you are not using the right tool for this job. But there are few ways to workaround this (again, I'm not saying that you SHOULD do it). Usually this is done in a combination with custom written Power App, embedded in your report in Power Apps visual. The idea is that the app will write to the database, and will refresh your report after that (if needed).
You can start here and I will recommend you to look at this in-depth session - Writing back data to PowerBI from your reports.
The answer is No if I am very straight forward. PBI is a analysis platform for data. There are probably some advance way to do that but, this is not logical or good idea to think about manipulating data from report or from any BI tools. You can search answers from different blog where the same questions asked. For more details, you can check below links-
help link 1
help link 2
I am working using Tableau and have to write down multiple different SQL each time, while making new data sources.
I have to save all changes on SQL for every data source.
Currently I would paste the SQL on notepad and save them on separate folder in my computer, along with description of the changes.
Is there any better way to do this?
Assuming you have permission to create objects in the database, begin by creating database views, As #Nick.McDermaid commented.
Then, instead of using Custom SQL data source in Tableau, just connect to the View as if it were a table.
If you need to track the changes to these SQL views of your data, you will need to learn how to use source control for the .sql files that can be scripted from within SQL Server Management Studio:
Your company or school may have a preferred source control system already in use, in which case you should use that. If they don't, or if you are learning at home, then Git and Subversion are popular open source choices.
There are many courses available on learning platforms like Coursera that will teach you how to learn how to use those systems.
I had similar problem as you.
We ended up writing the queries in SQL Editor SQL Work bench (https://www.sql-workbench.eu/), then managed the code history and performed code peer-review (logic, error check, etc) in team shared space (like confluence).
The reasons we did that is
1) SQL queries are much easy to write on Work Bench
2) Code review is a must! You will find through implementing a review process more mistakes than you could ever think about
3) The shared space is just really convenient as it is accessible by everyone, and all errors are documented. After sometimes you get a lot of visible knowledge accumulated.
I also totally agree with Nick as this is one step to a reporting solution. But developing a whole reporting server is heavy, costly and takes time. Unless management are really convinced of the importance of developing a reporting solution, you may have to get a workaround with queries and Tableau (at least that was the case for us)
A little late to the party, but I would suggest you simply version the tableau workbook. The contents of the workbook are XML, so perfect for versioning using file based tools (Dropbox, One Drive, etc.) or source control (git, etc.). The workbooks themselves are usually quite small, so just make sure to keep the extract data separate if you use it.
I want to have an access port for non-tech savvy individuals in which they could make reports of their own without needing to know SQL what-so-ever.
It would be best if I could create custom fields of myself, and then just let the users in the access port pick and choose whichever they like with a custom date range.
I've explored the options Google Data Studio offers, but it looks to me like it mostly puts an emphasis on data visualization.
In addition, my attempts to make custom queries with it were not successful, since the platform is rigid in terms of deciding which field is a metric and which is a dimension (and it does so inaccurately). This makes it hard to query reports as you normally would using BigQuery, which doesn't have these somewhat arbitrary limitations.
Perhaps I've misunderstood something about the platform due to my limited experience with it, but it looks like Data Studio isn't going to fit the bill for me.
EDIT: In addition, the platform should have a way of exporting said reports as CSV files, a feature that Data Studio doesn't have as far as I know.
It would be great to receive suggestions for a different platform which would better fit my needs, or even suggestions on how to make better use of Data Studio.
Have you looked at using a tool like redash (https://redash.io)? Assuming your GA360 data is in BigQuery you can connect redash to BQ. Then you can author queries and visualize.
You can also use the Google Could SDK to connect to BQ and run custom queries to generate new tables in BQ based on the GA360 session data. Then use redash, or any tool, to report/visualize.
Do there exist any (ideally free or open-source) tools for performing OLAP analyses on arbitrary tables in a relational database, without requiring any advance specification of dimensional hierarchies, cardinalities, or any other meta-information about the table beyond what can be extracted automatically from the table itself?
My inability to Google for anything like what I'm describing makes me suspect I'm using incorrect terminology and what I'm searching for isn't properly considered to be OLAP. If this is the case, what I specifically want is anything that would let technically unsophisticated users create cross-tab or contingency table aggregations using tables in a relational DB without needing to write elaborate SQL queries.
Or, in other words, I'd like something that mimics Excel's PivotTables on a larger scale. I appreciate that Excel does indeed generate extensive caches behind the scenes when you make a PivotTable, but it does this without the user having to explain to it which caches need creating. This is the functionality I'm trying to find elsewhere, if it exists.
The best options I know of are Excel and Access, but of course they are not open source. This space kinda got trampled in the explosion of interest in what is now called Business Intelligence and a lot of companies got bought by MS and others. It's pretty thin now as far as I can tell. I'll watch this thread though.
The most useful paradigm to attach to is I think spreadsheets and there's not much competition there any more. Google Docs spreadsheets can import csv etc. exported from databases, and there's a pivot chart available, but not much more.
The other place I've seen OLAP capabilities is in the Adobe Flex libraries to build on with ActionScript if you have any inclination in that direction. As usual, Adobe manages to get it about 90% right but doesn't quite provide a whole product.
icCube aims to setup an OLAP cube as simply as possible. It is not schema-agnostic, but I guess this is quite simple to define dimensions and facts from existing DB tables. Nevertheless, this could be not so "simple" depending on your tables - difficult to say without knowledge about them. I guess there's no generic easy solution ;-)
Then you can use Excel pivot table (amongst others) to access the cubes. Note as far as I know Excel does not do any caching neither aggregation when connecting to a cube. Indeed, it is generating all the required MDX requests to the cube.
Hope that helps.
Even with the advanced technologies and available database tools (even free alternatives) are available today, it seems that huge number of users are still very comfortable in using Excel IN EVERYTHING! That's why, as a database developer working as one of these users, I am forced to let them use Excel simply because they are very comfortable using it. Especially for the older people who seemed to never gonna let Excel go and embrace a new tool.
Currently, to make their experience as smooth as possible and at the same time, automated, I'm using a lot of database queries inside Excel be it view,SQL or stored procedures. Mostly on ad-hoc (but then became permanent) reports. My question is are there any hopes to improve this situation? I'm sure a lot of organizations are using this same method. Is it possible to completely replace this arrangement with something more logical and efficient both in data collection and reliability? I'm thinking about using Sharepoint. Am I on the right track?
I have also struggled with this problem in the past and can say that what worked for me was a two pronged approach.
Step 1 – Make a good alternative
It sounds like you have already done this, depending on the system there will always be some random report that someone needs to run to suit their “Business Need”. There is no way that you could cram all of these into your system as it would fill up with reports and the users would become snow blind.
Step 2 – User education
Show them the new way of making their own reports (Business objects SSRS whatever) and make sure they are comfortable with it. This is the hardest part as some people like their comfort blanket of excel and wont want to leave it. Give them some templates and some standard reports, maybe even pair develop one or two reports at their desk with them so they get the knack of it.
I will leave on a bit of a daily WTF, there was once this expert business manager who was an expert in business objects. She made reports left right and centre but she treated it like a giant version of excel and her work was littered with examples of this i.e. one report she wrote was to get the dealing totals for a year. No problem I hear you cry just do
SELECT SUM(DealAmount) where DealDate Between X and Y
Nope not our business expert, in here excel frame of mind this was too much like black magic so what she did was return a row for EVERY SINGLE DEAL done in that year and then aggregated it client side to give her a total. In I step and wow the users by reducing this 104mb report that took 17 minutes to run down to a 100kb report that ran in about 15 seconds.
I would go the other way around. And I mean by that, not making queries and database connections within Excel, but using some sort of Web Application to let users (through wizards) generate data they need, and export them to Excel to work.
That way you will have the following benefits:
No DB connections (and probably passwords) in your Excel files
No distribution problem of Excel files with new queries, views, etc.
Centralized approach to data retrieval
Excel for users used to it
Back in the day, I loved using Crystal Reports for ad-hoc reporting. I'm not sure about it's current status, as it seems that SAP has purchased the product: http://www.sap.com/solutions/sapbusinessobjects/sap-crystal-solutions/index.epx