Create a dynamic list of tables used by all the dashboards on the server - qlikview

Is it possible to create something which extracts all the database/table names being used by multiple dashboards? We have a lot of old and new dashboards and we wanted to extract which tables are being used by the dashboards.

You can give this a try
https://help.qlik.com/en-US/governance-dashboard/Content/QV_GovDashboard/What.htm
It has a data lineage tab that can show you some stuff and I imagine help you to build your own custom solution

Im not sure that this is possible. Or at least not easy to achieve. In order to extract such info:
loop through all apps and extract the script
parse the Qlik script and find/extract all sql statements
parse these statements and extract the database/tables
Extracting the scripts is the easiest part here.
Isolating the sql statements might be doable but then parsing the sql will have to be done by something.
There is no complete Qlik script parser so potentially using string manipulations to extract the sql statements (imo depends on the sql complexity). (I havent lost hope to complete qlik-script-parser but have to spend a lot of time on it in order to complete it)
There are tools (like sqlformat chose output to be List DB Object) that can extract tables and fields but not sure if automation can be achieved.

Related

What is the easiest/non-technical friendly way to upload and sort through a large dataset (400GB+)?

Would like to sort the (largely text) data by date tags first (separating the data by quarters, lets say, even if into different files). Then would like to perform standard functions like conditional sums on the data. Without a substantial programming/database background but with a willingness to spend a few days to learn, what's my best bet for a solution?
It sounds like you want to be able to do some queries on your data. I would look into a SQL database solution. The most difficult part would be getting your data into the database.
All of AWS's relational databases can import from text files:
Microsoft's SQL Server: http://msdn.microsoft.com/en-us/library/ms178129.aspx
Oracle: http://docs.oracle.com/cd/B28359_01/text.111/b28304/aload.htm
MySQL: http://dev.mysql.com/doc/refman/5.0/en/mysqlimport.html
I would base the decision purely on whichever one is easier to load your file. If you are budget constrained, you can download MySQL and not do this whole cloud thing. Just keep it on your local computer, assuming you have enough disk space to host the database.
After that's done, they all support SQL, which makes it very easy for you to query your data. If you don't want to write your own SQL, there are tools for you to create queries by drag and drop. But being a programmer, I highly recommend writing your own queries, or coming back here for some query help.

Is there an existing piece of software that allows you to (easily) build queries throught a webpage?

I would like to build arbitrary queries to a database, by allowing the user to build queries "on the fly". For every object/table, being able to select its attributes, and then "building" the query (that would translate into a SQL statement) and finally launching it, all through a web interface.
The ticketing system "rt" does that, for example, and another example would be the http://gatherer.wizards.com/Pages/Advanced.aspx webpage.
I'm currently programming in rails but any existing solution that implements this (or something similar) would be welcome.
Just be careful when creating dynamically generated queries like this that will need to be executed via sp_executesql (example: ms sql server), etc..... make sure you cover all of your bases to ensure that your application isnt vulnerable to SQL injection attacks as this type of development will essentially get one in a lot of trouble if its done incorrectly.. I would recommend storing all queries in a table and only reading queries from this table to help isolate the queries that are being ran in your application. Just identify them with a label, and allow the EU to choose the label from a dropdown list control on the frontend.
Good luck and I'm not sure of any software that will help assist
Not quite sure what your use case is here but i would say check out the
Doctrine ORM ( Object Relational Mapper )
**Edit
After reading more and looking at the example. I would only suggest Doctrine for a large website.
Then use Doctrines DQL syntax with some javascript/jquery magic for the forms.
Note that the queries you're referencing aren't arbitrary: they're on a very specific problem domain, on a specific set of sql tables.
That said, if I were you I'd look into how people are building sql queries with javascript. Something like these:
http://code.google.com/p/django-querybuilder/
http://css.dzone.com/articles/sqlike-sql-querying-engine?utm_source=feedburner&utm_medium=feed&utm_campaign=Feed%3A+zones%2Fria+(RIA+Zone)
http://thechangelog.com/post/4914956307/rel-arel-ported-to-node-js-with-some-changes
That'll at least get you a good idea of the underlying data structures.

Storing SQL in MySQL: Insert as Text?

Working in a team environment, each one of us has put together our own list of SQL statements that we use to help with our day to day job functions. As the case often is, there may be some redundancy with this, and we are often in need of each other's statements. To circumvent this, I'm looking to put together a small app that can be used to store and search for these SQL statements.
To begin with, I'm keeping it basic, just storing and searching the statements. This may build out to be the actual execution at some point in the future, but I'm not concerning myself with this at the time.
This will be built with PHP and MySQL - Should I store the SQL as text, or is there something that I need to be concerned with?
text should be fine.
you will also want a name (label) and possibly a descriptive field to let people know what it does in plain english.
Any particular reason these aren't just stored in the database as stored procedures?
Or in your version control system as scripts?

SQL Query builder in Delphi

I need to give users the ability to build a simple SQL query against our database. Our application is written in Delphi.
I am assuming only moderate levels of knowledge by the user, but they need the ability to build a simple select statement to be able to query against a couple of tables. If I can make this easy for them, that would be most wondrous.
Does anyone know of a tool or a set of components that I can use to help the users build SQL SELECT statements...
I've used the DevExpress ExpressFilter control to allow users to specify SQL where constraints before.
I've come across FastQueryBuilder http://fast-report.com/en/products/visual-query-builder.html
and
OpenQueryBuilder http://fast-report.com/en/products/free-query-builder.html
(apparently differing versions of the same tool) at Fast-Report.
It looks like the right idea. Anyone have any experience with it?
I've used SimpleQuery for years and am quite happy with it. (http://devtools.korzh.com/eq/vcl/) It's not completely intuitive, but once you get the hang of it, you can present fields to the user from your database(s) and they can combine any series of them with ANDs and ORs to make very complex queries. You get a separate SQL window to save the resulting code. I make The resulting dataset available for printout with PrintDat! or to save as CSV data from the Woll2Woll Infopower grid that I use for display. I occasionally allow for XLS export too with XLSReadWrite. It's all quite automatic at this point. Just take the SimpleQuery example and mold that to your particular needs. You should be able to have something very usable in a day. There is a trial and an example free query builder standalone program that will give you an idea of the final look.
TMS Query Studio? Good value at 75 Euro..
Try EMS Advanced Query Builder.
is a powerful component suite for
Borland® Delphi® and C++ Builder®
intended for visual building SQL
statements for the SELECT, INSERT,
UPDATE and DELETE clauses. It allows
you to build new queries visually
and/or graphically represent the
existing queries in your own
applications. The suite includes
components for working with standard
SQL, MS SQL, InterBase/Firebird,
MySQL, PostgreSQL and many more
dialects. Advanced Query Builder
enables users to make up large and
complicated SQL queries with unions
and subqueries for different servers
without any knowledge of the SQL
syntax.
(source: sqlmanager.net)
(source: sqlmanager.net)
I think the most powerful one is ActiveQueryBuilder, which now included with Delphi IDE (Since 2007 I think)
(source: activequerybuilder.com)
I've written a couple of similar things in Delphi. It's easy enough to allow the user to pick table and column names by querying the metadata and using lists and tree views for display. The difficulty comes when trying to implement things like joins. I've never come up with a good interface for this, and in my experience neither have many of the major data tools players.
You may want to try this free builder:
GSC Visual Query Builder v.0.7
Visual Query Builder v.1.0
and some other in Torry:
http://www.torry.net/pages.php?id=546

SQL query against a web page(or, advanced find)

Suppose I have a table. Now, I'm interested in Getting Useful Data Easily. This means I'd rather not drop it into Excel and go through contortions, nor somehow get it into CSV and then into a DB, and then into SQL.
I'd like to be able to execute a SQL query directly against a table in HTML.
Has anyone heard of a tool like this before? I've never seen one.
There's no tool to query an HTML table with SQL as far as I know, but XQuery is a standard language you can use to find elements in an XML or XHTML source in a manner very similar to using SQL to query databases.
No, there is absolutely no tool that lets you run SQL queries against an HTML table... but you could easily write some simple JavaScript if you had a specific query you wanted done.