Google Big Query: alternatives to browser SQL editor? [closed] - google-bigquery

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
I'm currently working a lot with Google Big Query and I absolutely hate querying in-browser. I'm used to connect to regular DB's through editors like Toad, Microsoft SQL Studio, Teradata Studio Express or Databeaver. I'm look for a similar tool that you guys would recommend for using on Google Big Query.
Only alternative I've found so far are razorsql and jetbrains datagrip (whereas the latter requires a custom connection (https://blog.jetbrains.com/datagrip/2018/07/10/using-bigquery-from-intellij-based-ide/)
Any idea's on alternatives out there?
Thanks in advance

There's a thread on reddit with some recommendations.
I personally like the UI and there's a new UI coming (but doesn't seem to change much in functionality, only styling).
Some of the recommendations are:
DataGrip
Aquafold
Metabase
Potens
PopSql
Plus a few other tools that aren't RDMBs per se but have the components to "run queries" not in the UI, like Mode Analytics.
I haven't personally used any of them, but take it for a spin and see which one you like the most! (share your insights here later as I'm also curious about it)

I'd recommend http://potens.io/, specifically Goliath which a great tool for data exploration. My team of ~20 non engineers uses Potens for data exploration and user friendly workflow automation (Magnus). Let me know if you have any questions.

I tried PopSQL (popsql.com) which looks like a nice alternative for the Web-UI and has the best Bigquery support of all the other tools. However it does not have many features and is not much better than the Web-UI. The main advantage is you can organise your stored-queries better, as you can use folders instead of the long list of stored queries in the webui.
There are some cons however: There is no autocomplete with table names and attribute names, and it does not have "save" buttons so you overwrite everything and sometimes things are not saved at all. It feels a bit beta, with some bugs and raw edges, so the payed full version is overpriced. The free version can only save 15 or so queries so it is of no use.
As for the other options I have tried:
Datagrip does not have a bigquery driver. There are some hacks to write your own driver but this has limited support.

Currently there is no official independent editor for BigQuery, but the BigQuery Web UI that you can use as the graphical web UI to query your data. However, you can file a feature request per instructions at Report Bugs and Request Features with Issue Trackers to request the independent editor tool for BigQuery.

Related

SAP GUI scripting disabled on server - other automation options? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
SAP GUI scripting is disabled on my employer's servers and I'm not in a position to change that, but I would very much like to automate some routine tasks of mine using some kind of scripting. I don't have (and I don't need) administrative access to the server, so this automation [afaik] has to revolve around SAP GUI in one way or another.
As an example, most repetitive task of mine that lacks automation is loading numerous reports with different parameters and exporting them to plain text files (to be aggregated and analyzed later in Excel).
What options should I look at? Mouse-and-keyboard automation tools come to mind first, but they are the least appealing ones at the moment.
For most of the options that are available, you will need at least the cooperation of the system administrators, and usually for good reasons - automation always gives you the option of destroying vast amounts of mission-critical data very fast. Depending on what you need to do - your question was not very clear on that - you might want to explore the following options:
Try to automate the data entry on the selection screen using selection variants. You can not only save static sets of data, but also include dynamic date calculations, and you can reduce the fields displayed until only the relevant fields are left.
Check whether you can schedule background jobs to have the reports run automatically at a given time and get the contents downloaded or mailed to you, depending on the capabilities of the report in question and the settings of your system.
For more complex flows, use the transaction recorder to check whether the task can be automated using batch input techniques. If this is possible, you could try to use the Legacy System Migration Workbench (LSMW) - although designed for data migration, you can use this tool as a "CSV-to-User-Command-Converter".
If you've come this far, it's not that hard to start writing ABAP programs, and we're back on topic for Stack Overflow... :-)
You can use Sikulli to automate tasks in SAP. They have a image recognition system where it can click buttons based on what it looks like. If you need to scroll through tables or something you are probably out of luck.
In my previous job I had to explain what SAP GUI scripting is to at least 2 dozen people and wait/follow for months to use it. Let say many people are still using my scripts there. It is always better to show your skills, So you better go proper channels. Also Using similar tools might not be welcomed If they are not enabling on their servers.

Best way to connect a spreadsheet to web database [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I'm using web based database for which I need to add spreadsheet capabilities to its front end. I was thinking that I could use Google Docs Spreadsheets. Their Google App Script seems to have the functionally that I need. In particular I can use the URLFetch service combined with onEdit events to keep the spreadsheet and DB in sync -- AJAX style. It also allows me a lot of flexibility in constructing, saving, and sharing the spreadsheets
However some things about Google App Script gave me pause. It runs server-side so it's difficult to debug locally. It doesn't have any sort of debugger with breakpoints or stepping. It can't import external modules or libraries. No JSLint. Without these I started getting that "Uh, oh, this is going to hurt" feeling.
So I'm wondering if there's a better way to bolt on browser accessible spreadsheet functionality to an existing web based database? Or are there best practices for getting the most out of Google App Script?
EDIT:
These are the potential solutions in order of what would be best for my application:
Browser based JavaScript spreadsheet engine. (May not exist.)
Python spreadsheet engine module that I can install on Google App Engine. (I haven't seen this either.)
An more robust and AJAXian approach to Google Spreadsheets. (See original question.)
Open source spreadsheet engines that I can install on EC2. (These seem to exist -- possibly SocialCalc, or Simple Spreadsheet. Recommendations?)
We use spreadsheet functionality on a web page, but rather than scripting all the features of a spreadsheet, we use a calculation engine which gives us the programmatic heart of spreadsheet functionality. A calculation engine knows how to calculate hundreds of types of formulas, handle dependencies (and the order between dependencies), cell formatting etc.
In my particular case, we use SpreadsheetGear - http://www.spreadsheetgear.com/products/spreadsheetgear.net.aspx
We create a HTML representation of a spreadsheet with cell navigation and various other features using some javascript. When we need the sheet to recalculate (eg F9 in Excel) we send the entire spreadsheet to the server, ask it to calculate everything and then refill the web page representation with the results. This may also write to the database depending on what's on the spreadsheet.
Perhaps I need your input at this point, to see if my answer is not too far off track.
So I did dig into this myself, and it did hurt a bit. Here are the particulars:
The bad:
You have to use their editor, no JSLint.
No debugger. The scripts run server side, so Firebug and other browser tools are zero help. The "View->Execution transcript" and "View->Logs" are a little help. But they don't seem to function in "onEdit" events.
The connection to the web goes through their UrlFetch Service which is doesn't have an asynchronous mode. And it doesn't function at all within "onEdit" events: http://code.google.com/p/google-apps-script-issues/issues/detail?id=185
They basically have three event types. "onLoad", "onInstall", and "onEdit". In particular I badly missed an "onClick" event.
No external library support.
The good:
It is real JavaScript, so it functions predictably, and I can use an external JSLint tool.
Shallow learning curve. Understanding sheets, ranges, and set/get values are the key concepts.
Lots of functionality for manipulating spreadsheet items, formatting, etc.
Google Data API Support? I haven't used this, but it looks like the way to go for connnecting to external web resources.
Well integrated, scripted updates on my spreadsheet are immediately visible to others viewing the spreadsheet.
Overall this environment has a ton of potential, with lots of UI capabilities, but it still seems to miss on some key functionality for doing real production development.

What features should a new SQL-based Rapid Development tool have? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I am interested in developing a new SQL-based RDS which can generate prototype CRUD-oriented 4GL code which can then be easily customized. So I am looking for opinions, suggestions, etc. in what features should a new RDS have?
I have looked at several products. FileMaker is more or less the type of product I'm looking for, but it's not SQL-based. Informix 4GL and 4Js are what comes closest, but the lead time for customizing/debugging seems to take a long time.
Depends on the market you are aiming for. In your place, my first decision would be whether to link it to a specific RDBMS implementation (which would be Oracle, SQL Server or mySQL) or to try to make it flexible to use amongst any of the major databases. The latter would obviously add massively to the complexity (development, testing, support).
The second would be the OS platform. Something that is specific to Windows (or perhaps Windows Vista/7 upwards) or can be used on Linux, OSX etc. 32bit/64bit would also be a consideration here. I'd need to know that it can run on 64-bit Windows 7 at least.
Thirdly, what do you develop your Rapid Development Tool in. Dot Net would pretty much be Windows only. Java or Python offer more choice.
Fourthly, are you making this as a commercial product, or as open source (or both). Bluntly, if I was looking at buying such a tool to develop my XYZ product in, I'd be looking at long-term support. If the tool doesn't have a five-year pedigree of ongoing support, I'd be very hesitant and would only consider it if it came from a company of the likes of Microsoft/IBM/Oracle/Apple with some serious history in the field. And even then I would be skeptical (search on Microsoft's Oslo/Quadrant/SQL Server Modeling).
I'd also be looking for professional tech writers for the documentation.
See the feature list of Oracle Application Express for some good ideas.
I've used a few of these tools and things that come to my mind are:
1/ automatic joining on foreign keys
2/ keeping columns in the same order as the table in the code
3/ dynamic code generation, no separate code generation stage
4/ undo on table changes
5/ prototyping features to enable actual data to be modelled during development
6/ hide column feature
7/ SQL import for those tricky bits
8/ code locking on tricky bits
9/ version control
10/ segmentation into namespaces
11/ Language templates for easy addition of extra code languages
I think going to an intermediate 4gl first would be good and then a second stage to render code in the language of choice would be good.
I hope this is of some use to you, good luck!

Documentation of a software project [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
I am working with a team that works on a very large software project, we have tons of Documentation that is written in MS WORD format with nohyperlinked indexes, no search ability.
Everyday we waste our time trying to find the exact document or reference.
I was thinking if there was way or even a professional tool that would convert all this into a wiki format and maybe with a little manual (painful) help be organised into something that improves the accessibility.
I use Google Desktop Search to make my life a little easier but its not the best solution
I just want to know if any of you faced similar problems and possible solutions to this issue.
Confluence wiki lets you import Word documents. I've been told it's a really good wiki with a lot of features.
Open Source: Trac Wiki
The Trac Wiki system is a popular open source option. There's even a thread on another Stack Exchange site about converting Word documents into the Track Wiki format. Some large scale or notable users of Trac include Red Hat, Django, Handbrake and SourceForge.
Commercial: Confluence
Confluence is another popular option, as mentioned by Alex Korban. A good example of a large-scale use of it is the Application Server 7 community documentation of software company JBoss.
2015 Update - PressGang and Corilla
I'll just add a quick update that I agree so strongly with this problem of discoverable content and collaboration that I've worked on solving it in a few different projects.
PressGang (the prototype)
Please refer to the PressGang CCMS project for an idea of what we did at Red Hat to build tools to solve this. The lead engineer did a run-through video that you can see on Vimeo, and I've created a public Amazon AMI if you wish to try it. It's not being maintained but it's all open source. Check out the repository or get involved in the community fork.
Corilla (the product)
That was an amazing experience to be a part of and inspired me to spin out Corilla, an open source technical writing startup tackling exactly the problems you mention. This will have both a commercial/enterprise version available as well as open source community version. There's a beta currently running that I would encourage you to get involved with - the problems you're experiencing will be great to shape the solutions being built.
Don't know about tools to convert MS Word to a Wiki, but if you save your documentation in ODF (The XML format from OpenDocument) it should be possible to import the documents in your own program, or transform them using XSL, and output them in Wiki format.
In the process you can try to guess some links based on string matching.
The difficult part, in a team used to write documentation in a word processor (without even using word processing basic features like indexes, table of contents and hyperlinks), is to convince them to use another system that is not a typewriter.

What's a quality development environment for writing Oracle SQL? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 9 years ago.
Improve this question
The project I'm working on has an Oracle backend, and we use SQL Navigator 5.5 for SQL development and it really sucks: the intellisense doesn't work for queries with more than one table and you can't browse the schema and write a query on the screen at the same time. Not fun.
I also tried using Oracle SQL Developer. That's not much of an improvement either - the graphics on the query results window get all fudged up and have to be repainted frequently by clicking around the screen.
The extent to which I'm writing SQL is pretty basic: a procedure here and there, mostly queries. Can someone recommend a decent tool, preferably one with a low cost?
You may want to check out PL/SQL Developer (http://www.allroundautomations.com/plsqldev.html). It does more than just PL/SQL. Cost is pretty reasonable. On Linux try Tora It is suppose to be a replacement for Toad. Have used to other 2 personally but not the latter.
I'm pretty sure that TOAD is the most used tool. I don't recall how much it costs now days, but I'm generally very satisfied with it even with a slightly older version.
All IDE:s for Oracle DB are more or less bad, TOAD is the best but still nothing comapared to real IDE like Visual Studio or Eclipse.
Main problem with TOAD is the default texteditor. Find is nightmare and you can't change shortcuts and auto alignment uses spaces even if you configure it to use tabs and other annoying features. Also cancelling queries or making query to disconnected session might crash it. Also the intellisense is kinda slow imo but it works which is better than what others offer.
There are several tools out there you could give a try.
TOAD or DbVisualizer are examples for tools which could make you happier than SQLDeveloper. Altough, most of the available tools are commercial.
SQL*Plus: it's free with the database, and always guaranteed to be there - and there's nothing you can't do with it :)
I used [CAST](http://en.wikipedia.org/wiki/CAST_(software) for 2004-6, and it's quite good. One of the principal reasons for using it was that it supports locking of PL/SQL packages when editing them. Now I use jEdit for almost all developments, and Oracle SQL Developer for easy lookup of table properties and such.
There is a freeware version of TOAD, though it's not as fully featured as the paid ones. You can find it at:
http://www.toadsoft.com/
There is fairly decent support in Emacs using sql-mode and plsql-mode. I wouldn't recommend it unless you are an Emacs user though.
Failing that Toad is fairly good, although it fairly heavy weight.
If you're looking for an open source solution SQuirreL SQL Client is pretty good. It isn't Oracle specific though. It works with any jdbc client.
Another open source tool which works really well is SQL Workbench (http://sql-workbench.net/). Works with all JDBC clients, but we've had alot of success with Oracle. Also includes some scripting support which can be helpful for importing/exporting large amounts of data.
I personally use PL/SQL Developer to develop and Spotlight on Oracle to profile.
Two more options (just for completeness):
Hora
Aqua Data Studio