How would you go for real time data streaming from a PostgreSQL DB on GCS to BQ?
Google has literally a ton of options and products on the left panel menu and from what i could find online, i saw that Datastream and Dataflow might be the way, but that example I found is with data from an oracle DB.
Would a database already on GCS only need dataflow?
I'm confused, the available material doesn't say much (at least to me, maybe it's my limitation) and i need to find a solution as I'm totally new to Google products.
Any help would be greatly appreciated.
Thanks.
Related
i have little experience working with local databases in flutter. I have recently found huge amount of data in sql database from one apk file. Now I would like to use it on my own flutter, but i cannot get inner data. Could you show me how to do that. Thanks a lot beforehand. Any kind of help is appreaciated.
I'm trying to figure out at what time people request content from a website (such as 'www.netflix.com'), do any of the avalible reports contain this data? If so how would I access it?
I've had a look around and can't see a table that has this data. Would anywhere else store it?
If you meant reports that available in BigQuery, I would suggest exploring BigQuery Public Datasets. At the moment I do not see any website access datasets but might still be useful for your reference.
I have a Firebase application which is uploading events with parameters. I need to be able to view those events in order to debug some issues we're having in production. I can only see the tables which are generated nightly in BigQuery. I can find references online saying that BigQuery allows viewing real time data. What I can't find is any straightforward instructions on how to create those views.
Is it possible? If so, can someone give me instructions that even a complete newb could follow?
We have decided to use the BigQuery APIs for information we want to see immediately in the database.
I am using BigQuery for SEO reasons. I am a search TC and I am a little confused why you are not using the Google Forum as I thought that was standard. What I want to use BigQuery for is to pull when my competitors change data on their website and which pages that were changed. So I need the URL that was changed and when it was changed (date) so I can also pull the page title and description to see what they are doing different than I am.
Is there anyone that knows how to use BigQuery to pull:
Date the page was changed
URL
Title
Description
We've switched to using Stack Overflow for support for many of our developer products, such as BigQuery. There's a great community here on StackOverflow, and the interface for formatting technical questions and interacting with the community is fantastic.
BigQuery does not collect the data for you-- it's a cloud service for performing ad hoc queries on massive datasets. Before first performing the queries, you need to upload the data to the service (as a CSV format).
So, if you have a job which collects this data -- URL, title, description, date and perhaps a hash of the webpage, you could potentially ingest a CSV file of this data into BigQuery and use it to understand when webpages have changed.
Of course, there are also 3rd-party services (such as Changedetection.com) which may be easier to use for your purposes.
We are trying to develop a company specific tracking software but not interested in Google or Piwik. Essentially we would have a JavaScript tracking code also. The data that it would capture, would that be best suited for traditional RDMS or can we get a NO SQL solution ?
Any thoughts or ideas welcome.
Creating xml files could do the trick for a no sql solution but web analytics can encompass a very large ammount of data depending on your "tracking software." You'll need some sort of relational data solution if you want to properly analyse the data and see trends such as how many unique visitors are using a specific browser.