When I try to launch a transfer from Amazon S3 to Google GCS using the google API from the console, my transfer keep stuck at the "calculating" step. I have been using this API a lot for four months and it is the first time I get this kind of behaviour. Maybe be this problem is linked with the last API version deployment.
Related
I'm trying to send data generated by a Google Home mesh network to a Splunk instance. I'd especially like to capture which devices are connected to which points throughout the day. This information is available in the app, but does not seem to be able to be streamed to a centralized logging platform. Is there a way I'm not seeing?
There are two methods of ingesting google cloud data supported by splunk.
--Push-based method : data is sent to a Splunk HTTP Event Collector (HEC) through a pub/sub to Splunk dataflow job.
--Pull-based method: data is fetched from the Google Cloud APIs through the Splunk Add-on for Google Cloud Platform.
It is generally recommended that you use the push-based method to ingest Google Cloud data in Splunk. Only in certain cases it is recommended to use the pull based method to ingest Google Cloud data into Splunk. These circumstances are as follows:
--Your Splunk deployment does not offer a Splunk HEC endpoint.
--Your log volume is low.
--You want to pull Cloud Monitoring metrics, Cloud Storage objects, or low-volume logs.
--You are already managing one or more Splunk heavy forwarders or are using a hosted Inputs Data Manger for Splunk Cloud.
More information as how to work around with the setup part and working around with this
problem can be found in the following link :
https://cloud.google.com/architecture/exporting-stackdriver-logging-for-splunk
I working on a project in which I’m facing a problem that I want to get real-time data from BigQuery dataset,
So I research hard I learn about web sockets like pusher and laravel-websocket, and I also test them also
but One thing I didn’t understand that how to do If I connect BigQuery SDK with my Laravel project then I didn’t find any event or method that BigQuery gives us to do so, because in my scenario most data comes from IoT devices and the device uses bigQueryAPI to feed data I also want that devices data real-time.
Then I found that there’s a way to connect BigQuery Dataset with firebase and then firebase gives us an event whenever row fed in the dataset.
I just want to know Is there’s any that without a firebase solution I can do it?
because the above solution is not free I have to pay firebase to avail of this feature?
thank you
IoT devices can post the data to pub/sub instead of BigQuery, from pub/sub one consumer(Ex:Dataflow) can read and insert the data into BigQuery and you can create consumer for your custom needs.
I am trying to set up a data extract in BigQuery to pull data from a Google Ads Account. But I keep getting Auth errors (AuthenticationError.NOT_ADS_USER) when the extract tries starting pulling data.
<ApiError><type>AuthenticationError.NOT_ADS_USER</type><trigge
What did I do?-
Set up Big Query and the data extract (as depicted here: https://cloud.google.com/bigquery-transfer/docs/adwords-transfer)
Invited and added the same email address that big query is using to the Google Ads account with read access level (at first) and then standard access level when the problem persisted.
Is there something that I missed, that I have to enable/accept/configure in order for Big query extract to be able to pull the data from Google Ads?
Thanks,
p.s I have another extract in the same big query account with a different Google Ads account that is working, I recall have followed the same steps.
I would recommend to remove and add the user account that you are using to access Google Ads. It's usually related to permissions related issue. Have you followed the steps mentioned here?
https://cloud.google.com/bigquery-transfer/docs/adwords-transfer#required_permissions
If yes, try remove and add again as suggested on both Google Ads and GCP. If it's not working, try creating a case with Google Cloud support team if you have that service enabled if not your marketing folks might have connections with Google to help you out.
I'm trying to build a data collection web endpoint.The Use case is similar to Google Analytics collect API. I want to add this endpoint(GET method) to all pages on the website and on-page load collect page info through this API.
Actually I'm thinking of doing this by using Google Cloud services like Endpoints, BQ(for storing the data).. I don't want to host it in any dedicated servers. Otherwise, I will be end up doing a lot for managing/monitoring the service.
Please suggest me how do I achieve this with Google Cloud Service? OR direct me to right direction if my idea is wrong
I suggest focussing on deciding where you want to code to run. There are several GCP options that don't require dedicated servers:
Google App Engine
Cloud Functions/Firebase Functions
Cloud Run (new!)
Look here to see which support Cloud Endpoints.
All of these products can support running code that takes the data from the request and sends it to the BigQuery API.
There are various ways of achieving what you want. David's answer is absolutely valid, but I would like to introduce Stackdriver Custom Metrics to the discussion.
Custom metrics are similar to regular Stackdriver Monitoring metrics, but you create your own time series (Stackdriver lingo described here) to keep track of whatever you want and clients can sent in their data through an API.
You could achieve the same thing with a compute solution (Google Cloud Functions for example) and a database (Google BigTable for example) and writing your own logic.. but Custom Metrics is an already built solution that includes dashboards and alerting policies while being a more managed solution.
I have used google map API but unfortunately I have discovered that they have an usage limits so I want to know that are there anyway o get an google map API without any usage limits, and if it is not how can I get any license?
Google maps api usage and limits
Do I need an API key?
Answer: in the early days of google maps you did not require an API key (it is still possible to not have one due to backwards compatibility) however, today, it is recommended that you generate an API key for google maps V3. Also, have in mind that some features are not available without the API key.
If you want to inform your self more about the API key, here is the official page that will tell you how to get started with it and how to include it to your project,
url: https://developers.google.com/maps/documentation/javascript/tutorial
What are the usage limits?
Answer: If your site gets 25 000 map loads or more every single day, for more than 90 days in a row, the google team will get in touch with you (they are well aware of your google maps usage). If however you don't think your google maps will generate such an amount of traffic then do the following:
Modify your application so that your usage is less than 25 000 map loads per day.
Enroll for automated billing of excess map loads in the Google Developers Console.
Purchase a Google Maps API for Work license.
Here is the official page for the limits,
url: https://developers.google.com/maps/documentation/javascript/usage#quota
To sum it all up:
If you can predict huge traffic on your website (and using the google maps API within your website), then once again, read carefully the docs and see what further steps need to be taken, otherwise you can experiment with google maps as much as you want.