I did a trial period for BigQuery and have been testing it out, but I checked the allow unlimited box for a query with a bad join and it wound up being billing tier 251 for 14GB, which is way over the $300 they give you. It hasn't shown up in my billing though so I'm not sure if this is covered by the free TB each month or if I'm going to discover a $17,000 bill at some point. How does this work? I am very worried
Related
For a small personal coding project I recently created a SQL database in Azure. For the past weeks I have been hardly using the database, out of 2 GB available space I have been using only 13 MB.
However, the database costs me 6,70 EUR per day and I don't understand why this is the case. Read a few topics/posts stating that the costs with similar use should be around 5-7 EUR per month, not per day.
This is the configuration for the database:
No elastic pool
General purpose, Gen5, 2 vCores
West Europe
Does anyone have an idea about what could be causing the costs per month to be so high?
You choosed the General purpose, Gen5, 2 vCores price tier. Here is the cost every month:
This means that you must pay for it no matter how many space you used. As you said you just used only 13M. So you must change the Pricing tier.
What I suggest you is configure you database price to Bacic which only cost you 4.99 USD per month. Basic price tier provides 5 DTUs and Max size 2GB for you.
You can change the price tier on the database overview site:
Hope this helps.
You're paying for the entire infrastructure is why. It really only saves on upfront cost. A dedicated server, Windows Server + SQL Server Web will run you, at least $5K. Performance wise, a dedicated server at a colo center will be a lot cheaper to run once you get the hardware. I know, I've switched several companies off of Azure and, instead of paying $2500/mo, they pay $200/mo (after the server) for 4U at a colo + $100/mo basic maintenance and 1TB/mo bandwidth, so it adds up. For example, I built 2 custom 1U servers (12 core/32GB) for $8500 and an opensource router for another $500 (pfSense), including OSes & SQL Server Web. Initial setup of both servers including SQL and the router for 16 IP Addresses was about $1K. Total cost was $10K up front. The equivalent horsepower and storage from Azure was $2500/mo. In 1 year on Azure it ran $30K! 1 year on colo (hosting + maintenance) was $13600, the following year was $3600. So far in 5 years, they saved ~$122,000. There was only 15mins of downtime during the entire period. Cloud hosting is a great idea, but it will never save you time nor money at the rates these company's charge. As far as downtime, I have been hosting for 2 decades and the worst downtime happened due to a network failure (that also took out multiple cloud providers) and it was 13 hours. The only other one was due to a fried router (about 3 hours). Just my take on it - Cloud hosting is still way too expensive for what you actually get & redundancy is nice but you can buy a new server every 2 months for the price difference (just get good equipment w/redundant power supplies and hot swap drives - in a 55 degree colo center, failures are rare)
It seems you don't know Azure offers a free tier. Please refer to this StackOverflow thread for details on how to take advantage of the free tier that supports databases of 32 MB of space.
If it is a small project you can run it on Ubuntu Linux and it's $3.80/month or $0.0052/hour.
On top of this, you can install MySql or SQL Express. I personally find MySql easier to access/configure
It's sure that Azure offers a free tier but still you can optimize it with very low cost if you use any purchased plan.
Here I provided some direction on the picture below that how to create free App Service Plan
Now let's see how we can optimize the cost for small database size for your purchased plan.
Go to the option of Create SQL Database
Click on the link Configure Database as per the below picture
Then Select the Basic option under DTU-Based as per the below picture
As the above picture shows, the default selected option is General Purpose option under VCore-Based section, so it costs $410 as it provides you 32 GB database.
As Basic option is selected, so DB Size is changed into 2 GB instead of 32 GB, hence cost is changed into just $5.64 instead of $410
According to the following link: Azure Database Pricing the hourly rate for a Basic Tier DTU service plan for a single database model is priced at $0.0068.
The recent billing invoice for my (Pay as you go) Subscription stated that the hourly rate for a 'Single Basic B DTUs' SQL Database is listed as $0.1610.
Without trying to get ahold of Microsoft, does someone know why their site advertises the basic DTU hourly rate at one price and have a different rate on invoices? Not sure if anyone has encountered something like this, just trying to get an idea why there was a change?
Any guidance or clarification would be greatly appreciated.
Double check your invoice, that price is quite close to the $0.1613 S7 plan. Also note the price varies per region, so check the region as well. If you still can't come up with an explanation, simply open up a support ticket from within the Azure portal, they typically respond in a reasonable time and will happily resolve billing discrepancies.
Google Cloud billing is not updating with the free trial (on monthly payments) and I can not change it to a faster update cycle. As per https://cloud.google.com/free-trial/docs/billing-during-free-trial the bill should come every month.
It is therefore not easy to see how much of the 300$ is left.
Is there any way to at least see how many TBs my queries used? This should be by far the biggest item on the bill.
I am concerned that I might get 'stuck' between some important queries that I otherwise could have managed better to have at least partial results available after the trial ends.
BigQuery analysis & storage costs should be listed under your GCP billing transactions:
https://console.cloud.google.com/billing/<INSERT_YOUR_BILLING_ID_HERE>/history?e=13803970,13803205
Another way to see how much you have queried is by enabling audit logging as described here.
Im new here and looking actually for this:
https://temboo.com/hardware/google-big-query-getting-started
Its going about how to connect Sensors to Google Bigquery,
but I actually don't know whether it is free or not.
My usage per month were around 1GB.
Please tell me what I can get for free there, I'm absolutely beginner and don't want get a big bill.
Thanks,
Petr
BigQuery charges for query processing and storage.
Query processing will likely be free in your case, since the first 1 TB per month is free, and you're only using 1 GB per month.
You will likely to have pay a small amount to store the data you're querying, but at that scale we're talking pennies ($0.02/GB/month).
Full pricing details:
https://cloud.google.com/bigquery/pricing
Is there anyway to determine how many requests or how much data you have in your remaining request limit amount for Bloomberg API?
from Bloomberg HelpDesk on April 2014 (this is valid for a basic desktop client):
We have 3 kind of limits..
You can have no more that 3500 real time fields open at the same time.
If you exceed this limit you will see "NA Limit" as error message and
you just need to delete some securities/ fields in order for the error
message to disappeared and to see the values.
We have also a daily limit. The Daily API limit is 500,000 hits/per
day. A "hit" is defined as one request for a single security/field
pairing. Therefore, if you request static data for 5 fields and 10
securities, that will translate into a total of 50 hits. so try to
refresh just the portion of the spreadsheet that really needs to be
refreshed and avoid refreshing it all or reopen it many times a day.
The last limit is a monthly limit. Our monthly limits comes from a
proprietary model. Only about 0.4% of our user database ever goes over
this limit. This limit is based on unique securities and depends on
the type of data being downloaded. For example some of the data on the
system such as intra-day is valued a little bit higher than historical
end of day for any given list of securities. We do not recommend more
than 5000 to 7000 unique identifiers per month and the limit upgrade
will only allow you to get data to complete your project. Once a
security is used once in a month then if you use it again it will not
count again towards the monthly limit.
We normally grant 2 resets per month in case you exceed your daily
limit and if you exceed your monthly limit we grant 1 extension per
month (10% more), if you breach the limit again you will then need to
wait for the midnight for the daily limit to be reset automatically or
the end of the month for the reset of the monthly.
Bloomberg do not state what the explicit limits are, and there is no programmatic way of finding out what the limits are or what proportion of your limits you have used.
The best information from Bloomberg that I have found is on the WAPI page (in the terminal). On the menus on the LHS, go to WAPI Home > API Resources > API Data Limits. There are two pages, 'Extended Rules and Usage Limits' and 'Managing Your API Data Limits' that shed some further light on the matter.
Broadly speaking... there is a daily limit of individual data requests (i.e. security/field pairs - but duplicates are counted for each request). However, your limit for subscriptions is based on the number of securities you are subscribing to concurrently - i.e. if you expect to be requesting the price of a security every 5 mins, you are much better off subscribing to that security's price. Then there is a monthly limit that is based on the number of unique securities that you are making requests for.
there is an upper limit on Bloomberg API, 500,000 hits per day.
-- information from Bloomberg Help Help
The daily limit is clearly stated - it is the monthly limit that is not to my knowledge disclosed in writing. I have been told the following in the context of discussions about Data Licence, which is one Bloomberg product for bulk data subscription. The monthly limit is expressed as a budget in $, and it is the equivalent price for your requests, priced under the Data Licence schema, which clearly is not secret if you enquire about that product. So why the secrecy about the budget? The reason it is commercially sensitive is that this budget is many times the monthly cost of the Terminal Licence, so clearly if you (a) know what it is and (b) either have access via API to the budget spent (nope) OR write software to 'count the cost' (not hard), then you could pony up a couple of terminals and vastly reduce your Data Licence spend. Bloomberg naturally frown on this sort of activity because it represents an arbitrage opportunity in their pricing model and it is not really 'playing nice'. They likewise do not like if you hit 'the wrong kind of data' too often or the monthly limit at all often, and these activities may prompt them to investigate your business model to be sure you are in compliance with all the T&C of the Data Addendum. Out of courtesy to Bloomberg I am not posting that budget number here, but you should be able to get it from your salesperson and confirm the validity of what I have said, because it may change at any time as it is not part of any contract.
I don't believe this is possible programmatically, however if you speak to the Bloomberg helpdesk they will be able to tell you whether you are near the limit, and reset it for you if necessary. Obviously they will only do that a certain number of times. I have not managed to get a definitive answer as to what the limit is, but it's designed to be large enough that you would not hit it just running spreadsheets, which have a limit of 3500 Bloomberg real-time formulas.
If you feel the download limit is not breached but you still get the error message, you can run the following steps to solve the issue:
Close Excel completely.
From the Windows "Start" menu, select All Programs > Bloomberg > Stop API Process. A command prompt window appears.
Press <Enter> to close the window.
From the Windows Start menu, select All Programs > Bloomberg > API Environment Diagnostics.
Click the Start button.
When the test is complete, if there are any red errors, click the "Repair" button.
Re-open Excel and test a formula.
500'000 data points is the approximate daily limit, however remember different types of data use up varying amounts. It is not 1 for 1. Typically requests for esoteric securities and fields will use up more data per request, than PX_LAST for AAPL US for example. Also there are different types of request, such as reference or historic, which will also consume your limit differently.
If you are requesting intra-day realtime data, these fields are typically not charged to your usage limit. Rather you have limits on how many times the realtime 'pipe' can be opened.
Bloomberg are typically very helpful at resetting your monthly data usage limit should you exceed it on an adhoc basis. This is not written company policy, but seems to be part of their customer care. If you are persistently breaching limits each month, they are likely to stop resetting your limits and try to move you to B-PIPE. But otherwise for my experience they are flexible