Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
There are several posts on SO that point to sources for downloading historical stock quotes, but these are all for currently listed symbols. The resulting dataset suffers from survivorship bias. Is there any source of all historical data, including delisted companies, preferably for free/cheap? I've found a few sources, but they're usually hundreds of dollars or more, require installing using some Windows client software, or just are on sketchy-looking websites. (End-of-day data is fine - I'm sure asking for intraday bid/ask is too much.)
Where do these data resellers in turn get their data from? What is the original source archive of data? (Some of these datasets date back to the '50s, so I don't think the answer is "they just record it themselves.") Do they cut deals with exchanges / do the exchanges have/sell this? Does the data exist in any public records? Thanks!
Norgate Investor Services is the cheapest I've found, but it will run you hundreds of dollars ( but less than 1000 ). Their source is Standard & Poors.
QuantQuote has survivorship bias free historical stock data, but they only offer it in minute/second/tick resolution, and it costs $$. They also have free daily resolution data of the S&P500. It's too bad Yahoo doesn't keep stock data around after a stock gets delisted.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I was recently asked the following question in an interview.
"How would you design a system to keep track of a million items at xyz.com ?
The xyz.com could update the prices maybe 2-3 times a day or once per month, so no guarentee on frequency.
Your system should show accurate prices for >95% of items at any given point of time and aim for 99%.
Also scale for 1billion items etc..
"
I asnwered along the lines of creating a distributed system app that would categorize items by priority (based on historical price fluctuations and 80/20 % rule etc) and do API calls more frequently for these.
But I was not allowed to use API calls.
I suggested scraping html content. (But the website can block my ip for such high load)
I basically want to know the resources that would help me anwering these type of questions. Prefer full length courses (Distributed systems ?) or books rather than quick-fix blogs.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
Our company is going to implement Big Query.
We saw many drawbacks in Big Query like
1. Only 1000 requests per day allowed.
2. No update delete allowed.
and so on...
Can u guys highlight some more drawbacks and also discuss on above two.
Please share any issues come during and after implementing Big Query.
Thanks in Advance.
"Only 1000 requests per day allowed"
Not true, fortunately! There is a limit of how many batch loads you can do to a table per day (1000, so one every 90 seconds), but this is about loading data, not querying it. And if you need to load data more frequently, you can use the streaming API for up to a 100,000 rows per second per table.
"No update delete allowed"
BigQuery is an analytical database which are not optimized for updates and deletes of individual rows. The analytical databases that support these operations usually do with caveats and performance costs. You can achieve the equivalent update and deletes with BigQuery by re-materializing your tables in just a couple minutes: https://stackoverflow.com/a/31663889/132438
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
I was doing some research and a need arized to get mtgox historic depth data, their api was not giving a lot of good info, it gives around ~6k bids and asks and most of them in last 3-4 months, while all of the dataset starts around 2010. I was looking for bid ask data with higher granularity, preferably for more currencies than just btc to usd.
Any help would be really appreciated.
You can access MtGox's full historical trade data via Google BigQuery.
See here for more information:
https://bitcointalk.org/index.php?topic=218980.0
I haven't seen anywhere that you could get the full order book history though. That would be an absolutely massive data set if it existed, so I doubt if anyone has such a thing available.
There is currently a new software developed to download all historical data from various exchanges and currencies. The release date expected is October 2013.
More details here: https://bitcointalk.org/index.php?topic=282154.0
The latest source for MtGox's data feed is: https://en.bitcoin.it/wiki/MtGox/API/Pubnub
And PubNub's Bitcoin Exchange Solution Kit based on the above: http://rtbitcoin.co/
Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 11 years ago.
Improve this question
Out of interest, how far can buying and selling shares be made automatic? How far do you think it can go? And what would you need to do it?
Very far.
There are real computers performing fully automated trades as we type.
I'd take a look at some trading APIs provided by E*TRADE https://us.etrade.com/e/t/activetrading/api or something similar. You can use virtually any language to call the API and execute trades, get quotes, and generate strategic algorithms.
However, a fair warning, unless you are a large corporation with lots of money to burn you need to be careful as a homegrown algorithmic trading system can be very dangerous.
I believe something like 70% of all trades on the market today are placed by computer systems automatically and not by people. It makes it harder for smaller individuals to compete because we simply don't have the financial resources to purchase the power and speed we need to compete.
Thanks,
Jeffrey Pry
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I am currently working on developing a program that stores Quantity as a decimal with four digit places for the very small minority of products that are sold in part (eg 100ml liquid being sold in variable amounts like 1mg-3.5mg).
I dislike this design as it causes problems for the entire program in deciding how to present the quantity and now some problems where there are products which incorrectly have decimal values.
Can you think of a better way to do this?
I'm looking for an alternative to a four decimal place field in the database.
The project uses SQL Server 2005.
Edit. I made this question more generic in order to obtain a larger potential audience
Store the quantity in the smallest amount sold. So if something is sold by the mg, store it that way. That way you don't have to store fractions, and deal with rounding.
Just create a looking table for the product so that you can store the value of how it is stored and displayed, as well as any other meta-data about the size.
In .NET, a System.Decimal is going to be your best option as System.Single and System.Double are not good at representing floating point numbers with accurate representational precision.
Can you provide a small, complete example that demonstrates your problem?