Ok, here is what I want to do. Given the name of a baseball player, I would like to automatically import his stats from baseball-reference.com and embed in my site.
They already allow you to do this manually by providing you with a 'share' button that will give you the html of the stats table. I would just like to automate this process.
I have no idea on where to start on something like this. Your advice is appreciated.
Thanks for the help. I never thought about this being against their EULA since they were letting you do it manually. Oh well.
Related
I'm new to python and scrapy. I had hoped I could combine the two to scrape some gambling websites. This is an example;
https://www.oddschecker.com/football/scottish/premiership/kilmarnock-v-aberdeen/winner
If i simply view source html for that page, the main table of odds isn't in the html. Which confuses me greatly. Have tried using scrapy on it but it's got the same issue.
What's going on at that page that nothing can see the data in the table? And what would be an easy way to scrape it?
Thanks!
It looks like the initial response is all JavaScript. Generally gambling and stats sites don't want to be scraped, and this looks like a means to thwart bots. I would think that this will deep on the difficult end of the pool, and you would likely be better off with something that employs a browser so you can render it.
I'm attempting a scrapy-with-splash project to get a few fields off the website "https://sailing-channels.com/by-subscribers". This site uses java to retrieve and delete listings as you scroll.
I've not had any luck getting the splash server to give me the whole set of data, or any of the detailed listings for that mater.
My first question is can splash even do this?
I really don't care how I get this data. I would prefer doing it with a program but any tool that can get me fields from this site in an .csv file would do the job. Anyone have any suggestions?
Thanks for any advice
Why do you want render it? They have pretty good API, check https://sailing-channels.com/api/channels/get?sort=subscribers&skip=0&take=5&_=1548520116425. So you can iterate, increasing skip argument and parsing json each time.
Looks like very promising way.
It must be the full url, not just history or type. I have seen, on statcounter.com, where they show stats regarding where the referrer of our site came from. I want to show the exact link like statcounter shows under a tab.
Does anyone know how to do this?
You will want to use Google Analytics.
Google will generate a unique javascript code for you, you then paste it in your html, which activates the service. It's very easy to get started and the dashboard is very robust considering it's free. To get started, you can check this out: http://www.google.com/analytics/learn/setupchecklist.html
I'm building a support form for my application. I'd like the user to include their Operating System and Browser they are using, catch is the general users either don't care to look this stuff up or don't know how. I'd like to add a text-input that automatically grabs this information for the user and displays it in the text field. Not sure how to go about doing this as I'm new to rails. I'm assuming I'd have to tie this in with javascript or a model that is attached to the form.
Any ideas would be appreciated.
you can read it on the server side so there is no need to bother the users.
request.env['HTTP_USER_AGENT'] should do the trick. You can also read some more information like the accept language or accept encoding. Just look at the keys like this: request.env.keys and check what could be of use to you.
I've been taking a quick look at QTP with one of our test team.
We've figured out some basics like recording a test, capturing text using output values, putting in check points, etc.
However, I can't figure out how to do the following...
We have a web form which a user fills in and submits. This will create an item and assign it an ID in the database. We can capture this ID.
A subsequent page then shows a list of all items, with a link available to open a specific item. This is a simple hyperlink on a web page.
How do I:
Check that a link exists on the page with the ID assigned from the creation step?
Click that link?
I'm sure that this must be possible, but I've been struggling grokking the tool so far, so this is a shameless cry for help to save me from having to study the docs.
Thanks.
Quickly look on Web GUI recognition principles in QTP. Get to the descriptive programming part.
Find Help for Link object in QTP.
Define your target Link object using descriptive programming.
It should be something like
Set objTargetLink = Browser("title:=...").Page("title:=...").Frame("title:=...").Link("id:=target_id")
Use
boolRC = objTargetLink.Exist(0)
To check if your link exists.
Use
sURL = objTargetLink.GetTOProperty("url")
to retrieve the actual url. You may get other properties the same way.
Use objTargetLink.Click to click on the link.
PS. Functional Test Automation is something different, though.
Ask your testing team to read about automation frameworks and automation requirements.
I have some stuff on my blog.
http://automation-beyond.com/2009/06/06/qa-test-automation-requirements-usability/
Thank you,
Albert Gareev
http://automation-beyond.com/