I'm trying to scrape data from HubSpot using ParseHub but not only do I have go through the login page everytime (which is easy because I've set it up to automatically enter my email and password), I also have go through two-factor authentication, which is not easy because: one, it's randomised so I can't keep putting the same six-digit number in; two: because I only have Google Authenticator on my phone which means it's impossible for me to scrape the six-digit number. Just to note, I am able to get past 2FA on Test Runs, because of the "Step in" feature of ParseHub. This allows me to manually enter the six-digit code everytime. Unfortunately, this feature is not available on the live run, which means the script basically fails at the 2FA stage and no data is scraped.
I've downloaded the Google Authenticator extension to my PC, but I have no idea how to have the same account for 2FA for HubSpot that I have on my phone, be present on my PC. If I was able to achieve that, I'm pretty sure I would be able to scrape the six-digit code.
Any help would be greatly appreciated.
Related
I am trying to read data from a webpage that requires a login. I could use twebbrowser and have the user login through that, however, the point is to not allow my app to handle any security credentials, even through twebbrowser. My hope is that the user would login on their default browser, and then my app would load the page as a logged-in user without any credentials going through my app. I swear there was a time, many moons ago, when I was able to do this. However I can't seem to get it to happen now. Is this possible? Or any other suggestions for connecting to a logged-in website without the credentials going through my app? Thanks in advance.
Additional info:
What I am trying to do is write an app that reads purchase history from a rapid-fire sale website (new item every few minutes) and keeps a running total in real time while also warning the user if something comes up for sale that they've already purchased (because the web site repeats things that didn't sell out). I prefer to keep my app as only a data aggregator, ie read-only, completely separated from logging in, purchasing, etc. I don't want people worrying about entering their password or credit card in my app.
I can see many related questions on SO, but none that answers exactly what I'm confused with.
I'm using Google Calendar API in a .NET desktop application that allows user to provide his/her username/password, logs in on his behalf and adds some events to the calendar. Now I want to do exactly the same thing for Tasks feature. I'm trying to use Google Tasks API for this, but have been told that I need to do some OAuth kind of authentication, and even before that, I need to go to my gmail account and set permissions and get my project "key" to enable it.
Now does every user of my application need to do these steps in their Gmail account? Or do I need to do this in MY gmail account once and then my application code will be able to use the generated project "key" to enable my users to add tasks to THEIR gmail tasks list?
Figured it out. For anyone having a hard time understanding this, here it is:
The "key" generation step needs to be done only once per application, not for each user who's going to use your application. To generate a key, login to your Google Account and go to Google APIs Console page. Click API Access button and that's where you can generate keys for different kinds of applications like browser apps, desktop apps, Android apps etc. After registration, you'll need to take Client ID, Client secret and API key from this page and put them into the code. Sample code (.NET) for task creation and several other Google features is available here.
Once your user runs your application, he'll be taken to his Google account in his default browser where he'll be asked if he wants to allow this application to write to his calendar/tasks list. This page will display your logo and description text too that you can provide at registration time. Once allowed, this step won't be required again in the next one hour (this may be adjustable, i don't know yet).
I've been struggling for some time now, trying to figure out how to make a button that redirects to a page asking for permission and then sending the partial information of their accounts to my database or email address and give other actions such as subscribing.
An example of who is doing this is socialblade and fullscreen, they're using it quite nicely and I would love to use this in the future.
Writing an android widget to display the most current Google checkout ending balance and can not figure out how to get this number directly. I have dissected the page but don't understand how the number is generated. I can gain access through a simple browser interface and view my daily ending balance but cant pull that single number for display on the widget. Any help would be grateful. Thanks.
There currently aren't any Authorization APIs to request balance.
In order to get this information, you would need to request the Google login credentials from the user which is something that Google probably doesn't want you to do(the reason why client login was deprecated).
I can't think of a great solution to this problem with out Google enabling an OAuth2 API.
Facebook allows you to create test user accounts that can only be used for testing purpose.
Does Twitter provide similar functionality ? I don't want to get my application blacklisted for creating fake user accounts; do I have to use my real user account for testing my application ? What strategies are your using for testing application with Twitter ?
As far as I know, there is nothing in Twitter's rules against creating account (unlike Facebook, where with the exception of test accounts, you're not allowed to create multiple accounts for testing purposes). So, you can just register the account like you normally would.
You might want to take a look at this post for some other tips for test accounts (hiding your tweets, deleting the account when you're done testing, etc.).
I'm currently creating a Twitter application and here are some of the strategies I'm using.
I create my accounts in combination with Gmail addresses. If I create a gmail account as user bob#gmail.com , I secure the Twitter name #bob on Twitter. That way it's kind of hard to forget where to email a lost password. I don't go crazy, as I don't need 100's of test accounts but I do have up to three.
I log on to my test accounts using Chrome because it will automatically recall your password as soon as you type in your Twitter name on the home page. That way it is easy to switch between them, but note that I find it hard sometimes to know which account is actually active because I'm constantly looking at other profiles. This gets confusing if I don't constantly look at the logged in user icon indicator.
Never, ever re-tweet anything unless you absolutely have to for a test case or use hash-tags unless for a test case. Unbelievably even on a completely un-publicized account, I had a few surprise Twitter users in my DB a few seconds after I re-tweeted a link.
on Localhost, close all your other browser windows while your testing. Especially if your calling the API through AJAX. You never know which sites you have open whom also call the Twitter API through AJAX, and this can seriously screw with your tests and rate limits. Especially when your developing live.
I would not recommend protecting your tweets. It's too limiting for most use cases.
For my site, I need to place a link in the tweets. Twitter will not
allow live links to http://localhost so you have to plan around
this and have a live test server sooner then you may anticipate.
Twitter has one of the easiest registration processes I have seen. You can quite easily create several test accounts; this is the only method I have used.
Here is a blog post about it.