I am trying to accept a HIT as a worker from sandbox but every time it shows following message.
"Your request for the Qualification "Categorization Masters" has been submitted to Requester Amazon Mechanical Turk"
Can anybody say what I am missing here.
The sandbox works just like the live site, as in you (as a worker) have to be meet the qualifications in order to accept a HIT. This means if you're trying to complete your own HIT in the sandbox just to try it out, you have two options:
Remove the QualificationRequirements from the HIT so you can complete it.
Obtain the necessary qualifications in the sandbox in order to satisfy the requirements.
Option 1 is easy (and is what you should do). Option 2 is difficult, or maybe impossible.
Related
Does anybody know where documentation for the Metacritic api is/if it still works. There used to be a Metacritic API at https://market.mashape.com/byroredux/metacritic-v2#get-user-details which disappeared today.
Otherwise I'm trying to scrape the site myself but keeping getting a blocked by a 429 Slow down. I got data like 3 times this hour and haven't been able to get anymore in the last 20 minutes which is making testing difficult and application possibly useless. Please let me know if there's anything else I can be doing to scape I don't know about.
I was using that API as well for an app I wrote a while ago. Looks like the creator removed it from Mashape. I just sent him an email to ask whether it'll be back up. I did find this scraper online. It only has a few endpoints but following the examples given you could easily add more. Let me know if you make any progress!
Edit: Looks like CBS requested it to be taken down. The ToS prohibits scraping:
[…] you agree not to do the following, or assist others to do the following:
Engage in unauthorized spidering, “scraping,” data mining or harvesting of Content, or use any other unauthorized automated means to gather data from or about the Services;
Though I was hoping for a Javascript way of doing this, the creator of the API also told me some info.
He says I was getting blocked for not having a User agent in the header and should use a 429 handling procedure i.e. re-request with longer pauses in between.
A PHP plugin available as well: http://datalinx.io/shop/metacritic-api/
I had to add a user agent like JCDJulian said and now it allows me to scrape. So for Ruby:
agent = Mechanize.new
agent.user_agent_alias = "Mac Firefox"
Then it stopped giving me the 403 Forbidden error.
Background
In our Apache configuration we use mod-auth-external (previously on Google Code) to invoke PAM authentication.
Now there is a request for proper handling of shadow-based password expiration:
If password is before warning period Apache should respond with HTTP status code 200. Nothing new here.
If password is in warning period (its validity end is near) Apache should respond with HTTP status code 200, but include somehow information about the warning period.
If password is in expiration period (it is no longer valid but user can still change it on his own) Apache should respond with HTTP status code 401 and include somehow information about expiration period.
If password is beyond expiration period (it is no longer valid and account was locked, administrator must unlock it) Apache should respond with HTTP status code 401 and include somehow information about the locked state.
(There are also corner cases of page missing or some other errors. It is not clear what to do then. But it seems that solving above points would allow to solve those corner cases as well.)
Our PAM authenticator (used through mod-auth-external) is able to differentiate those cases by adjusting return values. That we already have.
The problem is however how to get information from the authenticator to the associated action serving the page (either actual page with 200 status code or 401 error document).
Current investigations
It should be noted that there is significant difference between requirement 2 and requirements 3 and 4.
Requirements 3 and 4 alone are somewhat easier because they both involve our mod-auth-external authenticator returning error (access denied). So we only need to know how to get that error code in 401 error page. I even raised issue on that on mod-auth-external page.
Requirement 2 is much more difficult. In that case our authenticator must return 0 (access granted) and still somehow propagate information about the warning to whatever gets served in the end.
Logs parsing
Obvious (and ugly) idea is to parse logs. mod-auth-external description on Google Code Wiki mentions that authenticator return value gets written to Apache syslog. Also whatever authenticator prints to standard error stream gets logged as well.
This could be used to pass information from authenticator to some other entities.
The difficulty here is that it is not clear how to do it safely. What to print to be sure that "the other entity" will match properly current request with log entry. Mere URL doesn't seem to be enough since there can be multiple requests for the same URL at the same time. While I don't see anything more useful in what authenticator gets.
Another issue here is that it seems that to be able to parse the logs you have to have some non-trivial code running for "the other entity". And this complicates things further since how should we do it?
Another idea
If we could make the authenticator somehow modify "request session" (or whatever, maybe just environment? - I don't know, I'm new to Apache) to add arbitrary data to it we would be (almost) at home.
Our authenticator would somehow store "password status" and also possibly days remaining to the end of warning/expiration period (if applicable). Then upon serving 401 error page we would retrieve that back and use it to dynamically generate content of the page.
Or even better we would have it stored in session so that the other end could read that data directly. (For cases where it is not simply a browser showing page.)
But so far I fail to see how to do that.
Do you have any idea how to meet those requirements?
For over a month I got no answer here. Nor on GitHub issue that I opened for mod-auth-external.
So I ended doing a custom modification to our mod-auth-external. I don't like modifying third party software but this one seems dead anyway. And also it turned out we are using pretty old version (2.2.9 which I upgraded to 2.2.11, the last in 2.2.x line). Which already had some customizations anyway.
I explained details of the solution in a comment to my GitHub issue so I will not repeat them here.
I will however comment on shadow details as they were not mentioned there.
I had two choices: either use getspnam function to retrieve shadow data or to parse messages generated by PAM. First attempts based on getspnam function but in the end I used PAM messages. I didn't have strong reasons for any of those. However I decided to propagate in HTTP response not only shadow status but any PAM message that was generated and so it seemed easier to follow that way.
There is one IP (from China) which is trying to download my entire website. It downloads all my pages and loads the server significantly (I have more than 500 000 pages). Looking at the access logs I can tell it's definitely not a Google bot or any other search engine bot.
Temporarily I've banned it (using iptables rules), but it's not a solution for me, because some of my real users also have the same IP, so they are also banned and cannot acces the website.
Is there any way to prevent such kind of "user activity"? Maybe a mechanism which implements captcha if you try to request more than 5 requests a second or something?
P.S. I'm using Yii framework (PHP).
Any suggestions are greatly appreciated.
thank you!
You have answered your own question!
Make captcha appear if the request exceeds certain number per second or per minute!
You should use CCaptchaAction to implement, like this.
I guess the best way to monitor for suspicious user activity is really user session, CWebUser's getState()/setState(). Store current request time in user session, compare it to several previous values, show captcha if user makes requests too often.
Create new component, preload it via CWebApplication::$preload and check user activity in components init() function. This way you'll be able to turn bot check on and off easily.
While trying to debug my openid implementation with Google, which kept returning Apache 406 errors, I in the end discovered that my hosting company does not allow to pass a string containing "/id" as a GET parameter (something like "example.php?anyattribute=%2Fid" once URL encoded).
That's rather annoying as Google openid endpoint includes this death word "/id" (https://google.com/accounts/o8/id) so my app is returning 406 errors every time I log in with Google because of this. I contacted my hosting company who told me this has been deactivated for security purposes.
I could use POST instead, for sure. But has anyone got an idea why this could cause security problems ???
It can't, your host is being stupid. There's nothing magical about the string /id.
Sometimes people do stupid things with the string /id, like assuming no one is going to guess what follows, so that example.com/mysensitivedata/id/3/ shows my data because my user has id 3, and being the sneaky sort, I wonder what happens if I navigate to example.com/mysensitivedata/id/4/, and your site blindly lets me through to see someone else's stuff.
If that sort of attack breaks your site, no amount of mollycoddling by your host will help you anyway.
One reason a simple ID in the URL could be a security concern is that a user could see their ID and then type another one in, such as if its an integer they may select the next integer up, and potentially see another users info if it is not protected.
Authorize.net offers a "Silent POST" feature for their Automated Recurring Billing. It's supposed to POST data to a url of your choosing, telling you whether they were able to charge the customer, how much, etc. The problem is, it isn't very well documented.
Is there any way to test a post to that URL? I've signed up for a developer account, but there's no way to specify that URL like you could in the actual system. Hence, there doesn't seem to be a way to test it out.
If not, is there a list of possible values it could return? It appears to send x_first_name, x_amount - I've seen code that uses those values - but since I can't actually get it to send a response, I'm not sure.
Is there documentation for this feature anywhere? Or even class that implements it fully?
Better late then never: All About Authorize.Net’s Silent Post
I have not seen much on it only for AIM and SIM, you might just give them a call.
Log in to your Authorize.Net order processing account, and click on the Settings link (under ACCOUNT, in the left column). Then click on the "Silent Post URL" link in the Transaction Format Settings area. You can enter your silent post URL on the next page. The next page also contains a link to the documentation explaining the technical details. HTH
Here's a few more (somewhat) useful posts I found on the subject.
Merchant Account Services - gives some limited sample code (PHP)
Experts Exchange - lists a few helpful variables, gives an idea of what's being sent (ASP).
You still have to call your account rep for them to activate Silent Post URL with your account because that is not something that is enabled automatically
Our clients use the following tool to test silent post url requests sent from the Authorize.Net gateway.
Simply add the following url to your silent post settings and change the email address for the results to be delivered to an email of choice.
URL:
http://www.silentposturl.com/action/email/index.php?support#silentposturl.com