How can this be done with Mechanical Turk?
1- Worker fills in an evaluation test
2- The same worker uses an online e-learning site specifically on the topics that he/she failed.
3- The same worker passes again the evaluation test in order to confirm he/she made progress.
The test and the training can both be done online. Results and communications via email.
Thanks.
This sounds like a lot of manual work or custom programming.
You might want to use a Mechanical Turk Partner like TurkPrime.com. They work with many researchers to create customized solutions using what they call Concierge Services".
It sounds like manual work to me as well. If you do not want to program it then the manual option is to split it into three separate HITs and add a custom qualification after the first and second HIT. These qualifications would be required to accept the second and third HIT respectively.
Related
I'd like to use mturk to have 10 workers visit my website, log in with a test account, and enter some information on their profile. I don't want them to see each other's entries, so each worker should get login information for a different test account when they view the HIT.
This almost looks like what mturk's template feature is for -- I could upload a CSV with the information for each test account. But if I understand correctly, that will make 10 separate HITs, and allow one worker to do all 10 of them. Is there any way to have mturk put information that varies between workers into a single HIT?
Here are the solutions I'm currently aware of:
Use the CLI to automate creation of a bunch of different HITs. This would be a lot of work, and also make approving and retrieving the results cumbersome.
Direct workers to a survey website that's capable of doing what I want, and have them get the login information there.
Dynamically fill in part of the HIT using an AJAX request to an external website and database. That seems like crazy overkill for something so simple.
Are there other options?
As a requester, after I create HITs I'd like to preview them on the website both on the Sandbox and in Production for various reasons, ranging from final verification to checking up on a particular HIT.
But I have found that I am not able to view my own HITs since the qualifications I've set are beyond what my account when logged in as a worker can view. So I'd like to ask is there a way for a requester to be able to review their HITs even if they themselves do not having matching qualifications?
Options:
If they're your own user-defined qualifications, then you can assign them to yourself in the live server and in the sandbox.
If they're statistics-type qualifications (like number of HITs completed), you have to actually manually achieve them. You could do this by creating a bunch of HITs for yourself to complete in the sandbox. Tedious, but it's the only way to actually get the qualification.
If it's something like masters, there's no way to really make them work in the sandbox unless you follow (2), above, but the definition of master is not public anywhere so it's unclear how much you'd have to do to gain the qualification.
You can, of course, remove the qualifications in the sandbox just to test the HIT itself, but that's probably not exactly what you want.
To strengthen the authentication mechanism (web), I would like to log a user fingerprint for every attempt and apply pattern recognition to distinguish malicious attempts. For example if the user always logs in from European computers and there is an attempt made from China, the user is blocked until the user confirms (via email, for example) to allow logins from China.
I have a very, very small knowledge of pattern recognition from a university course. However, I cannot recall enough to start developing this service. What I know is that you should look at these various features:
Browser agent string, resulting in:
Operating system
Browser vendor
IP address, resulting in:
Location
Time stamp of login
Number of (failed) attempts (within session, or total)
You search for patterns and any extraordinary attempt is marked because it does not follow the average pattern. You probably will apply a threshold, so if a user logs in at night or has a new PC, it still works.
There are also a few requirements: first, the check of an attempt must be made real-time. You cannot block access after 2 minutes if the credentials were OK but you found out later on the attempt could have been malicious. Furthermore, all our apps are written in PHP, but PHP is probably too slow for this. I prefer to use Python then, but subsequently there is also a binding to Python required.
So the question is: where to start? What is the best approach to accomplish this? I can log all data in a key storage like Redis or document based like Mongo. How would I design a service which allows to validate a new attempt with certain features against a bulk of known other attempts? And return whether the attempt matches the average within a timely fashion, say 250ms.
What you want to do is called anomaly detection- wikipedia is a good place to start. As a first stab, you might want to try clustering:
you will need a data set. The good news is clustering is unsupervised, so you will not have to mark up a ton of login attempts as regular or malicious.
For a given user, keep a history of their past N logins (big brother warning!) and features of those logins. The features you have listed are a good start, but you can think of more.
apply a clustering algorithm to estimate what the average login is like. For every new attempt you can calculate the distance from the average and decide if it look malicious or not.
As a side not, you can go a long way without learning. My intuition is the location of the login and the number of failed attempts will get you most of the way there. A simple if-else might be good enough.
Theoretical question. Lets say I build an application for managing clients, products, bills and such. All without a user login.
Is it possible to get multi user (each one with his own clients,products,...) functionality after the main application is done?
Or should I think of the multi user in the first place? How flexible is rails at this part?
thanks in advance
dennym
What you are asking for is a pattern called Multitenant. There are different ways to do it, but none are provided by Rails natively.
You could start with this slide: http://www.slideshare.net/tardate/multitenancy-with-rails
And figure out if you want to do; partitioning based on data, Rbac, model or schema.
There are is one that offers it as a service that says that it is easy to do after you have your application: http://railskits.com/saas/
But in our experience, it was a bit outdated and missing some features that we wanted.
You can also take a look at other gems like: multitenant or act_as_tenant.
We ended up using act_as_tenant and doing it from the beginning.
I'm an absolute newbie in Mturk,and I'm definitely not a person who understands smth in programming. thus,I would appreciate a lot a simple answer on my question saying which button and where I should press(this is an upper limit of my computer skills).
the question is the following one:I want 200 people to participate in my survey. I want 1 pseron to answer on my survey just 1 time. where should I set those requirements? should I put the number 200 here:Number of assignments per HIT or there will be some other space where I will specify the number of my participants?
thanks a lot in advance!!!
I think you are looking for the MaxAssignments option of CreateHIT operation.
Here is the attached image for the developers guide
It says each worker can complete a HIT only once, so multiple assignments are guaranteed to be completed by multiple workers.
Hope this helps
I have seen workers try to complete my HITs more than once, even though there not meant to see the HIT again. Here are some workarounds to prevent this form happening.
Republish the same HIT. Workers can access an identical HIT if it's published under a different name.
You can add a qualification for every worker that has already taken your survey. Even without programming, this is fairly simple. You can download a list of your workers, add the qualification on the spreadsheet for all of them, and then upload the spreadsheet into MTurk. When you set up your HIT, you can add this qualification. In this case, you would add: (Name of your qualification) Has Not Been Granted.
Good luck.