Is is possible to use the Python Boto3 API to create HITs in a way they are not immediately available to workers?
The CreateHIT doc seems to talk about the HIT being available to workers immediately and I don't see a flag that let's you decide not to submit it.
I don't believe the API allows you do create a HIT while setting it to be inactive or inaccessible (outside of using Qualification Requirements to prevent workers from accessing it).
You could create a HIT Type using create_hit_type() and then later create a HIT using that HIT type (create_hit_with_hit_type()).
Related
I am planning to do a large scale crowdsourcing experiment on MTurk and would therefore like to do this using the API and Python, since I am very interested in the ReviewPolicies Feature. I tested this (not only in the sandbox, but also in the marketplace) and can't find my created test HIT (reward set to 0.01).
What could be the reason for this? Also I read in in some prior tasks, that HITs created with the API are not visible in the developer interface. But they must be visible to workers on the website interface, aren't they? If not, how will these HITs be found by workers on the dashboard/marketplace?
I published the HIT successfully on the marketplace (API) and I can see the HIT response. I expected to find this specific HIT also on the dashboard (signed in as a worker).
Background and Goal
I have a Debian/Linux VM on GCP which I manually start every morning and after it runs, it shuts down by itself using a Linux command. I want to automate the start of the VM by using the Cloud Scheduler. The question asked in GCP auto shutdown and startup using Google Cloud Schedulers has several answers and I am interested in pursuing the answer (https://stackoverflow.com/a/65062924/10322004) proposed by #nikelone because it seems to be simple and also it has been endorsed by #Damien and #RayFoss as being easy. I am a neophyte in these matters and I could not comprehend their replies fully. So this post was created to elicit more clear answers for a person like me.
What I have tried
I have gone to https://cloud.google.com/compute/docs/reference/rest/v1/instances/start (call this page A) and tried the API and was able to successfully start my already stopped VM when I clicked on the execute button. I presume that this means that my entries were fine and can be used in conjunction with appropriate software like Cloud Scheduler to perform the start function on a predefined schedule. But the problem is that I do not know or understand how to proceed from here. I give below my questions.
My Questions
On page A, the last three paragraphs are titled Authorization Scopes, IAM permissions, and Examples, and none of them say anything specific about what the user should do. Is it correct to assume that they have nothing to do with the Cloud Scheduler, but related to other methods to achieve the same goal? If this is not correct then my next question is what should I be doing to follow the statements in these three paragraphs?
Assuming that the answer to question 1 is "yes", meaning I can now start scheduling with the Cloud Scheduler, I next looked at the quickstart for Cloud Scheduler at https://cloud.google.com/scheduler/docs/quickstart (call this page B). The list of items to do is quite large including installing Cloud SDK, running a quite a few commands on the console, enabling some features, set up Pub/Sub, create a job, run the job and verify the results in Pub/Sub. This looks like a daunting set of tasks and I could not understand why it is necessary to jump through the hoops to use something that has already been achieved with just a few keystrokes earlier. So are these steps all necessary? Or is there a way to use the Cloud Scheduler directly without going through so many intermediate steps?
Now assume that the answer to question 2 is that I have to perform all steps stated on page B. If I run into some problem while accomplishing the tasks outlined on page B, my VM may get messed up irretrievably. Is there a way in which the Cloud Platform or its components can be used to reset my VM to its current state as of today, which is working fine? I really do not want to end up with something worse than what I have now.
To answer your questions:
Auth Scopes and IAM permissions are required for you to call the Compute Engine API methods such as instance.start & instance.stop. You need to set the right scope and the right IAM permission on your job or else it will fail. They are indeed related to the method that you're interested to call so you must keep them in mind. What you see on the examples are the ways to call the {API} using different programming languages so you don't need to pay attention to them as you will create the job through the Cloud Console. To further address this part, see the full steps I included below.
The answer that you're trying to follow uses HTTP target while the quickstart you've linked uses Pub/Sub and they are different with each other because they have separate use cases. This link shows a proper instruction how to create a scheduler job with an HTTP target. You can create this kind of job straight from the Cloud Console or a one-liner gcloud command. If your config is incorrect, the trigger will not execute the endpoint URL and you will see an error that you must fix.
Addressed on answer #2
Basically, you just need to follow the instructions to the link you've sent. However, I'll post it here as well along with my explanation:
Go to https://cloud.google.com/scheduler. Click on Go to Console. Click on Create Job. Fill up the required fields (those with red asterisks) when creating a Scheduler Job.
Select HTTP as target type.
Enter this as your URL (modify the capitalized words).
https://compute.googleapis.com/compute/v1/projects/PROJECT_ID/zones/INSTANCE_ZONE/instances/INSTANCE_NAME/start
Choose HTTP method POST.
Click show more and choose Auth Header "Add OAuth Token"
Enter your service account. This is used to pass an OAuth Token when your scheduler job calls the Compute API. Make sure that the service account you will use have the "Compute Instance Admin" role because this role contains the permissions to start/stop your instance. See this instruction how to grant access on a service account. If you're not sure what service account to use, feel free to use the Compute Engine default service account.
Add this on Scope:
https://www.googleapis.com/auth/cloud-platform
The description of this scope:
See, edit, configure, and delete your Google Cloud Platform data.
Repeat for Stop instance job and change URL in #3.
I was seeing this Deployment Manager template on Google Foundation Toolkit on Github
I am not able to get it what this line is for.
Github Repo
That is a reference to another function within the template (line 184). The reason this does it sequentially is that the API call to add a user to the DB only handles adding a single user at a time. The act of adding a user to CloudSQL through API cannot be done in a batch. Since DM creates all these resources through API call, the sequential calls are required
I created a data collection Job (HITs) using the mechanical turk website.
I would like to reject/approve jobs using python API, because that would accelerate the process.
I can approve the process using the python API, but that doesn't update the Assignment status on the website. Does anyone have any idea on this?
Thanks.
Unfortunately, HITs created through the website are not visible or manageable in the UI (and vice-versa). So, that's why you can't see or operate on them via API calls. You'll need to create the HITs
You can reuse the layouts created in the website, though. Check out this article: https://blog.mturk.com/tutorial-using-the-mturk-requester-website-together-with-python-and-boto-4a7ef0264b7e
I am trying to use BetterAuthorizationSample rather then go the so called "malicious" way of using setuid in order to get root privileges.
Currently I am using AuthorizationCreate(); with BLAuthentication to have root access to changing some files, but I am somewhat irritated by the fact that I have to constantly enter my password in every time the app launches.
So I came across Apple's method of a HelperTool, and I just can't figure it out.
I've been working with Cocoa for a couple months now, but this is just out of my reach, yet I still need it. How would I implement this tool to do simple root-privileged tasks?
Is there a simpler way to use the concept of a HelperTool, so that my users can just enter their password once and it would grant root-privileges forever?
The "modern" way to do a helper tool on Mac OS X is to ship it as part of your app, and use the ServiceManagement framework to deploy it. Your users enter their password once, when deploying the tool. That installs it as a launchd job; from then on you use any launchd on-demand mechanism to launch the helper and get it to do work for you.
Notice that the blog post linked above recommends that you protect subsequent invocations of the helper with an Authorization Services escalation, to avoid having an arbitrary privilege escalation that anyone can use. This seems like it somewhat impacts the "users can just enter their password once" benefit, although you can use AuthorizationRightSet() to create your app's authorization token in the policy database, so you can actually define whether users need to present passwords on first deployment.
The sample code from that post is on GitHub, and demonstrates using ServiceManagement to deploy the helper tool and Authorization Services to control access to it.