Is there a way to configure an S3 bucket to enable notifications/alerts whenever another version of an object is created? I am using versioning to prevent overriding an existing object but I would like also to be able to detect such events.
With versioning, an overwrite will not actually delete the existing object but it well add another object to the same key but with a different version. I would like to configure the bucket to receive a notification only when an object is created on top of another object (again, since we have versioning).
Related
I have a lambda function that gets notification whenever and s3 object gets added. The function then reads some metadata of the s3 object. Is there any way to include the metadata in the notification itself, rather than me having to go and read the head object.
The event notification data structure does not include object metadata or tags, and there is not a way to enable their inclusion.
New object is not cloned
$clonedObj = clone $repatNewsObj;
$this->extendednewsRepository->add($clonedObj);
$this->persistenceManager->persistAll();
This is not how you copy an extbase object with TYPO3. Sadly, just cloning it with PHP will not reset the object from the extbase point of view. The add method of the default repository will just ignore the "new" object if it already has a uid in the database. Hence, nothing is actually done by your persist call.
You could either create a new object yourself, setting each property manually, based on the source object or you can use the datahandler to copy your object. There is a blog post describing it (in German) as well as an older SO question discussing the same issue.
If you want to clone an object from one of your own extensions, best practice is to overwrite the __clone method and reset things like uid and so on there. If the model class comes from a third party extension, chose one of the two methods from above.
I need to replicate from one read/write Pimcore instance to another Pimcore instance read-only.
Is there a smarter way to achieve that than
- replicating the database (master slave model)
- "rsync"ing the media and class file system folders?
There are APIs to export and import contents and structure, but is this reliable with high volumes ?
Thanks
Possibility 1:
Never tried this, but you could post the object / asset / ... to an controller on the other instance. To do that, you need to extend the save-function of all objects or hook into the save event.
On the "slave" instance, you can then check if an object / asset... with this path and key exists, then update, otherwise create a new.
Possibility 2:
Do it with polling: Instance A tells B about it's objects, ... via a controller. B then iterates and clones it.
I want to create a AWS lambda event source to catch the action of upload a file via aws cli cp command, but it couldn't be triggered when i upload a file. Here is what i have done:
I configured the event source as following:
I have tried all the four option of Object Created event type, it just didn't work.
I use the aws cli as following:
aws s3 cp sample.html s3://ml.hengwei.me/data/
Is there anywhere i miss configured?
You are triggering your Lambda from the wrong event type.
Using the awscli to cp files up into S3 does not cause an s3:ObjectCreated:Copy event (which I believe relates to an S3 copy operation, copying an object from one bucket to another). In your case, the object is being uploaded to S3 and I presume that it results in either s3:ObjectCreated:Put or s3:ObjectCreated:CompleteMultipartUpload.
The events include:
s3:ObjectCreated:Put – An object was created by an HTTP PUT
operation.
s3:ObjectCreated:Post – An object was created by HTTP POST
operation.
s3:ObjectCreated:Copy – An object was created an S3 copy
operation.
s3:ObjectCreated:CompleteMultipartUpload – An object was
created by the completion of a S3 multi-part upload.
s3:ObjectCreated:* – An object was created by one of the event types
listed above or by a similar object creation event added in the
future.
Full list of events is here. Note that the awscli may or may not use multi-part upload so you need to handle both situations.
since we store our log files on S3 and to meet PCI requirements we have to be notified when someone tampers with the log files.
How can I be notified every time a put request is placed that replaces an existing object, or when an existing object is delete. The alert should not fire if a new object is created unless it replaces an existing one.
S3 does not currently provide deletion or overwrite-only notifications. Deletion notifications were added after the initial launch of the notification feature and can notify you when an object is deleted, but does not notify you when on object is implicitly deleted by overwrite.
However, S3 does have functionality to accomplish what you need, in a way that seems superior to what you are contemplating: object versioning and multi-factor authentication for deletion, both discussed here:
http://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html
With versioning enabled on the bucket, an overwrite of a file doesn't remove the old version of the file. Instead, each version of the file has an opaque string, assigned by S3, identifying the Version ID.
If someone overwrites a file, you would then have two versions of the same file in the bucket -- the original one and the new one -- so you not only have evidence of tampering, you also have the original file, undisturbed. Any object with more than one version in the bucket has, by definition, been overwritten at some point.
If you also enable Multi-Factor Authentication (MFA) Delete, then none of the versions of any object can be removed without access to the hardware or virtual MFA device.
As an developer of AWS utilities, tools, and libraries (3rd party; I'm not affiliated with Amazon), I am highly impressed by Amazon's implementation of object versioning in S3, because it works in such a way that client utilities that are unaware of versioning or that versioning is enabled on the bucket should not be affected in any way. This means you should be able to activate versioning on a bucket without changing anything in your existing code. For example:
fetching an object without an accompanying version id in the request simply fetches the newest version of the object
objects in versioned buckets aren't really deleted unless you explicitly delete a particular version; however, you can still "delete an object," and get the expected response back. Subsequently fetching the "deleted" object without specifying an accompanying version id still returns a 404 Not Found, as in the non-versioned environment, with the addition of an unobtrusive x-amz-delete-marker: header included in the response to indicate that the "latest version" of the object is in fact a delete marker placeholder. The individual versions of the "deleted" object remain accessible to version-aware code, unless purged.
other operations that are unrelated to versioning, which work on non-versioned buckets, continue to work the same way they did before versioning was enabled on the bucket.
But, again... with code that is version-aware, including the AWS console (two new buttons appear when you're looking at a versioned bucket -- you can choose to view it with a versioning-aware console view or versioning-unaware console view) you can iterate through the different versions of an object and fetch any version that has not been permanently removed... but preventing unauthorized removal of objects is the point of MFA delete.
Additionally, of course, there's bucket logging, which is typically only delayed by a few minutes from real-time and could be used to detect unusual activity... the history of which would be preserved by the bucket versioning.