I started working with Projected File system, but I'm not sure if I read the documentation correctly. PRJ_FILE_STATE_HYDRATED_PLACEHOLDER.
says
The item is a hydrated placeholder, i.e., the item's content has been written to disk.
For me this sounds like the file content (not the metadata) was written to disk.
However there is also PRJ_FILE_STATE_FULL which also sounds like the complete data (metadata + file content) is cached.
I would asume that PRJ_FILE_STATE_HYDRATED_PLACEHOLDER actually means
[...] the item's metadata has been written to dist.
Can someone confirm this?
I found another part in the documentation that was more detailed:
Hydrated placeholder
For files: The file’s content and metadata have been cached to the disk. Also referred to as a "partial file".
For directories: A directory that was created on disk as a placeholder never becomes a hydrated placeholder directory. This allows the provider to add or remove items from the directory in its backing store and have those changes be reflected in the local cache.
Related
I am using sensenet and react based client for front end. while copying a file from one document library to another I am getting the following error:
"Cannot copy a list item into an another list".
Can anybody tell me how i can solve this?
This behavior is currently by design. The reason is that content lists (doc lib, task list, etc.) may contain local list fields. If you had a document in a list with a custom metadata field filled with a value, you would loose that value if you copied the document to another list.
Workaround 1
If you do not need the list/library functionality (custom metadata fields, etc.) than store documents in a simple folder instead of a list. This will let you copy those documents wherever you want - even into a list. In this case you have to take care of setting the allowed child types (most likely File) somewhere on the parent chain (e.g. on the workspace), because you cannot set this value on simple folders.
Workaround 2
Copy files using a temp folder. It is allowed to copy a file from a list to a temp folder, and also copy a file from a folder into a list. I know, this is not very convenient and we are considering changing this behavior to make it more permissive, but this is how currently works.
I have two client programs which are using S3 to communicate some information. That information is a list of files.
Let's call the clients the "uploader" and "downloader":
The uploader does something like this:
upload file A
upload file B
upload file C
upload a SUCCESS marker file
The downloader does something lie this:
check for SUCCESS marker
if found, download A, B, C.
else, get data from somewhere else
and both of these programs are being run periodically. The uploader will populate a new directory when it is done, and the downloader will try to get the latest versions of A,B,C available.
Hopefully the intent is clear — I don't want the downloader to see a partial view, but rather get all of A,B,C or skip that directory.
However, I don't think that works, as written. Thanks to eventual consistency, the uploader's PUTs could be reordered into:
upload file B
upload a SUCCESS marker file
upload file A
...
And at this moment, the downloader might run, see the SUCCESS marker, and assume the directory is populated (which it is not).
So what's the right approach, here?
One idea is for the uploader to first upload A,B,C, then repeatedly check that the files are stored, and only after it sees all of them, then finally write the SUCCESS marker.
Would that work?
Stumbled upon similar issue in my project.
If the intention is to guarantee cross-file consistency (between files A,B,C) the only possible solution (purely within s3) is:
1) to put them as NEW objects
2) do not explicitly check for existence using HEAD or GET request prior to the put.
These two constraints above are required for fully consistent read-after-write behavior (https://aws.amazon.com/about-aws/whats-new/2015/08/amazon-s3-introduces-new-usability-enhancements/)
Each time you update the files, you need to generate a unique prefix (folder) name and put this name into your marker file (the manifest) which you are going to UPDATE.
The manifest will have a stable name but will be eventually consistent. Some clients may get the old version and some may get the new one.
The old manifest will point to the old “folder” and the new one will point the new “folder”. Thus each client will read only old files or only new files but never mixed, so cross file consistency will be achieved. Still different clients may end up having different versions. If the clients keep pulling the manifest and getting updated on change, they will eventually become consistent too.
Possible solution for client inconsistency is to move manifest meta data out of s3 into a consistent database (such as dynamo db)
A few obvious caveats with pure s3 approach:
1) requires full set of files to be uploaded each time (incremental updates are not possible)
2) needs eventual cleanup of old obsolete folders
3) clients need to keep pulling manifest to get updated
4) clients may be inconsistent between each other
It is possible to do this single copies in S3. Each file (A B C) will have prepended to it a unique hash or version code [e.g. md5sum generated from the concatenation of all three files.]
In addition the hash value will be uploaded to the bucket as well into a separate object.
When consuming the files, first read the hash file and compare to the last hash successfully consumed. If changed, then read the files and check the hash value within each. If they all match, the data is valid and may be used. If not, the downloaded files should be disgarded and downloaded again (after a suitable delay)..
This will catch the occassional race condition between write and read across multiple objects.
This works because the hash is repeated in all objects. The hash file is actually optional, serving as a low-cost and fast short cut for determining if the data is updated.
I'm storing a MP4 file in CoreData as an "Allows External Storage" NSData attribute. How can I get a path string or NSURL to this data? Also, what happens if the mp4 file in under 1 MB and gets stored internally? Will it then not be able to provide a path? Is there any easy way to tell whether or not the data was stored externally or internally?
Core Data doesn't provide any way to get the path. As you note, it doesn't even guarantee that there is a path, since it might not have used an external file. It's designed as something where you don't need to know whether a path exists or what it might be, and the API reflects that attitude.
Unofficially it's possible to locate the files. If your data store is called Foo.sqlite, the external files (if any) will be in a subdirectory in the same folder named .Foo_SUPPORT/_EXTERNAL_DATA/. The actual file names are just UUIDs though, so figuring out which one goes with which managed object is still not automatic. The best you could do would probably be something like tracking the modification date of a managed object and then looking for a file with the same mod date.
If you really need to get the file for some reason, you'll need to roll your own external reference code. This is pretty simple, really.
My iOS Application has been in review, but was rejected regarding the iOS Data Storage Guidelines. In the end, it was rejected because my Core Data database (sqlite) was located in the /Documents folder. I'm was aware, that this folder should only be used, it the data could not be recreated by my application. The reason why I chose to put it there anyway was, that one of the entities in my database contains an attribute telling if the given news has been read. This information cannot be recreated. Is this not enough to put the database in the /Documents folder?
Another thing is, that the current version of my application does not use this value to visualize if the news item has been read or not. So, should I tell the review-team about this attribute and argument why I think it should be placed in the document-folder -- or should I just move it to the /Library/Cache/?
The app review team wants you to split your data apart. Store the re-creatable parts in the Cache folder and the stuff that can't be re-created in the Documents folder. It's okay if there's a little bit of stuff in Documents that could theoretically be re-created—nobody will even notice a title or datestamp—but long text documents, video, audio, or images should be kept in the Cache folder if they can be downloaded again later.
There are a couple different ways you could do this:
Store the downloaded content in the Cache folder and only put the content's filename in your Core Data database (or calculate the filename from something else, like the SHA-1 hash of the URL it was downloaded from). Make sure your code will re-download any content that's not in the cache.
Use two Core Data stores with a single store coordinator. Note that you can't split an entity's attributes across two stores, so you may have to break some of your entities in half. Nor can you create a relationship from an object in one store to an object in another, so you'll have to store the object ID URI instead. See "Cross-Store Relationships" in the "Relationships and Fetched Properties" section of the Core Data Programming Guide for more details.
Whatever you do, keep in mind that iOS may purge your Cache folder at any time. Be prepared for files in your Cache folder to have disappeared; if that happens, you should re-download the files as the user requests them.
I'm currently working out the best structure for a document I'm trying to create. The document is basically a core data document that uses sqlite as its store, but uses the Apple provided NSPersistentDocument+FileWrapperSupport to enable file wrapper support.
The document makes heavy use of media, such as images, videos, audio files, etc. with potentially 1000s of files. So what I'm trying to do is create a structure similar to the following:
/myfile.ext/
/myfile.ext/store.sqlite
/myfile.ext/content/
/myfile.ext/content/images/*
/myfile.ext/content/videos/*
/myfile.ext/content/audio/*
Now, first of all I went down the route of creating a temporary directory and placing all of my media in there. Basically creating the paths and file names '/content/images/image1.jpg' as I wanted them to appear in the saved file wrapper, and then upon save I attempted to copy these all into the filewrapper...
What I found was that the files were indeed copied into the wrapper with the file structure I wanted, but when the actual wrapper was saved, these files all magically disappeared.
Great.
So, I trashed my existing solution and tried to use file wrappers instead. This solution involved creating a content wrapper file directory when a new document was created, or loading in a content directory file wrapper upon opening a document.
When an image was added/modified, I created the necessary directory wrappers inside this root content wrapper (i.e. an images directory wrapper if it didn't already exist, or any other intermediary directory wrappers that needed to be created) and then created a regular file wrapper for the media, removing any existing wrapper for that file name if one was there.
Saving the document was just a case of making sure the content file wrapper was added to the document file wrapper, and the document would save.
Well... it did. The first time. However, any attempts to make any subsequent changes i.e add an image, save. Then replace image, save. Did not behave as expected, only showing the image from the first save.
So, my question is... first of all, which of the above approaches is the correct one, if at all, and what am I doing that wrong for them to fail.
And secondly, as I expect to be managing 1000s of images, is using file wrappers the correct way to go about things at all.
With that much media in play, you should likely give your users control over whether the media resides in the document or only a reference to the media is included in the document, and the media resides elsewhere, such as in a library/repository managed by your application. Then they could save out a (potentially many times larger) copy with all references resolved.
You might want to zip/unzip any directory so that users don't get confused trying to attach the document to an email. I believe iWork has been doing this with its document bundles for a while now.
As far as what you are doing wrong, no-one can say, as you haven't provided any code demonstrating what you are doing.
Why don't you create a one-off application that lets you select files on disk and saves those files in a document using a file wrapper? This would let you tackle this functionality without any interference from other issues in your application. Once you understand how to use file wrappers, you can port the code back or just write new code that works.