I have multiple folders (4000+) with files included in every folder.
What is the Regular expression to select the files in all the folders in Pentaho "Get a file with SFTP" step.
I am afraid you cannot put wildcard in the directory name, nor to make a recursive search (in which case you could write the wildcard in the path name of the files).
You may find alternative in this other question.
Mind that PDI uses Apache's VFS almost everywhere. So you probably do not need the SFTP job entry. You may Copy File or Move File with their uri (spft://path/to/file) instead of their name. With the same trick you may also start the transformation Input File without copying it before.
Related
How do I setup inputs.conf in splunk to parse only JSON files found on multiple directories? I could define a single sourcetype (KV_MODE=json) in props.conf but not sure about the code in inputs.conf.
Currently, I have the file with multiple stanzas that would each specify the application log path having json files. Each stanza has a sourcetype defined in props.conf to point to json KV_mode. I would like to minimize the steps and consolidate into a single stanza if possible.
Each monitor stanza in Splunk monitors a single file path, although that path can contain wildcards. You could do something like [monitor:///.../*.json] to monitor any file anywhere with a json extension, but that would consume a crazy amount of resources.
You're better off with a separate stanza for each directory that contains JSON data. Maybe you can use wildcards to condense to a few entries.
All of them, however, can use the same sourcetype so there's no need to touch props.conf to monitor a new file path.
I am developing a zip extractor app for which if i unzip multiple times the same zip file it should extract like myfile-1, myfile-2, myfile-3 something like this .
example : there is sampleproject.zip in my desktop when i unzip it should be like sampleproject, sampleproject-1, sampleproject-2.
Any Suggestions.
Thanks in Advance!
Based on your comment I suggest you unzip your file to a temporary directory and then move its contents into the actual directory, handling any name clashes as you do that. In outline:
Use URLForDirectory:inDomain:appropriateForURL:create:error: to create a temporary directory suitable to unzip into. You should pass your the URL of your destinationPath for the appropriateForURL: parameter; this should give you a temporary directory on the same volume as destinationPath making placing the unzipped items into the right place moves rather than copies.
Unzip into the temporary directory returned by (1)
Now use NSFileManager calls to traverse the temporary directory moving each item found to destinationPath, renaming as needed to avoid name clashes.
Remove the temporary directory.
If you have problems implementing this algorithm ask a new question, show your code, explain your problem, and include a link back to this question so the thread can be followed. Someone will then undoubtedly help you with the next step.
HTH
Is it possible to get the actual file, or the file that gets copied from version control to a location?
This sounds confusing. Basically I have the file path of the version controlled file, but I need an actual path to the file because I need to make a cconsole command using powershell.exe. The file will look something like this
$/MyTeamProject/MyProject/Development/MyPowershellScript.ps1
Now, I am looking for a vb expression to see if I can get the actual file and make call the powershell.exe command from console. Any thoughts?
You may use VersionControlServer.GetItem(String path) to obtain a reference to the Item. Then use Item.DownloadFile() or Item.DownloadFile(String localPath) to copy the file locally. I have a variation of this that creates a shipment based on multiple changesets.
Matlab Central file exchange contains numerous helper scripts and functions. Is it possible to place them somewhere in order not to populate current (project) directory with helper files?
There is set of functionality that concerns itself with search path (documentation).
What Is the MATLAB Search Path?
The search path, or path is a subset of all the folders in the file system. MATLAB® software uses the search path to efficiently locate files used with MathWorks® products. MATLAB can access all files in the folders on the search path.
Add Folders to Search Path Upon Startup
The startup.m file is for specifying startup options. You can add folders to the search path by including addpath statements in startup.m. For example, to add the specified folders, /home/username/mytools to the search path, include this statement:
addpath /home/username/mytools
I'd like to list all files from a remote folder (let's say www.mysite.com/folder, and this folder is already configured through .htaccess for directory listing).
After listing, i'll need to copy the remote files to a local folder.
For listing/copying only local files, I was using NSFileManager, but this doesn't work for the remote ones. I've been looking for some reference on it, but couldn't find so far...
While NSFileManager can in fact handle URLs, it's not going to download the apache HTML page with the directory listing and parse it to do this... you'll have to do that yourself. This sounds like a strange thing to be doing however, so you may want to explain the reasoning and we may be able to suggest better alternatives. WebDAV comes to mind.
UPDATE: Based on your comment, why not put the resources in a .zip (or similar) file and download that? Then it's a single download and you can just extract it locally. Sounds like it would save a lot of headaches and would make it much easier to do things like checksum validations on the download(s).
Maybe it's not the best way, but - instead of get directory listing - we're going to keep a list of files that should be transfered (could be a .txt or .xml).
For downloading and tracking multiple requests, we're going to use ASINetworkQueues (more details can be found on http://allseeing-i.com/ASIHTTPRequest).
Another good suggestion, given by d11wqt (thank you for your help), is compressing the files and just make one single request.