I am doing some sort of science with a combination of Python, bash scripts and configuration files with weird syntaxes from old C programs. To run different tests, I have to tune (open text file, edit and save) a number of these files, but it is getting harder and harder to remember which files to edit. Also, the tasks are very repetitive and I end having a bunch of comment lines, switching on and off the settings for a specific run.
I dream of a tool that can keep a list of files (regardless of the file format) and allow me to centrally choose from a predefined list of "profiles", taking care of editing the script/config files. For example lets say I have a config and script file:
config.cfg
var1 = 1.0
#var1 = 1.5
script.sct
function(200.0)
#function(300.0)
I would like to instead have auxiliary text files:
config.cfg.tem
var1 = $$var1$$
script.sct.tem
function($$par1$$)
and in a central script or even better, a GUI, just switch from profile1 (e.g. var1=1.0,par1=200.0) to profile2 (e.g. var1=1.5,par1=300.0), and the tool to automatically update the text files.
Any idea if such a tools exists?
There are many, many processors which do this. Looking for a "templating system" will yield tons of elaborate tools.
OTOH, a very straight forward and simple way would be to use the C preprocessor. Define a settings include file were you #define your settings and then automatically include it when you process your templates.
$ cpp -include definitions_file input_template output_file
This is simple and has its shortcomings but works often good enough.
Related
Background: I am working with Angular (but my problem is not particular to any language or framework). In Angular, each component requires four separate files. So, we often find ourselves with 40+ files open. But, most of these files can be tiny, less than 20 lines each.
Many IDEs allow you to open your files in multiple windows. Each window can have a different panel, and each panel can have different tabs. This is great, but honestly, still isn't enough.
What I want: In addition to windows, panels, and tabs, I'd like to add another level of organization.
I speculate this has probably existed for decades, but I just don't know what it's called. At the very least, I speculate this has existed at least since Angular was a thing.
For example, here is a screenshot of VSCode with four files open across four panels. (Code taken from Angular dynamic component tutorial):
And here is a quick mockup showing what I'm looking for. Four files are open, but the three shortest ones are "concatenated" into one editor. Arrow-key down from the bottom of one file will bring you to the first line of the next file.
Notably, these files are not actually concatenated on-disk.
TLDR: What text editor can allow me to edit multiple files as if they were concatenated, as in the mockup above?
If the files stay as separate windows/tabs, the file editor would have to shrink each tab to a minimal height, and then tile them vertically. If any editor can do it, I suspect it would be emacs or vim. You might also be able to do it by opening separate editor windows and using a tiling window manager.
We can achieve a similar effect with some text editing magic. It would be something like:
Add a header to each file consisting of a unique separator (e.g. # === magic separator === filename my_file.js ===)
Use cat to combine all the files into one file
Edit this one file
When done, use the separator to break them up and put the text back into the original files
You could easily write some scripts for combining and splitting so you can do it quickly. You can also set up a background script that automatically runs the splitter as you edit the combined file. However, the combined file would essentially be a new file, so you could not view changes on it with git, and VS Code's CodeLens/Inline blame wouldn't work.
One option would be to develop your codebase with the combined files checked in to VCS, and then only have the splitter script as part of your "build" step. So you would make your changes, run ./build.sh which splits the files into some temp directory, and then run your application from there.
Lastly, and I hate to be snide, but the fact is that this problem is best solved by avoiding poorly designed frameworks that do not consider developer ergonomics. Many other languages give the developer much freedom and many tools to organize their code as they wish, rather than imposing constraints like requiring many small components to be in separate files. Java for example also had a similar problem (dunno if more recent versions fixed it) - you can only have one class per file, which creates a huge mess if you like having many small files. C# does not have this limitation and as a result C# codebases can be much tidier than Java codebases.
I'm trying to get a Cocoa development environment working in Emacs, and I'm 80% of the way there. The one feature I miss is Xcode's "Open Quickly", which basically performs a fuzzy match of the string you type against the filenames referenced in the Xcode workspace and the symbols defined in those files.
My problem is that our project is huge: if I generate a TAGS file using etags for the .h and .m files in our project's sub-directories, the result is over a gig in size and Emacs complains "TAGS file is large. Really open?", and if I say yes, then Emacs hangs and becomes essentially unusable. Of course, this is before I've even considered indexing tags for system libraries. I've also tried projectile, but unfortunately it's similarly unusable on a project of my size (on the order of a full minute to find a match).
It occurs to me that all the indexing information I really want is in the Xcode projects themselves, so if I had an Emacs package that could parse them and traverse their dependencies, that might be a start, but I'm not aware of any such package.
Any suggestions/solutions in this respect?
I've never found a single function quite as convenient as Xcode's "Open Quickly", but these days I use
helm-projectile-git-grep when I want to match on strings I know to be in the filenames, and
helm-git-grep for quick searches through the contents of the files themselves.
I've found that this gets me really close to what I wanted in my original question.
I recently recovered a 1.5TB external HDD that crashed. The program I used to recover the files was Active Undelete Enterprise, it's excellent. When the files were successfully recovered they were all saved with a .efs extension so files looked like mydocument.docx.efs. At first I thought they were encrypted and needed to be decrypted, I spent 10 mins on it and realized I just need to remove the .efs from the entire filename and the mydocument.docx works perfectly. Problem is now I have over 55,000 files within hundreds of folders where I need to simply remove the .efs after each file. Does anyone know how to do this?
From a command prompt window, navigate to the top level directory where these files reside.
Type the command
DIR /S/B >>filelist.txt
This command will give you a bare format file listing of the current directory plus all nested subdirectories without any extraneous information. The list will be contained in the text file named "filelist.txt" or whatever else you choose to call it. I would then use this text file in a text editor to convert every line of text from, for example,
C:\Users\dlucas\.gimp-2.8\mathmap\file1.png.efs
to
rename c:\Users\dlucas\.gimp-2.8\mathmap\file1.png.efs file1.png
to give a simple example of a file that I just found on my system using this method.
You will need to use a text editor with a columnar editing capability since you have to modify som many files. Old programmer's editors such as CodeWright made this really simple while modern editors such as Eclipse or Notepad++ make this a little more difficult and may require a columnar editing plugin, depending on version. You basically have to make a columnar copy of all of the text in the file, and then paste the copy off to the far right - far enough that a second column of filenames and paths won't overwrite any of the existing file names and paths. You can then use columnar editing features to select and delete the path names of the text in the 2nd column since the rename command requires that the 2nd argument be simply the base filename and extension without the path information. You can use the columnar editing features to prepend every line with "RENAME ". If you attempt to do this without columnar editing features, you will find it slow going!
An alternate way to do this is to use a command formed from a "regular expression" to create the rename command. If you are not familiar with "regular expressions", ask a programmer friend as this is not an easy topic to learn from scratch. If you are familiar with regular expressions, this is probably the simplest way to perform this task. I haven't used them in many years and no longer recall the exact syntax to use or I would tell you myself.
Regardless of what kind of editor you use, the goal is to turn this ASCII file list of paths and filenames into a batch file (simply rename file1.txt to file1.bat when you are finished editing). You can then run the batch file by typing file1.bat at a command prompt.
I have just run into this same problem myself using the same really wonderful tool that you used. I am writing this while waiting for the undelete program to finish. That it restores files with this extra extension seems very anti-intuitive so I will look for an option to make it not do this when it finishes. If I find one, I will post a new answer here that is more specific to this tool. Otherwise, I am going to have rename all kazillion files just as you had to.
You experienced this problem because the disk that you recovered your files to "does not support encryption", according to the Active# UNDELETE documentation. The documentation offers no further explanation of what kind of disks support encryption, etc.
They offer a Decrypt command that restores the file's proper names as a post processing step. Unfortunately, this requires that you "include" each and every file to be decrypted, with no support for wildcards and parsing subdirectories so that is a non-starter, in my opinion given that both of us have hundreds of thousands of files to be renamed.
I did find that by selecting a normal fixed (non-removable) hard drive as the destination of the recovery effort, that the resulting files do not end up encrypted (i.e., they are recovered with the proper file name and extension). I originally chose a large USB based flash drive and the files were stored in their "encrypted" state (not really encrypted, but possibly potentially so and thus they give the .efs extension). Of course, this meant that I had to run the command all over again after switching to a regular hard drive (takes about 16 hours to recover 80GB worth of files due to presence of many sector CRC errors).
I've seen GDIFF(Generic Diff Format) in wikipedia, and I wander is there any command line tool implements this standard. Now the best I have is LibXDiff, but it's a library, I'll need some extra work to make it run.
I know when it comes to binary-differ, VCDIFF(xdelta, etc) and bsdiff would have better compression rate, but in my case I really need a straight forward one. VCDIFF copies anything before current window(if my poor English reading was right about this article), and bsdiff's patch file format would be more complex.
update
Finally I found VCDIFF with xdelta3 is actually good and working, when "disable small string-matching" and "disable external decompression" is toggled, AND it has a pretty good "printdelta" command that prints very useful(for my app) information so that I don't really neet to extract VCDIFF format from the patch file.
Javaxdelta library implements xdelta and GDIFF patches. It can be used as command-line application like this:
# create patch
java -cp javaxdelta-2.0.1.jar:trove-1.0.2.jar com.nothome.delta.Delta source.file target.file patch.gdiff
# apply patch
java -cp javaxdelta-2.0.1.jar:trove-1.0.2.jar com.nothome.delta.GDiffPatcher unpatched.file patch.gdiff patched.file
I wrote once a wrapper around it to support directories patching (GDIFF files for directory are packed into one ZIP patch).
I currently have a directory with 98,000 individual archive transaction files. I need to search those files for user input strings and have the option to open the files as it finds them or at the end of the search. I'm using Notepad++ currently and, while functional, it's quite slow. I thought about writing my own, but I am only familiar with .NET and I'm a beginner. Also, I'm not sure how efficient that would be compared to NP++.
This tool would be used again and again so the dev time would definitely be worth it if it came to that. Is there some other tool out there that's already developed that would accomplish this?
Agent Ransack
I've been using it for years.
I recommend you using Astrogrep, a grep utility for Windows. You can open files as it finds them, and it shows you the line where the match was found, without having to open the file.
Assuming the archive transaction files are plain text, you can download CYGWIN which is an environment providing UNIX tools for Windows.
Once that's done, you can open a new Cygwin Bash Shell, then do cd 'c:\\foo' to get into the directory with your files, then do grep -F -r "my string" * to find your text. (The -F means it searches for that literal string as opposed to a regular expression and -r means recursive.)
Possibly overkill, but you could index the folder using Lucene, keep the index uptodate (as transaction files are added) and then searches will take trivial amounts of time, you can target the file, line and word number of each match for a given search string