My raw access logs file is 420 MB and when I try to open, the program says it is too big and can't open it. I tried opening it with notepad, notepad++ and Excel, and none of them could open it. And the file is only for 3 days worth of logs. How can I view the file?
Instead of reading whole file at a same time, I will suggest you some tool to split text file. For Windows file splitter called HJSplit, And for the linux you can use split command for this.
Related
I am using CGI DSPSTMF command to display stmf file on web browser. I am copying a spool file to a stmf file using CPYSPLF *STMF option. Once copied i am passing IFS location to DSPSTMF command but it is going to download automatically and when i open the download file i am getting all Junk data any idea why?
Also, i noticed it is using CONTTYPES file in CGILIB and on my server it is empty. What should be the values in it and what should i do show correct data instead of junk. I tried to use different methods to copy the file to IFS like used cpytostmf instead of cpysplf but on IFS file looks correct not the download version.
What CCSID is the resulting stream file tagged with?
use WRKLNK and option 8=Display attributes
If 65535, that tells the system the data is binary and it won't try to translate the EBCDIC to ASCII.
The correct fix is to properly configure your IBM i so that the stream file is tagged with it's correct CCSID.
Do a WRKSYSVAL QCCSID ... if your system is still set to 65535, that's the start of your problem. But this isn't programming related, you can try posting to Server Fault but you might get better responses on the Midrange mailing list
I have a text file. Now I have changed its file type from .txt to .abc. My VB.NET program loads the text into textboxes from that file. After changing the file type, however, other apps like NotePad and Word are able to open and read my .abc file.
Is there any way that only my application will be able to open/read from the file and no other app would be able to do so? What I mean is, suppose I have a PhotoShop document .psd file, no other app, rather that photoshop itself, can open it. How do I make my file unreadable by other apps?
There is no way to prevent an app that you don't develop from opening any file. The extensions are just there for helping us humans, and maybe a bit for the computer to know the default app you select for an extension.
Like you said, a .txt file can be opened by many many apps. You can open a .txt file with Notepad, Firefox, VSCode, and many others.
Same way, a .psd file can be opened by many many apps. You can open that .psd file with Photoshop, but also Notepad, Firefox, and VSCode, and probably the same apps as above.
The difference is which apps can read and understand the file.
In order to make a file not understandable by other apps, you need to make it into a format that cannot recognize, because you planned it "in secret".
Like Visual Vincent said above, you could encrypt the file in a way, or you can have a binary file, that basically only your app knows know to understand.
Since you dont own the app you want the file to be understood by, then you either have to accept that it can be opened by any app that can open files, or you can try to encrypt the file outside the app, or like zipping it with a password, and then decrypting or unzipping when you want to use it.
Firstly, any file can be read unless it is still open by a particular process or service. Even PhotoShop files can be 'read' by NotePad - try it!
So, an attempt at my first answer...
You can try a couple of methods to prevent opening the file, for instance, applying a file lock. As an example, SQL Server .mdf files are locked by the SQL Server service. This happens because the files are maintained in an open state, however; your application would have to remain running to keep these files open. Technically, though, the files can still be copied.
Another way is to set the hidden attribute for the file. This hides the file from the less savvy users, but it will be displayed if the user show's hidden files.
And my second answer: You refer to the format of files by saying only PhotoShop can read or write its own files (not true, but I know what you're saying).
The format of the file must be decided by yourself. You must determine how you are going to store the data that you output from your application. It looks like you have been attempting to write your application data into a text file. Perhaps you should try writing to binary files instead. Binary files, while not encrypted, as suggested by Visual Vincent in the comments to your question, still provide a more tailored approach to storing your data.
Binary files write raw binary data instead of humanised text. For instance, if you write an integer to the file it will appear as a string of four bytes, not your usual 123456789 textual format.
So, you really need to clarify what data you want to write to the file, decide on a set structure to your file (as you also have to be able to read it back in to your application) and then be able to write the information.
I've been trying to open this data: http://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing?sort=1&file=data%2Fdemo_gind.tsv.gz. I've already unzipped it and get the tsv file, but when I opened it in gedit, it looks like a binary file. Could anybody help me to open this file?
The file is correctly formatted even if not so readable for human beings.
TSV is a file extension for a tab-delimited file used with spreadsheet
software. TSV stands for Tab Separated Values. TSV files are used for
raw data and can be imported into and exported from spreadsheet
software. TSV files are essentially text files, and the raw data can
be viewed by text editors, though they are often used when moving raw
data between spreadsheets.
You can import it inside Excel or Open Office. Otherwise you may convert it by using online service (example google sheets).
Once you've unarchived the original .gz file there are two more steps required to view the data, as noted on Eurostat's website.
TSV files may be imported into Excel by (1) Saving on hard disk with
the suffix .tsv and (2) unzipping and (3) saving the table(s) as Text
(*.txt).
As per user74158's comment, decompress/unzip the tsv file. This can
likely be done with many different programs, I used 7zip and it
worked for me. On windows 7 I did this by right clicking, hovering
over 7zip, selecting extract files, tell 7zip where you'd like to
extract the files too and press OK.
Next go to the file, and change the .tsv file extension to .txt. Answer yes, you're sure you want to change the file extension and then you should be able to read the data.
Trying to write a fairly simple program that reads line by line a file from an ftp server, stores each line in a string, then writes that string to a local file on my drive. Now this is fairly easy and worked perfectly.
This is the part that gets tricky. Every time the ORIGINAL file on the ftp gets updated (a new line is added), I want that line to be added into my LOCAL file created earlier. I do NOT want to recopy the whole file contents if the files are different. I simply want to add the missing lines.
Is there any way this can be done ? Any tips will be appreciated. You don't have to submit a code. Let me know of how this can be done logically and I will try to write the code for it.
Regards
You can't read a file "line by line" from an FTP server via FTP itself.
FTP is short for File Transfer Protocol, which is for transferring files. There's no way to do anything else via that protocol. If you want to access it "line by line", you're going to have to use something other than FTP.
I have a large batch of assorted files, all missing their file extension.
I'm currently using Windows 7 Pro. I am able to "open with" and experiment to determine what application opens these files, and rename manually to suit.
However I would like some method to identify the correct file type (typically PDF, others include JPG, HTML, DOC, XLS and PPT), and batch rename to add the appropriate file extension.
I am able to open some files with notepad and review the first four bytes, which in some cases shows "%PDF".
I figure a small script would be able to inspect these bytes, and rename as appropriate. However not all files give such an easy method. HTML, JPG, DOC etc do not appear to give such an easy identifier.
This Powershell method appears to be close: https://superuser.com/questions/186942/renaming-multiple-file-extensions-based-on-a-condition
Difficulty here is focusing the method to work on file types with no extension; and then what to do with the files that don't have the first four bytes identifier?
Appreciate any help!!
EDIT: Solution using TriD seen here: http://mark0.net/soft-trid-e.html
And recursive method using Powershell to execute TriD here: http://mark0.net/forum/index.php?topic=550.0
You could probably save some time by getting a file utility for Windows (see What is the equivalent to the Linux File command for windows?) and then writing a simple script that maps from file type to extension.
EDIT: Looks like the TriD utility that's mentioned on that page can do what you want out of the box; see the -ae and -ce options)
Use python3.
import os,re
fldrPth = "path/to/folder" # relative to My Documents
os.chdir(fldrPth)
for i in os.listdir():
with open(i,'r') as doc:
st = doc.read(4)
os.rename(i,i+'.'+re.search(r'\w+',st).group())
Hopefully this would work.
I don't have test files to check the code. Take a backup and then run it and let me know if it works.