I want something that looks like a file handle but is really backed by an in-memory buffer to use for I/O redirects. How can I do this?
I just wrote a library which provides this, called "knob" [hackage]. You can use it to create Handles which reference/modify a ByteString:
import Data.ByteString (pack)
import Data.Knob
import System.IO
main = do
knob <- newKnob (pack [])
h <- newFileHandle knob "test.txt" WriteMode
hPutStrLn h "Hello world!"
hClose h
bytes <- Data.Knob.getContents knob
putStrLn ("Wrote bytes: " ++ show bytes)
If you can express what you want to do in terms of C or system calls you could use Haskell's Foreign Function Interface (FFI). I started to suggest using mmap, but on second thought I think mmap might be a mapping the wrong way even if you used it with the anonymous option.
You can find more information about the Haskell FFI at the haskell.org wiki.
This is actually a bug in the library design, and one that's annoyed me, too. I see two approaches to doing what you want, neither of which is terribly attractive.
Create a new typeclass, make the current handle an instance of it, write another instance to do the in-memory-data thing, and change all of your programs that need to use this facility. Possibly this is as simple as importing System.SIO (or whatever you want to call it) instead of System.IO. But if you use the custom I/O routines in libraries such as Data.ByteString, there's more work to be done there.
Rewrite the I/O libraries to extend them to support this. Not trivial, and a lot of work, but it wouldn't be particularly difficult work to do. However, then you've got a compatibility issue with systems that don't have this library.
This may not be possible. GHC, at least, seems to require a handle to have an OS file descriptor that is used for all read/write/seek operations.
See /libraries/base/IOBase.lhs from the GHC sources.
You may be able to get the same effect by enlisting the OS's help: create a temporary file, connect the handle to it and then memory map the file for the I/O redirects. This way, all the handle I/O would become visible in the memory mapped section.
To add a modern answer to this question, you could use createPipe from System.Process:
createPipe :: IO (Handle, Handle)
https://www.stackage.org/haddock/lts-10.3/process-1.6.1.0/System-Process.html#v:createPipe
It's not possible without modifying the compiler. This is because Handle is an abstract data type, not a typeclass.
Related
I'm looking for a way to "enter" a module in the REPL, so that I can access all symbols without qualification (not just the exported ones), and any function (re)defined at the REPL gets in the specified module. (Basically this is the functionality of Common Lisp's in-package macro.)
This would be useful in a REPL-oriented workflow, as I would be able to write the same code in the REPL as in the module I am developing.
The manual recommends a workflow where I qualify everything, but that seems annoying.
I started a package called REPLMods.jl for this a while back. It should probably be polished up, but I haven't had the time.
I spoke to core Julia members and there was interest in getting it merged into base once things were clean, but again, no time!
I know this isn't quite what you're asking, but just in case the 'obvious' had not occured to you (or future visitors to the question), assuming you loaded a module with an annoyingly cumbersome name, e.g.
import LaTeXStrings
and you don't want to have to type LaTeXStrings all the time just to explore its accessibles, i.e.
LaTeXStrings.[TAB]
you can just assign the imported module as a whole to another variable, i.e.
const l = LaTeXStrings
I'm sure in the absence of a more appropriate built-in solution, at least typing l.[TAB] as opposed to LaTeXStrings.[TAB]is a lot more tolerable :)
(I find it odd, in fact, that julia doesn't seem to support the import LaTeXStrings as l syntax ...)
It's 2020, I'm using Julia 1.4, and was unable to get REPLMods.jl to work. I think the following seem good enough for the time being:
ExportAll.jl - see Exporting all symbols in Julia for a discussion (just that one shouldn't ExportAll to replace normal export)
and Revise.jl
I'm trying to write a pam backdoor scanner, which may call fopen function in pam_sm_authenticate(normal file will not call fopen in this function) to store username and password, but I can't use external command such as "nm, readelf" or something like that, so the only way seems to scan pam_sm_authenticate function and find all call instructions and caculate the address to check if it is calling fopen, but it is too troublesome and i'm not very familiar with ELF file(I even dont know how to find offset of pam_sm_authenticate, I'm useing dlopen and dlsym to get the address..), so I wonder if there is a better or easy way to detect it? Thankyou.
TL;DR: building a robust "pam backdoor scanner" is theoretically impossible, so you should give up now and think about other ways to solve your problem.
Your question is very confusing, but I think what you are asking is: "can I determine programmatically whether pam_sm_authenticate calls fopen".
That is the wrong question to ask, for several reasons:
if pam_sm_authenticate calls foo, and foo calls fopen, then you still have a problem, so you really should be scanning pam_sm_authenticate and every function it calls (recursively),
the fopen is far from the only way to write files: you could also use open, or system (as in system("echo $secret > /tmp/backdoor"), or direct sys_open syscall, or a multitude of other hacks.
finally, the pam_sm_authenticate can use just-in-time compilation techniques to build arbitrary code (including code calling fopen) at runtime, and answering whether it does by examining its code is equivalent to solving the halting problem (i.e. impossible).
Would it be possible to take the source code from a SNES emulator (or any other game system emulator for that matter) and a game ROM for the system, and somehow create a single self-contained executable that lets you play that particular ROM without needing either the individual rom or the emulator itself to play? Would it be difficult, assuming you've already got the rom and the emulator source code to work with?
It shouldn't be too difficult if you have the emulator source code. You can use a method that is often used to store images in c source files.
Basically, what you need to do is create a char * variable in a header file, and store the contents of the rom file in that variable. You may want to write a script to automate this for you.
Then, you will need to alter the source code so that instead of reading the rom in from a file, it uses the in memory version of the rom, stored in your variable and included from your header file.
It may require a little bit of work if you need to emulate file pointers and such, or you may be lucky and find that the rom loading function just loads the whole file in at once. In this case it would probably be as simple as replacing the file load function with a function to return your pointer.
However, be careful for licensing issues. If the emulator is licensed under the GPL, you may not be legally allowed to store a proprietary file in the executable, so it would be worth checking that, especially before you release / distribute it (if you plan to do so).
Yes, more than possible, been done many times. Google: static binary translation. Graham Toal has a good howto paper on the subject, should show up early in the hits. There may be some code out there I may have left some code out there.
Completely removing the rom may be a bit more work than you think, but not using an emulator, definitely possible. Actually, both requirements are possible and you may be surprised how many of the handheld console games or set top box games are translated and not emulated. Esp platforms like those from Nintendo where there isnt enough processing power to emulate in real time.
You need a good emulator as a reference and/or write your own emulator as a reference. Then you need to write a disassembler, then you have that disassembler generate C code (please dont try to translate directly to another target, I made that mistake once, C is portable and the compilers will take care of a lot of dead code elimination for you). So an instruction of a make believe instruction set might be:
add r0,r0,#2
And that may translate into:
//add r0,r0,#2
r0=r0+2;
do_zflag(r0);
do_nflag(r0);
It looks like the SNES is related to the 6502 which is what Asteroids used, which is the translation I have been working on off and on for a while now as a hobby. The emulator you are using is probably written and tuned for runtime performance and may be difficult at best to use as a reference and to check in lock step with the translated code. The 6502 is nice because compared to say the z80 there really are not that many instructions. As with any variable word length instruction set the disassembler is your first big hurdle. Do not think linearly, think execution order, think like an emulator, you cannot linearly translate instructions from zero to N or N down to zero. You have to follow all the possible execution paths, marking bytes in the rom as being the first byte of an instruction, and not the first byte of an instruction. Some bytes you can decode as data and if you choose mark those, otherwise assume all other bytes are data or fill. Figuring out what to do with this data to get rid of the rom is the problem with getting rid of the rom. Some code addresses data directly others use register indirect meaning at translation time you have no idea where that data is or how much of it there is. Once you have marked all the starting bytes for instructions then it is a trivial task to walk the rom from zero to N disassembling and or translating.
Good luck, enjoy, it is well worth the experience.
I found the Wikipedia entry on the soft coding anti-pattern terse and confusing. So what is soft coding? In what settings is it a bad practice (anti-pattern)? Also, when could it be considered beneficial, and if so, how should it be implemented?
Short answer: Going to extremes to avoid Hard Coding and ending up with some monster convoluted abstraction layer to maintain that is worse than if the hard coded values had been there from the start. i.e. over engineering.
Like:
SpecialFileClass file = new SpecialFileClass( 200 ); // hard coded
SpecialFileClass file = new SpecialFileClass( DBConfig.Start().GetConnection().LookupValue("MaxBufferSizeOfSpecialFile").GetValue());
The main point of the Daily WTF article on soft coding is that because of premature optimization and fear a system that is very well defined and there is no duplicated knowledge is altered and becomes more complex without any need.
The main thing that you should keep in mind is if your changes actually improve your system and avoid to lightly label something as anti-pattern and avoid it by all means. Configuring your system and avoiding hardcoding is a simple cure for duplicated knowledge in your system (see point 11 : "DRY Don't Repeat Yourself" in The Pragmatic Programmer Quick Reference Guide) This is the driving need behind the suggestion of avoiding hardcoding. I.e. there should be ideally only one place in you system (that would be code or configuration) that should be altered if you have to change something as simple as an error message.
Ola, a good example of a real project that has the concept of softcoding built in to it is the Django project. Their settings.py file abstracts certain data settings so that you can make the changes there instead of embedding them within your code. You can also add values to that file if necessary and use them where necessary.
http://docs.djangoproject.com/en/dev/topics/settings/
Example:
This could be a snippet from the settings.py file:
num_rows = 20
Then within one of your files you could access that value:
from django.conf import settings
...
for x in xrange(settings.num_rows):
...
Soft-coding: it is process of inserting values from external source into computer program. like insert values through keyboard, command line interface. Soft-coding considered as good programming practice because developers can easily modify programs.
Hard-coding. Assign values to program during writing source code and make executable file of program.Now, it is very difficult process to change or modify the program source code values. like in block-chain technology, genesis block is hard-code that cannot changed or modified.
The ultimate in softcoding:
const float pi = 3.1415; // Don't want to hardcode this everywhere in case we ever need to ship to Indiana.
I'm writing a Linux kernel module that needs to open and read files. What's the best way to accomplish that?
Can I ask why are you trying to open a file?
I like to follow Linux development (out of curiosity, I'm not a kernel developer, I do Java), and I've seen discussion of this question before. I was able to find a LKML message about this, basically mentioning it's usually a bad idea. I'm almost positive that LWN covered it in the last year, but I'm having trouble finding the article.
If this is a private module (like for some custom hardware and the module won't be distributed) then you can do this, but I'm under the impression that if you are going to submit your code to the mainline then it may not be accepted.
Evan Teran mentioned sysfs, which seems like a good idea to me. If you really need to do harder custom stuff you could always make new ioctrls.
EDIT:
OK, I found the article I was looking for, it's from Linux Journal. It explains why doing this kind of stuff is generally a bad idea, then goes on to tell you exactly how to do it anyway.
assuming you can get pointers to the relavent function pointers to the open/read/close system calls, you can do something like this:
mm_segment_t fs = get_fs();
set_fs(KERNEL_DS);
fd = (*syscall_open)(file, flags, mode);
if(fd != -1) {
(*syscall_read)(fd, buf, size);
(*syscall_close)(fd);
}
set_fs(fs);
you will need to create the "syscall_*" function pointers I have shown though. I am sure there is a better way, but I believe that this would work.
Generally speaking, if you need to read/write files from a kernel module, you're doing something wrong architecturally.
There exist mechanisms (netlink for example - or just register a character device) to allow a kernel module to talk to a userspace helper process. That userspace helper process can do whatever it wants.
You could also implement a system call (or such like) to take a file descriptor opened in userspace and read/write it from the kernel.
This would probably be neater than trying to open files in kernel space.
There are some other things which already open files from kernel space, you could look at them (the loop driver springs to mind?).
/proc filesystem is also good for private use, and it's easy.
http://www.linuxtopia.org/online_books/Linux_Kernel_Module_Programming_Guide/x773.html
All of the kernel developers say that file I/O from kernel space is bad (especially if you're referring to these files by their paths) but the mainstream kernel does this when you load firmware. If you just need to read from files, use the
kernel_read_file_from_path(const char *path, void **buf, loff_t *size, loff_t max_size, enum kernel_read_file_id id)
function, which is what the firmware loader code uses, declared in include/linux/fs.h. This function returns a negative value on error.
I'm not really sure of the point of the id variable at the end, if you look at the code it's not really used, so just put something like READING_FIRMWARE there (no quotes).
buf is not null terminated, instead refer to its size in size. If you need it to be null terminated, create a string size + 1 bytes long and copy it over or rewrite the kernel_read_file() function (used by kernel_read_file_from_path(), defined in fs/exec.c) and add one to i_size where memory is allocated. (If you want to do this, you can redefine the kernel_read_file() function in your module with a different function name to avoid modifying the whole kernel.)
If you need to write to files, there is a kernel_write() function (analogous to kernel_read(), which is used by kernel_read_file() and therefore also by kernel_read_file_from_path()), but there is no kernel_write_file() or kernel_write_file_from_path() function. You can look at the code in the fs/exec.c file in the Linux kernel source tree where kernel_read_file() and kernel_read_file_from_path() are defined to write your own kernel_write_file() and kernel_write_file_from_path() functions that you can include in your module.
And as always, you can store a file's contents in a char pointer instead of a void pointer with this function by casting it.
You can also find some informations about sys_call_open in this Linux Kernel Module Programing Guide.