I read often of an "Image Generation" process in Smalltalk. The process seems to refer creating an image from scratch, from inside a Smalltalk.
But there is also a "Strip" process, which seems to involve removing objects to deploy a runtime.
What is the difference between both? There is any Smalltalk which supports image generation?
Term image generation often refers to process which starts from default vanilla image as shipped with installation, and loading of all code into it that is necessary for some project. This is done periodically during development to ensure that all code actually loads and works in the default image without problems.
Stripping is process that is (sometimes) done before deployment, from the image that contains all necessary code for the project, some of unused classes and methods are "stripped out" from the image. This is done to make deployed image smaller, or less dependent of external shared libraries, or for security reasons, or licensing reasons. For instance stripping might remove many classes related to the UI for the headless server. Or it might remove compiler to prevent user to change the code. In any case stripping is not exact science, since it is difficult to determine what can be removed and what not.
So with image generation you end up with the image that is larger than the one you have started with, and with stripping you end up with smaller image.
Related
Recently I'm reading malware analysis. I'm going through this Malware Repository (https://github.com/ytisf/theZoo). Here we can find malware binaries. Can binaries be in packed form? If so, how can we say that these binaries are packed or not?
PS: Packers compress a program and will try to hide internals from us(sort of compression or encryption). I got a doubt regarding this. Can binaries be in the packed form or not?
Edit2: In this repository, they just zipped it to be safe which is not actual packing I'm talking about. After unzipping, we will get a binary. Whether that can be in packed form or not?
First of all, the distinction you make between "packers" and archiver programs (ZIP, etc) or compression programs doesn't appear to have any basis.
A "packed" executable cannot be executed directly. It must be unpacked first. This is exactly the same as (say) a ZIP file containing malware, or a malware file that has been compressed with a standard compression program.
What about a "packed" executable that has been created by a program that does the "packing" in a secret way ... to evade detection? Well that won't work. The malware still has to be unpacked before it can be executed. So that means that the bad gut now has a second problem: getting the unpacker onto the victims machine. And once someone (an anti-hacker) gets hold of the super-secret unpacker, it is no longer secret. It can be reverse engineered ... or simply used as-is by an AV product on suspicious binary files.
The only practical use of "packing" that I can think of is to add self-unpacking functionality to the malware. The malware (as distributed) would consist of an executable with a small amount of code that implemented the unpacker. The rest of the executable would be packed code that implements the nasty stuff. When the user runs the malware, it would unpack the packed code, load it into memory and start executing.
However, there are potential ways to detect or prevent this kind of thing.
If the unpacker writes the executable code into a file prior to loading it, an AV product could detect that.
If the packer attempts to load code into itself, there are ways that could be blocked; e.g. using memory protection hardware + the OS, etc to stop the unpacker from creating memory segments containing executable code; see https://en.wikipedia.org/wiki/Executable_space_protection.
An AV could look for the signature in the packed code, or it cold look for a signature in the unpacker code.
In short, malware could use some kind of "packing" to hide itself, but there must be an executable component somewhere to unpack it.
If so, how can we say that these binaries are packed or not?
If the malware is distributed as a non-executable you figure out what is going to unpack it, and then see if that process is going to give you an executable.
If the malware is a self-unpacking executable, you reverse engineer the unpacking component to figure out how it works.
Not sure I'm using the right terminology, but if I have a website, and I make a change to one of the files, the underlying logic C#, whatever, do I have compile the whole application again before I deploy it?
How do I publish a small change?
You keep saying "the whole thing", but you don't really seem to understand what that means.
An MVC application is a Web Application project, which means that it's class files are compiled into an assembly. When you make a change to a class, the assembly that it's in must be recompiled (as well as any assemblies that are dependent upon it) and redeployed.
You don't have to recompile other assemblies that have not been changed, and you don't have to redeploy any unchanged files (like views or css or javascript).
If you have 100 pages, chances are, you probably going to split that up into several assemblies anyways. But even if you don't, it really doesn't take very long to compile even a 100 page web application (the pages themselves do not get compiled, it's only the .cs files). At most this should take 15-20 seconds on a medium powered workstation.
I think you're worrying about something that isn't really that big of a deal. You're probably more concerned about redeploying, but if you setup your publish system correctly, this is a single click, 10 second thing.
That depends on where you made the change.
If you made the change in a .cshtml or similar view file, or other content file (like javascript, css, images, etc.), just uploading the updated file will be sufficient.
If you changed a .cs (or vb, etc.) class however, you will need to recompile.
I noticed that after running into an issue last night, relaunching Pharo 3.0 didn't "undo" my working set - everything appeared to be as it was when I closed it. I saw where Fuel is included with Pharo now - does it automatically persist your session? I was under the impression that you had to do some tricks to make it actually work with your application.
Am I wrong?
Pharo uses an Image. The image basically is the snapshot of your memory contents when you use Pharo.
Upon startup this image is loaded from the image-file into memory and Pharo starts to run. The inverse happens when you save (snapshot) your session: the current state/memory is saved to the .image file. That includes all tools opened in the current session, all running processes and all live objects.
This has nothing to do with Fuel, which is a separate object serialization library.
There are two mechanisms in Pharo:
The image. The image is a memory snapshot containing all the objects (and in particular the compiled methods and classes as objects). When you save the image, you are saving the complete state of the system to disk. You can open an image (it loads the memory back and the execution continues where it stopped). In fact there is also another file that is called the change file. This file contains the textual representation of the classes and methods you edited. The tools are using this file to show you method code for example.
Now in addition to the concept of image (memory snapshot). The system records in permanence your code edition. After each compilation phase, the change is committed to the changes file. You can see what you did using the changeSorter or version browser (note that if you do not save your image, your changes will not be browsable using a changesorter because it is a simple tools). Now even if you did not save your image, your changes are logged in the changes file. There is a way to recover your changes using the "Recovery lost changes..." menu item under the Tools menu.
With this tools you can browse all the changes that have been recorded automatically and replay them. We are working on new tools for the future.
Now in general you should not rely on such tools. Using the Pharo distributed version management system (monticello) to create packages and publish them on forges such as SmalltalkHub.
Finally Fuel is an object serializer that is not used for saving Pharo snapshot. Fuel is a fast serializers that people used when they want to select what they serialize - usually graphs of objects.
All this information is also available in the free Pharo books: http://pharobyexample.org
and http://rmod.lille.inria.fr/pbe2/
My application has a plugin system that allows my users to write their own plugins that get loaded at runtime. Usually this is fine but in some cases two plugins use the same libraries that will cause a collision between those two.
Example:
Plugin A wants to use TouchJSON for working with JSON and thus the creator adds the TouchJSON code to the plugin source and it gets compiled and linked into the plugin binary. Later Plugin B also wants to use that same library and does exactly the same. Now when my app loads these two different plugins it detects this and spits out an warning like this:
Class CJSONScanner is implemented in
both [path_to_plugin_a] and
[path_to_plugin_b]. One of the two
will be used. Which one is undefined.
Since my app just loads plugins and makes sure they conform to a certain protocol I have no control over which plugins are loaded and if two or more use the same library.
As long as both plugins use the exact same version of the library this will probably work but as soon as the API changes in one plugin a bunch of problems will arise.
Is there anything I can do about this?
The bundle loading system provides no mean to pacifically resolve name conflicts. In fact, we're told to ensure ourselves that the problem doesn't happen, rather than what to do if it happens. (Obviously, in your case, that's not possible).
You could file a bug report with this issue.
If this is absolutely critical to your application, you may want to have bundles live in separate processes and use some kind of IPC, possibly NSDistantObject, to pass the data from your program to the plugin hosts. However, I'm fairly sure this is a bag of hurt, so if you don't have very clearly-defined interfaces that allow for distribution into different processes, it might be quite an undertaking.
In a single-process model, the only way to deal with this is to ensure that the shared code (more precisely, the shared Objective-C classes) is loaded once. There are two ways to do this:
Put the shared code in a framework.
Put the shared code in a loadable bundle, and load the bundle when the plug-in is loaded if the relevant classes aren’t already available (check using NSClassFromString()). The client code would also have to use NSClassFromString() rather than referring to classes directly.
Of course, if you aren’t in control of the plug-ins you can’t enforce either of these schemes. The best you can do is provide appropriate guidelines and possibly infrastructure; for instance, in the second case the loading could be handled by the application, perhaps by specifying a class to check for and the name of an embedded bundle to load if it isn’t available in the plug-in’s Info.plist.
OK, I'm working on a project right now and I need to create a graphic library.
The game I'm experimenting with is an RPG; this project is expected to contain many big graphic files to use and I would prefer not to load everything into memory at once, like I've done before with other smaller projects.
So, does anyone have experience with libraries such as this one? Here's what I've came up with:
Have graphic library files and paths in an XML file
Each entry in the XML file would be designated "PERMANENT" or "TEMPORARY", with perm. being that once loaded it stays in memory and won't be cleared (like menu-graphics)
The library that the XML file loads into would have a CLEAR command, that clears out all non-PERMANENT graphics
I have experience throwing everything into memory at startup, and with running the program running with the assumption that all necessary graphics are currently in memory. Are there any other considerations I might need to think about?
Ideally everything would be temporary and you would have a sensible evict function that chooses the right objects to victimize (based on access patterns) when your program decides it needs more memory.
There'll be some minimum amount of RAM your game needs to run, otherwise stuff will be constantly swapping, but this approach does mean you're not dumping objects marked TEMPORARY that you will just need to reload next frame because you happen to be using it currently.