Context
As a university project we want to change the the pharo vm to use an object-table and see what happens.
We use a pharo-vm clone from github and VMMaker. Building the VM works fine.
To get started we added a primitive that returns an incremented Integer:
InterpreterPrimitives>>primitiveIntegerIncrement
"increments an integer"
self pushInteger: self popInteger + 1 .
and modified StackInterpreter class>>initializePrimitiveTable accordingly
MaxPrimitiveIndex := 576.
"... and so on ..."
(575 primitiveFail)
(576 primitiveIntegerIncrement))
And it works.
Problem
When we make changes to the VM we want to test-run already in the SmalltalkImage so we do not need to compile and see it did not work.
Something like:
StackInterpreter test: '1 inc'
And then I can debug if the primitive is wrong or an error occurs. Of course there needs to be done much more but how can I achieve this?
What we tried
category VMMaker-InterpreterSimulation class StackInterpreterSimulator. Trying the code in the comments
DoIt
^ (StackInterpreterSimulator new openOn: Smalltalk imageName) test
errors:
displayForm := 'Display has not yet been installed' asDisplayText form.
the ByteString does not understand asDisplayText
(CogVMSimulator new openOn: Smalltalk imageName) test
(InterpreterSimulator new openOn: Smalltalk imageName) test
error:
PrimitiveFailed: primitive #basicNew: in Array class failed
I also found this screen cast but it only debugs the VM from outside using gbd: http://vimeo.com/22485382#
Our project is hosted here: http://smalltalkhub.com/#!/~kirstin/PharoObjectTable
Current Status
We started implementing an object table. The lookup of attributes can go throught the object table. Full object table support and no usage of direct pointes is very tricky since pointers are expected everywhere. So we use pointers into the object table to identify when a lookup should go through the OT. We also found all object creation primitives and add new objects to the table.
How long is your project and how many people are you ? To me what you try to do is quite some work. Do you have good knowledge about low level behavior ?
To answer your question, the main problem here is that the cog simulator is not maintained in the pharo vm fork. This is because no one in the pharo crew use the simulator. We only use external debugging from gdb. In fact the pharo folks work mostly on VM plugins, the core of the VM is mainly maintained and developed by Eliot Miranda which works on Squeak. Therefore we report to him when there's a bug in the VM core.
For your project you would have to split it in at least 2 steps:
step 1: make the object table work with the stack VM
step 2: make the JIT work with your object table
Note that for step 2 I would recommend not to change the way an object access its header, therefore having a VW-like object table where you have the fixed size header on the one in the the object table, and the fields of the objects (and maybe header extensions) in the heap.
So use the StackVMSimulator and build the StackVM first. When everything will work (including context), you can think about hacking the JIT. Recently Guillermo Polito ported the Stack VM to the build process (see PharoSVMBuilder instead of PharoVMBuilder), a guy reported problems with this builder but you could hack it a bit to make it work.
Now to make the simulator work on Pharo 2.0 (which is the Pharo version of the generator image you have), you have to open the monticello browser and merge from Eliot's branch the Cog package (repo MCHttpRepository location: 'http: //source. squeak. org/VMMaker'), but not the latest Cog, the one at around the same date as the current VMMaker package of pharo-vm because the latest Cog and VMMaker of Eliot's branch are not stable.
The alternative being to start from Eliot's build image and merge things from the pharo branch. Here are infos about how to build the squeak development image (http://www.mirandabanda.org/cogblog/build-image/).
Then Eliot gave me this script once:
| cos |
cos := CogVMSimulator newWithOptions: #(Cogit SistaStackToRegisterMappingCogit).
cos desiredNumStackPages: 8.
cos openOn: 'my/favourite.image'.
cos openAsMorph; toggleTranscript; halt; run
You don't need the SistaStackToRegisterMappingCogit option. I guess some similar script with the StackVMSimulator should work.
Lastly there are some documentation about the simulator but it is only for the CogSimulator (these documentations expects you already knows how the StackSimulator works, and just give you hints about how to use it with the JIT):
http://www.mirandabanda.org/cogblog/2008/12/12/simulate-out-of-the-bochs/
and in one of the video named "Cog VM (part x)", x being from 1 to 6, Eliot shows how he uses the simulator to disassemble x86, print the stack and inspect the heap.
Another tip, ask your questions on the pharo mailing list (pharo users or pharo dev), because here no one may notice your question (fortunately someone pointed me out your question this time).
And tell on the pharo mailing list if you managed to run the simulator in Pharo 2.0, some people (as me) are very interested in it. I was planning to do it at some point.
Good luck ! Nice project anyway.
The last time I tried to use the simulator is roughly a year ago, and I did not make it work.
However, there are a few patches, which I assume never got integrated that might be of help:
https://code.google.com/p/cog/issues/detail?id=106
https://code.google.com/p/cog/issues/detail?id=107
https://code.google.com/p/cog/issues/detail?id=108
Issue 107 includes a patch for your asDisplayText issue.
Related
I am porting an old (circa year 2000) IBM VAST 4.5 application to VA Smalltalk.
I managed to import the app from the VAST repository to my new VA Smalltalk repository with no error messages. But when I try to load the application this error occurs:
Error: 365 Cannot complete the load because CwItem can only be
defined by one of ('CwControls V 4.5a' 'CwWindowsControls V 8.6.0
[269]').
I understand that CwControls changed to CwWindowsControls but my Smalltalk knowledge is very rusty: I have no idea how to solve this.
Someone in the VA Smalltalk group mentioned a similar situation but his answer is a bit cryptic to me:
"CwControls app name is changed in new version as CwWindowsControls. When I create a empty app named CwControls, I can load apps uses controls in this app."
I know that I can create an empty app called CwControls but then my application will not compile as the methods normally provided by the original CwControls will not be present.
How can I solve this problem?
Thank you very much!!
It looks to me like CwWindowsControls already defines CwItem. When you load CwItem from the other project it is defined by CwControls though. I'm not familiar with VA but you might be able to resolve the issue by first deleting CwItem where it's defined by CwWindowsControls, then loading the other project and finally (possibly) refactor CwItem to be defined by CwWindowsControls.
I found a way to solve my problem.
My application needed another application called CwControls. But CwControls had changed its name to CwWindowsControls.
So, inspired by the answer by #MikeLeske, I just created an empty app called CwControls. That way VA Smalltalk let me continue the load.
Another error appeared then:
"Error: 365 Cannot complete the load because AbtCwPanel can only be defined by one of ('AbtRunWinCwControlsApp V 4.5' 'AbtWinRunViewsSubApp V 8.6.0 [269]')."
But the answer was similar: I just created an empty app called AbtRunWinCwControlsApp.
Now I have my 13 year old application loaded into my image. Time to work!!!
Thanks.
I had a similar problem, but with SstHttpSupport which became SstHttpCommunications and such.
I fixed this by "Managing" the application and removing the requirement for SstHttpSupport.
I then successfully ported the application into VAST 8.6.
Knowing that I no longer had SstHttpSupport, I add SstHttpCommunications as the new requirement in 8.6.
Hope this helps
After a search here on the forum I found a question like that, and it redirected me to a tutorial which gave em some basic instructions on manipulating SpringBoard with CapitainHook.
To start I'd like to do it with normal %hooks only. Any hint where I could start?
This little introduction is meant for whoever has a minimal knowledge on Objective-C and knows what he is doing.
NOTE: I will refer to the theos install path as $THEOS. This could be ~/theos, /var/theos, /usr/theos... Yeah.
The most popular way of creating MobileSubstrate extensions, also known as tweaks, is using Dustin Howett's theos build suite. Details follow:
What is theos?
So, we should start with what theos is not:
The Operating System
A Greek God
A compiler
And of course, what theos doesn't do:
Teaches you how to code.
Creates tweaks without having you to think
Sets up a whole building environment and/or installs the iOS SDK.
Theos is a cross-platform suite of development tools for managing, developing, and deploying iOS software without the use of Xcode, featuring:
A robust build system driven by GNU Make, which makes its Makefiles easily deployable through everywhere with theos installed too.
NIC, a project templating system which creates ready-to-build empty projects for varying purposes.
Logos, a built-in preprocessor-based library of directives designed to make MobileSubstrate extension development easy and with optimal code generation.
Automated packaging: Theos is capable of directly creating DEB packages for distribution in Cydia, the most popular mean of package distribution in the jailbreak scene.
How to install theos?
On OSX: Have the iOS SDK installed and follow these instructions.
On iOS: Install the BigBoss Recommended Tools package from Cydia and run installtheos3.
On Linux: Find a mean to have the toolchain installed, and follow these instructions.
On Windows: Nothing is impossible, but if you actually manage to do so, please let me know. :P
How to use theos?
This is a very asked question and too vague. Since theos is a whole suite of development tools, it doesn't make sense to ask How to use it, but more specifically, to ask How to create software using theos.
First of all, always have the Theos Makefile Reference in hand. It covers the basics of creating a theos Makefile, and that includes solving your linking issues adding a framework or private framework to the project.
Now, you can either create your own Makefile from scratch, create your little theos clone/symlink and start coding, but theos makes this step easier. You can just use nic.pl.
A very simple example of running NIC to create something can be found here. It's very straight-forward and sets you up right-away for programming.
Now, here's where we start getting back to topic.
Creating a tweak with theos
First of all, do not run NIC when inside $THEOS/bin. NIC will create the project directory exactly where you're running it from, and it avoids any project being created in $THEOS/bin. Therefore, you'll end up with a simple error which can be avoided by creating the project directory somewhere decent.
Run $THEOS/bin/nic.pl and choose the iphone/tweak template. You will be prompted by simple information which you may well know well how to answer, except for the last field: MobileSubstrate bundle filter.
Since a big part of MobileSubstrate is not just the hooker (the library which switches original methods/functions with yours), but also the loader (the part which gets your hooking to be inserted into certain processes), you have to supply this basic information for the Loader to know where to load your tweak. This field is but the bundle identifier for the application where this project will be inserted.
com.apple.springboard, the default option is the bundle identifier for SpringBoard, the application which is:
The iOS Homescreen
The launcher/displayer of common applications
The iOS Status Bar
Handler of some high-level essential background processes
Therefore, there's where many tweaks take place, altering behavior from something as trivial as app launching to something like how the whole homescreen UI looks like.
Programming a tweak with Logos
Now, the directory generated by NIC will contain:
The Theos Makefile, where you'll change information related to compiling
The control file, where you'll change packaging-related information
A symbolic link (or shortcut) to $THEOS named theos/
The main code file, defaulted as Tweak.xm. It is already added to the Makefile for compiling, so you can start coding right-away with it!
On knowing what to do
Now, you don't have SpringBoard's source code laying around, and you can't guess what methods to hook from nowhere. Therefore, you need a SpringBoard header set. For that, you need to use a tool named class-dump-z and run it into the SpringBoard binary (which is inside the iOS filesystem) to obtain header files including all class declarations and its methods inside the application.
From that (a deal of guessing and logging a method call is involved) you can start messing around with what you want in a tweak.
Of course, if you are not hooking SpringBoard you can use class-dump-z as you would in other binaries, such as UIKit, MobileSafari, etc.
Note that for when reversing App Store apps, they'll be encrypted. You'll need to decrypt those (I am unfortunately not allowed to tell you how-to), and then just run class-dump-z on them.
On obtaining private headers
Stuff like preference bundles require the headers for private frameworks, in that case the Preferences framework's headers. Else you'll get endless missing declaration errors (as I guess you could assume).
Getting them has the same logic applied the previous step. Run class-dump-z on, at this case, the Preferences binary and throw the headers at your INCLUDEPATH. The INCLUDEPATH is where the compiler will go looking for headers you include like #include <stdio.h>. Yes, stdio.h is inside one of the directories which build a compiler's INCLUDEPATH!
When compiling with a theos Makefile, $THEOS/include counts as part of your INCLUDEPATH, which means, you can just throw your dumped headers over there and include them later.
(Note that class-dumped headers aren't always perfect, so you're likely to have a couple of header-related compilation errors which can be easily fixed with something like removing a #import directive or changing it, or adding a couple of declarations.)
Code tips
You can't link against SpringBoard, so whenever you require a class from SpringBoard you have to use either the Logos %c directive or the objc_getClass function, as defined at <objc/runtime.h> to get it. Example: [%c(SBUIController) sharedInstance], [objc_getClass("SBUIController") sharedInstance].
When not knowing what a method does or how something works in SpringBoard, try disassembling it with IDA or others. I use IDA Demo (<- noob!) for my disassembling.
Looking at example code is amazingly helpful for both learning and figuring out how something works inside SpringBoard or others (again..). Great people at GitHub to have a projects looked at are rpetrich, chpwn, DHowett, EvilPenguin, and of course way more.
To also find about how SpringBoard and other works (...), have a look at a class's article at the iPhone Dev Wiki!
Epilogue
Wait, where's the good part? Where do I learn about coding in Tweak.xm?
Well, the original question was actually How to start MobileSubstrate tweaks programming?. You're all setup, hopefully with all headers placed, ready to type in make and see your project magically compiled with theos.
All you need to do is now to actually dig into your headers or your disassembly and go hooking, calling, etc.!
Logos Reference contains exactly how to hook and use other features of Logos, and the MobileSubstrate article on the devwiki is also a great read.
In case there is any doubt, don't hesitate joining the irc.saurik.com #theos IRC channel. It's a great way to discuss theos-related topics and ask questions. I'm mostly there, along with other greatly smart people ;)
You are looking for Theos created by DHowett.. Theos allows you to make tweaks, but it doesn't give you everything you need. You don't get every header for iOS, so you have to class-dump-z the frameworks/private-frameworks from the iOS SDK. Get started here: http://iphonedevwiki.net/index.php/Theos/Getting_Started, or join irc.saurik.net #theos for more help. You can also look at my projects that use theos: https://github.com/evilpenguin
You sound like you're looking for theos. Take a look at this, it should help get you started.
this is my first post, and it covers something which I've been trying to get working on and off for about a year now.
Essentially it boils down to the following: I have a copy of newlib which I'm trying to get working on an LPC2388 (an ARM7TDMI from NXP). This is on a linux box using arm-elf-gcc
The question I have is that I've been looking at a lot of the tutorials talking about porting newlib, and they all talk about the stubs (like exit, open, read/write, sbrk), and I have a pretty good idea of how to implement all of these functions. But where should I put them?
I have the newlib distribution from sources.redhat.com/pub/newlib/newlib-1.18.0.tar.gz and after poking around I found "syscalls.c" (in newlib-1.18.0/newlib/libc/sys/arm) which contains all of the stubs which I have to update, but they're all filled in with rather finished looking code (which does NOT seem to work without the crt0.S, which itself does not work with my chip).
Should I just be wiping out those functions myself, and re-writing them? Or should I write them somewhere else. Should I make a whole new folder in newlib/libc/sys with the name of my "architecture" and change the target to match?
I'm also curious if there's proper etiquette on distribution of something like this after releasing it as an open source project. I currently have a script which downloads binutils, arm-elf-gcc, newlib, and gdb, and compiles them. If I am modifying files which are in the newlib directory, should I hand a patch which my script auto-applies? Or should I add the modified newlib to the repository?
Thanks for bothering to read! Following this is a more detailed breakdown of what I'm doing.
For those who want/need more info about my setup:
I'm building a ARM videogame console based loosely on the Uzebox project ( http://belogic.com/uzebox/ ).
I've been doing all sorts of things pulling from a lot of different resources as I try and figure it out. You can read about the start of my adventures here (sparkfun forums, no one responds as I figure it out on my own): forum.sparkfun.com/viewtopic.php?f=11&t=22072
I followed all of this by reading through the Stackoverflow questions about porting newlib and saw a few of the different tutorials (like wiki.osdev.org/Porting_Newlib ) but they also suffer from telling me to implements stubs without mentioning where, who, what, when, or how!
But where should I put them?
You can put them where you like, so long as they exist in the final link. You might incorporate them in the libc library itself, or you might keep that generic, and have the syscalls as a separate target specific object file or library.
You may need to create your own target specific crt0.s and assemble and link it for your target.
A good tutorial by Miro Samek of Quantum Leaps on getting GNU/ARM development up and running is available here. The examples are based on an Atmel AT91 part so you will need to know a little about your NXP device to adapt the start-up code.
A ready made Newlib porting layer for LPC2xxx was available here, but the links ot teh files appear to be broken. The same porting layer is used in Martin Thomas' WinARM project. This is a Windows port of GNU ARM GCC, but the examples included in it are target specific not host specific.
You should only need to modify the porting layer on Newlib, and since it is target and application specific, you need not (in fact probably should not) submit your code to the project.
When I was using newlib that is exactly what I did, blew away crt0.s, syscalls.c and libcfunc.c. My personal preference was to link in the replacement for crt0.s and syscalls.c (rolled the few functions in libcfunc into the syscalls.c replacement) based on the embedded application.
I never had an interest in pushing any of that work back into the distro, so cannot help you there.
You are on the right path though, crt0.S and syscalls.c are where you want to work to customize for your target. Personally I was interested in a C library (and printf) and would primarily neuter all of the functions to return 0 or 1 or whatever it took to get the function to just work and not get in the way of linking, periodically making the file I/O functions operate on linked in data in rom/ram. Basically without replacing or modifying any other files in newlib I had a fair amount of success, so you are on the right path.
I cannot use the most recent dev Pharo release because of some strange issues with the compiler built into Pharo. Well. I was wondering if there is a quick way to install all the nifty extras into Pharo that the core image misses, as compared to the dev image.
With all non-core Pharo images come a script which was used to build that image. Just edit that file and drag&drop it on a new core.
You could also tell me what you don't like in the Pharo images so that I can enhance them.
There is also the script I published on the Pharo wiki that I use to build my images:
http://code.google.com/p/pharo/wiki/ImageBuildScripts
Of course it is very specific to my preferences and needs, but you can take it as an example and adapt it to your own needs.
CommandShell works with Pharo 9.10.10. You will hit several errors as you try to load the package due to Pharo lacking MVC, but you can simply proceed past the first bunch and abandon the last one (that tries to actually open a CommandShell in Morphic). At that point, you'll have a class called PipeableOSProcess that can be used very easily to grab output. For example:
(PipeableOSProcess command: 'ls /bin') output
will return the contents of your bin directory as a string.
Ok, OB itself can be easily downloaded using ScriptLoader loadSuperOB.
Damien adds (from comment below):
The problem with that approach is that nobody really maintains it.
Moreover, you miss some configuration steps to enhance the use of OB
(for example, you won't have the OB-based browsers if you ask for the
senders of a message from a workspace)
I'm new to VB6 but i'm currently in charge of maintaining a horror of editor like tool with plenty of forms, classes, modules and 3rd party tools all chunk together like the skin faces on that guy in the texas chainsaw massacre...
What i don't understand is why i get different results when i run the app in debugging mode, vs when i compiled it and run it on my devevelopment pc vs when i installed it on a different pc.
Yes i know i'm dumb, so please direct me to where i can find out more about this. I'm hoping to find out something like different linking, registry related etc connection that i'm simply not getting right now, i.e. something like wax on, wax off :P
The main pain in the neck is when i'm trying to debug some errors from my QA and i need to find a spare pc to test this on plus i can't directly debug because i don't know where the code is if i do it that manner.
Thanks.
i run the app in debugging mode vs when i compiled it and run it on my
devevelopment pc
When you compile you have the option of compiling to native code or pcode. The debugger runs using pcode only. Under rare circumstances when you compile to native code there will be a change in behavior. This particular is really rare. I used VB6 since it's release and I may get it once or twice a year. My application is a complex CAD/CAM creating shapes and running metal cutting machine and has two dozen DLLs. Not a typical situation. At home with my hobby software I never ran into this problem.
There are another class of errors that result from event sequencing problems. While VB6 isn't truly multi-tasking it has the ability to jump out of the current code block to process a event. If it re-enters the same block for the new event interesting things (to say the least) can result. I think this is the likely source of your problems as you software is an editor which is a highly interactive type of software.
In general the problem is fixed by reordering the effected areas. You find the effected area by inserting MsgBox or write to a text file to log where you are. I recommend logging to a text file as MsgBox tend to alter behavior that are timing or multi-tasking related.
Remember if a event fire while VB6 in the middle of a code block and there a DoEvents floating around then it will leave the code block process the event and return to the original code block. If it re-enters the same code block and you didn't mean for this to happen then you will have problems. And you will have different problems on different computers as the timing will be different for each.
The easiest way to deal with this type of issues is create some flag variables. In multi-tasking parlance they are known as semaphores or mutexes. WHen you enter a critical section of code, you set it true. When you leave the routine you set it to false. If it is already true when you enter that section of code you don't execute it.
when i installed it on a different pc.
These are usually the result of the wrong DLL installed. Most likely you have an older version while the target has a newer version. I would download the free Virtual PC and create a clean Window XP install to double check this.
If your problem is event timing this too can be different on different computers. This is found by logging (not MsgBox) suspect regions.
If you can display a screen shot or the text of your specific errors then I can help better.
The first thing to check would be the versions of all the dlls that your app depends on - including the service pack version of the VB6 dll.
Have you any more specific details about what's behaving differently?