I know how to write some 100 lines C, but I do not know how to read/organize larger source like Rebol. Somewhere was a tutorial with hostkit and dll, but it seems R3 is now statically linked. So I do not know where to look.
How would I write a native which gets a value and returns another? Where to put it in the source? What to obey, like telling the GC I created something in C?
Also, how can I embed R3 in other programms, to call it from Python or Node? I ask for Python/Node part comes later. But my learning-main should access R3 in a similar way. Means dll. And are there some typical hooks for startup/shutdown etc in such ffi's?
[Edit: forgot to mention: it is for Rebol 3.]
That's two questions. :-)
Regarding the first (about adding a native)... it's probably best elaborated as developer documentation on a Wiki. Questions in the rebol tag should generally be more about the language from a user's point of view. The answers regarding source code itself typically will be long, and there's really only a few people who can answer it, or care about the answer (what StackOverflow calls "too localized"). And it will wind up being more of a c question at that point, if anything.
So telling the developers to get their act together and write that documentation and put it in a centrally organized place is probably the best idea! :-P But I actually did try this myself. I added a set-env native to set environment variables from the interpreter, and you can look at the diffs it took to do that in GitHub for a bit of an idea.
add SET-ENV, tweaks to host api for environment string handling
An important thing to remember is that when you touch certain files, you have to run make prep which does a lot of scanning and code generation automatically. Adding a native is one of those things where you're definitely going to have to do that, each time you change to an interface that fundamental.
As for your second question, which is more of a user-facing question about interpreter embedding, one of the first places to look is just in how the simple REPL is implemented. There are two versions in the main repository right now, one for Posix and one for Windows.
https://github.com/rebol/r3/blob/master/src/os/host-main.c
So a string goes in and a string comes out. But there are more complicated forms of interaction, and you can get to them from reb-host.h, which includes these files from src/include
#include "reb-config.h"
#include "reb-c.h"
#include "reb-ext.h" // includes reb-defs.h
#include "reb-args.h"
#include "reb-device.h"
#include "reb-file.h"
#include "reb-event.h"
#include "reb-evtypes.h"
#include "reb-net.h"
#include "reb-filereq.h"
#include "reb-gob.h"
#include "reb-lib.h"
So you can look through those files for the API as it existed at the moment of the December-12th open-sourcing. Things will be evolving, and hopefully much better documented. But for now that's what seems to be available. You could link the host kit as a shared/DLL or static library, it depends on your build settings and what compiler you use.
Related
...is there anything I could do about it?
To be more precise, I would like to replace the caret "^" with something like "§" - granted, there's not much left on the keyboard that's not in use already.
After thinking about it for a while (dismissed using run script build phases along the way) I think the only way to do it would be a custom llvm build.
While I don't quite think I'm ready to deal with the internals of compilers, I have the naive hope that replacing one symbol with another isn't too hard. And the idea of building and running my own version of a compiler tickles me, be it just for a good deal of childish fun.
So I started poking around in the llvm sources, but - surprise - got nowhere so far.
If someone is familiar with these kind of things, could you please point me to a place to look at?
That would be awesome! Thanks!
Extending LLVM can be a bit of a hassle, especially considering how fast-moving the compiler team is, so it's a good thing you don't have to. The C preprocessor exists to perform the exact same thing you've outlined (text replacement). I'm fairly sure § isn't aliased to anything important, so #define § ^ should work great. If you still want to write your own module, LLVM provides instructions on how to extend their compiler.
Actually the code relevant for such a change isn't a part of LLVM at all, but a part of its Objective-C frontend, called Clang. Confusingly, "Clang" is also the name of the entire C/C++/ObjC compiler based on both Clang and LLVM.
While I don't quite think I'm ready to deal with the internals of compilers, I have the naive hope that replacing one symbol with another isn't too hard.
And you'll be right. What you're trying to do is very simple change.
In fact, if ^ was only used for blocks, it would be a trivial change - just modify the lexer to generate the "caret" token from § instead of ^: take a look at the lexer code to see what I mean (search for ^).
Unfortunately it's used for xor as well, so we'll have to modify both the lexer and the parser. The lexer to add a new token type and create that token from §, the parser to actually do something with it, e.g. by adding:
case tok::section: // 'section' is the token type you've added
Res = ParseBlockLiteralExpression();
break;
(and then fixing the assert at the beginning of ParseBlockLiteralExpression()).
You might run into some issues, though, as § isn't in ASCII - though as far as I know Clang should be able to deal with UTF-8 encoded files.
I'd like to be able to Alt-Click an instance variable (or a method) as part of the program i created and read what it's purpose is.
The fact that Xcode is telling me the class variable is declared at - is nice but not enough. In this case i'd like to see custom text i typed to describe what an asset really is. Additionally type of the ivar would also be useful to know.
How can this be done? In this case, i wonder what exactly did i mean by assets
I specifically wonder if this information can be viewed from inside Xcode, similar to how Eclipse shows JavaDoc content.
You would need to create a documentation set for your project and install it in Xcode. appledoc can help you with this. This is a command-line tool that can generate documentation in Apple's style from specially formatted comments in your headers. You can also integrate this into your build process with a run script build phase, so that documentation is always up-to-date.
For small projects, it's usually not worth the effort though and you're probably better off just adding comments to your header files and jumping there with Cmd-click (Ctrl+Cmd+left-arrow to go back to where you came from).
You'll probably want to take a look at Apple's documentation on Documentation Sets as well as their article on generating doc sets using Doxygen. The latter is based on Xcode 3.x, so how relevant it is is somewhat questionable, but it'd be a good idea to take a look nonetheless.
That said, if you decide to use Doxygen (alternatives like HeaderDoc can be used for documentation, but I'm not sure what's available to you as far as creating doc sets goes), it looks like the main point is you'll want to throw GENERATE_DOCSET=YES into your Doxyfile (or whatever you decide to call it). After that, you'd just throw the results into ~/Library/Developer/Shared/Documentation/DocSets (according to Doxygen's documentation). I don't know whether this works in Xcode 4.x - it's worth a shot though, and it'd be nice to hear back on it.
Note: most of this was based on this answer by Barry Wark. Figure credit is due there, since I wouldn't have bothered looking into this were it not for his answer.
After a search here on the forum I found a question like that, and it redirected me to a tutorial which gave em some basic instructions on manipulating SpringBoard with CapitainHook.
To start I'd like to do it with normal %hooks only. Any hint where I could start?
This little introduction is meant for whoever has a minimal knowledge on Objective-C and knows what he is doing.
NOTE: I will refer to the theos install path as $THEOS. This could be ~/theos, /var/theos, /usr/theos... Yeah.
The most popular way of creating MobileSubstrate extensions, also known as tweaks, is using Dustin Howett's theos build suite. Details follow:
What is theos?
So, we should start with what theos is not:
The Operating System
A Greek God
A compiler
And of course, what theos doesn't do:
Teaches you how to code.
Creates tweaks without having you to think
Sets up a whole building environment and/or installs the iOS SDK.
Theos is a cross-platform suite of development tools for managing, developing, and deploying iOS software without the use of Xcode, featuring:
A robust build system driven by GNU Make, which makes its Makefiles easily deployable through everywhere with theos installed too.
NIC, a project templating system which creates ready-to-build empty projects for varying purposes.
Logos, a built-in preprocessor-based library of directives designed to make MobileSubstrate extension development easy and with optimal code generation.
Automated packaging: Theos is capable of directly creating DEB packages for distribution in Cydia, the most popular mean of package distribution in the jailbreak scene.
How to install theos?
On OSX: Have the iOS SDK installed and follow these instructions.
On iOS: Install the BigBoss Recommended Tools package from Cydia and run installtheos3.
On Linux: Find a mean to have the toolchain installed, and follow these instructions.
On Windows: Nothing is impossible, but if you actually manage to do so, please let me know. :P
How to use theos?
This is a very asked question and too vague. Since theos is a whole suite of development tools, it doesn't make sense to ask How to use it, but more specifically, to ask How to create software using theos.
First of all, always have the Theos Makefile Reference in hand. It covers the basics of creating a theos Makefile, and that includes solving your linking issues adding a framework or private framework to the project.
Now, you can either create your own Makefile from scratch, create your little theos clone/symlink and start coding, but theos makes this step easier. You can just use nic.pl.
A very simple example of running NIC to create something can be found here. It's very straight-forward and sets you up right-away for programming.
Now, here's where we start getting back to topic.
Creating a tweak with theos
First of all, do not run NIC when inside $THEOS/bin. NIC will create the project directory exactly where you're running it from, and it avoids any project being created in $THEOS/bin. Therefore, you'll end up with a simple error which can be avoided by creating the project directory somewhere decent.
Run $THEOS/bin/nic.pl and choose the iphone/tweak template. You will be prompted by simple information which you may well know well how to answer, except for the last field: MobileSubstrate bundle filter.
Since a big part of MobileSubstrate is not just the hooker (the library which switches original methods/functions with yours), but also the loader (the part which gets your hooking to be inserted into certain processes), you have to supply this basic information for the Loader to know where to load your tweak. This field is but the bundle identifier for the application where this project will be inserted.
com.apple.springboard, the default option is the bundle identifier for SpringBoard, the application which is:
The iOS Homescreen
The launcher/displayer of common applications
The iOS Status Bar
Handler of some high-level essential background processes
Therefore, there's where many tweaks take place, altering behavior from something as trivial as app launching to something like how the whole homescreen UI looks like.
Programming a tweak with Logos
Now, the directory generated by NIC will contain:
The Theos Makefile, where you'll change information related to compiling
The control file, where you'll change packaging-related information
A symbolic link (or shortcut) to $THEOS named theos/
The main code file, defaulted as Tweak.xm. It is already added to the Makefile for compiling, so you can start coding right-away with it!
On knowing what to do
Now, you don't have SpringBoard's source code laying around, and you can't guess what methods to hook from nowhere. Therefore, you need a SpringBoard header set. For that, you need to use a tool named class-dump-z and run it into the SpringBoard binary (which is inside the iOS filesystem) to obtain header files including all class declarations and its methods inside the application.
From that (a deal of guessing and logging a method call is involved) you can start messing around with what you want in a tweak.
Of course, if you are not hooking SpringBoard you can use class-dump-z as you would in other binaries, such as UIKit, MobileSafari, etc.
Note that for when reversing App Store apps, they'll be encrypted. You'll need to decrypt those (I am unfortunately not allowed to tell you how-to), and then just run class-dump-z on them.
On obtaining private headers
Stuff like preference bundles require the headers for private frameworks, in that case the Preferences framework's headers. Else you'll get endless missing declaration errors (as I guess you could assume).
Getting them has the same logic applied the previous step. Run class-dump-z on, at this case, the Preferences binary and throw the headers at your INCLUDEPATH. The INCLUDEPATH is where the compiler will go looking for headers you include like #include <stdio.h>. Yes, stdio.h is inside one of the directories which build a compiler's INCLUDEPATH!
When compiling with a theos Makefile, $THEOS/include counts as part of your INCLUDEPATH, which means, you can just throw your dumped headers over there and include them later.
(Note that class-dumped headers aren't always perfect, so you're likely to have a couple of header-related compilation errors which can be easily fixed with something like removing a #import directive or changing it, or adding a couple of declarations.)
Code tips
You can't link against SpringBoard, so whenever you require a class from SpringBoard you have to use either the Logos %c directive or the objc_getClass function, as defined at <objc/runtime.h> to get it. Example: [%c(SBUIController) sharedInstance], [objc_getClass("SBUIController") sharedInstance].
When not knowing what a method does or how something works in SpringBoard, try disassembling it with IDA or others. I use IDA Demo (<- noob!) for my disassembling.
Looking at example code is amazingly helpful for both learning and figuring out how something works inside SpringBoard or others (again..). Great people at GitHub to have a projects looked at are rpetrich, chpwn, DHowett, EvilPenguin, and of course way more.
To also find about how SpringBoard and other works (...), have a look at a class's article at the iPhone Dev Wiki!
Epilogue
Wait, where's the good part? Where do I learn about coding in Tweak.xm?
Well, the original question was actually How to start MobileSubstrate tweaks programming?. You're all setup, hopefully with all headers placed, ready to type in make and see your project magically compiled with theos.
All you need to do is now to actually dig into your headers or your disassembly and go hooking, calling, etc.!
Logos Reference contains exactly how to hook and use other features of Logos, and the MobileSubstrate article on the devwiki is also a great read.
In case there is any doubt, don't hesitate joining the irc.saurik.com #theos IRC channel. It's a great way to discuss theos-related topics and ask questions. I'm mostly there, along with other greatly smart people ;)
You are looking for Theos created by DHowett.. Theos allows you to make tweaks, but it doesn't give you everything you need. You don't get every header for iOS, so you have to class-dump-z the frameworks/private-frameworks from the iOS SDK. Get started here: http://iphonedevwiki.net/index.php/Theos/Getting_Started, or join irc.saurik.net #theos for more help. You can also look at my projects that use theos: https://github.com/evilpenguin
You sound like you're looking for theos. Take a look at this, it should help get you started.
this is my first post, and it covers something which I've been trying to get working on and off for about a year now.
Essentially it boils down to the following: I have a copy of newlib which I'm trying to get working on an LPC2388 (an ARM7TDMI from NXP). This is on a linux box using arm-elf-gcc
The question I have is that I've been looking at a lot of the tutorials talking about porting newlib, and they all talk about the stubs (like exit, open, read/write, sbrk), and I have a pretty good idea of how to implement all of these functions. But where should I put them?
I have the newlib distribution from sources.redhat.com/pub/newlib/newlib-1.18.0.tar.gz and after poking around I found "syscalls.c" (in newlib-1.18.0/newlib/libc/sys/arm) which contains all of the stubs which I have to update, but they're all filled in with rather finished looking code (which does NOT seem to work without the crt0.S, which itself does not work with my chip).
Should I just be wiping out those functions myself, and re-writing them? Or should I write them somewhere else. Should I make a whole new folder in newlib/libc/sys with the name of my "architecture" and change the target to match?
I'm also curious if there's proper etiquette on distribution of something like this after releasing it as an open source project. I currently have a script which downloads binutils, arm-elf-gcc, newlib, and gdb, and compiles them. If I am modifying files which are in the newlib directory, should I hand a patch which my script auto-applies? Or should I add the modified newlib to the repository?
Thanks for bothering to read! Following this is a more detailed breakdown of what I'm doing.
For those who want/need more info about my setup:
I'm building a ARM videogame console based loosely on the Uzebox project ( http://belogic.com/uzebox/ ).
I've been doing all sorts of things pulling from a lot of different resources as I try and figure it out. You can read about the start of my adventures here (sparkfun forums, no one responds as I figure it out on my own): forum.sparkfun.com/viewtopic.php?f=11&t=22072
I followed all of this by reading through the Stackoverflow questions about porting newlib and saw a few of the different tutorials (like wiki.osdev.org/Porting_Newlib ) but they also suffer from telling me to implements stubs without mentioning where, who, what, when, or how!
But where should I put them?
You can put them where you like, so long as they exist in the final link. You might incorporate them in the libc library itself, or you might keep that generic, and have the syscalls as a separate target specific object file or library.
You may need to create your own target specific crt0.s and assemble and link it for your target.
A good tutorial by Miro Samek of Quantum Leaps on getting GNU/ARM development up and running is available here. The examples are based on an Atmel AT91 part so you will need to know a little about your NXP device to adapt the start-up code.
A ready made Newlib porting layer for LPC2xxx was available here, but the links ot teh files appear to be broken. The same porting layer is used in Martin Thomas' WinARM project. This is a Windows port of GNU ARM GCC, but the examples included in it are target specific not host specific.
You should only need to modify the porting layer on Newlib, and since it is target and application specific, you need not (in fact probably should not) submit your code to the project.
When I was using newlib that is exactly what I did, blew away crt0.s, syscalls.c and libcfunc.c. My personal preference was to link in the replacement for crt0.s and syscalls.c (rolled the few functions in libcfunc into the syscalls.c replacement) based on the embedded application.
I never had an interest in pushing any of that work back into the distro, so cannot help you there.
You are on the right path though, crt0.S and syscalls.c are where you want to work to customize for your target. Personally I was interested in a C library (and printf) and would primarily neuter all of the functions to return 0 or 1 or whatever it took to get the function to just work and not get in the way of linking, periodically making the file I/O functions operate on linked in data in rom/ram. Basically without replacing or modifying any other files in newlib I had a fair amount of success, so you are on the right path.
I have an ANSI C code that is about 10,000 lines long that I am trying to use in an iPhone project. When I compile the code with gcc on the command line, I type the following:
gcc -o myprog -O3 myprog.c
This program reads in large jpeg files and does some fancy processing on them, so I call it with the following
./myprog mypic.jpg
and from the command line, this takes take about 0.1 seconds.
I'm trying to import this code into an iPhone project but I'm not entirely sure how. I was able to get it to compile and run successfully by renaming myprog.c to myprog.h and then calling the functions in the C code from within a generic NSObject class. I added the O3 optimization to the project's Other C Flags. However, when I do this, the code on the simulator takes about 2 seconds to run and on the iPhone about 7 seconds to run which renders an unacceptable user experience.
Any tips on on hoe to get this going would be much appreciated.
It's hard to say for sure where the slowness comes from, or if there is any way around it, but right off the bat you've done something wrong.
You shouldn't have renamed a .c file to a .h file and included it. You should have written a .h (header) file that had the function, variable, and type declarations declared:
myprog.h:
#ifndef MYPROG_H_
#define MYPROG_H_
struct thing {
int a;
int b;
};
extern int woof;
int foo(void * buf, int size);
#endif /* MYPROG_H_ */
Then you should compile the .c file to an object file (or library) and link the main program against that. If you were to have included the .h file that was really just a renamed .c file into more than one source code files it may have resulted in having multiple versions of some data and code in your program.
You'll probably also want to go through and separate out any code in myprog.c that you won't be using in your iPhone program. I'll bet that there is plenty.
As far as why the program is slowing down, this could have to do with myprog being written to make use of some resources that aren't available on the iPhone. The first thing that comes to mind is large amounts of RAM, since many desktop applications are written as though available RAM is infinite, and I could see how some .jpg manipulation code could be written this way. The way to get around this would be to try to rework the algorithm so that it did not load as much of the picture at one time while working on it.
The second thing that come is floating point code. Floating point operations are common in image manipulation code, but often either not available or severely limited in embedded systems. In the case of iPhones they are available, but according to something I heard, their performance is noticeably hampered if you compile your code to thumb rather than regular ARM code. (I've never developed for an iPhone or its particular processor so I don't know for sure, but it is worth looking into).
Another place where things could be slowing down would be if there were some sort of translation between Objective C objects and C structures that you have somehow introduced and is happening a lot more often than it should need to. There are probably other slow downs that could happen because of this, but you might be able to test this theory out by creating a objective C program for your desktop that uses the myprog.c code in a manner similar to the iPhone program's use of it.
Another thing you probably should look into is profiling your iPhone program. Profiling determines (or only helps to determine, in some cases) where the program is spending its time. Knowing this doesn't necessarily tell you that the code that runs the most is bad or that anything about it could be improved, but it does tell you where to look. And sometimes you may look at the results and immediately know that some function that you thought was only going to be called once at the beginning of the program is actually being called repeatedly, which highly suggests that some improvement can be made.
I'm sure that a little searching will turn up how to go about this.