What Clever Solutions are There for Including Resources in a Static Lib? - objective-c

I have a static library Xcode 4 project that includes a home-brewed rendering engine, and I re-use this engine in multiple apps. This engine uses OpenGL ES 2.0, and by extension, shaders. As shaders got more complicated, I moved away from storing them as NSStrings in a source file, and now store them as standalone text files with the .vert and .frag extensions.
This works fine for apps that include the rendering engine in their own source; the shaders are simply added to the app's "Copy Bundle Resources" build phase, and loaded at runtime into NSStrings and compiled, linked, etc.
This strategy doesn't work at all if the rendering engine that loads these shaders is in a static library project; there is no bundle into which to copy resources. I'm currently forced to have every client project of the static lib rendering engine include their own copies of the shaders in their own "Copy Bundle Resources" build phase. This is a giant pain, and defeats a large part of the convenience of making the render engine into a static lib in the first place.
I suppose this is specific instance of the more general problem of "Resources in a Static Library". The best solution I can think of is copying the shader files' contents into strings in a header file, which are then included in the rendering engine's source. I may even be able to automate the conversion from .frag to .h with some "Run Scripts" build phase magic, but it seems unfortunately complicated.
Is there anything I'm missing?

For the benefit of posterity, I'll share the solution I ended up using. At a high level, the solution is to compile the resource in question into the application binary, thus obviating the need to also copy it to bundle resources.
I decided a generic and reliable way to compile any file data into the binary would be to store the file contents in a static byte array in a header file. Assuming there is already a header file created and added to the static lib target, I made the following bash script to read a file, and write its contents as a byte array of hex literals with C syntax. I then run the script in "Run Script" build phase before the Compile Sources and Copy Headers build phases:
#!/bin/bash
# Hexify.sh reads an input file, and hexdumps its contents to an output
# file in C-compliant syntax. The final argument is the name of the array.
infile=$1
outfile=$2
arrayName=$3
fileSize=$(stat -f "%z" $infile)
fileHexString=$(hexdump -ve '1/1 "0x%.2x, "' $infile)
prefix=$arrayName
suffix="Size"
variableName=$arrayName$suffix
nullTermination="0x00"
echo "//" > $headerFile
echo "// This file was automatically generated by a build script." >> $headerFile
echo "// Do not modify; the contents of this file will be overwritten on each build." >> $headerFile
echo "//" >> $headerFile
echo "" >> $headerFile;
echo "#ifndef some_arbitrary_include_guard" >> $headerFile
echo "#define some_arbitrary_include_guard" >> $headerFile
echo "" >> $headerFile
echo "static const int $variableName = $((fileSize+1));" >> $outfile
echo "static const char $arrayName[$variableName] = {" >> $outfile
echo -e "\t$fileHexString$nullTermination" >> $outfile
echo "};" >> $outfile
echo "#endif" >> $headerFile
So, for example, if I have a resource file example.txt:
Hello this
is a file
And I were to run ./Hexify.sh example.txt myHeader.h exampleArray, the header would look like this:
//
// This file was automatically generated by a build script.
// Do not modify; the contents of this file will be overwritten on each build.
//
#ifndef some_arbitrary_include_guard
#define some_arbitrary_include_guard
static const int exampleArraySize = 21;
static const char exampleArray[exampleArraySize] = {
0x48, 0x65, 0x6c, 0x6c, 0x6f, 0x20, 0x74, 0x68, 0x69, 0x73, 0x0a,
0x69, 0x73, 0x20, 0x61, 0x20, 0x66, 0x69, 0x6c, 0x65, 0x00
};
#endif
Now, at any point in time that I would have loaded said resource from the main bundle, I can instead refer to the byte array in that header file. Note that my version of the script adds a null terminated byte, which makes the data suitable for creating string objects. That may not apply in all cases.
As one final addendum, I apologize if that bash script makes any real bash programmers cringe; I have almost no idea what I'm doing with bash.

I feel your pain buddy, static libraries and resources don't go well together. I think the easiest way to do this is the one you already mentioned: Write a script that reads your shaders, escapes them properly and wraps them in C-compliant code.
I'm no expert, but maybe you could add the shader data to some section of your Mach-O executable upon linkage? But this eventually boils down to the same solution as mentioned above, with the only disadvantage that you're left with the ugly part of the job.
I'd go for the string constants using some shell script. PHP in my experience is pretty good at doing this kind of work. And of course bash scripts, but I'm not too good at that.

You could try to create a framework, it seems to fit your needs. There's an example on how to create such a framework for iOS on this page:
http://db-in.com/blog/2011/07/universal-framework-iphone-ios-2-0/
The guy that wrote the guide actually uses this technique to distribute his own iOS 3D engine project.
Edit: linked to newer version of the guide.

Related

CMake: How can I compile defines and flags as string constants into my C(++) program?

I'd like my C or C++ program that is built via CMake to be able to print (or otherwise make use of) the macro definitions and (other) C/C++ flags it was compiled with. So I want CMake to generate/configure a header or source file that defines respective strings constants and that is then built as part of/into my program.
CMake features several commands (like file() or execute_process()) that would be executed when (respectively before) the build system is generated and thus would allow me to write such a source file, but I'm having trouble with getting the effective macro definitions and flags used for my target. E.g. there seem to be COMPILE_DEFINITIONS for the directory, the target, and for the configuration. Is there a way to get the macro definitions/C(++) flags that are effectively used for building my target? And how do I best write them into a source file?
I've noticed, when using the Makefiles generator apparently a file "${CMAKE_CURRENT_BINARY_DIR}/CMakeFiles/MyTarget.dir/flags.make" is created, which seems to contain pretty much what I'm looking for. So if there's no other way, I can probably make use of that file, but obviously that won't work for other generators and it comes with its own challenges (the file is generated after execute_process()).
The approach I finally went with sets the CXX_COMPILER_LAUNCHER property to use a compiler wrapper script that injects the actual compiler command line into a source file. Since I have multiple libraries/executables to which I want to add the respective information, I use a CMake function that adds a source file containing the info to the target.
function(create_module_build_info _target _module _module_include_dir)
# generate BuildInfo.h and BuildInfo.cpp
set (BUILD_MODULE ${_module})
set (BUILD_MODULE_INCLUDE_DIR ${_module_include_dir})
configure_file(${CMAKE_SOURCE_DIR}/BuildInfo.h.in
${CMAKE_BINARY_DIR}/include/${_module_include_dir}/BuildInfo.h
#ONLY)
configure_file(${CMAKE_SOURCE_DIR}/BuildInfo.cpp.in
${CMAKE_CURRENT_BINARY_DIR}/BuildInfo.cpp
#ONLY)
# Set our wrapper script as a compiler launcher for the target. For
# BuildInfo.cpp we want to inject the build info.
get_property(_launcher TARGET ${_target} PROPERTY CXX_COMPILER_LAUNCHER)
set_property(TARGET ${_target} PROPERTY CXX_COMPILER_LAUNCHER
${CMAKE_SOURCE_DIR}/build_info_compiler_wrapper.sh ${_launcher})
get_property(_compile_flags SOURCE BuildInfo.cpp PROPERTY COMPILE_FLAGS)
set_property(SOURCE BuildInfo.cpp PROPERTY COMPILE_FLAGS
"${_compile_flags} -D_BUILD_INFO=${CMAKE_CURRENT_BINARY_DIR}/BuildInfo_generated.cpp,${_module}")
# add BuildInfo.cpp to target
target_sources(${_target} PRIVATE BuildInfo.cpp)
endfunction()
The function can simply be called after defining the target. Parameters are the target, a name that is used as a prefix of the constant name to be generated, and a name that is part of the path of the header file to be generated. The compiler flag -D_BUILD_INFO=... is only added to the generated source file and it will be used by the wrapper script as an indicator that the constant definition should be added to that source file. All other compiler lines are just invoked as is by the script.
The template source file "BuildInfo.cpp.in":
#include "#BUILD_MODULE_INCLUDE_DIR#/BuildInfo.h"
The template header file "BuildInfo.h.in":
#pragma once
#include <string>
extern const std::string #BUILD_MODULE#_COMPILER_COMMAND_LINE;
The compiler wrapper script "build_info_compiler_wrapper.sh":
#!/usr/bin/env bash
set -e
function createBuildInfoTempFile()
{
local source="$1"
local target="$2"
local prefix="$3"
local commandLine="$4"
cp "$source" "$target"
cat >> "$target" <<EOF
const std::string ${prefix}_COMPILER_COMMAND_LINE = "$commandLine";
EOF
}
# Process script arguments. We copy them to array variable args. If we find an
# argument "-D_BUILD_INFO=*", we remove it and will inject the build info
# variable definition into (a copy of) the input file.
generateBuildInfo=false
buildInfoTempFile=
buildInfoVariablePrefix=
args=()
while [ $# -ge 1 ]; do
case "$1" in
-D_BUILD_INFO=*)
if [[ ! "$1" =~ -D_BUILD_INFO=([^,]+),(.+) ]]; then
echo "error: failed to get arguments for build info generation" >&2
exit 1
fi
generateBuildInfo=true
buildInfoTempFile="${BASH_REMATCH[1]}"
buildInfoVariablePrefix="${BASH_REMATCH[2]}"
shift
continue
;;
esac
args+=("$1")
shift
done
if $generateBuildInfo; then
# We expect the last argument to be the source file. Check!
case "${args[-1]}" in
*.c|*.cxx|*.cpp|*.cc)
createBuildInfoTempFile "${args[-1]}" "$buildInfoTempFile" "$buildInfoVariablePrefix" "${args[*]}"
args[-1]="$buildInfoTempFile"
;;
*)
echo "error: Failed to find source file in compiler arguments for build info generation feature." >&2
exit 1
;;
esac
fi
"${args[#]}"
Obviously the script can be made smarter. E.g. instead of assuming it is the last argument it could find the actual index of the input source file. It could also process the command line to separate preprocessor definitions, include paths, and other flags.
Note that "-D_BUILD_INFO=..." argument is used instead of some parameter that the compiler wouldn't know (e.g. "--generate-build-info"), so that IDEs won't run into issues when passing the arguments directly to the compiler for whatever purposes.

Make a module out a bunch of .f source files

I downloaded some Fortran source code which contains around 50 or so .f files. Each .f file contains one subroutine or function. I'd like to conveniently stuff all of these into a Fortran module .mod file. Is there any way of doing this with gfortran (besides painstakingly writing out each sub/func prototype inside a module definition)?
This is more of a formatted comment than an answer, but you could try writing a source file something like this
module this_will_end_in_tears
contains
include 'subroutine1.f'
include 'subroutine2.f'
include 'subroutine2.f'
...
end module this_will_end_in_tears
If, as I suspect, the .f files contain fixed-form source, then you had better make sure the module is also in fixed-form.
I expect, as you may have guessed, that this will not compile the first time you try it, but it might (just might) save you a little time over doing the job right by, as you put it, painstakingly writing out each sub/func ...
Oh, and lest any of the Fortran-lovers out there cringe at this suggestion, note that I'm not saying it's a good way to proceed, just that it might save you a little time.
Good luck.
I made this bash script that takes two parameters 'the module name' and 'the file extension' then builds a module file. It's presently made for fixed-format. All the source files must be present in the same directory as the script. Of course you can modify it to be more flexible.
name=$1
ext=$2
modsrc=$name.$ext
echo -e " MODULE $name\n" > $modsrc
echo -e " implicit none\n" >> $modsrc
echo -e " contains\n" >> $modsrc
for i in *.f;
do
if [ "$i" != "$modsrc" ]; then
echo -e " include '$i'\n" >> $modsrc
fi
done
echo -e " END MODULE $name\n" >> $modsrc
I usually tackle things by compiling even .f90 code as -fixed. Then usually add in INTENT and IMPLICIT NONE. Almost always I have D-Lines for debug which seems to work easiest for me with -fixed form. Later adding in alignment pragmas and OpenMP reduction clauses and related vector stuff.

In an ELF file, how does the address for _start get detemined?

I've been reading the ELF specification and cannot figure out where the program entry point and _start address come from.
It seems like they should have to be in a pretty consistent place, but I made a few trivial programs, and _start is always in a different place.
Can anyone clarify?
The _start symbol may be defined in any object file. Normally it is generated automatically (it corresponds to main in C). You can generate it yourself, for instance in an assembler source file:
.globl _start
_start:
// assembly here
When the linker has processed all object files it looks for the _start symbol and puts its value in the e_entry field of the elf header. The loader takes the address from this field and makes a call to it after it has finished loading all sections in memory and is ready to execute the file.
Take a look at the linker script ld is using:
ld -verbose
The format is documented at: https://sourceware.org/binutils/docs-2.25/ld/Scripts.html
It determines basically everything about how the executable will be generated.
On Binutils 2.24 Ubuntu 14.04 64-bit, it contains the line:
ENTRY(_start)
which sets the entry point to the _start symbol (goes to the ELF header as mentioned by ctn)
And then:
. = SEGMENT_START("text-segment", 0x400000) + SIZEOF_HEADERS;
which sets the address of the first headers to 0x400000 + SIZEOF_HEADERS.
I have modified that address to 0x800000, passed my custom script with ld -T and it worked: readelf -s says that _start is at that address.
Another way to change it is to use the -Ttext-segment=0x800000 option.
The reason for using 0x400000 = 4Mb = getconf PAGE_SIZE is to start at the beginning of the second page as asked at: Why is the ELF execution entry point virtual address of the form 0x80xxxxx and not zero 0x0?
A question describes how to set _start from the command line: Why is the ELF entry point 0x8048000 not changeable with the "ld -e" option?
SIZEOF_HEADERS is the size of the ELF + program headers, which are at the beginning of the ELF file. That data gets loaded into the very beginning of the virtual memory space by Linux (TODO why?) In a minimal Linux x86-64 hello world with 2 program headers it is worth 0xb0, so that the _start symbol comes at 0x4000b0.
I'm not sure but try this link http://www.docstoc.com/docs/23942105/UNIX-ELF-File-Format
at page 8 it is shown where the entry point is if it is executable. Basically you need to calculate the offset and you got it.
Make sure to remember the little endianness of x86 ( i guess you use it) and reorder if you read bytewise edit: or maybe not i'm not quit sure about this to be honest.

SCONS: making a special script builder depend on output of another builder

I hope the title clarifies what I want to ask because it is a bit tricky.
I have a SCONS SConscript for every subdir as follows (doing it in linux, if it matters):
src_dir
compiler
SConscript
yacc srcs
scripts
legacy_script
data
SConscript
data files for the yacc
I use a variant_dir without copy, for example:
SConscript('src_dir/compiler/SConscript', variant_dir = 'obj_dir', duplicate = 0)
The resulting obj_dir after building the yacc is:
obj_dir
compiler
compiler_compiler.exe
Now here is the deal.
I have another SConscript in the data dir that needs to do 2 things:
1. compile the data with the yacc compiled compiler
2. Take the output of the compiler and run it with the legacy_script I can't change
(the legacy_script, takes the output of the compiled data and build some h files for another software to depend on)
number 1 is acheived easily:
linux_env.Command('[output1, output2]', 'data/data_files','compiler_compiler.exe data_files output1 output2')
my problem is number 2: How do I make the script runner depend on outputs of another target
And just to clarify it, I need to make SCONS run (and only if compiler_output changes):
src_dir/script/legacy_script obj_dir/data/compiler_output obj_dir/some_dir/script_output
(the script is usage is: legacy_script input_file output_file)
I hope I made myself clear, feel free to ask some more questions...
I've had a similar problem recently when I needed to compile Cheetah Templates first, which were then used from another Builder to generate HTML files from different sources.
If you define the build output of the first builder as source for the second builder, SCons will run them in the correct order and only if intermediate files have changed.
Wolfgang

Does Xcode's objective-c compiler optimize for bit shifts?

Does the objective-c compiler in Xcode know better, or is it faster if I use bit shift for multiplications and divisions by powers of 2?
NSInteger parentIndex = index >> 1; // integer division by 2
Isn't this a bit 1980's? Don't processors run these instructions in the same time these days? I remember back in my 68000 days when a div was 100+ cycles and a shift only 3 or 4... not sure this is the case any more as processors have moved on.
Why don't you get the compiler to generate the assembler file and have a look what it's generating and run some benchmarks.
I found this on the web which may help you... although it's for 'C' I think most of the options will be the same.
Q: How can I peek at the assembly code generated by GCC?
Q: How can I create a file where I can see the C code and its assembly
translation together?
A: Use the -S (note: capital S) switch to GCC, and it will emit the assembly code to a file with a .s extension. For example, the following command:
gcc -O2 -S -c foo.c
will leave the generated assembly code on the file foo.s.
If you want to see the C code together with the assembly it was converted to, use a command line like this:
gcc -c -g -Wa,-a,-ad [other GCC options] foo.c > foo.lst
which will output the combined C/assembly listing to the file foo.lst.
If you need to both get the assembly code and to compile/link the program, you can either give the -save-temps option to GCC (which will leave all the temporary files including the .s file in the current directory), or use the -Wa,aln=foo.s option which instructs the assembler to output the assembly translation of the C code (together with the hex machine code and some additional info) to the file named after the =.