I know that SHA256_Update() is implemented in libcrypto under openssl, yet, a simple grep can't find its definition:
$ ack SHA256_Update
fips/fips_standalone_sha1.c
76: SHA256_Update(md_ctx,key,len);
87: SHA256_Update(md_ctx,pad,SHA256_CBLOCK);
92: SHA256_Update(o_ctx,pad,SHA256_CBLOCK);
100: SHA256_Update(o_ctx,buf,sizeof buf);
154: SHA256_Update(&md_ctx,buf,l);
evp/m_sha1.c
114: { return SHA256_Update(ctx->md_data,data,count); }
sha/sha256.c
58: SHA256_Update(&c,d,n);
71: SHA256_Update(&c,d,n);
78:{ return SHA256_Update (c,data,len); }
116:#define HASH_UPDATE SHA256_Update
All these instances are where the function gets called, but not its definition. Yet, if I do "nm libcrypto.so |grep SHA256_Update", the entry can be found.
Weird...
Anyone could shed some light here?
md32_common.h is a "poor-man's template" for C.
It defines the structure of a general update function for any hash algorithm. Each algorithm provides the name for this general structure.
So in md32_common.h you will find this:
int HASH_UPDATE (HASH_CTX *c, const void *data_, size_t len)
And in sha/sha256.c you will find:
#define HASH_UPDATE SHA256_Update
So that when md32_common.h is included, you get the function SHA256_Update defined.
At the beginning of md32_common.h you will find a more complete explanation with an example.
Related
Context: I've been benchmarking the difference between using invokedynamic and manually generating bytecode (this is in the context of deciding whether a compiler targeting the JVM should emit more verbose "traditional" bytecode or just an invokedynamic call with a clever bootstrap method). In doing this, it has been pretty straightforward to map bytecode into MethodHandles combinators that are at least as fast, with the exception of tableswitch.
Question: Is there a trick to mimic tableswitch using MethodHandle? I tried mimicking it with a jump table: using a constant MethodHandle[], indexing into that with arrayElementGetter, then calling the found handle with MethodHandles.invoker. However, that ended up being around 50% slower than the original bytecode when I ran it through JMH.
Here's the code for producing the method handle:
private static MethodHandle makeProductElement(Class<?> receiverClass, List<MethodHandle> getters) {
MethodHandle[] boxedGetters = getters
.stream()
.map(getter -> getter.asType(getter.type().changeReturnType(java.lang.Object.class)))
.toArray(MethodHandle[]::new);
MethodHandle getGetter = MethodHandles // (I)H
.arrayElementGetter(MethodHandle[].class)
.bindTo(boxedGetters);
MethodHandle invokeGetter = MethodHandles.permuteArguments( // (RH)O
MethodHandles.invoker(MethodType.methodType(java.lang.Object.class, receiverClass)),
MethodType.methodType(java.lang.Object.class, receiverClass, MethodHandle.class),
1,
0
);
return MethodHandles.filterArguments(invokeGetter, 1, getGetter);
}
Here's the initial bytecode (which I'm trying to replace with one invokedynamic call)
public java.lang.Object productElement(int);
descriptor: (I)Ljava/lang/Object;
flags: (0x0001) ACC_PUBLIC
Code:
stack=3, locals=3, args_size=2
0: iload_1
1: istore_2
2: iload_2
3: tableswitch { // 0 to 2
0: 28
1: 38
2: 45
default: 55
}
28: aload_0
29: invokevirtual #62 // Method i:()I
32: invokestatic #81 // Method java/lang/Integer.valueOf:(I)Ljava/lang/Integer;
35: goto 67
38: aload_0
39: invokevirtual #65 // Method s:()Ljava/lang/String;
42: goto 67
45: aload_0
46: invokevirtual #68 // Method l:()J
49: invokestatic #85 // Method java/lang/Long.valueOf:(J)Ljava/lang/Long;
52: goto 67
55: new #87 // class java/lang/IndexOutOfBoundsException
58: dup
59: iload_1
60: invokestatic #93 // Method java/lang/Integer.toString:(I)Ljava/lang/String;
63: invokespecial #96 // Method java/lang/IndexOutOfBoundsException."<init>":(Ljava/lang/String;)V
66: athrow
67: areturn
The good thing about invokedynamic is that it allows to postpone the decision, how to implement the operation to the actual runtime. This is the trick behind LambdaMetafactory or StringConcatFactory which may return composed method handles, like in your example code, or dynamically generated code, at the particular implementation’s discretion.
There’s even a combined approach possible, generate classes which you compose to an operation, e.g. settling on the already existing LambdaMetafactory:
private static MethodHandle makeProductElement(
MethodHandles.Lookup lookup, Class<?> receiverClass, List<MethodHandle> getters)
throws Throwable {
Function[] boxedGetters = new Function[getters.size()];
MethodType factory = MethodType.methodType(Function.class);
for(int ix = 0; ix < boxedGetters.length; ix++) {
MethodHandle mh = getters.get(ix);
MethodType actual = mh.type().wrap(), generic = actual.erase();
boxedGetters[ix] = (Function)LambdaMetafactory.metafactory(lookup,
"apply", factory, generic, mh, actual).getTarget().invokeExact();
}
Object switcher = new Object() {
final Object get(Object receiver, int index) {
return boxedGetters[index].apply(receiver);
}
};
return lookup.bind(switcher, "get",
MethodType.methodType(Object.class, Object.class, int.class))
.asType(MethodType.methodType(Object.class, receiverClass, int.class));
}
This uses the LambdaMetafactory to generate a Function instance for each getter, similar to equivalent method references. Then, an actual class calling the right Function’s apply method is instantiated and a method handle to its get method returned.
This is a similar composition as your method handles, but with the reference implementation, no handles but fully materialized classes are used. I’d expect the composed handles and this approach to converge to the same performance for a very large number of invocations, but the materialized classes having a headstart for a medium number of invocations.
I added a first parameter MethodHandles.Lookup lookup which should be the lookup object received by the bootstrap method for the invokedynamic instruction. If used that way, the generated functions can access all methods the same way as the code containing the invokedynamic instruction, including private methods of that class.
Alternatively, you can generate a class containing a real switch instruction yourself. Using the ASM library, it may look like:
private static MethodHandle makeProductElement(
MethodHandles.Lookup lookup, Class<?> receiverClass, List<MethodHandle> getters)
throws ReflectiveOperationException {
ClassWriter cw = new ClassWriter(ClassWriter.COMPUTE_FRAMES);
cw.visit(V1_8, ACC_INTERFACE|ACC_ABSTRACT,
lookup.lookupClass().getName().replace('.', '/')+"$Switch", null,
"java/lang/Object", null);
MethodType type = MethodType.methodType(Object.class, receiverClass, int.class);
MethodVisitor mv = cw.visitMethod(ACC_STATIC|ACC_PUBLIC, "get",
type.toMethodDescriptorString(), null, null);
mv.visitCode();
Label defaultCase = new Label();
Label[] cases = new Label[getters.size()];
for(int ix = 0; ix < cases.length; ix++) cases[ix] = new Label();
mv.visitVarInsn(ALOAD, 0);
mv.visitVarInsn(ILOAD, 1);
mv.visitTableSwitchInsn(0, cases.length - 1, defaultCase, cases);
String owner = receiverClass.getName().replace('.', '/');
for(int ix = 0; ix < cases.length; ix++) {
mv.visitLabel(cases[ix]);
MethodHandle mh = getters.get(ix);
mv.visitMethodInsn(INVOKEVIRTUAL, owner, lookup.revealDirect(mh).getName(),
mh.type().dropParameterTypes(0, 1).toMethodDescriptorString(), false);
if(mh.type().returnType().isPrimitive()) {
Class<?> boxed = mh.type().wrap().returnType();
MethodType box = MethodType.methodType(boxed, mh.type().returnType());
mv.visitMethodInsn(INVOKESTATIC, boxed.getName().replace('.', '/'),
"valueOf", box.toMethodDescriptorString(), false);
}
mv.visitInsn(ARETURN);
}
mv.visitLabel(defaultCase);
mv.visitTypeInsn(NEW, "java/lang/IndexOutOfBoundsException");
mv.visitInsn(DUP);
mv.visitVarInsn(ILOAD, 1);
mv.visitMethodInsn(INVOKESTATIC, "java/lang/String",
"valueOf", "(I)Ljava/lang/String;", false);
mv.visitMethodInsn(INVOKESPECIAL, "java/lang/IndexOutOfBoundsException",
"<init>", "(Ljava/lang/String;)V", false);
mv.visitInsn(ATHROW);
mv.visitMaxs(-1, -1);
mv.visitEnd();
cw.visitEnd();
lookup = lookup.defineHiddenClass(
cw.toByteArray(), true, MethodHandles.Lookup.ClassOption.NESTMATE);
return lookup.findStatic(lookup.lookupClass(), "get", type);
}
This generates a new class with a static method containing the tableswitch instruction and the invocations (as well as the boxing conversions we now have to do ourselves). Also, it has the necessary code to create and throw an exception for out-of-bounds values. After generating the class, it returns a handle to that static method.
I don't know of your timeline. But it is likely there will be a MethodHandles.tableSwitch operation in Java 17. It is currently being integrated via https://github.com/openjdk/jdk/pull/3401/
Some more discussion about it here:
https://mail.openjdk.java.net/pipermail/core-libs-dev/2021-April/076105.html
The things is, tableswitch isn't always compiled to a jump table. For a small number of labels, like in your example, it's likely to act as a binary search. Thus using a tree of regular "if-then" MethodHandles will be the closest equivalent.
I'm working on a game written in Kotlin and was looking into improving GC churn. One of the major sources of churn are for-loops called in the main game/rendering loops that result in the allocation of iterators.
Turning to the documentation, I found this paragraph:
A for loop over an array is compiled to an index-based loop that does not create an iterator object.
If you want to iterate through an array or a list with an index, you can do it this way:
for (i in array.indices)
print(array[i])
Note that this “iteration through a range” is compiled down to optimal implementation with no extra objects created.
https://kotlinlang.org/docs/reference/control-flow.html#for-loops
Is this really true? To verify, I took this simple Kotlin program and inspected the generated byte code:
fun main(args: Array<String>) {
val arr = arrayOf(1, 2, 3)
for (i in arr.indices) {
println(arr[i])
}
}
According to the quote above, this should not result in any objects allocated, but get compiled down to a good old pre-Java-5 style for-loop. However, what I got was this:
41: aload_1
42: checkcast #23 // class "[Ljava/lang/Object;"
45: invokestatic #31 // Method kotlin/collections/ArraysKt.getIndices:([Ljava/lang/Object;)Lkotlin/ranges/IntRange;
48: dup
49: invokevirtual #37 // Method kotlin/ranges/IntRange.getFirst:()I
52: istore_2
53: invokevirtual #40 // Method kotlin/ranges/IntRange.getLast:()I
56: istore_3
57: iload_2
58: iload_3
59: if_icmpgt 93
This looks to me as if a method called getIndices is called that allocates a temporary IntRange object to back up bounds checking in this loop. How is this an "optimal implementation" with "no extra objects created", or am I missing something?
UPDATE:
So, after toying around a bit more and looking at the answers, the following appears to be true for Kotlin 1.0.2:
Arrays:
for (i in array.indices): range allocation
for (i in 0..array.size): no allocation
for (el in array): no allocation
array.forEach: no allocation
Collections:
for (i in coll.indices) range allocation
for (i in 0..coll.size): no allocation
for (el in coll): iterator allocation
coll.forEach: iterator allocation
To iterate an array without allocating extra objects you can use one of the following ways.
for-loop
for (e in arr) {
println(e)
}
forEach extension
arr.forEach {
println(it)
}
forEachIndexed extension, if you need to know index of each element
arr.forEachIndexed { index, e ->
println("$e at $index")
}
As far as I know the only allocation-less way to define a for loop is
for (i in 0..count - 1)
All other forms lead to either a Range allocation or an Iterator allocation. Unfortunately, you cannot even define an effective reverse for loop.
Here is an example of preparing a list and iterate with index and value.
val list = arrayListOf("1", "11", "111")
for ((index, value) in list.withIndex()) {
println("$index: $value")
}
Output:
0:1
1:11
2:111
Also, following code works similar,
val simplearray = arrayOf(1, 2, 3, 4, 5)
for ((index, value) in simplearray.withIndex()) {
println("$index: $value")
}
I am beginner to gtest. I trying to use ASSERT_THROW will compilation fail. Could anyone help on this:
class my_exp {};
int main(int argc, char *argv[])
{
EXPECT_THROW(throw my_exp(), my_exp); // this will pass
// This will through below compilation error
ASSERT_THROW(throw my_exp(), my_exp);
return 0;
}
Compilation output:
ERROR :
In file included from /usr/include/gtest/gtest.h:57:0,
from gtest.cpp:1:
gtest.cpp: In function ‘int main(int, char**)’:
gtest.cpp:12:3: error: void value not ignored as it ought to be
ASSERT_THROW(throw my_exp(), my_exp);
^
Short version
You write test in the wrong way, to write test you should put assertion inside test (macro TEST) or test fixtures (macro TEST_F).
Long version
1 . What's really happens?
To find out the real problem is not easy because the Google Testing Framework use macros which hide real code. To see code after macro substitution is required to perform preprocessing, something like this:
g++ -E main.cpp -o main.p
The result of preprocessing when using ASSERT_THROW will be looks like this (after formatting):
class my_exp {};
int main(int argc, char *argv[])
{
switch (0)
case 0:
default:
if (::testing::internal::ConstCharPtr gtest_msg = "") {
bool gtest_caught_expected = false;
try {
if (::testing::internal::AlwaysTrue () ) {
throw my_exp ();
};
} catch (my_exp const &) {
gtest_caught_expected = true;
} catch (...) {
gtest_msg.value = "Expected: throw my_exp() throws an exception of type my_exp.\n Actual: it throws a different type.";
goto gtest_label_testthrow_7;
} if (!gtest_caught_expected) {
gtest_msg.value = "Expected: throw my_exp() throws an exception of type my_exp.\n Actual: it throws nothing.";
goto gtest_label_testthrow_7;
}
}
else
gtest_label_testthrow_7:
return ::testing::internal::AssertHelper (::testing::TestPartResult::kFatalFailure, "main.cpp", 7, gtest_msg.value) = ::testing::Message ();
return 0;
}
For EXPECT_THROW result will be the same except some difference:
else
gtest_label_testthrow_7:
::testing::internal::AssertHelper (::testing::TestPartResult::kNonFatalFailure, "main.cpp", 7, gtest_msg.value) = ::testing::Message ();
2 . OK, the reason of different behaviour is found, let's continue.
In the file src/gtest.cc can be found AssertHelper class declaration including assignment operator which return void:
void AssertHelper::operator=(const Message& message) const
So now reason of the compiler complain is clarified.
3 . But why this problem is caused is not clear. Try realise why for ASSERT_THROW and EXPECT_THROW different code was generated. The answer is the macro from file include/gtest/internal/gtest-internal.h
#define GTEST_FATAL_FAILURE_(message) \
return GTEST_MESSAGE_(message, ::testing::TestPartResult::kFatalFailure)
#define GTEST_NONFATAL_FAILURE_(message) \
GTEST_MESSAGE_(message, ::testing::TestPartResult::kNonFatalFailure)
which contain return for fatal case.
4 . But now is question why this assertions usually works well?
To answer of this question try investigate code snipped which written in correct way when assertion is placed inside test:
#include <gtest/gtest.h>
class my_exp {};
TEST (MyExp, ThrowMyExp)
{
ASSERT_THROW (throw my_exp (), my_exp);
}
To exclude pollution of the answer I just notice that in such case the return statement for ASSERT_THROW also exist, but it is placed inside method:
void MyExp_ThrowMyExp_Test::TestBody ()
which return void! But in your example assertions are placed inside main function which return int. Looks like this is source of problem!
Try prove this point with simple snippet:
void f1 () {};
void f2 () {return f1();};
//int f2 () {return f1();}; // error here!
int main (int argc, char * argv [])
{
f2;
return 0;
}
5 . So the final answer is: the ASSERT_THROW macro contain return statement for expression which evaluates to void and when such expression is placed into function which return non void value the gcc complain about error.
P.S. But anyway I have no idea why for one case return is used but for other case is not.
Update: I've asked this question on GitHub and got the following answer:
ASSERT_XXX is used as a poor man's exception to allow it to work in
environments where exceptions are disabled. It does a return; instead.
It is meant to be used from within the TEST() methods, which return
void.
Update: I've just realised that this question described in the official documentation:
By placing it in a non-void function you'll get a confusing compile error > like "error: void value not ignored as it ought to be".
I'm trying to experiment with using splice (man 2 splice) to copy data from a UDP socket directly to a file. Unfortunately the first call to splice() returns EINVAL.
The man page states:
EINVAL Target file system doesn't support splicing; target file is opened in
append mode; neither of the descriptors refers to a pipe; or offset
given for nonseekable device.
However, I believe none of those conditions apply. I'm using Fedora 15 (kernel 2.6.40-4) so I believe splice() is supported on all filesystems. The target file should be irrelevant in the first call to splice, but for completeness I'm opening it via open(path, O_CREAT | O_WRONLY | O_TRUNC, S_IRUSR | S_IWUSR). Both calls use a pipe and neither call uses an offset besides NULL.
Here's my sample code:
int sz = splice(sock_fd, 0, mPipeFds[1], 0, 8192, SPLICE_F_MORE);
if (-1 == sz)
{
int err = errno;
LOG4CXX_ERROR(spLogger, "splice from: " << strerror(err));
return 0;
}
sz = splice(mPipeFds[0], 0, file_fd, 0, sz, SPLICE_F_MORE);
if (-1 == sz)
{
int err = errno;
LOG4CXX_ERROR(spLogger, "splice to: " << strerror(err));
}
return 0;
sock_fd is initialized by the following psuedocode:
int sock_fd = socket(AF_INET, SOCK_DGRAM, IPPROTO_UDP);
setsockopt(sock_fd, SOL_SOCKET, SO_REUSEADDR, &one, sizeof(one));
fcntl(sock_fd, F_SETFL, flags | O_NONBLOCK);
bind(sock_fd, ...);
Possibly related is that this code snippet is running inside a libevent loop. libevent is using epoll() to determine if the UDP socket is hot.
Found my answer. tl;dr - UDP isn't supported on the inbound side.
After enough Googling I stumbled upon a forum discussion and some test code which prints out a table of in/out fd types and their support:
$ ./a.out
in\out pipe reg chr unix tcp udp
pipe yes yes yes yes yes yes
reg yes no no no no no
chr yes no no no no no
unix no no no no no no
tcp yes no no no no no
udp no no no no no no
Yeah, it is definitely not supported for reading from a UDP socket, even in the latest kernels. References to the kernel source follow.
splice invokes do_splice in the kernel, which calls do_splice_to, which calls the splice_read member in the file_operations structure for the file.
For sockets, that structure is defined as socket_file_ops in net/socket.c, which initializes the splice_read field to sock_splice_read.
That function, in turn, contains this line of code:
if (unlikely(!sock->ops->splice_read))
return -EINVAL;
The ops field of the socket is a struct proto_ops. For an IPv4 UDP socket, it is initialized to inet_dgram_ops in net/ipv4/af_inet.c. Finally, that structure does not explicitly initialize the splice_read field of struct proto_ops; i.e., it initializes it to zero.
So sock_splice_read returns -EINVAL, and that propagates up.
I'm trying to allow users to simply hit Enter without typing anything, and use this to mean accepting a default value. scanf isn't doing what I want and the app still 'blocks': the next line of code doesn't run.
The only way is to actually type something THEN press Enter.
I tried using NSFileHandle and fileHandleWithStandardInput; however, it seems that the user is now forced to hit Ctrl-D to indicate EOF.
Someone suggested using fgets, but I cannot work out what to pass as 3rd parameter (of FILE* type). Tried stdin but it doesn't 'block'.
How do I accept input from a user, using Objective-C, and at the same time allow the user to simply hit Enter without being forced to type anything? How do I read a single line, even if that line is blank?
Assuming the code doesn't block and the next line runs immediately (as you seemed to indicate early in the question and in a comment), you have a common problem when mixing non-line-based and line-based input.
What happens is you have a newline left in the buffer, and fgets sees that, reads it, and returns, instead of doing what you really want: ignoring it, and then reading a line.
The solution is to simply do the ignoring part yourself, and then call fgets:
#include <stdio.h>
#include <string.h>
FILE* ignoreline(FILE* stream) {
for (int c; (c = fgetc(stream)) != EOF;) {
if (c == '\n') break;
}
return stream;
}
void example_use() {
char buf[1000];
ignoreline(stdin);
fgets(buf, sizeof buf, stdin);
// or, since it returns the stream, can be more compact:
fgets(buf, sizeof buf, ignoreline(stdin));
}
int main() { // error handling omitted
int n;
printf("Enter a number: ");
scanf("%d", &n);
char buf[1000];
printf("Enter a line: ");
ignoreline(stdin); // comment this line and compare the difference
fgets(buf, sizeof buf, stdin);
*strchr(buf, '\n') = '\0';
printf("You entered '%s'.\n", buf);
return 0;
}
Note that it is also common and encouraged to "pair" the ignoreline with the scanf (or other non-line-based input) to turn that into line-based input. You may want to modify it, in that case, so you can tell the difference between input of "42 abc" and "42" (in the "Enter a number" case). Some people just use fgets everywhere, then parse that line with sscanf, and while that works, it's not necessary.
I use getch(); in library conio.h
simply the program waits for any key to be pressed
If you're using Windows, you can use the ReadConsoleInput function (see MSDN for more on this) :
INPUT_RECORD keyin;
DWORD r;
while (ReadConsoleInput(GetStdHandle(STD_INPUT_HANDLE),&keyin,1,&r)) {
if (keyin.EventType!=KEY_EVENT) continue;
if (keyin.Event.KeyEvent.wVirtualKeyCode==VK_SPACE) break; ///use these VK codes to get any key's input
if (keyin.Event.KeyEvent.wVirtualKeyCode==VK_F1)
{
printf("You pressed F1\n");
}
if (keyin.Event.KeyEvent.wVirtualKeyCode==VK_F2)
{
printf("You pressed F2\n",);
}
}//end while loop
You don't need to hit enter after each key then.This works like a dream for me...
use getchar() to take input without using scanf function...