What happens if Xmx is not used? - jvm

According to the documentation of the JVM, if you use too big of a Xms paramenter, the JVM won't start. So, I ask, what happens if I don't use it? Is my VM allowed to grow indefinitely? Would it stop only when physical memory runs out?

XMX is the max heap size.
what happens if I don't use it?
If it is omitted, it uses the default. The default varies by JVM version, and what platform it's running on. The details of version 5 are here.
On server-class machines by default the following are selected.
...
maximum heap size of ΒΌ of physical memory up to 1Gbyte

If you don't specify -Xmx then you will get the default allocation for your operating system, your jvm, it's options and version.
Given the number things that might affect the value and the time it takes to navigate the documenation, it might just be easier to ask your jvm.
For instance on my Linux system:
$ java -XX:+PrintFlagsFinal -version 2>&1 | grep -i -E 'heapsize|permsize|version'
uintx AdaptivePermSizeWeight = 20 {product}
uintx ErgoHeapSizeLimit = 0 {product}
uintx InitialHeapSize := 66328448 {product}
uintx LargePageHeapSizeThreshold = 134217728 {product}
uintx MaxHeapSize := 1063256064 {product}
uintx MaxPermSize = 67108864 {pd product}
uintx PermSize = 16777216 {pd product}
java version "1.6.0_24"
as it defaults to -server, but with -client I get:
$ java -client -XX:+PrintFlagsFinal -version 2>&1 | grep -i -E 'heapsize|permsize|version'
uintx AdaptivePermSizeWeight = 20 {product}
uintx ErgoHeapSizeLimit = 0 {product}
uintx InitialHeapSize := 16777216 {product}
uintx LargePageHeapSizeThreshold = 134217728 {product}
uintx MaxHeapSize := 268435456 {product}
uintx MaxPermSize = 67108864 {pd product}
uintx PermSize = 12582912 {pd product}
java version "1.6.0_24"
While on my Windows system, I get:
C:\>java -XX:+PrintFlagsFinal -version 2>&1 | findstr /I "heapsize permsize version"
uintx AdaptivePermSizeWeight = 20 {product}
uintx ErgoHeapSizeLimit = 0 {product}
uintx InitialHeapSize := 16777216 {product}
uintx LargePageHeapSizeThreshold = 134217728 {product}
uintx MaxHeapSize := 268435456 {product}
uintx MaxPermSize = 67108864 {pd product}
uintx PermSize = 12582912 {pd product}
java version "1.6.0_21"
which are the -client settings and there appears to be no -server option:
C:\>java -server -XX:+PrintFlagsFinal -version 2>&1 | findstr /I "heapsize permsize version"
C:\>java -server -XX:+PrintFlagsFinal -version
Error: no `server' JVM at `C:\jdk\jre\bin\server\jvm.dll'.

Related

Early SSL Termination

SSL works from a VPN client but fails over the GCP-PaloAlto VPN tunnel.
The server is an on-prem Keycloak (version 13.0.0)
Attempt from VPN Client (Successful)
# openssl s_client -connect fqdn:443 -servername fqdn -tls1_2 --prexit
CONNECTED(00000005)
[...]
depth=0 CN = <fqdn>
verify return:1
---
Certificate chain
0 s:CN = <fqdn>
[...]
---
Server certificate
[...]
subject=CN = <fqdn>
issuer=C = US, O = Let's Encrypt, CN = R3
---
[...]
---
SSL handshake has read 4672 bytes and written 311 bytes
Verification: OK
---
New, TLSv1.2, Cipher is ECDHE-RSA-AES256-GCM-SHA384
Server public key is 2048 bit
Secure Renegotiation IS supported
[...]
---
closed
---
Certificate chain
0 s:CN = <fqdn>
[...]
---
Server certificate
[...]
subject=CN = <fqdn>
issuer=C = US, O = Let's Encrypt, CN = R3
[...]
tcpdump from the working client
19:32:09.582389 IP CLIENT.51013 > SERVER.https: Flags [S], seq 3647907685, win 65535, options [mss 1360,nop,wscale 6,nop,nop,TS val 350978343 ecr 0,sackOK,eol], length 0
19:32:09.582551 IP SERVER.https > CLIENT.51013: Flags [S.], seq 3495009322, ack 3647907686, win 65160, options [mss 1460,sackOK,TS val 2099619205 ecr 350978343,nop,wscale 7], length 0
19:32:09.596385 IP CLIENT.51013 > SERVER.https: Flags [.], ack 1, win 2064, options [nop,nop,TS val 350978356 ecr 2099619205], length 0
19:32:09.596385 IP CLIENT.51013 > SERVER.https: Flags [P.], seq 1:219, ack 1, win 2064, options [nop,nop,TS val 350978356 ecr 2099619205], length 218
19:32:09.596502 IP SERVER.https > CLIENT.51013: Flags [.], ack 219, win 508, options [nop,nop,TS val 2099619219 ecr 350978356], length 0
19:32:09.596976 IP SERVER.https > CLIENT.51013: Flags [P.], seq 1:2697, ack 219, win 508, options [nop,nop,TS val 2099619219 ecr 350978356], length 2696
19:32:09.597009 IP SERVER.https > CLIENT.51013: Flags [P.], seq 2697:4097, ack 219, win 508, options [nop,nop,TS val 2099619219 ecr 350978356], length 1400
19:32:09.599161 IP SERVER.https > CLIENT.51013: Flags [P.], seq 4097:4415, ack 219, win 508, options [nop,nop,TS val 2099619221 ecr 350978356], length 318
19:32:09.612262 IP CLIENT.51013 > SERVER.https: Flags [.], ack 1349, win 2043, options [nop,nop,TS val 350978368 ecr 2099619219], length 0
19:32:09.612262 IP CLIENT.51013 > SERVER.https: Flags [.], ack 2697, win 2022, options [nop,nop,TS val 350978368 ecr 2099619219], length 0
19:32:09.612262 IP CLIENT.51013 > SERVER.https: Flags [.], ack 4045, win 2000, options [nop,nop,TS val 350978368 ecr 2099619219], length 0
19:32:09.612262 IP CLIENT.51013 > SERVER.https: Flags [.], ack 4097, win 2000, options [nop,nop,TS val 350978368 ecr 2099619219], length 0
19:32:09.612262 IP CLIENT.51013 > SERVER.https: Flags [.], ack 4097, win 2048, options [nop,nop,TS val 350978368 ecr 2099619219], length 0
19:32:09.614194 IP CLIENT.51013 > SERVER.https: Flags [.], ack 4415, win 2043, options [nop,nop,TS val 350978369 ecr 2099619221], length 0
19:32:09.614194 IP CLIENT.51013 > SERVER.https: Flags [P.], seq 219:312, ack 4415, win 2048, options [nop,nop,TS val 350978370 ecr 2099619221], length 93
19:32:09.614248 IP SERVER.https > CLIENT.51013: Flags [.], ack 312, win 508, options [nop,nop,TS val 2099619236 ecr 350978370], length 0
19:32:09.614711 IP SERVER.https > CLIENT.51013: Flags [P.], seq 4415:4673, ack 312, win 508, options [nop,nop,TS val 2099619237 ecr 350978370], length 258
19:32:09.626178 IP CLIENT.51013 > SERVER.https: Flags [.], ack 4673, win 2043, options [nop,nop,TS val 350978384 ecr 2099619237], length 0
19:33:09.651599 IP SERVER.https > CLIENT.51013: Flags [P.], seq 4673:4704, ack 312, win 508, options [nop,nop,TS val 2099679274 ecr 350978384], length 31
19:33:09.651690 IP SERVER.https > CLIENT.51013: Flags [F.], seq 4704, ack 312, win 508, options [nop,nop,TS val 2099679274 ecr 350978384], length 0
19:33:09.678658 IP CLIENT.51013 > SERVER.https: Flags [.], ack 4704, win 2047, options [nop,nop,TS val 351038150 ecr 2099679274], length 0
19:33:09.678806 IP CLIENT.51013 > SERVER.https: Flags [.], ack 4705, win 2047, options [nop,nop,TS val 351038150 ecr 2099679274], length 0
19:33:09.680749 IP CLIENT.51013 > SERVER.https: Flags [P.], seq 312:343, ack 4705, win 2048, options [nop,nop,TS val 351038150 ecr 2099679274], length 31
19:33:09.680749 IP CLIENT.51013 > SERVER.https: Flags [F.], seq 343, ack 4705, win 2048, options [nop,nop,TS val 351038150 ecr 2099679274], length 0
19:33:09.680824 IP SERVER.https > CLIENT.51013: Flags [R], seq 3495014027, win 0, length 0
19:33:09.680888 IP SERVER.https > CLIENT.51013: Flags [R], seq 3495014027, win 0, length 0
Attempt from GCP Instance (Unsuccessful)
# openssl s_client -connect fqdn:443 -servername fqdn -tls1_2 --prexit
CONNECTED(00000003)
write:errno=0
---
no peer certificate available
---
No client certificate CA names sent
---
SSL handshake has read 0 bytes and written 212 bytes
Verification: OK
---
New, (NONE), Cipher is (NONE)
Secure Renegotiation IS NOT supported
[...]
SSL-Session:
Protocol : TLSv1.2
[...]
---
tcpdump from the client that doesn't work
19:36:01.818130 IP CLIENT.55282 > SERVER.https: Flags [S], seq 3880590205, win 65320, options [mss 1350,sackOK,TS val 3904950778 ecr 0,nop,wscale 7], length 0
19:36:01.818241 IP SERVER.https > CLIENT.55282: Flags [S.], seq 1198318204, ack 3880590206, win 65160, options [mss 1460,sackOK,TS val 2676835275 ecr 3904950778,nop,wscale 7], length 0
19:36:01.829890 IP CLIENT.55282 > SERVER.https: Flags [.], ack 1, win 511, options [nop,nop,TS val 3904950791 ecr 2676835275], length 0
19:37:01.890295 IP SERVER.https > CLIENT.55282: Flags [F.], seq 1, ack 1, win 510, options [nop,nop,TS val 2676895347 ecr 3904950791], length 0
19:37:02.105125 IP SERVER.https > CLIENT.55282: Flags [F.], seq 1, ack 1, win 510, options [nop,nop,TS val 2676895562 ecr 3904950791], length 0
19:37:02.321181 IP SERVER.https > CLIENT.55282: Flags [F.], seq 1, ack 1, win 510, options [nop,nop,TS val 2676895778 ecr 3904950791], length 0
19:37:02.753156 IP SERVER.https > CLIENT.55282: Flags [F.], seq 1, ack 1, win 510, options [nop,nop,TS val 2676896210 ecr 3904950791], length 0
19:37:03.617211 IP SERVER.https > CLIENT.55282: Flags [F.], seq 1, ack 1, win 510, options [nop,nop,TS val 2676897074 ecr 3904950791], length 0
19:37:05.345180 IP SERVER.https > CLIENT.55282: Flags [F.], seq 1, ack 1, win 510, options [nop,nop,TS val 2676898802 ecr 3904950791], length 0
19:37:08.769186 IP SERVER.https > CLIENT.55282: Flags [F.], seq 1, ack 1, win 510, options [nop,nop,TS val 2676902226 ecr 3904950791], length 0
19:37:15.681150 IP SERVER.https > CLIENT.55282: Flags [F.], seq 1, ack 1, win 510, options [nop,nop,TS val 2676909138 ecr 3904950791], length 0
Not sure if it matters, but a difference I have observed is the MTU of the networks.
VPN Client can ping the server with up to size 1372
GCP node can ping the server with up to size 1362
Turns out, it was a case of asynchronous routing. Two on-prem sites were advertising the same route but only one of them had the network.
In summary:
SYN: GCP -> SITE A
ACK, SYN: SITE A -> GCP
ACK: GCP -(through SITE B)-> SITE A
FIN: SITE A doesn't like that the ACK came though a different tunnel then the one used to send the SYN.
Terminates the connection and sends a FIN.

What is the format of special section in ELF

I try to write a program to generate ELF(based on Arm and execute through qemu-arm). Most format in ELF has been well illustrated
on wiki. But I can't find any spec describe the format of special section(e.g. .text .data(especially what I want to know)).
I tried to put some initialized global variable in .data section. What format should I write in ELF(.data section) if I have global statement like: int a = 10;
There is not special format for .text and .data.
When the static linker links several .o file,
it simply concatenates the .text and .data segments (while resolving relocations)
and places them in the final .so or executable file according to the linker script (see gcc -Wl,-verbose /dev/null).
The .data segment simply contains the initial values of the instanciated global variables.
The .text segment simply contains the machine code of the routines/functions.
Let's take this simple C file:
char x[5] = {0xba, 0xbb, 0xbc, 0xbd, 0xbe};
char f(int i) {
return x[i];
}
Let's compile it:
$ gcc -c -o test.o test.c
Let's dump the .data section, using elfcat:
$ elfcat test.o --section-name .data | xxd
00000000: babb bcbd be .....
We can clearly explain the content of .data section.
Let's dump the .text section:
$ elfcat test.o --section-name .text | xxd
00000000: 5548 89e5 897d fc8b 45fc 4898 488d 1500 UH...}..E.H.H...
00000010: 0000 000f b604 105d c3
Let's decompile this:
$ elfcat test.o --section-name .text > test.text
$ r2 -a x86 -b 64 -qc pd test.text
0x00000000 55 push rbp
0x00000001 4889e5 mov rbp, rsp
0x00000004 897dfc mov dword [rbp - 4], edi
0x00000007 8b45fc mov eax, dword [rbp - 4]
0x0000000a 4898 cdqe
0x0000000c 488d15000000. lea rdx, qword [0x00000013] ; 19
0x00000013 0fb60410 movzx eax, byte [rax + rdx]
0x00000017 5d pop rbp
0x00000018 c3 ret
Again, there is nothing special in the text segment: it only contains the machine code of the routines/functions of my program.
Notice however the relocation and symbol informations in other segments:
$ readelf -a test.o
[ ... ]
Relocation section '.rela.text' at offset 0x1b8 contains 1 entry:
Offset Info Type Sym. Value Sym. Name + Addend
00000000000f 000800000002 R_X86_64_PC32 0000000000000000 x - 4
Relocation section '.rela.eh_frame' at offset 0x1d0 contains 1 entry:
Offset Info Type Sym. Value Sym. Name + Addend
000000000020 000200000002 R_X86_64_PC32 0000000000000000 .text + 0
[...]
Symbol table '.symtab' contains 10 entries:
Num: Value Size Type Bind Vis Ndx Name
0: 0000000000000000 0 NOTYPE LOCAL DEFAULT UND
1: 0000000000000000 0 FILE LOCAL DEFAULT ABS test.c
2: 0000000000000000 0 SECTION LOCAL DEFAULT 1
3: 0000000000000000 0 SECTION LOCAL DEFAULT 3
4: 0000000000000000 0 SECTION LOCAL DEFAULT 4
5: 0000000000000000 0 SECTION LOCAL DEFAULT 6
6: 0000000000000000 0 SECTION LOCAL DEFAULT 7
7: 0000000000000000 0 SECTION LOCAL DEFAULT 5
8: 0000000000000000 5 OBJECT GLOBAL DEFAULT 3 x
9: 0000000000000000 25 FUNC GLOBAL DEFAULT 1 f

clang/LLVM project level optimization

So periodically I try LLVM as I have this theory it should outperform GNU. And then it sadly doesn't.
Part of the theory has to do with its ability to link modules/objects together and THEN optimize, where normally optimization happens on a per file/object basis.
Instead of using a generic one, I see how to build for a specific default target
rm -rf llvm-project
git clone https://github.com/llvm/llvm-project.git
cd llvm-project
git checkout llvmorg-10.0.0
mkdir build
cd build
cmake -DLLVM_ENABLE_PROJECTS='clang;lld' -DCMAKE_CROSSCOMPILING=True -DCMAKE_INSTALL_PREFIX=/opt/llvm/llvm10armv6m -DLLVM_DEFAULT_TARGET_TRIPLE=armv6m-none-eabi -DLLVM_TARGET_ARCH=ARM -DLLVM_TARGETS_TO_BUILD=ARM -G "Unix Makefiles" ../llvm
make -j 8
make -j 4
make
sudo make install
And the test files
test.c
unsigned int one ( void )
{
return(1);
}
unsigned int two ( void );
unsigned int testone ( void )
{
return(one());
}
unsigned int testtwo ( void )
{
return(two());
}
two.c
unsigned int two ( void )
{
return(2);
}
basic run
clang -O2 -fomit-frame-pointer -c test.c -o test.o
llvm-objdump -D test.o
00000000 one:
0: 01 20 movs r0, #1
2: 70 47 bx lr
00000004 testone:
4: 01 20 movs r0, #1
6: 70 47 bx lr
00000008 testtwo:
8: 80 b5 push {r7, lr}
a: ff f7 fe ff bl #-4
e: 80 bd pop {r7, pc}
as one would expect, one() has been inlined into testone().
The desire is to get testwo() inlined as well.
clang -fomit-frame-pointer -c -emit-llvm test.c -o test.bc
clang -fomit-frame-pointer -c -emit-llvm two.c -o two.bc
llvm-link test.bc two.bc -o both.bc
llc both.bc -o both.s
cat both.s
opt -O2 both.bc -o both.opt.bc
llc both.opt.bc -o both.opt.s
cat both.opt.s
gives
testone:
.fnstart
# %bb.0: # %entry
.save {r7, lr}
push {r7, lr}
bl one
pop {r7, pc}
testtwo:
.fnstart
# %bb.0: # %entry
.save {r7, lr}
push {r7, lr}
bl two
pop {r7, pc}
and
testone:
.fnstart
# %bb.0: # %entry
.save {r7, lr}
push {r7, lr}
bl one
pop {r7, pc}
testtwo:
.fnstart
# %bb.0: # %entry
.save {r7, lr}
push {r7, lr}
bl two
pop {r7, pc}
that is worse.
opt -std-link-opts both.bc -o both.opt.bc
same, no better
Now this works
clang -O2 -fomit-frame-pointer -c -emit-llvm test.c -o test.bc
clang -O2 -fomit-frame-pointer -c -emit-llvm two.c -o two.bc
llvm-link test.bc two.bc -o both.bc
opt -O2 both.bc -o both.opt.bc
llc both.opt.bc -o both.opt.s
cat both.opt.s
testone:
.fnstart
# %bb.0: # %entry
movs r0, #1
bx lr
testtwo:
.fnstart
# %bb.0: # %entry
movs r0, #2
bx lr
One would think that not optimizing the parts would give more meat for the optimization of the whole to chew on. Yes? Although this indicates otherwise.
clang -fomit-frame-pointer -c -emit-llvm test.c -o test.bc
clang -fomit-frame-pointer -c -emit-llvm two.c -o two.bc
llvm-link test.bc two.bc -o both.bc
opt -O3 both.bc -o both.opt.bc
llc both.opt.bc -o both.opt.s
cat both.opt.s
testone:
.fnstart
# %bb.0: # %entry
.save {r7, lr}
push {r7, lr}
bl one
movs r0, #1
pop {r7, pc}
testtwo:
.fnstart
# %bb.0: # %entry
.save {r7, lr}
push {r7, lr}
bl two
movs r0, #2
pop {r7, pc}
-O3 doesn't help either, and this output is as pretty bad it calls the function AND inlines it. What is going on there?!
llvm-dis both.opt.bc
cat both.opt.ll
; ModuleID = 'both.opt.bc'
source_filename = "llvm-link"
target datalayout = "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64"
target triple = "thumbv6m-none-unknown-eabi"
; Function Attrs: noinline nounwind optnone
define dso_local i32 #one() local_unnamed_addr #0 {
entry:
ret i32 1
}
; Function Attrs: noinline nounwind optnone
define dso_local i32 #testone() local_unnamed_addr #0 {
entry:
%call = call i32 #one()
ret i32 1
}
; Function Attrs: noinline nounwind optnone
define dso_local i32 #testtwo() local_unnamed_addr #0 {
entry:
%call = call i32 #two()
ret i32 2
}
; Function Attrs: noinline nounwind optnone
define dso_local i32 #two() local_unnamed_addr #0 {
entry:
ret i32 2
}
How does one undo that?
clang -O2 -fomit-frame-pointer -c -emit-llvm test.c -o test.bc
clang -O2 -fomit-frame-pointer -c -emit-llvm two.c -o two.bc
llvm-link test.bc two.bc -o both.bc
llvm-dis both.bc
cat both.ll
opt -O3 both.bc -o both.opt.bc
llvm-dis both.opt.bc
cat both.opt.ll
gives
; Function Attrs: norecurse nounwind readnone
define dso_local i32 #one() local_unnamed_addr #0 {
entry:
ret i32 1
}
; Function Attrs: norecurse nounwind readnone
define dso_local i32 #testone() local_unnamed_addr #0 {
entry:
ret i32 1
}
; Function Attrs: nounwind
define dso_local i32 #testtwo() local_unnamed_addr #1 {
entry:
%call = tail call i32 #two() #2
ret i32 %call
}
; Function Attrs: norecurse nounwind readnone
define dso_local i32 #two() local_unnamed_addr #0 {
entry:
ret i32 2
}
and
; Function Attrs: norecurse nounwind readnone
define dso_local i32 #one() local_unnamed_addr #0 {
entry:
ret i32 1
}
; Function Attrs: norecurse nounwind readnone
define dso_local i32 #testone() local_unnamed_addr #0 {
entry:
ret i32 1
}
; Function Attrs: norecurse nounwind readnone
define dso_local i32 #testtwo() local_unnamed_addr #0 {
entry:
ret i32 2
}
; Function Attrs: norecurse nounwind readnone
define dso_local i32 #two() local_unnamed_addr #0 {
entry:
ret i32 2
}
So is it correct that you have to apply the optimizations everywhere, at the file/object level in order to get the project level to optimize?
And then there is the question of tail call or leaf, etc optimization, if nothing else testtwo: even in the first case
clang -O2 -fomit-frame-pointer -c test.c -o test.o
could simply branch to two() and not setup a stack frame not do any of that. Or is this a thumb thing? b cant reach?
one:
0: b8 01 00 00 00 movl $1, %eax
5: c3 retq
testone:
10: b8 01 00 00 00 movl $1, %eax
15: c3 retq
testtwo:
20: e9 00 00 00 00 jmp 0 <testtwo+5>
In gnu the linker patches up any branch reaching or mode issues with trampolines
arm-none-eabi-gcc -c -O2 -mcpu=cortex-m0 test.c -o test.o
arm-none-eabi-objdump -D test.o
00000000 <one>:
0: 2001 movs r0, #1
2: 4770 bx lr
00000004 <testone>:
4: 2001 movs r0, #1
6: 4770 bx lr
00000008 <testtwo>:
8: b510 push {r4, lr}
a: f7ff fffe bl 0 <two>
e: bd10 pop {r4, pc}
Okay I stand corrected...
clang --version
clang version 10.0.0 (https://github.com/llvm/llvm-project.git d32170dbd5b0d54436537b6b75beaf44324e0c28)
Target: armv6m-none-unknown-eabi
Thread model: posix
InstalledDir: /opt/llvm/llvm10armv6m/bin
arm-none-eabi-gcc --version
arm-none-eabi-gcc (GCC) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
I guess the question is if one wants to do a project level optimization using llvm-link and opt, is optimization of each of the individual items required or is there a command line option I am missing. Not interested in compiler specific attributes that go into the source code itself, want the code infected with neither gcc nor llvm specifics.
After gcc 5.x.x the code got more bloated was hoping that llvm would have a chance but whenever I try this (on projects not just 10 lines of code) gcc ends up with fewer executed instructions, and/or fewer memory accesses, etc, etc. For simple demonstration functions like the ones above, with some exceptions they produce the same/equivalent output.
Is there something, another one of the tools, or command line options, that I am missing in order to get more out of clang/llvm?
Is it that this is too trivial of an example for the tool to shine?
EDIT based on answer
clang -c start.s -o start.o
clang -O2 -flto=thin -fomit-frame-pointer -c test.c
clang -O2 -flto=thin -fomit-frame-pointer -c two.c
ld.lld start.o test.o two.o -o test.elf
llvm-objdump -D test.elf
000110fc testtwo:
110fc: 02 20 movs r0, #2
110fe: 70 47 bx lr
00011100 two:
11100: 02 20 movs r0, #2
11102: 70 47 bx lr
so getting rid of the -emit-llvm and using lto basically gives the desired result.
Looking at the bc disassembly
clang -O2 -flto=thin -fomit-frame-pointer -c test.c
llvm-dis test.o
cat test.o.ll
; Function Attrs: norecurse nounwind readnone
define dso_local i32 #one() local_unnamed_addr #0 {
entry:
ret i32 1
}
; Function Attrs: norecurse nounwind readnone
define dso_local i32 #testone() local_unnamed_addr #0 {
entry:
ret i32 1
}
; Function Attrs: nounwind
define dso_local i32 #testtwo() local_unnamed_addr #1 {
entry:
%call = tail call i32 #two() #3
ret i32 %call
}
enables/adds the tail call. I really dislike using the compiler/shell as a linker (for embedded projects that have their own bootstrap and linker script), llvm-ldd usage wasn't easy to figure out or basically couldn't figure out, but ld.lld also supports the tlo stuff, so that worked out.
The answer is pretty easy actually: one should never want to use llc / opt / llvm-link for performing "end-user" project level kind of optimizations. These are developer-side tools with different defaults, thresholds, etc. Basically, they are just simple command-line frontends to various pieces of LLVM toolbox.
In order to perform the proper link-time-optimization you'd need to use the pipelines that were intended for such task. Basically, compiling everything using "clang -flto" and then linking everything again via "clang -flto" would work. Using LTO-aware linker like lld is a prerequisite as well.
Some further information about ThinLTO could also be found here: https://clang.llvm.org/docs/ThinLTO.html and http://blog.llvm.org/2016/06/thinlto-scalable-and-incremental-lto.html

autossh fails with exit status 1 and no error message

I'm trying to run autossh (on a VM running CentOS6), but it's exiting immediately with the help message. I think this is a system issue because when I run it with the exact same parameters on another computer (running Ubuntu 14.04) it works fine. It's also fine when I run the same command but with ssh instead of autossh. So I tried strace to see if anything's wrong there. But if there is I'm not sure what it is. Any ideas?
Here's the autossh command: autossh -oStrictHostKeyChecking=no -oServerAliveInterval=15 -oServerAliveCountMax=4 -L 3130:localhost:3130 -N -i /path/to/some.pem user#remotehost
Here's the strace output (note myserver is an entry in .ssh/config that contains the same parameters as the previous command):
execve("/usr/local/bin/autossh", ["autossh", "myserver"], [/* 55 vars */]) = 0
brk(0) = 0xefc000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f26193cc000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=36751, ...}) = 0
mmap(NULL, 36751, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f26193c3000
close(3) = 0
open("/lib64/libnsl.so.1", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p#\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=113432, ...}) = 0
mmap(NULL, 2198192, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f2618f95000
mprotect(0x7f2618fab000, 2093056, PROT_NONE) = 0
mmap(0x7f26191aa000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x15000) = 0x7f26191aa000
mmap(0x7f26191ac000, 6832, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f26191ac000
close(3) = 0
open("/lib64/libc.so.6", O_RDONLY) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\356\1\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1920936, ...}) = 0
mmap(NULL, 3750152, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f2618c01000
mprotect(0x7f2618d8b000, 2097152, PROT_NONE) = 0
mmap(0x7f2618f8b000, 20480, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x18a000) = 0x7f2618f8b000
mmap(0x7f2618f90000, 18696, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f2618f90000
close(3) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f26193c2000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f26193c1000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f26193c0000
arch_prctl(ARCH_SET_FS, 0x7f26193c1700) = 0
mprotect(0x7f2618f8b000, 16384, PROT_READ) = 0
mprotect(0x7f26191aa000, 4096, PROT_READ) = 0
mprotect(0x7f26193cd000, 4096, PROT_READ) = 0
munmap(0x7f26193c3000, 36751) = 0
write(2, "usage: autossh [-V] [-M monitor_"..., 69usage: autossh [-V] [-M monitor_port[:echo_port]] [-f] [SSH_OPTIONS]
) = 69
write(2, "\n", 1
) = 1
write(2, " -M specifies monitor port. M"..., 238 -M specifies monitor port. May be overridden by environment
variable AUTOSSH_PORT. 0 turns monitoring loop off.
Alternatively, a port for an echo service on the remote
machine may be specified. (Normally port 7.)
) = 238
write(2, " -f run in background (autoss"..., 85 -f run in background (autossh handles this, and does not
pass it to ssh.)
) = 85
write(2, " -V print autossh version and"..., 39 -V print autossh version and exit.
) = 39
write(2, "\n", 1
) = 1
write(2, "Environment variables are:\n", 27Environment variables are:
) = 27
write(2, " AUTOSSH_GATETIME - how lo"..., 259 AUTOSSH_GATETIME - how long must an ssh session be established
before we decide it really was established
(in seconds). Default is 30 seconds; use of -f
flag sets this to 0.
) = 259
write(2, " AUTOSSH_LOGFILE - file t"..., 107 AUTOSSH_LOGFILE - file to log to (default is to use the syslog
facility)
) = 107
write(2, " AUTOSSH_LOGLEVEL - level "..., 49 AUTOSSH_LOGLEVEL - level of log verbosity
) = 49
write(2, " AUTOSSH_MAXLIFETIME - set th"..., 65 AUTOSSH_MAXLIFETIME - set the maximum time to live (seconds)
) = 65
write(2, " AUTOSSH_MAXSTART - max ti"..., 69 AUTOSSH_MAXSTART - max times to restart (default is no limit)
) = 69
write(2, " AUTOSSH_MESSAGE - messag"..., 74 AUTOSSH_MESSAGE - message to append to echo string (max 64 bytes)
) = 74
write(2, " AUTOSSH_PATH - path t"..., 53 AUTOSSH_PATH - path to ssh if not default
) = 53
write(2, " AUTOSSH_PIDFILE - write "..., 49 AUTOSSH_PIDFILE - write pid to this file
) = 49
write(2, " AUTOSSH_POLL - how of"..., 70 AUTOSSH_POLL - how often to check the connection (seconds)
) = 70
write(2, " AUTOSSH_FIRST_POLL - time b"..., 71 AUTOSSH_FIRST_POLL - time before first connection check (seconds)
) = 71
write(2, " AUTOSSH_PORT - port t"..., 61 AUTOSSH_PORT - port to use for monitor connection
) = 61
write(2, " AUTOSSH_DEBUG - turn l"..., 104 AUTOSSH_DEBUG - turn logging to maximum verbosity and log to
stderr
) = 104
write(2, "\n", 1
) = 1
exit_group(1) = ?
+++ exited with 1 +++
I had exactly the same problem with autossh 1.4e on CentOS Linux 7. autossh stopped immediately and printed the help, without even trying to connect to SSH.
The solution was to specify -M 0 on the command line:
autossh -M 0 \
-oStrictHostKeyChecking=no \
-oServerAliveInterval=15 \
-oServerAliveCountMax=4 \
-L 3130:localhost:3130 \
-N -i /path/to/some.pem user#remotehost
Come on ... the autossh exits with
write(2, "usage: autossh [-V] [-M monitor_"..., 69usage: autossh [-V] [-M monitor_port[:echo_port]] [-f] [SSH_OPTIONS]
which means that you specified wrong arguments and it is trying to explain you how the syntax looks like. To see what is wrong, there are several things you can try:
Use -vvv switches to ssh to see more verbose log from the ssh.
Use AUTOSSH_DEBUG environment variable to get some debug logs from autossh.

Printing a string value in awk results in error

fatal: not enough arguments to satisfy format string
`%s SPT=80'
^ ran out for this one
This my code
for ((h = 1 ; h < 4 ; h++ )); do
x=$(awk -v i=h -v j=17 'FNR == 2 {printf "%s " $j}' newiptables.log)
echo $x
This is my file
Dec 26 09:17:51 localhost kernel: IN=eth0 OUT= MAC=00:10:c6:a8:da:68:00:90:7f:9c:50:5a:08:00 SRC=198.252.206.16 DST=10.128.1.225 LEN=313 TOS=0x00 PREC=0x00 TTL=64 ID=59334 PROTO=TCP SPT=80 DPT=56506 WINDOW=46535 RES=0x00 ACK PSH URGP=0
Dec 26 09:17:52 localhost kernel: IN=eth0 OUT= MAC=00:10:c6:a8:da:68:00:90:7f:9c:50:5a:08:00 SRC=198.252.206.16 DST=10.128.1.225 LEN=1440 TOS=0x00 PREC=0x00 TTL=64 ID=47303 PROTO=TCP SPT=80 DPT=56506 WINDOW=46535 RES=0x00 ACK URGP=0
Dec 26 09:17:52 localhost kernel: IN=eth0 OUT= MAC=00:10:c6:a8:da:68:00:90:7f:9c:50:5a:08:00 SRC=198.252.206.16 DST=10.128.1.225 LEN=1440 TOS=0x00 PREC=0x00 TTL=64 ID=47559 PROTO=TCP SPT=80 DPT=56506 WINDOW=46535 RES=0x00 ACK URGP=0
The problem is a missing comma in the printf command for awk:
awk -v i=h -v j=17 'FNR == 2 {printf "%s ", $j}' newiptables.log
^
|== This is needed
Quoting from the manual:
A simple printf statement looks like this:
printf format, item1, item2, ...