Fastest way to test a 128 bit NEON register for a value of 0 using intrinsics? - neon

Question

I'm looking for the fastest way to test if a 128 NEON register contains all zeros, using NEON intrinsics.
I'm currently using 3 OR operations, and 2 MOVs:
uint32x4_t vr = vorrq_u32(vcmp0, vcmp1);
uint64x2_t v0 = vreinterpretq_u64_u32(vr);
uint64x1_t v0or = vorr_u64(vget_high_u64(v0), vget_low_u64(v0));
uint32x2_t v1 = vreinterpret_u32_u64 (v0or);
uint32_t r = vget_lane_u32(v1, 0) | vget_lane_u32(v1, 1);
if (r == 0) { // do stuff }
This translates by gcc to the following assembly code:
VORR q9, q9, q10
VORR d16, d18, d19
VMOV.32 r3, d16[0]
VMOV.32 r2, d16[1]
VORRS r2, r2, r3
BEQ ...
Does anyone have an idea of a faster way?

Answer 1

While this answer may be a bit late, there is a simple way to do the test with only 3 instructions and no extra registers:
inline uint32_t is_not_zero(uint32x4_t v)
{
uint32x2_t tmp = vorr_u32(vget_low_u32(v), vget_high_u32(v));
return vget_lane_u32(vpmax_u32(tmp, tmp), 0);
}
The return value will be nonzero if any bit in the 128-bit NEON register was set.

Answer 2

If you're targeting AArch64 NEON, you can use the following to get a value to test with just two instructions:
inline uint64_t is_not_zero(uint32x4_t v)
{
uint64x2_t v64 = vreinterpretq_u64_u32(v);
uint32x2_t v32 = vqmovn_u64(v64);
uint64x1_t result = vreinterpret_u64_u32(v32);
return result[0];
}

Answer 3

You seem to be looking for intrinsics and this is the way:
inline bool is_zero(int32x4_t v) noexcept
{
v = v == int32x4{};
return !int32x2_t(
vtbl2_s8(
int8x8x2_t{
int8x8_t(vget_low_s32(v)),
int8x8_t(vget_high_s32(v))
},
int8x8_t{0, 4, 8, 12}
)
)[0];
}
Nils Pipenbrinck's answer has a flaw in that he assumes the QC, cumulative saturation flag to be clear.

Answer 4

If you have AArch64 you can do it even easier. They have a new instruction for designed for this.
inline uint32_t is_not_zero(uint32x4_t v)
{
return vaddvq_u32(v);
}

Answer 5

I'd avoid functions returning integer values that should only be interpreted as bool. A better way would be, for instance, defining a helper function to return maximum unsigned value of 4 lanes:
inline uint32_t max_lane_value_u32(const uint32x4_t& v)
{
#if defined(_WIN32) && defined(_ARM64_)
// Windows 64-bit
return neon_umaxvq32(v);
#elif defined(__LP64__)
// Linux/Android 64-bit
return vmaxvq_u32(v);
#else
// Windows/Linux/Android 32-bit
uint32x2_t result = vmax_u32(vget_low_u32(v), vget_high_u32(v));
return vget_lane_u32(vpmax_u32(result, result), 0);
#endif
}
you can then use:
if (0 == max_lane_value_u32(v))
{
...
}
in your code, and such function might also be useful elsewhere. Alternatively, you can use the exact same code to write a is_not_zero() function, but then it's best form to return a bool.
Note that the only reason you'd need to define a helper function is because vmaxvq_u32() is not available on 32-bit, and may not be aliased from neon_umaxvq32() in arm64_neon.h on Windows.