Compiling libtorch to WebAssembly with Emscripten using CMake build - cmake

Question

I am trying run my PyTorch model (exported to TorchScript) in the browser using WebAssembly. To do this, I need to compile libtorch using Emscripten. As PyTorch docs suggest using CMake as a compiler I stuck with that.
I've manage to compile libtorch using CMake without issues as specified here. Secondly, I also manage to compile a simple test program to WASM using Emscripten and CMake (for conciseness and focusing on the problem-at-hand I haven't included the CMakeLists.txt files for either of these)
Where the issues start is when I try to compile both together, here's my CMake file:
set(project "wasm-example")
cmake_minimum_required(VERSION 3.0.0 FATAL_ERROR)
set(CMAKE_TOOLCHAIN_FILE $HOME/.local/share/emsdk/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake)
set(CMAKE_BUILD_TYPE Debug)
project(${project})
SET_PROPERTY(GLOBAL PROPERTY TARGET_SUPPORTS_SHARED_LIBS TRUE)
set(CMAKE_FIND_ROOT_PATH_MODE_PACKAGE BOTH)
set(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY BOTH)
set(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE BOTH)
set(CMAKE_PREFIX_PATH $PWD/libtorch)
message(STATUS "Build type: ${CMAKE_BUILD_TYPE}")
message(STATUS "Using prefix path: ${CMAKE_PREFIX_PATH}")
message(STATUS "Using toolchain file: ${CMAKE_TOOLCHAIN_FILE}")
include(CTest)
enable_testing()
set(SOURCE_FILES main.cpp)
# process exported functions
set(exports_string "")
list(JOIN exports "," exports_string)
# set compiler and flags
set(CMAKE_C_COMPILER emcc)
set(CMAKE_CPP_COMPILER em++)
set(CMAKE_CXX_FLAGS "-s WASM=1 -s EXIT_RUNTIME=1 -s EXTRA_EXPORTED_RUNTIME_METHODS=\"[\"cwrap\", \"getValue\", \"setValue\"]\" ${TORCH_CXX_FLAGS}")
message(STATUS "Set CMAKE_CXX_FLAGS to: ${CMAKE_CXX_FLAGS}")
# find external target_link_libraries
find_package(Torch REQUIRED)
# specify the project
add_executable(${project} ${SOURCE_FILES})
target_link_libraries(${project} "${TORCH_LIBRARIES}")
set_property(TARGET ${project} PROPERTY CXX_STANDARD 14)
Note that I'm using the CPU-only distribution of libtorch, the CUDA-enabled version leads to even more issues because cuda is dynamic-only.
cmake ..src passes, however make throws several errors:
wasm-ld: error: unknown argument: --no-as-needed
wasm-ld: error: unknown argument: --as-needed
wasm-ld: error: unknown argument: --no-as-needed
wasm-ld: error: unknown argument: --as-needed
wasm-ld: error: unknown file type: /home/max/repos/sandbox/cpp/wasm-example/libtorch/lib/libtorch_cpu.so
wasm-ld: error: unknown file type: /home/max/repos/sandbox/cpp/wasm-example/libtorch/lib/libtorch.so
Clearly Emscripten is having some trouble with the dynamic nature of libtorch, so my first question would be if it's even possible to compile dynamic libraries using Emscripten? If this is possible, it would be interesting to try and compile GPU-enabled libtorch using Emscripten.
UPDATE
I tried to compile with the static CPU-only distribution of libtorch, but this leads to the exact same errors. However, an old GitHub issue on the Pytorch github was just revived stating that I'm not the only one with this problem. Hopefully this GitHub issue will address the issues I'm having.