Azure SQL sessions blocked in status PAGEIOLATCH_SH - azure-sql-database

Question

I've got a Spring application with Hibernate that sporadically stops working because all 30 connections of its connection pool are blocked. While these connections were blocked, I executed a bunch of queries to find the cause.
Each connection executed the same join statement.
The execution plan of that join looks like that
This query
SELECT *
FROM sys.dm_exec_requests
CROSS APPLY sys.dm_exec_sql_text(sql_handle)
returns for each connection something like:
status = suspended
command = SELECT
blocking_session_id = 0
wait_type = last_wait_type = PAGEIOLATCH_SH
wait_time = 3
cpu_time = 12909
total_elapsed_time = 3723943
logical_reads = 7986970
The 3 indexes involved have a size of about 16GB (last one in the screenshot above), 1 GB and 500 MB respectively.
This is happening inside a sql database in an elastic pool with 24 vcores, Gen5, max data size 2418 GB
The resource monitor of that elastic pool looked reasonable (arrow indicates the correct time):
Anything else I could check? Any ideas what could be the reason for this?

Answer 1

Inserting data on an increasing key (like an identity column) that means many parallel threads will be going after the last data page of that table if a good number of connections is trying to ingest data to the table, trying to add rows on that same data page.
If you put that table in memory (in-memory OLTP) you may not have a disk contention problem, but in this scenario that table is not in-memory so the disk contention is manifesting itself as pagelatch_ex wait.
Another option is to make the clustered index a GUID. That will make the insert distribute among all the data pages, but it may cause page splits.