2022-10-30

Submitted by cjallan on Thu, 01/05/2023 - 11:09

25 October 2022
The /DFS-L/SCRATCH area was also significantly affected by the bad striping bug. We do have a mitigation
and work-around now, but it may take some time to clean. The metadata server will need to go down at least
one more time, but I hope to have the cluster up again tonight. For now, /DFS-L is mounted on all login
nodes, but only a few compute nodes.

30 October
The patch applied to the /DFS-L filesystem did not prevent kernel panics and has been reverted. To get
the cluster mostly working again, /DFS-L will be remounted everywhere, but the /DFS-L/SCRATCH subdirectory
will be set unreadable to prevent triggering the bug. Getting this data clean is now the top priority.