ACCRE R9 Cluster Quick and Dirty Status
Report generated at Thu Aug 27 06:23:01 AM CDT 2026
Problem Nodes
HOSTNAMES STATE TIMESTAMP REASON COMMENT
cn1480 drain 2026-08-25T18:02:56 Prolog error (null)
cn1484 drain 2026-08-24T08:37:48 Eric - RTNA - Reboot Eric - RTNA - Reboot to try to clear bad OS state
cn1574 drain* 2026-08-18T09:48:33 Scott - RT100841 - r Scott - RT100841 - reseated blade need to hook up console
cn1575 drain* 2026-08-18T09:48:41 Scott - RT100841 - r Scott - RT100841 - reseated blade need to hook up console
cn1576 drain* 2026-08-18T09:48:48 Scott - RT100841 - r Scott - RT100841 - reseated blade need to hook up console
cn1577 drain* 2026-08-18T09:48:58 Scott - RT100841 - r Scott - RT100841 - reseated blade need to hook up console
cn1578 drain* 2026-08-18T13:09:39 Scott - RT- - RT1008 Scott - RT- - RT100841 - reseated blade need to hook up cons
cn1595 drain* 2026-07-27T16:58:32 Troy - RT100528 - Th Troy - RT100528 - Thermal issues
cn1700 down* 2026-08-25T09:05:13 Alex - Triage - rebo Alex - Triage - Reboot then Resume
Queue Summary (Batch)
GROUP USER ACTIVE_JOBS ACTIVE_CORES PENDING_JOBS PENDING_CORES
-----------------------------------------------------------------------------------------
accre_guests 0 0 1 100
haojz 0 0 1 100
-----------------------------------------------------------------------------------------
bias_group 1 10 0 0
biasds 1 10 0 0
-----------------------------------------------------------------------------------------
biostat_faculty 1 1 0 0
nianh1 1 1 0 0
-----------------------------------------------------------------------------------------
booth_lab 3 48 9 144
mathura 3 48 9 144
-----------------------------------------------------------------------------------------
brg_cores 1 8 0 0
kandelr 1 8 0 0
-----------------------------------------------------------------------------------------
cgg 0 0 1 64
liy110 0 0 1 64
-----------------------------------------------------------------------------------------
cms 1169 4582 452 1016
cmslocal 156 381 146 365
cmspilot 1013 4201 306 651
-----------------------------------------------------------------------------------------
coxlab 2 20 0 0
blostf1 1 10 0 0
mille131 1 10 0 0
-----------------------------------------------------------------------------------------
cqs_si 0 0 6 10
chenarsw 0 0 4 8
goodfeam 0 0 1 1
nichocm5 0 0 1 1
-----------------------------------------------------------------------------------------
das_lab 1 1 0 0
shiltmh1 1 1 0 0
-----------------------------------------------------------------------------------------
davis_lab 0 0 1 16
bluejor 0 0 1 16
-----------------------------------------------------------------------------------------
feng_lab 120 480 1260 5134
chenj96 0 0 3 90
jiangl1 120 480 1257 5044
-----------------------------------------------------------------------------------------
g_benntor_lab 1 1 0 0
mccorcl1 1 1 0 0
-----------------------------------------------------------------------------------------
g_gamazon_lab 1 8 0 0
bledsox 1 8 0 0
-----------------------------------------------------------------------------------------
goldring_group 10 50 44 217
mcgrawke 10 50 44 217
-----------------------------------------------------------------------------------------
gravity 28 40 90 174
ligo 28 40 90 174
-----------------------------------------------------------------------------------------
h_biostat_student 2 18 0 0
gaix1 1 10 0 0
yangc16 1 8 0 0
-----------------------------------------------------------------------------------------
h_darby_lab 38 76 0 0
mills37 38 76 0 0
-----------------------------------------------------------------------------------------
h_gelbard_lab 10 100 10 80
shinw5 10 100 10 80
-----------------------------------------------------------------------------------------
h_oguz_lab 0 0 2 2
lih30 0 0 2 2
-----------------------------------------------------------------------------------------
h_vangard_1 1 3 2 9
ramirm8 1 3 2 9
-----------------------------------------------------------------------------------------
isde-rer 0 0 1 8
maybergm 0 0 1 8
-----------------------------------------------------------------------------------------
l2_bioinfo1 2 4 0 0
liq17 2 4 0 0
-----------------------------------------------------------------------------------------
l3_runnoe_group 6 24 0 0
kaldorme 6 24 0 0
-----------------------------------------------------------------------------------------
l3_vuiis_cci 25 50 0 0
vuiis_daily_s 25 50 0 0
-----------------------------------------------------------------------------------------
lea_lab 51 324 317 2104
brassl1 20 80 108 432
longtial 1 4 0 0
songm6 30 240 209 1672
-----------------------------------------------------------------------------------------
nakagawa_lab 1 1 0 0
nakagat1 1 1 0 0
-----------------------------------------------------------------------------------------
nbody 2 512 1 256
smitm77 2 512 1 256
-----------------------------------------------------------------------------------------
neurogroup 1 5 0 0
roggeokk 1 5 0 0
-----------------------------------------------------------------------------------------
ng_lab 1 8 0 0
kimj119 1 8 0 0
-----------------------------------------------------------------------------------------
p_csb_meiler 120 1200 788 3796
agarwm5 0 0 5 259
tydingcw 120 1200 306 3060
yange8 0 0 477 477
-----------------------------------------------------------------------------------------
p_dsi 0 0 2 3
yangi1 0 0 2 3
-----------------------------------------------------------------------------------------
p_englot_group 0 0 1 24
redaa1 0 0 1 24
-----------------------------------------------------------------------------------------
p_meiler 0 0 1 3
yange8 0 0 1 3
-----------------------------------------------------------------------------------------
rer 10 14 0 0
cantrekb 9 10 0 0
hum6 1 4 0 0
-----------------------------------------------------------------------------------------
rokaslab 72 119 3444 3444
copea1 71 71 3444 3444
davidkt 1 48 0 0
-----------------------------------------------------------------------------------------
sbcs 1 10 0 0
shij19 1 10 0 0
-----------------------------------------------------------------------------------------
tk_lab 8 320 0 0
yoonh15 8 320 0 0
-----------------------------------------------------------------------------------------
tplab 1 36 0 0
zhanq21 1 36 0 0
-----------------------------------------------------------------------------------------
vgi 2 9 0 0
parkj71 1 5 0 0
salerl1 1 4 0 0
-----------------------------------------------------------------------------------------
womelsdorf_lab 33 340 122 1548
gerritcg 33 340 122 1548
-----------------------------------------------------------------------------------------
Totals: 1725 8422 6555 18152
Queue Summary (Batch GPU)
GROUP USER ACTIVE_JOBS ACTIVE_GPUS PENDING_JOBS PENDING_GPUS
-----------------------------------------------------------------------------------------
accre_guests_acc 2 2 0 0
lib13 1 1 0 0
wanj119 1 1 0 0
-----------------------------------------------------------------------------------------
csb_gpu_acc 38 44 5384 5387
gonzc11 0 0 5206 5206
karadim 24 27 0 0
lindsr1 7 7 177 177
liuy145 6 6 0 0
ranx 1 4 0 0
yangz24 0 0 1 4
-----------------------------------------------------------------------------------------
es3890_acc 1 1 0 0
sadridm 1 1 0 0
-----------------------------------------------------------------------------------------
h_oguz_lab_acc 2 2 0 0
wanj119 2 2 0 0
-----------------------------------------------------------------------------------------
p_dsi_acc 0 0 4 4
yangi1 0 0 4 4
-----------------------------------------------------------------------------------------
p_meiler_acc 1 8 1 1
agarwm5 0 0 1 1
kaot1 1 8 0 0
-----------------------------------------------------------------------------------------
taylor_group_acc 2 2 0 0
gersbaka 1 1 0 0
laaln1 1 1 0 0
-----------------------------------------------------------------------------------------
Totals: 46 59 5389 5392
Queue Summary (interactive)
GROUP USER ACTIVE_JOBS ACTIVE_CORES PENDING_JOBS PENDING_CORES
-----------------------------------------------------------------------------------------
booth_lab_int 1 2 0 0
mathura 1 2 0 0
-----------------------------------------------------------------------------------------
g_giri_group_int 1 4 0 0
breyem3 1 4 0 0
-----------------------------------------------------------------------------------------
h_vmac_int 1 6 0 0
janveva 1 6 0 0
-----------------------------------------------------------------------------------------
maiziezhou_lab_int 16 64 70 280
tangk10 16 64 70 280
-----------------------------------------------------------------------------------------
maiziezhou_lab_phd_int 1 8 0 0
chowx 1 8 0 0
-----------------------------------------------------------------------------------------
nbody_int 1 1 0 0
pater32 1 1 0 0
-----------------------------------------------------------------------------------------
rubinov_lab_int 2 25 0 0
abbasia 1 15 0 0
sardarn 1 10 0 0
-----------------------------------------------------------------------------------------
yang_lab_int 1 8 0 0
shaoq1 1 8 0 0
-----------------------------------------------------------------------------------------
Totals: 24 118 70 280
Queue Summary (interactive_gpu)
GROUP USER ACTIVE_JOBS ACTIVE_GPUS PENDING_JOBS PENDING_GPUS
-----------------------------------------------------------------------------------------
accre_dgx_iacc 1 4 0 0
rubinom 1 4 0 0
-----------------------------------------------------------------------------------------
dsi_dgx_iacc 5 11 3 12
chattec 1 1 0 0
jenkm22 2 8 3 12
schultls 2 2 0 0
-----------------------------------------------------------------------------------------
Totals: 6 15 3 12
Partition Summary
PARTITION AVAIL TIMELIMIT NODES STATE NODELIST
interactive up 14-00:00:0 10 mix cn[1287,1325-1326,1328-1330,1812-1815]
interactive up 14-00:00:0 2 alloc cn[1301-1302]
interactive up 14-00:00:0 19 idle cn[1322-1324,1707,1800-1811,1816-1818]
batch* up 14-00:00:0 6 drain* cn[1574-1578,1595]
batch* up 14-00:00:0 1 down* cn1700
batch* up 14-00:00:0 2 drain cn[1480,1484]
batch* up 14-00:00:0 143 mix cn[1203,1212,1218,1224,1227,1237,1241,1265,1280,1282,1294-1296,1354,1357-1358,1362,1369,1373-1376,1379,1381,1387,1389-1390,1395,1398,1422-1423,1426,1431,1435-1439,1441,1443,1445,1447,1449-1450,1452-1457,1460-1463,1467,1471-1473,1475-1479,1481,1496,1511,1520,1529-1536,1538,1543-1547,1551-1556,1558-1559,1562,1564-1567,1573,1579-1580,1582,1584,1586-1589,1592-1593,1596-1610,1612-1617,1619,1623-1626,1629,1632,1701-1706,1708-1710,2000]
batch* up 14-00:00:0 234 alloc cn[1202,1204-1211,1213,1215-1217,1219-1223,1226,1228-1236,1238-1240,1242,1257-1262,1264,1266-1275,1277-1279,1281,1283-1286,1288-1293,1297-1299,1303-1318,1320-1321,1327,1331-1353,1359-1361,1363-1368,1371-1372,1377-1378,1380,1382-1385,1388,1391-1394,1396-1397,1399-1411,1414-1421,1424-1425,1427,1430,1432,1434,1442,1446,1448,1458,1464,1466,1468-1470,1474,1482-1483,1485-1495,1497-1510,1512-1519,1522-1525,1527-1528,1537,1548-1550,1557,1561,1563,1568-1571,1581,1583,1585,1594,1618,1620-1622,1627-1628,1630-1631]
batch* up 14-00:00:0 1 idle cn1633
batch_gpu up 14-00:00:0 22 mix gpu[0059,0062-0067,0069-0070,0073-0076,0081-0082,0208,0300-0301],gracehopper[01-02],hgx[02-03]
batch_gpu up 14-00:00:0 2 alloc gpu[0071,0077]
batch_gpu up 14-00:00:0 15 idle gpu[0068,0072,0078-0080,0085,0302-0310]
interactive_gpu up 14-00:00:0 1 mix dgx03
interactive_gpu up 14-00:00:0 1 alloc dgx04
interactive_gpu up 14-00:00:0 5 idle dgx01,gpu[0058,0060-0061,0207]
sam up 2-02:00:00 2 idle cms-sam-[01-02]
reserved inact infinite 0 n/a