You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-25 - 12:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Thu Jun 25, 2026 00:45:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1261310999528,14cyclictest1360181-21ssh00:23:530
1261310999512,61cyclictest16950irq/224-800000023:00:030
1261310998329,35cyclictest1336782-21latency_hist23:35:020
1261322998213,41cyclictest1317726-21latency_hist22:55:023
1261310998126,13cyclictest1354655-21ssh00:12:110
1261310997817,44cyclictest16850irq/134-800000022:21:330
1261310997729,42cyclictest16850irq/134-800000022:45:000
1261310997628,16cyclictest1340098-21ssh23:41:350
126131099761,39cyclictest1333685-21ssh23:28:180
1261310997527,7cyclictest1366905-21ssh00:37:530
1261310997527,7cyclictest1366905-21ssh00:37:530
1261310997426,13cyclictest1323768-21ssh23:07:250
1261310997423,5cyclictest0-21swapper/000:28:170
1261310997423,5cyclictest0-21swapper/000:28:170
1261310997422,16cyclictest1356934-21ssh00:16:520
126132299734,4cyclictest131rcu_preempt22:00:023
126132299734,4cyclictest131rcu_preempt22:00:013
1261310997325,12cyclictest1283032-21ssh21:42:170
126131099722,39cyclictest1327870-21ssh23:15:530
126131099721,5cyclictest1261787-21cron19:20:010
126131099721,5cyclictest1261787-21cron19:20:000
1261310997128,13cyclictest1274058-21ssh21:23:390
1261310997122,33cyclictest12-21ksoftirqd/023:52:490
1261310997119,9cyclictest1346863-21ssh23:55:380
1261310997117,12cyclictest1363560-21ssh00:30:370
1261322997027,38cyclictest1273590-21sh21:22:313
126131099703,6cyclictest1277242-21sed21:30:030
1261310997025,14cyclictest1330242-21ssh23:20:430
1261310997025,14cyclictest1292017-21ssh22:01:020
1261310997024,14cyclictest1306135-21ssh22:30:210
1261310997016,12cyclictest1350344-21ssh00:03:080
1261310996924,12cyclictest1305923-21sed22:30:030
1261310996924,12cyclictest1297975-21ssh22:13:370
1261310996921,7cyclictest1289297-21ssh21:55:250
1261310996921,7cyclictest1289297-21ssh21:55:250
1261310996918,12cyclictest1270791-21ssh21:16:420
1261310996918,12cyclictest1270791-21ssh21:16:410
126132299681,14cyclictest3467602-21systemd-journal21:16:313
126132299681,14cyclictest3467602-21systemd-journal21:16:313
1261310996815,13cyclictest1351984-21ssh00:06:240
126132299672,13cyclictest1348253-21sshd23:58:513
126131099672,60cyclictest1300965-21sh22:20:000
126131099672,60cyclictest1300965-21sh22:20:000
1261310996722,12cyclictest1308532-21ssh22:35:240
126131099671,62cyclictest0-21swapper/022:50:030
1261310996627,34cyclictest16950irq/224-800000021:35:000
1261322996528,33cyclictest1320271-21latency_hist23:00:023
126132299651,60cyclictest1324359-21ssh23:08:423
126131099651,37cyclictest1342260-21ssh23:46:010
126132299642,56cyclictest1341114-21apt-get23:44:003
1261322996417,43cyclictest37-21ksoftirqd/323:29:223
1261310996422,5cyclictest309-21in:imuxsock21:05:010
126132299632,6cyclictest1265193-21munin-run20:25:003
126132299632,57cyclictest1311009-21ssh22:40:283
1261322996325,35cyclictest1266697-21sshd20:50:203
126132299632,4cyclictest1304484-21ssh22:27:053
126132299632,18cyclictest1354551-21sh00:11:573
126132299632,18cyclictest1295119-21ssh22:07:323
126132299632,14cyclictest1284718-21sshd21:45:423
126132299631,8cyclictest1338198-21sshd23:37:413
126132299631,6cyclictest1335584-21ssh23:32:143
126132299631,14cyclictest1344854-21ssh23:51:313
1261310996328,16cyclictest1295479-21ssh22:08:240
1261310996313,13cyclictest275-21dbus-daemon19:45:010
1261310996313,13cyclictest275-21dbus-daemon19:45:000
126132299622,19cyclictest1280732-21ssh21:37:373
126132299622,19cyclictest1280732-21ssh21:37:373
126132299621,9cyclictest1362522-21ssh00:28:473
126132299621,9cyclictest1362522-21ssh00:28:473
126132299621,9cyclictest1276349-21ssh21:28:273
126132299621,8cyclictest1310201-21ssh22:39:193
126132299621,7cyclictest1297219-21ssh22:11:543
126132299621,17cyclictest1366961-21sh00:38:003
126132299621,17cyclictest1366961-21sh00:38:003
1261310996226,5cyclictest1288509-21sh21:54:030
126131099621,34cyclictest1268441-21sshd21:11:420
126131099621,27cyclictest0-21swapper/023:13:330
126132299612,54cyclictest3467602-21systemd-journal00:05:513
126132299612,54cyclictest1322518-21munin-run23:05:003
126132299612,53cyclictest1292855-21ssh22:02:543
126132299611,8cyclictest1359517-21ssh00:22:193
126132299611,7cyclictest1302401-21ssh22:22:463
126132299611,53cyclictest1357547-21ssh00:18:173
126132299611,4cyclictest1307331-21ssh22:33:123
126132299602,5cyclictest1349044-21ssh00:00:053
126132299602,54cyclictest364-21snmpd23:21:323
126132299602,20cyclictest1326038-21ssh23:12:133
126132299601,8cyclictest1329371-21ssh23:19:263
126132299601,7cyclictest1314475-21ssh22:48:023
126132299601,6cyclictest1363674-21ssh00:30:523
126132299601,6cyclictest1283178-21ssh21:42:353
1261310996019,9cyclictest1319035-21ssh22:57:280
1261322995926,29cyclictest1265954-21latency_hist20:40:023
126132299592,5cyclictest1267280-21munin-run21:05:013
126132299592,52cyclictest1263831-21sshd19:55:043
126132299592,51cyclictest1263378-21munin-run19:50:013
126132299592,51cyclictest1263378-21munin-run19:50:003
126132299592,18cyclictest1-21systemd21:00:013
126132299591,4cyclictest1300730-21ssh22:19:333
126132299591,4cyclictest1300730-21ssh22:19:333
126131099592,6cyclictest1247743-21munin-node22:51:590
126132299582,52cyclictest364-21snmpd19:10:403
126132299582,51cyclictest1266230-21munin-run20:45:003
126132299581,6cyclictest1277871-21ssh21:31:263
1261318995850,5cyclictest1311116-21kworker/2:223:27:412
1261310995823,6cyclictest1338730-21sh23:38:550
1261310995814,15cyclictest275-21dbus-daemon19:35:010
1261310995814,15cyclictest275-21dbus-daemon19:35:000
126132299572,6cyclictest1266493-21munin-run20:50:013
126132299572,6cyclictest1264954-21munin-run20:20:003
126132299571,8cyclictest1263126-21munin-run19:45:023
126132299571,8cyclictest1263126-21munin-run19:45:013
126132299571,6cyclictest1268195-21ssh21:11:113
1261310995725,17cyclictest1286420-21ssh21:49:420
126132299562,49cyclictest1264149-21munin-run20:05:003
126132299562,48cyclictest1287048-21sshd21:50:323
126132299561,4cyclictest1341871-21ssh23:45:063
126132299561,14cyclictest364-21snmpd19:15:353
126132299561,14cyclictest364-21snmpd19:15:353
1261318995629,16cyclictest275-21dbus-daemon20:45:012
126131099563,21cyclictest1264076-21wc20:00:030
1261310995628,2cyclictest0-21swapper/021:37:310
1261310995628,2cyclictest0-21swapper/021:37:310
1261310995624,10cyclictest12-21ksoftirqd/019:10:350
126132299551,6cyclictest1265408-21sshd20:25:153
1261318995545,7cyclictest1311116-21kworker/2:223:10:102
1261318995542,7cyclictest1269836-21munin-plugin-st21:15:002
1261310995520,8cyclictest12-21ksoftirqd/019:55:020
1261318995446,5cyclictest1260923-21kworker/2:222:05:102
1261318995445,5cyclictest1311116-21kworker/2:200:20:352
1261318995435,14cyclictest3467602-21systemd-journal00:40:002
1261318995435,14cyclictest3467602-21systemd-journal00:39:592
126131099546,5cyclictest131rcu_preempt19:50:000
126131099546,5cyclictest131rcu_preempt19:50:000
126131099542,48cyclictest1261311-21ls19:10:000
126132299532,5cyclictest364-21snmpd19:21:463
1261322995323,27cyclictest1264649-21latency_hist20:15:023
126132299531,5cyclictest1264375-21sshd20:07:413
126132299531,4cyclictest1267490-21sshd21:06:413
1261318995345,5cyclictest1260923-21kworker/2:222:10:102
1261318995344,6cyclictest1294138-21kworker/2:022:36:342
1261318995344,5cyclictest1311116-21kworker/2:223:37:152
1261318995343,6cyclictest1311116-21kworker/2:200:01:422
1261318995340,7cyclictest1294138-21kworker/2:022:23:542
1261314995330,16cyclictest1320102-21munin-run23:00:011
126132299521,6cyclictest1262316-21sshd19:29:593
126132299521,4cyclictest1263594-21sshd19:53:423
1261318995244,5cyclictest1311116-21kworker/2:223:42:372
1261318995244,5cyclictest1311116-21kworker/2:223:16:142
1261318995244,5cyclictest1311116-21kworker/2:200:15:362
1261318995244,5cyclictest1301012-21kworker/2:122:43:242
1261318995243,6cyclictest1311116-21kworker/2:223:24:452
1261318995243,6cyclictest1301012-21kworker/2:123:46:372
1261318995242,7cyclictest1260923-21kworker/2:221:53:002
1261310995215,12cyclictest275-21dbus-daemon20:50:010
126132299513,41cyclictest1262833-21sh19:40:003
1261318995143,5cyclictest1311116-21kworker/2:223:00:292
1261318995143,5cyclictest1311116-21kworker/2:222:59:002
1261318995143,5cyclictest1301012-21kworker/2:123:51:212
1261318995143,5cyclictest1253951-21kworker/2:321:44:362
1261318995143,5cyclictest1253951-21kworker/2:321:19:512
1261318995143,5cyclictest1253951-21kworker/2:321:19:512
1261318995142,6cyclictest1294138-21kworker/2:022:29:332
1261318995141,7cyclictest1311116-21kworker/2:222:53:272
1261318995141,7cyclictest1260923-21kworker/2:220:11:422
1261318995141,6cyclictest1311116-21kworker/2:222:48:472
1261318995110,6cyclictest131rcu_preempt21:59:542
1261318995110,6cyclictest131rcu_preempt21:59:542
1261314995134,13cyclictest3467602-21systemd-journal19:25:001
1261310995121,7cyclictest12-21ksoftirqd/020:02:500
126132299502,5cyclictest1-21systemd20:35:003
1261318995042,5cyclictest1311116-21kworker/2:200:14:542
1261318995042,5cyclictest1301012-21kworker/2:122:32:402
1261318995042,5cyclictest1260923-21kworker/2:221:33:072
1261318995042,5cyclictest1260923-21kworker/2:221:28:392
1261318995042,5cyclictest1260923-21kworker/2:219:54:192
1261318995041,6cyclictest1311116-21kworker/2:200:34:552
1261318995041,6cyclictest1253951-21kworker/2:321:20:252
1261318995041,5cyclictest1311116-21kworker/2:200:27:162
1261318995041,5cyclictest1311116-21kworker/2:200:27:152
1261318995041,5cyclictest1301012-21kworker/2:123:30:232
1261318995040,6cyclictest1301012-21kworker/2:123:58:132
1261318995040,6cyclictest1260923-21kworker/2:222:02:422
1261318995033,11cyclictest30-21ksoftirqd/221:00:012
1261314995035,10cyclictest309-21in:imuxsock19:30:031
1261314995035,10cyclictest309-21in:imuxsock19:30:021
1261310995024,5cyclictest1262978-21latency_hist19:40:020
1261318994941,5cyclictest1311116-21kworker/2:223:09:332
1261318994941,5cyclictest1260923-21kworker/2:222:17:482
1261318994941,5cyclictest1260923-21kworker/2:222:17:482
1261318994940,6cyclictest1260923-21kworker/2:220:25:562
1261314994910,12cyclictest1266974-21cron21:00:011
1261318994840,5cyclictest1311116-21kworker/2:200:05:262
1261318994839,6cyclictest1260923-21kworker/2:221:35:262
1261318994839,6cyclictest1260923-21kworker/2:221:35:252
1261314994834,10cyclictest0-21swapper/123:05:011
1261314994829,15cyclictest23-21ksoftirqd/123:10:021
1261314994827,16cyclictest275-21dbus-daemon21:35:011
1261310994833,11cyclictest12-21ksoftirqd/020:20:000
1261310994829,13cyclictest3467602-21systemd-journal20:25:010
1261318994739,5cyclictest1260923-21kworker/2:221:04:412
1261318994739,5cyclictest1260923-21kworker/2:220:38:452
1261318994739,5cyclictest1253951-21kworker/2:321:46:182
1261318994731,12cyclictest3467602-21systemd-journal19:45:042
1261318994731,12cyclictest3467602-21systemd-journal19:45:032
1261318994730,12cyclictest309-21in:imuxsock20:55:002
1261310994727,14cyclictest275-21dbus-daemon20:10:010
126132299462,39cyclictest0-21swapper/319:30:443
126132299462,39cyclictest0-21swapper/319:30:433
1261318994638,5cyclictest1260923-21kworker/2:221:08:392
1261318994638,5cyclictest1260923-21kworker/2:220:04:252
1261318994638,5cyclictest1260923-21kworker/2:219:30:492
1261318994638,5cyclictest1260923-21kworker/2:219:30:482
1261318994629,11cyclictest30-21ksoftirqd/220:20:002
1261314994629,14cyclictest1-21systemd22:15:021
1261314994628,12cyclictest275-21dbus-daemon21:40:011
1261314994628,12cyclictest275-21dbus-daemon21:40:001
1261314994627,14cyclictest275-21dbus-daemon00:30:011
1261314994627,14cyclictest275-21dbus-daemon00:30:011
1261314994626,16cyclictest3467602-21systemd-journal20:40:001
126132299451,2cyclictest0-21swapper/319:10:023
1261318994537,5cyclictest1260923-21kworker/2:220:34:272
1261318994537,5cyclictest1260923-21kworker/2:219:15:072
1261318994537,5cyclictest1260923-21kworker/2:219:15:062
1261318994536,6cyclictest1260923-21kworker/2:220:08:532
1261314994527,13cyclictest1261528-21cron19:15:001
126131099451,30cyclictest0-21swapper/019:24:280
126131099451,28cyclictest0-21swapper/019:26:540
1261318994437,4cyclictest1260923-21kworker/2:220:49:482
1261318994436,5cyclictest1260923-21kworker/2:219:14:012
1261314994433,8cyclictest1341684-21cat23:45:011
1261314994431,9cyclictest3467602-21systemd-journal20:50:001
1261314994431,9cyclictest23-21ksoftirqd/100:14:501
1261314994431,10cyclictest23-21ksoftirqd/121:11:031
1261310994437,4cyclictest16950irq/224-800000020:30:000
1261310994427,3cyclictest121ksoftirqd/021:10:010
1261318994335,5cyclictest1260923-21kworker/2:220:21:052
1261314994332,8cyclictest23-21ksoftirqd/123:25:011
1261314994329,11cyclictest23-21ksoftirqd/121:10:021
1261314994327,12cyclictest3467602-21systemd-journal22:55:011
1261314994325,13cyclictest1-21systemd19:30:011
126131499424,4cyclictest131rcu_preempt00:35:381
126131499424,4cyclictest131rcu_preempt00:35:381
1261314994228,10cyclictest1300956-21cron22:20:001
1261314994228,10cyclictest1300956-21cron22:20:001
1261314994227,12cyclictest23-21ksoftirqd/100:04:071
1261314994224,13cyclictest3466785-21systemd-udevd21:30:001
1261314994221,15cyclictest1272227-21cron21:20:001
1261314994221,15cyclictest1272227-21cron21:20:001
126131499421,4cyclictest131rcu_preempt22:20:571
1261310994225,13cyclictest12-21ksoftirqd/020:35:000
1261310994213,6cyclictest131rcu_preempt20:45:010
1261318994134,4cyclictest1260923-21kworker/2:219:20:132
1261318994133,5cyclictest1260923-21kworker/2:219:43:312
1261318994133,5cyclictest1260923-21kworker/2:219:43:312
1261318994133,5cyclictest1260923-21kworker/2:219:28:142
126131499414,4cyclictest131rcu_preempt23:33:021
1261314994130,6cyclictest1261789-21cron19:20:011
1261314994130,6cyclictest1261789-21cron19:20:001
1261314994129,9cyclictest3467602-21systemd-journal22:48:411
1261314994127,11cyclictest23-21ksoftirqd/100:15:131
126131499412,4cyclictest131rcu_preempt23:15:051
126131099411,28cyclictest0-21swapper/020:50:400
126131499403,3cyclictest131rcu_preempt21:05:001
1261314994026,10cyclictest23-21ksoftirqd/123:13:361
1261314994026,10cyclictest23-21ksoftirqd/122:10:001
1261314994025,11cyclictest23-21ksoftirqd/122:35:001
1261310994021,15cyclictest275-21dbus-daemon20:40:010
126131099401,10cyclictest1267120-21latency_hist21:00:030
1261318993931,5cyclictest1260923-21kworker/2:219:56:482
126131499395,3cyclictest131rcu_preempt20:45:011
126131499394,3cyclictest131rcu_preempt22:37:571
126131499393,3cyclictest131rcu_preempt22:29:071
1261314993928,9cyclictest23-21ksoftirqd/119:50:021
1261314993928,9cyclictest23-21ksoftirqd/119:50:021
1261314993927,6cyclictest309-21in:imuxsock21:20:031
1261314993926,10cyclictest23-21ksoftirqd/122:03:541
1261314993926,10cyclictest23-21ksoftirqd/121:53:221
1261314993923,12cyclictest275-21dbus-daemon20:30:011
1261318993830,5cyclictest1260923-21kworker/2:219:35:072
1261314993828,8cyclictest23-21ksoftirqd/121:56:451
1261314993828,8cyclictest23-21ksoftirqd/121:56:441
1261314993826,9cyclictest23-21ksoftirqd/123:47:071
1261314993826,9cyclictest23-21ksoftirqd/121:45:521
1261314993825,9cyclictest23-21ksoftirqd/100:32:471
1261314993825,10cyclictest23-21ksoftirqd/123:25:211
1261314993822,13cyclictest1263980-21sed20:00:021
126131499373,3cyclictest131rcu_preempt00:05:441
1261314993728,7cyclictest23-21ksoftirqd/120:35:001
1261314993728,6cyclictest1337677-21sh23:36:351
1261314993724,10cyclictest23-21ksoftirqd/121:43:521
1261314993723,10cyclictest23-21ksoftirqd/100:00:001
1261314993720,4cyclictest23-21ksoftirqd/122:40:301
1261314993625,7cyclictest1264398-21munin-plugin-st20:10:011
1261314993624,9cyclictest23-21ksoftirqd/123:54:191
1261314993623,8cyclictest23-21ksoftirqd/120:25:001
1261314993622,10cyclictest23-21ksoftirqd/100:21:101
126131099362,13cyclictest1264616-21modprobe20:10:110
1261314993524,6cyclictest309-21in:imuxsock20:15:011
1261314993521,10cyclictest23-21ksoftirqd/119:40:021
1261314993521,10cyclictest1-21systemd20:05:011
1261314993426,6cyclictest23-21ksoftirqd/120:20:001
1261314993423,8cyclictest1263245-21sed19:45:031
1261314993423,8cyclictest1263245-21sed19:45:021
126131499331,29cyclictest48-21kcompactd019:50:361
1261318993123,6cyclictest30-21ksoftirqd/219:10:002
1261314993123,4cyclictest3467562-21sshd20:50:421
126131499251,3cyclictest131rcu_preempt19:10:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional