You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-16 - 08:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot3s.osadl.org (updated Sun Aug 16, 2026 00:45:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12191839910297,3cyclictest0-21swapper/023:55:020
1219183999490,2cyclictest0-21swapper/000:05:020
1219183999321,51cyclictest1278809-21sshd23:03:070
121918399911,3cyclictest150-21pr/ttyS200:20:030
121918399846,4cyclictest131rcu_preempt20:35:020
1219183998411,52cyclictest0-21swapper/000:23:410
1219183998321,17cyclictest1275478-21ssh22:56:070
1219183998320,36cyclictest1303595-21munin-run23:55:000
1219183998148,24cyclictest12-21ksoftirqd/021:05:010
1219183998126,21cyclictest1275014-21date22:55:030
1219183998122,17cyclictest1294033-21munin-run23:35:010
1219183998037,10cyclictest1234755-21munin-run21:30:010
1219183998012,37cyclictest0-21swapper/023:50:030
1219183998012,37cyclictest0-21swapper/023:50:020
1219183997926,20cyclictest1319980-21ssh00:29:340
121918399768,48cyclictest0-21swapper/023:39:130
1219183997537,10cyclictest364-21snmpd20:06:440
1219183997524,20cyclictest1245580-21ssh21:53:100
1219183997523,18cyclictest1220309-21cat19:30:020
1219183997523,10cyclictest1323794-21ssh00:37:210
1219183997454,17cyclictest16850irq/134-800000022:20:010
1219183997412,38cyclictest0-21swapper/021:21:250
1219193997357,11cyclictest37-21ksoftirqd/319:35:023
1219183997323,14cyclictest1307525-21ssh00:03:100
121918599722,64cyclictest1301246-21rm23:50:021
121918599722,64cyclictest1301246-21rm23:50:021
1219183997223,14cyclictest1297511-21ssh23:42:080
1219183997220,14cyclictest1284623-21modprobe23:15:020
1219183997113,31cyclictest0-21swapper/022:49:320
1219183997023,13cyclictest1321204-21ssh00:31:510
1219183996931,34cyclictest48-21kcompactd022:00:250
1219183996927,38cyclictest3467602-21systemd-journal22:29:180
1219183996927,38cyclictest3467602-21systemd-journal22:29:180
1219183996926,10cyclictest1284410-21unin-run23:15:010
1219183996733,11cyclictest1229054-21ssh21:17:490
1219183996728,33cyclictest1311896-21sshd00:12:220
1219183996725,9cyclictest1241710-21sh21:45:000
1219183996527,34cyclictest1-21systemd20:19:450
1219183996525,8cyclictest1263460-21ssh22:31:070
1219183996524,37cyclictest1237005-21ssh21:35:000
1219183996524,13cyclictest1239390-21systemd-run21:40:010
1219183996517,16cyclictest1291388-21ssh23:29:300
1219183996428,4cyclictest1220675-21munin-run19:40:010
1219183996419,11cyclictest1260456-21modprobe22:25:000
1219183996354,5cyclictest1279913-21sshd23:05:060
1219183996219,17cyclictest1287613-21ssh23:21:160
1219183996214,21cyclictest1251707-21ssh22:06:140
1219189996116,11cyclictest131rcu_preempt00:20:002
1219183996133,22cyclictest1243918-21ssh21:49:410
1219183996113,20cyclictest1247732-21rm21:57:440
1219189995942,8cyclictest1239395-21kworker/2:000:30:022
1219183995932,20cyclictest1221940-21modprobe20:00:030
1219185995814,6cyclictest131rcu_preempt20:35:011
1219183995837,16cyclictest1219359-21modprobe19:15:000
1219183995828,10cyclictest1221767-21latency_hist20:00:020
1219183995828,10cyclictest1221767-21latency_hist20:00:020
1219183995819,34cyclictest1222455-21sshd20:11:540
1219183995819,34cyclictest1222455-21sshd20:11:540
1219183995818,35cyclictest1220049-21cat19:25:010
1219183995742,12cyclictest1223745-21sshd20:36:470
1219183995716,16cyclictest0-21swapper/022:12:500
1219183995716,16cyclictest0-21swapper/022:12:500
121918399566,18cyclictest12-21ksoftirqd/019:20:020
121918399566,18cyclictest12-21ksoftirqd/019:20:020
1219183995643,10cyclictest1225307-21sshd21:06:580
1219183995642,10cyclictest1227663-21munin-run21:15:010
1219183995642,10cyclictest1221426-21sshd19:53:400
1219183995640,13cyclictest1222993-21sshd20:21:560
1219183995624,5cyclictest3467602-21systemd-journal19:45:000
1219183995624,5cyclictest3467602-21systemd-journal19:45:000
121918399562,38cyclictest1268637-21ssh22:41:490
1219183995617,35cyclictest0-21swapper/019:35:000
1219183995614,38cyclictest0-21swapper/021:00:010
1219183995614,38cyclictest0-21swapper/021:00:010
1219189995547,5cyclictest1221754-21kworker/2:221:50:452
1219189995444,6cyclictest1239395-21kworker/2:023:14:592
1219189995438,13cyclictest1-21systemd20:30:012
1219189995438,13cyclictest1-21systemd20:30:012
1219183995441,9cyclictest1223249-21sshd20:27:460
1219183995441,9cyclictest1223249-21sshd20:27:450
1219189995345,5cyclictest1239395-21kworker/2:022:12:572
1219189995345,5cyclictest1239395-21kworker/2:022:12:572
1219189995344,6cyclictest1239395-21kworker/2:022:04:562
1219183995339,11cyclictest1221171-21sshd19:47:400
1219183995240,9cyclictest1224530-21sshd20:52:580
1219183995217,31cyclictest0-21swapper/022:35:230
1219193995134,12cyclictest1-21systemd20:55:003
1219189995143,5cyclictest1221754-21kworker/2:221:35:022
1219189995142,6cyclictest1239395-21kworker/2:022:37:302
1219189995142,6cyclictest1239395-21kworker/2:000:00:212
1219189995142,5cyclictest1239395-21kworker/2:000:32:582
1219189995132,15cyclictest1279704-21sed23:05:022
1219185995136,11cyclictest3467602-21systemd-journal20:20:471
1219185995133,13cyclictest1248643-21systemctl22:00:001
1219183995135,11cyclictest48-21kcompactd020:47:010
1219183995119,9cyclictest0-21swapper/020:45:020
1219183995119,9cyclictest0-21swapper/020:45:020
121919399502,6cyclictest131rcu_preempt00:00:013
1219193995022,10cyclictest37-21ksoftirqd/321:33:353
1219189995042,5cyclictest1239395-21kworker/2:022:21:142
1219189995042,5cyclictest1239395-21kworker/2:021:57:202
1219189995042,5cyclictest1221754-21kworker/2:221:35:522
1219189995041,6cyclictest1239395-21kworker/2:022:41:132
1219189995013,8cyclictest131rcu_preempt23:06:342
121918599509,6cyclictest131rcu_preempt22:54:451
121918999494,6cyclictest131rcu_preempt23:33:372
1219189994941,5cyclictest1305948-21kworker/2:200:21:462
1219189994941,5cyclictest1305948-21kworker/2:200:06:142
121919399482,15cyclictest1247298-21ssh21:56:493
121918999484,3cyclictest131rcu_preempt21:19:372
1219189994840,5cyclictest1239395-21kworker/2:023:21:232
1219189994840,5cyclictest1221754-21kworker/2:221:48:392
1219189994840,5cyclictest1198906-21kworker/2:121:27:562
1219189994839,6cyclictest1239395-21kworker/2:023:26:102
1219189994839,5cyclictest1239395-21kworker/2:023:51:032
1219185994833,12cyclictest1-21systemd19:30:011
1219185994833,10cyclictest275-21dbus-daemon19:35:011
1219185994828,8cyclictest1289353-21sed23:25:021
121918599481,3cyclictest131rcu_preempt00:00:191
1219185994811,6cyclictest131rcu_preempt22:05:351
1219193994732,11cyclictest1315462-21cron00:20:003
1219189994739,5cyclictest1239395-21kworker/2:023:35:152
1219189994737,5cyclictest1270075-21kworker/2:323:00:012
1219189994732,11cyclictest309-21in:imuxsock23:45:082
1219189994732,11cyclictest309-21in:imuxsock23:45:072
1219189994732,11cyclictest1-21systemd22:35:002
1219185994737,6cyclictest1317760-21latency_hist00:25:011
1219185994735,9cyclictest23-21ksoftirqd/121:20:001
121918599473,4cyclictest131rcu_preempt23:44:081
1219193994631,11cyclictest1239503-21cat21:40:023
121919399462,14cyclictest1309348-21ssh00:06:583
121919399461,5cyclictest131rcu_preempt22:20:023
1219193994611,10cyclictest1241756-21sed21:45:013
1219189994639,5cyclictest1270075-21kworker/2:323:42:022
1219189994638,5cyclictest1239395-21kworker/2:023:59:112
121918599462,4cyclictest131rcu_preempt23:18:471
121918599462,11cyclictest1278106-21ssh23:01:401
121919399456,4cyclictest131rcu_preempt23:25:023
121919399453,3cyclictest131rcu_preempt22:05:413
121919399452,8cyclictest1317446-21ssh00:24:073
121919399451,13cyclictest1297383-21ssh23:41:413
121919399451,12cyclictest1286182-21ssh23:18:373
121918999457,9cyclictest131rcu_preempt20:00:002
121918999457,9cyclictest131rcu_preempt20:00:002
1219189994536,6cyclictest30-21ksoftirqd/220:10:012
1219189994531,10cyclictest3467602-21systemd-journal20:16:552
121918599456,5cyclictest131rcu_preempt22:29:281
121918599456,5cyclictest131rcu_preempt22:29:281
121918599454,3cyclictest131rcu_preempt23:50:461
1219185994534,8cyclictest1314294-21sshd00:17:221
1219185994510,7cyclictest23-21ksoftirqd/122:17:251
121919399443,4cyclictest131rcu_preempt22:46:573
1219193994419,9cyclictest37-21ksoftirqd/321:15:013
121918999447,7cyclictest30-21ksoftirqd/222:46:142
121918999446,7cyclictest131rcu_preempt21:10:002
121918999446,7cyclictest131rcu_preempt19:40:012
1219189994436,6cyclictest1239395-21kworker/2:022:54:152
1219189994436,5cyclictest1239395-21kworker/2:022:27:042
1219189994436,5cyclictest1239395-21kworker/2:022:27:032
1219189994436,5cyclictest1221754-21kworker/2:221:40:122
1219189994429,11cyclictest1-21systemd19:55:012
121918599444,3cyclictest131rcu_preempt22:42:241
121918599441,3cyclictest131rcu_preempt00:34:331
121918599441,11cyclictest1244860-21ssh21:51:241
1219185994410,8cyclictest0-21swapper/121:21:391
1219183994423,17cyclictest1219278-21latency_hist19:10:020
1219183994423,17cyclictest1219278-21latency_hist19:10:020
1219193994334,6cyclictest0-21swapper/319:10:023
1219193994334,6cyclictest0-21swapper/319:10:023
121919399433,3cyclictest131rcu_preempt23:11:033
1219193994331,8cyclictest1224792-21cron21:00:003
1219193994331,8cyclictest1224792-21cron21:00:003
1219193994329,11cyclictest1223891-21cat20:40:023
121919399432,3cyclictest150-21pr/ttyS222:57:363
121919399432,3cyclictest131rcu_preempt22:40:293
121919399431,4cyclictest131rcu_preempt23:33:153
121919399431,4cyclictest131rcu_preempt22:32:143
121919399431,3cyclictest131rcu_preempt00:27:203
121919399431,11cyclictest1321947-21ssh00:33:333
1219189994335,5cyclictest1198906-21kworker/2:119:33:092
1219189994334,6cyclictest1305948-21kworker/2:200:13:032
1219189994310,8cyclictest131rcu_preempt22:07:212
121918599439,5cyclictest131rcu_preempt23:09:491
1219185994331,8cyclictest1225542-21latency_hist21:10:031
121919399427,4cyclictest131rcu_preempt23:35:133
121919399424,3cyclictest131rcu_preempt20:20:013
121919399423,8cyclictest1258891-21ssh22:21:223
121919399423,3cyclictest131rcu_preempt00:39:513
1219193994230,9cyclictest120-21kswapd023:30:013
121919399422,7cyclictest1229913-21ssh21:19:473
1219193994226,12cyclictest1219356-21cron19:15:003
1219193994224,15cyclictest37-21ksoftirqd/300:14:243
1219193994224,13cyclictest1267606-21cron22:40:003
121919399422,3cyclictest131rcu_preempt22:55:013
121919399422,3cyclictest131rcu_preempt00:01:013
121919399421,6cyclictest1280939-21ssh23:07:413
121919399421,4cyclictest131rcu_preempt23:52:303
121919399421,4cyclictest131rcu_preempt23:04:403
121919399421,3cyclictest131rcu_preempt23:45:383
121919399421,3cyclictest131rcu_preempt23:45:383
121918999426,3cyclictest131rcu_preempt21:05:002
1219189994234,5cyclictest1198906-21kworker/2:120:12:212
1219189994234,5cyclictest1198906-21kworker/2:120:12:212
1219189994211,6cyclictest131rcu_preempt21:13:102
121918599424,7cyclictest0-21swapper/121:27:291
121918599423,3cyclictest131rcu_preempt22:50:021
121918599422,9cyclictest1239752-21ssh21:40:191
1219185994229,9cyclictest1-21systemd22:15:011
1219185994229,9cyclictest1-21systemd22:15:011
121918599422,4cyclictest131rcu_preempt22:34:381
1219185994222,11cyclictest0-21swapper/119:20:001
1219185994222,11cyclictest0-21swapper/119:20:001
121918599422,12cyclictest1292090-21ssh23:30:371
121918599421,14cyclictest346771850lldpd21:04:481
121918599421,11cyclictest1312354-21ssh00:13:201
1219193994129,9cyclictest37-21ksoftirqd/321:52:023
1219193994126,12cyclictest1251004-21systemd-run22:05:003
1219193994123,11cyclictest275-21dbus-daemon19:25:003
121919399411,3cyclictest131rcu_preempt22:26:073
121919399411,3cyclictest131rcu_preempt22:26:063
121919399411,3cyclictest131rcu_preempt22:11:053
121919399411,3cyclictest131rcu_preempt22:11:043
121919399411,10cyclictest1225343-21diskmemload21:23:233
121918999417,6cyclictest131rcu_preempt00:37:522
121918999415,5cyclictest131rcu_preempt20:40:012
121918999414,4cyclictest131rcu_preempt21:00:002
121918999414,4cyclictest131rcu_preempt21:00:002
121918999412,5cyclictest0-21swapper/221:21:372
121918599415,3cyclictest131rcu_preempt23:27:071
121918599413,32cyclictest1305881-40mandb00:00:001
1219185994130,7cyclictest1325095-21wc00:40:031
1219185994126,11cyclictest23-21ksoftirqd/121:46:191
1219185994122,15cyclictest275-21dbus-daemon21:40:011
121918599411,4cyclictest131rcu_preempt00:06:501
121919399403,4cyclictest131rcu_preempt20:15:023
121919399403,4cyclictest131rcu_preempt20:15:023
1219193994030,7cyclictest1220926-21sshd19:44:403
1219193994030,7cyclictest1220926-21sshd19:44:403
1219193994027,9cyclictest1221708-21systemctl20:00:003
1219193994027,9cyclictest1221708-21systemctl20:00:003
1219193994027,11cyclictest37-21ksoftirqd/320:05:013
121919399402,3cyclictest131rcu_preempt20:30:023
121919399402,3cyclictest131rcu_preempt20:30:023
121919399401,3cyclictest150-21pr/ttyS221:45:493
121918999401,5cyclictest0-21swapper/222:16:262
121918999401,3cyclictest131rcu_preempt23:17:582
1219185994028,9cyclictest3467602-21systemd-journal22:58:081
1219185994026,11cyclictest1219980-21sed19:25:011
1219185994025,11cyclictest23-21ksoftirqd/120:00:001
1219185994025,11cyclictest23-21ksoftirqd/120:00:001
1219185994025,10cyclictest3467602-21systemd-journal19:40:011
121919399396,4cyclictest131rcu_preempt19:48:403
121919399394,3cyclictest131rcu_preempt21:10:013
1219193993927,10cyclictest37-21ksoftirqd/321:26:183
121918999394,5cyclictest131rcu_preempt19:17:412
121918999394,5cyclictest131rcu_preempt19:17:412
1219189993930,5cyclictest309-21in:imuxsock19:14:592
1219189993926,10cyclictest1223009-21cron20:25:002
121918599396,25cyclictest3467602-21systemd-journal19:55:011
1219185993927,8cyclictest3467602-21systemd-journal20:55:001
121918599392,4cyclictest131rcu_preempt22:21:001
121918599392,4cyclictest131rcu_preempt21:31:281
1219193993825,9cyclictest3467602-21systemd-journal21:05:003
121919399381,3cyclictest131rcu_preempt19:17:403
121919399381,3cyclictest131rcu_preempt19:17:403
1219189993830,5cyclictest1198906-21kworker/2:120:02:142
1219189993828,6cyclictest309-21in:imuxsock20:35:002
121918999381,3cyclictest131rcu_preempt20:55:002
121918599383,3cyclictest131rcu_preempt00:26:581
1219185993829,7cyclictest23-21ksoftirqd/119:45:011
1219185993829,7cyclictest23-21ksoftirqd/119:45:011
1219185993825,10cyclictest23-21ksoftirqd/119:15:001
1219185993818,14cyclictest275-21dbus-daemon20:20:001
1219193993726,8cyclictest1224038-21sshd20:44:483
1219193993726,8cyclictest1224038-21sshd20:44:483
1219193993722,12cyclictest309-21in:imuxsock19:36:503
121919399371,4cyclictest131rcu_preempt19:52:393
121918999376,3cyclictest131rcu_preempt20:49:482
1219189993725,9cyclictest30-21ksoftirqd/219:10:022
1219189993725,9cyclictest30-21ksoftirqd/219:10:022
1219189993723,9cyclictest275-21dbus-daemon20:45:012
1219189993723,9cyclictest275-21dbus-daemon20:45:012
121918599375,4cyclictest131rcu_preempt22:05:021
121918599372,4cyclictest131rcu_preempt21:11:251
121919399364,4cyclictest131rcu_preempt19:30:013
121918999364,4cyclictest131rcu_preempt19:45:082
121918599365,7cyclictest23-21ksoftirqd/122:40:021
121918599365,6cyclictest131rcu_preempt23:13:081
121918599364,3cyclictest131rcu_preempt20:30:021
121918599364,3cyclictest131rcu_preempt20:30:021
121918599361,8cyclictest1294496-21ssh23:35:331
1219185993616,16cyclictest275-21dbus-daemon20:40:011
121919399352,4cyclictest131rcu_preempt20:25:003
121919399352,23cyclictest1224306-21munin-run20:50:003
1219193993517,14cyclictest275-21dbus-daemon20:10:013
121918599358,20cyclictest1221189-21modprobe19:50:001
121918599352,3cyclictest131rcu_preempt21:00:011
121918599352,3cyclictest131rcu_preempt21:00:001
121919399343,3cyclictest0-21swapper/320:34:463
1219189993417,13cyclictest275-21dbus-daemon19:45:022
1219189993417,13cyclictest275-21dbus-daemon19:45:022
121918999341,4cyclictest131rcu_preempt19:29:012
121918599349,16cyclictest3467602-21systemd-journal20:45:001
121918599349,16cyclictest3467602-21systemd-journal20:45:001
1219185993423,7cyclictest1222365-21cat20:10:021
121918599335,3cyclictest131rcu_preempt20:15:011
121918599335,3cyclictest131rcu_preempt20:15:001
121918599334,4cyclictest131rcu_preempt20:50:011
121918999324,4cyclictest131rcu_preempt19:25:002
1219185993225,4cyclictest1219110-21latency_hist19:10:031
121918599314,21cyclictest1221981-21munin-plugin-st20:05:001
1219185993023,4cyclictest1219110-21latency_hist19:10:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional