You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-28 - 14:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Thu May 28, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3211599250,24cyclictest9330-21ssh22:57:423
3211199250,1cyclictest0-21swapper/223:27:552
32105992523,1cyclictest0-21swapper/123:23:171
3210599250,24cyclictest0-21swapper/122:03:191
3210599250,24cyclictest0-21swapper/121:37:101
3210599250,1cyclictest24503-21awk22:10:011
3210099250,24cyclictest0-21swapper/022:52:490
32115992422,1cyclictest30791-21ssh22:16:093
3211599240,24cyclictest25323-21diskmemload21:22:403
3211599240,23cyclictest25323-21diskmemload00:07:403
3211599240,23cyclictest16814-21cron19:45:013
3211599240,23cyclictest13450-21taskset21:59:243
3211599240,23cyclictest0-21swapper/322:44:513
3211599240,1cyclictest30458-21munin-run21:45:003
3211599240,1cyclictest0-21swapper/321:12:083
32111992420,2cyclictest0-21swapper/221:28:202
3211199240,24cyclictest0-21swapper/223:59:182
3211199240,24cyclictest0-21swapper/223:41:122
3211199240,23cyclictest622-21gdbus19:45:012
3211199240,1cyclictest5323-21ssh21:52:492
32105992424,0cyclictest0-21swapper/123:04:371
32105992422,1cyclictest0-21swapper/121:50:001
32105992421,2cyclictest31161-21ssh00:18:161
3210599240,24cyclictest0-21swapper/123:41:351
3210599240,23cyclictest0-21swapper/123:19:441
3210599240,23cyclictest0-21swapper/123:10:471
3210599240,0cyclictest25323-21diskmemload23:09:351
32100992423,1cyclictest22327-21sh23:08:310
32100992422,1cyclictest6547-21cat00:25:010
32100992421,2cyclictest25323-21diskmemload22:33:160
3210099240,23cyclictest25323-21diskmemload00:26:480
3210099240,23cyclictest21764-21sh00:38:390
3210099240,23cyclictest13625-21sh21:59:390
3210099240,23cyclictest0-21swapper/023:41:030
3210099240,23cyclictest0-21swapper/022:04:400
3210099240,23cyclictest0-21swapper/000:30:400
32115992323,0cyclictest25323-21diskmemload23:41:123
32115992323,0cyclictest25323-21diskmemload00:25:453
32115992323,0cyclictest0-21swapper/323:46:493
32115992323,0cyclictest0-21swapper/322:30:463
32115992322,1cyclictest25323-21diskmemload23:06:013
32115992322,1cyclictest23257-21ssh00:10:063
32115992321,1cyclictest15133-21irqstats00:03:193
3211599230,22cyclictest27126-21/usr/sbin/munin23:13:213
3211599230,22cyclictest25323-21diskmemload21:50:353
3211599230,22cyclictest24111-21awk20:00:013
3211599230,22cyclictest10488-21/usr/sbin/munin23:58:243
3211599230,22cyclictest0-21swapper/323:16:013
3211599230,1cyclictest6275-21ssh00:24:463
3211599230,0cyclictest25323-21diskmemload23:30:193
3211599230,0cyclictest25323-21diskmemload21:26:513
3211599230,0cyclictest25323-21diskmemload00:36:123
3211599230,0cyclictest0-21swapper/323:50:573
3211599230,0cyclictest0-21swapper/323:27:553
3211599230,0cyclictest0-21swapper/323:04:283
32111992323,0cyclictest25323-21diskmemload23:46:492
32111992323,0cyclictest25323-21diskmemload22:30:462
32111992323,0cyclictest0-21swapper/223:18:052
32111992323,0cyclictest0-21swapper/222:52:522
32111992323,0cyclictest0-21swapper/222:21:392
32111992323,0cyclictest0-21swapper/221:57:212
32111992323,0cyclictest0-21swapper/221:42:582
32111992323,0cyclictest0-21swapper/200:01:202
32111992320,3cyclictest0-21swapper/200:15:322
32111992320,2cyclictest16311-21ssh00:33:252
3211199230,2cyclictest1652-21ssh22:18:252
3211199230,23cyclictest21143-21ssh22:07:432
3211199230,23cyclictest0-21swapper/223:01:262
3211199230,22cyclictest17592-21ssh23:34:512
3211199230,0cyclictest25323-21diskmemload22:44:512
3211199230,0cyclictest0-21swapper/222:39:312
32105992323,0cyclictest0-21swapper/123:52:521
32105992323,0cyclictest0-21swapper/123:30:221
32105992323,0cyclictest0-21swapper/122:33:051
32105992323,0cyclictest0-21swapper/100:21:591
32105992321,1cyclictest0-21swapper/123:56:301
32105992321,1cyclictest0-21swapper/121:52:011
32105992321,1cyclictest0-21swapper/119:33:141
32105992321,1cyclictest0-21swapper/100:07:511
3210599230,2cyclictest8206-21ssh23:26:081
3210599230,23cyclictest25323-21diskmemload22:26:461
3210599230,22cyclictest4671-21sh21:20:481
3210599230,22cyclictest30627-21apt-get21:15:021
3210599230,22cyclictest0-21swapper/122:21:211
3210599230,22cyclictest0-21swapper/100:30:381
3210599230,22cyclictest0-21swapper/100:27:121
3210599230,0cyclictest25323-21diskmemload23:36:561
3210599230,0cyclictest25323-21diskmemload22:35:221
3210599230,0cyclictest0-21swapper/121:34:231
3210599230,0cyclictest0-21swapper/121:29:141
3210599230,0cyclictest0-21swapper/121:10:191
32100992323,0cyclictest0-21swapper/023:26:300
32100992322,0cyclictest0-21swapper/000:11:590
32100992321,1cyclictest0-21swapper/023:22:510
32100992321,1cyclictest0-21swapper/023:10:340
32100992321,1cyclictest0-21swapper/023:01:240
3210099231,21cyclictest0-21swapper/023:30:370
3210099230,23cyclictest16578-21ssh21:32:450
3210099230,23cyclictest0-21swapper/022:12:280
3210099230,23cyclictest0-21swapper/022:06:530
3210099230,22cyclictest25323-21diskmemload22:44:320
3210099230,22cyclictest2410-21turbostat.cron19:15:000
3210099230,22cyclictest21690-21grep19:55:010
3210099230,22cyclictest19381-21sh22:35:570
3210099230,22cyclictest0-21swapper/023:36:030
3210099230,22cyclictest0-21swapper/022:46:430
3210099230,22cyclictest0-21swapper/021:42:540
3210099230,22cyclictest0-21swapper/021:35:200
3210099230,22cyclictest0-21swapper/021:26:030
3210099230,22cyclictest0-21swapper/021:15:280
3210099230,1cyclictest0-21swapper/000:18:010
3210099230,0cyclictest0-21swapper/022:21:460
32115992222,0cyclictest25323-21diskmemload22:52:523
32115992222,0cyclictest0-21swapper/322:07:243
32115992220,1cyclictest8769-21ssh22:25:523
32115992220,1cyclictest4063-21ssh23:22:503
32115992220,1cyclictest28892-21ssh00:15:383
32115992220,1cyclictest26997-21apt-get20:08:143
32115992220,1cyclictest18966-21ssh22:35:143
32115992220,1cyclictest0-21swapper/322:00:133
32115992220,1cyclictest0-21swapper/300:31:443
3211599221,21cyclictest327-21sh21:17:213
3211599221,1cyclictest25323-21diskmemload23:36:303
3211599220,22cyclictest0-21swapper/321:35:373
3211599220,21cyclictest32215-21ssh21:47:213
3211599220,21cyclictest3085-21sh22:20:153
3211599220,21cyclictest0-21swapper/322:10:243
3211599220,21cyclictest0-21swapper/319:23:133
3211599220,1cyclictest0-21swapper/322:49:523
32111992222,0cyclictest0-21swapper/223:23:422
32111992222,0cyclictest0-21swapper/223:06:012
32111992222,0cyclictest0-21swapper/222:56:342
32111992222,0cyclictest0-21swapper/222:26:582
32111992222,0cyclictest0-21swapper/200:06:192
32111992221,1cyclictest0-21swapper/221:45:122
32111992221,0cyclictest0-21swapper/200:10:022
32111992217,1cyclictest30333-21ssh21:14:372
3211199220,22cyclictest0-21swapper/221:16:202
3211199220,21cyclictest5347-21apt-get19:23:132
3211199220,21cyclictest25323-21diskmemload23:53:162
3211199220,21cyclictest18401-21ssh23:35:372
3211199220,21cyclictest0-21swapper/200:25:022
3211199220,1cyclictest8825-21ssh21:24:242
3211199220,1cyclictest16534-21ssh21:32:412
3211199220,0cyclictest0-21swapper/221:35:372
3211199220,0cyclictest0-21swapper/219:58:142
32105992222,0cyclictest0-21swapper/122:58:171
32105992222,0cyclictest0-21swapper/122:16:551
32105992220,1cyclictest606-21dbus-daemon19:30:001
32105992220,1cyclictest25034-21ssh22:41:301
3210599220,22cyclictest0-21swapper/122:50:161
3210599220,21cyclictest0-21swapper/123:45:241
3210599220,21cyclictest0-21swapper/122:45:441
3210599220,21cyclictest0-21swapper/121:41:191
3210599220,21cyclictest0-21swapper/119:18:131
3210599220,21cyclictest0-21swapper/100:14:101
3210599220,21cyclictest0-21swapper/100:02:401
3210599220,1cyclictest0-21swapper/100:36:321
3210599220,0cyclictest25323-21diskmemload22:10:421
3210599220,0cyclictest0-21swapper/121:56:211
3210599220,0cyclictest0-21swapper/120:08:131
32100992222,0cyclictest25323-21diskmemload22:27:590
32100992220,1cyclictest0-21swapper/023:53:190
32100992220,1cyclictest0-21swapper/022:58:260
3210099220,22cyclictest0-21swapper/023:15:150
3210099220,22cyclictest0-21swapper/022:15:060
3210099220,21cyclictest7241-21sh23:55:440
3210099220,21cyclictest0-21swapper/021:50:590
3210099220,21cyclictest0-21swapper/020:23:120
3210099220,21cyclictest0-21swapper/020:03:140
3210099220,1cyclictest0-21swapper/023:46:580
3210099220,1cyclictest0-21swapper/000:02:270
3210099220,0cyclictest0-21swapper/000:06:340
32115992120,1cyclictest16194-21apt-get20:53:133
32115992119,1cyclictest0-21swapper/320:50:013
3211599210,20cyclictest1800-21apt-get20:23:123
3211599210,20cyclictest0-21swapper/320:18:133
3211599210,0cyclictest0-21swapper/321:32:413
3211599210,0cyclictest0-21swapper/319:38:133
32111992121,0cyclictest7741-21apt-get19:28:142
32111992119,1cyclictest15683-21perf20:50:012
3211199210,1cyclictest31788-21apt-get20:18:132
3211199210,1cyclictest29529-21ssh22:45:152
3211199210,1cyclictest17731-21ssh00:35:092
3211199210,0cyclictest0-21swapper/222:10:242
3211199210,0cyclictest0-21swapper/222:00:142
3211199210,0cyclictest0-21swapper/220:08:142
32105992119,1cyclictest6061-21cron20:30:001
32105992119,1cyclictest13799-21apt-get20:48:141
32100992120,1cyclictest0-21swapper/021:08:140
3210099210,20cyclictest13308-21perf20:45:010
3210099210,20cyclictest0-21swapper/020:13:120
3210099210,20cyclictest0-21swapper/019:23:130
32115992020,0cyclictest0-21swapper/319:28:143
32115992020,0cyclictest0-21swapper/319:13:143
32111992020,0cyclictest0-21swapper/220:53:132
32111992019,0cyclictest23452-21apt-get21:08:142
3211199200,0cyclictest0-21swapper/220:23:122
32105992020,0cyclictest0-21swapper/119:46:171
32105992018,1cyclictest18589-21apt-get20:58:151
32100992020,0cyclictest0-21swapper/020:18:130
32100992020,0cyclictest0-21swapper/019:48:120
32115991919,0cyclictest0-21swapper/321:08:143
32115991917,1cyclictest2930-21apt-get19:18:133
32105991919,0cyclictest16194-21apt-get20:53:131
32105991919,0cyclictest12533-21apt-get19:38:131
32105991919,0cyclictest0-21swapper/120:33:141
32105991916,2cyclictest24104-21sendmail-msp20:00:011
32100991917,1cyclictest14438-21apt19:40:000
32115991817,1cyclictest19738-21apt-get19:53:133
32115991815,2cyclictest0-21swapper/320:43:123
32111991818,0cyclictest4198-21apt-get20:28:142
32111991818,0cyclictest0-21swapper/219:53:132
32111991817,0cyclictest0-21swapper/219:18:132
32105991818,0cyclictest0-21swapper/120:18:131
32105991817,0cyclictest0-21swapper/120:38:141
32100991818,0cyclictest0-21swapper/020:53:130
32100991818,0cyclictest0-21swapper/020:38:140
32115991717,0cyclictest0-21swapper/320:58:153
32115991717,0cyclictest0-21swapper/320:38:143
32115991716,1cyclictest10141-21apt-get19:33:143
32111991717,0cyclictest0-21swapper/220:58:152
32111991717,0cyclictest0-21swapper/220:38:142
32111991717,0cyclictest0-21swapper/219:33:142
32111991716,1cyclictest2418-21perf19:15:002
32111991716,0cyclictest29391-21apt-get20:13:132
32105991717,0cyclictest0-21swapper/121:03:131
32105991717,0cyclictest0-21swapper/119:43:131
32105991717,0cyclictest0-21swapper/119:13:141
32105991716,1cyclictest0-21swapper/120:43:131
32100991717,0cyclictest0-21swapper/020:10:000
32100991717,0cyclictest0-21swapper/019:58:140
32100991716,1cyclictest15702-21munin-run20:50:010
32115991616,0cyclictest0-21swapper/320:33:143
32115991614,1cyclictest21675-21gltestperf21:03:163
32111991616,0cyclictest0-21swapper/220:43:122
32111991616,0cyclictest0-21swapper/220:33:142
32100991616,0cyclictest0-21swapper/019:28:140
32100991616,0cyclictest0-21swapper/019:18:130
32100991614,1cyclictest0-21swapper/020:58:190
32100991614,1cyclictest0-21swapper/019:43:140
32115991515,0cyclictest0-21swapper/320:13:133
32111991514,1cyclictest24604-21apt-get20:03:142
32105991515,0cyclictest0-21swapper/120:23:121
32105991515,0cyclictest0-21swapper/120:03:131
32105991515,0cyclictest0-21swapper/119:23:131
3210599150,7cyclictest313-21systemd-journal19:55:011
3210099150,14cyclictest11186-21ntp_states19:33:200
3211599140,14cyclictest0-21swapper/320:00:293
3211599140,14cyclictest0-21swapper/319:46:423
32111991414,0cyclictest0-21swapper/219:35:282
3211199140,14cyclictest0-21swapper/221:00:162
3211199140,14cyclictest0-21swapper/219:49:052
3210599140,14cyclictest0-21swapper/120:10:091
3210599140,0cyclictest0-21swapper/121:05:071
32100991414,0cyclictest0-21swapper/020:25:070
3210099140,14cyclictest0-21swapper/020:31:130
3210099140,0cyclictest0-21swapper/021:00:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional