You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-13 - 08:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Sun Sep 13, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3911992619,2cyclictest3747-21ssh22:42:512
390799262,1cyclictest16888-21sh22:55:451
390799260,25cyclictest26208-21kworker/u8:121:36:061
391199250,24cyclictest29549-21diskmemload23:23:382
391199250,24cyclictest16308-21perf19:35:012
3898992521,3cyclictest0-21swapper/022:27:160
389899251,23cyclictest62550irq/127-eno123:52:410
3912992422,1cyclictest0-21swapper/321:48:193
391299240,23cyclictest29549-21diskmemload23:55:193
391299240,23cyclictest0-21swapper/319:42:073
391299240,1cyclictest0-21swapper/321:58:063
3911992422,1cyclictest1493-21ssh23:40:542
3911992421,2cyclictest19815-21ssh22:57:292
391199240,20cyclictest0-21swapper/222:17:182
3907992421,2cyclictest10824-21ssh22:20:221
390799240,2cyclictest9199-21cron21:20:011
390799240,24cyclictest0-21swapper/122:36:401
390799240,24cyclictest0-21swapper/100:10:591
390799240,23cyclictest10792-21date22:50:011
390799240,23cyclictest0-21swapper/100:28:571
390799240,1cyclictest0-21swapper/123:44:421
390799240,0cyclictest0-21swapper/123:01:171
390799240,0cyclictest0-21swapper/100:37:321
389899240,24cyclictest0-21swapper/021:12:520
389899240,23cyclictest29549-21diskmemload23:15:130
389899240,23cyclictest29549-21diskmemload22:56:320
389899240,23cyclictest29549-21diskmemload21:21:060
389899240,23cyclictest29549-21diskmemload00:21:340
389899240,23cyclictest24756-21latency_hist21:00:000
389899240,23cyclictest0-21swapper/021:28:250
389899240,1cyclictest0-21swapper/000:18:220
389899240,0cyclictest29549-21diskmemload22:53:490
389899240,0cyclictest0-21swapper/023:36:180
389899240,0cyclictest0-21swapper/000:31:020
389899240,0cyclictest0-21swapper/000:02:110
3912992323,0cyclictest29549-21diskmemload23:09:503
3912992323,0cyclictest29549-21diskmemload22:08:123
3912992323,0cyclictest0-21swapper/322:51:243
3912992323,0cyclictest0-21swapper/321:40:023
3912992323,0cyclictest0-21swapper/321:23:123
3912992323,0cyclictest0-21swapper/321:13:593
3912992322,1cyclictest7729-21rm21:17:533
3912992321,1cyclictest21995-21ssh22:01:223
3912992321,1cyclictest16796-21ssh22:25:513
391299230,2cyclictest13830-21cron19:30:003
391299230,23cyclictest0-21swapper/300:15:153
391299230,22cyclictest29549-21diskmemload21:38:583
391299230,22cyclictest28038-21ssh00:34:263
391299230,22cyclictest27409-21ssh22:35:443
391299230,22cyclictest19028-21sh23:27:103
391299230,22cyclictest0-21swapper/323:40:563
391299230,0cyclictest0-21swapper/322:40:183
391299230,0cyclictest0-21swapper/300:09:153
3911992323,0cyclictest29549-21diskmemload23:38:242
3911992323,0cyclictest29549-21diskmemload00:18:472
3911992323,0cyclictest0-21swapper/222:39:022
3911992323,0cyclictest0-21swapper/200:11:582
3911992323,0cyclictest0-21swapper/200:06:262
3911992321,1cyclictest0-21swapper/200:20:492
3911992318,5cyclictest0-21swapper/221:46:192
3911992318,4cyclictest0-21swapper/221:10:472
391199230,23cyclictest29549-21diskmemload22:04:242
391199230,23cyclictest29549-21diskmemload21:26:372
391199230,23cyclictest0-21swapper/222:20:372
391199230,23cyclictest0-21swapper/200:37:562
391199230,22cyclictest9679-21vmstat23:47:182
391199230,22cyclictest7382-21ssh22:47:112
391199230,22cyclictest27127-21sh21:36:302
391199230,22cyclictest23310-21ssh00:00:282
391199230,22cyclictest17853-21ssh00:25:172
391199230,22cyclictest10533-21ssh23:19:032
391199230,0cyclictest0-21swapper/223:06:092
391199230,0cyclictest0-21swapper/223:02:082
391199230,0cyclictest0-21swapper/222:13:582
3907992323,0cyclictest0-21swapper/122:53:011
3907992323,0cyclictest0-21swapper/122:25:351
3907992323,0cyclictest0-21swapper/100:20:351
3907992323,0cyclictest0-21swapper/100:01:391
3907992321,1cyclictest6720-21rm00:15:011
3907992321,1cyclictest0-21swapper/123:31:361
3907992321,1cyclictest0-21swapper/123:31:361
3907992321,1cyclictest0-21swapper/121:27:111
3907992320,2cyclictest28510-21sh00:05:061
3907992320,2cyclictest13112-21ssh21:22:431
390799230,2cyclictest22235-21ssh23:59:331
390799230,23cyclictest0-21swapper/123:12:441
390799230,23cyclictest0-21swapper/121:57:321
390799230,22cyclictest5421-21apt-get23:15:021
390799230,22cyclictest32142-21ssh23:38:371
390799230,22cyclictest24776-21ssh00:32:081
390799230,22cyclictest22392-21rm22:31:101
390799230,22cyclictest0-21swapper/122:01:501
390799230,22cyclictest0-21swapper/121:31:361
3898992323,0cyclictest62550irq/127-eno122:38:580
3898992323,0cyclictest29549-21diskmemload23:44:420
3898992323,0cyclictest29549-21diskmemload23:21:000
3898992323,0cyclictest0-21swapper/022:24:120
3898992323,0cyclictest0-21swapper/020:17:090
3898992322,1cyclictest29549-21diskmemload22:15:340
3898992321,1cyclictest29041-21sh00:35:350
3898992321,1cyclictest22619-21sh23:30:180
3898992321,1cyclictest22619-21sh23:30:180
3898992321,1cyclictest0-21swapper/022:49:520
3898992321,1cyclictest0-21swapper/000:27:010
3898992320,2cyclictest16886-21sh21:56:540
389899230,22cyclictest28174-21cpuspeed_turbos22:07:100
389899230,22cyclictest21984-21diskmemload22:01:210
389899230,22cyclictest20837-21sh23:28:120
389899230,22cyclictest0-21swapper/023:47:090
389899230,22cyclictest0-21swapper/021:42:130
389899230,22cyclictest0-21swapper/021:15:090
389899230,22cyclictest0-21swapper/000:06:000
389899230,1cyclictest0-21swapper/000:13:480
3912992222,0cyclictest0-21swapper/323:15:263
3912992222,0cyclictest0-21swapper/322:58:463
3912992222,0cyclictest0-21swapper/322:16:323
3912992222,0cyclictest0-21swapper/321:50:283
3912992220,1cyclictest28081-21diskmemload23:35:193
3912992220,1cyclictest25203-21apt-get21:02:103
3912992220,1cyclictest17620-21ntp_states21:27:163
391299220,2cyclictest12201-21ssh23:20:593
391299220,22cyclictest6530-21ssh23:45:183
391299220,22cyclictest0-21swapper/323:54:263
391299220,21cyclictest29549-21diskmemload23:00:023
391299220,21cyclictest29549-21diskmemload21:30:483
391299220,21cyclictest21784-21sh22:30:243
391299220,21cyclictest19974-21perf20:50:003
391299220,21cyclictest1626-21apt-get23:12:103
391299220,21cyclictest10772-21ssh22:20:183
391299220,21cyclictest0-21swapper/322:45:243
391299220,21cyclictest0-21swapper/320:37:093
391299220,1cyclictest29362-21ssh00:36:003
391299220,1cyclictest2054-21missed_timers22:12:133
391299220,1cyclictest18670-21ssh00:26:283
391299220,1cyclictest0-21swapper/323:31:363
391299220,1cyclictest0-21swapper/323:31:363
391299220,1cyclictest0-21swapper/300:11:173
3911992222,0cyclictest0-21swapper/222:32:382
3911992222,0cyclictest0-21swapper/221:40:492
3911992221,0cyclictest0-21swapper/223:27:082
3911992220,1cyclictest24466-21sh00:31:452
3911992220,1cyclictest17558-21diskstats21:57:112
3911992220,1cyclictest11880-21apt-get19:27:122
3911992220,1cyclictest10291-21sh21:50:152
3911992219,3cyclictest0-21swapper/222:09:352
3911992219,2cyclictest11430-21cron19:25:002
391199220,2cyclictest11158-21ssh22:50:192
391199220,22cyclictest29549-21diskmemload21:21:582
391199220,21cyclictest29554-21idleruntime-cro21:10:002
391199220,21cyclictest29554-21idleruntime-cro21:10:002
391199220,21cyclictest16736-21ssh22:25:472
391199220,21cyclictest0-21swapper/223:50:452
391199220,21cyclictest0-21swapper/221:30:032
391199220,1cyclictest23524-21munin-run19:50:012
391199220,1cyclictest22374-21awk20:55:002
3907992222,0cyclictest29549-21diskmemload21:47:311
3907992222,0cyclictest0-21swapper/123:06:431
3907992222,0cyclictest0-21swapper/122:10:331
3907992220,1cyclictest30709-21latency_hist20:05:001
3907992220,1cyclictest22378-21perf20:55:001
3907992220,1cyclictest0-21swapper/122:40:381
390799220,21cyclictest3516-21apt-get20:17:091
390799220,21cyclictest28329-21munin-run20:00:001
390799220,21cyclictest13128-21apt-get20:37:091
390799220,21cyclictest11956-21ssh23:20:401
390799220,21cyclictest0-21swapper/123:25:091
390799220,21cyclictest0-21swapper/122:07:301
390799220,1cyclictest31867-21sh21:40:301
390799220,1cyclictest11875-21ssh23:50:081
3898992222,0cyclictest62550irq/127-eno122:31:070
3898992222,0cyclictest0-21swapper/023:10:320
3898992222,0cyclictest0-21swapper/022:40:150
3898992221,1cyclictest27247-21rm21:36:390
3898992220,1cyclictest29549-21diskmemload23:00:340
3898992220,1cyclictest20388-21apt-get20:52:110
3898992220,1cyclictest0-21swapper/023:58:300
3898992220,1cyclictest0-21swapper/021:30:530
3898992220,1cyclictest0-21swapper/020:30:010
389899220,21cyclictest6263-21munin-run19:15:000
389899220,21cyclictest6250-21apt-get19:15:020
389899220,21cyclictest29549-21diskmemload22:10:030
389899220,21cyclictest0-21swapper/021:50:220
389899220,21cyclictest0-21swapper/020:35:010
389899220,1cyclictest28404-21ssh23:06:510
389899220,0cyclictest0-21swapper/021:46:440
3912992121,0cyclictest0-21swapper/319:32:113
3912992121,0cyclictest0-21swapper/300:21:323
3912992120,1cyclictest0-21swapper/320:52:113
3912992120,1cyclictest0-21swapper/320:32:113
391299210,21cyclictest0-21swapper/300:00:163
391299210,0cyclictest0-21swapper/319:37:093
3911992121,0cyclictest0-21swapper/223:30:262
3911992121,0cyclictest0-21swapper/223:30:262
3911992121,0cyclictest0-21swapper/223:12:092
3911992120,1cyclictest0-21swapper/223:55:122
3911992119,1cyclictest19982-21munin-run20:50:002
3911992119,1cyclictest15167-21turbostat20:40:012
3907992120,1cyclictest15600-21apt-get20:42:101
3907992118,2cyclictest622-21gdbus19:35:011
390799210,21cyclictest0-21swapper/121:50:461
390799210,1cyclictest6747-21ssh23:45:341
3898992119,1cyclictest606-21dbus-daemon20:15:000
389899210,0cyclictest0-21swapper/019:52:100
389899210,0cyclictest0-21swapper/019:27:120
391299200,0cyclictest31133-21apt-get20:07:073
3911992020,0cyclictest0-21swapper/220:22:102
3911992019,1cyclictest0-21swapper/220:07:072
3911992018,1cyclictest7068-21apt-get19:17:102
391199200,20cyclictest0-21swapper/220:32:102
3907992020,0cyclictest0-21swapper/120:57:091
3907992018,1cyclictest21102-21sh19:45:011
3898992020,0cyclictest0-21swapper/020:37:090
3898992020,0cyclictest0-21swapper/019:37:090
3912991919,0cyclictest0-21swapper/319:57:103
3912991919,0cyclictest0-21swapper/319:15:023
3911991919,0cyclictest0-21swapper/220:12:092
391199190,0cyclictest0-21swapper/219:12:062
390799192,11cyclictest793-21lldpd19:17:111
3907991919,0cyclictest0-21swapper/121:07:111
3907991919,0cyclictest0-21swapper/121:07:111
3907991919,0cyclictest0-21swapper/120:07:071
3907991917,1cyclictest4270-21apt-get19:12:061
3898991919,0cyclictest0-21swapper/021:02:100
3898991919,0cyclictest0-21swapper/020:02:110
3898991919,0cyclictest0-21swapper/019:22:090
3912991818,0cyclictest0-21swapper/320:22:103
3912991818,0cyclictest0-21swapper/320:17:093
3911991818,0cyclictest0-21swapper/220:57:092
3911991818,0cyclictest0-21swapper/219:37:092
3911991816,1cyclictest10319-21munin-run20:30:012
3907991818,0cyclictest10737-21apt-get20:32:111
3907991817,1cyclictest25203-21apt-get21:02:091
3907991816,1cyclictest5945-21apt-get20:22:101
3898991818,0cyclictest0-21swapper/020:42:100
3898991818,0cyclictest0-21swapper/019:32:100
389899180,17cyclictest0-21swapper/020:47:070
3912991715,1cyclictest21533-21apt-get19:47:093
3911991717,0cyclictest0-21swapper/219:57:092
3911991716,1cyclictest19138-21apt-get19:42:072
3907991716,1cyclictest0-21swapper/120:27:111
3898991717,0cyclictest0-21swapper/021:07:110
3898991717,0cyclictest0-21swapper/021:07:110
3898991717,0cyclictest0-21swapper/019:57:100
3912991616,0cyclictest0-21swapper/320:57:093
3912991614,1cyclictest606-21dbus-daemon19:25:003
391299160,15cyclictest6503-21apt-config19:15:003
3911991614,1cyclictest5532-21munin-run20:20:002
3907991616,0cyclictest0-21swapper/120:47:071
3907991616,0cyclictest0-21swapper/119:22:091
3898991616,0cyclictest0-21swapper/020:07:070
3898991616,0cyclictest0-21swapper/019:42:070
3912991514,1cyclictest28746-21apt-get20:02:113
391299150,1cyclictest9141-21irqstats20:27:143
391199150,14cyclictest25889-21sh19:55:002
3907991515,0cyclictest0-21swapper/119:47:091
3898991515,0cyclictest0-21swapper/019:47:090
3898991513,1cyclictest0-21swapper/020:21:150
391299140,14cyclictest0-21swapper/321:07:073
391299140,14cyclictest0-21swapper/321:07:073
391299140,14cyclictest0-21swapper/320:11:593
391299140,14cyclictest0-21swapper/319:50:203
391299140,0cyclictest0-21swapper/320:40:393
3911991414,0cyclictest0-21swapper/221:00:322
391199140,14cyclictest0-21swapper/220:40:292
391199140,14cyclictest0-21swapper/220:00:122
3907991414,0cyclictest0-21swapper/119:50:161
3907991414,0cyclictest0-21swapper/119:35:401
390799140,0cyclictest0-21swapper/120:10:301
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional