You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-26 - 05:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue May 26, 2026 00:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23795992624,1cyclictest14468-21apt-get22:33:031
23804992523,1cyclictest0-21swapper/300:22:383
2380499250,1cyclictest0-21swapper/323:15:563
2379799250,24cyclictest18858-21apt-get00:03:052
2379799250,24cyclictest18217-21diskmemload23:20:532
2379799250,24cyclictest18217-21diskmemload21:52:132
2379799250,24cyclictest0-21swapper/223:11:212
2379599250,24cyclictest8343-21apt-get20:53:051
2379599250,24cyclictest32359-21latency_hist22:20:001
2379599250,24cyclictest2166-21ssh21:53:111
2379599250,24cyclictest0-21swapper/123:09:131
2379599250,1cyclictest2658-21chrt21:24:591
23791992523,1cyclictest27018-21cat23:13:050
2379199250,24cyclictest24242-21apt-get19:13:050
2379199250,24cyclictest18217-21diskmemload21:19:430
23804992423,1cyclictest0-21swapper/300:07:083
23804992422,1cyclictest5894-21cron22:25:003
2380499240,23cyclictest21089-21apt-get20:13:053
23797992423,1cyclictest18217-21diskmemload23:55:512
23797992423,1cyclictest0-21swapper/222:37:122
23797992419,4cyclictest0-21swapper/221:42:212
2379799240,23cyclictest18217-21diskmemload21:21:172
2379799240,23cyclictest0-21swapper/223:06:272
2379799240,23cyclictest0-21swapper/200:32:022
2379799240,0cyclictest0-21swapper/223:52:162
23795992424,0cyclictest0-21swapper/123:26:251
2379599240,23cyclictest12099-21awk00:25:011
2379599240,0cyclictest18217-21diskmemload00:04:561
2379599240,0cyclictest0-21swapper/122:58:391
2379599240,0cyclictest0-21swapper/100:12:591
23791992424,0cyclictest18217-21diskmemload21:23:310
23791992424,0cyclictest0-21swapper/023:03:220
23791992423,1cyclictest62550irq/127-eno100:32:170
23791992423,1cyclictest11035-21ssh23:55:070
23791992423,1cyclictest0-21swapper/023:44:010
23791992420,3cyclictest24426-21cron23:10:000
2379199240,24cyclictest0-21swapper/022:33:410
2379199240,24cyclictest0-21swapper/021:52:250
2379199240,24cyclictest0-21swapper/021:40:580
2379199240,23cyclictest313-21systemd-journal21:35:000
2379199240,0cyclictest0-21swapper/021:26:110
23804992323,0cyclictest18217-21diskmemload23:47:363
23804992323,0cyclictest18217-21diskmemload22:14:353
23804992323,0cyclictest0-21swapper/322:59:153
23804992323,0cyclictest0-21swapper/300:15:573
23804992321,1cyclictest0-21swapper/323:32:343
23804992321,1cyclictest0-21swapper/321:45:023
2380499231,21cyclictest0-21swapper/321:13:153
2380499230,23cyclictest0-21swapper/322:19:053
2380499230,23cyclictest0-21swapper/322:07:383
2380499230,22cyclictest31432-21munin-run23:45:013
2380499230,22cyclictest25788-21turbostat20:20:003
2380499230,22cyclictest23554-21ssh22:40:093
2380499230,22cyclictest18217-21diskmemload22:31:053
2380499230,22cyclictest18217-21diskmemload21:51:533
2380499230,22cyclictest18217-21diskmemload21:23:123
2380499230,22cyclictest18217-21diskmemload00:27:153
2380499230,22cyclictest18217-21diskmemload00:10:483
2380499230,22cyclictest0-21swapper/300:33:503
2380499230,0cyclictest18217-21diskmemload23:55:483
2380499230,0cyclictest18217-21diskmemload23:06:383
2380499230,0cyclictest18217-21diskmemload23:03:213
2380499230,0cyclictest0-21swapper/323:11:423
2380499230,0cyclictest0-21swapper/322:38:563
2380499230,0cyclictest0-21swapper/321:15:483
2380499230,0cyclictest0-21swapper/300:35:223
23797992323,0cyclictest0-21swapper/223:45:252
23797992323,0cyclictest0-21swapper/222:05:382
23797992321,2cyclictest28071-21sh23:42:492
23797992321,1cyclictest0-21swapper/223:36:212
23797992321,1cyclictest0-21swapper/223:31:272
23797992321,1cyclictest0-21swapper/221:16:532
23797992320,2cyclictest13235-21perf23:00:012
23797992319,0cyclictest0-21swapper/222:30:042
2379799230,3cyclictest0-21swapper/221:10:562
2379799230,2cyclictest18217-21diskmemload22:00:012
2379799230,22cyclictest17792-21perf00:30:012
2379799230,22cyclictest0-21swapper/221:36:512
2379799230,1cyclictest0-21swapper/221:31:032
2379799230,0cyclictest0-21swapper/223:04:252
2379799230,0cyclictest0-21swapper/222:29:442
2379799230,0cyclictest0-21swapper/200:37:472
23795992323,0cyclictest0-21swapper/123:00:171
23795992323,0cyclictest0-21swapper/121:30:521
23795992322,1cyclictest18217-21diskmemload00:27:491
23795992321,1cyclictest0-21swapper/100:15:541
2379599230,2cyclictest22903-21sh21:13:491
2379599230,2cyclictest20197-21munin-plugin-st23:35:011
2379599230,2cyclictest1885-21ssh23:48:001
2379599230,23cyclictest18217-21diskmemload22:38:331
2379599230,23cyclictest0-21swapper/122:50:391
2379599230,23cyclictest0-21swapper/100:30:141
2379599230,22cyclictest5645-21sh23:50:331
2379599230,22cyclictest30662-21perf20:30:011
2379599230,22cyclictest26754-21ssh22:15:011
2379599230,22cyclictest26298-21ssh21:46:031
2379599230,22cyclictest26207-21munin-run19:15:001
2379599230,22cyclictest22681-21sh00:05:371
2379599230,22cyclictest13368-21cron19:55:011
2379599230,22cyclictest1084-21sh22:20:381
2379599230,22cyclictest0-21swapper/121:15:101
2379599230,0cyclictest18217-21diskmemload22:26:001
2379599230,0cyclictest0-21swapper/123:38:551
2379599230,0cyclictest0-21swapper/123:18:111
23791992323,0cyclictest18217-21diskmemload21:36:070
23791992323,0cyclictest0-21swapper/022:57:470
23791992323,0cyclictest0-21swapper/022:39:490
23791992323,0cyclictest0-21swapper/022:26:190
23791992323,0cyclictest0-21swapper/022:16:120
23791992322,1cyclictest0-21swapper/000:14:110
23791992320,2cyclictest15070-21ssh23:30:290
2379199230,23cyclictest0-21swapper/023:50:430
2379199230,23cyclictest0-21swapper/023:24:250
2379199230,22cyclictest9838-21idleruntime-cro22:00:010
2379199230,22cyclictest26613-21ssh00:08:400
2379199230,22cyclictest0-21swapper/023:18:050
2379199230,22cyclictest0-21swapper/022:00:100
2379199230,22cyclictest0-21swapper/000:38:050
2379199230,21cyclictest27695-21kworker/u8:222:20:280
2379199230,0cyclictest18217-21diskmemload22:13:410
2379199230,0cyclictest18217-21diskmemload22:05:330
2379199230,0cyclictest0-21swapper/021:10:260
23804992221,1cyclictest0-21swapper/321:42:003
23804992221,1cyclictest0-21swapper/321:00:023
23804992220,1cyclictest0-21swapper/323:20:013
23804992220,1cyclictest0-21swapper/320:05:003
2380499220,22cyclictest0-21swapper/321:30:323
2380499220,21cyclictest606-21dbus-daemon19:50:013
2380499220,21cyclictest18217-21diskmemload23:35:063
2380499220,21cyclictest15012-21sh21:36:053
2380499220,21cyclictest1475-21apt-get19:33:023
2380499220,21cyclictest0-21swapper/323:25:243
2380499220,21cyclictest0-21swapper/322:29:043
2380499220,21cyclictest0-21swapper/320:33:043
2380499220,1cyclictest29157-21sh22:45:063
2380499220,1cyclictest0-21swapper/321:26:393
2380499220,0cyclictest18217-21diskmemload22:50:473
2380499220,0cyclictest0-21swapper/300:00:573
23797992222,0cyclictest0-21swapper/222:00:322
23797992222,0cyclictest0-21swapper/200:17:502
23797992221,1cyclictest0-21swapper/222:12:592
23797992220,1cyclictest29111-21apt-get19:23:052
23797992220,1cyclictest18217-21diskmemload23:25:412
23797992220,1cyclictest0-21swapper/222:51:182
23797992220,1cyclictest0-21swapper/221:28:052
23797992220,1cyclictest0-21swapper/200:20:222
2379799220,21cyclictest23907-21rm22:40:342
2379799220,21cyclictest15758-21cron20:00:002
2379799220,21cyclictest0-21swapper/222:48:122
2379799220,21cyclictest0-21swapper/222:20:172
2379799220,21cyclictest0-21swapper/222:15:082
2379799220,21cyclictest0-21swapper/221:45:522
2379799220,21cyclictest0-21swapper/200:07:122
2379799220,1cyclictest18217-21diskmemload23:16:442
2379799220,1cyclictest0-21swapper/200:14:532
23795992222,0cyclictest18217-21diskmemload23:12:121
23795992222,0cyclictest18217-21diskmemload22:48:461
23795992222,0cyclictest0-21swapper/123:55:071
23795992222,0cyclictest0-21swapper/121:38:041
23795992221,1cyclictest0-21swapper/123:25:011
23795992220,1cyclictest606-21dbus-daemon19:45:011
2379599220,21cyclictest18217-21diskmemload22:40:041
2379599220,21cyclictest18217-21diskmemload22:05:361
2379599220,21cyclictest1118-21apt-get20:38:021
2379599220,21cyclictest0-21swapper/123:41:351
2379599220,21cyclictest0-21swapper/120:15:021
2379599220,21cyclictest0-21swapper/119:33:021
2379599220,1cyclictest10242-21sh22:00:161
2379599220,0cyclictest0-21swapper/121:56:361
23791992222,0cyclictest0-21swapper/023:38:060
23791992222,0cyclictest0-21swapper/023:25:450
23791992222,0cyclictest0-21swapper/022:50:580
23791992222,0cyclictest0-21swapper/019:18:030
23791992220,1cyclictest30640-21idleruntime-cro20:30:010
2379199220,22cyclictest0-21swapper/000:15:190
2379199220,21cyclictest313-21systemd-journal20:25:010
2379199220,21cyclictest22984-21cron20:15:000
2379199220,21cyclictest18217-21diskmemload00:25:370
2379199220,1cyclictest6876-21ssh00:20:200
2379199220,1cyclictest17097-21ssh00:00:420
2379199220,1cyclictest0-21swapper/021:46:570
2379199220,0cyclictest0-21swapper/023:45:020
23804992121,0cyclictest0-21swapper/319:53:033
23804992119,1cyclictest5417-21sh23:50:153
23804992118,2cyclictest26188-21turbostat.cron19:15:003
2380499210,20cyclictest0-21swapper/319:28:033
23797992120,1cyclictest7842-21latency_hist20:50:002
23797992119,1cyclictest3402-21perf19:35:012
23797992119,1cyclictest31500-21apt-get19:28:032
2379799210,20cyclictest26690-21apt-get19:18:032
2379799210,20cyclictest0-21swapper/220:55:022
2379799210,20cyclictest0-21swapper/220:38:032
2379799210,20cyclictest0-21swapper/220:18:042
2379799210,20cyclictest0-21swapper/219:13:052
2379799210,0cyclictest0-21swapper/220:03:042
23795992121,0cyclictest0-21swapper/120:33:041
2379599210,1cyclictest20024-21ssh21:40:091
2379199210,21cyclictest0-21swapper/020:38:030
2379199210,20cyclictest15838-21dump-pmu-power21:05:000
2379199210,1cyclictest6191-21awk19:40:010
2379199210,1cyclictest29192-21ssh22:45:090
2379199210,0cyclictest0-21swapper/022:40:570
2379199210,0cyclictest0-21swapper/020:53:040
23804992020,0cyclictest0-21swapper/320:53:053
23804992019,0cyclictest0-21swapper/320:43:043
2380499200,20cyclictest0-21swapper/319:38:033
2380499200,0cyclictest0-21swapper/320:28:043
23797992020,0cyclictest4277-21apt-get19:38:022
23795992020,0cyclictest5946-21apt-get20:48:051
23795992018,1cyclictest15855-21munin-run21:05:001
23791992020,0cyclictest3550-21apt-get20:43:040
23791992020,0cyclictest0-21swapper/020:48:050
23791992020,0cyclictest0-21swapper/019:58:050
2379199200,0cyclictest0-21swapper/019:53:040
2379199200,0cyclictest0-21swapper/019:28:030
23804991917,1cyclictest0-21swapper/320:38:033
23797991919,0cyclictest0-21swapper/220:13:052
23797991918,0cyclictest18694-21apt-get20:08:042
23795991919,0cyclictest0-21swapper/120:43:051
23795991917,1cyclictest0-21swapper/119:30:001
23791991919,0cyclictest0-21swapper/019:43:040
2379199190,18cyclictest0-21swapper/020:08:040
23804991818,0cyclictest0-21swapper/321:08:033
23804991818,0cyclictest0-21swapper/319:23:053
23795991818,0cyclictest16334-21apt-get21:08:031
23795991818,0cyclictest0-21swapper/119:38:021
23791991818,0cyclictest0-21swapper/019:48:040
23791991816,1cyclictest0-21swapper/019:25:000
23804991717,0cyclictest11132-21apt-get20:58:043
23797991717,0cyclictest0-21swapper/221:08:032
23797991717,0cyclictest0-21swapper/221:03:042
23791991717,0cyclictest0-21swapper/020:33:040
23804991616,0cyclictest0-21swapper/319:18:033
23804991614,1cyclictest20310-21sed20:08:133
23804991614,1cyclictest15787-21grep20:00:003
2379799160,1cyclictest10265-21ls20:55:002
2379799160,15cyclictest11906-21df_inode19:53:052
2379799160,15cyclictest0-21swapper/219:46:362
23795991616,0cyclictest0-21swapper/120:58:041
2379599160,15cyclictest31068-21munin-run19:25:001
2379599160,15cyclictest10981-21cron19:50:011
23791991616,0cyclictest0-21swapper/020:58:050
23791991614,1cyclictest0-21swapper/019:35:010
23804991515,0cyclictest0-21swapper/319:43:033
23797991515,0cyclictest0-21swapper/220:28:042
2379599150,1cyclictest27679-21mailstats20:23:141
23791991513,1cyclictest313-21systemd-journal20:20:000
23804991414,0cyclictest0-21swapper/320:46:223
2380499140,0cyclictest0-21swapper/320:21:043
23797991412,1cyclictest0-21swapper/219:40:242
2379799140,14cyclictest0-21swapper/220:22:332
2379799140,0cyclictest0-21swapper/220:40:162
2379799140,0cyclictest0-21swapper/220:32:462
23795991414,0cyclictest21089-21apt-get20:13:051
23795991414,0cyclictest0-21swapper/120:08:041
2379599140,14cyclictest0-21swapper/120:01:421
23791991414,0cyclictest0-21swapper/021:07:520
2379199140,14cyclictest0-21swapper/020:00:100
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional