You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 13:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Nov 22, 2025 00:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2868899262,23cyclictest19671-21apt-get22:12:070
28690992523,1cyclictest31077-21perf19:15:002
2868999250,1cyclictest24101-21cron22:20:001
2868899250,24cyclictest593-21dbus-daemon00:25:000
2869199240,23cyclictest16601-21apt-get23:12:093
2869199240,1cyclictest0-21swapper/321:00:013
28688992420,3cyclictest27258-21idleruntime-cro21:20:000
2869199230,22cyclictest13507-21apt-get19:47:093
2869099231,21cyclictest11175-21apt-get00:07:072
2869099231,21cyclictest0-21swapper/221:57:092
2869099230,22cyclictest27195-21cron21:20:002
2869099230,22cyclictest14773-21apt-get20:55:022
2868999230,22cyclictest0-21swapper/119:42:061
28688992323,0cyclictest0-21swapper/019:47:090
28691992220,1cyclictest612-21gdbus21:40:003
2869199220,21cyclictest20292-21idleruntime-cro20:00:003
2869199220,21cyclictest20292-21idleruntime-cro20:00:003
2869199220,1cyclictest32022-21cron21:30:003
2869199220,1cyclictest29746-21apt-get22:32:093
28690992221,1cyclictest11809-21apt-get23:02:082
28690992220,2cyclictest596-21in:imuxsock21:05:012
28690992220,1cyclictest5168-21perf20:35:002
28690992220,1cyclictest0-21swapper/219:32:072
2869099220,21cyclictest32058-21awk21:30:002
2869099220,21cyclictest25572-21apt-get00:37:092
2869099220,21cyclictest0-21swapper/222:40:012
2869099220,21cyclictest0-21swapper/221:52:092
2869099220,1cyclictest2045-21grep21:35:002
28689992220,1cyclictest313-21systemd-journal23:04:591
28689992219,3cyclictest4459-21perf21:40:001
2868999220,21cyclictest0-21swapper/123:12:091
2868999220,21cyclictest0-21swapper/119:47:091
2868999220,1cyclictest0-21swapper/120:57:061
2868999220,1cyclictest0-21swapper/100:40:001
2868999220,0cyclictest8385-21apt-get00:02:081
28688992221,1cyclictest0-21swapper/023:07:090
28688992220,1cyclictest17977-21awk00:20:010
28688992220,1cyclictest16863-21latency_hist22:05:000
28688992220,1cyclictest10358-21perf00:05:000
2868899220,21cyclictest27965-21apt-get20:17:070
2868899220,21cyclictest17572-21perf21:00:010
2868899220,21cyclictest10387-21apt-get20:47:060
2868899220,1cyclictest646-21perf23:44:590
2868899220,1cyclictest5523-21unixbench_multi19:27:150
2868899220,1cyclictest12056-21munin-plugin-st21:55:000
2868899220,1cyclictest0-21swapper/023:02:080
28691992121,0cyclictest0-21swapper/321:12:063
28691992120,1cyclictest3480-21apt-get23:52:073
28691992120,1cyclictest14775-21perf20:55:003
28691992120,0cyclictest0-21swapper/323:27:073
28691992120,0cyclictest0-21swapper/300:37:093
28691992119,1cyclictest640-21grep23:45:003
2869199210,20cyclictest0-21swapper/323:57:103
28690992119,1cyclictest29634-21sh21:25:002
28690992119,1cyclictest0-21swapper/200:15:012
2869099210,21cyclictest0-21swapper/219:37:062
2869099210,20cyclictest2455-21apt-get21:37:082
2869099210,20cyclictest0-21swapper/222:27:102
2869099210,20cyclictest0-21swapper/200:02:072
2869099210,0cyclictest24545-21apt-get22:22:092
2869099210,0cyclictest24545-21apt-get22:22:092
2869099210,0cyclictest0-21swapper/200:17:092
28689992121,0cyclictest30394-21apt-get20:22:081
28689992121,0cyclictest0-21swapper/122:27:101
28689992120,1cyclictest9644-21idleruntime-cro21:50:001
28689992120,1cyclictest22843-21apt-get21:12:051
28689992119,1cyclictest3063-21grep23:49:591
2868999210,20cyclictest0-21swapper/123:27:081
2868999210,20cyclictest0-21swapper/120:52:101
2868999210,20cyclictest0-21swapper/100:07:081
28688992121,0cyclictest0-21swapper/022:57:080
28688992121,0cyclictest0-21swapper/022:32:080
28688992121,0cyclictest0-21swapper/020:32:100
28688992120,1cyclictest2198-21apt-get22:42:060
28688992120,1cyclictest21473-21apt-get23:22:110
28688992120,1cyclictest0-21swapper/021:02:070
28688992120,0cyclictest5881-21apt-get23:57:110
28688992119,1cyclictest29327-21munin-run22:30:000
2868899210,21cyclictest0-21swapper/019:52:080
2868899210,20cyclictest23866-21apt-get23:27:080
2868899210,0cyclictest27661-21apt-get21:22:070
2868899210,0cyclictest20779-21apt-get00:27:060
2868899210,0cyclictest0-21swapper/021:12:060
28691992020,0cyclictest27338-21apt-get22:27:103
28691992020,0cyclictest0-21swapper/323:07:093
28691992020,0cyclictest0-21swapper/322:47:073
28691992020,0cyclictest0-21swapper/322:12:073
28691992020,0cyclictest0-21swapper/321:07:103
28691992020,0cyclictest0-21swapper/320:37:083
28691992020,0cyclictest0-21swapper/320:07:063
28691992020,0cyclictest0-21swapper/319:32:063
28691992019,0cyclictest0-21swapper/300:32:083
28691992017,2cyclictest313-21systemd-journal22:19:593
2869199200,20cyclictest0-21swapper/323:02:083
2869199200,20cyclictest0-21swapper/322:52:073
2869199200,0cyclictest3898-21apt-get19:27:053
2869199200,0cyclictest20722-21apt-get20:02:093
2869199200,0cyclictest17279-21apt-get22:07:103
2869199200,0cyclictest0-21swapper/321:40:023
28690992020,0cyclictest7988-21apt-get20:42:062
28690992020,0cyclictest0-21swapper/223:27:082
28690992020,0cyclictest0-21swapper/223:22:112
28690992020,0cyclictest0-21swapper/222:42:082
28690992020,0cyclictest0-21swapper/222:02:082
28690992020,0cyclictest0-21swapper/220:22:082
28690992020,0cyclictest0-21swapper/220:02:092
28690992020,0cyclictest0-21swapper/200:27:062
28690992019,1cyclictest14206-21apt-get23:07:092
28690992019,0cyclictest0-21swapper/220:25:012
28690992019,0cyclictest0-21swapper/219:27:062
2869099200,0cyclictest0-21swapper/223:57:102
2869099200,0cyclictest0-21swapper/220:37:082
28689992020,0cyclictest6992-21apt-get22:52:071
28689992020,0cyclictest0-21swapper/123:42:131
28689992020,0cyclictest0-21swapper/121:57:091
28689992020,0cyclictest0-21swapper/120:27:071
28689992019,0cyclictest25573-21apt-get20:12:121
28689992017,2cyclictest10667-21cron19:40:001
2868999200,0cyclictest0-21swapper/120:42:061
2868999200,0cyclictest0-21swapper/120:07:061
2868999200,0cyclictest0-21swapper/100:17:091
28688992020,0cyclictest6295-21apt-get19:32:070
28688992020,0cyclictest0-21swapper/021:37:070
28688992020,0cyclictest0-21swapper/019:37:060
28688992020,0cyclictest0-21swapper/000:32:080
28688992020,0cyclictest0-21swapper/000:07:080
28688992019,1cyclictest24118-21perf22:20:000
28688992018,1cyclictest2767-21munin-run20:30:000
28688992018,1cyclictest18306-21apt-get19:57:090
28688992018,1cyclictest18306-21apt-get19:57:080
2868899200,0cyclictest11109-21apt-get19:42:050
2868899200,0cyclictest0-21swapper/023:52:080
28691991919,0cyclictest9392-21apt-get22:57:083
28691991919,0cyclictest32189-21apt-get22:37:073
28691991919,0cyclictest31492-21apt-get19:17:073
28691991919,0cyclictest25247-21apt-get21:17:083
28691991919,0cyclictest0-21swapper/323:32:103
28691991919,0cyclictest0-21swapper/323:22:103
28691991919,0cyclictest0-21swapper/321:52:093
28691991919,0cyclictest0-21swapper/321:22:063
28691991919,0cyclictest0-21swapper/321:02:083
28691991919,0cyclictest0-21swapper/320:42:053
28691991918,0cyclictest28667-21apt-get23:37:083
28690991919,0cyclictest0-21swapper/223:12:082
28690991919,0cyclictest0-21swapper/222:32:092
2869099190,19cyclictest0-21swapper/222:52:072
2869099190,18cyclictest0-21swapper/223:52:082
28689991919,0cyclictest17279-21apt-get22:07:101
28689991919,0cyclictest10065-21apt-get21:52:081
28689991919,0cyclictest0-21swapper/122:37:081
28689991919,0cyclictest0-21swapper/120:47:071
28689991919,0cyclictest0-21swapper/119:57:091
28689991919,0cyclictest0-21swapper/119:57:081
28689991919,0cyclictest0-21swapper/119:27:061
28689991919,0cyclictest0-21swapper/100:27:061
28688991919,0cyclictest20393-21apt-get21:07:100
28688991919,0cyclictest0-21swapper/020:42:050
28688991918,1cyclictest19002-21apt-get23:17:110
28688991917,1cyclictest5574-21apt-get20:37:080
28691991818,0cyclictest0-21swapper/322:42:083
28691991818,0cyclictest0-21swapper/322:22:083
28691991818,0cyclictest0-21swapper/322:22:073
28691991818,0cyclictest0-21swapper/320:47:073
28691991818,0cyclictest0-21swapper/320:32:093
28691991818,0cyclictest0-21swapper/320:22:073
28691991818,0cyclictest0-21swapper/319:22:063
28691991818,0cyclictest0-21swapper/300:12:113
28691991817,1cyclictest29962-21awk20:20:003
28690991818,0cyclictest0-21swapper/219:47:092
28690991817,0cyclictest23169-21apt-get20:07:062
28690991816,1cyclictest598-21rs:main2
28690991816,1cyclictest598-21rs:main2
2869099180,8cyclictest0-21swapper/220:52:102
28689991818,0cyclictest0-21swapper/122:57:071
28689991818,0cyclictest0-21swapper/122:42:081
28689991818,0cyclictest0-21swapper/122:02:091
28689991818,0cyclictest0-21swapper/121:42:081
28689991818,0cyclictest0-21swapper/121:17:091
28689991818,0cyclictest0-21swapper/120:37:081
28689991818,0cyclictest0-21swapper/119:12:071
28689991817,1cyclictest17994-21apt-get21:02:071
28689991816,1cyclictest0-21swapper/119:25:011
28689991815,2cyclictest8290-21awk19:35:001
28688991817,0cyclictest1486-21apt-get19:22:060
28691991717,0cyclictest0-21swapper/300:17:083
28691991716,0cyclictest18383-21apt-get00:22:063
28690991717,0cyclictest0-21swapper/222:07:092
28690991717,0cyclictest0-21swapper/221:12:062
28690991717,0cyclictest0-21swapper/220:12:102
28690991717,0cyclictest0-21swapper/219:42:052
28690991716,1cyclictest31096-21apt-get23:42:132
28690991716,0cyclictest4598-21apt-get22:47:082
28690991715,1cyclictest20393-21apt-get21:07:092
28690991715,1cyclictest11379-21latency_hist23:00:002
28689991717,0cyclictest0-21swapper/122:12:071
28689991717,0cyclictest0-21swapper/119:17:071
28689991717,0cyclictest0-21swapper/100:22:071
28689991717,0cyclictest0-21swapper/100:12:111
28689991716,1cyclictest0-21swapper/123:37:081
28689991715,1cyclictest0-21swapper/123:54:591
28689991711,5cyclictest20627-21unixbench_multi23:17:201
28688991717,0cyclictest0-21swapper/022:47:070
28688991717,0cyclictest0-21swapper/022:22:080
28688991717,0cyclictest0-21swapper/022:22:080
28688991717,0cyclictest0-21swapper/021:47:100
28688991717,0cyclictest0-21swapper/021:34:590
28691991616,0cyclictest0-21swapper/321:32:083
28691991616,0cyclictest0-21swapper/320:27:063
28691991616,0cyclictest0-21swapper/319:42:063
2869199160,15cyclictest20948-21cron23:20:003
28690991616,0cyclictest0-21swapper/200:32:092
28689991616,0cyclictest0-21swapper/123:07:091
28689991616,0cyclictest0-21swapper/121:27:081
28689991616,0cyclictest0-21swapper/120:32:101
28689991614,1cyclictest22442-21sendmail-mta21:10:001
28689991614,1cyclictest0-21swapper/123:22:111
28688991616,0cyclictest0-21swapper/023:47:070
28688991616,0cyclictest0-21swapper/023:37:070
28688991616,0cyclictest0-21swapper/022:52:070
28688991616,0cyclictest0-21swapper/022:37:070
28688991616,0cyclictest0-21swapper/022:07:100
28688991616,0cyclictest0-21swapper/020:22:080
28688991616,0cyclictest0-21swapper/020:12:110
28691991515,0cyclictest0-21swapper/300:07:093
2869099155,10cyclictest3007-21munin-node19:17:022
28690991515,0cyclictest0-21swapper/222:12:062
2869099151,14cyclictest1068-21apt-get23:47:082
2869099150,1cyclictest28249-21perf23:35:002
2869099150,1cyclictest0-21swapper/220:47:142
2869099150,13cyclictest781-21lldpd23:39:402
2869099150,13cyclictest0-21swapper/219:52:092
28689991515,0cyclictest0-21swapper/123:32:101
2868999150,14cyclictest0-21swapper/122:45:351
28688991515,0cyclictest0-21swapper/021:57:090
28688991515,0cyclictest0-21swapper/020:07:060
28688991513,1cyclictest0-21swapper/000:37:080
28691991414,0cyclictest0-21swapper/300:02:083
2869199140,14cyclictest0-21swapper/323:45:373
2869199140,14cyclictest0-21swapper/322:03:013
2869199140,14cyclictest0-21swapper/321:55:063
2869199140,14cyclictest0-21swapper/321:45:473
2869199140,14cyclictest0-21swapper/320:11:223
2869199140,14cyclictest0-21swapper/319:50:333
2869199140,14cyclictest0-21swapper/319:36:003
2869199140,14cyclictest0-21swapper/319:11:073
2869199140,14cyclictest0-21swapper/300:25:213
28690991414,0cyclictest0-21swapper/222:17:072
2869099140,14cyclictest0-21swapper/223:19:542
2869099140,14cyclictest0-21swapper/221:45:542
2869099140,14cyclictest0-21swapper/219:20:122
2869099140,14cyclictest0-21swapper/200:24:442
2869099140,0cyclictest28876-21memory20:17:112
28689991414,0cyclictest0-21swapper/122:33:461
28689991414,0cyclictest0-21swapper/121:23:361
2868999140,14cyclictest0-21swapper/120:15:061
2868999140,14cyclictest0-21swapper/120:01:091
2868999140,14cyclictest0-21swapper/119:50:171
2868999140,14cyclictest0-21swapper/100:32:271
2868999140,0cyclictest0-21swapper/121:31:591
28688991414,0cyclictest0-21swapper/023:32:150
28688991414,0cyclictest0-21swapper/023:12:020
28688991414,0cyclictest0-21swapper/019:15:200
28688991414,0cyclictest0-21swapper/000:11:020
2868899140,14cyclictest0-21swapper/021:40:520
2868899140,0cyclictest0-21swapper/021:25:380
2868899140,0cyclictest0-21swapper/020:52:130
2868899140,0cyclictest0-21swapper/020:01:410
2868899140,0cyclictest0-21swapper/019:10:400
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional