You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 06:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Sun Nov 23, 2025 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16150992422,1cyclictest8590-21apt-get00:27:153
1614899240,23cyclictest15895-21basename21:20:001
16150992320,2cyclictest13080-21perf21:15:003
1615099230,23cyclictest0-21swapper/320:42:143
1615099230,22cyclictest13413-21awk20:10:013
1615099230,0cyclictest18185-21apt-get20:20:003
1615099230,0cyclictest15184-21apt-get22:27:143
16149992321,1cyclictest0-21swapper/220:19:592
1614999230,22cyclictest3738-21perf19:50:002
1614999230,22cyclictest0-21swapper/222:22:142
1614999230,0cyclictest0-21swapper/220:02:152
16148992321,1cyclictest0-21swapper/123:50:001
1614899230,2cyclictest313-21systemd-journal23:25:001
16147992320,2cyclictest24391-21awk22:45:000
1614799230,2cyclictest23413-21wc20:30:000
1614799230,22cyclictest313-21systemd-journal22:05:000
1614799230,22cyclictest29164-21apt22:55:000
16150992220,1cyclictest20945-21apt-get19:20:013
16150992220,1cyclictest0-21swapper/323:00:003
1615099220,21cyclictest7832-21awk21:05:003
1615099220,21cyclictest4396-21apt-get23:12:143
1615099220,21cyclictest28538-21dump-pmu-power19:34:593
1615099220,21cyclictest0-21swapper/321:42:133
1615099220,21cyclictest0-21swapper/321:27:133
1615099220,1cyclictest7480-21sh22:10:003
16149992221,0cyclictest21681-21apt-get23:47:142
16149992220,1cyclictest21440-21apt-get20:27:152
16149992220,1cyclictest0-21swapper/219:25:002
1614999220,0cyclictest0-21swapper/200:37:132
16148992222,0cyclictest0-21swapper/119:27:131
16148992220,1cyclictest0-21swapper/120:50:001
1614899220,21cyclictest30718-21apt-get21:52:131
1614899220,21cyclictest26479-21apt-get23:57:151
1614899220,21cyclictest26042-21dump-pmu-power23:55:001
1614899220,21cyclictest1361-21apt-get00:12:141
1614899220,21cyclictest0-21swapper/121:47:151
16147992221,1cyclictest18956-21apt-get19:17:130
16147992221,0cyclictest0-21swapper/000:02:130
16147992220,1cyclictest593-21dbus-daemon23:20:000
1614799220,21cyclictest766-21snmpd22:40:000
1614799220,21cyclictest28961-21apt-get19:37:120
1614799220,21cyclictest17155-21cron22:29:590
1614799220,21cyclictest0-21swapper/023:32:130
1614799220,21cyclictest0-21swapper/021:07:150
1614799220,21cyclictest0-21swapper/020:17:130
16150992121,0cyclictest8302-21apt-get21:07:153
16150992121,0cyclictest3774-21apt-get00:17:153
16150992121,0cyclictest26133-21perf19:29:593
16150992121,0cyclictest24076-21apt-get23:52:133
16150992121,0cyclictest0-21swapper/300:22:123
16150992120,1cyclictest6795-21apt-get23:17:123
16150992120,0cyclictest10987-21apt-get00:32:163
16150992120,0cyclictest0-21swapper/319:42:153
16150992119,1cyclictest15349-21cron00:40:003
1615099210,20cyclictest0-21swapper/323:02:143
1615099210,20cyclictest0-21swapper/320:57:163
1615099210,20cyclictest0-21swapper/320:52:153
1615099210,20cyclictest0-21swapper/320:02:143
1615099210,1cyclictest22405-21apt-get22:42:133
1615099210,0cyclictest21110-21apt-get21:32:133
1615099210,0cyclictest0-21swapper/300:12:143
16149992121,0cyclictest0-21swapper/222:07:162
16149992119,1cyclictest24372-21perf22:45:002
16149992119,1cyclictest23098-21awk21:35:002
1614999210,21cyclictest0-21swapper/200:32:162
1614999210,20cyclictest4157-21apt-get19:52:132
1614999210,20cyclictest3438-21apt-get20:57:162
1614999210,20cyclictest0-21swapper/223:17:142
1614999210,20cyclictest0-21swapper/223:12:142
1614999210,20cyclictest0-21swapper/222:32:132
1614999210,20cyclictest0-21swapper/222:27:142
1614999210,20cyclictest0-21swapper/200:12:132
1614999210,0cyclictest0-21swapper/220:37:142
16148992121,0cyclictest0-21swapper/123:07:151
16148992121,0cyclictest0-21swapper/119:42:151
16148992121,0cyclictest0-21swapper/119:37:121
16148992120,1cyclictest677-21apt-get21:57:141
16148992120,1cyclictest0-21swapper/122:42:131
16148992119,1cyclictest31594-21munin-run23:00:001
16148992119,1cyclictest0-21swapper/120:54:591
1614899210,20cyclictest0-21swapper/119:22:131
1614899210,20cyclictest0-21swapper/100:22:131
1614899210,0cyclictest0-21swapper/120:17:131
1614899210,0cyclictest0-21swapper/119:57:141
16147992120,0cyclictest13377-21apt-get00:37:120
16147992119,1cyclictest23086-21perf21:35:000
16147992119,1cyclictest14729-21cron22:25:000
1614799210,21cyclictest0-21swapper/023:42:130
1614799210,21cyclictest0-21swapper/020:22:130
1614799210,20cyclictest0-21swapper/021:47:140
1614799210,20cyclictest0-21swapper/020:37:150
1614799210,0cyclictest24167-21apt-get19:27:130
16150992020,0cyclictest0-21swapper/321:17:143
16150992020,0cyclictest0-21swapper/319:12:153
16150992019,0cyclictest9666-21apt-get23:22:153
16150992019,0cyclictest14459-21apt-get23:32:123
16150992019,0cyclictest0-21swapper/322:52:143
16150992018,1cyclictest932-21turbostat00:10:003
16150992018,1cyclictest30308-21munin-run21:50:013
16150992018,1cyclictest12367-21munin-run22:19:593
1615099200,0cyclictest0-21swapper/320:47:153
1615099200,0cyclictest0-21swapper/320:32:153
1615099200,0cyclictest0-21swapper/319:47:133
16149992020,0cyclictest11098-21apt-get21:12:132
16149992020,0cyclictest0-21swapper/223:52:132
16149992020,0cyclictest0-21swapper/223:20:022
16149992020,0cyclictest0-21swapper/223:02:132
16149992020,0cyclictest0-21swapper/222:52:142
16149992020,0cyclictest0-21swapper/222:02:112
16149992020,0cyclictest0-21swapper/221:07:152
16149992020,0cyclictest0-21swapper/220:32:152
16149992020,0cyclictest0-21swapper/219:32:142
16149992020,0cyclictest0-21swapper/200:02:132
16149992019,1cyclictest18280-21turbostat21:25:002
16149992019,0cyclictest0-21swapper/219:57:152
16149992018,1cyclictest21005-21idleruntime-cro20:25:002
1614999200,20cyclictest0-21swapper/222:12:152
1614999200,20cyclictest0-21swapper/200:22:132
1614999200,0cyclictest0-21swapper/223:57:162
1614999200,0cyclictest0-21swapper/221:52:142
1614999200,0cyclictest0-21swapper/200:27:172
16148992020,0cyclictest0-21swapper/123:02:131
16148992020,0cyclictest0-21swapper/122:37:131
16148992020,0cyclictest0-21swapper/122:22:141
16148992020,0cyclictest0-21swapper/120:57:151
16148992020,0cyclictest0-21swapper/120:27:151
16148992020,0cyclictest0-21swapper/100:17:151
16148992019,1cyclictest9030-21apt-get20:02:141
16148992019,0cyclictest31385-21apt-get00:07:131
16148992018,1cyclictest11427-21apt-get20:07:141
1614899200,1cyclictest8590-21apt-get00:27:161
1614899200,19cyclictest0-21swapper/122:27:141
1614899200,0cyclictest0-21swapper/120:32:151
16147992020,0cyclictest13063-21apt-get21:15:010
16147992020,0cyclictest0-21swapper/023:27:140
16147992020,0cyclictest0-21swapper/022:07:170
16147992020,0cyclictest0-21swapper/020:42:130
16147992019,1cyclictest0-21swapper/020:47:150
16147992019,0cyclictest0-21swapper/022:32:120
16147992018,1cyclictest593-21dbus-daemon21:25:000
16147992018,1cyclictest313-21systemd-journal00:25:010
16147992018,1cyclictest0-21swapper/019:40:000
1614799200,20cyclictest0-21swapper/019:22:130
1614799200,1cyclictest24043-21/usr/sbin/munin23:52:230
1614799200,0cyclictest0-21swapper/021:42:130
1614799200,0cyclictest0-21swapper/019:12:150
16150991919,0cyclictest23508-21apt-get21:37:143
16150991919,0cyclictest0-21swapper/322:47:143
16150991919,0cyclictest0-21swapper/300:02:143
16150991917,1cyclictest8609-21munin-run20:00:003
1615099190,18cyclictest0-21swapper/323:42:143
16149991919,0cyclictest31776-21apt-get19:42:152
16149991919,0cyclictest19987-21apt-get22:37:132
16149991919,0cyclictest0-21swapper/223:42:142
16149991919,0cyclictest0-21swapper/223:07:142
16149991919,0cyclictest0-21swapper/221:47:152
16149991919,0cyclictest0-21swapper/221:37:142
16149991919,0cyclictest0-21swapper/221:27:142
16149991919,0cyclictest0-21swapper/220:42:142
16149991919,0cyclictest0-21swapper/219:27:132
16149991918,1cyclictest0-21swapper/221:02:132
16149991917,2cyclictest31544-21sh22:59:592
16149991917,1cyclictest30924-21sh19:40:002
16149991917,1cyclictest0-21swapper/220:10:002
1614999190,0cyclictest0-21swapper/221:17:132
16148991919,0cyclictest0-21swapper/123:12:141
16148991919,0cyclictest0-21swapper/119:17:131
16148991919,0cyclictest0-21swapper/100:32:171
16148991917,1cyclictest5093-21apt22:05:001
16148991917,1cyclictest29168-21dump-pmu-power22:54:591
16148991917,1cyclictest16435-21perf23:35:001
1614899190,18cyclictest5851-21apt-get21:02:121
1614899190,18cyclictest20671-21idleruntime-cro21:30:001
16147991919,0cyclictest18712-21apt-get21:27:130
16147991919,0cyclictest0-21swapper/000:32:170
1614799190,4cyclictest0-21swapper/020:12:160
16150991818,0cyclictest0-21swapper/320:27:143
16149991818,0cyclictest24802-21apt-get22:47:132
1614999180,17cyclictest15798-21awk20:15:002
16148991818,0cyclictest0-21swapper/123:37:131
16148991818,0cyclictest0-21swapper/121:37:141
16148991818,0cyclictest0-21swapper/121:12:141
16148991817,1cyclictest22-21ksoftirqd/119:55:001
16148991817,0cyclictest10372-21apt-get22:17:151
16148991816,1cyclictest0-21swapper/120:20:011
1614899180,0cyclictest0-21swapper/119:47:131
16147991818,0cyclictest16863-21apt-get23:37:130
16147991818,0cyclictest0-21swapper/022:12:150
16147991818,0cyclictest0-21swapper/021:02:130
16147991818,0cyclictest0-21swapper/019:57:140
1614799180,17cyclictest0-21swapper/023:58:440
16150991717,0cyclictest17589-21apt-get22:32:133
16150991717,0cyclictest16229-21apt-get20:17:133
16150991717,0cyclictest0-21swapper/323:27:133
16150991717,0cyclictest0-21swapper/320:37:143
16150991717,0cyclictest0-21swapper/319:37:133
16149991717,0cyclictest0-21swapper/219:12:152
16149991716,1cyclictest12060-21apt-get23:27:142
16148991717,0cyclictest0-21swapper/123:27:141
16148991717,0cyclictest0-21swapper/121:07:141
16148991717,0cyclictest0-21swapper/120:42:141
16148991715,1cyclictest0-21swapper/121:35:001
16147991717,0cyclictest11098-21apt-get21:12:140
16147991717,0cyclictest0-21swapper/023:22:160
16147991717,0cyclictest0-21swapper/022:47:140
16147991717,0cyclictest0-21swapper/022:17:150
16147991717,0cyclictest0-21swapper/000:07:130
16147991716,0cyclictest0-21swapper/020:52:150
16147991715,1cyclictest313-21systemd-journal21:00:010
16150991616,0cyclictest0-21swapper/323:57:163
1615099160,13cyclictest4799-21hddtemp_smartct19:52:153
16149991616,0cyclictest0-21swapper/223:37:132
16149991616,0cyclictest0-21swapper/222:17:152
16149991614,1cyclictest0-21swapper/222:00:002
16148991616,0cyclictest25924-21apt-get21:42:131
16148991616,0cyclictest0-21swapper/123:42:141
16148991616,0cyclictest0-21swapper/121:22:141
16148991616,0cyclictest0-21swapper/120:37:151
16148991616,0cyclictest0-21swapper/100:37:131
16148991616,0cyclictest0-21swapper/100:02:121
16148991613,2cyclictest8840-21sendmail-msp23:20:001
16147991616,0cyclictest0-21swapper/023:12:130
16147991616,0cyclictest0-21swapper/023:07:150
16147991616,0cyclictest0-21swapper/021:52:140
16147991616,0cyclictest0-21swapper/020:07:140
16147991616,0cyclictest0-21swapper/020:02:150
16147991616,0cyclictest0-21swapper/019:52:130
16147991616,0cyclictest0-21swapper/019:32:140
16147991616,0cyclictest0-21swapper/000:27:160
16147991614,1cyclictest23839-21apt-get20:32:140
16147991613,2cyclictest5777-21idleruntime-cro00:20:010
16147991613,2cyclictest3063-21grep00:12:230
1614799160,10cyclictest593-21dbus-daemon22:57:220
16150991515,0cyclictest0-21swapper/322:12:143
16150991515,0cyclictest0-21swapper/321:57:153
16150991515,0cyclictest0-21swapper/321:52:143
1615099150,15cyclictest15130-21gzip20:12:223
1615099150,14cyclictest0-21swapper/321:22:153
1615099150,0cyclictest2004-21apt-get23:07:153
16149991515,0cyclictest31051-21apt-get20:47:152
16149991514,1cyclictest0-21swapper/200:17:222
1614999150,11cyclictest0-21swapper/223:32:132
16147991515,0cyclictest0-21swapper/021:37:140
16150991414,0cyclictest0-21swapper/319:20:023
1615099140,14cyclictest0-21swapper/323:35:543
1615099140,14cyclictest0-21swapper/322:21:203
1615099140,14cyclictest0-21swapper/322:01:303
1614999140,14cyclictest0-21swapper/221:41:052
1614999140,14cyclictest0-21swapper/200:05:152
1614999140,0cyclictest0-21swapper/220:50:302
1614999140,0cyclictest0-21swapper/219:17:142
16148991414,0cyclictest0-21swapper/120:12:151
1614899140,14cyclictest0-21swapper/122:46:171
1614899140,14cyclictest0-21swapper/122:30:371
1614899140,14cyclictest0-21swapper/119:30:111
1614899140,0cyclictest0-21swapper/122:10:031
1614899140,0cyclictest0-21swapper/122:05:321
1614799140,14cyclictest0-21swapper/023:01:120
1614799140,14cyclictest0-21swapper/021:57:560
1614799140,0cyclictest0-21swapper/023:46:210
16150991313,0cyclictest0-21swapper/323:47:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional