You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-20 - 07:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Sun Sep 20, 2026 00:42:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1253926996352,7cyclictest3734659-21cups-browsed21:51:270
1253926995324,17cyclictest151ktimers/022:16:130
1253930995243,6cyclictest1285473-21systemd-run20:55:012
1253930995230,18cyclictest1396057-21munin-plugin-st00:05:002
1253930995129,19cyclictest1321532-21ssh22:01:122
1253930994823,3cyclictest0-21swapper/223:56:222
1253930994823,3cyclictest0-21swapper/223:56:222
125392699482,35cyclictest151ktimers/019:21:200
1253930994732,12cyclictest1341682-21kworker/u8:9-xprtiod22:50:002
1253930994727,17cyclictest0-21swapper/200:25:012
1253930994721,4cyclictest1283838-21unixbench_multi20:46:262
1253930994719,8cyclictest1265263-21ntp_kernel_err19:46:162
1253927994739,5cyclictest1368890-21systemd-run23:20:021
1253927994739,5cyclictest1368890-21systemd-run23:20:021
1253927994734,10cyclictest1309625-21sh21:41:171
1253927994638,5cyclictest1411137-21munin-run00:30:011
1253927994637,5cyclictest0-21swapper/123:46:201
125393099454,13cyclictest1342881-21ssh22:36:132
125393099454,13cyclictest1342881-21ssh22:36:132
1253930994522,3cyclictest0-21swapper/221:46:192
1253926994536,6cyclictest0-21swapper/000:16:160
1253930994437,3cyclictest0-21swapper/223:51:142
1253927994436,5cyclictest1280994-21systemd-run20:40:011
125392799434,30cyclictest1321851-21ssh22:01:181
1253930994234,5cyclictest1393066-21cups-browsed00:11:152
1253930994221,3cyclictest0-21swapper/200:26:062
1253930994213,10cyclictest421-21dbus-daemon22:41:032
1253930994213,10cyclictest421-21dbus-daemon22:41:032
125392799425,30cyclictest1333943-21ssh22:21:181
1253927994230,8cyclictest1361241-21sh23:06:221
1253926994234,5cyclictest1-21systemd21:45:010
125393099413,13cyclictest1327813-21sshd22:11:172
1253927994135,3cyclictest0-21swapper/100:31:151
125392799413,31cyclictest1330818-21ssh22:16:131
1253927994132,6cyclictest1346526-21tail22:41:271
1253927994132,6cyclictest1346526-21tail22:41:271
1253926994131,7cyclictest12650irq/32-eth022:01:220
1253926994116,17cyclictest151ktimers/023:56:180
1253926994116,17cyclictest151ktimers/023:56:180
125393099405,15cyclictest1336886-21ssh22:26:142
1253930994032,5cyclictest1308398-21cron21:40:002
1253930994032,4cyclictest0-21swapper/222:56:172
1253930994022,14cyclictest1000-21mta-sts-daemon00:07:422
1253927994031,6cyclictest1308586-21latency_hist21:40:021
1253927994029,8cyclictest1305364-21/usr/sbin/munin21:35:021
1253926994030,5cyclictest151ktimers/023:21:200
125393099393,32cyclictest1291256-21ssh21:11:152
1253930993931,3cyclictest0-21swapper/223:41:182
1253930993922,4cyclictest1003-21mta-sts-daemon23:00:232
1253927993931,5cyclictest1325350-21unixbench-2d22:06:281
1253927993931,5cyclictest1282495-21systemd-run20:45:011
125392799392,30cyclictest1327865-21ssh22:11:191
125392699394,26cyclictest1352040-21ssh22:51:180
1253926993930,7cyclictest1304111-21ssh21:31:280
1253926993929,8cyclictest1404730-21kworker/u8:15-rpciod00:21:170
1253930993830,4cyclictest0-21swapper/221:26:192
1253930993829,6cyclictest1279482-21systemd-run20:35:012
1253930993829,5cyclictest1379773-21grep23:36:282
1253927993830,3cyclictest0-21swapper/123:38:291
1253927993828,4cyclictest0-21swapper/121:51:201
125392799382,26cyclictest1397312-21ssh00:06:161
1253926993832,4cyclictest1356769-21munin-run23:00:010
1253926993830,5cyclictest1333276-21aten_r2power_fr22:21:050
1253926993829,6cyclictest1293910-21ssh21:16:090
1253926993826,6cyclictest151ktimers/020:31:150
1253930993727,7cyclictest1304109-21tail21:31:282
1253930993725,9cyclictest1294308-21ntp_kernel_pll_21:16:162
1253927993730,4cyclictest1408150-21systemd-run00:25:021
1253927993730,4cyclictest1353816-21latency_hist22:55:021
1253927993728,7cyclictest0-21swapper/100:00:011
1253927993728,7cyclictest0-21swapper/100:00:011
1253926993731,3cyclictest1001-21mta-sts-daemon23:31:160
1253927993628,5cyclictest1267412-21cut19:55:011
1253927993627,6cyclictest420-21avahi-daemon00:16:461
1253927993625,8cyclictest1276431-21munin-plugin-st20:25:001
1253926993629,4cyclictest1326574-21munin-run22:10:010
1253926993621,12cyclictest1368901-21latency_hist23:20:020
1253926993621,12cyclictest1368901-21latency_hist23:20:020
1253930993527,5cyclictest1331347-21ssh22:16:262
1253930993527,5cyclictest1282482-21idleruntime-cro20:45:012
1253930993526,5cyclictest1286998-21systemd-run21:00:012
1253927993525,7cyclictest1348991-21ssh22:46:161
1253927993524,7cyclictest1255364-21unixbench_singl19:11:251
1253926993530,2cyclictest0-21swapper/000:10:170
1253926993528,4cyclictest1361117-21ntp_kernel_pll_23:06:170
1253926993527,5cyclictest1265071-21kernelversion19:46:130
125392699351,29cyclictest1394404-21sh00:01:210
1253930993425,5cyclictest1002-21mta-sts-daemon21:42:192
1253930993424,7cyclictest1362840-21systemd-run23:10:012
1253927993428,3cyclictest0-21swapper/123:11:141
1253927993428,3cyclictest0-21swapper/121:56:201
1253927993426,5cyclictest1289972-21idleruntime-cro21:10:011
1253927993423,8cyclictest1286354-21ntp_kernel_pll_20:56:161
125392799342,25cyclictest1257815-21sshd19:21:161
1253927993418,11cyclictest421-21dbus-daemon23:30:361
1253926993426,5cyclictest1341654-21cron22:35:010
1253926993425,4cyclictest0-21swapper/023:41:170
1253926993424,7cyclictest1371902-21kworker/u8:7-xprtiod00:09:100
125392699341,27cyclictest1327818-21sshd22:11:170
1253930993328,2cyclictest0-21swapper/223:35:012
1253930993324,5cyclictest1268737-21/usr/sbin/munin19:56:272
1253930993324,5cyclictest1268737-21/usr/sbin/munin19:56:272
1253930993323,7cyclictest1280986-21sendmail20:40:012
1253927993326,4cyclictest1344731-21ssh22:40:031
1253927993326,4cyclictest1344731-21ssh22:40:031
1253927993325,5cyclictest421-21dbus-daemon21:15:011
1253927993325,5cyclictest1265935-21systemd-run19:50:011
1253927993323,7cyclictest1336175-21/usr/sbin/munin22:26:101
1253926993327,4cyclictest191rcuc/021:56:170
1253926993325,5cyclictest1276419-21cron20:25:000
1253926993321,9cyclictest0-21swapper/000:31:120
1253926993312,16cyclictest665-21snmpd22:47:200
125393099323,23cyclictest1364063-21ssh23:11:192
1253930993226,3cyclictest0-21swapper/221:56:242
1253930993225,4cyclictest1262913-21sendmail19:40:012
125392799324,22cyclictest1297487-21ssh21:21:191
125392799322,5cyclictest171rcu_preempt21:46:181
1253927993225,3cyclictest0-21swapper/123:51:031
1253927993225,3cyclictest0-21swapper/122:35:021
1253927993224,5cyclictest1266873-21kworker/u8:5+xprtiod20:05:001
1253927993224,5cyclictest1266873-21kworker/u8:5+xprtiod20:05:001
1253927993223,6cyclictest420-21avahi-daemon20:55:021
1253927993223,3cyclictest0-21swapper/123:01:151
125392799322,23cyclictest1400441-21ssh00:11:201
1253926993226,4cyclictest0-21swapper/021:01:210
1253926993225,4cyclictest1357667-21fschecks_time23:01:080
1253926993224,5cyclictest151ktimers/020:51:130
1253926993224,5cyclictest151ktimers/020:01:210
1253926993224,5cyclictest151ktimers/020:01:210
1253926993223,7cyclictest0-21swapper/000:26:070
1253926993220,7cyclictest12650irq/32-eth022:26:130
1253930993125,3cyclictest0-21swapper/222:25:022
1253930993124,4cyclictest1001-21mta-sts-daemon22:08:472
1253930993123,5cyclictest1372073-21latency_hist23:25:022
1253930993123,5cyclictest1353753-21idleruntime-cro22:55:002
1253930993123,5cyclictest1296862-21aten2.4-expect21:21:072
1253930993122,6cyclictest1412919-21unixbench-2d00:31:282
1253930993121,6cyclictest1368860-21idleruntime-cro23:20:012
1253930993121,6cyclictest1368860-21idleruntime-cro23:20:012
1253927993122,6cyclictest3734659-21cups-browsed20:20:011
1253927993122,4cyclictest0-21swapper/123:21:161
1253926993125,3cyclictest0-21swapper/021:46:150
1253926993123,6cyclictest1364645-21unixbench_multi23:11:320
1253926993122,7cyclictest0-21swapper/021:11:160
1253926993119,9cyclictest1274431-21proc_pri20:16:200
125392699311,5cyclictest0-21swapper/023:26:130
1253930993023,4cyclictest1002-21mta-sts-daemon23:26:072
1253930993022,4cyclictest0-21swapper/219:31:162
1253927993022,4cyclictest0-21swapper/100:36:101
1253927993022,3cyclictest0-21swapper/121:26:151
125392799302,21cyclictest1373266-21ssh23:26:201
1253927993015,9cyclictest421-21dbus-daemon00:01:191
125392699309,16cyclictest151ktimers/020:26:110
125392699303,21cyclictest12650irq/32-eth023:46:180
1253926993024,4cyclictest1271838-21unixbench_singl20:06:380
1253926993021,7cyclictest1417167-21munin-plugin-st00:40:010
1253926993019,7cyclictest3734659-21cups-browsed21:22:170
1253926993019,7cyclictest12650irq/32-eth022:41:170
1253926993019,7cyclictest12650irq/32-eth022:41:170
1253926993018,11cyclictest151ktimers/019:51:120
1253930992923,3cyclictest0-21swapper/219:45:012
1253930992923,3cyclictest0-21swapper/200:36:302
1253930992922,4cyclictest0-21swapper/219:11:162
1253930992921,5cyclictest1276002-21mailstats20:21:302
1253930992921,4cyclictest0-21swapper/223:46:192
1253930992920,6cyclictest1274795-21unixbench_multi20:16:252
125392799292,22cyclictest3734659-21cups-browsed19:36:151
1253927992921,5cyclictest1288329-21unixbench-2d21:01:271
1253927992921,4cyclictest0-21swapper/121:16:091
1253927992921,3cyclictest0-21swapper/123:41:041
1253926992924,3cyclictest151ktimers/021:06:190
125392699291,3cyclictest0-21swapper/020:36:150
1253926992910,12cyclictest12650irq/32-eth023:36:150
1253930992821,4cyclictest1273399-21kworker/u8:5+xprtiod20:30:032
1253930992821,4cyclictest1003-21mta-sts-daemon20:11:232
1253930992820,4cyclictest0-21swapper/221:51:272
1253927992821,4cyclictest0-21swapper/119:35:021
1253927992820,5cyclictest1268903-21munin-run20:00:001
1253927992820,5cyclictest1268903-21munin-run20:00:001
1253927992815,5cyclictest1002-21mta-sts-daemon20:31:201
1253926992820,6cyclictest0-21swapper/021:00:010
125392699281,22cyclictest1306734-21ssh21:36:230
1253930992721,3cyclictest1000-21mta-sts-daemon19:51:182
1253930992721,3cyclictest0-21swapper/222:31:132
1253930992721,3cyclictest0-21swapper/221:06:202
1253930992719,5cyclictest1405097-21ssh00:20:012
1253930992719,5cyclictest1287526-21gltestperf21:01:102
1253927992716,8cyclictest1277916-21sh20:30:011
125392699279,12cyclictest12650irq/32-eth021:26:150
1253926992719,6cyclictest1388843-21unixbench_singl23:51:260
1253926992714,9cyclictest0-21swapper/022:36:170
1253926992714,9cyclictest0-21swapper/022:36:170
1253927992618,6cyclictest1283318-21gcc20:46:131
1253927992618,2cyclictest0-21swapper/122:56:081
1253926992619,5cyclictest1256849-21head19:16:320
1253930992519,3cyclictest0-21swapper/219:25:012
1253927992517,5cyclictest0-21swapper/120:11:201
1253926992512,8cyclictest12650irq/32-eth019:56:160
1253926992512,8cyclictest12650irq/32-eth019:56:160
1253930992419,2cyclictest0-21swapper/219:20:012
1253930992417,5cyclictest0-21swapper/220:01:242
1253930992417,5cyclictest0-21swapper/220:01:242
1253927992417,3cyclictest0-21swapper/120:06:211
1253926992418,4cyclictest1259827-21unixbench_multi19:26:440
1253926992417,5cyclictest428-21mta-sts-daemon20:40:160
1253930992317,3cyclictest0-21swapper/220:10:012
1253927992317,3cyclictest0-21swapper/119:20:011
1253927992310,11cyclictest271ktimers/119:41:141
125392699238,9cyclictest12650irq/32-eth020:46:170
125392699232,15cyclictest1272801-21ntpq20:11:150
1253926992317,4cyclictest1264424-21munin-plugin-st19:45:010
1253926992317,4cyclictest1253925-21cyclictest19:11:160
1253926992317,3cyclictest0-21swapper/019:31:120
1253927992214,5cyclictest1258952-21fschecks_time19:26:101
125392699221,15cyclictest1262314-21sshd19:36:160
125393199212,14cyclictest0-21swapper/321:21:193
125393199211,15cyclictest0-21swapper/320:46:183
125393099219,8cyclictest421-21dbus-daemon19:10:022
1253930992113,5cyclictest1259811-21unixbench_multi19:26:442
125393199202,13cyclictest0-21swapper/321:46:173
125393199202,13cyclictest0-21swapper/319:26:173
125393199201,3cyclictest0-21swapper/323:46:083
125393199201,3cyclictest0-21swapper/322:21:213
125393199201,3cyclictest0-21swapper/319:41:313
125393199201,14cyclictest0-21swapper/322:01:163
125393199201,14cyclictest0-21swapper/320:56:203
125393199201,14cyclictest0-21swapper/320:01:163
125393199201,14cyclictest0-21swapper/320:01:163
125393199201,13cyclictest0-21swapper/320:36:153
125393199201,13cyclictest0-21swapper/300:31:163
125393199200,14cyclictest0-21swapper/320:30:463
125392799201,14cyclictest265-21systemd-journal19:10:011
1253926992016,2cyclictest0-21swapper/019:10:010
125393199191,3cyclictest0-21swapper/323:01:173
125393199191,3cyclictest0-21swapper/322:57:323
125393199191,3cyclictest0-21swapper/322:46:153
125393199191,3cyclictest0-21swapper/321:11:163
125393199191,3cyclictest0-21swapper/320:41:133
125393199191,3cyclictest0-21swapper/320:16:203
125393199191,3cyclictest0-21swapper/319:36:143
125393199191,3cyclictest0-21swapper/300:21:173
125393199191,14cyclictest0-21swapper/319:11:143
125393199191,13cyclictest0-21swapper/323:41:143
125393199191,13cyclictest0-21swapper/323:26:133
125393199191,13cyclictest0-21swapper/323:21:153
125393199191,13cyclictest0-21swapper/323:16:213
125393199191,13cyclictest0-21swapper/323:16:213
125393199191,13cyclictest0-21swapper/323:06:213
125393199191,13cyclictest0-21swapper/322:16:133
125393199191,13cyclictest0-21swapper/321:01:183
125393199191,13cyclictest0-21swapper/319:46:213
125393199191,13cyclictest0-21swapper/300:26:213
125393199191,13cyclictest0-21swapper/300:16:203
125393199191,12cyclictest0-21swapper/319:51:123
125393199181,3cyclictest0-21swapper/323:36:213
125393199181,3cyclictest0-21swapper/323:31:173
125393199181,3cyclictest0-21swapper/322:41:143
125393199181,3cyclictest0-21swapper/322:41:143
125393199181,3cyclictest0-21swapper/322:31:143
125393199181,3cyclictest0-21swapper/322:11:203
125393199181,3cyclictest0-21swapper/321:51:203
125393199181,3cyclictest0-21swapper/321:36:203
125393199181,3cyclictest0-21swapper/321:16:183
125393199181,3cyclictest0-21swapper/320:21:233
125393199181,3cyclictest0-21swapper/320:06:183
125393199181,3cyclictest0-21swapper/319:31:183
125393199181,3cyclictest0-21swapper/300:36:193
125393199181,3cyclictest0-21swapper/300:11:193
125393199181,3cyclictest0-21swapper/300:06:143
125393199181,13cyclictest0-21swapper/323:56:143
125393199181,13cyclictest0-21swapper/323:56:143
125393199181,13cyclictest0-21swapper/323:51:143
125393199181,13cyclictest0-21swapper/323:11:243
125393199181,13cyclictest0-21swapper/322:26:193
125393199181,12cyclictest0-21swapper/322:51:163
125393199181,12cyclictest0-21swapper/322:36:133
125393199181,12cyclictest0-21swapper/322:36:133
125393199181,12cyclictest0-21swapper/322:06:163
125393199181,12cyclictest0-21swapper/321:26:153
125393199181,12cyclictest0-21swapper/321:06:193
125393199181,12cyclictest0-21swapper/320:26:113
125393199181,12cyclictest0-21swapper/320:11:133
125393199181,12cyclictest0-21swapper/319:56:193
125393199181,12cyclictest0-21swapper/319:56:193
125393199181,12cyclictest0-21swapper/319:21:153
125393199181,12cyclictest0-21swapper/319:16:143
125393199180,3cyclictest0-21swapper/321:56:153
125393199180,3cyclictest0-21swapper/321:31:103
125393199180,13cyclictest0-21swapper/321:41:183
125393199172,11cyclictest0-21swapper/300:01:053
125393199171,3cyclictest0-21swapper/320:51:033
125393199170,2cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional