You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-24 - 19:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Thu Sep 24, 2026 12:42:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1194984997338,32cyclictest1270613-21/usr/sbin/munin10:15:021
1194984995846,5cyclictest0-21swapper/111:36:241
1194985995244,5cyclictest1285797-21munin-run10:40:012
1194985995041,5cyclictest0-21swapper/211:25:002
1194985995041,5cyclictest0-21swapper/211:25:002
1194985995037,10cyclictest1235547-21mailstats09:16:242
1194985995037,10cyclictest1235547-21mailstats09:16:242
119498499493,37cyclictest1326744-21ssh11:46:231
1194983994841,4cyclictest1219407-21kworker/u8:12-flush-0:2109:16:230
1194983994841,4cyclictest1219407-21kworker/u8:12-flush-0:2109:16:230
1194985994638,5cyclictest1316242-21munin-plugin-st11:30:002
1194985994638,5cyclictest1316242-21munin-plugin-st11:30:002
119498499463,36cyclictest421112-21cupsd09:16:181
119498499463,36cyclictest421112-21cupsd09:16:181
1194985994540,2cyclictest0-21swapper/211:46:212
1194985994435,5cyclictest1246305-21ssh09:35:012
1194983994436,4cyclictest0-21swapper/010:01:210
1194985994335,5cyclictest1337618-21latency_hist12:05:032
1194985994335,5cyclictest1337618-21latency_hist12:05:032
1194984994333,7cyclictest1240198-21munin-run09:25:011
1194985994234,5cyclictest1344434-21uniq12:16:072
1194984994236,3cyclictest0-21swapper/111:00:011
1194984994231,8cyclictest421117-21cups-browsed08:55:011
1194983994235,5cyclictest1313214-21munin-plugin-st11:25:000
1194983994235,5cyclictest1313214-21munin-plugin-st11:25:000
119498599414,34cyclictest1308318-21ssh11:16:122
1194984994131,8cyclictest1209283-21needreboot07:56:151
1194984994130,5cyclictest0-21swapper/110:51:251
1194983994130,4cyclictest1345428-21unixbench_singl12:16:270
1194985994032,5cyclictest1258438-21cron09:55:012
1194984994032,5cyclictest1273616-21ssh10:20:001
1194984994030,7cyclictest0-21swapper/108:45:001
119498399403,32cyclictest1293214-21ssh10:51:230
1194985993930,6cyclictest1205419-21unixbench_singl07:41:282
1194984993929,8cyclictest1351526-21unixbench_singl12:26:281
1194983993933,4cyclictest1249334-21ssh09:40:000
119498599386,28cyclictest1271982-21ssh10:16:242
1194985993830,5cyclictest1301136-21latency_hist11:05:022
1194985993830,5cyclictest1276700-21systemd-run10:25:012
1194985993830,5cyclictest1250108-21aten_r2power_vo09:41:062
1194985993830,5cyclictest1222512-21fschecks_time08:41:072
1194985993824,10cyclictest421-21dbus-daemon09:20:112
119498499383,27cyclictest1357156-21ssh12:36:231
1194984993831,3cyclictest0-21swapper/112:20:011
1194984993828,6cyclictest420-21avahi-daemon10:06:281
1194983993833,2cyclictest0-21swapper/011:00:020
1194983993831,4cyclictest1210013-21latency_hist08:00:020
1194983993824,11cyclictest1200934-21unixbench_singl07:26:320
1194985993726,8cyclictest1247352-21latency09:36:122
1194985993725,9cyclictest1302225-21needreboot11:06:162
1194984993729,5cyclictest1225042-21latency_hist08:50:021
1194984993729,5cyclictest1225042-21latency_hist08:50:021
119498499371,26cyclictest1299297-21ssh11:01:211
1194983993731,4cyclictest1230986-21ssh09:10:010
1194983993731,3cyclictest1208395-21unixbench_singl07:51:270
1194983993729,5cyclictest420-21avahi-daemon11:43:470
1194983993725,10cyclictest1245016-21sed09:31:280
1194983993724,9cyclictest0-21swapper/012:21:140
1194983993724,11cyclictest1221846-21tail08:36:280
119498599364,25cyclictest1232300-21ssh09:11:172
1194985993630,2cyclictest0-21swapper/210:05:012
1194985993627,6cyclictest421-21dbus-daemon12:10:462
1194985993626,7cyclictest1334453-21latency_hist12:00:012
1194985993626,7cyclictest1290473-21switchtime10:46:252
1194984993630,3cyclictest0-21swapper/110:41:201
1194984993628,5cyclictest421-21dbus-daemon10:25:011
1194984993628,5cyclictest1261476-21cron10:00:011
1194984993628,5cyclictest1227999-21systemd-run09:00:021
119498599354,27cyclictest1350935-21ssh12:26:162
1194985993529,3cyclictest0-21swapper/209:56:282
1194985993528,4cyclictest1211496-21kworker/u8:1-events_unbound08:31:082
119498499353,25cyclictest1311587-21ssh11:21:221
119498499353,25cyclictest1311587-21ssh11:21:221
1194984993527,5cyclictest1329925-21systemctl11:51:261
1194984993526,6cyclictest421-21dbus-daemon09:50:011
119498499351,29cyclictest1305398-21ssh11:11:211
1194983993530,3cyclictest1002-21mta-sts-daemon10:38:370
1194983993529,4cyclictest1255433-21cut09:50:010
1194983993529,4cyclictest1241796-21switchtime09:26:270
1194983993529,4cyclictest1198010-21systemd-run07:20:010
1194983993528,4cyclictest1218976-21munin-plugin-st08:30:000
1194983993522,10cyclictest1223365-21unixbench_multi08:41:260
1194983993514,13cyclictest151ktimers/009:56:160
119498599344,21cyclictest1356984-21sh12:36:152
1194985993428,4cyclictest1194982-21cyclictest11:51:152
1194985993426,5cyclictest421117-21cups-browsed08:46:132
1194985993426,5cyclictest421117-21cups-browsed08:46:132
1194985993426,4cyclictest0-21swapper/209:10:032
119498599342,26cyclictest1287100-21rm10:41:172
119498599341,23cyclictest1281090-21ssh10:31:232
1194984993422,9cyclictest1284510-21timerwakeupswit10:36:271
1194984993422,8cyclictest1281285-21mailstats10:31:291
119498499341,25cyclictest1232196-21ssh09:11:141
1194983993428,4cyclictest1310144-21sh11:20:010
1194983993428,4cyclictest1265167-21aten_r2power_en10:06:060
1194985993327,3cyclictest0-21swapper/212:31:162
1194985993326,4cyclictest1349821-21latency_hist12:25:022
1194985993325,5cyclictest1305119-21latency11:11:142
1194985993325,5cyclictest1213315-21aten_r2power_po08:11:082
1194985993321,8cyclictest1277349-21aten_r2power_fr10:26:062
1194984993326,4cyclictest1324152-21grep11:41:281
1194984993326,4cyclictest1279914-21latency_hist10:30:021
1194984993325,5cyclictest1230009-21fschecks_time09:06:111
1194984993325,5cyclictest1211672-21latency_hist08:05:021
1194984993324,6cyclictest1204506-21fschecks_count07:41:101
119498499331,26cyclictest1302337-21rm11:06:201
1194983993326,3cyclictest0-21swapper/011:35:010
1194983993324,6cyclictest1356994-21ntp_kernel_pll_12:36:160
1194983993321,10cyclictest12650irq/32-eth009:51:170
1194985993225,3cyclictest0-21swapper/210:06:222
1194985993224,5cyclictest1226460-21cron08:55:002
1194985993223,7cyclictest1209239-21missed_timers07:56:142
119498599322,24cyclictest1293150-21ssh10:51:212
1194985993221,8cyclictest1254164-21sort09:46:282
1194984993226,3cyclictest1237379-21kworker/u8:15-xprtiod09:55:011
1194984993226,3cyclictest0-21swapper/109:31:231
1194984993225,4cyclictest1248133-21unixbench_multi09:36:281
1194984993224,5cyclictest1264547-21munin-plugin-st10:05:011
1194984993224,5cyclictest1200998-21systemd-run07:30:021
1194984993221,9cyclictest1195788-21needreboot07:11:151
1194984993219,10cyclictest1336023-21mailstats12:01:261
1194984993219,10cyclictest1336023-21mailstats12:01:261
1194983993225,5cyclictest420-21avahi-daemon11:47:480
1194983993225,4cyclictest1335832-21ntp_offset12:01:200
1194983993225,4cyclictest1335832-21ntp_offset12:01:200
1194983993225,4cyclictest1217500-21kworker/u8:4-rpciod09:21:370
1194983993225,4cyclictest1001-21mta-sts-daemon12:31:240
1194983993220,9cyclictest1273633-21grep10:20:000
119498599313,20cyclictest421-21dbus-daemon07:46:222
1194985993127,2cyclictest0-21swapper/211:40:002
1194985993127,2cyclictest0-21swapper/208:20:012
1194985993125,3cyclictest0-21swapper/210:11:302
1194985993124,3cyclictest0-21swapper/211:31:192
1194985993123,5cyclictest1211479-21munin-plugin-st08:05:012
1194984993124,4cyclictest1333267-21sed11:56:281
1194984993124,4cyclictest1003-21mta-sts-daemon10:49:311
1194984993124,3cyclictest0-21swapper/112:11:221
1194984993123,5cyclictest1354593-21unixbench_singl12:31:331
1194984993123,5cyclictest1313803-21aten_r2power_cu11:26:051
1194984993123,5cyclictest1313803-21aten_r2power_cu11:26:051
1194984993123,5cyclictest1204017-21latency_hist07:40:021
1194984993121,7cyclictest1310142-21latency_hist11:20:031
1194983993126,3cyclictest0-21swapper/011:04:200
1194983993122,7cyclictest1000-21mta-sts-daemon11:11:150
119498599303,22cyclictest1241490-21ssh09:26:192
1194985993024,3cyclictest0-21swapper/212:06:182
1194985993024,3cyclictest0-21swapper/212:06:182
1194985993022,3cyclictest0-21swapper/211:41:182
1194985993021,6cyclictest485-21in:imuxsock07:55:012
1194985993019,8cyclictest1297954-21sh11:00:012
1194984993024,3cyclictest0-21swapper/109:45:011
1194984993022,5cyclictest1347732-21ssh12:21:121
1194983993024,4cyclictest1350401-21aten_r2power_vo12:26:060
1194983993024,4cyclictest1301909-21uniq11:06:110
1194983993023,5cyclictest567-21gdbus07:40:010
1194983993023,4cyclictest421117-21cups-browsed08:54:400
1194983993022,5cyclictest421117-21cups-browsed07:12:130
1194983993022,5cyclictest420-21avahi-daemon11:35:200
1194983993021,6cyclictest1287413-21switchtime10:41:270
119498399301,24cyclictest12650irq/32-eth010:31:200
1194985992923,3cyclictest0-21swapper/208:21:282
1194984992923,4cyclictest1000-21mta-sts-daemon08:15:011
1194984992921,5cyclictest1229358-21tail09:01:301
1194984992921,5cyclictest1213046-21latency_hist08:10:021
1194984992921,5cyclictest1199485-21munin-run07:25:011
1194983992924,2cyclictest0-21swapper/007:10:010
1194983992923,4cyclictest1252437-21sh09:45:010
1194983992921,6cyclictest151ktimers/011:56:190
1194983992919,7cyclictest1341261-21basename12:11:120
1194983992919,7cyclictest1000-21mta-sts-daemon10:11:150
1194983992918,9cyclictest0-21swapper/008:31:280
1194985992820,5cyclictest1227947-21cron09:00:012
119498499283,17cyclictest1003-21mta-sts-daemon08:31:221
1194984992823,3cyclictest1211506-21kworker/u8:6-rpciod08:20:011
1194984992822,3cyclictest1002-21mta-sts-daemon07:31:191
1194984992820,4cyclictest1000-21mta-sts-daemon09:26:451
1194984992813,10cyclictest421-21dbus-daemon07:51:181
1194983992823,3cyclictest1314089-21fschecks_count11:26:120
1194983992823,3cyclictest1314089-21fschecks_count11:26:120
1194983992821,5cyclictest421117-21cups-browsed07:45:370
1194983992821,5cyclictest421117-21cups-browsed07:41:130
1194983992821,4cyclictest1276675-21cron10:25:000
1194983992821,4cyclictest1215975-21grep08:20:010
1194983992821,4cyclictest1001-21mta-sts-daemon10:46:100
1194983992820,6cyclictest1202255-21timerwakeupswit07:31:270
1194983992820,4cyclictest1316811-21kworker/u8:1-rpciod11:50:430
1194985992721,3cyclictest0-21swapper/208:36:282
1194985992721,3cyclictest0-21swapper/207:40:012
1194985992721,3cyclictest0-21swapper/207:30:012
1194984992721,3cyclictest0-21swapper/112:06:271
1194984992721,3cyclictest0-21swapper/112:06:271
1194983992722,2cyclictest0-21swapper/012:06:050
1194983992722,2cyclictest0-21swapper/012:06:050
119498599263,16cyclictest1197958-21cron07:20:002
1194985992618,4cyclictest1189647-21kworker/u8:9-xprtiod07:12:142
119498499261,17cyclictest665-21snmpd07:16:201
1194983992621,2cyclictest0-21swapper/010:26:090
1194983992620,3cyclictest1224583-21systemctl08:46:230
1194983992620,3cyclictest1224583-21systemctl08:46:230
1194983992618,6cyclictest151ktimers/008:11:190
1194985992515,7cyclictest1229144-21switchtime09:01:292
1194985992514,8cyclictest1198836-21ntp_kernel_err07:21:172
1194984992519,2cyclictest0-21swapper/111:35:011
1194984992516,6cyclictest1003-21mta-sts-daemon08:26:061
1194983992518,5cyclictest1212991-21systemd-run08:10:010
1194983992516,6cyclictest12650irq/32-eth008:01:090
119498399251,12cyclictest1232339-21ntpq09:11:190
1194985992417,3cyclictest0-21swapper/208:10:022
119498599241,15cyclictest1218404-21awk08:26:182
1194984992418,3cyclictest0-21swapper/108:25:011
1194983992417,5cyclictest151ktimers/008:21:150
1194983992415,4cyclictest151ktimers/007:21:230
1194985992317,3cyclictest0-21swapper/207:31:082
1194985992317,3cyclictest0-21swapper/207:10:022
1194984992317,3cyclictest0-21swapper/107:10:021
1194984992316,4cyclictest1222096-21latency_hist08:40:021
1194984992316,4cyclictest1206984-21systemd-run07:50:011
119498399222,15cyclictest1227309-21ntpq08:56:150
1194983992215,5cyclictest421117-21cups-browsed09:03:400
119498699213,10cyclictest0-21swapper/310:51:213
119498699212,10cyclictest0-21swapper/307:46:223
119498699194,8cyclictest0-21swapper/311:41:193
119498699192,9cyclictest0-21swapper/311:16:163
119498699191,10cyclictest0-21swapper/310:31:253
119498699183,7cyclictest0-21swapper/311:46:163
119498699183,7cyclictest0-21swapper/309:31:183
119498699181,7cyclictest0-21swapper/311:56:203
119498699181,10cyclictest0-21swapper/309:41:193
119498699175,6cyclictest0-21swapper/310:46:143
119498699174,7cyclictest0-21swapper/310:20:013
119498699172,9cyclictest0-21swapper/312:11:273
119498699172,8cyclictest0-21swapper/309:51:133
119498699172,8cyclictest0-21swapper/309:46:163
119498699171,9cyclictest0-21swapper/312:26:143
119498699171,9cyclictest0-21swapper/310:36:183
119498699163,6cyclictest0-21swapper/312:31:263
119498699163,6cyclictest0-21swapper/311:11:183
119498699162,8cyclictest0-21swapper/311:26:223
119498699162,8cyclictest0-21swapper/311:26:223
119498699162,8cyclictest0-21swapper/310:01:173
119498699162,6cyclictest0-21swapper/311:31:143
119498699162,10cyclictest0-21swapper/310:21:233
119498699161,9cyclictest0-21swapper/312:06:223
119498699161,9cyclictest0-21swapper/312:06:223
119498699161,8cyclictest0-21swapper/308:11:243
119498699153,6cyclictest0-21swapper/309:26:173
119498699152,8cyclictest0-21swapper/312:36:153
119498699152,8cyclictest0-21swapper/309:36:163
119498699151,9cyclictest0-21swapper/310:10:013
119498699151,8cyclictest0-21swapper/311:01:213
119498699151,8cyclictest0-21swapper/310:41:173
119498699151,8cyclictest0-21swapper/308:36:233
119498699151,7cyclictest0-21swapper/312:21:173
119498699145,6cyclictest0-21swapper/312:01:243
119498699145,6cyclictest0-21swapper/312:01:243
119498699144,7cyclictest0-21swapper/311:36:233
119498699143,6cyclictest0-21swapper/311:51:173
119498699143,6cyclictest0-21swapper/309:56:213
119498699142,8cyclictest0-21swapper/309:16:073
119498699142,8cyclictest0-21swapper/309:16:073
119498699142,6cyclictest0-21swapper/308:31:163
119498699142,6cyclictest0-21swapper/307:36:153
119498699141,8cyclictest0-21swapper/310:26:183
119498699141,8cyclictest0-21swapper/309:21:213
119498699141,6cyclictest0-21swapper/311:06:093
119498699141,6cyclictest0-21swapper/308:51:073
119498699141,5cyclictest0-21swapper/308:16:153
119498699140,6cyclictest0-21swapper/309:01:213
119498699135,6cyclictest0-21swapper/312:16:173
119498699132,7cyclictest0-21swapper/309:11:293
119498699132,7cyclictest0-21swapper/308:25:013
119498699132,6cyclictest0-21swapper/308:56:093
119498699132,6cyclictest0-21swapper/308:10:013
119498699131,8cyclictest0-21swapper/310:11:233
119498699131,7cyclictest0-21swapper/307:56:233
119498699131,6cyclictest0-21swapper/309:06:243
119498699122,6cyclictest0-21swapper/308:46:133
119498699122,6cyclictest0-21swapper/308:46:133
119498699122,5cyclictest0-21swapper/307:26:193
119498699121,8cyclictest0-21swapper/310:56:163
119498699121,6cyclictest0-21swapper/311:21:133
119498699121,6cyclictest0-21swapper/311:21:133
119498699121,6cyclictest0-21swapper/307:31:183
119498699121,5cyclictest0-21swapper/307:21:173
119498699111,7cyclictest0-21swapper/308:01:223
119498699101,6cyclictest0-21swapper/308:41:143
119498699101,6cyclictest0-21swapper/307:16:203
119498699101,5cyclictest0-21swapper/308:26:073
119498699101,5cyclictest0-21swapper/307:51:083
119498699101,5cyclictest0-21swapper/307:41:113
11949869991,5cyclictest0-21swapper/307:11:093
11949869982,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional