You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-01 - 15:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Wed Jul 01, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1167550995529,23cyclictest1234225-21systemd10:00:020
1167550995119,30cyclictest0-21swapper/009:00:010
116755299502,10cyclictest171rcu_preempt10:15:382
1167551995042,5cyclictest1178066-21systemd-run07:45:011
1167551994940,6cyclictest1328838-21cron12:35:011
116755099491,16cyclictest1186379-21ntpq08:10:330
1167552994834,10cyclictest1276930-21sh11:10:012
1167550994728,15cyclictest0-21swapper/009:35:360
116755299461,35cyclictest1247587-21ssh10:20:412
1167550994615,27cyclictest1310910-21/usr/sbin/munin12:05:230
1167550994615,27cyclictest1310910-21/usr/sbin/munin12:05:230
116755099460,23cyclictest1262709-21ssh10:45:350
1167552994538,4cyclictest0-21swapper/211:30:222
1167552994537,5cyclictest421-21dbus-daemon08:35:012
1167551994537,5cyclictest1319734-21systemd-run12:20:011
1167551994533,5cyclictest1179522-21kworker/u8:2-events_unbound09:25:361
1167551994533,5cyclictest1179522-21kworker/u8:2-events_unbound09:25:361
116755199452,34cyclictest1244336-21ssh10:15:331
1167550994516,26cyclictest0-21swapper/012:40:010
116755299447,6cyclictest171rcu_preempt12:35:412
1167550994421,19cyclictest0-21swapper/009:25:360
1167550994421,19cyclictest0-21swapper/009:25:360
1167552994333,7cyclictest1302993-21grep11:50:452
1167551994335,5cyclictest1252571-21latency_hist10:30:021
116755199431,35cyclictest1308512-21sh12:00:311
1167550994316,4cyclictest1293734-21unixbench-2d11:35:450
1167552994234,5cyclictest1203531-21diskmemload12:30:472
1167552994234,5cyclictest1170889-21aten_r2power_fr07:20:242
1167552994228,10cyclictest1188537-21munin-plugin-st08:20:012
1167551994236,3cyclictest0-21swapper/111:30:251
116755099429,16cyclictest0-21swapper/011:10:240
1167550994233,6cyclictest1270781-21munin-run11:00:010
1167550994219,2cyclictest0-21swapper/010:25:020
1167550994219,20cyclictest1284905-21sshd11:21:220
116755299414,30cyclictest1210721-21sh09:20:332
116755299414,30cyclictest1210721-21sh09:20:332
1167552994133,4cyclictest0-21swapper/209:50:022
116755099412,15cyclictest1302515-21ntp_states11:50:380
116755099412,15cyclictest1204685-21ntpq09:10:380
1167550994120,4cyclictest1241522-21ssh10:10:440
1167550994120,3cyclictest0-21swapper/011:05:460
1167550994116,5cyclictest0-21swapper/011:05:020
1167550994115,5cyclictest371393-21cups-browsed07:17:470
1167552994028,8cyclictest1229040-21needreboot09:50:322
116755299402,28cyclictest371393-21cups-browsed12:05:382
116755299402,28cyclictest371393-21cups-browsed12:05:382
1167551994032,5cyclictest1209663-21ssh09:20:001
1167551994032,5cyclictest1209663-21ssh09:20:001
1167551994025,3cyclictest171rcu_preempt10:55:451
116755099403,7cyclictest1317692-21ntpq12:15:380
1167550994029,7cyclictest420-21avahi-daemon10:16:060
116755099402,15cyclictest1173017-21ntp_states07:25:410
1167550994020,17cyclictest0-21swapper/008:45:380
1167550994020,17cyclictest0-21swapper/008:45:380
1167552993933,2cyclictest0-21swapper/208:00:012
1167552993932,4cyclictest1167549-21cyclictest07:35:352
1167552993931,5cyclictest1319724-21sendmail12:20:012
1167552993931,5cyclictest1270786-21idleruntime-cro11:00:012
1167552993929,7cyclictest1194540-21basename08:40:012
1167551993932,4cyclictest0-21swapper/110:00:011
1167550993930,5cyclictest1250680-21proc_pri10:25:420
1167552993830,5cyclictest1225058-21systemd-run09:45:012
1167552993824,10cyclictest1209662-21latency_hist09:20:022
1167552993824,10cyclictest1209662-21latency_hist09:20:022
1167551993832,3cyclictest0-21swapper/108:00:011
1167551993830,5cyclictest1193037-21munin-plugin-st08:35:011
1167551993829,7cyclictest1307479-21ssh12:00:011
1167551993825,5cyclictest750-21ntpd11:20:321
116755099386,10cyclictest12650irq/32-eth009:50:350
1167550993817,15cyclictest12650irq/32-eth011:55:350
116755099381,18cyclictest1238338-21ntpq10:05:380
1167552993729,5cyclictest1324338-21sed12:25:482
1167552993726,7cyclictest420-21avahi-daemon10:39:572
1167552993726,7cyclictest420-21avahi-daemon10:39:572
116755299371,31cyclictest1238491-21ssh10:05:442
1167551993728,6cyclictest420-21avahi-daemon10:11:421
1167551993728,5cyclictest1274200-21/usr/sbin/munin11:05:291
1167551993724,9cyclictest371393-21cups-browsed11:38:071
1167550993731,4cyclictest1296376-21ssh11:40:380
1167550993712,6cyclictest12650irq/32-eth012:10:370
1167552993627,6cyclictest1203534-21grep09:10:012
116755299361,27cyclictest1216959-21ssh09:30:512
1167551993628,5cyclictest0-21swapper/112:30:021
1167551993624,6cyclictest0-21swapper/109:45:361
116755099362,17cyclictest1207628-21rm09:15:340
116755099362,17cyclictest1207628-21rm09:15:340
116755299353,25cyclictest1320779-21rm12:20:362
1167552993527,5cyclictest1308144-21fschecks_count12:00:252
1167552993525,7cyclictest1186056-21sort08:10:282
1167552993525,6cyclictest420-21avahi-daemon11:12:162
1167552993525,6cyclictest1-21systemd10:00:012
1167552993524,5cyclictest0-21swapper/208:45:362
1167552993524,5cyclictest0-21swapper/208:45:362
1167551993528,3cyclictest0-21swapper/111:15:021
1167551993527,5cyclictest1255591-21latency_hist10:35:021
1167551993527,5cyclictest1203549-21sh09:10:021
1167550993528,4cyclictest1199048-21systemd-run08:55:010
1167550993528,3cyclictest0-21swapper/011:15:370
116755099352,16cyclictest1320824-21ntpq12:20:380
1167550993512,9cyclictest12650irq/32-eth007:20:320
116755099351,18cyclictest1225043-21awk09:45:010
116755099351,17cyclictest1226581-21sed09:45:450
1167552993427,4cyclictest1283012-21munin-run11:20:012
1167552993427,4cyclictest1221956-21awk09:40:012
116755299342,24cyclictest1253616-21ssh10:30:382
1167551993428,3cyclictest0-21swapper/109:40:271
1167551993428,3cyclictest0-21swapper/109:35:011
1167551993427,4cyclictest1247674-21sendmail_mailqu10:20:421
1167551993427,4cyclictest1196037-21idleruntime-cro08:45:011
1167551993424,7cyclictest1260031-21switchtime10:40:441
1167551993423,8cyclictest1265650-21missed_timers10:50:341
1167551993423,8cyclictest1256494-21missed_timers10:35:311
1167551993423,8cyclictest1256494-21missed_timers10:35:311
1167551993420,6cyclictest0-21swapper/109:40:021
116755099343,11cyclictest12650irq/32-eth012:00:320
116755099342,13cyclictest12650irq/32-eth011:45:330
116755099341,25cyclictest1265815-21ssh10:50:400
1167551993327,3cyclictest0-21swapper/110:00:461
1167551993326,3cyclictest0-21swapper/111:15:361
1167550993317,3cyclictest1001-21mta-sts-daemon08:30:480
1167550993316,6cyclictest371393-21cups-browsed10:33:420
1167550993310,6cyclictest1292172-21latency_hist11:35:030
116755299326,18cyclictest1314658-21ssh12:10:342
1167552993222,7cyclictest1259479-21latency10:40:282
1167552993221,6cyclictest1298328-21cron11:45:012
1167551993226,3cyclictest0-21swapper/110:05:361
1167551993224,5cyclictest420-21avahi-daemon07:30:381
1167551993224,5cyclictest1331884-21cron12:40:011
1167551993224,5cyclictest1302774-21switchtime11:50:441
1167551993223,5cyclictest1000-21mta-sts-daemon12:15:001
1167551993222,7cyclictest1210065-21aten_r2power_cu09:20:241
1167551993222,7cyclictest1210065-21aten_r2power_cu09:20:241
116755199322,25cyclictest1296329-21ssh11:40:351
1167550993226,3cyclictest0-21swapper/009:32:250
1167550993223,5cyclictest421-21dbus-daemon10:40:020
1167550993223,5cyclictest421-21dbus-daemon10:40:020
1167550993213,12cyclictest151ktimers/007:55:380
1167552993123,5cyclictest1177604-21sendmail_mailqu07:40:412
1167552993123,3cyclictest0-21swapper/211:35:302
1167552993122,6cyclictest420-21avahi-daemon10:01:572
1167552993121,6cyclictest1204058-21aten_r2power_vo09:10:252
1167551993125,3cyclictest0-21swapper/110:50:011
1167551993125,3cyclictest0-21swapper/109:00:011
1167551993123,5cyclictest1190061-21systemd-run08:25:011
1167551993122,6cyclictest421-21dbus-daemon07:55:011
1167551993121,5cyclictest1261773-21kworker/u8:11-xprtiod11:00:291
1167550993125,3cyclictest0-21swapper/008:20:020
1167550993124,5cyclictest439-21systemd-logind08:10:010
1167552993023,4cyclictest1271714-21missed_timers11:00:302
1167552993022,5cyclictest0-21swapper/209:30:002
1167552993022,5cyclictest0-21swapper/209:30:002
1167552993016,10cyclictest421-21dbus-daemon11:47:392
116755299301,20cyclictest1250488-21rm10:25:352
1167551993024,3cyclictest0-21swapper/107:25:411
1167551993023,4cyclictest1205191-21head09:10:461
1167551993022,6cyclictest1172079-21latency_hist07:25:021
1167551993022,5cyclictest439-21systemd-logind08:40:011
1167551993022,5cyclictest1188575-21systemd-run08:20:021
1167551993021,6cyclictest371393-21cups-browsed08:12:491
1167551993021,5cyclictest1322756-21latency_hist12:25:021
116755099303,21cyclictest1326886-21ntpq12:30:360
1167550993023,4cyclictest0-21swapper/012:25:350
1167550993016,12cyclictest1259898-21mailstats10:40:410
116755299297,13cyclictest1003-21mta-sts-daemon07:15:342
1167552992923,3cyclictest0-21swapper/210:55:002
1167552992922,3cyclictest0-21swapper/207:30:382
1167552992920,6cyclictest485-21in:imuxsock08:10:012
1167552992919,8cyclictest1191217-21switchtime08:25:452
1167552992919,6cyclictest371391-21cupsd11:27:392
1167551992923,3cyclictest0-21swapper/109:50:471
1167551992923,3cyclictest0-21swapper/107:15:011
1167551992922,4cyclictest1179452-21sed07:45:451
1167551992922,4cyclictest1179452-21sed07:45:451
1167551992922,4cyclictest1000-21mta-sts-daemon09:00:341
1167551992921,4cyclictest0-21swapper/111:45:391
1167551992915,10cyclictest421-21dbus-daemon11:27:511
1167550992923,4cyclictest1183075-21fschecks_time08:00:270
1167550992922,4cyclictest1189072-21fschecks_count08:20:270
116755099292,20cyclictest1210849-21ssh09:20:390
116755099292,20cyclictest1210849-21ssh09:20:390
1167550992920,7cyclictest1194522-21idleruntime-cro08:40:010
116755099291,14cyclictest1287244-21ntpq11:25:390
116755299284,17cyclictest1241401-21ssh10:10:392
116755299284,13cyclictest1305487-21ssh11:55:352
116755299282,19cyclictest1262753-21sh10:45:362
116755299282,18cyclictest958-21sendmail-mta07:45:342
116755299282,18cyclictest958-21sendmail-mta07:45:342
1167552992821,5cyclictest1167549-21cyclictest07:55:002
1167552992817,8cyclictest439-21systemd-logind09:00:022
1167552992816,9cyclictest1168436-21ntp_kernel_pll_07:10:402
1167552992816,6cyclictest0-21swapper/211:20:332
116755199284,18cyclictest1311616-21ssh12:05:341
116755199284,18cyclictest1311616-21ssh12:05:341
1167550992823,3cyclictest1202031-21cron09:05:010
1167550992823,2cyclictest0-21swapper/010:02:410
1167550992822,3cyclictest0-21swapper/009:10:010
1167552992720,4cyclictest1173534-21cron07:30:012
1167552992719,5cyclictest371393-21cups-browsed08:42:092
1167551992719,5cyclictest371393-21cups-browsed08:27:291
1167551992719,5cyclictest1176530-21munin-plugin-st07:40:011
1167552992620,2cyclictest341rcuc/208:50:362
1167552992619,4cyclictest1183986-21unixbench_singl08:00:482
1167551992620,4cyclictest1000-21mta-sts-daemon08:50:301
1167551992619,3cyclictest0-21swapper/107:20:011
1167551992618,4cyclictest1183547-21/usr/sbin/munin08:00:421
1167550992621,3cyclictest1002-21mta-sts-daemon07:50:000
1167550992621,3cyclictest1002-21mta-sts-daemon07:50:000
1167550992620,3cyclictest1180943-21unixbench_singl07:50:460
1167550992617,7cyclictest1195060-21fschecks_count08:40:280
1167551992521,2cyclictest0-21swapper/108:05:311
1167550992518,5cyclictest567-21gdbus07:38:090
1167550992518,5cyclictest1178066-21systemd-run07:45:010
1167550992516,7cyclictest1174059-21fschecks_count07:30:280
1167552992415,5cyclictest371393-21cups-browsed08:23:382
116755299232,16cyclictest1201418-21ntpq09:00:342
116755199232,15cyclictest1197020-21ntpq08:45:391
116755199232,15cyclictest1197020-21ntpq08:45:391
1167550992313,8cyclictest265-21systemd-journal08:30:020
116755399212,12cyclictest0-21swapper/312:10:353
1167550992116,3cyclictest1000-21mta-sts-daemon07:10:220
116755399202,9cyclictest0-21swapper/309:10:343
1167551992015,2cyclictest0-21swapper/107:10:021
116755399193,9cyclictest0-21swapper/310:55:323
116755399193,10cyclictest0-21swapper/310:20:463
116755399191,8cyclictest0-21swapper/307:40:333
116755399184,6cyclictest0-21swapper/312:30:353
116755399183,9cyclictest0-21swapper/310:05:373
116755399182,9cyclictest0-21swapper/311:30:393
116755399182,9cyclictest0-21swapper/310:15:373
116755399182,7cyclictest0-21swapper/308:25:393
116755399182,10cyclictest0-21swapper/311:55:353
116755399182,10cyclictest0-21swapper/309:15:243
116755399182,10cyclictest0-21swapper/309:15:243
116755399181,7cyclictest0-21swapper/308:05:443
116755399181,11cyclictest0-21swapper/310:30:473
116755399173,8cyclictest0-21swapper/312:00:323
116755399172,9cyclictest0-21swapper/311:17:223
116755399172,9cyclictest0-21swapper/311:10:313
116755399172,9cyclictest0-21swapper/310:35:253
116755399172,9cyclictest0-21swapper/310:35:253
116755399172,9cyclictest0-21swapper/309:45:383
116755399172,8cyclictest0-21swapper/310:40:383
116755399171,9cyclictest0-21swapper/309:30:443
116755399171,7cyclictest0-21swapper/307:55:333
116755399171,6cyclictest0-21swapper/308:45:333
116755399171,6cyclictest0-21swapper/308:45:333
116755399171,11cyclictest0-21swapper/311:50:343
116755399171,10cyclictest0-21swapper/310:00:383
116755299171,8cyclictest1167555-21systemd-run07:10:012
116755399163,7cyclictest0-21swapper/311:20:333
116755399162,9cyclictest0-21swapper/312:05:343
116755399162,9cyclictest0-21swapper/312:05:343
116755399162,9cyclictest0-21swapper/309:50:293
116755399162,8cyclictest0-21swapper/312:25:393
116755399162,8cyclictest0-21swapper/311:35:283
116755399162,8cyclictest0-21swapper/310:25:353
116755399162,8cyclictest0-21swapper/310:10:393
116755399162,8cyclictest0-21swapper/309:25:293
116755399162,8cyclictest0-21swapper/309:25:293
116755399162,8cyclictest0-21swapper/308:40:013
116755399162,8cyclictest0-21swapper/307:25:413
116755399161,9cyclictest0-21swapper/311:45:303
116755399161,9cyclictest0-21swapper/309:55:453
116755399161,7cyclictest0-21swapper/310:50:283
116755399161,6cyclictest0-21swapper/309:05:343
116755399161,6cyclictest0-21swapper/307:20:363
116755399160,7cyclictest0-21swapper/307:45:393
116755399160,7cyclictest0-21swapper/307:45:393
116755099166,7cyclictest421-21dbus-daemon07:10:010
116755399153,8cyclictest0-21swapper/311:05:303
116755399153,8cyclictest0-21swapper/309:35:283
116755399153,7cyclictest0-21swapper/309:20:333
116755399153,7cyclictest0-21swapper/309:20:333
116755399152,8cyclictest0-21swapper/312:35:393
116755399152,8cyclictest0-21swapper/312:20:273
116755399152,8cyclictest0-21swapper/311:40:323
116755399152,8cyclictest0-21swapper/311:25:393
116755399152,8cyclictest0-21swapper/310:45:383
116755399152,7cyclictest0-21swapper/309:40:323
116755399151,6cyclictest0-21swapper/308:00:353
116755399151,6cyclictest0-21swapper/307:35:353
116755399142,6cyclictest0-21swapper/308:10:263
116755399142,6cyclictest0-21swapper/307:50:343
116755399141,8cyclictest0-21swapper/312:15:323
116755399141,8cyclictest0-21swapper/311:00:323
116755399133,6cyclictest0-21swapper/308:15:383
116755399132,5cyclictest0-21swapper/308:40:343
116755399131,7cyclictest0-21swapper/308:20:363
116755399131,7cyclictest0-21swapper/307:11:223
116755399122,6cyclictest0-21swapper/307:15:413
116755399122,5cyclictest0-21swapper/309:00:323
116755399121,6cyclictest0-21swapper/308:50:363
116755399121,6cyclictest0-21swapper/307:30:363
116755399111,8cyclictest0-21swapper/308:30:283
116755399111,7cyclictest0-21swapper/308:55:353
11675539993,5cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional