You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-10 - 09:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Fri Jul 10, 2026 00:43:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
371352996232,20cyclictest12650irq/32-eth022:30:430
371352996044,12cyclictest535347-21latency_hist00:40:030
371353995818,27cyclictest395562-21/usr/sbin/munin20:30:291
371353995818,27cyclictest395562-21/usr/sbin/munin20:30:291
371354995647,5cyclictest510865-21ssh00:00:012
371354995647,4cyclictest0-21swapper/223:50:462
371354995344,6cyclictest485-21in:imuxsock22:40:012
371354995244,5cyclictest526090-21systemd-run00:25:012
371352995023,24cyclictest532123-21cron00:35:010
371354994842,3cyclictest0-21swapper/221:20:012
371353994839,6cyclictest2803590-21cups-browsed21:59:421
371352994841,4cyclictest0-21swapper/022:10:430
371354994736,5cyclictest0-21swapper/222:15:442
371353994739,5cyclictest495695-21systemd-run23:35:011
371353994739,5cyclictest495695-21systemd-run23:35:011
371352994726,18cyclictest1000-21mta-sts-daemon22:55:300
371354994639,3cyclictest0-21swapper/223:50:012
371354994636,6cyclictest1-21systemd23:20:022
371354994635,8cyclictest420-21avahi-daemon23:39:002
371353994638,5cyclictest413530-21idleruntime-cro21:20:001
371352994637,7cyclictest151ktimers/000:00:460
371352994633,10cyclictest450110-21munin-plugin-st22:20:010
371353994537,5cyclictest478018-21aten_r2power_vo23:05:301
371352994537,5cyclictest417857-21proc_pri21:25:480
371352994515,27cyclictest502454-21fschecks_time23:45:320
371352994438,4cyclictest510908-21lockfile-create00:00:010
371352994411,10cyclictest414901-21grep21:20:500
371352994332,7cyclictest0-21swapper/022:00:020
371352994321,4cyclictest383333-21cron19:50:010
371352994320,21cyclictest488029-21xargs23:20:490
371354994233,6cyclictest2803590-21cups-browsed21:35:512
371354994231,7cyclictest1000-21mta-sts-daemon23:01:462
37135399424,30cyclictest499856-21ssh23:40:411
371353994234,5cyclictest520087-21latency_hist00:15:021
371353994234,5cyclictest421-21dbus-daemon00:40:021
371353994234,5cyclictest419670-21systemd-run21:30:011
371352994235,4cyclictest442459-21timerandwakeup22:05:490
371352994217,5cyclictest496511-21kernelversion23:35:340
37135499413,31cyclictest478573-21ssh23:05:412
371354994129,9cyclictest383354-21sh19:50:012
371353994133,5cyclictest454826-21unixbench_singl22:25:531
371352994134,4cyclictest407394-21munin-run21:10:010
37135299412,12cyclictest12650irq/32-eth023:05:470
371352994120,2cyclictest0-21swapper/022:40:320
371352994120,2cyclictest0-21swapper/022:40:320
371354994034,3cyclictest0-21swapper/221:45:012
371354994033,3cyclictest0-21swapper/200:20:012
37135499401,32cyclictest530270-21ssh00:30:442
371353994034,3cyclictest0-21swapper/122:20:011
371353994030,5cyclictest0-21swapper/100:25:421
371352994022,4cyclictest518571-21unixbench_multi00:10:520
37135299402,19cyclictest469458-21rm22:50:410
37135499395,23cyclictest408517-21ssh21:10:402
37135499395,23cyclictest408517-21ssh21:10:402
371354993931,5cyclictest432957-21ssh21:50:392
371354993930,5cyclictest529121-21systemd-run00:30:012
37135299399,11cyclictest12650irq/32-eth000:15:370
371352993911,18cyclictest12650irq/32-eth021:40:430
371354993827,7cyclictest420-21avahi-daemon22:11:162
371354993826,6cyclictest457422-21proc_pri22:30:472
37135499382,31cyclictest442035-21ssh22:05:372
371354993823,12cyclictest511290-21/usr/sbin/munin00:00:402
371353993831,4cyclictest1001-21mta-sts-daemon23:17:211
371353993828,8cyclictest378152-21needreboot19:30:351
371353993827,8cyclictest523991-21missed_timers00:20:371
371352993831,4cyclictest440991-21ssh22:05:000
371352993831,4cyclictest440991-21ssh22:05:000
371352993829,6cyclictest420-21avahi-daemon00:25:500
371352993821,4cyclictest1002-21mta-sts-daemon00:23:170
371352993815,16cyclictest12650irq/32-eth023:40:380
37135499374,23cyclictest481695-21ssh23:10:442
371354993729,4cyclictest1002-21mta-sts-daemon21:48:132
371354993729,4cyclictest0-21swapper/222:40:392
371354993729,4cyclictest0-21swapper/222:40:392
371354993728,5cyclictest1002-21mta-sts-daemon00:36:192
371354993728,4cyclictest0-21swapper/223:30:382
371354993728,4cyclictest0-21swapper/223:30:382
371354993726,7cyclictest427-21polkitd21:08:542
37135499372,28cyclictest417855-21rm21:25:482
37135499370,4cyclictest171rcu_preempt20:00:462
371353993730,4cyclictest471364-21cron22:55:001
371353993730,4cyclictest1002-21mta-sts-daemon23:39:511
371353993729,6cyclictest1003-21mta-sts-daemon22:20:491
371352993729,4cyclictest0-21swapper/021:45:380
371352993726,7cyclictest12650irq/32-eth021:50:430
37135499363,25cyclictest420765-21ssh21:30:402
371354993627,6cyclictest0-21swapper/222:25:462
371353993629,5cyclictest371351-21cyclictest22:57:111
371353993629,4cyclictest508360-21aten_r2power_po23:55:301
371353993627,6cyclictest567-21gdbus00:30:491
371353993614,15cyclictest421-21dbus-daemon20:35:421
37135499355,23cyclictest2803590-21cups-browsed19:25:382
371354993528,4cyclictest518631-21unixbench_singl00:10:522
371354993527,5cyclictest514452-21aten_r2power_po00:05:312
371353993529,3cyclictest0-21swapper/121:45:011
371353993528,4cyclictest459887-21fschecks_time22:35:311
371353993528,3cyclictest0-21swapper/100:05:341
371353993526,6cyclictest371358-21true19:10:011
371353993525,5cyclictest1001-21mta-sts-daemon22:02:451
371353993525,5cyclictest1001-21mta-sts-daemon22:02:451
371352993526,7cyclictest466289-21needreboot22:45:370
371352993526,7cyclictest466289-21needreboot22:45:370
371354993427,4cyclictest1003-21mta-sts-daemon19:50:372
371354993426,5cyclictest440995-21cron22:05:002
371354993426,5cyclictest440995-21cron22:05:002
371354993425,6cyclictest439-21systemd-logind19:20:022
371354993425,6cyclictest388364-21fschecks_count20:05:312
371354993425,4cyclictest0-21swapper/223:20:332
37135399343,23cyclictest481566-21ssh23:10:381
371353993426,5cyclictest390836-21munin-plugin-st20:15:011
371353993426,5cyclictest387865-21systemd-run20:05:011
371353993424,3cyclictest1000-21mta-sts-daemon22:43:441
371353993424,3cyclictest1000-21mta-sts-daemon22:43:441
371352993427,5cyclictest1001-21mta-sts-daemon23:15:520
371352993419,8cyclictest12650irq/32-eth023:10:430
371354993325,5cyclictest437953-21sendmail22:00:012
371354993325,5cyclictest407391-21diskmemload23:30:022
371354993325,4cyclictest0-21swapper/223:40:292
371354993324,6cyclictest392330-21cron20:20:012
371354993323,7cyclictest2803590-21cups-browsed22:20:052
371354993322,5cyclictest0-21swapper/222:50:402
371353993327,3cyclictest0-21swapper/122:45:521
371353993327,3cyclictest0-21swapper/122:45:521
371353993326,4cyclictest1000-21mta-sts-daemon23:51:091
371353993326,4cyclictest1000-21mta-sts-daemon21:51:091
371352993328,3cyclictest425782-21systemctl21:40:010
371352993326,4cyclictest1003-21mta-sts-daemon22:20:270
371352993326,3cyclictest0-21swapper/022:35:510
371352993326,3cyclictest0-21swapper/021:13:510
371352993326,3cyclictest0-21swapper/021:13:510
371354993211,12cyclictest421-21dbus-daemon22:45:472
371354993211,12cyclictest421-21dbus-daemon22:45:472
371353993226,3cyclictest0-21swapper/121:40:011
371353993225,4cyclictest502440-21fschecks_count23:45:321
371353993225,4cyclictest409023-21grep21:10:491
371353993225,4cyclictest409023-21grep21:10:491
371353993225,3cyclictest0-21swapper/123:26:541
371353993225,3cyclictest0-21swapper/121:45:381
371353993225,3cyclictest0-21swapper/121:30:341
371353993224,5cyclictest392351-21idleruntime-cro20:20:011
371353993224,5cyclictest382358-21fschecks_count19:45:311
371353993224,4cyclictest511846-21missed_timers00:00:401
371352993225,4cyclictest1003-21mta-sts-daemon21:30:190
371352993224,5cyclictest0-21swapper/023:30:340
371352993224,5cyclictest0-21swapper/023:30:340
371352993224,4cyclictest151ktimers/023:25:400
371354993126,2cyclictest0-21swapper/223:00:002
371354993123,5cyclictest406014-21latency_hist21:05:032
371354993121,4cyclictest0-21swapper/219:40:422
371354993120,8cyclictest420-21avahi-daemon19:40:022
371353993125,3cyclictest0-21swapper/122:15:011
371353993125,3cyclictest0-21swapper/100:15:271
371353993124,3cyclictest0-21swapper/123:00:411
371353993123,4cyclictest0-21swapper/119:40:421
371353993122,6cyclictest0-21swapper/123:20:271
371353993119,9cyclictest456570-21aten_r2power_cu22:30:291
371353993119,9cyclictest1-21systemd21:10:021
371353993111,14cyclictest265-21systemd-journal19:50:441
371352993125,2cyclictest0-21swapper/021:15:380
37135299311,24cyclictest454185-21ssh22:25:390
371354993024,3cyclictest0-21swapper/219:25:012
371354993021,5cyclictest0-21swapper/220:25:012
371352993024,4cyclictest516944-21ssh00:10:000
371352993018,7cyclictest506352-21timerwakeupswit23:50:510
371354992922,4cyclictest1002-21mta-sts-daemon19:55:322
371354992921,5cyclictest395256-21head20:25:502
371354992921,4cyclictest1002-21mta-sts-daemon21:24:472
371354992918,8cyclictest372235-21ntp_kernel_pll_19:10:402
371354992917,9cyclictest396228-21ntp_kernel_err20:30:372
371354992917,9cyclictest396228-21ntp_kernel_err20:30:372
37135399293,21cyclictest442065-21ssh22:05:391
371353992921,5cyclictest1002-21mta-sts-daemon19:55:361
37135299295,10cyclictest12650irq/32-eth020:30:290
37135299295,10cyclictest12650irq/32-eth020:30:290
371352992919,7cyclictest475327-21latency23:00:350
371352992918,8cyclictest402875-21cron20:55:010
371353992820,5cyclictest374343-21munin-plugin-st19:20:011
371352992821,4cyclictest389854-21fschecks_count20:10:330
371352992820,5cyclictest151ktimers/019:40:430
371352992818,8cyclictest376662-21needreboot19:25:360
371352992818,6cyclictest151ktimers/020:45:380
371354992718,4cyclictest0-21swapper/220:45:362
371353992720,3cyclictest0-21swapper/120:20:361
37135399271,21cyclictest414577-21ssh21:20:381
37135299276,16cyclictest428-21mta-sts-daemon19:10:400
371352992713,3cyclictest0-21swapper/019:20:010
37135299271,22cyclictest405407-21ps21:00:500
371352992711,10cyclictest12650irq/32-eth019:30:360
37135499262,19cyclictest2803590-21cups-browsed20:40:402
371353992619,4cyclictest398856-21cpuspeed20:40:281
371353992619,4cyclictest1002-21mta-sts-daemon20:50:401
371353992618,5cyclictest401284-21unixbench_multi20:45:481
371353992612,10cyclictest421-21dbus-daemon19:25:381
371352992613,9cyclictest12650irq/32-eth020:20:390
371352992613,3cyclictest0-21swapper/019:50:280
371353992517,5cyclictest421-21dbus-daemon20:10:011
371353992517,5cyclictest372840-21munin-plugin-st19:15:011
371354992418,4cyclictest398099-21timerandwakeup20:35:482
371354992415,7cyclictest403243-21aten_r2power_po20:55:302
371353992415,6cyclictest405897-21munin-plugin-st21:05:011
37135299243,11cyclictest428-21mta-sts-daemon19:20:350
37135299242,18cyclictest387233-21awk20:00:410
371352992414,7cyclictest627-21lldpd20:05:420
37135299241,17cyclictest665-21snmpd19:55:350
37135499232,16cyclictest402354-21ntp_states20:50:442
37135499232,15cyclictest390186-21ntpq20:10:392
37135399233,16cyclictest1001-21mta-sts-daemon20:25:401
371353992314,6cyclictest375272-21ntpq19:20:391
37135399231,16cyclictest379851-21ps19:35:441
37135299231,19cyclictest404385-21idleruntime-cro21:00:010
37135499222,15cyclictest378312-21ntp_states19:30:432
371353992216,4cyclictest371351-21cyclictest20:59:331
371352992216,4cyclictest391846-21proc_pri20:15:440
371352992215,5cyclictest394736-21ntp_kernel_pll_20:25:380
371352992214,4cyclictest151ktimers/020:40:370
371352992213,5cyclictest151ktimers/019:35:400
371352992013,5cyclictest397408-21uniq20:35:310
371354991913,3cyclictest0-21swapper/219:10:032
37135599183,8cyclictest0-21swapper/322:05:283
37135599182,9cyclictest0-21swapper/300:25:473
37135599181,7cyclictest0-21swapper/321:45:283
371352991810,3cyclictest366053-21kworker/u8:7-nfsiod19:10:020
37135599172,9cyclictest0-21swapper/322:30:403
37135599172,8cyclictest0-21swapper/322:15:523
37135599164,5cyclictest0-21swapper/320:55:413
37135599163,9cyclictest0-21swapper/300:30:503
37135599163,7cyclictest0-21swapper/322:00:463
37135599163,7cyclictest0-21swapper/322:00:463
37135599162,8cyclictest0-21swapper/321:50:433
37135599162,7cyclictest0-21swapper/323:45:383
37135599161,7cyclictest0-21swapper/321:05:403
37135599161,6cyclictest0-21swapper/320:15:373
37135599153,7cyclictest0-21swapper/323:45:013
37135599152,7cyclictest0-21swapper/323:20:393
37135599152,7cyclictest0-21swapper/321:10:443
37135599152,7cyclictest0-21swapper/321:10:443
37135599152,7cyclictest0-21swapper/320:30:293
37135599152,7cyclictest0-21swapper/320:30:293
37135599151,9cyclictest0-21swapper/322:10:433
37135599151,8cyclictest0-21swapper/323:35:363
37135599151,8cyclictest0-21swapper/323:15:413
37135599151,8cyclictest0-21swapper/322:40:403
37135599151,8cyclictest0-21swapper/322:40:403
37135599151,8cyclictest0-21swapper/322:25:423
37135599151,8cyclictest0-21swapper/321:55:333
37135599151,8cyclictest0-21swapper/321:35:383
37135599151,8cyclictest0-21swapper/321:15:493
37135599151,7cyclictest0-21swapper/322:50:373
37135599151,7cyclictest0-21swapper/300:35:283
37135599151,6cyclictest0-21swapper/320:50:413
37135599144,7cyclictest0-21swapper/322:35:313
37135599143,7cyclictest0-21swapper/322:45:523
37135599143,7cyclictest0-21swapper/322:45:523
37135599143,4cyclictest0-21swapper/320:05:393
37135599142,8cyclictest0-21swapper/323:50:423
37135599142,8cyclictest0-21swapper/300:15:373
37135599142,7cyclictest0-21swapper/323:10:373
37135599142,7cyclictest0-21swapper/323:05:293
37135599142,7cyclictest0-21swapper/323:00:533
37135599142,7cyclictest0-21swapper/321:20:383
37135599142,6cyclictest0-21swapper/319:25:383
37135599142,5cyclictest0-21swapper/320:45:373
37135599141,8cyclictest0-21swapper/322:55:463
37135599141,8cyclictest0-21swapper/322:20:393
37135599141,7cyclictest0-21swapper/323:55:353
37135599141,6cyclictest0-21swapper/319:55:363
37135599141,6cyclictest0-21swapper/319:20:363
37135599141,10cyclictest0-21swapper/300:10:313
37135599133,6cyclictest0-21swapper/323:30:373
37135599133,6cyclictest0-21swapper/323:30:373
37135599132,8cyclictest0-21swapper/321:40:323
37135599132,7cyclictest0-21swapper/300:05:493
37135599131,8cyclictest0-21swapper/321:30:323
37135599131,7cyclictest0-21swapper/319:15:283
37135599131,5cyclictest0-21swapper/319:50:363
37135599131,5cyclictest0-21swapper/319:40:363
37135599122,7cyclictest0-21swapper/320:45:013
37135599122,6cyclictest0-21swapper/300:20:343
37135599122,5cyclictest0-21swapper/319:35:383
37135599122,5cyclictest0-21swapper/319:30:323
37135599121,8cyclictest0-21swapper/300:00:393
37135599121,7cyclictest0-21swapper/323:25:313
37135599121,7cyclictest0-21swapper/321:30:023
37135599121,6cyclictest0-21swapper/320:35:363
37135599113,4cyclictest0-21swapper/319:10:443
37135599112,5cyclictest0-21swapper/320:25:013
37135599111,7cyclictest0-21swapper/321:00:283
37135599111,6cyclictest0-21swapper/320:25:393
37135599111,6cyclictest0-21swapper/319:45:393
37135599101,6cyclictest0-21swapper/320:10:333
37135599101,6cyclictest0-21swapper/320:00:313
3713559992,5cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional