You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-05 - 07:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Tue May 05, 2026 00:43:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
893774996449,10cyclictest1080647-21unixbench_multi00:16:042
893772995342,4cyclictest990410-21cron22:25:010
893772995229,18cyclictest261-21systemd-journal00:35:460
893772994940,6cyclictest0-21swapper/021:55:450
893772994940,6cyclictest0-21swapper/021:55:450
893773994842,3cyclictest0-21swapper/123:10:011
893773994842,3cyclictest0-21swapper/123:10:011
893772994738,6cyclictest1018479-21sendmail23:00:010
893772994734,5cyclictest421-21dbus-daemon23:50:010
893774994640,3cyclictest0-21swapper/220:55:012
893774994636,5cyclictest0-21swapper/223:50:532
893773994640,3cyclictest986428-21systemd22:20:021
893773994638,5cyclictest0-21swapper/100:35:461
893773994638,4cyclictest420-21avahi-daemon00:34:291
893774994535,6cyclictest933813-21ssh21:15:012
893773994540,3cyclictest0-21swapper/123:55:481
893772994531,11cyclictest0-21swapper/021:25:010
893772994436,5cyclictest3304446-21cups-browsed23:11:290
893772994431,4cyclictest171rcu_preempt23:25:010
893773994335,5cyclictest1027271-21cpu23:10:381
893773994322,4cyclictest0-21swapper/121:10:521
893772994337,3cyclictest0-21swapper/023:35:530
893772994337,3cyclictest0-21swapper/023:35:530
893774994233,6cyclictest1016390-21grep22:56:002
893772994236,4cyclictest1002449-21ssh22:40:000
893772994236,3cyclictest0-21swapper/022:05:480
893773994135,3cyclictest0-21swapper/122:20:481
893772994134,4cyclictest1088683-21unixbench_singl00:26:000
893774994032,5cyclictest1014666-21latency_hist22:55:022
893774994031,6cyclictest1058752-21ls23:50:022
893774994031,5cyclictest991532-21missed_timers22:25:452
893773994035,3cyclictest0-21swapper/121:00:001
893773994033,4cyclictest0-21swapper/122:35:451
893773994032,5cyclictest920687-21unixbench_singl20:35:571
893773994031,6cyclictest421-21dbus-daemon21:23:031
893772994032,5cyclictest420-21avahi-daemon21:13:530
893772994031,6cyclictest3304446-21cups-browsed23:02:300
893772994031,6cyclictest3304446-21cups-browsed23:02:300
893772994014,22cyclictest963517-21ssh21:50:520
893774993931,5cyclictest1-21systemd22:10:022
893774993931,5cyclictest1088579-21unixbench-2d00:26:002
893774993929,6cyclictest1032400-21unixbench-2d23:16:002
893774993926,9cyclictest938955-21missed_timers21:20:432
893774993920,14cyclictest1047874-21sshd23:35:452
893774993920,14cyclictest1047874-21sshd23:35:452
893773993930,7cyclictest911759-21/usr/sbin/munin20:10:021
89377499383,26cyclictest951337-21ssh21:35:532
893774993832,4cyclictest893771-21cyclictest23:10:512
893774993830,5cyclictest893783-21systemd-run19:10:012
893774993829,6cyclictest1-21systemd00:35:012
893774993829,6cyclictest1055-21mta-sts-daemon21:17:512
893774993826,9cyclictest918564-21needreboot20:30:442
893773993832,3cyclictest0-21swapper/123:35:451
893773993832,3cyclictest0-21swapper/123:35:451
893773993831,4cyclictest695-21cron22:15:011
893773993830,5cyclictest1012364-21unixbench_singl22:51:011
893773993828,7cyclictest902751-21munin-run19:40:001
893772993833,3cyclictest151ktimers/019:55:510
893772993828,7cyclictest1010488-21latency_hist22:50:020
893772993821,7cyclictest12450irq/33-eth023:55:500
893772993821,3cyclictest0-21swapper/023:15:420
893774993730,4cyclictest974319-21cron22:05:002
893774993727,7cyclictest986375-21munin-plugin-st22:20:012
893774993727,6cyclictest1038612-21sh23:25:012
893773993730,4cyclictest0-21swapper/100:10:541
893773993728,6cyclictest420-21avahi-daemon23:52:341
893773993728,6cyclictest3304441-21cupsd23:20:501
893772993717,15cyclictest171rcu_preempt23:51:230
893774993629,5cyclictest893771-21cyclictest00:14:572
893774993628,5cyclictest1002419-21cron22:40:002
893774993628,5cyclictest0-21swapper/222:15:012
89377399367,10cyclictest28-21ksoftirqd/119:20:011
893773993630,3cyclictest0-21swapper/121:40:521
893773993629,4cyclictest0-21swapper/120:35:011
893773993624,4cyclictest261rcuc/122:49:261
893773993621,3cyclictest171rcu_preempt21:55:531
893773993621,3cyclictest171rcu_preempt21:55:531
893772993630,4cyclictest1092500-21timerwakeupswit00:31:020
893772993630,4cyclictest1086742-21idleruntime-cro00:25:010
893772993629,4cyclictest439-21systemd-logind20:30:010
893772993617,17cyclictest941888-21kworker/u8:9+rpciod21:30:470
89377299361,28cyclictest1068095-21ssh00:00:510
893774993527,5cyclictest1095349-21ssh00:35:372
893774993526,7cyclictest659-21snmpd00:24:142
893773993526,6cyclictest1054-21mta-sts-daemon19:20:491
89377399352,28cyclictest1080003-21ssh00:15:511
893772993529,4cyclictest3304446-21cups-browsed22:40:390
893772993528,4cyclictest12350irq/32-eth021:15:460
893772993528,3cyclictest0-21swapper/020:45:500
893772993527,5cyclictest974353-21systemd-run22:05:010
893772993527,5cyclictest421-21dbus-daemon23:07:160
893772993527,5cyclictest421-21dbus-daemon23:07:160
893772993523,9cyclictest12350irq/32-eth023:30:450
893774993428,3cyclictest0-21swapper/221:10:022
893774993427,4cyclictest1022533-21ls23:05:012
893774993427,4cyclictest1022533-21ls23:05:012
893774993425,5cyclictest907515-21aten_r2power_cu19:55:372
893774993422,9cyclictest351ktimers/221:50:462
89377499342,22cyclictest1003811-21ssh22:40:492
893773993428,3cyclictest0-21swapper/122:30:521
893773993428,3cyclictest0-21swapper/122:30:521
893773993427,4cyclictest962114-21ssh21:50:011
893773993426,5cyclictest942572-21aten_r2power_vo21:25:381
893773993426,5cyclictest420-21avahi-daemon21:39:241
893773993426,5cyclictest1054693-21cut23:45:001
893773993426,5cyclictest1046690-21latency_hist23:35:021
893773993425,6cyclictest420-21avahi-daemon22:05:041
893773993422,2cyclictest171rcu_preempt20:00:021
893773993415,15cyclictest421-21dbus-daemon22:59:451
893772993428,4cyclictest916292-21systemd-run20:25:010
893772993428,4cyclictest1071692-21ssh00:05:450
893772993428,3cyclictest0-21swapper/021:26:010
893772993427,4cyclictest0-21swapper/021:50:010
893772993425,6cyclictest0-21swapper/000:15:000
893772993421,10cyclictest0-21swapper/000:15:460
893772993416,3cyclictest171rcu_preempt22:32:100
893772993416,3cyclictest171rcu_preempt22:32:100
893772993410,5cyclictest14-21ksoftirqd/019:25:460
893774993327,3cyclictest0-21swapper/222:20:472
893774993326,4cyclictest1051-21mta-sts-daemon22:30:232
893774993326,4cyclictest1051-21mta-sts-daemon22:30:232
893774993325,5cyclictest923694-21sed20:45:592
893774993325,5cyclictest0-21swapper/200:00:012
89377399332,7cyclictest171rcu_preempt23:45:521
893773993327,3cyclictest0-21swapper/100:30:021
893773993326,3cyclictest0-21swapper/100:20:491
893773993325,5cyclictest925235-21cron20:55:011
893773993325,5cyclictest923861-21latency_hist20:50:021
89377399332,27cyclictest1031870-21ssh23:15:491
893773993310,4cyclictest261rcuc/121:00:461
893772993327,4cyclictest901253-21munin-plugin-st19:35:000
893772993327,4cyclictest901253-21munin-plugin-st19:35:000
893772993325,6cyclictest421-21dbus-daemon21:40:520
893772993325,5cyclictest516-21in:imuxsock19:45:000
893772993322,8cyclictest3304441-21cupsd23:25:480
893772993320,3cyclictest171rcu_preempt23:42:350
893772993310,5cyclictest14-21ksoftirqd/021:05:520
89377499325,3cyclictest171rcu_preempt00:00:532
89377499323,21cyclictest1039891-21ssh23:25:482
893774993226,3cyclictest0-21swapper/219:40:002
893774993224,5cyclictest917651-21tail20:26:022
89377399327,5cyclictest28-21ksoftirqd/122:00:511
89377399324,7cyclictest28-21ksoftirqd/122:40:131
893773993225,3cyclictest0-21swapper/123:00:401
893773993225,3cyclictest0-21swapper/123:00:401
893773993224,2cyclictest171rcu_preempt00:00:421
893773993223,6cyclictest547-21gmain21:15:581
893773993223,6cyclictest1071724-21/usr/sbin/munin00:05:451
893773993223,2cyclictest171rcu_preempt23:28:081
893773993221,4cyclictest261rcuc/119:30:481
893773993221,4cyclictest261rcuc/119:30:481
893773993220,3cyclictest171rcu_preempt20:20:021
89377299323,24cyclictest1011691-21ssh22:50:480
893772993226,4cyclictest911684-21unixbench_singl20:05:580
893772993226,3cyclictest0-21swapper/022:15:410
893772993220,9cyclictest0-21swapper/022:10:580
89377499319,5cyclictest171rcu_preempt21:30:472
89377499319,4cyclictest171rcu_preempt19:25:472
89377499314,23cyclictest1043971-21ssh23:30:502
893774993124,4cyclictest962116-21/usr/sbin/munin21:50:012
893774993124,3cyclictest0-21swapper/222:00:022
893774993124,3cyclictest0-21swapper/222:00:022
893774993123,5cyclictest914790-21systemd-run20:20:012
893774993123,5cyclictest907273-21systemd-run19:55:022
893774993120,8cyclictest753-21ntpd19:45:452
89377399312,24cyclictest963364-21ssh21:50:461
893773993121,7cyclictest421-21dbus-daemon19:10:011
893773993113,10cyclictest915633-21ntpq20:20:461
893772993113,12cyclictest951260-21diskmemload21:35:500
89377499303,8cyclictest0-21swapper/221:25:492
893774993024,3cyclictest0-21swapper/220:15:012
893774993023,4cyclictest1007921-21proc_pri22:45:542
89377499302,23cyclictest1052147-21ssh23:40:532
893774993018,9cyclictest955763-21timerwakeupswit21:40:592
89377399307,5cyclictest28-21ksoftirqd/120:25:441
893773993023,4cyclictest903132-21aten_r2power_po19:40:361
893773993022,5cyclictest990942-21/usr/sbin/munin22:25:421
893773993016,2cyclictest171rcu_preempt20:00:391
89377399301,23cyclictest947235-21ssh21:30:501
89377299308,5cyclictest14-21ksoftirqd/019:20:500
893772993025,3cyclictest926736-21sh21:00:000
893772993023,4cyclictest922408-21latency_hist20:45:020
893772993023,3cyclictest151ktimers/020:00:510
893772993016,10cyclictest421-21dbus-daemon20:50:470
893774992923,3cyclictest0-21swapper/219:45:002
893774992923,3cyclictest0-21swapper/219:20:012
893774992921,6cyclictest421-21dbus-daemon19:35:012
893774992921,6cyclictest421-21dbus-daemon19:35:012
893774992920,6cyclictest915296-21sort20:20:402
89377399294,5cyclictest28-21ksoftirqd/119:10:471
893773992921,6cyclictest907183-21grep19:50:591
893772992923,4cyclictest927283-21fschecks_time21:00:400
89377299291,22cyclictest991690-21rm22:25:490
893774992822,3cyclictest0-21swapper/200:08:592
893774992820,5cyclictest921097-21aten_r2power_fr20:40:382
89377399286,5cyclictest28-21ksoftirqd/119:45:361
893773992820,5cyclictest912282-21fschecks_time20:10:401
893773992811,2cyclictest0-21swapper/119:25:581
89377299288,6cyclictest14-21ksoftirqd/020:35:540
89377299288,5cyclictest14-21ksoftirqd/019:10:500
893772992822,4cyclictest3304446-21cups-browsed19:37:540
893772992822,4cyclictest0-21swapper/020:15:400
89377499276,3cyclictest171rcu_preempt20:40:012
89377499272,20cyclictest1023874-21ssh23:05:512
89377499272,20cyclictest1023874-21ssh23:05:512
893773992721,4cyclictest271ktimers/120:40:481
893773992711,2cyclictest171rcu_preempt21:05:531
89377299278,5cyclictest14-21ksoftirqd/019:10:020
893772992722,3cyclictest913323-21latency_hist20:15:020
89377499269,3cyclictest171rcu_preempt19:10:372
893774992619,4cyclictest923732-21kworker/u8:0-xprtiod21:05:012
893774992618,5cyclictest3304446-21cups-browsed20:05:392
893774992614,2cyclictest36-21ksoftirqd/220:55:462
89377299266,5cyclictest14-21ksoftirqd/019:45:460
89377299264,3cyclictest14-21ksoftirqd/019:50:540
89377299263,5cyclictest14-21ksoftirqd/020:30:540
893772992618,6cyclictest895271-21kworker/u8:4-nfsiod19:20:010
89377499254,4cyclictest171rcu_preempt20:00:512
89377499251,5cyclictest897619-21ntpq19:20:462
89377599203,9cyclictest0-21swapper/321:55:453
89377599203,9cyclictest0-21swapper/321:55:453
89377599192,8cyclictest0-21swapper/322:10:493
89377599182,7cyclictest0-21swapper/322:50:563
89377599172,9cyclictest0-21swapper/323:50:453
89377599162,7cyclictest0-21swapper/323:15:533
89377599161,8cyclictest0-21swapper/300:30:583
89377599161,7cyclictest0-21swapper/323:30:453
89377599152,8cyclictest0-21swapper/322:07:473
89377599152,7cyclictest0-21swapper/323:35:533
89377599152,7cyclictest0-21swapper/323:35:533
89377599152,7cyclictest0-21swapper/323:30:013
89377599152,7cyclictest0-21swapper/322:15:543
89377599151,8cyclictest0-21swapper/321:35:463
89377599151,7cyclictest0-21swapper/321:40:473
89377599151,6cyclictest0-21swapper/320:35:443
89377599144,7cyclictest0-21swapper/319:15:483
89377599142,8cyclictest0-21swapper/323:55:453
89377599142,6cyclictest0-21swapper/323:20:583
89377599142,5cyclictest0-21swapper/300:35:463
89377599141,7cyclictest0-21swapper/323:45:493
89377599141,7cyclictest0-21swapper/321:15:483
89377599141,7cyclictest0-21swapper/320:10:483
89377599131,7cyclictest0-21swapper/323:40:453
89377599131,7cyclictest0-21swapper/323:10:513
89377599131,7cyclictest0-21swapper/322:55:513
89377599131,7cyclictest0-21swapper/322:35:453
89377599131,7cyclictest0-21swapper/322:30:453
89377599131,7cyclictest0-21swapper/322:30:453
89377599131,7cyclictest0-21swapper/322:25:473
89377599131,7cyclictest0-21swapper/321:10:483
89377599131,7cyclictest0-21swapper/300:15:563
89377599131,7cyclictest0-21swapper/300:05:523
89377599131,6cyclictest0-21swapper/321:25:523
89377599131,6cyclictest0-21swapper/321:05:493
89377599131,6cyclictest0-21swapper/320:40:483
89377599131,6cyclictest0-21swapper/320:05:473
89377599131,6cyclictest0-21swapper/319:30:463
89377599131,6cyclictest0-21swapper/319:30:463
89377599131,5cyclictest0-21swapper/321:50:463
89377599122,6cyclictest0-21swapper/300:20:363
89377599121,8cyclictest0-21swapper/323:05:483
89377599121,8cyclictest0-21swapper/323:05:483
89377599121,7cyclictest0-21swapper/323:00:483
89377599121,7cyclictest0-21swapper/323:00:483
89377599121,7cyclictest0-21swapper/322:45:513
89377599121,7cyclictest0-21swapper/322:40:343
89377599121,7cyclictest0-21swapper/321:45:473
89377599121,7cyclictest0-21swapper/321:30:493
89377599121,7cyclictest0-21swapper/321:20:483
89377599121,6cyclictest0-21swapper/320:55:483
89377599121,6cyclictest0-21swapper/320:50:513
89377599121,6cyclictest0-21swapper/320:45:493
89377599121,6cyclictest0-21swapper/320:20:483
89377599121,6cyclictest0-21swapper/319:50:503
89377599121,6cyclictest0-21swapper/300:25:463
89377599121,5cyclictest0-21swapper/321:00:493
89377599121,5cyclictest0-21swapper/319:25:463
89377599111,6cyclictest0-21swapper/322:20:473
89377599111,6cyclictest0-21swapper/322:00:473
89377599111,6cyclictest0-21swapper/320:30:463
89377599111,6cyclictest0-21swapper/319:40:503
89377599111,6cyclictest0-21swapper/300:10:493
89377599111,6cyclictest0-21swapper/300:00:433
89377599111,5cyclictest0-21swapper/320:15:413
89377599111,5cyclictest0-21swapper/320:00:473
89377599111,5cyclictest0-21swapper/319:35:533
89377599111,5cyclictest0-21swapper/319:20:513
89377599101,5cyclictest0-21swapper/319:10:463
8937759991,5cyclictest0-21swapper/320:25:413
8937759991,4cyclictest0-21swapper/319:55:523
8937759991,4cyclictest0-21swapper/319:45:393
8937759981,4cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional