You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-12 - 19:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Wed Aug 12, 2026 12:43:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4058141995644,8cyclictest3822-21awk12:00:011
4058140995648,6cyclictest4155184-21cut10:50:010
4058140995447,4cyclictest12650irq/32-eth011:15:570
4058142995041,6cyclictest4191764-21latency_hist11:50:022
4058142995041,6cyclictest4121783-21latency_hist09:55:022
405814199502,35cyclictest5045-21ssh12:00:591
405814199492,13cyclictest4107584-21ssh09:30:561
4058141994921,25cyclictest0-21swapper/109:15:531
4058140994939,7cyclictest4159755-21timerandwakeup10:56:070
4058141994738,6cyclictest4064150-21systemd-run07:30:011
4058141994635,7cyclictest421-21dbus-daemon11:25:581
4058141994635,6cyclictest0-21swapper/109:25:581
4058141994630,12cyclictest0-21swapper/111:40:471
4058141994624,18cyclictest4102016-21unixbench_multi09:21:071
4058142994536,5cyclictest4125765-21missed_timers10:00:522
4058142994437,3cyclictest0-21swapper/210:35:012
4058142994436,5cyclictest421-21dbus-daemon10:55:012
4058142994436,5cyclictest3284321-21cupsd09:20:502
4058141994419,3cyclictest0-21swapper/110:06:011
4058140994430,4cyclictest4067125-21munin-plugin-st07:40:010
4058142994335,5cyclictest4100323-21systemd-run09:20:012
4058142994334,4cyclictest0-21swapper/208:30:012
4058141994335,4cyclictest0-21swapper/109:10:491
4058141994318,3cyclictest0-21swapper/111:36:081
4058141994314,5cyclictest4168257-21kernelversion11:10:501
4058140994337,3cyclictest0-21swapper/009:50:590
4058142994234,5cyclictest12956-21cron12:15:012
4058142994231,8cyclictest4097230-21ssh09:15:012
4058142994231,7cyclictest1002-21mta-sts-daemon08:15:002
4058142994228,6cyclictest421-21dbus-daemon09:40:522
4058141994234,5cyclictest4149127-21systemd-run10:40:011
405814199422,18cyclictest23364-21sh12:31:001
4058141994217,3cyclictest0-21swapper/110:40:561
4058141994216,4cyclictest4083626-21cron08:35:011
4058141994216,4cyclictest4073105-21cron08:00:001
405814199421,15cyclictest4135133-21ntpq10:16:001
4058140994233,5cyclictest12650irq/32-eth012:35:540
4058140994230,8cyclictest12650irq/32-eth008:32:080
4058142994131,6cyclictest3284324-21cups-browsed09:36:482
4058142994131,6cyclictest3284324-21cups-browsed09:36:482
405814299412,30cyclictest4150235-21ssh10:40:542
405814199414,8cyclictest4162466-21ssh11:01:001
405814199413,27cyclictest26393-21ssh12:35:571
4058142994032,5cyclictest25162-21munin-plugin-st12:35:002
4058142994031,6cyclictest6912-21latency_hist12:05:022
405814299402,3cyclictest171rcu_preempt12:20:582
4058141994022,15cyclictest0-21swapper/108:55:011
405814199402,17cyclictest11056-21ssh12:10:551
4058141994016,3cyclictest0-21swapper/109:56:061
4058141994016,3cyclictest0-21swapper/108:01:001
4058140994033,4cyclictest3842-21sendmail-msp12:00:010
4058140994032,5cyclictest4107806-21sendmail_mailqu09:31:020
4058142993931,4cyclictest0-21swapper/209:55:562
4058141993929,6cyclictest420-21avahi-daemon12:18:081
405814199392,14cyclictest4159568-21ps10:56:031
405814199392,11cyclictest428-21mta-sts-daemon09:45:391
405814199391,15cyclictest4165435-21ntpq11:05:531
4058140993930,6cyclictest421-21dbus-daemon12:25:410
4058140993929,6cyclictest4134632-21fschecks_count10:15:490
4058140993926,10cyclictest4076936-21needreboot08:10:520
4058140993922,4cyclictest171rcu_preempt10:05:590
4058140993914,20cyclictest4186855-21ssh11:41:000
4058142993831,4cyclictest4167353-21latency_hist11:10:022
4058142993830,5cyclictest4173436-21grep11:20:012
4058142993829,5cyclictest4176938-21tail11:25:452
4058142993827,8cyclictest4087452-21needreboot08:45:512
405814199382,8cyclictest4131989-21ssh10:10:551
4058141993815,3cyclictest0-21swapper/109:53:331
405814199381,15cyclictest8100-21ntpq12:06:021
405814099383,30cyclictest4192886-21ssh11:50:580
405814099383,29cyclictest4101500-21sh09:20:570
4058140993830,5cyclictest420-21avahi-daemon08:40:020
405814299373,7cyclictest171rcu_preempt08:20:592
4058142993731,3cyclictest0-21swapper/210:35:572
405814199379,9cyclictest20667-21timerwakeupswit12:26:051
405814199374,10cyclictest3284324-21cups-browsed09:44:431
4058141993731,3cyclictest0-21swapper/108:20:001
4058141993729,5cyclictest4068658-21systemd-run07:45:011
405814199372,17cyclictest17163-21ssh12:20:551
405814199371,17cyclictest4141131-21ntpq10:25:571
405814199371,16cyclictest4155933-21forks10:50:501
4058140993732,3cyclictest4182538-21grep11:35:010
4058140993731,4cyclictest4091251-21latency_hist09:00:020
4058140993730,3cyclictest0-21swapper/010:30:550
4058142993629,4cyclictest4193466-21grep11:51:082
4058142993628,4cyclictest0-21swapper/211:55:512
4058142993627,5cyclictest0-21swapper/210:15:482
4058142993626,7cyclictest1003-21mta-sts-daemon12:15:292
4058141993629,4cyclictest1001-21mta-sts-daemon11:23:121
4058141993621,4cyclictest1003-21mta-sts-daemon08:10:491
4058141993617,2cyclictest0-21swapper/110:00:511
4058140993629,4cyclictest4087169-21fschecks_time08:45:470
4058140993628,5cyclictest5520-21unixbench-2d12:01:090
405814299353,26cyclictest4141099-21ssh10:25:562
4058142993527,5cyclictest4186935-21proc_pri11:41:032
4058142993527,5cyclictest4133861-21idleruntime-cro10:15:012
4058142993527,4cyclictest0-21swapper/211:21:112
405814199358,10cyclictest420-21avahi-daemon09:36:491
405814199358,10cyclictest420-21avahi-daemon09:36:491
4058141993527,5cyclictest4173458-21idleruntime-cro11:20:011
4058141993525,7cyclictest4091139-21sendmail09:00:011
4058140993528,4cyclictest10074-21latency_hist12:10:030
4058140993527,5cyclictest421-21dbus-daemon12:15:020
4058140993526,6cyclictest420-21avahi-daemon11:08:310
4058140993525,7cyclictest4064079-21head07:26:070
4058140993524,8cyclictest4150765-21timerwakeupswit10:41:080
4058140993524,6cyclictest151ktimers/007:10:590
4058140993523,3cyclictest171rcu_preempt10:26:030
4058140993513,9cyclictest171rcu_preempt10:21:000
4058142993429,2cyclictest0-21swapper/208:20:002
4058142993425,6cyclictest1-21systemd07:55:022
4058141993426,5cyclictest4139981-21kworker/u8:11+xprtiod10:35:011
405814099343,24cyclictest4110626-21ssh09:35:590
405814099343,24cyclictest4110626-21ssh09:35:590
4058140993428,4cyclictest4170404-21munin-plugin-st11:15:010
4058140993427,5cyclictest421-21dbus-daemon09:45:010
4058140993427,4cyclictest4095840-21unixbench_multi09:11:060
4058140993426,5cyclictest4079658-21sort08:20:500
4058140993423,9cyclictest4158229-21systemd-run10:55:020
4058140993423,8cyclictest14603-21timerwakeupswit12:16:070
4058140993421,10cyclictest0-21swapper/011:35:470
405814099340,4cyclictest171rcu_preempt09:55:550
4058142993327,3cyclictest0-21swapper/212:40:012
4058142993326,3cyclictest0-21swapper/209:10:012
4058142993325,4cyclictest4080654-21kworker/u8:16-xprtiod09:00:012
4058142993325,4cyclictest0-21swapper/208:32:052
4058142993322,7cyclictest7811-21kernelversion12:05:552
4058141993322,8cyclictest4069532-21ntp_kernel_pll_07:45:581
4058140993327,4cyclictest4061164-21systemd-run07:20:010
405814099332,23cyclictest23360-21ssh12:30:590
4058140993320,10cyclictest0-21swapper/011:20:570
4058142993228,2cyclictest0-21swapper/210:25:012
4058142993226,3cyclictest0-21swapper/209:30:012
4058142993226,3cyclictest0-21swapper/207:40:012
4058142993225,4cyclictest4084655-21proc_pri08:36:002
4058142993224,5cyclictest1001-21mta-sts-daemon10:07:422
4058142993220,7cyclictest4162566-21ssh11:01:062
405814199325,8cyclictest1001-21mta-sts-daemon11:30:561
4058141993224,5cyclictest4080641-21grep08:25:011
405814199321,16cyclictest4192950-21ntpq11:51:001
405814099325,12cyclictest171rcu_preempt10:10:580
4058140993226,4cyclictest4164311-21latency_hist11:05:020
4058140993212,5cyclictest14-21ksoftirqd/010:35:550
4058142993125,3cyclictest0-21swapper/211:00:012
4058142993124,5cyclictest694-21cron07:25:012
4058142993124,4cyclictest1000-21mta-sts-daemon07:35:022
4058142993121,8cyclictest4074038-21ntp_kernel_pll_08:01:052
4058142993120,3cyclictest0-21swapper/211:31:002
4058141993121,8cyclictest4082147-21sh08:30:011
4058140993125,3cyclictest0-21swapper/008:01:000
4058140993123,3cyclictest0-21swapper/009:25:580
405814299307,16cyclictest1003-21mta-sts-daemon07:10:572
4058142993023,4cyclictest4169055-21tail11:11:062
4058142993023,3cyclictest0-21swapper/207:26:002
4058142993022,6cyclictest0-21swapper/209:05:022
4058142993020,8cyclictest4075444-21needreboot08:05:542
4058142993020,8cyclictest4075444-21needreboot08:05:542
4058141993022,5cyclictest4139996-21systemd10:25:021
4058141993022,5cyclictest4093099-21cpuspeed09:05:461
4058141993019,8cyclictest1-21systemd11:49:041
4058140993024,4cyclictest4177008-21aten_r2power_fr11:25:460
4058140993024,4cyclictest4062658-21systemd-run07:25:010
4058140993022,5cyclictest151ktimers/008:40:590
4058140993020,4cyclictest1001-21mta-sts-daemon08:05:520
4058140993020,4cyclictest1001-21mta-sts-daemon08:05:520
4058140993016,3cyclictest171rcu_preempt09:00:590
4058140993014,3cyclictest171rcu_preempt09:05:550
4058140993011,16cyclictest0-21swapper/012:22:390
405814299293,18cyclictest4183816-21ssh11:36:012
4058142992923,3cyclictest0-21swapper/207:45:012
4058142992922,4cyclictest0-21swapper/208:00:012
4058142992922,3cyclictest0-21swapper/209:46:032
4058142992917,5cyclictest1003-21mta-sts-daemon10:48:432
4058141992923,3cyclictest0-21swapper/108:45:011
4058140992923,4cyclictest4127787-21munin-plugin-st10:05:010
4058140992916,3cyclictest171rcu_preempt08:51:030
4058140992914,3cyclictest171rcu_preempt07:31:000
405814099290,3cyclictest4187594-21kworker/0:0+events_power_efficient11:47:240
4058142992821,4cyclictest4088592-21cpuspeed08:50:492
4058142992820,5cyclictest20844-21unixbench_singl12:26:062
4058142992820,5cyclictest0-21swapper/207:20:012
4058142992819,6cyclictest750-21ntpd08:40:462
4058141992822,3cyclictest0-21swapper/108:40:011
4058141992822,3cyclictest0-21swapper/107:15:011
4058141992821,3cyclictest0-21swapper/110:46:061
4058141992818,6cyclictest420-21avahi-daemon08:05:471
4058141992818,6cyclictest420-21avahi-daemon08:05:471
4058140992822,3cyclictest0-21swapper/007:41:110
4058140992821,4cyclictest421-21dbus-daemon08:00:010
4058140992814,3cyclictest171rcu_preempt09:45:570
4058142992716,7cyclictest4106883-21/usr/sbin/munin09:30:522
4058141992720,4cyclictest4065496-21tail07:31:061
4058140992714,4cyclictest171rcu_preempt08:25:540
4058140992712,3cyclictest171rcu_preempt09:15:550
4058141992618,5cyclictest4070640-21fschecks_count07:50:481
4058141992617,6cyclictest420-21avahi-daemon08:46:021
4058140992616,7cyclictest4078462-21ntp_kernel_err08:15:540
4058141992519,3cyclictest0-21swapper/109:05:011
4058140992515,2cyclictest4043245-21kworker/u8:4+xprtiod07:50:020
4058140992511,4cyclictest171rcu_preempt07:51:000
4058141992417,4cyclictest4065895-21aten_r2power_cu07:35:441
4058140992420,2cyclictest0-21swapper/007:10:010
405814299233,14cyclictest694-21cron07:46:012
405814199232,15cyclictest4062007-21ntpq07:21:001
4058141992317,3cyclictest0-21swapper/107:15:491
405814399193,6cyclictest0-21swapper/310:01:003
405814399183,9cyclictest0-21swapper/310:56:023
405814299181,12cyclictest4058095-21/usr/sbin/munin07:10:022
405814399174,7cyclictest0-21swapper/312:06:003
405814399174,7cyclictest0-21swapper/309:10:583
405814399172,8cyclictest0-21swapper/310:26:043
405814399172,8cyclictest0-21swapper/308:32:053
405814399172,7cyclictest0-21swapper/311:25:573
405814399172,6cyclictest0-21swapper/307:55:533
405814399171,9cyclictest0-21swapper/312:01:013
405814399171,8cyclictest0-21swapper/311:01:053
405814199177,8cyclictest4058144-21sh07:10:011
405814399164,6cyclictest0-21swapper/311:40:553
405814399163,5cyclictest0-21swapper/311:50:553
405814399162,8cyclictest0-21swapper/310:35:543
405814399161,8cyclictest0-21swapper/310:46:043
405814399154,8cyclictest0-21swapper/309:56:003
405814399154,6cyclictest0-21swapper/310:11:003
405814399154,5cyclictest0-21swapper/312:21:003
405814399153,7cyclictest0-21swapper/309:46:003
405814399152,8cyclictest0-21swapper/312:30:553
405814399152,8cyclictest0-21swapper/312:15:573
405814399152,8cyclictest0-21swapper/310:40:563
405814399152,7cyclictest0-21swapper/309:15:533
405814399152,6cyclictest0-21swapper/309:01:073
405814399151,9cyclictest0-21swapper/312:00:013
405814399151,9cyclictest0-21swapper/310:30:473
405814399151,8cyclictest0-21swapper/312:25:563
405814399151,6cyclictest0-21swapper/308:15:573
405814399151,6cyclictest0-21swapper/307:45:563
405814399144,7cyclictest0-21swapper/309:50:593
405814399143,8cyclictest0-21swapper/311:16:033
405814399143,6cyclictest0-21swapper/312:10:493
405814399143,6cyclictest0-21swapper/309:36:063
405814399143,6cyclictest0-21swapper/309:36:063
405814399142,7cyclictest0-21swapper/312:35:523
405814399142,7cyclictest0-21swapper/309:25:543
405814399141,8cyclictest0-21swapper/311:46:013
405814399141,7cyclictest0-21swapper/311:10:563
405814399133,6cyclictest0-21swapper/310:05:483
405814399133,6cyclictest0-21swapper/309:10:013
405814399132,7cyclictest0-21swapper/311:05:453
405814399132,7cyclictest0-21swapper/310:15:573
405814399132,6cyclictest0-21swapper/311:35:513
405814399132,6cyclictest0-21swapper/311:30:553
405814399132,6cyclictest0-21swapper/309:20:553
405814399132,6cyclictest0-21swapper/308:01:003
405814399131,6cyclictest0-21swapper/308:51:053
405814399123,5cyclictest0-21swapper/309:30:503
405814399122,7cyclictest0-21swapper/307:25:583
405814399122,6cyclictest0-21swapper/309:40:483
405814399122,5cyclictest0-21swapper/311:21:033
405814399122,5cyclictest0-21swapper/308:45:473
405814399121,7cyclictest0-21swapper/310:20:543
405814399121,7cyclictest0-21swapper/308:25:563
405814399121,7cyclictest0-21swapper/307:30:593
405814399121,7cyclictest0-21swapper/307:10:543
405814399121,6cyclictest0-21swapper/310:50:473
405814399121,6cyclictest0-21swapper/308:35:563
405814399121,6cyclictest0-21swapper/308:20:563
405814399112,6cyclictest0-21swapper/308:10:013
405814399112,6cyclictest0-21swapper/308:10:013
405814399111,7cyclictest0-21swapper/308:10:563
405814399111,6cyclictest0-21swapper/309:00:023
405814399104,4cyclictest0-21swapper/307:35:553
405814399103,4cyclictest0-21swapper/307:15:563
405814399102,5cyclictest431ktimers/307:10:013
405814399102,5cyclictest0-21swapper/307:50:483
405814399101,7cyclictest0-21swapper/307:25:023
40581439992,5cyclictest0-21swapper/308:45:013
40581439991,4cyclictest0-21swapper/307:45:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional