You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-28 - 12:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Tue Apr 28, 2026 00:43:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3951387995944,10cyclictest261-21systemd-journal00:30:071
3951387995944,10cyclictest261-21systemd-journal00:30:071
3951388995849,6cyclictest1-21systemd21:10:012
3951388995849,6cyclictest1-21systemd21:10:012
3951387995444,6cyclictest4042514-21idleruntime-cro22:20:011
3951386995315,33cyclictest3988723-21ssh21:10:470
3951386995146,3cyclictest4150698-21grep00:35:010
3951386995146,3cyclictest4150698-21grep00:35:010
3951387995039,5cyclictest0-21swapper/120:25:431
3951387994940,6cyclictest4054439-21latency_hist22:35:021
3951388994838,4cyclictest0-21swapper/223:35:462
3951387994839,6cyclictest421-21dbus-daemon00:00:011
3951387994839,6cyclictest3972409-21systemd-run20:20:011
3951387994838,7cyclictest4142682-21true00:25:011
3951386994639,4cyclictest4062415-21systemd-run22:45:010
3951386994635,8cyclictest421-21dbus-daemon19:45:010
3951386994635,8cyclictest421-21dbus-daemon19:45:010
3951388994540,2cyclictest0-21swapper/221:15:442
3951388994538,4cyclictest4082351-21munin-plugin-st23:10:012
3951388994536,6cyclictest4050539-21latency_hist22:30:022
3951387994535,7cyclictest659-21snmpd23:30:521
3951387994535,7cyclictest659-21snmpd23:30:521
3951387994535,6cyclictest420-21avahi-daemon21:27:211
395138699454,33cyclictest4115756-21ssh23:50:440
3951387994424,4cyclictest171rcu_preempt21:35:431
3951386994435,6cyclictest1017-21sendmail-mta23:25:430
3951386994429,12cyclictest4119593-21missed_timers23:55:380
3951388994336,3cyclictest0-21swapper/222:55:462
3951388994335,5cyclictest3966382-21sendmail20:00:012
3951388994334,5cyclictest421-21dbus-daemon21:15:012
3951387994326,13cyclictest421-21dbus-daemon22:55:021
3951387994326,13cyclictest421-21dbus-daemon22:55:021
3951388994232,6cyclictest4015411-21latency_hist21:45:022
3951387994232,6cyclictest3996821-21proc_pri21:20:511
3951387994231,7cyclictest2147018-21cups-browsed21:45:311
3951387994231,7cyclictest2147018-21cups-browsed21:45:311
3951387994229,10cyclictest4043809-21awk22:20:501
3951388994132,5cyclictest0-21swapper/223:20:442
395138799412,30cyclictest4048336-21ssh22:25:581
3951386994132,7cyclictest4102419-21munin-plugin-st23:35:010
3951386994132,7cyclictest4102419-21munin-plugin-st23:35:010
3951388994033,4cyclictest3973879-21debian-sa120:25:012
3951388994032,5cyclictest4118555-21systemd-run23:55:012
3951387994033,3cyclictest0-21swapper/123:50:431
3951387994033,3cyclictest0-21swapper/121:40:341
395138799403,32cyclictest4083684-21ssh23:10:481
3951387994030,7cyclictest2147018-21cups-browsed22:45:431
3951387994010,13cyclictest28-21ksoftirqd/121:30:451
3951388993927,9cyclictest4127976-21gcc00:05:482
395138899392,30cyclictest4055719-21ssh22:35:472
395138799393,29cyclictest659-21snmpd00:15:501
3951387993932,3cyclictest4130625-21kworker/u8:7-xprtiod00:10:011
3951387993930,6cyclictest3978385-21idleruntime-cro20:40:011
3951387993930,6cyclictest2147018-21cups-browsed23:07:441
3951387993930,6cyclictest0-21swapper/123:05:011
3951387993930,5cyclictest3992506-21latency21:15:451
3951388993829,6cyclictest1-21systemd22:15:532
3951387993831,4cyclictest4154713-21ssh00:40:001
3951387993830,5cyclictest3991436-21grep21:15:001
395138699385,27cyclictest4071595-21ssh22:55:440
3951386993832,4cyclictest3954391-21sendmail19:20:010
3951386993831,5cyclictest421-21dbus-daemon00:10:010
3951386993831,4cyclictest4038605-21cron22:15:010
3951386993830,5cyclictest3999938-21aten_r2power_en21:25:320
3951386993826,10cyclictest3987377-21diskmemload22:25:370
395138699382,29cyclictest4028210-21ssh22:00:470
3951386993820,14cyclictest151ktimers/020:20:420
3951388993730,4cyclictest4100267-21grep23:30:532
3951388993730,4cyclictest4100267-21grep23:30:532
395138899372,26cyclictest4139967-21ssh00:20:472
3951387993730,3cyclictest0-21swapper/122:40:011
3951387993728,5cyclictest0-21swapper/123:25:451
3951387993727,7cyclictest600-21gmain22:00:011
3951387993727,7cyclictest600-21gmain22:00:011
3951387993726,8cyclictest3979891-21sh20:45:011
3951386993732,3cyclictest4138654-21sendmail00:20:010
3951386993724,7cyclictest261-21systemd-journal20:40:010
3951388993629,4cyclictest3976844-21kworker/u8:2+xprtiod20:40:012
3951388993629,4cyclictest0-21swapper/219:20:012
3951388993628,5cyclictest4078376-21systemd-run23:05:012
3951388993628,4cyclictest4017122-21unixbench_multi21:45:572
3951388993628,4cyclictest4017122-21unixbench_multi21:45:572
3951388993627,6cyclictest4030925-21latency_hist22:05:022
3951388993626,6cyclictest2147018-21cups-browsed22:12:182
3951388993626,6cyclictest2147018-21cups-browsed21:50:172
3951388993626,6cyclictest2147018-21cups-browsed21:50:172
3951387993630,3cyclictest0-21swapper/123:20:011
3951387993627,5cyclictest0-21swapper/123:00:001
3951387993623,10cyclictest3956692-21needreboot19:25:411
3951387993620,11cyclictest261-21systemd-journal00:03:211
3951387993620,11cyclictest261-21systemd-journal00:03:211
395138799361,28cyclictest4111704-21sh23:45:441
3951386993629,5cyclictest420-21avahi-daemon21:15:150
3951386993626,7cyclictest3999383-21cron21:25:010
3951386993626,6cyclictest1054-21mta-sts-daemon00:35:160
395138699362,27cyclictest4067804-21grep22:50:510
395138699362,27cyclictest4067804-21grep22:50:510
3951388993527,4cyclictest0-21swapper/200:10:492
3951388993526,6cyclictest2147018-21cups-browsed20:14:202
3951388993525,8cyclictest4031833-21sh22:05:392
3951387993529,3cyclictest0-21swapper/122:05:411
3951387993528,5cyclictest3951385-21cyclictest23:40:531
3951387993526,6cyclictest421-21dbus-daemon21:55:011
3951387993526,6cyclictest421-21dbus-daemon21:55:011
3951386993529,3cyclictest4094419-21kworker/u8:13-xprtiod23:45:350
3951386993529,3cyclictest0-21swapper/022:35:390
3951386993526,7cyclictest3944120-21kworker/u8:2+xprtiod19:50:010
3951388993428,3cyclictest0-21swapper/223:25:432
3951388993428,3cyclictest0-21swapper/200:00:012
3951388993426,5cyclictest4126636-21ssh00:05:012
3951388993426,5cyclictest4126636-21ssh00:05:012
3951388993426,5cyclictest4043650-21needreboot22:20:442
3951388993426,3cyclictest0-21swapper/222:50:472
3951388993426,3cyclictest0-21swapper/222:50:472
3951388993426,3cyclictest0-21swapper/222:30:442
3951388993426,3cyclictest0-21swapper/221:25:412
3951388993425,6cyclictest4059887-21mailstats22:40:512
3951388993425,5cyclictest0-21swapper/200:35:422
3951388993424,7cyclictest4122567-21cupsd00:15:122
3951388993422,9cyclictest3968686-21needreboot20:05:452
395138799343,25cyclictest4036045-21ssh22:10:481
3951387993427,4cyclictest1051-21mta-sts-daemon23:35:091
3951387993424,6cyclictest4027515-21head22:00:341
3951387993423,8cyclictest3967141-21missed_timers20:00:401
3951387993420,3cyclictest171rcu_preempt00:10:451
3951386993428,4cyclictest4008195-21fschecks_time21:35:370
3951386993425,7cyclictest0-21swapper/023:20:010
3951386993425,6cyclictest0-21swapper/022:20:010
3951386993413,16cyclictest12450irq/33-eth000:25:500
3951387993326,3cyclictest0-21swapper/121:00:001
3951387993326,3cyclictest0-21swapper/121:00:001
3951387993325,5cyclictest4060224-21unixbench_multi22:40:551
3951387993325,5cyclictest3952208-21/usr/sbin/munin19:10:411
3951387993318,3cyclictest171rcu_preempt19:40:451
3951387993318,3cyclictest171rcu_preempt19:40:451
3951387993315,3cyclictest171rcu_preempt00:25:461
3951386993320,10cyclictest12350irq/32-eth021:50:490
3951386993320,10cyclictest12350irq/32-eth021:50:490
3951386993317,10cyclictest12350irq/32-eth023:10:440
395138699331,27cyclictest659-21snmpd20:30:460
3951388993224,5cyclictest4007448-21latency_hist21:35:022
3951388993224,3cyclictest0-21swapper/223:40:472
3951388993223,6cyclictest3976894-21systemd-run20:35:012
3951388993223,6cyclictest2147015-21cupsd22:46:272
3951388993221,8cyclictest421-21dbus-daemon00:35:022
3951388993221,8cyclictest421-21dbus-daemon00:35:022
395138799325,4cyclictest171rcu_preempt23:20:471
3951386993224,5cyclictest1-21systemd19:55:020
395138699322,26cyclictest4004616-21ssh21:30:440
395138699322,25cyclictest659-21snmpd23:35:460
3951386993221,8cyclictest12350irq/32-eth021:55:460
3951386993221,8cyclictest12350irq/32-eth021:55:460
395138699321,23cyclictest4131854-21ssh00:10:440
395138899314,22cyclictest4024319-21sh21:55:462
395138899314,22cyclictest4024319-21sh21:55:462
3951388993125,3cyclictest0-21swapper/223:20:012
3951388993123,5cyclictest4007909-21/usr/sbin/munin21:35:332
3951388993123,5cyclictest0-21swapper/220:55:412
3951388993123,5cyclictest0-21swapper/220:55:412
3951388993123,4cyclictest3952218-21ntp_kernel_err19:10:412
3951388993122,6cyclictest4054390-21kworker/u8:15-nfsiod23:10:062
395138799318,17cyclictest428-21mta-sts-daemon21:00:431
395138799318,17cyclictest428-21mta-sts-daemon21:00:431
3951387993122,6cyclictest3970856-21cron20:15:011
395138699313,22cyclictest4123915-21ssh00:00:470
395138699313,22cyclictest4123915-21ssh00:00:470
3951386993125,4cyclictest4142655-21cron00:25:010
3951386993124,5cyclictest3957059-21switchtime19:25:540
3951386993124,5cyclictest2147018-21cups-browsed23:20:120
395138699312,23cyclictest4079648-21ssh23:05:480
395138699311,25cyclictest4107736-21sh23:40:440
3951388993022,5cyclictest3960363-21grep19:40:002
3951387993024,3cyclictest0-21swapper/120:00:011
3951386993023,5cyclictest2147018-21cups-browsed20:49:460
3951386993022,5cyclictest4007385-21kworker/u8:0+xprtiod21:50:020
3951386993022,5cyclictest4007385-21kworker/u8:0+xprtiod21:50:020
3951386993020,8cyclictest0-21swapper/021:40:340
3951388992922,4cyclictest4112391-21unixbench_singl23:45:582
3951388992922,3cyclictest0-21swapper/221:00:422
3951388992922,3cyclictest0-21swapper/221:00:422
3951388992921,5cyclictest3982858-21sh20:55:002
3951388992921,5cyclictest3982858-21sh20:55:002
3951388992920,7cyclictest3951390-21sh19:10:012
395138799296,17cyclictest428-21mta-sts-daemon20:20:471
3951387992923,3cyclictest0-21swapper/119:55:011
3951387992921,5cyclictest3980323-21cpu20:45:361
3951387992921,4cyclictest0-21swapper/121:10:011
3951387992921,4cyclictest0-21swapper/121:10:011
3951387992911,4cyclictest171rcu_preempt19:30:451
395138699299,16cyclictest4032017-21ntpq22:05:430
3951388992822,3cyclictest0-21swapper/221:20:562
3951387992821,4cyclictest1056-21mta-sts-daemon19:35:481
3951387992820,5cyclictest518-21rs:main1
3951387992820,5cyclictest1-21systemd19:22:481
395138699283,19cyclictest12350irq/32-eth019:35:480
3951386992824,2cyclictest0-21swapper/019:55:540
3951386992823,2cyclictest0-21swapper/020:50:400
3951386992823,2cyclictest0-21swapper/020:50:400
3951386992820,6cyclictest439-21systemd-logind20:45:010
3951386992820,6cyclictest4063249-21/usr/sbin/munin22:45:400
3951386992814,12cyclictest4050870-21acpi22:30:330
3951388992719,5cyclictest4143483-21fschecks_count00:25:372
395138799276,12cyclictest421-21dbus-daemon20:50:411
395138799276,12cyclictest421-21dbus-daemon20:50:411
3951387992719,4cyclictest0-21swapper/120:30:531
3951387992712,2cyclictest171rcu_preempt19:45:471
3951386992721,3cyclictest0-21swapper/023:00:590
3951386992720,4cyclictest3984284-21head20:55:540
3951386992720,4cyclictest3984284-21head20:55:540
3951386992719,6cyclictest0-21swapper/020:15:020
3951386992710,11cyclictest12350irq/32-eth022:20:530
3951388992619,4cyclictest3964852-21cron19:55:012
3951388992618,5cyclictest3974984-21mailstats20:25:522
3951388992618,5cyclictest2147018-21cups-browsed20:20:012
3951386992620,4cyclictest3971926-21wc20:15:510
3951386992614,9cyclictest0-21swapper/020:25:550
3951388992520,2cyclictest0-21swapper/220:50:002
3951388992518,3cyclictest0-21swapper/220:40:482
3951388992517,5cyclictest1-21systemd19:25:022
3951386992517,6cyclictest0-21swapper/019:20:350
3951386992514,7cyclictest151ktimers/021:05:450
3951386992514,7cyclictest151ktimers/021:05:450
3951388992420,2cyclictest0-21swapper/220:05:012
3951388992418,3cyclictest0-21swapper/219:40:562
3951388992418,3cyclictest0-21swapper/219:40:562
3951388992417,5cyclictest0-21swapper/219:45:552
395138799245,3cyclictest171rcu_preempt19:10:021
3951387992415,3cyclictest171rcu_preempt19:15:461
3951386992417,5cyclictest2147018-21cups-browsed21:00:340
3951386992417,5cyclictest2147018-21cups-browsed21:00:340
3951386992416,6cyclictest695-21cron20:00:030
3951386992316,5cyclictest3957911-21gltestperf19:30:360
395138899222,15cyclictest3958290-21ntpq19:30:442
3951388992216,3cyclictest0-21swapper/219:30:012
395138699222,16cyclictest3952262-21ntpq19:10:430
3951386992217,3cyclictest3957393-21kworker/u8:5+rpciod20:05:540
395138999182,10cyclictest0-21swapper/322:55:483
3951386991814,2cyclictest0-21swapper/019:10:020
395138999172,9cyclictest0-21swapper/322:40:423
395138999172,8cyclictest0-21swapper/322:30:493
395138999172,8cyclictest0-21swapper/322:10:463
395138999163,7cyclictest0-21swapper/321:45:513
395138999163,7cyclictest0-21swapper/321:45:513
395138999163,7cyclictest0-21swapper/321:10:343
395138999162,9cyclictest0-21swapper/323:20:563
395138999162,8cyclictest0-21swapper/323:05:483
395138999162,8cyclictest0-21swapper/322:15:523
395138999162,8cyclictest0-21swapper/321:35:503
395138999162,8cyclictest0-21swapper/300:15:503
395138999162,7cyclictest0-21swapper/322:35:453
395138999162,5cyclictest0-21swapper/300:00:443
395138999162,5cyclictest0-21swapper/300:00:443
395138999161,9cyclictest0-21swapper/322:50:443
395138999161,9cyclictest0-21swapper/322:50:443
395138999161,8cyclictest0-21swapper/300:05:493
395138999154,7cyclictest0-21swapper/321:30:443
395138999153,7cyclictest0-21swapper/323:35:433
395138999152,8cyclictest0-21swapper/323:30:443
395138999152,8cyclictest0-21swapper/323:30:443
395138999152,7cyclictest0-21swapper/323:10:483
395138999152,7cyclictest0-21swapper/322:20:493
395138999152,7cyclictest0-21swapper/321:05:423
395138999152,7cyclictest0-21swapper/321:05:423
395138999152,6cyclictest0-21swapper/300:10:423
395138999152,5cyclictest0-21swapper/320:35:433
395138999151,8cyclictest0-21swapper/323:40:433
395138999151,7cyclictest0-21swapper/321:50:433
395138999151,7cyclictest0-21swapper/321:50:433
395138999151,7cyclictest0-21swapper/319:45:423
395138999151,7cyclictest0-21swapper/319:30:473
395138999143,6cyclictest0-21swapper/323:00:453
395138999142,8cyclictest0-21swapper/320:25:433
395138999142,7cyclictest0-21swapper/323:55:413
395138999142,7cyclictest0-21swapper/323:25:463
395138999142,7cyclictest0-21swapper/323:15:463
395138999142,7cyclictest0-21swapper/322:05:333
395138999142,7cyclictest0-21swapper/321:55:463
395138999142,7cyclictest0-21swapper/321:55:463
395138999142,5cyclictest0-21swapper/319:20:023
395138999141,7cyclictest0-21swapper/300:20:383
395138999141,6cyclictest0-21swapper/319:25:433
395138999134,7cyclictest0-21swapper/321:25:413
395138999133,6cyclictest0-21swapper/321:20:383
395138999133,5cyclictest0-21swapper/322:25:363
395138999132,7cyclictest0-21swapper/323:50:443
395138999132,7cyclictest0-21swapper/322:00:073
395138999132,6cyclictest0-21swapper/321:15:343
395138999132,6cyclictest0-21swapper/319:50:433
395138999132,5cyclictest0-21swapper/321:00:403
395138999132,5cyclictest0-21swapper/321:00:403
395138999132,5cyclictest0-21swapper/319:10:433
395138999131,8cyclictest0-21swapper/322:45:453
395138999131,8cyclictest0-21swapper/300:35:373
395138999131,7cyclictest0-21swapper/321:40:423
395138999131,6cyclictest0-21swapper/320:50:483
395138999131,6cyclictest0-21swapper/320:50:483
395138999131,6cyclictest0-21swapper/320:45:443
395138999131,6cyclictest0-21swapper/319:40:453
395138999131,6cyclictest0-21swapper/319:40:453
395138999131,6cyclictest0-21swapper/300:25:373
395138999122,6cyclictest0-21swapper/320:55:423
395138999122,6cyclictest0-21swapper/320:55:423
395138999122,6cyclictest0-21swapper/319:40:013
395138999121,7cyclictest0-21swapper/323:45:443
395138999121,6cyclictest0-21swapper/320:10:443
395138999121,6cyclictest0-21swapper/319:55:423
395138999112,5cyclictest0-21swapper/320:20:353
395138999112,4cyclictest0-21swapper/319:20:403
395138999111,6cyclictest0-21swapper/300:30:073
395138999111,6cyclictest0-21swapper/300:30:073
395138999111,5cyclictest0-21swapper/320:05:353
395138999110,5cyclictest0-21swapper/320:40:463
395138999104,4cyclictest0-21swapper/320:30:423
395138999102,4cyclictest0-21swapper/320:15:433
395138999101,5cyclictest0-21swapper/320:00:373
39513899991,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional