You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-27 - 16:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Mon Jul 27, 2026 12:43:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
398677099493,37cyclictest4023969-21ssh09:10:520
3986772994638,5cyclictest4113434-21ssh11:40:012
3986772994635,7cyclictest3196310-21cups-browsed11:05:462
3986771994637,6cyclictest4101504-21idleruntime-cro11:20:011
3986771994637,6cyclictest4101504-21idleruntime-cro11:20:011
3986772994234,5cyclictest3994270-21systemd-run07:35:012
3986771994234,5cyclictest4062178-21sh10:15:011
3986771994234,5cyclictest3196310-21cups-browsed08:25:021
3986771994233,6cyclictest1-21systemd12:00:121
3986772994133,5cyclictest4062176-21systemd-run10:15:012
3986771994134,4cyclictest1002-21mta-sts-daemon11:10:521
3986771994032,5cyclictest4018239-21cron08:55:011
3986771994028,8cyclictest4063601-21switchtime10:15:571
3986770994032,5cyclictest1-21systemd10:00:010
3986772993931,5cyclictest4056247-21latency_hist10:05:032
3986772993928,8cyclictest4110451-21sh11:35:012
3986771993932,4cyclictest4149296-21systemctl12:40:011
3986771993932,4cyclictest4022752-21diskmemload09:50:021
3986771993932,3cyclictest0-21swapper/110:35:521
3986771993931,5cyclictest3196310-21cups-browsed11:47:281
3986770993932,4cyclictest4018772-21fschecks_time08:55:410
3986770993930,6cyclictest4137327-21grep12:20:010
3986772993831,4cyclictest3992784-21systemd-run07:30:012
3986772993827,4cyclictest0-21swapper/210:05:532
3986770993831,4cyclictest4122380-21systemd-run11:55:020
3986770993831,4cyclictest4122380-21systemd-run11:55:020
3986770993831,4cyclictest4090315-21latency11:00:420
3986770993831,4cyclictest4016778-21systemd-run08:50:010
3986770993830,5cyclictest4077828-21aten_r2power_en10:40:370
3986770993829,6cyclictest421-21dbus-daemon12:01:130
3986770993828,4cyclictest4062886-21fschecks_time10:15:400
3986770993826,5cyclictest0-21swapper/007:15:500
3986772993729,5cyclictest4096753-21proc_pri11:10:552
3986772993727,6cyclictest3196307-21cupsd09:21:342
3986772993726,8cyclictest4053123-21sh10:00:012
3986771993730,3cyclictest0-21swapper/107:50:011
3986771993729,5cyclictest1-21systemd09:00:011
3986771993727,7cyclictest4030316-21switchtime09:21:001
3986771993726,7cyclictest3196310-21cups-browsed10:44:071
3986772993628,5cyclictest421-21dbus-daemon11:05:012
3986772993625,8cyclictest4141297-21missed_timers12:25:442
3986771993629,4cyclictest4140528-21latency_hist12:25:031
3986771993629,4cyclictest4140528-21latency_hist12:25:031
3986771993628,5cyclictest1-21systemd09:15:011
3986770993631,3cyclictest4040978-21munin-run09:40:000
3986770993628,5cyclictest1-21systemd09:35:020
3986770993628,5cyclictest1-21systemd09:35:020
3986772993528,2cyclictest0-21swapper/209:45:472
3986772993527,5cyclictest4119960-21aten_r2power_vo11:50:402
3986772993527,5cyclictest4119960-21aten_r2power_vo11:50:402
3986772993527,5cyclictest3196310-21cups-browsed09:29:282
3986772993526,6cyclictest3196310-21cups-browsed07:50:442
3986771993528,4cyclictest4122357-21grep11:55:011
3986771993528,4cyclictest4122357-21grep11:55:011
3986771993528,4cyclictest4015295-21latency_hist08:45:021
3986771993527,5cyclictest3991262-21cron07:25:011
3986771993526,6cyclictest420-21avahi-daemon09:19:301
3986770993530,3cyclictest4098494-21kworker/u8:4-xprtiod11:15:010
3986770993524,8cyclictest4050066-21sh09:55:010
3986770993524,7cyclictest0-21swapper/012:30:420
398677099351,29cyclictest4108596-21ssh11:30:500
3986772993426,5cyclictest4085051-21unixbench_multi10:51:012
3986772993425,6cyclictest439-21systemd-logind07:20:022
3986771993427,4cyclictest4001769-21idleruntime-cro08:00:011
3986771993427,4cyclictest1002-21mta-sts-daemon12:12:041
3986771993426,5cyclictest4111151-21sh11:35:411
3986771993426,5cyclictest4000119-21tail07:50:561
3986771993426,5cyclictest3992784-21systemd-run07:30:011
3986770993428,3cyclictest0-21swapper/010:58:120
3986770993426,6cyclictest4139071-21unixbench_singl12:21:000
3986770993426,6cyclictest4139071-21unixbench_singl12:21:000
3986770993423,4cyclictest4093913-21cat11:05:580
398677099342,26cyclictest4033031-21ssh09:25:490
3986770993419,11cyclictest12650irq/32-eth012:25:530
398677099340,29cyclictest4081454-21ssh10:45:470
398677099340,29cyclictest4081454-21ssh10:45:470
3986772993327,3cyclictest0-21swapper/212:00:492
3986772993325,5cyclictest4074466-21latency_hist10:35:032
3986772993325,5cyclictest4063863-21head10:15:592
3986772993324,7cyclictest4014580-21needreboot08:40:472
3986772993324,6cyclictest3196310-21cups-browsed11:29:402
398677199333,25cyclictest4072498-21ssh10:30:511
3986771993327,3cyclictest0-21swapper/107:20:011
3986771993326,4cyclictest4054805-21kworker/u8:0-xprtiod11:20:371
3986771993326,4cyclictest4054805-21kworker/u8:0-xprtiod11:20:371
3986771993326,4cyclictest4040970-21cron09:40:001
3986771993324,5cyclictest4135202-21kernelversion12:15:441
3986771993323,7cyclictest1000-21mta-sts-daemon09:30:531
3986771993323,7cyclictest1000-21mta-sts-daemon09:30:531
3986771993322,8cyclictest4020628-21ntp_kernel_pll_09:00:511
398677199332,26cyclictest4054329-21ssh10:00:531
398677099333,5cyclictest171rcu_preempt12:10:490
3986770993326,5cyclictest694-21cron10:55:000
3986772993227,2cyclictest0-21swapper/212:00:002
3986772993226,3cyclictest0-21swapper/212:10:012
3986772993225,5cyclictest0-21swapper/210:20:352
3986772993225,4cyclictest4133011-21grep12:11:002
3986772993225,4cyclictest4089514-21latency_hist11:00:022
3986772993224,5cyclictest421-21dbus-daemon09:20:022
3986772993224,5cyclictest4149499-21latency_hist12:40:022
3986772993224,5cyclictest3196307-21cupsd11:25:012
3986772993224,5cyclictest3196307-21cupsd11:25:012
3986772993223,5cyclictest4114258-21/usr/sbin/munin11:40:432
398677199326,22cyclictest4084551-21rm10:50:511
3986771993226,3cyclictest0-21swapper/112:25:581
3986771993224,5cyclictest4022752-21diskmemload09:40:551
3986771993224,5cyclictest1002-21mta-sts-daemon08:15:461
3986771993222,6cyclictest421-21dbus-daemon10:25:031
3986771993215,11cyclictest4089859-21aten_r2power_cu11:00:361
398677099323,19cyclictest12650irq/32-eth011:25:490
3986770993227,2cyclictest0-21swapper/010:30:020
3986770993226,4cyclictest4006273-21systemd-run08:15:010
3986770993224,5cyclictest4116406-21systemd-run11:45:010
398677099322,24cyclictest4066401-21ssh10:20:500
3986770993221,8cyclictest3990583-21needreboot07:20:440
398677099321,4cyclictest0-21swapper/010:05:490
3986772993125,3cyclictest0-21swapper/209:30:562
3986772993125,3cyclictest0-21swapper/209:30:562
3986772993122,5cyclictest4074769-21aten2.4-expect10:35:382
398677199313,24cyclictest4022222-21sshd09:05:531
3986771993120,8cyclictest4068688-21/usr/sbin/munin10:25:551
3986771993119,9cyclictest4125348-21munin-plugin-st12:00:001
3986770993126,2cyclictest0-21swapper/010:10:520
3986770993125,3cyclictest4113428-21awk11:40:010
3986770993124,4cyclictest4101498-21sendmail11:20:010
3986770993124,4cyclictest4101498-21sendmail11:20:010
3986770993121,7cyclictest4045713-21sed09:46:000
3986770993118,10cyclictest4029406-21aten_r2power_po09:20:390
398677299303,22cyclictest4081553-21ssh10:45:502
398677299303,22cyclictest4081553-21ssh10:45:502
3986772993023,4cyclictest4010888-21kworker/u8:5-rpciod09:35:402
3986772993022,5cyclictest4024399-21grep09:11:002
3986772993022,4cyclictest4078547-21diskmemload10:40:512
3986772993021,7cyclictest4048480-21switchtime09:50:552
3986772993020,7cyclictest4020405-21kernelversion09:00:462
3986772993017,9cyclictest421-21dbus-daemon07:40:492
3986771993025,2cyclictest0-21swapper/110:10:021
3986771993024,3cyclictest0-21swapper/112:08:571
3986771993023,4cyclictest4146333-21latency_hist12:35:011
398677199302,22cyclictest4032999-21ssh09:25:471
3986771993021,6cyclictest4003189-21grep08:00:591
3986771993020,7cyclictest4115825-21sshd11:43:121
3986770993024,2cyclictest0-21swapper/010:40:010
3986770993023,5cyclictest4119373-21munin-plugin-st11:50:020
3986770993023,4cyclictest4004793-21latency_hist08:10:020
398677099302,23cyclictest4026948-21ssh09:15:490
3986770993021,6cyclictest3196310-21cups-browsed12:08:220
398677099300,18cyclictest12650irq/32-eth007:35:470
3986772992923,3cyclictest4095700-21kworker/u8:3-xprtiod11:20:012
3986772992923,3cyclictest4095700-21kworker/u8:3-xprtiod11:20:012
3986772992923,3cyclictest0-21swapper/209:00:012
3986772992922,4cyclictest1000-21mta-sts-daemon10:29:192
3986772992921,5cyclictest4138530-21ntp_offset12:20:502
3986772992921,5cyclictest4138530-21ntp_offset12:20:502
3986772992921,5cyclictest3988197-21unixbench_singl07:11:012
3986772992921,5cyclictest3196310-21cups-browsed12:16:502
398677299291,6cyclictest171rcu_preempt12:30:472
398677199293,22cyclictest4048156-21ssh09:50:451
3986771992923,3cyclictest0-21swapper/110:45:471
3986771992923,3cyclictest0-21swapper/110:45:471
3986771992922,5cyclictest0-21swapper/111:00:021
3986771992921,5cyclictest421-21dbus-daemon07:10:011
3986771992921,5cyclictest4095662-21latency_hist11:10:031
3986771992921,5cyclictest4009794-21fschecks_time08:25:411
3986771992920,7cyclictest4108474-21needreboot11:30:451
3986770992923,4cyclictest3992636-21tail07:25:580
3986770992923,4cyclictest1001-21mta-sts-daemon11:58:440
3986770992922,4cyclictest1000-21mta-sts-daemon11:20:410
3986770992922,4cyclictest1000-21mta-sts-daemon11:20:410
3986770992921,5cyclictest4053587-21aten_r2power_en10:00:380
3986770992920,7cyclictest4017563-21needreboot08:50:440
3986770992913,11cyclictest151ktimers/007:45:540
398677099291,23cyclictest4008766-21sshd08:21:020
3986772992821,4cyclictest4004755-21latency_hist08:10:012
3986772992820,5cyclictest3995751-21munin-run07:40:012
398677299281,23cyclictest4042216-21ssh09:40:522
3986771992823,2cyclictest0-21swapper/111:25:431
398677199282,22cyclictest4051155-21rm09:55:461
3986771992820,5cyclictest4013749-21cron08:40:011
3986770992822,4cyclictest4147026-21ssh12:35:410
3986770992820,5cyclictest4001733-21cron08:00:010
3986770992820,5cyclictest12650irq/32-eth009:40:520
3986772992720,4cyclictest0-21swapper/207:25:012
3986772992719,5cyclictest3997807-21gltestperf07:45:432
3986771992720,4cyclictest4006273-21systemd-run08:15:011
3986771992719,5cyclictest3995639-21unixbench_multi07:36:001
3986770992723,2cyclictest0-21swapper/010:35:010
3986770992720,4cyclictest4022758-21munin-plugin-st09:10:010
3986770992718,6cyclictest3999564-21needreboot07:50:430
3986770992718,3cyclictest0-21swapper/007:45:010
3986772992618,5cyclictest4001733-21cron08:00:012
3986772992615,7cyclictest3196310-21cups-browsed08:20:012
398677299261,20cyclictest4117610-21ssh11:45:522
3986771992619,4cyclictest4016697-21unixbench_singl08:46:021
3986771992618,5cyclictest3988269-21systemd-run07:15:011
3986771992616,5cyclictest3996025-21aten_r2power_cu07:40:361
398677299257,13cyclictest4013282-21wc08:35:532
3986772992518,4cyclictest4001789-21kworker/u8:11-xprtiod08:10:252
3986772992517,5cyclictest4003144-21tail08:00:592
3986772992516,7cyclictest4022749-21latency_hist09:10:032
398677199252,17cyclictest4004780-21munin-run08:10:021
3986771992519,3cyclictest0-21swapper/107:30:431
3986771992518,3cyclictest0-21swapper/108:30:521
3986770992519,4cyclictest1000-21mta-sts-daemon08:25:550
3986770992516,7cyclictest4013741-21cron08:40:010
3986772992418,2cyclictest0-21swapper/208:35:022
3986772992417,4cyclictest4008238-21cpuspeed08:20:392
3986772992415,6cyclictest4017551-21grep08:50:432
398677099242,17cyclictest3196310-21cups-browsed08:00:500
3986770992416,6cyclictest4007494-21timerandwakeup08:15:560
3986770992416,6cyclictest3988248-21latency_hist07:15:030
3986772992316,5cyclictest0-21swapper/208:50:022
3986770992317,3cyclictest665-21snmpd08:41:220
3986770992316,5cyclictest1001-21mta-sts-daemon08:30:570
3986770992316,3cyclictest0-21swapper/009:00:530
3986772992217,2cyclictest0-21swapper/208:25:592
3986772992115,3cyclictest0-21swapper/207:10:022
398677399202,9cyclictest0-21swapper/311:35:523
398677399202,10cyclictest0-21swapper/310:30:463
398677399201,9cyclictest0-21swapper/310:40:553
398677399192,9cyclictest0-21swapper/310:50:533
398677399191,12cyclictest0-21swapper/309:55:493
398677099191,15cyclictest3993615-21ntpq07:30:460
398677099191,14cyclictest3986727-21munin-run07:10:010
398677399183,10cyclictest0-21swapper/311:05:463
398677399182,9cyclictest0-21swapper/312:30:473
398677399182,9cyclictest0-21swapper/311:10:513
398677399182,9cyclictest0-21swapper/309:15:483
398677399182,8cyclictest0-21swapper/311:40:473
398677399182,8cyclictest0-21swapper/309:50:343
398677399182,8cyclictest0-21swapper/309:40:373
398677399182,7cyclictest0-21swapper/308:10:513
398677399182,10cyclictest0-21swapper/311:50:393
398677399182,10cyclictest0-21swapper/311:50:393
398677399182,10cyclictest0-21swapper/310:20:513
398677399173,8cyclictest0-21swapper/312:00:523
398677399173,8cyclictest0-21swapper/311:55:433
398677399173,8cyclictest0-21swapper/310:05:473
398677399173,8cyclictest0-21swapper/309:50:023
398677399173,8cyclictest0-21swapper/309:31:023
398677399173,8cyclictest0-21swapper/309:31:023
398677399173,7cyclictest0-21swapper/309:25:483
398677399172,9cyclictest0-21swapper/310:55:503
398677399172,9cyclictest0-21swapper/308:00:503
398677399172,7cyclictest0-21swapper/307:20:483
398677399172,10cyclictest0-21swapper/311:20:593
398677399172,10cyclictest0-21swapper/311:20:593
398677399171,7cyclictest0-21swapper/308:20:543
398677399171,10cyclictest0-21swapper/312:25:423
398677399165,8cyclictest0-21swapper/312:05:403
398677399164,7cyclictest0-21swapper/309:10:533
398677399163,9cyclictest0-21swapper/310:35:433
398677399162,7cyclictest0-21swapper/311:30:413
398677399162,7cyclictest0-21swapper/310:15:383
398677399162,7cyclictest0-21swapper/307:40:493
398677399162,6cyclictest0-21swapper/311:00:363
398677399162,10cyclictest0-21swapper/312:35:423
398677399161,9cyclictest0-21swapper/312:20:463
398677399161,9cyclictest0-21swapper/312:20:463
398677399161,9cyclictest0-21swapper/311:15:493
398677399161,9cyclictest0-21swapper/311:15:493
398677399161,9cyclictest0-21swapper/309:35:483
398677399161,9cyclictest0-21swapper/308:25:483
398677399161,7cyclictest0-21swapper/307:10:503
398677399157,5cyclictest431ktimers/307:10:013
398677399154,4cyclictest0-21swapper/308:45:503
398677399153,7cyclictest0-21swapper/310:00:503
398677399153,5cyclictest0-21swapper/309:00:503
398677399152,8cyclictest0-21swapper/312:15:493
398677399152,8cyclictest0-21swapper/312:10:413
398677399152,8cyclictest0-21swapper/311:25:483
398677399152,8cyclictest0-21swapper/310:46:123
398677399152,8cyclictest0-21swapper/310:46:123
398677399152,8cyclictest0-21swapper/308:35:023
398677399152,7cyclictest0-21swapper/307:15:533
398677399152,6cyclictest0-21swapper/311:45:123
398677399151,9cyclictest0-21swapper/310:10:383
398677399151,8cyclictest0-21swapper/310:25:393
398677399151,7cyclictest0-21swapper/308:05:493
398677399150,7cyclictest0-21swapper/307:30:473
398677399143,5cyclictest0-21swapper/308:50:453
398677399143,5cyclictest0-21swapper/307:35:483
398677399142,9cyclictest0-21swapper/309:20:493
398677399142,7cyclictest0-21swapper/308:15:503
398677399141,7cyclictest0-21swapper/307:45:483
398677399141,6cyclictest0-21swapper/308:35:463
398677399141,6cyclictest0-21swapper/307:25:453
398677399131,8cyclictest0-21swapper/309:05:483
398677399131,6cyclictest0-21swapper/308:40:483
398677399131,6cyclictest0-21swapper/307:50:443
398677399122,7cyclictest0-21swapper/309:00:013
398677399122,6cyclictest0-21swapper/307:55:513
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional