You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-18 - 18:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Sat Jul 18, 2026 12:43:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
383187299566,38cyclictest3965737-21ssh11:50:432
3831870995326,20cyclictest12650irq/32-eth010:10:470
3831870994940,7cyclictest3858810-21kworker/u8:0+xprtiod09:00:010
3831870994637,5cyclictest3886273-21latency_hist09:40:020
3831870994637,5cyclictest3886273-21latency_hist09:40:020
383187099452,34cyclictest3962861-21grep11:45:500
3831870994523,15cyclictest151ktimers/007:25:460
3831871994436,5cyclictest3872566-21grep09:15:591
3831871994434,7cyclictest3921045-21grep10:35:581
3831871994434,7cyclictest3898141-21sh10:00:011
3831870994419,16cyclictest151ktimers/009:25:460
3831872994334,6cyclictest3913671-21/usr/sbin/munin10:25:372
3831872994334,6cyclictest3913671-21/usr/sbin/munin10:25:372
3831872994333,7cyclictest421-21dbus-daemon09:31:402
3831872994233,5cyclictest3941722-21grep11:10:512
3831872994233,5cyclictest3875539-21unixbench-2d09:20:562
3831872994231,7cyclictest1001-21mta-sts-daemon12:34:432
3831872994231,7cyclictest1001-21mta-sts-daemon12:34:432
3831870994220,15cyclictest12650irq/32-eth011:35:420
3831870994220,15cyclictest12650irq/32-eth011:35:420
3831872994134,4cyclictest3864817-21sh09:00:012
3831872994133,5cyclictest3946469-21munin-plugin-st11:20:012
3831872994132,6cyclictest420-21avahi-daemon09:30:012
3831871994134,3cyclictest0-21swapper/112:20:011
3831871994132,6cyclictest1003-21mta-sts-daemon12:01:301
383187099414,30cyclictest3983908-21ssh12:20:440
3831870994132,6cyclictest3959288-21gltestperf11:40:370
3831872994034,3cyclictest0-21swapper/212:05:012
3831872994032,5cyclictest3033423-21cups-browsed12:10:022
3831872994030,7cyclictest420-21avahi-daemon09:19:222
3831870994032,5cyclictest427-21polkitd12:00:010
3831870994032,5cyclictest3946471-21awk11:20:010
3831870994032,5cyclictest0-21swapper/008:50:440
3831870994031,6cyclictest3860363-21systemd-run08:45:010
3831872993931,5cyclictest3887757-21tail09:40:572
3831872993926,10cyclictest3983718-21latency12:20:392
3831871993931,5cyclictest3874613-21fschecks_time09:20:361
3831871993930,6cyclictest420-21avahi-daemon10:06:221
3831871993929,6cyclictest421-21dbus-daemon11:20:011
3831871993925,8cyclictest3935576-21sh11:00:461
3831870993927,8cyclictest3033423-21cups-browsed12:00:300
3831870993920,15cyclictest12650irq/32-eth010:50:510
3831872993830,5cyclictest3982835-21systemd-run12:20:012
3831871993830,5cyclictest3988842-21cron12:30:011
3831871993829,5cyclictest3990116-21processes12:30:491
3831871993829,5cyclictest3990116-21processes12:30:491
383187199382,28cyclictest3905283-21ssh10:10:431
3831870993827,5cyclictest0-21swapper/010:05:430
3831872993730,4cyclictest0-21swapper/210:10:012
3831872993727,7cyclictest3033423-21cups-browsed09:10:342
383187299371,31cyclictest3911348-21sh10:20:422
383187299371,31cyclictest3911348-21sh10:20:422
3831871993728,5cyclictest3961609-21systemd-run11:45:011
3831871993727,6cyclictest3033419-21cupsd10:55:121
3831870993729,5cyclictest485-21in:imuxsock08:00:010
3831870993724,10cyclictest3991859-21cron12:35:010
3831870993724,10cyclictest3991859-21cron12:35:010
3831870993710,21cyclictest665-21snmpd08:20:440
3831872993629,4cyclictest3934408-21sendmail11:00:012
3831872993629,4cyclictest3845340-21sh07:55:012
3831872993628,5cyclictest3910254-21grep10:20:012
3831872993627,6cyclictest265-21systemd-journal07:20:002
3831872993626,7cyclictest3033423-21cups-browsed10:48:572
3831871993629,4cyclictest3929068-21fschecks_count10:50:371
3831871993626,6cyclictest1-21systemd12:20:511
3831871993624,9cyclictest3968690-21needreboot11:55:401
3831870993626,7cyclictest3873933-21systemd-run09:20:010
3831870993623,9cyclictest0-21swapper/010:55:450
3831870993620,11cyclictest151ktimers/007:15:400
3831872993527,5cyclictest3865174-21aten_r2power_fr09:00:342
3831872993527,5cyclictest3852860-21sendmail08:20:002
3831872993527,5cyclictest3852860-21sendmail08:20:002
3831872993527,4cyclictest1000-21mta-sts-daemon08:05:482
3831872993527,4cyclictest1000-21mta-sts-daemon08:05:482
3831872993526,7cyclictest3988852-21grep12:30:012
3831872993526,5cyclictest0-21swapper/210:35:482
383187199353,24cyclictest3923630-21ssh10:40:501
3831871993528,4cyclictest3936198-21kworker/u8:2-rpciod11:10:331
3831871993527,5cyclictest3895120-21ssh09:55:011
3831871993526,7cyclictest3846835-21/usr/sbin/munin08:00:021
3831871993526,6cyclictest3965148-21aten_r2power_fr11:50:321
383187099353,27cyclictest3992953-21sh12:35:430
3831870993528,4cyclictest3840976-21latency_hist07:40:020
3831870993516,17cyclictest14-21ksoftirqd/011:30:490
3831870993516,17cyclictest14-21ksoftirqd/011:30:490
3831871993428,3cyclictest0-21swapper/108:40:011
3831871993427,3cyclictest0-21swapper/111:30:441
3831871993427,3cyclictest0-21swapper/111:30:441
3831871993426,5cyclictest3964817-21latency_hist11:50:021
3831871993426,5cyclictest1002-21mta-sts-daemon09:31:511
383187099346,22cyclictest3887178-21sshd09:40:440
383187099343,12cyclictest3893204-21ssh09:50:420
3831870993425,6cyclictest0-21swapper/008:10:460
3831870993425,6cyclictest0-21swapper/008:10:460
3831870993425,5cyclictest0-21swapper/009:50:010
3831870993425,4cyclictest0-21swapper/011:51:310
383187099342,26cyclictest3896250-21ssh09:55:450
383187099342,26cyclictest12650irq/32-eth012:25:480
383187099342,24cyclictest265-21systemd-journal10:35:440
3831872993328,2cyclictest0-21swapper/210:05:012
3831872993326,5cyclictest0-21swapper/208:00:012
3831872993325,5cyclictest3951149-21unixbench_multi11:25:542
3831872993323,7cyclictest3033423-21cups-browsed10:44:222
383187299332,26cyclictest3890221-21ssh09:45:452
3831872993319,8cyclictest3033423-21cups-browsed08:00:512
3831871993326,4cyclictest0-21swapper/109:10:461
3831871993326,3cyclictest0-21swapper/110:45:471
3831871993324,6cyclictest3859370-21fschecks_count08:40:351
3831871993324,5cyclictest3033423-21cups-browsed08:25:031
3831870993327,4cyclictest3914926-21grep10:25:560
3831870993327,4cyclictest3914926-21grep10:25:560
3831870993325,5cyclictest3908585-21sendmail_mailtr10:15:520
3831870993324,3cyclictest3870860-21cron09:15:010
3831870993314,15cyclictest151ktimers/008:05:450
3831870993314,15cyclictest151ktimers/008:05:450
383187299328,17cyclictest1002-21mta-sts-daemon08:45:472
3831872993227,2cyclictest0-21swapper/210:15:012
3831872993224,5cyclictest3858842-21systemctl08:40:012
3831872993224,5cyclictest3033423-21cups-browsed11:09:412
3831872993224,3cyclictest0-21swapper/211:20:512
3831872993220,9cyclictest3962491-21latency11:45:392
383187299321,24cyclictest3917600-21ssh10:30:502
383187299321,24cyclictest3917600-21ssh10:30:502
3831871993225,4cyclictest3866257-21unixbench_singl09:00:551
3831871993224,5cyclictest3911634-21mailstats10:20:511
3831871993224,5cyclictest3911634-21mailstats10:20:511
3831871993224,5cyclictest3863354-21systemd-run08:55:011
3831871993223,6cyclictest0-21swapper/109:40:441
383187099325,5cyclictest171rcu_preempt07:50:450
3831870993225,4cyclictest3943457-21cron11:15:010
383187099322,10cyclictest3981192-21cat12:15:540
3831870993217,10cyclictest3874974-21ssh09:20:430
3831870993215,9cyclictest3974964-21sh12:05:480
3831872993124,3cyclictest0-21swapper/211:40:002
3831872993124,3cyclictest0-21swapper/211:40:002
3831872993123,5cyclictest265-21systemd-journal10:51:102
3831872993120,8cyclictest3970927-21latency_hist12:00:022
3831872993120,7cyclictest3033423-21cups-browsed07:30:442
3831872993118,10cyclictest351ktimers/208:10:422
3831872993118,10cyclictest351ktimers/208:10:422
3831872993114,11cyclictest1003-21mta-sts-daemon09:55:482
3831871993126,2cyclictest0-21swapper/110:30:451
3831871993126,2cyclictest0-21swapper/110:30:451
3831871993126,2cyclictest0-21swapper/110:15:441
3831871993125,3cyclictest0-21swapper/111:25:011
3831871993125,3cyclictest0-21swapper/109:25:541
3831871993124,5cyclictest3956624-21ntp_kernel_pll_11:35:431
3831871993124,5cyclictest3956624-21ntp_kernel_pll_11:35:431
3831871993124,4cyclictest3033419-21cupsd07:45:361
3831871993124,4cyclictest1000-21mta-sts-daemon08:30:441
3831871993123,5cyclictest3898865-21fschecks_time10:00:351
3831871993123,5cyclictest3033423-21cups-browsed07:25:361
3831871993121,7cyclictest3833368-21systemd-run07:15:011
3831870993123,4cyclictest0-21swapper/010:50:010
383187099312,23cyclictest3950649-21ntpq11:25:450
383187099312,23cyclictest3938511-21ssh11:05:420
383187099312,11cyclictest3911407-21ntpq10:20:450
383187099312,11cyclictest3911407-21ntpq10:20:450
3831870993114,14cyclictest151ktimers/009:00:450
3831872993025,2cyclictest0-21swapper/209:40:002
3831872993025,2cyclictest0-21swapper/209:40:002
3831872993022,5cyclictest3979805-21systemd-run12:15:012
3831872993022,5cyclictest3961609-21systemd-run11:45:012
3831872993019,8cyclictest3936128-21head11:00:562
3831871993025,3cyclictest3867476-21kworker/u8:15-xprtiod09:40:001
3831871993025,3cyclictest3867476-21kworker/u8:15-xprtiod09:40:001
3831871993023,4cyclictest3914112-21kernelversion10:25:391
3831871993023,4cyclictest3914112-21kernelversion10:25:391
383187199302,24cyclictest3838770-21sshd07:30:451
3831871993021,6cyclictest420-21avahi-daemon08:10:011
3831871993021,6cyclictest420-21avahi-daemon08:10:011
3831870993022,5cyclictest1-21systemd08:20:010
3831870993022,5cyclictest1-21systemd08:20:010
383187099302,20cyclictest3917417-21sh10:30:440
383187099302,20cyclictest3917417-21sh10:30:440
3831870993018,8cyclictest3934810-21/usr/sbin/munin11:00:350
3831872992924,2cyclictest0-21swapper/209:55:012
3831872992921,5cyclictest3841397-21fschecks_time07:40:362
3831872992918,9cyclictest351ktimers/208:40:492
383187199299,10cyclictest3950691-21ssh11:25:471
3831871992923,3cyclictest0-21swapper/109:45:561
3831871992920,6cyclictest420-21avahi-daemon12:11:191
3831871992920,6cyclictest0-21swapper/107:40:011
3831871992919,6cyclictest0-21swapper/111:09:411
3831870992923,2cyclictest0-21swapper/010:00:420
3831870992922,4cyclictest3835392-21fschecks_count07:20:350
3831870992921,4cyclictest0-21swapper/008:35:010
3831870992920,6cyclictest3880518-21aten2.4-expect09:30:340
3831870992920,4cyclictest0-21swapper/010:40:460
383187099291,22cyclictest3977946-21ntpq12:10:480
3831872992820,5cyclictest1-21systemd07:26:442
3831871992823,2cyclictest0-21swapper/112:40:011
3831871992821,4cyclictest3834856-21munin-plugin-st07:20:001
3831871992820,5cyclictest3851283-21unixbench_singl08:10:561
3831871992820,5cyclictest3851283-21unixbench_singl08:10:561
3831871992820,5cyclictest1-21systemd07:55:011
3831871992819,7cyclictest3851856-21fschecks_count08:15:361
3831871992819,7cyclictest3851856-21fschecks_count08:15:361
3831870992822,4cyclictest3925478-21kworker/u8:8-rpciod11:20:420
3831870992822,4cyclictest3820770-21kworker/u8:6+rpciod07:35:010
383187099282,19cyclictest3855279-21sshd08:25:440
3831870992821,5cyclictest12650irq/32-eth009:05:500
3831872992721,4cyclictest3831869-21cyclictest11:30:442
3831872992721,4cyclictest3831869-21cyclictest11:30:442
3831872992720,4cyclictest3867728-21unixbench_multi09:05:542
3831872992720,4cyclictest3862366-21fschecks_time08:50:372
3831872992718,5cyclictest3992704-21kernelversion12:35:392
3831871992720,4cyclictest3974190-21aten_r2power_en12:05:321
3831870992719,6cyclictest3858830-21cron08:40:010
3831872992619,4cyclictest3854349-21cron08:25:002
3831872992618,5cyclictest3033423-21cups-browsed08:31:192
3831872992615,7cyclictest1-21systemd07:15:012
3831871992621,2cyclictest0-21swapper/107:45:011
3831871992618,5cyclictest3853964-21systemctl08:20:511
3831870992617,5cyclictest12650irq/32-eth008:00:450
3831870992616,7cyclictest3832729-21ntp_kernel_pll_07:10:450
3831872992517,4cyclictest0-21swapper/207:35:402
3831871992519,3cyclictest0-21swapper/109:10:011
3831871992418,3cyclictest1002-21mta-sts-daemon07:20:041
3831871992411,10cyclictest3863813-21df08:55:361
383187099241,9cyclictest3841164-21aten_r2power_en07:40:320
383187099241,11cyclictest3842877-21cut07:45:360
383187199232,17cyclictest623-21sshd08:45:451
3831872992216,4cyclictest3831869-21cyclictest07:20:482
3831872992214,5cyclictest3854751-21tail08:25:332
3831871992215,4cyclictest1000-21mta-sts-daemon08:00:561
3831870992215,4cyclictest3861850-21idleruntime-cro08:50:010
3831870992214,5cyclictest265-21systemd-journal07:10:010
3831870992214,5cyclictest265-21systemd-journal07:10:010
383187399213,10cyclictest0-21swapper/310:25:493
383187399213,10cyclictest0-21swapper/310:25:493
383187399211,10cyclictest0-21swapper/312:05:443
383187299212,14cyclictest3843257-21ntpq07:45:442
383187399192,8cyclictest0-21swapper/309:35:413
383187399192,8cyclictest0-21swapper/309:35:413
383187399192,7cyclictest0-21swapper/309:00:453
383187399191,10cyclictest0-21swapper/309:55:433
383187399184,5cyclictest0-21swapper/310:55:493
383187399182,9cyclictest0-21swapper/310:15:483
383187399182,8cyclictest0-21swapper/312:00:493
383187399182,8cyclictest0-21swapper/310:40:473
383187399182,8cyclictest0-21swapper/309:45:453
3831872991812,3cyclictest0-21swapper/207:10:022
3831872991812,3cyclictest0-21swapper/207:10:022
383187399172,8cyclictest0-21swapper/311:50:443
383187399172,8cyclictest0-21swapper/311:10:423
383187399172,8cyclictest0-21swapper/311:05:463
383187399172,7cyclictest0-21swapper/312:30:433
383187399172,7cyclictest0-21swapper/312:30:433
383187399172,7cyclictest0-21swapper/312:25:463
383187399172,7cyclictest0-21swapper/309:30:413
383187399171,8cyclictest0-21swapper/311:40:463
383187399171,7cyclictest0-21swapper/307:50:483
383187399163,8cyclictest0-21swapper/312:10:493
383187399163,8cyclictest0-21swapper/311:30:423
383187399163,8cyclictest0-21swapper/311:30:423
383187399162,8cyclictest0-21swapper/310:50:513
383187399162,7cyclictest0-21swapper/311:45:433
383187399162,7cyclictest0-21swapper/308:15:473
383187399162,7cyclictest0-21swapper/308:15:473
383187399161,9cyclictest0-21swapper/310:30:423
383187399161,9cyclictest0-21swapper/310:30:423
383187399161,8cyclictest0-21swapper/308:45:453
383187399161,7cyclictest0-21swapper/311:25:433
383187399161,7cyclictest0-21swapper/309:10:483
3831871991610,3cyclictest0-21swapper/107:10:011
3831871991610,3cyclictest0-21swapper/107:10:011
383187399154,6cyclictest0-21swapper/310:00:483
383187399153,6cyclictest0-21swapper/311:20:483
383187399153,6cyclictest0-21swapper/308:35:453
383187399152,8cyclictest0-21swapper/310:45:533
383187399152,7cyclictest0-21swapper/310:05:443
383187399151,8cyclictest0-21swapper/312:20:423
383187399151,7cyclictest0-21swapper/307:35:403
383187399142,7cyclictest0-21swapper/310:10:443
383187399142,7cyclictest0-21swapper/308:00:443
383187399141,8cyclictest0-21swapper/311:00:463
383187399141,8cyclictest0-21swapper/310:20:453
383187399141,8cyclictest0-21swapper/310:20:453
383187399141,8cyclictest0-21swapper/309:15:443
383187399141,6cyclictest0-21swapper/309:05:423
383187399141,6cyclictest0-21swapper/308:30:423
383187399141,6cyclictest0-21swapper/308:20:423
383187399141,6cyclictest0-21swapper/307:40:443
383187399134,6cyclictest0-21swapper/310:35:443
383187399133,7cyclictest0-21swapper/309:50:483
383187399133,6cyclictest0-21swapper/311:55:423
383187399133,6cyclictest0-21swapper/311:40:013
383187399133,6cyclictest0-21swapper/311:40:013
383187399132,7cyclictest0-21swapper/307:50:023
383187399132,6cyclictest0-21swapper/307:30:443
383187399131,7cyclictest0-21swapper/312:15:483
383187399131,7cyclictest0-21swapper/309:40:433
383187399131,7cyclictest0-21swapper/308:55:423
383187399131,6cyclictest0-21swapper/309:25:343
383187399131,6cyclictest0-21swapper/308:40:413
383187399131,6cyclictest0-21swapper/308:05:433
383187399131,6cyclictest0-21swapper/308:05:433
383187399131,6cyclictest0-21swapper/307:55:423
383187399131,6cyclictest0-21swapper/307:20:433
383187399123,6cyclictest0-21swapper/309:20:423
383187399123,5cyclictest0-21swapper/311:15:363
383187399122,6cyclictest0-21swapper/308:25:443
383187399122,5cyclictest0-21swapper/308:50:443
383187399122,5cyclictest0-21swapper/308:10:443
383187399122,5cyclictest0-21swapper/308:10:443
383187399112,6cyclictest0-21swapper/312:35:343
383187399112,4cyclictest0-21swapper/307:15:403
383187399111,6cyclictest0-21swapper/307:10:453
383187399101,5cyclictest0-21swapper/307:25:313
38318739982,5cyclictest0-21swapper/307:10:013
38318739982,5cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional