You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-14 - 06:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Tue Jul 14, 2026 00:43:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3711733996656,5cyclictest0-21swapper/222:15:432
3711733996656,5cyclictest0-21swapper/222:15:432
3711733995343,6cyclictest3744738-21systemd-run21:00:012
371173399522,42cyclictest3851266-21ssh00:00:402
3711731995243,5cyclictest3748920-21ssh21:10:450
3711731995239,10cyclictest3814077-21munin-run23:00:010
3711731995125,3cyclictest3779057-21ssh22:00:390
3711733995038,7cyclictest3775954-21needreboot21:55:382
3711733995036,7cyclictest1003-21mta-sts-daemon21:25:472
3711732995043,3cyclictest0-21swapper/122:55:011
371173399492,38cyclictest3866240-21ssh00:25:402
3711731994937,6cyclictest151ktimers/022:05:400
3711732994842,3cyclictest0-21swapper/121:15:421
3711732994842,3cyclictest0-21swapper/121:15:421
3711733994638,3cyclictest0-21swapper/200:20:012
3711733994638,3cyclictest0-21swapper/200:20:012
371173399461,39cyclictest3767003-21ssh21:40:402
371173399455,5cyclictest3833251-21ssh23:30:432
371173399455,5cyclictest3833251-21ssh23:30:432
371173399454,31cyclictest3845205-21ssh23:50:392
3711733994535,7cyclictest3750785-21idleruntime-cro21:15:012
3711733994535,6cyclictest3848798-21unixbench_multi23:55:542
3711731994532,10cyclictest3799024-21latency_hist22:35:010
3711731994433,7cyclictest3743211-21cut20:55:010
3711731994430,5cyclictest3871810-21cpuspeed00:35:320
3711733994333,4cyclictest0-21swapper/221:30:392
3711732994324,13cyclictest421-21dbus-daemon22:40:421
3711731994336,4cyclictest3802056-21tr22:40:010
3711731994335,5cyclictest3826122-21grep23:20:010
3711732994234,5cyclictest3771948-21ssh21:50:011
3711732994234,4cyclictest3826122-21grep23:20:011
3711732994230,8cyclictest1961693-21cups-browsed23:47:181
3711731994218,17cyclictest3836319-21ssh23:35:460
3711733994130,7cyclictest420-21avahi-daemon00:05:122
3711733994130,7cyclictest1961693-21cups-browsed23:13:012
3711731994129,9cyclictest3850238-21cups-browsed00:30:410
371173399408,19cyclictest3824234-21ssh23:15:432
371173399401,33cyclictest3764077-21ssh21:35:452
371173399400,8cyclictest3747714-21diskmemload23:40:412
3711732994032,5cyclictest3802061-21rm22:40:011
3711732994031,5cyclictest3776087-21ssh21:55:441
3711732994028,8cyclictest3830623-21timerandwakeup23:25:561
3711732994028,8cyclictest3830623-21timerandwakeup23:25:561
3711731994031,6cyclictest1961693-21cups-browsed21:43:250
3711733993927,9cyclictest3793053-21ssh22:25:012
371173299397,4cyclictest171rcu_preempt23:20:431
371173299397,4cyclictest171rcu_preempt23:20:431
3711732993932,4cyclictest3726704-21grep20:00:011
3711732993931,5cyclictest3747714-21diskmemload23:05:021
3711732993930,6cyclictest3759910-21latency_hist21:30:011
371173299392,31cyclictest3839258-21ssh23:40:431
3711731993932,5cyclictest421-21dbus-daemon21:10:020
3711731993931,5cyclictest421-21dbus-daemon00:05:450
3711733993829,5cyclictest3793596-21cpu22:25:312
371173299384,30cyclictest3773136-21ssh21:50:451
3711732993830,5cyclictest3787034-21systemd-run22:15:021
371173199383,32cyclictest3776021-21ssh21:55:410
3711731993827,8cyclictest3760826-21missed_timers21:30:350
3711733993729,5cyclictest3862725-21aten_r2power_vo00:20:302
3711733993729,5cyclictest3829143-21cron23:25:012
3711733993729,5cyclictest3829143-21cron23:25:012
3711733993729,5cyclictest3820192-21latency_hist23:10:022
3711733993727,7cyclictest1961693-21cups-browsed22:04:432
3711733993725,9cyclictest3737216-21cut20:35:012
3711733993725,9cyclictest3737216-21cut20:35:012
3711732993728,6cyclictest1000-21mta-sts-daemon23:38:541
3711732993727,6cyclictest420-21avahi-daemon00:13:161
3711731993730,4cyclictest3865203-21cron00:25:000
3711731993730,4cyclictest3793589-21cpu22:25:310
3711733993630,3cyclictest0-21swapper/221:10:012
3711733993628,4cyclictest0-21swapper/221:20:372
3711733993627,6cyclictest420-21avahi-daemon23:00:472
3711733993627,6cyclictest3869562-21mailstats00:30:502
3711732993628,5cyclictest3790028-21idleruntime-cro22:20:001
3711732993628,5cyclictest3790028-21idleruntime-cro22:20:001
3711732993628,5cyclictest3716220-21cron19:25:011
371173299361,30cyclictest3833298-21ssh23:30:441
371173299361,30cyclictest3833298-21ssh23:30:441
3711731993629,5cyclictest0-21swapper/021:40:010
3711733993528,3cyclictest0-21swapper/222:40:312
3711733993527,5cyclictest3802032-21cron22:40:012
3711733993527,4cyclictest0-21swapper/222:10:422
3711733993526,4cyclictest0-21swapper/222:35:012
3711733993525,6cyclictest1961690-21cupsd23:48:242
3711733993524,8cyclictest3716229-21/usr/sbin/munin19:25:022
3711733993524,7cyclictest3725207-21sh19:55:012
3711733993520,3cyclictest171rcu_preempt19:10:392
3711732993527,5cyclictest3743504-21aten_r2power_en20:55:301
3711732993527,5cyclictest3725207-21cron19:55:011
3711732993527,3cyclictest0-21swapper/123:15:011
3711732993527,3cyclictest0-21swapper/120:20:431
3711732993525,6cyclictest3869176-21needreboot00:30:371
3711731993529,3cyclictest1000-21mta-sts-daemon21:25:450
3711731993526,6cyclictest3832675-21aten_r2power_vo23:30:310
3711731993526,6cyclictest3832675-21aten_r2power_vo23:30:310
3711731993523,5cyclictest3713208-21cron19:15:010
3711731993522,10cyclictest1961693-21cups-browsed21:15:420
3711731993522,10cyclictest1961693-21cups-browsed21:15:420
3711733993428,3cyclictest0-21swapper/200:15:012
3711733993427,4cyclictest0-21swapper/223:40:002
3711733993425,6cyclictest1961693-21cups-browsed22:55:572
3711733993425,5cyclictest3784003-21ssh22:10:012
3711732993425,6cyclictest1961693-21cups-browsed21:05:461
371173299341,28cyclictest3812258-21ssh22:55:451
371173299341,24cyclictest3851439-21ssh00:00:521
371173199342,29cyclictest3788175-21ssh22:15:440
371173199342,29cyclictest3788175-21ssh22:15:440
3711731993422,9cyclictest3784535-21aten_r2power_vo22:10:310
3711732993327,3cyclictest0-21swapper/122:20:521
3711732993326,4cyclictest0-21swapper/119:20:011
3711732993326,3cyclictest0-21swapper/100:20:471
3711732993325,5cyclictest421-21dbus-daemon19:40:011
3711732993323,6cyclictest3848821-21unixbench_multi23:55:541
371173299331,24cyclictest3854421-21ssh00:05:511
3711731993326,3cyclictest0-21swapper/022:50:420
3711731993325,5cyclictest420-21avahi-daemon19:25:470
3711731993325,5cyclictest420-21avahi-daemon19:25:470
3711731993325,4cyclictest3737361-21latency_hist20:35:020
3711731993325,4cyclictest3737361-21latency_hist20:35:020
371173199332,25cyclictest3830181-21sh23:25:440
371173199332,25cyclictest3830181-21sh23:25:440
371173199331,25cyclictest3866340-21ssh00:25:450
3711733993227,2cyclictest0-21swapper/219:40:002
3711733993225,5cyclictest341rcuc/220:25:432
3711733993224,5cyclictest1961693-21cups-browsed22:50:262
3711733993224,3cyclictest3778228-21kworker/u8:14-nfsiod22:50:012
3711733993223,7cyclictest265-21systemd-journal23:30:012
3711733993223,7cyclictest265-21systemd-journal23:30:012
3711733993221,8cyclictest3717038-21needreboot19:25:382
3711733993221,8cyclictest3717038-21needreboot19:25:382
371173299323,23cyclictest3797208-21ssh22:30:451
3711732993225,3cyclictest0-21swapper/121:10:391
3711732993225,3cyclictest0-21swapper/120:40:011
3711732993225,3cyclictest0-21swapper/120:40:011
3711732993223,6cyclictest485-21in:imuxsock21:41:371
3711732993223,6cyclictest420-21avahi-daemon23:52:591
3711732993223,6cyclictest3723732-21true19:50:021
3711732993221,8cyclictest3761354-21timerandwakeup21:30:501
3711731993227,2cyclictest0-21swapper/020:20:010
3711731993222,7cyclictest3775200-21kworker/u8:12-rpciod00:00:020
371173199321,24cyclictest3818190-21ssh23:05:410
3711733993124,3cyclictest0-21swapper/220:15:012
3711733993123,5cyclictest1961693-21cups-browsed21:15:532
3711733993123,5cyclictest1961693-21cups-browsed21:15:532
3711733993116,4cyclictest171rcu_preempt20:20:462
3711732993124,4cyclictest1003-21mta-sts-daemon00:28:041
3711732993123,5cyclictest3794447-21switchtime22:25:511
371173299312,20cyclictest3755082-21ssh21:21:001
3711732993120,8cyclictest3741715-21munin-plugin-st20:50:011
3711731993126,3cyclictest3728167-21head20:00:580
3711731993125,4cyclictest421-21dbus-daemon23:45:010
3711731993123,5cyclictest3817135-21sh23:05:020
3711731993122,7cyclictest3847156-21cron23:55:000
3711733993022,4cyclictest3743208-21munin-plugin-st20:55:012
3711733993015,11cyclictest421-21dbus-daemon00:39:292
371173299307,16cyclictest1000-21mta-sts-daemon23:05:421
3711732993023,4cyclictest3732706-21cron20:20:011
3711732993023,4cyclictest1001-21mta-sts-daemon00:17:501
3711732993023,4cyclictest1001-21mta-sts-daemon00:17:501
3711732993023,3cyclictest0-21swapper/121:35:311
3711732993022,5cyclictest1003-21mta-sts-daemon20:10:011
3711732993022,4cyclictest0-21swapper/122:00:391
371173199306,17cyclictest428-21mta-sts-daemon19:55:450
3711731993025,3cyclictest0-21swapper/021:20:330
3711731993024,4cyclictest3802729-21fschecks_count22:40:330
3711731993024,3cyclictest0-21swapper/020:40:000
3711731993024,3cyclictest0-21swapper/020:40:000
3711731993022,5cyclictest3859852-21/usr/sbin/munin00:15:370
3711731993022,5cyclictest3859852-21/usr/sbin/munin00:15:370
3711731993022,5cyclictest1003-21mta-sts-daemon20:05:450
3711731993021,5cyclictest3829262-21latency_hist23:25:030
3711731993021,5cyclictest3829262-21latency_hist23:25:030
3711731993020,7cyclictest12650irq/32-eth019:15:410
3711731993011,16cyclictest151ktimers/021:50:410
3711733992922,4cyclictest3739741-21proc_pri20:40:502
371173399292,20cyclictest3770203-21ssh21:45:472
3711733992921,5cyclictest3732573-21unixbench_multi20:15:522
3711732992921,5cyclictest3728217-21cron20:05:011
3711732992918,7cyclictest0-21swapper/119:25:441
3711732992918,7cyclictest0-21swapper/119:25:441
3711732992916,8cyclictest3730561-21sshd20:10:371
3711731992924,3cyclictest3709149-21kworker/u8:6-xprtiod19:41:000
3711733992821,4cyclictest0-21swapper/220:00:532
3711733992819,7cyclictest3729735-21sh20:10:012
371173399281,20cyclictest3773022-21ssh21:50:402
371173299284,18cyclictest428-21mta-sts-daemon00:35:471
3711732992821,3cyclictest0-21swapper/122:10:021
3711732992818,7cyclictest3805970-21missed_timers22:45:361
3711731992822,4cyclictest3711730-21cyclictest21:46:400
371173199282,15cyclictest3806156-21ntpq22:45:400
3711731992821,4cyclictest3851574-21ssh00:00:570
3711731992820,5cyclictest3844315-21latency_hist23:50:020
371173399277,15cyclictest351ktimers/219:30:422
371173399272,18cyclictest36-21ksoftirqd/220:45:422
3711733992720,4cyclictest3722747-21fschecks_time19:45:352
3711733992718,7cyclictest3714019-21needreboot19:15:382
3711732992721,3cyclictest1001-21mta-sts-daemon19:10:541
3711732992720,3cyclictest0-21swapper/121:00:341
3711732992719,5cyclictest3735722-21idleruntime-cro20:30:011
371173299271,19cyclictest665-21snmpd20:30:431
371173299271,19cyclictest665-21snmpd20:30:431
3711731992720,3cyclictest3750805-21kworker/u8:5-rpciod22:21:520
3711731992716,3cyclictest3719205-21cron19:35:000
371173399269,5cyclictest3707856-21kworker/u8:1+events_unbound19:10:022
371173399262,18cyclictest443-21gpm20:35:432
371173399262,18cyclictest443-21gpm20:35:432
3711733992613,3cyclictest171rcu_preempt19:45:012
3711731992619,5cyclictest1000-21mta-sts-daemon20:20:300
3711731992619,4cyclictest3816356-21kworker/u8:7-xprtiod23:10:350
3711733992517,5cyclictest3745232-21fschecks_count21:00:342
371173299254,14cyclictest421-21dbus-daemon19:30:441
3711732992517,4cyclictest0-21swapper/120:40:551
3711731992518,3cyclictest3778228-21kworker/u8:14-rpciod00:10:370
3711731992514,9cyclictest0-21swapper/020:40:550
3711732992415,6cyclictest3721934-21threads19:41:011
3711731992418,3cyclictest1003-21mta-sts-daemon20:55:180
3711731992417,5cyclictest1000-21mta-sts-daemon21:02:270
3711731992412,10cyclictest0-21swapper/019:45:490
371173399237,3cyclictest171rcu_preempt19:55:352
3711732992314,6cyclictest265-21systemd-journal20:50:371
371173199232,15cyclictest3724556-21ntpq19:50:410
3711731992317,4cyclictest1961693-21cups-browsed20:10:560
3711731992313,7cyclictest3714687-21kworker/u8:0+xprtiod19:25:010
3711731992216,4cyclictest3720697-21cron19:40:000
3711731992215,4cyclictest3735734-21systemd-run20:30:010
371173199219,7cyclictest1001-21mta-sts-daemon20:45:410
3711732992015,2cyclictest0-21swapper/119:10:011
371173499191,7cyclictest0-21swapper/321:40:403
371173499191,10cyclictest0-21swapper/323:45:413
371173499184,8cyclictest0-21swapper/323:55:443
371173499173,10cyclictest0-21swapper/322:20:413
371173499171,9cyclictest0-21swapper/321:50:443
371173499171,8cyclictest0-21swapper/323:50:393
371173499171,8cyclictest0-21swapper/322:45:383
371173499162,9cyclictest0-21swapper/321:30:013
371173499162,8cyclictest0-21swapper/322:25:443
371173499162,8cyclictest0-21swapper/321:35:423
371173499162,7cyclictest0-21swapper/323:15:403
371173499161,9cyclictest0-21swapper/300:35:443
371173499161,8cyclictest0-21swapper/323:25:443
371173499161,8cyclictest0-21swapper/323:25:443
371173499161,8cyclictest0-21swapper/300:15:473
371173499161,8cyclictest0-21swapper/300:15:473
371173499161,7cyclictest0-21swapper/322:35:393
371173499161,7cyclictest0-21swapper/322:15:393
371173499161,7cyclictest0-21swapper/322:15:393
371173499161,7cyclictest0-21swapper/300:05:433
371173199166,7cyclictest3709149-21kworker/u8:6+xprtiod19:10:010
371173499154,6cyclictest0-21swapper/321:45:423
371173499153,6cyclictest0-21swapper/321:10:393
371173499152,8cyclictest0-21swapper/321:30:373
371173499151,8cyclictest0-21swapper/323:35:433
371173499151,8cyclictest0-21swapper/321:20:583
371173499151,8cyclictest0-21swapper/321:15:453
371173499151,8cyclictest0-21swapper/321:15:453
371173499151,7cyclictest0-21swapper/323:20:393
371173499151,7cyclictest0-21swapper/323:20:393
371173499151,7cyclictest0-21swapper/320:05:463
371173499151,6cyclictest0-21swapper/320:30:393
371173499151,6cyclictest0-21swapper/320:30:393
371173499151,6cyclictest0-21swapper/319:55:423
371173499151,5cyclictest0-21swapper/320:50:403
371173499144,6cyclictest0-21swapper/322:10:313
371173499142,7cyclictest0-21swapper/322:30:353
371173499142,6cyclictest0-21swapper/323:40:413
371173499142,6cyclictest0-21swapper/319:20:473
371173499141,8cyclictest0-21swapper/322:55:403
371173499141,8cyclictest0-21swapper/300:00:523
371173499141,7cyclictest0-21swapper/300:30:443
371173499141,6cyclictest0-21swapper/320:40:503
371173499141,6cyclictest0-21swapper/320:20:443
371173499133,6cyclictest0-21swapper/322:55:013
371173499132,7cyclictest0-21swapper/323:30:343
371173499132,7cyclictest0-21swapper/323:30:343
371173499132,7cyclictest0-21swapper/300:10:363
371173499132,6cyclictest0-21swapper/323:05:313
371173499132,6cyclictest0-21swapper/319:30:013
371173499132,6cyclictest0-21swapper/319:30:013
371173499131,8cyclictest0-21swapper/321:55:433
371173499131,7cyclictest0-21swapper/323:00:533
371173499131,7cyclictest0-21swapper/300:25:373
371173499131,7cyclictest0-21swapper/300:20:293
371173499131,6cyclictest0-21swapper/322:00:333
371173499131,6cyclictest0-21swapper/319:50:433
371173499131,6cyclictest0-21swapper/319:45:473
371173499131,6cyclictest0-21swapper/319:15:393
371173499131,6cyclictest0-21swapper/319:10:393
371173499131,5cyclictest0-21swapper/319:40:453
371173499122,6cyclictest0-21swapper/322:40:313
371173499122,6cyclictest0-21swapper/322:05:353
371173499122,6cyclictest0-21swapper/321:10:013
371173499122,5cyclictest0-21swapper/323:10:303
371173499121,6cyclictest0-21swapper/320:45:403
371173499113,5cyclictest0-21swapper/320:10:373
371173499112,6cyclictest0-21swapper/319:40:013
371173499112,5cyclictest0-21swapper/321:00:373
371173499112,5cyclictest0-21swapper/320:15:423
371173499111,6cyclictest0-21swapper/320:55:343
371173499111,5cyclictest0-21swapper/320:25:393
371173499111,5cyclictest0-21swapper/319:30:393
371173499104,4cyclictest0-21swapper/320:35:393
371173499104,4cyclictest0-21swapper/320:35:393
371173499101,5cyclictest0-21swapper/320:00:423
37117349981,5cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional