You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-07 - 04:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Thu May 07, 2026 00:43:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
628250995544,7cyclictest717947-21latency_hist22:20:021
628249995239,10cyclictest811094-21timerandwakeup00:25:580
628249995039,8cyclictest792461-21unixbench_multi00:01:000
628251994941,5cyclictest0-21swapper/222:45:012
628249994738,6cyclictest439-21systemd-logind21:30:010
628249994637,6cyclictest790599-21cups-browsed00:00:010
628251994537,5cyclictest643176-21tail19:55:592
628251994537,5cyclictest643176-21tail19:55:592
628251994434,8cyclictest791241-21sh00:00:382
628250994434,5cyclictest0-21swapper/122:50:501
628249994430,11cyclictest3027766-21cups-browsed23:40:460
628251994335,5cyclictest748820-21sh23:01:002
628250994334,4cyclictest0-21swapper/123:45:501
628249994328,12cyclictest722636-21needreboot22:25:450
628249994322,4cyclictest171rcu_preempt19:50:440
628251994234,5cyclictest779644-21latency_hist23:45:022
628251994233,5cyclictest796213-21unixbench_singl00:06:022
628250994234,5cyclictest691716-21systemd-run21:45:011
628250994234,5cyclictest691716-21systemd-run21:45:011
628250994233,6cyclictest0-21swapper/121:20:011
628250994233,5cyclictest765818-21fschecks_time23:25:431
628249994233,6cyclictest805708-21systemd-run00:20:020
62824999423,34cyclictest659-21snmpd23:20:500
628251994133,5cyclictest711106-21aten2.4-expect22:10:372
628250994134,4cyclictest746971-21ssh23:00:011
628250994125,12cyclictest677410-21latency21:25:441
62825099411,34cyclictest758956-21ssh23:15:511
628249994132,6cyclictest420-21avahi-daemon23:31:460
628249994130,8cyclictest700026-21fschecks_time21:55:420
62824999412,29cyclictest704199-21ssh22:00:480
62825199406,30cyclictest708131-21ssh22:05:522
628251994034,4cyclictest1055-21mta-sts-daemon20:25:502
628251994033,3cyclictest0-21swapper/219:40:012
628251994032,5cyclictest726972-21sed22:31:002
628251994032,5cyclictest726972-21sed22:31:002
628251994032,5cyclictest706795-21systemd-run22:05:012
628251994032,5cyclictest439-21systemd-logind20:20:012
628251994031,5cyclictest696220-21cpuspeed21:50:382
628250994032,5cyclictest637289-21systemctl19:40:011
628249994031,6cyclictest739918-21latency_hist22:50:020
628249994018,16cyclictest12350irq/32-eth023:15:550
628251993930,6cyclictest682104-21unixbench_singl21:30:592
628251993923,6cyclictest420-21avahi-daemon21:17:042
628250993930,5cyclictest775874-21munin-run23:40:001
628250993929,6cyclictest687940-21systemd-run21:40:011
62824999395,28cyclictest696701-21ssh21:50:480
628249993931,5cyclictest659-21snmpd19:20:010
628249993929,6cyclictest688587-21aten_r2power_po21:40:380
628249993929,6cyclictest688587-21aten_r2power_po21:40:380
628249993928,8cyclictest708895-21sshd22:06:280
628251993831,5cyclictest624375-21kworker/u8:6-rpciod19:20:012
628251993830,5cyclictest772238-21cron23:35:002
628251993830,5cyclictest717805-21cron22:20:012
628251993829,6cyclictest753-21ntpd20:51:002
628251993829,4cyclictest0-21swapper/223:45:542
628251993810,5cyclictest171rcu_preempt22:50:472
628250993829,6cyclictest261-21systemd-journal20:35:011
628250993828,5cyclictest0-21swapper/100:35:491
62824999383,30cyclictest814564-21ssh00:30:510
628249993832,3cyclictest0-21swapper/022:10:540
628249993830,5cyclictest730421-21xargs22:35:590
628249993828,7cyclictest420-21avahi-daemon22:53:450
628249993825,9cyclictest693513-21unixbench_multi21:46:000
628251993731,3cyclictest0-21swapper/200:35:002
628251993728,6cyclictest656788-21idleruntime-cro20:45:012
628251993726,8cyclictest818904-21sed00:35:592
62825199372,27cyclictest803132-21ssh00:15:492
628250993729,5cyclictest728823-21ssh22:35:011
628250993729,5cyclictest728823-21ssh22:35:011
628250993728,6cyclictest421-21dbus-daemon21:35:011
628250993727,7cyclictest748104-21needreboot23:00:461
628249993729,6cyclictest795535-21ssh00:05:480
628249993723,5cyclictest12350irq/32-eth021:10:510
628251993631,2cyclictest0-21swapper/220:40:012
628251993624,9cyclictest678049-21timerandwakeup21:25:592
62825199361,31cyclictest773585-21ssh23:35:532
628250993631,2cyclictest0-21swapper/122:05:011
628250993628,6cyclictest711092-21aten_r2power_po22:10:381
628250993628,5cyclictest673830-21proc_pri21:20:561
628250993628,5cyclictest421-21dbus-daemon22:00:011
62824999364,25cyclictest733771-21ssh22:40:490
62824999363,26cyclictest748246-21ssh23:00:500
628249993622,9cyclictest12350irq/32-eth023:10:460
62824999362,22cyclictest799297-21ssh00:10:490
628251993528,4cyclictest676344-21idleruntime-cro21:25:012
628251993528,3cyclictest0-21swapper/223:20:522
628251993527,5cyclictest790614-21ssh00:00:012
628251993527,3cyclictest0-21swapper/221:35:482
628251993524,6cyclictest0-21swapper/222:20:492
628250993527,5cyclictest794392-21systemd-run00:05:011
628250993527,5cyclictest707594-21uniq22:05:401
628250993527,5cyclictest695463-21cron21:50:001
628250993527,5cyclictest649818-21fschecks_count20:20:451
628250993526,4cyclictest0-21swapper/122:35:511
628250993525,6cyclictest722906-21proc_pri22:25:551
628250993525,4cyclictest0-21swapper/100:25:521
628250993524,9cyclictest652284-21/usr/sbin/munin20:30:021
62824999353,19cyclictest780892-21ssh23:45:510
628249993528,4cyclictest744975-21find22:56:000
628249993525,6cyclictest787000-21latency_hist23:55:020
628249993524,8cyclictest728992-21latency_hist22:35:020
628249993524,8cyclictest728992-21latency_hist22:35:020
628249993511,3cyclictest171rcu_preempt00:36:040
62825199343,23cyclictest730072-21ssh22:35:512
628250993427,3cyclictest0-21swapper/100:10:491
628250993425,6cyclictest805774-21latency_hist00:20:021
628249993422,9cyclictest1-21systemd22:16:300
62824999342,29cyclictest0-21swapper/021:05:500
62824999341,9cyclictest14-21ksoftirqd/023:35:530
628251993325,5cyclictest689757-21unixbench_multi21:41:002
628251993325,5cyclictest689757-21unixbench_multi21:41:002
628251993323,8cyclictest810595-21needreboot00:25:452
628251993318,9cyclictest0-21swapper/200:20:562
628251993318,9cyclictest0-21swapper/200:20:562
62825199331,24cyclictest799292-21rm00:10:492
628250993325,5cyclictest762224-21fschecks_time23:20:421
628250993325,5cyclictest640223-21sed19:46:021
628250993325,4cyclictest0-21swapper/121:10:491
628250993321,5cyclictest0-21swapper/123:30:501
628249993326,4cyclictest721507-21cron22:25:010
628249993325,5cyclictest421-21dbus-daemon20:55:010
628249993322,5cyclictest0-21swapper/020:10:500
62824999332,24cyclictest766339-21ssh23:25:540
62824999332,22cyclictest673612-21ssh21:20:480
628249993318,11cyclictest0-21swapper/020:25:030
62825199324,22cyclictest784620-21ssh23:50:542
628251993226,3cyclictest0-21swapper/222:55:582
628251993226,3cyclictest0-21swapper/221:45:412
628251993225,4cyclictest664304-21kworker/u8:14-xprtiod22:30:012
628251993224,5cyclictest761403-21munin-run23:20:012
628251993224,5cyclictest420-21avahi-daemon19:42:242
62825199322,23cyclictest737529-21ssh22:45:542
62825199321,26cyclictest665699-21ssh21:10:542
628250993224,5cyclictest656779-21cron20:45:011
628250993224,5cyclictest421-21dbus-daemon00:00:001
628250993221,8cyclictest1054-21mta-sts-daemon19:20:531
628250993220,9cyclictest637969-21latency19:40:431
628250993215,8cyclictest664284-21diskmemload21:50:491
628249993227,3cyclictest637264-21kworker/u8:3-xprtiod19:40:010
628249993225,5cyclictest261-21systemd-journal21:20:010
628249993222,6cyclictest0-21swapper/000:20:430
628249993222,6cyclictest0-21swapper/000:20:430
628249993221,8cyclictest681301-21missed_timers21:30:420
628249993217,3cyclictest171rcu_preempt20:55:460
628251993124,4cyclictest1054-21mta-sts-daemon20:00:492
628251993123,5cyclictest662806-21systemd-run21:05:012
62825199312,21cyclictest659-21snmpd23:10:482
628250993125,3cyclictest0-21swapper/123:45:011
628250993124,4cyclictest721682-21ssh22:25:031
628250993123,5cyclictest736094-21grep22:45:001
628250993123,5cyclictest642298-21fschecks_count19:55:401
628250993123,5cyclictest642298-21fschecks_count19:55:401
62825099311,25cyclictest814456-21ssh00:30:481
62824999318,6cyclictest646263-21kworker/0:0-events20:35:540
62824999314,3cyclictest171rcu_preempt19:10:430
628249993119,2cyclictest171rcu_preempt21:35:470
628249993115,3cyclictest171rcu_preempt23:05:480
628249993115,3cyclictest171rcu_preempt19:20:480
628251993023,5cyclictest0-21swapper/223:10:022
628251993023,4cyclictest1056-21mta-sts-daemon20:45:442
628251993022,6cyclictest0-21swapper/219:25:012
628251993022,5cyclictest428-21mta-sts-daemon19:53:142
628251993021,6cyclictest0-21swapper/222:00:002
62825199301,22cyclictest766413-21grep23:25:562
628250993024,3cyclictest0-21swapper/121:10:001
628250993023,3cyclictest691721-21kworker/u8:15-xprtiod23:10:011
628250993022,5cyclictest646228-21unixbench_singl20:06:001
628250993022,5cyclictest261-21systemd-journal20:05:001
628250993021,5cyclictest647768-21cron20:15:001
628250993019,9cyclictest261-21systemd-journal22:47:171
628250993018,9cyclictest806693-21latency00:20:441
628250993018,9cyclictest806693-21latency00:20:441
628249993014,3cyclictest171rcu_preempt19:30:530
628251992921,4cyclictest661138-21unixbench_multi20:55:592
628251992920,7cyclictest639559-21missed_timers19:45:472
628251992920,6cyclictest663304-21fschecks_count21:05:412
628250992923,3cyclictest0-21swapper/121:00:001
628250992921,5cyclictest655262-21cron20:40:011
62825099291,18cyclictest784464-21ssh23:50:471
62824999295,3cyclictest171rcu_preempt21:00:490
628249992920,7cyclictest644160-21ntp_kernel_pll_20:00:460
628249992919,8cyclictest633560-21needreboot19:25:460
628251992821,4cyclictest647766-21cron20:15:002
628251992821,4cyclictest635738-21munin-plugin-st19:35:012
628251992814,12cyclictest351ktimers/220:20:582
62825099282,19cyclictest795688-21ntpq00:05:551
628250992820,5cyclictest634145-21unixbench_multi19:25:591
62824999288,14cyclictest171rcu_preempt20:50:010
628249992822,3cyclictest0-21swapper/020:05:520
628249992821,4cyclictest631270-21kworker/u8:4-xprtiod19:40:450
628249992819,6cyclictest643309-21systemd-run20:00:010
628249992819,6cyclictest643309-21systemd-run20:00:010
628249992814,3cyclictest171rcu_preempt20:15:490
628249992812,14cyclictest638781-21kworker/u8:7-flush-0:2120:32:240
62825199275,15cyclictest428-21mta-sts-daemon19:10:492
628251992720,3cyclictest0-21swapper/219:10:012
628251992714,4cyclictest171rcu_preempt20:30:502
62825099274,7cyclictest171rcu_preempt20:50:491
628250992721,3cyclictest0-21swapper/120:50:011
628250992718,4cyclictest0-21swapper/123:10:521
62824999272,3cyclictest171rcu_preempt20:25:410
628249992720,5cyclictest639647-21ntpq19:45:490
628250992617,6cyclictest630482-21missed_timers19:15:431
628249992615,3cyclictest653760-21kworker/u8:6-xprtiod20:44:580
628251992518,4cyclictest646124-21unixbench-2d20:05:592
62825099256,14cyclictest1054-21mta-sts-daemon19:50:451
628250992519,3cyclictest0-21swapper/120:20:001
628250992419,2cyclictest0-21swapper/119:10:021
628250992317,2cyclictest0-21swapper/121:05:011
628250992315,5cyclictest629737-21grep19:15:011
628251992216,3cyclictest0-21swapper/219:25:422
62825099222,15cyclictest635184-21ntpq19:30:511
62825299203,8cyclictest0-21swapper/322:40:463
628249992014,3cyclictest151ktimers/019:10:010
62825299183,9cyclictest0-21swapper/323:45:483
62825299183,6cyclictest0-21swapper/322:25:553
62825299182,9cyclictest0-21swapper/322:45:513
62825299173,6cyclictest0-21swapper/323:05:493
62825299172,8cyclictest0-21swapper/322:05:533
62825299164,6cyclictest0-21swapper/300:05:563
62825299162,8cyclictest0-21swapper/300:25:523
62825299162,7cyclictest0-21swapper/323:20:483
62825299162,7cyclictest0-21swapper/321:30:463
62825299161,7cyclictest0-21swapper/323:10:463
62825299161,7cyclictest0-21swapper/300:20:533
62825299161,7cyclictest0-21swapper/300:20:533
62825299161,6cyclictest0-21swapper/320:10:453
62825299161,6cyclictest0-21swapper/319:10:473
62825299152,8cyclictest0-21swapper/300:10:493
62825299152,7cyclictest0-21swapper/321:45:503
62825299152,7cyclictest0-21swapper/300:15:463
62825299152,5cyclictest0-21swapper/319:55:513
62825299152,5cyclictest0-21swapper/319:55:513
62825299152,5cyclictest0-21swapper/319:37:593
62825299151,8cyclictest0-21swapper/322:16:013
62825299151,8cyclictest0-21swapper/300:30:503
62825299151,7cyclictest0-21swapper/322:20:493
62825299151,7cyclictest0-21swapper/322:00:503
62825299151,7cyclictest0-21swapper/321:25:483
62825299151,7cyclictest0-21swapper/320:05:523
62825299151,6cyclictest0-21swapper/319:45:523
62825299143,7cyclictest0-21swapper/322:10:483
62825299143,6cyclictest0-21swapper/323:35:533
62825299143,6cyclictest0-21swapper/321:10:483
62825299142,7cyclictest0-21swapper/323:50:393
62825299142,7cyclictest0-21swapper/321:35:493
62825299142,5cyclictest0-21swapper/320:50:483
62825299142,5cyclictest0-21swapper/319:25:473
62825299142,4cyclictest0-21swapper/320:30:463
62825299141,7cyclictest0-21swapper/323:00:493
62825299141,7cyclictest0-21swapper/321:40:483
62825299141,7cyclictest0-21swapper/321:40:483
62825299141,6cyclictest0-21swapper/320:20:513
62825299141,6cyclictest0-21swapper/320:15:473
62825299141,6cyclictest0-21swapper/320:00:493
62825299134,5cyclictest0-21swapper/321:15:503
62825299133,6cyclictest0-21swapper/320:25:493
62825299132,6cyclictest0-21swapper/300:00:393
62825299132,5cyclictest0-21swapper/323:30:433
62825299131,8cyclictest0-21swapper/323:40:503
62825299131,8cyclictest0-21swapper/322:50:423
62825299131,7cyclictest0-21swapper/323:55:503
62825299131,7cyclictest0-21swapper/321:50:493
62825299131,6cyclictest0-21swapper/320:35:553
62825299131,5cyclictest0-21swapper/319:50:463
62825299123,6cyclictest0-21swapper/321:00:523
62825299122,6cyclictest0-21swapper/322:30:433
62825299122,6cyclictest0-21swapper/322:30:433
62825299121,7cyclictest0-21swapper/300:35:513
62825299121,6cyclictest0-21swapper/321:20:493
62825299121,6cyclictest0-21swapper/321:10:003
62825299114,4cyclictest0-21swapper/319:40:463
62825299113,5cyclictest0-21swapper/322:35:403
62825299112,5cyclictest0-21swapper/321:55:383
62825299111,6cyclictest0-21swapper/322:55:433
62825299111,6cyclictest0-21swapper/320:40:483
62825299111,6cyclictest0-21swapper/319:20:463
62825299111,5cyclictest0-21swapper/323:25:393
62825299110,7cyclictest0-21swapper/323:15:393
62825299104,3cyclictest0-21swapper/319:30:473
62825299102,4cyclictest0-21swapper/320:45:503
62825299101,6cyclictest0-21swapper/319:15:373
62825299101,5cyclictest0-21swapper/320:55:493
6282529982,4cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional