You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-29 - 23:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Wed Apr 29, 2026 12:43:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2696248996955,10cyclictest2757300-40mandb09:40:391
2696249995634,18cyclictest1-21systemd10:00:012
269624999542,20cyclictest2798653-21ssh10:30:512
2696249995336,14cyclictest2757540-21if_wlan009:40:402
2696249995330,20cyclictest2699240-21idleruntime-cro07:20:002
2696249995327,3cyclictest0-21swapper/209:50:012
269624999513,23cyclictest62450lldpd09:05:482
2696249995128,19cyclictest0-21swapper/209:30:022
2696249995126,21cyclictest2750065-21switchtime09:30:562
2696249995125,6cyclictest420-21avahi-daemon12:15:392
2696248995043,4cyclictest0-21swapper/112:10:011
2696249994921,5cyclictest2836465-21/usr/sbin/munin11:20:372
2696247994941,4cyclictest1-21systemd10:30:010
2696249994838,5cyclictest2824940-21nm-online11:04:592
2696249994838,5cyclictest2824940-21nm-online11:04:592
2696249994737,6cyclictest2768845-21latency_hist09:55:022
2696249994721,4cyclictest2683718-21kworker/u8:10-xprtiod07:45:432
269624999452,33cyclictest2782363-21ssh10:10:472
269624999452,33cyclictest2782363-21ssh10:10:472
2696249994434,7cyclictest2789183-21munin-run10:20:012
2696248994437,3cyclictest0-21swapper/111:10:391
2696248994437,3cyclictest0-21swapper/111:10:391
2696248994435,6cyclictest420-21avahi-daemon11:53:371
2696248994432,9cyclictest2772926-21systemd10:00:021
2696247994440,2cyclictest0-21swapper/008:40:000
2696247994433,8cyclictest2850745-21idleruntime-cro11:40:010
2696249994335,5cyclictest2828884-21systemd-run11:10:012
2696249994335,5cyclictest2828884-21systemd-run11:10:012
2696249994333,6cyclictest2892306-21tail12:30:562
2696248994335,5cyclictest2850731-21munin-plugin-st11:40:011
2696248994335,5cyclictest2745060-21aten_r2power_vo09:25:351
2696248994331,8cyclictest1895805-21cups-browsed11:20:461
2696247994335,5cyclictest1-21systemd11:15:010
2696247994335,5cyclictest1-21systemd11:15:010
2696248994234,5cyclictest2884425-21unixbench_singl12:20:571
2696249994132,4cyclictest0-21swapper/211:40:012
2696249994129,6cyclictest0-21swapper/211:30:422
2696249994128,8cyclictest420-21avahi-daemon09:10:472
2696248994130,7cyclictest420-21avahi-daemon11:33:161
2696247994124,12cyclictest12350irq/32-eth009:35:420
2696249994031,5cyclictest2862636-21latency_hist11:55:012
2696249994031,5cyclictest0-21swapper/212:40:002
2696248994032,5cyclictest2752515-21systemd-run09:35:011
269624899401,32cyclictest2774238-21ssh10:00:481
2696247994033,4cyclictest2777015-21latency_hist10:05:020
2696247994030,7cyclictest2862644-21munin-run11:55:010
2696249993931,5cyclictest2711204-21cron08:00:012
2696249993931,5cyclictest1-21systemd12:00:012
269624899397,25cyclictest2826245-21sh11:05:451
269624899397,25cyclictest2826245-21sh11:05:451
2696248993931,5cyclictest2822864-21grep11:00:551
2696248993931,5cyclictest2822864-21grep11:00:551
2696247993933,4cyclictest2734425-21kworker/u8:1-nfsiod09:40:380
2696247993926,9cyclictest2825499-21/usr/sbin/munin11:05:520
2696247993926,9cyclictest2825499-21/usr/sbin/munin11:05:520
2696247993916,19cyclictest1895805-21cups-browsed11:40:470
2696249993830,5cyclictest2854699-21latency_hist11:45:012
269624899385,26cyclictest2833705-21ssh11:15:441
2696248993830,5cyclictest2705266-21systemd-run07:40:011
2696248993830,5cyclictest2705266-21systemd-run07:40:011
2696248993828,8cyclictest2879283-21sh12:15:361
269624899381,4cyclictest171rcu_preempt09:45:461
2696247993829,6cyclictest1895805-21cups-browsed12:16:010
2696249993729,5cyclictest2774376-21sendmail_mailqu10:00:522
2696249993724,10cyclictest1056-21mta-sts-daemon08:30:452
2696249993724,10cyclictest1056-21mta-sts-daemon08:30:452
2696248993729,5cyclictest2734057-21unixbench_multi09:10:561
2696248993726,8cyclictest2732216-21diskmemload09:20:461
269624799373,27cyclictest2745637-21ssh09:25:450
2696247993731,4cyclictest2890522-21systemd-run12:30:020
2696247993731,4cyclictest2890522-21systemd-run12:30:020
2696247993731,4cyclictest2782943-21unixbench_singl10:10:570
2696247993731,4cyclictest2782943-21unixbench_singl10:10:570
2696247993730,5cyclictest421-21dbus-daemon07:50:010
2696247993730,4cyclictest2817349-21kworker/u8:7+xprtiod11:04:590
2696247993730,4cyclictest2817349-21kworker/u8:7+xprtiod11:04:590
2696247993730,4cyclictest2737701-21proc_pri09:15:500
2696247993727,7cyclictest2866605-21sh12:00:010
269624799371,30cyclictest547-21gmain07:55:430
2696249993630,3cyclictest0-21swapper/209:20:012
2696249993628,5cyclictest2803145-21unixbench_singl10:35:562
2696249993628,5cyclictest2793267-21munin-run10:25:012
2696249993628,5cyclictest2744382-21cron09:25:012
269624899364,25cyclictest2851996-21ssh11:40:441
269624899363,25cyclictest2753854-21ssh09:35:481
2696248993630,3cyclictest0-21swapper/111:00:001
2696248993628,5cyclictest2781079-21systemd-run10:10:021
2696248993628,5cyclictest2781079-21systemd-run10:10:021
2696248993628,5cyclictest2732216-21diskmemload10:50:011
2696248993628,4cyclictest0-21swapper/111:45:441
2696248993627,5cyclictest2805271-21kworker/u8:8-xprtiod10:40:541
2696248993627,5cyclictest2725243-21fschecks_count08:45:381
2696247993628,5cyclictest2706761-21systemd-run07:45:010
2696247993627,6cyclictest0-21swapper/009:05:490
2696247993627,4cyclictest0-21swapper/010:55:450
2696247993622,11cyclictest0-21swapper/011:15:340
2696247993620,11cyclictest151ktimers/008:45:520
2696247993615,17cyclictest2891735-21ssh12:30:450
269624999354,22cyclictest547-21gmain07:40:472
2696249993530,2cyclictest0-21swapper/208:40:002
2696249993526,6cyclictest1055-21mta-sts-daemon12:30:022
2696249993526,6cyclictest1055-21mta-sts-daemon12:30:022
269624899359,6cyclictest2732216-21diskmemload12:35:451
2696248993529,3cyclictest0-21swapper/108:55:011
2696248993528,4cyclictest2717752-21fschecks_count08:20:361
2696248993526,5cyclictest2801414-21latency_hist10:35:021
2696248993525,7cyclictest439-21systemd-logind12:15:011
2696248993524,8cyclictest2700117-21ntp_kernel_pll_07:20:461
2696247993529,3cyclictest0-21swapper/012:15:010
2696247993528,4cyclictest1055-21mta-sts-daemon11:20:520
269624799352,26cyclictest2806490-21rm10:40:440
269624999345,24cyclictest2830136-21ssh11:10:462
269624999345,24cyclictest2830136-21ssh11:10:462
2696249993425,6cyclictest2708626-21aten_r2power_vo07:50:342
2696249993425,6cyclictest0-21swapper/210:55:402
269624899343,24cyclictest2802487-21ssh10:35:431
2696248993428,3cyclictest0-21swapper/112:05:011
2696248993428,3cyclictest0-21swapper/109:15:471
2696248993427,3cyclictest0-21swapper/110:10:511
2696248993427,3cyclictest0-21swapper/110:10:511
2696248993424,7cyclictest421-21dbus-daemon12:33:151
2696247993428,3cyclictest0-21swapper/011:30:420
2696247993427,5cyclictest2718665-21head08:20:540
2696247993426,5cyclictest2789651-21sshd10:20:280
2696247993426,5cyclictest2787005-21unixbench_singl10:15:560
269624799342,26cyclictest2814593-21ssh10:50:570
269624799342,26cyclictest2814593-21ssh10:50:570
2696247993420,10cyclictest261-21systemd-journal07:10:010
2696247993420,10cyclictest261-21systemd-journal07:10:010
2696247993416,14cyclictest151ktimers/009:00:430
2696249993326,4cyclictest2795034-21timerwakeupswit10:25:582
2696249993325,5cyclictest2817105-21munin-run10:55:012
2696249993325,5cyclictest2817105-21munin-run10:55:012
2696249993325,5cyclictest2703724-21sh07:35:002
2696249993325,5cyclictest2703724-21sh07:35:002
2696249993324,6cyclictest2753187-21aten_r2power_vo09:35:342
2696248993326,4cyclictest427-21polkitd10:52:381
2696248993326,4cyclictest427-21polkitd10:52:381
2696248993325,5cyclictest2721719-21munin-plugin-st08:35:001
2696248993325,5cyclictest2721719-21munin-plugin-st08:35:001
2696247993324,6cyclictest2872290-21timerwakeupswit12:05:560
2696247993324,6cyclictest1895805-21cups-browsed09:14:380
269624799331,26cyclictest151ktimers/009:45:460
2696249993225,4cyclictest2729195-21cron09:00:012
2696249993222,6cyclictest0-21swapper/210:45:012
2696248993224,5cyclictest2724725-21debian-sa108:45:011
269624899321,26cyclictest2863854-21ssh11:55:451
2696247993226,3cyclictest2858685-21kworker/u8:9-nfsiod12:40:010
2696247993223,5cyclictest2867098-21aten_r2power_cu12:00:330
269624799322,24cyclictest2778211-21ssh10:05:450
269624799322,24cyclictest2778211-21ssh10:05:450
2696247993213,13cyclictest12350irq/32-eth009:55:480
2696249993123,5cyclictest421-21dbus-daemon08:20:022
2696249993123,5cyclictest2842985-21ssh11:30:012
2696249993122,6cyclictest421-21dbus-daemon10:10:022
2696249993122,6cyclictest421-21dbus-daemon10:10:022
2696249993122,6cyclictest2727314-21systemctl08:50:562
2696249993120,4cyclictest0-21swapper/211:15:462
2696249993119,9cyclictest421-21dbus-daemon12:12:102
269624899313,21cyclictest2786355-21ssh10:15:441
2696248993124,4cyclictest2706734-21cron07:45:001
2696248993123,5cyclictest2890510-21idleruntime-cro12:30:011
2696248993123,5cyclictest2890510-21idleruntime-cro12:30:011
2696248993123,5cyclictest0-21swapper/108:00:011
2696248993122,6cyclictest0-21swapper/110:30:021
2696247993123,5cyclictest420-21avahi-daemon10:35:520
2696247993123,5cyclictest2765801-21missed_timers09:50:410
2696247993122,6cyclictest1895803-21cupsd09:32:030
2696249993022,5cyclictest421-21dbus-daemon08:50:012
269624899308,12cyclictest2005418-21munin-node08:00:451
2696248993025,2cyclictest0-21swapper/110:20:331
2696248993023,3cyclictest0-21swapper/111:26:011
2696248993023,3cyclictest0-21swapper/109:50:451
2696247993023,4cyclictest2732216-21diskmemload11:46:060
2696247993021,6cyclictest2709567-21unixbench-2d07:50:540
2696249992923,3cyclictest0-21swapper/212:20:462
269624999292,20cyclictest2871878-21ssh12:05:472
2696249992918,8cyclictest2813203-21idleruntime-cro10:50:012
2696249992918,8cyclictest2718079-21ntp_kernel_err08:20:412
2696248992924,2cyclictest0-21swapper/107:45:441
2696248992922,4cyclictest2729637-21cpu09:00:351
2696248992919,7cyclictest1-21systemd09:00:011
269624799296,15cyclictest2005418-21munin-node08:55:480
2696247992923,4cyclictest2705214-21cron07:40:000
2696247992923,4cyclictest2705214-21cron07:40:000
2696247992923,3cyclictest0-21swapper/010:30:440
2696247992922,4cyclictest2740890-21aten2.4-expect09:20:340
269624799292,20cyclictest659-21snmpd07:20:480
269624799292,20cyclictest12350irq/32-eth008:15:450
2696247992916,9cyclictest1895803-21cupsd08:25:460
2696247992916,9cyclictest1895803-21cupsd08:25:460
2696249992821,4cyclictest2697726-21idleruntime-cro07:15:002
2696249992820,5cyclictest516-21in:imuxsock07:10:022
2696249992820,5cyclictest516-21in:imuxsock07:10:022
2696249992819,7cyclictest2713907-21switchtime08:05:552
2696248992821,4cyclictest2708453-21/usr/sbin/munin07:50:541
2696248992819,6cyclictest2715735-21true08:15:011
2696248992819,6cyclictest1056-21mta-sts-daemon07:15:471
2696247992823,3cyclictest0-21swapper/012:25:020
2696247992821,5cyclictest548-21gdbus11:25:520
2696247992820,5cyclictest1895805-21cups-browsed08:00:400
2696247992818,7cyclictest2727706-21munin-plugin-st08:55:020
2696249992721,4cyclictest2868408-21unixbench_multi12:00:562
2696249992720,3cyclictest0-21swapper/208:10:522
2696249992719,5cyclictest420-21avahi-daemon07:25:572
2696249992719,5cyclictest420-21avahi-daemon07:25:572
2696247992719,5cyclictest2779702-21kworker/u8:0-xprtiod10:45:450
2696249992620,3cyclictest0-21swapper/208:30:002
2696249992620,3cyclictest0-21swapper/208:30:002
2696249992617,6cyclictest1055-21mta-sts-daemon11:45:272
2696248992610,10cyclictest1895805-21cups-browsed08:15:431
269624799267,11cyclictest171rcu_preempt07:15:470
2696247992610,6cyclictest0-21swapper/008:30:490
2696247992610,6cyclictest0-21swapper/008:30:490
269624999255,14cyclictest1055-21mta-sts-daemon08:40:472
2696249992519,3cyclictest0-21swapper/207:20:392
2696249992517,5cyclictest420-21avahi-daemon08:01:352
269624899252,18cyclictest2722634-21ntpq08:35:451
2696248992519,3cyclictest0-21swapper/107:35:001
2696248992519,3cyclictest0-21swapper/107:35:001
2696248992518,4cyclictest0-21swapper/108:05:371
2696247992516,7cyclictest2703597-21sed07:30:530
2696247992516,7cyclictest2703597-21sed07:30:530
269624999242,17cyclictest659-21snmpd09:00:482
2696248992419,2cyclictest0-21swapper/108:25:341
2696248992419,2cyclictest0-21swapper/108:25:341
2696248992417,4cyclictest1054-21mta-sts-daemon07:29:591
2696248992417,4cyclictest1054-21mta-sts-daemon07:29:591
2696248992415,7cyclictest2732235-21sh09:10:011
2696249992318,2cyclictest0-21swapper/207:35:452
2696249992318,2cyclictest0-21swapper/207:35:452
2696248992317,2cyclictest0-21swapper/107:10:381
269624799233,15cyclictest2697184-21ntp_states07:10:470
2696247992318,2cyclictest0-21swapper/008:05:560
269624799231,7cyclictest171rcu_preempt08:10:460
2696247992317,4cyclictest2724716-21cron08:45:010
2696247992215,4cyclictest12350irq/32-eth007:25:450
2696247992215,4cyclictest12350irq/32-eth007:25:450
269625099213,8cyclictest0-21swapper/311:45:443
269625099213,11cyclictest0-21swapper/309:25:513
269625099203,9cyclictest0-21swapper/309:55:473
269625099202,10cyclictest0-21swapper/309:20:493
2696248992015,3cyclictest0-21swapper/107:10:021
2696248992015,3cyclictest0-21swapper/107:10:021
269625099183,10cyclictest0-21swapper/309:45:463
269625099182,9cyclictest0-21swapper/309:30:473
269625099182,8cyclictest0-21swapper/310:25:463
269625099173,8cyclictest0-21swapper/311:20:473
269625099173,10cyclictest0-21swapper/309:40:373
269625099172,9cyclictest0-21swapper/311:40:473
269625099172,10cyclictest0-21swapper/311:30:423
269625099171,9cyclictest0-21swapper/310:10:493
269625099171,9cyclictest0-21swapper/310:10:493
269625099171,8cyclictest0-21swapper/309:35:423
269625099162,8cyclictest0-21swapper/311:25:473
269625099162,7cyclictest0-21swapper/311:15:443
269625099161,9cyclictest0-21swapper/311:50:423
269625099161,8cyclictest0-21swapper/310:35:433
269625099161,7cyclictest0-21swapper/307:10:453
269625099153,8cyclictest0-21swapper/311:00:443
269625099153,8cyclictest0-21swapper/311:00:443
269625099152,7cyclictest0-21swapper/311:55:443
269625099152,7cyclictest0-21swapper/311:05:483
269625099152,7cyclictest0-21swapper/311:05:483
269625099152,7cyclictest0-21swapper/310:00:463
269625099152,6cyclictest0-21swapper/307:25:013
269625099151,9cyclictest0-21swapper/309:50:523
269625099151,8cyclictest0-21swapper/310:55:503
269625099151,8cyclictest0-21swapper/310:50:513
269625099151,8cyclictest0-21swapper/310:50:513
269625099151,8cyclictest0-21swapper/310:40:433
269625099151,7cyclictest0-21swapper/310:45:433
269625099151,7cyclictest0-21swapper/308:55:453
269625099151,7cyclictest0-21swapper/308:15:443
269625099151,6cyclictest0-21swapper/307:50:453
269625099142,8cyclictest0-21swapper/310:15:443
269625099142,7cyclictest0-21swapper/312:35:443
269625099142,7cyclictest0-21swapper/312:30:443
269625099142,7cyclictest0-21swapper/312:05:473
269625099142,7cyclictest0-21swapper/311:10:423
269625099142,7cyclictest0-21swapper/311:10:423
269625099142,7cyclictest0-21swapper/310:05:443
269625099142,7cyclictest0-21swapper/310:05:443
269625099142,7cyclictest0-21swapper/307:40:473
269625099142,5cyclictest0-21swapper/308:20:453
269625099141,9cyclictest0-21swapper/309:15:463
269625099141,8cyclictest0-21swapper/312:00:493
269625099141,8cyclictest0-21swapper/310:20:513
269625099141,7cyclictest0-21swapper/308:00:493
269625099141,6cyclictest0-21swapper/308:35:423
269625099141,6cyclictest0-21swapper/307:35:443
269625099141,6cyclictest0-21swapper/307:35:443
269625099132,7cyclictest0-21swapper/312:25:473
269625099132,7cyclictest0-21swapper/312:25:473
269625099132,7cyclictest0-21swapper/308:30:443
269625099132,7cyclictest0-21swapper/308:30:443
269625099132,6cyclictest0-21swapper/311:35:453
269625099131,8cyclictest0-21swapper/312:20:463
269625099131,8cyclictest0-21swapper/312:15:483
269625099131,8cyclictest0-21swapper/310:30:373
269625099131,7cyclictest0-21swapper/309:10:353
269625099131,7cyclictest0-21swapper/307:15:463
269625099131,6cyclictest0-21swapper/308:25:473
269625099131,6cyclictest0-21swapper/308:25:473
269625099131,6cyclictest0-21swapper/308:10:443
269625099121,7cyclictest0-21swapper/307:35:013
269625099121,7cyclictest0-21swapper/307:35:013
269625099121,6cyclictest0-21swapper/312:10:093
269625099121,6cyclictest0-21swapper/308:40:423
269625099121,4cyclictest0-21swapper/307:45:413
269625099111,6cyclictest0-21swapper/309:00:403
269625099111,6cyclictest0-21swapper/308:50:493
269625099104,5cyclictest0-21swapper/308:05:483
269625099101,6cyclictest0-21swapper/309:05:493
269625099101,6cyclictest0-21swapper/308:45:353
269625099101,6cyclictest0-21swapper/307:55:453
269625099101,5cyclictest0-21swapper/307:25:413
269625099101,5cyclictest0-21swapper/307:25:413
26962509991,4cyclictest0-21swapper/307:10:033
26962509991,4cyclictest0-21swapper/307:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional