You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-25 - 12:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Mon May 25, 2026 00:43:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3005984995734,5cyclictest3100345-21unixbench-2d22:21:091
3005985995042,5cyclictest3054114-21sh21:25:002
3005983994942,4cyclictest3054801-21aten_r2power_en21:25:470
3005985994838,7cyclictest3200753-21/usr/sbin/munin00:26:022
3005984994839,6cyclictest1-21systemd23:15:011
3005984994839,5cyclictest3179094-21grep00:00:001
3005983994839,6cyclictest12350irq/32-eth023:51:020
3005985994637,6cyclictest3150900-21unin-run23:25:022
3005984994635,8cyclictest3082287-21sh22:00:011
3005984994538,3cyclictest0-21swapper/122:40:001
3005984994538,3cyclictest0-21swapper/122:40:001
3005983994538,5cyclictest1-21systemd00:00:010
3005983994437,4cyclictest3185117-21grep00:06:130
3005983994436,5cyclictest3131627-21fschecks_count23:00:520
3005985994334,6cyclictest3106507-21unin-run22:30:022
3005985994332,9cyclictest439-21systemd-logind19:35:012
3005985994329,10cyclictest1-21systemd22:00:012
3005984994335,5cyclictest3158926-21systemd-run23:35:011
300598399434,8cyclictest3047404-21ssh21:16:000
300598399434,8cyclictest3047404-21ssh21:16:000
3005983994324,16cyclictest12350irq/32-eth023:20:580
3005985994234,5cyclictest3207316-21systemd-run00:35:012
3005984994233,6cyclictest3185113-21unixbench-2d00:06:131
3005983994234,5cyclictest3074212-21systemd21:50:020
3005983994234,5cyclictest3074212-21systemd21:50:020
3005984994135,3cyclictest0-21swapper/121:30:001
3005983994135,4cyclictest3195207-21idleruntime-cro00:20:000
3005983994135,3cyclictest0-21swapper/022:15:540
3005983994134,4cyclictest3091203-21rm22:10:490
300598399413,34cyclictest3180519-21ssh00:01:000
3005983994130,8cyclictest261-21systemd-journal00:25:000
3005985994033,4cyclictest1055-21mta-sts-daemon20:10:562
3005985994030,7cyclictest3050189-21latency_hist21:20:022
3005985994030,7cyclictest3050189-21latency_hist21:20:022
3005984994032,5cyclictest3180375-21needreboot00:00:551
3005984994032,5cyclictest3126720-21latency_hist22:55:021
3005984994032,5cyclictest3050058-21munin-run21:20:011
3005984994032,5cyclictest3050058-21munin-run21:20:011
3005984994027,10cyclictest3051385-21sh21:20:561
3005983994033,4cyclictest3034488-21systemd-run20:45:020
3005983994033,4cyclictest3020994-21systemd-run20:00:010
3005983994032,6cyclictest3188094-21sh00:10:520
3005983994032,6cyclictest3188094-21sh00:10:520
3005983994028,9cyclictest0-21swapper/021:40:470
3005985993932,4cyclictest3038949-21awk21:00:012
3005985993931,5cyclictest3197172-21unixbench_multi00:21:102
3005985993931,5cyclictest3066088-21cron21:40:012
3005985993931,5cyclictest0-21swapper/222:36:112
3005985993931,5cyclictest0-21swapper/222:36:112
3005985993930,6cyclictest1225109-21cups-browsed22:48:232
3005984993930,6cyclictest3096293-21unixbench-2d22:16:101
3005984993930,6cyclictest3083832-21proc_pri22:01:041
3005984993930,5cyclictest0-21swapper/121:31:011
3005984993930,4cyclictest0-21swapper/122:30:571
3005984993930,4cyclictest0-21swapper/122:30:571
3005984993929,7cyclictest3007487-21systemd-run19:15:011
3005984993929,7cyclictest3007487-21systemd-run19:15:011
3005984993929,6cyclictest3134748-21systemd-run23:05:011
3005984993927,9cyclictest3138733-21sh23:10:011
3005983993925,12cyclictest3038941-21idleruntime-cro21:00:010
3005983993925,11cyclictest0-21swapper/022:50:010
3005983993923,13cyclictest3207341-21latency_hist00:35:020
3005985993832,3cyclictest0-21swapper/200:40:012
3005985993832,3cyclictest0-21swapper/200:40:012
3005985993832,3cyclictest0-21swapper/200:06:042
3005985993830,5cyclictest3175732-21aten_r2power_en23:55:472
3005985993830,5cyclictest3154875-21munin-plugin-st23:30:002
3005985993830,5cyclictest3154875-21munin-plugin-st23:30:002
3005985993829,6cyclictest3179076-21cups-browsed00:15:022
3005985993829,6cyclictest3179076-21cups-browsed00:15:022
3005985993829,6cyclictest3098403-21ssh22:20:012
300598599382,28cyclictest3140051-21ssh23:10:572
3005984993830,4cyclictest0-21swapper/100:36:021
3005984993830,4cyclictest0-21swapper/100:36:021
3005983993831,5cyclictest3050681-21/usr/sbin/munin21:21:060
3005983993831,4cyclictest3162940-21cron23:40:000
3005983993831,4cyclictest0-21swapper/022:41:000
3005985993728,6cyclictest1225109-21cups-browsed22:33:432
3005985993728,6cyclictest1225109-21cups-browsed22:33:432
3005984993730,5cyclictest3078204-21cron21:55:001
3005984993730,5cyclictest3078204-21cron21:55:001
3005984993729,5cyclictest3191194-21munin-run00:15:011
3005984993729,5cyclictest3191194-21munin-run00:15:011
3005984993729,5cyclictest1054-21mta-sts-daemon00:26:081
3005983993729,5cyclictest420-21avahi-daemon22:54:370
3005983993723,11cyclictest0-21swapper/019:26:080
3005985993629,3cyclictest0-21swapper/223:40:582
3005985993628,5cyclictest3146814-21run-parts23:20:012
3005985993628,5cyclictest3094493-21latency_hist22:15:022
3005985993628,5cyclictest3027011-21latency_hist20:20:012
3005985993628,5cyclictest1-21systemd19:45:022
3005985993628,4cyclictest0-21swapper/221:11:052
3005984993628,5cyclictest3040459-21cron21:05:011
3005984993628,5cyclictest0-21swapper/120:25:011
3005984993628,5cyclictest0-21swapper/120:25:011
3005984993627,5cyclictest3192578-21sshd00:16:011
3005984993627,5cyclictest0-21swapper/123:36:021
3005984993626,6cyclictest3063623-21proc_pri21:36:051
3005983993631,3cyclictest3126653-21kworker/u8:3-rpciod23:16:040
3005983993630,3cyclictest0-21swapper/019:25:010
3005983993627,5cyclictest0-21swapper/023:45:560
3005983993623,8cyclictest3107957-21sh22:31:020
3005983993623,8cyclictest3107957-21sh22:31:020
3005985993530,2cyclictest0-21swapper/223:40:012
3005985993528,4cyclictest3090336-21sh22:10:012
3005985993527,5cyclictest0-21swapper/200:00:492
3005985993525,5cyclictest3156791-21/usr/sbin/munin23:31:112
3005984993527,5cyclictest3041959-21diskmemload22:30:011
3005984993527,5cyclictest0-21swapper/121:10:011
3005984993526,5cyclictest3008970-21munin-run19:20:001
3005984993526,5cyclictest3008970-21munin-run19:20:001
3005984993523,8cyclictest3168717-21switchtime23:46:081
3005984993520,13cyclictest3035954-21cron20:50:011
3005983993527,5cyclictest1225109-21cups-browsed22:24:440
3005983993525,6cyclictest3135987-21needreboot23:05:570
300598399352,27cyclictest3200541-21ssh00:25:540
3005983993522,11cyclictest0-21swapper/023:30:490
300598399351,11cyclictest3113682-21sh22:38:210
300598399351,11cyclictest3113682-21sh22:38:210
3005985993428,3cyclictest3020989-21kworker/u8:11-xprtiod20:05:012
3005985993427,4cyclictest3134730-21idleruntime-cro23:05:012
3005985993426,5cyclictest3126639-21munin-run22:55:012
3005985993417,10cyclictest3041959-21diskmemload22:55:562
3005984993427,4cyclictest1055-21mta-sts-daemon22:59:391
3005984993427,3cyclictest0-21swapper/122:12:051
3005984993426,5cyclictest3152835-21unixbench_multi23:26:111
3005984993426,5cyclictest3152835-21unixbench_multi23:26:111
3005984993425,6cyclictest421-21dbus-daemon22:10:011
3005984993425,6cyclictest1225109-21cups-browsed23:53:421
3005983993427,4cyclictest3082245-21idleruntime-cro22:00:000
3005983993420,8cyclictest3208655-21sh00:35:590
3005983993420,8cyclictest3208655-21sh00:35:590
3005983993418,13cyclictest3066156-21latency_hist21:40:020
300598399341,28cyclictest151ktimers/020:20:560
300598399341,28cyclictest151ktimers/020:20:560
3005985993328,2cyclictest0-21swapper/223:05:582
3005985993328,2cyclictest0-21swapper/220:00:012
3005985993325,5cyclictest1225109-21cups-browsed21:51:302
3005985993325,5cyclictest1225109-21cups-browsed21:51:302
3005985993324,5cyclictest3083195-21grep22:00:502
3005985993318,11cyclictest1056-21mta-sts-daemon00:16:062
3005984993326,4cyclictest3197213-21head00:21:101
3005984993325,5cyclictest3148823-21unixbench_singl23:21:101
3005984993324,6cyclictest1-21systemd19:35:011
300598399336,17cyclictest151ktimers/020:30:560
3005983993326,4cyclictest0-21swapper/021:10:500
3005983993320,4cyclictest3166994-21cron23:45:010
3005985993226,3cyclictest0-21swapper/222:20:572
3005985993226,3cyclictest0-21swapper/220:40:002
3005985993225,3cyclictest0-21swapper/221:26:032
3005985993224,5cyclictest3014115-21kernelversion19:35:532
3005985993224,5cyclictest1225109-21cups-browsed19:29:362
3005984993226,3cyclictest3054149-21kworker/u8:5-xprtiod22:42:091
3005984993226,3cyclictest0-21swapper/122:50:001
3005984993224,5cyclictest3163910-21ssh23:40:511
3005984993224,5cyclictest3031977-21fschecks_count20:35:521
300598399325,16cyclictest12350irq/32-eth022:00:580
3005983993225,3cyclictest0-21swapper/022:55:570
3005983993224,3cyclictest0-21swapper/021:30:510
3005983993219,10cyclictest3040457-21cron21:05:010
3005983993214,9cyclictest151ktimers/020:50:590
3005985993123,5cyclictest3006673-21latency19:10:542
3005985993123,5cyclictest3006673-21latency19:10:542
3005985993123,3cyclictest0-21swapper/221:41:032
300598599312,24cyclictest3071468-21ssh21:45:552
300598599312,24cyclictest3071468-21ssh21:45:552
300598599312,22cyclictest3172483-21ssh23:51:012
3005985993119,9cyclictest3017370-21ntp_kernel_pll_19:45:582
3005984993125,3cyclictest3054234-21kworker/u8:7-xprtiod23:20:011
3005984993124,4cyclictest3204413-21latency00:30:531
3005984993122,6cyclictest3014562-21mailstats19:36:071
3005984993122,5cyclictest0-21swapper/121:40:561
3005983993122,6cyclictest3016477-21/usr/sbin/munin19:45:020
3005983993122,6cyclictest12350irq/32-eth022:05:590
3005983993121,8cyclictest518-21rs:main0
3005983993121,8cyclictest518-21rs:main0
300598399312,15cyclictest12350irq/32-eth022:25:570
3005985993022,5cyclictest3115810-21needreboot22:40:532
3005985993021,7cyclictest3058351-21sh21:30:102
300598499302,21cyclictest3071544-21ssh21:45:581
300598499302,21cyclictest3071544-21ssh21:45:581
3005983993022,6cyclictest3024858-21ntp_kernel_pll_20:10:580
300598399302,19cyclictest3152201-21ssh23:25:570
300598399302,19cyclictest3152201-21ssh23:25:570
3005985992922,3cyclictest0-21swapper/223:45:592
3005984992922,3cyclictest0-21swapper/121:11:021
3005984992921,5cyclictest1056-21mta-sts-daemon20:06:101
3005983992921,5cyclictest3140601-21timerwakeupswit23:11:100
3005984992820,5cyclictest421-21dbus-daemon19:51:381
3005983992820,5cyclictest3014863-21unixbench_multi19:36:100
3005983992819,6cyclictest151ktimers/020:45:580
3005983992816,6cyclictest3008831-21unixbench_multi19:16:080
3005983992816,6cyclictest3008831-21unixbench_multi19:16:080
300598399281,21cyclictest151ktimers/020:16:000
3005985992722,2cyclictest0-21swapper/219:25:012
3005985992720,4cyclictest1051-21mta-sts-daemon19:51:102
3005985992719,5cyclictest1225109-21cups-browsed21:06:062
3005985992716,8cyclictest3028490-21systemd-run20:25:012
3005985992716,8cyclictest3028490-21systemd-run20:25:012
3005984992721,3cyclictest0-21swapper/120:05:011
3005984992719,5cyclictest3037158-21find20:51:081
3005983992721,4cyclictest3019669-21latency_hist19:55:020
3005983992720,5cyclictest439-21systemd-logind21:06:070
3005983992718,6cyclictest1056-21mta-sts-daemon19:32:110
3005983992713,11cyclictest3006003-21cat19:10:010
3005985992621,2cyclictest0-21swapper/221:05:012
3005985992620,3cyclictest0-21swapper/220:26:092
3005985992620,3cyclictest0-21swapper/220:26:092
3005985992618,5cyclictest3023281-21needreboot20:05:572
3005984992618,5cyclictest3024578-21if_wlan020:10:511
300598399263,18cyclictest3021939-21cron20:01:000
3005983992618,5cyclictest421-21dbus-daemon20:30:010
3005983992618,5cyclictest421-21dbus-daemon20:30:010
3005985992517,5cyclictest3034457-21cron20:45:012
3005985992517,5cyclictest3008838-21unixbench_multi19:16:082
3005985992517,5cyclictest3008838-21unixbench_multi19:16:082
3005984992519,3cyclictest0-21swapper/120:00:011
3005984992517,5cyclictest3031170-21threads20:31:071
3005984992517,5cyclictest1225109-21cups-browsed20:55:511
3005983992519,3cyclictest1051-21mta-sts-daemon20:36:120
3005983992512,2cyclictest0-21swapper/019:11:040
3005983992512,2cyclictest0-21swapper/019:11:040
300598399251,19cyclictest1-21systemd20:10:010
3005985992419,3cyclictest1054-21mta-sts-daemon20:45:572
3005985992419,2cyclictest0-21swapper/220:50:522
3005985992419,2cyclictest0-21swapper/220:35:002
300598499243,16cyclictest1056-21mta-sts-daemon19:40:561
3005984992417,4cyclictest261-21systemd-journal20:19:511
3005984992417,4cyclictest0-21swapper/119:26:081
3005983992419,3cyclictest3017969-21munin-plugin-st19:50:010
300598499232,16cyclictest3033888-21ntpq20:41:041
3005984992316,4cyclictest1054-21mta-sts-daemon19:49:051
3005984992314,6cyclictest1225109-21cups-browsed20:25:441
3005984992314,6cyclictest1225109-21cups-browsed20:25:441
300598499222,14cyclictest3009887-21ntpq19:20:581
3005984992110,8cyclictest3006044-21wc19:10:011
300598699173,7cyclictest0-21swapper/323:25:573
300598699173,7cyclictest0-21swapper/323:25:573
300598699152,7cyclictest0-21swapper/322:41:003
300598699151,8cyclictest0-21swapper/321:55:573
300598699151,6cyclictest0-21swapper/320:25:553
300598699151,6cyclictest0-21swapper/320:25:553
300598699151,6cyclictest0-21swapper/300:16:003
300598699151,5cyclictest0-21swapper/320:10:553
300598599153,9cyclictest3005939-21/usr/sbin/munin19:10:012
300598699142,7cyclictest0-21swapper/322:46:063
300598699142,6cyclictest0-21swapper/322:15:563
300598699142,6cyclictest0-21swapper/322:00:593
300598699142,6cyclictest0-21swapper/320:51:003
300598699142,5cyclictest0-21swapper/320:30:593
300598699141,7cyclictest0-21swapper/323:41:043
300598699141,7cyclictest0-21swapper/300:11:063
300598699141,7cyclictest0-21swapper/300:11:063
300598699141,6cyclictest0-21swapper/323:30:583
300598699141,6cyclictest0-21swapper/322:10:573
300598699141,6cyclictest0-21swapper/320:41:053
300598699141,6cyclictest0-21swapper/319:26:003
300598699141,6cyclictest0-21swapper/300:30:583
300598699133,5cyclictest0-21swapper/321:20:563
300598699132,5cyclictest0-21swapper/319:40:543
300598699132,4cyclictest0-21swapper/320:45:593
300598699131,7cyclictest0-21swapper/322:31:053
300598699131,7cyclictest0-21swapper/322:31:053
300598699131,7cyclictest0-21swapper/300:25:573
300598699131,7cyclictest0-21swapper/300:21:023
300598699131,7cyclictest0-21swapper/300:00:583
300598699131,6cyclictest0-21swapper/321:05:583
300598699131,5cyclictest0-21swapper/319:55:583
300598699131,5cyclictest0-21swapper/319:10:583
300598699131,5cyclictest0-21swapper/319:10:583
300598699122,6cyclictest0-21swapper/322:25:543
300598699121,7cyclictest0-21swapper/323:35:543
300598699121,7cyclictest0-21swapper/323:00:493
300598699121,7cyclictest0-21swapper/322:55:553
300598699121,7cyclictest0-21swapper/322:21:033
300598699121,7cyclictest0-21swapper/321:15:583
300598699121,7cyclictest0-21swapper/321:15:583
300598699121,6cyclictest0-21swapper/323:50:533
300598699121,6cyclictest0-21swapper/323:15:573
300598699121,6cyclictest0-21swapper/322:35:593
300598699121,6cyclictest0-21swapper/322:35:593
300598699121,6cyclictest0-21swapper/321:30:533
300598699121,6cyclictest0-21swapper/321:01:003
300598699121,5cyclictest0-21swapper/319:46:013
300598699120,6cyclictest0-21swapper/319:30:583
300598699114,5cyclictest0-21swapper/300:36:003
300598699114,5cyclictest0-21swapper/300:36:003
300598699112,5cyclictest0-21swapper/320:21:023
300598699112,5cyclictest0-21swapper/320:21:023
300598699112,5cyclictest0-21swapper/300:06:003
300598699111,7cyclictest0-21swapper/321:40:513
300598699111,6cyclictest0-21swapper/323:55:563
300598699111,6cyclictest0-21swapper/323:10:513
300598699111,6cyclictest0-21swapper/321:50:493
300598699111,6cyclictest0-21swapper/321:50:493
300598699111,6cyclictest0-21swapper/321:45:553
300598699111,6cyclictest0-21swapper/321:45:553
300598699111,6cyclictest0-21swapper/321:10:523
300598699111,5cyclictest0-21swapper/322:05:473
300598699111,5cyclictest0-21swapper/320:00:573
300598699103,5cyclictest0-21swapper/321:35:493
300598699103,5cyclictest0-21swapper/321:25:473
300598699102,4cyclictest0-21swapper/323:20:473
300598699102,4cyclictest0-21swapper/319:51:063
300598699101,6cyclictest0-21swapper/323:45:473
300598699101,6cyclictest0-21swapper/322:50:503
300598699101,6cyclictest0-21swapper/319:36:033
300598699101,5cyclictest0-21swapper/323:05:493
300598699101,5cyclictest0-21swapper/320:05:513
300598699101,5cyclictest0-21swapper/319:15:593
300598699101,5cyclictest0-21swapper/319:15:593
30059869993,3cyclictest0-21swapper/320:15:573
30059869991,5cyclictest0-21swapper/320:55:563
30059869991,5cyclictest0-21swapper/320:35:513
30059869991,4cyclictest0-21swapper/319:20:593
30059869971,3cyclictest0-21swapper/319:10:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional