You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-18 - 08:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sat Jul 18, 2026 00:43:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2894565996352,7cyclictest1149763-21cups-browsed19:40:461
2894565995347,3cyclictest0-21swapper/119:20:441
289456499521,26cyclictest3040535-21ssh00:10:450
2894564995115,28cyclictest2953091-21ssh21:45:440
2894564995042,5cyclictest3006365-21systemd-run23:15:020
2894564994435,6cyclictest2983369-21ssh22:35:400
2894566994335,5cyclictest3006334-21idleruntime-cro23:15:012
289456599432,33cyclictest3019398-21ssh23:35:411
2894565994234,5cyclictest3025611-21proc_pri23:45:491
2894565994232,7cyclictest1000-21mta-sts-daemon19:30:421
2894566994133,5cyclictest3033348-21cron00:00:012
2894566994132,6cyclictest3024513-21latency_hist23:45:032
2894566994132,6cyclictest1149763-21cups-browsed22:13:072
2894565994134,3cyclictest0-21swapper/122:45:411
2894565994133,5cyclictest3045464-21latency_hist00:20:021
2894564994133,5cyclictest2977637-21sendmail_mailst22:25:490
2894565994034,3cyclictest0-21swapper/121:20:011
2894565994032,5cyclictest2932111-21sh21:10:531
2894565994031,6cyclictest2967569-21tail22:10:321
2894565994031,6cyclictest1149763-21cups-browsed22:30:321
2894565994031,5cyclictest1003-21mta-sts-daemon00:01:331
2894565994030,6cyclictest2930543-21munin-run21:10:011
2894564994030,7cyclictest3045420-21plymouth00:20:010
2894566993933,3cyclictest0-21swapper/221:35:012
2894566993931,5cyclictest2909551-21idleruntime-cro20:00:012
289456599393,26cyclictest2950094-21ssh21:40:471
289456599393,26cyclictest2950094-21ssh21:40:471
2894565993930,6cyclictest2944143-21sendmail_mailtr21:30:511
2894565993930,5cyclictest2920064-21systemd-run20:35:011
2894565993928,7cyclictest420-21avahi-daemon22:26:081
2894565993926,4cyclictest0-21swapper/123:22:171
289456599392,29cyclictest2983415-21ssh22:35:421
289456699383,29cyclictest3052506-21ssh00:30:422
289456599389,10cyclictest2930544-21diskmemload23:50:441
289456599383,26cyclictest2986404-21ssh22:40:411
2894565993832,4cyclictest2894563-21cyclictest00:30:411
2894565993830,5cyclictest2894563-21cyclictest20:25:451
2894565993830,4cyclictest0-21swapper/122:10:011
2894565993830,3cyclictest0-21swapper/121:40:011
2894565993828,7cyclictest2959143-21ntp_kernel_pll_21:55:451
2894565993828,6cyclictest2940445-21fschecks_count21:25:361
2894565993827,8cyclictest2974454-21sh22:20:481
2894565993826,9cyclictest2958018-21/usr/sbin/munin21:55:021
2894564993831,4cyclictest3022752-21switchtime23:40:520
2894564993830,6cyclictest2997306-21ssh23:00:010
2894564993830,5cyclictest1-21systemd22:55:010
2894564993826,8cyclictest2937668-21missed_timers21:20:380
2894566993729,6cyclictest2894563-21cyclictest21:40:362
2894566993729,6cyclictest2894563-21cyclictest21:40:362
2894565993729,5cyclictest1001-21mta-sts-daemon23:40:541
2894565993728,5cyclictest3057408-21ssh00:40:011
2894565993727,6cyclictest2998971-21unixbench_multi23:00:541
2894565993727,5cyclictest1003-21mta-sts-daemon00:26:191
2894565993724,9cyclictest3033423-21cups-browsed00:00:031
2894564993730,5cyclictest421-21dbus-daemon22:25:010
2894564993730,4cyclictest3000956-21ssh23:05:330
2894564993729,5cyclictest3007834-21timerandwakeup23:15:520
2894564993729,5cyclictest2980441-21ntp_kernel_pll_22:30:420
2894564993726,9cyclictest265-21systemd-journal23:05:020
2894564993718,4cyclictest0-21swapper/000:00:030
2894566993627,6cyclictest3043504-21ntp_kernel_pll_00:15:442
2894566993627,6cyclictest2934957-21/usr/sbin/munin21:15:512
2894566993627,6cyclictest1001-21mta-sts-daemon21:20:462
2894565993626,6cyclictest2961758-21fschecks_count22:00:351
2894565993625,7cyclictest1001-21mta-sts-daemon23:25:511
2894564993629,4cyclictest151ktimers/020:40:460
2894564993628,5cyclictest2943903-21ntpq21:30:440
2894564993626,7cyclictest151ktimers/019:10:420
2894566993528,4cyclictest2988364-21kworker/u8:9-xprtiod22:50:022
2894566993527,4cyclictest0-21swapper/223:20:422
2894566993526,6cyclictest485-21in:imuxsock23:20:002
2894566993525,7cyclictest420-21avahi-daemon00:29:212
2894566993524,8cyclictest2953007-21ntp_kernel_err21:45:412
289456699352,27cyclictest2983427-21ssh22:35:422
2894565993527,4cyclictest3037653-21proc_pri00:05:491
2894565993525,6cyclictest2926557-21fschecks_time20:55:371
2894564993527,4cyclictest151ktimers/000:30:410
2894564993526,7cyclictest2960659-21sshd21:58:490
2894566993428,3cyclictest0-21swapper/219:20:012
2894566993426,5cyclictest3027378-21true23:50:012
2894566993426,4cyclictest0-21swapper/221:40:012
2894566993424,7cyclictest420-21avahi-daemon22:09:492
289456699342,26cyclictest2971356-21ssh22:15:452
289456699342,26cyclictest2971356-21ssh22:15:452
2894565993426,3cyclictest0-21swapper/119:35:431
2894565993425,6cyclictest3007022-21fschecks_count23:15:341
2894565993425,4cyclictest0-21swapper/121:20:501
2894565993424,6cyclictest2908045-21grep19:55:011
2894565993423,5cyclictest0-21swapper/100:12:181
2894564993428,4cyclictest3019664-21sshd23:35:500
2894564993428,4cyclictest2967265-21latency_hist22:10:020
2894564993427,5cyclictest3028041-21fschecks_count23:50:360
2894564993427,5cyclictest2946539-21ssh21:35:350
2894564993427,4cyclictest3056065-21unixbench_multi00:35:590
2894564993427,4cyclictest2989162-21kernelversion22:45:370
2894564993424,8cyclictest421-21dbus-daemon22:15:490
2894564993424,8cyclictest421-21dbus-daemon22:15:490
2894566993326,4cyclictest2926034-21sh20:55:012
2894566993326,4cyclictest2915535-21cron20:20:012
2894566993325,5cyclictest439-21systemd-logind19:45:012
2894566993325,5cyclictest2932202-21unixbench_multi21:10:542
2894566993325,5cyclictest2908067-21systemd-run19:55:022
2894566993323,7cyclictest2986183-21kernelversion22:40:372
2894566993322,6cyclictest3019896-21timerwakeupswit23:35:532
289456699332,25cyclictest2995604-21ssh22:55:542
2894565993325,5cyclictest2914058-21systemd-run20:15:011
2894565993324,5cyclictest2911051-21cron20:05:011
2894565993324,4cyclictest0-21swapper/123:30:381
2894565993323,6cyclictest1149763-21cups-browsed20:35:391
2894565993323,6cyclictest1149763-21cups-browsed20:35:391
2894565993317,9cyclictest2930544-21diskmemload22:15:421
2894565993317,9cyclictest2930544-21diskmemload22:15:421
2894564993326,4cyclictest0-21swapper/023:20:350
2894564993324,7cyclictest2927524-21cron21:00:010
2894564993324,6cyclictest2912562-21sh20:10:010
2894564993322,8cyclictest2956454-21switchtime21:50:550
2894566993226,3cyclictest0-21swapper/223:25:462
2894566993226,3cyclictest0-21swapper/200:40:012
2894566993224,5cyclictest1003-21mta-sts-daemon00:02:372
2894566993223,7cyclictest2927563-21sh21:00:012
289456599323,21cyclictest3046560-21ssh00:20:451
2894565993225,4cyclictest2912490-21unixbench_singl20:05:531
2894564993225,3cyclictest0-21swapper/022:45:010
2894564993221,8cyclictest151ktimers/021:05:420
2894564993216,9cyclictest12650irq/32-eth023:30:510
2894566993125,3cyclictest0-21swapper/221:25:492
2894566993125,3cyclictest0-21swapper/220:30:522
2894566993124,4cyclictest1002-21mta-sts-daemon23:33:442
2894566993123,5cyclictest3000334-21cron23:05:012
2894566993123,5cyclictest2962724-21unixbench_multi22:00:542
2894566993123,4cyclictest0-21swapper/221:50:472
2894566993122,6cyclictest265-21systemd-journal22:55:012
2894565993125,3cyclictest0-21swapper/120:45:011
2894565993123,5cyclictest3001043-21fschecks_time23:05:351
2894565993122,6cyclictest3004365-21netstat23:10:411
2894565993122,6cyclictest2909414-21unixbench_multi19:55:551
2894565993122,4cyclictest0-21swapper/122:55:011
289456599312,21cyclictest665-21snmpd21:45:421
289456499313,22cyclictest3046493-21ssh00:20:420
2894566993024,3cyclictest0-21swapper/220:25:012
2894566993021,6cyclictest265-21systemd-journal19:50:022
2894566993018,8cyclictest3037906-21timerandwakeup00:05:532
2894565993021,4cyclictest2906573-21systemd19:50:021
2894564993023,5cyclictest0-21swapper/020:30:010
2894564993023,4cyclictest3035046-21unixbench_multi00:00:570
2894564993023,4cyclictest2970178-21cron22:15:010
2894564993019,9cyclictest421-21dbus-daemon19:50:010
2894564993018,9cyclictest0-21swapper/019:40:530
2894564993016,7cyclictest2930544-21diskmemload21:25:440
289456699294,16cyclictest428-21mta-sts-daemon00:20:442
2894566992922,3cyclictest0-21swapper/223:50:422
2894566992921,5cyclictest2930106-21sendmail_mailqu21:05:492
2894566992920,6cyclictest2980229-21latency22:30:372
2894566992920,6cyclictest265-21systemd-journal00:15:022
2894566992919,7cyclictest2913777-21timerandwakeup20:10:542
2894565992921,3cyclictest0-21swapper/122:55:541
2894565992920,6cyclictest0-21swapper/120:20:401
289456499293,21cyclictest3049488-21ssh00:25:400
289456499292,14cyclictest2950140-21sshd21:40:490
289456499292,14cyclictest2950140-21sshd21:40:490
2894564992921,4cyclictest12750irq/33-eth019:26:490
2894564992921,4cyclictest12750irq/33-eth019:26:490
2894564992919,7cyclictest3026015-21unixbench_multi23:45:540
2894564992918,8cyclictest265-21systemd-journal20:50:010
2894566992821,5cyclictest2911163-21latency_hist20:05:022
2894566992821,4cyclictest2923053-21cut20:45:002
2894564992821,5cyclictest0-21swapper/000:05:440
2894564992819,7cyclictest3013448-21grep23:25:440
2894564992817,9cyclictest1001-21mta-sts-daemon22:02:360
2894564992816,9cyclictest0-21swapper/020:20:010
2894566992721,4cyclictest2894563-21cyclictest20:06:392
2894566992721,3cyclictest0-21swapper/221:55:322
2894566992720,3cyclictest0-21swapper/223:05:502
2894566992720,3cyclictest0-21swapper/222:20:402
2894566992719,5cyclictest2977035-21fschecks_time22:25:352
2894566992719,5cyclictest2921438-21unixbench_multi20:35:532
2894566992719,5cyclictest2921438-21unixbench_multi20:35:532
2894566992718,6cyclictest2898819-21timerandwakeup19:20:522
289456599278,13cyclictest421-21dbus-daemon19:15:471
2894565992721,3cyclictest0-21swapper/120:55:011
2894565992720,3cyclictest0-21swapper/121:00:361
2894564992721,3cyclictest0-21swapper/021:15:330
2894564992719,6cyclictest2910317-21missed_timers20:00:370
2894564992719,6cyclictest2899003-21unixbench_singl19:20:530
2894564992713,11cyclictest0-21swapper/019:40:000
2894566992620,3cyclictest0-21swapper/219:30:002
2894566992620,3cyclictest0-21swapper/219:30:002
2894566992619,5cyclictest2895617-21mailstats19:10:502
2894566992616,7cyclictest2902054-21cut19:35:012
2894565992620,3cyclictest0-21swapper/120:20:011
289456499263,13cyclictest428-21mta-sts-daemon20:20:460
2894564992622,3cyclictest2894563-21cyclictest20:51:590
289456499261,21cyclictest2913560-21ps20:10:500
2894566992517,5cyclictest2903545-21munin-run19:40:002
2894565992518,4cyclictest271ktimers/119:25:441
2894565992518,4cyclictest271ktimers/119:25:441
2894564992516,4cyclictest2909657-21latency_hist20:00:020
289456499247,12cyclictest12650irq/32-eth021:10:410
2894564992419,3cyclictest2880532-21kworker/u8:4-xprtiod19:55:010
2894564992418,4cyclictest2928077-21fschecks_time21:00:360
2894566992315,5cyclictest2928058-21fschecks_count21:00:352
289456699231,15cyclictest2917941-21ntpq20:25:442
2894565992317,3cyclictest0-21swapper/119:10:551
2894565992312,8cyclictest271ktimers/120:45:421
2894564992315,5cyclictest0-21swapper/020:30:520
2894566992217,2cyclictest0-21swapper/220:50:022
2894565992214,4cyclictest0-21swapper/119:10:011
2894564992215,5cyclictest485-21in:imuxsock19:18:490
289456499221,18cyclictest2921010-21ntpq20:35:470
289456499221,18cyclictest2921010-21ntpq20:35:470
289456499221,18cyclictest2901491-21ntpq19:30:440
2894566991812,3cyclictest0-21swapper/219:10:012
289456499173,8cyclictest2894520-21munin-run19:10:010
289456799164,5cyclictest0-21swapper/323:40:413
289456799164,5cyclictest0-21swapper/323:25:453
289456799163,8cyclictest0-21swapper/321:30:493
289456799163,7cyclictest0-21swapper/319:40:463
289456799157,4cyclictest0-21swapper/322:30:313
289456799153,7cyclictest0-21swapper/321:20:543
289456799152,8cyclictest0-21swapper/323:00:443
289456799152,8cyclictest0-21swapper/322:55:523
289456799152,8cyclictest0-21swapper/321:45:453
289456799151,8cyclictest0-21swapper/323:55:483
289456799151,8cyclictest0-21swapper/323:30:413
289456799151,8cyclictest0-21swapper/322:50:443
289456799151,7cyclictest0-21swapper/321:35:483
289456799146,4cyclictest0-21swapper/323:23:013
289456799145,7cyclictest0-21swapper/300:10:433
289456799143,7cyclictest0-21swapper/322:25:403
289456799142,7cyclictest0-21swapper/322:10:443
289456799142,6cyclictest0-21swapper/321:25:323
289456799142,6cyclictest0-21swapper/319:10:433
289456799142,5cyclictest0-21swapper/320:05:413
289456799141,9cyclictest0-21swapper/322:40:353
289456799141,7cyclictest0-21swapper/323:15:353
289456799141,7cyclictest0-21swapper/322:05:403
289456799141,7cyclictest0-21swapper/321:55:433
289456799141,7cyclictest0-21swapper/320:30:413
289456799141,7cyclictest0-21swapper/319:20:413
289456799141,7cyclictest0-21swapper/300:30:463
289456799141,7cyclictest0-21swapper/300:25:473
289456799141,6cyclictest0-21swapper/321:05:413
289456799141,6cyclictest0-21swapper/320:20:433
289456799141,6cyclictest0-21swapper/319:35:453
289456799136,4cyclictest0-21swapper/322:45:443
289456799136,4cyclictest0-21swapper/321:40:443
289456799136,4cyclictest0-21swapper/321:40:443
289456799135,5cyclictest0-21swapper/300:20:543
289456799135,4cyclictest0-21swapper/323:05:473
289456799132,7cyclictest0-21swapper/300:35:393
289456799132,6cyclictest0-21swapper/322:35:313
289456799132,6cyclictest0-21swapper/300:15:443
289456799132,5cyclictest0-21swapper/320:15:453
289456799131,7cyclictest0-21swapper/320:15:023
289456799131,7cyclictest0-21swapper/300:05:443
289456799131,7cyclictest0-21swapper/300:00:583
289456799131,6cyclictest0-21swapper/322:15:323
289456799131,6cyclictest0-21swapper/322:15:323
289456799131,6cyclictest0-21swapper/319:30:473
289456799131,5cyclictest0-21swapper/320:00:413
289456799127,3cyclictest0-21swapper/322:00:483
289456799126,4cyclictest0-21swapper/323:48:503
289456799126,4cyclictest0-21swapper/322:20:423
289456799125,4cyclictest0-21swapper/320:55:393
289456799121,7cyclictest0-21swapper/323:35:323
289456799121,7cyclictest0-21swapper/323:10:323
289456799121,7cyclictest0-21swapper/321:50:353
289456799121,6cyclictest0-21swapper/321:15:413
289456799121,6cyclictest0-21swapper/321:10:363
289456799121,6cyclictest0-21swapper/321:00:463
289456799121,5cyclictest0-21swapper/320:40:423
289456799121,5cyclictest0-21swapper/319:15:473
289456799120,6cyclictest0-21swapper/319:55:443
289456799116,3cyclictest0-21swapper/320:45:323
289456799115,3cyclictest0-21swapper/320:25:393
289456799112,5cyclictest0-21swapper/319:10:013
289456799111,5cyclictest0-21swapper/323:50:243
289456799105,3cyclictest0-21swapper/320:50:493
289456799105,3cyclictest0-21swapper/319:25:363
289456799105,3cyclictest0-21swapper/319:25:363
289456799104,3cyclictest0-21swapper/319:45:433
28945679994,3cyclictest0-21swapper/319:50:313
28945679991,5cyclictest0-21swapper/320:35:333
28945679991,5cyclictest0-21swapper/320:35:333
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional