You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-17 - 09:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Wed Jun 17, 2026 00:43:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3171022995946,10cyclictest3313081-21munin-run00:00:010
3171023995543,9cyclictest3307911-21sshd23:50:281
317102299526,36cyclictest3336006-21ssh00:35:290
3171024994841,3cyclictest0-21swapper/221:50:392
3171022994723,17cyclictest151ktimers/023:00:310
3171024994639,4cyclictest3260131-21idleruntime-cro22:35:012
317102399457,34cyclictest3214480-21ssh21:20:421
3171022994522,15cyclictest151ktimers/020:25:280
3171023994436,4cyclictest0-21swapper/123:55:271
317102399435,29cyclictest3332836-21ssh00:30:301
3171023994334,6cyclictest3172528-21systemd-run19:15:021
317102399433,29cyclictest3217551-21ssh21:25:361
3171023994318,3cyclictest171rcu_preempt21:45:281
317102499423,33cyclictest3214378-21ssh21:20:352
3171024994230,8cyclictest3273414-21latency22:55:242
3171023994234,5cyclictest3244628-21latency_hist22:10:021
3171023994234,5cyclictest3244628-21latency_hist22:10:021
3171023994233,5cyclictest3224114-21timerandwakeup21:35:381
3171023994225,3cyclictest171rcu_preempt19:40:271
3171022994237,3cyclictest0-21swapper/021:45:270
3171022994219,20cyclictest3239517-21rm22:00:350
3171022994219,20cyclictest3239517-21rm22:00:350
3171024994134,4cyclictest3322579-21latency_hist00:15:022
317102399414,28cyclictest3236199-21ssh21:55:271
3171023994122,4cyclictest171rcu_preempt21:40:321
317102299413,30cyclictest3223744-21ssh21:35:300
3171022994131,5cyclictest0-21swapper/000:00:260
3171022994124,12cyclictest12650irq/32-eth023:25:340
3171024994033,4cyclictest0-21swapper/222:25:272
3171024994032,4cyclictest3334965-21kworker/u8:100:35:012
3171024994031,6cyclictest3247682-21true22:15:012
3171024994031,6cyclictest3247682-21true22:15:012
3171024994030,6cyclictest3263820-21cpuspeed22:40:192
317102399403,26cyclictest3317243-21ssh00:05:311
3171023994032,5cyclictest3204525-21fschecks_count21:00:181
3171023994031,6cyclictest1-21systemd19:20:021
3171022994034,4cyclictest3280365-21timerwakeupswit23:05:430
3171022994034,4cyclictest3247778-21latency_hist22:15:020
3171022994034,4cyclictest3247778-21latency_hist22:15:020
3171024993932,3cyclictest3278878-21kworker/u8:9-xprtiod00:00:012
3171024993931,4cyclictest3171027-21kworker/u8:10-rpciod19:20:022
3171024993928,7cyclictest3313177-21cups-browsed00:37:502
3171023993931,5cyclictest1000-21mta-sts-daemon23:30:301
3171023993931,4cyclictest0-21swapper/100:35:311
3171023993930,4cyclictest0-21swapper/123:35:261
3171022993930,7cyclictest3275754-21systemd-run23:00:010
317102299392,34cyclictest3324121-21users00:15:410
3171024993831,4cyclictest3199536-21kworker/u8:13+xprtiod21:50:022
3171024993830,5cyclictest3327156-21ssh00:20:412
3171024993830,5cyclictest3198056-21latency_hist20:40:012
3171024993829,4cyclictest0-21swapper/222:45:282
317102499382,27cyclictest3248745-21ssh22:15:332
317102399385,25cyclictest3220561-21ssh21:30:291
3171023993831,4cyclictest3210272-21latency_hist21:15:021
3171023993828,7cyclictest420-21avahi-daemon21:19:581
3171023993828,7cyclictest1003-21mta-sts-daemon22:45:011
317102299385,23cyclictest3316267-21kworker/u8:10+rpciod00:30:290
317102299383,32cyclictest3326633-21ssh00:20:290
3171024993729,5cyclictest3204016-21sendmail21:00:012
3171024993728,6cyclictest3208547-21sed21:10:392
317102399379,3cyclictest171rcu_preempt00:15:311
3171023993731,4cyclictest0-21swapper/122:25:291
3171023993730,4cyclictest3285040-21ssh23:15:011
3171023993728,6cyclictest3279483-21fschecks_time23:05:221
3171022993730,4cyclictest3254828-21needreboot22:25:250
3171022993727,7cyclictest3329949-21find00:25:350
3171022993726,9cyclictest151ktimers/023:20:250
3171022993715,16cyclictest12650irq/32-eth023:50:300
3171024993628,5cyclictest0-21swapper/219:15:012
317102399363,28cyclictest3251916-21ssh22:20:321
3171023993613,17cyclictest421-21dbus-daemon20:50:021
317102399361,28cyclictest3258092-21ssh22:30:301
3171022993627,7cyclictest3298418-21needreboot23:35:240
317102299362,28cyclictest3236185-21ssh21:55:260
317102299362,26cyclictest3267351-21ssh22:45:260
3171024993527,5cyclictest421-21dbus-daemon19:40:012
3171024993526,6cyclictest420-21avahi-daemon20:00:352
3171023993526,6cyclictest420-21avahi-daemon22:03:261
3171023993526,6cyclictest420-21avahi-daemon22:03:261
3171023993526,6cyclictest1427483-21cups-browsed19:37:471
3171023993524,3cyclictest261rcuc/100:00:251
3171023993518,4cyclictest171rcu_preempt20:50:321
3171022993523,8cyclictest12650irq/32-eth000:10:290
317102299351,33cyclictest0-21swapper/020:10:330
317102299351,31cyclictest665-21snmpd23:40:560
3171024993426,5cyclictest3281948-21latency_hist23:10:012
3171024993426,3cyclictest0-21swapper/222:00:262
3171024993426,3cyclictest0-21swapper/222:00:262
3171023993425,5cyclictest3263260-21sendmail22:40:011
3171023993414,4cyclictest171rcu_preempt20:20:251
317102299346,20cyclictest12650irq/32-eth022:20:290
3171022993428,4cyclictest3235187-21munin-plugin-st21:55:000
3171022993428,4cyclictest3220293-21kernelversion21:30:230
3171022993426,4cyclictest0-21swapper/022:15:300
3171022993425,6cyclictest3286678-21head23:15:410
3171022993424,8cyclictest151ktimers/019:25:340
3171022993423,9cyclictest3179313-21needreboot19:35:240
3171022993414,15cyclictest12650irq/32-eth021:10:260
3171024993328,2cyclictest0-21swapper/221:20:022
3171024993326,3cyclictest3183028-21kworker/u8:14-xprtiod23:50:162
3171024993325,5cyclictest3183035-21kworker/u8:15+xprtiod20:35:012
3171024993325,5cyclictest3183035-21kworker/u8:15+xprtiod20:35:012
3171024993324,4cyclictest0-21swapper/222:50:282
3171024993323,7cyclictest694-21cron22:10:002
3171024993323,7cyclictest694-21cron22:10:002
317102499331,26cyclictest3261151-21ssh22:35:282
3171023993326,5cyclictest57-21kcompactd023:20:301
3171023993325,5cyclictest3245732-21proc_pri22:10:321
3171023993325,5cyclictest3245732-21proc_pri22:10:321
3171023993324,4cyclictest0-21swapper/123:50:011
3171023993317,4cyclictest171rcu_preempt23:15:311
3171022993323,8cyclictest0-21swapper/020:40:400
3171024993226,4cyclictest3171021-21cyclictest20:45:302
3171024993226,3cyclictest0-21swapper/223:45:312
3171024993225,3cyclictest0-21swapper/223:10:282
3171024993224,5cyclictest3175519-21cron19:25:012
3171024993222,8cyclictest3191171-21latency20:15:222
317102499322,23cyclictest3226824-21ssh21:40:252
3171024993221,8cyclictest3186032-21sh20:00:012
3171024993221,6cyclictest3207013-21diskmemload22:20:272
3171024993213,10cyclictest3207013-21diskmemload23:00:312
3171023993225,5cyclictest0-21swapper/122:50:011
3171023993225,3cyclictest0-21swapper/119:10:011
3171023993224,5cyclictest421-21dbus-daemon22:56:491
3171023993217,3cyclictest171rcu_preempt00:10:201
3171022993226,4cyclictest3181631-21latency_hist19:45:020
3171022993221,8cyclictest0-21swapper/022:10:020
3171022993221,8cyclictest0-21swapper/022:10:020
317102499313,23cyclictest3329768-21ntpq00:25:292
3171024993126,3cyclictest0-21swapper/223:35:342
3171024993125,3cyclictest0-21swapper/221:30:012
3171024993123,5cyclictest3314722-21unixbench_multi00:00:402
3171024993123,5cyclictest3182043-21fschecks_time19:45:222
317102399315,21cyclictest428-21mta-sts-daemon20:25:281
3171023993125,3cyclictest0-21swapper/122:20:011
3171023993122,5cyclictest0-21swapper/120:20:011
3171023993122,4cyclictest0-21swapper/123:25:351
3171023993120,7cyclictest0-21swapper/123:03:061
3171023993120,4cyclictest3233061-21ntpq21:50:261
3171023993119,8cyclictest3302124-21timerwakeupswit23:40:371
3171023993115,3cyclictest0-21swapper/120:40:011
317102399310,5cyclictest0-21swapper/119:20:341
317102299314,22cyclictest420-21avahi-daemon22:35:320
3171022993124,5cyclictest421-21dbus-daemon20:20:010
317102299312,24cyclictest3270520-21sh22:50:290
317102299312,22cyclictest665-21snmpd21:15:270
3171022993121,7cyclictest3207013-21diskmemload22:30:350
3171022993113,12cyclictest12650irq/32-eth023:10:260
3171024993023,4cyclictest3288927-21modprobe23:20:222
3171024993022,5cyclictest1-21systemd20:55:022
317102499302,20cyclictest3220595-21ssh21:30:302
3171024993021,5cyclictest0-21swapper/200:16:312
3171024993019,7cyclictest3178399-21unixbench_multi19:30:392
317102499301,22cyclictest3301745-21sh23:40:302
3171023993023,4cyclictest3188052-21fschecks_time20:05:221
3171023993022,5cyclictest3176915-21unixbench_multi19:25:401
317102399301,4cyclictest171rcu_preempt00:25:271
3171022993025,3cyclictest3172404-21unixbench_multi19:10:390
3171022993022,5cyclictest3217955-21/usr/sbin/munin21:25:430
317102299301,12cyclictest3295334-21netstat23:30:250
3171024992922,3cyclictest0-21swapper/221:55:272
3171024992921,5cyclictest3285677-21gltestperf23:15:202
3171024992920,3cyclictest0-21swapper/220:10:302
3171023992922,3cyclictest0-21swapper/100:20:211
3171023992921,5cyclictest0-21swapper/120:00:011
3171022992924,3cyclictest1002-21mta-sts-daemon21:22:120
3171022992921,6cyclictest3177708-21latency19:30:230
3171022992916,11cyclictest1001-21mta-sts-daemon19:50:180
3171022992915,6cyclictest171rcu_preempt20:30:260
3171022992915,6cyclictest171rcu_preempt20:30:260
317102499282,7cyclictest171rcu_preempt20:20:282
3171024992823,2cyclictest0-21swapper/221:40:012
3171024992822,4cyclictest1001-21mta-sts-daemon00:08:402
3171024992822,3cyclictest0-21swapper/221:00:182
3171024992821,4cyclictest3296003-21unixbench_singl23:30:392
3171024992820,5cyclictest3183262-21aten_r2power_cu19:50:152
317102399286,5cyclictest171rcu_preempt20:40:271
3171023992821,4cyclictest3184024-21proc_pri19:50:351
3171023992821,4cyclictest0-21swapper/120:15:021
3171023992813,4cyclictest171rcu_preempt19:45:321
3171022992821,3cyclictest0-21swapper/000:10:010
317102299281,23cyclictest3226943-21ntp_states21:40:320
3171024992717,7cyclictest3292319-21ntpq23:25:322
3171023992720,4cyclictest439-21systemd-logind19:35:011
3171023992720,4cyclictest0-21swapper/121:05:381
3171022992721,3cyclictest0-21swapper/020:00:330
3171022992720,5cyclictest1427483-21cups-browsed23:45:340
3171022992720,3cyclictest3244163-21kworker/u8:1-xprtiod22:40:310
317102399262,19cyclictest3271368-21ssh22:51:211
3171022992617,4cyclictest0-21swapper/020:45:260
3171022992614,3cyclictest0-21swapper/020:25:010
3171024992519,3cyclictest0-21swapper/219:45:012
3171024992518,5cyclictest3188470-21/usr/sbin/munin20:05:312
3171024992514,8cyclictest3176188-21latency19:25:242
317102399259,3cyclictest171rcu_preempt20:00:271
3171023992520,2cyclictest0-21swapper/121:00:011
317102399251,6cyclictest3195887-21ntpq20:30:271
317102399251,6cyclictest3195887-21ntpq20:30:271
317102299252,14cyclictest151ktimers/019:15:270
3171022992514,8cyclictest0-21swapper/020:40:000
317102599241,3cyclictest0-21swapper/321:50:303
317102599241,3cyclictest0-21swapper/300:20:343
3171024992419,2cyclictest0-21swapper/220:25:322
317102299246,14cyclictest665-21snmpd21:00:240
3171022992419,4cyclictest3171021-21cyclictest20:50:390
3171022992412,10cyclictest151ktimers/019:55:310
317102299241,19cyclictest3182467-21ntp_states19:45:320
317102599232,6cyclictest0-21swapper/320:45:303
317102599231,4cyclictest0-21swapper/323:05:313
317102599231,4cyclictest0-21swapper/321:45:293
317102499231,17cyclictest3206374-21awk21:05:262
317102599223,14cyclictest0-21swapper/321:25:313
317102599221,4cyclictest0-21swapper/323:20:283
317102599221,4cyclictest0-21swapper/322:55:293
317102599221,4cyclictest0-21swapper/300:25:323
317102599221,4cyclictest0-21swapper/300:10:283
317102599221,3cyclictest0-21swapper/323:25:313
317102599221,3cyclictest0-21swapper/322:50:293
317102599221,3cyclictest0-21swapper/321:40:253
317102599221,3cyclictest0-21swapper/319:40:283
317102599221,3cyclictest0-21swapper/300:00:253
317102599221,17cyclictest0-21swapper/321:00:273
317102599220,3cyclictest0-21swapper/323:30:273
3171024992216,3cyclictest0-21swapper/220:45:002
3171022992217,3cyclictest0-21swapper/021:10:020
317102299221,18cyclictest3203424-21ntpq20:55:300
317102299221,17cyclictest665-21snmpd19:10:010
317102599213,13cyclictest0-21swapper/322:00:283
317102599213,13cyclictest0-21swapper/322:00:283
317102599212,4cyclictest0-21swapper/323:10:263
317102599212,4cyclictest0-21swapper/320:15:273
317102599212,3cyclictest0-21swapper/320:00:263
317102599212,15cyclictest0-21swapper/323:45:253
317102599211,5cyclictest0-21swapper/322:15:283
317102599211,5cyclictest0-21swapper/300:15:273
317102599211,4cyclictest0-21swapper/323:50:333
317102599211,4cyclictest0-21swapper/323:35:223
317102599211,4cyclictest0-21swapper/323:15:263
317102599211,4cyclictest0-21swapper/323:00:333
317102599211,4cyclictest0-21swapper/322:40:173
317102599211,4cyclictest0-21swapper/322:35:293
317102599211,4cyclictest0-21swapper/322:10:253
317102599211,4cyclictest0-21swapper/322:10:253
317102599211,4cyclictest0-21swapper/321:30:343
317102599211,4cyclictest0-21swapper/321:05:323
317102599211,4cyclictest0-21swapper/320:55:023
317102599211,4cyclictest0-21swapper/320:05:313
317102599211,4cyclictest0-21swapper/319:50:283
317102599211,4cyclictest0-21swapper/319:45:273
317102599211,4cyclictest0-21swapper/319:30:273
317102599211,4cyclictest0-21swapper/319:15:273
317102599211,3cyclictest0-21swapper/323:40:303
317102599211,3cyclictest0-21swapper/322:45:303
317102599211,3cyclictest0-21swapper/322:30:273
317102599211,3cyclictest0-21swapper/322:20:323
317102599211,3cyclictest0-21swapper/321:35:303
317102599211,3cyclictest0-21swapper/321:20:303
317102599211,3cyclictest0-21swapper/320:35:253
317102599211,3cyclictest0-21swapper/320:25:303
317102599211,3cyclictest0-21swapper/319:35:293
317102599211,3cyclictest0-21swapper/319:10:313
317102599211,3cyclictest0-21swapper/300:05:323
317102599211,2cyclictest0-21swapper/319:55:263
317102599211,16cyclictest0-21swapper/300:35:203
317102599211,15cyclictest0-21swapper/321:10:263
317102599210,3cyclictest0-21swapper/322:05:323
317102599210,3cyclictest0-21swapper/322:05:323
317102599210,3cyclictest0-21swapper/321:55:253
317102599210,3cyclictest0-21swapper/321:15:053
3171022992111,7cyclictest3174391-21aten_r2power_po19:20:160
317102599201,4cyclictest0-21swapper/320:30:313
317102599201,4cyclictest0-21swapper/320:30:313
317102599201,4cyclictest0-21swapper/300:30:183
317102599201,3cyclictest0-21swapper/323:55:193
317102599201,3cyclictest0-21swapper/322:25:213
317102599201,3cyclictest0-21swapper/320:55:173
317102599201,3cyclictest0-21swapper/320:20:253
317102599201,3cyclictest0-21swapper/319:25:273
317102599201,3cyclictest0-21swapper/319:20:263
317102599200,3cyclictest0-21swapper/320:40:273
317102599200,3cyclictest0-21swapper/320:10:303
3171022992010,7cyclictest3187970-21/usr/sbin/munin20:05:190
317102599191,4cyclictest0-21swapper/319:10:013
317102499178,6cyclictest421-21dbus-daemon19:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional