You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-21 - 20:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Sun Jun 21, 2026 12:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3261632995433,3cyclictest171rcu_preempt07:25:302
3261632994926,12cyclictest351ktimers/210:40:352
3261630994940,5cyclictest0-21swapper/011:55:260
3261632994843,2cyclictest0-21swapper/211:40:022
3261632994538,4cyclictest3328534-21kworker/u8:1-xprtiod11:00:012
3261632994534,6cyclictest0-21swapper/211:25:302
3261632994534,5cyclictest0-21swapper/208:40:302
3261631994537,3cyclictest0-21swapper/111:25:011
326163199453,36cyclictest3351066-21ssh10:35:301
326163299449,26cyclictest3391083-21ssh11:40:292
326163299449,26cyclictest3391083-21ssh11:40:292
3261632994434,4cyclictest0-21swapper/209:55:312
326163299443,36cyclictest3409486-21ssh12:10:282
326163299442,35cyclictest3332661-21ssh10:05:342
3261630994438,4cyclictest3396191-21latency_hist11:50:010
3261630994438,4cyclictest3396191-21latency_hist11:50:010
3261632994333,7cyclictest2465353-21cups-browsed07:15:232
3261632994319,16cyclictest351ktimers/212:15:292
3261630994327,10cyclictest12650irq/32-eth009:15:300
326163299421,33cyclictest3338777-21sshd10:15:302
326163299421,33cyclictest3338777-21sshd10:15:302
326163199422,33cyclictest3314205-21ssh09:35:321
3261630994234,5cyclictest3297618-21grep09:10:010
3261630994233,6cyclictest3343902-21basename10:25:000
3261632994129,8cyclictest3323711-21switchtime09:50:402
3261630994133,5cyclictest3411579-21latency_hist12:15:010
3261630994131,7cyclictest2465353-21cups-browsed10:56:100
326163099411,31cyclictest3384934-21sshd11:30:300
3261632994028,7cyclictest3352911-21sshd10:39:312
326163299402,30cyclictest3372716-21ssh11:10:402
326163199401,31cyclictest3354170-21ssh10:40:311
326163099409,27cyclictest3261629-21cyclictest10:45:340
3261630994033,4cyclictest3314548-21timerwakeupswit09:35:380
3261630994033,3cyclictest0-21swapper/008:20:310
3261630994032,5cyclictest3265490-21sshd07:20:300
3261632993932,4cyclictest3357821-21unixbench_singl10:45:422
3261632993931,5cyclictest3377825-21ssh11:20:032
326163199398,23cyclictest3307983-21ssh09:25:311
326163199393,29cyclictest3363541-21grep10:55:371
3261631993931,5cyclictest3383972-21cat11:30:031
3261631993931,4cyclictest0-21swapper/107:15:011
3261630993932,5cyclictest2465353-21cups-browsed11:40:020
326163299383,28cyclictest3366487-21ssh11:00:322
3261632993830,5cyclictest665-21snmpd09:00:362
3261632993830,5cyclictest0-21swapper/211:35:012
3261632993827,7cyclictest3423854-21latency_hist12:35:012
326163299382,28cyclictest3394088-21ssh11:45:282
326163299382,28cyclictest3394088-21ssh11:45:282
3261632993816,3cyclictest171rcu_preempt08:55:282
3261631993821,11cyclictest3297613-21diskmemload11:55:281
326163199381,27cyclictest3406434-21ssh12:05:301
3261630993823,12cyclictest3379059-21switchtime11:20:400
326163099382,28cyclictest3326517-21sshd09:55:310
326163099381,30cyclictest3317301-21ssh09:40:320
326163299375,3cyclictest171rcu_preempt08:30:312
3261632993729,5cyclictest3331587-21cron10:05:012
3261632993729,5cyclictest3316911-21if_wlan009:40:232
3261632993725,9cyclictest3362337-21/usr/sbin/munin10:55:012
326163199373,26cyclictest3332735-21ssh10:05:371
3261631993730,3cyclictest0-21swapper/110:45:321
3261630993731,3cyclictest0-21swapper/009:50:300
3261630993730,4cyclictest3337743-21grep10:15:020
3261630993727,7cyclictest3406976-21unixbench_multi12:05:420
326163099372,28cyclictest3305013-21ssh09:20:390
3261632993630,3cyclictest3272140-21kworker/u8:12-xprtiod08:00:002
3261632993630,3cyclictest0-21swapper/209:15:012
3261632993628,5cyclictest3306917-21systemd-run09:25:012
326163299362,23cyclictest3320376-21ssh09:45:332
3261631993628,5cyclictest0-21swapper/112:40:011
3261631993627,6cyclictest3289137-21fschecks_count08:40:211
3261631993627,5cyclictest0-21swapper/111:30:301
3261630993629,4cyclictest3412193-21tr12:15:220
3261630993624,9cyclictest3347682-21if_wlan010:30:240
3261630993617,12cyclictest3281095-21kworker/u8:3+rpciod09:30:330
326163299353,19cyclictest3341931-21ssh10:20:322
3261632993526,6cyclictest485-21in:imuxsock11:25:012
3261632993517,3cyclictest171rcu_preempt10:30:302
3261631993528,4cyclictest3388563-21unixbench_singl11:35:411
3261631993528,4cyclictest0-21swapper/110:25:321
3261631993527,5cyclictest3371589-21systemd-run11:10:011
326163199352,27cyclictest3291035-21sshd08:45:301
3261630993529,4cyclictest3291497-21unixbench_multi08:45:390
3261630993526,6cyclictest3333285-21vmstat10:05:460
3261632993425,6cyclictest3273544-21head07:45:382
326163199343,24cyclictest3323349-21ssh09:50:291
3261631993427,4cyclictest3319260-21cron09:45:011
3261631993426,5cyclictest3391210-21ps11:40:331
3261631993426,5cyclictest3391210-21ps11:40:331
3261631993426,5cyclictest3329227-21if_eth010:00:251
3261631993426,5cyclictest3298408-21latency09:10:271
3261631993425,6cyclictest3294523-21unixbench_singl08:55:391
3261631993425,4cyclictest0-21swapper/111:50:311
3261631993425,4cyclictest0-21swapper/111:50:311
326163199341,25cyclictest3415725-21ssh12:20:311
3261630993426,5cyclictest3299162-21unixbench-2d09:10:420
3261630993424,6cyclictest623-21sshd07:25:310
3261632993327,3cyclictest0-21swapper/208:40:012
3261632993326,3cyclictest0-21swapper/210:25:212
3261632993325,5cyclictest3415192-21ssh12:20:202
3261632993324,5cyclictest3293101-21debian-sa108:55:012
3261632993322,8cyclictest3279295-21switchtime08:05:402
3261631993326,4cyclictest3402971-21gltestperf12:00:221
3261631993326,4cyclictest3375895-21sendmail_mailqu11:15:411
3261631993326,4cyclictest3261629-21cyclictest10:50:381
3261631993326,4cyclictest0-21swapper/107:40:011
3261631993326,3cyclictest0-21swapper/107:55:001
3261631993326,3cyclictest0-21swapper/107:55:001
3261631993325,5cyclictest3374637-21munin-run11:15:011
3261631993325,4cyclictest3267623-21munin-plugin-st07:30:011
3261631993325,4cyclictest0-21swapper/109:45:231
3261631993322,8cyclictest3409910-21timerandwakeup12:10:391
3261631993321,8cyclictest3301072-21aten_r2power_en09:15:181
3261630993326,5cyclictest3369009-21cpu11:05:210
3261630993326,4cyclictest3356186-21sh10:45:000
3261630993326,4cyclictest1002-21mta-sts-daemon11:27:410
3261630993320,10cyclictest3426891-21cron12:40:010
3261632993224,4cyclictest0-21swapper/209:30:302
3261632993223,6cyclictest1-21systemd09:40:012
3261631993224,5cyclictest3305436-21unixbench_singl09:20:441
3261631993223,5cyclictest0-21swapper/109:55:341
3261631993222,5cyclictest0-21swapper/110:15:311
3261631993222,5cyclictest0-21swapper/110:15:311
326163199322,24cyclictest3421966-21ssh12:30:361
326163199322,22cyclictest3347958-21ssh10:30:291
3261630993226,3cyclictest0-21swapper/011:00:350
3261630993226,3cyclictest0-21swapper/007:50:010
3261630993225,3cyclictest0-21swapper/012:30:310
3261630993224,5cyclictest3415462-21latency12:20:240
3261630993224,4cyclictest0-21swapper/011:10:420
3261630993222,5cyclictest151ktimers/008:00:290
3261630993222,5cyclictest12650irq/32-eth007:40:300
3261632993125,4cyclictest3261629-21cyclictest08:25:312
3261632993123,5cyclictest3307518-21fschecks_count09:25:222
3261632993122,7cyclictest3268286-21latency07:30:252
326163299312,23cyclictest3369634-21ssh11:05:342
3261631993124,3cyclictest0-21swapper/108:00:301
3261631993123,5cyclictest3288604-21grep08:40:011
3261631993123,3cyclictest0-21swapper/111:50:001
3261631993123,3cyclictest0-21swapper/111:50:001
3261631993121,8cyclictest3292274-21latency08:50:241
3261631993114,12cyclictest421-21dbus-daemon12:30:011
3261630993125,4cyclictest3329161-21fschecks_time10:00:240
3261630993124,4cyclictest3288603-21/usr/sbin/munin08:40:020
326163299302,25cyclictest351ktimers/208:20:292
3261632993021,6cyclictest1001-21mta-sts-daemon10:12:492
3261631993021,5cyclictest3272146-21systemd-run07:45:011
3261631993020,7cyclictest0-21swapper/109:10:021
3261630993024,4cyclictest3377704-21ssh11:20:010
3261630993024,3cyclictest0-21swapper/008:20:020
3261630993022,5cyclictest0-21swapper/007:40:010
3261630993019,9cyclictest3288578-21kworker/u8:2-rpciod08:45:020
326163299299,14cyclictest2465353-21cups-browsed11:50:302
326163299299,14cyclictest2465353-21cups-browsed11:50:302
3261632992922,5cyclictest3403012-21sh12:00:232
3261632992921,5cyclictest3278121-21ls08:05:002
3261632992921,5cyclictest0-21swapper/207:40:012
3261632992919,8cyclictest3419061-21switchtime12:25:392
3261632992913,4cyclictest171rcu_preempt08:45:332
326163199295,17cyclictest3335685-21sshd10:10:301
3261631992922,4cyclictest3310987-21ntp_kernel_err09:30:281
3261631992922,4cyclictest3276620-21idleruntime-cro08:00:001
326163199292,22cyclictest3280515-21sshd08:10:301
3261631992921,5cyclictest3269229-21latency_hist07:35:021
3261631992918,8cyclictest3412353-21latency12:15:241
3261630992924,2cyclictest0-21swapper/010:50:310
3261630992922,5cyclictest3350344-21/usr/sbin/munin10:35:230
326163099292,14cyclictest3344867-21ssh10:25:260
3261630992916,8cyclictest12650irq/32-eth009:25:290
3261630992916,11cyclictest3287181-21latency_hist08:35:020
3261630992910,16cyclictest3418767-21ssh12:25:300
3261632992823,2cyclictest0-21swapper/212:10:012
3261632992822,3cyclictest0-21swapper/212:00:002
3261632992821,4cyclictest1000-21mta-sts-daemon09:17:572
3261632992820,5cyclictest3426948-21systemd12:40:022
3261632992819,6cyclictest3262136-21fschecks_count07:10:252
3261632992815,2cyclictest171rcu_preempt08:15:342
326163199281,19cyclictest3283526-21sshd08:20:311
3261630992822,3cyclictest0-21swapper/008:05:300
3261630992815,10cyclictest0-21swapper/011:50:260
3261630992815,10cyclictest0-21swapper/011:50:260
3261630992815,10cyclictest0-21swapper/007:10:430
326163299276,5cyclictest0-21swapper/209:05:312
326163299275,14cyclictest428-21mta-sts-daemon07:50:322
326163299275,14cyclictest428-21mta-sts-daemon07:50:322
3261631992720,4cyclictest3367082-21unixbench_singl11:00:441
326163199271,17cyclictest265-21systemd-journal07:45:301
326163099277,15cyclictest12650irq/32-eth011:40:320
326163099277,15cyclictest12650irq/32-eth011:40:320
3261630992721,4cyclictest3280140-21fschecks_time08:10:220
3261630992719,6cyclictest151ktimers/007:55:290
3261630992718,7cyclictest3405398-21munin-plugin-st12:05:010
3261632992610,12cyclictest627-21lldpd07:20:342
3261631992616,7cyclictest3341701-21missed_timers10:20:251
326163099262,19cyclictest3338742-21ssh10:15:290
326163099262,19cyclictest3338742-21ssh10:15:290
3261630992619,5cyclictest1002-21mta-sts-daemon07:50:300
3261630992619,5cyclictest1002-21mta-sts-daemon07:50:300
326163299256,3cyclictest171rcu_preempt08:10:322
3261631992517,5cyclictest0-21swapper/108:30:011
3261631992517,4cyclictest3266120-21cron07:25:011
326163399241,7cyclictest0-21swapper/309:30:303
326163199244,16cyclictest428-21mta-sts-daemon07:15:351
3261631992416,5cyclictest2465353-21cups-browsed08:19:251
3261630992420,3cyclictest3261629-21cyclictest08:58:510
3261630992419,2cyclictest0-21swapper/007:20:010
3261630992417,5cyclictest567-21gdbus08:50:250
3261630992417,5cyclictest1002-21mta-sts-daemon09:48:510
3261630992412,7cyclictest12650irq/32-eth007:30:300
326163399233,5cyclictest0-21swapper/311:30:313
326163399231,6cyclictest0-21swapper/310:45:293
3261631992315,5cyclictest3287036-21head08:30:411
3261630992316,5cyclictest3285424-21timerwakeupswit08:25:390
326163399221,8cyclictest0-21swapper/309:20:393
326163399221,7cyclictest0-21swapper/311:55:303
3261632992212,2cyclictest171rcu_preempt07:40:282
326163199222,15cyclictest3295589-21ntpq09:00:361
3261631992216,3cyclictest0-21swapper/108:05:411
326163099221,17cyclictest3295224-21awk09:00:240
326163399212,7cyclictest0-21swapper/311:45:303
326163399212,7cyclictest0-21swapper/311:45:303
326163399212,4cyclictest0-21swapper/310:25:263
326163399203,9cyclictest0-21swapper/311:10:403
326163399202,9cyclictest0-21swapper/309:10:363
326163399202,7cyclictest0-21swapper/310:30:293
326163399202,6cyclictest0-21swapper/310:35:303
326163399202,6cyclictest0-21swapper/310:20:353
326163399201,11cyclictest0-21swapper/311:20:293
326163299207,3cyclictest36-21ksoftirqd/207:10:022
3261631992015,2cyclictest0-21swapper/107:10:011
326163399191,5cyclictest0-21swapper/311:25:303
326163399183,9cyclictest0-21swapper/310:55:373
326163399183,8cyclictest0-21swapper/311:05:373
326163399183,5cyclictest0-21swapper/308:50:313
326163399182,8cyclictest0-21swapper/309:15:283
326163399172,9cyclictest0-21swapper/309:40:303
326163399162,9cyclictest0-21swapper/311:50:223
326163399162,9cyclictest0-21swapper/311:50:223
326163399162,8cyclictest0-21swapper/312:10:293
326163399162,7cyclictest0-21swapper/311:35:283
326163399161,8cyclictest0-21swapper/312:15:283
326163399161,6cyclictest0-21swapper/307:30:283
326163099161,13cyclictest3261635-21systemd-run07:10:010
326163399153,9cyclictest0-21swapper/310:00:293
326163399153,6cyclictest0-21swapper/312:20:303
326163399153,5cyclictest0-21swapper/307:10:343
326163399152,9cyclictest0-21swapper/310:15:253
326163399152,9cyclictest0-21swapper/310:15:253
326163399152,5cyclictest0-21swapper/308:15:283
326163399151,7cyclictest0-21swapper/307:25:313
326163399144,7cyclictest0-21swapper/310:10:333
326163399143,7cyclictest0-21swapper/312:30:313
326163399143,5cyclictest0-21swapper/312:00:223
326163399142,7cyclictest0-21swapper/310:40:203
326163399142,7cyclictest0-21swapper/308:05:303
326163399142,6cyclictest0-21swapper/309:25:273
326163399141,7cyclictest0-21swapper/309:35:263
326163399141,7cyclictest0-21swapper/308:35:303
326163399134,5cyclictest0-21swapper/309:00:333
326163399133,5cyclictest0-21swapper/308:40:303
326163399132,8cyclictest0-21swapper/308:20:313
326163399132,6cyclictest0-21swapper/309:55:213
326163399132,6cyclictest0-21swapper/308:50:013
326163399132,6cyclictest0-21swapper/308:00:003
326163399132,6cyclictest0-21swapper/307:20:303
326163399132,5cyclictest0-21swapper/310:05:283
326163399131,7cyclictest0-21swapper/309:45:233
326163399124,6cyclictest0-21swapper/312:05:303
326163399122,6cyclictest0-21swapper/311:00:123
326163399122,6cyclictest0-21swapper/309:50:253
326163399121,7cyclictest0-21swapper/307:50:233
326163399121,7cyclictest0-21swapper/307:50:233
326163399121,6cyclictest0-21swapper/308:00:303
326163399121,6cyclictest0-21swapper/307:45:213
326163399114,4cyclictest0-21swapper/307:40:333
326163399113,6cyclictest0-21swapper/311:40:163
326163399113,6cyclictest0-21swapper/311:40:163
326163399112,7cyclictest0-21swapper/310:50:313
326163399111,6cyclictest0-21swapper/312:35:183
326163399111,6cyclictest0-21swapper/312:25:183
326163399111,6cyclictest0-21swapper/311:15:193
326163399111,6cyclictest0-21swapper/309:10:023
326163399101,6cyclictest0-21swapper/307:35:203
32616339993,5cyclictest0-21swapper/308:25:283
32616339992,5cyclictest0-21swapper/307:20:023
32616339991,6cyclictest0-21swapper/308:55:273
32616339991,4cyclictest0-21swapper/308:10:303
32616339981,6cyclictest0-21swapper/308:30:253
32616339971,2cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional