You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-01 - 01:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Tue Mar 31, 2026 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
315558823630,1sleep3471rcuc/310:20:013
2965317995549,4cyclictest3176438-21apt-get10:41:211
29663332510,2sleep375-21kworker/3:1-events07:11:313
2965315995130,17cyclictest3056539-21latency_hist08:40:000
2965315994933,13cyclictest3105991-21cron09:30:000
2965315994832,13cyclictest3036507-21latency_hist08:20:010
2965317994638,5cyclictest3071429-21latency_hist08:55:011
2965317994631,12cyclictest280-21systemd-journal08:27:211
2965317994536,6cyclictest280-21systemd-journal12:07:291
2965317994535,7cyclictest3075539-21ntpq08:56:351
2965317994534,8cyclictest280-21systemd-journal09:47:201
2965317994532,10cyclictest3061519-21latency_hist08:45:011
2965317994529,14cyclictest3190908-21systemd-run10:55:001
2965320994426,16cyclictest280-21systemd-journal07:15:052
2965315994426,16cyclictest3135732-21apt10:00:000
2965317994325,16cyclictest3105990-21cron09:30:001
2965315994326,15cyclictest3126792-21irqcore09:51:300
2965317994233,7cyclictest3272787-21kworker/u8:0+flush-8:012:21:271
2965317994232,8cyclictest280-21systemd-journal10:35:001
2965317994222,18cyclictest3106327-21/usr/sbin/munin09:31:361
2965317994222,18cyclictest3106327-21/usr/sbin/munin09:31:361
2965317994222,17cyclictest280-21systemd-journal11:40:001
2965315994226,14cyclictest2972163-21irqcore07:15:050
2965317994123,16cyclictest3115903-21idleruntime-cro09:40:001
2965315994031,7cyclictest3190070-21ntpq10:51:380
2965315994028,9cyclictest2977751-21diskstats07:21:280
2965315994024,14cyclictest1-21systemd10:25:010
2965315994022,16cyclictest0-21swapper/007:11:290
2965315994021,17cyclictest3170486-21grep10:31:420
2965315994019,4cyclictest3085933-21unixbench_singl09:06:410
2965315994017,6cyclictest3260253-21latency_hist12:05:000
2965317993930,7cyclictest3255090-21apt-get12:00:021
2965317993930,7cyclictest3220864-21apt-get11:26:221
2965317993919,7cyclictest280-21systemd-journal10:25:011
2965315993930,6cyclictest3250128-21latency_hist11:55:010
2965315993920,17cyclictest3075747-21mailstats08:56:390
30502822380,3sleep2371rcuc/208:31:362
2965317993812,3cyclictest30139762kthreadcore07:56:331
2965315993823,13cyclictest3110938-21cron09:35:000
2965315993823,13cyclictest3110938-21cron09:35:000
2965315993819,3cyclictest3031265-21unixbench_singl08:11:430
2965315993811,12cyclictest3046604-21awk08:30:010
2965320993731,4cyclictest280-21systemd-journal07:25:212
2965320993722,13cyclictest3226150-21fschecks_time11:31:282
2965320993722,13cyclictest3226150-21fschecks_time11:31:282
2965320993717,16cyclictest280-21systemd-journal09:30:002
2965317993727,7cyclictest2965151-21latency_hist07:10:011
2965317993720,3cyclictest0-21swapper/111:41:451
2965317993719,3cyclictest2977479-21acpi07:21:241
2965315993731,4cyclictest2997190-21kworker/u8:3+flush-8:007:51:260
2965315993725,9cyclictest3175829-21apt-get10:40:010
2965315993713,6cyclictest3120892-21latency_hist09:45:000
30588182360,5sleep3451irq_work/308:41:323
2965320993629,5cyclictest1403894-21kworker/u8:1+flush-8:009:36:262
2965320993623,11cyclictest3265857-21fschecks_time12:11:282
2965317993632,3cyclictest3185086-21needreboot10:46:361
2965317993622,12cyclictest3244846-21unixbench_multi11:46:431
2965317993618,3cyclictest0-21swapper/111:21:401
2965317993618,2cyclictest0-21swapper/111:16:391
2965315993626,8cyclictest2991948-21latency_hist07:35:010
2965315993619,2cyclictest3210293-21grep11:11:440
2965317993516,3cyclictest0-21swapper/109:25:001
296531599353,3cyclictest32842342ntpq12:26:370
2965315993518,4cyclictest3294649-21unixbench_singl12:36:430
2965315993518,2cyclictest3145922-21sshd10:11:200
2965315993513,2cyclictest421rcuog/208:23:220
2965317993419,3cyclictest3040863-21mailstats08:21:421
2965317993418,4cyclictest3170751-21kworker/u8:2+events_unbound12:12:061
2965317993418,3cyclictest3077460-21cut09:01:301
2965317993417,6cyclictest280-21systemd-journal10:11:191
2965317993417,3cyclictest0-21swapper/111:05:011
2965317993417,2cyclictest0-21swapper/108:46:351
2965317993417,15cyclictest0-21swapper/109:56:261
296531599349,2cyclictest0-21swapper/009:01:320
2965315993418,2cyclictest1-21systemd07:30:000
296531599341,7cyclictest407-21mta-sts-daemon12:21:040
2965320993321,10cyclictest3231049-21fschecks_time11:36:272
2965320993317,14cyclictest3200453-21unixbench_singl11:01:442
296532099331,25cyclictest2992305-21apt-get07:36:262
2965320993310,11cyclictest32842342ntpq12:26:372
296532099330,22cyclictest161rcu_preempt09:16:272
2965317993317,2cyclictest0-21swapper/109:51:261
2965315993325,5cyclictest3269835-21grep12:11:440
2965315993318,2cyclictest880-21mta-sts-daemon11:31:360
2965315993318,2cyclictest880-21mta-sts-daemon11:31:360
29642622332,25sleep02964312-21collect207:06:350
296532099322,23cyclictest3176438-21apt-get10:41:222
2965320993217,13cyclictest3120891-21idleruntime-cro09:45:002
2965320993216,13cyclictest3018056-21kthreadcore08:01:312
2965320993214,16cyclictest3046518-21gpgv08:30:012
296532099321,22cyclictest161rcu_preempt10:12:292
296532099321,21cyclictest161rcu_preempt10:35:172
2965317993218,5cyclictest2992305-21apt-get07:36:241
2965317993210,2cyclictest0-21swapper/110:56:001
2965315993221,8cyclictest3065656-21ntpq08:46:360
296532099313,21cyclictest2965574-21apt-get07:11:242
2965320993122,7cyclictest3161137-21cp10:25:012
2965320993117,12cyclictest3080963-21unixbench_multi09:01:432
2965315993126,3cyclictest3254199-21needreboot11:56:350
2965315993123,6cyclictest3057001-21aten2_r5power_p08:41:250
2965315993111,8cyclictest3139844-21ntpq10:01:370
2965320993015,13cyclictest3205343-21unixbench_singl11:06:432
296532099301,24cyclictest3238633-21kthreadcore11:41:352
296532099300,22cyclictest161rcu_preempt08:52:352
2965317993025,3cyclictest3082262-21irqcore09:06:281
2965317993025,3cyclictest2974678-21kthreadcore07:16:321
296531599303,13cyclictest3210954-21taskset11:16:240
2965315993022,6cyclictest3070600-21ntpq08:51:380
296531599301,27cyclictest2960292-21kworker/0:1+events08:01:360
2965320992925,3cyclictest3155521-21munin-run10:20:002
2965320992924,3cyclictest3110607-21unixbench_multi09:31:442
2965320992924,3cyclictest3110607-21unixbench_multi09:31:442
2965320992922,5cyclictest280-21systemd-journal07:45:252
2965320992922,5cyclictest280-21systemd-journal07:45:252
2965320992914,13cyclictest3086701-21aten2_r5power_p09:11:262
296532099291,24cyclictest3215912-21aten2_r5power_c11:21:252
296532099291,24cyclictest2991669-21/usr/sbin/munin07:31:432
2965320992910,16cyclictest3096773-21fschecks_count09:21:272
2965317992925,2cyclictest2970413-21munin-node07:51:341
2965317992925,2cyclictest0-21swapper/110:21:421
2965317992924,3cyclictest3275085-21munin-run12:20:001
2965317992924,3cyclictest3120886-21cron09:45:001
2965317992924,3cyclictest2986688-21unixbench_singl07:26:421
2965315992926,2cyclictest3006840-21awk07:50:000
2965315992926,2cyclictest3006840-21awk07:50:000
2965315992925,3cyclictest3001601-21unixbench_multi07:41:450
2965315992924,3cyclictest3235947-21cpuspeed_turbos11:41:280
2965315992920,7cyclictest280-21systemd-journal12:10:010
2965315992919,8cyclictest3090335-21ntpq09:11:360
2965315992919,8cyclictest280-21systemd-journal10:40:010
2965315992919,7cyclictest3270751-21sshd12:16:290
296532099289,16cyclictest3287316-21tr12:31:322
2965320992816,10cyclictest3052276-21fschecks_count08:36:292
296532099281,23cyclictest3250465-21munin-node11:56:222
2965320992812,14cyclictest3012358-21cpuspeed_turbos07:56:262
2965320992812,13cyclictest0-21swapper/207:41:432
2965320992811,14cyclictest0-21swapper/207:21:252
2965320992810,16cyclictest3083829-21kthreadcore09:06:302
296532099280,26cyclictest3244458-21/usr/sbin/munin11:46:392
296532099280,25cyclictest3141328-21/usr/sbin/munin10:06:292
296532099280,24cyclictest3260585-21/usr/sbin/munin12:06:232
2965317992824,3cyclictest3136828-21irqrtprio10:01:311
2965317992824,2cyclictest0-21swapper/111:11:391
2965317992824,2cyclictest0-21swapper/108:11:231
2965317992823,3cyclictest3086848-21cpuspeed_turbos09:11:271
2965317992821,5cyclictest409-21smartd10:19:471
2965315992824,3cyclictest3016768-21munin-run08:00:010
2965315992818,8cyclictest3155513-21latency_hist10:20:010
2965320992713,12cyclictest0-21swapper/211:16:302
296532099271,23cyclictest3166858-21ps10:31:302
296532099271,23cyclictest3030847-21/usr/sbin/munin08:11:402
296532099271,22cyclictest3294938-21idleruntime-cro12:40:002
296532099271,22cyclictest3276736-21kthreadcore12:21:322
296532099271,22cyclictest3193520-21kthreadcore10:56:322
296532099271,22cyclictest3123119-21kthreadcore09:46:342
296532099271,22cyclictest3071970-21/usr/sbin/munin08:56:232
2965320992712,13cyclictest3221082-21cpuspeed_turbos11:26:252
2965320992712,13cyclictest0-21swapper/211:51:352
2965320992712,13cyclictest0-21swapper/210:26:322
296532099270,24cyclictest3134896-21/usr/sbin/munin09:56:362
296532099270,23cyclictest3270405-21seq12:15:312
296532099270,23cyclictest3056922-21/usr/sbin/munin08:41:242
296532099270,19cyclictest161rcu_preempt09:52:232
2965317992723,2cyclictest0-21swapper/110:06:301
296531599278,17cyclictest3046548-21kworker/0:2+events09:46:340
2965315992723,2cyclictest0-21swapper/011:40:000
2965315992721,4cyclictest3200401-21unixbench_multi11:01:440
2965315992720,5cyclictest3185688-21unixbench_singl10:46:440
296532099266,17cyclictest3022161-21cut08:06:252
2965320992614,10cyclictest0-21swapper/210:01:312
296532099261,22cyclictest3033326-21cut08:16:322
2965320992612,12cyclictest0-21swapper/208:46:332
2965320992610,14cyclictest0-21swapper/210:46:262
296532099260,24cyclictest3260064-21seq12:02:432
296532099260,23cyclictest3186335-21seq10:51:242
296532099260,22cyclictest3206059-21/usr/sbin/munin11:11:252
2965317992622,2cyclictest0-21swapper/108:06:421
2965317992619,5cyclictest280-21systemd-journal08:17:001
296531599267,17cyclictest3046548-21kworker/0:2+events11:26:370
2965315992622,3cyclictest3290033-21latency_hist12:35:010
2965315992620,4cyclictest3091783-21sh09:16:250
296532099251,20cyclictest3036911-21aten2_r5power_e08:21:262
296532099251,15cyclictest421rcuog/207:51:322
296532099250,21cyclictest2965363-21wc07:10:012
296531799252,18cyclictest3175902-21munin-run10:40:011
2965317992521,2cyclictest0-21swapper/111:56:421
2965317992521,2cyclictest0-21swapper/111:06:351
2965317992521,2cyclictest0-21swapper/108:35:001
2965317992521,2cyclictest0-21swapper/108:01:421
2965317992521,2cyclictest0-21swapper/107:11:411
2965317992519,4cyclictest2997190-21kworker/u8:3+flush-8:009:16:521
2965315992521,2cyclictest0-21swapper/011:00:000
2965315992520,4cyclictest3049191-21sed08:31:350
2965315992520,3cyclictest3220496-21cron11:25:000
2965317992420,2cyclictest0-21swapper/111:31:311
2965317992420,2cyclictest0-21swapper/111:31:311
2965317992420,2cyclictest0-21swapper/107:46:291
2965317992420,2cyclictest0-21swapper/107:46:291
2965317992420,2cyclictest0-21swapper/107:31:371
2965315992418,5cyclictest3115610-21unixbench_singl09:36:410
2965317992319,2cyclictest3052707-21irqrtprio08:36:321
2965317992317,4cyclictest3285061-21sh12:30:011
2965317992316,5cyclictest3290351-21apt-get12:36:211
2965315992319,3cyclictest3205305-21unixbench_multi11:06:430
2965315992319,3cyclictest0-21swapper/007:36:310
2965315992318,3cyclictest3244340-21ntp_kernel_err11:46:350
2965317992216,4cyclictest1403894-21kworker/u8:1+flush-8:007:41:481
296531599227,14cyclictest2960292-21kworker/0:1+events08:09:260
2965317992117,3cyclictest3289138-21ntp_kernel_err12:31:351
2965317992112,6cyclictest3249214-21grep11:51:341
2965315992113,5cyclictest3165917-21latency_hist10:30:010
2965315992111,8cyclictest3096732-21diskstats09:21:270
296531599192,13cyclictest3141047-21taskset10:06:250
29602792193,8sleep30-21swapper/307:05:013
31165382167,6sleep30-21swapper/309:41:293
32444022154,7sleep30-21swapper/311:46:383
32803882145,5sleep30-21swapper/312:26:233
32155722132,6sleep30-21swapper/311:20:013
31555882132,6sleep30-21swapper/310:20:023
30804532132,7sleep30-21swapper/309:01:363
30465162132,6sleep30-21swapper/308:30:023
30173892132,7sleep30-21swapper/308:01:283
29969622132,6sleep30-21swapper/307:40:013
32900232122,6sleep30-21swapper/312:35:023
32853382122,6sleep30-21swapper/312:31:003
32651732122,5sleep30-21swapper/312:10:013
32649772122,6sleep30-21swapper/312:07:293
32504942122,6sleep30-21swapper/311:56:253
32361712122,6sleep30-21swapper/311:41:323
32300962122,6sleep30-21swapper/311:31:453
32300962122,6sleep30-21swapper/311:31:453
32083012122,6sleep30-21swapper/311:11:343
32055172122,6sleep30-21swapper/311:08:213
32007342122,6sleep30-21swapper/311:05:003
31908712122,5sleep30-21swapper/310:55:013
31863152123,5sleep30-21swapper/310:51:243
31810472122,6sleep30-21swapper/310:45:023
31809542122,6sleep30-21swapper/310:44:213
31752622122,6sleep30-21swapper/310:36:433
31667112122,6sleep30-21swapper/310:31:293
31656382122,6sleep30-21swapper/310:26:443
31360222122,6sleep30-21swapper/310:00:393
31308072122,5sleep30-21swapper/309:55:013
31264552122,6sleep30-21swapper/309:51:283
31150572122,6sleep30-21swapper/309:36:353
31083952122,6sleep30-21swapper/309:31:343
31083952122,6sleep30-21swapper/309:31:343
31016882123,5sleep30-21swapper/309:26:283
30870882122,6sleep30-21swapper/309:11:313
30813332122,5sleep30-21swapper/309:05:013
30362232122,6sleep30-21swapper/308:16:453
30272412122,6sleep30-21swapper/308:11:283
30260572122,6sleep30-21swapper/308:06:423
30075762122,6sleep30-21swapper/307:51:293
30027902122,6sleep30-21swapper/307:46:313
30027902122,6sleep30-21swapper/307:46:313
29963802122,6sleep30-21swapper/307:36:443
29829092122,6sleep30-21swapper/307:26:293
29763152122,6sleep30-21swapper/307:16:383
32767332112,5sleep30-21swapper/312:21:333
32705062112,6sleep30-21swapper/312:16:283
32556722112,5sleep30-21swapper/312:01:233
32458032112,5sleep30-21swapper/311:51:273
32315042112,5sleep30-21swapper/311:36:303
32211122112,5sleep30-21swapper/311:26:273
32147852112,5sleep30-21swapper/311:16:403
31458172112,5sleep30-21swapper/310:10:023
31447132112,5sleep30-21swapper/310:06:373
31212502112,5sleep30-21swapper/309:46:273
30963472112,5sleep30-21swapper/309:20:023
30914312112,6sleep30-21swapper/309:15:163
30715752112,5sleep30-21swapper/308:55:023
30675662110,7sleep30-21swapper/308:51:333
30623772112,5sleep30-21swapper/308:46:313
30520842112,5sleep30-21swapper/308:36:283
30420852112,5sleep30-21swapper/308:26:283
30367432112,5sleep30-21swapper/308:20:023
30120512112,6sleep30-21swapper/307:55:383
29872112112,5sleep30-21swapper/307:30:023
29779552112,5sleep30-21swapper/307:21:313
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional