You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-22 - 03:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Sun Mar 22, 2026 00:44:14)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30242552730,2sleep3471rcuc/319:15:013
32088752700,3sleep0201rcuc/022:16:400
30182862632,32sleep20-21swapper/219:06:462
3019207995349,3cyclictest3056188-21kworker/u8:2+events_unbound22:40:001
30433482480,4chrt0-21swapper/019:31:400
3019207994541,3cyclictest3185064-21kworker/u8:0+events_unbound00:11:361
3019211994433,9cyclictest280-21systemd-journal19:18:142
3019207994437,6cyclictest3170472-21kworker/u8:3+flush-8:023:01:421
3019207994434,8cyclictest3284015-21ntpq23:31:421
3019205994435,6cyclictest3055933-21latency_hist19:45:010
3019205994432,9cyclictest3250041-21awk23:00:000
3019205994432,9cyclictest3250041-21awk23:00:000
3019211994333,7cyclictest3070750-21awk20:00:002
3019207994237,4cyclictest3046010-21kworker/u8:1+events_unbound20:10:011
3019205994233,6cyclictest3205058-21latency_hist22:15:010
3019207994137,3cyclictest3056188-21kworker/u8:2+events_unbound19:56:441
3019207994136,3cyclictest3056188-21kworker/u8:2+events_unbound22:21:381
3019207994132,7cyclictest280-21systemd-journal23:20:011
3019205994133,6cyclictest3066154-21/usr/sbin/munin19:56:410
3019207994036,3cyclictest3170472-21kworker/u8:3+events_unbound22:01:371
3019207994035,4cyclictest3170472-21kworker/u8:3+flush-8:022:16:341
3019205994034,4cyclictest3091061-21kworker/u8:3+events_unbound21:01:330
3019205994030,8cyclictest3234160-21ntpq22:41:440
3019207993934,4cyclictest3056188-21kworker/u8:2+flush-8:021:56:381
3019207993932,6cyclictest3185064-21kworker/u8:0+flush-8:022:51:451
3019207993932,6cyclictest3185064-21kworker/u8:0+flush-8:000:01:431
3019207993931,5cyclictest3165291-21latency_hist21:35:011
3019207993931,3cyclictest3140757-21kworker/u8:0+flush-8:021:16:441
3019205993932,5cyclictest280-21systemd-journal20:28:580
3019207993832,5cyclictest3309640-21kworker/u8:2+flush-8:000:36:441
3019207993832,4cyclictest3275200-21kworker/u8:1+events_unbound23:46:411
3019205993833,4cyclictest3056188-21kworker/u8:2+flush-8:019:51:340
301920599380,36cyclictest1-21systemd00:39:140
3019211993727,7cyclictest3140529-21awk21:10:002
3019207993732,3cyclictest3170472-21kworker/u8:3+events_unbound22:46:381
3019207993731,4cyclictest3056188-21kworker/u8:2+events_unbound20:31:511
3019207993730,4cyclictest280-21systemd-journal20:43:141
31000482360,2sleep3481ktimers/320:26:403
3019207993631,4cyclictest2190942-21kworker/u8:2+events_unbound19:21:461
3019207993631,3cyclictest3170472-21kworker/u8:3+flush-8:021:51:351
3019207993631,3cyclictest3075836-21kworker/u8:0+events_unbound20:16:491
3019207993630,4cyclictest3056188-21kworker/u8:2+flush-8:022:31:351
3019207993630,4cyclictest3056188-21kworker/u8:2+flush-8:021:46:441
3019207993630,4cyclictest3040727-21kworker/u8:3+flush-8:019:36:371
3019207993625,8cyclictest3134783-21ntpq21:01:441
3019205993631,4cyclictest3040727-21kworker/u8:3+flush-8:020:01:470
3019211993527,5cyclictest3065816-21cat19:55:002
3019207993531,3cyclictest3056188-21kworker/u8:2+flush-8:020:20:311
3019207993528,5cyclictest280-21systemd-journal22:59:141
3019207993528,5cyclictest280-21systemd-journal22:59:141
3019207993528,5cyclictest2190942-21kworker/u8:2+flush-8:019:26:471
3019207993527,6cyclictest3056188-21kworker/u8:2+flush-8:021:11:511
3019207993520,6cyclictest3040727-21kworker/u8:3+flush-8:019:46:471
3019205993531,3cyclictest3120732-21ls20:50:000
3019205993530,4cyclictest3185064-21kworker/u8:0+flush-8:022:51:320
3019205993526,7cyclictest3110856-21latency_hist20:40:010
3019207993428,5cyclictest3056188-21kworker/u8:2+flush-8:023:11:471
3019207993428,4cyclictest3154584-21ntpq21:21:431
3019207993428,4cyclictest3056188-21kworker/u8:2+events_unbound20:51:311
3019207993427,5cyclictest280-21systemd-journal00:25:001
3019205993429,3cyclictest3019063-21kworker/u8:1+flush-8:019:26:370
30144472344,10sleep10-21swapper/119:05:491
3019207993329,3cyclictest3185064-21kworker/u8:0+events_unbound23:45:011
3019207993329,3cyclictest3056188-21kworker/u8:2+events_unbound23:23:141
3019207993328,4cyclictest3185064-21kworker/u8:0+flush-8:023:06:441
3019207993328,3cyclictest3294770-21kworker/u8:3+flush-8:000:26:501
3019207993327,5cyclictest3040727-21kworker/u8:3+events_unbound20:01:431
3019207993322,8cyclictest3308753-21ntpq23:56:431
3019207993228,3cyclictest3185064-21kworker/u8:0+events_unbound22:11:331
3019207993227,3cyclictest3041929-21irqrtprio19:31:381
3019207993225,5cyclictest3309640-21kworker/u8:2+events_unbound00:16:311
3019207993225,5cyclictest3170472-21kworker/u8:3+flush-8:022:26:481
3019205993226,4cyclictest3185064-21kworker/u8:0+events_unbound23:01:470
3019205993224,6cyclictest3334240-21unixbench_multi00:21:500
3019207993126,3cyclictest3046010-21kworker/u8:1+flush-8:020:11:401
3019207993120,7cyclictest280-21systemd-journal22:08:141
3019205993127,3cyclictest3075836-21kworker/u8:0+flush-8:020:21:460
3019205993126,4cyclictest3185064-21kworker/u8:0+events_unbound22:01:490
3019205993125,4cyclictest3102049-21irqrtprio20:31:370
3019205993124,5cyclictest3150130-21unixbench_singl21:16:490
3019211993026,2cyclictest3175190-21cron21:45:002
3019207993027,2cyclictest2190942-21kworker/u8:2+events_unbound19:11:381
3019207993025,4cyclictest3056188-21kworker/u8:2+flush-8:021:26:351
3019205993024,5cyclictest3155366-21cron21:25:000
3019205993024,4cyclictest3185064-21kworker/u8:0+flush-8:022:36:490
3019211992926,2cyclictest3091046-21sendmail20:20:002
3019207992924,4cyclictest3019063-21kworker/u8:1+events_unbound19:16:491
3019207992924,3cyclictest3185064-21kworker/u8:0+flush-8:022:41:361
3019207992923,4cyclictest3096253-21kworker/u8:1+flush-8:021:36:461
3019205992925,3cyclictest3170472-21kworker/u8:3+events_unbound21:51:330
3019205992925,2cyclictest0-21swapper/019:40:000
3019205992924,3cyclictest3185064-21kworker/u8:0+flush-8:000:01:490
3019205992921,6cyclictest3200114-21cron22:10:000
3019205992921,6cyclictest3140919-21aten2_r5power_c21:11:340
3019205992919,8cyclictest3285466-21diskstats23:36:340
3019205992824,3cyclictest3096253-21kworker/u8:1+flush-8:021:06:340
3019205992824,3cyclictest3056188-21kworker/u8:2+events_unbound22:31:380
3019205992824,2cyclictest3160326-21cron21:30:000
3019205992823,4cyclictest3170472-21kworker/u8:3+flush-8:022:49:520
3019205992823,3cyclictest882-21mta-sts-daemon22:29:140
3019205992823,3cyclictest3329544-21latency_hist00:20:000
3019205992822,5cyclictest3056373-21aten2_r5power_p19:46:310
3019205992821,5cyclictest3320010-21aten2_r5power_v00:11:320
3019205992821,5cyclictest3189701-21unixbench_singl21:56:500
3019205992820,6cyclictest3121075-21/usr/sbin/munin20:51:400
3019205992817,2cyclictest161rcu_preempt20:41:460
3019211992722,3cyclictest3134692-21grep21:01:422
3019211992722,3cyclictest3055914-21cron19:45:002
3019211992721,4cyclictest3080822-21journal-offline20:08:552
3019207992722,3cyclictest3004367-21kworker/u8:0+flush-8:019:41:511
3019207992721,5cyclictest3096253-21kworker/u8:1+flush-8:021:41:321
3019207992720,5cyclictest3304625-21latency_hist23:55:011
3019205992723,3cyclictest3264465-21unixbench_multi23:11:480
3019205992723,3cyclictest3019063-21kworker/u8:1+flush-8:019:21:350
3019205992723,3cyclictest2190942-21kworker/u8:2+events_unbound19:11:490
3019205992720,5cyclictest3269433-21unixbench_singl23:16:480
3019211992623,2cyclictest3309579-21cron00:00:002
3019211992622,3cyclictest3229686-21unixbench_multi22:36:492
3019211992622,2cyclictest3040511-21grep19:26:482
3019211992621,4cyclictest3329389-21munin-run00:20:002
3019211992621,3cyclictest3259569-21unixbench_multi23:06:502
3019211992618,6cyclictest3145496-21awk21:15:002
3019211992612,9cyclictest3300083-21apt-get23:51:312
3019207992622,3cyclictest3185064-21kworker/u8:0+events_unbound23:26:371
3019207992622,3cyclictest3096253-21kworker/u8:1+flush-8:020:46:351
3019207992621,4cyclictest3096253-21kworker/u8:1+flush-8:020:26:421
3019207992620,5cyclictest3309640-21kworker/u8:2+events_unbound00:31:431
3019207992616,8cyclictest3129760-21ntpq20:56:411
3019207992616,8cyclictest3129760-21ntpq20:56:411
3019205992624,1cyclictest3185064-21kworker/u8:0+events_unbound23:56:470
3019205992620,4cyclictest3280480-21cpuspeed_turbos23:31:320
3019205992620,4cyclictest3214343-21needreboot22:21:420
3019205992620,4cyclictest3090708-21unixbench_singl20:16:500
3019205992615,8cyclictest3335183-21diskstats00:26:340
33379372250,2sleep3471rcuc/300:26:403
3019211992522,2cyclictest3230645-21cpuspeed_turbos22:41:342
3019211992522,2cyclictest3166576-21kthreadcore21:36:362
3019211992522,2cyclictest3019497-21aten2_r5power_c19:11:322
3019211992521,2cyclictest3185673-21uniq21:56:342
3019211992521,2cyclictest3115817-21cron20:45:002
3019211992521,2cyclictest3101436-21aten2_r5power_v20:31:322
3019211992518,6cyclictest3288898-21needreboot23:36:422
3019207992520,4cyclictest3056188-21kworker/u8:2+events_unbound20:36:341
3019205992521,3cyclictest3185064-21kworker/u8:0+events_unbound23:06:380
3019205992521,3cyclictest20977-21cron19:20:000
3019205992521,2cyclictest3304635-21sh23:55:000
3019205992520,4cyclictest0-21swapper/021:40:000
3019205992520,3cyclictest3319259-21unixbench_singl00:06:490
3019205992519,4cyclictest3175207-21latency_hist21:45:010
3019211992415,7cyclictest280-21systemd-journal00:39:142
3019207992420,2cyclictest3285937-21irqrtprio23:36:381
3019207992419,3cyclictest0-21swapper/121:06:371
3019205992421,2cyclictest3180106-21apt21:50:000
3019205992420,3cyclictest3046010-21kworker/u8:1+events_unbound20:11:390
3019205992420,2cyclictest0-21swapper/000:31:370
3019205992418,3cyclictest0-21swapper/021:31:450
3019205992416,6cyclictest3295821-21irqrtprio23:46:360
3019205992411,8cyclictest161rcu_preempt20:06:480
3019211992319,2cyclictest3081547-21aten2_r5power_c20:11:332
3019211992318,3cyclictest3270714-21fschecks_time23:21:342
3019211992318,3cyclictest3195787-21tr22:06:342
301921199221,9cyclictest32088792kthreadcore22:16:402
3019211992219,2cyclictest3184734-21unixbench_singl21:51:502
3019211992219,2cyclictest3156026-21fschecks_count21:26:342
3019211992218,2cyclictest3239615-21unixbench_singl22:46:492
3019211992214,6cyclictest3180112-21turbostat.cron21:50:002
3019211992213,6cyclictest3145844-21apt-get21:16:312
3019211992211,8cyclictest3096022-21awk20:25:002
3019207992217,3cyclictest0-21swapper/100:06:381
3019205992217,3cyclictest0-21swapper/020:56:390
3019205992217,3cyclictest0-21swapper/020:56:390
31823892210,2sleep375-21kworker/3:1+events21:51:393
3019211992118,2cyclictest3096604-21cpuspeed_turbos20:26:322
3019211992117,2cyclictest0-21swapper/223:20:002
3019211992117,2cyclictest0-21swapper/222:15:002
3019211992111,8cyclictest280-21systemd-journal23:46:142
3019207992117,2cyclictest880-21mta-sts-daemon19:51:321
301920599212,18cyclictest0-21swapper/023:21:480
3019205992117,2cyclictest0-21swapper/023:26:420
3019205992116,3cyclictest0-21swapper/023:41:400
3019211992016,3cyclictest3060857-21turbostat.cron19:50:002
3019211992016,2cyclictest3210861-21tr22:21:342
3019211992016,2cyclictest3126260-21tr20:56:312
3019211992016,2cyclictest3126260-21tr20:56:312
3019211992016,2cyclictest0-21swapper/220:05:002
3019211992013,5cyclictest0-21swapper/219:31:292
3019211991916,2cyclictest0-21swapper/200:01:382
3019211991915,2cyclictest3344053-21unixbench_multi00:31:492
3019211991915,2cyclictest3290403-21tr23:41:332
3019211991915,2cyclictest0-21swapper/223:31:422
3019211991915,2cyclictest0-21swapper/223:30:002
3019211991915,2cyclictest0-21swapper/200:25:002
3019211991911,6cyclictest3191796-21cut22:01:382
31973902180,2sleep3481ktimers/322:06:403
301921199189,7cyclictest3245085-21sed22:55:002
3019211991813,4cyclictest0-21swapper/221:21:382
3019211991810,6cyclictest3120727-21cron20:50:002
30188402183,8sleep30-21swapper/319:07:403
31356922170,6sleep30-21swapper/321:05:003
301921199179,6cyclictest3162846-21tr21:31:392
301921199178,7cyclictest3262765-21tr23:11:382
3019211991713,2cyclictest0-21swapper/222:31:372
3019211991713,2cyclictest0-21swapper/222:26:342
3019211991712,3cyclictest3319530-21ls00:10:002
301921199171,12cyclictest3049953-21ntpq19:36:432
301921199170,8cyclictest32530161kthreadcore23:01:402
301920599172,9cyclictest3019203-21latency_hist19:10:010
301921199162,13cyclictest0-21swapper/222:59:472
301921199162,13cyclictest0-21swapper/222:59:472
301921199162,13cyclictest0-21swapper/220:50:522
301921199162,13cyclictest0-21swapper/219:21:252
301921199162,13cyclictest0-21swapper/200:12:572
3019211991613,2cyclictest0-21swapper/220:36:472
3019211991613,2cyclictest0-21swapper/200:26:112
32749622132,6sleep30-21swapper/323:25:023
32263772132,7sleep30-21swapper/322:36:383
31793822132,7sleep30-21swapper/321:46:463
31305902132,7sleep30-21swapper/321:00:003
31305902132,7sleep30-21swapper/321:00:003
33443942122,6sleep30-21swapper/300:35:003
33246052122,6sleep30-21swapper/300:15:023
33136812122,6sleep30-21swapper/300:01:413
33042632122,6sleep30-21swapper/323:51:503
32950772122,6sleep30-21swapper/323:46:293
32856892122,6sleep30-21swapper/323:36:373
32830602122,6sleep30-21swapper/323:31:403
32651932122,6sleep30-21swapper/323:16:313
32612792122,6sleep30-21swapper/323:11:373
32574522122,6sleep30-21swapper/323:06:403
32540762122,7sleep30-21swapper/323:01:423
32454532122,6sleep30-21swapper/322:56:333
32454532122,6sleep30-21swapper/322:56:333
32444522122,6sleep30-21swapper/322:51:473
32356272122,6sleep30-21swapper/322:46:343
32340322122,6sleep30-21swapper/322:41:443
32102472122,5sleep30-21swapper/322:20:013
32077842122,6sleep30-21swapper/322:16:403
32002662122,6sleep30-21swapper/322:10:023
31408402122,6sleep30-21swapper/321:11:153
31148442122,6sleep30-21swapper/320:41:423
30861062122,5sleep30-21swapper/320:15:013
30858182122,6sleep30-21swapper/320:11:523
30608732122,5sleep30-21swapper/319:50:013
30595182122,6sleep30-21swapper/319:46:403
30509852122,5sleep30-21swapper/319:40:023
30497582122,6sleep30-21swapper/319:36:413
30423262122,6sleep30-21swapper/319:31:403
30359262122,5sleep30-21swapper/319:25:013
30313552122,6sleep30-21swapper/319:21:323
33487652112,5sleep30-21swapper/300:36:493
33299282112,6sleep30-21swapper/300:20:563
33198232112,5sleep30-21swapper/300:10:523
33195222112,6sleep30-21swapper/300:10:003
33087522112,5sleep30-21swapper/323:56:453
32910562112,5sleep30-21swapper/323:41:393
32702902112,6sleep30-21swapper/323:20:523
32204042112,6sleep30-21swapper/322:30:163
32159352112,5sleep30-21swapper/322:26:353
31943482112,5sleep30-21swapper/322:01:433
31850602111,5sleep30-21swapper/321:55:013
31715012112,5sleep30-21swapper/321:41:373
31661202112,5sleep30-21swapper/321:36:363
31603162112,5sleep30-21swapper/321:30:013
31556602112,6sleep30-21swapper/321:25:523
31505062112,5sleep30-21swapper/321:20:023
31463472112,6sleep30-21swapper/321:16:373
31368282112,5sleep30-21swapper/321:06:383
31213472112,5sleep30-21swapper/320:51:333
31161452112,5sleep30-21swapper/320:46:153
31063082112,6sleep30-21swapper/320:36:323
31011832112,5sleep30-21swapper/320:30:023
30917892112,5sleep30-21swapper/320:21:353
30761622112,4sleep30-21swapper/320:05:023
30711062112,5sleep30-21swapper/320:01:293
30665572112,5sleep30-21swapper/319:56:353
30262362112,6sleep30-21swapper/319:15:143
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional