You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-28 - 18:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot4s.osadl.org (updated Sat Mar 28, 2026 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
381768622280,5sleep375-21kworker/3:1+events10:06:343
370949521090,5chrt0-21swapper/208:16:372
370949521090,5chrt0-21swapper/208:16:372
37651402850,4sleep30-21swapper/309:11:363
38919602790,2chrt281rcuc/111:21:321
3640954997048,18cyclictest3831219-21latency_hist10:20:010
38057542580,2chrt451irq_work/309:51:413
38306522570,9sleep20-21swapper/210:16:442
3640958995243,6cyclictest3746926-21latency_hist08:55:011
3640958995039,9cyclictest3672848-21apt-get07:41:241
3640954995038,9cyclictest3935297-21ntpq12:01:370
3640954995038,9cyclictest3935297-21ntpq12:01:370
3640958994930,16cyclictest3941090-21latency_hist12:10:001
3640958994723,21cyclictest3652655-21latency_hist07:20:011
3640954994736,8cyclictest3716993-21latency_hist08:25:000
3640954994640,4cyclictest3881523-21apt-get11:11:260
3640958994528,15cyclictest3721618-21grep08:26:461
3640954994536,6cyclictest3682401-21latency_hist07:50:010
3640954994535,7cyclictest3677469-21latency_hist07:45:010
3640958994334,7cyclictest3657943-21/usr/sbin/munin07:26:381
3640958994233,6cyclictest3702138-21latency_hist08:10:011
3640958994223,16cyclictest3728199-21irqrtprio08:36:341
3640954994235,5cyclictest280-21systemd-journal07:13:190
3640954994233,6cyclictest3861125-21latency_hist10:50:010
3640954994213,23cyclictest3712097-21kworker/0:1+events09:46:260
3640958994121,9cyclictest3966188-21base6412:35:011
3640954994135,4cyclictest3951576-21apt-get12:21:260
3640958994019,7cyclictest3641178-21/usr/sbin/munin07:11:361
3640958993928,9cyclictest3815564-21ntpq10:01:361
3640958993928,9cyclictest3815564-21ntpq10:01:361
3640954993831,5cyclictest3846033-21cron10:35:000
3640954993813,23cyclictest3931257-21kworker/0:0+events12:26:420
3640958993722,3cyclictest3816986-21cpuspeed_turbos10:06:281
3640958993721,3cyclictest3910283-21needreboot11:36:371
3640958993721,2cyclictest3902337-21irqrtprio11:31:331
3640958993721,2cyclictest3852083-21irqrtprio10:41:331
3640958993716,8cyclictest3781825-21apt-get09:31:221
3640954993730,5cyclictest3782116-21grep09:31:270
364095499372,28cyclictest3640809-21kworker/0:2+events07:21:260
3640958993621,2cyclictest3761011-21grep09:06:431
3640958993617,7cyclictest3806874-21apt-get09:56:231
3640954993626,8cyclictest3780690-21ntpq09:26:400
3640954993613,21cyclictest3640809-21kworker/0:2+events07:16:360
364095499361,28cyclictest3712097-21kworker/0:1+events11:26:260
3640954993612,22cyclictest3712097-21kworker/0:1+events08:26:440
3640954993612,17cyclictest3712097-21kworker/0:1+events08:41:260
36402242364,11sleep10-21swapper/107:06:461
3640958993527,6cyclictest3866441-21apt-get10:56:251
3640958993525,8cyclictest280-21systemd-journal07:36:221
3640954993529,4cyclictest3915760-21unixbench_multi11:41:450
364095499352,27cyclictest3712097-21kworker/0:1+events11:50:020
3640958993425,7cyclictest280-21systemd-journal07:54:221
364095499343,29cyclictest3712097-21kworker/0:1+events09:01:420
364095499343,29cyclictest3712097-21kworker/0:1+events08:51:340
3640954993427,5cyclictest3921054-21cron11:50:000
364095499342,30cyclictest3712097-21kworker/0:1+events09:51:260
364095499341,27cyclictest3712097-21kworker/0:1+events08:56:260
3640954993410,22cyclictest3712097-21kworker/0:1+events11:31:400
364095499331,29cyclictest3712097-21kworker/0:1+events09:25:000
3640954993310,21cyclictest3931257-21kworker/0:0+events12:36:260
3640958993227,4cyclictest3742230-21kworker/u8:2+events_unbound09:01:341
3640958993224,6cyclictest280-21systemd-journal07:57:221
364095499329,18cyclictest3712097-21kworker/0:1+events08:35:060
364095499328,22cyclictest3712097-21kworker/0:1+events11:36:260
364095499323,27cyclictest3640809-21kworker/0:2+events07:56:260
3640954993224,5cyclictest3663632-21irqrtprio07:31:340
364095499322,29cyclictest3712097-21kworker/0:1+events09:07:100
364095499321,29cyclictest3931257-21kworker/0:0+events12:15:000
3640954993210,20cyclictest3831257-21kworker/0:0+events10:21:260
3640954993210,17cyclictest3712097-21kworker/0:1+events12:00:000
3640963993127,2cyclictest3662248-21unixbench_singl07:26:462
3640954993126,4cyclictest2799977-21kworker/u8:3+flush-8:008:06:310
364095499312,27cyclictest3640809-21kworker/0:2+events07:26:380
3640954993121,7cyclictest3855933-21apt-get10:45:010
3640954993112,17cyclictest3712097-21kworker/0:1+events10:06:400
3640958993027,2cyclictest3848560-21kthreadcore10:36:351
3640958993020,8cyclictest3920238-21ntpq11:46:381
3640954993026,2cyclictest3686447-21needreboot07:51:340
3640954993022,6cyclictest3876252-21apt-key11:05:010
364095499302,26cyclictest3712097-21kworker/0:1+events11:16:340
364095499302,26cyclictest3712097-21kworker/0:1+events10:54:500
364095499302,26cyclictest3712097-21kworker/0:1+events10:36:340
3640954993021,6cyclictest3841085-21latency_hist10:30:010
364095499301,28cyclictest3712097-21kworker/0:1+events10:56:440
3640954993010,16cyclictest3931257-21kworker/0:0+events12:30:060
3640963992925,2cyclictest3821764-21aten2_r5power_p10:11:262
3640963992924,3cyclictest3896794-21fschecks_time11:26:302
3640963992924,3cyclictest3668157-21fschecks_count07:36:312
3640963992923,4cyclictest3741623-21unixbench_multi08:46:452
3640963992920,7cyclictest280-21systemd-journal11:10:242
3640958992925,3cyclictest3886375-21kworker/u8:0+flush-8:011:16:331
3640958992924,3cyclictest280-21systemd-journal09:55:001
3640958992922,5cyclictest280-21systemd-journal08:49:251
3640958992922,5cyclictest280-21systemd-journal07:33:221
3640958992919,8cyclictest3970372-21ntpq12:36:391
364095499299,19cyclictest3712097-21kworker/0:1+events09:59:360
3640954992922,5cyclictest3671601-21ntpq07:36:400
3640954992922,4cyclictest3727348-21kworker/u8:0+flush-8:010:01:400
3640954992922,4cyclictest3727348-21kworker/u8:0+flush-8:010:01:400
364095499291,27cyclictest3640809-21kworker/0:2+events08:16:400
364095499291,27cyclictest3640809-21kworker/0:2+events08:16:400
3640954992911,16cyclictest3931257-21kworker/0:0+events12:16:240
3640954992911,16cyclictest3931257-21kworker/0:0+events12:06:300
3640954992910,14cyclictest3712097-21kworker/0:1+events08:45:260
3640963992824,2cyclictest3935839-21unixbench_singl12:01:452
3640963992824,2cyclictest3935839-21unixbench_singl12:01:452
3640963992821,5cyclictest3876399-21base6411:05:012
3640963992819,7cyclictest3921157-21cp11:50:002
3640958992822,4cyclictest3712039-21latency_hist08:20:001
3640958992822,4cyclictest3712039-21latency_hist08:20:001
3640958992816,9cyclictest280-21systemd-journal08:00:221
364095499286,17cyclictest3712097-21kworker/0:1+events11:00:580
364095499282,25cyclictest3640809-21kworker/0:2+events08:02:460
3640954992811,15cyclictest3640809-21kworker/0:2+events08:11:440
3640954992810,16cyclictest3712097-21kworker/0:1+events09:16:400
38871172270,5sleep3451irq_work/311:16:333
3640963992723,2cyclictest3891800-21fschecks_time11:21:302
3640963992722,4cyclictest3791059-21unixbench_multi09:36:452
3640963992722,3cyclictest3876650-21aten2_r5power_p11:06:282
3640963992722,3cyclictest3742544-21cpuspeed_turbos08:51:282
3640958992723,2cyclictest3878735-21kthreadcore11:06:361
3640958992723,2cyclictest0-21swapper/111:30:001
3640958992722,3cyclictest3956204-21munin-plugin-st12:25:001
3640958992719,6cyclictest3772211-21diskstats09:21:311
3640958992719,5cyclictest3791414-21latency_hist09:40:011
364095499279,17cyclictest3712097-21kworker/0:1+events09:38:560
364095499274,22cyclictest3712097-21kworker/0:1+events09:11:020
3640954992720,5cyclictest3825485-21ntpq10:11:380
3640954992718,6cyclictest3795535-21ntpq09:41:360
3640963992622,2cyclictest3682052-21unixbench_multi07:46:442
3640958992620,3cyclictest0-21swapper/112:01:341
3640958992620,3cyclictest0-21swapper/112:01:341
364095499267,14cyclictest3712097-21kworker/0:1+events08:30:280
364096399257,3cyclictest37165822timerwakeupswit08:21:452
3640963992522,2cyclictest3946964-21fschecks_time12:16:292
3640963992521,3cyclictest3797044-21fschecks_count09:46:302
3640963992521,2cyclictest3722501-21cpuspeed_turbos08:31:252
3640963992521,2cyclictest3645508-21unixbench_singl07:11:442
3640958992521,2cyclictest0-21swapper/110:20:011
3640958992519,4cyclictest3840757-21unixbench_multi10:26:481
3640958992519,4cyclictest3826002-21unixbench_singl10:11:441
3640954992519,4cyclictest3891776-21fschecks_count11:21:300
3640963992420,2cyclictest3727744-21fschecks_count08:36:312
3640963992419,3cyclictest3850617-21unixbench_multi10:36:462
3640963992419,3cyclictest3692944-21fschecks_time08:01:302
3640963992417,5cyclictest3956187-21cron12:25:002
3640958992420,2cyclictest0-21swapper/111:56:401
3640958992418,3cyclictest0-21swapper/109:46:411
3640958992410,5cyclictest0-21swapper/110:46:351
3640963992319,2cyclictest3648341-21fschecks_count07:16:302
3640963992317,4cyclictest3742230-21kworker/u8:2+flush-8:009:31:272
3640958992319,3cyclictest3960813-21unixbench_singl12:26:471
3640958992319,3cyclictest3713132-21irqrtprio08:21:321
3640958992319,2cyclictest0-21swapper/109:26:291
3640958992318,3cyclictest3653119-21aten2_r5power_p07:21:271
3640963992212,7cyclictest3855921-21cron10:45:002
364096399220,3cyclictest39273662cut11:56:322
3640958992219,2cyclictest3947205-21irqcore12:16:311
3640958992219,2cyclictest0-21swapper/111:01:381
3640958992218,2cyclictest0-21swapper/112:11:371
3640958992218,2cyclictest0-21swapper/110:21:341
3640958992218,2cyclictest0-21swapper/109:41:321
3640958992218,2cyclictest0-21swapper/108:11:301
3640958992218,2cyclictest0-21swapper/107:46:271
3640958992217,3cyclictest3865754-21unixbench_multi10:51:451
3640958992217,3cyclictest0-21swapper/109:11:281
3640958992216,5cyclictest3662562-21kworker/u8:2+flush-8:008:31:371
39681692210,11sleep30-21swapper/312:36:353
3640963992117,3cyclictest3762280-21cpuspeed_turbos09:11:282
3640963992117,2cyclictest3771262-21unixbench_singl09:16:452
3640963992117,2cyclictest0-21swapper/210:05:002
3640963992117,2cyclictest0-21swapper/210:05:002
3640963992116,4cyclictest3751832-21sendmail09:00:002
3640963992116,3cyclictest0-21swapper/211:40:002
3640963992115,3cyclictest0-21swapper/209:00:562
3640958992117,3cyclictest3846042-21idleruntime-cro10:35:001
3640958992117,3cyclictest3747873-21irqcore08:56:331
3640958992116,3cyclictest3886134-21cron11:15:001
3640958992115,4cyclictest0-21swapper/111:51:321
3640958992115,4cyclictest0-21swapper/111:41:431
3640958992115,4cyclictest0-21swapper/108:41:291
36362032211,12sleep2280-21systemd-journal07:06:192
3640963992016,2cyclictest3870130-21grep10:56:372
3640963992016,2cyclictest0-21swapper/212:35:002
3640963992012,6cyclictest3901470-21sed11:30:012
3640958992016,2cyclictest0-21swapper/109:16:321
364096399196,9cyclictest3682757-21apt-get07:51:232
364096399192,12cyclictest3717348-21apt-get08:26:272
3640963991916,2cyclictest3772312-21fschecks_time09:21:312
3640963991915,2cyclictest3781109-21unixbench_multi09:26:452
3640963991915,2cyclictest0-21swapper/212:15:012
3640963991915,2cyclictest0-21swapper/212:06:432
3640963991914,3cyclictest3865728-21unixbench_multi10:51:452
364096399191,13cyclictest3672852-21taskset07:41:262
36399412193,8sleep30-21swapper/307:06:403
37105532180,5sleep3451irq_work/308:16:383
37105532180,5sleep3451irq_work/308:16:383
364096399185,8cyclictest3702498-21apt-get08:11:262
364096399182,11cyclictest3806874-21apt-get09:56:252
3640963991814,2cyclictest3856558-21aten2_r5power_c10:46:282
3640963991814,2cyclictest0-21swapper/210:21:342
3640963991813,4cyclictest0-21swapper/211:51:512
3640963991813,3cyclictest3653373-21grep07:21:292
3640963991813,2cyclictest0-21swapper/210:31:372
3640963991812,4cyclictest0-21swapper/208:06:352
3640963991810,6cyclictest3796383-21cron09:45:002
3640954991814,2cyclictest0-21swapper/007:10:010
364096399177,7cyclictest3818679-21cat10:06:352
3640963991713,2cyclictest0-21swapper/212:36:132
3640963991713,2cyclictest0-21swapper/209:51:282
3640963991712,3cyclictest0-21swapper/210:26:372
3640963991711,2cyclictest0-21swapper/207:58:562
364096399168,6cyclictest3887466-21seq11:16:342
3640963991613,2cyclictest0-21swapper/208:43:202
3640963991612,2cyclictest0-21swapper/212:26:312
3640963991611,3cyclictest0-21swapper/211:45:002
37870112150,3sleep375-21kworker/3:1-events09:36:283
3640963991511,2cyclictest0-21swapper/209:06:322
3640963991510,3cyclictest0-21swapper/207:31:462
39348342140,2sleep3471rcuc/312:01:353
39348342140,2sleep3471rcuc/312:01:353
39611252133,5sleep30-21swapper/312:30:013
39554422132,7sleep30-21swapper/312:21:433
38602702132,7sleep30-21swapper/310:46:403
38510232132,6sleep30-21swapper/310:40:023
37668122132,6sleep30-21swapper/309:15:023
37609632133,6sleep30-21swapper/309:06:433
37419912132,6sleep30-21swapper/308:50:023
36923922132,6sleep30-21swapper/308:00:023
39603942122,6sleep30-21swapper/312:26:443
39508242122,6sleep30-21swapper/312:16:453
39402912122,6sleep30-21swapper/312:06:383
39257192121,7sleep30-21swapper/311:51:533
39164782122,6sleep30-21swapper/311:46:273
39112162122,5sleep30-21swapper/311:40:013
39053062122,6sleep30-21swapper/311:31:383
38814252122,6sleep30-21swapper/311:10:323
38762302122,5sleep30-21swapper/311:05:023
38710262121,7sleep30-21swapper/311:00:003
38463312122,6sleep30-21swapper/310:35:523
38378842122,6sleep30-21swapper/310:26:373
38262942122,5sleep30-21swapper/310:15:013
38103482122,6sleep30-21swapper/309:56:353
37971772122,6sleep30-21swapper/309:46:323
37765542122,5sleep30-21swapper/309:25:013
37722472122,6sleep30-21swapper/309:21:323
37476622122,6sleep30-21swapper/308:56:323
37419082122,6sleep30-21swapper/308:49:263
37266082122,6sleep30-21swapper/308:31:443
36877002123,6sleep30-21swapper/307:56:283
36826262122,5sleep30-21swapper/307:50:023
36732892122,6sleep30-21swapper/307:41:323
36675282122,5sleep30-21swapper/307:35:013
36672332122,6sleep30-21swapper/307:31:483
39414512111,6sleep30-21swapper/312:11:273
39265432112,6sleep30-21swapper/311:56:023
39064272112,5sleep30-21swapper/311:35:123
39009772112,6sleep30-21swapper/311:29:383
38952472112,5sleep30-21swapper/311:21:433
38713002112,5sleep30-21swapper/311:00:163
38617272112,6sleep30-21swapper/310:51:293
38413062112,5sleep30-21swapper/310:30:023
38355932112,5sleep30-21swapper/310:21:443
38216492112,5sleep30-21swapper/310:11:243
38122302111,6sleep30-21swapper/310:01:293
38122302111,6sleep30-21swapper/310:01:293
37916562112,5sleep30-21swapper/309:40:023
37820832112,5sleep30-21swapper/309:31:273
37521042112,5sleep30-21swapper/309:00:253
37328142112,5sleep30-21swapper/308:41:313
37298152112,6sleep30-21swapper/308:36:373
37173152112,5sleep30-21swapper/308:26:223
37122952112,5sleep30-21swapper/308:20:163
37021522112,5sleep30-21swapper/308:10:013
36979582112,6sleep30-21swapper/308:06:323
36820322112,5sleep30-21swapper/307:46:453
36578782113,5sleep30-21swapper/307:25:383
36535562112,6sleep30-21swapper/307:21:333
36499392112,5sleep30-21swapper/307:16:353
36411092112,6sleep30-21swapper/307:10:333
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional