You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-26 - 19:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Thu Mar 26, 2026 12:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
41076582740,5chrt0-21swapper/007:41:360
4073234996251,8cyclictest4078163-21latency_hist07:15:011
4073229996242,17cyclictest4154453-21latency_hist08:30:000
316732590,5sleep30-21swapper/309:41:343
40705702570,6chrt451irq_work/307:06:353
4073234995548,5cyclictest34422-21kworker/u8:1+events_unbound10:00:011
4073229995545,7cyclictest4124805-21latency_hist08:00:010
1390782530,1chrt75-21kworker/3:1-events11:30:013
117552530,2chrt141ktimers/009:21:360
4073229995135,14cyclictest79568-21base6410:30:000
4073229995135,14cyclictest79568-21base6410:30:000
41186192500,4sleep2371rcuc/207:51:372
4073229995040,7cyclictest153765-21latency_hist11:45:000
4073229994929,18cyclictest94493-21latency_hist10:45:010
1493582490,5sleep375-21kworker/3:1+events11:41:283
4073229994826,20cyclictest69518-21idleruntime-cro10:20:000
4073229994536,6cyclictest4119867-21latency_hist07:55:010
4073229994535,7cyclictest208777-21latency_hist12:40:010
4073234994436,6cyclictest34932-21apt-key09:45:011
4073229994429,13cyclictest280-21systemd-journal08:06:380
4073229994425,3cyclictest5007-21cron09:15:000
4073234994232,8cyclictest168008-21ntpq11:56:401
4073234994132,6cyclictest280-21systemd-journal07:48:381
4073229994131,8cyclictest189388-21diskstats12:21:300
4073229994131,8cyclictest189388-21diskstats12:21:300
41364222400,4sleep0521rcuop/308:11:340
4073234994033,5cyclictest4089943-21latency_hist07:25:011
4073234994028,9cyclictest4099009-21ntpq07:31:391
4073234993730,5cyclictest4104799-21apt-get07:40:011
4073234993729,6cyclictest280-21systemd-journal09:30:001
4073229993627,7cyclictest55053-21/usr/sbin/munin10:06:440
4073234993525,8cyclictest123327-21ntpq11:11:471
4073229993528,5cyclictest280-21systemd-journal11:17:070
4073229993425,6cyclictest4144565-21latency_hist08:20:000
4073234993326,5cyclictest4073083-21kworker/u8:2+flush-8:007:20:001
4073234993325,6cyclictest194047-21/usr/sbin/munin12:26:361
407322999331,26cyclictest4063288-21kworker/0:0+events08:25:000
407322999331,26cyclictest4063288-21kworker/0:0+events08:25:000
4073239993222,8cyclictest4104916-21cp07:40:002
4073234993225,5cyclictest73632-21ntpq10:21:401
4073234993225,5cyclictest73632-21ntpq10:21:401
4073234993224,6cyclictest84586-21latency_hist10:35:001
4073234993222,8cyclictest4183322-21ntpq08:56:401
407322999329,21cyclictest94517-21kworker/0:1+events12:18:360
4073229993223,6cyclictest99427-21latency_hist10:50:010
407322999322,26cyclictest4063288-21kworker/0:0+events07:36:260
4073234993125,4cyclictest129047-21cron11:20:001
4073229993127,2cyclictest124116-21sh11:15:000
407322999312,27cyclictest4063288-21kworker/0:0+events08:01:320
4073234993025,3cyclictest57226-21kthreadcore10:06:351
4073234993025,3cyclictest186339-21kthreadcore12:16:361
4073234993025,3cyclictest136509-21kthreadcore11:26:351
4073234993024,4cyclictest2694856-21kworker/u8:4+events_unbound08:01:451
4073234993024,3cyclictest4091026-21irqrtprio07:26:331
4073234993018,10cyclictest4124055-21ntpq07:56:421
407322999302,26cyclictest94517-21kworker/0:1+events11:36:380
4073229993021,6cyclictest104365-21latency_hist10:55:010
407322999301,28cyclictest4063288-21kworker/0:0+events08:51:340
4073234992922,5cyclictest4143709-21ntpq08:16:381
4073234992920,7cyclictest5011-21latency_hist09:15:011
4073234992919,5cyclictest0-21swapper/110:51:281
4073234992918,8cyclictest280-21systemd-journal11:05:001
4073229992926,2cyclictest4194024-21cron09:10:000
4073229992925,3cyclictest198694-21systemd-run12:30:000
4073229992924,3cyclictest4158516-21needreboot08:31:390
4073229992922,5cyclictest280-21systemd-journal11:09:220
407322999291,26cyclictest94517-21kworker/0:1+events11:21:360
4073239992824,2cyclictest4085469-21aten2_r5power_v07:21:302
4073239992821,5cyclictest34671-21apt-get09:45:002
4073239992819,7cyclictest280-21systemd-journal11:20:002
4073234992822,4cyclictest4149872-21/usr/sbin/munin08:26:451
4073234992821,4cyclictest0-21swapper/112:06:341
407322999288,18cyclictest94517-21kworker/0:1+events11:01:400
407322999282,25cyclictest94517-21kworker/0:1+events11:56:360
4073229992810,16cyclictest4063288-21kworker/0:0+events10:36:280
4073239992723,2cyclictest75037-21cpuspeed_turbos10:26:282
4073239992723,2cyclictest75037-21cpuspeed_turbos10:26:282
4073239992723,2cyclictest194422-21fschecks_time12:26:292
4073239992721,4cyclictest173754-21apt-get12:05:002
4073239992717,8cyclictest280-21systemd-journal11:55:012
4073234992722,3cyclictest159512-21cpuspeed_turbos11:51:291
4073234992722,3cyclictest149907-21irqrtprio11:41:331
4073234992721,4cyclictest24755-21kworker/u8:3+events_unbound10:11:331
4073234992720,5cyclictest89549-21latency_hist10:40:011
4073234992720,5cyclictest4134694-21latency_hist08:10:011
4073234992719,6cyclictest199090-21aten2_r5power_e12:31:281
4073234992715,8cyclictest280-21systemd-journal09:21:111
4073229992721,4cyclictest4188784-21unixbench_singl09:01:460
4073229992710,16cyclictest4073093-21kworker/0:1+events07:11:420
4073229992710,15cyclictest94517-21kworker/0:1+events12:01:350
1684372270,7sleep375-21kworker/3:1+mm_percpu_wq11:56:453
41414272260,10sleep375-21kworker/3:1+mm_percpu_wq08:16:343
4073239992623,2cyclictest188405-21unixbench_multi12:16:462
4073239992622,3cyclictest4073519-21aten2_r5power_c07:11:282
4073239992622,2cyclictest119597-21aten2_r5power_e11:11:272
4073239992620,4cyclictest99100-21grep10:46:462
4073239992620,4cyclictest4149183-21unixbench_multi08:21:472
4073239992620,4cyclictest4149183-21unixbench_multi08:21:472
4073234992622,2cyclictest880-21mta-sts-daemon12:05:001
4073234992620,4cyclictest4104926-21apt-key07:40:011
4073234992620,4cyclictest193738-21systemd-run12:25:001
4073234992620,4cyclictest193738-21systemd-run12:25:001
4073234992620,3cyclictest0-21swapper/109:31:451
4073234992618,6cyclictest4144943-21apt-get08:21:271
4073234992618,6cyclictest4144943-21apt-get08:21:271
4073234992615,8cyclictest143842-21cron11:35:001
407322999269,16cyclictest94517-21kworker/0:1+events11:35:000
4073229992623,2cyclictest4089061-21needreboot07:21:390
4073229992615,9cyclictest4094119-21ntpq07:26:410
4073229992610,14cyclictest94517-21kworker/0:1+events11:26:340
4073239992521,2cyclictest4135163-21aten2_r5power_v08:11:282
4073239992520,3cyclictest40300-21aten2_r5power_p09:51:312
4073239992520,3cyclictest0-21swapper/208:40:002
4073239992519,5cyclictest208776-21apt12:40:012
407323499252,16cyclictest589-21runrttasks10:56:281
4073234992520,3cyclictest179972-21irqrtprio12:11:331
4073234992519,4cyclictest4169025-21unixbench_singl08:41:471
4073234992519,4cyclictest148484-21unixbench_multi11:36:471
4073234992519,3cyclictest0-21swapper/112:36:271
4073234992519,3cyclictest0-21swapper/110:01:271
4073234992519,2cyclictest0-21swapper/111:46:441
4073234992517,5cyclictest4159400-21cron08:35:001
4073234992515,8cyclictest4188242-21ntpq09:01:381
407322999257,16cyclictest4063288-21kworker/0:0+events09:16:460
407322999256,17cyclictest4063288-21kworker/0:0+events09:26:280
4073229992520,3cyclictest34658-21cron09:45:000
4073229992515,8cyclictest0-21swapper/008:40:010
4073229992515,7cyclictest64569-21latency_hist10:15:000
4073239992421,2cyclictest15294-21aten2_r5power_e09:26:292
4073234992419,3cyclictest95082-21cut10:46:291
4073234992419,3cyclictest4160489-21irqrtprio08:36:341
4073234992419,3cyclictest0-21swapper/109:46:391
4073234992418,4cyclictest92826-21kthreadcore10:41:361
4073234992418,4cyclictest34422-21kworker/u8:1+events_unbound11:10:001
4073234992418,3cyclictest0-21swapper/110:17:121
4073234992416,5cyclictest0-21swapper/109:16:461
407322999248,15cyclictest4063288-21kworker/0:0+events10:21:460
407322999248,15cyclictest4063288-21kworker/0:0+events10:21:460
407322999248,14cyclictest4063288-21kworker/0:0+events07:31:460
407322999247,15cyclictest94517-21kworker/0:1+events11:51:400
407322999247,15cyclictest4063288-21kworker/0:0+events10:00:000
407322999246,16cyclictest4063288-21kworker/0:0+events08:46:280
4073229992420,3cyclictest84283-21unixbench_singl10:31:460
4073229992419,3cyclictest203995-21apt-key12:35:010
4073229992418,4cyclictest183526-21unixbench_singl12:11:460
4073229992418,4cyclictest0-21swapper/009:36:350
22932240,5sleep2371rcuc/209:11:352
407323999232,14cyclictest114607-21apt-get11:06:282
4073239992319,2cyclictest45260-21aten2_r5power_v09:56:302
4073239992319,2cyclictest4081344-21kthreadcore07:16:342
4073239992319,2cyclictest179310-21aten2_r5power_e12:11:282
4073239992319,2cyclictest0-21swapper/212:35:012
4073234992317,4cyclictest4159432-21kworker/u8:1-ext4-rsv-conversion08:55:001
4073234992317,3cyclictest0-21swapper/111:21:331
4073234992316,5cyclictest4073083-21kworker/u8:2+flush-8:007:51:441
4073234992314,7cyclictest43941-21ntpq09:51:411
4073234992313,7cyclictest0-21swapper/108:46:261
4073229992319,2cyclictest0-21swapper/009:46:400
4073229992317,4cyclictest4167338-21runrttasks08:41:360
4073229992312,9cyclictest280-21systemd-journal09:34:520
407323999222,14cyclictest89918-21apt-get10:41:282
4073239992219,2cyclictest4179896-21fschecks_time08:56:312
4073239992219,2cyclictest4164782-21aten2_r5power_e08:41:292
4073239992219,2cyclictest10544-21cpuspeed_turbos09:21:312
4073239992218,3cyclictest25403-21fschecks_time09:36:322
4073239992217,3cyclictest4144254-21unixbench_singl08:16:462
4073234992218,2cyclictest0-21swapper/108:11:271
4073234992217,2cyclictest0-21swapper/109:06:451
4073234992214,5cyclictest0-21swapper/110:26:291
4073234992214,5cyclictest0-21swapper/110:26:291
407322999227,13cyclictest4063288-21kworker/0:0+events07:16:380
407322999226,15cyclictest94517-21kworker/0:1+events11:47:360
407322999224,17cyclictest94517-21kworker/0:1+events12:06:400
4073229992217,3cyclictest0-21swapper/010:56:460
40684462224,12sleep10-21swapper/107:05:301
407323999216,3cyclictest41775142kthreadcore08:51:362
4073239992118,2cyclictest178593-21unixbench_singl12:06:472
4073239992117,2cyclictest154440-21aten2_r5power_p11:46:282
4073239992117,2cyclictest0-21swapper/209:20:002
4073239992116,3cyclictest64211-21unixbench_multi10:11:462
4073239992112,7cyclictest23187-21tr09:31:352
4073234992115,4cyclictest0-21swapper/109:36:301
4073229992117,2cyclictest0-21swapper/008:56:310
407323999202,12cyclictest4149899-21apt-get08:26:282
407323999202,12cyclictest4110322-21apt-get07:46:272
4073239992017,2cyclictest129696-21fschecks_count11:21:302
4073239992016,3cyclictest153803-21systemd-run11:45:002
4073239992016,2cyclictest0-21swapper/210:35:002
4073239992016,2cyclictest0-21swapper/207:31:442
4073239992015,4cyclictest4173278-21missed_timers08:46:372
4073239992015,4cyclictest113941-21unixbench_singl11:01:462
4073239992012,6cyclictest89528-21cron10:40:002
4073239992011,6cyclictest143097-21seq11:31:422
407322999205,13cyclictest4063288-21kworker/0:0+events07:46:400
4073229992016,2cyclictest0-21swapper/009:51:370
4073229992010,7cyclictest54883-21cat10:05:010
407323999192,12cyclictest144213-21apt-get11:36:282
4073239991914,4cyclictest4073083-21kworker/u8:2+flush-8:008:31:442
40722072192,10sleep24072239-21ntpq07:06:392
407323999182,12cyclictest4125183-21apt-get08:01:272
4073239991815,2cyclictest164280-21aten2_r5power_e11:56:282
4073239991815,2cyclictest0-21swapper/209:01:382
4073239991814,2cyclictest4130358-21cpuspeed_turbos08:06:292
4073239991814,2cyclictest0-21swapper/212:21:382
4073239991814,2cyclictest0-21swapper/212:21:382
4073239991814,2cyclictest0-21swapper/210:56:412
4073239991814,2cyclictest0-21swapper/210:05:012
4073239991814,2cyclictest0-21swapper/209:46:372
4073239991814,2cyclictest0-21swapper/209:06:362
4073239991810,6cyclictest57313-21tr10:06:352
4073239991713,2cyclictest0-21swapper/211:26:392
407323999171,11cyclictest64922-21apt-get10:16:272
41294742160,3sleep30-21swapper/308:01:453
4073239991613,2cyclictest4090657-21fschecks_time07:26:302
4073239991612,3cyclictest0-21swapper/210:21:472
4073239991612,3cyclictest0-21swapper/210:21:472
4073239991612,2cyclictest0-21swapper/207:40:132
4073239991611,3cyclictest0-21swapper/210:51:382
407323999161,12cyclictest4120757-21ps07:56:342
407322999169,5cyclictest4073280-21ls07:10:010
1189882169,4sleep30-21swapper/311:07:393
686092152,8sleep30-21swapper/310:16:383
41038032154,7sleep30-21swapper/307:36:383
2041742142,8sleep30-21swapper/312:36:273
895752132,6sleep30-21swapper/310:40:023
41743172132,6sleep30-21swapper/308:50:023
40914822133,6sleep30-21swapper/307:26:353
40740722132,7sleep30-21swapper/307:11:343
2030342133,6sleep30-21swapper/312:31:463
1241022132,6sleep30-21swapper/311:15:013
960192122,6sleep30-21swapper/310:46:363
56172123,6sleep30-21swapper/309:16:313
492592122,6sleep30-21swapper/309:56:483
419642122,6sleep30-21swapper/309:51:383
41893332123,6sleep30-21swapper/309:05:113
41846812122,6sleep30-21swapper/309:01:303
41808252122,6sleep30-21swapper/308:56:363
41689702122,6sleep30-21swapper/308:41:473
41628912122,6sleep30-21swapper/308:36:383
41557752122,6sleep30-21swapper/308:31:363
41464762122,6sleep30-21swapper/308:21:373
41464762122,6sleep30-21swapper/308:21:373
41347172122,5sleep30-21swapper/308:10:013
41247862122,6sleep30-21swapper/307:59:583
41099482122,5sleep30-21swapper/307:45:013
41056892122,6sleep30-21swapper/307:41:323
40952462122,6sleep30-21swapper/307:31:293
40842582122,6sleep30-21swapper/307:16:443
352112122,6sleep30-21swapper/309:46:123
289752123,5sleep30-21swapper/309:36:453
191732122,6sleep30-21swapper/309:26:463
1743082122,6sleep30-21swapper/312:06:223
1691802122,6sleep30-21swapper/312:01:283
1485272122,6sleep30-21swapper/311:36:483
1390782122,6sleep30-21swapper/311:30:023
1244472123,5sleep30-21swapper/311:16:253
1042202122,6sleep30-21swapper/310:53:223
103012122,6sleep30-21swapper/309:21:283
8872112,5sleep30-21swapper/309:11:333
849132112,6sleep30-21swapper/310:36:243
811162112,5sleep30-21swapper/310:31:353
744712112,5sleep30-21swapper/310:25:013
744712112,5sleep30-21swapper/310:25:013
741812112,6sleep30-21swapper/310:21:483
741812112,6sleep30-21swapper/310:21:483
599432112,5sleep30-21swapper/310:11:123
549442112,5sleep30-21swapper/310:05:023
504972112,5sleep30-21swapper/310:01:333
41696522112,5sleep30-21swapper/308:46:253
41497792112,6sleep30-21swapper/308:25:143
41301432112,5sleep30-21swapper/308:06:283
41155452113,5sleep30-21swapper/307:51:313
40853602112,5sleep30-21swapper/307:21:293
206042112,5sleep30-21swapper/309:31:313
1974682112,6sleep30-21swapper/312:26:363
1891132112,5sleep30-21swapper/312:21:263
1891132112,5sleep30-21swapper/312:21:263
1841872112,5sleep30-21swapper/312:16:293
1795072112,5sleep30-21swapper/312:11:313
1608142112,5sleep30-21swapper/311:51:373
1547002112,5sleep30-21swapper/311:46:313
1296812112,5sleep30-21swapper/311:21:313
1141912112,6sleep30-21swapper/311:04:253
1049572112,5sleep30-21swapper/310:56:313
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional