You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-17 - 16:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Mon Aug 17, 2026 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
297079623710,3sleep3575-21kworker/3:2-events07:47:413
293355599207201,4cyclictest3183637-21apt-get11:20:042
322442521920,2sleep3575-21kworker/3:2+events12:02:423
293355099119114,3cyclictest0-21swapper/111:20:031
29335459910994,13cyclictest0-21swapper/011:20:030
32577802880,1chrt3257781-21/usr/sbin/munin12:32:571
29384582700,4sleep00-21swapper/007:15:000
2933545996153,5cyclictest3074612-21latency_hist09:30:010
2933550995646,7cyclictest2970337-21latency_hist07:45:011
2933550995550,3cyclictest2901704-21kworker/u8:0+flush-8:007:15:051
2933550995546,6cyclictest3228628-21latency_hist12:05:001
30111272540,4sleep00-21swapper/008:27:470
31148292530,3chrt3114834-21cpuspeed_turbos10:12:422
31148292530,3chrt3114834-21cpuspeed_turbos10:12:422
2933550995137,9cyclictest275-21systemd-journal09:18:061
2933545995141,8cyclictest2945998-21/usr/sbin/munin07:22:570
2933545994944,4cyclictest3149260-21kworker/u8:1+events_unbound10:57:420
2933545994940,7cyclictest3118346-21ntpq10:12:510
2933545994940,7cyclictest3118346-21ntpq10:12:510
31499152480,2sleep3451irq_work/310:47:463
2933550994739,6cyclictest3208147-21ntpq11:42:501
2933550994738,7cyclictest275-21systemd-journal08:42:471
2933545994741,4cyclictest2928507-21kworker/u8:3+flush-8:007:57:440
2933545994739,6cyclictest3183639-21latency_hist11:20:010
29461832440,5sleep30-21swapper/307:22:413
2933545994435,7cyclictest3119541-21/usr/sbin/munin10:17:530
2933545994435,7cyclictest3119541-21/usr/sbin/munin10:17:530
2933545994435,6cyclictest3020019-21latency_hist08:35:010
2933550994336,5cyclictest3178700-21latency_hist11:15:011
2933545994233,6cyclictest3238493-21latency_hist12:15:000
32121022410,6sleep20-21swapper/211:47:482
2933550994135,5cyclictest3010276-21kworker/u8:0+flush-8:010:27:531
2933550994133,5cyclictest3094347-21latency_hist09:50:011
2933545994131,8cyclictest3055522-21diskstats09:12:410
2933550994034,4cyclictest3059760-21latency_hist09:15:011
2933550994033,5cyclictest3234009-21apt-get12:12:391
2933550994033,5cyclictest3060020-21kworker/u8:3+flush-8:010:58:001
2933550994029,8cyclictest275-21systemd-journal08:03:061
2933545994034,4cyclictest2955823-21kworker/u8:1+flush-8:008:07:490
2933555993935,3cyclictest2955823-21kworker/u8:1+events_unbound08:58:012
2933555993935,3cyclictest2955823-21kworker/u8:1+events_unbound08:25:022
2933550993829,7cyclictest3005531-21/usr/sbin/munin08:22:571
2933555993733,3cyclictest2037181-21kworker/u8:2+events_unbound07:25:032
2933545993730,5cyclictest3149476-21apt-get10:47:400
293354599372,30cyclictest3010956-21kworker/0:3+events08:35:340
30681982360,6sleep00-21swapper/009:22:490
2933550993630,4cyclictest3010276-21kworker/u8:0+flush-8:009:40:001
2933550993627,7cyclictest2965391-21latency_hist07:40:021
2933545993629,5cyclictest2975248-21latency_hist07:50:010
2933545993628,6cyclictest3243852-21/usr/sbin/munin12:22:570
293354599362,32cyclictest3248400-21kworker/0:2+events12:29:360
293354599362,30cyclictest3010956-21kworker/0:3+events08:51:480
293354599362,29cyclictest3194062-21kworker/0:0+events11:51:580
293354599362,29cyclictest3168828-21kworker/0:1+events11:06:300
293354599362,28cyclictest3010956-21kworker/0:3+events08:45:220
293354599361,30cyclictest3194062-21kworker/0:0+events11:45:340
293354599361,30cyclictest2923624-21kworker/0:0+events08:02:280
293354599360,30cyclictest3248400-21kworker/0:2+events12:35:340
29322632360,30sleep0201rcuc/007:07:490
2933550993527,6cyclictest275-21systemd-journal12:15:071
293354599352,29cyclictest3010956-21kworker/0:3+events09:00:380
293354599351,32cyclictest3010956-21kworker/0:3+events08:57:280
2933555993430,3cyclictest3060020-21kworker/u8:3+events_unbound10:43:012
2933555993424,3cyclictest2928507-21kworker/u8:3+events_unbound07:33:022
2933550993427,5cyclictest3248365-21latency_hist12:25:011
2933550993425,7cyclictest3134147-21kworker/u8:2+events_unbound11:27:511
2933550993425,7cyclictest2994285-21ntpq08:07:531
2933545993429,3cyclictest3144070-21turbostat.cron10:40:000
293354599342,30cyclictest3248400-21kworker/0:2+events12:33:460
293354599342,26cyclictest3010956-21kworker/0:3+events10:42:060
293354599342,26cyclictest2923624-21kworker/0:0+events08:10:140
293354599341,31cyclictest3010956-21kworker/0:3+events10:08:040
293354599341,29cyclictest3010956-21kworker/0:3+events10:20:200
293354599341,28cyclictest3168828-21kworker/0:1+events11:31:310
293354599341,28cyclictest3010956-21kworker/0:3+events08:40:280
293354599341,27cyclictest2923624-21kworker/0:0+events08:15:080
293354599341,27cyclictest2923624-21kworker/0:0+events08:15:080
30145402330,2sleep3575-21kworker/3:2+events08:27:583
2933550993328,4cyclictest3188880-21kworker/u8:0+events_unbound11:50:001
2933550993327,5cyclictest2955823-21kworker/u8:1+flush-8:008:12:531
2933550993327,4cyclictest3134367-21apt-get10:32:381
2933545993327,4cyclictest3183408-21kworker/u8:3+flush-8:011:57:430
293354599332,30cyclictest3168828-21kworker/0:1+events11:26:200
293354599332,30cyclictest3010956-21kworker/0:3+events10:28:140
293354599332,29cyclictest3194062-21kworker/0:0+events11:38:160
2933545993322,9cyclictest275-21systemd-journal09:10:000
293354599332,26cyclictest3168828-21kworker/0:1+events11:10:140
293354599331,31cyclictest3010956-21kworker/0:3+events09:48:540
293354599331,26cyclictest3010956-21kworker/0:3+events09:50:140
293354599330,31cyclictest3010956-21kworker/0:3+events10:54:340
2933550993227,3cyclictest3140281-21irqrtprio10:37:481
2933550993224,6cyclictest2955861-21/usr/sbin/munin07:32:541
293354599329,21cyclictest3194062-21kworker/0:0+events12:07:280
293354599329,21cyclictest3194062-21kworker/0:0+events12:07:280
293354599322,29cyclictest2923624-21kworker/0:0+events07:44:580
293354599322,26cyclictest3194062-21kworker/0:0+events11:40:040
293354599321,30cyclictest3010956-21kworker/0:3+events09:44:440
293354599321,30cyclictest3010956-21kworker/0:3+events09:36:060
293354599321,30cyclictest3010956-21kworker/0:3+events09:30:360
293354599321,30cyclictest2923624-21kworker/0:0+events07:29:500
293354599321,29cyclictest2923624-21kworker/0:0+events08:20:460
293354599321,29cyclictest2923624-21kworker/0:0+events07:38:480
293354599321,27cyclictest2923624-21kworker/0:0+events07:50:280
2933550993127,3cyclictest2955823-21kworker/u8:1+flush-8:008:55:371
2933550993125,5cyclictest3010276-21kworker/u8:0+flush-8:010:22:581
2933550993125,4cyclictest2037181-21kworker/u8:2+events_unbound08:37:551
2933550993124,5cyclictest3060020-21kworker/u8:3+flush-8:009:52:591
2933550993124,5cyclictest2928507-21kworker/u8:3+flush-8:008:17:581
2933550993124,5cyclictest2928507-21kworker/u8:3+flush-8:008:17:581
2933545993127,3cyclictest3060326-21aten2_r5power_v09:17:420
293354599312,27cyclictest3010956-21kworker/0:3+events09:57:280
293354599312,24cyclictest3010956-21kworker/0:3+events10:00:020
293354599311,29cyclictest2923624-21kworker/0:0+events07:32:220
293354599311,26cyclictest3010956-21kworker/0:3+events11:00:180
29289252315,19sleep10-21swapper/107:07:391
293355599303,12cyclictest3060449-21seq09:17:432
2933555993026,3cyclictest3243657-21kworker/u8:2+events_unbound12:35:292
2933550993026,3cyclictest3150196-21irqrtprio10:47:481
2933550993024,4cyclictest2938859-21latency_hist07:15:011
2933550993022,6cyclictest3050332-21apt-get09:07:381
2933545993025,3cyclictest8450irq/124-ahci[0000:00:12.0]07:15:050
293354599302,23cyclictest3010956-21kworker/0:3+events10:30:120
2933555992925,3cyclictest2928507-21kworker/u8:3+events_unbound07:17:092
2933555992923,4cyclictest3049634-21unixbench_multi09:02:582
2933555992914,3cyclictest161rcu_preempt07:52:462
2933550992924,3cyclictest3258995-21cpuspeed_turbos12:37:411
2933550992924,3cyclictest2037181-21kworker/u8:2+events_unbound09:32:431
2933550992923,4cyclictest3166451-21grep11:02:471
2933550992922,5cyclictest3109222-21latency_hist10:05:011
2933550992922,5cyclictest2928507-21kworker/u8:3+events_unbound07:57:481
2933550992921,6cyclictest3232863-21ntpq12:07:541
2933550992921,6cyclictest3232863-21ntpq12:07:541
293354599292,26cyclictest3194062-21kworker/0:0+events12:01:060
293354599290,28cyclictest3194062-21kworker/0:0+events12:17:280
2933555992824,2cyclictest3203111-21needreboot11:37:502
2933550992822,4cyclictest3188880-21kworker/u8:0+events_unbound11:37:581
2933550992820,5cyclictest2980207-21latency_hist07:55:011
293355599275,10cyclictest39-21ksoftirqd/212:15:002
2933555992723,2cyclictest3239004-21aten2_r5power_e12:17:402
2933555992722,3cyclictest3248844-21aten2_r5power_c12:27:402
2933555992713,4cyclictest161rcu_preempt08:33:002
293355599266,8cyclictest3168189-21cut11:02:552
293355599265,12cyclictest0-21swapper/207:47:442
293355599262,6cyclictest0-21swapper/211:11:432
2933555992622,3cyclictest3010276-21kworker/u8:0+events_unbound09:51:132
2933555992622,2cyclictest3228353-21unixbench_multi12:02:572
2933555992622,2cyclictest3000687-21aten2_r5power_p08:17:402
2933555992622,2cyclictest3000687-21aten2_r5power_p08:17:402
2933555992621,4cyclictest3030947-21sed08:47:472
2933555992618,2cyclictest161rcu_preempt08:38:022
293355599260,9cyclictest31327792kthreadcore10:27:492
2933550992622,3cyclictest2037181-21kworker/u8:2+flush-8:009:21:141
2933550992622,2cyclictest0-21swapper/110:07:551
2933550992621,4cyclictest3010276-21kworker/u8:0+flush-8:010:18:021
2933550992621,4cyclictest3010276-21kworker/u8:0+flush-8:010:18:011
2933550992621,3cyclictest3219472-21cut11:57:431
2933550992621,3cyclictest1-21systemd08:50:011
2933550992621,3cyclictest0-21swapper/109:57:411
2933550992620,3cyclictest0-21swapper/110:42:441
2933550992619,4cyclictest275-21systemd-journal08:27:441
2933555992521,3cyclictest3138657-21grep10:32:582
293355599250,10cyclictest3006357-21cut08:22:472
2933550992521,3cyclictest3134147-21kworker/u8:2+flush-8:011:18:491
2933550992521,3cyclictest2946248-21cpuspeed_turbos07:22:421
2933550992520,4cyclictest2955823-21kworker/u8:1+flush-8:008:32:441
2933550992519,4cyclictest3243657-21kworker/u8:2+flush-8:012:27:461
2933550992519,4cyclictest3049629-21unixbench_multi09:02:581
2933550992518,5cyclictest3194277-21kworker/u8:1+events_unbound11:32:581
2933550992518,5cyclictest2037181-21kworker/u8:2+events_unbound09:27:421
293355599246,4cyclictest39-21ksoftirqd/212:35:012
293355599243,9cyclictest3054977-21sort09:10:012
293355599242,3cyclictest0-21swapper/210:18:022
293355599242,3cyclictest0-21swapper/210:18:022
2933555992420,3cyclictest3194277-21kworker/u8:1+flush-8:011:31:372
2933555992420,2cyclictest3229134-21aten2_r5power_e12:07:412
2933555992420,2cyclictest3229134-21aten2_r5power_e12:07:412
2933555992419,3cyclictest3025731-21fschecks_time08:42:432
293355599240,12cyclictest3056541-21cut09:12:462
293355099242,16cyclictest3084955-21chrt09:42:401
2933550992420,2cyclictest3039752-21cron08:55:001
2933550992420,2cyclictest0-21swapper/110:52:441
2933550992419,3cyclictest3218553-21unixbench_singl11:52:591
2933550992416,6cyclictest2970788-21apt-get07:47:391
293355599237,6cyclictest39-21ksoftirqd/212:25:012
293355599237,2cyclictest29380052switchtime07:12:592
293355599234,10cyclictest0-21swapper/207:37:482
293355599232,5cyclictest0-21swapper/210:37:492
293355599232,5cyclictest0-21swapper/209:42:492
2933555992315,2cyclictest161rcu_preempt09:22:402
293355599230,8cyclictest3035609-21grep08:52:452
293355599230,6cyclictest3204404-21apt-get11:42:392
2933550992317,4cyclictest3114603-21apt-get10:12:391
2933550992317,4cyclictest3114603-21apt-get10:12:391
2933550992316,4cyclictest0-21swapper/111:07:461
293355599228,6cyclictest421rcuog/210:01:042
293355599225,2cyclictest0-21swapper/207:57:552
293355599222,6cyclictest0-21swapper/209:30:392
2933555992219,2cyclictest2966203-21fschecks_time07:42:432
2933555992217,3cyclictest3159585-21cpuspeed_turbos10:57:422
2933555992212,2cyclictest161rcu_preempt08:13:302
2933555992211,2cyclictest161rcu_preempt09:27:392
293355599220,9cyclictest39-21ksoftirqd/209:49:092
293355599220,9cyclictest3218036-21awk11:52:532
293355599220,8cyclictest2992973-21kthreadcore08:07:492
2933550992218,2cyclictest0-21swapper/107:27:451
293355599213,4cyclictest0-21swapper/209:37:462
293355599212,7cyclictest39-21ksoftirqd/210:10:002
293355599212,4cyclictest0-21swapper/209:57:502
293355599212,4cyclictest0-21swapper/208:02:412
293355599212,11cyclictest39-21ksoftirqd/210:50:012
2933555992117,3cyclictest3149260-21kworker/u8:1+flush-8:011:08:222
2933555992110,2cyclictest161rcu_preempt07:22:212
293355599210,9cyclictest3220156-21cut11:57:472
293355599210,12cyclictest3155294-21irqrtprio10:52:482
29332002214,10sleep20-21swapper/207:08:012
293355599202,5cyclictest0-21swapper/210:22:482
2933555992016,2cyclictest0-21swapper/211:25:012
2933555992010,2cyclictest161rcu_preempt11:15:072
303406121910,5sleep30-21swapper/308:47:533
29289492183,8sleep30-21swapper/307:07:413
32083942142,8sleep30-21swapper/311:42:573
31665432143,7sleep30-21swapper/311:02:483
32229522133,6sleep30-21swapper/311:57:533
32187432132,6sleep30-21swapper/311:55:003
32031372132,7sleep30-21swapper/311:37:513
31589322132,6sleep30-21swapper/310:55:013
31431102132,7sleep30-21swapper/310:37:523
31035252132,7sleep30-21swapper/309:57:513
30846882132,6sleep30-21swapper/309:40:023
32580482122,6sleep30-21swapper/312:33:003
32483872122,6sleep30-21swapper/312:25:023
32476442122,7sleep30-21swapper/312:22:533
32340422121,7sleep30-21swapper/312:12:423
32098852122,6sleep30-21swapper/311:47:463
31983952122,6sleep30-21swapper/311:32:573
31895352122,6sleep30-21swapper/311:27:403
31842302122,6sleep30-21swapper/311:21:343
31790712122,6sleep30-21swapper/311:17:003
31693332122,6sleep30-21swapper/311:07:413
31582792122,6sleep30-21swapper/310:52:573
31450292122,6sleep30-21swapper/310:42:463
31339312122,6sleep30-21swapper/310:30:023
31236212122,6sleep30-21swapper/310:18:023
31236212122,6sleep30-21swapper/310:18:023
30943602122,5sleep30-21swapper/309:50:013
30898932122,6sleep30-21swapper/309:47:443
30800002122,6sleep30-21swapper/309:37:413
30753062122,6sleep30-21swapper/309:32:433
30696892122,5sleep30-21swapper/309:25:013
30598872122,5sleep30-21swapper/309:15:023
30548032122,6sleep30-21swapper/309:10:003
30443162122,6sleep30-21swapper/308:57:563
30391522122,6sleep30-21swapper/308:52:593
30256662122,6sleep30-21swapper/308:42:443
30207072122,6sleep30-21swapper/308:37:423
30072982122,6sleep30-21swapper/308:22:493
30044562122,6sleep30-21swapper/308:17:563
30044562122,6sleep30-21swapper/308:17:563
29954622122,6sleep30-21swapper/308:12:423
29853622122,6sleep30-21swapper/308:00:063
29809812122,6sleep30-21swapper/307:57:453
29765832122,6sleep30-21swapper/307:52:483
29563362122,6sleep30-21swapper/307:32:463
29544852122,6sleep30-21swapper/307:27:523
32630992112,5sleep30-21swapper/312:38:063
32398422112,6sleep30-21swapper/312:17:483
32289632113,5sleep30-21swapper/312:06:323
32289632113,5sleep30-21swapper/312:06:323
31739852112,5sleep30-21swapper/311:10:023
31292172113,5sleep30-21swapper/310:25:123
31242732112,6sleep30-21swapper/310:20:063
31144342112,5sleep30-21swapper/310:10:473
31144342112,5sleep30-21swapper/310:10:473
31094402112,5sleep30-21swapper/310:05:023
31085722112,5sleep30-21swapper/310:02:573
30650062112,5sleep30-21swapper/309:20:363
30552442112,6sleep30-21swapper/309:12:373
30453452112,6sleep30-21swapper/309:02:183
29903712112,6sleep30-21swapper/308:06:043
29656812112,6sleep30-21swapper/307:40:483
29606332112,5sleep30-21swapper/307:35:063
29410732112,5sleep30-21swapper/307:17:413
29336662112,5sleep30-21swapper/307:10:023
30152772102,5sleep30-21swapper/308:30:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional