You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-26 - 15:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Sun Jul 26, 2026 12:44:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
226877521240,2sleep30-21swapper/307:15:033
234360921080,3sleep00-21swapper/010:45:120
226642628040,14sleep10-21swapper/107:09:081
226630327740,14sleep00-21swapper/007:07:340
226590026645,14sleep20-21swapper/207:05:262
226622926341,15sleep30-21swapper/307:06:373
23120542510,3sleep00-21swapper/009:15:130
23048852490,3sleep03056-21PK-Backend08:55:160
23292542470,4sleep33056-21PK-Backend10:05:133
23630062420,4chrt0-21swapper/011:40:170
23565072420,4sleep10-21swapper/111:20:211
226666099367,5cyclictest13-21ksoftirqd/007:15:010
226666099346,7cyclictest13-21ksoftirqd/011:35:000
2266668993314,10cyclictest42-21ksoftirqd/311:45:003
2266668993213,4cyclictest42-21ksoftirqd/309:59:303
2266660993211,10cyclictest13-21ksoftirqd/008:45:010
2266668993114,5cyclictest42-21ksoftirqd/311:55:013
2266668993110,9cyclictest42-21ksoftirqd/308:10:023
226666099318,11cyclictest13-21ksoftirqd/009:05:010
226666099312,6cyclictest13-21ksoftirqd/011:20:000
2266660993111,5cyclictest13-21ksoftirqd/012:00:000
2266660993011,5cyclictest13-21ksoftirqd/008:00:000
226666899291,2cyclictest141rcu_preempt09:15:023
226666899286,6cyclictest42-21ksoftirqd/310:40:003
226666899283,4cyclictest42-21ksoftirqd/311:40:003
2266668992823,3cyclictest411rcuc/310:15:003
2266668992813,4cyclictest42-21ksoftirqd/309:19:443
2266668992813,4cyclictest42-21ksoftirqd/308:44:363
226666099287,10cyclictest13-21ksoftirqd/009:15:020
226666099285,7cyclictest13-21ksoftirqd/010:40:170
226666099285,11cyclictest13-21ksoftirqd/008:15:010
226666899274,2cyclictest141rcu_preempt10:35:013
226666899273,3cyclictest42-21ksoftirqd/307:50:013
2266668992713,4cyclictest42-21ksoftirqd/311:20:193
226666099277,10cyclictest13-21ksoftirqd/011:30:000
226666099276,10cyclictest13-21ksoftirqd/008:20:000
226666099275,11cyclictest13-21ksoftirqd/007:35:130
226666099271,7cyclictest13-21ksoftirqd/008:35:000
226666099270,7cyclictest13-21ksoftirqd/011:00:010
226666899268,5cyclictest42-21ksoftirqd/310:45:003
226666899263,3cyclictest42-21ksoftirqd/312:05:003
226666899263,2cyclictest141rcu_preempt12:25:003
2266668992611,4cyclictest42-21ksoftirqd/312:16:373
2266668992611,4cyclictest42-21ksoftirqd/311:25:123
2266668992610,4cyclictest42-21ksoftirqd/310:45:213
226666299267,5cyclictest141rcu_preempt10:14:591
226666099268,5cyclictest13-21ksoftirqd/010:40:000
226666099264,2cyclictest141rcu_preempt12:15:010
226666099263,4cyclictest13-21ksoftirqd/011:25:010
226666099262,7cyclictest13-21ksoftirqd/007:20:010
2266660992610,5cyclictest13-21ksoftirqd/007:25:000
226666099260,6cyclictest13-21ksoftirqd/010:14:590
226666099260,6cyclictest13-21ksoftirqd/009:45:000
226666899256,8cyclictest42-21ksoftirqd/311:55:023
226666899253,2cyclictest141rcu_preempt08:15:013
226666899252,6cyclictest42-21ksoftirqd/310:24:593
2266668992521,3cyclictest411rcuc/307:25:213
2266668992510,4cyclictest42-21ksoftirqd/308:24:463
2266668992510,3cyclictest42-21ksoftirqd/309:40:543
226666299259,4cyclictest26-21ksoftirqd/111:05:001
226666099250,8cyclictest13-21ksoftirqd/008:50:210
226666899248,2cyclictest141rcu_preempt12:28:353
226666899248,14cyclictest2282755-21unin-run07:55:013
226666899247,3cyclictest141rcu_preempt08:54:053
226666899246,3cyclictest141rcu_preempt10:03:563
226666899243,3cyclictest42-21ksoftirqd/312:35:003
226666899243,2cyclictest141rcu_preempt12:15:013
226666899242,4cyclictest42-21ksoftirqd/307:15:013
226666899242,2cyclictest141rcu_preempt09:35:013
226666299249,4cyclictest26-21ksoftirqd/109:50:001
226666299249,3cyclictest141rcu_preempt11:00:001
2266662992414,2cyclictest2374788-21turbostat12:14:591
226666299241,21cyclictest2293913-21apt08:25:121
226666099245,7cyclictest13-21ksoftirqd/007:33:030
226666099244,9cyclictest13-21ksoftirqd/007:25:180
226666099244,6cyclictest13-21ksoftirqd/008:35:200
226666099243,6cyclictest13-21ksoftirqd/012:35:000
226666099243,6cyclictest13-21ksoftirqd/010:20:130
226666099242,3cyclictest13-21ksoftirqd/012:20:000
226666099242,12cyclictest13-21ksoftirqd/011:35:130
226666899239,4cyclictest42-21ksoftirqd/312:10:003
226666899239,4cyclictest42-21ksoftirqd/312:10:003
226666899237,8cyclictest42-21ksoftirqd/307:40:183
226666899236,6cyclictest3056-21PK-Backend11:12:093
226666899234,3cyclictest141rcu_preempt08:47:093
226666899233,1cyclictest141rcu_preempt10:55:023
226666899232,9cyclictest42-21ksoftirqd/309:00:133
2266668992319,3cyclictest411rcuc/307:35:203
2266668992319,3cyclictest411rcuc/307:20:203
2266668992311,3cyclictest141rcu_preempt11:01:463
226666899230,3cyclictest42-21ksoftirqd/309:50:003
2266665992315,6cyclictest2321946-21cat09:45:012
226666099236,15cyclictest2339229-21cron10:35:000
226666099234,8cyclictest13-21ksoftirqd/010:50:190
226666099234,7cyclictest13-21ksoftirqd/010:00:160
226666099232,8cyclictest13-21ksoftirqd/011:45:130
226666099232,6cyclictest13-21ksoftirqd/012:25:000
2266660992321,1cyclictest0-21swapper/008:06:010
2266660992319,3cyclictest161rcuc/009:55:000
226666099231,8cyclictest13-21ksoftirqd/012:05:200
226666099231,8cyclictest13-21ksoftirqd/012:05:200
2266660992315,4cyclictest2334027-21dump-pmu-power10:19:590
2266660992315,4cyclictest2334027-21dump-pmu-power10:19:590
139099230,15rtkit-daemon0-21swapper/007:43:440
226666899226,1cyclictest141rcu_preempt09:24:593
226666899223,6cyclictest42-21ksoftirqd/312:35:223
226666899223,4cyclictest42-21ksoftirqd/308:20:013
226666899223,1cyclictest141rcu_preempt09:55:003
226666899222,2cyclictest141rcu_preempt09:40:013
2266668992219,2cyclictest411rcuc/311:50:003
2266668992218,3cyclictest411rcuc/308:05:003
226666899221,10cyclictest42-21ksoftirqd/309:05:413
2266668992210,3cyclictest42-21ksoftirqd/310:15:223
2266668992210,3cyclictest42-21ksoftirqd/310:15:223
2266665992220,1cyclictest0-21swapper/210:25:492
2266665992220,1cyclictest0-21swapper/207:59:012
226666299222,2cyclictest141rcu_preempt11:40:011
2266662992217,4cyclictest26-21ksoftirqd/109:25:001
2266662992213,3cyclictest26-21ksoftirqd/108:50:181
2266662992212,3cyclictest26-21ksoftirqd/109:45:001
2266662992211,4cyclictest26-21ksoftirqd/111:15:001
226666099225,4cyclictest13-21ksoftirqd/012:35:170
226666099224,2cyclictest2291656-21munin-run08:20:000
226666099222,8cyclictest13-21ksoftirqd/011:51:210
226666099222,7cyclictest13-21ksoftirqd/008:05:000
226666099222,6cyclictest13-21ksoftirqd/007:50:200
2266660992219,2cyclictest161rcuc/008:46:160
226666099220,21cyclictest0-21swapper/009:25:300
23529072210,2sleep20-21swapper/211:10:232
22804512210,2sleep13056-21PK-Backend07:45:191
226666899215,6cyclictest2351454-21chrt11:07:483
2266668992114,5cyclictest2355116-21apt-get11:19:593
2266668992110,3cyclictest42-21ksoftirqd/309:26:313
226666899210,20cyclictest2360604-21systemd-run11:35:003
226666899210,19cyclictest3056-21PK-Backend08:32:023
2266665992119,1cyclictest0-21swapper/212:19:192
2266665992119,1cyclictest0-21swapper/212:03:102
2266665992119,1cyclictest0-21swapper/211:58:302
2266665992119,1cyclictest0-21swapper/211:41:332
2266665992119,1cyclictest0-21swapper/210:30:432
2266665992119,1cyclictest0-21swapper/210:10:352
2266665992119,1cyclictest0-21swapper/210:01:562
2266665992119,1cyclictest0-21swapper/209:23:102
2266665992119,1cyclictest0-21swapper/209:15:062
2266665992119,1cyclictest0-21swapper/209:01:402
2266665992119,1cyclictest0-21swapper/208:50:392
2266665992119,1cyclictest0-21swapper/208:23:492
226666599210,3cyclictest2367294-21taskset11:52:282
226666599210,20cyclictest0-21swapper/212:34:532
226666599210,20cyclictest0-21swapper/212:22:032
226666599210,20cyclictest0-21swapper/210:20:552
226666599210,20cyclictest0-21swapper/209:07:412
226666599210,20cyclictest0-21swapper/208:07:452
226666599210,20cyclictest0-21swapper/207:26:022
226666299218,3cyclictest26-21ksoftirqd/109:33:431
226666299218,3cyclictest26-21ksoftirqd/109:15:121
226666299216,7cyclictest26-21ksoftirqd/111:45:161
226666299216,3cyclictest141rcu_preempt11:45:011
2266662992119,1cyclictest0-21swapper/112:27:531
226666299211,2cyclictest141rcu_preempt07:45:001
2266662992112,3cyclictest26-21ksoftirqd/110:25:011
226666299210,8cyclictest2300812-21cp08:45:001
226666099212,7cyclictest13-21ksoftirqd/009:55:160
226666099212,7cyclictest13-21ksoftirqd/009:30:190
226666099212,6cyclictest13-21ksoftirqd/012:00:130
2266660992119,1cyclictest0-21swapper/011:10:270
2266660992119,1cyclictest0-21swapper/009:07:180
2266660992117,2cyclictest161rcuc/009:20:540
2266660992115,4cyclictest2323617-21sh09:50:000
226666099211,3cyclictest13-21ksoftirqd/012:30:000
2266660992110,2cyclictest141rcu_preempt10:07:300
226666099210,7cyclictest2320150-21cron09:40:000
226666099210,20cyclictest0-21swapper/010:26:130
226666099210,20cyclictest0-21swapper/007:48:500
226666899206,6cyclictest2295255-21chrt08:28:403
226666899206,5cyclictest42-21ksoftirqd/308:59:073
226666899202,2cyclictest141rcu_preempt11:00:023
2266668992016,3cyclictest411rcuc/307:33:033
2266668992014,4cyclictest42-21ksoftirqd/308:00:003
226666899200,19cyclictest3056-21PK-Backend08:39:263
2266665992018,1cyclictest0-21swapper/211:01:182
2266665992018,1cyclictest0-21swapper/209:33:372
2266665992018,1cyclictest0-21swapper/208:59:032
2266665992018,1cyclictest0-21swapper/208:34:022
2266665992018,1cyclictest0-21swapper/208:27:132
2266665992018,1cyclictest0-21swapper/208:16:462
2266665992018,1cyclictest0-21swapper/207:53:202
2266665992013,6cyclictest2342174-21strings10:40:192
226666599200,1cyclictest0-21swapper/212:07:492
226666599200,1cyclictest0-21swapper/212:07:492
226666599200,1cyclictest0-21swapper/208:02:532
226666599200,19cyclictest3056-21PK-Backend11:23:482
226666599200,19cyclictest0-21swapper/211:15:512
226666599200,19cyclictest0-21swapper/210:49:412
226666599200,19cyclictest0-21swapper/209:58:452
226666599200,19cyclictest0-21swapper/209:26:542
226666599200,19cyclictest0-21swapper/209:12:082
226666599200,19cyclictest0-21swapper/208:37:262
226666599200,19cyclictest0-21swapper/208:13:402
226666599200,19cyclictest0-21swapper/207:17:512
226666299209,3cyclictest26-21ksoftirqd/110:15:231
226666299209,3cyclictest26-21ksoftirqd/110:15:221
226666299207,2cyclictest141rcu_preempt09:50:191
226666299206,6cyclictest26-21ksoftirqd/108:45:191
226666299206,2cyclictest141rcu_preempt08:39:011
2266662992018,1cyclictest0-21swapper/111:29:591
2266662992017,2cyclictest3056-21PK-Backend08:59:471
2266662992010,4cyclictest26-21ksoftirqd/110:50:011
2266662992010,3cyclictest26-21ksoftirqd/108:35:001
226666299200,8cyclictest2316691-21latency_hist09:30:011
226666299200,1cyclictest3056-21PK-Backend09:03:591
226666299200,19cyclictest3056-21PK-Backend07:39:161
226666299200,19cyclictest3056-21PK-Backend07:32:131
226666299200,19cyclictest0-21swapper/111:31:211
226666299200,19cyclictest0-21swapper/110:51:531
226666299200,19cyclictest0-21swapper/110:25:301
226666299200,19cyclictest0-21swapper/107:55:381
226666299200,19cyclictest0-21swapper/107:52:471
226666299200,18cyclictest3056-21PK-Backend11:50:131
2266660992018,1cyclictest0-21swapper/011:06:070
2266660992016,3cyclictest161rcuc/011:05:010
2266660992015,4cyclictest13-21ksoftirqd/008:30:000
2266665991917,1cyclictest0-21swapper/212:28:522
2266665991917,1cyclictest0-21swapper/212:11:542
2266665991917,1cyclictest0-21swapper/211:36:062
2266665991917,1cyclictest0-21swapper/211:06:252
2266665991917,1cyclictest0-21swapper/210:06:292
2266665991917,1cyclictest0-21swapper/209:45:412
2266665991916,2cyclictest0-21swapper/207:40:212
226666599190,1cyclictest0-21swapper/210:37:462
226666599190,1cyclictest0-21swapper/210:16:032
226666599190,1cyclictest0-21swapper/210:16:032
226666599190,1cyclictest0-21swapper/209:39:322
226666599190,1cyclictest0-21swapper/207:45:502
226666599190,1cyclictest0-21swapper/207:31:162
226666599190,1cyclictest0-21swapper/207:13:512
226666599190,18cyclictest0-21swapper/208:45:392
226666299199,2cyclictest141rcu_preempt11:17:101
226666299199,2cyclictest141rcu_preempt10:05:201
226666299199,2cyclictest141rcu_preempt09:12:181
226666299195,6cyclictest26-21ksoftirqd/110:01:321
2266662991917,1cyclictest0-21swapper/112:32:321
2266662991916,2cyclictest2375299-21cstates12:15:141
226666299190,1cyclictest3056-21PK-Backend08:11:391
226666299190,1cyclictest3056-21PK-Backend08:04:581
226666299190,1cyclictest3056-21PK-Backend07:26:591
226666299190,1cyclictest3056-21PK-Backend07:15:021
226666299190,1cyclictest3056-21PK-Backend07:13:261
226666299190,1cyclictest0-21swapper/111:08:571
226666299190,1cyclictest0-21swapper/109:55:091
226666299190,1cyclictest0-21swapper/109:35:011
226666299190,18cyclictest3056-21PK-Backend12:02:371
226666299190,18cyclictest0-21swapper/109:06:351
226666899180,5cyclictest3056-21PK-Backend10:29:253
2266665991816,1cyclictest0-21swapper/212:38:582
2266665991816,1cyclictest0-21swapper/211:30:392
2266665991816,1cyclictest0-21swapper/209:51:502
226666599180,1cyclictest0-21swapper/207:37:312
226666299189,2cyclictest141rcu_preempt10:30:031
226666299185,3cyclictest26-21ksoftirqd/110:35:111
226666299184,2cyclictest141rcu_preempt12:05:171
226666299184,2cyclictest141rcu_preempt12:05:171
226666299180,4cyclictest3056-21PK-Backend08:20:131
226666299180,1cyclictest3056-21PK-Backend08:16:331
2266665991715,1cyclictest0-21swapper/211:27:542
2266665991712,3cyclictest2364888-21missed_timers11:45:172
226666299176,2cyclictest141rcu_preempt08:10:011
226666299173,2cyclictest141rcu_preempt10:40:191
226666599169,5cyclictest2345711-21ntpq10:50:182
2266665991611,4cyclictest2299971-21seq08:40:212
226666599160,6cyclictest2348173-21sh11:00:002
226666299169,2cyclictest26-21ksoftirqd/107:20:221
226666299161,3cyclictest141rcu_preempt12:35:431
226666299161,3cyclictest141rcu_preempt12:22:441
226666299161,3cyclictest141rcu_preempt11:57:261
2266665991411,2cyclictest2270768-21/usr/sbin/munin07:20:192
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional