You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-10 - 19:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Sun May 10, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
57074721220,1sleep10-21swapper/107:34:551
62688121110,2sleep33056-21PK-Backend10:15:013
6031932930,3sleep20-21swapper/209:05:182
56124928364,13sleep30-21swapper/307:06:183
56153627839,14sleep00-21swapper/007:09:570
56098127848,20sleep20-21swapper/207:05:282
56127427338,13sleep10-21swapper/107:06:361
5924642560,1sleep10-21swapper/108:35:141
5817252440,2sleep10-21swapper/108:05:111
56170099367,10cyclictest597253-21latency_hist08:50:001
56170099347,8cyclictest621620-21latency_hist10:00:021
56170099338,8cyclictest1095-21runrttasks10:33:181
56170099326,14cyclictest650337-21/usr/sbin/munin11:20:131
56170099317,9cyclictest665513-21seq12:04:451
56170099307,9cyclictest628487-21sshd10:19:261
56170099307,10cyclictest568106-21latency07:25:171
56170099305,15cyclictest635810-21/usr/sbin/munin10:40:131
56170099305,14cyclictest619383-21cat09:50:191
56170099305,14cyclictest0-21swapper/111:05:131
5786102290,4sleep23056-21PK-Backend07:55:132
56170099298,11cyclictest605473-21taskset09:11:381
56170099298,11cyclictest605473-21taskset09:11:371
56170099297,9cyclictest591042-21missed_timers08:30:171
56170099297,13cyclictest0-21swapper/107:15:081
56170099296,8cyclictest584676-21taskset08:11:451
56170099296,11cyclictest667332-21sed12:10:011
56170099295,14cyclictest659946-21cat11:45:221
56170099295,14cyclictest0-21swapper/112:30:211
56170099295,10cyclictest587982-21timerwakeupswit08:20:221
6438572280,2sleep33056-21PK-Backend11:00:193
56170099287,8cyclictest658122-21timerandwakeup11:40:221
56170099285,13cyclictest603556-21timerwakeupswit09:05:231
56170099285,13cyclictest0-21swapper/112:15:011
56170099276,9cyclictest630756-21cat10:25:131
56170099275,8cyclictest0-21swapper/112:15:121
56170099275,13cyclictest0-21swapper/108:00:121
56170099274,8cyclictest0-21swapper/111:30:131
56170099274,8cyclictest0-21swapper/110:10:121
561700992723,3cyclictest0-21swapper/107:10:141
56169799273,22cyclictest0-21swapper/010:50:160
56170599268,5cyclictest34-21ksoftirqd/211:19:592
56170599264,7cyclictest34-21ksoftirqd/211:00:022
56170099266,6cyclictest614958-21apt09:40:111
56170099265,8cyclictest0-21swapper/108:40:181
56170099265,8cyclictest0-21swapper/108:25:131
56170099264,9cyclictest0-21swapper/107:20:121
56170099264,9cyclictest0-21swapper/107:20:121
6731452250,2sleep10-21swapper/112:25:141
56170099256,5cyclictest26-21ksoftirqd/110:35:131
56170099255,8cyclictest571092-21cpuspeed07:35:131
56170099254,9cyclictest640721-21sshd10:51:261
56170099254,8cyclictest0-21swapper/111:55:201
56170099254,8cyclictest0-21swapper/109:45:201
56170099253,9cyclictest0-21swapper/111:50:181
561707992420,2cyclictest42-21ksoftirqd/307:55:013
56170099246,8cyclictest0-21swapper/111:15:171
56170099245,8cyclictest0-21swapper/109:05:001
56170099245,7cyclictest0-21swapper/112:35:111
56170099244,8cyclictest0-21swapper/108:50:191
139099240,3rtkit-daemon1389-21rtkit-daemon12:04:430
139099240,3rtkit-daemon1389-21rtkit-daemon10:47:081
561707992318,3cyclictest613402-21tr09:35:133
561705992318,4cyclictest34-21ksoftirqd/207:45:012
561705992316,6cyclictest610946-21cron09:30:002
56170099234,7cyclictest0-21swapper/108:15:171
56170099233,8cyclictest0-21swapper/111:35:041
56170099233,8cyclictest0-21swapper/111:15:011
56170099233,8cyclictest0-21swapper/111:15:011
561707992210,4cyclictest3056-21PK-Backend08:54:453
56170599225,2cyclictest141rcu_preempt10:25:212
561705992220,1cyclictest0-21swapper/212:32:562
561705992220,1cyclictest0-21swapper/209:32:072
561705992220,1cyclictest0-21swapper/209:19:172
561705992218,3cyclictest34-21ksoftirqd/208:30:002
56170599220,21cyclictest0-21swapper/209:03:492
561700992215,5cyclictest622393-21cut10:00:171
561700992212,8cyclictest609247-21idleruntime-cro09:25:001
561700992212,8cyclictest577853-21apt-config07:55:011
561700992211,9cyclictest573219-21runrttasks07:40:141
561700992210,11cyclictest624029-21idleruntime10:05:161
561697992220,1cyclictest0-21swapper/012:06:530
561697992220,1cyclictest0-21swapper/010:07:230
561697992220,1cyclictest0-21swapper/008:23:400
561697992220,1cyclictest0-21swapper/008:13:470
561697992219,2cyclictest0-21swapper/012:19:380
56169799220,21cyclictest0-21swapper/008:39:290
561707992118,2cyclictest3056-21PK-Backend10:43:213
56170799210,20cyclictest3056-21PK-Backend12:27:113
56170799210,20cyclictest3056-21PK-Backend11:14:233
56170799210,20cyclictest3056-21PK-Backend11:14:233
56170799210,20cyclictest3056-21PK-Backend10:38:273
56170799210,20cyclictest3056-21PK-Backend09:04:113
56170799210,19cyclictest3056-21PK-Backend12:35:213
56170799210,19cyclictest3056-21PK-Backend11:21:413
56170799210,19cyclictest3056-21PK-Backend09:57:043
56170799210,19cyclictest3056-21PK-Backend07:41:593
56170799210,19cyclictest3056-21PK-Backend07:36:113
561705992119,1cyclictest0-21swapper/212:29:322
561705992119,1cyclictest0-21swapper/212:04:582
561705992119,1cyclictest0-21swapper/211:59:102
561705992119,1cyclictest0-21swapper/210:48:322
561705992119,1cyclictest0-21swapper/209:57:022
561705992119,1cyclictest0-21swapper/209:54:592
561705992119,1cyclictest0-21swapper/209:39:032
561705992119,1cyclictest0-21swapper/207:38:242
561705992113,6cyclictest575630-21awk07:45:222
561705992113,6cyclictest436-20systemd-journal07:55:012
56170599210,20cyclictest0-21swapper/211:45:102
56170599210,20cyclictest0-21swapper/211:14:212
56170599210,20cyclictest0-21swapper/211:14:212
56170599210,20cyclictest0-21swapper/208:54:162
56170599210,20cyclictest0-21swapper/208:40:022
56170599210,20cyclictest0-21swapper/207:11:542
56170099219,10cyclictest641654-21df_inode10:55:151
561700992112,8cyclictest610947-21cron09:30:001
561700992112,7cyclictest606886-21python309:15:201
56169799218,4cyclictest917-21in:imuxsock12:35:000
561697992119,1cyclictest3056-21PK-Backend10:25:340
561697992119,1cyclictest0-21swapper/011:35:370
561697992119,1cyclictest0-21swapper/010:31:290
561697992119,1cyclictest0-21swapper/010:19:270
561697992119,1cyclictest0-21swapper/009:37:490
561697992119,1cyclictest0-21swapper/009:05:040
561697992118,2cyclictest0-21swapper/012:29:240
56169799210,20cyclictest0-21swapper/011:51:330
56169799210,20cyclictest0-21swapper/008:45:100
56169799210,20cyclictest0-21swapper/007:23:170
56169799210,20cyclictest0-21swapper/007:23:170
56169799210,19cyclictest3056-21PK-Backend07:45:200
56169799210,19cyclictest0-21swapper/009:20:180
139099210,4rtkit-daemon1389-21rtkit-daemon09:55:320
139099210,18rtkit-daemon1317-21systemd11:26:321
56170799209,4cyclictest3056-21PK-Backend07:23:263
56170799209,4cyclictest3056-21PK-Backend07:23:253
56170799207,8cyclictest141rcu_preempt10:20:213
561707992018,1cyclictest3056-21PK-Backend09:40:223
561707992017,2cyclictest3056-21PK-Backend08:55:533
561707992017,2cyclictest3056-21PK-Backend07:12:463
561707992016,2cyclictest3056-21PK-Backend09:05:133
56170799200,1cyclictest3056-21PK-Backend11:07:073
56170799200,1cyclictest3056-21PK-Backend10:47:143
56170799200,19cyclictest3056-21PK-Backend11:37:153
56170799200,19cyclictest3056-21PK-Backend10:52:073
56170799200,19cyclictest3056-21PK-Backend09:24:533
56170799200,19cyclictest3056-21PK-Backend09:19:133
56170799200,19cyclictest3056-21PK-Backend08:31:333
561705992018,1cyclictest0-21swapper/212:37:092
561705992018,1cyclictest0-21swapper/212:23:432
561705992018,1cyclictest0-21swapper/211:42:192
561705992018,1cyclictest0-21swapper/211:33:492
561705992018,1cyclictest0-21swapper/211:25:302
561705992018,1cyclictest0-21swapper/211:07:532
561705992018,1cyclictest0-21swapper/211:02:122
561705992018,1cyclictest0-21swapper/210:22:242
561705992018,1cyclictest0-21swapper/210:00:052
561705992018,1cyclictest0-21swapper/208:14:402
561705992018,1cyclictest0-21swapper/208:06:362
561705992018,1cyclictest0-21swapper/208:04:472
561705992018,1cyclictest0-21swapper/207:34:462
56170599200,1cyclictest0-21swapper/210:54:282
56170599200,19cyclictest0-21swapper/211:22:392
56170599200,19cyclictest0-21swapper/209:47:022
56170599200,19cyclictest0-21swapper/209:14:232
56170599200,19cyclictest0-21swapper/209:14:232
56170599200,19cyclictest0-21swapper/208:21:102
56170599200,18cyclictest3056-21PK-Backend08:55:172
56170599200,10cyclictest141rcu_preempt12:15:002
561700992018,1cyclictest0-21swapper/109:39:541
561700992018,1cyclictest0-21swapper/107:45:021
561700992011,8cyclictest579761-21perf08:00:011
561700992011,7cyclictest671167-21cpuspeed_turbos12:20:111
561700992011,7cyclictest612955-21apt09:35:011
561697992018,1cyclictest0-21swapper/011:46:470
561697992018,1cyclictest0-21swapper/010:45:220
561697992018,1cyclictest0-21swapper/010:11:290
561697992018,1cyclictest0-21swapper/010:00:270
561697992018,1cyclictest0-21swapper/009:43:440
561697992018,1cyclictest0-21swapper/009:32:550
561697992018,1cyclictest0-21swapper/009:28:020
561697992018,1cyclictest0-21swapper/009:04:020
561697992013,5cyclictest436-20systemd-journal08:33:240
56169799200,19cyclictest0-21swapper/012:10:530
56169799200,19cyclictest0-21swapper/008:52:400
56169799200,19cyclictest0-21swapper/007:58:050
6083442190,3chrt331rcuc/209:20:182
56170799196,3cyclictest612212-21grep09:30:173
561707991917,1cyclictest3056-21PK-Backend08:18:503
561707991917,1cyclictest3056-21PK-Backend07:28:193
561707991917,1cyclictest3056-21PK-Backend07:15:573
561707991916,2cyclictest3056-21PK-Backend07:59:293
56170799190,1cyclictest3056-21PK-Backend10:06:523
56170799190,1cyclictest3056-21PK-Backend08:45:333
56170799190,1cyclictest0-21swapper/311:44:323
56170799190,1cyclictest0-21swapper/311:25:193
56170799190,1cyclictest0-21swapper/308:41:413
56170799190,18cyclictest3056-21PK-Backend08:26:053
56170799190,13cyclictest3056-21PK-Backend09:10:203
56170799190,13cyclictest3056-21PK-Backend09:10:203
56170599196,3cyclictest141rcu_preempt07:15:212
561705991917,1cyclictest0-21swapper/210:19:182
561705991911,6cyclictest634738-21ntpq10:35:182
56170599190,3cyclictest141rcu_preempt07:23:262
56170599190,3cyclictest141rcu_preempt07:23:262
56170599190,18cyclictest0-21swapper/212:16:192
56170599190,18cyclictest0-21swapper/210:40:342
56170599190,18cyclictest0-21swapper/208:45:432
56170099198,9cyclictest628894-21cpuspeed10:20:111
561700991912,5cyclictest643152-21grep11:00:121
561700991910,7cyclictest978-21snmpd08:55:131
561697991917,1cyclictest3056-21PK-Backend11:29:560
561697991917,1cyclictest0-21swapper/011:05:290
561697991917,1cyclictest0-21swapper/010:38:260
561697991917,1cyclictest0-21swapper/010:22:300
561697991917,1cyclictest0-21swapper/009:13:360
561697991917,1cyclictest0-21swapper/009:13:360
561697991917,1cyclictest0-21swapper/008:58:210
561697991917,1cyclictest0-21swapper/007:50:070
561697991911,6cyclictest642783-21pluginstate11:00:010
561697991911,6cyclictest1087-21gdbus11:02:100
56169799190,1cyclictest0-21swapper/008:04:070
56169799190,1cyclictest0-21swapper/007:38:590
56169799190,18cyclictest0-21swapper/011:15:370
56169799190,18cyclictest0-21swapper/009:16:260
56169799190,18cyclictest0-21swapper/008:15:360
56169799190,18cyclictest0-21swapper/008:09:210
56169799190,17cyclictest436-20systemd-journal10:45:010
561707991816,1cyclictest3056-21PK-Backend11:31:473
561707991816,1cyclictest3056-21PK-Backend10:15:513
561707991816,1cyclictest3056-21PK-Backend07:30:573
561707991815,2cyclictest3056-21PK-Backend10:59:373
56170599186,5cyclictest34-21ksoftirqd/207:25:422
561705991816,1cyclictest0-21swapper/210:05:122
561705991816,1cyclictest0-21swapper/208:37:112
561705991816,1cyclictest0-21swapper/208:31:312
56170599181,2cyclictest141rcu_preempt08:15:592
56170599180,8cyclictest660748-21tr11:50:132
56170599180,1cyclictest0-21swapper/210:34:272
56169799189,7cyclictest676991-21awk12:35:180
561697991816,1cyclictest0-21swapper/011:21:050
561697991816,1cyclictest0-21swapper/011:13:480
561697991816,1cyclictest0-21swapper/011:13:470
56169799181,4cyclictest618159-21apt09:50:010
561697991811,5cyclictest671656-21cat12:20:170
561697991811,5cyclictest664020-21ls12:00:000
561697991811,5cyclictest569749-21cat07:30:160
56169799181,13cyclictest653431-21apt11:30:120
56169799180,1cyclictest3056-21PK-Backend08:40:310
56169799180,1cyclictest0-21swapper/007:42:580
56170799176,10cyclictest42-21ksoftirqd/309:51:133
561707991715,1cyclictest3056-21PK-Backend08:24:173
561707991715,1cyclictest0-21swapper/312:05:413
561707991714,2cyclictest3056-21PK-Backend08:12:353
561707991712,3cyclictest3056-21PK-Backend08:00:103
561707991710,5cyclictest575188-21cat07:45:183
561707991710,2cyclictest42-21ksoftirqd/312:20:003
56170599177,4cyclictest34-21ksoftirqd/210:14:242
56170599177,2cyclictest141rcu_preempt09:40:182
56170599174,3cyclictest141rcu_preempt12:10:002
561697991715,1cyclictest0-21swapper/008:26:300
561697991715,1cyclictest0-21swapper/007:25:550
561697991711,5cyclictest564660-21cat07:15:160
56169799170,15cyclictest657897-21ntpq11:40:190
561707991614,1cyclictest3056-21PK-Backend12:20:013
561707991614,1cyclictest3056-21PK-Backend11:19:523
561707991614,1cyclictest3056-21PK-Backend08:38:033
561707991614,1cyclictest0-21swapper/309:47:183
561705991614,1cyclictest0-21swapper/211:39:292
561697991611,4cyclictest562899-21sed07:10:220
561697991610,5cyclictest619390-21sed09:50:190
56170799159,4cyclictest631608-21sed10:25:223
56170799158,2cyclictest3056-21PK-Backend11:50:223
56170799152,3cyclictest42-21ksoftirqd/310:00:133
561707991513,1cyclictest3056-21PK-Backend12:00:543
561707991511,2cyclictest42-21ksoftirqd/312:11:043
561707991510,3cyclictest632419-21cpuspeed_turbos10:30:123
56170799150,4cyclictest675783-21chrt12:32:543
56170799150,4cyclictest3056-21PK-Backend08:05:453
56170799150,3cyclictest3056-21PK-Backend11:49:163
56170799142,4cyclictest141rcu_preempt09:25:243
56170799130,5cyclictest3056-21PK-Backend11:55:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional