You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-24 - 05:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Thu Sep 24, 2026 00:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
229000321140,3sleep126-21ksoftirqd/122:35:191
223123921050,3sleep10-21swapper/119:50:021
224907421000,2sleep20-21swapper/220:40:152
221593027941,14sleep30-21swapper/319:08:223
139099720,59rtkit-daemon0-21swapper/119:05:281
221597926445,12sleep00-21swapper/019:08:590
22948782620,2sleep20-21swapper/222:50:192
221598626242,14sleep20-21swapper/219:09:062
23154562500,2sleep00-21swapper/023:50:010
23154562500,2sleep00-21swapper/023:50:000
22453562470,2sleep00-21swapper/020:30:150
22379302470,3sleep22237928-21cron20:10:002
22926502410,4sleep00-21swapper/022:45:000
23106062400,2sleep13056-21PK-Backend23:35:141
23049332380,4sleep02304929-21latency_hist23:20:010
23007022380,4sleep13056-21PK-Backend23:05:221
22172482290,8sleep2341ksoftirqd/219:10:212
22267752270,7sleep1261ksoftirqd/119:35:231
221618499266,11cyclictest34-21ksoftirqd/222:10:002
2216184992419,3cyclictest34-21ksoftirqd/219:25:002
2216184992416,6cyclictest2255264-21cron21:00:002
2216182992320,2cyclictest3056-21PK-Backend23:30:031
2216182992313,7cyclictest436-20systemd-journal20:49:591
221618299230,21cyclictest3056-21PK-Backend21:40:331
221618799221,7cyclictest2313780-21apt23:45:103
221618799221,7cyclictest2313780-21apt23:45:103
2216187992214,6cyclictest2331751-21df00:35:153
2216184992220,1cyclictest0-21swapper/220:48:262
221618499220,3cyclictest2232014-21ntp_states19:50:192
2216182992220,1cyclictest0-21swapper/123:58:501
2216182992213,7cyclictest1-21systemd19:15:041
221618299220,20cyclictest3056-21PK-Backend23:24:221
221618299220,20cyclictest3056-21PK-Backend22:25:081
221618299220,20cyclictest3056-21PK-Backend19:43:191
2216178992220,1cyclictest0-21swapper/020:40:530
2216178992215,5cyclictest2232728-21cron19:55:000
22696262210,4sleep00-21swapper/021:40:010
221618799210,20cyclictest3056-21PK-Backend23:59:063
221618799210,20cyclictest0-21swapper/323:21:483
221618799210,19cyclictest3056-21PK-Backend20:38:103
221618799210,19cyclictest3056-21PK-Backend00:12:043
2216184992119,1cyclictest0-21swapper/223:20:412
2216184992119,1cyclictest0-21swapper/222:48:522
2216184992119,1cyclictest0-21swapper/222:26:472
2216184992119,1cyclictest0-21swapper/222:23:092
2216184992119,1cyclictest0-21swapper/222:13:162
2216184992119,1cyclictest0-21swapper/219:44:452
2216184992119,1cyclictest0-21swapper/219:31:272
2216184992119,1cyclictest0-21swapper/219:17:342
2216184992119,1cyclictest0-21swapper/200:29:032
2216184992119,1cyclictest0-21swapper/200:20:582
2216184992119,1cyclictest0-21swapper/200:07:462
221618499210,20cyclictest0-21swapper/223:46:032
221618499210,20cyclictest0-21swapper/223:46:032
221618499210,20cyclictest0-21swapper/223:11:092
221618499210,20cyclictest0-21swapper/222:15:062
221618499210,20cyclictest0-21swapper/219:39:112
221618499210,1cyclictest0-21swapper/223:59:142
221618299212,13cyclictest2258751-21apt-get21:10:031
2216182992119,1cyclictest3056-21PK-Backend20:55:051
2216182992119,1cyclictest0-21swapper/123:04:081
2216182992119,1cyclictest0-21swapper/122:15:281
2216182992119,1cyclictest0-21swapper/121:23:091
2216182992119,1cyclictest0-21swapper/120:36:121
2216182992119,1cyclictest0-21swapper/119:46:301
2216182992119,1cyclictest0-21swapper/100:24:451
221618299210,2cyclictest1788620-21WebKitWebProces19:12:031
221618299210,19cyclictest3056-21PK-Backend22:32:581
2216178992119,1cyclictest0-21swapper/023:23:150
2216178992119,1cyclictest0-21swapper/022:17:110
2216178992119,1cyclictest0-21swapper/021:32:430
2216178992119,1cyclictest0-21swapper/020:38:280
2216178992119,1cyclictest0-21swapper/020:25:240
2216178992119,1cyclictest0-21swapper/019:39:340
2216178992119,1cyclictest0-21swapper/019:14:190
2216178992119,1cyclictest0-21swapper/000:18:180
2216178992118,2cyclictest3056-21PK-Backend23:31:530
2216178992114,5cyclictest2317948-21ntp_kernel_err23:55:180
221617899210,20cyclictest0-21swapper/023:35:180
221617899210,20cyclictest0-21swapper/022:50:030
221617899210,20cyclictest0-21swapper/000:05:330
221617899210,19cyclictest3056-21PK-Backend22:35:220
221617899210,19cyclictest3056-21PK-Backend22:20:350
2216187992018,1cyclictest3056-21PK-Backend00:26:383
2216187992018,1cyclictest0-21swapper/322:41:063
2216187992018,1cyclictest0-21swapper/321:49:553
2216187992017,2cyclictest3056-21PK-Backend20:40:333
2216187992016,3cyclictest1895000-21snapd22:23:433
2216187992015,4cyclictest978-21snmpd22:54:063
2216187992014,4cyclictest2220270-21python319:19:133
2216187992012,6cyclictest2275149-21expr21:55:113
221618799201,17cyclictest2273495-21expr21:50:163
221618799200,1cyclictest3056-21PK-Backend23:52:383
221618799200,1cyclictest3056-21PK-Backend21:16:103
221618799200,1cyclictest0-21swapper/322:00:443
221618799200,19cyclictest3056-21PK-Backend23:03:103
221618799200,19cyclictest0-21swapper/323:13:173
2216184992018,1cyclictest3056-21PK-Backend00:18:072
2216184992018,1cyclictest0-21swapper/223:41:232
2216184992018,1cyclictest0-21swapper/223:01:012
2216184992018,1cyclictest0-21swapper/222:38:302
2216184992018,1cyclictest0-21swapper/221:15:432
2216184992018,1cyclictest0-21swapper/220:50:022
2216184992018,1cyclictest0-21swapper/220:50:012
2216184992018,1cyclictest0-21swapper/220:27:172
2216184992018,1cyclictest0-21swapper/200:04:572
2216184992015,3cyclictest2247047-21tr20:35:102
2216184992014,5cyclictest2175-21JS10:27:482
2216184992014,4cyclictest2272873-21cron21:50:012
2216184992013,5cyclictest1895000-21snapd21:01:282
221618499201,11cyclictest34-21ksoftirqd/221:10:002
221618499200,19cyclictest0-21swapper/223:28:122
221618499200,19cyclictest0-21swapper/222:31:132
221618499200,19cyclictest0-21swapper/222:04:102
221618499200,19cyclictest0-21swapper/221:26:182
221618499200,19cyclictest0-21swapper/219:49:312
221618499200,19cyclictest0-21swapper/200:30:512
2216182992018,1cyclictest3056-21PK-Backend22:55:021
2216182992018,1cyclictest3056-21PK-Backend21:48:441
2216182992018,1cyclictest3056-21PK-Backend00:37:161
2216182992018,1cyclictest0-21swapper/120:51:051
2216182992018,1cyclictest0-21swapper/120:51:051
2216182992011,7cyclictest2285518-21cron22:25:011
221618299200,1cyclictest3056-21PK-Backend19:27:161
221618299200,19cyclictest3056-21PK-Backend22:46:441
221618299200,19cyclictest0-21swapper/123:48:291
221618299200,19cyclictest0-21swapper/123:48:291
221618299200,19cyclictest0-21swapper/122:11:491
221618299200,19cyclictest0-21swapper/120:44:431
221618299200,19cyclictest0-21swapper/120:08:481
221618299200,19cyclictest0-21swapper/100:03:291
221618299200,18cyclictest3056-21PK-Backend23:43:501
2216178992018,1cyclictest0-21swapper/020:51:470
2216178992018,1cyclictest0-21swapper/020:51:470
2216178992018,1cyclictest0-21swapper/020:06:310
2216178992018,1cyclictest0-21swapper/000:21:560
2216178992012,6cyclictest2279338-21ntp_states22:05:190
2216178992012,6cyclictest2251452-21kworker/u8:121:10:230
221617899200,19cyclictest0-21swapper/021:18:100
139099200,14rtkit-daemon3056-21PK-Backend21:11:173
2216187991917,1cyclictest3056-21PK-Backend21:28:193
2216187991917,1cyclictest3056-21PK-Backend00:00:083
2216187991917,1cyclictest0-21swapper/319:33:483
221618799191,4cyclictest2303980-21python323:15:203
2216187991912,5cyclictest436-20systemd-journal20:55:343
221618799190,1cyclictest3056-21PK-Backend20:15:133
221618799190,1cyclictest0-21swapper/323:33:593
221618799190,1cyclictest0-21swapper/323:08:373
221618799190,1cyclictest0-21swapper/321:44:563
221618799190,1cyclictest0-21swapper/321:39:563
221618799190,1cyclictest0-21swapper/319:43:563
221618799190,18cyclictest3056-21PK-Backend22:59:313
221618799190,18cyclictest0-21swapper/322:45:463
221618799190,18cyclictest0-21swapper/322:29:093
221618799190,18cyclictest0-21swapper/321:34:553
221618799190,18cyclictest0-21swapper/319:47:543
2216184991917,1cyclictest0-21swapper/200:36:192
2216184991914,3cyclictest978-21snmpd23:17:462
2216184991913,4cyclictest34-21ksoftirqd/223:35:012
2216184991913,4cyclictest34-21ksoftirqd/222:00:002
2216184991912,5cyclictest2235822-21vmstat20:00:262
2216184991910,2cyclictest34-21ksoftirqd/221:10:012
221618499190,1cyclictest0-21swapper/222:41:222
221618499190,1cyclictest0-21swapper/221:30:302
221618499190,1cyclictest0-21swapper/220:22:442
221618499190,1cyclictest0-21swapper/219:59:522
221618499190,18cyclictest0-21swapper/222:58:112
221618499190,18cyclictest0-21swapper/200:10:372
2216182991917,1cyclictest3056-21PK-Backend00:32:501
2216182991917,1cyclictest0-21swapper/122:03:451
2216182991917,1cyclictest0-21swapper/120:26:521
2216182991917,1cyclictest0-21swapper/120:21:311
2216182991913,4cyclictest2279160-21cat22:05:161
2216182991912,5cyclictest2264918-21cat21:25:171
2216182991911,6cyclictest2240701-21awk20:15:211
221618299191,14cyclictest2261006-21apt21:15:111
221618299190,1cyclictest3056-21PK-Backend23:16:041
221618299190,1cyclictest3056-21PK-Backend21:56:091
221618299190,1cyclictest0-21swapper/100:26:341
221618299190,1cyclictest0-21swapper/100:15:391
221618299190,1cyclictest0-21swapper/100:12:491
2216178991917,1cyclictest0-21swapper/023:42:070
2216178991917,1cyclictest0-21swapper/022:46:380
2216178991917,1cyclictest0-21swapper/022:04:540
2216178991917,1cyclictest0-21swapper/021:01:400
2216178991917,1cyclictest0-21swapper/020:19:160
2216178991915,2cyclictest2315516-21apt23:50:120
2216178991912,5cyclictest2251306-21sed20:45:210
2216178991911,6cyclictest2236362-21cat20:05:000
221617899190,1cyclictest0-21swapper/019:22:320
221617899190,18cyclictest0-21swapper/019:40:220
221617899190,18cyclictest0-21swapper/000:14:390
221617899190,18cyclictest0-21swapper/000:01:550
2216187991816,1cyclictest3056-21PK-Backend20:22:093
2216187991816,1cyclictest0-21swapper/322:30:113
2216187991816,1cyclictest0-21swapper/319:35:243
2216187991815,2cyclictest3056-21PK-Backend20:27:293
2216187991815,1cyclictest3056-21PK-Backend20:50:423
2216187991815,1cyclictest3056-21PK-Backend20:50:413
2216187991815,1cyclictest0-21swapper/322:17:403
2216187991812,5cyclictest42-21ksoftirqd/323:35:123
2216187991811,5cyclictest42-21ksoftirqd/323:25:193
2216184991816,1cyclictest0-21swapper/223:54:352
2216184991816,1cyclictest0-21swapper/223:36:452
2216184991816,1cyclictest0-21swapper/223:05:412
2216184991816,1cyclictest0-21swapper/220:18:582
2216182991816,1cyclictest3056-21PK-Backend22:51:251
2216182991816,1cyclictest3056-21PK-Backend22:44:551
2216182991816,1cyclictest3056-21PK-Backend21:34:451
2216182991816,1cyclictest0-21swapper/123:54:101
2216182991816,1cyclictest0-21swapper/120:33:491
2216182991816,1cyclictest0-21swapper/120:00:011
2216182991815,2cyclictest3056-21PK-Backend23:14:151
2216182991810,6cyclictest2258400-21sshd21:05:471
2216178991816,1cyclictest0-21swapper/023:12:470
2216178991816,1cyclictest0-21swapper/023:04:100
2216178991816,1cyclictest0-21swapper/022:25:360
2216178991816,1cyclictest0-21swapper/021:29:320
2216178991816,1cyclictest0-21swapper/019:33:050
2216178991814,2cyclictest2217865-21latency_hist19:15:010
2216178991812,4cyclictest2242796-21sshd20:20:460
2216178991811,5cyclictest2281042-21cat22:10:210
2216178991811,5cyclictest2257374-21ls21:05:140
2216178991811,5cyclictest2254190-21diskstats20:55:140
221618799179,6cyclictest2258751-21apt-get21:10:013
2216187991715,1cyclictest3056-21PK-Backend22:08:493
2216187991715,1cyclictest3056-21PK-Backend20:49:063
2216187991715,1cyclictest3056-21PK-Backend20:13:363
2216187991715,1cyclictest3056-21PK-Backend00:22:123
2216187991711,4cyclictest42-21ksoftirqd/322:35:143
221618799171,14cyclictest2233373-21df19:55:143
221618799170,15cyclictest2312052-21apt23:40:143
221618799170,15cyclictest2223425-21users19:25:223
221618799170,14cyclictest2329632-21/usr/sbin/munin00:30:143
221618499179,6cyclictest2264194-21cron21:25:012
2216184991713,3cyclictest978-21snmpd21:37:252
2216184991710,6cyclictest2273143-21apt21:50:162
2216184991710,5cyclictest2245246-21sort20:30:142
2216182991715,1cyclictest3056-21PK-Backend19:55:151
2216182991714,2cyclictest3056-21PK-Backend19:22:231
2216182991713,3cyclictest2321579-21rt-features00:05:221
221617899177,9cyclictest2233682-21needreboot19:55:180
2216178991715,1cyclictest0-21swapper/023:29:170
2216178991715,1cyclictest0-21swapper/021:59:060
2216178991715,1cyclictest0-21swapper/021:42:500
2216178991712,3cyclictest2330308-21needreboot00:30:180
2216178991712,3cyclictest2239671-21idleruntime-cro20:15:010
2216178991711,5cyclictest2297182-21sed22:55:270
2216178991710,5cyclictest2263809-21uptime21:20:240
221617899170,3cyclictest2329399-21awk00:30:000
221617899170,16cyclictest0-21swapper/021:51:220
139099170,7rtkit-daemon0-21swapper/221:40:082
221618799169,5cyclictest2220598-21sort19:20:123
2216187991614,1cyclictest3056-21PK-Backend22:10:373
2216187991614,1cyclictest3056-21PK-Backend00:16:443
2216187991612,3cyclictest42-21ksoftirqd/319:50:023
221618499165,4cyclictest34-21ksoftirqd/220:10:172
221618299168,6cyclictest2238784-21memory20:10:171
2216182991613,2cyclictest3056-21PK-Backend21:00:391
221617899169,5cyclictest0-21swapper/019:28:090
2216178991614,1cyclictest0-21swapper/021:45:130
2216178991611,4cyclictest2287649-21tr22:30:130
221617899161,13cyclictest2331525-21cpuspeed_turbos00:35:140
2216178991611,3cyclictest2230745-21idleruntime-cro19:49:590
2216178991610,4cyclictest2301242-21cat23:10:010
221618799154,2cyclictest141rcu_preempt19:15:003
2216187991513,1cyclictest3056-21PK-Backend20:32:493
221618799150,3cyclictest2256288-21ntp_states21:00:173
2216182991513,1cyclictest3056-21PK-Backend19:32:371
221618799149,4cyclictest2236973-21sed20:05:203
2216187991412,1cyclictest3056-21PK-Backend00:06:373
2216187991410,3cyclictest3056-21PK-Backend20:01:063
221618299149,4cyclictest3056-21PK-Backend21:50:041
221618299149,4cyclictest2308321-21cut23:30:001
221618299128,3cyclictest3056-21PK-Backend21:39:571
139099120,4rtkit-daemon3056-21PK-Backend21:20:133
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional