You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-23 - 22:11

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackcslot7s.osadl.org (updated Wed Sep 23, 2026 12:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
193286922450,16sleep00-21swapper/007:05:140
195018121190,4sleep1251rcuc/107:55:011
193250428047,23sleep10-21swapper/107:05:121
193337327841,9sleep30-21swapper/307:05:263
193382726847,15sleep20-21swapper/207:08:232
19494582470,10sleep013-21ksoftirqd/007:50:180
19461472470,2sleep1251rcuc/107:40:211
19725082330,2sleep20-21swapper/208:55:252
19810742280,3chrt0-21swapper/209:20:192
193409299289,10cyclictest13-21ksoftirqd/010:44:460
193409699260,24cyclictest2044230-21basename12:19:591
139099260,4rtkit-daemon1389-21rtkit-daemon12:18:500
139099260,4rtkit-daemon1389-21rtkit-daemon10:27:560
1934092992515,7cyclictest436-20systemd-journal07:25:000
1934092992413,9cyclictest2023136-21/usr/sbin/munin11:20:220
139099240,3rtkit-daemon1389-21rtkit-daemon10:51:050
1934102992320,2cyclictest3056-21PK-Backend12:11:433
1934102992319,2cyclictest3056-21PK-Backend12:36:313
1934102992319,2cyclictest3056-21PK-Backend07:29:353
193410299231,3cyclictest2036680-21python311:55:203
1934096992320,2cyclictest3056-21PK-Backend10:38:001
1934096992315,6cyclictest2000233-21ntp_states10:15:191
193409299236,6cyclictest13-21ksoftirqd/010:15:220
1934092992321,1cyclictest0-21swapper/008:28:210
1934092992319,3cyclictest161rcuc/011:35:110
1934092992319,3cyclictest13-21ksoftirqd/010:02:380
193410299221,9cyclictest1-21systemd11:55:003
1934102992219,2cyclictest3056-21PK-Backend12:15:223
1934102992219,2cyclictest3056-21PK-Backend10:49:513
1934102992219,2cyclictest3056-21PK-Backend10:14:083
193410299220,20cyclictest3056-21PK-Backend10:07:253
193410299220,20cyclictest3056-21PK-Backend09:15:553
193410299220,20cyclictest3056-21PK-Backend07:10:433
1934096992220,1cyclictest0-21swapper/112:29:401
1934096992220,1cyclictest0-21swapper/110:14:281
1934096992220,1cyclictest0-21swapper/109:52:041
1934096992220,1cyclictest0-21swapper/108:35:181
1934096992218,2cyclictest2002702-21apt-get10:25:021
1934096992214,6cyclictest2034836-21ntp_states11:50:241
1934096992214,6cyclictest1941435-21cron07:30:001
193409299225,7cyclictest13-21ksoftirqd/011:15:210
1934092992220,1cyclictest0-21swapper/012:13:300
1934092992220,1cyclictest0-21swapper/008:44:290
1934092992220,1cyclictest0-21swapper/007:45:550
1934092992219,2cyclictest161rcuc/008:47:520
1934092992219,2cyclictest161rcuc/007:18:300
139099220,4rtkit-daemon1389-21rtkit-daemon11:55:110
20398222210,3sleep20-21swapper/212:05:152
19912542210,4sleep33056-21PK-Backend09:50:193
193410299218,11cyclictest1957140-21unin-run08:15:003
1934102992119,1cyclictest3056-21PK-Backend11:26:413
1934102992119,1cyclictest3056-21PK-Backend10:41:333
1934102992119,1cyclictest3056-21PK-Backend09:45:023
1934102992119,1cyclictest3056-21PK-Backend09:43:593
1934102992119,1cyclictest3056-21PK-Backend09:36:163
1934102992119,1cyclictest3056-21PK-Backend09:12:023
1934102992119,1cyclictest0-21swapper/312:31:173
1934102992119,1cyclictest0-21swapper/307:48:353
1934102992116,4cyclictest2019-21gmain12:25:003
1934102992115,5cyclictest42-21ksoftirqd/308:15:123
193410299210,8cyclictest2028103-21latency_hist11:35:003
193410299210,20cyclictest3056-21PK-Backend10:59:103
193410299210,20cyclictest0-21swapper/308:35:333
193410299210,19cyclictest3056-21PK-Backend10:25:313
193410299210,19cyclictest3056-21PK-Backend07:52:393
193410299210,19cyclictest3056-21PK-Backend07:35:303
1934098992115,5cyclictest1095-21runrttasks08:11:542
1934098992113,6cyclictest1945122-21date07:40:012
193409899210,19cyclictest1998934-21chrt10:11:312
1934096992119,1cyclictest0-21swapper/112:23:391
1934096992119,1cyclictest0-21swapper/112:13:301
1934096992119,1cyclictest0-21swapper/111:44:381
1934096992119,1cyclictest0-21swapper/111:17:341
1934096992119,1cyclictest0-21swapper/110:45:311
1934096992119,1cyclictest0-21swapper/109:20:411
1934096992119,1cyclictest0-21swapper/107:57:471
1934096992119,1cyclictest0-21swapper/107:35:221
193409699210,20cyclictest0-21swapper/110:58:421
193409699210,20cyclictest0-21swapper/110:06:441
193409699210,20cyclictest0-21swapper/109:40:141
193409699210,20cyclictest0-21swapper/108:01:521
193409299214,7cyclictest13-21ksoftirqd/012:30:230
193409299214,7cyclictest13-21ksoftirqd/012:00:240
193409299214,7cyclictest13-21ksoftirqd/011:25:150
193409299214,6cyclictest13-21ksoftirqd/009:50:220
193409299213,7cyclictest13-21ksoftirqd/012:05:190
193409299212,7cyclictest13-21ksoftirqd/007:55:210
1934092992119,1cyclictest0-21swapper/011:11:520
1934092992119,1cyclictest0-21swapper/009:58:180
1934092992119,1cyclictest0-21swapper/009:39:310
1934092992119,1cyclictest0-21swapper/009:10:250
1934092992119,1cyclictest0-21swapper/009:01:260
1934092992119,1cyclictest0-21swapper/008:59:240
1934092992119,1cyclictest0-21swapper/008:38:350
1934092992119,1cyclictest0-21swapper/008:16:520
1934092992119,1cyclictest0-21swapper/008:06:240
1934092992117,3cyclictest161rcuc/010:55:350
1934092992116,3cyclictest2019418-21ls11:10:000
1934092992116,3cyclictest2019418-21ls11:10:000
193409299210,20cyclictest0-21swapper/012:38:180
193409299210,20cyclictest0-21swapper/009:08:220
193409299210,1cyclictest0-21swapper/009:40:330
139099210,3rtkit-daemon1389-21rtkit-daemon10:35:200
139099210,15rtkit-daemon0-21swapper/008:14:220
139099210,14rtkit-daemon3056-21PK-Backend08:30:131
139099210,14rtkit-daemon0-21swapper/010:21:530
139099210,14rtkit-daemon0-21swapper/009:23:550
20247302200,3sleep10-21swapper/111:25:001
193410299203,4cyclictest0-21swapper/308:25:113
1934102992018,1cyclictest3056-21PK-Backend12:28:393
1934102992018,1cyclictest3056-21PK-Backend10:52:413
1934102992018,1cyclictest3056-21PK-Backend10:20:513
1934102992018,1cyclictest3056-21PK-Backend09:56:513
1934102992018,1cyclictest3056-21PK-Backend09:23:393
1934102992018,1cyclictest3056-21PK-Backend08:50:273
1934102992018,1cyclictest3056-21PK-Backend07:30:373
1934102992017,2cyclictest3056-21PK-Backend11:05:533
1934102992017,2cyclictest3056-21PK-Backend11:05:523
1934102992017,2cyclictest3056-21PK-Backend10:31:123
1934102992017,2cyclictest3056-21PK-Backend07:57:483
193410299200,1cyclictest3056-21PK-Backend07:18:133
193410299200,19cyclictest3056-21PK-Backend12:00:473
193410299200,19cyclictest3056-21PK-Backend11:39:123
193410299200,19cyclictest3056-21PK-Backend10:03:343
193410299200,19cyclictest3056-21PK-Backend09:27:313
193410299200,19cyclictest3056-21PK-Backend09:07:103
193410299200,19cyclictest3056-21PK-Backend09:01:153
193410299200,19cyclictest3056-21PK-Backend08:00:523
193410299200,19cyclictest0-21swapper/308:41:273
193410299200,19cyclictest0-21swapper/308:20:463
193410299200,19cyclictest0-21swapper/307:43:283
1934098992018,1cyclictest0-21swapper/211:14:032
1934098992018,1cyclictest0-21swapper/210:53:292
1934098992018,1cyclictest0-21swapper/210:03:072
1934098992018,1cyclictest0-21swapper/208:37:222
1934098992018,1cyclictest0-21swapper/207:46:532
1934098992017,2cyclictest1894998-21snapd10:44:292
1934098992013,5cyclictest1958653-21sshd08:17:472
193409899200,1cyclictest0-21swapper/210:06:312
1934096992018,1cyclictest0-21swapper/112:00:471
1934096992018,1cyclictest0-21swapper/111:02:341
1934096992018,1cyclictest0-21swapper/110:30:441
1934096992018,1cyclictest0-21swapper/110:24:001
1934096992018,1cyclictest0-21swapper/109:25:331
1934096992018,1cyclictest0-21swapper/109:16:491
1934096992018,1cyclictest0-21swapper/109:12:571
1934096992018,1cyclictest0-21swapper/107:33:201
1934096992013,6cyclictest2009673-21sed10:40:211
1934096992013,5cyclictest2049866-21expr12:35:111
193409699200,9cyclictest19846712awk09:30:221
193409699200,1cyclictest0-21swapper/111:39:111
193409699200,19cyclictest0-21swapper/111:59:461
193409699200,19cyclictest0-21swapper/110:53:001
193409699200,19cyclictest0-21swapper/108:51:071
193409299204,9cyclictest13-21ksoftirqd/011:48:470
1934092992018,1cyclictest0-21swapper/010:11:570
1934092992018,1cyclictest0-21swapper/009:46:280
1934092992018,1cyclictest0-21swapper/009:29:450
1934092992018,1cyclictest0-21swapper/008:00:160
1934092992018,1cyclictest0-21swapper/007:42:520
1934092992017,2cyclictest0-21swapper/007:30:010
1934092992016,3cyclictest161rcuc/007:39:130
1934092992013,5cyclictest2031556-21sshd11:41:470
1934092992011,7cyclictest2027004-21df_inode11:30:150
193409299200,1cyclictest0-21swapper/008:52:270
193409299200,19cyclictest0-21swapper/011:02:180
139099200,13rtkit-daemon0-21swapper/108:26:061
139099200,13rtkit-daemon0-21swapper/009:15:080
139099200,12rtkit-daemon0-21swapper/009:30:060
1934102991917,1cyclictest3056-21PK-Backend12:08:053
1934102991917,1cyclictest3056-21PK-Backend11:49:053
1934102991917,1cyclictest3056-21PK-Backend11:18:373
1934102991917,1cyclictest3056-21PK-Backend10:15:103
1934102991917,1cyclictest3056-21PK-Backend09:32:243
1934102991917,1cyclictest3056-21PK-Backend08:55:203
1934102991917,1cyclictest0-21swapper/310:36:523
1934102991911,6cyclictest1955367-21cron08:10:003
193410299190,1cyclictest3056-21PK-Backend11:41:013
193410299190,1cyclictest3056-21PK-Backend11:23:033
193410299190,1cyclictest3056-21PK-Backend11:10:323
193410299190,1cyclictest3056-21PK-Backend11:00:113
1934098991917,1cyclictest0-21swapper/208:32:562
1934098991915,3cyclictest1979363-21grep09:15:182
1934098991912,5cyclictest2048073-21cat12:30:162
193409899190,1cyclictest0-21swapper/212:18:192
193409899190,18cyclictest0-21swapper/208:24:382
1934096991917,1cyclictest3056-21PK-Backend08:14:561
1934096991917,1cyclictest0-21swapper/111:45:391
1934096991917,1cyclictest0-21swapper/111:13:561
1934096991917,1cyclictest0-21swapper/111:09:171
1934096991917,1cyclictest0-21swapper/111:09:171
1934096991917,1cyclictest0-21swapper/109:02:081
1934096991912,5cyclictest1994469-21expr10:00:141
193409699190,1cyclictest0-21swapper/112:08:521
193409699190,18cyclictest3056-21PK-Backend08:21:391
193409299196,3cyclictest13-21ksoftirqd/010:30:220
193409299195,8cyclictest13-21ksoftirqd/007:11:080
1934092991917,1cyclictest0-21swapper/012:27:020
193409299190,7cyclictest13-21ksoftirqd/011:54:590
193409299190,1cyclictest0-21swapper/010:48:060
193409299190,1cyclictest0-21swapper/010:07:030
193409299190,18cyclictest0-21swapper/012:21:470
139099190,13rtkit-daemon0-21swapper/008:21:460
139099190,12rtkit-daemon0-21swapper/008:30:070
1934102991816,1cyclictest3056-21PK-Backend07:23:533
1934102991812,4cyclictest1962723-21cpuspeed_turbos08:30:143
1934098991816,1cyclictest0-21swapper/211:45:072
1934098991816,1cyclictest0-21swapper/209:36:512
1934098991816,1cyclictest0-21swapper/207:42:272
1934098991816,1cyclictest0-21swapper/207:33:002
1934098991814,3cyclictest1937987-21latency_hist07:20:002
1934098991813,4cyclictest978-21snmpd11:01:022
1934098991813,3cyclictest1935206-21wc07:10:212
1934098991812,4cyclictest2006518-21date10:35:012
1934098991811,6cyclictest883-21dbus-daemon10:50:012
1934098991811,5cyclictest436-20systemd-journal09:48:242
1934098991811,5cyclictest2050422-21ntpq12:35:172
1934098991811,5cyclictest1969092-21cat08:45:222
1934098991810,6cyclictest2008153-21apt-get10:40:012
1934096991816,1cyclictest3056-21PK-Backend08:46:131
1934096991816,1cyclictest0-21swapper/108:59:031
1934096991816,1cyclictest0-21swapper/107:20:561
1934096991814,3cyclictest2026357-21cron11:30:001
1934096991812,4cyclictest26-21ksoftirqd/107:15:021
1934096991811,6cyclictest883-21dbus-daemon09:10:001
1934096991810,6cyclictest2028095-21apt-get11:35:011
1934092991812,4cyclictest13-21ksoftirqd/007:35:010
193410299178,8cyclictest1968709-21needreboot08:45:173
1934098991715,1cyclictest0-21swapper/210:59:172
1934098991715,1cyclictest0-21swapper/208:41:002
1934098991715,1cyclictest0-21swapper/208:04:362
193409899171,2cyclictest141rcu_preempt08:55:002
1934098991712,3cyclictest2021592-21tr11:15:142
1934098991710,5cyclictest2046382-21expr12:25:132
1934098991710,5cyclictest2029092-21interrupts11:35:162
1934098991710,5cyclictest1954047-21tr08:05:152
1934098991710,5cyclictest1951185-21cat07:55:192
1934096991715,1cyclictest3056-21PK-Backend08:17:471
1934096991715,1cyclictest0-21swapper/109:36:221
1934096991711,5cyclictest1989487-21smartctl09:45:151
1934096991710,5cyclictest26-21ksoftirqd/109:55:201
193409699170,16cyclictest3056-21PK-Backend08:41:331
193409899169,5cyclictest883-21dbus-daemon08:29:592
193409899169,5cyclictest2045396-21grep12:20:212
193409899168,7cyclictest2038596-21grep12:00:242
1934098991614,1cyclictest0-21swapper/211:41:072
1934098991612,3cyclictest978-21snmpd11:21:362
1934098991611,3cyclictest2036331-21latency11:55:172
1934098991611,3cyclictest1983624-21turbostat.cron09:30:002
1934098991610,4cyclictest34-21ksoftirqd/209:30:152
1934098991610,4cyclictest2034658-21cat11:50:222
1934098991610,4cyclictest1975477-21expr09:05:142
1934096991611,3cyclictest1954888-21timerandwakeup08:05:241
193409899159,5cyclictest2042525-21sort12:15:012
193409899158,5cyclictest883-21dbus-daemon10:25:012
193409899158,5cyclictest1999839-21expr10:15:122
1934098991513,1cyclictest0-21swapper/209:58:542
1934098991511,3cyclictest1990806-21/usr/sbin/munin09:50:242
1934098991510,4cyclictest1977456-21sed09:10:182
193409899150,14cyclictest3056-21PK-Backend07:51:192
1934096991513,1cyclictest3056-21PK-Backend12:34:411
193409899149,4cyclictest2002757-21date10:25:012
193409899148,5cyclictest2027221-21wc11:30:212
193409899148,5cyclictest1938798-21grep07:20:172
1934098991412,1cyclictest0-21swapper/211:28:242
1934098991410,3cyclictest1988816-21cron09:45:012
193409899140,1cyclictest0-21swapper/211:05:052
193409899140,1cyclictest0-21swapper/211:05:042
193409899140,13cyclictest0-21swapper/209:04:062
1934096991411,2cyclictest1935412-21sshd07:10:461
1934096991410,3cyclictest3056-21PK-Backend07:45:591
193409899128,3cyclictest1940106-21cpuspeed_turbos07:25:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional