You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-28 - 20:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Tue Jul 28, 2026 12:44:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
341733321190,2sleep30-21swapper/308:05:123
346187621060,3sleep13056-21PK-Backend10:10:211
339700927740,13sleep30-21swapper/307:07:383
339703727539,14sleep10-21swapper/107:08:011
339718527239,9sleep00-21swapper/007:09:520
34862772700,4sleep13056-21PK-Backend11:20:191
339685826444,13sleep20-21swapper/207:05:412
34310242570,9chrt26-21ksoftirqd/108:45:001
34689392520,3sleep10-21swapper/110:30:231
34298552520,2sleep00-21swapper/008:40:140
34186082500,2sleep10-21swapper/108:09:561
34186262490,3sleep23418619-21cron08:10:002
34370872480,2sleep00-21swapper/009:00:180
34669132410,3sleep10-21swapper/110:25:171
34314902400,3sleep20-21swapper/208:45:132
34502092380,2sleep00-21swapper/009:40:000
339731699253,20cyclictest3431019-21apt08:44:593
139099250,3rtkit-daemon1389-21rtkit-daemon08:57:591
34561482240,3sleep30-21swapper/309:55:153
3397314992419,4cyclictest34-21ksoftirqd/212:23:332
3397314992419,4cyclictest34-21ksoftirqd/211:52:142
3397316992315,6cyclictest3454685-21python309:50:223
3397316992313,8cyclictest957-21gdbus12:23:333
3397314992314,7cyclictest3416375-21awk08:00:222
3397309992315,6cyclictest3407060-21df07:35:141
3397309992311,10cyclictest3413511-21cat07:55:001
139099230,16rtkit-daemon3056-21PK-Backend09:00:043
139099230,16rtkit-daemon3056-21PK-Backend08:59:243
139099230,14rtkit-daemon0-21swapper/210:58:182
35134352221,7sleep234-21ksoftirqd/212:40:002
3397316992216,4cyclictest3403416-21grep07:25:133
3397316992215,5cyclictest436-20systemd-journal11:25:003
3397316992215,5cyclictest2174-21JS10:27:483
3397316992215,4cyclictest2176-21JS10:27:483
339731699220,20cyclictest3056-21PK-Backend08:48:223
339731499229,11cyclictest3451851-21unin-run09:45:002
3397314992220,1cyclictest0-21swapper/212:08:432
3397314992220,1cyclictest0-21swapper/211:32:332
3397314992219,2cyclictest0-21swapper/209:11:322
3397314992214,6cyclictest3500387-21awk12:00:212
339730599220,21cyclictest0-21swapper/012:02:520
139099220,4rtkit-daemon1389-21rtkit-daemon11:04:370
139099220,15rtkit-daemon3056-21PK-Backend07:52:263
139099220,15rtkit-daemon3056-21PK-Backend07:19:143
35003572210,4sleep13056-21PK-Backend12:00:201
3397316992119,1cyclictest0-21swapper/311:04:213
3397316992119,1cyclictest0-21swapper/310:10:083
3397316992119,1cyclictest0-21swapper/307:36:093
3397316992118,2cyclictest3056-21PK-Backend07:12:093
3397316992114,5cyclictest3446606-21cron09:29:593
3397316992113,6cyclictest3439301-21vmstat09:05:243
339731699210,8cyclictest3433769-21ntp_states08:50:193
3397314992119,1cyclictest0-21swapper/211:43:492
3397314992119,1cyclictest0-21swapper/210:53:002
3397314992119,1cyclictest0-21swapper/210:45:092
3397314992119,1cyclictest0-21swapper/210:40:352
3397314992119,1cyclictest0-21swapper/210:21:502
3397314992119,1cyclictest0-21swapper/210:09:122
3397314992119,1cyclictest0-21swapper/210:00:272
3397314992119,1cyclictest0-21swapper/209:55:412
3397314992119,1cyclictest0-21swapper/209:16:332
3397314992118,2cyclictest3503644-21strings12:10:192
3397314992114,5cyclictest3453507-21sshd09:49:372
3397314992114,5cyclictest3420423-21latency_hist08:15:002
339731499210,20cyclictest0-21swapper/212:27:082
339731499210,20cyclictest0-21swapper/211:59:372
339731499210,20cyclictest0-21swapper/211:21:182
339731499210,20cyclictest0-21swapper/209:51:422
339731499210,20cyclictest0-21swapper/209:32:362
339731499210,20cyclictest0-21swapper/208:21:522
339731499210,20cyclictest0-21swapper/207:35:342
3397309992119,1cyclictest0-21swapper/111:37:061
3397309992118,2cyclictest3056-21PK-Backend12:17:551
3397309992114,5cyclictest3414562-21python307:55:211
339730999210,20cyclictest0-21swapper/110:25:001
339730999210,19cyclictest3056-21PK-Backend10:47:041
339730999210,19cyclictest3056-21PK-Backend10:38:121
339730999210,19cyclictest3056-21PK-Backend09:13:091
339730999210,19cyclictest3056-21PK-Backend07:27:231
339730599213,15cyclictest3462483-21munin-run10:15:000
3397305992113,6cyclictest3486989-21cron11:25:000
3397305992113,6cyclictest3476340-21cron10:55:000
339730599210,20cyclictest0-21swapper/011:45:060
339730599210,20cyclictest0-21swapper/011:36:150
339730599210,20cyclictest0-21swapper/008:02:340
339730599210,19cyclictest3056-21PK-Backend12:25:220
339730599210,19cyclictest3056-21PK-Backend10:25:300
139099210,4rtkit-daemon1389-21rtkit-daemon10:00:030
139099210,14rtkit-daemon0-21swapper/211:06:112
339731699205,13cyclictest3487318-21cpu11:25:123
3397316992018,1cyclictest0-21swapper/312:28:033
3397316992018,1cyclictest0-21swapper/311:57:073
3397316992018,1cyclictest0-21swapper/311:30:253
3397316992018,1cyclictest0-21swapper/310:43:273
3397316992018,1cyclictest0-21swapper/310:34:003
3397316992018,1cyclictest0-21swapper/309:34:313
3397316992018,1cyclictest0-21swapper/307:42:113
3397316992015,3cyclictest3452826-21ntp_states09:45:193
3397316992014,4cyclictest3444090-21strings09:20:183
3397316992014,4cyclictest3444090-21strings09:20:183
3397316992013,5cyclictest3504708-21apt12:15:113
3397316992013,5cyclictest3423355-21ntpq08:20:173
3397316992012,6cyclictest3420363-21apt-get08:15:013
339731699200,1cyclictest3056-21PK-Backend11:10:173
339731699200,19cyclictest3056-21PK-Backend10:18:333
339731699200,19cyclictest3056-21PK-Backend10:18:323
339731699200,19cyclictest0-21swapper/312:31:493
339731699200,19cyclictest0-21swapper/311:52:003
339731699200,19cyclictest0-21swapper/310:08:313
339731699200,17cyclictest3491605-21python311:35:243
339731499202,2cyclictest141rcu_preempt08:44:542
3397314992018,1cyclictest0-21swapper/211:25:502
3397314992018,1cyclictest0-21swapper/211:18:212
3397314992018,1cyclictest0-21swapper/211:02:592
3397314992018,1cyclictest0-21swapper/210:25:342
3397314992018,1cyclictest0-21swapper/210:18:532
3397314992018,1cyclictest0-21swapper/210:18:532
3397314992018,1cyclictest0-21swapper/208:25:302
3397314992018,1cyclictest0-21swapper/207:51:572
3397314992012,6cyclictest436-20systemd-journal10:35:002
3397314992012,6cyclictest3494254-21awk11:45:132
3397314992012,6cyclictest3408484-21apt07:40:112
339731499200,19cyclictest0-21swapper/211:39:162
339731499200,19cyclictest0-21swapper/209:35:122
339731499200,19cyclictest0-21swapper/207:48:182
3397309992018,1cyclictest3056-21PK-Backend12:29:121
3397309992018,1cyclictest0-21swapper/112:37:081
3397309992018,1cyclictest0-21swapper/112:33:101
3397309992018,1cyclictest0-21swapper/107:19:591
339730999200,1cyclictest3056-21PK-Backend09:54:451
339730999200,1cyclictest3056-21PK-Backend09:25:321
339730999200,19cyclictest3056-21PK-Backend08:22:591
339730999200,19cyclictest1389-21rtkit-daemon08:36:111
339730999200,19cyclictest0-21swapper/111:12:191
339730999200,19cyclictest0-21swapper/108:46:261
339730999200,18cyclictest3056-21PK-Backend11:00:561
339730999200,18cyclictest3056-21PK-Backend09:47:491
3397305992018,1cyclictest0-21swapper/008:27:480
3397305992017,2cyclictest0-21swapper/010:15:170
3397305992017,2cyclictest0-21swapper/010:15:160
339730599200,1cyclictest0-21swapper/008:09:090
339730599200,19cyclictest3056-21PK-Backend09:10:480
339730599200,19cyclictest3056-21PK-Backend07:55:580
339730599200,19cyclictest0-21swapper/012:20:490
339730599200,19cyclictest0-21swapper/012:14:400
339730599200,19cyclictest0-21swapper/012:08:460
339730599200,19cyclictest0-21swapper/011:44:190
339730599200,19cyclictest0-21swapper/011:26:000
339730599200,19cyclictest0-21swapper/011:17:560
339730599200,19cyclictest0-21swapper/010:57:210
339730599200,19cyclictest0-21swapper/010:05:230
339730599200,19cyclictest0-21swapper/009:50:290
339730599200,19cyclictest0-21swapper/009:48:060
339730599200,19cyclictest0-21swapper/009:28:110
339730599200,19cyclictest0-21swapper/008:19:250
339730599200,19cyclictest0-21swapper/007:38:270
339730599200,19cyclictest0-21swapper/007:12:530
139099200,13rtkit-daemon0-21swapper/208:58:522
34880162190,3sleep10-21swapper/111:25:201
3397316991917,1cyclictest3056-21PK-Backend08:37:193
3397316991917,1cyclictest0-21swapper/312:14:383
3397316991917,1cyclictest0-21swapper/312:03:363
3397316991917,1cyclictest0-21swapper/311:45:313
3397316991917,1cyclictest0-21swapper/309:45:003
3397316991912,5cyclictest42-21ksoftirqd/310:00:133
3397316991912,5cyclictest3469686-21apt10:35:133
339731699190,1cyclictest3056-21PK-Backend10:55:223
339731699190,1cyclictest0-21swapper/310:26:093
339731699190,1cyclictest0-21swapper/307:24:343
339731699190,18cyclictest0-21swapper/310:46:243
339731699190,18cyclictest0-21swapper/309:35:123
3397314991917,1cyclictest0-21swapper/208:33:142
3397314991917,1cyclictest0-21swapper/208:16:372
3397314991917,1cyclictest0-21swapper/207:56:102
3397314991914,3cyclictest3437964-21turbostat.cron09:05:012
339731499190,1cyclictest3056-21PK-Backend07:11:002
339731499190,1cyclictest0-21swapper/211:12:252
339731499190,1cyclictest0-21swapper/210:38:592
339731499190,1cyclictest0-21swapper/207:31:092
339731499190,18cyclictest0-21swapper/209:06:332
3397309991917,1cyclictest0-21swapper/108:34:421
3397309991916,2cyclictest3056-21PK-Backend11:16:381
3397309991916,2cyclictest3056-21PK-Backend08:15:221
339730999190,2cyclictest0-21swapper/109:40:231
339730999190,1cyclictest3056-21PK-Backend12:21:421
339730999190,1cyclictest3056-21PK-Backend12:06:401
339730999190,1cyclictest3056-21PK-Backend11:51:061
339730999190,1cyclictest3056-21PK-Backend11:42:131
339730999190,1cyclictest3056-21PK-Backend09:30:331
339730999190,1cyclictest0-21swapper/109:56:211
339730999190,1cyclictest0-21swapper/107:20:471
339730999190,18cyclictest0-21swapper/110:16:561
339730999190,18cyclictest0-21swapper/110:16:561
339730999190,18cyclictest0-21swapper/109:00:101
339730999190,18cyclictest0-21swapper/107:30:481
3397305991917,1cyclictest3056-21PK-Backend07:45:450
3397305991917,1cyclictest0-21swapper/011:10:050
3397305991917,1cyclictest0-21swapper/010:20:230
3397305991917,1cyclictest0-21swapper/009:59:480
3397305991917,1cyclictest0-21swapper/008:24:380
3397305991917,1cyclictest0-21swapper/008:11:470
3397305991917,1cyclictest0-21swapper/007:25:360
3397305991915,3cyclictest3431957-21tail08:45:190
3397305991912,6cyclictest883-21dbus-daemon12:35:010
3397305991912,5cyclictest3512748-21awk12:35:210
3397305991912,5cyclictest3442690-21cat09:15:220
3397305991911,6cyclictest3405826-21python307:30:210
339730599190,1cyclictest3056-21PK-Backend08:50:270
339730599190,1cyclictest0-21swapper/009:41:100
339730599190,1cyclictest0-21swapper/008:32:210
339730599190,18cyclictest0-21swapper/011:33:180
339730599190,18cyclictest0-21swapper/011:09:180
339730599190,18cyclictest0-21swapper/010:36:330
339730599190,18cyclictest0-21swapper/008:37:420
339730599190,18cyclictest0-21swapper/007:22:130
339731699189,3cyclictest0-21swapper/309:15:153
3397316991816,1cyclictest0-21swapper/307:55:363
3397316991815,2cyclictest3056-21PK-Backend11:06:323
3397316991812,5cyclictest3492770-21expr11:40:133
3397316991811,5cyclictest3484517-21ntpq11:15:183
339731699181,15cyclictest3416298-21python308:00:213
3397316991811,5cyclictest3411081-21swap07:45:203
3397314991816,1cyclictest0-21swapper/210:11:352
3397314991816,1cyclictest0-21swapper/209:29:112
3397314991816,1cyclictest0-21swapper/208:35:372
3397314991812,5cyclictest3444300-21awk09:20:222
3397314991812,5cyclictest3444300-21awk09:20:212
3397314991811,6cyclictest3434443-21sshd08:54:362
3397309991816,1cyclictest3056-21PK-Backend07:48:321
3397309991816,1cyclictest0-21swapper/110:00:211
3397309991811,5cyclictest3398089-21cat07:10:161
339730999180,1cyclictest3056-21PK-Backend11:59:581
339730999180,1cyclictest0-21swapper/110:56:041
3397305991816,1cyclictest0-21swapper/011:56:560
3397305991816,1cyclictest0-21swapper/011:53:110
3397305991816,1cyclictest0-21swapper/009:23:120
3397305991816,1cyclictest0-21swapper/009:23:110
3397305991816,1cyclictest0-21swapper/007:51:110
3397305991813,3cyclictest3438714-21kernelversion09:05:180
3397305991811,5cyclictest3435603-21awk08:55:200
339730599180,1cyclictest0-21swapper/007:44:440
3397316991711,5cyclictest3440877-21sed09:10:213
3397316991711,4cyclictest3475458-21load10:50:173
339731499179,6cyclictest1009-21lldpd07:29:242
3397314991712,3cyclictest3505680-21timerandwakeup12:15:222
3397314991711,4cyclictest34-21ksoftirqd/212:30:022
339730999178,7cyclictest3449205-21memory09:35:201
3397309991715,1cyclictest3056-21PK-Backend11:45:581
3397309991715,1cyclictest3056-21PK-Backend10:41:571
3397309991715,1cyclictest3056-21PK-Backend08:04:211
3397309991715,1cyclictest3056-21PK-Backend07:40:281
3397309991712,3cyclictest3419776-21switchtime08:10:231
3397309991711,5cyclictest3489625-21cat11:30:171
3397309991711,4cyclictest436-20systemd-journal11:07:371
3397305991715,1cyclictest0-21swapper/010:33:360
3397305991715,1cyclictest0-21swapper/009:30:480
3397305991710,6cyclictest3506162-21cron12:19:590
339731699160,15cyclictest3056-21PK-Backend08:27:543
3397314991614,1cyclictest0-21swapper/207:24:192
3397314991614,1cyclictest0-21swapper/207:19:182
3397309991614,1cyclictest3056-21PK-Backend12:13:221
3397309991611,3cyclictest3438200-21/usr/sbin/munin09:05:141
3397309991611,3cyclictest3424969-21missed_timers08:25:171
3397309991610,5cyclictest3056-21PK-Backend10:10:001
3397316991513,1cyclictest3056-21PK-Backend10:20:413
339731699150,1cyclictest3056-21PK-Backend08:30:033
339730599158,5cyclictest3399419-21expr07:15:020
3397305991513,1cyclictest0-21swapper/010:43:160
3397309991412,1cyclictest3056-21PK-Backend10:54:411
339730599148,5cyclictest0-21swapper/010:47:090
34436842130,3sleep13056-21PK-Backend09:20:121
34436842130,3sleep13056-21PK-Backend09:20:121
339730999137,5cyclictest3443167-21perf09:19:591
339730999137,4cyclictest3433641-21grep08:50:161
339731699129,2cyclictest3056-21PK-Backend07:34:243
339731699120,11cyclictest3056-21PK-Backend12:37:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional