You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-26 - 17:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System (updated Thu Mar 26, 2026 12:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25601722340,7sleep13056-21PK-Backend07:05:141
30003421460,4sleep33056-21PK-Backend09:10:013
31212421080,4sleep33056-21PK-Backend09:45:013
34826121030,4sleep20-21swapper/211:25:162
139099940,54rtkit-daemon0-21swapper/207:07:082
25706327939,13sleep00-21swapper/007:09:450
25682327940,14sleep30-21swapper/307:06:393
3028122500,4sleep33056-21PK-Backend09:15:223
3254192450,4sleep00-21swapper/010:20:180
139099400,4rtkit-daemon265426-21memory07:30:173
139099390,3rtkit-daemon0-21swapper/310:58:423
139099380,4rtkit-daemon3056-21PK-Backend12:00:183
139099370,3rtkit-daemon0-21swapper/308:41:193
139099370,3rtkit-daemon0-21swapper/308:04:423
139099370,2rtkit-daemon0-21swapper/310:49:563
2942782300,1sleep10-21swapper/108:50:221
257244992714,9cyclictest883-21dbus-daemon11:20:030
257255992616,6cyclictest268369-21apt07:40:113
257255992515,8cyclictest42-21ksoftirqd/307:54:593
25724999241,4cyclictest1317-21systemd09:46:461
257244992421,2cyclictest161rcuc/011:56:050
257244992421,2cyclictest161rcuc/009:43:450
257244992421,2cyclictest161rcuc/009:28:160
139099230,15rtkit-daemon0-21swapper/207:47:372
257255992214,6cyclictest318601-21python310:00:203
257249992220,1cyclictest3056-21PK-Backend08:07:031
257249992220,1cyclictest3056-21PK-Backend07:49:261
257249992220,1cyclictest0-21swapper/112:02:471
257249992220,1cyclictest0-21swapper/110:24:321
257249992220,1cyclictest0-21swapper/109:58:501
257249992220,1cyclictest0-21swapper/109:58:501
257249992220,1cyclictest0-21swapper/109:38:161
257249992215,5cyclictest26-21ksoftirqd/112:10:141
257249992211,9cyclictest312122-21cat09:45:011
25724999220,21cyclictest3056-21PK-Backend08:24:071
25724999220,20cyclictest3056-21PK-Backend10:51:501
25724999220,20cyclictest3056-21PK-Backend10:51:501
257244992220,1cyclictest0-21swapper/008:20:040
257244992219,2cyclictest161rcuc/011:07:340
257244992219,2cyclictest161rcuc/010:58:580
257244992219,2cyclictest161rcuc/009:17:110
257244992218,3cyclictest161rcuc/012:20:140
257244992215,4cyclictest320138-21strings10:05:170
257244992214,6cyclictest349842-21df11:30:130
139099220,6rtkit-daemon272925-21chrt07:50:282
139099220,15rtkit-daemon3056-21PK-Backend09:02:313
139099220,14rtkit-daemon3056-21PK-Backend08:58:193
2866602210,11sleep234-21ksoftirqd/208:30:182
257255992118,2cyclictest3056-21PK-Backend12:34:023
25725599210,19cyclictest3056-21PK-Backend12:10:153
257251992114,5cyclictest34-21ksoftirqd/211:40:012
25725199210,20cyclictest0-21swapper/212:01:362
25725199210,20cyclictest0-21swapper/210:23:082
257249992119,1cyclictest3056-21PK-Backend08:47:381
257249992119,1cyclictest3056-21PK-Backend08:31:581
257249992119,1cyclictest3056-21PK-Backend07:31:551
257249992119,1cyclictest0-21swapper/111:37:051
257249992119,1cyclictest0-21swapper/111:08:051
257249992119,1cyclictest0-21swapper/110:12:421
257249992119,1cyclictest0-21swapper/109:51:541
257249992119,1cyclictest0-21swapper/109:29:441
257249992119,1cyclictest0-21swapper/109:18:201
25724999210,20cyclictest3056-21PK-Backend10:57:311
25724999210,19cyclictest3056-21PK-Backend08:03:391
25724999210,19cyclictest3056-21PK-Backend07:42:011
25724999210,19cyclictest3056-21PK-Backend07:38:501
25724499216,7cyclictest13-21ksoftirqd/008:06:290
25724499213,8cyclictest13-21ksoftirqd/008:55:200
257244992119,1cyclictest0-21swapper/012:04:410
257244992119,1cyclictest0-21swapper/011:49:180
257244992119,1cyclictest0-21swapper/010:52:410
257244992119,1cyclictest0-21swapper/010:52:410
257244992119,1cyclictest0-21swapper/010:11:320
257244992119,1cyclictest0-21swapper/009:04:470
257244992119,1cyclictest0-21swapper/008:50:400
257244992119,1cyclictest0-21swapper/008:31:210
257244992119,1cyclictest0-21swapper/007:35:370
257244992119,1cyclictest0-21swapper/007:20:370
257244992119,1cyclictest0-21swapper/007:11:380
25724499211,7cyclictest13-21ksoftirqd/012:25:420
257244992117,2cyclictest352696-21basename11:40:000
25724499210,20cyclictest0-21swapper/012:13:450
25724499210,20cyclictest0-21swapper/011:13:500
25724499210,20cyclictest0-21swapper/010:43:560
25724499210,20cyclictest0-21swapper/010:28:150
25724499210,20cyclictest0-21swapper/009:38:530
25724499210,20cyclictest0-21swapper/008:41:480
25724499210,20cyclictest0-21swapper/008:03:360
25724499210,20cyclictest0-21swapper/007:27:060
25724499210,20cyclictest0-21swapper/007:18:260
139099210,15rtkit-daemon0-21swapper/309:35:443
139099210,14rtkit-daemon3056-21PK-Backend12:35:223
2621282200,3chrt0-21swapper/307:20:173
257255992018,1cyclictest0-21swapper/308:10:403
257255992013,6cyclictest259919-21expr07:15:133
257255992013,5cyclictest285866-21cut08:30:003
25725599200,1cyclictest3056-21PK-Backend10:05:173
25725599200,19cyclictest0-21swapper/308:15:133
25725599200,18cyclictest3056-21PK-Backend10:13:293
257251992018,1cyclictest0-21swapper/212:30:562
257251992018,1cyclictest0-21swapper/211:46:502
257251992018,1cyclictest0-21swapper/211:21:012
257251992018,1cyclictest0-21swapper/211:01:402
257251992018,1cyclictest0-21swapper/210:34:092
257251992018,1cyclictest0-21swapper/209:16:032
257251992018,1cyclictest0-21swapper/207:23:152
25725199200,1cyclictest0-21swapper/212:05:152
25725199200,19cyclictest0-21swapper/212:22:382
25725199200,19cyclictest0-21swapper/212:14:212
25725199200,19cyclictest0-21swapper/211:30:332
25725199200,19cyclictest0-21swapper/211:05:332
25725199200,19cyclictest0-21swapper/210:59:382
25725199200,19cyclictest0-21swapper/210:44:582
25725199200,19cyclictest0-21swapper/210:26:132
25725199200,19cyclictest0-21swapper/210:15:112
25725199200,19cyclictest0-21swapper/210:01:042
25725199200,19cyclictest0-21swapper/209:54:562
25725199200,19cyclictest0-21swapper/209:11:092
25725199200,19cyclictest0-21swapper/208:43:042
25725199200,19cyclictest0-21swapper/208:04:582
257249992018,1cyclictest3056-21PK-Backend08:25:561
257249992018,1cyclictest3056-21PK-Backend07:58:391
257249992018,1cyclictest0-21swapper/112:31:411
257249992018,1cyclictest0-21swapper/112:21:471
257249992018,1cyclictest0-21swapper/109:00:581
257249992017,2cyclictest3056-21PK-Backend07:22:151
257249992013,5cyclictest295124-21expr08:55:121
25724999200,1cyclictest0-21swapper/112:25:251
25724999200,1cyclictest0-21swapper/111:57:201
25724999200,19cyclictest3056-21PK-Backend08:10:271
25724999200,19cyclictest0-21swapper/111:53:421
25724999200,19cyclictest0-21swapper/111:41:441
25724999200,19cyclictest0-21swapper/111:34:141
25724999200,19cyclictest0-21swapper/110:31:291
25724499205,2cyclictest141rcu_preempt11:27:090
25724499204,6cyclictest13-21ksoftirqd/009:31:430
257244992018,1cyclictest0-21swapper/012:33:470
257244992018,1cyclictest0-21swapper/011:50:200
257244992018,1cyclictest0-21swapper/010:02:330
257244992018,1cyclictest0-21swapper/009:52:310
257244992018,1cyclictest0-21swapper/009:10:140
257244992018,1cyclictest0-21swapper/008:36:340
257244992017,2cyclictest161rcuc/009:21:510
257244992016,3cyclictest161rcuc/009:57:200
257244992016,3cyclictest161rcuc/009:57:190
25724499200,8cyclictest329231-21taskset10:30:250
25724499200,1cyclictest0-21swapper/012:08:190
25724499200,19cyclictest0-21swapper/012:35:360
25724499200,19cyclictest0-21swapper/011:00:260
25724499200,19cyclictest0-21swapper/010:36:120
25724499200,19cyclictest0-21swapper/007:55:110
139099200,14rtkit-daemon3056-21PK-Backend11:54:543
139099200,14rtkit-daemon3056-21PK-Backend11:35:233
25725599198,2cyclictest2673212ntpq07:35:193
257255991917,1cyclictest3056-21PK-Backend10:22:403
257255991912,5cyclictest436-20systemd-journal11:05:003
257255991912,5cyclictest42-21ksoftirqd/307:25:183
257255991912,5cyclictest292591-21sshd08:45:363
257251991917,1cyclictest3056-21PK-Backend09:35:222
257251991917,1cyclictest3056-21PK-Backend08:50:212
257251991917,1cyclictest0-21swapper/212:38:132
257251991917,1cyclictest0-21swapper/212:25:292
257251991917,1cyclictest0-21swapper/211:56:092
257251991917,1cyclictest0-21swapper/210:36:132
257251991917,1cyclictest0-21swapper/210:05:092
257251991917,1cyclictest0-21swapper/209:31:312
257251991917,1cyclictest0-21swapper/208:20:062
257251991917,1cyclictest0-21swapper/208:11:142
257251991917,1cyclictest0-21swapper/207:58:572
257251991917,1cyclictest0-21swapper/207:44:302
257251991911,6cyclictest285670-21cron08:30:002
25725199190,1cyclictest0-21swapper/210:50:532
25725199190,1cyclictest0-21swapper/210:50:532
25725199190,1cyclictest0-21swapper/210:12:072
25725199190,1cyclictest0-21swapper/209:56:592
25725199190,1cyclictest0-21swapper/209:56:592
25725199190,1cyclictest0-21swapper/209:05:412
25725199190,1cyclictest0-21swapper/207:26:112
25725199190,18cyclictest0-21swapper/209:02:042
25725199190,18cyclictest0-21swapper/208:56:362
25724999196,3cyclictest3467222ntpq11:20:181
257249991917,1cyclictest3056-21PK-Backend10:46:091
257249991917,1cyclictest3056-21PK-Backend10:42:161
257249991917,1cyclictest3056-21PK-Backend08:16:291
257249991917,1cyclictest3056-21PK-Backend07:53:381
257249991917,1cyclictest0-21swapper/112:37:551
257249991917,1cyclictest0-21swapper/111:11:571
257249991917,1cyclictest0-21swapper/109:22:121
257249991913,5cyclictest308578-21systemd-run09:35:011
25724999190,1cyclictest3056-21PK-Backend11:01:231
25724999190,1cyclictest3056-21PK-Backend10:37:221
25724999190,1cyclictest3056-21PK-Backend08:43:121
25724999190,1cyclictest3056-21PK-Backend07:11:201
25724999190,1cyclictest0-21swapper/112:09:021
25724999190,1cyclictest0-21swapper/110:25:341
25724999190,1cyclictest0-21swapper/109:11:391
25724999190,18cyclictest3056-21PK-Backend07:29:321
25724999190,18cyclictest0-21swapper/109:08:011
257244991917,1cyclictest0-21swapper/010:45:580
257244991917,1cyclictest0-21swapper/008:48:510
257244991917,1cyclictest0-21swapper/008:17:280
257244991910,7cyclictest345079-21users11:15:220
25724499190,18cyclictest3056-21PK-Backend07:50:580
25724499190,18cyclictest0-21swapper/009:06:360
139099190,15rtkit-daemon352688-21apt-get11:40:032
139099190,13rtkit-daemon3056-21PK-Backend08:54:453
139099190,13rtkit-daemon0-21swapper/308:21:563
257255991816,1cyclictest3056-21PK-Backend09:34:553
257255991815,2cyclictest3056-21PK-Backend12:23:193
257255991815,2cyclictest3056-21PK-Backend12:05:433
257255991815,2cyclictest3056-21PK-Backend11:58:473
257255991815,2cyclictest3056-21PK-Backend10:16:533
257255991814,3cyclictest42-21ksoftirqd/307:55:003
257255991812,5cyclictest287023-21awk08:30:223
257255991812,4cyclictest42-21ksoftirqd/308:35:173
257255991812,3cyclictest270099-21apt07:45:113
257255991811,5cyclictest42-21ksoftirqd/312:25:133
25725599180,1cyclictest0-21swapper/310:40:053
25725599180,16cyclictest329882-21/usr/sbin/munin10:35:113
257251991816,1cyclictest0-21swapper/211:11:282
257251991816,1cyclictest0-21swapper/209:42:042
257251991813,3cyclictest345470-21latency_hist11:20:012
257251991811,6cyclictest260686-21wc07:15:232
257251991810,6cyclictest281966-21cron08:17:002
25725199180,1cyclictest0-21swapper/209:22:452
25725199180,1cyclictest0-21swapper/208:49:202
257249991816,1cyclictest3056-21PK-Backend08:37:121
257249991816,1cyclictest0-21swapper/111:15:501
257249991816,1cyclictest0-21swapper/110:07:491
257249991811,5cyclictest323853-21ls10:15:211
257249991810,6cyclictest366221-21python312:15:211
25724499181,7cyclictest13-21ksoftirqd/007:45:120
257244991817,1cyclictest0-21swapper/012:16:360
257244991816,1cyclictest0-21swapper/011:41:000
257244991814,3cyclictest161rcuc/007:30:070
257244991812,5cyclictest312947-21uname09:45:190
257255991715,1cyclictest3056-21PK-Backend10:28:283
257255991711,4cyclictest335454-21seq10:50:133
257255991711,4cyclictest335454-21seq10:50:133
25725599170,15cyclictest277655-21df_inode08:05:123
257251991715,1cyclictest3056-21PK-Backend11:50:422
257251991715,1cyclictest0-21swapper/212:17:122
257251991715,1cyclictest0-21swapper/210:47:012
257251991715,1cyclictest0-21swapper/209:27:392
257251991712,3cyclictest1095-21runrttasks07:35:192
257251991711,5cyclictest265894-21cat07:30:202
25725199171,14cyclictest277457-21cpuspeed_turbos08:05:102
257251991711,4cyclictest257816-21forks07:10:142
25724999179,6cyclictest317211-21apt-get10:00:031
257244991714,2cyclictest161rcuc/008:25:550
257244991714,2cyclictest161rcuc/008:10:280
257244991713,3cyclictest268540-21cpuspeed_turbos07:40:110
257244991712,3cyclictest323132-21cstates10:15:120
25725599168,2cyclictest141rcu_preempt09:25:153
257255991614,1cyclictest3056-21PK-Backend09:20:363
25725599161,13cyclictest257400-21/usr/sbin/munin07:10:153
25725599160,7cyclictest328801-21head10:30:183
25725199169,5cyclictest288022-21expr08:35:122
257251991614,1cyclictest0-21swapper/209:45:572
257249991614,1cyclictest3056-21PK-Backend07:15:261
257249991610,5cyclictest356491-21wc11:50:011
25725599159,5cyclictest316786-21grep09:55:223
25725599159,5cyclictest316786-21grep09:55:223
25725599159,4cyclictest312957-21cut09:45:193
25725599153,11cyclictest0-21swapper/311:05:093
257255991510,4cyclictest3056-21PK-Backend09:50:193
25725599150,14cyclictest3056-21PK-Backend11:21:483
25725599150,14cyclictest3056-21PK-Backend11:10:413
25724999151,11cyclictest347736-21apt11:25:121
257255991412,1cyclictest3056-21PK-Backend11:25:483
257255991410,3cyclictest3056-21PK-Backend11:42:183
257255991410,3cyclictest3056-21PK-Backend09:10:133
25725599140,1cyclictest3056-21PK-Backend11:19:253
25725599139,3cyclictest3056-21PK-Backend12:16:463
257255991311,1cyclictest3056-21PK-Backend11:48:043
257255991311,1cyclictest3056-21PK-Backend11:30:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional