You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-06 - 03:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackcslot7s.osadl.org (updated Sat Jun 06, 2026 00:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
296401621180,18sleep2331rcuc/219:05:212
305237221150,4sleep20-21swapper/223:15:152
299849821080,2sleep03056-21PK-Backend20:40:200
30402072950,2sleep234-21ksoftirqd/222:40:142
139099790,69rtkit-daemon0-21swapper/319:09:323
296458627537,13sleep10-21swapper/119:06:501
296433427143,19sleep00-21swapper/019:05:280
30805402480,2sleep33056-21PK-Backend00:35:123
30687392430,5sleep30-21swapper/300:00:193
30162712370,2sleep13056-21PK-Backend21:30:211
30067502270,4sleep23006752-21cpuspeed_turbos21:05:122
30067502270,4sleep23006752-21cpuspeed_turbos21:05:112
139099260,4rtkit-daemon1389-21rtkit-daemon21:37:353
296496499250,23cyclictest2989037-21chrt20:15:122
139099250,4rtkit-daemon1389-21rtkit-daemon22:39:090
139099250,16rtkit-daemon3078501-21apt00:30:112
139099240,17rtkit-daemon0-21swapper/200:37:422
139099240,16rtkit-daemon3056-21PK-Backend21:00:571
139099240,15rtkit-daemon0-21swapper/222:35:232
2964964992315,7cyclictest2019-21gmain20:55:012
139099230,4rtkit-daemon1389-21rtkit-daemon23:35:520
139099230,4rtkit-daemon1389-21rtkit-daemon20:46:050
139099230,19rtkit-daemon1020-21ntpd20:07:042
139099230,18rtkit-daemon1020-21ntpd21:13:502
139099230,18rtkit-daemon1020-21ntpd21:13:502
139099230,16rtkit-daemon3056-21PK-Backend21:57:001
139099230,16rtkit-daemon3056-21PK-Backend21:40:521
139099230,16rtkit-daemon3056-21PK-Backend21:38:151
30300162220,3chrt0-21swapper/222:10:162
30300162220,3chrt0-21swapper/222:10:162
2964968992219,2cyclictest3056-21PK-Backend22:12:503
2964968992219,2cyclictest3056-21PK-Backend22:12:493
296496899220,20cyclictest3033811-21switchtime22:20:213
2964960992220,1cyclictest0-21swapper/100:34:231
2964960992219,2cyclictest3056-21PK-Backend19:47:081
2964960992219,2cyclictest3056-21PK-Backend00:29:301
2964960992219,2cyclictest3056-21PK-Backend00:13:421
2964960992218,3cyclictest0-21swapper/122:50:191
139099220,4rtkit-daemon1389-21rtkit-daemon22:58:320
139099220,4rtkit-daemon1389-21rtkit-daemon22:23:160
139099220,16rtkit-daemon3056-21PK-Backend22:29:303
139099220,16rtkit-daemon3056-21PK-Backend20:22:041
139099220,15rtkit-daemon0-21swapper/120:25:241
296496899219,4cyclictest42-21ksoftirqd/320:17:283
296496899217,8cyclictest42-21ksoftirqd/322:35:133
2964968992119,1cyclictest3056-21PK-Backend23:21:323
2964968992119,1cyclictest3056-21PK-Backend22:34:343
2964968992119,1cyclictest0-21swapper/323:50:043
2964968992119,1cyclictest0-21swapper/322:59:213
2964968992119,1cyclictest0-21swapper/321:52:213
2964968992119,1cyclictest0-21swapper/321:31:543
2964968992119,1cyclictest0-21swapper/321:05:243
2964968992119,1cyclictest0-21swapper/321:05:243
2964968992119,1cyclictest0-21swapper/321:01:453
2964968992119,1cyclictest0-21swapper/320:07:113
2964968992119,1cyclictest0-21swapper/300:08:493
2964968992118,2cyclictest3056-21PK-Backend23:17:403
2964968992117,3cyclictest3024582-21df_inode21:55:133
2964968992116,4cyclictest42-21ksoftirqd/321:45:013
2964968992116,3cyclictest42-21ksoftirqd/321:20:003
2964968992116,3cyclictest2156-21gnome-shell00:12:093
296496899210,20cyclictest0-21swapper/323:55:593
296496899210,20cyclictest0-21swapper/320:11:513
296496899210,20cyclictest0-21swapper/319:35:273
296496899210,19cyclictest3056-21PK-Backend23:10:573
296496899210,19cyclictest3056-21PK-Backend19:30:333
296496899210,19cyclictest3056-21PK-Backend19:20:133
296496499219,4cyclictest34-21ksoftirqd/200:10:002
2964964992118,2cyclictest0-21swapper/222:48:422
2964964992113,6cyclictest436-20systemd-journal21:20:002
2964964992113,6cyclictest3020878-21fw_packets21:45:162
296496099214,15cyclictest2966618-21cron19:15:001
2964960992119,1cyclictest0-21swapper/123:22:511
2964960992119,1cyclictest0-21swapper/123:14:061
2964960992119,1cyclictest0-21swapper/122:28:231
2964960992119,1cyclictest0-21swapper/120:51:511
2964960992119,1cyclictest0-21swapper/119:41:271
2964960992119,1cyclictest0-21swapper/119:39:251
2964960992118,2cyclictest3056-21PK-Backend23:54:281
2964960992118,2cyclictest3056-21PK-Backend22:57:511
2964960992118,2cyclictest3056-21PK-Backend21:23:071
2964960992117,2cyclictest2971124-21apt19:25:121
2964960992115,4cyclictest3041667-21tr22:45:121
2964960992112,7cyclictest3003125-21apt20:55:121
296496099210,20cyclictest3056-21PK-Backend20:46:021
296496099210,20cyclictest0-21swapper/123:06:221
296496099210,20cyclictest0-21swapper/122:42:361
296496099210,20cyclictest0-21swapper/122:06:051
296496099210,1cyclictest0-21swapper/119:16:401
296496099210,19cyclictest3056-21PK-Backend20:39:141
296496099210,19cyclictest3056-21PK-Backend20:15:061
2964957992113,6cyclictest3028414-21ntp_states22:05:190
296495799210,6cyclictest436-20systemd-journal00:34:590
296495799210,20cyclictest0-21swapper/023:00:190
296495799210,20cyclictest0-21swapper/021:59:020
296495799210,20cyclictest0-21swapper/020:37:500
296495799210,20cyclictest0-21swapper/020:01:200
296495799210,20cyclictest0-21swapper/019:28:010
296495799210,1cyclictest0-21swapper/020:50:210
139099210,17rtkit-daemon1020-21ntpd19:52:342
139099210,16rtkit-daemon3056-21PK-Backend20:06:361
139099210,15rtkit-daemon3056-21PK-Backend21:13:071
139099210,15rtkit-daemon3056-21PK-Backend21:13:061
139099210,15rtkit-daemon3056-21PK-Backend20:55:293
139099210,15rtkit-daemon3056-21PK-Backend20:53:213
2964968992018,1cyclictest3056-21PK-Backend19:15:333
2964968992018,1cyclictest0-21swapper/323:36:113
2964968992018,1cyclictest0-21swapper/320:22:463
296496899201,7cyclictest2998230-21memory20:40:173
296496899201,7cyclictest2996763-21/usr/sbin/munin20:35:203
2964968992014,5cyclictest3079247-21ntp_states00:30:183
2964968992013,5cyclictest436-20systemd-journal23:48:153
2964968992012,6cyclictest3020877-21awk21:45:143
2964968992012,2cyclictest42-21ksoftirqd/322:45:193
296496899200,1cyclictest3056-21PK-Backend23:07:043
296496899200,1cyclictest0-21swapper/323:30:163
296496899200,19cyclictest3056-21PK-Backend23:03:133
296496899200,19cyclictest3056-21PK-Backend22:17:303
296496899200,19cyclictest3056-21PK-Backend22:06:223
296496899200,19cyclictest3056-21PK-Backend19:29:333
296496899200,19cyclictest0-21swapper/323:41:053
296496899200,19cyclictest0-21swapper/321:11:403
296496899200,19cyclictest0-21swapper/321:11:403
296496899200,19cyclictest0-21swapper/320:46:523
296496899200,19cyclictest0-21swapper/319:49:203
2964964992018,1cyclictest0-21swapper/223:45:272
2964964992018,1cyclictest0-21swapper/223:38:172
2964964992018,1cyclictest0-21swapper/223:22:222
2964964992018,1cyclictest0-21swapper/222:50:452
2964964992018,1cyclictest0-21swapper/222:19:352
2964964992018,1cyclictest0-21swapper/221:39:482
2964964992018,1cyclictest0-21swapper/221:20:212
2964964992018,1cyclictest0-21swapper/220:48:382
2964964992018,1cyclictest0-21swapper/220:36:402
2964964992018,1cyclictest0-21swapper/219:31:522
2964964992018,1cyclictest0-21swapper/219:10:292
2964964992018,1cyclictest0-21swapper/200:12:232
2964964992017,2cyclictest0-21swapper/220:04:372
2964964992016,3cyclictest34-21ksoftirqd/219:20:002
2964964992014,5cyclictest3005227-21diskstats21:00:132
2964964992012,6cyclictest3075767-21ntp_states00:20:172
2964964992012,6cyclictest2998376-21ntpq20:40:182
296496499201,1cyclictest0-21swapper/223:06:272
2964964992011,7cyclictest3025284-21users21:55:212
2964964992011,7cyclictest3014605-21users21:25:222
296496499200,1cyclictest3056-21PK-Backend19:20:162
296496499200,1cyclictest0-21swapper/223:00:322
296496499200,19cyclictest0-21swapper/223:40:032
296496499200,19cyclictest0-21swapper/223:13:222
296496499200,19cyclictest0-21swapper/222:55:392
296496499200,19cyclictest0-21swapper/222:27:202
296496499200,19cyclictest0-21swapper/220:32:492
296496499200,19cyclictest0-21swapper/220:11:322
296496499200,19cyclictest0-21swapper/219:56:122
296496099208,10cyclictest2970882-21unin-run19:25:001
2964960992018,1cyclictest3056-21PK-Backend21:09:281
2964960992018,1cyclictest3056-21PK-Backend21:09:281
2964960992018,1cyclictest3056-21PK-Backend20:31:231
2964960992018,1cyclictest3056-21PK-Backend00:00:571
2964960992018,1cyclictest0-21swapper/123:02:301
2964960992018,1cyclictest0-21swapper/122:18:161
2964960992018,1cyclictest0-21swapper/119:31:541
2964960992017,2cyclictest3056-21PK-Backend22:22:561
2964960992013,5cyclictest3037054-21ntpq22:30:191
2964960992012,6cyclictest2988708-21cron20:15:001
296496099200,19cyclictest3056-21PK-Backend22:04:171
296496099200,19cyclictest0-21swapper/123:15:081
2964957992018,1cyclictest0-21swapper/023:29:120
2964957992018,1cyclictest0-21swapper/023:07:020
2964957992018,1cyclictest0-21swapper/022:25:110
2964957992018,1cyclictest0-21swapper/022:01:530
2964957992018,1cyclictest0-21swapper/020:19:250
2964957992018,1cyclictest0-21swapper/020:05:400
2964957992018,1cyclictest0-21swapper/019:37:480
2964957992015,3cyclictest3032599-21sh22:20:000
296495799200,1cyclictest0-21swapper/021:42:260
296495799200,1cyclictest0-21swapper/021:21:230
296495799200,19cyclictest0-21swapper/023:55:410
296495799200,19cyclictest0-21swapper/023:10:540
296495799200,19cyclictest0-21swapper/022:10:110
296495799200,19cyclictest0-21swapper/022:10:100
296495799200,19cyclictest0-21swapper/021:17:440
296495799200,19cyclictest0-21swapper/000:17:190
139099200,3rtkit-daemon1389-21rtkit-daemon23:40:040
139099200,14rtkit-daemon3056-21PK-Backend21:26:381
139099200,13rtkit-daemon3056-21PK-Backend21:16:071
139099200,13rtkit-daemon0-21swapper/020:12:390
139099200,13rtkit-daemon0-21swapper/019:40:060
2964968991917,1cyclictest3056-21PK-Backend22:44:073
2964968991917,1cyclictest3056-21PK-Backend19:10:403
296496899191,5cyclictest2994548-21df20:30:113
2964968991914,3cyclictest2982165-21diskstats19:55:133
2964968991913,4cyclictest3073485-21/usr/sbin/munin00:15:163
2964968991912,6cyclictest42-21ksoftirqd/300:24:053
296496899190,7cyclictest2981134-21/usr/sbin/munin19:50:223
296496899190,1cyclictest3056-21PK-Backend19:40:213
296496899190,1cyclictest0-21swapper/323:28:143
296496899190,18cyclictest0-21swapper/320:00:483
296496499196,4cyclictest34-21ksoftirqd/220:55:132
2964964991917,1cyclictest0-21swapper/223:25:262
2964964991917,1cyclictest0-21swapper/222:00:302
2964964991917,1cyclictest0-21swapper/221:40:492
2964964991917,1cyclictest0-21swapper/220:28:092
2964964991917,1cyclictest0-21swapper/219:38:482
2964964991915,3cyclictest978-21snmpd20:23:262
2964964991912,6cyclictest3058623-21basename23:35:002
2964964991912,5cyclictest436-20systemd-journal19:49:292
2964964991912,5cyclictest3073342-21cat00:15:122
2964964991912,5cyclictest3022675-21cat21:50:122
296496499190,1cyclictest0-21swapper/219:29:492
296496499190,18cyclictest3056-21PK-Backend22:20:372
296496499190,18cyclictest0-21swapper/223:59:202
296496499190,18cyclictest0-21swapper/223:52:232
2964960991917,1cyclictest3056-21PK-Backend23:29:341
2964960991917,1cyclictest3056-21PK-Backend21:45:241
2964960991917,1cyclictest3056-21PK-Backend19:53:511
2964960991917,1cyclictest3056-21PK-Backend00:21:111
296496099190,1cyclictest3056-21PK-Backend23:37:521
296496099190,1cyclictest3056-21PK-Backend19:56:421
296496099190,1cyclictest0-21swapper/122:12:351
296496099190,1cyclictest0-21swapper/122:12:351
2964957991917,1cyclictest3056-21PK-Backend23:45:560
2964957991917,1cyclictest0-21swapper/023:54:410
2964957991917,1cyclictest0-21swapper/021:38:480
2964957991917,1cyclictest0-21swapper/021:05:150
2964957991917,1cyclictest0-21swapper/021:05:150
2964957991917,1cyclictest0-21swapper/020:24:530
2964957991917,1cyclictest0-21swapper/000:04:270
2964957991912,5cyclictest3051945-21apt23:15:120
2964957991911,6cyclictest3054544-21python323:20:220
296495799190,1cyclictest3056-21PK-Backend00:20:230
296495799190,1cyclictest0-21swapper/023:30:130
296495799190,1cyclictest0-21swapper/021:26:510
296495799190,1cyclictest0-21swapper/019:20:310
296495799190,1cyclictest0-21swapper/019:17:400
296495799190,1cyclictest0-21swapper/000:13:130
296495799190,18cyclictest0-21swapper/022:52:360
296495799190,18cyclictest0-21swapper/022:48:440
296495799190,18cyclictest0-21swapper/022:43:020
296495799190,18cyclictest0-21swapper/020:33:110
139099190,13rtkit-daemon3056-21PK-Backend21:50:411
296496899186,3cyclictest141rcu_preempt21:24:593
2964968991812,4cyclictest2993311-21python320:25:203
2964968991811,2cyclictest42-21ksoftirqd/322:00:313
2964968991811,2cyclictest42-21ksoftirqd/300:26:543
296496899180,1cyclictest0-21swapper/321:27:133
2964964991816,1cyclictest0-21swapper/222:32:132
2964964991816,1cyclictest0-21swapper/221:33:182
2964964991814,3cyclictest3068536-21netstat00:00:172
2964964991810,6cyclictest3076768-21apt00:25:122
2964960991816,1cyclictest3056-21PK-Backend23:55:301
2964960991816,1cyclictest3056-21PK-Backend23:30:361
2964960991816,1cyclictest3056-21PK-Backend00:08:141
2964960991816,1cyclictest0-21swapper/100:38:291
296496099180,17cyclictest3056-21PK-Backend00:18:201
2964957991816,1cyclictest0-21swapper/021:45:150
2964957991816,1cyclictest0-21swapper/019:55:110
2964957991816,1cyclictest0-21swapper/000:36:320
2964957991811,5cyclictest436-20systemd-journal19:15:000
2964957991811,5cyclictest3015280-21awk21:30:110
2964968991711,2cyclictest42-21ksoftirqd/322:53:203
2964964991715,1cyclictest0-21swapper/222:09:012
2964964991715,1cyclictest0-21swapper/219:41:522
2964960991710,5cyclictest3038827-21ntpq22:35:201
296495799179,6cyclictest2979531-21apt-get19:50:000
2964957991715,1cyclictest0-21swapper/021:00:490
2964957991715,1cyclictest0-21swapper/000:06:310
2964957991710,6cyclictest3037480-21seq22:30:580
2964957991710,5cyclictest2972991-21grep19:30:130
2964960991614,1cyclictest3056-21PK-Backend20:02:371
2964960991614,1cyclictest0-21swapper/123:45:221
2964960991614,1cyclictest0-21swapper/120:40:151
2964960991611,3cyclictest3061588-21timerandwakeup23:40:201
2964957991614,1cyclictest0-21swapper/021:13:190
2964957991614,1cyclictest0-21swapper/021:13:190
2964957991614,1cyclictest0-21swapper/020:58:580
2964957991614,1cyclictest0-21swapper/020:25:530
2964957991610,5cyclictest2980971-21grep19:50:210
296495799158,5cyclictest3077262-21sed00:25:160
296495799150,13cyclictest3022622-21tr21:50:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional