You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-01 - 07:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Fri May 01, 2026 00:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
366768521190,3sleep30-21swapper/323:55:133
139099850,51rtkit-daemon0-21swapper/019:07:190
139099710,58rtkit-daemon0-21swapper/119:09:511
356686426844,16sleep30-21swapper/319:08:233
356684826540,19sleep20-21swapper/219:08:092
36574892500,3sleep126-21ksoftirqd/123:25:151
36422812500,4sleep13056-21PK-Backend22:40:201
35945742490,2sleep20-21swapper/220:25:122
36485872460,4chrt0-21swapper/123:00:131
35817042460,3sleep33581705-21idleruntime-cro19:50:003
35909362440,7sleep1261ksoftirqd/120:15:121
139099300,4rtkit-daemon1389-21rtkit-daemon20:09:282
139099270,4rtkit-daemon1389-21rtkit-daemon20:29:183
139099260,4rtkit-daemon1389-21rtkit-daemon21:09:572
139099260,4rtkit-daemon1389-21rtkit-daemon20:59:492
139099260,4rtkit-daemon1389-21rtkit-daemon20:43:072
36603992250,4sleep00-21swapper/023:35:010
35828392250,4sleep33582840-21switchtime19:50:203
139099250,3rtkit-daemon1389-21rtkit-daemon21:02:372
36090092240,4chrt3056-21PK-Backend21:05:211
356713999242,18cyclictest0-21swapper/300:30:123
3567139992417,5cyclictest3669180-21apt00:00:123
3567139992318,3cyclictest3633566-21users22:15:253
3567139992317,4cyclictest3674767-21ls00:20:003
3567135992315,6cyclictest3655002-21cron23:20:002
139099230,4rtkit-daemon1389-21rtkit-daemon21:10:182
139099230,16rtkit-daemon3056-21PK-Backend23:22:141
139099230,15rtkit-daemon3056-21PK-Backend21:59:131
3567139992220,1cyclictest0-21swapper/323:42:243
3567139992220,1cyclictest0-21swapper/320:48:593
3567139992220,1cyclictest0-21swapper/319:55:053
3567139992219,2cyclictest0-21swapper/321:32:263
356713999221,7cyclictest3641162-21perf22:40:003
356713999220,3cyclictest1788620-21WebKitWebProces22:20:453
3567135992214,6cyclictest1-21systemd20:05:002
356713299220,20cyclictest3056-21PK-Backend21:50:251
356713299220,20cyclictest0-21swapper/123:05:151
3567130992220,1cyclictest0-21swapper/021:27:500
3567130992219,2cyclictest0-21swapper/023:07:270
3567130992219,2cyclictest0-21swapper/021:07:220
3567130992219,2cyclictest0-21swapper/000:05:410
3567130992214,6cyclictest3649819-21/usr/sbin/munin23:05:010
3567130992214,6cyclictest13-21ksoftirqd/021:30:390
356713099220,20cyclictest0-21swapper/020:20:580
139099220,19rtkit-daemon3194-21gmain21:17:542
139099220,15rtkit-daemon3056-21PK-Backend22:29:191
139099220,15rtkit-daemon3056-21PK-Backend00:36:351
139099220,14rtkit-daemon3056-21PK-Backend00:11:471
3567139992119,1cyclictest3056-21PK-Backend20:34:543
3567139992119,1cyclictest0-21swapper/323:54:343
3567139992119,1cyclictest0-21swapper/322:13:013
3567139992119,1cyclictest0-21swapper/321:37:533
3567139992119,1cyclictest0-21swapper/320:42:373
3567139992119,1cyclictest0-21swapper/319:27:003
3567139992119,1cyclictest0-21swapper/319:27:003
3567139992119,1cyclictest0-21swapper/300:39:573
3567139992119,1cyclictest0-21swapper/300:24:423
3567139992118,2cyclictest3056-21PK-Backend19:12:063
3567139992113,6cyclictest3576219-21cron19:35:013
3567139992112,7cyclictest436-20systemd-journal22:05:003
3567139992111,6cyclictest0-21swapper/322:25:123
356713999210,20cyclictest3056-21PK-Backend20:35:413
356713999210,20cyclictest0-21swapper/323:24:473
356713999210,20cyclictest0-21swapper/321:53:013
356713999210,20cyclictest0-21swapper/320:23:383
356713999210,20cyclictest0-21swapper/320:09:193
356713599215,14cyclictest3653266-21cron23:15:002
356713599210,19cyclictest0-21swapper/223:53:422
356713299210,20cyclictest3056-21PK-Backend00:29:101
356713299210,20cyclictest0-21swapper/123:36:241
356713299210,20cyclictest0-21swapper/120:36:101
356713299210,19cyclictest3056-21PK-Backend21:49:391
356713299210,19cyclictest3056-21PK-Backend21:35:451
356713299210,19cyclictest3056-21PK-Backend20:51:051
356713299210,19cyclictest3056-21PK-Backend20:14:201
356713299210,19cyclictest3056-21PK-Backend00:02:341
3567130992119,1cyclictest0-21swapper/023:17:340
3567130992119,1cyclictest0-21swapper/022:20:030
3567130992119,1cyclictest0-21swapper/019:25:430
3567130992119,1cyclictest0-21swapper/019:25:420
3567130992119,1cyclictest0-21swapper/000:38:260
3567130992119,1cyclictest0-21swapper/000:15:270
3567130992118,2cyclictest3056-21PK-Backend19:20:560
356713099210,20cyclictest0-21swapper/021:36:020
356713099210,20cyclictest0-21swapper/021:01:410
356713099210,20cyclictest0-21swapper/020:56:480
356713099210,20cyclictest0-21swapper/020:51:410
356713099210,20cyclictest0-21swapper/000:28:520
356713099210,20cyclictest0-21swapper/000:03:380
356713099210,19cyclictest0-21swapper/021:50:080
139099210,3rtkit-daemon1389-21rtkit-daemon19:51:182
139099210,14rtkit-daemon3056-21PK-Backend23:55:331
139099210,14rtkit-daemon3056-21PK-Backend23:50:410
139099210,14rtkit-daemon3056-21PK-Backend22:18:231
36277282200,3sleep20-21swapper/222:00:152
3567139992018,1cyclictest3056-21PK-Backend23:34:063
3567139992018,1cyclictest0-21swapper/323:47:523
3567139992018,1cyclictest0-21swapper/322:42:153
3567139992018,1cyclictest0-21swapper/322:32:493
3567139992018,1cyclictest0-21swapper/322:05:173
3567139992018,1cyclictest0-21swapper/321:45:303
3567139992018,1cyclictest0-21swapper/321:22:253
3567139992018,1cyclictest0-21swapper/321:04:213
3567139992018,1cyclictest0-21swapper/320:01:353
3567139992018,1cyclictest0-21swapper/319:16:393
3567139992018,1cyclictest0-21swapper/300:14:083
3567139992017,2cyclictest3056-21PK-Backend23:13:383
3567139992014,4cyclictest3589254-21df20:10:163
356713999200,19cyclictest3056-21PK-Backend23:03:443
356713999200,19cyclictest0-21swapper/323:25:483
356713999200,19cyclictest0-21swapper/300:05:223
356713599209,9cyclictest3589503-21missed_timers20:10:212
3567135992018,1cyclictest0-21swapper/220:54:222
3567135992018,1cyclictest0-21swapper/220:45:362
3567135992018,1cyclictest0-21swapper/219:58:042
356713599200,19cyclictest0-21swapper/223:55:442
356713599200,19cyclictest0-21swapper/222:32:162
356713599200,19cyclictest0-21swapper/222:15:402
356713599200,19cyclictest0-21swapper/221:57:012
356713599200,19cyclictest0-21swapper/221:50:532
356713599200,19cyclictest0-21swapper/221:49:172
356713599200,19cyclictest0-21swapper/220:23:122
356713599200,19cyclictest0-21swapper/219:41:292
356713599200,19cyclictest0-21swapper/219:26:072
356713599200,19cyclictest0-21swapper/219:26:072
356713599200,18cyclictest0-21swapper/223:45:102
356713599200,18cyclictest0-21swapper/223:29:222
3567132992018,1cyclictest3056-21PK-Backend00:30:121
3567132992018,1cyclictest0-21swapper/121:25:241
3567132992018,1cyclictest0-21swapper/120:20:021
3567132992018,1cyclictest0-21swapper/119:10:461
3567132992017,2cyclictest3056-21PK-Backend20:26:301
3567132992017,2cyclictest3056-21PK-Backend00:09:161
3567132992013,5cyclictest436-20systemd-journal19:54:561
356713299200,1cyclictest3056-21PK-Backend21:24:371
356713299200,1cyclictest3056-21PK-Backend21:19:301
356713299200,19cyclictest3056-21PK-Backend23:53:481
356713299200,19cyclictest3056-21PK-Backend21:33:011
356713299200,19cyclictest3056-21PK-Backend20:45:291
356713299200,19cyclictest3056-21PK-Backend19:29:261
356713299200,19cyclictest3056-21PK-Backend19:29:251
356713299200,19cyclictest0-21swapper/123:17:591
356713299200,19cyclictest0-21swapper/121:10:511
3567130992018,1cyclictest0-21swapper/023:48:580
3567130992018,1cyclictest0-21swapper/023:41:410
3567130992018,1cyclictest0-21swapper/022:53:080
3567130992018,1cyclictest0-21swapper/021:24:060
3567130992018,1cyclictest0-21swapper/020:33:360
3567130992018,1cyclictest0-21swapper/020:29:040
3567130992018,1cyclictest0-21swapper/020:12:400
3567130992018,1cyclictest0-21swapper/000:10:340
3567130992016,3cyclictest1095-21runrttasks23:10:060
356713099201,4cyclictest3630500-21munin-plugin-st22:10:000
356713099200,1cyclictest0-21swapper/019:46:170
356713099200,19cyclictest0-21swapper/023:20:250
356713099200,19cyclictest0-21swapper/022:10:430
356713099200,19cyclictest0-21swapper/021:55:280
356713099200,19cyclictest0-21swapper/021:41:290
356713099200,19cyclictest0-21swapper/019:55:230
356713099200,19cyclictest0-21swapper/000:32:440
139099200,15rtkit-daemon3056-21PK-Backend00:22:411
139099200,14rtkit-daemon3056-21PK-Backend22:49:051
139099200,14rtkit-daemon3056-21PK-Backend19:16:431
36037542190,3chrt0-21swapper/320:50:193
3567139991917,1cyclictest3056-21PK-Backend23:07:233
3567139991917,1cyclictest3056-21PK-Backend22:57:303
3567139991917,1cyclictest3056-21PK-Backend21:10:493
3567139991917,1cyclictest0-21swapper/321:59:093
356713999190,1cyclictest3056-21PK-Backend21:29:433
356713999190,18cyclictest0-21swapper/323:39:343
3567135991917,1cyclictest0-21swapper/222:56:092
3567135991917,1cyclictest0-21swapper/222:38:192
3567135991917,1cyclictest0-21swapper/222:25:272
3567135991917,1cyclictest0-21swapper/221:44:102
3567135991917,1cyclictest0-21swapper/220:38:272
3567135991917,1cyclictest0-21swapper/200:36:412
3567135991917,1cyclictest0-21swapper/200:28:572
3567135991917,1cyclictest0-21swapper/200:22:142
3567135991917,1cyclictest0-21swapper/200:07:342
3567135991914,4cyclictest1095-21runrttasks22:20:202
3567135991913,4cyclictest3642361-21threads22:40:212
3567135991911,6cyclictest3572030-21ntp_states19:20:182
356713599190,1cyclictest3056-21PK-Backend23:20:022
356713599190,1cyclictest0-21swapper/223:36:522
356713599190,1cyclictest0-21swapper/223:32:122
356713599190,1cyclictest0-21swapper/222:53:322
356713599190,1cyclictest0-21swapper/221:29:442
356713599190,1cyclictest0-21swapper/219:12:012
356713599190,18cyclictest0-21swapper/222:45:072
356713599190,18cyclictest0-21swapper/222:14:052
356713599190,18cyclictest0-21swapper/221:24:372
356713599190,18cyclictest0-21swapper/220:30:432
356713599190,18cyclictest0-21swapper/219:38:382
356713599190,18cyclictest0-21swapper/219:16:352
356713599190,18cyclictest0-21swapper/200:01:392
3567132991917,1cyclictest3056-21PK-Backend23:45:301
3567132991912,6cyclictest3584678-21sed19:55:211
356713299190,1cyclictest3056-21PK-Backend23:41:521
356713299190,1cyclictest3056-21PK-Backend22:20:061
356713299190,1cyclictest3056-21PK-Backend20:43:061
356713299190,1cyclictest3056-21PK-Backend19:45:351
356713299190,1cyclictest0-21swapper/123:30:571
356713299190,1cyclictest0-21swapper/122:51:571
356713299190,18cyclictest3056-21PK-Backend22:55:211
356713299190,18cyclictest0-21swapper/123:13:191
3567130991917,1cyclictest0-21swapper/022:00:490
3567130991917,1cyclictest0-21swapper/020:01:040
3567130991917,1cyclictest0-21swapper/019:33:330
356713099190,1cyclictest3056-21PK-Backend20:38:560
356713099190,1cyclictest0-21swapper/023:38:500
356713099190,1cyclictest0-21swapper/023:28:430
356713099190,1cyclictest0-21swapper/021:16:010
356713099190,1cyclictest0-21swapper/020:06:460
356713099190,1cyclictest0-21swapper/019:40:500
356713099190,1cyclictest0-21swapper/019:17:450
356713099190,18cyclictest3056-21PK-Backend19:35:220
139099190,13rtkit-daemon3056-21PK-Backend19:39:531
139099190,13rtkit-daemon0-21swapper/000:23:300
3567139991816,1cyclictest0-21swapper/321:15:083
3567139991816,1cyclictest0-21swapper/319:35:043
3567139991814,3cyclictest3620494-21tr21:40:113
3567139991814,3cyclictest3590982-21df20:15:123
3567139991812,5cyclictest3676995-21expr00:25:163
356713999181,15cyclictest3643524-21df_inode22:45:143
356713999180,5cyclictest3604735-21acpi20:55:113
356713999180,16cyclictest3608753-21latency21:05:183
3567135991816,1cyclictest0-21swapper/223:06:172
3567135991816,1cyclictest0-21swapper/220:15:492
3567135991816,1cyclictest0-21swapper/219:48:452
3567135991816,1cyclictest0-21swapper/219:33:102
3567135991811,5cyclictest436-20systemd-journal00:15:422
356713299183,6cyclictest26-21ksoftirqd/119:45:011
3567132991816,1cyclictest3056-21PK-Backend22:10:131
356713299180,1cyclictest3056-21PK-Backend20:00:551
356713299180,1cyclictest0-21swapper/120:31:371
3567130991816,1cyclictest0-21swapper/023:55:540
3567130991816,1cyclictest0-21swapper/022:25:020
3567130991816,1cyclictest0-21swapper/022:18:270
3567130991816,1cyclictest0-21swapper/021:46:220
3567130991816,1cyclictest0-21swapper/021:13:510
3567130991810,6cyclictest3647704-21awk22:55:230
356713999179,6cyclictest3571647-21df19:20:123
3567139991715,1cyclictest3056-21PK-Backend22:54:523
3567135991715,1cyclictest0-21swapper/223:41:322
3567135991715,1cyclictest0-21swapper/221:30:322
3567135991710,5cyclictest3629097-21cat22:05:122
3567132991715,1cyclictest3056-21PK-Backend22:37:441
3567132991715,1cyclictest3056-21PK-Backend21:03:481
3567132991715,1cyclictest3056-21PK-Backend20:07:371
3567132991715,1cyclictest3056-21PK-Backend19:33:511
3567130991715,1cyclictest0-21swapper/022:48:550
3567130991715,1cyclictest0-21swapper/022:40:300
3567130991715,1cyclictest0-21swapper/022:37:530
3567130991715,1cyclictest0-21swapper/020:45:190
3567130991715,1cyclictest0-21swapper/019:51:450
3567130991710,5cyclictest3568383-21cat19:10:220
139099170,3rtkit-daemon0-21swapper/022:30:400
3567139991614,1cyclictest0-21swapper/323:18:183
356713599169,6cyclictest3679220-21ntpq00:30:192
3567135991610,5cyclictest3673166-21sed00:15:002
3567132991614,1cyclictest3056-21PK-Backend22:33:311
3567132991614,1cyclictest3056-21PK-Backend22:08:371
3567132991614,1cyclictest3056-21PK-Backend21:41:131
3567130991614,1cyclictest0-21swapper/020:42:070
3567130991611,3cyclictest3590763-21cpuspeed_turbos20:15:120
356713999157,6cyclictest978-21snmpd19:40:113
3567135991513,1cyclictest0-21swapper/223:00:492
356713299158,5cyclictest3571987-21awk19:20:171
3567132991513,1cyclictest3056-21PK-Backend00:17:001
3567132991513,1cyclictest0-21swapper/122:00:191
139099150,13rtkit-daemon0-21swapper/221:37:242
356713299129,2cyclictest3056-21PK-Backend20:55:371
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional