You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-30 - 09:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackcslot7s.osadl.org (updated Mon Mar 30, 2026 00:44:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
224437221180,3sleep1251rcuc/120:25:131
224437221180,3sleep1251rcuc/120:25:131
221947021140,2sleep10-21swapper/119:15:121
22561692980,2sleep10-21swapper/121:00:011
221684327657,13sleep10-21swapper/119:09:471
221684327657,13sleep10-21swapper/119:09:471
221669226545,14sleep00-21swapper/019:07:500
221669226545,14sleep00-21swapper/019:07:500
221662726343,14sleep20-21swapper/219:06:592
221662726343,14sleep20-21swapper/219:06:592
221659126342,14sleep30-21swapper/319:06:333
221659126342,14sleep30-21swapper/319:06:323
23200292470,3sleep22320027-21grep00:00:112
22637472430,3sleep10-21swapper/121:20:141
2217020993017,11cyclictest2275502-21unin-run21:55:001
139099280,4rtkit-daemon1389-21rtkit-daemon22:28:421
22339232260,3sleep20-21swapper/219:55:112
2217025992621,4cyclictest2331448-21nm-dispatcher00:31:242
221702099250,18cyclictest0-21swapper/119:22:281
221702099240,22cyclictest2227123-21ntp_states19:35:181
221701799241,7cyclictest2303733-21apt23:15:000
139099240,16rtkit-daemon3056-21PK-Backend23:05:123
22866412230,4sleep33056-21PK-Backend22:25:123
221702099237,3cyclictest0-21swapper/122:05:021
221702099231,20cyclictest2304854-21python323:15:201
221702099230,22cyclictest1389-21rtkit-daemon21:11:051
221702099230,1cyclictest0-21swapper/119:33:291
2217017992321,1cyclictest0-21swapper/022:43:570
2217017992314,7cyclictest883-21dbus-daemon20:05:010
221701799230,21cyclictest0-21swapper/020:10:120
22636972220,4sleep33056-21PK-Backend21:20:123
2217027992220,1cyclictest3056-21PK-Backend21:00:233
2217027992219,2cyclictest3056-21PK-Backend21:05:573
2217027992219,2cyclictest3056-21PK-Backend20:15:413
2217027992219,2cyclictest3056-21PK-Backend20:10:563
2217027992218,2cyclictest2297636-21memory22:55:143
221702799221,17cyclictest2244847-21python320:25:193
221702799221,17cyclictest2244847-21python320:25:193
221702799220,21cyclictest3056-21PK-Backend21:14:423
221702799220,20cyclictest3056-21PK-Backend23:22:383
221702799220,20cyclictest3056-21PK-Backend21:34:023
221702799220,20cyclictest3056-21PK-Backend21:34:023
2217025992220,1cyclictest0-21swapper/223:08:382
2217025992220,1cyclictest0-21swapper/222:37:422
2217025992220,1cyclictest0-21swapper/222:01:532
2217020992214,6cyclictest2254791-21fw_packets20:55:141
2217017992220,1cyclictest0-21swapper/023:52:100
2217017992220,1cyclictest0-21swapper/023:34:470
2217017992220,1cyclictest0-21swapper/023:34:470
2217017992220,1cyclictest0-21swapper/022:26:050
2217017992220,1cyclictest0-21swapper/022:22:270
2217017992220,1cyclictest0-21swapper/020:33:380
2217017992220,1cyclictest0-21swapper/019:40:040
2217017992220,1cyclictest0-21swapper/000:38:000
2217017992220,1cyclictest0-21swapper/000:01:370
2217017992219,2cyclictest0-21swapper/021:55:160
221701799220,21cyclictest0-21swapper/023:38:390
221701799220,21cyclictest0-21swapper/022:52:420
139099220,15rtkit-daemon3056-21PK-Backend00:03:453
2217027992119,1cyclictest3056-21PK-Backend22:01:273
2217027992119,1cyclictest3056-21PK-Backend19:29:183
2217027992119,1cyclictest0-21swapper/322:51:223
2217027992119,1cyclictest0-21swapper/322:42:383
2217027992119,1cyclictest0-21swapper/321:57:143
2217027992119,1cyclictest0-21swapper/321:45:393
2217027992119,1cyclictest0-21swapper/319:40:273
2217027992118,1cyclictest3056-21PK-Backend19:35:473
221702799210,20cyclictest3056-21PK-Backend00:21:123
221702799210,20cyclictest3056-21PK-Backend00:14:163
221702799210,19cyclictest3056-21PK-Backend23:55:513
221702799210,19cyclictest3056-21PK-Backend23:39:283
221702799210,19cyclictest3056-21PK-Backend20:07:453
221702799210,19cyclictest3056-21PK-Backend00:25:043
2217025992119,1cyclictest0-21swapper/223:50:152
2217025992119,1cyclictest0-21swapper/223:30:002
2217025992119,1cyclictest0-21swapper/223:30:002
2217025992119,1cyclictest0-21swapper/223:26:082
2217025992119,1cyclictest0-21swapper/223:18:252
2217025992119,1cyclictest0-21swapper/223:10:402
2217025992119,1cyclictest0-21swapper/222:58:512
2217025992119,1cyclictest0-21swapper/222:55:002
2217025992119,1cyclictest0-21swapper/222:32:012
2217025992119,1cyclictest0-21swapper/222:17:002
2217025992119,1cyclictest0-21swapper/221:58:292
2217025992119,1cyclictest0-21swapper/221:54:162
2217025992119,1cyclictest0-21swapper/221:45:502
2217025992119,1cyclictest0-21swapper/221:42:262
2217025992119,1cyclictest0-21swapper/221:38:132
2217025992119,1cyclictest0-21swapper/221:33:142
2217025992119,1cyclictest0-21swapper/221:33:132
2217025992119,1cyclictest0-21swapper/221:06:512
2217025992119,1cyclictest0-21swapper/220:37:102
2217025992119,1cyclictest0-21swapper/219:42:212
2217025992119,1cyclictest0-21swapper/219:20:522
2217025992119,1cyclictest0-21swapper/219:13:482
2217025992119,1cyclictest0-21swapper/200:18:342
2217025992112,7cyclictest2638-21gdbus19:31:192
221702599210,20cyclictest0-21swapper/222:06:052
221702599210,20cyclictest0-21swapper/219:54:242
221702599210,20cyclictest0-21swapper/200:12:252
221702099212,17cyclictest2322268-21strings00:05:181
221702099210,20cyclictest0-21swapper/122:21:131
221702099210,20cyclictest0-21swapper/121:36:181
221702099210,20cyclictest0-21swapper/120:13:301
221702099210,20cyclictest0-21swapper/100:01:371
221702099210,19cyclictest0-21swapper/121:30:101
221702099210,19cyclictest0-21swapper/121:30:091
2217017992119,1cyclictest3056-21PK-Backend23:47:030
2217017992119,1cyclictest0-21swapper/022:33:360
2217017992119,1cyclictest0-21swapper/021:46:040
2217017992119,1cyclictest0-21swapper/021:23:410
2217017992119,1cyclictest0-21swapper/020:24:590
2217017992119,1cyclictest0-21swapper/020:05:260
2217017992119,1cyclictest0-21swapper/019:55:590
2217017992119,1cyclictest0-21swapper/019:51:540
2217017992119,1cyclictest0-21swapper/019:49:510
2217017992119,1cyclictest0-21swapper/019:36:130
2217017992119,1cyclictest0-21swapper/000:23:060
2217017992119,1cyclictest0-21swapper/000:12:040
2217017992113,6cyclictest2293970-21diskstats22:45:150
221701799210,20cyclictest22612222sleep021:13:530
221701799210,20cyclictest0-21swapper/023:24:260
221701799210,20cyclictest0-21swapper/022:15:100
221701799210,20cyclictest0-21swapper/021:50:160
221701799210,20cyclictest0-21swapper/019:18:000
221701799210,20cyclictest0-21swapper/019:10:440
221701799210,20cyclictest0-21swapper/000:31:040
139099210,15rtkit-daemon3056-21PK-Backend22:37:093
139099210,14rtkit-daemon3056-21PK-Backend23:53:363
139099210,14rtkit-daemon0-21swapper/223:37:452
139099210,14rtkit-daemon0-21swapper/220:07:462
2217027992018,1cyclictest3056-21PK-Backend23:19:483
2217027992018,1cyclictest3056-21PK-Backend23:00:553
2217027992018,1cyclictest3056-21PK-Backend22:14:183
2217027992018,1cyclictest0-21swapper/300:38:433
221702799200,1cyclictest3056-21PK-Backend21:50:393
221702799200,19cyclictest3056-21PK-Backend20:57:133
221702799200,19cyclictest3056-21PK-Backend19:13:373
221702799200,19cyclictest0-21swapper/323:25:293
221702799200,19cyclictest0-21swapper/322:46:293
221702799200,19cyclictest0-21swapper/320:31:303
221702799200,19cyclictest0-21swapper/319:50:133
221702799200,19cyclictest0-21swapper/319:47:093
2217025992018,1cyclictest0-21swapper/222:28:102
2217025992018,1cyclictest0-21swapper/221:27:262
2217025992018,1cyclictest0-21swapper/221:27:252
2217025992018,1cyclictest0-21swapper/221:13:262
2217025992018,1cyclictest0-21swapper/220:45:212
2217025992018,1cyclictest0-21swapper/220:27:292
2217025992018,1cyclictest0-21swapper/220:27:292
2217025992018,1cyclictest0-21swapper/220:24:532
221702599200,1cyclictest0-21swapper/223:41:512
221702599200,1cyclictest0-21swapper/200:28:342
221702599200,19cyclictest0-21swapper/222:12:202
221702599200,19cyclictest0-21swapper/220:59:272
221702599200,19cyclictest0-21swapper/220:53:522
221702599200,19cyclictest0-21swapper/200:20:372
2217020992018,1cyclictest0-21swapper/120:06:481
2217020992013,6cyclictest2319392-21sshd23:57:471
221702099200,19cyclictest3056-21PK-Backend23:41:091
221702099200,19cyclictest0-21swapper/123:36:291
221702099200,19cyclictest0-21swapper/121:28:341
221702099200,19cyclictest0-21swapper/121:28:341
221702099200,19cyclictest0-21swapper/121:15:431
221702099200,19cyclictest0-21swapper/120:37:301
2217017992018,1cyclictest0-21swapper/023:19:460
2217017992018,1cyclictest0-21swapper/023:05:200
2217017992018,1cyclictest0-21swapper/022:08:420
2217017992018,1cyclictest0-21swapper/021:41:050
2217017992018,1cyclictest0-21swapper/021:36:040
2217017992018,1cyclictest0-21swapper/020:27:370
2217017992018,1cyclictest0-21swapper/020:27:360
2217017992018,1cyclictest0-21swapper/000:28:010
2217017992014,4cyclictest13-21ksoftirqd/022:15:020
2217017992013,5cyclictest883-21dbus-daemon21:18:030
2217017992012,6cyclictest2257275-21awk21:00:220
221701799200,1cyclictest0-21swapper/020:58:320
221701799200,19cyclictest0-21swapper/023:25:270
221701799200,19cyclictest0-21swapper/023:00:260
221701799200,19cyclictest0-21swapper/020:46:020
221701799200,19cyclictest0-21swapper/019:26:520
139099200,4rtkit-daemon3056-21PK-Backend20:35:173
139099200,14rtkit-daemon0-21swapper/220:32:412
2217027991917,1cyclictest3056-21PK-Backend23:12:183
2217027991917,1cyclictest3056-21PK-Backend22:16:083
2217027991917,1cyclictest3056-21PK-Backend21:17:533
2217027991917,1cyclictest0-21swapper/321:40:393
2217027991917,1cyclictest0-21swapper/319:55:203
2217027991914,3cyclictest2269453-21grep21:35:153
221702799190,1cyclictest3056-21PK-Backend19:32:563
221702799190,1cyclictest0-21swapper/300:19:093
221702799190,18cyclictest3056-21PK-Backend00:32:493
221702799190,16cyclictest2251672-21ntp_states20:45:173
2217025991917,1cyclictest0-21swapper/222:43:232
2217025991917,1cyclictest0-21swapper/219:35:382
2217025991917,1cyclictest0-21swapper/219:29:572
2217025991917,1cyclictest0-21swapper/200:35:442
221702599190,1cyclictest3056-21PK-Backend21:20:022
221702599190,1cyclictest0-21swapper/220:15:412
221702599190,1cyclictest0-21swapper/219:48:172
221702599190,18cyclictest0-21swapper/220:01:472
2217020991917,1cyclictest3056-21PK-Backend19:43:091
2217020991917,1cyclictest0-21swapper/122:11:461
2217020991917,1cyclictest0-21swapper/120:15:061
2217020991917,1cyclictest0-21swapper/120:02:221
2217020991917,1cyclictest0-21swapper/100:27:471
221702099190,1cyclictest0-21swapper/123:51:031
221702099190,1cyclictest0-21swapper/123:01:011
221702099190,1cyclictest0-21swapper/119:57:291
221702099190,18cyclictest0-21swapper/119:27:281
221702099190,17cyclictest3056-21PK-Backend00:38:281
221701799192,12cyclictest2266541-21apt-get21:30:030
221701799192,12cyclictest2266541-21apt-get21:30:030
2217017991917,1cyclictest0-21swapper/023:57:190
2217017991917,1cyclictest0-21swapper/020:19:590
2217017991913,4cyclictest2298532-21cut23:00:000
2217017991912,5cyclictest2258726-21cat21:05:170
2217017991912,4cyclictest2221861-21ntp_kernel_err19:20:170
221701799190,1cyclictest0-21swapper/022:36:270
221701799190,1cyclictest0-21swapper/020:38:510
221701799190,1cyclictest0-21swapper/000:19:010
221701799190,18cyclictest3056-21PK-Backend20:40:410
221701799190,18cyclictest0-21swapper/022:00:170
139099190,13rtkit-daemon3056-21PK-Backend20:00:453
139099190,13rtkit-daemon0-21swapper/223:58:112
2217027991816,1cyclictest3056-21PK-Backend22:32:293
2217027991816,1cyclictest3056-21PK-Backend19:24:513
2217027991816,1cyclictest0-21swapper/320:50:173
2217027991815,2cyclictest3056-21PK-Backend22:06:273
2217027991813,3cyclictest2313139-21irqstats23:40:163
2217027991813,3cyclictest2249946-21ntp_states20:40:183
2217027991811,5cyclictest436-20systemd-journal23:50:003
221702799180,1cyclictest3056-21PK-Backend23:33:473
221702799180,1cyclictest3056-21PK-Backend23:33:473
2217025991816,1cyclictest0-21swapper/223:03:442
2217025991816,1cyclictest0-21swapper/220:43:452
2217025991816,1cyclictest0-21swapper/219:19:032
221702599180,16cyclictest2257859-21perf21:05:012
2217020991816,1cyclictest3056-21PK-Backend21:48:211
2217020991816,1cyclictest0-21swapper/123:33:521
2217020991816,1cyclictest0-21swapper/123:33:521
2217020991816,1cyclictest0-21swapper/122:56:361
2217020991816,1cyclictest0-21swapper/122:06:471
2217020991816,1cyclictest0-21swapper/121:05:011
2217020991816,1cyclictest0-21swapper/100:31:251
2217020991816,1cyclictest0-21swapper/100:12:121
2217020991815,2cyclictest3056-21PK-Backend20:43:261
2217020991810,6cyclictest2306687-21sed23:20:211
221702099180,1cyclictest0-21swapper/123:26:031
221702099180,1cyclictest0-21swapper/122:19:231
221702099180,17cyclictest0-21swapper/100:15:031
2217017991816,1cyclictest0-21swapper/019:33:220
2217017991813,3cyclictest2265676-21missed_timers21:25:220
2217017991813,3cyclictest2265676-21missed_timers21:25:210
23143692170,2sleep10-21swapper/123:45:121
221702799178,7cyclictest3056-21PK-Backend00:10:003
2217027991715,1cyclictest3056-21PK-Backend22:22:223
2217027991715,1cyclictest3056-21PK-Backend19:19:383
2217025991715,1cyclictest0-21swapper/222:21:402
2217025991712,3cyclictest2305858-21cpuspeed_turbos23:20:122
2217025991711,4cyclictest2321732-21cpuspeed_turbos00:05:142
221702099178,7cyclictest1788610-21gdbus19:51:211
2217020991715,1cyclictest3056-21PK-Backend22:42:561
2217020991715,1cyclictest0-21swapper/119:11:521
221702099171,12cyclictest2290128-21apt22:35:111
221701799179,6cyclictest883-21dbus-daemon23:45:000
221701799179,6cyclictest1-21systemd00:10:000
2217027991614,1cyclictest0-21swapper/320:24:403
2217025991611,4cyclictest2262191-21sed21:15:182
221702099169,5cyclictest2296033-21awk22:50:151
2217020991614,1cyclictest3056-21PK-Backend22:45:331
2217020991614,1cyclictest0-21swapper/100:20:301
2217020991612,3cyclictest2302341-21cpuspeed23:10:111
2217020991612,3cyclictest2301997-21idleruntime-cro23:10:001
2217020991610,4cyclictest2271019-21sed21:40:161
2217027991510,3cyclictest2266512-21cron21:30:003
2217027991510,3cyclictest2266512-21cron21:29:593
2217025991513,1cyclictest0-21swapper/222:48:162
2217025991512,2cyclictest3056-21PK-Backend23:45:082
221702099159,4cyclictest913-21gdbus21:55:011
221702099156,7cyclictest2289502-21anacron22:30:381
2217020991513,1cyclictest3056-21PK-Backend20:33:311
221702099150,14cyclictest3056-21PK-Backend19:46:331
2217025991412,1cyclictest0-21swapper/220:14:522
221702099149,3cyclictest2242935-21netstat20:20:181
221701799149,3cyclictest1095-21runrttasks20:51:580
221702099136,5cyclictest2251641-21ntpq20:45:171
2217020991311,1cyclictest3056-21PK-Backend20:51:501
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional