You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-10 - 05:59

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #c, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackcslot7s.osadl.org (updated Fri Jul 10, 2026 00:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
138034521170,2sleep00-21swapper/000:25:160
129658921110,3sleep13056-21PK-Backend20:25:171
133526121030,4sleep20-21swapper/222:15:272
135265021020,4sleep30-21swapper/323:05:203
13699182970,3sleep20-21swapper/223:55:152
126732128039,14sleep10-21swapper/119:05:061
126735627645,21sleep30-21swapper/319:05:123
126854427144,17sleep20-21swapper/219:06:442
126849526939,23sleep00-21swapper/019:06:050
12914732590,3sleep20-21swapper/220:10:192
13152842500,3sleep30-21swapper/321:20:133
12915622470,4sleep30-21swapper/320:10:193
139099270,4rtkit-daemon1389-21rtkit-daemon21:53:361
139099260,4rtkit-daemon1389-21rtkit-daemon20:05:221
139099260,4rtkit-daemon1389-21rtkit-daemon19:32:181
139099250,4rtkit-daemon1389-21rtkit-daemon19:23:541
1268940992518,1cyclictest0-21swapper/200:26:592
13325702240,4chrt3056-21PK-Backend22:10:103
139099230,4rtkit-daemon1389-21rtkit-daemon22:53:531
1268943992315,6cyclictest2164-21gdbus20:18:193
126894399230,21cyclictest1310633-21python321:05:223
1268938992314,7cyclictest1020-21ntpd22:10:171
1268938992312,9cyclictest1381466-21unin-run00:30:001
126893899230,21cyclictest3056-21PK-Backend20:20:191
1268933992315,6cyclictest1301969-21python320:40:190
1268943992220,1cyclictest3056-21PK-Backend22:49:583
1268943992220,1cyclictest3056-21PK-Backend19:14:223
126894099220,21cyclictest0-21swapper/220:39:122
1268938992219,2cyclictest3056-21PK-Backend23:23:371
1268938992219,1cyclictest3056-21PK-Backend23:10:181
1268938992214,6cyclictest2638-21gdbus20:03:191
126893899220,20cyclictest3056-21PK-Backend21:41:511
126893899220,20cyclictest3056-21PK-Backend20:56:341
1268943992115,4cyclictest1314722-21turbostat.cron21:20:003
126894399210,20cyclictest0-21swapper/323:29:393
126894399210,19cyclictest3056-21PK-Backend22:35:333
1268940992119,1cyclictest0-21swapper/219:20:162
126894099210,2cyclictest886-21NetworkManager00:07:242
126894099210,20cyclictest0-21swapper/220:25:122
1268938992119,1cyclictest3056-21PK-Backend23:51:021
1268938992119,1cyclictest3056-21PK-Backend21:04:321
1268938992119,1cyclictest0-21swapper/122:35:031
1268938992119,1cyclictest0-21swapper/121:57:121
1268938992119,1cyclictest0-21swapper/119:55:451
1268938992119,1cyclictest0-21swapper/119:48:561
1268938992119,1cyclictest0-21swapper/100:38:531
1268938992118,2cyclictest3056-21PK-Backend20:10:461
126893899210,20cyclictest0-21swapper/122:44:501
126893899210,20cyclictest0-21swapper/120:45:201
126893899210,19cyclictest3056-21PK-Backend23:27:091
126893899210,19cyclictest3056-21PK-Backend21:15:481
126893899210,19cyclictest3056-21PK-Backend20:32:071
126893399210,20cyclictest0-21swapper/023:50:040
126893399210,20cyclictest0-21swapper/023:41:270
126893399210,20cyclictest0-21swapper/022:47:390
126893399210,20cyclictest0-21swapper/022:00:350
126893399210,20cyclictest0-21swapper/020:51:470
126893399210,20cyclictest0-21swapper/020:09:410
126893399210,20cyclictest0-21swapper/019:48:270
12994282200,4sleep03056-21PK-Backend20:35:110
1268943992018,1cyclictest3056-21PK-Backend22:01:333
1268943992018,1cyclictest3056-21PK-Backend21:58:283
1268943992018,1cyclictest3056-21PK-Backend21:51:453
1268943992018,1cyclictest3056-21PK-Backend20:04:383
126894399201,6cyclictest883-21dbus-daemon23:05:013
126894399201,6cyclictest1356913-21unin-run23:20:013
1268943992012,6cyclictest1367159-21awk23:45:223
1268943992011,7cyclictest1295630-21cron20:25:013
126894399200,19cyclictest3056-21PK-Backend20:45:343
1268940992018,1cyclictest0-21swapper/223:38:382
1268940992018,1cyclictest0-21swapper/222:11:532
1268940992018,1cyclictest0-21swapper/221:51:102
1268940992018,1cyclictest0-21swapper/220:43:182
1268940992018,1cyclictest0-21swapper/220:00:462
1268940992018,1cyclictest0-21swapper/219:30:452
1268940992018,1cyclictest0-21swapper/219:29:222
1268940992018,1cyclictest0-21swapper/200:19:292
1268940992014,5cyclictest1317025-21ls21:25:152
1268940992013,5cyclictest1352213-21expr23:05:132
1268940992013,5cyclictest1322777-21python321:40:222
1268940992013,5cyclictest1279142-21rt-features19:35:202
126894099200,1cyclictest0-21swapper/221:58:002
126894099200,1cyclictest0-21swapper/220:33:112
126894099200,19cyclictest0-21swapper/222:53:372
126894099200,19cyclictest0-21swapper/222:42:152
126894099200,19cyclictest0-21swapper/222:05:382
126894099200,19cyclictest0-21swapper/221:31:162
126894099200,19cyclictest0-21swapper/220:50:062
126894099200,19cyclictest0-21swapper/220:46:022
1268938992018,1cyclictest3056-21PK-Backend21:35:141
1268938992018,1cyclictest0-21swapper/123:56:431
1268938992018,1cyclictest0-21swapper/122:45:381
1268938992018,1cyclictest0-21swapper/122:22:341
1268938992018,1cyclictest0-21swapper/100:21:091
1268938992018,1cyclictest0-21swapper/100:15:141
1268938992018,1cyclictest0-21swapper/100:00:281
1268938992017,2cyclictest3056-21PK-Backend21:12:311
126893899200,1cyclictest3056-21PK-Backend23:43:581
126893899200,19cyclictest3056-21PK-Backend23:08:091
126893899200,19cyclictest0-21swapper/122:34:151
126893899200,19cyclictest0-21swapper/122:05:221
126893899200,19cyclictest0-21swapper/100:30:021
126893899200,19cyclictest0-21swapper/100:06:231
126893399207,4cyclictest13-21ksoftirqd/021:20:130
1268933992018,1cyclictest0-21swapper/023:37:080
1268933992018,1cyclictest0-21swapper/021:57:510
1268933992018,1cyclictest0-21swapper/020:19:280
1268933992018,1cyclictest0-21swapper/019:53:540
1268933992018,1cyclictest0-21swapper/000:18:510
1268933992017,2cyclictest0-21swapper/023:16:520
1268933992017,2cyclictest0-21swapper/019:30:220
1268933992013,6cyclictest1-21systemd00:00:000
126893399200,1cyclictest0-21swapper/021:31:210
126893399200,19cyclictest0-21swapper/022:52:320
126893399200,19cyclictest0-21swapper/022:42:450
126893399200,19cyclictest0-21swapper/022:37:530
126893399200,19cyclictest0-21swapper/021:49:390
126893399200,19cyclictest0-21swapper/021:12:480
126893399200,19cyclictest0-21swapper/021:07:210
126893399200,19cyclictest0-21swapper/020:30:240
126893399200,19cyclictest0-21swapper/019:36:570
126893399200,19cyclictest0-21swapper/019:12:230
126893399200,19cyclictest0-21swapper/000:00:190
1268943991917,1cyclictest0-21swapper/323:40:563
1268943991916,2cyclictest3056-21PK-Backend22:30:323
126894399191,4cyclictest1321653-21/usr/sbin/munin21:40:013
1268943991914,3cyclictest1322653-21ntp_states21:40:193
1268943991913,4cyclictest1298504-21python320:30:213
1268943991912,5cyclictest1301818-21awk20:40:173
1268943991911,6cyclictest1308664-21cat21:00:183
126894399190,1cyclictest3056-21PK-Backend19:16:383
126894399190,1cyclictest3056-21PK-Backend00:25:433
126894399190,1cyclictest0-21swapper/321:46:453
126894399190,1cyclictest0-21swapper/321:31:113
126894399190,18cyclictest0-21swapper/320:29:053
1268940991917,1cyclictest0-21swapper/223:33:312
1268940991917,1cyclictest0-21swapper/223:28:242
1268940991917,1cyclictest0-21swapper/223:13:042
1268940991917,1cyclictest0-21swapper/222:57:552
1268940991917,1cyclictest0-21swapper/221:45:442
1268940991917,1cyclictest0-21swapper/221:20:552
1268940991917,1cyclictest0-21swapper/221:05:082
1268940991917,1cyclictest0-21swapper/219:42:482
1268940991917,1cyclictest0-21swapper/200:35:162
1268940991912,6cyclictest1293449-21sed20:15:212
1268940991911,6cyclictest1294173-21awk20:20:092
126894099190,1cyclictest0-21swapper/223:45:562
126894099190,1cyclictest0-21swapper/223:01:272
126894099190,1cyclictest0-21swapper/221:16:032
126894099190,18cyclictest0-21swapper/219:18:552
126893899192,13cyclictest1349958-21apt23:00:121
1268938991917,1cyclictest3056-21PK-Backend23:46:071
1268938991917,1cyclictest3056-21PK-Backend23:37:431
1268938991917,1cyclictest3056-21PK-Backend23:32:501
1268938991917,1cyclictest3056-21PK-Backend21:21:501
1268938991917,1cyclictest3056-21PK-Backend20:39:171
1268938991917,1cyclictest3056-21PK-Backend19:35:301
1268938991917,1cyclictest0-21swapper/119:44:501
1268938991912,4cyclictest1330579-21munin-run22:04:591
126893899190,1cyclictest3056-21PK-Backend21:26:301
126893899190,1cyclictest0-21swapper/122:28:011
126893899190,1cyclictest0-21swapper/120:42:361
126893899190,1cyclictest0-21swapper/100:11:291
126893899190,18cyclictest0-21swapper/119:51:391
126893899190,18cyclictest0-21swapper/119:29:291
1268933991917,1cyclictest0-21swapper/023:05:170
1268933991917,1cyclictest0-21swapper/023:04:290
1268933991917,1cyclictest0-21swapper/022:32:590
1268933991917,1cyclictest0-21swapper/022:12:510
1268933991917,1cyclictest0-21swapper/021:04:380
1268933991917,1cyclictest0-21swapper/020:55:530
1268933991917,1cyclictest0-21swapper/020:29:020
1268933991917,1cyclictest0-21swapper/019:17:570
1268933991917,1cyclictest0-21swapper/000:09:120
1268933991911,6cyclictest1336960-21python322:20:220
126893399190,1cyclictest0-21swapper/023:34:100
126893399190,1cyclictest0-21swapper/022:18:180
126893399190,1cyclictest0-21swapper/021:52:240
126893399190,1cyclictest0-21swapper/021:18:160
126893399190,1cyclictest0-21swapper/019:58:000
126893399190,1cyclictest0-21swapper/000:23:580
126893399190,18cyclictest0-21swapper/023:13:560
126893399190,18cyclictest0-21swapper/022:07:240
126893399190,18cyclictest0-21swapper/021:42:150
126893399190,18cyclictest0-21swapper/000:38:460
126893399190,17cyclictest1275924-21sshd19:25:290
126894399189,7cyclictest1312576-21users21:10:203
1268943991816,1cyclictest3056-21PK-Backend22:19:233
1268943991816,1cyclictest3056-21PK-Backend00:00:423
1268943991816,1cyclictest0-21swapper/300:10:373
126894399181,5cyclictest1353655-21/usr/sbin/munin23:10:193
1268943991815,2cyclictest3056-21PK-Backend22:56:543
1268943991814,3cyclictest978-21snmpd23:50:403
1268943991811,5cyclictest1307049-21ntpq20:55:193
126894399181,15cyclictest1300920-21latency_hist20:40:013
1268943991811,5cyclictest1276310-21munin-plugin-st19:30:013
1268943991810,6cyclictest1317428-21ntpq21:25:213
1268940991816,1cyclictest0-21swapper/223:53:122
1268940991816,1cyclictest0-21swapper/223:21:082
1268940991816,1cyclictest0-21swapper/219:50:242
1268940991816,1cyclictest0-21swapper/219:10:312
1268940991816,1cyclictest0-21swapper/200:12:452
1268940991812,5cyclictest1283381-21wc19:50:012
1268940991811,5cyclictest1320764-21memory21:35:182
1268940991810,6cyclictest1364486-21fw_packets23:40:162
1268940991810,6cyclictest1308757-21ntpq21:00:192
1268938991816,1cyclictest3056-21PK-Backend23:18:431
1268938991816,1cyclictest3056-21PK-Backend21:05:541
1268938991816,1cyclictest3056-21PK-Backend20:51:561
1268938991816,1cyclictest3056-21PK-Backend19:10:161
1268938991816,1cyclictest0-21swapper/122:57:351
1268933991816,1cyclictest0-21swapper/023:29:500
1268933991816,1cyclictest0-21swapper/020:24:230
1268933991816,1cyclictest0-21swapper/020:13:130
1268933991816,1cyclictest0-21swapper/019:23:460
1268933991816,1cyclictest0-21swapper/000:12:570
1268933991811,6cyclictest1303805-21sed20:45:200
126893399181,15cyclictest1321647-21perf21:40:000
13706162170,3sleep33056-21PK-Backend23:55:233
126894399179,6cyclictest1289829-21python320:05:213
1268943991715,1cyclictest0-21swapper/300:15:163
1268943991714,2cyclictest3056-21PK-Backend19:50:393
1268943991711,5cyclictest1373279-21cat00:05:153
1268943991711,5cyclictest1305747-21sshd20:51:273
1268943991710,5cyclictest1361385-21awk23:30:183
126894399170,5cyclictest1338767-21find22:25:223
126894399170,5cyclictest1279478-21taskset19:36:343
126894399170,15cyclictest1363085-21ntp_states23:35:203
126894399170,15cyclictest1357891-21ntp_states23:20:183
1268940991715,1cyclictest0-21swapper/222:46:342
1268940991715,1cyclictest0-21swapper/222:00:092
1268940991715,1cyclictest0-21swapper/200:00:422
1268940991714,2cyclictest0-21swapper/220:09:452
1268940991711,5cyclictest1340278-21awk22:30:192
126894099171,12cyclictest1311505-21apt21:10:112
1268940991710,5cyclictest1306510-21cat20:55:122
126893899179,5cyclictest1270980-21cpu19:15:021
1268938991715,1cyclictest3056-21PK-Backend21:33:521
1268938991715,1cyclictest3056-21PK-Backend20:16:131
126893899170,15cyclictest3056-21PK-Backend21:45:091
126893399179,6cyclictest436-20systemd-journal23:00:000
1268933991715,1cyclictest0-21swapper/023:47:550
1268933991715,1cyclictest0-21swapper/023:23:210
1268933991715,1cyclictest0-21swapper/019:41:020
1268933991712,4cyclictest1288127-21sed20:00:220
1268933991711,5cyclictest1382075-21df00:30:140
1268933991711,5cyclictest1317161-21awk21:25:190
1268943991614,1cyclictest3056-21PK-Backend22:42:293
1268943991614,1cyclictest3056-21PK-Backend19:45:043
1268943991614,1cyclictest3056-21PK-Backend00:36:593
1268943991611,3cyclictest1277225-21needreboot19:30:173
1268943991610,4cyclictest1331851-21grep22:05:223
126894099169,6cyclictest1379702-21cron00:25:012
1268940991614,1cyclictest0-21swapper/200:34:282
1268940991612,2cyclictest1336772-21cut22:20:192
1268940991611,3cyclictest1341479-21cpuspeed_turbos22:35:112
1268940991610,4cyclictest1355622-21cat23:15:152
1268938991610,4cyclictest1334858-21tr22:15:211
1268933991614,1cyclictest0-21swapper/022:28:060
1268943991513,1cyclictest3056-21PK-Backend00:32:193
1268943991513,1cyclictest3056-21PK-Backend00:23:473
126894399150,14cyclictest3056-21PK-Backend22:22:273
126894399150,13cyclictest1280005-21/usr/sbin/munin19:40:143
1268940991513,1cyclictest0-21swapper/222:28:422
126894399149,4cyclictest3056-21PK-Backend19:20:393
126894399140,13cyclictest3056-21PK-Backend22:52:283
1268940991410,3cyclictest1286942-21cron19:59:592
1268943991311,1cyclictest3056-21PK-Backend19:55:053
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional