You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-09 - 13:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Thu Apr 09, 2026 00:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1272267994138,2cyclictest1340414-21kworker/u8:3+events_unbound20:15:160
1272269993630,4cyclictest1308713-21ntpq19:40:201
1272267993629,5cyclictest1297985-21ntpq19:30:200
127226799360,34cyclictest1-21systemd21:55:000
1272267993531,3cyclictest1371307-21kworker/u8:0+flush-259:020:50:000
1272273993429,3cyclictest1548175-21latency_hist23:35:002
1272273993428,4cyclictest1427208-21ntpq21:35:182
1272273993327,4cyclictest1278228-21diskstats19:15:122
1272277993229,2cyclictest1288368-21kworker/u8:2+flush-259:019:35:173
1272273993229,2cyclictest1288368-21kworker/u8:2+flush-259:019:40:172
1272267993228,3cyclictest1355354-21kworker/u8:4+events_unbound22:15:010
1272267993227,4cyclictest1282980-21latency_hist19:20:010
1272267993226,5cyclictest252-21systemd-journal00:20:000
1272277993128,2cyclictest1537509-21gpgv23:25:013
1272277993128,2cyclictest1537509-21gpgv23:25:003
1272277993026,3cyclictest252-21systemd-journal21:25:003
1272273993026,3cyclictest252-21systemd-journal00:00:372
1272269993026,3cyclictest1511990-21apt-get23:00:131
1272267993027,2cyclictest1454879-21kworker/u8:0+events_unbound22:05:160
1272267993027,2cyclictest1251412-21kworker/u8:4+flush-259:019:50:010
1272277992923,4cyclictest252-21systemd-journal00:08:393
1272273992926,2cyclictest1443923-21kworker/u8:3+events_unbound22:00:142
1272273992926,2cyclictest1251412-21kworker/u8:4+events_unbound19:10:012
1272267992926,2cyclictest3281578-21kworker/u8:0+flush-259:019:55:240
1272267992924,3cyclictest1355354-21kworker/u8:4+flush-259:020:55:160
1272277992825,2cyclictest1355354-21kworker/u8:4+flush-259:022:30:213
1272273992825,2cyclictest1553388-21kworker/u8:1+events_unbound23:40:112
1272273992824,3cyclictest1324706-21kworker/u8:2+flush-259:020:10:162
1272269992822,4cyclictest1543066-21kworker/u8:0+flush-259:000:00:251
1272267992825,2cyclictest1464818-21kworker/u8:1+events_unbound22:25:200
1272267992821,5cyclictest1604055-21ntpq00:25:230
1272273992724,2cyclictest1397069-21kworker/u8:1+flush-259:021:30:232
1272273992724,2cyclictest1355354-21kworker/u8:4+flush-259:021:05:002
1272267992722,4cyclictest1345816-21apt-get20:20:100
1272277992622,3cyclictest1459817-21latency_hist22:10:013
1272277992619,5cyclictest1277568-21cp19:15:013
1272273992623,3cyclictest1288368-21kworker/u8:2+events_unbound19:25:242
1272273992623,2cyclictest1496539-21kworker/u8:3+flush-259:022:55:252
1272273992623,2cyclictest1475487-21kworker/u8:2+flush-259:023:45:242
1272273992623,2cyclictest1314353-21kworker/u8:1+flush-259:020:10:012
1272273992622,3cyclictest1475523-21apt-get22:25:132
1272273992622,3cyclictest1475487-21kworker/u8:2+flush-259:023:35:242
1272273992622,3cyclictest1475487-21kworker/u8:2+flush-259:023:35:232
1272273992622,3cyclictest1412532-21latency_hist21:25:012
1272269992622,3cyclictest1485846-21apt-get22:35:111
1272277992522,2cyclictest1371307-21kworker/u8:0+flush-259:021:10:263
1272277992521,3cyclictest1355354-21kworker/u8:4+flush-259:020:50:243
1272277992520,3cyclictest1433639-21gpgv21:45:013
1272277992519,4cyclictest1573977-21cp00:00:003
1272277992517,7cyclictest1599737-21kworker/u8:0+events_unbound00:25:013
1272273992522,2cyclictest1496539-21kworker/u8:3+flush-259:023:25:232
1272273992522,2cyclictest1496539-21kworker/u8:3+flush-259:022:50:172
1272273992522,2cyclictest1454879-21kworker/u8:0+flush-259:022:05:222
1272269992521,3cyclictest1324706-21kworker/u8:2+flush-259:020:10:251
1272269992520,3cyclictest1569080-21apt-get23:55:101
1272267992521,3cyclictest1355354-21kworker/u8:4+events_unbound21:00:220
1272267992520,4cyclictest1558504-21latency_hist23:45:000
1272277992420,2cyclictest211-21jbd2/nvme0n1p2-820:20:283
1272273992418,4cyclictest1344653-21ntpq20:15:202
1272269992421,2cyclictest1314353-21kworker/u8:1+flush-259:020:25:001
1272269992420,3cyclictest1496539-21kworker/u8:3+flush-259:023:15:231
1272267992421,2cyclictest1282835-21kworker/u8:1+events_unbound19:20:140
1272267992418,4cyclictest252-21systemd-journal22:42:280
1272267992418,4cyclictest1568822-21latency_hist23:55:000
1272277992319,3cyclictest1387038-21apt-get21:00:113
1272273992320,2cyclictest1314353-21kworker/u8:1+events_unbound19:50:222
1272269992319,3cyclictest1543066-21kworker/u8:0+flush-259:000:05:241
1272267992320,2cyclictest1599737-21kworker/u8:0+flush-259:000:35:240
1272267992319,3cyclictest1251412-21kworker/u8:4+flush-259:019:10:160
1272267992319,2cyclictest3281578-21kworker/u8:0+flush-259:019:40:220
1272277992219,2cyclictest1314353-21kworker/u8:1+flush-259:020:05:263
1272277992218,3cyclictest252-21systemd-journal21:00:003
1272273992219,2cyclictest1584078-21kworker/u8:3+flush-259:000:30:242
1272273992219,2cyclictest1475487-21kworker/u8:2+flush-259:023:20:242
1272273992219,2cyclictest1475487-21kworker/u8:2+flush-259:023:20:242
1272273992219,2cyclictest1397069-21kworker/u8:1+flush-259:021:45:222
1272273992219,2cyclictest1288368-21kworker/u8:2+events_unbound19:45:162
1272273992219,2cyclictest1282835-21kworker/u8:1+flush-259:019:20:222
1272273992218,3cyclictest252-21systemd-journal20:50:022
1272273992218,3cyclictest1553388-21kworker/u8:1+events_unbound00:24:382
1272269992218,3cyclictest1584078-21kworker/u8:3+flush-259:000:10:231
1272267992219,3cyclictest1355354-21kworker/u8:4+events_unbound22:30:240
1272267992219,2cyclictest1584078-21kworker/u8:3+events_unbound00:10:220
1272267992219,2cyclictest1475487-21kworker/u8:2+events_unbound23:20:260
1272267992219,2cyclictest1475487-21kworker/u8:2+events_unbound23:20:260
1272267992219,2cyclictest1314353-21kworker/u8:1+events_unbound19:50:120
1272267992218,3cyclictest1464818-21kworker/u8:1+events_unbound22:36:290
1272267992218,3cyclictest1371307-21kworker/u8:0+events_unbound21:35:240
1272277992119,1cyclictest3281578-21kworker/u8:0+flush-259:019:45:253
1272273992118,2cyclictest1599737-21kworker/u8:0+flush-259:000:35:122
1272273992118,2cyclictest1563649-21kworker/u8:3+events_unbound23:50:172
1272273992118,2cyclictest1496539-21kworker/u8:3+flush-259:023:30:212
1272269992117,2cyclictest1563649-21kworker/u8:3+flush-259:023:50:151
1272269992115,4cyclictest252-21systemd-journal19:19:261
1272267992118,2cyclictest1496539-21kworker/u8:3+events_unbound23:30:180
1272267992118,2cyclictest1496539-21kworker/u8:3+events_unbound23:30:180
1272267992118,2cyclictest1397069-21kworker/u8:1+flush-259:021:45:170
1272267992116,4cyclictest509-21rs:main0
1272277992017,2cyclictest1464818-21kworker/u8:1+flush-259:022:25:263
1272273992016,3cyclictest1454879-21kworker/u8:0+events_unbound22:10:122
1272273992015,3cyclictest0-21swapper/220:25:002
1272273992014,4cyclictest252-21systemd-journal22:35:002
1272269992017,2cyclictest3281578-21kworker/u8:0+flush-259:019:55:201
1272269992017,2cyclictest301rcuc/120:50:001
1272269992017,2cyclictest1459820-21latency_hist22:10:001
1272269992016,3cyclictest1288368-21kworker/u8:2+events_unbound19:45:141
1272269992015,3cyclictest1599737-21kworker/u8:0+flush-259:000:35:051
1272267992016,3cyclictest1397069-21kworker/u8:1+flush-259:021:58:050
1272277991916,2cyclictest1360621-21kworker/u8:5+events_unbound20:40:153
1272277991915,3cyclictest1574243-21/usr/sbin/munin00:00:223
1272277991914,3cyclictest252-21systemd-journal22:00:283
1272273991916,2cyclictest1584078-21kworker/u8:3+events_unbound00:25:392
1272273991916,2cyclictest1475487-21kworker/u8:2+events_unbound22:45:232
1272273991916,2cyclictest1355354-21kworker/u8:4+events_unbound21:55:152
1272273991916,2cyclictest1355354-21kworker/u8:4+events_unbound21:13:302
1272273991916,2cyclictest1251412-21kworker/u8:4+events_unbound19:57:262
1272273991915,3cyclictest1251412-21kworker/u8:4+events_unbound19:10:112
1272273991913,5cyclictest1522017-21latency_hist23:10:002
1272269991917,1cyclictest1417960-21apt-get21:30:101
1272269991916,2cyclictest1553388-21kworker/u8:1+flush-259:000:20:181
1272269991915,3cyclictest1548434-21apt-get23:35:111
1272269991915,3cyclictest1548434-21apt-get23:35:111
1272269991915,3cyclictest1496539-21kworker/u8:3+flush-259:023:10:231
1272269991915,3cyclictest1330377-21apt-get20:05:121
1272269991914,4cyclictest252-21systemd-journal21:55:001
1272269991914,4cyclictest1423138-21apt-get21:35:091
1272267991916,2cyclictest1397069-21kworker/u8:1+flush-259:022:00:210
1272267991915,3cyclictest1475487-21kworker/u8:2+flush-259:023:15:300
1272267991915,3cyclictest1360621-21kworker/u8:5+events_unbound20:40:180
1272267991913,4cyclictest1562766-21ntpq23:45:220
1272277991815,2cyclictest1371307-21kworker/u8:0+flush-259:021:50:003
1272277991815,2cyclictest0-21swapper/319:30:013
1272277991814,3cyclictest1330159-21kworker/u8:0+flush-259:020:29:593
1272273991816,2cyclictest1355354-21kworker/u8:4+events_unbound20:35:152
1272273991815,2cyclictest3281578-21kworker/u8:0+flush-259:019:35:212
1272273991815,2cyclictest1584078-21kworker/u8:3+flush-259:000:15:172
1272269991815,2cyclictest1584078-21kworker/u8:3+flush-259:000:35:001
1272269991815,2cyclictest1496579-21apt-get22:45:171
1272269991815,2cyclictest1325192-21apt-get20:00:111
1272267991815,2cyclictest1584078-21kworker/u8:3+events_unbound00:30:230
1272267991815,2cyclictest1397069-21kworker/u8:1+events_unbound21:30:220
1272267991814,3cyclictest1496539-21kworker/u8:3+flush-259:023:10:300
127226799180,16cyclictest1-21systemd21:20:000
1272277991713,3cyclictest1501707-21/usr/sbin/munin22:50:233
1272273991714,2cyclictest1491852-21irqcore22:40:162
1272273991714,2cyclictest1355354-21kworker/u8:4+events_unbound20:55:192
1272273991713,3cyclictest1496539-21kworker/u8:3+events_unbound23:15:312
1272273991712,4cyclictest1485821-21/usr/sbin/munin22:35:142
1272269991713,3cyclictest1517148-21apt-get23:05:111
1272267991714,2cyclictest60350irq/132-enp0s29f1:rx-022:55:190
1272267991714,2cyclictest1574046-21kworker/u8:2+flush-259:000:00:180
1272267991714,2cyclictest1355354-21kworker/u8:4+events_unbound20:25:270
1272267991714,2cyclictest1314353-21kworker/u8:1+events_unbound20:05:220
1272267991714,2cyclictest1314353-21kworker/u8:1+events_unbound20:05:010
1272267991713,3cyclictest1355354-21kworker/u8:4+events_unbound22:17:280
1272277991613,2cyclictest1496539-21kworker/u8:3+events_unbound23:10:163
1272277991613,2cyclictest1355354-21kworker/u8:4+flush-259:022:35:243
1272277991611,4cyclictest1407347-21latency_hist21:20:013
1272273991614,2cyclictest1574046-21kworker/u8:2+events_unbound00:05:152
1272273991614,2cyclictest1496539-21kworker/u8:3+events_unbound23:10:262
1272273991613,2cyclictest1355354-21kworker/u8:4+flush-259:020:45:262
1272273991612,3cyclictest1355354-21kworker/u8:4+flush-259:021:05:102
1272273991612,3cyclictest1355354-21kworker/u8:4+events_unbound22:18:282
1272269991614,1cyclictest0-21swapper/121:05:001
1272269991613,2cyclictest1555897-21kthreadcore23:40:151
1272269991612,3cyclictest1464818-21kworker/u8:1+flush-259:022:25:121
1272269991610,4cyclictest1314888-21/usr/sbin/munin19:50:131
1272267991614,1cyclictest60350irq/132-enp0s29f1:rx-021:20:240
1272267991614,1cyclictest60350irq/132-enp0s29f1:rx-021:05:230
1272267991614,1cyclictest60350irq/132-enp0s29f1:rx-020:10:220
1272267991613,2cyclictest60350irq/132-enp0s29f1:rx-000:05:200
1272267991613,2cyclictest1563649-21kworker/u8:3+flush-259:023:55:230
1272267991613,2cyclictest1475487-21kworker/u8:2+flush-259:022:45:310
1272267991613,2cyclictest1475487-21kworker/u8:2+events_unbound23:35:140
1272267991613,2cyclictest1475487-21kworker/u8:2+events_unbound23:35:130
1272267991612,3cyclictest1553388-21kworker/u8:1+events_unbound00:20:120
1272277991512,2cyclictest1355354-21kworker/u8:4+flush-259:021:55:193
1272277991511,3cyclictest1610486-21diskstats00:35:133
1272277991510,4cyclictest1584078-21kworker/u8:3+flush-259:000:13:083
1272273991513,1cyclictest0-21swapper/220:45:002
1272273991512,2cyclictest1355354-21kworker/u8:4+events_unbound22:20:212
1272273991512,2cyclictest1288368-21kworker/u8:2+events_unbound19:30:012
1272273991511,3cyclictest1355354-21kworker/u8:4+flush-259:021:25:332
1272273991511,2cyclictest1584968-21irqcore00:10:152
1272269991513,1cyclictest1272481-21apt-get19:10:101
1272269991511,3cyclictest1480684-21apt-get22:30:121
1272269991511,3cyclictest1397069-21kworker/u8:1+flush-259:021:15:001
1272267991514,1cyclictest60350irq/132-enp0s29f1:rx-022:24:170
1272267991513,1cyclictest60350irq/132-enp0s29f1:rx-021:12:140
1272267991512,2cyclictest60350irq/132-enp0s29f1:rx-020:54:110
1272267991512,2cyclictest1496539-21kworker/u8:3+events_unbound23:25:220
1272267991512,2cyclictest1475487-21kworker/u8:2+events_unbound23:00:220
1272267991511,3cyclictest1355354-21kworker/u8:4+events_unbound20:35:170
1272267991511,3cyclictest1288368-21kworker/u8:2+flush-259:019:25:110
1272277991412,1cyclictest1507267-21irqcore22:55:243
1272277991412,1cyclictest0-21swapper/319:55:013
1272277991411,2cyclictest1462368-21kthreadcore22:10:183
1272273991411,2cyclictest1371307-21kworker/u8:0+events_unbound21:50:152
127227399141,11cyclictest1-21systemd20:32:282
1272273991410,3cyclictest1324706-21kworker/u8:2+events_unbound20:01:332
1272269991412,1cyclictest0-21swapper/119:40:011
1272269991411,2cyclictest1590140-21irqcore00:15:161
1272269991411,2cyclictest1475487-21kworker/u8:2+events_unbound23:45:251
1272269991411,2cyclictest0-21swapper/121:10:001
1272269991410,3cyclictest1449373-21apt-get22:00:111
1272269991410,3cyclictest1340414-21kworker/u8:3+flush-259:020:30:231
1272267991412,1cyclictest60350irq/132-enp0s29f1:rx-020:30:180
127227799139,3cyclictest1563202-21unixbench_singl23:45:253
127227799131,10cyclictest1288143-21awk19:25:003
1272277991310,2cyclictest1584078-21kworker/u8:3+flush-259:000:15:153
1272277991310,1cyclictest0-21swapper/322:40:213
127227399139,3cyclictest1371307-21kworker/u8:0+events_unbound21:17:012
127227399138,4cyclictest1475487-21kworker/u8:2+flush-259:023:00:152
127226999138,4cyclictest1444141-21/usr/sbin/munin21:55:141
1272269991310,2cyclictest0-21swapper/122:54:591
127226799139,3cyclictest1355354-21kworker/u8:4+events_unbound21:25:190
127227799129,2cyclictest1553662-21cpuspeed_turbos23:40:113
127227799129,2cyclictest1543025-21latency_hist23:30:003
127227799129,2cyclictest1496539-21kworker/u8:3+flush-259:023:30:163
127227799129,2cyclictest1496539-21kworker/u8:3+flush-259:023:30:163
127227799129,2cyclictest1324494-21unixbench_singl19:55:243
127227799129,2cyclictest1224529-21kworker/u8:2+flush-259:019:15:163
127227799128,3cyclictest1564018-21cut23:50:133
127227799128,3cyclictest1464818-21kworker/u8:1+events_unbound22:15:233
127227799127,3cyclictest1422880-21turbostat21:35:003
127227799126,4cyclictest252-21systemd-journal21:35:103
127227799121,3cyclictest1298836-21latency_hist19:35:003
1272277991210,1cyclictest0-21swapper/321:50:193
1272273991210,1cyclictest1351365-21irqcore20:25:172
127226999129,2cyclictest211-21jbd2/nvme0n1p2-800:25:171
127226999128,3cyclictest0-21swapper/121:50:001
127226999127,4cyclictest1491346-21cat22:40:011
127226999126,4cyclictest252-21systemd-journal21:18:281
127227799118,1cyclictest0-21swapper/320:10:163
127227799115,4cyclictest1527599-21diskstats23:15:123
127226999119,1cyclictest0-21swapper/121:30:001
127226999118,2cyclictest1496539-21kworker/u8:3+events_unbound22:55:261
127226999118,2cyclictest1430644-21kthreadcore21:40:171
127226999117,3cyclictest252-21systemd-journal23:33:281
127226999117,3cyclictest252-21systemd-journal23:33:271
127226999117,3cyclictest252-21systemd-journal19:25:001
127226999117,3cyclictest1360621-21kworker/u8:5+flush-259:020:40:241
127227799108,1cyclictest0-21swapper/323:35:133
127227799108,1cyclictest0-21swapper/323:35:133
127227799108,1cyclictest0-21swapper/320:15:163
127227799107,2cyclictest1340414-21kworker/u8:3+events_unbound20:30:133
127227799101,7cyclictest1500583-21ntpq22:45:263
127227799100,2cyclictest1397039-21latency_hist21:10:003
127226999108,1cyclictest1292561-21sensors19:25:241
127226999107,2cyclictest1532673-21cpuspeed_turbos23:20:141
127226999107,2cyclictest1532673-21cpuspeed_turbos23:20:131
127226999107,2cyclictest1460506-21grep22:10:161
127226999107,2cyclictest0-21swapper/122:20:151
127226999106,3cyclictest252-21systemd-journal19:34:471
12722699997,1cyclictest0-21swapper/123:25:151
12722699996,2cyclictest0-21swapper/121:25:001
12722699986,1cyclictest0-21swapper/122:15:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional