You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-13 - 05:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Thu Aug 13, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1426456994845,2cyclictest1411112-21kworker/u8:2+flush-259:019:15:112
1426450994642,3cyclictest1609193-21kworker/u8:3+flush-259:023:10:000
1426460994538,5cyclictest1545991-21http21:05:013
1426450994237,3cyclictest335557-21systemd-journal20:54:140
1426450994237,3cyclictest335557-21systemd-journal20:54:130
1426456993835,2cyclictest1452736-21kworker/u8:2+flush-259:019:45:252
1426454993835,2cyclictest1499534-21kworker/u8:0+events_unbound21:10:221
1426456993732,4cyclictest1718520-21apt-get23:50:112
1426456993732,4cyclictest1551577-21latency_hist21:10:012
1426456993731,4cyclictest1739411-21latency_hist00:10:022
1426456993730,3cyclictest1452736-21kworker/u8:2+events_unbound20:25:152
1426454993734,2cyclictest1739646-21kworker/u8:2+events_unbound00:25:141
1426454993734,2cyclictest1525335-21kworker/u8:1+events_unbound20:55:251
1426454993734,2cyclictest1420922-21kworker/u8:0+flush-259:019:45:161
1426454993734,2cyclictest1420922-21kworker/u8:0+events_unbound19:31:441
1426460993631,3cyclictest1457399-21ntpq19:35:213
1426456993629,5cyclictest1488301-21ntpq20:05:202
1426454993633,2cyclictest1452736-21kworker/u8:2+events_unbound19:40:131
1426460993532,2cyclictest1545986-21kworker/u8:2+flush-259:022:35:003
1426456993530,4cyclictest335557-21systemd-journal22:17:282
1426454993533,2cyclictest1718283-21kworker/u8:3+flush-259:000:00:231
1426454993532,2cyclictest1545986-21kworker/u8:2+events_unbound22:40:131
1426454993532,2cyclictest1420922-21kworker/u8:0+events_unbound19:39:451
1426454993531,3cyclictest1494626-21apt-get20:15:121
1426454993529,4cyclictest1687223-21latency_hist23:20:001
1426454993431,2cyclictest1545986-21kworker/u8:2+events_unbound22:45:231
1426450993429,3cyclictest1545986-21kworker/u8:2+flush-259:022:10:140
1426460993329,3cyclictest1671231-21kworker/u8:0+flush-259:023:20:153
1426456993329,3cyclictest1509818-21kworker/u8:3+flush-259:020:35:232
1426456993329,3cyclictest1499534-21kworker/u8:0+flush-259:021:20:202
1426454993331,2cyclictest1525335-21kworker/u8:1+events_unbound20:45:201
1426454993330,2cyclictest1609193-21kworker/u8:3+flush-259:022:55:181
1426454993330,2cyclictest1609193-21kworker/u8:3+events_unbound23:05:241
1426454993330,2cyclictest1437088-21kworker/u8:1+flush-259:019:19:541
1426454993230,2cyclictest1609193-21kworker/u8:3+events_unbound23:10:131
1426454993229,2cyclictest1525335-21kworker/u8:1+events_unbound20:50:101
1426454993229,2cyclictest1525335-21kworker/u8:1+events_unbound20:50:101
1426460993128,2cyclictest1682269-21kworker/u8:1+flush-259:023:15:233
1426456993127,3cyclictest1545986-21kworker/u8:2+flush-259:022:20:162
1426454993129,2cyclictest1452736-21kworker/u8:2+flush-259:019:50:191
1426454993128,2cyclictest1671231-21kworker/u8:0+events_unbound23:55:141
1426454993128,2cyclictest1582987-21kworker/u8:1+events_unbound22:05:141
1426454993128,2cyclictest1499534-21kworker/u8:0+flush-259:020:30:191
1426450993126,4cyclictest1557368-21base6421:15:010
1426460993023,5cyclictest335557-21systemd-journal23:00:293
1426456993026,3cyclictest1525585-21apt-get20:45:112
1426454993028,2cyclictest1545986-21kworker/u8:2+flush-259:021:05:231
1426454993028,2cyclictest1452736-21kworker/u8:2+events_unbound20:40:121
1426454993027,2cyclictest1760359-21kworker/u8:1+flush-259:000:38:471
1426454993027,2cyclictest1609193-21kworker/u8:3+events_unbound22:15:151
1426454993027,2cyclictest1509818-21kworker/u8:3+events_unbound20:35:171
1426454993026,3cyclictest1713312-21kworker/u8:1+events_unbound00:10:211
1426454993026,3cyclictest1671231-21kworker/u8:0+events_unbound00:30:161
1426456992926,2cyclictest1603629-21kworker/u8:0+events_unbound22:00:242
1426454992927,2cyclictest1499534-21kworker/u8:0+events_unbound21:35:201
1426460992823,3cyclictest1734228-21latency_hist00:05:013
1426456992825,2cyclictest1671231-21kworker/u8:0+events_unbound23:30:222
1426456992825,2cyclictest1609193-21kworker/u8:3+flush-259:023:05:202
1426456992824,3cyclictest1420922-21kworker/u8:0+events_unbound20:04:512
1426456992823,4cyclictest1650508-21latency_hist22:45:002
1426454992827,1cyclictest1545986-21kworker/u8:2+events_unbound21:48:281
1426454992825,2cyclictest1713312-21kworker/u8:1+events_unbound23:45:221
1426454992825,2cyclictest1545986-21kworker/u8:2+events_unbound23:00:141
1426454992825,2cyclictest1545986-21kworker/u8:2+events_unbound22:25:221
1426454992825,2cyclictest1499534-21kworker/u8:0+events_unbound21:40:221
1426454992824,3cyclictest1545999-21gpgv21:04:591
1426454992824,3cyclictest1443047-21apt-get19:25:121
1426450992824,3cyclictest1420922-21kworker/u8:0+flush-259:019:10:230
1426460992723,3cyclictest1671231-21kworker/u8:0+flush-259:023:05:163
1426456992723,3cyclictest1687481-21apt-get23:20:132
1426456992722,3cyclictest1671231-21kworker/u8:0+flush-259:000:00:172
1426454992725,2cyclictest1609193-21kworker/u8:3+events_unbound23:35:161
1426454992724,2cyclictest1671231-21kworker/u8:0+flush-259:000:05:231
1426454992724,2cyclictest1671231-21kworker/u8:0+events_unbound23:40:431
1426454992724,2cyclictest1420922-21kworker/u8:0+flush-259:020:00:241
1426454992724,2cyclictest1420922-21kworker/u8:0+events_unbound19:55:011
1426450992721,4cyclictest1489203-21latency_hist20:10:000
1426450992720,5cyclictest1727777-21ntpq23:55:220
1426460992622,3cyclictest1609193-21kworker/u8:3+flush-259:022:55:163
1426460992622,3cyclictest1499534-21kworker/u8:0+flush-259:020:50:243
1426460992622,3cyclictest1499534-21kworker/u8:0+flush-259:020:50:243
1426460992622,3cyclictest1499534-21kworker/u8:0+flush-259:020:30:163
1426460992621,3cyclictest1567741-21/usr/sbin/munin21:25:213
1426460992620,4cyclictest1644414-21ntpq22:35:193
1426456992624,2cyclictest1671231-21kworker/u8:0+flush-259:023:15:152
1426456992623,2cyclictest1671231-21kworker/u8:0+flush-259:000:25:002
1426456992622,3cyclictest1707503-21kworker/u8:4+flush-259:023:40:202
1426456992622,3cyclictest1609193-21kworker/u8:3+events_unbound22:53:292
1426456992622,3cyclictest1545986-21kworker/u8:2+flush-259:022:30:172
1426456992622,3cyclictest1525335-21kworker/u8:1+flush-259:020:50:162
1426456992622,3cyclictest1525335-21kworker/u8:1+flush-259:020:50:162
1426456992622,3cyclictest1499534-21kworker/u8:0+events_unbound21:41:292
1426454992623,2cyclictest1671231-21kworker/u8:0+flush-259:000:15:261
1426454992623,2cyclictest1671231-21kworker/u8:0+events_unbound23:21:311
1426454992622,3cyclictest1603629-21kworker/u8:0+events_unbound22:00:141
1426460992520,4cyclictest335557-21systemd-journal22:30:003
1426460992518,5cyclictest335557-21systemd-journal00:35:013
1426456992522,3cyclictest1582987-21kworker/u8:1+events_unbound22:14:292
1426456992522,2cyclictest1754648-21kworker/u8:3+flush-259:000:35:182
1426456992522,2cyclictest1671231-21kworker/u8:0+events_unbound23:57:312
1426456992522,2cyclictest1420922-21kworker/u8:0+flush-259:019:10:242
1426454992523,2cyclictest1713312-21kworker/u8:1+events_unbound00:20:141
1426454992522,2cyclictest1645354-21kworker/u8:0+flush-259:022:50:241
1426454992522,2cyclictest1420922-21kworker/u8:0+events_unbound19:10:131
1426450992520,3cyclictest1499782-21apt-get20:20:120
1426450992519,4cyclictest1499534-21kworker/u8:0+events_unbound21:45:000
1426460992420,3cyclictest1452736-21kworker/u8:2+events_unbound19:55:003
1426456992421,2cyclictest1609193-21kworker/u8:3+flush-259:022:25:222
1426456992421,2cyclictest1545986-21kworker/u8:2+flush-259:021:30:212
1426456992421,2cyclictest1499534-21kworker/u8:0+events_unbound20:20:242
1426456992420,3cyclictest1426434-21kworker/u8:3+events_unbound19:27:452
1426456992418,4cyclictest1677063-21cat23:10:012
1426454992422,2cyclictest1713312-21kworker/u8:1+flush-259:023:50:171
1426454992421,2cyclictest1593705-21kworker/u8:3+events_unbound21:50:211
1426454992418,4cyclictest1437419-21/usr/sbin/munin19:20:131
1426454992418,2cyclictest1545986-21kworker/u8:2+events_unbound21:26:281
1426460992319,3cyclictest1734499-21apt-get00:05:163
1426456992320,2cyclictest1545986-21kworker/u8:2+events_unbound23:01:312
1426456992319,3cyclictest1545986-21kworker/u8:2+events_unbound21:59:292
1426454992321,2cyclictest1556783-21kworker/u8:1+events_unbound21:15:221
1426454992320,2cyclictest1545986-21kworker/u8:2+flush-259:021:20:261
1426454992320,2cyclictest1545986-21kworker/u8:2+events_unbound22:20:191
1426454992320,2cyclictest1452736-21kworker/u8:2+events_unbound20:05:031
1426450992317,4cyclictest335557-21systemd-journal22:59:290
1426450992317,4cyclictest1603150-21ntpq21:55:220
1426460992218,3cyclictest1499534-21kworker/u8:0+events_unbound20:25:093
1426460992216,4cyclictest335557-21systemd-journal23:45:293
1426456992219,2cyclictest1452736-21kworker/u8:2+flush-259:019:40:452
1426456992218,3cyclictest1545986-21kworker/u8:2+events_unbound22:45:172
1426454992219,2cyclictest1545986-21kworker/u8:2+events_unbound21:30:201
1426454992215,5cyclictest1489447-21/usr/sbin/munin20:10:201
1426460992118,2cyclictest1489245-21kworker/u8:1+flush-259:020:10:053
1426460992118,2cyclictest1420922-21kworker/u8:0+flush-259:019:25:163
1426460992117,3cyclictest1671231-21kworker/u8:0+events_unbound00:19:263
1426460992117,3cyclictest1545986-21kworker/u8:2+flush-259:021:45:243
1426460992115,4cyclictest1525316-21latency_hist20:45:013
1426456992118,2cyclictest507-21in:imuxsock22:40:002
1426456992118,2cyclictest1545986-21kworker/u8:2+flush-259:021:25:202
1426456992118,2cyclictest1525335-21kworker/u8:1+events_unbound20:55:172
1426456992115,4cyclictest1546413-21cat21:05:012
1426454992118,2cyclictest1671231-21kworker/u8:0+events_unbound23:30:261
1426454992118,2cyclictest1609193-21kworker/u8:3+events_unbound22:35:181
1426454992117,3cyclictest1682269-21kworker/u8:1+events_unbound23:28:301
1426450992115,4cyclictest1509792-21latency_hist20:30:000
1426450992114,5cyclictest1743747-21ntpq00:10:220
1426450992114,4cyclictest1682034-21latency_hist23:15:000
1426460992017,2cyclictest1452736-21kworker/u8:2+events_unbound20:19:593
1426460992016,3cyclictest1452736-21kworker/u8:2+flush-259:020:05:163
1426460992016,3cyclictest1452736-21kworker/u8:2+flush-259:019:40:193
1426460992015,4cyclictest1603629-21kworker/u8:0+events_unbound22:00:283
1426456992017,2cyclictest1499534-21kworker/u8:0+events_unbound21:10:142
1426456992016,3cyclictest1609193-21kworker/u8:3+flush-259:023:35:162
1426456992016,3cyclictest1452736-21kworker/u8:2+events_unbound20:40:252
1426450992016,3cyclictest1452736-21kworker/u8:2+flush-259:020:40:150
1426450992014,4cyclictest1436318-21ntpq19:15:200
1426460991916,2cyclictest1713312-21kworker/u8:1+events_unbound00:10:153
1426460991914,3cyclictest1514971-21turbostat20:39:553
1426460991913,4cyclictest335557-21systemd-journal21:00:003
1426456991916,2cyclictest1593705-21kworker/u8:3+flush-259:021:50:162
1426456991916,2cyclictest1452736-21kworker/u8:2+events_unbound19:35:452
1426456991915,3cyclictest1452736-21kworker/u8:2+flush-259:019:50:202
1426456991913,4cyclictest1452271-21ntpq19:30:212
1426454991917,2cyclictest1582987-21kworker/u8:1+events_unbound22:10:181
1426454991911,2cyclictest191rcu_preempt22:33:291
1426450991916,2cyclictest1463454-21idleruntime-cro19:45:000
1426450991912,5cyclictest1494352-21latency_hist20:15:000
1426460991815,2cyclictest1609193-21kworker/u8:3+flush-259:023:10:163
1426460991814,3cyclictest1452736-21kworker/u8:2+flush-259:019:45:183
1426460991814,2cyclictest0-21swapper/300:00:003
1426460991813,4cyclictest507-21in:imuxsock23:45:003
1426460991813,4cyclictest1697949-21diskstats23:30:143
1426460991812,4cyclictest1604006-21latency_hist22:00:013
1426456991815,2cyclictest1671231-21kworker/u8:0+events_unbound00:15:232
1426456991815,2cyclictest1545986-21kworker/u8:2+events_unbound21:45:152
1426456991814,3cyclictest1556783-21kworker/u8:1+events_unbound21:15:152
1426456991814,3cyclictest1499534-21kworker/u8:0+events_unbound21:35:182
1426456991812,4cyclictest335557-21systemd-journal23:30:002
1426454991815,2cyclictest1499534-21kworker/u8:0+flush-259:020:20:251
1426454991815,2cyclictest1499534-21kworker/u8:0+events_unbound20:25:001
1426450991813,4cyclictest1489245-21kworker/u8:1+events_unbound20:15:180
1426450991813,3cyclictest1729227-21gpgv00:00:020
1426450991811,5cyclictest1759296-21ntpq00:25:230
1426460991714,2cyclictest1556783-21kworker/u8:1+flush-259:021:15:203
1426460991714,2cyclictest1545986-21kworker/u8:2+events_unbound21:08:153
1426460991713,3cyclictest1499534-21kworker/u8:0+events_unbound20:20:183
1426456991714,2cyclictest1713312-21kworker/u8:1+flush-259:000:10:172
1426456991714,2cyclictest1499534-21kworker/u8:0+events_unbound20:30:532
1426456991713,3cyclictest1760359-21kworker/u8:1+events_unbound00:30:302
1426456991713,3cyclictest1671231-21kworker/u8:0+events_unbound00:25:232
1426456991711,4cyclictest1442538-21cat19:25:012
1426454991711,3cyclictest1545986-21kworker/u8:2+events_unbound21:56:301
1426450991713,3cyclictest1452736-21kworker/u8:2+flush-259:019:35:560
1426450991711,4cyclictest1625114-21diskstats22:20:140
1426450991710,5cyclictest1541250-21diskstats21:00:130
1426460991613,2cyclictest1420922-21kworker/u8:0+flush-259:020:00:153
1426460991612,3cyclictest1713312-21kworker/u8:1+flush-259:023:50:163
1426460991612,3cyclictest1593705-21kworker/u8:3+events_unbound21:50:113
1426460991612,3cyclictest1545986-21kworker/u8:2+flush-259:021:10:153
1426460991612,3cyclictest1452736-21kworker/u8:2+events_unbound19:58:443
1426460991610,4cyclictest1707877-21latency_hist23:40:003
1426456991613,2cyclictest1545986-21kworker/u8:2+events_unbound23:00:002
1426450991614,2cyclictest60350irq/132-enp0s29f1:rx-022:50:270
1426450991614,2cyclictest60350irq/132-enp0s29f1:rx-021:24:190
1426450991614,1cyclictest60350irq/132-enp0s29f1:rx-020:35:240
1426450991613,2cyclictest60350irq/132-enp0s29f1:rx-023:16:030
1426450991613,2cyclictest60350irq/132-enp0s29f1:rx-022:31:410
1426450991613,2cyclictest60350irq/132-enp0s29f1:rx-020:00:230
1426450991613,2cyclictest60350irq/132-enp0s29f1:rx-019:45:250
1426450991613,2cyclictest1609205-21sh22:05:000
1426450991613,2cyclictest1557171-21latency_hist21:15:010
1426450991612,3cyclictest1551574-21idleruntime-cro21:10:000
1426450991612,2cyclictest1650503-21turbostat22:45:000
1426460991512,3cyclictest1452736-21kworker/u8:2+events_unbound19:34:593
1426460991512,2cyclictest1754648-21kworker/u8:3+events_unbound00:35:133
1426460991512,2cyclictest1499534-21kworker/u8:0+events_unbound21:23:283
1426460991511,3cyclictest1713312-21kworker/u8:1+flush-259:000:20:253
1426460991511,3cyclictest1671231-21kworker/u8:0+events_unbound23:25:153
1426460991511,3cyclictest1582987-21kworker/u8:1+events_unbound22:10:233
1426456991512,2cyclictest1489245-21kworker/u8:1+events_unbound20:10:202
1426456991511,3cyclictest1452736-21kworker/u8:2+events_unbound19:55:272
142645099158,5cyclictest1540821-21latency_hist21:00:010
1426450991513,2cyclictest60350irq/132-enp0s29f1:rx-000:08:180
1426450991513,1cyclictest60350irq/132-enp0s29f1:rx-022:27:080
1426450991513,1cyclictest60350irq/132-enp0s29f1:rx-022:06:510
1426450991513,1cyclictest60350irq/132-enp0s29f1:rx-019:25:250
1426450991512,2cyclictest60350irq/132-enp0s29f1:rx-022:35:150
1426450991512,2cyclictest60350irq/132-enp0s29f1:rx-022:15:220
1426450991512,2cyclictest60350irq/132-enp0s29f1:rx-019:58:300
1426450991512,2cyclictest1769566-21ntp_kernel_pll_00:35:210
1426450991511,2cyclictest335557-21systemd-journal23:30:300
1426450991510,3cyclictest60350irq/132-enp0s29f1:rx-000:30:220
1426460991410,3cyclictest1426434-21kworker/u8:3+events_unbound19:20:113
142645099141,4cyclictest1510482-21cut20:30:190
1426450991413,1cyclictest60350irq/132-enp0s29f1:rx-023:02:260
1426450991413,1cyclictest60350irq/132-enp0s29f1:rx-021:45:150
1426450991412,1cyclictest60350irq/132-enp0s29f1:rx-019:30:160
1426450991411,2cyclictest60350irq/132-enp0s29f1:rx-023:40:110
1426450991411,2cyclictest60350irq/132-enp0s29f1:rx-023:35:120
1426450991411,2cyclictest60350irq/132-enp0s29f1:rx-021:50:420
1426450991411,2cyclictest60350irq/132-enp0s29f1:rx-000:20:120
142646099139,3cyclictest1525335-21kworker/u8:1+events_unbound20:48:123
1426460991310,2cyclictest1609193-21kworker/u8:3+events_unbound22:55:003
1426460991310,2cyclictest1499534-21kworker/u8:0+events_unbound21:35:143
1426456991310,2cyclictest1452736-21kworker/u8:2+events_unbound20:19:482
1426450991311,1cyclictest60350irq/132-enp0s29f1:rx-023:52:300
1426450991311,1cyclictest60350irq/132-enp0s29f1:rx-019:50:230
1426450991310,2cyclictest61150irq/133-enp0s29f1:tx-019:20:140
1426450991310,2cyclictest60350irq/132-enp0s29f1:rx-022:45:120
142646099129,2cyclictest1671231-21kworker/u8:0+flush-259:000:25:313
142646099129,2cyclictest1545986-21kworker/u8:2+events_unbound21:30:133
142646099128,3cyclictest1588120-21latency_hist21:45:003
142646099128,3cyclictest1545986-21kworker/u8:2+flush-259:022:48:393
142646099128,3cyclictest1545986-21kworker/u8:2+events_unbound22:20:153
142646099125,5cyclictest1431596-21apt-get19:15:023
142646099120,10cyclictest1650512-21awk22:45:003
142645699128,3cyclictest1582987-21kworker/u8:1+events_unbound22:06:002
142645099129,2cyclictest60350irq/132-enp0s29f1:rx-021:25:150
142645099129,2cyclictest60350irq/132-enp0s29f1:rx-020:45:190
142645099129,2cyclictest60350irq/132-enp0s29f1:rx-000:15:210
1426450991211,1cyclictest60350irq/132-enp0s29f1:rx-023:49:510
142646099118,2cyclictest1545986-21kworker/u8:2+flush-259:022:17:053
142645699118,2cyclictest1692372-21kworker/u8:2+events_unbound23:49:322
142645099119,2cyclictest60350irq/132-enp0s29f1:rx-023:20:250
142645099118,2cyclictest1578139-21cpuspeed_turbos21:35:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional