You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-10 - 20:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Tue Mar 10, 2026 12:44:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1187495994134,5cyclictest1275574-21latency_hist08:35:001
1187502994034,4cyclictest1328411-21latency_hist09:25:003
1187502993330,2cyclictest1197542-21kworker/u8:1+flush-259:008:00:013
1187502993330,2cyclictest1197542-21kworker/u8:1+flush-259:008:00:003
1187502993225,5cyclictest1420454-21ntpq10:50:183
1187495993225,5cyclictest252-21systemd-journal09:49:591
1187495993127,3cyclictest1473388-21kworker/u8:3+flush-259:012:14:591
1187492993125,4cyclictest252-21systemd-journal10:06:190
1187500993027,2cyclictest1260207-21kworker/u8:3+flush-259:008:30:002
1187495993025,3cyclictest1353996-21kworker/u8:2+flush-259:010:15:141
1187495993025,3cyclictest1353996-21kworker/u8:2+flush-259:010:15:141
1187500992721,4cyclictest252-21systemd-journal11:33:172
1187500992720,5cyclictest1260608-21diskstats08:20:132
1187495992722,3cyclictest1224359-21/usr/sbin/munin07:45:141
1187492992723,3cyclictest1343854-21kworker/u8:3+flush-259:010:00:260
1187492992722,4cyclictest1322833-21apt-get09:20:030
1187500992622,2cyclictest0-21swapper/209:20:002
1187500992522,2cyclictest1353996-21kworker/u8:2+flush-259:011:15:192
1187495992522,2cyclictest1254967-21kworker/u8:1+flush-259:008:55:211
1187492992521,3cyclictest1279763-21kworker/u8:2+events_unbound08:50:000
1187502992420,3cyclictest1473388-21kworker/u8:3+flush-259:012:10:003
1187500992420,3cyclictest1306942-21kworker/u8:0+flush-259:009:15:012
1187495992417,5cyclictest252-21systemd-journal10:45:001
1187500992319,2cyclictest1353996-21kworker/u8:2+flush-259:011:25:012
1187500992317,4cyclictest1405861-21latency_hist10:40:002
1187495992318,3cyclictest1353996-21kworker/u8:2+flush-259:012:05:181
1187492992317,4cyclictest1333555-21latency_hist09:30:000
1187492992316,5cyclictest1224331-21cp07:45:000
1187500992218,3cyclictest1353996-21kworker/u8:2+events_unbound12:07:182
1187500992218,3cyclictest1254967-21kworker/u8:1+flush-259:009:45:002
1187500992217,3cyclictest1473388-21kworker/u8:3+events_unbound11:45:182
1187492992218,3cyclictest1395539-21kworker/u8:3+flush-259:010:50:000
1187492992217,4cyclictest1254141-21ntpq08:10:210
1187492992216,4cyclictest252-21systemd-journal12:16:180
1187502992117,3cyclictest1244934-21kworker/u8:3+flush-259:008:05:263
1187502992115,4cyclictest1312791-21apt-get09:10:113
1187500992118,2cyclictest1380032-21kworker/u8:5+events_unbound10:15:192
1187500992118,2cyclictest1380032-21kworker/u8:5+events_unbound10:15:182
1187500992118,2cyclictest1279763-21kworker/u8:2+flush-259:008:38:482
1187500992116,3cyclictest1446915-21kworker/u8:1+events_unbound11:50:162
1187500992114,4cyclictest1494011-21latency_hist12:05:002
1187495992117,3cyclictest1223814-21gpgv07:45:011
1187495992117,3cyclictest1223814-21gpgv07:45:001
1187492992118,2cyclictest1234458-21kworker/u8:2+events_unbound08:00:000
1187492992118,2cyclictest1234458-21kworker/u8:2+events_unbound08:00:000
1187492992116,3cyclictest1473388-21kworker/u8:3+events_unbound11:50:000
1187502992014,5cyclictest252-21systemd-journal08:15:003
1187500992016,3cyclictest1509507-21kworker/u8:0+flush-259:012:25:172
1187500992016,3cyclictest1380031-21kworker/u8:0+events_unbound10:20:192
1187500992015,3cyclictest1519982-21/usr/sbin/munin12:30:232
1187500992015,3cyclictest1519982-21/usr/sbin/munin12:30:232
1187500992014,4cyclictest1359295-21latency_hist09:55:012
1187500992011,6cyclictest252-21systemd-journal07:45:002
1187500992011,6cyclictest252-21systemd-journal07:44:592
1187495992017,2cyclictest1307181-21apt-get09:05:111
1187495992014,4cyclictest1323046-21cat09:20:001
1187495992014,4cyclictest1284953-21ntpq08:40:201
1187492992017,2cyclictest1380032-21kworker/u8:5+events_unbound10:35:000
1187492992016,3cyclictest1494238-21kworker/u8:4+events_unbound12:15:000
1187492992016,3cyclictest1431905-21kworker/u8:0+events_unbound12:00:170
1187492992016,3cyclictest1380032-21kworker/u8:5+flush-259:010:45:010
1187502991916,1cyclictest0-21swapper/309:45:003
1187502991915,3cyclictest1317927-21apt-get09:15:103
1187502991915,3cyclictest1279763-21kworker/u8:2+flush-259:009:05:013
1187502991914,4cyclictest1254967-21kworker/u8:1+flush-259:008:55:193
1187500991917,1cyclictest0-21swapper/209:24:592
1187500991915,3cyclictest1306942-21kworker/u8:0+events_unbound09:26:202
1187500991915,3cyclictest1254967-21kworker/u8:1+flush-259:009:30:202
1187500991915,3cyclictest1197542-21kworker/u8:1+events_unbound07:25:182
1187500991915,2cyclictest1255019-21sh08:15:002
1187500991912,5cyclictest252-21systemd-journal10:35:012
1187500991912,5cyclictest1514630-21latency_hist12:25:012
1187495991916,2cyclictest509-21rs:main1
1187495991915,3cyclictest2971-21cron10:14:591
1187495991915,3cyclictest1187702-21apt-get07:10:111
1187495991915,2cyclictest0-21swapper/111:15:011
1187495991913,4cyclictest1254967-21kworker/u8:1+flush-259:009:40:171
1187492991916,2cyclictest1369869-21kworker/u8:1+flush-259:010:14:590
1187492991916,2cyclictest1364477-21kworker/u8:4+flush-259:010:55:260
1187492991915,3cyclictest1380031-21kworker/u8:0+events_unbound10:20:130
1187500991815,2cyclictest1353996-21kworker/u8:2+events_unbound11:45:002
1187500991815,2cyclictest1279763-21kworker/u8:2+flush-259:009:00:242
1187500991814,3cyclictest1254967-21kworker/u8:1+events_unbound08:40:132
1187492991814,3cyclictest1380032-21kworker/u8:5+events_unbound10:15:210
1187492991814,3cyclictest1380032-21kworker/u8:5+events_unbound10:15:200
1187492991813,3cyclictest252-21systemd-journal11:24:130
1187492991812,4cyclictest252-21systemd-journal08:58:510
1187502991714,2cyclictest1307603-21irqcore09:05:153
1187502991714,2cyclictest1306942-21kworker/u8:0+flush-259:009:30:003
1187502991714,2cyclictest0-21swapper/311:49:593
1187500991714,2cyclictest1353996-21kworker/u8:2+flush-259:010:55:002
1187500991714,2cyclictest1343854-21kworker/u8:3+flush-259:010:00:192
1187500991714,2cyclictest1343834-21sh09:40:002
1187500991713,3cyclictest1473388-21kworker/u8:3+events_unbound12:14:222
1187500991713,3cyclictest1431905-21kworker/u8:0+flush-259:011:40:012
1187500991713,3cyclictest1369869-21kworker/u8:1+events_unbound10:10:162
1187500991713,3cyclictest1254967-21kworker/u8:1+events_unbound08:30:182
1187500991713,3cyclictest1182340-21kworker/u8:0+events_unbound07:10:212
1187495991714,1cyclictest0-21swapper/111:59:591
1187495991711,4cyclictest1415288-21ntpq10:45:231
1187495991711,4cyclictest1380032-21kworker/u8:5+flush-259:010:25:171
1187495991710,5cyclictest1451430-21ntpq11:20:231
1187492991714,2cyclictest1431673-21latency_hist11:05:000
1187492991714,2cyclictest1339388-21irqcore09:35:170
1187492991714,1cyclictest0-21swapper/011:44:590
1187492991713,3cyclictest1431905-21kworker/u8:0+events_unbound11:35:190
1187492991713,3cyclictest1254967-21kworker/u8:1+events_unbound08:25:000
1187492991713,3cyclictest1198063-21apt-get07:20:100
1187492991712,4cyclictest1354393-21apt-get09:50:110
1187492991712,3cyclictest1446915-21kworker/u8:1+events_unbound11:30:170
1187502991613,2cyclictest1509471-21apt12:20:003
1187502991610,4cyclictest252-21systemd-journal12:22:183
1187500991613,2cyclictest1254967-21kworker/u8:1+events_unbound09:05:132
1187500991612,3cyclictest1364477-21kworker/u8:4+events_unbound10:48:192
1187500991612,3cyclictest1234458-21kworker/u8:2+events_unbound08:15:162
1187500991612,2cyclictest0-21swapper/207:19:592
1187495991613,2cyclictest1286524-21irqcore08:45:171
1187495991612,2cyclictest1426491-21cron11:00:011
1187495991611,4cyclictest252-21systemd-journal07:29:591
1187495991611,4cyclictest1312766-21/usr/sbin/munin09:10:191
1187492991614,1cyclictest60350irq/132-enp0s29f1:rx-012:05:260
1187492991613,2cyclictest60350irq/132-enp0s29f1:rx-012:35:170
1187492991613,2cyclictest60350irq/132-enp0s29f1:rx-011:55:570
1187492991613,2cyclictest60350irq/132-enp0s29f1:rx-011:27:590
1187492991613,2cyclictest60350irq/132-enp0s29f1:rx-007:33:510
1187492991613,2cyclictest1234458-21kworker/u8:2+events_unbound08:15:190
1187492991612,3cyclictest1244934-21kworker/u8:3+events_unbound08:05:150
1187492991611,4cyclictest1244691-21latency_hist08:05:000
118749299160,14cyclictest1354139-21turbostat09:50:000
118750299159,4cyclictest1280696-21latency_hist08:40:003
1187502991512,2cyclictest1288419-21kthreadcore08:45:203
1187502991510,4cyclictest509-21rs:main3
118750099159,4cyclictest1296824-21base6408:55:022
1187500991512,2cyclictest1369869-21kworker/u8:1+events_unbound10:05:152
1187500991512,2cyclictest1149694-21kworker/u8:3+events_unbound07:20:212
1187500991511,3cyclictest1473388-21kworker/u8:3+events_unbound12:38:182
1187500991511,3cyclictest1279763-21kworker/u8:2+events_unbound08:50:202
1187500991511,3cyclictest1244934-21kworker/u8:3+events_unbound08:05:232
1187495991512,2cyclictest1323935-21irqcore09:20:171
1187495991512,2cyclictest1196839-21needreboot07:15:211
1187492991513,2cyclictest60350irq/132-enp0s29f1:rx-007:27:440
1187492991513,1cyclictest60350irq/132-enp0s29f1:rx-011:50:150
1187492991513,1cyclictest60350irq/132-enp0s29f1:rx-010:52:470
1187492991513,1cyclictest60350irq/132-enp0s29f1:rx-008:40:270
1187492991513,1cyclictest60350irq/132-enp0s29f1:rx-007:53:080
1187492991513,1cyclictest60350irq/132-enp0s29f1:rx-007:15:240
1187492991512,2cyclictest60350irq/132-enp0s29f1:rx-007:35:250
1187492991511,3cyclictest1400986-21apt-get10:35:110
118750299148,4cyclictest1218888-21diskstats07:40:133
118750299148,4cyclictest1218888-21diskstats07:40:123
118750299145,1cyclictest13852712ls10:20:013
118750299145,1cyclictest13852712ls10:20:003
1187502991411,2cyclictest1334238-21irqcore09:30:163
1187502991411,2cyclictest1234458-21kworker/u8:2+flush-259:008:15:163
1187502991411,2cyclictest1203690-21grep07:25:163
1187502991411,2cyclictest0-21swapper/310:15:003
1187502991410,3cyclictest1172107-21kworker/u8:5+flush-259:007:10:263
118750099147,5cyclictest1457069-21ntpq11:25:222
118750099141,3cyclictest1364447-21awk10:00:002
1187500991411,2cyclictest1473388-21kworker/u8:3+events_unbound11:59:192
1187500991411,2cyclictest1431905-21kworker/u8:0+events_unbound11:05:202
1187500991411,2cyclictest1343854-21kworker/u8:3+flush-259:009:49:592
1187500991411,2cyclictest0-21swapper/211:15:012
1187500991410,3cyclictest252-21systemd-journal12:20:002
1187500991410,3cyclictest1353996-21kworker/u8:2+flush-259:010:40:212
1187500991410,3cyclictest1217853-21kworker/u8:4+flush-259:007:45:162
1187500991410,3cyclictest1197542-21kworker/u8:1+flush-259:008:00:212
118749599149,4cyclictest252-21systemd-journal07:22:181
118749599148,4cyclictest1473365-21latency_hist11:44:591
1187495991411,2cyclictest1509795-21cpuspeed_turbos12:20:121
1187495991411,2cyclictest0-21swapper/111:55:001
1187495991410,3cyclictest1254967-21kworker/u8:1+flush-259:009:30:241
118749299148,4cyclictest1312739-21cat09:10:010
1187492991413,1cyclictest60350irq/132-enp0s29f1:rx-007:40:160
1187492991413,1cyclictest60350irq/132-enp0s29f1:rx-007:40:150
1187492991412,1cyclictest60350irq/132-enp0s29f1:rx-012:30:240
1187492991412,1cyclictest60350irq/132-enp0s29f1:rx-012:30:230
1187492991412,1cyclictest60350irq/132-enp0s29f1:rx-012:27:580
1187492991412,1cyclictest60350irq/132-enp0s29f1:rx-011:10:150
1187492991412,1cyclictest60350irq/132-enp0s29f1:rx-011:06:380
1187492991412,1cyclictest60350irq/132-enp0s29f1:rx-009:40:190
1187492991412,1cyclictest60350irq/132-enp0s29f1:rx-009:30:050
1187492991412,1cyclictest60350irq/132-enp0s29f1:rx-009:00:230
1187492991412,1cyclictest60350irq/132-enp0s29f1:rx-008:29:450
1187492991410,3cyclictest1260207-21kworker/u8:3+flush-259:008:30:100
118750299139,3cyclictest1385457-21apt-get10:20:103
118750299136,5cyclictest1405853-21awk10:40:003
1187502991311,1cyclictest0-21swapper/310:29:593
1187502991310,2cyclictest1411402-21cpuspeed_turbos10:45:153
1187502991310,2cyclictest1353996-21kworker/u8:2+flush-259:009:55:153
1187502991310,2cyclictest1228274-21needreboot07:45:193
1187502991310,2cyclictest1202996-21cron07:25:003
1187500991310,2cyclictest1197542-21kworker/u8:1+flush-259:007:59:452
1187500991310,2cyclictest1197542-21kworker/u8:1+flush-259:007:59:442
1187500991310,2cyclictest1187484-21cyclictest10:25:132
118749599139,3cyclictest1353996-21kworker/u8:2+flush-259:012:30:161
118749599139,3cyclictest1353996-21kworker/u8:2+flush-259:012:30:161
118749599139,3cyclictest1343854-21kworker/u8:3+flush-259:010:04:241
118749599139,3cyclictest1182340-21kworker/u8:0+flush-259:007:50:241
118749599138,3cyclictest252-21systemd-journal09:59:591
118749599137,4cyclictest1458169-21apt-get11:30:111
118749599131,6cyclictest1218470-21munin-run07:39:591
1187495991310,2cyclictest1370014-21cpuspeed_turbos10:05:131
1187495991310,2cyclictest1-21systemd10:35:011
1187495991310,2cyclictest0-21swapper/111:40:011
118749299139,3cyclictest1353996-21kworker/u8:2+flush-259:011:15:170
118749299138,4cyclictest60350irq/132-enp0s29f1:rx-009:55:180
1187492991312,1cyclictest60350irq/132-enp0s29f1:rx-012:23:450
1187492991311,1cyclictest60350irq/132-enp0s29f1:rx-009:17:200
1187492991311,1cyclictest60350irq/132-enp0s29f1:rx-009:05:140
118750299129,2cyclictest0-21swapper/312:15:003
118750299129,2cyclictest0-21swapper/309:49:593
118750299128,3cyclictest1254967-21kworker/u8:1+flush-259:008:25:013
118750299127,4cyclictest1468209-21latency_hist11:40:013
1187502991210,1cyclictest0-21swapper/310:10:003
1187502991210,1cyclictest0-21swapper/309:35:173
1187502991210,1cyclictest0-21swapper/308:55:003
118750099128,3cyclictest1426127-21kworker/u8:1+events_unbound11:04:192
118750099128,3cyclictest1364477-21kworker/u8:4+events_unbound10:55:222
118750099128,3cyclictest1285838-21kworker/u8:0+flush-259:008:45:162
118749599129,2cyclictest1508993-21unixbench_multi12:15:251
118749599129,2cyclictest1426997-21uniq11:00:141
118749599129,2cyclictest0-21swapper/111:45:171
118749599127,4cyclictest1353996-21kworker/u8:2+flush-259:012:35:311
118749599121,2cyclictest1249888-21awk08:10:001
118749299129,2cyclictest1291473-21fschecks_count08:50:150
118749299129,2cyclictest1188106-21irqcore07:10:160
118750299119,1cyclictest0-21swapper/311:10:163
118750299119,1cyclictest0-21swapper/307:20:003
118750299118,2cyclictest1410595-21unixbench_multi10:40:233
118750299118,2cyclictest1395846-21cpuspeed_turbos10:30:123
118750299117,3cyclictest1353996-21kworker/u8:2+flush-259:012:35:163
118750099115,4cyclictest1217544-21ntpq07:35:212
118749599115,2cyclictest1493992-21cron12:05:001
118750299108,1cyclictest0-21swapper/311:40:173
118749599107,1cyclictest1328857-21fschecks_count09:25:151
118749599106,3cyclictest252-21systemd-journal08:35:171
118749599105,3cyclictest252-21systemd-journal10:55:001
118749599105,3cyclictest252-21systemd-journal08:20:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional