You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-11 - 14:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sat Jul 11, 2026 00:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1221699993633,2cyclictest1330747-21kworker/u8:0+flush-259:022:10:223
1221692993631,3cyclictest1268544-21/usr/sbin/munin19:55:151
1221699993532,2cyclictest1268344-21kworker/u8:3+flush-259:020:10:303
1221699993431,2cyclictest1330747-21kworker/u8:0+events_unbound21:17:253
1221699993330,2cyclictest1242586-21kworker/u8:0+events_unbound19:55:243
1221688993224,5cyclictest1538190-21latency_hist00:15:010
1221699993128,2cyclictest1517169-21kworker/u8:0+flush-259:000:02:363
1221699993128,2cyclictest1304801-21kworker/u8:2+events_unbound20:45:253
1221699993128,2cyclictest1163264-21kworker/u8:1+events_unbound19:37:253
1221696993128,2cyclictest1454982-21kworker/u8:2+events_unbound23:40:012
1221692993126,4cyclictest1532576-21latency_hist00:10:011
1221699993026,3cyclictest1341073-21kworker/u8:2+events_unbound22:40:143
1221688993026,3cyclictest1325404-21kworker/u8:4+flush-259:021:05:010
1221688993013,4cyclictest1429881-21diskstats22:30:130
1221699992926,2cyclictest1397726-21kworker/u8:1+events_unbound22:40:003
1221699992926,2cyclictest1330747-21kworker/u8:0+flush-259:021:40:233
1221699992926,2cyclictest1289195-21kworker/u8:1+flush-259:020:40:453
1221699992926,2cyclictest1268344-21kworker/u8:3+events_unbound20:30:123
1221699992926,2cyclictest1188736-21kworker/u8:3+events_unbound19:11:083
1221699992825,2cyclictest1341073-21kworker/u8:2+events_unbound21:35:133
1221699992825,2cyclictest1330747-21kworker/u8:0+events_unbound22:15:253
1221699992823,3cyclictest1227359-21/usr/sbin/munin19:15:143
1221696992825,2cyclictest1543354-21kworker/u8:2+events_unbound00:35:212
1221692992825,2cyclictest1491306-21kworker/u8:3+flush-259:000:00:011
1221692992825,2cyclictest1188736-21kworker/u8:3+events_unbound19:20:001
122168899289,5cyclictest1258024-21latency_hist19:45:010
1221688992820,6cyclictest1521442-21ntpq23:55:220
1221699992724,2cyclictest1491306-21kworker/u8:3+events_unbound00:25:143
1221699992724,2cyclictest1242586-21kworker/u8:0+events_unbound20:05:143
1221699992724,2cyclictest1242586-21kworker/u8:0+events_unbound19:40:213
1221696992724,2cyclictest1491306-21kworker/u8:3+flush-259:000:00:212
1221688992712,2cyclictest0-21swapper/021:10:000
1221699992623,2cyclictest1341073-21kworker/u8:2+events_unbound21:30:153
1221699992623,2cyclictest1330747-21kworker/u8:0+events_unbound21:06:263
1221696992623,2cyclictest1517169-21kworker/u8:0+flush-259:000:05:222
1221692992621,4cyclictest1377033-21latency_hist21:40:011
1221688992619,5cyclictest335557-21systemd-journal23:55:000
1221688992619,5cyclictest1454453-21ntpq22:50:210
1221699992522,2cyclictest1491306-21kworker/u8:3+flush-259:023:30:233
1221699992522,2cyclictest1397726-21kworker/u8:1+events_unbound22:55:213
1221699992522,2cyclictest1397726-21kworker/u8:1+events_unbound22:55:213
1221699992522,2cyclictest1330747-21kworker/u8:0+events_unbound21:51:273
1221699992522,2cyclictest1242586-21kworker/u8:0+flush-259:019:50:153
1221699992521,3cyclictest1491306-21kworker/u8:3+events_unbound23:41:193
1221699992519,4cyclictest1424554-21apt-get22:25:093
1221699992421,2cyclictest1330747-21kworker/u8:0+flush-259:021:10:513
1221699992421,2cyclictest1268344-21kworker/u8:3+events_unbound20:50:483
1221696992421,2cyclictest1481107-21kworker/u8:1+events_unbound23:24:272
1221696992421,2cyclictest1242586-21kworker/u8:0+events_unbound20:05:252
1221692992421,2cyclictest1491306-21kworker/u8:3+flush-259:023:30:221
1221692992420,3cyclictest1454982-21kworker/u8:2+events_unbound23:52:251
1221692992418,4cyclictest335557-21systemd-journal22:15:001
1221688992420,3cyclictest1517169-21kworker/u8:0+flush-259:000:20:200
1221699992320,2cyclictest1465681-21kworker/u8:3+flush-259:023:05:213
1221699992320,2cyclictest1397726-21kworker/u8:1+events_unbound22:45:233
1221699992319,3cyclictest1289195-21kworker/u8:1+events_unbound20:22:253
1221696992320,2cyclictest1454982-21kworker/u8:2+events_unbound23:50:132
1221696992320,2cyclictest1188736-21kworker/u8:3+flush-259:019:10:242
1221696992319,3cyclictest1341073-21kworker/u8:2+flush-259:021:25:222
1221696992319,3cyclictest1330747-21kworker/u8:0+events_unbound21:15:182
1221696992319,3cyclictest1163264-21kworker/u8:1+events_unbound19:40:132
1221688992320,2cyclictest1273643-21gpgv19:59:590
1221688992315,5cyclictest1263006-21kworker/u8:2+flush-259:020:15:240
1221699992219,2cyclictest1543354-21kworker/u8:2+events_unbound00:35:153
1221699992219,2cyclictest1397726-21kworker/u8:1+events_unbound22:30:053
1221699992219,2cyclictest1325404-21kworker/u8:4+flush-259:021:00:203
1221699992218,3cyclictest1330747-21kworker/u8:0+events_unbound21:22:253
1221696992219,3cyclictest1263006-21kworker/u8:2+flush-259:020:10:052
1221692992218,3cyclictest1341073-21kworker/u8:2+flush-259:022:25:161
1221692992217,4cyclictest335557-21systemd-journal00:40:001
1221688992215,5cyclictest1325378-21latency_hist20:50:010
1221699992118,2cyclictest1454982-21kworker/u8:2+events_unbound23:11:133
1221699992117,3cyclictest1263006-21kworker/u8:2+events_unbound20:15:073
1221699992117,3cyclictest1263006-21kworker/u8:2+events_unbound20:01:133
1221696992118,2cyclictest1188736-21kworker/u8:3+events_unbound19:23:242
1221696992116,4cyclictest1480211-21ntpq23:15:222
1221696992116,4cyclictest1299613-21latency_hist20:25:012
1221692992119,1cyclictest1188736-21kworker/u8:3+events_unbound19:10:251
1221692992118,2cyclictest1454982-21kworker/u8:2+events_unbound23:22:261
1221692992118,2cyclictest1341073-21kworker/u8:2+events_unbound22:42:251
1221692992118,2cyclictest1340867-21kworker/u8:1+flush-259:021:30:311
1221692992118,2cyclictest1289195-21kworker/u8:1+events_unbound20:37:261
1221692992117,3cyclictest1418916-21kworker/u8:4+events_unbound22:30:261
1221692992117,3cyclictest1330747-21kworker/u8:0+events_unbound22:07:271
1221688992116,4cyclictest1341073-21kworker/u8:2+flush-259:021:25:160
1221699992017,2cyclictest1491306-21kworker/u8:3+events_unbound23:46:463
1221699992017,2cyclictest1418949-21apt-get22:20:103
1221699992016,3cyclictest1491306-21kworker/u8:3+events_unbound23:55:203
1221699992016,3cyclictest1341073-21kworker/u8:2+events_unbound21:25:223
1221699992015,2cyclictest1163264-21kworker/u8:1+flush-259:019:30:013
1221699992014,4cyclictest1382458-21apt-get21:45:103
1221696992017,2cyclictest1330747-21kworker/u8:0+events_unbound21:20:142
1221692992017,2cyclictest1163264-21kworker/u8:1+flush-259:019:40:231
1221692992016,3cyclictest1424044-21base6422:25:001
1221692992016,3cyclictest1397726-21kworker/u8:1+events_unbound22:53:271
1221692992016,3cyclictest1242586-21kworker/u8:0+flush-259:020:00:251
1221688992013,5cyclictest1221441-21latency_hist19:10:010
1221688992013,2cyclictest60350irq/132-enp0s29f1:rx-022:43:260
1221688992010,2cyclictest60350irq/132-enp0s29f1:rx-020:10:230
1221699991916,2cyclictest1330747-21kworker/u8:0+events_unbound22:00:173
1221699991914,4cyclictest335557-21systemd-journal21:00:003
1221696991916,2cyclictest1517169-21kworker/u8:0+flush-259:000:20:212
1221696991916,2cyclictest1491306-21kworker/u8:3+events_unbound23:34:272
1221696991916,2cyclictest1330747-21kworker/u8:0+flush-259:020:55:202
1221696991915,3cyclictest1341073-21kworker/u8:2+flush-259:021:35:562
1221696991915,3cyclictest1289195-21kworker/u8:1+events_unbound20:39:272
1221692991916,2cyclictest1304801-21kworker/u8:2+events_unbound20:45:181
1221692991916,2cyclictest1289195-21kworker/u8:1+events_unbound20:42:251
1221692991915,3cyclictest509-21rs:main1
1221692991915,3cyclictest1434850-21kworker/u8:3+events_unbound22:36:101
1221692991915,3cyclictest1268344-21kworker/u8:3+events_unbound21:04:271
1221692991915,3cyclictest1268344-21kworker/u8:3+events_unbound20:20:161
1221688991913,1cyclictest60350irq/132-enp0s29f1:rx-020:55:240
1221688991913,1cyclictest60350irq/132-enp0s29f1:rx-020:50:220
1221699991815,2cyclictest1517169-21kworker/u8:0+events_unbound00:05:233
1221699991815,2cyclictest1397726-21kworker/u8:1+flush-259:023:00:193
1221699991815,2cyclictest1397726-21kworker/u8:1+flush-259:023:00:193
1221699991815,2cyclictest1289195-21kworker/u8:1+flush-259:020:25:193
1221699991815,2cyclictest1268344-21kworker/u8:3+events_unbound20:35:073
1221699991814,3cyclictest1188736-21kworker/u8:3+events_unbound19:30:163
1221696991815,2cyclictest1517169-21kworker/u8:0+events_unbound00:26:382
1221696991815,2cyclictest1418916-21kworker/u8:4+events_unbound22:30:202
1221696991815,2cyclictest1340867-21kworker/u8:1+flush-259:021:05:212
1221696991815,2cyclictest1263006-21kworker/u8:2+events_unbound20:15:212
1221696991814,3cyclictest1377294-21apt-get21:40:112
1221696991814,3cyclictest1341073-21kworker/u8:2+events_unbound21:46:012
1221696991814,3cyclictest1330747-21kworker/u8:0+events_unbound22:06:252
1221696991813,4cyclictest1279032-21latency_hist20:05:002
1221692991816,1cyclictest1341073-21kworker/u8:2+events_unbound21:45:591
1221692991815,2cyclictest1397726-21kworker/u8:1+events_unbound23:03:271
1221692991815,2cyclictest1397726-21kworker/u8:1+events_unbound23:03:271
1221692991815,2cyclictest1330747-21kworker/u8:0+flush-259:021:15:161
1221692991815,2cyclictest1325404-21kworker/u8:4+flush-259:020:55:261
1221692991814,3cyclictest1491306-21kworker/u8:3+flush-259:000:00:211
1221692991814,3cyclictest1454982-21kworker/u8:2+events_unbound23:10:001
1221692991814,3cyclictest1263006-21kworker/u8:2+events_unbound20:15:381
1221688991813,4cyclictest1289195-21kworker/u8:1+flush-259:020:39:590
1221688991812,1cyclictest60350irq/132-enp0s29f1:rx-023:13:230
1221699991714,2cyclictest1491306-21kworker/u8:3+events_unbound00:10:173
1221699991714,2cyclictest1330747-21kworker/u8:0+events_unbound22:05:023
1221696991715,2cyclictest1517169-21kworker/u8:0+events_unbound23:59:262
1221696991714,2cyclictest1491306-21kworker/u8:3+events_unbound00:13:332
1221696991714,2cyclictest1397726-21kworker/u8:1+events_unbound22:38:272
1221696991714,2cyclictest1289195-21kworker/u8:1+events_unbound20:50:102
1221696991713,3cyclictest1517169-21kworker/u8:0+events_unbound00:30:232
1221696991713,3cyclictest1454982-21kworker/u8:2+events_unbound23:13:262
1221696991713,3cyclictest1445050-21idleruntime-cro22:45:002
1221696991713,3cyclictest1268344-21kworker/u8:3+events_unbound20:31:262
1221696991713,3cyclictest1163264-21kworker/u8:1+flush-259:019:45:202
1221692991714,2cyclictest1330747-21kworker/u8:0+events_unbound22:16:261
1221692991714,2cyclictest1330747-21kworker/u8:0+events_unbound22:00:231
1221692991714,2cyclictest1289195-21kworker/u8:1+events_unbound20:25:211
1221692991714,2cyclictest1163264-21kworker/u8:1+events_unbound19:25:241
1221692991713,3cyclictest1481107-21kworker/u8:1+flush-259:023:25:261
1221692991713,3cyclictest1434850-21kworker/u8:3+events_unbound22:46:111
1221692991712,3cyclictest1387661-21/usr/sbin/munin21:50:171
1221692991711,5cyclictest335557-21systemd-journal19:46:231
1221688991714,2cyclictest60350irq/132-enp0s29f1:rx-000:25:150
1221688991713,3cyclictest60350irq/132-enp0s29f1:rx-023:15:200
1221688991713,3cyclictest1242586-21kworker/u8:0+events_unbound20:00:180
1221688991710,5cyclictest1381266-21ntpq21:40:190
1221699991613,2cyclictest1454982-21kworker/u8:2+events_unbound23:52:273
1221699991613,2cyclictest1454982-21kworker/u8:2+events_unbound23:26:143
1221696991613,2cyclictest1491306-21kworker/u8:3+events_unbound00:15:172
1221696991613,2cyclictest1397726-21kworker/u8:1+events_unbound23:00:242
1221696991613,2cyclictest1397726-21kworker/u8:1+events_unbound23:00:242
1221696991613,2cyclictest1304801-21kworker/u8:2+events_unbound20:40:162
1221696991613,2cyclictest1242586-21kworker/u8:0+flush-259:019:55:212
1221696991612,3cyclictest1454982-21kworker/u8:2+flush-259:023:40:222
1221696991612,3cyclictest1397726-21kworker/u8:1+events_unbound22:56:262
1221696991612,3cyclictest1397726-21kworker/u8:1+events_unbound22:56:262
1221696991612,3cyclictest1163264-21kworker/u8:1+events_unbound19:31:242
1221692991612,3cyclictest1361783-21/usr/sbin/munin21:25:151
1221692991612,3cyclictest1340867-21kworker/u8:1+flush-259:021:10:161
1221692991612,3cyclictest1289195-21kworker/u8:1+flush-259:020:50:161
1221692991612,2cyclictest0-21swapper/123:15:001
122168899169,5cyclictest1424335-21cat22:25:000
1221688991614,2cyclictest60350irq/132-enp0s29f1:rx-000:00:140
1221688991614,1cyclictest60350irq/132-enp0s29f1:rx-020:05:250
1221688991613,2cyclictest61150irq/133-enp0s29f1:tx-023:40:150
1221688991613,2cyclictest60350irq/132-enp0s29f1:rx-023:30:290
1221688991613,2cyclictest60350irq/132-enp0s29f1:rx-022:25:140
1221688991613,2cyclictest60350irq/132-enp0s29f1:rx-020:20:030
1221688991612,2cyclictest60350irq/132-enp0s29f1:rx-021:20:180
1221699991512,2cyclictest1517169-21kworker/u8:0+events_unbound00:30:313
1221699991511,3cyclictest1538455-21apt-get00:15:123
1221696991512,2cyclictest1454982-21kworker/u8:2+events_unbound23:45:122
1221696991512,2cyclictest1418916-21kworker/u8:4+events_unbound22:25:132
1221696991512,2cyclictest1163264-21kworker/u8:1+events_unbound19:35:262
1221696991511,3cyclictest1340867-21kworker/u8:1+events_unbound21:31:262
1221692991512,2cyclictest1163264-21kworker/u8:1+events_unbound19:35:251
1221692991512,2cyclictest1163264-21kworker/u8:1+events_unbound19:20:261
1221692991511,3cyclictest1491306-21kworker/u8:3+events_unbound23:40:131
1221692991511,3cyclictest1340867-21kworker/u8:1+flush-259:021:06:351
1221692991511,3cyclictest1268344-21kworker/u8:3+flush-259:020:30:161
1221692991511,3cyclictest1263006-21kworker/u8:2+events_unbound20:10:221
1221688991514,1cyclictest60350irq/132-enp0s29f1:rx-021:30:390
1221688991513,1cyclictest60350irq/132-enp0s29f1:rx-020:44:270
1221688991513,1cyclictest60350irq/132-enp0s29f1:rx-019:50:240
1221688991513,1cyclictest60350irq/132-enp0s29f1:rx-019:10:120
1221688991513,1cyclictest1237451-21turbostat19:25:010
1221688991512,2cyclictest60350irq/132-enp0s29f1:rx-023:05:530
1221688991512,2cyclictest60350irq/132-enp0s29f1:rx-022:05:250
1221688991512,2cyclictest60350irq/132-enp0s29f1:rx-021:52:260
1221688991512,2cyclictest60350irq/132-enp0s29f1:rx-021:35:220
1221688991512,2cyclictest60350irq/132-enp0s29f1:rx-000:38:530
1221688991511,3cyclictest60350irq/132-enp0s29f1:rx-019:40:010
1221688991510,3cyclictest60350irq/132-enp0s29f1:rx-023:45:110
1221688991510,3cyclictest1397726-21kworker/u8:1+flush-259:022:55:160
1221688991510,3cyclictest1397726-21kworker/u8:1+flush-259:022:55:160
1221699991410,3cyclictest1517169-21kworker/u8:0+events_unbound00:24:563
122169699143,6cyclictest1340847-21munin-run21:05:002
1221696991412,2cyclictest1418916-21kworker/u8:4+events_unbound22:20:232
1221696991411,2cyclictest1445291-21kworker/u8:0+events_unbound23:05:142
1221696991411,2cyclictest1320879-21irqcore20:45:172
1221696991410,3cyclictest1299268-21kworker/u8:0+events_unbound20:26:252
1221692991411,2cyclictest1491306-21kworker/u8:3+flush-259:023:35:301
1221692991411,2cyclictest1491306-21kworker/u8:3+events_unbound00:11:021
1221692991410,3cyclictest1517169-21kworker/u8:0+events_unbound00:33:361
1221692991410,3cyclictest1491306-21kworker/u8:3+flush-259:000:15:161
1221692991410,3cyclictest1330747-21kworker/u8:0+flush-259:021:43:071
1221688991412,1cyclictest60350irq/132-enp0s29f1:rx-023:35:220
1221688991412,1cyclictest60350irq/132-enp0s29f1:rx-022:45:240
1221688991412,1cyclictest60350irq/132-enp0s29f1:rx-021:45:210
1221688991412,1cyclictest60350irq/132-enp0s29f1:rx-020:30:150
1221688991411,2cyclictest60350irq/132-enp0s29f1:rx-023:20:250
1221688991411,2cyclictest60350irq/132-enp0s29f1:rx-022:02:360
1221688991410,2cyclictest60350irq/132-enp0s29f1:rx-023:25:130
1221688991410,2cyclictest0-21swapper/019:35:000
1221699991310,2cyclictest1163264-21kworker/u8:1+flush-259:019:45:063
122169699135,1cyclictest0-21swapper/223:30:002
1221696991310,2cyclictest1341073-21kworker/u8:2+events_unbound21:55:152
1221696991310,2cyclictest1340867-21kworker/u8:1+events_unbound21:10:232
122169299137,5cyclictest335557-21systemd-journal00:29:591
1221692991310,2cyclictest1457942-21sort22:55:171
1221692991310,2cyclictest1457942-21sort22:55:171
122168899139,2cyclictest1435904-21kthreadcore22:35:160
122168899137,4cyclictest335557-21systemd-journal00:33:350
1221688991311,1cyclictest60350irq/132-enp0s29f1:rx-021:15:160
1221688991311,1cyclictest60350irq/132-enp0s29f1:rx-020:25:130
1221688991311,1cyclictest60350irq/132-enp0s29f1:rx-019:15:150
1221688991310,2cyclictest1543311-21cron00:20:000
122169699128,3cyclictest1263006-21kworker/u8:2+events_unbound19:50:342
122169699127,4cyclictest1413808-21apt-get22:15:122
122169299129,2cyclictest1330747-21kworker/u8:0+flush-259:021:20:251
122168899129,2cyclictest1412566-21needreboot22:10:180
122169999117,3cyclictest1341073-21kworker/u8:2+events_unbound21:56:423
122169699118,2cyclictest1397726-21kworker/u8:1+flush-259:022:50:122
122169299118,2cyclictest1480071-21needreboot23:15:191
122169299118,2cyclictest1341073-21kworker/u8:2+flush-259:021:55:121
122169299118,2cyclictest1263006-21kworker/u8:2+flush-259:019:50:191
122169299106,3cyclictest1454982-21kworker/u8:2+events_unbound23:45:151
122169299106,3cyclictest1242586-21kworker/u8:0+flush-259:020:06:411
12216929975,1cyclictest0-21swapper/100:20:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional