You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-23 - 15:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Mon Mar 23, 2026 12:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
995774994235,5cyclictest1176965-21cp10:05:002
995770994238,3cyclictest1182464-21kworker/u8:3+flush-259:010:20:151
995770993936,2cyclictest1244222-21kworker/u8:1+events_unbound11:40:231
995768993933,4cyclictest1125567-21/usr/sbin/munin09:15:240
995770993835,2cyclictest1249566-21kworker/u8:0+flush-259:011:15:161
995770993835,2cyclictest1182464-21kworker/u8:3+flush-259:010:15:161
995778993733,3cyclictest1182464-21kworker/u8:3+flush-259:010:20:183
995770993733,3cyclictest1275198-21kworker/u8:2+events_unbound12:30:171
995770993633,2cyclictest1073863-21kworker/u8:2+events_unbound09:00:171
995778993531,2cyclictest1005997-21kworker/u8:0+events_unbound08:15:203
995770993532,2cyclictest990645-21kworker/u8:2+events_unbound07:30:191
995770993531,3cyclictest1073863-21kworker/u8:2+flush-259:009:05:241
995770993531,2cyclictest1203002-21kworker/u8:0+flush-259:010:35:201
995770993530,3cyclictest1037466-21kworker/u8:1+flush-259:008:15:211
995768993529,4cyclictest1073863-21kworker/u8:2+flush-259:009:10:160
995778993430,2cyclictest1275198-21kworker/u8:2+flush-259:012:15:173
995778993429,3cyclictest1048347-21/usr/sbin/munin08:00:143
995774993431,2cyclictest1322034-21kworker/u8:0+flush-259:012:35:272
995770993431,2cyclictest1182464-21kworker/u8:3+flush-259:010:25:221
995770993431,2cyclictest1073863-21kworker/u8:2+events_unbound10:50:231
995770993430,3cyclictest1005997-21kworker/u8:0+events_unbound07:50:141
995770993428,4cyclictest252-21systemd-journal09:43:251
995778993327,4cyclictest1327601-21diskstats12:30:143
995770993329,3cyclictest1275198-21kworker/u8:2+flush-259:011:50:151
995770993328,3cyclictest1228726-21kworker/u8:3+events_unbound11:00:121
995770993229,2cyclictest1255128-21kworker/u8:4+events_unbound11:20:241
995770993228,2cyclictest1073863-21kworker/u8:2+events_unbound09:50:191
995770993227,3cyclictest1182464-21kworker/u8:3+flush-259:010:30:151
995770993128,2cyclictest1005997-21kworker/u8:0+flush-259:009:55:111
995770993127,3cyclictest990645-21kworker/u8:2+events_unbound07:35:191
995778993027,2cyclictest1203002-21kworker/u8:0+events_unbound10:55:253
995778993026,3cyclictest1151299-21kworker/u8:4+flush-259:009:40:243
995778993026,2cyclictest1203002-21kworker/u8:0+flush-259:010:40:163
995778993024,4cyclictest1317091-21/usr/sbin/munin12:20:123
995770993027,2cyclictest1005997-21kworker/u8:0+flush-259:009:45:241
995770993026,3cyclictest1275198-21kworker/u8:2+flush-259:012:20:201
995770993026,3cyclictest1079232-21kworker/u8:4+events_unbound08:50:231
995770993026,3cyclictest1073863-21kworker/u8:2+events_unbound10:05:181
995770992926,2cyclictest957719-21kworker/u8:1+flush-259:007:10:241
99576899290,27cyclictest1-21systemd07:50:000
995770992824,3cyclictest1275198-21kworker/u8:2+events_unbound11:55:161
995778992723,3cyclictest1275277-21gpgv11:40:013
995770992723,3cyclictest1275198-21kworker/u8:2+flush-259:012:25:211
995770992723,3cyclictest1275198-21kworker/u8:2+flush-259:012:15:241
995770992723,3cyclictest1005997-21kworker/u8:0+events_unbound08:20:121
995770992722,3cyclictest1275198-21kworker/u8:2+events_unbound12:00:181
995768992721,4cyclictest1218420-21latency_hist10:45:010
995778992620,4cyclictest1275674-21base6411:40:013
995774992621,4cyclictest1182464-21kworker/u8:3+flush-259:010:30:002
995770992623,2cyclictest1005997-21kworker/u8:0+events_unbound10:00:171
995770992622,3cyclictest1114633-21kworker/u8:1+events_unbound09:15:151
995770992622,3cyclictest1005997-21kworker/u8:0+events_unbound09:30:201
995770992621,4cyclictest1322034-21kworker/u8:0+flush-259:012:35:171
995768992621,4cyclictest1232975-21ntpq10:55:210
995770992522,2cyclictest1228726-21kworker/u8:3+events_unbound11:10:251
995770992521,3cyclictest1073863-21kworker/u8:2+flush-259:009:35:231
995768992521,2cyclictest1073863-21kworker/u8:2+flush-259:009:30:170
995778992421,2cyclictest990645-21kworker/u8:2+flush-259:007:45:053
995778992419,3cyclictest1249566-21kworker/u8:0+flush-259:011:15:163
995774992419,3cyclictest1255128-21kworker/u8:4+flush-259:011:30:142
995770992420,3cyclictest1255128-21kworker/u8:4+events_unbound11:25:141
995770992420,3cyclictest1073863-21kworker/u8:2+flush-259:011:10:011
995770992419,4cyclictest1058660-21apt-get08:10:091
995768992419,4cyclictest1264897-21kworker/u8:0+flush-259:012:10:160
995778992318,4cyclictest1182464-21kworker/u8:3+flush-259:010:15:263
995774992317,4cyclictest509-21rs:main2
995770992319,3cyclictest1005997-21kworker/u8:0+events_unbound09:20:231
99577099230,22cyclictest1037447-21munin-run07:50:001
995768992319,3cyclictest990645-21kworker/u8:2+events_unbound07:55:190
995778992218,3cyclictest252-21systemd-journal11:33:253
995778992218,3cyclictest1073863-21kworker/u8:2+events_unbound08:25:213
995778992218,3cyclictest1005997-21kworker/u8:0+events_unbound09:20:253
995778992216,4cyclictest1151070-21latency_hist09:40:013
995778992216,4cyclictest1145069-21ntpq09:30:213
995770992218,3cyclictest1005997-21kworker/u8:0+events_unbound10:10:151
995768992218,3cyclictest1296208-21kworker/u8:3+flush-259:012:20:140
995768992217,4cyclictest1187800-21/usr/sbin/munin10:15:190
995778992118,2cyclictest1079232-21kworker/u8:4+flush-259:008:55:253
995778992118,2cyclictest1037466-21kworker/u8:1+events_unbound08:10:263
995778992117,3cyclictest990645-21kworker/u8:2+events_unbound07:20:133
995774992118,2cyclictest1145939-21munin-run09:35:002
995774992117,3cyclictest1079232-21kworker/u8:4+flush-259:008:35:002
995774992116,3cyclictest1264897-21kworker/u8:0+flush-259:011:40:162
995770992118,2cyclictest990645-21kworker/u8:2+events_unbound07:55:151
995770992118,2cyclictest1264897-21kworker/u8:0+events_unbound12:10:211
995770992118,2cyclictest1264897-21kworker/u8:0+events_unbound11:30:261
995770992118,2cyclictest1005997-21kworker/u8:0+events_unbound08:00:231
995770992118,2cyclictest1005997-21kworker/u8:0+events_unbound07:25:141
995768992117,3cyclictest990645-21kworker/u8:2+flush-259:008:00:170
995768992115,4cyclictest1305657-21ntpq12:05:190
995778992017,2cyclictest1005997-21kworker/u8:0+flush-259:007:25:243
995778992016,3cyclictest1275198-21kworker/u8:2+flush-259:012:00:163
995778992016,3cyclictest1203002-21kworker/u8:0+events_unbound10:50:143
995778992016,3cyclictest1073863-21kworker/u8:2+flush-259:009:10:213
995778992016,3cyclictest1037466-21kworker/u8:1+flush-259:008:20:243
995778992016,2cyclictest1130462-21idleruntime-cro09:20:003
995774992016,3cyclictest1317116-21apt-get12:20:092
995774992015,3cyclictest1005997-21kworker/u8:0+flush-259:009:55:152
995774992013,5cyclictest1042553-21awk07:55:002
995770992017,2cyclictest1079232-21kworker/u8:4+flush-259:008:40:161
995770992016,3cyclictest1255128-21kworker/u8:4+events_unbound11:35:131
995770992016,3cyclictest1079232-21kworker/u8:4+events_unbound08:30:231
995770992013,5cyclictest1031399-21ntpq07:40:211
995768992013,5cyclictest1238138-21ntpq11:00:210
995778991915,3cyclictest252-21systemd-journal07:19:183
995774991914,3cyclictest1073863-21kworker/u8:2+flush-259:010:05:202
995770991913,3cyclictest1296208-21kworker/u8:3+events_unbound12:05:191
995768991916,1cyclictest0-21swapper/007:55:000
995768991915,3cyclictest1213298-21kworker/u8:1+flush-259:010:45:140
995768991915,3cyclictest1213298-21kworker/u8:1+flush-259:010:45:140
995778991815,2cyclictest1005997-21kworker/u8:0+events_unbound09:45:193
99577499185,6cyclictest1016400-21munin-run07:30:012
995770991815,2cyclictest1228726-21kworker/u8:3+events_unbound10:56:271
995770991814,3cyclictest3143626-21kworker/u8:3+events_unbound07:18:241
995770991814,3cyclictest1079232-21kworker/u8:4+events_unbound08:45:261
995770991813,3cyclictest3143626-21kworker/u8:3+events_unbound07:10:011
995768991814,3cyclictest1275198-21kworker/u8:2+flush-259:012:30:160
995768991813,4cyclictest1280952-21kworker/u8:3+flush-259:011:50:010
995768991813,4cyclictest1094687-21/usr/sbin/munin08:45:200
995778991714,2cyclictest990645-21kworker/u8:2+events_unbound07:30:213
995778991714,2cyclictest1228726-21kworker/u8:3+flush-259:011:10:253
995778991713,3cyclictest1079232-21kworker/u8:4+events_unbound08:40:143
995778991713,3cyclictest1005997-21kworker/u8:0+events_unbound07:50:203
995774991714,2cyclictest990645-21kworker/u8:2+flush-259:008:10:002
995774991713,2cyclictest0-21swapper/212:05:012
995770991714,2cyclictest1203002-21kworker/u8:0+events_unbound10:40:241
995770991713,3cyclictest1005997-21kworker/u8:0+flush-259:007:20:231
995768991714,2cyclictest60350irq/132-enp0s29f1:rx-010:25:270
995768991714,1cyclictest0-21swapper/009:55:000
995768991713,2cyclictest0-21swapper/012:40:000
995768991711,4cyclictest1170896-21ntpq09:55:200
995778991613,3cyclictest1005997-21kworker/u8:0+events_unbound07:35:033
995778991613,2cyclictest1079232-21kworker/u8:4+events_unbound08:45:223
995778991613,2cyclictest1073863-21kworker/u8:2+events_unbound09:05:243
995778991613,2cyclictest1005997-21kworker/u8:0+flush-259:007:40:253
995778991612,3cyclictest1280952-21kworker/u8:3+events_unbound11:45:183
99577499161,8cyclictest1006088-21munin-run07:20:012
995774991613,2cyclictest1037466-21kworker/u8:1+flush-259:008:25:002
995774991611,3cyclictest1120380-21/usr/sbin/munin09:10:192
99577499161,13cyclictest1059792-21sshd08:10:162
995774991610,4cyclictest1255128-21kworker/u8:4+flush-259:011:20:172
995774991610,3cyclictest1275198-21kworker/u8:2+flush-259:012:05:142
99577099161,7cyclictest1079007-21munin-run08:30:001
995770991613,2cyclictest1079232-21kworker/u8:4+events_unbound08:35:221
995770991613,2cyclictest1073863-21kworker/u8:2+events_unbound08:57:261
995770991612,3cyclictest1037466-21kworker/u8:1+flush-259:008:05:291
995768991614,1cyclictest60350irq/132-enp0s29f1:rx-008:40:170
995768991613,2cyclictest60350irq/132-enp0s29f1:rx-011:35:160
995768991613,2cyclictest60350irq/132-enp0s29f1:rx-011:30:200
995768991613,2cyclictest60350irq/132-enp0s29f1:rx-011:15:160
995768991613,2cyclictest60350irq/132-enp0s29f1:rx-010:20:210
995768991613,2cyclictest60350irq/132-enp0s29f1:rx-008:50:140
995768991613,2cyclictest60350irq/132-enp0s29f1:rx-008:25:180
995768991613,2cyclictest60350irq/132-enp0s29f1:rx-008:05:190
995768991612,2cyclictest60350irq/132-enp0s29f1:rx-009:00:200
995768991611,3cyclictest1275847-21apt-get11:40:130
995768991611,3cyclictest1208123-21latency_hist10:35:000
995768991611,3cyclictest1186664-21ntpq10:10:200
995778991513,2cyclictest1228726-21kworker/u8:3+events_unbound11:05:233
995778991512,2cyclictest957719-21kworker/u8:1+events_unbound07:10:153
995778991512,2cyclictest1264897-21kworker/u8:0+events_unbound12:06:093
995778991512,2cyclictest1005997-21kworker/u8:0+flush-259:008:55:003
995778991512,1cyclictest0-21swapper/309:50:193
995778991511,3cyclictest1322034-21kworker/u8:0+events_unbound12:35:213
995778991511,3cyclictest1182464-21kworker/u8:3+flush-259:010:35:003
995778991511,3cyclictest1037466-21kworker/u8:1+flush-259:008:05:253
995778991511,3cyclictest1005997-21kworker/u8:0+events_unbound10:10:193
99577499159,4cyclictest1275198-21kworker/u8:2+flush-259:011:50:182
99577499159,4cyclictest1203002-21kworker/u8:0+flush-259:010:40:152
995774991512,2cyclictest1073863-21kworker/u8:2+flush-259:009:05:172
995774991512,2cyclictest0-21swapper/207:35:182
995774991512,1cyclictest0-21swapper/211:00:002
995774991511,3cyclictest1203002-21kworker/u8:0+flush-259:010:35:252
995774991511,3cyclictest1079232-21kworker/u8:4+flush-259:008:50:302
995774991511,3cyclictest1073863-21kworker/u8:2+flush-259:011:10:462
995774991511,3cyclictest1005997-21kworker/u8:0+flush-259:010:10:102
995774991511,2cyclictest0-21swapper/208:40:002
995774991510,3cyclictest1005997-21kworker/u8:0+flush-259:008:15:162
995770991512,2cyclictest1005997-21kworker/u8:0+flush-259:009:25:161
995770991511,3cyclictest1213298-21kworker/u8:1+flush-259:010:45:101
995770991511,3cyclictest1213298-21kworker/u8:1+flush-259:010:45:101
995770991511,2cyclictest0-21swapper/109:15:001
995768991513,1cyclictest60350irq/132-enp0s29f1:rx-012:01:550
995768991513,1cyclictest60350irq/132-enp0s29f1:rx-010:50:150
995768991513,1cyclictest60350irq/132-enp0s29f1:rx-009:45:240
995768991513,1cyclictest60350irq/132-enp0s29f1:rx-008:31:250
995768991513,1cyclictest60350irq/132-enp0s29f1:rx-008:15:370
995768991512,2cyclictest60350irq/132-enp0s29f1:rx-012:29:120
995768991512,2cyclictest60350irq/132-enp0s29f1:rx-012:17:260
995768991512,2cyclictest60350irq/132-enp0s29f1:rx-011:10:110
995768991512,2cyclictest60350irq/132-enp0s29f1:rx-009:35:230
995768991512,2cyclictest60350irq/132-enp0s29f1:rx-009:25:120
995768991512,2cyclictest60350irq/132-enp0s29f1:rx-008:39:330
995768991512,2cyclictest60350irq/132-enp0s29f1:rx-007:35:230
995768991512,2cyclictest60350irq/132-enp0s29f1:rx-007:31:260
995768991512,2cyclictest60350irq/132-enp0s29f1:rx-007:12:090
995768991510,3cyclictest60350irq/132-enp0s29f1:rx-010:35:140
995778991410,3cyclictest1275198-21kworker/u8:2+flush-259:011:55:103
995774991411,2cyclictest1140308-21unixbench_multi09:25:242
995774991410,2cyclictest1327441-21apt-get12:30:122
995768991413,1cyclictest60350irq/132-enp0s29f1:rx-011:46:020
995768991413,1cyclictest60350irq/132-enp0s29f1:rx-007:24:170
995768991412,1cyclictest60350irq/132-enp0s29f1:rx-011:55:190
995768991412,1cyclictest60350irq/132-enp0s29f1:rx-011:23:330
995768991412,1cyclictest60350irq/132-enp0s29f1:rx-009:42:380
995768991412,1cyclictest60350irq/132-enp0s29f1:rx-009:05:240
995768991412,1cyclictest60350irq/132-enp0s29f1:rx-008:20:230
995768991412,1cyclictest60350irq/132-enp0s29f1:rx-007:40:230
995768991412,1cyclictest60350irq/132-enp0s29f1:rx-007:25:190
995768991412,1cyclictest60350irq/132-enp0s29f1:rx-007:15:190
995768991411,2cyclictest60350irq/132-enp0s29f1:rx-010:07:400
995768991411,2cyclictest60350irq/132-enp0s29f1:rx-009:20:140
995768991411,2cyclictest60350irq/132-enp0s29f1:rx-008:10:190
99577899137,4cyclictest1084424-21/usr/sbin/munin08:35:153
99577499139,3cyclictest1291284-21apt-get11:55:102
99577499139,3cyclictest1005997-21kworker/u8:0+flush-259:007:45:212
99577499137,4cyclictest252-21systemd-journal09:05:002
995768991312,1cyclictest60350irq/132-enp0s29f1:rx-011:05:160
995768991310,2cyclictest60350irq/132-enp0s29f1:rx-008:55:140
99577899129,2cyclictest1079686-21irqcore08:30:163
99577899128,3cyclictest1255128-21kworker/u8:4+flush-259:011:23:023
99577899128,3cyclictest1244222-21kworker/u8:1+events_unbound11:25:223
99577899121,9cyclictest1140774-21cron09:30:003
99577499129,2cyclictest1131141-21sed09:20:162
99577499129,2cyclictest1028386-21kthreadcore07:40:172
99577499128,3cyclictest957719-21kworker/u8:1+flush-259:007:10:242
99577499128,3cyclictest401ktimers/209:40:202
99577499128,3cyclictest1228726-21kworker/u8:3+flush-259:011:05:002
99577499128,2cyclictest1047227-21grep07:55:252
99577499121,2cyclictest1094457-21awk08:45:002
995768991211,1cyclictest60350irq/132-enp0s29f1:rx-010:03:320
99577899119,1cyclictest1005997-21kworker/u8:0+events_unbound10:00:223
99577899119,1cyclictest0-21swapper/310:10:003
99577899118,2cyclictest1322322-21cpuspeed_turbos12:25:113
99577899117,2cyclictest1114999-21awk09:05:003
99577899116,4cyclictest990645-21kworker/u8:2+flush-259:007:55:183
99577499118,2cyclictest0-21swapper/210:45:232
99577499118,2cyclictest0-21swapper/210:45:222
99577499118,1cyclictest0-21swapper/212:25:152
99577499117,2cyclictest0-21swapper/210:15:162
99577499106,2cyclictest1006454-21cut07:20:122
9957749996,2cyclictest0-21swapper/209:15:212
9957749995,3cyclictest401ktimers/210:20:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional