You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-18 - 21:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sat Jul 18, 2026 12:44:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2192985995043,5cyclictest2240454-21gpgv07:55:000
2192993994742,3cyclictest2339332-21kworker/u8:3+events_unbound09:55:232
2192993994542,2cyclictest2444149-21kworker/u8:0+events_unbound11:30:232
2192993994437,5cyclictest2329487-21latency_hist09:20:002
2192993994339,3cyclictest2397984-21kworker/u8:0+events_unbound10:30:232
2192985994235,5cyclictest2449549-21latency_hist11:15:010
2192993994137,3cyclictest2277849-21kworker/u8:2+events_unbound08:35:132
2192996994037,2cyclictest2428876-21kworker/u8:3+events_unbound11:10:233
2192996994037,2cyclictest2349987-21kworker/u8:1+flush-259:011:05:253
2192996994036,2cyclictest2423646-21kworker/u8:2+events_unbound10:55:243
2192993994036,3cyclictest2182761-21kworker/u8:0+events_unbound07:25:012
2192996993936,2cyclictest2349987-21kworker/u8:1+events_unbound10:15:233
2192993993934,2cyclictest2349987-21kworker/u8:1+events_unbound11:00:002
2192985993931,5cyclictest2203735-21diskstats07:20:130
2192996993835,2cyclictest2485689-21kworker/u8:2+flush-259:012:30:003
2192996993834,3cyclictest2266645-21kworker/u8:0+events_unbound08:20:223
2192993993835,2cyclictest2366747-21kworker/u8:0+events_unbound10:10:222
2192993993833,4cyclictest2465050-21latency_hist11:30:012
2192996993734,2cyclictest2277849-21kworker/u8:2+events_unbound08:30:243
2192996993733,3cyclictest2485689-21kworker/u8:2+events_unbound11:45:513
2192996993633,2cyclictest2453710-21kworker/u8:4+events_unbound11:25:203
2192989993629,5cyclictest335557-21systemd-journal12:25:001
2192993993531,3cyclictest2182761-21kworker/u8:0+events_unbound07:15:132
2192985993530,4cyclictest2282818-21apt-get08:35:100
2192993993431,2cyclictest2224518-21kworker/u8:0+events_unbound07:45:242
2192993993430,3cyclictest2323874-21kworker/u8:1+events_unbound09:30:232
2192993993430,3cyclictest2266645-21kworker/u8:0+flush-259:008:27:272
2192993993430,3cyclictest2266645-21kworker/u8:0+events_unbound08:45:122
2192996993330,2cyclictest2234786-21kworker/u8:2+events_unbound08:05:223
2192996993329,3cyclictest2454989-21apt-get11:20:123
2192996993329,3cyclictest2224518-21kworker/u8:0+events_unbound07:40:213
2192993993330,2cyclictest2234786-21kworker/u8:2+events_unbound08:00:242
2192993993329,3cyclictest2387054-21kworker/u8:2+events_unbound10:28:232
2192985993325,5cyclictest335557-21systemd-journal10:15:190
2192996993229,2cyclictest2485689-21kworker/u8:2+events_unbound11:55:133
2192996993229,2cyclictest2408315-21kworker/u8:3+events_unbound10:45:223
2192996993229,2cyclictest2182761-21kworker/u8:0+events_unbound07:15:113
2192996993228,3cyclictest2313480-21kworker/u8:0+events_unbound09:35:233
2192993993229,2cyclictest2349987-21kworker/u8:1+events_unbound10:20:202
2192993993228,3cyclictest2349987-21kworker/u8:1+events_unbound09:50:222
2192996993128,2cyclictest2224518-21kworker/u8:0+events_unbound07:45:223
2192996993126,2cyclictest2485842-21kworker/u8:3+events_unbound11:50:253
2192993993129,2cyclictest2428876-21kworker/u8:3+flush-259:011:10:252
2192993993127,3cyclictest2408315-21kworker/u8:3+flush-259:010:45:262
2192989993124,5cyclictest2360930-21latency_hist09:50:011
2192985993126,4cyclictest2533367-21apt-get12:35:130
2192985993124,4cyclictest2517597-21latency_hist12:20:000
2192985993123,6cyclictest2485284-21ntpq11:45:220
2192993993027,2cyclictest2485689-21kworker/u8:2+events_unbound11:55:152
2192993993027,2cyclictest2344636-21kworker/u8:2+events_unbound09:45:002
2192993993027,2cyclictest2277849-21kworker/u8:2+flush-259:008:30:252
2192993993027,2cyclictest2245980-21kworker/u8:3+events_unbound09:05:262
2192993993026,3cyclictest2501613-21kworker/u8:1+events_unbound12:10:142
2192993993023,5cyclictest2355357-21latency_hist09:45:002
2192989993024,4cyclictest2272019-21/usr/sbin/munin08:25:151
2192996992926,2cyclictest2444149-21kworker/u8:0+events_unbound11:41:243
2192996992926,2cyclictest2313480-21kworker/u8:0+events_unbound09:28:233
2192996992925,3cyclictest2453710-21kworker/u8:4+events_unbound11:39:223
2192996992925,3cyclictest2323874-21kworker/u8:1+events_unbound09:20:133
2192996992923,4cyclictest2382053-21latency_hist10:10:013
2192993992926,2cyclictest2208459-21kworker/u8:2+events_unbound07:25:222
2192989992924,4cyclictest2245980-21kworker/u8:3+flush-259:008:55:181
2192989992922,5cyclictest2298060-21latency_hist08:50:011
2192996992826,2cyclictest2266645-21kworker/u8:0+flush-259:008:25:203
2192996992825,2cyclictest2256233-21kworker/u8:1+flush-259:008:15:253
2192996992825,2cyclictest2245980-21kworker/u8:3+events_unbound09:00:253
2192996992825,2cyclictest2224518-21kworker/u8:0+events_unbound07:56:233
2192996992823,4cyclictest2366285-21gpgv09:55:013
2192996992822,4cyclictest2538482-21cat12:40:003
2192996992822,2cyclictest2344636-21kworker/u8:2+events_unbound09:45:183
2192993992825,2cyclictest2517459-21kworker/u8:0+events_unbound12:25:152
2192993992824,3cyclictest2192962-21kworker/u8:1+events_unbound07:30:142
2192985992823,4cyclictest2298589-21base6408:50:020
2192985992822,4cyclictest335557-21systemd-journal11:35:000
2192985992821,5cyclictest335557-21systemd-journal08:50:000
2192996992721,3cyclictest2234786-21kworker/u8:2+events_unbound07:55:013
2192993992724,2cyclictest2501850-21cp12:05:002
2192993992724,2cyclictest2214563-21kworker/u8:3+events_unbound07:39:222
2192993992723,3cyclictest2313480-21kworker/u8:0+flush-259:009:10:162
2192989992725,1cyclictest2339332-21kworker/u8:3+flush-259:009:59:591
2192989992722,4cyclictest2235429-21apt-get07:50:111
2192985992721,4cyclictest2459898-21latency_hist11:25:010
2192996992623,2cyclictest2397984-21kworker/u8:0+events_unbound10:30:203
2192996992623,2cyclictest2349987-21kworker/u8:1+events_unbound09:43:223
2192996992623,2cyclictest2339332-21kworker/u8:3+flush-259:009:55:263
2192996992623,2cyclictest2234786-21kworker/u8:2+flush-259:008:00:203
2192996992622,3cyclictest2339332-21kworker/u8:3+events_unbound10:12:243
2192996992620,4cyclictest335557-21systemd-journal07:40:013
2192993992623,2cyclictest2224518-21kworker/u8:0+events_unbound08:10:192
2192993992622,3cyclictest2245980-21kworker/u8:3+flush-259:008:55:162
2192989992620,4cyclictest2241028-21apt-get07:55:111
2192985992623,2cyclictest2224518-21kworker/u8:0+flush-259:008:05:160
2192996992522,2cyclictest2453710-21kworker/u8:4+events_unbound12:05:173
2192996992522,2cyclictest2266645-21kworker/u8:0+events_unbound08:45:153
2192993992523,2cyclictest2453710-21kworker/u8:4+events_unbound12:05:162
2192993992522,2cyclictest2266645-21kworker/u8:0+events_unbound08:22:232
2192993992521,3cyclictest2453710-21kworker/u8:4+events_unbound11:53:212
2192989992520,4cyclictest2449811-21apt-get11:15:131
2192989992520,3cyclictest335557-21systemd-journal10:40:221
2192989992519,4cyclictest2538282-21gpgv12:40:001
2192989992519,4cyclictest2219192-21apt-get07:35:011
2192985992521,3cyclictest2429126-21apt-get10:55:130
2192985992519,4cyclictest2453710-21kworker/u8:4+events_unbound11:35:180
2192985992519,4cyclictest1-21systemd12:00:000
2192996992421,2cyclictest2517459-21kworker/u8:0+events_unbound12:33:233
2192996992421,2cyclictest2256233-21kworker/u8:1+flush-259:008:10:083
2192996992421,2cyclictest2208459-21kworker/u8:2+events_unbound07:31:213
2192996992420,3cyclictest2501613-21kworker/u8:1+events_unbound12:13:243
2192996992420,3cyclictest2453710-21kworker/u8:4+events_unbound12:17:243
2192996992419,3cyclictest2434324-21apt-get11:00:133
2192993992421,2cyclictest2501613-21kworker/u8:1+events_unbound12:35:442
2192993992421,2cyclictest2366747-21kworker/u8:0+events_unbound10:00:132
2192993992421,2cyclictest2234786-21kworker/u8:2+events_unbound07:53:212
2192985992420,3cyclictest2366747-21kworker/u8:0+events_unbound10:10:180
2192985992420,3cyclictest2323874-21kworker/u8:1+events_unbound09:22:250
2192985992419,4cyclictest2313480-21kworker/u8:0+flush-259:009:25:150
2192996992320,3cyclictest2349987-21kworker/u8:1+events_unbound10:53:243
2192996992320,2cyclictest2323874-21kworker/u8:1+flush-259:009:30:303
2192996992320,2cyclictest2266645-21kworker/u8:0+events_unbound08:40:223
2192996992320,2cyclictest2245980-21kworker/u8:3+events_unbound09:06:223
2192993992320,2cyclictest2444149-21kworker/u8:0+events_unbound11:40:142
2192993992320,2cyclictest2428876-21kworker/u8:3+events_unbound11:04:232
2192993992320,2cyclictest2298102-21kworker/u8:2+events_unbound08:50:242
2192993992319,3cyclictest2453710-21kworker/u8:4+events_unbound12:15:132
2192993992319,3cyclictest2313480-21kworker/u8:0+flush-259:009:35:162
2192989992319,3cyclictest2366747-21kworker/u8:0+flush-259:010:05:001
2192985992319,3cyclictest2517459-21kworker/u8:0+events_unbound12:30:240
2192985992319,3cyclictest2182761-21kworker/u8:0+flush-259:007:10:260
2192985992317,4cyclictest335557-21systemd-journal10:33:230
2192985992316,5cyclictest2265741-21ntpq08:15:220
2192996992219,2cyclictest2491193-21kworker/u8:0+events_unbound12:00:233
2192993992219,2cyclictest2397984-21kworker/u8:0+events_unbound10:40:212
2192989992216,4cyclictest2292897-21latency_hist08:45:021
2192989992216,4cyclictest2225063-21/usr/sbin/munin07:40:151
2192985992218,3cyclictest2313480-21kworker/u8:0+events_unbound09:35:110
2192985992218,3cyclictest2224518-21kworker/u8:0+events_unbound07:55:150
2192996992118,2cyclictest2517459-21kworker/u8:0+events_unbound12:22:233
2192996992118,2cyclictest2208459-21kworker/u8:2+events_unbound07:25:143
2192993992118,2cyclictest2349987-21kworker/u8:1+events_unbound11:20:232
2192993992118,2cyclictest2256233-21kworker/u8:1+events_unbound08:17:242
2192993992117,3cyclictest2266645-21kworker/u8:0+events_unbound08:40:132
2192993992115,4cyclictest335557-21systemd-journal09:04:222
2192989992117,3cyclictest2428876-21kworker/u8:3+flush-259:011:00:261
2192989992117,3cyclictest2277627-21apt-get08:30:131
2192989992116,4cyclictest2171586-21kworker/u8:3+flush-259:007:20:161
2192985992118,2cyclictest2349987-21kworker/u8:1+events_unbound09:50:230
2192985992117,3cyclictest2485842-21kworker/u8:3+events_unbound11:50:230
2192985992117,3cyclictest2475819-21kworker/u8:1+events_unbound11:40:220
2192985992117,3cyclictest2192962-21kworker/u8:1+events_unbound07:26:210
2192985992117,2cyclictest2371950-21apt-get10:00:110
2192985992114,5cyclictest2443509-21ntpq11:05:210
2192996992017,2cyclictest2349987-21kworker/u8:1+events_unbound10:27:233
2192996992017,2cyclictest2182761-21kworker/u8:0+events_unbound07:10:173
2192993992014,3cyclictest2323874-21kworker/u8:1+events_unbound09:20:162
2192989992017,2cyclictest2501683-21sh12:04:591
2192989992015,4cyclictest2234786-21kworker/u8:2+flush-259:008:05:011
2192985992017,2cyclictest2491193-21kworker/u8:0+flush-259:012:00:180
2192993991916,2cyclictest2444149-21kworker/u8:0+events_unbound11:15:262
2192993991916,2cyclictest2349987-21kworker/u8:1+events_unbound10:50:202
2192985991916,2cyclictest2349987-21kworker/u8:1+events_unbound10:50:130
2192985991915,3cyclictest2349987-21kworker/u8:1+events_unbound10:25:160
2192985991915,3cyclictest2344636-21kworker/u8:2+events_unbound09:40:240
2192985991914,3cyclictest2501613-21kworker/u8:1+flush-259:012:05:510
2192996991816,1cyclictest0-21swapper/310:40:003
2192996991815,2cyclictest2313480-21kworker/u8:0+events_unbound09:10:173
2192993991815,2cyclictest2453710-21kworker/u8:4+events_unbound11:45:222
2192989991812,4cyclictest2517459-21kworker/u8:0+flush-259:012:25:191
2192989991812,4cyclictest2349987-21kworker/u8:1+flush-259:011:10:001
2192989991810,7cyclictest2203319-21munin-run07:20:001
2192985991814,3cyclictest2428876-21kworker/u8:3+events_unbound11:00:150
2192985991814,3cyclictest2313480-21kworker/u8:0+flush-259:009:05:160
2192985991814,3cyclictest2245980-21kworker/u8:3+events_unbound08:55:150
2192996991714,2cyclictest2182761-21kworker/u8:0+events_unbound07:20:263
2192989991712,4cyclictest335557-21systemd-journal12:15:381
2192985991714,2cyclictest60350irq/132-enp0s29f1:rx-009:15:120
2192985991713,3cyclictest2517459-21kworker/u8:0+events_unbound12:25:170
2192985991713,3cyclictest2313480-21kworker/u8:0+events_unbound09:32:230
2192985991713,3cyclictest2234786-21kworker/u8:2+flush-259:008:00:150
2192985991712,3cyclictest2339332-21kworker/u8:3+events_unbound09:55:190
2192985991710,5cyclictest2423709-21latency_hist10:50:010
2192996991613,2cyclictest2366747-21kworker/u8:0+flush-259:010:00:263
2192993991613,2cyclictest2234786-21kworker/u8:2+events_unbound08:05:272
2192993991611,3cyclictest2349987-21kworker/u8:1+events_unbound11:05:132
219298999169,5cyclictest335557-21systemd-journal11:58:221
2192989991612,3cyclictest2208704-21apt-get07:25:101
2192989991611,4cyclictest2444149-21kworker/u8:0+flush-259:011:30:251
2192985991614,1cyclictest60350irq/132-enp0s29f1:rx-008:40:110
2192985991613,2cyclictest2245980-21kworker/u8:3+events_unbound09:00:150
2192985991613,2cyclictest2224518-21kworker/u8:0+flush-259:008:10:250
2192985991613,2cyclictest2224518-21kworker/u8:0+events_unbound07:40:230
2192985991612,2cyclictest0-21swapper/010:24:590
2192985991610,4cyclictest2418580-21latency_hist10:45:010
2192996991512,2cyclictest2387054-21kworker/u8:2+events_unbound10:24:243
2192996991512,2cyclictest2245980-21kworker/u8:3+flush-259:008:55:223
2192993991512,2cyclictest2453710-21kworker/u8:4+flush-259:011:35:252
2192993991512,2cyclictest2339332-21kworker/u8:3+flush-259:010:05:232
2192993991512,2cyclictest2182761-21kworker/u8:0+events_unbound07:10:272
2192989991512,2cyclictest2366448-21apt-key09:55:011
2192989991511,3cyclictest2261442-21sh08:15:011
2192989991510,4cyclictest2392336-21apt-get10:20:011
2192989991510,3cyclictest2349987-21kworker/u8:1+flush-259:010:15:181
2192985991513,1cyclictest60350irq/132-enp0s29f1:rx-011:25:230
2192985991512,2cyclictest60350irq/132-enp0s29f1:rx-012:10:240
2192985991512,2cyclictest60350irq/132-enp0s29f1:rx-010:37:480
2192985991512,2cyclictest60350irq/132-enp0s29f1:rx-009:45:140
2192985991512,2cyclictest2444149-21kworker/u8:0+events_unbound11:16:220
2192996991411,2cyclictest2397984-21kworker/u8:0+events_unbound10:40:293
2192993991411,2cyclictest2234786-21kworker/u8:2+events_unbound07:59:362
219298999149,3cyclictest0-21swapper/109:10:011
219298999141,12cyclictest0-21swapper/108:40:001
2192989991410,2cyclictest0-21swapper/109:19:591
219298599148,4cyclictest335557-21systemd-journal07:30:200
2192985991413,1cyclictest60350irq/132-enp0s29f1:rx-008:29:000
2192985991412,1cyclictest60350irq/132-enp0s29f1:rx-008:21:430
2192985991412,1cyclictest60350irq/132-enp0s29f1:rx-007:35:160
2192985991412,1cyclictest60350irq/132-enp0s29f1:rx-007:18:520
2192985991411,2cyclictest60350irq/132-enp0s29f1:rx-007:46:190
219299699133,2cyclictest50-21ksoftirqd/311:30:013
219299399136,1cyclictest41-21ksoftirqd/207:40:172
2192993991310,2cyclictest2517459-21kworker/u8:0+events_unbound12:20:132
2192993991310,2cyclictest2397984-21kworker/u8:0+flush-259:010:35:272
2192993991310,2cyclictest2392319-21cron10:20:002
2192993991310,2cyclictest2313480-21kworker/u8:0+events_unbound09:25:262
219298999139,3cyclictest2214563-21kworker/u8:3+flush-259:007:45:271
219298999139,3cyclictest2208459-21kworker/u8:2+flush-259:007:35:161
2192989991310,2cyclictest2501613-21kworker/u8:1+events_unbound12:10:201
2192989991310,2cyclictest2423646-21kworker/u8:2+events_unbound10:55:191
2192989991310,1cyclictest0-21swapper/110:29:591
219298599139,3cyclictest2245980-21kworker/u8:3+events_unbound08:31:230
2192985991310,2cyclictest60350irq/132-enp0s29f1:rx-012:22:430
219299699129,2cyclictest2313480-21kworker/u8:0+events_unbound09:15:023
219299699129,2cyclictest2266645-21kworker/u8:0+events_unbound08:50:043
219299399128,2cyclictest2532304-21cut12:30:222
219298999129,2cyclictest2486463-21cpuspeed_turbos11:50:131
219298999126,4cyclictest2345003-21latency_hist09:35:001
219298999125,5cyclictest2465045-21apt-get11:30:011
2192989991210,1cyclictest0-21swapper/108:10:001
219298599129,2cyclictest60350irq/132-enp0s29f1:rx-009:10:140
2192985991211,1cyclictest60350irq/132-enp0s29f1:rx-010:05:170
219299699114,1cyclictest0-21swapper/311:15:203
219298999119,1cyclictest0-21swapper/112:05:221
219298999118,2cyclictest2323424-21ntp_kernel_pll_09:10:231
219298999118,2cyclictest2266934-21cpuspeed_turbos08:20:121
219298999117,3cyclictest2418840-21apt-get10:45:131
219298999116,3cyclictest301rcuc/111:20:171
219298999115,4cyclictest2245980-21kworker/u8:3+flush-259:009:00:171
219299699108,2cyclictest2266645-21kworker/u8:0+events_unbound08:35:493
219298999104,4cyclictest2528188-21/usr/sbin/munin12:30:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional