You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-25 - 13:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Sat Apr 25, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2325788994943,4cyclictest2558877-21cp22:55:001
2325788994540,3cyclictest2346324-21kworker/u8:1+flush-259:019:45:171
2325785994239,2cyclictest2564273-21kworker/u8:3+events_unbound00:10:230
2325795994036,3cyclictest2610509-21kworker/u8:0+flush-259:000:15:203
2325785994037,2cyclictest2475418-21kworker/u8:2+flush-259:021:45:000
2325785994037,2cyclictest2387505-21kworker/u8:1+flush-259:021:10:230
2325785993932,5cyclictest2414071-21diskstats20:35:130
2325791993731,4cyclictest2372423-21diskstats19:55:132
2325785993730,5cyclictest2365963-21ntpq19:45:220
2325785993633,2cyclictest2393071-21kworker/u8:3+flush-259:020:30:210
2325785993633,2cyclictest2346324-21kworker/u8:1+flush-259:019:50:200
2325785993629,5cyclictest2340252-21ntpq19:20:210
2325791993529,4cyclictest252-21systemd-journal19:25:242
2325785993532,2cyclictest2393071-21kworker/u8:3+flush-259:021:00:000
2325785993532,2cyclictest2387505-21kworker/u8:1+events_unbound20:15:150
2325785993528,5cyclictest2569810-21diskstats23:05:130
2325791993431,2cyclictest2393071-21kworker/u8:3+events_unbound20:45:162
2325788993430,3cyclictest2393071-21kworker/u8:3+flush-259:020:50:001
2325788993428,4cyclictest252-21systemd-journal19:52:241
2325785993431,2cyclictest2573652-21kworker/u8:1+flush-259:023:30:210
2325785993430,3cyclictest2475418-21kworker/u8:2+events_unbound21:55:010
2325788993327,4cyclictest2625907-21latency_hist00:00:011
2325785993330,2cyclictest2387505-21kworker/u8:1+flush-259:021:04:590
2325785993326,5cyclictest2511063-21ntpq22:05:200
2325788993226,4cyclictest2512209-21/usr/sbin/munin22:10:131
2325785993228,3cyclictest2304429-21kworker/u8:3+flush-259:019:10:230
2325791993126,4cyclictest509-21rs:main2
2325788993127,3cyclictest2475418-21kworker/u8:2+flush-259:022:45:191
2325788993126,3cyclictest2620770-21kworker/u8:2+events_unbound00:35:011
2325788993124,5cyclictest2407629-21ntpq20:25:221
2325785993127,3cyclictest2573652-21kworker/u8:1+events_unbound23:45:230
2325785993127,3cyclictest2507070-21kworker/u8:0+flush-259:022:10:230
2325785993127,3cyclictest2304429-21kworker/u8:3+events_unbound19:25:130
2325785993127,3cyclictest2282574-21kworker/u8:2+flush-259:019:55:160
232578599310,29cyclictest1-21systemd21:30:000
2325795993024,4cyclictest2449579-21latency_hist21:10:013
2325791993027,2cyclictest2387505-21kworker/u8:1+events_unbound20:55:182
2325791993026,3cyclictest2573652-21kworker/u8:1+flush-259:023:35:172
2325791993026,3cyclictest2573652-21kworker/u8:1+flush-259:023:35:162
2325785993026,3cyclictest2610509-21kworker/u8:0+events_unbound00:00:200
2325785993026,3cyclictest2610509-21kworker/u8:0+events_unbound00:00:010
2325788992923,4cyclictest2475658-21apt-get21:35:121
2325788992922,5cyclictest2542385-21ntpq22:35:211
2325788992921,5cyclictest2636189-21latency_hist00:10:001
2325785992925,3cyclictest2393071-21kworker/u8:3+events_unbound20:50:120
2325785992923,4cyclictest2480417-21apt-get21:38:200
2325791992824,3cyclictest2282574-21kworker/u8:2+events_unbound20:00:142
2325785992824,3cyclictest2610509-21kworker/u8:0+events_unbound00:15:150
2325785992824,3cyclictest2573652-21kworker/u8:1+events_unbound23:26:240
2325785992824,3cyclictest2475418-21kworker/u8:2+flush-259:022:45:240
2325785992824,3cyclictest2393071-21kworker/u8:3+events_unbound20:41:280
2325785992824,3cyclictest2387505-21kworker/u8:1+events_unbound21:30:230
2325795992721,4cyclictest2366870-21latency_hist19:50:003
2325791992724,2cyclictest2538156-21kworker/u8:0+flush-259:022:35:242
2325791992723,3cyclictest2393071-21kworker/u8:3+events_unbound20:20:262
2325791992722,4cyclictest252-21systemd-journal21:09:592
2325791992722,4cyclictest252-21systemd-journal19:45:242
2325785992723,3cyclictest2543505-21kworker/u8:3+events_unbound22:40:180
2325785992723,3cyclictest2393071-21kworker/u8:3+events_unbound20:45:230
2325785992723,3cyclictest2387505-21kworker/u8:1+flush-259:020:10:200
2325785992723,2cyclictest2475418-21kworker/u8:2+flush-259:022:55:210
2325795992621,3cyclictest2387505-21kworker/u8:1+flush-259:021:35:163
2325795992620,4cyclictest2485807-21latency_hist21:45:013
2325788992622,3cyclictest2564273-21kworker/u8:3+flush-259:023:50:211
2325788992622,3cyclictest2449476-21kworker/u8:0+flush-259:021:50:261
2325788992622,3cyclictest2366902-21kworker/u8:3+flush-259:020:00:261
2325788992620,4cyclictest2449821-21/usr/sbin/munin21:10:131
2325788992619,5cyclictest2398445-21/usr/sbin/munin20:20:121
2325785992622,3cyclictest2564273-21kworker/u8:3+flush-259:023:10:200
2325785992622,3cyclictest2564273-21kworker/u8:3+events_unbound23:15:250
2325785992622,3cyclictest2507070-21kworker/u8:0+flush-259:022:15:160
2325785992622,3cyclictest2304429-21kworker/u8:3+events_unbound19:15:230
2325785992622,3cyclictest2282574-21kworker/u8:2+flush-259:020:00:210
2325791992522,2cyclictest2387505-21kworker/u8:1+events_unbound21:30:192
2325788992521,3cyclictest2387505-21kworker/u8:1+flush-259:022:25:221
2325785992521,3cyclictest2475418-21kworker/u8:2+flush-259:023:00:240
2325785992521,3cyclictest2304429-21kworker/u8:3+events_unbound19:31:190
2325791992420,3cyclictest2393071-21kworker/u8:3+flush-259:021:25:252
2325791992420,3cyclictest2387505-21kworker/u8:1+flush-259:020:39:592
2325791992420,3cyclictest2282574-21kworker/u8:2+events_unbound20:05:212
2325788992419,3cyclictest509-21rs:main1
2325788992418,4cyclictest2506429-21http22:05:011
2325785992419,4cyclictest2564273-21kworker/u8:3+events_unbound00:29:260
2325785992418,5cyclictest2496347-21/usr/sbin/munin21:55:130
2325795992319,3cyclictest2590009-21kworker/u8:2+flush-259:023:40:003
2325795992319,3cyclictest2590009-21kworker/u8:2+flush-259:023:40:003
2325795992319,3cyclictest2393071-21kworker/u8:3+flush-259:021:15:003
2325795992318,3cyclictest2351721-21apt-get19:35:113
2325788992320,1cyclictest2564273-21kworker/u8:3+flush-259:023:25:011
2325788992319,3cyclictest2475418-21kworker/u8:2+flush-259:021:45:181
2325788992319,3cyclictest2393071-21kworker/u8:3+flush-259:020:54:591
2325788992318,4cyclictest2559348-21/usr/sbin/munin22:55:131
2325788992318,4cyclictest2465332-21/usr/sbin/munin21:25:141
2325788992318,3cyclictest2387505-21kworker/u8:1+flush-259:021:30:161
2325785992319,3cyclictest2573652-21kworker/u8:1+events_unbound23:36:590
2325785992319,3cyclictest2573652-21kworker/u8:1+events_unbound23:36:590
2325785992319,3cyclictest2573652-21kworker/u8:1+events_unbound23:20:210
2325785992319,3cyclictest2475418-21kworker/u8:2+flush-259:022:50:240
2325785992319,3cyclictest2393071-21kworker/u8:3+events_unbound21:16:370
2325785992318,4cyclictest2522449-21kworker/u8:3+events_unbound22:31:520
2325795992216,4cyclictest252-21systemd-journal20:41:283
2325795992216,4cyclictest2382751-21latency_hist20:05:013
2325791992219,2cyclictest2282574-21kworker/u8:2+flush-259:019:35:212
2325791992218,3cyclictest2538156-21kworker/u8:0+flush-259:023:05:152
2325791992217,4cyclictest2620770-21kworker/u8:2+events_unbound00:00:122
2325791992217,3cyclictest2387505-21kworker/u8:1+events_unbound22:05:242
2325785992219,2cyclictest2282574-21kworker/u8:2+events_unbound19:36:160
2325791992117,3cyclictest2564273-21kworker/u8:3+events_unbound23:50:152
2325791992117,3cyclictest2496340-21kworker/u8:3+flush-259:022:00:252
2325791992117,3cyclictest2449476-21kworker/u8:0+flush-259:021:50:212
2325791992117,3cyclictest2387505-21kworker/u8:1+flush-259:021:55:212
2325791992116,4cyclictest2393071-21kworker/u8:3+flush-259:021:10:182
2325788992116,3cyclictest252-21systemd-journal20:20:011
2325785992118,2cyclictest2282574-21kworker/u8:2+flush-259:019:40:210
2325785992117,3cyclictest2393071-21kworker/u8:3+events_unbound20:20:140
2325795992016,3cyclictest2393071-21kworker/u8:3+flush-259:021:25:173
2325791992017,2cyclictest2387505-21kworker/u8:1+flush-259:021:50:002
2325791992016,3cyclictest2564273-21kworker/u8:3+flush-259:000:05:242
2325791992016,3cyclictest2304429-21kworker/u8:3+events_unbound19:10:252
2325791992015,3cyclictest252-21systemd-journal19:16:142
2325788992017,1cyclictest2387505-21kworker/u8:1+flush-259:021:05:001
2325788992014,4cyclictest252-21systemd-journal22:23:241
2325788992014,4cyclictest2517746-21base6422:15:011
2325785992016,3cyclictest2590009-21kworker/u8:2+events_unbound23:40:240
2325785992016,3cyclictest2393071-21kworker/u8:3+events_unbound21:25:010
2325795991913,4cyclictest2475418-21kworker/u8:2+flush-259:022:55:153
232579599191,16cyclictest1-21systemd22:00:003
2325791991916,2cyclictest2558719-21gpgv22:55:002
2325791991916,2cyclictest2543505-21kworker/u8:3+events_unbound22:40:212
2325791991915,3cyclictest2610509-21kworker/u8:0+flush-259:000:15:252
2325791991915,3cyclictest2538156-21kworker/u8:0+events_unbound23:02:252
2325791991915,3cyclictest2522449-21kworker/u8:3+flush-259:022:30:182
2325791991914,3cyclictest2475418-21kworker/u8:2+flush-259:022:10:162
2325788991915,3cyclictest2507070-21kworker/u8:0+flush-259:022:05:361
2325788991914,3cyclictest252-21systemd-journal20:45:001
2325788991913,4cyclictest2594235-21ntpq23:25:211
2325788991913,4cyclictest252-21systemd-journal21:41:421
2325788991913,4cyclictest2346324-21kworker/u8:1+flush-259:019:40:161
2325788991913,4cyclictest2325982-21/usr/sbin/munin19:10:181
2325788991912,5cyclictest2393071-21kworker/u8:3+flush-259:020:30:171
2325785991916,2cyclictest2538156-21kworker/u8:0+events_unbound22:35:240
2325785991915,3cyclictest2475418-21kworker/u8:2+events_unbound22:04:250
232579599189,8cyclictest2429022-21basename20:50:003
232579599188,9cyclictest2605618-21/usr/sbin/munin23:40:153
2325795991812,4cyclictest252-21systemd-journal22:25:243
232579599180,3cyclictest2387938-21cat20:10:013
2325791991814,3cyclictest2620770-21kworker/u8:2+flush-259:000:35:152
2325791991814,3cyclictest2564273-21kworker/u8:3+events_unbound00:10:162
2325791991814,3cyclictest2393071-21kworker/u8:3+flush-259:021:15:262
2325791991814,3cyclictest2282574-21kworker/u8:2+events_unbound19:21:162
2325788991813,4cyclictest252-21systemd-journal23:46:521
2325788991812,5cyclictest252-21systemd-journal20:06:221
2325785991814,3cyclictest2393071-21kworker/u8:3+events_unbound21:05:570
2325785991814,3cyclictest2387505-21kworker/u8:1+events_unbound21:48:260
2325795991713,2cyclictest2331146-21apt-get19:15:113
232579599171,2cyclictest2599342-21ntpq23:30:193
232579599171,2cyclictest2345401-21ntpq19:25:213
2325795991712,3cyclictest252-21systemd-journal00:09:233
232579599170,3cyclictest2533214-21/usr/sbin/munin22:30:153
232579599170,16cyclictest2439295-21awk21:00:003
232579599170,15cyclictest2491109-21cat21:50:013
2325791991714,2cyclictest2393071-21kworker/u8:3+events_unbound21:35:212
2325791991714,2cyclictest2387505-21kworker/u8:1+events_unbound21:00:192
2325791991713,3cyclictest2564273-21kworker/u8:3+events_unbound00:31:552
2325791991713,3cyclictest2346324-21kworker/u8:1+flush-259:019:50:162
2325791991712,3cyclictest252-21systemd-journal22:49:592
232579599165,9cyclictest0-21swapper/300:05:013
232579599161,3cyclictest2666366-21sed00:35:233
232579599161,3cyclictest2412780-21ntpq20:30:223
232579599161,3cyclictest2377413-21sed20:00:003
232579599161,3cyclictest2371997-21awk19:55:013
232579599161,2cyclictest2522687-21cron22:20:003
232579599161,2cyclictest2506419-21munin-run22:05:003
232579599161,1cyclictest2619847-21ntpq23:50:203
232579599161,1cyclictest2511046-21ntpq22:05:203
232579599161,13cyclictest252-21systemd-journal19:34:593
232579599160,4cyclictest2579680-21idleruntime-cro23:15:013
232579599160,3cyclictest2640409-21ntpq00:10:203
232579599160,2cyclictest2542172-21latency22:35:193
232579599160,14cyclictest2615728-21cat23:50:003
232579599160,14cyclictest2558989-21apt-config22:55:003
2325791991612,3cyclictest2573652-21kworker/u8:1+flush-259:023:15:152
2325791991612,3cyclictest2475418-21kworker/u8:2+events_unbound22:20:232
2325791991612,3cyclictest2393071-21kworker/u8:3+events_unbound20:41:262
2325791991612,3cyclictest2387505-21kworker/u8:1+flush-259:020:15:002
2325791991612,3cyclictest2387505-21kworker/u8:1+events_unbound21:43:252
2325791991611,4cyclictest2564273-21kworker/u8:3+events_unbound23:11:252
2325791991611,4cyclictest252-21systemd-journal20:33:242
232578899169,5cyclictest252-21systemd-journal23:40:001
232578899169,5cyclictest252-21systemd-journal23:40:001
232578899168,5cyclictest2547537-21ntpq22:40:211
2325788991612,3cyclictest2304429-21kworker/u8:3+flush-259:019:15:251
2325788991610,4cyclictest252-21systemd-journal20:39:591
2325788991610,4cyclictest2460171-21apt-get21:20:091
2325785991613,2cyclictest2475418-21kworker/u8:2+events_unbound22:20:150
232579599151,5cyclictest2398565-21tr20:20:123
232579599151,4cyclictest2652180-21/usr/sbin/munin00:25:153
232579599151,4cyclictest2327027-21kthreadcore19:10:173
232579599151,3cyclictest2621424-21cut23:55:173
232579599151,3cyclictest2527831-21basename22:25:003
232579599151,3cyclictest252-21systemd-journal19:45:013
2325795991512,2cyclictest2338575-21kthreadcore19:20:183
2325795991510,4cyclictest2387505-21kworker/u8:1+flush-259:021:01:293
232579599150,5cyclictest2569892-21tune2fs23:05:133
232579599150,4cyclictest2660893-21/usr/sbin/munin00:30:193
232579599150,4cyclictest2473845-21grep21:30:193
232579599150,3cyclictest2646951-21tune2fs00:20:143
232579599150,3cyclictest2585502-21grep23:20:173
232579599150,3cyclictest2414322-21irqcore20:35:163
232579599150,2cyclictest2569374-21cron23:05:003
232579599150,2cyclictest2543543-21taskset22:40:133
232579599150,2cyclictest2429532-21tune2fs20:50:133
232579599150,2cyclictest2397320-21ntpq20:15:223
232579199159,4cyclictest2610499-21latency_hist23:45:002
2325791991513,1cyclictest0-21swapper/223:50:012
2325791991511,3cyclictest2475418-21kworker/u8:2+flush-259:022:17:302
2325791991511,3cyclictest2393071-21kworker/u8:3+events_unbound20:25:142
2325791991511,3cyclictest2387505-21kworker/u8:1+events_unbound22:26:522
232578899159,4cyclictest252-21systemd-journal00:17:241
2325788991511,2cyclictest0-21swapper/120:00:011
2325785991513,1cyclictest60350irq/132-enp0s29f1:rx-000:32:430
2325785991513,1cyclictest60350irq/132-enp0s29f1:rx-000:24:310
2325785991512,2cyclictest60350irq/132-enp0s29f1:rx-020:05:180
2325785991512,2cyclictest60350irq/132-enp0s29f1:rx-000:06:100
232579599144,8cyclictest2514954-21tr22:10:173
232579599142,3cyclictest2491189-21apt-get21:50:123
232579599141,2cyclictest2455140-21tr21:15:133
2325795991411,1cyclictest0-21swapper/320:10:153
232579599141,10cyclictest2590233-21runrttasks23:25:123
232579599140,4cyclictest2460583-21cut21:20:133
232579599140,3cyclictest2404019-21ps20:25:163
232579599140,2cyclictest2584001-21ntpq23:15:223
232579599140,2cyclictest2549107-21ps22:45:193
232579199149,4cyclictest2282574-21kworker/u8:2+flush-259:019:41:482
2325791991410,3cyclictest2564273-21kworker/u8:3+events_unbound23:56:242
2325791991410,3cyclictest2564273-21kworker/u8:3+events_unbound23:25:182
232578899148,4cyclictest2387505-21kworker/u8:1+flush-259:021:10:001
2325788991411,2cyclictest2392184-21ntp_kernel_pll_20:10:221
2325788991410,3cyclictest2564273-21kworker/u8:3+flush-259:023:10:221
2325785991410,3cyclictest2620770-21kworker/u8:2+events_unbound00:37:030
2325785991410,3cyclictest2564273-21kworker/u8:3+events_unbound23:50:270
2325785991410,3cyclictest2387505-21kworker/u8:1+events_unbound20:26:370
232579199139,3cyclictest2387505-21kworker/u8:1+flush-259:021:22:012
2325791991310,3cyclictest2475418-21kworker/u8:2+flush-259:022:55:102
232578899139,3cyclictest2439289-21cron21:00:001
2325791991210,1cyclictest0-21swapper/219:35:012
232578899129,2cyclictest2341491-21cpuspeed_turbos19:25:121
232579199117,3cyclictest2564273-21kworker/u8:3+flush-259:023:22:312
232579199117,3cyclictest2372664-21kworker/u8:0+events_unbound20:15:282
232578899118,1cyclictest0-21swapper/123:15:251
232578899110,2cyclictest2641342-21awk00:15:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional