You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-27 - 13:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4s.osadl.org (updated Mon Jul 27, 2026 00:44:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2166709995046,3cyclictest2177150-21gpgv19:20:000
2166716994640,4cyclictest2478871-21latency_hist00:10:011
2166723994438,4cyclictest2510278-21latency_hist00:40:003
2166719994237,4cyclictest509-21rs:main2
2166709994236,4cyclictest335557-21systemd-journal19:50:010
2166709994139,2cyclictest2245078-21kworker/u8:1+events_unbound21:35:310
2166709994138,2cyclictest2271268-21kworker/u8:0+events_unbound21:47:300
2166719994037,2cyclictest2338906-21kworker/u8:3+flush-259:022:50:002
2166709994037,2cyclictest2390635-21kworker/u8:2+events_unbound22:46:210
2166709994037,2cyclictest2245078-21kworker/u8:1+events_unbound21:20:300
2166716993933,4cyclictest2151555-21kworker/u8:4+flush-259:019:15:181
2166719993834,3cyclictest2338906-21kworker/u8:3+flush-259:022:45:002
2166716993834,3cyclictest2297562-21kworker/u8:4+flush-259:021:45:011
2166716993831,5cyclictest2292301-21latency_hist21:10:001
2166709993835,2cyclictest2380665-21kworker/u8:1+events_unbound00:33:300
2166709993835,2cyclictest2338906-21kworker/u8:3+events_unbound22:31:180
2166723993732,3cyclictest2182660-21latency_hist19:25:013
2166709993735,2cyclictest3320283-21kworker/u8:1+events_unbound19:29:290
2166709993734,3cyclictest2297562-21kworker/u8:4+events_unbound21:58:300
2166709993734,2cyclictest2380665-21kworker/u8:1+events_unbound23:49:310
2166709993734,2cyclictest2188493-21kworker/u8:2+events_unbound20:40:250
2166709993734,2cyclictest2188493-21kworker/u8:2+events_unbound19:55:150
2166709993734,2cyclictest2182538-21kworker/u8:3+events_unbound20:01:290
2166716993629,5cyclictest509-21rs:main1
2166709993633,2cyclictest2473732-21kworker/u8:0+events_unbound00:15:230
2166709993633,2cyclictest2396211-21kworker/u8:0+events_unbound23:00:230
2166709993633,2cyclictest2182538-21kworker/u8:3+events_unbound19:50:220
2166709993533,2cyclictest2370331-21kworker/u8:0+events_unbound22:26:100
2166709993532,2cyclictest2338906-21kworker/u8:3+events_unbound22:14:290
2166723993430,3cyclictest2245078-21kworker/u8:1+flush-259:021:20:183
2166719993430,2cyclictest2245078-21kworker/u8:1+flush-259:021:30:182
2166719993428,4cyclictest335557-21systemd-journal00:05:012
2166709993431,2cyclictest2473732-21kworker/u8:0+events_unbound00:08:300
2166709993431,2cyclictest2245078-21kworker/u8:1+events_unbound21:15:190
2166709993431,2cyclictest2219245-21kworker/u8:1+events_unbound20:16:300
2166719993329,3cyclictest2437697-21kworker/u8:4+events_unbound23:34:302
2166716993328,3cyclictest2188493-21kworker/u8:2+flush-259:020:45:011
2166716993325,5cyclictest2468526-21latency_hist00:00:011
2166709993327,4cyclictest2287563-21diskstats21:05:140
2166723993229,2cyclictest2188493-21kworker/u8:2+events_unbound20:40:013
2166709993229,2cyclictest2432354-21kworker/u8:3+events_unbound23:25:150
2166709993229,2cyclictest2271268-21kworker/u8:0+events_unbound21:00:140
2166709993229,2cyclictest2219245-21kworker/u8:1+events_unbound20:08:290
2166723993127,2cyclictest2271268-21kworker/u8:0+events_unbound21:00:233
2166719993128,2cyclictest2448052-21kworker/u8:3+events_unbound23:58:312
2166719993128,2cyclictest2271268-21kworker/u8:0+flush-259:021:55:242
2166709993128,2cyclictest2271268-21kworker/u8:0+events_unbound21:26:010
2166723993026,3cyclictest2239607-21kworker/u8:3+flush-259:021:10:013
2166719993026,3cyclictest2245078-21kworker/u8:1+events_unbound21:16:002
2166716993024,4cyclictest2254907-21ntpq20:30:211
2166709993024,4cyclictest2401344-21latency_hist22:55:000
216670999301,27cyclictest2297536-21true21:15:010
2166723992926,2cyclictest2437697-21kworker/u8:4+events_unbound23:30:203
2166723992925,3cyclictest2245078-21kworker/u8:1+events_unbound20:27:283
2166719992923,2cyclictest2229465-21kworker/u8:0+events_unbound20:10:012
2166716992922,5cyclictest2489463-21/usr/sbin/munin00:20:151
2166723992824,3cyclictest2380665-21kworker/u8:1+events_unbound00:25:023
2166723992824,3cyclictest2338906-21kworker/u8:3+events_unbound22:08:303
2166723992824,3cyclictest2239607-21kworker/u8:3+flush-259:020:45:173
2166719992825,2cyclictest2245078-21kworker/u8:1+flush-259:021:05:202
2166719992825,2cyclictest2245078-21kworker/u8:1+events_unbound21:24:292
2166719992825,2cyclictest2182538-21kworker/u8:3+events_unbound19:31:272
2166709992826,2cyclictest2188493-21kworker/u8:2+events_unbound20:20:270
2166709992825,3cyclictest2473732-21kworker/u8:0+events_unbound00:10:550
2166709992825,2cyclictest2380665-21kworker/u8:1+events_unbound00:25:180
2166716992721,4cyclictest335557-21systemd-journal20:00:001
2166716992720,5cyclictest2245078-21kworker/u8:1+flush-259:020:25:161
2166723992622,3cyclictest2271268-21kworker/u8:0+flush-259:021:59:303
2166719992623,2cyclictest2432354-21kworker/u8:3+events_unbound23:25:182
2166719992623,2cyclictest2338906-21kworker/u8:3+events_unbound22:16:102
2166716992621,3cyclictest2182538-21kworker/u8:3+flush-259:020:05:001
2166716992620,4cyclictest2375512-21latency_hist22:30:001
2166709992624,2cyclictest3320283-21kworker/u8:1+events_unbound19:20:150
2166709992624,2cyclictest2473732-21kworker/u8:0+events_unbound00:20:210
2166709992624,2cyclictest2411696-21kworker/u8:2+events_unbound23:05:170
2166709992620,4cyclictest335557-21systemd-journal22:18:290
2166723992522,2cyclictest2219245-21kworker/u8:1+flush-259:020:15:003
2166723992521,3cyclictest2245078-21kworker/u8:1+events_unbound20:55:153
2166723992519,4cyclictest2239607-21kworker/u8:3+events_unbound20:20:183
2166723992518,5cyclictest2313069-21latency_hist21:30:003
2166719992522,2cyclictest2271268-21kworker/u8:0+flush-259:022:05:112
2166719992522,2cyclictest2239607-21kworker/u8:3+flush-259:020:35:252
2166719992522,2cyclictest2188493-21kworker/u8:2+flush-259:019:55:212
2166719992518,3cyclictest2182538-21kworker/u8:3+events_unbound19:40:192
2166709992522,2cyclictest2380665-21kworker/u8:1+events_unbound00:38:300
2166709992522,2cyclictest2271268-21kworker/u8:0+events_unbound22:01:090
2166709992521,3cyclictest2437697-21kworker/u8:4+events_unbound23:36:310
2166723992421,2cyclictest2338906-21kworker/u8:3+flush-259:022:00:303
2166723992421,2cyclictest2219245-21kworker/u8:1+flush-259:020:15:303
2166723992421,2cyclictest2182538-21kworker/u8:3+events_unbound20:06:293
2166723992420,3cyclictest2448052-21kworker/u8:3+events_unbound23:51:303
2166723992420,3cyclictest2338906-21kworker/u8:3+events_unbound23:01:243
2166723992420,3cyclictest2166916-21kworker/u8:0+events_unbound19:19:313
2166719992421,3cyclictest2494160-21kworker/u8:2+events_unbound00:34:302
2166719992421,2cyclictest3320283-21kworker/u8:1+events_unbound19:20:282
2166719992421,2cyclictest2271268-21kworker/u8:0+events_unbound22:00:022
2166719992421,2cyclictest2245078-21kworker/u8:1+events_unbound20:25:212
2166719992421,2cyclictest2239607-21kworker/u8:3+events_unbound20:45:212
2166719992420,3cyclictest2411696-21kworker/u8:2+events_unbound23:18:302
2166719992420,3cyclictest2380665-21kworker/u8:1+events_unbound23:40:142
2166719992420,3cyclictest2380665-21kworker/u8:1+events_unbound23:20:192
2166719992419,3cyclictest2380665-21kworker/u8:1+events_unbound00:10:002
2166716992418,4cyclictest509-21rs:main1
2166709992421,2cyclictest2271268-21kworker/u8:0+events_unbound22:05:190
2166709992420,3cyclictest2297562-21kworker/u8:4+flush-259:021:50:160
2166723992319,3cyclictest2297562-21kworker/u8:4+events_unbound21:50:133
2166723992317,4cyclictest2203740-21latency_hist19:45:023
2166719992320,2cyclictest2473732-21kworker/u8:0+flush-259:000:15:172
2166719992320,2cyclictest2411696-21kworker/u8:2+flush-259:023:10:172
2166719992319,3cyclictest2338906-21kworker/u8:3+flush-259:022:25:212
2166719992319,3cyclictest2323632-21kworker/u8:2+flush-259:021:40:252
2166719992319,3cyclictest2239607-21kworker/u8:3+events_unbound20:30:122
2166719992319,2cyclictest2370331-21kworker/u8:0+events_unbound22:35:202
2166719992317,3cyclictest2229465-21kworker/u8:0+events_unbound20:10:222
2166716992319,3cyclictest2396211-21kworker/u8:0+flush-259:022:55:001
2166723992218,3cyclictest2380665-21kworker/u8:1+events_unbound00:24:293
2166723992216,4cyclictest335557-21systemd-journal19:28:283
2166719992219,2cyclictest2182538-21kworker/u8:3+events_unbound19:50:442
2166719992218,3cyclictest2473732-21kworker/u8:0+events_unbound00:20:152
2166719992218,3cyclictest2338906-21kworker/u8:3+flush-259:022:34:592
2166719992215,5cyclictest2198588-21latency_hist19:40:012
2166716992217,4cyclictest2271268-21kworker/u8:0+flush-259:021:05:011
2166716992215,5cyclictest335557-21systemd-journal00:11:291
2166709992219,2cyclictest2448052-21kworker/u8:3+events_unbound23:52:320
2166709992217,4cyclictest2386107-21apt-get22:40:130
2166723992118,2cyclictest2380665-21kworker/u8:1+events_unbound23:19:313
2166723992117,3cyclictest2473732-21kworker/u8:0+events_unbound00:16:303
2166719992118,2cyclictest2354633-21kworker/u8:2+flush-259:022:20:202
2166719992116,3cyclictest2271268-21kworker/u8:0+events_unbound21:10:192
2166719992114,3cyclictest3320283-21kworker/u8:1+events_unbound19:15:342
2166709992118,2cyclictest2411696-21kworker/u8:2+events_unbound23:12:310
2166709992118,2cyclictest2380665-21kworker/u8:1+events_unbound23:23:290
2166723992017,2cyclictest2380665-21kworker/u8:1+events_unbound00:03:313
2166723992016,3cyclictest2380665-21kworker/u8:1+flush-259:023:29:593
2166723992016,1cyclictest0-21swapper/319:15:003
2166719992017,2cyclictest2166916-21kworker/u8:0+flush-259:019:10:162
2166719992016,3cyclictest2448052-21kworker/u8:3+events_unbound23:54:302
216671699205,8cyclictest2406533-21awk22:59:591
2166716992015,3cyclictest2448052-21kworker/u8:3+flush-259:023:45:001
2166716992014,4cyclictest2468804-21apt-get00:00:131
216670999205,6cyclictest2442801-21cat23:35:010
2166709992018,2cyclictest2229465-21kworker/u8:0+events_unbound20:10:460
2166709992017,2cyclictest2239607-21kworker/u8:3+events_unbound20:45:180
2166709992016,3cyclictest2188493-21kworker/u8:2+events_unbound20:33:300
2166709992014,4cyclictest2327699-21ntpq21:40:220
2166723991917,1cyclictest2338906-21kworker/u8:3+events_unbound22:10:143
2166723991916,2cyclictest2370331-21kworker/u8:0+events_unbound22:40:203
2166723991916,2cyclictest0-21swapper/323:40:003
2166719991916,2cyclictest2473732-21kworker/u8:0+events_unbound00:10:252
2166719991916,2cyclictest2219245-21kworker/u8:1+flush-259:020:15:192
2166719991916,2cyclictest2188493-21kworker/u8:2+events_unbound20:40:162
2166716991914,3cyclictest335557-21systemd-journal20:40:001
2166716991911,5cyclictest2369423-21ntpq22:20:221
2166709991916,2cyclictest2245078-21kworker/u8:1+events_unbound20:25:200
2166709991912,5cyclictest2384955-21ntpq22:35:240
216670999190,17cyclictest509-21rs:main0
2166723991815,2cyclictest2188493-21kworker/u8:2+events_unbound19:35:253
2166723991814,3cyclictest2239607-21kworker/u8:3+events_unbound20:50:163
2166719991815,2cyclictest2271268-21kworker/u8:0+events_unbound21:25:312
2166719991815,2cyclictest2245078-21kworker/u8:1+flush-259:021:35:262
2166719991814,3cyclictest2271268-21kworker/u8:0+flush-259:021:00:272
2166719991814,3cyclictest2245078-21kworker/u8:1+flush-259:020:55:172
2166719991813,2cyclictest2297562-21kworker/u8:4+events_unbound21:45:262
2166716991815,2cyclictest2233825-21ntp_kernel_pll_20:10:231
2166716991813,3cyclictest2333727-21kworker/u8:1+flush-259:021:55:011
2166716991812,4cyclictest2380665-21kworker/u8:1+flush-259:023:50:001
2166709991815,2cyclictest2448052-21kworker/u8:3+events_unbound23:42:520
2166709991811,5cyclictest2467636-21ntpq23:55:210
2166723991714,2cyclictest2188493-21kworker/u8:2+flush-259:020:40:253
2166723991713,3cyclictest2380665-21kworker/u8:1+events_unbound23:20:173
2166723991713,3cyclictest2338906-21kworker/u8:3+events_unbound22:29:313
2166723991713,3cyclictest2188493-21kworker/u8:2+flush-259:019:55:243
2166723991712,4cyclictest2500217-21/usr/sbin/munin00:30:153
2166723991711,4cyclictest2477981-21ntpq00:05:223
2166719991714,2cyclictest2338906-21kworker/u8:3+events_unbound23:07:312
2166716991714,2cyclictest2437477-21idleruntime-cro23:30:001
2166716991713,3cyclictest2271268-21kworker/u8:0+events_unbound21:25:201
2166716991713,3cyclictest2182538-21kworker/u8:3+flush-259:019:43:331
2166716991712,4cyclictest2297562-21kworker/u8:4+flush-259:021:45:271
2166716991711,4cyclictest2245078-21kworker/u8:1+flush-259:021:20:201
2166709991714,2cyclictest2239607-21kworker/u8:3+flush-259:020:50:180
2166709991714,2cyclictest2166916-21kworker/u8:0+events_unbound19:10:480
2166709991713,3cyclictest2297562-21kworker/u8:4+flush-259:022:25:000
2166723991613,2cyclictest2323632-21kworker/u8:2+events_unbound21:41:303
2166723991613,2cyclictest2239607-21kworker/u8:3+flush-259:020:30:153
2166723991613,2cyclictest2214034-21idleruntime-cro19:55:003
2166723991612,3cyclictest2390635-21kworker/u8:2+events_unbound22:50:113
2166723991612,3cyclictest2188493-21kworker/u8:2+events_unbound19:30:153
2166719991612,3cyclictest2396211-21kworker/u8:0+events_unbound22:55:202
2166719991612,3cyclictest2239607-21kworker/u8:3+events_unbound20:24:302
2166719991611,2cyclictest2239607-21kworker/u8:3+events_unbound20:50:212
216671699169,5cyclictest2228666-21ntpq20:05:241
216671699161,8cyclictest2188269-21idleruntime-cro19:30:001
2166716991612,3cyclictest2411696-21kworker/u8:2+flush-259:023:15:251
2166716991610,4cyclictest335557-21systemd-journal23:01:221
2166709991614,2cyclictest2188493-21kworker/u8:2+events_unbound19:40:260
2166709991613,2cyclictest2239607-21kworker/u8:3+events_unbound20:35:280
2166723991511,3cyclictest2396211-21kworker/u8:0+events_unbound22:55:233
2166723991511,3cyclictest2297562-21kworker/u8:4+events_unbound21:45:243
2166719991512,2cyclictest2396211-21kworker/u8:0+events_unbound23:00:212
2166719991512,2cyclictest2380665-21kworker/u8:1+events_unbound23:45:212
2166719991512,2cyclictest2338906-21kworker/u8:3+events_unbound22:11:302
2166716991511,3cyclictest2411696-21kworker/u8:2+flush-259:023:05:221
2166716991511,3cyclictest2271268-21kworker/u8:0+flush-259:022:05:001
2166716991510,4cyclictest2188493-21kworker/u8:2+flush-259:019:45:231
2166716991510,3cyclictest335557-21systemd-journal20:48:291
216670999155,2cyclictest16-21ksoftirqd/020:55:190
2166709991512,2cyclictest60350irq/132-enp0s29f1:rx-000:01:030
2166709991512,2cyclictest2411696-21kworker/u8:2+events_unbound23:16:250
2166723991411,2cyclictest2245078-21kworker/u8:1+events_unbound21:36:293
2166719991411,2cyclictest2380665-21kworker/u8:1+flush-259:000:35:162
2166719991411,2cyclictest2188493-21kworker/u8:2+flush-259:019:45:262
2166719991411,2cyclictest2166916-21kworker/u8:0+flush-259:019:25:222
216671699149,4cyclictest2313334-21apt-get21:30:101
216671699148,4cyclictest2495068-21apt-get00:25:151
216671699147,5cyclictest2193674-21/usr/sbin/munin19:35:171
2166716991410,3cyclictest2448052-21kworker/u8:3+flush-259:023:55:001
2166716991410,2cyclictest2417538-21irqcore23:10:151
2166709991412,1cyclictest60350irq/132-enp0s29f1:rx-021:30:130
2166709991412,1cyclictest60350irq/132-enp0s29f1:rx-019:30:060
216672399139,3cyclictest2354633-21kworker/u8:2+events_unbound22:20:183
216672399139,3cyclictest2182538-21kworker/u8:3+events_unbound20:00:433
216672399138,3cyclictest2380665-21kworker/u8:1+events_unbound23:45:193
2166723991310,2cyclictest2416844-21apt23:10:003
2166723991310,2cyclictest2380665-21kworker/u8:1+events_unbound23:40:213
2166723991310,2cyclictest2338906-21kworker/u8:3+events_unbound22:15:193
2166719991310,2cyclictest2494160-21kworker/u8:2+events_unbound00:26:202
216671699139,2cyclictest0-21swapper/123:20:171
216671699139,2cyclictest0-21swapper/119:12:291
216671699138,4cyclictest2271268-21kworker/u8:0+flush-259:021:10:271
216671699137,4cyclictest509-21rs:main1
216671699137,4cyclictest2391233-21/usr/sbin/munin22:45:241
216671699137,4cyclictest2354667-21apt-get22:10:121
216672399129,2cyclictest2370331-21kworker/u8:0+events_unbound22:38:293
216672399129,2cyclictest2271268-21kworker/u8:0+flush-259:021:19:353
216672399129,2cyclictest2245078-21kworker/u8:1+events_unbound21:12:293
216672399128,3cyclictest2411696-21kworker/u8:2+events_unbound23:13:303
216671999128,3cyclictest2338906-21kworker/u8:3+events_unbound22:50:032
216671699128,3cyclictest2437697-21kworker/u8:4+flush-259:023:35:301
216671699128,3cyclictest2188493-21kworker/u8:2+flush-259:020:17:331
216671699127,3cyclictest2349465-21apt-get22:05:111
216671699126,4cyclictest490-21dbus-daemon21:20:011
216672399118,2cyclictest2390635-21kworker/u8:2+flush-259:022:45:153
216672399118,2cyclictest2245078-21kworker/u8:1+flush-259:021:30:193
216672399117,3cyclictest509-21rs:main3
216671699118,1cyclictest0-21swapper/122:44:591
216671699112,7cyclictest509-21rs:main1
21667239996,2cyclictest2380665-21kworker/u8:1+events_unbound23:55:133
21667239996,2cyclictest0-21swapper/322:34:593
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional