You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-19 - 12:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Sat Sep 19, 2026 00:44:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2239495995148,2cyclictest2421480-21kworker/u8:3+events_unbound23:15:233
2239492994943,4cyclictest2390101-21kworker/u8:1+flush-259:021:35:262
2239481994540,4cyclictest2504449-21apt-get23:25:120
2239481994338,3cyclictest2260695-21apt-get19:30:100
2239488994135,4cyclictest2255229-21latency_hist19:25:011
2239481993835,2cyclictest2363801-21kworker/u8:0+flush-259:022:59:590
2239495993730,5cyclictest2394745-21ntpq21:35:233
2239488993733,3cyclictest2343129-21kworker/u8:3+flush-259:020:55:251
2239495993532,2cyclictest2462907-21kworker/u8:2+flush-259:023:15:003
2239495993532,2cyclictest2462907-21kworker/u8:2+flush-259:023:14:593
2239495993531,3cyclictest2515230-21apt-get23:35:133
2239488993531,3cyclictest2363801-21kworker/u8:0+events_unbound22:50:171
2239492993429,4cyclictest2411365-21/usr/sbin/munin21:55:132
2239488993429,3cyclictest2520386-21/usr/sbin/munin23:40:151
2239481993430,3cyclictest2234101-21kworker/u8:3+flush-259:019:35:220
2239495993329,3cyclictest509-21rs:main3
2239481993328,3cyclictest2530741-21apt-get23:50:120
2239495993229,2cyclictest2363801-21kworker/u8:0+flush-259:021:50:163
2239495993226,5cyclictest2291856-21diskstats20:00:133
2239492993127,3cyclictest335557-21systemd-journal19:35:502
2239488993126,3cyclictest2541210-21apt-get00:00:101
2239481993126,4cyclictest335557-21systemd-journal21:15:030
2239492992923,4cyclictest2483591-21latency_hist23:05:002
2239488992925,3cyclictest2421480-21kworker/u8:3+flush-259:023:05:241
2239481992925,3cyclictest2363801-21kworker/u8:0+flush-259:022:20:160
2239492992823,4cyclictest2493921-21kworker/u8:0+flush-259:023:30:002
2239492992822,4cyclictest335557-21systemd-journal22:44:242
2239488992825,2cyclictest2520377-21kworker/u8:3+events_unbound23:55:141
2239488992825,2cyclictest2447370-21kworker/u8:1+flush-259:022:49:591
2239488992824,3cyclictest2473500-21apt-get22:55:111
2239488992824,3cyclictest2462907-21kworker/u8:2+flush-259:023:20:171
2239488992824,3cyclictest2343129-21kworker/u8:3+events_unbound21:30:131
2239488992824,3cyclictest2239960-21apt-get19:10:111
2239488992823,3cyclictest2567106-21apt-get00:25:121
2239488992822,4cyclictest2551352-21latency_hist00:10:011
2239481992822,4cyclictest335557-21systemd-journal21:02:240
2239495992724,2cyclictest2421480-21kworker/u8:3+events_unbound22:25:173
2239495992724,2cyclictest2186232-21kworker/u8:1+flush-259:019:20:103
2239495992723,3cyclictest2561925-21/usr/sbin/munin00:20:133
2239495992723,3cyclictest2186232-21kworker/u8:1+events_unbound19:30:213
2239495992721,5cyclictest335557-21systemd-journal20:30:013
2239492992722,4cyclictest335557-21systemd-journal23:45:012
2239492992722,4cyclictest2432079-21apt-get22:15:132
2239488992722,3cyclictest2322776-21apt-get20:30:111
2239481992724,2cyclictest2421480-21kworker/u8:3+flush-259:022:15:170
2239481992722,3cyclictest2493921-21kworker/u8:0+flush-259:023:30:150
2239481992721,4cyclictest2291435-21latency_hist20:00:010
2239488992621,4cyclictest335557-21systemd-journal21:36:141
2239488992621,4cyclictest2401083-21apt-get21:45:121
2239488992620,5cyclictest2411390-21apt-get21:55:111
2239488992620,4cyclictest2337973-21latency_hist20:45:001
2239481992622,3cyclictest2504416-21kworker/u8:1+flush-259:023:35:180
2239481992621,4cyclictest2411168-21kworker/u8:2+flush-259:022:10:180
2239481992621,3cyclictest2296840-21kworker/u8:3+flush-259:020:05:170
223948199261,23cyclictest509-21rs:main0
2239495992522,3cyclictest2363801-21kworker/u8:0+events_unbound21:20:133
2239495992522,2cyclictest2421480-21kworker/u8:3+events_unbound22:40:003
2239495992522,2cyclictest2363801-21kworker/u8:0+flush-259:022:55:163
2239495992521,3cyclictest2411168-21kworker/u8:2+events_unbound21:55:223
2239495992520,3cyclictest2483829-21/usr/sbin/munin23:05:213
2239495992519,4cyclictest2520377-21kworker/u8:3+flush-259:023:45:013
2239495992519,4cyclictest2510025-21/usr/sbin/munin23:30:203
2239495992518,5cyclictest2286247-21latency_hist19:55:013
2239492992521,3cyclictest2343129-21kworker/u8:3+flush-259:021:30:242
2239481992519,4cyclictest2577213-21latency_hist00:35:010
2239495992420,3cyclictest2363801-21kworker/u8:0+events_unbound21:40:113
2239495992420,3cyclictest2270761-21kworker/u8:2+events_unbound20:20:153
2239492992421,2cyclictest2363801-21kworker/u8:0+flush-259:022:55:212
2239488992420,3cyclictest2421480-21kworker/u8:3+events_unbound23:10:241
2239488992420,3cyclictest2421480-21kworker/u8:3+events_unbound23:10:241
2239495992319,3cyclictest2363801-21kworker/u8:0+events_unbound22:45:253
2239495992318,3cyclictest2421480-21kworker/u8:3+flush-259:023:00:263
2239495992317,4cyclictest335557-21systemd-journal22:05:233
2239492992319,3cyclictest2245114-21apt-get19:15:112
2239488992319,3cyclictest2421480-21kworker/u8:3+flush-259:023:00:201
2239488992318,4cyclictest2270761-21kworker/u8:2+events_unbound20:10:151
2239488992318,4cyclictest2270761-21kworker/u8:2+events_unbound19:45:171
2239488992318,3cyclictest2363801-21kworker/u8:0+flush-259:021:15:241
2239488992317,5cyclictest2509732-21cp23:30:011
2239495992218,3cyclictest2520377-21kworker/u8:3+events_unbound00:00:213
2239495992218,3cyclictest2363801-21kworker/u8:0+events_unbound21:45:213
2239495992217,4cyclictest2363801-21kworker/u8:0+events_unbound22:20:013
2239495992217,4cyclictest2363801-21kworker/u8:0+events_unbound21:14:253
2239495992216,4cyclictest2390101-21kworker/u8:1+flush-259:022:00:173
2239492992218,3cyclictest2520377-21kworker/u8:3+flush-259:023:55:172
2239492992218,3cyclictest2286480-21kworker/u8:0+flush-259:020:00:012
2239488992218,3cyclictest2421480-21kworker/u8:3+flush-259:022:10:001
2239488992218,3cyclictest2186232-21kworker/u8:1+flush-259:019:30:231
2239495992116,4cyclictest2343129-21kworker/u8:3+events_unbound21:05:283
2239492992117,3cyclictest2270761-21kworker/u8:2+events_unbound20:10:202
2239488992117,3cyclictest3838517-21kworker/u8:0+events_unbound19:40:121
2239488992117,3cyclictest2530498-21kworker/u8:0+events_unbound23:50:241
2239488992117,3cyclictest2363801-21kworker/u8:0+flush-259:021:25:141
2239488992115,4cyclictest2332793-21latency_hist20:40:011
2239481992118,2cyclictest2473225-21cron22:55:000
2239481992117,3cyclictest2463146-21apt-get22:45:090
2239481992116,4cyclictest509-21rs:main0
2239481992116,4cyclictest335557-21systemd-journal21:30:000
2239481992114,5cyclictest2462025-21ntpq22:40:230
2239495992016,3cyclictest2551585-21kworker/u8:0+flush-259:000:38:343
2239495992016,3cyclictest2327676-21kworker/u8:0+flush-259:020:40:003
2239495992016,3cyclictest2296840-21kworker/u8:3+flush-259:020:10:243
2239492992014,4cyclictest335557-21systemd-journal20:26:192
2239488992015,3cyclictest509-21rs:main1
2239481992017,2cyclictest2270761-21kworker/u8:2+events_unbound20:50:140
2239495991916,2cyclictest2551585-21kworker/u8:0+events_unbound00:28:223
2239495991916,2cyclictest2541176-21kworker/u8:5+events_unbound00:10:133
2239495991916,2cyclictest2247432-21kthreadcore19:15:183
2239495991915,3cyclictest2390101-21kworker/u8:1+events_unbound21:31:153
2239495991915,3cyclictest2234101-21kworker/u8:3+events_unbound19:40:103
2239495991912,5cyclictest2301786-21latency_hist20:10:003
2239495991910,6cyclictest2311247-21ntpq20:15:203
2239492991915,3cyclictest335557-21systemd-journal22:30:232
2239492991915,3cyclictest2369170-21store21:15:002
2239488991916,2cyclictest2421480-21kworker/u8:3+events_unbound22:15:241
2239488991915,3cyclictest2551585-21kworker/u8:0+events_unbound00:32:271
2239488991915,3cyclictest2270761-21kworker/u8:2+events_unbound20:15:171
2239481991916,2cyclictest2971-21cron19:30:000
2239481991916,2cyclictest2234101-21kworker/u8:3+flush-259:019:40:210
2239481991915,3cyclictest2551585-21kworker/u8:0+flush-259:000:35:260
2239481991915,3cyclictest2411168-21kworker/u8:2+flush-259:021:55:240
2239481991914,4cyclictest2239933-21/usr/sbin/munin19:10:140
2239495991816,2cyclictest2270761-21kworker/u8:2+flush-259:021:00:183
2239492991815,2cyclictest2520136-21idleruntime-cro23:39:592
2239492991815,2cyclictest0-21swapper/219:10:012
2239492991812,4cyclictest2348265-21latency_hist20:55:012
2239488991814,3cyclictest2504416-21kworker/u8:1+flush-259:023:45:231
2239488991814,3cyclictest2421480-21kworker/u8:3+events_unbound22:35:241
2239488991814,3cyclictest2281091-21kworker/u8:1+events_unbound19:50:131
2239488991812,4cyclictest2577453-21/usr/sbin/munin00:35:171
2239488991812,4cyclictest2317588-21apt-get20:25:101
2239481991815,2cyclictest335557-21systemd-journal00:13:170
2239481991814,3cyclictest2452812-21apt-get22:35:120
2239495991714,2cyclictest2421480-21kworker/u8:3+events_unbound23:25:253
2239495991714,2cyclictest2270761-21kworker/u8:2+events_unbound21:18:233
2239495991714,2cyclictest2206902-21kworker/u8:2+flush-259:019:10:193
2239495991713,3cyclictest2535864-21kworker/u8:2+flush-259:023:55:213
2239495991713,3cyclictest2520377-21kworker/u8:3+flush-259:000:08:233
2239495991710,5cyclictest2337041-21ntpq20:40:203
2239492991714,2cyclictest509-21rs:main2
2239492991714,2cyclictest2447370-21kworker/u8:1+flush-259:022:39:592
2239492991713,3cyclictest507-21in:imuxsock21:00:002
2239492991713,3cyclictest2510017-21kworker/u8:4+flush-259:023:30:162
2239492991712,3cyclictest2430944-21ntpq22:10:232
2239492991712,3cyclictest2385289-21latency_hist21:30:002
2239488991714,2cyclictest2462907-21kworker/u8:2+flush-259:023:15:151
2239488991714,2cyclictest2421480-21kworker/u8:3+events_unbound22:25:141
2239488991714,2cyclictest2363801-21kworker/u8:0+events_unbound21:40:211
2239488991714,2cyclictest2296840-21kworker/u8:3+events_unbound20:05:121
2239488991713,3cyclictest3838517-21kworker/u8:0+flush-259:019:25:261
2239488991713,3cyclictest2363801-21kworker/u8:0+flush-259:022:20:231
2239481991713,3cyclictest2270761-21kworker/u8:2+flush-259:020:20:240
2239481991711,4cyclictest335557-21systemd-journal21:34:240
2239495991613,2cyclictest2439586-21kthreadcore22:20:163
2239495991612,3cyclictest2541176-21kworker/u8:5+events_unbound00:31:253
2239495991612,3cyclictest2343129-21kworker/u8:3+events_unbound21:25:243
2239495991612,3cyclictest2270761-21kworker/u8:2+events_unbound20:49:233
2239492991614,1cyclictest0-21swapper/200:30:012
2239492991612,3cyclictest3838517-21kworker/u8:0+flush-259:019:50:002
2239488991613,2cyclictest2363801-21kworker/u8:0+events_unbound22:30:191
2239488991613,2cyclictest2281091-21kworker/u8:1+flush-259:020:00:161
2239488991612,3cyclictest2270761-21kworker/u8:2+flush-259:020:45:251
2239481991614,1cyclictest60350irq/132-enp0s29f1:rx-023:05:190
2239481991613,2cyclictest61150irq/133-enp0s29f1:tx-020:40:240
2239481991613,2cyclictest60350irq/132-enp0s29f1:rx-023:40:210
2239481991613,2cyclictest60350irq/132-enp0s29f1:rx-020:25:100
2239481991613,2cyclictest2364118-21cpuspeed_turbos21:10:120
2239481991612,3cyclictest2363801-21kworker/u8:0+flush-259:022:25:360
2239495991511,3cyclictest2551585-21kworker/u8:0+flush-259:000:15:293
2239495991511,3cyclictest2421480-21kworker/u8:3+events_unbound22:31:273
2239495991510,3cyclictest2286480-21kworker/u8:0+events_unbound19:55:113
2239492991513,1cyclictest2421483-21systemd-run22:05:012
2239492991512,2cyclictest2501613-21kthreadcore23:20:182
2239492991511,3cyclictest2363801-21kworker/u8:0+flush-259:021:54:282
2239492991511,3cyclictest2307193-21kworker/u8:1+flush-259:020:21:032
2239488991512,2cyclictest2520377-21kworker/u8:3+flush-259:000:20:161
2239488991512,2cyclictest2380018-21apt-key21:25:001
2239481991513,2cyclictest60350irq/132-enp0s29f1:rx-020:58:500
2239481991513,1cyclictest60350irq/132-enp0s29f1:rx-023:20:180
2239481991513,1cyclictest60350irq/132-enp0s29f1:rx-021:20:150
2239481991513,1cyclictest60350irq/132-enp0s29f1:rx-021:05:230
2239481991513,1cyclictest60350irq/132-enp0s29f1:rx-000:15:150
2239481991513,1cyclictest60350irq/132-enp0s29f1:rx-000:05:160
2239481991512,2cyclictest60350irq/132-enp0s29f1:rx-022:00:120
2239481991512,2cyclictest60350irq/132-enp0s29f1:rx-019:21:340
2239481991511,3cyclictest2363801-21kworker/u8:0+flush-259:021:45:260
2239481991511,3cyclictest2363801-21kworker/u8:0+flush-259:021:40:170
2239495991411,2cyclictest2343129-21kworker/u8:3+flush-259:020:55:163
223949299141,7cyclictest2582353-21awk00:40:012
223948899149,4cyclictest2343351-21apt-get20:50:121
2239488991410,3cyclictest2270761-21kworker/u8:2+events_unbound21:10:151
2239488991410,3cyclictest2270761-21kworker/u8:2+events_unbound21:06:071
2239488991410,3cyclictest2239511-21store19:10:011
2239481991412,1cyclictest60350irq/132-enp0s29f1:rx-023:10:350
2239481991412,1cyclictest60350irq/132-enp0s29f1:rx-023:10:340
2239481991412,1cyclictest60350irq/132-enp0s29f1:rx-023:00:130
2239481991412,1cyclictest60350irq/132-enp0s29f1:rx-020:45:160
2239481991412,1cyclictest60350irq/132-enp0s29f1:rx-020:10:160
2239481991412,1cyclictest60350irq/132-enp0s29f1:rx-020:00:230
2239481991411,2cyclictest60350irq/132-enp0s29f1:rx-021:50:140
2239481991411,2cyclictest60350irq/132-enp0s29f1:rx-020:20:000
2239481991411,2cyclictest60350irq/132-enp0s29f1:rx-000:20:110
2239481991410,3cyclictest2422133-21irqcore22:05:170
2239481991410,3cyclictest2327676-21kworker/u8:0+flush-259:020:35:240
223949599139,3cyclictest3838517-21kworker/u8:0+events_unbound19:45:113
223949599139,3cyclictest3838517-21kworker/u8:0+events_unbound19:36:213
223949599139,3cyclictest2504416-21kworker/u8:1+events_unbound23:45:243
223949599139,3cyclictest2270761-21kworker/u8:2+flush-259:020:31:303
223949599138,3cyclictest2421480-21kworker/u8:3+events_unbound22:42:263
2239495991310,2cyclictest0-21swapper/319:25:193
223949299139,3cyclictest2462907-21kworker/u8:2+flush-259:023:10:002
2239492991310,2cyclictest2530049-21unixbench_singl23:45:242
2239492991310,2cyclictest0-21swapper/220:09:592
223948899138,3cyclictest2416565-21/usr/sbin/munin22:00:221
2239488991311,1cyclictest2460279-21kthreadcore22:40:201
2239488991311,1cyclictest2317309-21cron20:25:001
2239488991310,2cyclictest2270761-21kworker/u8:2+flush-259:021:00:151
2239481991312,1cyclictest60350irq/132-enp0s29f1:rx-023:47:100
2239481991312,1cyclictest60350irq/132-enp0s29f1:rx-019:16:250
2239481991311,2cyclictest60350irq/132-enp0s29f1:rx-020:31:570
2239481991311,1cyclictest60350irq/132-enp0s29f1:rx-000:00:230
2239481991310,2cyclictest60350irq/132-enp0s29f1:rx-000:25:130
2239481991310,2cyclictest2276625-21irqcore19:45:160
223949599129,2cyclictest2530498-21kworker/u8:0+events_unbound23:54:233
223949599129,2cyclictest2239475-21cyclictest22:10:193
223949299128,3cyclictest2462907-21kworker/u8:2+flush-259:023:15:352
223948899129,2cyclictest2519669-21unixbench_multi23:35:261
223948899129,2cyclictest2406480-21fschecks_time21:50:131
223948899129,1cyclictest0-21swapper/119:15:121
2239481991210,1cyclictest60350irq/132-enp0s29f1:rx-021:35:240
223949599118,2cyclictest2270761-21kworker/u8:2+flush-259:020:50:143
223949299119,1cyclictest0-21swapper/219:55:002
223949299118,2cyclictest2374879-21cpuspeed_turbos21:20:132
223949299118,2cyclictest2363801-21kworker/u8:0+flush-259:021:40:242
223949299118,2cyclictest2353772-21cpuspeed_turbos21:00:112
223949299118,2cyclictest2327676-21kworker/u8:0+flush-259:020:35:172
223949299118,2cyclictest2255695-21fschecks_count19:25:152
223949299117,3cyclictest2541176-21kworker/u8:5+flush-259:000:10:112
223949299117,2cyclictest2373649-21ntpq21:15:202
223949299117,2cyclictest0-21swapper/222:24:592
223949299116,4cyclictest2560827-21ntpq00:15:232
223949299116,4cyclictest2343072-21cron20:50:002
223948899118,2cyclictest2541176-21kworker/u8:5+flush-259:000:10:161
223948899118,2cyclictest2290951-21unixbench_multi19:55:241
223948899117,3cyclictest2510017-21kworker/u8:4+events_unbound00:15:191
223949599101,7cyclictest2499757-21sed23:20:183
223949299107,2cyclictest2546166-21idleruntime-cro00:05:012
223949299107,2cyclictest2463405-21fschecks_time22:45:122
223949299107,2cyclictest2261110-21irqcore19:30:152
223949299106,3cyclictest2186232-21kworker/u8:1+flush-259:019:20:152
223949299105,3cyclictest2337900-21sshd20:44:232
223949299105,3cyclictest0-21swapper/222:05:252
223948899106,3cyclictest2411168-21kworker/u8:2+flush-259:022:10:221
22394929997,1cyclictest0-21swapper/220:00:132
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional