You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-10 - 20:38

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4s.osadl.org (updated Wed Jun 10, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2209738994035,4cyclictest335557-21systemd-journal10:55:000
2209741993933,4cyclictest2526596-21latency_hist12:15:011
2209746993834,3cyclictest2432098-21kworker/u8:1+flush-259:011:25:212
2209746993834,3cyclictest2365011-21kworker/u8:2+flush-259:010:25:002
220973899381,35cyclictest1-21systemd10:10:000
2209741993630,4cyclictest335557-21systemd-journal10:52:341
2209738993528,5cyclictest2327535-21ntpq09:00:190
2209738993527,5cyclictest2256435-21latency_hist07:55:010
220973899350,33cyclictest1-21systemd10:05:000
220973899350,33cyclictest1-21systemd10:05:000
2209746993431,2cyclictest2277013-21kworker/u8:0+flush-259:009:20:192
2209748993328,3cyclictest2230712-21latency_hist07:30:003
2209746993329,3cyclictest2365011-21kworker/u8:2+flush-259:010:30:212
2209746993329,3cyclictest2282362-21kworker/u8:2+events_unbound08:25:172
2209738993326,5cyclictest2483580-21ntpq11:30:210
2209746993228,3cyclictest2365011-21kworker/u8:2+flush-259:009:45:202
2209746993127,3cyclictest2354441-21kworker/u8:1+flush-259:009:30:162
2209746993127,3cyclictest2225575-21kworker/u8:1+flush-259:007:30:212
2209738993122,6cyclictest335557-21systemd-journal11:54:590
2209748993026,3cyclictest2375288-21kworker/u8:4+flush-259:010:10:003
2209746993027,2cyclictest2365011-21kworker/u8:2+events_unbound10:10:142
2209746992926,2cyclictest2495499-21kworker/u8:0+flush-259:011:45:182
2209746992926,2cyclictest2432098-21kworker/u8:1+flush-259:011:30:182
2209746992925,3cyclictest2390810-21kworker/u8:0+flush-259:010:40:242
2209738992925,2cyclictest2287581-21apt-get08:25:110
2209748992822,5cyclictest2390773-21latency_hist10:05:003
2209748992822,5cyclictest2390773-21latency_hist10:05:003
2209746992825,2cyclictest2432098-21kworker/u8:1+flush-259:011:15:002
2209746992823,3cyclictest2432098-21kworker/u8:1+flush-259:012:00:002
2209746992821,4cyclictest335557-21systemd-journal12:37:202
2209741992824,3cyclictest2406472-21/usr/sbin/munin10:20:131
2209741992824,3cyclictest2246418-21/usr/sbin/munin07:45:151
2209741992824,3cyclictest2246418-21/usr/sbin/munin07:45:141
2209738992821,5cyclictest2447985-21diskstats11:00:140
2209738992821,5cyclictest2265778-21ntpq08:00:210
2209748992721,4cyclictest2220534-21cat07:20:013
2209746992723,3cyclictest2516329-21kworker/u8:3+events_unbound12:25:182
2209746992723,3cyclictest2432098-21kworker/u8:1+events_unbound12:20:142
2209746992723,3cyclictest2432098-21kworker/u8:1+events_unbound12:20:132
2209746992723,3cyclictest2292482-21kworker/u8:3+flush-259:009:35:222
2209746992722,4cyclictest2272113-21apt-get08:10:102
2209746992722,3cyclictest2338502-21kworker/u8:2+events_unbound09:25:162
2209738992721,4cyclictest335557-21systemd-journal12:09:330
2209748992620,4cyclictest2292457-21latency_hist08:30:013
2209746992623,2cyclictest2241039-21kworker/u8:3+flush-259:008:00:152
2209746992619,2cyclictest2505058-21kworker/u8:2+events_unbound12:00:212
2209738992620,4cyclictest2256857-21diskstats07:55:120
2209746992522,2cyclictest2277013-21kworker/u8:0+flush-259:008:20:212
2209746992522,2cyclictest2241039-21kworker/u8:3+events_unbound07:46:362
2209746992522,2cyclictest2241039-21kworker/u8:3+events_unbound07:46:352
2209746992421,2cyclictest2484534-21kworker/u8:3+flush-259:011:35:172
2209746992421,2cyclictest2292482-21kworker/u8:3+flush-259:008:50:112
2209746992420,3cyclictest2266712-21kworker/u8:1+events_unbound08:09:342
2209746992417,5cyclictest2447552-21awk11:00:002
2209738992418,4cyclictest2537202-21/usr/sbin/munin12:25:120
2209738992418,4cyclictest2301868-21ntpq08:35:210
2209746992320,2cyclictest2277013-21kworker/u8:0+flush-259:009:05:162
2209738992320,2cyclictest2292482-21kworker/u8:3+flush-259:008:40:100
220974699227,7cyclictest2370160-21basename09:45:002
2209746992219,2cyclictest2292482-21kworker/u8:3+flush-259:008:55:212
2209746992218,3cyclictest2542796-21kworker/u8:2+flush-259:012:30:182
2209746992218,3cyclictest2292482-21kworker/u8:3+events_unbound09:10:252
2209746992218,3cyclictest2277013-21kworker/u8:0+flush-259:009:15:122
2209741992217,4cyclictest335557-21systemd-journal10:10:001
2209741992216,4cyclictest335557-21systemd-journal08:25:011
2209748992119,1cyclictest0-21swapper/312:25:003
2209748992119,1cyclictest0-21swapper/312:25:003
2209748992117,3cyclictest2479614-21apt-get11:30:113
2209746992118,2cyclictest2516329-21kworker/u8:3+events_unbound12:15:172
2209746992117,3cyclictest2292482-21kworker/u8:3+flush-259:009:00:212
2209741992116,4cyclictest2547319-21latency_hist12:35:011
2209741992114,6cyclictest0-21swapper/108:20:001
2209741992110,9cyclictest2390755-21cron10:05:001
2209741992110,9cyclictest2390755-21cron10:05:001
2209738992117,3cyclictest2339477-21apt-get09:15:100
2209738992116,4cyclictest2396163-21kworker/u8:3+flush-259:010:20:150
2209738992114,5cyclictest2353834-21ntpq09:25:200
2209748992015,4cyclictest335557-21systemd-journal10:40:003
2209748992015,4cyclictest2500399-21apt-get11:49:593
2209746992017,2cyclictest2396162-21kworker/u8:1+events_unbound10:15:242
2209746992017,2cyclictest2375288-21kworker/u8:4+events_unbound09:56:352
2209746992016,3cyclictest2432098-21kworker/u8:1+events_unbound11:20:232
2209746992016,3cyclictest2365011-21kworker/u8:2+flush-259:009:50:152
2209741992017,2cyclictest2266693-21sh08:05:001
2209738992016,3cyclictest2354441-21kworker/u8:1+flush-259:009:34:450
2209738992014,4cyclictest2516329-21kworker/u8:3+flush-259:012:30:160
2209748991916,2cyclictest2468294-21gpgv11:20:013
2209748991916,2cyclictest2220012-21idleruntime-cro07:20:003
2209748991915,3cyclictest2971-21cron10:55:003
2209748991913,4cyclictest2505990-21latency_hist11:55:003
2209746991916,2cyclictest2516329-21kworker/u8:3+flush-259:012:05:152
2209746991916,2cyclictest2421797-21kworker/u8:3+flush-259:010:50:172
2209746991916,2cyclictest2396162-21kworker/u8:1+events_unbound10:29:532
2209746991916,2cyclictest2365011-21kworker/u8:2+events_unbound10:01:362
2209746991916,2cyclictest2365011-21kworker/u8:2+events_unbound10:01:362
2209746991915,3cyclictest2516329-21kworker/u8:3+events_unbound12:13:342
2209746991915,3cyclictest155503-21kworker/u8:0+flush-259:007:40:102
2209746991914,3cyclictest2375288-21kworker/u8:4+events_unbound10:05:192
2209746991913,4cyclictest2468295-21apt-key11:20:012
2209741991916,2cyclictest2531813-21turbostat.cron12:20:001
2209741991915,3cyclictest335557-21systemd-journal08:55:341
2209741991915,3cyclictest2478924-21latency_hist11:30:001
2209741991914,4cyclictest2536120-21ntpq12:20:191
2209741991914,4cyclictest2536120-21ntpq12:20:191
2209741991914,4cyclictest2337919-21ntpq09:10:201
2209741991912,5cyclictest2499523-21ntpq11:45:221
2209741991912,5cyclictest2271853-21latency_hist08:10:011
2209738991915,3cyclictest2225575-21kworker/u8:1+events_unbound07:35:160
2209738991912,4cyclictest2344806-21diskstats09:20:130
2209748991814,3cyclictest2442419-21kworker/u8:2+flush-259:011:05:153
2209748991814,3cyclictest2277013-21kworker/u8:0+flush-259:009:20:163
2209746991816,1cyclictest2442419-21kworker/u8:2+events_unbound11:01:342
2209746991815,2cyclictest2225575-21kworker/u8:1+flush-259:007:30:002
2209746991815,2cyclictest155503-21kworker/u8:0+events_unbound07:20:132
2209746991814,3cyclictest2277013-21kworker/u8:0+flush-259:008:45:122
220974199188,8cyclictest0-21swapper/108:30:231
2209741991814,3cyclictest2354441-21kworker/u8:1+flush-259:009:35:001
2209741991814,3cyclictest155503-21kworker/u8:0+flush-259:007:20:001
2209741991812,5cyclictest2426008-21ntpq10:35:201
2209738991815,2cyclictest2500398-21sh11:49:590
2209738991815,2cyclictest2287316-21sh08:24:590
2209738991815,1cyclictest0-21swapper/011:20:010
2209738991813,3cyclictest2396194-21apt-get10:10:090
2209738991811,4cyclictest335557-21systemd-journal11:59:350
2209748991715,1cyclictest0-21swapper/308:19:593
2209748991714,2cyclictest2323587-21apt-get09:00:113
2209748991712,4cyclictest2479029-21cp11:30:003
2209746991714,2cyclictest2421797-21kworker/u8:3+events_unbound10:47:352
2209746991714,2cyclictest155503-21kworker/u8:0+flush-259:007:15:152
2209746991711,4cyclictest2501063-21/usr/sbin/munin11:50:202
220974199171,5cyclictest2241002-21cron07:39:591
220974199170,4cyclictest335557-21systemd-journal09:45:001
220974199170,4cyclictest2452692-21awk11:05:001
220974199170,2cyclictest2375328-21cat09:50:001
2209738991714,2cyclictest60350irq/132-enp0s29f1:rx-010:30:200
2209738991713,2cyclictest60350irq/132-enp0s29f1:rx-011:20:120
2209738991712,3cyclictest2396162-21kworker/u8:1+flush-259:010:15:220
2209738991710,5cyclictest2427358-21diskstats10:40:130
220974899163,3cyclictest2296707-21ntpq08:30:183
2209748991614,1cyclictest0-21swapper/310:00:003
2209748991613,2cyclictest2354703-21cron09:30:003
2209748991613,2cyclictest2298307-21grep08:35:183
2209748991613,1cyclictest0-21swapper/310:30:003
2209746991610,4cyclictest335557-21systemd-journal08:36:342
220974199165,9cyclictest0-21swapper/111:35:141
220974199164,10cyclictest2313564-21perl08:50:111
220974199162,3cyclictest2401353-21apt-get10:15:091
220974199161,4cyclictest513-21gdbus12:30:001
220974199161,4cyclictest2350234-21ps09:25:151
220974199161,4cyclictest2328462-21dump-pmu-power09:05:011
220974199161,3cyclictest513-21gdbus10:35:001
220974199161,3cyclictest1-21systemd11:24:341
220974199161,2cyclictest2515402-21ntpq12:00:201
220974199161,2cyclictest2493952-21ntpq11:40:221
2209741991612,3cyclictest2505058-21kworker/u8:2+flush-259:011:50:241
2209741991612,3cyclictest2432098-21kworker/u8:1+flush-259:011:30:011
2209741991612,3cyclictest2390810-21kworker/u8:0+flush-259:010:50:001
2209741991612,3cyclictest2225815-21apt-get07:25:111
220974199161,1cyclictest2260730-21ntpq07:55:201
220974199160,4cyclictest2313054-21turbostat08:50:011
220974199160,4cyclictest2303044-21apt-get08:40:101
220974199160,3cyclictest2443068-21ps10:55:151
2209738991613,2cyclictest60350irq/132-enp0s29f1:rx-011:25:130
2209738991613,2cyclictest60350irq/132-enp0s29f1:rx-011:14:510
2209738991613,2cyclictest2241039-21kworker/u8:3+events_unbound07:44:330
2209738991612,3cyclictest61150irq/133-enp0s29f1:tx-009:45:150
2209738991612,2cyclictest155503-21kworker/u8:0+events_unbound07:15:010
2209748991512,2cyclictest0-21swapper/312:15:183
2209748991512,2cyclictest0-21swapper/308:00:003
2209748991511,3cyclictest2209946-21apt-get07:10:093
220974899150,4cyclictest2364117-21ld09:35:193
220974899150,4cyclictest2234972-21ld07:30:203
2209746991513,2cyclictest2432098-21kworker/u8:1+events_unbound11:05:182
2209746991512,2cyclictest2277013-21kworker/u8:0+events_unbound08:15:192
2209746991511,3cyclictest2277013-21kworker/u8:0+events_unbound08:32:352
220974199154,1cyclictest0-21swapper/107:30:171
220974199152,2cyclictest2379406-21missed_timers09:50:181
2209741991513,1cyclictest0-21swapper/111:55:221
220974199151,2cyclictest2463000-21cron11:15:001
220974199151,2cyclictest2456933-21ntpq11:05:201
220974199151,2cyclictest2364054-21ntpq09:35:181
220974199151,2cyclictest2213952-21ntpq07:10:201
220974199150,7cyclictest2301961-21ntpq08:35:241
220974199150,6cyclictest2400250-21ntpq10:10:191
220974199150,5cyclictest2224691-21ntpq07:20:201
220974199150,4cyclictest335557-21systemd-journal12:40:001
220974199150,4cyclictest2463457-21grep11:15:141
220974199150,4cyclictest2431629-21grep10:40:241
220974199150,4cyclictest2333626-21sh09:10:011
220974199150,4cyclictest2287584-21taskset08:25:121
220974199150,4cyclictest2251640-21tr07:50:121
220974199150,3cyclictest2416575-21idleruntime-cro10:30:001
220974199150,3cyclictest2381325-21irqcore09:55:151
220974199150,3cyclictest2344788-21cpuspeed_turbos09:20:121
220974199150,2cyclictest2246176-21apt07:45:001
2209738991513,1cyclictest60350irq/132-enp0s29f1:rx-012:35:230
2209738991513,1cyclictest60350irq/132-enp0s29f1:rx-012:20:130
2209738991513,1cyclictest60350irq/132-enp0s29f1:rx-012:20:120
2209738991513,1cyclictest60350irq/132-enp0s29f1:rx-008:30:210
2209738991512,2cyclictest61150irq/133-enp0s29f1:tx-009:05:160
2209738991512,2cyclictest60350irq/132-enp0s29f1:rx-009:55:090
220974899148,3cyclictest0-21swapper/308:47:573
220974899143,9cyclictest2267035-21grep08:05:133
220974899142,3cyclictest2525721-21ntpq12:10:213
220974899142,3cyclictest2337908-21ntpq09:10:203
220974899142,2cyclictest2432813-21sed10:45:173
220974899142,2cyclictest2317271-21ntpq08:50:193
220974899141,8cyclictest2245101-21memory07:40:173
2209748991411,2cyclictest2490958-21kthreadcore11:40:183
220974699149,4cyclictest335557-21systemd-journal08:45:012
2209746991411,2cyclictest2257119-21irqcore07:55:162
2209746991411,2cyclictest155503-21kworker/u8:0+events_unbound07:35:072
2209746991410,3cyclictest2484534-21kworker/u8:3+flush-259:011:40:302
220974199141,2cyclictest2520589-21ntpq12:05:231
220974199141,2cyclictest2272116-21seq08:10:111
220974199140,3cyclictest2209791-21latency_hist07:09:591
220974199140,2cyclictest2339603-21grep09:15:121
220973899149,3cyclictest2505058-21kworker/u8:2+flush-259:012:00:210
220973899146,6cyclictest2493971-21ntpq11:40:230
2209738991413,1cyclictest60350irq/132-enp0s29f1:rx-009:50:230
2209738991413,1cyclictest60350irq/132-enp0s29f1:rx-008:54:280
2209738991412,1cyclictest60350irq/132-enp0s29f1:rx-010:55:240
2209738991412,1cyclictest60350irq/132-enp0s29f1:rx-009:35:180
2209738991412,1cyclictest60350irq/132-enp0s29f1:rx-008:45:240
2209738991412,1cyclictest60350irq/132-enp0s29f1:rx-008:10:150
2209738991412,1cyclictest2250313-21needreboot07:45:210
2209738991412,1cyclictest2250313-21needreboot07:45:200
2209738991411,2cyclictest60350irq/132-enp0s29f1:rx-012:10:160
2209738991411,2cyclictest60350irq/132-enp0s29f1:rx-008:15:170
2209738991411,2cyclictest60350irq/132-enp0s29f1:rx-008:05:120
2209738991411,2cyclictest60350irq/132-enp0s29f1:rx-007:20:110
2209738991410,3cyclictest2484754-21apt-get11:35:110
2209738991410,2cyclictest2220008-21cron07:20:000
220974899136,5cyclictest2458075-21date11:10:013
220974899132,7cyclictest50-21ksoftirqd/312:30:223
220974899132,2cyclictest50-21ksoftirqd/311:00:173
220974899132,2cyclictest2375502-21/usr/sbin/munin09:50:223
220974899132,2cyclictest2262817-21irqstats08:00:163
220974899131,7cyclictest2442825-21diskstats10:55:133
220974899131,2cyclictest2401501-21df10:15:103
220974899131,2cyclictest2322396-21ntpq08:55:183
2209748991310,2cyclictest2506319-21cpuspeed_turbos11:55:143
220974699139,3cyclictest155503-21kworker/u8:0+events_unbound07:11:262
2209746991310,2cyclictest2241039-21kworker/u8:3+events_unbound07:53:092
220973899138,3cyclictest2390810-21kworker/u8:0+events_unbound10:45:190
2209738991312,1cyclictest60350irq/132-enp0s29f1:rx-007:26:220
2209738991311,1cyclictest60350irq/132-enp0s29f1:rx-010:25:210
2209738991311,1cyclictest60350irq/132-enp0s29f1:rx-009:40:180
220974899129,2cyclictest2547645-21cpuspeed_turbos12:35:143
220974899129,2cyclictest2401078-21cron10:15:003
220974899129,2cyclictest2303156-21cpuspeed_turbos08:40:113
220974899129,2cyclictest0-21swapper/307:35:203
220974899128,1cyclictest191rcu_preempt12:00:173
220974899121,3cyclictest2521416-21sshd12:09:333
220974899121,2cyclictest2473337-21users11:20:253
220974899121,2cyclictest2328887-21diskstats09:05:123
2209748991210,1cyclictest0-21swapper/310:45:003
220974899120,3cyclictest2256054-21sshd07:50:343
220974899120,2cyclictest2407479-21irqstats10:20:163
220974699128,3cyclictest2421797-21kworker/u8:3+events_unbound10:39:172
220973899129,2cyclictest60350irq/132-enp0s29f1:rx-011:05:150
220973899127,4cyclictest60350irq/132-enp0s29f1:rx-009:10:120
2209738991210,2cyclictest60350irq/132-enp0s29f1:rx-010:35:070
2209738991210,1cyclictest60350irq/132-enp0s29f1:rx-008:55:580
220974899116,1cyclictest191rcu_preempt10:30:133
220974899111,6cyclictest2282685-21fw_forwarded_lo08:20:153
220974899111,6cyclictest2272275-21diskstats08:10:133
220974899111,4cyclictest50-21ksoftirqd/312:25:173
220974899111,3cyclictest50-21ksoftirqd/311:35:203
220974899111,2cyclictest2246404-21runrttasks07:45:023
220974899111,2cyclictest2246404-21runrttasks07:45:023
220973899111,3cyclictest2531803-21latency_hist12:20:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional