You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-14 - 23:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot4s.osadl.org (updated Mon Sep 14, 2026 12:44:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2562639995752,3cyclictest2734976-21kworker/u8:0+flush-259:010:30:222
2562630994943,4cyclictest2766490-21latency_hist10:25:010
2562643994235,5cyclictest2708857-21latency_hist09:30:003
2562639994239,2cyclictest2862582-21kworker/u8:0+events_unbound12:05:232
2562639994239,2cyclictest2783207-21kworker/u8:0+flush-259:011:30:142
2562639994036,3cyclictest2567694-21kworker/u8:2+events_unbound07:25:212
2562639993734,2cyclictest2734976-21kworker/u8:0+flush-259:010:10:232
2562643993630,4cyclictest2588885-21/usr/sbin/munin07:35:253
2562639993632,3cyclictest2897502-21kworker/u8:4+events_unbound12:35:142
2562639993629,3cyclictest2862582-21kworker/u8:0+events_unbound12:00:172
2562634993623,4cyclictest2823848-21ntpq11:15:221
2562639993532,2cyclictest2714457-21kworker/u8:3+flush-259:009:45:242
2562639993532,2cyclictest2614280-21kworker/u8:2+flush-259:009:25:212
2562639993531,3cyclictest2887424-21kworker/u8:2+events_unbound12:30:132
2562639993531,3cyclictest2677413-21kworker/u8:0+events_unbound09:35:152
2562630993531,3cyclictest2862582-21kworker/u8:0+flush-259:012:00:240
2562630993531,3cyclictest2756451-21apt-get10:15:110
2562630993530,3cyclictest2714679-21apt-get09:35:120
2562643993430,3cyclictest2614289-21kworker/u8:4+flush-259:008:20:233
2562639993431,2cyclictest2819635-21kworker/u8:2+events_unbound11:40:202
2562639993431,2cyclictest2583530-21kworker/u8:1+events_unbound07:35:242
2562639993430,2cyclictest2614289-21kworker/u8:4+events_unbound09:00:192
2562639993330,2cyclictest2856168-21kworker/u8:1+events_unbound11:55:202
2562639993330,2cyclictest2567694-21kworker/u8:2+events_unbound07:15:222
2562639993329,3cyclictest2603859-21kworker/u8:0+flush-259:007:55:162
2562639993229,2cyclictest2793321-21kworker/u8:4+events_unbound10:55:202
2562639993229,2cyclictest2567694-21kworker/u8:2+flush-259:007:20:232
2562639993228,2cyclictest2783207-21kworker/u8:0+events_unbound11:25:172
2562639993227,3cyclictest2593837-21kworker/u8:3+events_unbound07:50:172
2562643993127,2cyclictest2766713-21kworker/u8:2+events_unbound10:35:203
2562643993126,4cyclictest2547081-21kworker/u8:1+flush-259:007:10:163
2562643993125,4cyclictest2609592-21apt-get07:55:113
2562643993025,3cyclictest2567694-21kworker/u8:2+flush-259:007:20:163
2562639993027,2cyclictest2734976-21kworker/u8:0+flush-259:010:00:232
2562639993027,2cyclictest2724756-21kworker/u8:1+flush-259:010:05:152
2562639993027,2cyclictest2614280-21kworker/u8:2+events_unbound09:50:162
2562639993026,3cyclictest2846066-21kworker/u8:3+events_unbound11:45:202
2562630993024,4cyclictest2677661-21apt-get09:00:130
2562643992925,3cyclictest2897502-21kworker/u8:4+flush-259:012:35:183
2562643992925,3cyclictest2614280-21kworker/u8:2+flush-259:008:20:013
2562643992924,3cyclictest2740915-21apt-get10:00:103
2562639992926,2cyclictest2783207-21kworker/u8:0+events_unbound10:45:182
2562639992926,2cyclictest2614289-21kworker/u8:4+events_unbound08:35:212
2562639992926,2cyclictest2614280-21kworker/u8:2+events_unbound08:10:242
2562639992925,3cyclictest2724756-21kworker/u8:1+flush-259:011:05:162
2562639992925,3cyclictest2547081-21kworker/u8:1+flush-259:007:10:202
2562639992924,4cyclictest2614280-21kworker/u8:2+events_unbound09:10:212
2562643992824,2cyclictest2766539-21systemd-run10:25:003
2562643992821,5cyclictest335557-21systemd-journal12:15:003
2562643992821,4cyclictest2646406-21latency_hist08:30:013
2562639992825,2cyclictest2856168-21kworker/u8:1+events_unbound11:53:382
2562639992825,2cyclictest2714457-21kworker/u8:3+flush-259:009:40:192
2562639992824,3cyclictest2830129-21kworker/u8:1+events_unbound11:35:232
2562639992824,3cyclictest2793321-21kworker/u8:4+flush-259:010:50:162
2562639992824,3cyclictest2734976-21kworker/u8:0+events_unbound10:15:202
2562634992824,3cyclictest2636362-21apt-get08:20:121
256263099280,26cyclictest509-21rs:main0
2562643992722,3cyclictest2614280-21kworker/u8:2+flush-259:009:45:163
2562639992724,2cyclictest2766713-21kworker/u8:2+flush-259:010:35:152
2562639992723,2cyclictest2614280-21kworker/u8:2+events_unbound09:15:172
2562639992721,4cyclictest2766485-21apt-get10:25:022
2562634992721,4cyclictest2851250-21/usr/sbin/munin11:45:131
2562630992723,3cyclictest2787704-21apt-get10:44:590
2562630992722,4cyclictest2567694-21kworker/u8:2+flush-259:007:20:240
2562643992619,5cyclictest2676530-21ntpq08:55:203
2562643992619,5cyclictest2676530-21ntpq08:55:193
2562639992623,2cyclictest2614289-21kworker/u8:4+events_unbound08:20:132
2562639992623,2cyclictest2614280-21kworker/u8:2+flush-259:008:55:262
2562639992623,2cyclictest2614280-21kworker/u8:2+flush-259:008:55:252
2562639992622,3cyclictest2819635-21kworker/u8:2+events_unbound11:15:192
2562634992621,3cyclictest2782738-21/usr/sbin/munin10:40:141
2562634992619,4cyclictest335557-21systemd-journal11:10:001
2562643992522,2cyclictest2724756-21kworker/u8:1+flush-259:009:55:213
2562643992521,3cyclictest2614289-21kworker/u8:4+flush-259:008:40:233
2562643992520,3cyclictest2651827-21apt-get08:35:123
2562639992522,2cyclictest2614289-21kworker/u8:4+events_unbound08:45:212
2562639992522,2cyclictest2567694-21kworker/u8:2+events_unbound07:40:162
2562639992521,3cyclictest2862582-21kworker/u8:0+flush-259:012:20:262
2562639992521,3cyclictest2862582-21kworker/u8:0+events_unbound12:25:232
2562634992520,3cyclictest2862582-21kworker/u8:0+flush-259:012:25:131
2562634992519,4cyclictest2750160-21ntpq10:05:231
2562630992521,3cyclictest2819635-21kworker/u8:2+flush-259:011:40:490
2562630992521,3cyclictest2677413-21kworker/u8:0+flush-259:009:26:070
2562643992419,4cyclictest2567694-21kworker/u8:2+flush-259:007:45:193
2562639992420,3cyclictest2856168-21kworker/u8:1+events_unbound12:15:282
2562639992420,3cyclictest2614289-21kworker/u8:4+events_unbound09:06:382
2562634992418,4cyclictest2682761-21cp09:05:011
2562634992418,4cyclictest2578599-21apt-get07:25:131
2562630992419,4cyclictest2871924-21apt-get12:05:110
2562630992419,4cyclictest2830129-21kworker/u8:1+flush-259:011:35:170
2562630992419,3cyclictest2819635-21kworker/u8:2+flush-259:011:55:000
2562630992417,5cyclictest2788257-21gpgv10:45:010
2562643992318,3cyclictest2862582-21kworker/u8:0+flush-259:012:20:163
2562639992319,3cyclictest2677413-21kworker/u8:0+events_unbound09:30:162
2562634992316,5cyclictest2730551-21/usr/sbin/munin09:50:141
2562630992319,3cyclictest2688441-21apt-get09:10:120
2562630992318,4cyclictest2814672-21/usr/sbin/munin11:10:180
2562643992218,2cyclictest2614280-21kworker/u8:2+flush-259:008:45:153
2562639992219,2cyclictest2614280-21kworker/u8:2+flush-259:008:30:382
2562639992219,2cyclictest2567694-21kworker/u8:2+events_unbound07:45:132
2562634992215,5cyclictest2755362-21ntpq10:10:211
2562630992217,3cyclictest2730122-21gpgv09:50:000
2562643992117,3cyclictest2819635-21kworker/u8:2+flush-259:011:40:173
2562643992117,3cyclictest2614289-21kworker/u8:4+events_unbound08:50:223
2562643992116,4cyclictest2567694-21kworker/u8:2+events_unbound07:40:253
2562639992118,2cyclictest2614280-21kworker/u8:2+events_unbound09:20:162
2562639992117,3cyclictest2724756-21kworker/u8:1+events_unbound10:20:032
2562639992116,3cyclictest2614289-21kworker/u8:4+flush-259:008:40:152
2562630992117,3cyclictest2614289-21kworker/u8:4+flush-259:009:00:000
2562630992117,3cyclictest2614289-21kworker/u8:4+flush-259:008:59:590
2562630992117,3cyclictest2593837-21kworker/u8:3+flush-259:007:50:250
2562630992116,4cyclictest2766713-21kworker/u8:2+flush-259:010:35:190
2562639992017,2cyclictest2777293-21kworker/u8:3+events_unbound11:10:252
2562639992017,2cyclictest2724756-21kworker/u8:1+events_unbound09:55:172
2562639992016,3cyclictest2793321-21kworker/u8:4+flush-259:011:00:132
2562634992017,2cyclictest0-21swapper/110:00:011
2562630992016,3cyclictest2614280-21kworker/u8:2+flush-259:009:15:230
2562643991916,2cyclictest2777293-21kworker/u8:3+events_unbound10:40:213
2562643991915,3cyclictest2614289-21kworker/u8:4+events_unbound09:05:163
2562643991914,4cyclictest2588632-21cron07:35:013
2562643991913,4cyclictest2846066-21kworker/u8:3+flush-259:011:50:173
2562643991913,4cyclictest2624903-21sh08:09:593
2562639991916,2cyclictest2856168-21kworker/u8:1+flush-259:012:10:212
2562639991916,2cyclictest2614289-21kworker/u8:4+events_unbound08:50:132
256263499193,4cyclictest2865602-21ntpq11:55:211
2562634991911,6cyclictest2819613-21latency_hist11:15:001
2562630991915,2cyclictest2619718-21sh08:04:590
2562643991815,2cyclictest2862582-21kworker/u8:0+flush-259:012:00:213
2562643991814,3cyclictest2724756-21kworker/u8:1+events_unbound11:05:233
2562643991813,3cyclictest2856168-21kworker/u8:1+events_unbound11:55:243
2562643991810,6cyclictest335557-21systemd-journal08:05:003
2562639991814,3cyclictest2614289-21kworker/u8:4+events_unbound08:26:392
2562639991814,3cyclictest2614280-21kworker/u8:2+events_unbound08:07:372
256263499183,4cyclictest2901836-21ld12:30:211
256263499183,4cyclictest2593438-21modprobe07:35:371
2562634991815,2cyclictest2829919-21sh11:25:001
2562634991814,3cyclictest2614289-21kworker/u8:4+flush-259:009:05:151
2562634991814,3cyclictest2613578-21ntp_kernel_err07:55:211
2562634991814,2cyclictest2622298-21kthreadcore08:05:181
2562634991811,5cyclictest335557-21systemd-journal07:47:361
2562630991814,3cyclictest2567694-21kworker/u8:2+flush-259:007:35:260
2562630991813,3cyclictest2835794-21apt-get11:30:110
2562643991715,1cyclictest0-21swapper/310:15:013
2562643991714,2cyclictest2682647-21apt-key09:05:003
2562643991714,2cyclictest2567694-21kworker/u8:2+flush-259:007:15:223
2562643991713,3cyclictest2846066-21kworker/u8:3+events_unbound11:45:163
2562643991712,3cyclictest2777293-21kworker/u8:3+flush-259:011:10:183
2562639991714,2cyclictest2603859-21kworker/u8:0+flush-259:008:00:192
2562639991714,2cyclictest2552407-21kworker/u8:0+events_unbound07:30:112
2562639991712,3cyclictest2614280-21kworker/u8:2+flush-259:008:15:172
256263499174,4cyclictest2799506-21cat10:55:141
256263499174,2cyclictest2728981-21ntpq09:45:191
2562634991711,4cyclictest2614280-21kworker/u8:2+flush-259:009:30:241
256263499171,14cyclictest1-21systemd09:15:381
2562630991714,2cyclictest60350irq/132-enp0s29f1:rx-012:36:510
2562630991714,2cyclictest2567694-21kworker/u8:2+events_unbound07:15:250
2562630991713,3cyclictest60350irq/132-enp0s29f1:rx-011:25:210
2562630991713,3cyclictest2793321-21kworker/u8:4+flush-259:010:55:160
2562630991710,5cyclictest2687317-21ntpq09:05:220
2562643991613,2cyclictest2862582-21kworker/u8:0+flush-259:012:15:173
2562643991612,3cyclictest2714653-21/usr/sbin/munin09:35:203
2562643991612,3cyclictest2614280-21kworker/u8:2+events_unbound09:20:153
2562643991611,4cyclictest2830129-21kworker/u8:1+flush-259:011:35:153
2562643991611,3cyclictest2793321-21kworker/u8:4+flush-259:011:00:193
2562643991611,3cyclictest2724756-21kworker/u8:1+events_unbound10:30:143
2562639991613,2cyclictest2783787-21kthreadcore10:40:172
256263499169,6cyclictest2631183-21apt-get08:15:121
256263499168,5cyclictest2583885-21cpuspeed_turbos07:30:111
256263499166,8cyclictest2667110-21awk08:49:591
256263499165,2cyclictest2860478-21ntpq11:50:201
256263499163,3cyclictest2672034-21sshd08:52:381
2562634991612,3cyclictest2882017-21basename12:15:001
2562634991612,3cyclictest2793967-21irqcore10:50:151
2562634991611,3cyclictest2887458-21apt-get12:20:111
2562634991610,4cyclictest2840709-21latency_hist11:35:011
256263099169,5cyclictest2671353-21ntpq08:50:200
2562630991614,1cyclictest60350irq/132-enp0s29f1:rx-008:15:260
2562630991613,2cyclictest2793321-21kworker/u8:4+flush-259:010:50:210
2562630991612,2cyclictest2614280-21kworker/u8:2+flush-259:009:30:160
2562630991612,2cyclictest0-21swapper/007:59:590
2562630991610,4cyclictest335557-21systemd-journal07:10:020
2562643991511,3cyclictest2614280-21kworker/u8:2+flush-259:009:35:013
2562643991511,3cyclictest2614280-21kworker/u8:2+flush-259:008:10:253
2562643991510,4cyclictest2783207-21kworker/u8:0+events_unbound11:30:193
2562643991510,3cyclictest2614280-21kworker/u8:2+events_unbound09:10:173
2562639991512,2cyclictest2777293-21kworker/u8:3+events_unbound11:20:522
256263499159,4cyclictest2625445-21cat08:10:011
256263499157,2cyclictest0-21swapper/109:40:001
2562634991511,3cyclictest2714457-21kworker/u8:3+flush-259:009:40:181
2562634991510,4cyclictest2734976-21kworker/u8:0+flush-259:010:00:141
2562634991510,3cyclictest2777563-21/usr/sbin/munin10:35:171
2562630991513,2cyclictest60350irq/132-enp0s29f1:rx-011:48:150
2562630991513,1cyclictest60350irq/132-enp0s29f1:rx-012:15:240
2562630991513,1cyclictest60350irq/132-enp0s29f1:rx-012:10:140
2562630991513,1cyclictest60350irq/132-enp0s29f1:rx-010:00:240
2562630991513,1cyclictest60350irq/132-enp0s29f1:rx-009:50:230
2562630991513,1cyclictest60350irq/132-enp0s29f1:rx-007:45:220
2562630991513,1cyclictest60350irq/132-enp0s29f1:rx-007:25:240
2562630991512,2cyclictest60350irq/132-enp0s29f1:rx-011:05:170
2562630991512,2cyclictest60350irq/132-enp0s29f1:rx-010:33:000
2562630991512,2cyclictest60350irq/132-enp0s29f1:rx-010:25:180
2562630991512,2cyclictest60350irq/132-enp0s29f1:rx-010:11:060
2562630991512,2cyclictest60350irq/132-enp0s29f1:rx-008:49:580
2562630991512,2cyclictest60350irq/132-enp0s29f1:rx-008:20:320
2562643991410,3cyclictest2862582-21kworker/u8:0+flush-259:012:05:213
2562643991410,3cyclictest2734976-21kworker/u8:0+flush-259:010:25:213
256263499149,4cyclictest2830129-21kworker/u8:1+flush-259:011:35:161
256263499148,4cyclictest335557-21systemd-journal07:15:001
256263499148,4cyclictest2693320-21cron09:15:001
256263499148,4cyclictest2646402-21turbostat.cron08:29:591
256263499146,6cyclictest2886340-21ntpq12:15:221
256263499146,5cyclictest2672600-21cut08:55:111
256263499146,5cyclictest2672600-21cut08:55:111
256263499146,5cyclictest2573510-21grep07:20:121
256263499143,6cyclictest2907738-21runrttasks12:38:251
256263499143,3cyclictest2772367-21/usr/sbin/munin10:30:111
256263499143,2cyclictest2598050-21ntpq07:40:221
2562630991412,2cyclictest60350irq/132-enp0s29f1:rx-012:31:560
2562630991412,2cyclictest60350irq/132-enp0s29f1:rx-008:07:300
2562630991412,1cyclictest60350irq/132-enp0s29f1:rx-012:20:220
2562630991412,1cyclictest60350irq/132-enp0s29f1:rx-011:55:130
2562630991412,1cyclictest60350irq/132-enp0s29f1:rx-007:40:260
2562630991411,2cyclictest60350irq/132-enp0s29f1:rx-011:20:250
2562630991411,2cyclictest60350irq/132-enp0s29f1:rx-008:35:130
2562630991411,2cyclictest60350irq/132-enp0s29f1:rx-008:32:130
2562630991410,3cyclictest60350irq/132-enp0s29f1:rx-008:10:210
256264399136,4cyclictest2583487-21latency_hist07:30:013
2562643991310,2cyclictest2799344-21fschecks_count10:55:143
256263499138,3cyclictest2772111-21cron10:30:001
256263499138,3cyclictest2762050-21ps10:20:161
256263499138,3cyclictest2619699-21cron08:04:591
256263499138,3cyclictest2568653-21tr07:15:171
256263499137,5cyclictest2835100-21grep11:25:241
256263499137,4cyclictest2867160-21irqcore12:00:161
256263499137,4cyclictest2646812-21cpuspeed_turbos08:30:121
256263499136,5cyclictest2788472-21tr10:45:121
256263499135,6cyclictest2847792-21grep11:40:181
256263499135,5cyclictest2709291-21dpkg09:30:011
256263499134,3cyclictest2809258-21awk11:05:001
256263499133,2cyclictest2872230-21hddtemp_smartct12:05:131
256263499132,3cyclictest0-21swapper/108:37:431
2562634991310,2cyclictest2701067-21kthreadcore09:20:181
256263099139,3cyclictest2641715-21fschecks_count08:25:140
256263099137,4cyclictest2820308-21tr11:15:180
2562630991311,1cyclictest60350irq/132-enp0s29f1:rx-009:20:230
2562630991311,1cyclictest60350irq/132-enp0s29f1:rx-007:31:250
2562630991310,2cyclictest60350irq/132-enp0s29f1:rx-008:43:550
256264399128,3cyclictest2783207-21kworker/u8:0+events_unbound10:45:223
256264399128,2cyclictest2734976-21kworker/u8:0+events_unbound10:15:173
256264399126,4cyclictest2614289-21kworker/u8:4+flush-259:008:34:423
256264399125,5cyclictest2730437-21cat09:50:003
256263499127,4cyclictest2660964-21needreboot08:40:171
256263499127,3cyclictest2604552-21cpuspeed_turbos07:50:131
256263499124,6cyclictest2756804-21if_enp0s30f410:15:141
256264399119,1cyclictest0-21swapper/309:15:243
256264399117,3cyclictest2831642-21sort11:25:163
256264399117,3cyclictest2593837-21kworker/u8:3+flush-259:007:50:233
256264399115,4cyclictest2897493-21awk12:30:003
256263099119,1cyclictest60350irq/132-enp0s29f1:rx-009:40:190
2562630991110,1cyclictest60350irq/132-enp0s29f1:rx-010:05:110
256264399107,2cyclictest0-21swapper/312:35:003
256264399106,3cyclictest2821230-21tr11:15:193
25626439995,3cyclictest2746862-21cut10:05:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional