You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-24 - 17:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot3s.osadl.org (updated Wed Dec 24, 2025 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12912239163,25sleep00-21swapper/007:09:360
9812204168,24sleep20-21swapper/207:05:352
11632202166,24sleep10-21swapper/107:07:561
13152200164,24sleep30-21swapper/307:09:523
1027121850,3sleep1141399cyclictest12:20:211
14199910399,3cyclictest0-21swapper/207:28:302
141999103100,2cyclictest0-21swapper/211:59:122
1419999289,2cyclictest0-21swapper/212:31:262
277652710,1sleep30-21swapper/309:20:173
143872530,6sleep339-21rcuc/307:35:233
141399491,15cyclictest0-21swapper/112:11:401
1419994016,16cyclictest10150-21/usr/sbin/munin09:55:282
1419994013,13cyclictest21684-21latency_hist11:35:022
141399394,7cyclictest24330-40dnf09:10:301
141399382,33cyclictest1112-21munin-run09:35:011
1419993713,12cyclictest11165-21sh12:25:012
1419993712,14cyclictest11398-21/usr/sbin/munin12:25:152
1419993711,14cyclictest0-21swapper/207:49:322
275072360,1sleep20-21swapper/209:20:162
1419993613,13cyclictest499-21uname10:45:172
1419993613,12cyclictest29320-21/usr/sbin/munin09:25:132
1419993612,21cyclictest31312-21tune2fs08:15:152
1419993612,14cyclictest26977-21timerwakeupswit10:30:262
1419993611,14cyclictest13934-21uptime10:00:292
1413993615,4cyclictest13623-21cut11:15:161
141399361,19cyclictest11846-21ntp_states11:10:221
1405993611,12cyclictest6782-21munin-run12:15:000
1419993512,13cyclictest17373-21/usr/sbin/munin11:25:262
1419993511,14cyclictest2587-21gunzip09:35:272
140599351,27cyclictest0-21swapper/008:06:460
141999342,20cyclictest12455-21kworker/2:008:10:472
1419993415,10cyclictest9538-21users09:50:282
1419993413,8cyclictest645-21/usr/sbin/munin08:20:132
1419993413,8cyclictest16422-21memory07:40:192
1419993412,10cyclictest16035-21unixbench-2d10:05:252
1419993411,13cyclictest13344-21fschecks_count11:15:142
1419993410,13cyclictest19389-21ntpq09:00:232
1419993410,13cyclictest19389-21ntpq09:00:232
141399345,18cyclictest0-21swapper/111:08:441
141999338,14cyclictest0-21swapper/211:00:012
1419993314,9cyclictest3051-21ntp_states10:50:222
1419993310,7cyclictest15277-21latency_hist07:40:022
1419993310,13cyclictest1926-21awk12:00:262
141399334,3cyclictest0-21swapper/112:15:351
1413993317,3cyclictest12767-21timerandwakeup12:25:251
1413993316,2cyclictest0-21swapper/107:27:531
140599331,26cyclictest0-21swapper/007:27:090
141999329,14cyclictest0-21swapper/207:10:172
1419993215,6cyclictest23468-21tr10:25:112
1419993214,7cyclictest12857-21taskset07:32:142
1419993213,8cyclictest22211-21ntp_states10:20:212
141399321,10cyclictest397-21latency_hist08:20:021
140599322,14cyclictest23127-21munin-run10:25:020
140599321,24cyclictest0-21swapper/011:18:560
140599321,23cyclictest22502-21diskmemload10:33:040
140599321,23cyclictest0-21swapper/009:56:070
140599321,22cyclictest0-21swapper/010:43:020
140599321,15cyclictest7116-21munin-run08:35:020
141999319,9cyclictest0-21swapper/210:35:182
141999319,12cyclictest0-21swapper/209:20:022
141999318,10cyclictest10975-21expr11:10:122
1419993112,12cyclictest0-21swapper/212:11:092
141399312,7cyclictest30555-21/usr/sbin/munin11:55:251
141399311,7cyclictest0-21swapper/109:27:311
141399311,12cyclictest0-21swapper/110:40:351
141399311,12cyclictest0-21swapper/109:04:161
141399311,12cyclictest0-21swapper/109:04:161
140599311,23cyclictest16813-21kworker/0:211:25:250
140599311,23cyclictest0-21swapper/012:05:330
141999309,9cyclictest835-21runrttasks10:41:002
141999309,7cyclictest22502-21diskmemload11:47:432
141999308,7cyclictest0-21swapper/211:40:162
141999308,12cyclictest22502-21diskmemload10:19:072
141999308,12cyclictest22502-21diskmemload10:19:072
141999308,10cyclictest0-21swapper/211:05:002
141999302,23cyclictest12455-21kworker/2:007:56:392
1419993020,7cyclictest23077-21/usr/sbin/munin09:10:152
1419993014,8cyclictest3906-21diskstats09:40:152
1419993010,12cyclictest8250-21mailstats12:15:232
141399308,3cyclictest0-21swapper/110:56:121
141399301,9cyclictest16993-21expr10:10:141
141399301,14cyclictest0-21swapper/109:53:211
141399301,11cyclictest0-21swapper/110:01:071
141399301,10cyclictest22502-21diskmemload11:04:041
141399300,15cyclictest0-21swapper/111:36:281
141399300,14cyclictest0-21swapper/108:24:161
141399300,14cyclictest0-21swapper/107:50:391
141399300,14cyclictest0-21swapper/107:50:391
141399300,12cyclictest0-21swapper/111:48:121
140599301,14cyclictest30555-21/usr/sbin/munin11:55:170
140599301,14cyclictest0-21swapper/010:58:480
140599301,13cyclictest0-21swapper/008:44:380
141999296,13cyclictest87250irq/129-enp1s0-08:45:212
1419992919,7cyclictest25364-21/usr/sbin/munin08:00:212
1419992913,7cyclictest4528-21/usr/sbin/munin07:15:152
1419992912,5cyclictest15174-21/usr/sbin/munin11:20:252
1419992911,6cyclictest801-21snmpd10:12:212
1419992911,5cyclictest0-21swapper/212:10:022
141399297,4cyclictest19294-21df_inode10:15:141
141399297,4cyclictest19294-21df_inode10:15:141
141399294,8cyclictest751-21rngd08:46:261
1413992912,3cyclictest0-21swapper/109:41:301
141399291,12cyclictest0-21swapper/110:46:191
141399290,14cyclictest0-21swapper/108:33:431
140599299,3cyclictest0-21swapper/008:57:010
140599293,13cyclictest31189-21cstates08:15:140
140599291,17cyclictest0-21swapper/008:26:160
140599291,14cyclictest0-21swapper/012:23:170
140599291,13cyclictest0-21swapper/012:38:140
247632280,1sleep10-21swapper/111:40:171
1419992822,4cyclictest5582-21tune2fs08:30:162
1419992819,6cyclictest8147-21irqstats08:35:172
1419992819,6cyclictest26504-21/usr/sbin/munin08:05:202
1419992818,7cyclictest9529-21/usr/sbin/munin08:40:252
1419992817,8cyclictest15794-21/usr/sbin/munin12:35:182
141399281,10cyclictest0-21swapper/110:07:231
1413992810,3cyclictest25319-21tune2fs09:15:161
141399280,15cyclictest0-21swapper/109:24:011
140599281,18cyclictest0-21swapper/010:10:330
140599281,14cyclictest8358-21munin-run11:05:010
140599281,13cyclictest0-21swapper/010:15:500
140599281,13cyclictest0-21swapper/010:15:500
140599281,13cyclictest0-21swapper/007:37:530
140599281,13cyclictest0-21swapper/007:11:360
140599281,12cyclictest0-21swapper/009:43:130
140599281,12cyclictest0-21swapper/008:53:570
140599281,12cyclictest0-21swapper/007:50:240
140599281,12cyclictest0-21swapper/007:50:240
140599281,11cyclictest0-21swapper/011:40:450
140599281,11cyclictest0-21swapper/010:38:400
141999277,12cyclictest12050irq/123-ahci[0009:35:022
141999277,11cyclictest87150irq/128-enp1s0-11:35:162
1419992720,4cyclictest3647-21/usr/sbin/munin08:25:162
1419992720,4cyclictest20583-21/usr/sbin/munin07:50:152
1419992720,4cyclictest20583-21/usr/sbin/munin07:50:152
1419992720,4cyclictest14958-21ntp_kernel_pll_08:50:202
1419992719,6cyclictest16127-21/usr/sbin/munin08:55:182
1419992719,5cyclictest21596-21ntp_states09:05:202
1419992718,6cyclictest8766-21/usr/sbin/munin11:05:122
1419992718,6cyclictest29463-21/usr/sbin/munin11:50:232
1419992717,7cyclictest6725-21/usr/sbin/munin07:20:182
1419992717,7cyclictest5755-21/usr/sbin/munin09:45:222
141399279,3cyclictest28454-40dnf08:09:551
141399278,2cyclictest0-21swapper/107:20:281
141399271,19cyclictest0-21swapper/112:09:281
141399271,12cyclictest0-21swapper/112:03:151
141399270,11cyclictest0-21swapper/109:35:451
140599271,12cyclictest0-21swapper/012:02:130
140599271,12cyclictest0-21swapper/011:54:310
140599271,11cyclictest0-21swapper/012:30:220
140599271,11cyclictest0-21swapper/009:46:290
140599271,11cyclictest0-21swapper/008:14:540
140599271,11cyclictest0-21swapper/008:03:020
140599271,11cyclictest0-21swapper/007:17:120
140599271,10cyclictest0-21swapper/011:21:510
140599271,10cyclictest0-21swapper/007:55:040
140599270,12cyclictest0-21swapper/012:28:320
141399265,3cyclictest0-21swapper/108:25:341
141399261,15cyclictest0-21swapper/111:29:281
141399261,15cyclictest0-21swapper/110:28:271
141399261,14cyclictest0-21swapper/108:59:461
141399261,12cyclictest0-21swapper/112:36:351
141399260,7cyclictest0-21swapper/107:15:341
141399260,12cyclictest0-21swapper/112:32:011
141399260,10cyclictest0-21swapper/111:50:471
140599262,9cyclictest6814-21kworker/0:308:36:060
140599261,13cyclictest0-21swapper/011:08:300
140599261,13cyclictest0-21swapper/010:51:460
140599261,12cyclictest0-21swapper/010:04:230
140599261,11cyclictest0-21swapper/012:16:050
140599261,11cyclictest0-21swapper/009:54:290
140599261,10cyclictest0-21swapper/009:27:370
140599261,10cyclictest0-21swapper/008:45:420
141399254,3cyclictest0-21swapper/111:21:121
141399252,2cyclictest87150irq/128-enp1s0-08:42:401
141399251,14cyclictest0-21swapper/107:43:371
141399251,13cyclictest0-21swapper/107:30:151
141399251,12cyclictest0-21swapper/109:57:211
141399250,10cyclictest0-21swapper/110:54:351
140599253,6cyclictest751-21rngd09:17:090
140599252,11cyclictest0-21swapper/011:12:140
140599251,9cyclictest0-21swapper/009:22:290
140599251,8cyclictest0-21swapper/009:13:570
140599251,10cyclictest0-21swapper/011:45:410
140599251,10cyclictest0-21swapper/010:25:380
140599251,10cyclictest0-21swapper/009:39:010
140599251,10cyclictest0-21swapper/009:02:290
140599251,10cyclictest0-21swapper/009:02:290
141399247,3cyclictest0-21swapper/108:03:511
141399241,14cyclictest0-21swapper/108:35:381
140599241,11cyclictest0-21swapper/007:20:470
140599241,10cyclictest0-21swapper/011:31:410
140599241,10cyclictest0-21swapper/010:48:240
140599241,10cyclictest0-21swapper/007:44:050
313642231,18sleep330555-21/usr/sbin/munin11:55:213
1423992312,2cyclictest19342-21awk11:30:003
141399238,3cyclictest0-21swapper/110:32:111
141399237,4cyclictest0-21swapper/110:35:151
141399233,3cyclictest0-21swapper/109:05:321
141399231,14cyclictest0-21swapper/111:33:161
141399231,13cyclictest0-21swapper/110:20:511
140599239,3cyclictest0-21swapper/007:33:090
140599231,11cyclictest0-21swapper/010:07:210
140599231,11cyclictest0-21swapper/009:30:150
140599231,10cyclictest0-21swapper/011:38:450
141399221,12cyclictest0-21swapper/109:45:471
140599225,2cyclictest0-21swapper/009:05:170
140599221,9cyclictest0-21swapper/008:22:460
140599221,9cyclictest0-21swapper/007:45:580
142399218,4cyclictest12050irq/123-ahci[0007:15:013
142399211,7cyclictest986-1kworker/u9:109:43:523
142399211,7cyclictest0-21swapper/312:33:253
142399211,11cyclictest31515-21/usr/sbin/munin09:30:183
141399211,13cyclictest0-21swapper/107:12:431
141399211,10cyclictest14267-21df_inode08:50:141
141399210,2cyclictest86950irq/126-enp1s0-07:35:191
58092201,15sleep34528-21/usr/sbin/munin07:15:253
142399202,10cyclictest24295-21/usr/sbin/munin08:00:153
142399201,6cyclictest0-21swapper/310:37:033
142399201,6cyclictest0-21swapper/309:50:413
142399201,4cyclictest0-21swapper/309:29:273
1413992013,2cyclictest0-21swapper/107:56:291
1413992012,3cyclictest0-21swapper/107:46:481
141399201,17cyclictest0-21swapper/108:10:191
142399199,3cyclictest801-21snmpd08:24:353
142399197,4cyclictest0-21swapper/310:05:023
142399193,13cyclictest14543-21grep08:50:153
142399191,6cyclictest0-21swapper/312:12:273
142399191,5cyclictest0-21swapper/312:36:023
1423991915,3cyclictest0-21swapper/307:40:373
142399191,4cyclictest0-21swapper/312:25:153
142399191,3cyclictest0-21swapper/310:57:273
142399191,11cyclictest23346-21/usr/sbin/munin10:25:113
142399191,10cyclictest6376-21/usr/sbin/munin11:00:203
142399191,10cyclictest32132-21/usr/sbin/munin10:45:243
142399191,10cyclictest2612-21/usr/sbin/munin12:05:193
142399191,10cyclictest10779-21/usr/sbin/munin11:10:163
142399190,8cyclictest0-21swapper/309:37:233
169362181,3sleep30-21swapper/311:22:293
142399188,3cyclictest801-21snmpd07:51:543
142399188,3cyclictest801-21snmpd07:51:543
142399185,4cyclictest3831-21sshd10:53:393
142399183,5cyclictest0-21swapper/311:53:183
142399182,4cyclictest32747-21/usr/sbin/munin12:00:193
142399181,8cyclictest31962-21ntp_states08:15:213
142399181,7cyclictest0-21swapper/311:49:063
142399181,7cyclictest0-21swapper/309:46:073
142399181,6cyclictest30521-21/usr/sbin/munin10:40:173
142399181,4cyclictest0-21swapper/311:05:593
142399181,4cyclictest0-21swapper/310:30:373
1423991812,4cyclictest7767-21awk12:15:173
142399181,14cyclictest21695-21fschecks_time10:20:163
212142171,2sleep30-21swapper/311:31:003
142399178,6cyclictest764-21ntpd07:25:213
142399176,3cyclictest23960-21/usr/sbin/munin11:40:123
142399175,3cyclictest262332switchtime09:15:253
142399173,3cyclictest139382users10:00:303
142399171,9cyclictest0-21swapper/312:20:143
142399171,7cyclictest18954-21/usr/sbin/munin10:15:183
142399171,7cyclictest18954-21/usr/sbin/munin10:15:183
1423991712,3cyclictest24136-21sort09:10:253
1423991711,3cyclictest18009-21cat07:45:143
142399171,10cyclictest0-21swapper/308:35:233
142399161,6cyclictest22158-21cut11:35:153
142399161,6cyclictest0-21swapper/308:11:083
142399161,6cyclictest0-21swapper/307:30:203
142399161,4cyclictest5142-21/usr/sbin/munin08:30:203
142399161,3cyclictest86950irq/126-enp1s0-11:15:123
142399161,2cyclictest0-21swapper/309:04:053
142399161,2cyclictest0-21swapper/309:04:053
1423991612,3cyclictest0-21swapper/308:41:463
1423991612,3cyclictest0-21swapper/308:26:233
142399161,14cyclictest0-21swapper/309:07:213
142399161,14cyclictest0-21swapper/308:45:013
1423991611,3cyclictest26504-21/usr/sbin/munin08:05:183
1423991611,3cyclictest22101-21/usr/sbin/munin07:55:163
142399161,13cyclictest18092-21sh09:00:003
1423991611,3cyclictest17629-21latency10:10:203
1423991610,3cyclictest11109-21cat09:55:203
142399159,3cyclictest6740-21bc07:20:153
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional