You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-25 - 00:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot5s.osadl.org (updated Wed Jun 24, 2026 12:44:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2332769995950,5cyclictest2375067-21munin-run09:10:013
2332764995529,21cyclictest0-21swapper/208:20:162
2332764995345,5cyclictest2444267-21latency_hist12:30:012
2332764995239,9cyclictest311ktimers/212:20:112
2332764994926,19cyclictest0-21swapper/207:20:012
2332756994915,7cyclictest131rcu_preempt10:45:120
2332764994820,25cyclictest2419824-21cron11:20:002
2332760994841,5cyclictest2397312-21kworker/u8:2+events_unbound10:23:171
2332769994736,7cyclictest2421126-21grep11:20:263
2332769994639,4cyclictest2334727-21cat07:15:013
2332764994632,10cyclictest311ktimers/209:25:102
233276499461,11cyclictest2437543-21apt-get12:10:112
2332756994639,5cyclictest2336514-21kworker/u8:0+events_unbound07:57:500
233276499431,9cyclictest2395033-21/usr/sbin/munin10:05:272
2332769994234,4cyclictest2423262-21munin-plugin-st11:30:003
2332764994216,23cyclictest2415802-21awk11:05:272
233276499421,10cyclictest2389080-21/usr/sbin/munin09:50:152
2332760994234,6cyclictest231ktimers/110:05:141
2332756994236,4cyclictest2421225-21kworker/u8:0+events_unbound11:28:340
2332756994236,4cyclictest2355731-21kworker/u8:1+events_unbound08:22:530
233276499411,8cyclictest2413370-21/usr/sbin/munin11:00:182
233276499411,8cyclictest2354527-21/usr/sbin/munin08:10:162
233276499411,37cyclictest2434696-21expr12:00:252
233276499411,37cyclictest2421369-21cron11:23:002
233276499411,33cyclictest2382334-21cut09:30:162
233276499411,33cyclictest2382334-21cut09:30:152
233276499410,2cyclictest0-21swapper/210:45:102
2332760994134,4cyclictest2371663-21sendmail09:00:011
2332756994134,4cyclictest2334393-21munin-plugin-st07:15:010
2332764994033,4cyclictest2368248-21idleruntime-cro08:50:012
2332764994033,4cyclictest2334499-21latency_hist07:15:002
2332764994032,4cyclictest0-21swapper/210:00:012
233276499401,9cyclictest2436569-21/usr/sbin/munin12:05:252
233276499401,9cyclictest2370289-21diskstats08:55:132
233276499401,8cyclictest2395974-21/usr/sbin/munin10:10:252
233276499401,4cyclictest2393023-21meminfo10:00:192
233276499401,36cyclictest2386807-21sshd09:42:432
233276499401,34cyclictest2439349-21df12:15:132
233276499401,34cyclictest2372052-21fschecks_count09:00:132
2332764994010,27cyclictest2377825-21cat09:15:262
233276499400,3cyclictest0-21swapper/210:40:112
233276499400,3cyclictest0-21swapper/210:40:102
2332760994031,5cyclictest2419851-21sendmail11:19:591
233276499393,30cyclictest2416605-21cut11:10:142
233276499392,4cyclictest2427876-21/usr/sbin/munin11:40:252
233276499392,4cyclictest2427876-21/usr/sbin/munin11:40:242
233276499392,30cyclictest2361410-21apt-key08:30:002
233276499391,9cyclictest2398129-21/usr/sbin/munin10:15:192
233276499391,8cyclictest2387919-21/usr/sbin/munin09:45:242
233276499391,8cyclictest2349434-21/usr/sbin/munin07:55:122
233276499391,8cyclictest2342616-21fschecks_count07:35:162
233276499391,8cyclictest2339692-21/usr/sbin/munin07:25:222
233276499391,5cyclictest2432449-21/usr/sbin/munin11:55:152
233276499391,3cyclictest0-21swapper/211:50:122
233276499391,3cyclictest0-21swapper/208:40:112
233276499391,35cyclictest2363505-21/usr/sbin/munin08:35:172
233276499391,33cyclictest2380193-21cron09:25:012
233276499391,33cyclictest2344417-21/usr/sbin/munin07:40:182
233276499391,32cyclictest2401316-21grep10:25:152
2332764993911,25cyclictest2426651-21latency_hist11:39:592
2332764993910,26cyclictest204-21dbus-daemon08:02:482
233276499390,3cyclictest0-21swapper/209:05:142
233276499390,3cyclictest0-21swapper/207:20:122
2332760993929,6cyclictest2402968-21fschecks_count10:30:171
2332760993927,5cyclictest2410959-21kworker/u8:2+events_unbound10:58:221
2332756993933,4cyclictest2428803-21kworker/u8:4+events_unbound11:58:410
2332756993930,7cyclictest121ktimers/007:15:110
2332769993826,8cyclictest2399171-21sendmail10:20:003
233276499382,3cyclictest2444685-21df12:30:142
233276499381,6cyclictest2346015-21/usr/sbin/munin07:45:152
233276499381,6cyclictest2346015-21/usr/sbin/munin07:45:152
233276499381,5cyclictest2409735-21df_inode10:50:152
2332764993815,19cyclictest0-21swapper/208:30:142
233276499381,32cyclictest2375407-21df_inode09:10:142
233276499381,32cyclictest2357159-21grep08:15:262
233276499381,32cyclictest2353150-21meminfo08:05:222
233276499380,3cyclictest0-21swapper/210:35:122
2332760993829,5cyclictest0-21swapper/112:23:431
2332756993832,4cyclictest2393818-21kworker/u8:3+events_unbound10:03:110
2332769993726,7cyclictest2355401-21grep08:10:253
2332769993724,9cyclictest2338826-21sh07:25:013
233276499373,31cyclictest2423599-21diskstats11:30:142
233276499372,31cyclictest2402803-21/usr/sbin/munin10:30:282
233276499372,31cyclictest2340947-21fschecks_time07:30:162
233276499371,9cyclictest2447039-21swap12:35:232
233276499371,5cyclictest2421892-21df_inode11:25:132
233276499371,3cyclictest0-21swapper/211:45:122
233276499371,3cyclictest0-21swapper/211:45:122
233276499371,31cyclictest2400864-21cron10:24:592
233276099374,6cyclictest131rcu_preempt08:10:121
2332756993730,5cyclictest2337351-21kworker/u8:4+events_unbound07:52:490
2332756993726,7cyclictest2362527-21sort08:30:250
2332756993712,6cyclictest131rcu_preempt12:20:120
2332769993629,4cyclictest2332750-21cyclictest09:55:013
233276499361,9cyclictest2368704-21fw_packets08:50:162
233276499361,4cyclictest2411697-21/usr/sbin/munin10:55:172
233276499361,4cyclictest2384459-21/usr/sbin/munin09:35:262
2332760993626,6cyclictest0-21swapper/107:50:151
2332756993626,6cyclictest0-21swapper/012:20:000
2332769993527,4cyclictest0-21swapper/308:45:263
2332769993525,7cyclictest2386589-21unixbench_singl09:40:243
2332769993525,6cyclictest2392332-21cron10:00:013
233276499351,8cyclictest2347684-21taskset07:50:132
2332760993526,4cyclictest0-21swapper/109:48:101
233275699357,21cyclictest11-21ksoftirqd/009:45:000
2332756993529,4cyclictest2393818-21kworker/u8:3+events_unbound10:13:140
2332756993528,4cyclictest2390424-21idleruntime-cro09:55:010
2332756993528,4cyclictest2381900-21idleruntime-cro09:30:000
2332769993425,4cyclictest0-21swapper/311:55:153
2332760993423,5cyclictest2355731-21kworker/u8:1+events_unbound08:27:551
2332760993422,7cyclictest0-21swapper/107:55:221
2332756993427,5cyclictest2431133-21kworker/u8:0+events_unbound12:38:450
2332756993427,5cyclictest2350651-21kworker/u8:1+events_unbound08:02:500
2332756993421,9cyclictest151rcuc/007:45:120
2332756993421,9cyclictest151rcuc/007:45:110
2332756993413,7cyclictest11-21ksoftirqd/010:00:010
2332760993327,4cyclictest2355731-21kworker/u8:1+events_unbound09:28:081
2332760993325,6cyclictest2438691-21kworker/u8:3+events_unbound12:33:441
2332760993325,4cyclictest0-21swapper/110:00:141
2332760993322,7cyclictest0-21swapper/110:40:251
2332760993322,7cyclictest0-21swapper/110:40:251
2332760993322,7cyclictest0-21swapper/107:20:161
233275699339,12cyclictest131rcu_preempt11:40:000
233275699338,6cyclictest11-21ksoftirqd/007:45:010
2332756993327,4cyclictest2444155-21idleruntime-cro12:30:000
2332756993327,4cyclictest2401675-21kworker/u8:3+events_unbound10:53:210
2332756993326,5cyclictest2388393-21kworker/u8:3+events_unbound09:48:110
2332756993326,4cyclictest2405977-21munin-plugin-st10:40:000
2332756993319,12cyclictest2336514-21kworker/u8:0+events_unbound07:25:000
2332769993226,3cyclictest0-21swapper/309:40:003
2332769993224,4cyclictest0-21swapper/311:00:163
233276099325,5cyclictest131rcu_preempt07:40:131
2332760993222,8cyclictest231ktimers/109:20:151
2332760993222,6cyclictest0-21swapper/111:45:261
2332760993222,6cyclictest0-21swapper/111:45:261
233275699329,5cyclictest131rcu_preempt10:45:000
233275699329,5cyclictest131rcu_preempt10:44:590
233275699323,8cyclictest11-21ksoftirqd/012:10:100
2332756993224,6cyclictest151rcuc/010:55:100
2332756993219,5cyclictest131rcu_preempt09:30:240
2332756993219,5cyclictest131rcu_preempt09:30:240
2332756993210,5cyclictest131rcu_preempt11:20:010
2332769993123,4cyclictest0-21swapper/312:20:243
2332769993123,4cyclictest0-21swapper/310:00:253
2332769993123,4cyclictest0-21swapper/307:30:163
2332769993122,5cyclictest2388232-21unixbench_multi09:45:253
2332760993122,4cyclictest0-21swapper/110:50:191
2332760993121,6cyclictest0-21swapper/111:30:001
233275699313,22cyclictest0-21swapper/011:09:400
2332769993024,4cyclictest2376396-21kworker/u8:0+events_unbound09:13:063
2332769993023,4cyclictest2354173-21kthreadd08:10:013
2332760993023,5cyclictest231ktimers/110:45:121
2332760993018,7cyclictest0-21swapper/108:00:271
233275699309,12cyclictest131rcu_preempt08:55:190
233275699309,11cyclictest131rcu_preempt12:00:190
233275699308,13cyclictest11-21ksoftirqd/009:20:250
233275699307,13cyclictest131rcu_preempt09:15:010
2332756993022,4cyclictest0-21swapper/009:05:010
2332756993011,5cyclictest131rcu_preempt07:30:120
233275699300,16cyclictest2428363-21cron11:45:000
233275699300,16cyclictest2428363-21cron11:45:000
2332769992922,4cyclictest2333175-21kworker/u8:1+events_unbound07:15:033
2332769992922,3cyclictest0-21swapper/311:40:163
2332769992922,3cyclictest0-21swapper/311:40:163
2332769992921,4cyclictest0-21swapper/308:20:243
2332769992919,6cyclictest0-21swapper/311:05:263
2332760992923,4cyclictest2333757-21timerandwakeup07:10:251
2332760992917,7cyclictest0-21swapper/111:35:171
2332760992916,9cyclictest1-21systemd09:08:031
233275699299,12cyclictest131rcu_preempt08:25:210
2332756992923,4cyclictest2381829-21kworker/u8:0+events_unbound10:28:430
2332756992922,4cyclictest0-21swapper/012:30:250
2332756992922,4cyclictest0-21swapper/011:55:000
2332756992921,5cyclictest2412723-21kworker/u8:0+events_unbound11:00:140
2332756992921,5cyclictest0-21swapper/011:30:140
2332756992921,4cyclictest0-21swapper/011:15:010
2332756992910,5cyclictest131rcu_preempt07:30:000
2332769992819,4cyclictest0-21swapper/311:13:263
2332769992817,7cyclictest2419829-21cron11:20:003
2332760992816,7cyclictest204-21dbus-daemon12:38:451
2332760992813,11cyclictest24-21ksoftirqd/107:35:241
233275699289,11cyclictest11-21ksoftirqd/009:35:160
233275699287,11cyclictest131rcu_preempt08:45:020
2332756992821,5cyclictest2401675-21kworker/u8:3+events_unbound10:34:020
2332756992820,6cyclictest2410959-21kworker/u8:2+events_unbound12:08:410
2332756992815,5cyclictest131rcu_preempt09:15:260
2332756992813,5cyclictest2355731-21kworker/u8:1+events_unbound08:37:090
233275699281,24cyclictest2357574-21sh08:20:000
2332769992720,3cyclictest0-21swapper/307:35:273
2332769992719,4cyclictest0-21swapper/310:05:283
2332769992719,4cyclictest0-21swapper/308:15:163
2332760992721,3cyclictest0-21swapper/110:20:011
2332760992720,3cyclictest0-21swapper/109:30:211
2332760992720,3cyclictest0-21swapper/109:30:201
2332760992720,3cyclictest0-21swapper/107:15:041
2332760992719,4cyclictest0-21swapper/110:35:251
2332760992719,4cyclictest0-21swapper/108:20:251
2332760992716,5cyclictest2381829-21kworker/u8:0+events_unbound09:36:261
2332760992713,8cyclictest0-21swapper/108:35:281
2332756992716,3cyclictest131rcu_preempt10:15:160
2332756992710,10cyclictest131rcu_preempt08:05:260
2332769992617,6cyclictest2407834-21latency_hist10:45:013
2332769992617,6cyclictest2407834-21latency_hist10:45:003
2332769992614,8cyclictest2402676-21wc10:30:013
233276099262,4cyclictest131rcu_preempt12:05:121
2332760992619,5cyclictest2421225-21kworker/u8:0+events_unbound11:32:381
2332760992618,5cyclictest0-21swapper/111:15:011
2332760992616,6cyclictest0-21swapper/111:20:251
2332760992616,6cyclictest0-21swapper/108:45:161
2332760992610,8cyclictest131rcu_preempt10:10:241
233275699269,5cyclictest131rcu_preempt08:45:160
233275699268,11cyclictest131rcu_preempt10:05:150
233275699266,11cyclictest11-21ksoftirqd/007:35:250
2332756992619,5cyclictest2428803-21kworker/u8:4+events_unbound11:48:070
2332756992619,5cyclictest2428803-21kworker/u8:4+events_unbound11:48:060
2332756992619,4cyclictest2371473-21kworker/u8:3+events_unbound09:05:530
2332756992618,5cyclictest151rcuc/010:20:120
2332756992615,3cyclictest131rcu_preempt11:25:010
2332756992610,12cyclictest11-21ksoftirqd/008:55:000
2332769992518,4cyclictest0-21swapper/312:05:283
2332769992517,5cyclictest2362529-21timerandwakeup08:30:253
2332769992517,5cyclictest0-21swapper/308:40:153
2332769992517,5cyclictest0-21swapper/308:30:013
2332769992515,6cyclictest0-21swapper/311:45:263
2332769992515,6cyclictest0-21swapper/311:45:253
2332769992513,8cyclictest2447334-21grep12:35:253
233276099254,12cyclictest24-21ksoftirqd/107:30:111
2332760992519,3cyclictest0-21swapper/110:00:011
2332760992518,5cyclictest2431133-21kworker/u8:0+events_unbound12:10:131
2332760992518,3cyclictest0-21swapper/107:25:251
2332760992515,7cyclictest2416214-21auditctl11:10:011
2332760992515,4cyclictest2353842-21kworker/u8:0+events_unbound08:30:151
2332760992511,7cyclictest24-21ksoftirqd/112:00:231
2332760992511,11cyclictest24-21ksoftirqd/109:50:161
2332769992419,3cyclictest2350750-21latency_hist08:00:013
2332769992415,6cyclictest0-21swapper/312:05:013
2332769992415,5cyclictest0-21swapper/309:20:013
2332769992415,5cyclictest0-21swapper/308:50:243
2332769992412,9cyclictest2444418-21sed12:30:013
2332760992417,5cyclictest2410959-21kworker/u8:2+events_unbound11:40:471
2332760992417,5cyclictest2410959-21kworker/u8:2+events_unbound11:40:461
2332760992417,3cyclictest0-21swapper/112:15:231
2332760992415,7cyclictest2366284-21kworker/u8:0+events_unbound08:42:341
2332760992414,7cyclictest24-21ksoftirqd/108:05:201
233276099240,12cyclictest2414641-21latency_hist11:05:021
2332769992315,4cyclictest0-21swapper/310:45:283
2332769992314,6cyclictest0-21swapper/308:35:223
2332769992313,7cyclictest2370268-21/usr/sbin/munin08:55:133
2332760992316,5cyclictest0-21swapper/109:45:001
2332760992315,5cyclictest0-21swapper/111:50:151
2332760992315,4cyclictest0-21swapper/110:25:261
2332760992315,4cyclictest0-21swapper/109:00:241
233276099230,2cyclictest0-21swapper/109:15:001
2332769992216,4cyclictest2337351-21kworker/u8:4-ext4-rsv-conversion07:30:013
2332769992216,3cyclictest0-21swapper/312:10:223
2332769992216,3cyclictest0-21swapper/310:55:003
2332769992216,3cyclictest0-21swapper/310:10:173
2332769992216,3cyclictest0-21swapper/309:25:243
2332769992215,4cyclictest2404501-21/usr/sbin/munin10:35:153
2332769992215,3cyclictest0-21swapper/312:15:143
2332769992214,5cyclictest0-21swapper/308:05:013
2332769992214,4cyclictest0-21swapper/307:45:253
2332769992214,4cyclictest0-21swapper/307:45:253
2332769992213,6cyclictest0-21swapper/312:30:163
2332769992212,6cyclictest0-21swapper/309:20:263
2332769992211,6cyclictest204-21dbus-daemon09:33:083
2332769992211,6cyclictest204-21dbus-daemon09:33:083
2332760992216,4cyclictest2345578-21kworker/u8:2+events_unbound08:18:201
2332760992210,10cyclictest24-21ksoftirqd/112:25:151
2332769992115,3cyclictest0-21swapper/311:35:163
2332769992115,3cyclictest0-21swapper/310:30:223
2332760992115,4cyclictest2410959-21kworker/u8:2+events_unbound11:58:581
233276999209,8cyclictest2348355-21wc07:50:233
2332769992013,3cyclictest0-21swapper/309:03:033
2332769992011,5cyclictest2400312-21/usr/sbin/munin10:20:243
2332769991913,3cyclictest2411501-21fschecks_count10:55:143
2332769991912,5cyclictest391ktimers/307:40:233
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional