You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-01 - 18:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Wed Jul 01, 2026 12:44:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
75332234164,57sleep20-21swapper/207:08:332
73752233163,57sleep10-21swapper/107:06:311
74442232164,55sleep30-21swapper/307:07:263
56862201163,25sleep00-21swapper/007:05:090
2064821720,5sleep0773399cyclictest08:50:170
862921610,4sleep0773399cyclictest08:20:270
774399107104,2cyclictest0-21swapper/208:16:212
233272610,1sleep20-21swapper/212:35:252
176682490,5sleep2774399cyclictest07:30:122
212442480,2sleep10-21swapper/108:50:221
295402470,1sleep10-21swapper/109:10:161
773899381,10cyclictest0-21swapper/107:48:161
773899371,18cyclictest31848-21/usr/sbin/munin08:00:261
773899358,5cyclictest9473-40dnf09:38:141
773899351,9cyclictest18504-21irqstats12:25:181
773899351,9cyclictest0-21swapper/111:00:111
7738993516,3cyclictest1035-21sshd09:20:121
773899351,10cyclictest17662-21ps09:55:221
773899350,1cyclictest0-21swapper/110:50:011
7738993412,3cyclictest10827-21/usr/sbin/munin07:15:231
773899341,11cyclictest652-21smart_sda10:30:251
273052340,8sleep232-21ksoftirqd/209:05:152
273052340,8sleep232-21ksoftirqd/209:05:152
773899331,4cyclictest15559-21df07:25:151
7738993314,3cyclictest15271-21idleruntime-cro12:20:011
773899331,11cyclictest18796-21cpuspeed_turbos10:00:121
773399335,13cyclictest30046-21idleruntime-cro11:40:010
773899319,3cyclictest5751-21smart_sda11:55:251
773899311,3cyclictest3646-21munin-run10:40:011
773899311,3cyclictest24012-21diskstats11:25:131
773899311,10cyclictest0-21swapper/112:20:171
773899311,10cyclictest0-21swapper/107:11:161
773399314,12cyclictest28809-21awk09:10:010
773399314,12cyclictest28809-21awk09:10:010
773899301,4cyclictest19272-21/usr/sbin/munin11:15:131
7738993012,3cyclictest3556-21memory08:10:191
7738993012,3cyclictest10750-21sed08:25:271
773899300,4cyclictest0-21swapper/112:13:341
774399291,10cyclictest0-21swapper/211:53:392
7738992915,2cyclictest0-21swapper/108:24:151
773899291,4cyclictest1495-21ntp_states08:05:231
773899290,17cyclictest0-21swapper/110:51:191
7733992911,16cyclictest4618-21munin-run08:15:010
773399290,13cyclictest21867-21awk12:35:000
774399281,13cyclictest0-21swapper/208:11:022
773899281,3cyclictest30599-21df11:40:131
773399289,16cyclictest11828-21munin-run09:45:010
773399281,14cyclictest17194-21munin-run07:30:010
107232280,4sleep339-21rcuc/308:25:283
773899279,4cyclictest0-21swapper/110:40:081
773899271,4cyclictest13638-21/usr/sbin/munin08:35:201
773899271,4cyclictest10597-21taskset12:06:401
773899271,3cyclictest22761-21df08:55:161
773899270,18cyclictest0-21swapper/110:11:291
7733992710,14cyclictest11-21rcuc/008:20:020
110392270,1sleep30-21swapper/309:40:233
775099265,12cyclictest808-21snmpd10:22:273
774399260,12cyclictest0-21swapper/212:26:512
773899261,11cyclictest0-21swapper/107:21:131
7733992611,8cyclictest0-21swapper/009:55:150
773399261,13cyclictest3042-21awk09:25:010
773399261,13cyclictest12457-21basename11:00:020
775099251,6cyclictest0-21swapper/312:38:083
774399250,1cyclictest0-21swapper/211:29:182
773899251,11cyclictest28799-21diskmemload09:20:021
272792251,2sleep227369-21proc_pri07:50:232
775099241,15cyclictest1040-21/usr/sbin/munin09:20:233
774399240,12cyclictest0-21swapper/209:34:472
773899242,3cyclictest6356-21sort09:30:331
773899241,11cyclictest210412awk12:30:231
773399245,16cyclictest9908-21munin-run07:15:020
299052240,2sleep30-21swapper/310:25:153
172182240,5sleep230-21rcuc/211:10:122
775099237,13cyclictest186471/usr/sbin/munin10:00:143
774399231,12cyclictest0-21swapper/209:49:512
774399230,9cyclictest0-21swapper/207:58:062
773899231,16cyclictest0-21swapper/111:12:411
773899231,11cyclictest0-21swapper/111:32:371
773899231,11cyclictest0-21swapper/111:32:371
96202220,1sleep00-21swapper/012:05:160
775099224,12cyclictest0-21swapper/310:55:283
7750992217,3cyclictest19657-21ls12:30:003
775099221,11cyclictest8816-21wc12:05:013
774399221,3cyclictest25025-21munin-run10:15:012
773899227,3cyclictest7531-21latency12:00:171
773899227,3cyclictest26798-21gltestperf07:50:161
773899226,3cyclictest30271-21chrt07:58:561
773899221,18cyclictest16123-21mailstats11:05:231
773899221,18cyclictest11791-21diskstats08:30:141
773899221,12cyclictest28613-21cut11:35:151
773899221,12cyclictest0-21swapper/110:55:241
773899221,10cyclictest28799-21diskmemload12:36:161
773899221,10cyclictest28799-21diskmemload09:29:091
773399222,11cyclictest23051-21/usr/sbin/munin10:10:240
44812220,5sleep29-21rcu_preempt10:40:142
775099213,4cyclictest0-21swapper/310:34:023
775099211,11cyclictest14877-21/usr/sbin/munin11:05:123
775099210,10cyclictest0-21swapper/311:51:033
774399216,4cyclictest2516-21cut10:35:182
774399211,14cyclictest0-21swapper/210:16:032
773899219,11cyclictest18050-21kworker/u8:210:05:261
773899218,11cyclictest693-21kworker/u8:009:40:241
773899217,3cyclictest808-21snmpd10:17:481
773899215,3cyclictest0-21swapper/107:33:481
773899211,14cyclictest0-21swapper/109:46:081
773899211,12cyclictest528-21fschecks_time11:45:181
773899211,11cyclictest25537-21memory09:00:201
773399211,10cyclictest18412-21needreboot07:30:180
773399210,5cyclictest0-21swapper/011:21:410
114302210,6sleep3775099cyclictest12:10:123
94152200,4sleep3775099cyclictest10:50:233
775099208,4cyclictest0-21swapper/311:47:393
775099204,11cyclictest0-21swapper/310:12:263
775099203,6cyclictest9-21rcu_preempt10:00:003
775099201,8cyclictest0-21swapper/312:06:203
775099201,12cyclictest0-21swapper/310:37:523
775099201,11cyclictest25247-21/usr/sbin/munin10:15:143
774399201,5cyclictest0-21swapper/212:23:302
773899206,3cyclictest0-21swapper/109:06:251
773899206,3cyclictest0-21swapper/109:06:251
773899205,3cyclictest0-21swapper/109:54:161
773899201,8cyclictest0-21swapper/111:20:171
773899201,18cyclictest0-21swapper/110:27:321
773899201,11cyclictest29259-21sshd10:23:101
773899201,11cyclictest19631-21chrt07:35:121
773399207,4cyclictest6621-21fschecks_time10:45:160
773399207,3cyclictest0-21swapper/011:10:140
773399202,10cyclictest17691-21/usr/sbin/munin12:25:280
773399201,6cyclictest0-21swapper/010:52:160
773399201,6cyclictest0-21swapper/009:10:140
773399201,5cyclictest0-21swapper/011:34:460
773399201,5cyclictest0-21swapper/011:34:460
773399201,13cyclictest0-21swapper/007:45:150
773399201,11cyclictest18031-21/usr/sbin/munin08:45:210
773399201,10cyclictest8466-21irqstats09:35:170
773399201,10cyclictest3862-21head09:25:160
64732200,4sleep36458-21munin-run12:00:013
145112200,2sleep081ktimersoftd/012:15:230
775099198,3cyclictest693-21kworker/u8:009:45:133
775099198,3cyclictest21464-21/usr/sbin/munin11:20:203
775099197,3cyclictest0-21swapper/307:47:023
775099196,11cyclictest87350irq/126-enp1s0-07:50:213
775099194,3cyclictest14251-21sort12:15:193
775099191,7cyclictest0-21swapper/309:31:163
775099191,4cyclictest28790-21idleruntime-cro09:10:013
775099191,4cyclictest28790-21idleruntime-cro09:10:013
775099191,14cyclictest0-21swapper/310:46:333
775099191,10cyclictest808-21snmpd08:10:363
775099191,10cyclictest23237-21irqstats08:55:193
775099191,10cyclictest0-21swapper/311:10:113
775099190,9cyclictest0-21swapper/312:20:093
775099190,10cyclictest0-21swapper/307:36:163
774399191,6cyclictest0-21swapper/211:40:552
774399191,4cyclictest0-21swapper/210:54:472
774399191,3cyclictest0-21swapper/210:20:082
7743991912,4cyclictest20410-21pmu-power11:15:222
774399191,16cyclictest0-21swapper/207:15:242
774399191,13cyclictest0-21swapper/207:12:342
774399191,10cyclictest0-21swapper/208:50:262
7738991915,3cyclictest0-21swapper/107:40:131
773899191,16cyclictest0-21swapper/111:54:251
773899191,10cyclictest16018-21cstates08:40:131
773399198,3cyclictest9243-21/usr/sbin/munin08:25:200
773399198,3cyclictest1834-21sshd11:47:100
773399191,9cyclictest6464-21needreboot09:30:330
773399191,6cyclictest0-21swapper/011:51:140
7733991914,3cyclictest22836-21idleruntime-cro10:10:000
773399191,15cyclictest16076-21processes11:05:240
775099188,3cyclictest0-21swapper/311:30:523
775099188,3cyclictest0-21swapper/311:30:523
775099187,2cyclictest41-21ksoftirqd/312:31:453
775099186,5cyclictest0-21swapper/311:40:053
775099185,4cyclictest2440-21cat08:10:023
775099182,5cyclictest19386-21munin-run07:35:013
775099182,14cyclictest14560-21wc08:35:193
775099181,9cyclictest30919-21sshd09:14:113
775099181,7cyclictest21811-21/usr/sbin/munin07:40:133
775099181,7cyclictest20764-21uniq08:50:163
775099181,5cyclictest0-21swapper/309:39:123
7750991814,3cyclictest0-21swapper/308:41:483
775099181,3cyclictest0-21swapper/311:00:173
7750991812,4cyclictest2924-21kworker/u8:107:20:113
7750991812,4cyclictest0-21swapper/310:44:493
7750991812,3cyclictest14594-21df_inode09:50:133
7750991812,3cyclictest10132-21timerwakeupswit07:15:013
775099181,11cyclictest0-21swapper/310:07:143
775099181,10cyclictest12703-21smart_sda08:30:253
775099180,9cyclictest0-21swapper/309:28:263
775099180,8cyclictest0-21swapper/311:15:383
775099180,12cyclictest0-21swapper/309:15:063
774399183,3cyclictest773-21NetworkManager07:22:532
774399181,9cyclictest0-21swapper/209:01:382
774399181,9cyclictest0-21swapper/208:04:062
774399181,8cyclictest32235-21fschecks_count10:30:152
774399181,7cyclictest0-21swapper/209:25:512
774399181,6cyclictest0-21swapper/210:28:272
774399181,5cyclictest11366-21sed10:55:172
774399181,5cyclictest10670-21irqstats09:40:192
774399181,5cyclictest0-21swapper/211:58:002
774399181,5cyclictest0-21swapper/211:21:512
774399181,5cyclictest0-21swapper/211:06:192
7743991812,3cyclictest827-21runrttasks10:02:482
774399181,15cyclictest0-21swapper/209:11:382
774399181,15cyclictest0-21swapper/208:07:362
774399181,11cyclictest0-21swapper/212:14:132
774399181,11cyclictest0-21swapper/209:51:072
773899184,3cyclictest0-21swapper/108:46:521
7738991812,3cyclictest5649-21irqstats08:15:161
773399188,2cyclictest808-21snmpd10:42:310
773399181,9cyclictest808-21snmpd10:02:220
773399181,8cyclictest0-21swapper/012:10:150
773399181,7cyclictest32153-21unixbench_singl08:00:280
773399181,7cyclictest0-21swapper/011:55:480
7733991812,4cyclictest0-21swapper/009:04:520
7733991811,5cyclictest0-21swapper/007:50:270
773399181,10cyclictest22428-21/usr/sbin/munin08:55:270
7733991810,5cyclictest0-21swapper/008:30:150
288282180,4sleep230-21rcuc/211:35:162
182212181,3sleep20-21swapper/209:57:422
148872181,2sleep00-21swapper/007:23:480
775099172,14cyclictest0-21swapper/309:01:523
775099171,4cyclictest0-21swapper/307:15:163
7750991711,4cyclictest0-21swapper/311:28:033
775099171,14cyclictest0-21swapper/308:47:053
775099171,11cyclictest0-21swapper/311:37:453
775099171,10cyclictest0-21swapper/308:20:143
774399178,2cyclictest0-21swapper/210:46:462
774399176,3cyclictest30-21rcuc/208:37:132
774399174,2cyclictest87550irq/128-enp1s0-12:15:142
774399171,6cyclictest8092-21ntp_kernel_pll_08:20:202
774399171,6cyclictest23774-21crond07:45:012
774399171,6cyclictest0-21swapper/212:00:292
774399171,6cyclictest0-21swapper/207:35:212
774399171,4cyclictest0-21swapper/212:06:462
774399171,4cyclictest0-21swapper/211:31:192
774399171,4cyclictest0-21swapper/211:31:192
7743991712,3cyclictest19887-21acpi12:30:122
774399171,14cyclictest0-21swapper/209:16:222
773399174,6cyclictest31374-21sshd10:27:100
773399174,2cyclictest87350irq/126-enp1s0-10:38:230
773399172,9cyclictest0-21swapper/009:51:290
773399171,6cyclictest0-21swapper/012:39:300
773399171,6cyclictest0-21swapper/009:48:050
773399171,3cyclictest0-21swapper/011:04:040
7733991713,3cyclictest0-21swapper/012:03:100
7733991712,3cyclictest29032-21uptime10:20:260
7733991712,3cyclictest21811-21/usr/sbin/munin07:40:160
7733991711,4cyclictest16410-21memory12:20:190
773399171,14cyclictest0-21swapper/011:15:090
773399171,14cyclictest0-21swapper/008:06:520
7733991711,3cyclictest23661-21/usr/sbin/munin11:25:130
7733991711,3cyclictest13816-21cat08:35:140
773399170,16cyclictest0-21swapper/007:55:560
258452171,2sleep20-21swapper/207:48:302
19032170,2sleep20-21swapper/211:48:062
775099161,6cyclictest4836-21/usr/sbin/munin08:15:113
775099161,3cyclictest15636-21tune2fs07:25:163
775099161,12cyclictest28401-21/usr/sbin/munin07:55:243
774399165,3cyclictest18623-21gltestperf08:45:162
774399161,6cyclictest13406-21latency_hist08:35:012
774399161,5cyclictest8484-21kernelversion09:35:182
774399161,5cyclictest0-21swapper/211:00:412
774399161,4cyclictest0-21swapper/210:05:292
7743991612,3cyclictest0-21swapper/208:55:142
774399161,14cyclictest0-21swapper/208:27:432
7743991611,3cyclictest2297-21sendmail_mailqu09:20:242
7743991611,3cyclictest15224-21acpi07:25:142
773399169,4cyclictest31806-21gltestperf09:15:160
773399164,7cyclictest0-21swapper/007:35:140
773399161,6cyclictest10827-21/usr/sbin/munin07:15:230
773399161,6cyclictest0-21swapper/010:16:230
7733991612,3cyclictest0-21swapper/010:32:420
7733991611,3cyclictest30569-21cat11:40:120
7750991510,3cyclictest31531-21cat08:00:213
7743991512,2cyclictest0-21swapper/208:40:082
773399150,14cyclictest0-21swapper/008:41:560
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional