You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-11 - 02:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Wed Jun 10, 2026 12:45:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
112982181120,30sleep10-21swapper/107:06:351
114182169101,61sleep20-21swapper/207:08:082
114042163108,27sleep30-21swapper/307:07:573
112202156101,28sleep00-21swapper/007:05:330
122112700,2sleep30-21swapper/308:25:123
303792670,1sleep10-21swapper/110:20:191
159362630,3sleep10-21swapper/110:57:431
22552620,2sleep10-21swapper/110:00:131
324952610,1sleep30-21swapper/309:58:223
317712570,1sleep00-21swapper/010:45:220
233642560,8sleep334-21ksoftirqd/309:51:013
233642560,8sleep334-21ksoftirqd/309:51:003
199632550,1sleep20-21swapper/212:12:172
44682530,2chrt0-21swapper/010:25:170
95312520,1sleep10-21swapper/110:29:341
72482520,1sleep00-21swapper/011:39:480
143472520,2sleep00-21swapper/010:32:380
138412520,2sleep00-21swapper/010:55:380
1183899526,35cyclictest18544-21missed_timers12:35:202
299752500,2sleep00-21swapper/009:55:500
262382490,3sleep10-21swapper/110:17:341
163642480,1sleep30-21swapper/311:45:253
1183899471,32cyclictest14296-21/usr/sbin/munin09:45:152
1183899470,27cyclictest29595-21ssh09:32:032
207842460,1sleep10-21swapper/109:25:211
1183899461,29cyclictest25595-21ssh12:16:102
1183899461,27cyclictest4841-21rm09:14:172
1183899461,27cyclictest29447-21/usr/sbin/munin11:55:222
1183899460,30cyclictest0-21swapper/212:31:072
1183899460,27cyclictest31204-21diskmemload11:01:352
1183899460,25cyclictest27833-21/usr/sbin/munin07:45:202
111392460,3sleep334-21ksoftirqd/309:18:493
280282450,1sleep328027-21ssh11:06:163
1183999451,23cyclictest31551-21chrt10:20:533
1183899451,30cyclictest31204-21diskmemload10:19:342
1183899451,29cyclictest19306-21ssh09:25:022
1183899451,28cyclictest16305-21ssh10:58:052
11838994510,25cyclictest10960-21sh09:42:042
11838994510,21cyclictest6290-21sh10:50:222
1183999442,22cyclictest0-21swapper/312:10:113
1183999441,29cyclictest0-21swapper/311:14:333
1183899448,26cyclictest31204-21diskmemload12:25:442
1183899442,23cyclictest2175-21ntp_states11:35:202
1183899441,29cyclictest15774-21ssh09:21:402
1183899441,29cyclictest0-21swapper/211:25:562
1183899441,27cyclictest19313-21ssh10:35:512
1183899441,25cyclictest13734-21sh10:08:292
1183899440,29cyclictest0-21swapper/208:24:482
1183899440,27cyclictest30078-21ssh09:55:562
1183899440,26cyclictest2011-21ssh11:10:562
1183899440,25cyclictest26077-21sh10:40:512
1183899440,25cyclictest26077-21sh10:40:512
1183999437,20cyclictest10865-21sh11:18:023
1183899431,27cyclictest12106-21sh11:19:162
1183899431,25cyclictest30146-21ssh11:08:242
1183899431,24cyclictest555-21rm12:21:492
1183899431,24cyclictest10517-21ssh12:05:202
1183899430,28cyclictest0-21swapper/211:45:272
1183899430,27cyclictest7333-21ssh10:27:232
1183899430,27cyclictest21757-21ssh10:14:502
1183899430,26cyclictest11882-21ssh11:42:412
1183999427,20cyclictest23427-21sh11:27:053
1183899421,26cyclictest31204-21diskmemload10:20:102
1183899421,26cyclictest3088-21ssh09:36:002
1183899421,25cyclictest4070-21rm10:49:322
1183899421,25cyclictest30370-21ssh11:32:172
1183899421,22cyclictest5829-21ssh10:02:212
1183899420,32cyclictest13733-21taskset08:26:072
1183899420,29cyclictest0-21swapper/212:04:162
1183899420,29cyclictest0-21swapper/208:05:422
1183899420,27cyclictest12884-21ssh10:31:102
1183899420,27cyclictest0-21swapper/207:57:562
1183899420,26cyclictest25953-21ssh09:53:352
1183899420,26cyclictest25953-21ssh09:53:352
1183899420,26cyclictest22535-21chrt08:48:592
1183899420,25cyclictest7416-21ssh09:15:242
1183899420,24cyclictest25438-21ssh11:52:322
1183999411,24cyclictest2038-21chrt10:47:303
1183899411,28cyclictest0-21swapper/207:23:572
1183899411,26cyclictest2738-21chrt08:03:002
1183899411,22cyclictest24806-21latency_hist08:55:002
1183899410,31cyclictest9336-21chrt08:17:412
1183899410,31cyclictest15905-21chrt08:31:332
1183899410,26cyclictest26557-21chrt07:42:382
1183899410,25cyclictest17846-21ssh11:23:162
296942400,1sleep00-21swapper/009:05:140
1183999406,21cyclictest31204-21diskmemload09:29:043
1183999404,23cyclictest7555-21kworker/u8:110:25:283
1183999401,25cyclictest21518-21ssh12:37:333
1183899401,25cyclictest6818-21taskset08:10:312
1183899401,25cyclictest6818-21taskset08:10:312
1183899401,25cyclictest26834-21chrt08:59:182
1183899401,23cyclictest16420-21/usr/sbin/munin08:35:182
1183899400,27cyclictest20066-21chrt07:28:172
47882390,2sleep30-21swapper/310:01:193
276082390,2sleep10-21swapper/107:45:181
1183999398,18cyclictest30684-21sh09:33:093
1183999391,8cyclictest17849-21sh10:59:373
63132380,1chrt0-21swapper/111:38:531
1183899381,24cyclictest30866-21taskset09:05:532
1183899380,25cyclictest13666-21chrt07:13:532
1183899380,24cyclictest30796-21chrt07:52:132
1183899380,24cyclictest22101-21chrt07:31:552
273252370,8chrt34-21ksoftirqd/311:54:243
1183999375,19cyclictest0-21swapper/311:04:163
1183899370,24cyclictest28819-21taskset09:02:032
1183899370,23cyclictest24143-21chrt07:35:402
1183899370,22cyclictest0-21swapper/207:19:162
11838993612,21cyclictest19346-21date08:40:162
202932350,1sleep00-21swapper/009:25:160
1183999356,15cyclictest0-21swapper/309:11:063
1183999351,8cyclictest14158-21sh12:08:153
1183999351,7cyclictest7369-21sh12:03:243
25372340,10sleep122-21ksoftirqd/110:48:001
207572340,1sleep00-21swapper/011:49:350
1183999340,8cyclictest11196-21sh09:42:203
1183999340,3cyclictest31204-21diskmemload11:40:253
88932330,2chrt0-21swapper/011:40:200
170012330,9sleep122-21ksoftirqd/108:35:171
1183999331,29cyclictest29581-21grep07:50:153
1183999331,18cyclictest0-21swapper/309:36:533
11839993311,20cyclictest2334-21wc08:00:253
1183999330,4cyclictest5163-21rm12:25:173
1183999330,2cyclictest6828-21ssh10:50:333
174482320,2chrt0-21swapper/111:22:511
1183999322,2cyclictest0-21swapper/312:20:093
1183999321,4cyclictest624-21gdbus07:35:223
1183999321,18cyclictest18576-21ssh10:35:243
11839993211,18cyclictest21590-21which07:30:223
11839993210,19cyclictest30267-21sh11:32:103
11839993110,18cyclictest5034-21cron08:10:003
1183999310,4cyclictest19451-21chrt07:25:223
106612310,2chrt0-21swapper/111:17:511
317522300,2sleep00-21swapper/011:57:060
1183999309,18cyclictest18111-21sed07:25:003
1183999301,26cyclictest13893-21ssh09:20:213
1183999301,26cyclictest10914-21ssh10:30:153
1183999301,25cyclictest14538-21ssh11:20:223
11839993010,17cyclictest9539-21idleruntime-cro08:20:003
1183999300,7cyclictest16859-21ssh12:34:373
1183999300,3cyclictest31204-21diskmemload11:35:173
1183999300,28cyclictest0-21swapper/308:14:003
1183999300,28cyclictest0-21swapper/308:14:003
1183799301,2cyclictest0-21swapper/108:13:461
1183799301,2cyclictest0-21swapper/108:13:461
1183999298,19cyclictest0-21swapper/310:10:133
1183999291,8cyclictest23279-21grep10:15:173
1183999291,3cyclictest2358-21cat12:00:013
1183999291,2cyclictest0-21swapper/312:15:103
1183999290,7cyclictest25582-21sed10:40:253
1183999290,7cyclictest25582-21sed10:40:253
1183999290,5cyclictest236342sleep308:50:163
1183999285,20cyclictest14712-21fschecks_time08:30:133
1183999284,21cyclictest8952-21sh10:05:133
1183999281,24cyclictest28622-21sed09:00:243
1183999277,17cyclictest19503-21grep08:40:183
1183999275,19cyclictest11148-21timerwakeupswit08:20:243
1183999274,19cyclictest30300-21pmu-power09:05:203
1183999273,20cyclictest27992-21pmu-power07:45:233
1183999272,6cyclictest16447-21apt-get08:35:133
1183999272,20cyclictest0-21swapper/308:45:123
1183999271,10cyclictest14797-21fschecks_time09:45:153
11839992710,14cyclictest25431-21hwlatdetect07:40:163
1183799271,24cyclictest0-21swapper/108:17:191
1183799270,25cyclictest0-21swapper/108:05:171
11836992715,7cyclictest14327-21apt-get09:45:110
1183699271,24cyclictest0-21swapper/008:14:020
1183699271,24cyclictest0-21swapper/008:14:020
226122260,2chrt0-21swapper/011:50:190
1183999269,14cyclictest12737-21netstat07:10:183
1183999266,17cyclictest14912-21ntp_kernel_pll_07:15:183
1183999261,22cyclictest31253-21/usr/sbin/munin07:55:213
1183999260,7cyclictest26893-21cron09:00:003
1183699261,1cyclictest0-21swapper/008:05:520
11837992518,5cyclictest22-21ksoftirqd/111:30:001
11836992523,1cyclictest0-21swapper/008:16:460
11836992519,4cyclictest12-21ksoftirqd/011:25:000
11836992517,6cyclictest20481-21cron08:44:590
11836992515,5cyclictest5480-21sh12:25:210
1183699251,1cyclictest0-21swapper/007:36:050
289752240,2sleep00-21swapper/011:30:530
251412240,2sleep0131rcuc/010:16:300
1183799241,21cyclictest0-21swapper/111:04:201
11836992415,3cyclictest26485-21ssh09:54:060
11836992415,3cyclictest26485-21ssh09:54:060
1183799239,11cyclictest11398-21fschecks_time12:30:151
1183799233,13cyclictest31204-21diskmemload12:20:101
11836992316,5cyclictest31204-21diskmemload09:43:170
11836992312,4cyclictest12-21ksoftirqd/007:30:000
172742220,3sleep017273-21grep10:10:230
1183799221,4cyclictest0-21swapper/110:35:261
11837992214,4cyclictest31204-21diskmemload12:05:131
11837992213,6cyclictest560-21dbus-daemon07:15:001
11837992211,3cyclictest22-21ksoftirqd/110:55:001
11837992210,3cyclictest272152ssh11:05:301
1183699226,3cyclictest141rcu_preempt10:40:010
11836992212,7cyclictest10452-21awk08:20:170
234492210,3sleep123455-21needreboot07:35:191
1183799212,14cyclictest763-21ssh09:35:101
1183799212,14cyclictest21624-21ssh11:50:121
11837992116,3cyclictest10506-21latency_hist12:30:011
11837992112,6cyclictest26771-21basename07:45:001
1183799211,10cyclictest29412-21sed09:55:241
11837992110,7cyclictest27572-21grep11:30:181
1183699211,9cyclictest37832sh11:12:420
11836992115,4cyclictest2331-21timerwakeupswit08:00:250
11836992113,6cyclictest14771-21ls07:15:170
1183699211,11cyclictest2574-21fschecks_count10:00:150
253302200,2chrt0-21swapper/007:40:150
1183799202,14cyclictest14345-21ssh09:45:111
1183799202,13cyclictest15663-21ssh10:10:101
1183799201,8cyclictest640-21snmpd11:57:301
1183799201,4cyclictest0-21swapper/112:14:231
11837992013,4cyclictest29425-21sh12:20:001
11837992011,6cyclictest14673-21gdbus11:45:011
11836992016,3cyclictest7555-21kworker/u8:110:06:000
11836992012,6cyclictest27559-21seq11:05:490
11836992012,6cyclictest27443-21perl07:45:160
11836992012,6cyclictest21913-21seq08:45:240
1183699201,16cyclictest31204-21diskmemload12:35:250
11836992011,6cyclictest29840-21cat07:50:170
11836992011,6cyclictest19218-21gltestperf11:00:160
11836992010,9cyclictest24257-21ssh11:27:550
11836992010,7cyclictest16988-21cat07:20:180
11836992010,6cyclictest23084-21apt-get12:15:110
1183799193,14cyclictest20206-21cron07:30:001
1183799193,12cyclictest31204-21diskmemload09:30:111
1183799192,6cyclictest32157-21apt-get11:10:111
1183799192,12cyclictest14300-21apt-get08:30:111
11837991916,2cyclictest0-21swapper/108:21:251
11837991912,5cyclictest14070-21basename08:30:001
11837991912,4cyclictest18967-21mailstats12:35:231
11837991911,6cyclictest31265-21wc09:10:001
11837991911,6cyclictest24795-21grep08:55:001
11837991911,6cyclictest16098-21seq10:34:231
11837991911,5cyclictest52750irq/139-enp3s0-12:00:231
11837991911,5cyclictest15538-21sh09:21:261
1183699198,7cyclictest4946-21apt-get10:50:120
1183699192,12cyclictest23922-21ssh10:40:120
1183699192,12cyclictest23922-21ssh10:40:110
1183699192,11cyclictest22876-21apt-get08:50:100
1183699192,11cyclictest12160-21apt-get08:25:110
1183699191,8cyclictest31204-21diskmemload09:11:550
1183699191,7cyclictest31204-21diskmemload12:23:250
1183699191,6cyclictest2214-21timerandwakeup09:35:230
1183699191,5cyclictest31204-21diskmemload12:05:010
11836991914,3cyclictest7217-21rt-features09:15:210
11836991914,3cyclictest31924-21interrupts07:55:160
11836991913,4cyclictest29573-21sh09:32:020
11836991912,5cyclictest18021-21seq08:36:130
11836991911,6cyclictest19662-21seq12:11:580
11836991911,6cyclictest17369-21date12:35:000
11836991911,5cyclictest22176-21seq07:32:530
11836991910,6cyclictest27475-21cat09:00:130
1183799189,6cyclictest28612-21grep09:00:241
1183799188,7cyclictest20481-21cron08:45:011
1183799188,7cyclictest0-21swapper/109:15:111
1183799181,8cyclictest8858-21awk09:40:221
11837991812,4cyclictest2925-21idleruntime-cro08:05:001
11837991812,3cyclictest729-21cron08:00:001
11837991811,5cyclictest9968-21ntp_kernel_pll_10:05:211
11837991811,5cyclictest26967-21sed09:00:001
11837991810,5cyclictest31204-21diskmemload09:10:181
11837991810,5cyclictest0-21swapper/109:50:141
11837991810,5cyclictest0-21swapper/109:50:141
11836991813,3cyclictest6364-21rm12:02:230
11836991811,5cyclictest30340-21ntpq10:20:180
11836991811,5cyclictest25654-21interrupts08:55:160
11836991811,5cyclictest14489-21expr08:30:110
11836991810,6cyclictest7037-21tune2fs11:15:150
11836991810,6cyclictest13356-21sed07:10:240
11836991810,5cyclictest14032-21sendmail_mailqu09:20:210
1183799179,6cyclictest22-21ksoftirqd/107:35:001
1183799178,6cyclictest16174-21acpi07:20:111
1183799172,11cyclictest23904-21apt-get10:40:131
1183799172,11cyclictest23904-21apt-get10:40:121
11837991710,5cyclictest31109-21sed07:55:011
11837991710,5cyclictest22-21ksoftirqd/111:40:121
1183799161,2cyclictest0-21swapper/108:46:051
1183799161,13cyclictest14159-21cut07:15:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional