You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-27 - 21:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Wed May 27, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2438821730,4sleep1425399cyclictest09:58:441
2560021700,5sleep0425299cyclictest11:56:390
1887021160,5sleep0425299cyclictest11:49:310
4252999541,7cyclictest7950irq/56-dwc_otg_10:40:140
4252999121,16cyclictest0-21swapper/008:45:160
60392880,2sleep16033-21apt-config10:55:051
4252998637,5cyclictest7950irq/56-dwc_otg_09:55:340
237292840,4sleep323720-21ssh11:55:173
196172840,8sleep013-21rcuc/009:52:350
4252998234,29cyclictest1133-21runrttasks12:15:140
105492810,2sleep3381ktimersoftd/308:55:253
425299792,44cyclictest0-21swapper/011:40:140
4252997646,23cyclictest0-21swapper/007:55:200
425299752,52cyclictest16541-21grep12:25:210
4253997128,25cyclictest27833-21ntpq11:20:271
4253997128,25cyclictest27833-21ntpq11:20:271
425599701,55cyclictest21592-21ssh09:55:223
4253997029,13cyclictest26933-21ssh09:23:061
4253997029,13cyclictest26933-21ssh09:23:061
4253996929,20cyclictest29528-21ls09:25:331
425599681,36cyclictest8050irq/56-dwc_otg_09:50:053
4253996832,25cyclictest19498-21/usr/sbin/munin11:50:321
4253996832,25cyclictest19498-21/usr/sbin/munin11:50:321
4253996830,11cyclictest10469-21apt-get11:00:161
4253996828,20cyclictest12556-21ssh10:23:301
425599671,44cyclictest21501-21ssh10:34:413
4253996730,18cyclictest12337-21ssh09:44:281
425299672,44cyclictest0-21swapper/009:20:160
425299672,44cyclictest0-21swapper/009:20:160
425299671,43cyclictest0-21swapper/007:15:310
425299671,30cyclictest2919850ssh12:00:320
425599661,43cyclictest7950irq/56-dwc_otg_11:20:333
425599661,43cyclictest7950irq/56-dwc_otg_11:20:333
4253996623,25cyclictest8050irq/56-dwc_otg_11:35:191
4253996623,15cyclictest20146-21chrt10:32:141
425299663,28cyclictest0-21swapper/007:20:150
425599656,41cyclictest8050irq/56-dwc_otg_10:55:103
425599651,51cyclictest22526-21ssh12:32:043
425599651,49cyclictest25564-21ssh11:17:573
4253996534,15cyclictest26566-21/usr/sbin/munin08:10:231
4253996525,16cyclictest27077-21ssh10:01:051
425599643,11cyclictest7950irq/56-dwc_otg_09:40:163
425599641,47cyclictest393-21sh12:05:283
425599641,41cyclictest26040-21/usr/sbin/munin08:10:263
4253996427,17cyclictest15289-21ssh09:47:141
425299641,25cyclictest2940-21ssh11:30:160
4252996411,32cyclictest2512-21grep10:10:310
239252640,5sleep1425399cyclictest11:15:301
425599631,50cyclictest674-21ssh10:09:343
425599631,50cyclictest15887-21ssh12:25:043
425599631,47cyclictest8050irq/56-dwc_otg_10:12:073
425599631,46cyclictest24040-21ssh09:20:063
425599631,46cyclictest24040-21ssh09:20:063
425599631,45cyclictest7029-21ntpq08:45:253
425599631,45cyclictest5392-21ssh10:15:083
425599631,45cyclictest5392-21ssh10:15:083
425599631,42cyclictest14562-21ssh11:44:123
4253996332,12cyclictest28588-21/usr/sbin/munin08:15:281
4253996327,16cyclictest2075-21rm11:29:091
425599621,49cyclictest6254-21ssh09:36:183
425599621,49cyclictest11066-21ssh10:20:463
425599621,46cyclictest0-21swapper/308:00:153
425599621,45cyclictest17655-21ssh12:25:433
425599621,45cyclictest15517-21ssh10:26:203
425599621,44cyclictest4119-21ssh10:51:523
425599621,43cyclictest24438-21ssh10:37:303
4253996236,11cyclictest17414-21df09:50:191
4253996227,13cyclictest17673-21ssh12:25:451
4253996226,22cyclictest8960-21ntp_offset08:50:271
425299625,46cyclictest0-21swapper/009:45:150
4252996234,6cyclictest10772-21timerwakeupswit10:20:320
4252996233,6cyclictest0-21swapper/011:25:170
4252996219,29cyclictest0-21swapper/009:05:150
37172623,12sleep00-21swapper/007:06:450
425599611,51cyclictest9991-21ssh12:16:543
425599611,49cyclictest11503-21/usr/sbin/munin07:30:313
425599611,47cyclictest17794-21ssh09:12:043
425599611,47cyclictest1165-21snmpd07:22:313
425599611,44cyclictest22517-21ssh11:53:363
425599611,44cyclictest22517-21ssh11:53:363
425599611,41cyclictest17774-21ssh11:08:523
425599611,41cyclictest13744-21if_wlan007:35:233
425599611,40cyclictest28485-21ls08:15:253
425599611,40cyclictest0-21swapper/311:35:153
4253996129,8cyclictest6754-21/usr/sbin/munin08:45:221
4253996126,22cyclictest12191-21/usr/sbin/munin07:30:261
4253996124,21cyclictest5127-21ssh11:32:081
4253996122,16cyclictest12889-21ssh11:41:101
4252996129,6cyclictest0-21swapper/012:05:180
4252996124,25cyclictest30701-21fschecks_time10:45:200
4252996124,25cyclictest30701-21fschecks_time10:45:200
425599602,42cyclictest0-21swapper/311:29:103
425599601,44cyclictest24012-21latency_hist09:20:053
425599601,38cyclictest17174-21apt-get09:50:173
4253996031,16cyclictest26501-21/usr/sbin/munin10:40:201
4253996030,21cyclictest1111-21ssh10:48:581
4253996030,21cyclictest1111-21ssh10:48:581
4253996029,20cyclictest553-21sensors_temp12:05:301
4253996025,25cyclictest22510-21taskset12:32:021
4253996022,22cyclictest20945-21ssh11:12:061
4253996021,10cyclictest7950irq/56-dwc_otg_07:35:301
4253996020,21cyclictest15502-21ssh10:26:181
425599595,43cyclictest31251-21ssh09:28:123
425599593,9cyclictest7850irq/56-dwc_otg12:00:143
425599591,48cyclictest17843-21ssh11:47:383
425599591,44cyclictest26920-21ssh10:00:483
425599591,35cyclictest8050irq/56-dwc_otg_09:31:233
4253995930,11cyclictest30080-21ssh12:02:041
4253995929,12cyclictest12022-21ssh12:20:101
4253995928,12cyclictest26967-21ssh11:59:091
4253995925,21cyclictest9117-21seq07:20:511
4253995925,17cyclictest31559-21ssh10:06:431
4253995924,19cyclictest9985-21ssh12:16:531
4253995918,25cyclictest8050irq/56-dwc_otg_09:36:281
425299595,29cyclictest0-21swapper/007:10:170
425599581,41cyclictest4327-21latency_hist08:40:043
425599581,38cyclictest26477-21ssh10:40:193
4253995845,8cyclictest15372-21apt-get11:45:141
4253995829,13cyclictest17966-21ssh11:09:131
4253995827,18cyclictest17777-21ssh09:12:021
4253995821,16cyclictest3322-21runrttasks09:33:381
425299582,38cyclictest26742-21idleruntime-cro11:20:050
425299581,41cyclictest0-21swapper/009:30:330
425299581,40cyclictest3881-21grep08:35:330
425299581,31cyclictest6080-21dpkg10:55:050
425599571,52cyclictest24452-21df_inode08:05:203
425599571,44cyclictest5371-21ssh11:32:343
425599571,38cyclictest22362-21ssh11:14:403
425499575,16cyclictest3793-21sh12:10:172
4253995725,11cyclictest24234-21ssh10:37:081
4253995723,11cyclictest5792-21taskset07:14:351
4253995722,21cyclictest3602-21ssh10:12:181
4252995732,6cyclictest6212-21ssh10:55:050
4252995731,5cyclictest7850irq/56-dwc_otg10:35:050
4252995724,16cyclictest8050irq/56-dwc_otg_08:10:170
425299571,37cyclictest0-21swapper/010:00:260
425599561,41cyclictest32743-21taskset08:26:343
425599561,40cyclictest0-21swapper/310:46:083
425599561,40cyclictest0-21swapper/310:46:083
425599561,40cyclictest0-21swapper/307:27:273
4253995628,11cyclictest27149-21ssh12:37:561
4253995627,19cyclictest32716-21taskset08:26:141
4252995613,4cyclictest15718-21aten_rbpower_po09:10:190
425599556,30cyclictest0-21swapper/312:10:193
425599551,50cyclictest7064-21ls07:15:323
425599551,50cyclictest22322-21cat08:00:053
425599551,50cyclictest19044-21diskstats07:50:213
425599551,39cyclictest11790-21users11:00:333
425599551,37cyclictest26377-21ssh12:36:323
4253995527,12cyclictest30893-21chrt08:20:541
4253995525,11cyclictest6952-21ssh12:13:551
425299553,29cyclictest20564-21sh09:15:250
4252995523,16cyclictest0-21swapper/010:05:290
425299551,38cyclictest24908-21tr08:05:260
425599542,37cyclictest6114-21cron08:45:043
425599541,42cyclictest9033-21ps08:50:293
425599541,36cyclictest0-21swapper/307:43:433
4253995427,12cyclictest23775-21taskset08:02:221
4253995424,10cyclictest18829-21apt-get07:50:161
4253995421,19cyclictest22303-21ssh09:17:271
4253995421,17cyclictest11430-21latency_hist07:30:051
4253995415,14cyclictest25797-21cron08:10:041
4252995426,14cyclictest8050irq/56-dwc_otg_08:30:110
4252995425,5cyclictest11576-21aten_rbpower_en07:30:180
4252995425,14cyclictest7950irq/56-dwc_otg_11:10:150
425299542,38cyclictest0-21swapper/012:33:240
425299542,29cyclictest25070-21sh12:35:240
72052530,14sleep09-21ksoftirqd/009:37:580
425599531,41cyclictest30886-21seq08:20:493
425599531,39cyclictest0-21swapper/309:09:393
4253995324,10cyclictest0-21swapper/110:20:051
4253995324,10cyclictest0-21swapper/110:20:051
4252995328,4cyclictest6310-21proc_pri10:15:280
4252995328,4cyclictest6310-21proc_pri10:15:280
425299532,34cyclictest14894-21ssh11:05:190
425299532,29cyclictest14989-21grep10:25:320
425299532,29cyclictest0-21swapper/009:43:000
425299531,29cyclictest19678-21switchtime07:50:300
425599521,47cyclictest5262-21timerwakeupswit07:10:313
425499528,7cyclictest399050kworker/u8:407:15:162
4253995224,11cyclictest8558-21ssh10:57:121
4253995220,13cyclictest8050irq/56-dwc_otg_08:56:171
425299521,30cyclictest30428-21grep08:20:280
425299521,29cyclictest12005-21grep09:00:190
425299521,26cyclictest10764-21ntp_kernel_pll_08:55:280
425599511,40cyclictest915-21/usr/sbin/munin08:30:283
4253995124,12cyclictest6285-21latency_hist08:45:051
4253995124,11cyclictest16354-21taskset07:41:041
4253995123,17cyclictest18156-21runrttasks07:46:021
425299512,8cyclictest8050irq/56-dwc_otg_08:25:160
425299511,29cyclictest10451-21/usr/sbin/munin11:00:200
425599501,46cyclictest16955-21aten_rbpower_en07:45:183
4253995019,12cyclictest4373-21latency_hist08:40:051
4253995017,12cyclictest7260-21users07:15:341
425299501,30cyclictest10467-21grep07:25:300
425299501,27cyclictest7798-21missed_timers11:35:260
4252995012,6cyclictest4949-21timerwakeupswit12:10:320
425299501,26cyclictest20751-21grep11:50:320
425299501,26cyclictest20751-21grep11:50:320
4254994923,6cyclictest7565-21if_eth011:35:232
4252994916,5cyclictest15351-21sh12:24:060
425599481,32cyclictest8050irq/56-dwc_otg_09:01:193
4253994821,19cyclictest8050irq/56-dwc_otg_09:07:041
4253994821,13cyclictest22160-21taskset07:59:461
4253994818,12cyclictest141-21systemd-journal08:34:171
425299489,17cyclictest0-21swapper/009:25:240
425299482,7cyclictest0-21swapper/007:45:160
4252994822,14cyclictest0-21swapper/008:50:060
425299481,25cyclictest5057-21kernelversion08:40:240
180762480,3sleep218077-21unixbench_singl07:45:322
425499477,25cyclictest0-21swapper/208:55:192
4254994723,5cyclictest24493-21fschecks_time09:20:212
4254994723,5cyclictest24493-21fschecks_time09:20:212
425499463,5cyclictest7950irq/56-dwc_otg_10:15:152
425499463,5cyclictest7950irq/56-dwc_otg_10:15:152
4252994623,5cyclictest28363-21unixbench_singl11:20:330
4252994623,5cyclictest28363-21unixbench_singl11:20:330
425499457,23cyclictest0-21swapper/208:25:192
4253994517,9cyclictest133472chrt09:04:011
425299456,27cyclictest0-21swapper/010:35:170
425299455,7cyclictest7950irq/56-dwc_otg_07:40:150
425499441,37cyclictest8050irq/56-dwc_otg_12:35:062
425499437,23cyclictest0-21swapper/210:05:172
425499433,11cyclictest7950irq/56-dwc_otg_10:50:132
425299431,5cyclictest29000-21unixbench_singl08:15:320
4254994219,14cyclictest0-21swapper/211:45:172
425499421,12cyclictest26969-21timerandwakeup08:10:302
425499415,12cyclictest31-21ksoftirqd/207:35:162
425499412,9cyclictest0-21swapper/210:10:132
4254994119,7cyclictest1885950wget07:50:182
4252994118,14cyclictest0-21swapper/008:03:200
425499401,3cyclictest8050irq/56-dwc_otg_11:30:162
4252994020,12cyclictest0-21swapper/007:36:080
425499394,5cyclictest7850irq/56-dwc_otg08:05:332
4254993923,5cyclictest31-21ksoftirqd/210:45:312
4254993923,5cyclictest31-21ksoftirqd/210:45:312
425499382,27cyclictest31-21ksoftirqd/209:10:162
4254993712,5cyclictest8050irq/56-dwc_otg_11:10:322
425499371,25cyclictest5137-21missed_timers08:40:252
425499371,25cyclictest23859-21ssh11:15:302
425499366,15cyclictest8050irq/56-dwc_otg_12:20:192
425499365,7cyclictest7850irq/56-dwc_otg10:40:142
425499363,5cyclictest21331-21apt-get09:55:172
4254993622,4cyclictest8122-21latency_hist08:50:052
425499356,15cyclictest8050irq/56-dwc_otg_09:25:232
425499355,5cyclictest8050irq/56-dwc_otg_11:05:192
425499353,6cyclictest0-21swapper/207:30:322
425499353,5cyclictest0-21swapper/207:10:322
425499352,4cyclictest60-21kswapd010:35:142
425499351,20cyclictest3447-21latency_hist12:10:052
425499348,7cyclictest0-21swapper/211:25:302
425499345,19cyclictest24535-21ssh12:35:182
425499344,16cyclictest8050irq/56-dwc_otg_07:55:342
4254993411,15cyclictest0-21swapper/211:58:592
24182348,12sleep30-21swapper/307:05:143
425499338,6cyclictest8050irq/56-dwc_otg_10:55:222
425499337,5cyclictest0-21swapper/209:50:172
425499335,13cyclictest8050irq/56-dwc_otg_07:20:312
425499332,6cyclictest7850irq/56-dwc_otg09:45:302
425499332,6cyclictest31-21ksoftirqd/208:50:212
425499331,5cyclictest8445-21idleruntime-cro09:40:042
425499331,3cyclictest0-21swapper/208:35:162
4254993310,6cyclictest0-21swapper/211:50:292
4254993310,6cyclictest0-21swapper/211:50:292
425499329,4cyclictest0-21swapper/209:30:072
425499327,12cyclictest8050irq/56-dwc_otg_12:26:142
425499326,13cyclictest8050irq/56-dwc_otg_10:34:142
425499323,4cyclictest7850irq/56-dwc_otg08:15:212
425499323,13cyclictest7950irq/56-dwc_otg_08:35:042
425499321,5cyclictest16297-21ssh10:27:462
4254993213,11cyclictest14816-21ssh11:44:412
4254993211,4cyclictest0-21swapper/210:04:182
4254993210,3cyclictest7850irq/56-dwc_otg11:20:202
4254993210,3cyclictest7850irq/56-dwc_otg11:20:202
425499316,8cyclictest15264-21diskmemload10:20:232
425499311,8cyclictest22741-21fschecks_time08:00:222
4254993116,4cyclictest11051-21sh09:42:132
425499311,5cyclictest0-21swapper/209:17:442
4254993114,5cyclictest29692-21sh12:01:222
425499311,19cyclictest7850irq/56-dwc_otg12:18:332
425499306,14cyclictest0-21swapper/207:25:192
425499302,4cyclictest10552-21aten_rbpower_en11:00:172
4254993014,5cyclictest14207-21cat09:05:242
425499281,12cyclictest7850irq/56-dwc_otg09:00:232
37842285,14sleep10-21swapper/107:07:311
4254992713,12cyclictest25661-21kworker/dying08:20:182
425499262,18cyclictest7950irq/56-dwc_otg_07:40:582
24122244,14sleep20-21swapper/207:05:092
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional