You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-17 - 14:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Fri Apr 17, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
208722570,2sleep30-21swapper/304:26:023
208722570,2sleep30-21swapper/304:26:023
826021490,4sleep0367999cyclictest07:38:070
36799910426,53cyclictest0-21swapper/007:05:570
719021020,25sleep27950irq/56-dwc_otg_04:40:442
1428921020,2sleep00-21swapper/008:23:040
865921010,6sleep3369899cyclictest06:11:103
2829821000,2sleep211-21rcu_preempt09:17:052
3679999629,56cyclictest17481-21kernelversion07:11:050
80242940,2sleep30-21swapper/304:41:063
3679999428,50cyclictest805-21fschecks_time05:51:020
3679999331,43cyclictest7618-21awk06:10:440
3679999329,49cyclictest20708-21ssh07:52:310
3679999326,56cyclictest4185-21iwlist04:31:020
3679999326,48cyclictest31017-21fschecks_count09:20:590
3679999326,48cyclictest31017-21fschecks_count09:20:590
3679999232,37cyclictest13591-21/usr/sbin/munin04:56:070
3679999225,55cyclictest357-21ssh08:06:210
3679999224,52cyclictest26372-21latency_hist09:15:460
3679999224,52cyclictest26372-21latency_hist09:15:460
3679999126,51cyclictest23577-21latency_hist06:40:450
3679998925,47cyclictest3038-21taskset08:47:480
3679998924,48cyclictest0-21swapper/006:02:040
3679998824,43cyclictest0-21swapper/008:25:580
231142880,1sleep00-21swapper/009:50:450
3679998729,41cyclictest22024-21cut05:20:590
3679998728,44cyclictest26485-21rm08:37:030
3679998726,46cyclictest21353-21/usr/sbin/munin08:31:100
3679998724,48cyclictest0-21swapper/008:56:020
3679998724,48cyclictest0-21swapper/006:41:030
3679998625,42cyclictest0-21swapper/006:16:110
3679998624,48cyclictest0-21swapper/005:26:090
3679998525,47cyclictest0-21swapper/009:59:510
3679998423,46cyclictest0-21swapper/008:42:120
3679998421,48cyclictest1373-21latency_hist07:30:450
42462830,1sleep10-21swapper/108:49:551
3679998330,45cyclictest9612-21ssh08:17:210
3679998323,43cyclictest0-21swapper/007:16:380
3679998224,42cyclictest1467-21ssh06:51:460
3679998223,47cyclictest0-21swapper/006:56:130
3679998223,47cyclictest0-21swapper/005:39:590
3679998134,31cyclictest26243-21ssh09:54:140
3679998126,12cyclictest15116-21ssh09:01:320
3679998126,12cyclictest15116-21ssh09:01:320
3679998125,44cyclictest17381-21ssh09:43:100
3679998125,44cyclictest17381-21ssh09:43:100
3679998125,42cyclictest8472-21ssh09:32:040
3679998123,45cyclictest0-21swapper/009:25:590
3679998122,45cyclictest0-21swapper/005:45:160
3679998032,38cyclictest31151-21ssh06:49:240
3679998026,40cyclictest0-21swapper/008:01:110
3679998024,44cyclictest7765-21chrt08:53:360
3679998024,44cyclictest0-21swapper/007:22:100
3679998022,50cyclictest0-21swapper/006:32:470
3679997926,31cyclictest9586-21grep07:01:120
3679997926,31cyclictest14761-21cut09:40:450
3679997923,48cyclictest12658-21taskset06:21:230
3679997923,45cyclictest0-21swapper/007:47:200
3679997922,47cyclictest0-21swapper/006:29:400
367999786,51cyclictest0-21swapper/007:30:580
3679997828,47cyclictest11599-21ssh07:41:310
3679997824,40cyclictest0-21swapper/008:12:130
3679997731,28cyclictest21276-21sh09:09:520
3679997727,12cyclictest28945-21chrt09:18:160
3679997624,40cyclictest0-21swapper/004:36:550
3679997624,38cyclictest0-21swapper/005:09:560
203922760,2sleep20-21swapper/207:51:582
3679997524,48cyclictest0-21swapper/004:42:050
207642750,5sleep1368799cyclictest09:46:471
3687997422,34cyclictest14073-21latency_hist07:45:451
3679997428,30cyclictest0-21swapper/007:58:030
3679997428,30cyclictest0-21swapper/007:58:030
3679997224,33cyclictest21443-21taskset05:17:540
3679997223,34cyclictest26952-21chrt05:31:420
234962720,4sleep223487-21apt-get05:25:452
3679997123,35cyclictest0-21swapper/004:53:030
144432710,5sleep114447-21modprobe05:00:441
3679997024,33cyclictest0-21swapper/005:12:230
3679996924,29cyclictest16121-21chrt05:04:050
3679996924,29cyclictest10800-21chrt04:50:200
368799679,49cyclictest0-21swapper/107:49:111
368999665,5cyclictest7750irq/56-dwc_otg09:21:092
368999665,5cyclictest7750irq/56-dwc_otg09:21:092
368799661,11cyclictest12308-21cron07:05:451
3679996622,31cyclictest0-21swapper/005:59:190
368799655,5cyclictest10489-21wc08:56:121
368799651,7cyclictest1607-21sed05:51:131
3679996520,30cyclictest0-21swapper/006:13:060
3679996433,29cyclictest31158-21dpkg05:45:540
368999632,6cyclictest9956-21fschecks_time06:16:012
368799623,4cyclictest7750irq/56-dwc_otg08:01:001
368799622,4cyclictest0-21swapper/105:15:551
368999615,6cyclictest4800-21chrt04:31:102
368999615,5cyclictest0-21swapper/207:25:562
368999614,5cyclictest7850irq/56-dwc_otg_05:51:132
368999614,3cyclictest7950irq/56-dwc_otg_07:30:572
368999613,5cyclictest7850irq/56-dwc_otg_04:56:012
368799615,6cyclictest0-21swapper/107:25:561
368999606,10cyclictest0-21swapper/208:19:132
368999601,11cyclictest7950irq/56-dwc_otg_05:41:042
3689996010,7cyclictest21265-21aten_rbpower_en07:15:582
368799601,6cyclictest31699-21switchtime09:21:081
368799601,6cyclictest31699-21switchtime09:21:081
369899593,39cyclictest7950irq/56-dwc_otg_08:40:583
3689995916,40cyclictest16817-21fschecks_time05:06:032
368999591,5cyclictest0-21swapper/206:51:372
368999591,4cyclictest19545-21ssh09:06:512
368799593,47cyclictest0-21swapper/109:54:271
368799592,4cyclictest0-21swapper/107:17:071
368799591,15cyclictest17461-21ssh09:43:191
368799591,15cyclictest17461-21ssh09:43:191
368799591,13cyclictest0-21swapper/108:31:191
369899585,39cyclictest7950irq/56-dwc_otg_08:45:583
368999586,4cyclictest7950irq/56-dwc_otg_09:46:082
368999585,10cyclictest7950irq/56-dwc_otg_08:26:102
368999583,9cyclictest7950irq/56-dwc_otg_08:56:132
368799583,4cyclictest7850irq/56-dwc_otg_05:21:101
368799581,5cyclictest7950irq/56-dwc_otg_07:36:081
3687995810,45cyclictest1947950wget06:35:581
170242580,4sleep317018-21ssh08:26:093
368999573,4cyclictest14833-21diskmemload09:42:222
368999573,4cyclictest14833-21diskmemload09:42:222
368999572,4cyclictest0-21swapper/208:07:472
368799572,4cyclictest1551-21apt-get07:30:561
368799572,3cyclictest0-21swapper/106:48:491
3687995718,37cyclictest7079-21fschecks_time09:31:031
3687995717,31cyclictest1561-21sh06:51:571
368799571,4cyclictest7950irq/56-dwc_otg_07:08:331
368799571,3cyclictest7750irq/56-dwc_otg06:44:591
368999567,7cyclictest7750irq/56-dwc_otg07:56:052
368999567,7cyclictest7750irq/56-dwc_otg07:56:052
368999563,5cyclictest0-21swapper/206:49:012
368999561,5cyclictest25577-21systemd-journal08:52:162
3689995611,38cyclictest27616-21ssh07:22:392
368799566,4cyclictest7950irq/56-dwc_otg_09:15:581
368799566,4cyclictest7750irq/56-dwc_otg08:51:031
368799562,5cyclictest0-21swapper/104:30:501
368799562,4cyclictest9450irq/48-DMA09:12:251
368799561,3cyclictest0-21swapper/109:13:461
368799561,3cyclictest0-21swapper/109:13:461
368799561,3cyclictest0-21swapper/108:20:581
3687995610,4cyclictest0-21swapper/108:11:121
368999555,8cyclictest7950irq/56-dwc_otg_05:46:012
368999551,5cyclictest0-21swapper/207:40:092
368799552,9cyclictest7950irq/56-dwc_otg_06:35:451
368799551,3cyclictest0-21swapper/104:41:311
368799551,10cyclictest30965-21ssh10:00:181
368999545,4cyclictest0-21swapper/209:15:142
368999545,4cyclictest0-21swapper/209:15:142
368999544,3cyclictest0-21swapper/207:48:342
368999543,5cyclictest12431-21cat07:05:452
368999543,3cyclictest7950irq/56-dwc_otg_08:41:522
368999541,5cyclictest27981-21ssh08:01:082
3689995412,5cyclictest24823-21aten_rbpower_vo08:35:592
368799542,3cyclictest7950irq/56-dwc_otg_07:13:521
368799542,14cyclictest10039-21if_err_wlan006:16:021
368799542,11cyclictest7950irq/56-dwc_otg_08:25:491
368799541,46cyclictest0-21swapper/107:25:361
368799541,3cyclictest0-21swapper/107:54:261
3687995411,7cyclictest24826-21aten_rbpower_vo08:35:591
368799541,10cyclictest30915-21latency_hist05:45:451
368999536,5cyclictest7850irq/56-dwc_otg_08:23:052
368999533,4cyclictest7850irq/56-dwc_otg_06:21:072
368999531,6cyclictest13244-21latency07:06:052
368999531,3cyclictest0-21swapper/206:41:042
3689995313,4cyclictest29180-21latency_hist05:40:442
368799539,5cyclictest0-21swapper/105:10:581
368799535,8cyclictest7950irq/56-dwc_otg_08:42:231
368799532,6cyclictest25358-21ssh07:58:131
368799532,6cyclictest25358-21ssh07:58:131
3687995316,35cyclictest15192-21ls05:01:041
368799531,13cyclictest7950irq/56-dwc_otg_09:37:481
368799531,11cyclictest0-21swapper/108:20:171
368799531,11cyclictest0-21swapper/104:46:001
368799531,10cyclictest0-21swapper/108:06:331
368999523,6cyclictest7850irq/56-dwc_otg_09:01:102
368999523,6cyclictest7850irq/56-dwc_otg_09:01:102
368999521,5cyclictest7750irq/56-dwc_otg07:42:002
3689995214,4cyclictest6362-21latency_hist09:30:452
3689995211,4cyclictest19346-21sed05:11:112
3689995210,5cyclictest0-21swapper/208:30:562
368799525,9cyclictest7950irq/56-dwc_otg_05:40:091
368799523,5cyclictest6370-21munin-run09:30:441
368799521,8cyclictest26041-21grep05:30:591
3687995217,32cyclictest8038-21cpuspeed06:11:001
207852520,2sleep337-21rcuc/309:09:013
369899516,14cyclictest7950irq/56-dwc_otg_07:50:573
368999513,4cyclictest7850irq/56-dwc_otg_06:35:432
368999512,5cyclictest0-21swapper/206:07:442
368999512,4cyclictest7850irq/56-dwc_otg_09:54:152
368999511,4cyclictest0-21swapper/207:11:152
368999511,3cyclictest7850irq/56-dwc_otg_08:14:482
368999511,3cyclictest0-21swapper/206:59:492
368799513,4cyclictest7950irq/56-dwc_otg_06:27:051
368799513,11cyclictest7950irq/56-dwc_otg_09:01:441
368799513,11cyclictest7950irq/56-dwc_otg_09:01:441
368799511,5cyclictest31495-21perl05:46:011
368799511,4cyclictest0-21swapper/104:36:001
3687995111,5cyclictest6801-21cut06:06:071
368799511,13cyclictest6161-21ssh06:57:281
368999504,6cyclictest0-21swapper/205:56:132
368999504,4cyclictest7850irq/56-dwc_otg_06:36:092
368999502,8cyclictest0-21swapper/210:00:202
368999501,4cyclictest7950irq/56-dwc_otg_09:37:502
368999501,4cyclictest7950irq/56-dwc_otg_08:47:552
368999501,3cyclictest0-21swapper/209:33:452
368799502,45cyclictest0-21swapper/105:06:031
368999493,5cyclictest7850irq/56-dwc_otg_04:46:002
368999491,4cyclictest11074-21/usr/sbin/munin04:51:042
368799495,7cyclictest7950irq/56-dwc_otg_06:05:201
368999483,3cyclictest7950irq/56-dwc_otg_04:42:342
368999482,6cyclictest0-21swapper/205:26:062
368999481,4cyclictest0-21swapper/205:15:552
3689994811,5cyclictest145012chrt06:26:512
368799482,3cyclictest7950irq/56-dwc_otg_06:21:071
368799481,9cyclictest7750irq/56-dwc_otg05:26:201
369899477,28cyclictest0-21swapper/304:45:493
369899477,26cyclictest0-21swapper/306:05:503
369899467,25cyclictest0-21swapper/305:45:573
369899467,23cyclictest0-21swapper/306:25:593
369899461,12cyclictest5757-21cpuspeed04:35:583
368999461,5cyclictest0-21swapper/205:32:132
368799465,4cyclictest7750irq/56-dwc_otg04:51:111
3679994631,13cyclictest3673-21cyclictest04:30:460
369899457,25cyclictest0-21swapper/306:15:573
369899455,6cyclictest37-21rcuc/309:40:453
369899447,23cyclictest0-21swapper/309:10:503
369899447,23cyclictest0-21swapper/309:10:503
369899444,7cyclictest0-21swapper/309:15:563
368799441,5cyclictest3585-21taskset05:57:031
29742443,35sleep00-21swapper/004:26:240
369899438,19cyclictest37-21rcuc/305:20:593
369899437,24cyclictest0-21swapper/305:25:523
369899437,22cyclictest0-21swapper/305:15:573
369899436,24cyclictest0-21swapper/307:20:583
369899427,25cyclictest37-21rcuc/307:19:473
369899427,23cyclictest0-21swapper/305:30:553
369899426,25cyclictest0-21swapper/307:45:593
369899421,24cyclictest0-21swapper/309:45:583
369899412,30cyclictest0-21swapper/308:00:033
369899412,30cyclictest0-21swapper/308:00:033
3698994112,18cyclictest0-21swapper/308:03:013
369899407,23cyclictest0-21swapper/304:55:573
369899406,16cyclictest7850irq/56-dwc_otg_08:14:383
369899405,20cyclictest4126-21ssh06:55:573
368999407,6cyclictest25577-21systemd-journal06:01:072
3689994015,12cyclictest14685-21acpi05:00:552
369899397,5cyclictest39-21ksoftirqd/309:21:003
369899397,5cyclictest39-21ksoftirqd/309:21:003
369899397,19cyclictest39-21ksoftirqd/307:36:013
369899397,13cyclictest7850irq/56-dwc_otg_07:06:093
369899396,23cyclictest0-21swapper/309:25:583
369899396,17cyclictest37-21rcuc/306:40:483
369899396,15cyclictest37-21rcuc/308:09:313
3698993914,12cyclictest381ktimersoftd/308:36:463
369899389,18cyclictest381ktimersoftd/306:49:233
369899388,18cyclictest37-21rcuc/307:44:153
369899387,21cyclictest0-21swapper/305:00:583
3698993813,11cyclictest381ktimersoftd/308:20:113
3698993810,16cyclictest39-21ksoftirqd/308:31:103
3698993719,9cyclictest7950irq/56-dwc_otg_07:33:173
3698993718,5cyclictest908-21runrttasks09:31:143
3698993717,5cyclictest30854-21sh10:00:063
3698993714,8cyclictest17668-21unixbench_singl05:06:133
3698993712,5cyclictest39-21ksoftirqd/309:04:243
3698993712,5cyclictest39-21ksoftirqd/309:04:243
369899368,17cyclictest37-21rcuc/304:34:113
369899367,16cyclictest37-21rcuc/306:54:353
369899362,12cyclictest39-21ksoftirqd/305:51:063
3698993614,5cyclictest39-21ksoftirqd/308:50:543
369899357,14cyclictest39-21ksoftirqd/309:43:143
369899357,14cyclictest39-21ksoftirqd/309:43:143
369899352,7cyclictest12817-21zcat08:21:093
369899352,2cyclictest0-21swapper/305:35:543
369899352,15cyclictest37-21rcuc/306:24:323
3698993516,4cyclictest381ktimersoftd/307:11:053
115722350,2sleep30-21swapper/304:51:043
369899346,8cyclictest39-21ksoftirqd/307:01:033
369899346,8cyclictest381ktimersoftd/309:51:133
369899345,15cyclictest39-21ksoftirqd/306:38:003
369899345,14cyclictest7850irq/56-dwc_otg_07:28:063
369899341,7cyclictest0-21swapper/306:02:283
368999341,11cyclictest0-21swapper/206:15:202
369899337,7cyclictest37-21rcuc/308:56:043
369899324,14cyclictest381ktimersoftd/306:32:283
369899301,5cyclictest7850irq/56-dwc_otg_05:15:443
369899282,5cyclictest0-21swapper/305:55:593
3698992816,5cyclictest29675-21fschecks_time05:41:023
33132282,17sleep10-21swapper/104:30:141
33132282,17sleep10-21swapper/104:30:141
29662253,14sleep20-21swapper/204:26:182
29662253,14sleep20-21swapper/204:26:182
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional