You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-17 - 20:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Fri Apr 17, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1101221680,5sleep3638299cyclictest20:19:563
2922521450,5sleep1638099cyclictest21:20:571
498321380,5sleep0637999cyclictest18:54:090
1404021130,3sleep30-21swapper/316:51:023
6379991037,60cyclictest0-21swapper/019:35:580
2321921010,2sleep00-21swapper/018:36:250
5082940,2sleep30-21swapper/319:27:053
5082940,2sleep30-21swapper/319:27:053
6379999057,27cyclictest0-21swapper/020:20:500
6379998833,5cyclictest7950irq/56-dwc_otg_16:51:110
6379998435,24cyclictest0-21swapper/016:55:570
637999827,13cyclictest0-21swapper/021:05:550
637999811,23cyclictest10673-21cat18:11:020
205482810,2sleep20-21swapper/220:31:022
31312800,2sleep13132-21sh20:10:401
6379997922,11cyclictest7850irq/56-dwc_otg_18:40:590
6379997922,11cyclictest7850irq/56-dwc_otg_18:40:590
6379997918,5cyclictest0-21swapper/019:16:150
637999781,21cyclictest0-21swapper/019:41:130
6379997728,5cyclictest7850irq/56-dwc_otg_21:51:140
6379997728,5cyclictest7850irq/56-dwc_otg_21:51:140
6379997417,17cyclictest12917-21fschecks_time21:01:020
286892720,1sleep20-21swapper/221:59:032
638099715,50cyclictest7850irq/56-dwc_otg_21:51:111
638099715,50cyclictest7850irq/56-dwc_otg_21:51:111
6380997112,40cyclictest25577-21systemd-journal21:29:051
637999711,25cyclictest27928-21modprobe17:30:440
638099681,5cyclictest19544-21ntp_states19:11:071
637999682,47cyclictest0-21swapper/020:05:560
6379996822,4cyclictest7750irq/56-dwc_otg18:24:400
637999671,16cyclictest5715-21ssh21:31:050
279112670,4sleep1221ktimersoftd/121:57:371
638099666,37cyclictest7850irq/56-dwc_otg_19:36:131
637999666,46cyclictest6667-21cut16:31:010
6379996653,7cyclictest0-21swapper/020:25:550
637999662,55cyclictest0-21swapper/017:36:010
638099656,41cyclictest7950irq/56-dwc_otg_19:30:581
638099653,44cyclictest7750irq/56-dwc_otg18:46:451
638099647,44cyclictest27903-21cron17:30:441
638099646,36cyclictest23-21ksoftirqd/118:26:051
638099643,7cyclictest20206-21ssh20:30:571
638099642,38cyclictest14276-21apt19:05:441
6379996444,6cyclictest0-21swapper/017:50:450
638099634,6cyclictest10488-21fschecks_time18:10:591
638099632,49cyclictest7750irq/56-dwc_otg21:00:581
638099631,5cyclictest7850irq/56-dwc_otg_19:30:461
638099631,5cyclictest7850irq/56-dwc_otg_19:30:461
6379996324,22cyclictest0-21swapper/019:50:570
6379996323,10cyclictest0-21swapper/017:15:570
6379996323,10cyclictest0-21swapper/017:15:570
638099629,35cyclictest0-21swapper/121:42:241
638099624,40cyclictest14980-21fschecks_time19:06:021
638099621,6cyclictest7950irq/56-dwc_otg_21:46:001
6380996210,30cyclictest0-21swapper/119:41:101
638099618,41cyclictest14769-21head16:51:111
638099615,40cyclictest7950irq/56-dwc_otg_21:37:151
6379996136,4cyclictest7850irq/56-dwc_otg_18:45:560
638099602,3cyclictest0-21swapper/119:20:451
638099602,39cyclictest0-21swapper/121:11:091
638099602,39cyclictest0-21swapper/121:11:091
638099601,45cyclictest0-21swapper/117:31:011
637999601,29cyclictest11501-21http20:20:450
638099597,33cyclictest0-21swapper/118:35:571
638099594,4cyclictest7750irq/56-dwc_otg20:01:051
638099593,43cyclictest7850irq/56-dwc_otg_19:55:571
638099593,43cyclictest7850irq/56-dwc_otg_19:55:571
638099592,4cyclictest8056-21fw_packets20:16:021
6379995915,23cyclictest28462-21/usr/sbin/munin20:41:130
638099585,41cyclictest7750irq/56-dwc_otg17:36:001
638099584,42cyclictest7750irq/56-dwc_otg18:55:591
6380995815,24cyclictest0-21swapper/119:45:561
6379995816,4cyclictest0-21swapper/020:50:570
638099575,40cyclictest7950irq/56-dwc_otg_20:41:131
638099572,3cyclictest7850irq/56-dwc_otg_21:00:041
638099571,6cyclictest0-21swapper/116:40:421
638099571,4cyclictest0-21swapper/119:20:571
638099571,46cyclictest5154-21ssh18:54:271
6379995722,5cyclictest0-21swapper/016:35:570
638099567,34cyclictest7750irq/56-dwc_otg17:01:071
638099563,4cyclictest7750irq/56-dwc_otg18:40:551
638099563,4cyclictest7750irq/56-dwc_otg18:40:551
637999562,22cyclictest0-21swapper/021:15:550
154892560,3sleep06378-21cyclictest21:42:380
638099554,4cyclictest9450irq/48-DMA09:12:251
638099551,7cyclictest2041-21ssh20:47:121
638099551,5cyclictest6835-21sh21:31:591
637999554,26cyclictest0-21swapper/017:00:560
6379995527,7cyclictest7850irq/56-dwc_otg_18:00:450
6379995527,5cyclictest1789-21proc_pri21:26:090
637999551,23cyclictest0-21swapper/017:10:570
638099547,36cyclictest7950irq/56-dwc_otg_16:40:581
638099542,4cyclictest0-21swapper/120:25:451
638099542,42cyclictest0-21swapper/120:35:561
6380995412,39cyclictest19418-21fschecks_time17:06:001
637999546,18cyclictest17145-21kworker/u8:419:00:510
6379995423,5cyclictest10504-21grep21:36:120
637999541,31cyclictest22678-21latency_hist19:15:450
638099533,4cyclictest0-21swapper/118:33:041
638099531,5cyclictest917-21snmpd20:54:471
638099531,5cyclictest5934-21taskset20:13:131
638099531,5cyclictest18408-21ssh21:06:511
6379995320,7cyclictest0-21swapper/019:45:550
637999531,28cyclictest0-21swapper/019:10:450
638199528,26cyclictest7750irq/56-dwc_otg21:05:552
638099524,2cyclictest0-21swapper/121:15:551
638099522,6cyclictest24601-21aten_rbpower_en17:20:581
638099521,8cyclictest463-21grep17:41:131
6379995229,12cyclictest7950irq/56-dwc_otg_20:31:000
6379995226,7cyclictest25721-21grep17:21:120
637999522,29cyclictest7410-21ssh20:15:450
637999521,21cyclictest0-21swapper/021:11:040
637999521,21cyclictest0-21swapper/021:11:040
6381995125,5cyclictest24713-21idleruntime-cro21:15:452
6381995125,5cyclictest24713-21idleruntime-cro21:15:452
638099513,4cyclictest0-21swapper/117:45:551
638099512,5cyclictest22446-21runrttasks17:15:061
638099512,4cyclictest0-21swapper/117:55:571
638099511,41cyclictest0-21swapper/118:16:461
638099511,3cyclictest7750irq/56-dwc_otg20:30:161
6379995122,7cyclictest18768-21unixbench_multi21:46:140
6379995122,14cyclictest7950irq/56-dwc_otg_18:31:120
6379995119,15cyclictest0-21swapper/020:40:080
637999511,23cyclictest0-21swapper/018:05:560
637999511,23cyclictest0-21swapper/018:05:560
638099505,42cyclictest8438-21wget18:05:581
638099505,42cyclictest8438-21wget18:05:581
638099503,6cyclictest0-21swapper/116:46:101
638099503,6cyclictest0-21swapper/116:46:101
638099502,5cyclictest4230-21runrttasks17:52:171
638099501,7cyclictest6732-21fschecks_time16:31:021
637999505,22cyclictest0-21swapper/019:20:570
637999502,4cyclictest0-21swapper/019:00:450
6379995021,5cyclictest908-21runrttasks20:00:560
638099491,8cyclictest7645-21grep18:01:121
638099491,8cyclictest0-21swapper/116:58:591
638099481,6cyclictest15417-21latency_hist18:25:451
637999486,7cyclictest7738-21unixbench_singl18:01:120
638199477,25cyclictest0-21swapper/216:30:582
6380994710,9cyclictest0-21swapper/117:15:571
6380994710,9cyclictest0-21swapper/117:15:571
6379994725,14cyclictest29112-21/usr/sbin/munin21:21:090
637999472,24cyclictest0-21swapper/019:25:590
637999472,24cyclictest0-21swapper/019:25:590
637999472,22cyclictest0-21swapper/017:06:110
6379994720,14cyclictest0-21swapper/017:50:590
637999471,28cyclictest27103-21unixbench_singl21:56:140
637999471,26cyclictest23631-21cut19:55:560
637999471,26cyclictest23631-21cut19:55:560
637999471,16cyclictest29327-21unixbench_singl17:31:130
6379994623,3cyclictest0-21swapper/020:55:580
6379994620,4cyclictest4037-21latency_hist20:50:450
637999461,24cyclictest11985-21tr18:15:570
638299457,23cyclictest0-21swapper/317:05:563
638299451,4cyclictest7950irq/56-dwc_otg_19:10:553
6379994515,5cyclictest17273-21ls18:30:450
638299446,23cyclictest0-21swapper/318:25:583
638199447,25cyclictest0-21swapper/217:30:582
637999441,15cyclictest11196-21sed16:41:120
59142442,14sleep00-21swapper/016:28:200
111552440,12sleep27950irq/56-dwc_otg_19:01:072
638299437,24cyclictest0-21swapper/321:30:573
638199437,26cyclictest0-21swapper/221:00:582
638199436,7cyclictest31-21ksoftirqd/220:26:122
638199436,23cyclictest0-21swapper/217:25:572
6381994323,12cyclictest29-21rcuc/217:50:562
637999431,22cyclictest0-21swapper/019:31:070
638299427,25cyclictest0-21swapper/321:00:583
6381994212,13cyclictest31-21ksoftirqd/221:45:452
637999421,27cyclictest917-21snmpd17:44:320
638299417,20cyclictest0-21swapper/316:30:583
638299417,19cyclictest0-21swapper/316:55:493
638299416,25cyclictest0-21swapper/316:40:573
6382994126,5cyclictest39-21ksoftirqd/319:56:053
6382994126,5cyclictest39-21ksoftirqd/319:56:053
6382994124,8cyclictest7950irq/56-dwc_otg_20:40:573
6382994110,11cyclictest39-21ksoftirqd/321:21:063
6381994112,5cyclictest0-21swapper/221:25:442
637999412,21cyclictest0-21swapper/016:45:540
637999412,21cyclictest0-21swapper/016:45:540
638299406,19cyclictest0-21swapper/317:25:573
638299401,23cyclictest0-21swapper/317:10:583
638299401,15cyclictest23056-21wget19:16:023
638199401,7cyclictest20058-21cat17:06:092
261112400,1sleep20-21swapper/221:16:112
6382993917,6cyclictest20430-21sh20:31:003
638299391,6cyclictest3528-21apt-get20:10:573
638199392,6cyclictest0-21swapper/219:10:572
638299386,22cyclictest15832-21ssh20:25:453
638299382,14cyclictest0-21swapper/319:52:423
6382993817,5cyclictest29721-21idleruntime-cro17:35:443
638299381,20cyclictest10968-21ls19:40:453
638199387,13cyclictest7850irq/56-dwc_otg_18:46:142
638199387,12cyclictest7850irq/56-dwc_otg_19:35:592
638199385,24cyclictest21800-21ssh18:35:582
638199384,22cyclictest10282-21cut16:41:012
638199381,8cyclictest15897-21grep18:26:022
6381993816,6cyclictest31-21ksoftirqd/221:25:482
6381993814,7cyclictest0-21swapper/220:36:192
6381993811,5cyclictest31-21ksoftirqd/221:00:452
6382993720,6cyclictest1960650ssh21:47:353
6382993714,15cyclictest7850irq/56-dwc_otg_20:40:453
638199379,14cyclictest16622-21grep16:56:122
638199378,12cyclictest7850irq/56-dwc_otg_17:25:442
638199377,20cyclictest0-21swapper/219:55:562
638199377,20cyclictest0-21swapper/219:55:562
638199375,13cyclictest0-21swapper/218:34:042
638199373,25cyclictest0-21swapper/216:50:572
638299367,16cyclictest0-21swapper/318:35:583
638299362,14cyclictest0-21swapper/319:05:253
638299361,9cyclictest37-21rcuc/321:00:453
638299361,26cyclictest9512-21ssh21:36:013
638199369,8cyclictest7750irq/56-dwc_otg19:20:532
638199367,5cyclictest5158-21proc_pri20:51:102
638199365,10cyclictest7750irq/56-dwc_otg17:41:132
6381993621,5cyclictest31-21ksoftirqd/217:46:052
638299355,15cyclictest39-21ksoftirqd/321:06:403
638199359,4cyclictest0-21swapper/218:52:422
638199352,8cyclictest0-21swapper/218:55:562
638199351,5cyclictest14485-21ntp_states18:21:072
45692356,17sleep14608-21/usr/sbin/munin16:25:581
638299347,15cyclictest0-21swapper/317:45:583
638299345,13cyclictest7950irq/56-dwc_otg_20:30:443
638299344,19cyclictest7950irq/56-dwc_otg_18:43:133
638299344,19cyclictest7950irq/56-dwc_otg_18:43:133
638299342,16cyclictest5115-21ntp_states20:51:093
638199349,6cyclictest31-21ksoftirqd/218:16:152
638199349,6cyclictest31-21ksoftirqd/218:11:082
638199349,3cyclictest0-21swapper/220:19:172
638199347,5cyclictest7750irq/56-dwc_otg19:55:452
638199342,16cyclictest29-21rcuc/219:44:292
638199341,27cyclictest31687-21ls17:40:452
6381993412,5cyclictest12635-21/usr/sbin/munin16:46:112
6381993412,5cyclictest12635-21/usr/sbin/munin16:46:112
638199341,24cyclictest0-21swapper/218:44:542
638199341,24cyclictest0-21swapper/218:44:542
638199341,23cyclictest3153-21cut19:31:012
6381993412,13cyclictest14674-21ssh20:23:522
6381993410,4cyclictest7750irq/56-dwc_otg21:35:552
6381993410,18cyclictest0-21swapper/219:09:442
249092341,29sleep324912-21ssh21:54:423
249092341,29sleep324912-21ssh21:54:423
638299338,5cyclictest7750irq/56-dwc_otg19:41:023
638299333,23cyclictest7950irq/56-dwc_otg_21:10:563
638299333,23cyclictest7950irq/56-dwc_otg_21:10:563
638299331,4cyclictest7950irq/56-dwc_otg_18:05:553
638299331,4cyclictest7950irq/56-dwc_otg_18:05:553
6382993311,5cyclictest0-21swapper/321:26:083
638199339,12cyclictest301ktimersoftd/220:04:162
638199338,5cyclictest7850irq/56-dwc_otg_19:26:022
638199338,5cyclictest7850irq/56-dwc_otg_19:26:022
638199336,5cyclictest7850irq/56-dwc_otg_17:16:142
638199336,5cyclictest7850irq/56-dwc_otg_17:16:142
638199336,5cyclictest7850irq/56-dwc_otg_16:36:132
638199335,5cyclictest24151-21grep19:16:152
638199334,6cyclictest31-21ksoftirqd/220:41:062
6381993320,7cyclictest7950irq/56-dwc_otg_20:47:142
6381993316,11cyclictest8565-21ssh21:35:072
638199331,5cyclictest0-21swapper/221:46:532
59482332,24sleep30-21swapper/316:28:483
258492331,4sleep325834-21apt-get21:55:593
638299325,17cyclictest0-21swapper/320:05:573
638299321,9cyclictest0-21swapper/317:01:123
638299321,7cyclictest0-21swapper/317:20:553
638299321,3cyclictest7850irq/56-dwc_otg_21:40:573
6382993212,6cyclictest30468-21aten_rbpower_vo18:46:003
6382993212,4cyclictest16357-21timerandwakeup19:46:113
6382993212,10cyclictest7850irq/56-dwc_otg_17:50:593
6382993211,11cyclictest39-21ksoftirqd/318:35:453
638199325,12cyclictest7850irq/56-dwc_otg_19:45:582
6381993213,4cyclictest29-21rcuc/221:50:572
6381993213,4cyclictest29-21rcuc/221:50:572
6381993212,5cyclictest6740-21cpuspeed18:00:592
177682320,5sleep3638299cyclictest19:09:343
638299319,12cyclictest381ktimersoftd/319:31:513
638299316,15cyclictest0-21swapper/316:35:583
6382993125,4cyclictest39-21ksoftirqd/318:56:123
638299312,14cyclictest7850irq/56-dwc_otg_17:16:113
638299312,14cyclictest7850irq/56-dwc_otg_17:16:113
638299311,2cyclictest0-21swapper/316:46:013
638299311,2cyclictest0-21swapper/316:46:013
6382993112,4cyclictest0-21swapper/318:51:353
6382993110,4cyclictest27381-21fschecks_time19:21:003
638199317,6cyclictest7950irq/56-dwc_otg_20:05:582
6381993111,6cyclictest0-21swapper/220:14:592
638299304,13cyclictest28650-21ssh20:01:113
638299302,7cyclictest7850irq/56-dwc_otg_18:21:123
638299302,15cyclictest0-21swapper/318:00:593
638299301,22cyclictest0-21swapper/317:36:073
638299301,13cyclictest7950irq/56-dwc_otg_18:17:093
638299301,13cyclictest32723-21aten_rbpower_po20:46:003
638199309,5cyclictest7850irq/56-dwc_otg_17:00:562
6381993015,6cyclictest31-21ksoftirqd/218:10:452
6381993015,6cyclictest31-21ksoftirqd/218:10:452
638299297,12cyclictest37-21rcuc/318:11:063
638299295,6cyclictest32205-21if_eth017:41:033
638299291,12cyclictest39-21ksoftirqd/321:20:173
638199292,7cyclictest20939-21awk17:10:552
638199291,2cyclictest0-21swapper/217:55:562
638299281,4cyclictest7950irq/56-dwc_otg_17:55:573
58382273,15sleep20-21swapper/216:27:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional