You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-21 - 22:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Thu May 21, 2026 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
102219910032,50cyclictest101ktimersoftd/007:35:100
1022199986,72cyclictest0-21swapper/007:25:140
93402900,2sleep30-21swapper/310:50:513
64752890,7sleep3381ktimersoftd/311:26:503
64752890,7sleep3381ktimersoftd/311:26:503
1022199864,69cyclictest0-21swapper/011:50:070
160862840,8sleep2301ktimersoftd/207:25:192
10221998428,26cyclictest0-21swapper/007:15:150
1022199826,51cyclictest30108-21runrttasks08:05:140
10221997728,35cyclictest0-21swapper/007:10:040
10221997611,50cyclictest0-21swapper/008:30:010
84452750,5sleep11022299cyclictest09:32:181
52232730,4sleep11022299cyclictest10:45:541
1022499731,52cyclictest32692-21/usr/sbin/munin11:20:263
1022199731,55cyclictest19921-21ssh11:05:010
63362710,1sleep00-21swapper/011:26:350
63362710,1sleep00-21swapper/011:26:350
273832710,6sleep01022199cyclictest11:13:350
10332710,5sleep21022399cyclictest09:24:182
10221996956,5cyclictest101ktimersoftd/010:15:290
1022199682,45cyclictest0-21swapper/011:55:140
10221996814,21cyclictest0-21swapper/010:55:260
10221996814,21cyclictest0-21swapper/010:55:260
1022499671,51cyclictest5467-21ssh12:05:173
1022199671,57cyclictest27855-21tr09:55:230
1022199671,57cyclictest27855-21tr09:55:230
243322660,1sleep30-21swapper/312:27:413
147512660,2sleep30-21swapper/308:50:263
1022499661,36cyclictest5046-21ssh10:45:343
10221996651,4cyclictest11599-21ssh10:55:000
1022499652,52cyclictest25627-21sed07:50:283
1022499651,54cyclictest6699-21ssh10:09:563
1022499651,51cyclictest28911-21ssh09:56:233
1022499651,51cyclictest28911-21ssh09:56:233
1022499651,45cyclictest21965-21ssh11:06:133
1022499651,40cyclictest7109-21ssh10:10:143
1022499641,44cyclictest8501-21apt-get08:35:123
10222996412,29cyclictest23-21ksoftirqd/111:20:141
1022199642,54cyclictest0-21swapper/008:00:020
1022499636,42cyclictest8050irq/56-dwc_otg_10:55:243
1022499636,42cyclictest8050irq/56-dwc_otg_10:55:243
1022499633,54cyclictest10226-21runrttasks07:10:043
1022499631,42cyclictest3537-21wc10:45:023
1022499631,40cyclictest10635-21sed12:10:303
1022199631,31cyclictest0-21swapper/011:45:140
244032620,2sleep20-21swapper/211:49:202
1022499626,38cyclictest7950irq/56-dwc_otg_09:45:203
1022499623,39cyclictest8050irq/56-dwc_otg_09:15:193
1022499621,50cyclictest31466-21ssh09:20:473
1022499621,47cyclictest31041-21tail12:35:273
1022499621,47cyclictest0-21swapper/311:53:113
1022499621,43cyclictest25775-21rm09:53:143
1022499621,43cyclictest10307-21apt-get08:40:133
1022399622,25cyclictest25172-21ssh11:50:072
10223996213,18cyclictest7950irq/56-dwc_otg_07:15:152
1022199623,48cyclictest0-21swapper/009:20:200
1022499611,47cyclictest17004-21ssh10:21:003
1022499611,45cyclictest26858-21/usr/sbin/munin12:30:263
1022499611,41cyclictest10764-21ssh11:32:063
10221996143,10cyclictest101ktimersoftd/007:21:090
10221996136,14cyclictest101ktimersoftd/012:30:300
319472600,2sleep00-21swapper/010:00:240
1022499602,40cyclictest21095-21idleruntime-cro09:10:023
1022499602,40cyclictest21095-21idleruntime-cro09:10:023
1022499601,43cyclictest26926-21if_wlan007:55:213
1022499601,42cyclictest4753-21ssh12:04:473
10222996040,15cyclictest10220-21cyclictest07:10:041
10221996041,13cyclictest101ktimersoftd/009:50:240
1022499594,25cyclictest20992-21ssh11:45:143
1022499591,48cyclictest31387-21ssh11:18:193
1022499591,43cyclictest15020-21ssh12:15:493
1022499591,42cyclictest23255-21ssh10:29:523
1022499591,42cyclictest23255-21ssh10:29:523
1022499591,42cyclictest19204-21ssh11:03:453
1022499591,41cyclictest26223-21ssh10:32:433
1022499591,40cyclictest21097-21diskmemload09:26:333
10221995927,15cyclictest8050irq/56-dwc_otg_10:45:170
1022499582,52cyclictest15923-21apt-get08:55:123
1022499581,42cyclictest22599-21taskset07:44:463
1022499581,39cyclictest16105-21ssh09:40:543
1022499581,39cyclictest15092-21chrt07:20:543
10221995838,8cyclictest101ktimersoftd/012:04:240
1022199581,6cyclictest12377-21munin-run11:35:010
1022199581,25cyclictest0-21swapper/008:15:150
193392570,2sleep119342-21aten_rbpower_po07:35:131
1022499571,45cyclictest0-21swapper/310:36:003
1022499571,42cyclictest12163-21gzip10:15:273
1022499571,40cyclictest8050irq/56-dwc_otg_11:40:213
1022499571,35cyclictest24063-21ssh09:12:473
1022199575,32cyclictest13-21rcuc/012:17:460
10221995734,13cyclictest0-21swapper/008:37:330
10221995734,12cyclictest16778-21ssh09:42:080
1022199572,28cyclictest0-21swapper/010:40:100
10221995717,12cyclictest0-21swapper/010:25:020
1022499561,47cyclictest2223-21ssh10:04:203
1022499561,38cyclictest5207-21perl08:25:183
10221995636,10cyclictest0-21swapper/012:38:390
10221995633,14cyclictest101ktimersoftd/010:10:130
1022199561,8cyclictest25774-21unixbench_multi07:50:290
1022199561,47cyclictest13231-21grep08:45:300
1022199561,39cyclictest25639-21ls12:30:020
1022499556,5cyclictest8050irq/56-dwc_otg_08:20:123
1022499552,43cyclictest8050irq/56-dwc_otg_11:37:503
1022499551,47cyclictest31809-21ssh11:57:323
1022499551,38cyclictest19572-21ssh12:21:333
10221995535,9cyclictest25714-21ssh09:15:150
10221995535,5cyclictest101ktimersoftd/011:40:150
10221995524,4cyclictest0-21swapper/010:05:190
1022199551,39cyclictest387-21tr11:20:160
1022499541,50cyclictest18346-21ls09:00:223
1022499541,50cyclictest15671-21apt-get07:25:143
1022499541,40cyclictest692-21sshd08:11:093
1022499541,40cyclictest692-21sshd08:11:093
1022499541,35cyclictest1165-21snmpd08:08:313
10222995429,16cyclictest12237-21sh12:13:191
10222995412,17cyclictest21126-21wc09:10:021
10222995412,17cyclictest21126-21wc09:10:021
1022199544,30cyclictest0-21swapper/009:45:200
10221995435,4cyclictest7950irq/56-dwc_otg_12:20:210
1022499538,30cyclictest0-21swapper/309:30:383
1022499534,9cyclictest8050irq/56-dwc_otg_07:30:123
1022499533,43cyclictest20605-21seq07:37:173
1022499532,48cyclictest0-21swapper/307:15:133
1022499532,47cyclictest6696-21apt-get08:30:123
10222995311,24cyclictest11694-21wc10:55:021
1022199536,31cyclictest0-21swapper/008:50:190
10221995337,4cyclictest10335-21aten_rbpower_cu08:40:140
1022199532,33cyclictest0-21swapper/008:55:110
1022199531,7cyclictest18583-21grep07:30:290
1022499522,33cyclictest8050irq/56-dwc_otg_11:10:193
1022499521,37cyclictest22856-21taskset07:45:063
1022499521,32cyclictest10612-21ssh09:35:173
10222995217,21cyclictest25145-21/usr/sbin/munin07:50:211
10222995217,16cyclictest26844-21/usr/sbin/munin07:55:201
10221995235,5cyclictest101ktimersoftd/011:37:200
1022199521,34cyclictest0-21swapper/007:45:130
1022499511,47cyclictest12374-21/usr/sbin/munin08:45:173
1022499511,35cyclictest29523-21runrttasks08:01:003
10222995115,16cyclictest12440-21cat10:55:191
10222995115,16cyclictest12440-21cat10:55:191
10221995130,9cyclictest101ktimersoftd/009:37:140
10221995126,17cyclictest10720-21vmstat12:10:320
10221995120,8cyclictest101ktimersoftd/009:10:020
10221995120,8cyclictest101ktimersoftd/009:10:020
10221995120,13cyclictest5734-21ssh09:30:010
1022199511,35cyclictest0-21swapper/011:15:150
1022199511,31cyclictest22164-21grep07:40:290
1022199511,28cyclictest20403-21grep10:25:250
1022199511,28cyclictest20403-21grep10:25:250
1022499501,45cyclictest1323-21/usr/sbin/munin08:15:163
10222995018,13cyclictest12278-21ls10:15:281
10222995016,16cyclictest880-21ssh12:39:001
10222995011,23cyclictest28161-21ssh11:53:251
10222995011,19cyclictest0-21swapper/109:45:161
10222995010,13cyclictest30737-21ssh11:17:081
1022199507,6cyclictest0-21swapper/009:30:300
10221995033,9cyclictest23500-21sh11:09:030
1022399493,12cyclictest0-21swapper/207:10:042
10222994913,21cyclictest24103-21ssh09:50:301
10222994912,15cyclictest863-21ssh10:40:321
10222994911,21cyclictest23519-21ssh11:47:431
1022199496,33cyclictest13-21rcuc/008:21:350
10221994935,8cyclictest101ktimersoftd/009:04:380
10221994925,15cyclictest101ktimersoftd/010:35:280
1022199491,24cyclictest24673-21switchtime10:30:280
1022299487,21cyclictest17191-21ssh10:21:201
10222994813,19cyclictest23908-21ssh09:12:301
10222994813,15cyclictest0-21swapper/110:27:011
10222994813,15cyclictest0-21swapper/110:27:011
1022199482,24cyclictest0-21swapper/012:05:300
1022199481,28cyclictest0-21swapper/009:10:310
10222994717,16cyclictest28899-21ssh09:56:221
10222994717,16cyclictest28899-21ssh09:56:221
10222994716,13cyclictest17201-21fschecks_time11:40:181
10222994714,12cyclictest25986-21ssh10:32:171
10222994710,21cyclictest15909-21/usr/sbin/munin08:55:191
1022199471,3cyclictest0-21swapper/008:15:020
1022199471,3cyclictest0-21swapper/008:15:020
86262463,34sleep00-21swapper/007:05:190
1022299469,18cyclictest5979-21ssh11:25:561
1022299469,18cyclictest5979-21ssh11:25:561
1022299469,11cyclictest16814-21ssh09:42:111
1022299468,23cyclictest0-21swapper/108:00:141
10222994624,6cyclictest7950irq/56-dwc_otg_11:32:051
10222994615,17cyclictest13859-21rm09:39:231
10222994613,20cyclictest27756-21ssh10:35:061
10222994613,20cyclictest20259-21aten_rbpower_cu11:05:141
10222994612,12cyclictest26751-21sh12:30:251
10222994611,19cyclictest19413-21ssh11:04:081
10222994611,15cyclictest3668-21rm12:02:481
10222994611,12cyclictest16838-21ssh12:19:081
10222994513,18cyclictest7127-21ssh12:06:331
10222994513,17cyclictest24064-21rm12:27:121
165442440,1sleep20-21swapper/209:41:422
1022399447,22cyclictest0-21swapper/208:40:142
1022299448,20cyclictest7360-21/usr/sbin/munin10:10:171
1022299447,23cyclictest0-21swapper/107:40:141
1022299446,20cyclictest24228-21taskset07:47:271
1022299442,5cyclictest8050irq/56-dwc_otg_09:20:141
10222994424,12cyclictest31108-21diskmemload11:56:151
10222994422,8cyclictest8050irq/56-dwc_otg_08:40:161
10222994410,17cyclictest27022-21ssh09:15:311
10222994313,17cyclictest21097-21diskmemload11:37:061
10222994310,19cyclictest2267-21ssh10:04:241
1022399427,25cyclictest0-21swapper/209:15:152
1022399426,22cyclictest0-21swapper/208:45:162
10223994215,5cyclictest19435-21sh11:04:102
1022299429,11cyclictest26631-21ssh11:12:121
1022299428,15cyclictest8050irq/56-dwc_otg_12:21:391
1022299427,19cyclictest7950irq/56-dwc_otg_08:35:171
1022199422,27cyclictest0-21swapper/008:30:290
1022399415,4cyclictest0-21swapper/211:20:142
10223994119,5cyclictest1133-21runrttasks12:34:512
10222994121,8cyclictest0-21swapper/107:21:141
19332400,3sleep2301ktimersoftd/210:42:292
1022399402,3cyclictest7850irq/56-dwc_otg10:00:132
1022199402,19cyclictest0-21swapper/008:02:520
1022399394,5cyclictest29-21rcuc/211:40:152
10223993914,15cyclictest29-21rcuc/209:50:212
1022399387,6cyclictest31-21ksoftirqd/210:23:422
1022399387,20cyclictest0-21swapper/211:30:132
1022399381,8cyclictest31-21ksoftirqd/209:10:202
10223993816,8cyclictest31-21ksoftirqd/207:50:202
10223993813,15cyclictest31-21ksoftirqd/210:35:242
1022299389,9cyclictest7850irq/56-dwc_otg08:35:011
1022299388,12cyclictest6408-21chrt08:29:391
1022299384,17cyclictest21097-21diskmemload09:25:301
10222993813,11cyclictest13647-21rm08:48:361
1022399377,23cyclictest0-21swapper/208:35:142
1022399375,20cyclictest23713-21ssh10:30:152
1022399374,7cyclictest31-21ksoftirqd/212:10:312
10223993723,5cyclictest7858-21rm12:07:532
10223993722,9cyclictest301ktimersoftd/210:15:302
10223993720,10cyclictest301ktimersoftd/210:52:332
10223993712,5cyclictest4044-21fschecks_time10:45:182
1022299375,20cyclictest5383-21runrttasks10:07:321
10222993713,21cyclictest0-21swapper/108:15:161
1022399365,24cyclictest29-21rcuc/207:37:162
10223993618,5cyclictest19925-21if_eth009:05:202
10223993618,5cyclictest19925-21if_eth009:05:202
1022399361,4cyclictest0-21swapper/211:16:152
10223993613,5cyclictest31-21ksoftirqd/212:25:222
10223993611,9cyclictest21097-21diskmemload11:08:232
1022299368,12cyclictest18747-21taskset07:31:351
1022299366,13cyclictest0-21swapper/108:51:141
10222993622,11cyclictest3412-21perl08:20:181
1022399359,13cyclictest29-21rcuc/208:34:482
1022399357,15cyclictest0-21swapper/209:00:142
10223993516,10cyclictest29-21rcuc/207:47:422
10223993512,9cyclictest301ktimersoftd/210:55:232
10223993512,9cyclictest301ktimersoftd/210:55:232
1022399348,14cyclictest7850irq/56-dwc_otg07:55:192
1022399346,18cyclictest31-21ksoftirqd/210:10:152
1022399346,13cyclictest0-21swapper/211:59:002
10223993418,9cyclictest1133-21runrttasks12:20:192
10223993417,9cyclictest32362-21/usr/sbin/munin08:10:202
10223993417,9cyclictest32362-21/usr/sbin/munin08:10:202
10223993416,4cyclictest4804-21fschecks_time11:25:202
10223993416,4cyclictest4804-21fschecks_time11:25:202
10223993413,4cyclictest1435-21fschecks_time12:00:182
10223993411,6cyclictest27649-21latency09:55:202
10223993411,6cyclictest27649-21latency09:55:202
10223993410,15cyclictest1479-21modprobe12:40:012
1022299345,16cyclictest0-21swapper/108:05:141
1022299341,21cyclictest8050irq/56-dwc_otg_08:10:271
1022299341,21cyclictest8050irq/56-dwc_otg_08:10:271
10222993410,5cyclictest19279-21latency_hist09:05:021
1022399337,4cyclictest8050irq/56-dwc_otg_08:10:022
10223993315,12cyclictest301ktimersoftd/208:03:392
1022399331,24cyclictest24937-21ls11:10:212
1022299336,11cyclictest13524-21taskset07:18:581
1022399328,5cyclictest0-21swapper/209:25:112
1022399328,4cyclictest0-21swapper/210:05:032
1022399325,16cyclictest6416-21fschecks_time09:30:192
1022399323,4cyclictest18388-21apt-get09:45:072
10223993210,5cyclictest0-21swapper/210:26:302
10223993210,5cyclictest0-21swapper/210:26:302
1022299327,14cyclictest0-21swapper/107:25:141
94642310,2sleep10-21swapper/107:05:321
1022399314,18cyclictest29-21rcuc/209:37:412
1022399312,20cyclictest0-21swapper/212:15:502
1022399312,19cyclictest0-21swapper/207:42:232
1022399302,16cyclictest7850irq/56-dwc_otg07:22:292
10223993013,10cyclictest13324-21sed11:35:222
10223993011,13cyclictest188502chrt07:32:562
97112295,17sleep20-21swapper/207:07:032
1022399297,10cyclictest4319-1kworker/u9:108:52:582
1022399296,14cyclictest0-21swapper/208:55:072
1022399293,18cyclictest7850irq/56-dwc_otg08:24:562
1022399286,13cyclictest0-21swapper/208:17:222
87542274,8sleep30-21swapper/307:05:223
1022399276,4cyclictest8050irq/56-dwc_otg_08:25:192
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional