You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-09 - 10:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sat May 09, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2381921470,5sleep10-21swapper/104:30:571
23954991086,66cyclictest0-21swapper/004:46:010
239549910127,40cyclictest18188-21kworker/u8:305:56:030
45522880,5sleep32395799cyclictest06:32:153
45522880,5sleep32395799cyclictest06:32:153
26142860,1sleep10-21swapper/109:46:151
26142860,1sleep10-21swapper/109:46:151
2395499795,10cyclictest0-21swapper/006:26:070
23954997920,4cyclictest13798-21grep05:31:150
2395499781,7cyclictest13780-21latency_hist09:20:570
2395499771,26cyclictest0-21swapper/008:40:560
272362760,2sleep30-21swapper/309:36:223
92832750,2sleep10-21swapper/108:36:091
2395499752,26cyclictest12400-21sed08:00:570
23954997516,6cyclictest7773-21aten_rbpower_cu07:16:080
2395499751,33cyclictest0-21swapper/006:31:140
2395499751,33cyclictest0-21swapper/006:31:140
2395499751,22cyclictest0-21swapper/009:46:190
2395499751,22cyclictest0-21swapper/009:46:190
23954997426,4cyclictest7850irq/56-dwc_otg_09:01:060
2395499741,30cyclictest0-21swapper/004:37:270
138622740,2sleep30-21swapper/310:00:303
2395499722,5cyclictest0-21swapper/004:56:120
2395499721,7cyclictest29661-21unixbench_singl07:41:230
2395499712,34cyclictest0-21swapper/006:51:080
2395499712,25cyclictest0-21swapper/008:20:560
2395499711,27cyclictest0-21swapper/007:02:040
2395499711,24cyclictest0-21swapper/009:06:070
2395499702,30cyclictest0-21swapper/007:13:170
23954997014,16cyclictest0-21swapper/008:23:500
23954996920,5cyclictest7850irq/56-dwc_otg_07:36:090
2395499686,16cyclictest0-21swapper/009:28:300
2395499671,21cyclictest12596-21/usr/sbin/munin08:01:160
2395499663,44cyclictest0-21swapper/004:30:590
2395499658,54cyclictest0-21swapper/005:46:080
2395499651,30cyclictest3983-21latency_hist07:50:560
239472650,5sleep20-21swapper/204:30:582
280492640,2sleep20-21swapper/208:58:172
98692630,2sleep10-21swapper/109:55:451
23954996329,17cyclictest9-21ksoftirqd/008:26:070
2395499631,43cyclictest27465-21modprobe07:00:560
2395499631,43cyclictest27465-21modprobe07:00:560
2395499631,43cyclictest16258-21proc_pri06:46:200
2395499612,42cyclictest26618-21fschecks_time09:36:140
2395499611,44cyclictest5380-21cut08:31:120
185472610,5sleep02395499cyclictest07:28:400
23954996040,10cyclictest25573-21ls08:55:570
23954996040,10cyclictest25573-21ls08:55:570
230532606,40sleep30-21swapper/304:26:223
2395499591,42cyclictest6752-21ntpq05:11:180
23954995912,9cyclictest0-21swapper/008:56:050
24152580,2sleep10-21swapper/106:30:561
2395599585,22cyclictest7950irq/56-dwc_otg_06:51:071
2395499587,29cyclictest0-21swapper/006:41:180
2395499583,33cyclictest0-21swapper/006:21:180
23954995827,5cyclictest0-21swapper/008:06:070
2395499582,37cyclictest0-21swapper/009:41:160
2395499581,38cyclictest0-21swapper/004:41:120
2395499575,30cyclictest0-21swapper/006:11:230
23954995722,18cyclictest0-21swapper/008:46:100
23954995629,13cyclictest12616-21grep05:26:240
23954995611,6cyclictest20825-21fschecks_time05:51:130
2395499553,8cyclictest11890-21ssh07:21:080
2395499552,32cyclictest0-21swapper/009:56:240
23954995521,11cyclictest0-21swapper/005:36:060
2395499551,29cyclictest0-21swapper/005:16:090
2395499541,40cyclictest7374-21fschecks_time06:36:140
2395699533,7cyclictest7950irq/56-dwc_otg_09:41:092
23954995323,5cyclictest22408-21grep09:31:120
2395499532,34cyclictest0-21swapper/008:14:070
2395499531,4cyclictest31307-21/usr/sbin/munin04:51:240
2395499531,24cyclictest0-21swapper/009:11:080
2395499524,25cyclictest9-21ksoftirqd/005:06:210
2395499521,5cyclictest31912-21apt-get07:06:070
23954995124,8cyclictest30424-21proc_pri06:16:200
2395499511,34cyclictest20744-21/usr/sbin/munin07:31:160
2395499511,30cyclictest8151-21latency_hist07:55:560
23954995015,20cyclictest0-21swapper/006:01:070
2395599496,26cyclictest15422-21ssh06:46:091
2395499491,33cyclictest7037-21/usr/sbin/munin09:51:220
39802480,3sleep3381ktimersoftd/309:08:283
2395599477,25cyclictest0-21swapper/107:11:081
2395499472,28cyclictest13610-21uniq08:41:120
23954994710,27cyclictest0-21swapper/009:21:330
23954994710,27cyclictest0-21swapper/009:21:330
23957994523,12cyclictest7010-21sh07:53:583
23954994517,3cyclictest0-21swapper/005:05:560
2395599447,25cyclictest0-21swapper/107:06:091
2395499441,24cyclictest17614-21ntpq05:41:190
23957994317,6cyclictest28135-21fschecks_time07:01:123
2395599437,24cyclictest0-21swapper/108:11:021
2395599434,22cyclictest0-21swapper/104:30:581
2395499436,23cyclictest0-21swapper/005:21:090
2395499436,23cyclictest0-21swapper/005:21:090
2395799427,23cyclictest0-21swapper/309:11:103
23957994215,17cyclictest7950irq/56-dwc_otg_07:26:063
2395699425,6cyclictest7850irq/56-dwc_otg_07:51:142
23956994214,13cyclictest0-21swapper/208:26:232
2395799412,6cyclictest7950irq/56-dwc_otg_09:46:063
2395799412,6cyclictest7950irq/56-dwc_otg_09:46:063
2395599414,29cyclictest0-21swapper/107:46:591
2395599413,27cyclictest0-21swapper/108:20:571
2395499412,22cyclictest0-21swapper/006:05:580
163792410,2sleep20-21swapper/209:23:332
163792410,2sleep20-21swapper/209:23:332
2395799401,6cyclictest0-21swapper/306:06:073
2395699401,5cyclictest0-21swapper/209:51:102
2395599403,15cyclictest7750irq/56-dwc_otg05:06:071
2395599401,30cyclictest21985-21latency_hist09:30:561
207532400,4sleep231-21ksoftirqd/205:51:122
113502401,34sleep20-21swapper/207:59:232
2395799397,22cyclictest0-21swapper/309:41:093
2395799391,7cyclictest14176-21cut09:21:133
2395799391,7cyclictest14176-21cut09:21:133
23957993917,11cyclictest9934-21sh09:55:533
2395699399,4cyclictest0-21swapper/209:30:292
2395699395,9cyclictest7850irq/56-dwc_otg_04:51:092
2395699395,6cyclictest29244-21apt08:21:072
2395699391,7cyclictest27370-21munin-run04:40:562
23956993914,5cyclictest20368-21run-parts08:10:362
2395699391,13cyclictest13132-21ssh07:21:252
2395599392,27cyclictest27455-21cron07:00:561
2395599392,27cyclictest27455-21cron07:00:561
23955993917,4cyclictest6078-21timerandwakeup08:31:221
2395599391,23cyclictest32038-21missed_timers06:21:171
23955993911,8cyclictest21-21rcuc/108:26:241
2395799389,6cyclictest39-21ksoftirqd/306:56:063
2395799389,6cyclictest39-21ksoftirqd/306:56:063
23957993822,5cyclictest19633-21kworker/3:308:16:123
2395699388,4cyclictest0-21swapper/209:50:362
2395699388,4cyclictest0-21swapper/209:50:362
2395699388,4cyclictest0-21swapper/209:08:482
2395699387,6cyclictest0-21swapper/207:04:502
2395699387,4cyclictest0-21swapper/206:40:412
23956993815,4cyclictest32694-21aten_rbpower_po07:46:092
23955993825,10cyclictest7950irq/56-dwc_otg_08:56:051
2395599382,21cyclictest0-21swapper/107:52:351
2395599381,5cyclictest0-21swapper/104:45:571
2395599381,13cyclictest2054550ssh08:10:561
103852381,2sleep10-21swapper/109:16:161
2395799371,7cyclictest20575-21idleruntime-cro08:10:563
2395699377,4cyclictest7850irq/56-dwc_otg_04:46:202
2395699376,4cyclictest0-21swapper/206:47:572
2395699376,21cyclictest31-21ksoftirqd/205:01:092
2395699375,5cyclictest7850irq/56-dwc_otg_09:16:122
23956993720,4cyclictest17416-21/usr/sbin/munin08:46:132
23956993715,5cyclictest21024-21aten_rbpower_cu08:11:082
23956993712,17cyclictest0-21swapper/207:30:132
2395599377,25cyclictest0-21swapper/108:01:021
2395599375,20cyclictest17463-21ssh08:46:091
2395599374,21cyclictest0-21swapper/106:36:111
23955993717,13cyclictest30073-21latency_hist09:40:561
2395599371,20cyclictest9682-21/usr/sbin/munin05:21:131
2395599371,20cyclictest9682-21/usr/sbin/munin05:21:131
2395799364,5cyclictest28369-21ssh07:41:063
23957993617,6cyclictest2840-21sh07:48:533
23957993616,7cyclictest11125-21sh07:58:583
2395699367,4cyclictest0-21swapper/209:57:072
2395699366,6cyclictest7850irq/56-dwc_otg_08:01:162
2395699365,5cyclictest1376350ssh08:41:132
2395599362,7cyclictest16001-21ssh07:26:061
2395599361,19cyclictest0-21swapper/104:51:051
2395799352,22cyclictest30622-21rm09:01:193
2395799351,6cyclictest13103-21grep07:21:243
2395699358,4cyclictest0-21swapper/206:42:532
2395699358,4cyclictest0-21swapper/206:32:432
2395699358,4cyclictest0-21swapper/206:32:432
2395699357,5cyclictest32427-21ls06:21:232
2395699357,4cyclictest0-21swapper/208:31:442
2395699356,3cyclictest0-21swapper/207:37:472
2395699352,5cyclictest0-21swapper/205:26:082
23956993514,14cyclictest13963-21/usr/sbin/munin05:31:172
2395699351,3cyclictest7950irq/56-dwc_otg_08:36:062
2395599354,22cyclictest0-21swapper/105:57:471
2395599353,21cyclictest0-21swapper/105:19:021
2395599351,20cyclictest0-21swapper/107:24:121
210812350,1sleep30-21swapper/309:29:383
2395799345,20cyclictest12572-21ssh08:01:023
2395799342,19cyclictest7950irq/56-dwc_otg_07:36:233
23957993414,10cyclictest2409-21diskmemload08:41:033
23957993413,5cyclictest25384-21cron08:55:563
23957993413,5cyclictest25384-21cron08:55:563
23957993413,4cyclictest25761-21aten_rbpower_en08:56:083
2395799341,22cyclictest0-21swapper/305:11:223
2395799341,14cyclictest18666-21grep08:46:233
2395699349,4cyclictest0-21swapper/209:36:092
2395699349,3cyclictest0-21swapper/208:17:532
2395699348,12cyclictest0-21swapper/209:05:562
2395699347,5cyclictest7850irq/56-dwc_otg_05:06:112
2395699346,4cyclictest0-21swapper/209:13:222
2395699341,5cyclictest23963-21cut06:56:122
2395699341,5cyclictest23963-21cut06:56:122
2395699341,20cyclictest32579-21ssh07:06:172
2395599347,17cyclictest0-21swapper/109:21:091
2395599347,17cyclictest0-21swapper/109:21:091
2395599346,18cyclictest0-21swapper/109:11:101
2395599346,15cyclictest0-21swapper/109:41:091
2395599344,16cyclictest1348-21latency_hist09:05:561
2395599341,6cyclictest0-21swapper/107:19:091
2395599341,4cyclictest2468-21/usr/sbin/munin05:01:251
2395599341,23cyclictest0-21swapper/105:40:021
2395599341,19cyclictest0-21swapper/109:32:541
23955993410,4cyclictest0-21swapper/107:41:181
2395799337,4cyclictest0-21swapper/309:16:013
2395799336,16cyclictest1714-21ls06:26:203
2395799335,10cyclictest7750irq/56-dwc_otg07:31:233
2395799332,6cyclictest0-21swapper/304:46:153
23957993319,5cyclictest30963-21sh08:22:093
23957993319,5cyclictest14389-21sh06:44:423
2395799331,6cyclictest4274-21/usr/sbin/munin05:06:073
2395799331,16cyclictest7850irq/56-dwc_otg_05:36:203
2395699331,5cyclictest28200-21perl06:11:122
2395599339,6cyclictest0-21swapper/107:34:351
2395599337,24cyclictest0-21swapper/106:41:071
2395599336,13cyclictest7750irq/56-dwc_otg08:55:561
2395599336,13cyclictest7750irq/56-dwc_otg08:55:561
2395599333,7cyclictest0-21swapper/105:26:081
2395599333,5cyclictest8557-21aten_rbpower_vo07:56:101
2395599333,22cyclictest0-21swapper/108:41:141
23955993315,4cyclictest4595-21sh06:32:191
23955993315,4cyclictest4595-21sh06:32:191
23955993312,3cyclictest0-21swapper/108:21:161
2395599331,13cyclictest16953-21wget05:41:101
2395799328,5cyclictest7850irq/56-dwc_otg_04:40:563
2395799322,5cyclictest0-21swapper/306:25:563
2395699327,5cyclictest7850irq/56-dwc_otg_07:41:092
2395699322,13cyclictest7850irq/56-dwc_otg_05:41:132
23956993218,4cyclictest908-21runrttasks08:55:372
23956993218,4cyclictest908-21runrttasks08:55:372
23956993214,10cyclictest22798-21idleruntime05:56:132
23956993211,7cyclictest0-21swapper/206:06:112
2395599326,4cyclictest0-21swapper/107:02:461
2395599322,4cyclictest7850irq/56-dwc_otg_06:11:061
2395599322,11cyclictest0-21swapper/105:00:441
2395599321,23cyclictest2521-21switchtime09:06:221
2395799316,12cyclictest7950irq/56-dwc_otg_07:11:193
2395799315,13cyclictest0-21swapper/304:56:183
23957993117,5cyclictest32637-21netstat07:06:173
23957993117,4cyclictest908-21runrttasks06:37:233
2395799311,6cyclictest1708-21ssh08:26:193
23957993113,9cyclictest2409-21diskmemload08:38:533
23957993113,5cyclictest22863-21sh08:12:253
23957993112,11cyclictest246902sleep304:31:203
23957993111,4cyclictest24867-21cat06:01:163
2395799311,13cyclictest0-21swapper/306:47:373
2395799311,11cyclictest0-21swapper/307:18:203
23957993110,5cyclictest0-21swapper/308:31:243
2395699319,6cyclictest7850irq/56-dwc_otg_05:21:162
2395699319,6cyclictest7850irq/56-dwc_otg_05:21:162
2395699318,10cyclictest0-21swapper/206:54:352
2395699316,4cyclictest0-21swapper/207:18:182
2395699314,18cyclictest29-21rcuc/207:32:412
2395699311,7cyclictest0-21swapper/209:34:162
23956993114,8cyclictest908-21runrttasks05:18:532
23956993111,5cyclictest2483-21latency_hist06:30:562
2395599314,4cyclictest0-21swapper/107:36:151
2395599312,6cyclictest20475-21latency_hist05:50:571
2395599311,23cyclictest0-21swapper/104:49:271
2395599311,21cyclictest0-21swapper/104:36:271
2395599311,14cyclictest26308-21grep06:06:121
2395799301,22cyclictest0-21swapper/305:31:483
2395799301,21cyclictest0-21swapper/305:57:133
2395799301,10cyclictest0-21swapper/309:31:333
23957993010,4cyclictest0-21swapper/306:52:133
2395699307,9cyclictest7950irq/56-dwc_otg_04:56:182
2395699304,11cyclictest29-21rcuc/205:36:212
23956993010,5cyclictest0-21swapper/207:12:042
2395599301,22cyclictest13712-21if_eth005:31:141
2395599301,19cyclictest1732-21kworker/u8:409:56:241
223532305,10sleep00-21swapper/004:26:110
2395799294,5cyclictest7750irq/56-dwc_otg06:11:083
2395799293,11cyclictest7750irq/56-dwc_otg05:01:193
2395799291,7cyclictest13064-21apt05:30:553
2395799291,11cyclictest0-21swapper/305:50:513
2395699292,17cyclictest24235-21/usr/sbin/munin06:01:192
2395599291,17cyclictest0-21swapper/106:01:101
2395799281,18cyclictest0-21swapper/304:41:113
2395799281,15cyclictest7950irq/56-dwc_otg_05:21:173
2395799281,15cyclictest7950irq/56-dwc_otg_05:21:173
2395799281,14cyclictest0-21swapper/305:52:033
2395599282,5cyclictest20524-21/usr/sbin/munin05:51:231
2395599282,16cyclictest0-21swapper/106:16:081
2395599282,15cyclictest0-21swapper/105:11:251
2395799272,18cyclictest0-21swapper/305:16:563
2395699274,9cyclictest7950irq/56-dwc_otg_06:16:232
2395699271,9cyclictest0-21swapper/204:43:252
224852277,9sleep20-21swapper/204:26:132
2395799261,14cyclictest0-21swapper/304:51:083
2395799261,11cyclictest0-21swapper/305:45:303
2395699261,4cyclictest7750irq/56-dwc_otg05:46:072
2395699261,3cyclictest0-21swapper/205:11:092
23957992513,9cyclictest7950irq/56-dwc_otg_06:16:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional