You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-28 - 08:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sun Jun 28, 2026 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
797023460,2sleep20-21swapper/219:05:322
91109912610,91cyclictest4040-21ls00:05:320
911099931,6cyclictest26526-21unixbench_singl19:55:380
911099931,6cyclictest15475-21munin-plugin-st00:20:090
911099922,4cyclictest32067-21tune2fs00:00:270
911099911,4cyclictest27281-21fschecks_time23:15:270
911099901,5cyclictest32445-21apt-get20:15:090
911099901,5cyclictest0-21swapper/019:25:210
911099891,7cyclictest18691-21unixbench_singl21:45:370
911099891,6cyclictest28519-21apt21:20:090
911099891,6cyclictest28519-21apt21:20:090
316272890,5sleep3911399cyclictest22:01:313
316272890,5sleep3911399cyclictest22:01:313
175292880,2sleep00-21swapper/000:21:230
911099851,5cyclictest10142-21wc20:40:340
911099851,5cyclictest10142-21wc20:40:340
911099851,3cyclictest0-21swapper/019:50:230
911099841,5cyclictest10690-21ssh23:35:090
911099831,6cyclictest3705-21ssh21:28:370
241582830,2sleep00-21swapper/021:52:580
911099822,6cyclictest23301-21chrt22:30:340
911099821,5cyclictest16688-21aten_rbpower_po21:00:230
911099821,4cyclictest1589-21aten_rbpower_cu22:05:240
911099818,60cyclictest19962-21diskmemload23:46:210
9110998145,5cyclictest8050irq/56-dwc_otg_22:50:220
911099813,71cyclictest11310-21/usr/sbin/munin22:55:350
911099813,71cyclictest11310-21/usr/sbin/munin22:55:350
218562790,2sleep20-21swapper/223:09:062
911099782,4cyclictest23329-21idleruntime-cro19:50:090
911099781,4cyclictest17208-21grep21:45:100
911099781,4cyclictest17208-21grep21:45:100
9110997731,26cyclictest0-21swapper/019:35:210
9110997731,26cyclictest0-21swapper/019:35:210
911099771,4cyclictest11181-21rm21:37:050
911099762,46cyclictest0-21swapper/023:35:330
911099761,63cyclictest6210-21ssh23:29:360
911099761,5cyclictest0-21swapper/023:12:210
911099761,4cyclictest8296-21/usr/sbin/munin00:10:340
911099754,60cyclictest1388-21latency_hist22:05:100
911099754,60cyclictest1388-21latency_hist22:05:100
911099753,3cyclictest0-21swapper/022:25:220
911099751,56cyclictest1690-21ssh22:44:590
911099749,54cyclictest0-21swapper/021:14:270
911099745,45cyclictest0-21swapper/023:00:230
911099742,57cyclictest31768-21ssh21:23:240
911099741,6cyclictest32445-21apt-get20:15:120
911099741,5cyclictest11652-21runrttasks22:16:370
911099741,57cyclictest28205-21ssh22:36:410
911099741,57cyclictest17406-21ssh23:42:250
9110997412,39cyclictest0-21swapper/023:50:370
911099731,5cyclictest0-21swapper/020:57:300
9112997243,23cyclictest11344-21/usr/sbin/munin22:55:362
9112997243,23cyclictest11344-21/usr/sbin/munin22:55:362
911099726,48cyclictest6467-21ntp_states20:30:330
911099725,45cyclictest0-21swapper/000:35:200
911099721,5cyclictest27732-21ssh21:56:550
911099721,50cyclictest8275-21ssh21:34:200
9111997112,29cyclictest23-21ksoftirqd/122:55:351
9111997112,29cyclictest23-21ksoftirqd/122:55:351
9110997121,11cyclictest0-21swapper/022:45:210
911099711,8cyclictest0-21swapper/023:57:350
911099711,4cyclictest0-21swapper/000:34:100
911099711,47cyclictest0-21swapper/022:10:330
911099711,46cyclictest30964-21ssh23:20:130
911099711,46cyclictest30964-21ssh23:20:130
297812700,2sleep20-21swapper/222:39:342
9111996913,24cyclictest7950irq/56-dwc_otg_00:05:311
911099692,3cyclictest0-21swapper/000:25:470
9110996918,27cyclictest8050irq/56-dwc_otg_22:25:090
911099691,4cyclictest0-21swapper/019:44:590
911099681,5cyclictest0-21swapper/020:49:030
9110996718,7cyclictest0-21swapper/020:50:180
911099671,5cyclictest28410-21chrt20:01:110
911099671,5cyclictest0-21swapper/023:07:240
911099671,4cyclictest0-21swapper/020:37:410
9112996617,22cyclictest3894-21ssh00:05:292
911099661,5cyclictest0-21swapper/019:19:520
9112996519,21cyclictest28382-21ssh23:15:552
911099651,46cyclictest0-21swapper/020:21:160
911099641,5cyclictest10413-21taskset19:11:450
131142640,2sleep20-21swapper/200:15:532
911099631,5cyclictest0-21swapper/019:22:340
911099621,5cyclictest5224-21chrt20:26:270
911099621,5cyclictest0-21swapper/020:09:380
9113996134,11cyclictest3932-21ssh00:05:303
911099611,5cyclictest0-21swapper/021:05:390
9113996012,18cyclictest8050irq/56-dwc_otg_22:55:373
9113996012,18cyclictest8050irq/56-dwc_otg_22:55:373
911099601,41cyclictest0-21swapper/019:34:010
911399563,13cyclictest26890-21ssh23:15:193
9112995615,19cyclictest29164-21ssh21:59:332
911299553,33cyclictest0-21swapper/222:40:212
9112995523,12cyclictest7371-21sed20:35:092
9112995520,16cyclictest18592-21ls21:45:362
9112995515,24cyclictest10285-21cat20:40:362
9112995515,24cyclictest10285-21cat20:40:362
9112995514,26cyclictest27530-21chrt23:55:162
327592550,1sleep30-21swapper/323:21:233
327592550,1sleep30-21swapper/323:21:233
296062550,3sleep329605-21ssh22:00:103
9112995414,21cyclictest16293-21ssh23:01:272
911299532,11cyclictest7950irq/56-dwc_otg_20:50:182
9112995319,18cyclictest18487-21/usr/sbin/munin22:25:282
9112995318,22cyclictest14942-21/usr/sbin/munin19:25:282
9112995316,20cyclictest4480-21ssh22:47:322
9112995314,20cyclictest0-21swapper/219:50:292
9112995312,27cyclictest4674-21ssh23:26:462
9112995311,21cyclictest16589-21ssh00:20:332
9112995215,20cyclictest16539-21ssh23:40:492
9112995125,8cyclictest317-21apt-get20:15:212
9112995121,8cyclictest12755-21apt-get19:20:202
9112995117,21cyclictest27052-21ssh23:54:492
9112995116,7cyclictest19883-21memory23:45:312
9112995114,20cyclictest17681-21ssh22:25:062
9112995025,11cyclictest12430-21rm22:18:022
9112995017,18cyclictest26038-21ntpq19:55:322
9112995016,12cyclictest0-21swapper/221:40:212
9112995016,12cyclictest0-21swapper/221:40:212
911199504,36cyclictest0-21swapper/120:50:191
9112994918,19cyclictest10749-21latency_hist23:35:092
9112994916,17cyclictest681-21ssh00:00:492
911199493,4cyclictest7850irq/56-dwc_otg21:45:211
911399481,9cyclictest0-21swapper/323:50:203
9112994816,11cyclictest1665-21ssh22:05:252
9112994815,8cyclictest8689-21ssh22:13:462
9112994815,13cyclictest10336-21unixbench_multi21:35:382
9112994814,22cyclictest32648-21rm23:21:102
9112994814,22cyclictest32648-21rm23:21:102
911199482,5cyclictest0-21swapper/121:35:221
911299479,22cyclictest7850irq/56-dwc_otg21:20:352
911299479,13cyclictest0-21swapper/221:16:092
911299479,13cyclictest0-21swapper/221:16:092
911299477,25cyclictest0-21swapper/220:55:232
9112994716,13cyclictest2124-21ssh21:25:512
9112994715,21cyclictest30453-21ssh00:37:232
9112994713,18cyclictest6919-21ssh21:31:502
9112994712,20cyclictest23878-21ssh21:14:322
911199474,35cyclictest0-21swapper/120:10:201
911199474,33cyclictest0-21swapper/123:45:231
911199471,4cyclictest0-21swapper/123:25:221
911399461,8cyclictest0-21swapper/320:45:203
911299469,7cyclictest7950irq/56-dwc_otg_00:30:242
911299469,21cyclictest9664-21sh00:12:072
9112994617,12cyclictest23228-21ssh21:51:162
9112994616,11cyclictest21521-21ssh00:26:062
9112994613,14cyclictest23526-21latency_hist19:50:092
911199466,25cyclictest0-21swapper/120:05:231
9111994627,14cyclictest8050irq/56-dwc_otg_21:55:201
9111994615,11cyclictest221ktimersoftd/100:00:291
85202465,35sleep00-21swapper/019:06:080
6412460,2sleep30-21swapper/300:00:443
911299457,25cyclictest0-21swapper/223:35:232
911299457,24cyclictest0-21swapper/220:05:142
911199455,6cyclictest0-21swapper/121:15:221
911199455,6cyclictest0-21swapper/121:15:221
911199452,7cyclictest0-21swapper/123:40:221
9111994518,12cyclictest23-21ksoftirqd/122:47:471
911299448,23cyclictest0-21swapper/220:45:232
911299447,22cyclictest0-21swapper/219:10:232
9112994416,13cyclictest22713-21ssh23:10:142
9112994411,20cyclictest32224-21ssh22:02:352
9112994411,20cyclictest32224-21ssh22:02:352
9112994410,19cyclictest6241-21sh22:50:192
9112994410,12cyclictest7950irq/56-dwc_otg_21:00:272
911199447,20cyclictest0-21swapper/121:25:221
911199446,24cyclictest0-21swapper/120:45:231
911199446,24cyclictest0-21swapper/120:20:231
911199445,12cyclictest14510-21ssh23:00:211
9111994410,16cyclictest21-21rcuc/123:35:091
95382430,5sleep1911199cyclictest00:11:531
911399435,7cyclictest0-21swapper/319:15:213
911299437,23cyclictest0-21swapper/220:20:232
9112994317,11cyclictest23478-21users22:30:362
911199436,24cyclictest0-21swapper/120:15:151
128492421,4sleep312996-21ssh21:40:103
911299417,24cyclictest0-21swapper/219:40:232
9112994113,19cyclictest17787-21chrt19:33:402
911299411,13cyclictest7950irq/56-dwc_otg_19:15:232
911199418,14cyclictest8050irq/56-dwc_otg_21:05:241
911199417,24cyclictest0-21swapper/119:20:231
9111994114,6cyclictest23-21ksoftirqd/100:20:241
911199411,19cyclictest32108-21cat00:40:101
9112994030,7cyclictest18396-21apt-get21:05:212
911199405,25cyclictest0-21swapper/120:40:231
911199405,25cyclictest0-21swapper/120:40:231
911199403,4cyclictest7950irq/56-dwc_otg_22:40:211
911199403,18cyclictest11-21rcu_preempt22:20:351
9111994023,4cyclictest505-21cron19:35:091
9111994022,9cyclictest7850irq/56-dwc_otg22:08:121
911199402,27cyclictest0-21swapper/119:35:211
911199402,27cyclictest0-21swapper/119:35:211
9111994020,9cyclictest7850irq/56-dwc_otg22:31:001
911399398,14cyclictest8050irq/56-dwc_otg_21:20:333
911399392,6cyclictest8050irq/56-dwc_otg_20:40:223
911399392,6cyclictest8050irq/56-dwc_otg_20:40:223
911299396,22cyclictest7950irq/56-dwc_otg_20:35:362
9112993914,10cyclictest18145-21/usr/sbin/munin19:35:292
9112993914,10cyclictest18145-21/usr/sbin/munin19:35:292
9111993930,5cyclictest221ktimersoftd/122:00:201
9111993930,5cyclictest221ktimersoftd/122:00:201
9111993921,7cyclictest7850irq/56-dwc_otg22:25:251
9111993914,6cyclictest0-21swapper/100:27:511
911199391,23cyclictest0-21swapper/119:55:231
254402390,2sleep111-21rcu_preempt00:30:461
97532381,33sleep3750-21snmpd23:33:313
9111993810,11cyclictest221ktimersoftd/123:15:291
911399371,8cyclictest23075-21ssh23:50:103
9113993718,5cyclictest14100-21unixbench_multi20:50:353
9113993717,5cyclictest30129-21latency_hist22:40:093
9113993714,12cyclictest19962-21diskmemload23:02:243
911399371,25cyclictest6005-21cut20:30:263
911199375,7cyclictest221ktimersoftd/123:55:091
9111993715,9cyclictest221ktimersoftd/123:21:051
9111993715,9cyclictest221ktimersoftd/123:21:051
9111993715,5cyclictest22798-21unixbench_multi21:50:381
9111993714,10cyclictest7850irq/56-dwc_otg23:07:241
911199371,24cyclictest12619-21ls19:20:101
911199371,23cyclictest15079-21cut20:55:261
223372371,32sleep322347-21ssh21:11:513
168502371,4sleep10-21swapper/121:44:551
168502371,4sleep10-21swapper/121:44:551
911399367,19cyclictest0-21swapper/320:35:223
911399365,8cyclictest19903-21grep23:05:373
911399365,21cyclictest17371-21ssh21:45:163
9113993617,6cyclictest381ktimersoftd/323:40:323
9113993617,4cyclictest1392-21/usr/sbin/munin20:15:373
9113993611,6cyclictest0-21swapper/321:50:273
9112993610,6cyclictest7950irq/56-dwc_otg_20:05:102
9112993610,10cyclictest0-21swapper/220:12:512
911199367,22cyclictest0-21swapper/119:45:221
911199361,7cyclictest7850irq/56-dwc_otg19:10:381
9111993615,12cyclictest7850irq/56-dwc_otg21:20:291
911199361,23cyclictest27691-21latency20:00:271
911199361,20cyclictest221ktimersoftd/100:17:171
9111993611,13cyclictest7850irq/56-dwc_otg19:25:201
911399358,13cyclictest8050irq/56-dwc_otg_23:35:323
911399357,14cyclictest8050irq/56-dwc_otg_00:30:393
911399355,3cyclictest0-21swapper/323:11:333
911399355,14cyclictest7950irq/56-dwc_otg_21:30:093
911199358,9cyclictest7950irq/56-dwc_otg_19:50:231
911199355,17cyclictest21-21rcuc/122:50:331
911199351,2cyclictest0-21swapper/123:55:201
9111993512,9cyclictest221ktimersoftd/123:35:101
911399347,12cyclictest8050irq/56-dwc_otg_22:55:093
911399346,13cyclictest8050irq/56-dwc_otg_22:40:343
911399344,5cyclictest7850irq/56-dwc_otg00:27:383
911399342,14cyclictest8050irq/56-dwc_otg_21:15:283
911399342,14cyclictest8050irq/56-dwc_otg_21:15:283
911399341,8cyclictest3369-21grep22:45:383
9113993410,5cyclictest3734-21idleruntime-cro20:25:093
9113993410,12cyclictest8050irq/56-dwc_otg_23:25:263
9112993419,12cyclictest4166-21ls20:25:232
911199347,13cyclictest23-21ksoftirqd/121:11:371
911199342,8cyclictest8895-21latency_hist21:35:101
9111993416,9cyclictest221ktimersoftd/122:37:471
9111993414,14cyclictest7850irq/56-dwc_otg22:11:201
9111993412,6cyclictest991250wget22:15:221
911399338,13cyclictest8050irq/56-dwc_otg_21:42:353
911399338,13cyclictest8050irq/56-dwc_otg_21:42:353
911399336,15cyclictest8050irq/56-dwc_otg_22:20:243
911399336,15cyclictest0-21swapper/320:05:233
911399332,20cyclictest0-21swapper/319:35:383
911399332,20cyclictest0-21swapper/319:35:383
9113993317,4cyclictest25379-21aten_rbpower_cu19:55:233
168022330,4sleep316808-21switchtime00:20:373
911399327,15cyclictest0-21swapper/322:10:223
911399327,15cyclictest0-21swapper/320:00:203
911399326,14cyclictest0-21swapper/320:10:213
911399324,8cyclictest7950irq/56-dwc_otg_19:30:093
9113993223,6cyclictest9818-21ssh00:12:253
911399321,8cyclictest20706-21seq19:40:343
911399321,21cyclictest0-21swapper/319:10:283
911199325,5cyclictest6863-21unixbench_multi20:30:371
911199322,13cyclictest23-21ksoftirqd/121:00:241
77532324,16sleep37847-21/usr/sbin/munin19:05:303
911399315,7cyclictest8050irq/56-dwc_otg_22:25:213
911399311,5cyclictest8804-21ssh21:35:093
9113993113,5cyclictest14356-21latency_hist19:25:093
9113993112,5cyclictest37-21rcuc/321:00:093
911399311,22cyclictest0-21swapper/322:15:523
911199316,12cyclictest8050irq/56-dwc_otg_23:10:241
911399305,5cyclictest7850irq/56-dwc_otg22:05:193
911399305,4cyclictest0-21swapper/322:30:183
911399302,14cyclictest7950irq/56-dwc_otg_00:40:103
9113993020,7cyclictest11747-21aten_rbpower_en00:15:243
911399301,11cyclictest0-21swapper/319:50:263
9113993010,13cyclictest5382-21taskset20:28:283
911199306,10cyclictest7850irq/56-dwc_otg20:35:221
911199301,14cyclictest21-21rcuc/119:44:321
911399297,10cyclictest8050irq/56-dwc_otg_19:45:333
911399293,14cyclictest7950irq/56-dwc_otg_21:05:153
911399292,15cyclictest8050irq/56-dwc_otg_23:55:193
911199296,4cyclictest23-21ksoftirqd/120:26:231
87292298,14sleep10-21swapper/119:08:481
911399286,9cyclictest7950irq/56-dwc_otg_21:01:513
911399271,16cyclictest0-21swapper/319:31:473
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional