You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-17 - 11:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Thu Sep 17, 2026 00:43:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4218991267,74cyclictest0-21swapper/019:30:300
4218991251,5cyclictest0-21swapper/023:14:310
4218991042,88cyclictest0-21swapper/023:07:210
4218991042,88cyclictest0-21swapper/023:07:210
4218999915,53cyclictest0-21swapper/019:45:360
421899982,5cyclictest22923-21apt-get00:05:240
421899982,5cyclictest22923-21apt-get00:05:240
4218999825,41cyclictest0-21swapper/023:45:360
4218999825,41cyclictest0-21swapper/023:45:360
421899962,72cyclictest23132-21latency_hist00:05:230
421899961,78cyclictest22606-21/usr/sbin/munin20:00:370
421899961,78cyclictest22606-21/usr/sbin/munin20:00:370
421899942,77cyclictest0-21swapper/022:54:380
421899942,77cyclictest0-21swapper/022:54:380
421899941,72cyclictest0-21swapper/021:45:510
79822930,6sleep1421999cyclictest00:26:021
421899932,74cyclictest0-21swapper/023:21:500
421899932,74cyclictest0-21swapper/023:21:500
421899932,73cyclictest13373-21/usr/sbin/munin19:35:320
4218999319,53cyclictest7382-21sh21:40:460
421899921,74cyclictest0-21swapper/023:55:370
421899921,4cyclictest0-21swapper/023:50:510
4219999116,34cyclictest7950irq/56-dwc_otg_23:30:291
421899917,70cyclictest0-21swapper/021:25:360
4218999118,51cyclictest15379-21ssh21:50:470
223302910,7sleep2422099cyclictest22:40:492
421899905,23cyclictest22409-21ssh22:00:340
421899902,79cyclictest0-21swapper/020:14:530
421899901,63cyclictest28543-21grep00:10:520
4218998924,49cyclictest7850irq/56-dwc_otg_21:35:360
4218998924,49cyclictest7850irq/56-dwc_otg_21:35:360
421899891,77cyclictest0-21swapper/022:47:340
421899882,71cyclictest0-21swapper/000:21:590
421899882,71cyclictest0-21swapper/000:21:590
421899881,70cyclictest0-21swapper/021:05:440
421899881,3cyclictest0-21swapper/022:32:120
128782880,8sleep1421999cyclictest23:51:001
421899871,72cyclictest27062-21cat21:25:220
130622860,6sleep37950irq/56-dwc_otg_21:48:033
421899852,67cyclictest0-21swapper/023:41:120
421899842,5cyclictest520-21rs:main0
421899832,69cyclictest0-21swapper/000:36:250
421899831,72cyclictest0-21swapper/022:44:440
421899831,71cyclictest0-21swapper/022:19:570
421899821,7cyclictest18715-21df21:55:370
421899812,61cyclictest0-21swapper/020:29:240
421899812,61cyclictest0-21swapper/020:29:240
421899811,62cyclictest0-21swapper/021:17:070
421899811,3cyclictest0-21swapper/022:37:140
421899811,3cyclictest0-21swapper/022:37:140
421899802,58cyclictest0-21swapper/022:12:000
421899802,57cyclictest0-21swapper/019:15:440
421899801,5cyclictest18691-21ssh21:14:310
421899801,3cyclictest0-21swapper/000:33:450
421899792,66cyclictest0-21swapper/019:58:330
421899792,65cyclictest0-21swapper/023:26:420
421899792,62cyclictest0-21swapper/023:39:040
421899792,53cyclictest0-21swapper/022:22:100
421899792,53cyclictest0-21swapper/022:22:100
421899791,64cyclictest0-21swapper/022:07:150
421899791,5cyclictest6474-21ssh23:02:050
421899791,56cyclictest31415-21aten_rbpower_po21:30:350
30602790,2sleep20-21swapper/223:40:052
4218997830,10cyclictest0-21swapper/023:15:350
421899782,65cyclictest0-21swapper/022:57:050
421899782,58cyclictest0-21swapper/022:27:120
322412780,2sleep20-21swapper/222:12:162
421899773,69cyclictest32173-21ssh00:15:490
4218997718,3cyclictest0-21swapper/023:30:340
421899762,58cyclictest0-21swapper/019:24:160
4220997529,11cyclictest7850irq/56-dwc_otg_21:50:332
421899752,65cyclictest0-21swapper/020:10:220
421899751,3cyclictest0-21swapper/019:53:590
107112750,4sleep2422099cyclictest23:07:452
107112750,4sleep2422099cyclictest23:07:452
421899737,53cyclictest29486-21cron20:20:210
421899731,5cyclictest0-21swapper/020:49:150
422199723,26cyclictest7850irq/56-dwc_otg_20:30:283
138722720,2sleep20-21swapper/223:11:152
421899712,5cyclictest0-21swapper/020:56:530
421899712,56cyclictest0-21swapper/021:04:490
421899711,5cyclictest4447-21cat20:40:220
421899711,3cyclictest0-21swapper/019:42:170
422099701,36cyclictest0-21swapper/222:15:402
421899701,5cyclictest0-21swapper/020:54:180
421899692,55cyclictest0-21swapper/019:28:430
421899671,3cyclictest0-21swapper/020:31:530
421899662,6cyclictest0-21swapper/019:11:060
421899651,3cyclictest0-21swapper/020:41:350
421899651,3cyclictest0-21swapper/020:41:350
421899651,3cyclictest0-21swapper/020:24:370
23782653,11sleep10-21swapper/119:05:251
23782653,11sleep10-21swapper/119:05:251
422099599,31cyclictest0-21swapper/219:30:302
422099581,5cyclictest25251-21ssh22:45:362
422199568,30cyclictest0-21swapper/323:30:283
422099558,30cyclictest0-21swapper/220:35:352
422099552,5cyclictest7750irq/56-dwc_otg22:00:342
422099552,5cyclictest7750irq/56-dwc_otg00:25:342
421999559,30cyclictest0-21swapper/121:40:371
422199549,33cyclictest0-21swapper/319:40:343
422099542,9cyclictest0-21swapper/219:20:332
421999549,29cyclictest0-21swapper/119:30:291
422199533,9cyclictest7950irq/56-dwc_otg_20:35:323
421999537,30cyclictest0-21swapper/119:25:341
422099526,5cyclictest7950irq/56-dwc_otg_23:00:342
421999527,37cyclictest0-21swapper/100:20:351
421999527,37cyclictest0-21swapper/100:20:351
421999527,29cyclictest0-21swapper/120:40:351
421999527,29cyclictest0-21swapper/120:40:351
4219995234,8cyclictest7950irq/56-dwc_otg_00:15:491
422099518,27cyclictest0-21swapper/219:50:362
421999518,29cyclictest0-21swapper/119:15:361
421999518,27cyclictest0-21swapper/119:50:361
421999517,27cyclictest7850irq/56-dwc_otg_23:15:351
66312500,7sleep1421999cyclictest22:20:491
66312500,7sleep1421999cyclictest22:20:491
422099508,25cyclictest0-21swapper/220:30:282
422199498,26cyclictest0-21swapper/322:50:363
422199498,26cyclictest0-21swapper/322:50:363
422099492,5cyclictest7750irq/56-dwc_otg23:30:332
421999498,27cyclictest0-21swapper/121:30:351
421999498,26cyclictest0-21swapper/123:05:361
421999498,26cyclictest0-21swapper/123:05:361
421999496,27cyclictest7750irq/56-dwc_otg22:42:381
421999492,35cyclictest0-21swapper/123:20:311
421999492,35cyclictest0-21swapper/123:20:311
29162490,5sleep30-21swapper/319:05:433
29162490,5sleep30-21swapper/319:05:433
421999488,24cyclictest0-21swapper/123:35:361
4219994816,5cyclictest15358-21ls21:10:221
422199478,27cyclictest0-21swapper/321:25:283
422099478,28cyclictest0-21swapper/221:25:282
4220994717,11cyclictest7950irq/56-dwc_otg_19:35:332
360524710,29sleep00-21swapper/019:06:010
360524710,29sleep00-21swapper/019:06:010
153482470,3sleep115337-21ssh00:35:491
4221994625,12cyclictest24606-21ssh22:44:483
422099468,27cyclictest0-21swapper/221:55:352
422099467,8cyclictest7850irq/56-dwc_otg_20:00:342
422099467,8cyclictest7850irq/56-dwc_otg_20:00:342
4219994621,6cyclictest4638-21switchtime23:40:501
422199458,11cyclictest0-21swapper/323:52:343
422199451,26cyclictest5954-21ls19:15:223
422099456,24cyclictest29-21rcuc/223:55:382
422099453,3cyclictest0-21swapper/220:40:342
422099453,3cyclictest0-21swapper/220:40:342
421999458,29cyclictest0-21swapper/121:00:351
421999453,30cyclictest0-21swapper/122:10:221
4219994519,10cyclictest1289-21/usr/sbin/munin22:55:461
421999451,28cyclictest8275-21ls23:05:231
159482450,2sleep1221ktimersoftd/123:15:211
422199448,5cyclictest14747-21ls21:50:363
4221994420,5cyclictest631-21apt-get20:30:223
4221994420,5cyclictest631-21apt-get20:30:223
422099446,27cyclictest27279-21ssh00:10:362
422099444,18cyclictest29-21rcuc/221:35:492
422099444,18cyclictest29-21rcuc/221:35:492
4220994428,6cyclictest14709-21ssh23:54:392
422099441,24cyclictest7850irq/56-dwc_otg_22:55:542
421999446,26cyclictest7750irq/56-dwc_otg22:26:361
421999444,25cyclictest23-21ksoftirqd/122:30:331
4219994419,6cyclictest23607-21unixbench_multi22:00:511
4219994414,6cyclictest21-21rcuc/100:05:301
4219994414,6cyclictest21-21rcuc/100:05:301
421999441,30cyclictest1448-21ls22:15:221
422199438,6cyclictest7950irq/56-dwc_otg_00:05:363
422199438,6cyclictest7950irq/56-dwc_otg_00:05:363
4220994313,16cyclictest0-21swapper/220:25:352
4220994313,16cyclictest0-21swapper/220:25:352
421999436,29cyclictest27374-21ssh21:25:281
4219994319,5cyclictest16748-21sh19:45:221
4219994318,6cyclictest7750irq/56-dwc_otg23:25:371
4219994312,7cyclictest7750irq/56-dwc_otg22:15:561
422199425,22cyclictest26322-21ssh22:05:353
4221994216,16cyclictest381ktimersoftd/321:43:013
422099426,18cyclictest29-21rcuc/200:24:202
422099426,18cyclictest29-21rcuc/200:24:202
422099424,25cyclictest7750irq/56-dwc_otg21:40:462
422099421,11cyclictest17963-21ls22:35:432
422099421,11cyclictest17963-21ls22:35:432
421999427,20cyclictest0-21swapper/120:35:351
421999421,4cyclictest0-21swapper/100:30:391
422199418,29cyclictest0-21swapper/320:15:283
422199412,5cyclictest0-21swapper/300:25:473
422199411,27cyclictest19391-21missed_timers19:50:453
422199411,25cyclictest8122-21latency_hist23:05:233
422099419,19cyclictest0-21swapper/219:25:342
422099418,20cyclictest29-21rcuc/221:30:352
422099416,18cyclictest29-21rcuc/200:00:462
4220994125,10cyclictest301ktimersoftd/219:55:332
4220994122,7cyclictest29789-21grep20:20:362
4220994115,12cyclictest7750irq/56-dwc_otg21:45:522
421999419,28cyclictest0-21swapper/120:30:281
421999416,28cyclictest14577-21ssh21:50:351
4219994118,12cyclictest28429-21grep00:10:521
4219994113,6cyclictest15681-21aten_rbpower_vo21:10:371
422199408,20cyclictest0-21swapper/320:45:363
422199402,17cyclictest7950irq/56-dwc_otg_21:25:233
422199401,28cyclictest13414-21ls21:05:233
422099408,19cyclictest31-21ksoftirqd/221:25:222
422099408,15cyclictest7850irq/56-dwc_otg_00:35:372
422099407,18cyclictest29-21rcuc/219:15:442
422099406,25cyclictest6837-21ssh22:20:512
422099406,25cyclictest6837-21ssh22:20:512
422099406,17cyclictest29-21rcuc/223:24:332
422099406,17cyclictest29-21rcuc/223:24:332
422099405,23cyclictest29-21rcuc/222:27:102
4220994023,6cyclictest17908-21ssh21:12:582
4220994020,9cyclictest7750irq/56-dwc_otg00:19:562
422099401,27cyclictest16694-21ls23:15:422
421999404,24cyclictest7750irq/56-dwc_otg22:53:191
421999404,24cyclictest7750irq/56-dwc_otg22:53:191
4219994026,7cyclictest23-21ksoftirqd/121:45:581
422199398,28cyclictest0-21swapper/321:30:353
422199396,7cyclictest7750irq/56-dwc_otg19:15:413
422199392,18cyclictest0-21swapper/300:35:483
4221993918,5cyclictest4909-21unixbench_singl23:40:523
422199391,28cyclictest24556-21ls20:05:403
422199391,25cyclictest21507-21timerandwakeup19:55:493
422199391,21cyclictest11498-21apt-key19:30:243
422099397,20cyclictest301ktimersoftd/223:29:232
422099396,5cyclictest7950irq/56-dwc_otg_19:40:332
422099396,22cyclictest23295-21ssh00:05:292
422099396,22cyclictest23295-21ssh00:05:292
422099393,23cyclictest29-21rcuc/222:35:012
422099392,21cyclictest7850irq/56-dwc_otg_19:11:062
4220993916,11cyclictest22893-21sh21:20:012
422099391,27cyclictest30205-21timerandwakeup22:50:502
422099391,27cyclictest30205-21timerandwakeup22:50:502
421999392,26cyclictest18529-21timerwakeupswit22:35:521
421999392,26cyclictest18529-21timerwakeupswit22:35:521
28452396,20sleep22915-21/usr/sbin/munin19:05:432
28452396,20sleep22915-21/usr/sbin/munin19:05:432
177562390,4sleep3381ktimersoftd/323:57:553
153712390,2sleep30-21swapper/323:14:153
422199386,19cyclictest27726-21http23:30:223
422199386,16cyclictest7950irq/56-dwc_otg_23:15:463
422199382,8cyclictest5332-21munin-run22:20:223
4221993823,10cyclictest0-21swapper/321:15:413
4221993815,5cyclictest3505-21missed_timers21:35:453
4221993815,5cyclictest3505-21missed_timers21:35:453
422099389,20cyclictest7850irq/56-dwc_otg_20:15:232
422099389,16cyclictest29-21rcuc/221:04:442
422099389,10cyclictest29-21rcuc/222:06:532
422099382,27cyclictest10087-21aten_rbpower_en20:55:372
4220993818,6cyclictest10496-21/usr/sbin/munin00:30:512
422099381,20cyclictest7950irq/56-dwc_otg_19:45:452
422099381,18cyclictest29-21rcuc/223:48:242
422099381,18cyclictest29-21rcuc/223:48:242
421999385,5cyclictest5817-21munin-run19:15:211
421999384,5cyclictest0-21swapper/120:05:321
127252381,33sleep30-21swapper/322:29:343
422199378,26cyclictest0-21swapper/320:40:353
422199378,26cyclictest0-21swapper/320:40:353
422199374,7cyclictest7750irq/56-dwc_otg21:10:433
422199371,26cyclictest1762-21ls22:55:533
4221993712,5cyclictest24349-21ssh22:02:083
4221993710,15cyclictest7950irq/56-dwc_otg_22:35:583
4221993710,15cyclictest7950irq/56-dwc_otg_22:35:583
422099378,17cyclictest0-21swapper/220:15:292
422099375,21cyclictest3588-21ssh23:40:282
421999373,22cyclictest24105-21wc20:05:231
421999373,22cyclictest24105-21wc20:05:231
421999372,4cyclictest0-21swapper/120:55:421
4219993715,13cyclictest0-21swapper/122:45:361
422199369,13cyclictest7850irq/56-dwc_otg_20:21:103
422199368,6cyclictest7850irq/56-dwc_otg_23:20:393
422199368,6cyclictest7850irq/56-dwc_otg_23:20:393
422199368,17cyclictest0-21swapper/320:50:363
422199367,16cyclictest7950irq/56-dwc_otg_19:25:233
422199362,23cyclictest0-21swapper/300:20:373
422199362,23cyclictest0-21swapper/300:20:373
422199362,19cyclictest7950irq/56-dwc_otg_23:35:413
422199361,5cyclictest23150-21ssh00:05:243
4221993614,8cyclictest13120-21sh00:33:273
4221993610,5cyclictest7950irq/56-dwc_otg_00:15:393
422099362,10cyclictest24233-21grep20:05:352
421999368,16cyclictest0-21swapper/123:55:351
421999365,24cyclictest18503-21ssh21:55:351
421999365,14cyclictest7850irq/56-dwc_otg_19:40:231
421999364,7cyclictest0-21swapper/100:00:531
421999363,23cyclictest7850irq/56-dwc_otg_20:10:431
4219993614,5cyclictest1180-21runrttasks20:21:191
421999361,25cyclictest9499-21ls19:25:221
4219993610,18cyclictest7602-21ssh23:45:361
4219993610,18cyclictest7602-21ssh23:45:361
4221993527,6cyclictest20348-21rm21:56:583
422199352,22cyclictest0-21swapper/320:03:593
422199352,22cyclictest0-21swapper/320:03:593
422199351,9cyclictest30715-21if_wlan022:10:423
422199351,4cyclictest7950irq/56-dwc_otg_21:05:353
422199351,22cyclictest0-21swapper/320:12:413
422199351,20cyclictest6048-21ls22:20:403
422199351,20cyclictest6048-21ls22:20:403
421999355,12cyclictest7850irq/56-dwc_otg_20:45:531
4219993518,7cyclictest7950irq/56-dwc_otg_21:20:441
4219993510,4cyclictest0-21swapper/121:15:371
422199349,11cyclictest7950irq/56-dwc_otg_19:25:493
422199343,20cyclictest13851-21sed19:35:423
422199343,11cyclictest7750irq/56-dwc_otg23:45:453
422199343,11cyclictest7750irq/56-dwc_otg23:45:453
422199342,22cyclictest0-21swapper/320:58:373
422099346,18cyclictest29-21rcuc/220:49:342
422099342,17cyclictest13617-21/usr/sbin/munin21:05:442
421999347,12cyclictest7850irq/56-dwc_otg_21:35:231
421999347,12cyclictest7850irq/56-dwc_otg_21:35:231
421999346,17cyclictest0-21swapper/119:45:401
4219993426,5cyclictest20705-21wget19:55:361
421999341,6cyclictest32184-21grep20:25:451
421999341,6cyclictest32184-21grep20:25:451
421999341,21cyclictest7750irq/56-dwc_otg20:15:461
422199338,15cyclictest7950irq/56-dwc_otg_19:45:433
422199334,14cyclictest0-21swapper/322:35:223
422199331,6cyclictest27249-21apt-get00:10:363
4219993314,5cyclictest8442-21/usr/sbin/munin20:50:401
422199322,22cyclictest0-21swapper/323:06:163
422199322,22cyclictest0-21swapper/323:06:163
4221993220,9cyclictest25513-21fschecks_count22:45:403
422099322,6cyclictest0-21swapper/220:50:472
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional