You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-19 - 00:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Sat Jul 18, 2026 12:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2661821770,7sleep21470499cyclictest11:12:202
277821750,2sleep31470599cyclictest08:05:413
14702991513,125cyclictest0-21swapper/010:55:370
14702991252,61cyclictest7950irq/56-dwc_otg_11:00:470
14705991176,85cyclictest17126-21/usr/sbin/munin11:00:473
14705991042,75cyclictest12335-21fschecks_time10:55:403
14704999911,56cyclictest12239-21diskstats10:55:382
1470299963,77cyclictest0-21swapper/008:55:200
27852950,4sleep314701-21cyclictest09:20:543
50632930,3sleep21470499cyclictest12:05:302
50632930,3sleep21470499cyclictest12:05:302
1470299932,72cyclictest22416-21grep09:45:510
296852920,2sleep30-21swapper/307:50:403
296852920,2sleep30-21swapper/307:50:403
1470299921,71cyclictest125-21systemd-journal10:39:440
14702999024,49cyclictest21263-21switchtime10:25:490
1470299901,5cyclictest12473-21runrttasks09:34:280
1470499875,59cyclictest17038-21open_inodes11:00:452
1470299877,64cyclictest0-21swapper/010:34:190
1470299871,4cyclictest4939-21timerwakeupswit08:10:490
1470299861,5cyclictest21584-21sed11:05:490
1470299861,4cyclictest1421-21ssh09:20:300
1470299852,70cyclictest61-21kswapd011:50:310
1470299852,65cyclictest0-21swapper/008:55:400
1470299851,67cyclictest16750-21ssh09:40:050
1470299851,67cyclictest16750-21ssh09:40:050
1470299851,5cyclictest0-21swapper/010:10:150
1470299851,5cyclictest0-21swapper/009:26:090
1470299821,6cyclictest25608-21tr07:40:340
1470299821,63cyclictest14941-21taskset10:18:190
1470299821,4cyclictest16520-21wc11:40:200
1470299812,64cyclictest0-21swapper/009:42:480
1470299811,7cyclictest8711-21cut10:10:390
139592810,5sleep313965-21netstat09:35:413
139592810,5sleep313965-21netstat09:35:413
1470299803,65cyclictest19246-21ssh10:24:000
1470299801,4cyclictest14255-21sed08:35:460
1470299796,63cyclictest0-21swapper/009:59:250
1470299792,9cyclictest0-21swapper/009:54:090
1470299791,7cyclictest21269-21tune2fs11:45:380
1470299788,61cyclictest2812-21ssh12:01:510
1470299781,59cyclictest7488-21rm11:28:590
1470299772,66cyclictest0-21swapper/009:18:080
1470299772,58cyclictest18271-21kworker/u8:109:12:280
1470299771,63cyclictest30318-21sed11:55:460
1470299769,51cyclictest8729-21ssh12:10:070
1470299769,51cyclictest8729-21ssh12:10:070
14702997613,47cyclictest23543-21chrt07:35:200
1470299759,53cyclictest30471-21sh11:16:500
1470299757,52cyclictest11425-21ssh12:12:300
1470299752,4cyclictest0-21swapper/010:55:210
1470299751,56cyclictest24087-21fschecks_time07:35:370
1470299741,4cyclictest0-21swapper/008:05:370
1470299731,5cyclictest14603-21sed12:15:490
1470299731,5cyclictest0-21swapper/010:02:040
1470299731,57cyclictest1273-21ssh12:40:180
14702997311,45cyclictest0-21swapper/010:41:320
77332720,1sleep10-21swapper/111:29:261
302702720,3sleep31470599cyclictest11:16:283
235072720,2sleep0101ktimersoftd/012:26:540
1470299721,6cyclictest15940-21taskset07:11:030
14702997211,49cyclictest2164-21ssh11:21:480
116022720,2sleep20-21swapper/210:55:172
275992710,2sleep20-21swapper/212:31:392
1470299712,8cyclictest0-21swapper/008:30:320
1470299711,5cyclictest17509-21cat11:40:420
1470299711,5cyclictest17509-21cat11:40:420
1470499701,51cyclictest3559-21ssh11:24:222
1470299705,56cyclictest18882-21ssh12:21:010
1470299701,54cyclictest10260-21ssh11:31:300
1470499691,41cyclictest10951-21ssh10:13:162
1470499691,38cyclictest23318-21ssh10:29:172
14702996926,24cyclictest8050irq/56-dwc_otg_08:05:190
1470299681,4cyclictest27387-21ssh11:13:440
317762670,3sleep331745-21ssh10:40:203
1470299672,4cyclictest0-21swapper/007:59:440
1470299671,55cyclictest28636-21ssh12:33:330
14702996710,46cyclictest25606-21diskmemload10:49:470
1470599665,5cyclictest7850irq/56-dwc_otg09:10:523
14702996620,26cyclictest7950irq/56-dwc_otg_08:30:200
1470499651,50cyclictest30067-21ssh10:37:012
1470499651,40cyclictest6832-21ssh09:26:062
1470499651,40cyclictest14899-21ssh10:18:132
1470399653,14cyclictest17142-21sensors11:00:471
1470299651,4cyclictest0-21swapper/008:41:120
1470599644,12cyclictest8050irq/56-dwc_otg_09:15:303
14705996419,6cyclictest17087-21sh10:20:433
1470499641,51cyclictest23585-21ssh09:48:032
1470499641,50cyclictest32225-21sh09:17:472
1470299642,6cyclictest0-21swapper/007:24:290
1470299642,5cyclictest0-21swapper/008:19:070
1470299641,50cyclictest21468-21chrt07:27:140
1470499631,49cyclictest29418-21ssh09:15:032
1470499631,48cyclictest27755-21ssh09:53:312
1470299632,48cyclictest18461-21cat08:50:210
1470299632,48cyclictest18461-21cat08:50:210
1470599628,15cyclictest8050irq/56-dwc_otg_10:15:473
14704996218,28cyclictest0-21swapper/211:50:412
1470499621,45cyclictest4079-21ssh10:05:142
1470499621,40cyclictest4931-21ls08:10:492
1470299621,7cyclictest0-21swapper/009:01:200
1470299621,47cyclictest0-21swapper/009:06:380
1470499611,41cyclictest5759-21vmstat11:25:492
1470499611,39cyclictest19657-21ssh09:43:042
1470499611,38cyclictest30704-21ssh09:56:342
1470499611,38cyclictest25783-21df_inode07:40:362
1470299612,4cyclictest0-21swapper/007:48:420
1470299611,48cyclictest30418-21chrt07:51:300
1470299611,48cyclictest30418-21chrt07:51:300
1470299611,46cyclictest0-21swapper/007:16:290
1470499601,47cyclictest11183-21ssh11:33:122
1470499601,39cyclictest1655-21ssh09:20:362
1470499601,39cyclictest12315-21ssh09:34:092
1470399602,8cyclictest12231-21ssh10:55:371
1470599597,13cyclictest0-21swapper/311:10:483
1470499592,35cyclictest8084-21munin-run10:10:202
1470499591,44cyclictest13315-21/usr/sbin/munin12:15:312
1470499591,43cyclictest31732-21ssh11:19:062
1470499591,40cyclictest7850irq/56-dwc_otg07:35:202
1470499591,40cyclictest21368-21/usr/sbin/munin11:45:392
1470499591,35cyclictest27152-21runrttasks10:34:032
143902592,13sleep00-21swapper/007:09:480
143902592,13sleep00-21swapper/007:09:480
1470599581,13cyclictest8050irq/56-dwc_otg_09:40:343
1470499582,32cyclictest8050irq/56-dwc_otg_11:40:462
1470499582,32cyclictest8050irq/56-dwc_otg_11:40:462
1470499581,42cyclictest0-21swapper/208:30:372
1470499581,41cyclictest11750-21rm12:13:062
1470499581,33cyclictest4590-21ssh10:45:462
1470499571,42cyclictest16733-21ssh09:40:042
1470499571,42cyclictest16733-21ssh09:40:042
1470499571,40cyclictest23183-21ssh12:26:202
1470499571,35cyclictest30870-21ssh11:56:232
1470499571,35cyclictest19017-21ssh10:23:342
1470299572,7cyclictest0-21swapper/008:24:410
1470599562,12cyclictest8050irq/56-dwc_otg_12:15:393
1470499566,39cyclictest2772-21/usr/sbin/munin08:05:412
1470499561,38cyclictest8984-21taskset08:22:162
1470599558,4cyclictest0-21swapper/309:58:163
1470499552,39cyclictest20079-21apt-get08:55:252
1470599549,4cyclictest0-21swapper/310:25:443
1470499541,37cyclictest14009-21ssh11:35:492
1470499541,37cyclictest0-21swapper/210:42:232
1470599536,5cyclictest8050irq/56-dwc_otg_09:50:433
1470499531,40cyclictest2841-21ssh12:01:542
1470499531,30cyclictest8050irq/56-dwc_otg_07:35:412
1470499522,45cyclictest16523-21/usr/sbin/munin07:15:392
1470499521,30cyclictest0-21swapper/212:35:332
1470599513,4cyclictest8050irq/56-dwc_otg_10:02:363
1470499512,45cyclictest30307-21grep07:50:482
1470499512,45cyclictest30307-21grep07:50:482
1470499511,46cyclictest21109-21timerandwakeup07:25:462
1470499511,40cyclictest0-21swapper/207:11:052
1470499511,37cyclictest8050irq/56-dwc_otg_08:02:562
14703995126,7cyclictest28997-21latency_hist07:50:201
1470499501,44cyclictest6354-21/usr/sbin/munin08:15:412
1470499501,39cyclictest21678-21ssh11:05:532
1470499501,36cyclictest14265-21users08:35:462
1470499501,34cyclictest7850irq/56-dwc_otg08:30:202
1470499501,31cyclictest0-21swapper/212:24:022
1470499491,43cyclictest19437-21ls08:50:472
1470499491,43cyclictest17375-21/usr/sbin/munin08:45:402
1470499491,43cyclictest17375-21/usr/sbin/munin08:45:402
1470499491,35cyclictest0-21swapper/207:57:012
1470499481,43cyclictest23051-21sensors09:00:472
1470499481,35cyclictest0-21swapper/207:45:592
1470499471,42cyclictest24034-21apt-get09:05:312
1470399477,20cyclictest7850irq/56-dwc_otg09:55:331
14703994721,6cyclictest505-21cron09:35:191
1470599467,36cyclictest0-21swapper/308:15:333
14705994624,6cyclictest29808-21cat12:35:203
1470399461,24cyclictest30534-21ssh09:15:411
1470599457,22cyclictest0-21swapper/307:25:323
1470499451,41cyclictest15266-21basename08:40:382
1470399458,27cyclictest0-21swapper/108:55:351
277282440,2sleep10-21swapper/110:35:101
14703994437,5cyclictest221ktimersoftd/112:30:471
1470399442,22cyclictest7850irq/56-dwc_otg10:20:301
14703994417,14cyclictest0-21swapper/112:05:371
14703994417,14cyclictest0-21swapper/112:05:371
1470499431,39cyclictest18357-21apt07:20:302
1470399438,29cyclictest0-21swapper/109:05:331
14703994316,14cyclictest25606-21diskmemload09:46:381
14703994316,14cyclictest12040-21sh10:15:201
14705994221,6cyclictest6097-21netstat09:25:423
1470399427,25cyclictest0-21swapper/111:10:341
14703994220,6cyclictest403750wget08:10:361
14703994215,18cyclictest0-21swapper/109:41:101
85402410,1sleep30-21swapper/312:09:463
85402410,1sleep30-21swapper/312:09:463
1470599416,4cyclictest4866-21ssh10:45:493
1470399412,26cyclictest28351-21cpuspeed10:35:371
1470399411,16cyclictest8050irq/56-dwc_otg_12:20:381
282662400,26sleep18050irq/56-dwc_otg_11:54:101
1470399405,17cyclictest8050irq/56-dwc_otg_08:20:261
1470399402,7cyclictest32095-21ssh10:40:321
14703994016,5cyclictest14459-21grep09:35:481
14703994016,5cyclictest14459-21grep09:35:481
1470399401,30cyclictest30842-21latency_hist07:55:201
1470399401,30cyclictest30842-21latency_hist07:55:201
1470399394,19cyclictest8050irq/56-dwc_otg_09:30:201
1470399393,20cyclictest8050irq/56-dwc_otg_09:10:441
14703993928,7cyclictest25117-21aten_rbpower_cu09:50:331
14703993913,6cyclictest8050irq/56-dwc_otg_11:45:331
34502380,2sleep10-21swapper/109:22:131
1470399387,25cyclictest0-21swapper/107:10:351
1470399381,26cyclictest20642-21uniq10:25:401
1470399380,3cyclictest8050irq/56-dwc_otg_11:30:311
1470599376,5cyclictest8050irq/56-dwc_otg_09:00:353
14705993718,5cyclictest727-21runrttasks11:33:003
1470399376,25cyclictest0-21swapper/108:45:321
1470399376,25cyclictest0-21swapper/108:45:321
14703993726,8cyclictest23554-21latency_hist07:35:201
14703993713,11cyclictest23-21ksoftirqd/110:15:361
1470399370,24cyclictest0-21swapper/110:02:261
46972362,5sleep34706-21diskstats10:05:363
1470599366,15cyclictest7950irq/56-dwc_otg_09:45:323
14705993631,3cyclictest381ktimersoftd/307:15:303
14705993620,5cyclictest7850irq/56-dwc_otg12:05:203
14705993617,5cyclictest727-21runrttasks11:50:363
14705993613,9cyclictest8050irq/56-dwc_otg_07:15:203
1470399368,16cyclictest7950irq/56-dwc_otg_12:10:501
1470399367,15cyclictest8050irq/56-dwc_otg_11:10:211
1470399367,13cyclictest7850irq/56-dwc_otg10:50:321
1470399362,3cyclictest8050irq/56-dwc_otg_08:25:321
1470399361,5cyclictest5251-21ssh10:05:441
1470399361,5cyclictest17018-21awk11:40:351
1470399361,5cyclictest17018-21awk11:40:351
1470599355,20cyclictest9226-21ssh12:10:353
1470599351,14cyclictest8050irq/56-dwc_otg_08:40:203
1470399354,4cyclictest8050irq/56-dwc_otg_08:00:331
1470399353,9cyclictest7850irq/56-dwc_otg07:20:311
14703993510,12cyclictest7850irq/56-dwc_otg08:10:191
1470599342,23cyclictest8050irq/56-dwc_otg_07:20:313
1470599342,21cyclictest0-21swapper/307:30:543
1470599342,20cyclictest17591-21switchtime08:45:443
1470599342,20cyclictest17591-21switchtime08:45:443
1470599342,13cyclictest8050irq/56-dwc_otg_09:32:503
14705993410,6cyclictest7850irq/56-dwc_otg10:30:383
1470399349,4cyclictest0-21swapper/112:28:531
1470399349,13cyclictest0-21swapper/110:45:501
14703993416,5cyclictest25606-21diskmemload11:55:301
14703993410,4cyclictest0-21swapper/112:20:121
1470599333,8cyclictest2527-21ssh11:22:293
14705993312,4cyclictest14963-21grep08:40:343
1470399337,17cyclictest0-21swapper/109:00:351
1470399337,12cyclictest0-21swapper/112:35:441
14703993312,5cyclictest28890-21fschecks_count11:15:381
1470599328,5cyclictest0-21swapper/311:40:493
1470599328,5cyclictest0-21swapper/311:40:493
1470599327,6cyclictest9650irq/80-mmc012:00:203
1470599325,5cyclictest8050irq/56-dwc_otg_11:35:453
1470599323,15cyclictest8050irq/56-dwc_otg_08:55:193
14705993210,4cyclictest0-21swapper/311:28:593
14705993210,4cyclictest0-21swapper/310:12:113
14705993210,11cyclictest8050irq/56-dwc_otg_10:40:463
1470399325,14cyclictest7950irq/56-dwc_otg_08:00:191
1470399321,5cyclictest0-21swapper/112:05:091
1470599317,5cyclictest18377-21/usr/sbin/munin12:20:463
1470599313,12cyclictest7950irq/56-dwc_otg_12:29:163
14705993118,5cyclictest11041-21sh08:30:203
1470599311,16cyclictest0-21swapper/310:55:013
1470599311,11cyclictest8050irq/56-dwc_otg_11:09:133
14705993110,4cyclictest0-21swapper/312:39:073
1470399315,17cyclictest8050irq/56-dwc_otg_07:35:331
1470399312,5cyclictest0-21swapper/111:35:481
1470399312,11cyclictest20307-21latency_hist08:55:201
1470399311,21cyclictest11773-21latency08:30:411
1470599308,4cyclictest0-21swapper/311:49:403
1470599306,6cyclictest39-21ksoftirqd/309:05:383
1470599305,5cyclictest21565-21grep08:55:513
1470599303,4cyclictest7850irq/56-dwc_otg07:55:313
1470599303,12cyclictest7950irq/56-dwc_otg_07:40:343
1470599302,19cyclictest8850-21sed08:20:483
1470599301,18cyclictest4971-21ls08:10:493
1470599301,16cyclictest304-21/usr/sbin/munin08:00:493
1470399307,7cyclictest0-21swapper/111:20:371
1470399302,2cyclictest0-21swapper/108:15:321
130952305,15sleep10-21swapper/107:05:361
130952305,15sleep10-21swapper/107:05:361
128922305,17sleep212893-21aten_rbpower_cu07:05:332
128922305,17sleep212893-21aten_rbpower_cu07:05:332
1470599294,12cyclictest0-21swapper/307:45:313
1470399298,6cyclictest7850irq/56-dwc_otg07:25:361
14703992911,4cyclictest727-21runrttasks07:15:201
1470599283,4cyclictest7850irq/56-dwc_otg08:35:203
1470599281,20cyclictest0-21swapper/307:37:513
1470399289,11cyclictest0-21swapper/107:40:341
1470399286,4cyclictest7850irq/56-dwc_otg08:35:461
1470399281,19cyclictest16469-21latency_hist08:45:211
132262287,10sleep30-21swapper/307:05:383
132262287,10sleep30-21swapper/307:05:383
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional