You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-16 - 10:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Thu Jul 16, 2026 00:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1076224320,3sleep20-21swapper/219:05:412
12015991353,52cyclictest10303-21dhclient-script00:10:350
120159911851,46cyclictest25588-21latency_hist23:50:190
120159911446,32cyclictest12260-21sh22:15:250
12015991123,86cyclictest727-21runrttasks23:30:380
12015991055,77cyclictest0-21swapper/022:30:520
12015991054,63cyclictest0-21swapper/022:50:190
12015991043,80cyclictest29488-21sed22:35:410
12015991033,81cyclictest0-21swapper/023:35:230
120159910315,62cyclictest0-21swapper/021:35:360
120159910015,56cyclictest0-21swapper/022:20:320
12015999923,55cyclictest1087-21grep22:00:490
1201599938,67cyclictest9-21ksoftirqd/022:50:200
1201599913,72cyclictest0-21swapper/000:15:410
1201599909,61cyclictest0-21swapper/022:40:480
12015999012,59cyclictest17925-21diskstats23:40:360
12015998847,25cyclictest7950irq/56-dwc_otg_00:05:390
12015998847,25cyclictest7950irq/56-dwc_otg_00:05:390
12015998534,25cyclictest0-21swapper/000:20:460
12018998425,35cyclictest0-21swapper/300:05:413
12018998425,35cyclictest0-21swapper/300:05:413
1201599846,63cyclictest0-21swapper/022:55:480
1201599846,63cyclictest0-21swapper/022:55:480
12015998224,41cyclictest9-21ksoftirqd/023:27:020
12018998017,33cyclictest0-21swapper/322:20:323
1201599802,60cyclictest0-21swapper/023:50:210
1201699793,51cyclictest0-21swapper/123:50:191
1201899773,53cyclictest25701-21ls23:50:193
25802760,2sleep20-21swapper/222:03:242
1201799745,27cyclictest30030-21ssh23:55:292
12017997419,18cyclictest13120-21apt-get23:35:222
12016997429,22cyclictest7850irq/56-dwc_otg22:20:321
1201799732,27cyclictest25836-21ls22:30:492
12016997313,27cyclictest9650irq/80-mmc000:05:411
12016997313,27cyclictest9650irq/80-mmc000:05:411
12016997312,36cyclictest12014-21cyclictest22:40:461
12015997315,7cyclictest0-21swapper/021:20:190
1201899714,42cyclictest29424-21sh22:35:413
1201599717,39cyclictest0-21swapper/020:30:310
1201799708,12cyclictest17921-21ssh23:40:352
1201699703,40cyclictest7850irq/56-dwc_otg23:35:211
12016997014,33cyclictest25983-21grep22:30:511
1201599701,57cyclictest18787-21grep20:55:460
1201699692,38cyclictest7950irq/56-dwc_otg_22:55:471
1201699692,38cyclictest7950irq/56-dwc_otg_22:55:471
12016996917,24cyclictest7186-21ssh23:27:001
12015996910,45cyclictest25606-21fschecks_time23:10:350
234342680,2sleep20-21swapper/222:28:362
1201599689,49cyclictest3206-21aten_rbpower_cu20:15:320
1201599682,53cyclictest0-21swapper/023:05:350
1201599681,53cyclictest21358-21fschecks_time19:35:360
12018996625,26cyclictest3550-21kworker/3:023:35:203
12017996626,20cyclictest7950irq/56-dwc_otg_23:27:022
1201699665,41cyclictest0-21swapper/122:00:481
12015996635,15cyclictest0-21swapper/021:20:210
1201599663,48cyclictest0-21swapper/020:05:470
1201599663,48cyclictest0-21swapper/020:05:470
1201599661,51cyclictest0-21swapper/020:40:320
12018996514,31cyclictest8050irq/56-dwc_otg_22:15:393
1201699654,13cyclictest31476-21ssh21:20:201
12016996518,30cyclictest12957-21ssh22:15:411
1201599651,52cyclictest7137-21cron21:30:180
1201599651,52cyclictest7137-21cron21:30:180
1201799646,7cyclictest7950irq/56-dwc_otg_21:40:312
1201699643,29cyclictest9693-21cut23:30:371
1201599641,53cyclictest0-21swapper/023:55:450
1201899632,56cyclictest31482-21ls21:20:193
12018996312,12cyclictest8050irq/56-dwc_otg_23:50:203
12017996331,15cyclictest9648-21fschecks_time23:30:372
12017996322,10cyclictest8507-21sed22:50:192
12017996314,26cyclictest22143-21fschecks_time23:45:362
12017996313,29cyclictest8050irq/56-dwc_otg_00:20:482
12017996311,27cyclictest7950irq/56-dwc_otg_22:40:472
1201699635,13cyclictest0-21swapper/121:20:211
1201799625,12cyclictest0-21swapper/222:35:422
1201799613,16cyclictest14145-21unixbench_singl22:55:482
1201799613,16cyclictest14145-21unixbench_singl22:55:482
12015996137,5cyclictest29902-21ssh23:15:380
12015996111,38cyclictest0-21swapper/020:35:360
1201799605,8cyclictest7950irq/56-dwc_otg_20:05:322
1201799605,8cyclictest7950irq/56-dwc_otg_20:05:322
1201699604,15cyclictest0-21swapper/122:35:411
12016996011,28cyclictest11990-21fschecks_time21:35:391
1201599602,38cyclictest20334-21grep19:30:460
12015996017,32cyclictest28635-21if_wlan021:55:380
12015996017,32cyclictest28635-21if_wlan021:55:380
1201599601,50cyclictest20839-21fschecks_count22:25:350
1201899596,8cyclictest7850irq/56-dwc_otg22:55:473
1201899596,8cyclictest7850irq/56-dwc_otg22:55:473
1201899594,8cyclictest7950irq/56-dwc_otg_23:30:363
1201699598,33cyclictest0-21swapper/123:50:201
12018995832,11cyclictest8617-21ssh22:50:213
12018995813,25cyclictest7950irq/56-dwc_otg_00:15:423
12017995812,29cyclictest0-21swapper/200:10:362
12016995810,31cyclictest0-21swapper/123:40:371
12016995810,21cyclictest0-21swapper/122:50:191
1201599582,46cyclictest125-21systemd-journal20:25:190
1201599582,44cyclictest21259-21wget21:05:350
1201599581,47cyclictest0-21swapper/021:40:450
1201599581,43cyclictest17277-21fschecks_count23:00:340
102622580,6sleep30-21swapper/319:05:343
12018995723,18cyclictest39-21ksoftirqd/322:00:483
1201699573,16cyclictest7850irq/56-dwc_otg00:10:321
1201599572,32cyclictest21000-21ssh21:45:460
1201799563,41cyclictest31330-21latency_hist21:20:202
1201799563,10cyclictest6596-21memory00:05:422
1201799563,10cyclictest6596-21memory00:05:422
12017995611,10cyclictest7850irq/56-dwc_otg22:15:412
12016995638,13cyclictest221ktimersoftd/100:15:411
1201599562,44cyclictest0-21swapper/023:20:380
1201599562,44cyclictest0-21swapper/023:20:380
1201899553,14cyclictest8050irq/56-dwc_otg_23:40:353
12018995523,16cyclictest31114-21kworker/3:223:27:003
1201799554,16cyclictest125-21systemd-journal21:35:392
12016995517,23cyclictest0-21swapper/121:40:311
1201599551,43cyclictest26974-21grep19:50:390
1201599551,42cyclictest4684-21awk22:05:390
1201899543,16cyclictest0-21swapper/322:30:483
12017995413,19cyclictest8050irq/56-dwc_otg_22:20:292
12016995441,9cyclictest19403-21ls00:20:471
1201899538,9cyclictest7950irq/56-dwc_otg_21:35:393
12018995336,12cyclictest727-21runrttasks22:40:473
12015995311,24cyclictest0-21swapper/019:15:300
1201599524,37cyclictest0-21swapper/021:10:310
1201599524,37cyclictest0-21swapper/021:10:310
1201599522,40cyclictest25808-21sed19:45:470
1201599521,35cyclictest2970-21grep00:00:470
12015995211,14cyclictest0-21swapper/021:50:300
237292510,2sleep30-21swapper/322:29:093
1201899514,24cyclictest0-21swapper/300:10:383
12018995114,12cyclictest7950irq/56-dwc_otg_00:20:473
1201799513,25cyclictest7850irq/56-dwc_otg00:15:412
1201699517,8cyclictest7503-21ssh21:30:321
1201699513,27cyclictest8050irq/56-dwc_otg_22:50:201
1201599517,35cyclictest21894-1kworker/0:0H00:35:250
12015995120,5cyclictest0-21swapper/022:10:390
1201599511,5cyclictest15642-21latency_hist20:50:180
1201899503,14cyclictest8440-21sed22:50:183
12015995020,5cyclictest28315-21aten_rbpower_cu19:55:320
1201599501,43cyclictest0-21swapper/019:20:400
12015995010,31cyclictest0-21swapper/020:00:290
12015994931,9cyclictest0-21swapper/021:30:430
12015994925,4cyclictest7950irq/56-dwc_otg_00:30:180
1201899483,24cyclictest0-21swapper/321:20:213
1201799483,22cyclictest0-21swapper/222:50:212
1201599481,37cyclictest0-21swapper/020:10:390
1201899476,25cyclictest0-21swapper/319:40:313
1201799477,26cyclictest0-21swapper/220:40:312
1201699478,29cyclictest0-21swapper/100:35:311
1201799468,25cyclictest0-21swapper/200:35:242
1201799467,24cyclictest0-21swapper/220:10:242
1201699467,26cyclictest0-21swapper/120:40:311
12016994623,5cyclictest23-21ksoftirqd/122:25:411
1201599462,23cyclictest0-21swapper/019:10:370
12015994616,16cyclictest0-21swapper/019:40:460
1201599461,24cyclictest20172-21wc21:00:410
12015994610,21cyclictest22793-21diskmemload00:30:260
115812462,15sleep00-21swapper/019:08:190
237062450,5sleep31201899cyclictest23:08:073
1201899457,23cyclictest0-21swapper/319:50:323
1201599451,30cyclictest0-21swapper/020:25:330
1201599451,30cyclictest0-21swapper/020:25:330
1201599451,21cyclictest18611-21grep19:25:460
1201599451,21cyclictest18611-21grep19:25:460
1201799443,14cyclictest0-21swapper/223:50:202
12017994412,4cyclictest8050irq/56-dwc_otg_20:30:292
1201799432,9cyclictest31559-21ssh21:20:212
1201799431,26cyclictest1728-21ssh00:00:312
1201599431,19cyclictest13767-21kworker/u8:020:50:320
1201899426,24cyclictest0-21swapper/319:35:323
1201799422,3cyclictest0-21swapper/220:55:302
12017994221,11cyclictest7850irq/56-dwc_otg21:30:372
1201799421,5cyclictest8050irq/56-dwc_otg_19:30:302
1201699427,25cyclictest0-21swapper/120:30:311
1201799416,13cyclictest24031-21ssh21:50:312
12017994111,15cyclictest7950irq/56-dwc_otg_22:12:252
12017994110,14cyclictest7950irq/56-dwc_otg_19:15:302
1201699418,22cyclictest23-21ksoftirqd/100:30:301
1201699417,32cyclictest3364-21ssh21:25:321
1201699417,32cyclictest3364-21ssh21:25:321
1201699417,25cyclictest0-21swapper/121:00:291
1201699417,22cyclictest0-21swapper/120:10:241
1201699416,5cyclictest21-21rcuc/119:20:181
12018994016,15cyclictest3950-21ssh22:05:283
1201799404,11cyclictest0-21swapper/200:25:292
1201799402,14cyclictest7950irq/56-dwc_otg_19:55:372
1201699408,24cyclictest0-21swapper/119:20:311
1201699405,10cyclictest7850irq/56-dwc_otg21:05:341
12016994021,5cyclictest328450wget20:15:331
1201699402,10cyclictest0-21swapper/121:50:311
1201699401,21cyclictest2507-21ssh00:00:421
262962391,4sleep20-21swapper/221:14:012
262962391,4sleep20-21swapper/221:14:012
1201899396,21cyclictest0-21swapper/323:55:303
1201899392,15cyclictest1125050ssh21:35:183
12017993910,4cyclictest7850irq/56-dwc_otg23:10:422
12017993910,16cyclictest8050irq/56-dwc_otg_21:55:482
12017993910,16cyclictest8050irq/56-dwc_otg_21:55:482
1201699394,4cyclictest7850irq/56-dwc_otg20:25:301
1201699394,4cyclictest7850irq/56-dwc_otg20:25:301
1201699393,20cyclictest0-21swapper/123:55:291
1201699392,18cyclictest0-21swapper/122:05:411
12016993915,16cyclictest21-21rcuc/120:38:491
1201899389,6cyclictest8050irq/56-dwc_otg_00:35:303
12017993819,13cyclictest7850irq/56-dwc_otg23:16:162
12017993816,4cyclictest7950irq/56-dwc_otg_19:25:462
12017993816,4cyclictest7950irq/56-dwc_otg_19:25:462
1201699386,7cyclictest23-21ksoftirqd/121:00:191
1201699383,6cyclictest23-21ksoftirqd/100:25:371
1201699381,27cyclictest14964-21timerandwakeup20:45:461
1201899377,20cyclictest0-21swapper/320:55:313
12018993719,10cyclictest3706-21sh23:23:113
12018993719,10cyclictest3706-21sh23:23:113
1201799379,5cyclictest8050irq/56-dwc_otg_20:50:362
1201799378,4cyclictest0-21swapper/200:30:372
1201799373,5cyclictest8050irq/56-dwc_otg_21:45:302
12017993710,14cyclictest8050irq/56-dwc_otg_21:25:292
12017993710,14cyclictest8050irq/56-dwc_otg_21:25:292
12016993715,8cyclictest4944-21latency_hist23:25:191
12016993715,8cyclictest4944-21latency_hist23:25:191
12016993711,12cyclictest0-21swapper/120:00:321
12018993626,4cyclictest8050irq/56-dwc_otg_21:50:353
1201899361,24cyclictest16678-21timerwakeupswit19:20:463
12018993610,5cyclictest25878-21sh00:29:393
1201799368,5cyclictest1404-21fschecks_time23:20:362
1201799368,5cyclictest1404-21fschecks_time23:20:362
1201799366,13cyclictest24911-21latency_hist23:10:192
12017993612,4cyclictest7850irq/56-dwc_otg19:20:442
12017993611,6cyclictest7850irq/56-dwc_otg19:55:182
1201899357,13cyclictest0-21swapper/323:15:353
1201899351,24cyclictest13224-21timerwakeupswit20:40:453
1201799356,5cyclictest8050irq/56-dwc_otg_23:05:182
12017993516,5cyclictest7598-21sh22:10:032
1201699359,6cyclictest0-21swapper/121:55:461
1201699359,6cyclictest0-21swapper/121:55:461
12016993521,11cyclictest0-21swapper/121:10:361
12016993521,11cyclictest0-21swapper/121:10:361
12016993512,6cyclictest277972chrt23:13:041
1201699351,26cyclictest22708-21ls19:40:181
115122350,5sleep31201899cyclictest22:14:403
12018993415,4cyclictest10690-21fschecks_time20:35:363
12018993415,10cyclictest23046-21ssh21:10:223
12018993415,10cyclictest23046-21ssh21:10:223
12018993411,5cyclictest30490-21sh21:57:403
12018993411,5cyclictest30490-21sh21:57:403
1201799347,15cyclictest0-21swapper/220:15:322
12017993418,8cyclictest7950irq/56-dwc_otg_20:00:412
12017993413,3cyclictest0-21swapper/221:00:272
1201799341,14cyclictest8050irq/56-dwc_otg_19:10:302
1201699344,8cyclictest9307-21grep22:10:471
12016993427,5cyclictest23-21ksoftirqd/119:10:461
1201699341,26cyclictest25006-21grep19:45:361
1201899333,10cyclictest8050irq/56-dwc_otg_23:00:453
1201899332,14cyclictest8050irq/56-dwc_otg_19:30:203
12018993312,14cyclictest2148-21/usr/sbin/munin00:00:363
1201799338,5cyclictest7850irq/56-dwc_otg21:05:492
1201799337,11cyclictest8050irq/56-dwc_otg_20:25:352
1201799337,11cyclictest8050irq/56-dwc_otg_20:25:352
1201799336,6cyclictest8050irq/56-dwc_otg_19:45:332
1201699336,14cyclictest7850irq/56-dwc_otg20:00:191
12016993312,16cyclictest0-21swapper/120:05:321
12016993312,16cyclictest0-21swapper/120:05:321
12016993311,5cyclictest29623-21/usr/sbin/munin23:15:351
12018993214,8cyclictest17956-21sh21:42:243
12018993214,5cyclictest15564-21cron20:50:183
12018993213,4cyclictest12324-21fschecks_time19:10:363
12018993212,6cyclictest27862-21grep00:30:463
1201799328,7cyclictest21667-21missed_timers19:35:402
1201799325,13cyclictest8050irq/56-dwc_otg_20:45:422
1201799322,21cyclictest0-21swapper/220:21:342
1201799321,17cyclictest0-21swapper/219:40:552
1201699326,15cyclictest0-21swapper/123:00:301
1201699324,4cyclictest7950irq/56-dwc_otg_23:10:181
1201899318,6cyclictest8050irq/56-dwc_otg_20:30:463
1201899317,6cyclictest7950irq/56-dwc_otg_20:25:193
1201899315,7cyclictest25828-21unixbench_singl19:45:473
1201899312,6cyclictest8050irq/56-dwc_otg_21:45:323
1201899312,5cyclictest7950irq/56-dwc_otg_23:10:383
12018993121,4cyclictest20122-21ntp_states21:00:403
1201899311,3cyclictest0-21swapper/319:30:323
1201699315,3cyclictest0-21swapper/121:45:301
116542315,16sleep10-21swapper/119:09:171
1201899304,5cyclictest28317-21grep19:55:323
1201899302,6cyclictest7950irq/56-dwc_otg_20:15:343
12018993014,9cyclictest727-21runrttasks19:26:293
1201799301,6cyclictest0-21swapper/220:39:002
1201699301,15cyclictest7850irq/56-dwc_otg19:25:381
1201699301,15cyclictest7850irq/56-dwc_otg19:25:381
1201899297,13cyclictest0-21swapper/321:10:193
1201899295,5cyclictest8050irq/56-dwc_otg_20:10:343
1201899291,24cyclictest14130-21cut19:15:333
1201899291,10cyclictest7176-21ssh21:30:193
1201899291,10cyclictest7176-21ssh21:30:193
1201699297,11cyclictest23197-21fschecks_time19:40:351
1201699292,7cyclictest5275-21fschecks_time20:20:351
1201899281,4cyclictest0-21swapper/320:05:463
1201899281,4cyclictest0-21swapper/320:05:463
1201699284,19cyclictest16919-21tail20:50:481
1201699283,6cyclictest7850irq/56-dwc_otg19:30:331
1201699283,5cyclictest7850irq/56-dwc_otg19:50:361
1201899274,7cyclictest8050irq/56-dwc_otg_20:50:213
12018992721,4cyclictest8493-21latency_hist20:30:193
12018992721,4cyclictest8493-21latency_hist20:30:193
1201899251,11cyclictest8050irq/56-dwc_otg_20:05:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional