You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-29 - 11:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Sat Aug 29, 2026 00:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3198321100,2sleep10-21swapper/122:54:311
2335221010,2sleep30-21swapper/323:22:313
2335221010,2sleep30-21swapper/323:22:313
155182860,2sleep20-21swapper/200:30:582
31957997913,40cyclictest5704-21grep23:00:421
3195799725,8cyclictest7750irq/56-dwc_otg20:55:261
31957997211,29cyclictest17460-21ssh23:55:241
31954997211,25cyclictest19069-21sh21:20:210
133332710,2sleep10-21swapper/122:30:431
133332710,2sleep10-21swapper/122:30:431
3195799708,36cyclictest7950irq/56-dwc_otg_22:20:371
3195799706,47cyclictest0-21swapper/121:49:111
3195499702,31cyclictest7750irq/56-dwc_otg20:10:240
32422690,4sleep33240-21ssh21:40:253
3195799696,46cyclictest7750irq/56-dwc_otg20:10:371
3195799696,38cyclictest7750irq/56-dwc_otg00:35:291
3195799691,7cyclictest7750irq/56-dwc_otg22:00:301
3195799683,51cyclictest7850irq/56-dwc_otg_19:35:401
3195799671,54cyclictest21123-21/usr/sbin/munin22:40:351
3195799671,47cyclictest25463-21apt-config00:05:121
3195499672,37cyclictest0-21swapper/000:00:230
129122670,2sleep10-21swapper/121:12:251
195182660,2sleep30-21swapper/322:39:193
3195799658,40cyclictest2794-21cron21:40:121
31957996513,37cyclictest8841-21cpu21:05:251
31954996516,28cyclictest101ktimersoftd/020:00:230
31954996516,28cyclictest101ktimersoftd/020:00:230
3195799647,36cyclictest0-21swapper/123:14:121
3195799638,32cyclictest0-21swapper/122:55:391
3195799634,38cyclictest7950irq/56-dwc_otg_00:30:231
3195799633,3cyclictest7950irq/56-dwc_otg_20:05:271
3195799631,7cyclictest31517-21systemd-journal00:05:131
3195799631,7cyclictest31517-21systemd-journal00:05:131
3195499633,32cyclictest32709-21ssh22:55:240
3195499632,8cyclictest29909-21ssh00:10:240
3195799627,38cyclictest0-21swapper/123:05:311
3195799627,38cyclictest0-21swapper/122:35:531
15862610,1sleep10-21swapper/122:16:551
3195799603,4cyclictest0-21swapper/123:15:401
3195799603,43cyclictest0-21swapper/100:30:131
3195799603,35cyclictest0-21swapper/100:15:411
31957996026,16cyclictest14934-21ssh21:15:231
3195799601,6cyclictest23453-21ssh21:25:291
3195799601,4cyclictest7750irq/56-dwc_otg23:40:291
3195799601,43cyclictest17991-21ssh21:57:471
3195499602,6cyclictest9450irq/48-DMA08:56:170
31954996017,23cyclictest0-21swapper/021:50:230
3195799594,3cyclictest7950irq/56-dwc_otg_19:55:261
3195799592,46cyclictest0-21swapper/120:25:221
3195499597,11cyclictest0-21swapper/019:45:240
3195499596,12cyclictest0-21swapper/022:05:240
3195799584,3cyclictest0-21swapper/121:50:531
3195799581,44cyclictest0-21swapper/122:26:411
3195499587,25cyclictest0-21swapper/021:15:180
3195499584,36cyclictest0-21swapper/023:55:240
3195999571,25cyclictest7950irq/56-dwc_otg_20:55:262
3195799571,8cyclictest21387-21ssh21:22:281
3195799571,43cyclictest0-21swapper/123:39:421
3195799571,2cyclictest0-21swapper/123:45:361
199262570,8sleep33196099cyclictest23:18:483
3195799561,4cyclictest0-21swapper/123:23:271
3195799561,4cyclictest0-21swapper/123:23:271
3195799561,39cyclictest7750irq/56-dwc_otg23:30:301
3195499569,11cyclictest0-21swapper/019:30:240
31954995616,29cyclictest1180-21runrttasks19:40:390
3195499561,31cyclictest0-21swapper/019:25:230
75442550,1sleep30-21swapper/321:45:283
3195799558,35cyclictest24265-1kworker/u9:223:26:551
3195799552,5cyclictest3927-21ntp_states21:40:361
31957995518,34cyclictest27903-21sh22:49:351
3195799551,4cyclictest0-21swapper/119:32:501
3195499555,9cyclictest0-21swapper/023:00:250
31954995532,11cyclictest23374-21fschecks_time20:15:300
140002550,2sleep0101ktimersoftd/000:30:190
43052540,2sleep00-21swapper/022:20:320
3195799544,6cyclictest0-21swapper/121:00:221
3195799543,36cyclictest0-21swapper/123:50:381
3195799543,36cyclictest0-21swapper/123:50:381
3195799542,8cyclictest30173-21latency_hist20:35:131
3195799541,5cyclictest28695-21ssh22:10:351
3195799541,4cyclictest7750irq/56-dwc_otg20:16:241
3195799541,43cyclictest0-21swapper/100:12:451
3195499544,34cyclictest0-21swapper/023:50:230
3195499544,34cyclictest0-21swapper/023:50:230
3195499541,32cyclictest13807-21apt-key00:30:130
31954995410,7cyclictest0-21swapper/021:40:220
31954995410,6cyclictest0-21swapper/019:35:230
3195799531,4cyclictest6878-21ls00:20:401
3195499536,18cyclictest10330-21ls21:10:120
31954995332,14cyclictest8958-21ls23:45:130
3195499532,32cyclictest0-21swapper/023:08:020
3196099522,11cyclictest7750irq/56-dwc_otg00:10:243
3195799522,7cyclictest7850irq/56-dwc_otg_21:31:361
3195799521,5cyclictest31465-21users20:35:401
3195799521,5cyclictest2725-21taskset20:46:311
3195799521,38cyclictest0-21swapper/120:00:261
3195799521,38cyclictest0-21swapper/120:00:261
3195499523,26cyclictest0-21swapper/020:40:260
3195499522,7cyclictest25791-21ssh00:05:240
3195499522,7cyclictest25791-21ssh00:05:240
31954995223,4cyclictest0-21swapper/022:25:410
31954995223,16cyclictest16952-21sh23:15:260
3195799514,35cyclictest7850irq/56-dwc_otg_19:52:401
3195799512,5cyclictest0-21swapper/120:22:281
3195799511,6cyclictest1214-21latency_hist20:45:121
3195499518,21cyclictest0-21swapper/021:28:330
3195499516,4cyclictest0-21swapper/022:45:260
3195499513,26cyclictest6176-21unixbench_multi20:55:450
3195499512,31cyclictest30780-21latency20:35:310
3195499512,29cyclictest0-21swapper/022:35:400
31954995117,23cyclictest0-21swapper/022:50:270
31954995113,16cyclictest7850irq/56-dwc_otg_00:35:190
3195499511,30cyclictest26627-21grep20:25:130
3195999507,29cyclictest5114-21ssh23:40:252
3195799501,5cyclictest0-21swapper/119:40:221
3195799501,4cyclictest23845-21aten_rbpower_cu22:05:261
3195499507,29cyclictest15547-21ssh21:55:250
3195499502,28cyclictest10259-21diskmemload21:10:400
3195499501,19cyclictest0-21swapper/021:00:280
3195799492,6cyclictest4319-21unixbench_singl19:20:401
3195799491,7cyclictest4385-21grep20:50:401
3195799491,7cyclictest4385-21grep20:50:401
3195499498,21cyclictest0-21swapper/022:40:310
3195499494,6cyclictest28643-21ssh21:30:420
3195499493,25cyclictest3576-21fschecks_time20:50:290
3195499493,25cyclictest3576-21fschecks_time20:50:290
3195499492,8cyclictest0-21swapper/020:05:230
3195499492,36cyclictest0-21swapper/022:33:500
3195499492,36cyclictest0-21swapper/022:33:500
3195499491,30cyclictest9552-21fschecks_time23:45:290
31959994835,5cyclictest12228-21ssh22:30:282
31959994835,5cyclictest12228-21ssh22:30:282
3195799482,3cyclictest0-21swapper/119:30:121
3195799481,7cyclictest13215-21ls19:45:381
31954994826,6cyclictest7850irq/56-dwc_otg_00:15:290
3195499481,27cyclictest2816-21awk21:40:120
3195499481,26cyclictest15744-21ssh23:13:440
3195499481,24cyclictest28811-21ssh22:10:360
294832480,3sleep30-21swapper/322:50:383
3195999471,2cyclictest0-21swapper/200:35:232
3195799471,4cyclictest32049-21cut19:10:251
3195499473,5cyclictest5708-21ssh00:20:220
31954994723,3cyclictest0-21swapper/021:45:370
31954994712,16cyclictest21123-21sh23:20:270
31954994712,16cyclictest21123-21sh23:20:270
3195999467,25cyclictest0-21swapper/219:15:252
3195999467,24cyclictest0-21swapper/219:35:252
3195999461,7cyclictest7950irq/56-dwc_otg_20:50:242
3195999461,7cyclictest7950irq/56-dwc_otg_20:50:242
3195799462,4cyclictest2150-21/usr/sbin/munin19:15:371
31954994631,3cyclictest0-21swapper/019:15:330
3195499461,16cyclictest0-21swapper/022:01:120
3195499452,8cyclictest3235-21aten_rbpower_en19:20:260
3195499452,22cyclictest26346-21unixbench-2d23:25:400
3195499452,10cyclictest811-21latency_hist23:35:130
31954994513,12cyclictest0-21swapper/019:50:230
31954994512,4cyclictest0-21swapper/019:10:180
315332453,13sleep00-21swapper/019:09:280
31954994421,10cyclictest0-21swapper/020:30:240
31954994420,15cyclictest0-21swapper/020:26:100
3195499441,28cyclictest0-21swapper/023:39:470
3195999436,24cyclictest0-21swapper/219:45:252
31954994319,14cyclictest0-21swapper/020:47:320
3195499431,21cyclictest17417-21munin-run20:00:120
60762420,3sleep3381ktimersoftd/323:01:093
3196099427,22cyclictest0-21swapper/320:50:253
3196099427,22cyclictest0-21swapper/320:50:253
3195999427,23cyclictest19094-21ssh21:20:242
3195999418,24cyclictest10620-21ssh21:10:252
151682410,2sleep30-21swapper/323:12:403
3196099407,18cyclictest0-21swapper/319:50:253
3196099406,20cyclictest8660-21ssh23:05:273
3195999407,23cyclictest7750irq/56-dwc_otg22:00:352
3195999403,12cyclictest0-21swapper/223:10:462
95722391,2sleep20-21swapper/223:45:302
3196099397,19cyclictest0-21swapper/319:45:253
3195999397,22cyclictest0-21swapper/223:15:262
3195999397,22cyclictest0-21swapper/220:45:242
3195999381,25cyclictest26464-21ls00:05:332
3195999381,25cyclictest26464-21ls00:05:332
3195999381,22cyclictest0-21swapper/222:20:042
3196099371,18cyclictest0-21swapper/323:40:013
3195999376,22cyclictest0-21swapper/223:35:252
3195999376,15cyclictest0-21swapper/220:10:252
3195999372,22cyclictest7750irq/56-dwc_otg21:35:122
3195999371,24cyclictest25517-21date00:05:132
3196099366,15cyclictest7950irq/56-dwc_otg_00:35:403
3196099361,24cyclictest1398-21ls20:45:133
31960993610,14cyclictest13724-21latency_hist00:30:133
3195999367,5cyclictest7950irq/56-dwc_otg_23:00:382
293872360,12sleep37950irq/56-dwc_otg_21:32:043
3196099358,14cyclictest0-21swapper/322:47:373
3196099355,19cyclictest0-21swapper/322:04:443
3196099354,5cyclictest7950irq/56-dwc_otg_19:20:263
31960993524,4cyclictest7750irq/56-dwc_otg00:00:233
31960993522,4cyclictest10935-21fschecks_time21:10:293
3195999359,5cyclictest7950irq/56-dwc_otg_00:25:132
3195999357,16cyclictest0-21swapper/222:10:242
3195999357,16cyclictest0-21swapper/219:20:252
3195999356,8cyclictest7950irq/56-dwc_otg_19:45:132
31959993526,6cyclictest26458-21idleruntime-cro20:25:122
31959993515,4cyclictest1885-21sh21:38:332
3196099344,15cyclictest7950irq/56-dwc_otg_20:45:283
31960993413,4cyclictest0-21swapper/323:25:423
3195999348,19cyclictest0-21swapper/222:44:062
3195999346,7cyclictest27781-21grep20:25:412
3195999345,17cyclictest7950irq/56-dwc_otg_23:20:242
3195999345,17cyclictest7950irq/56-dwc_otg_23:20:242
3196099334,5cyclictest32024-21/usr/sbin/munin22:15:373
3196099334,19cyclictest29290-21ssh23:30:253
3196099333,9cyclictest7950irq/56-dwc_otg_22:43:363
3196099333,14cyclictest7950irq/56-dwc_otg_20:15:263
31960993311,4cyclictest0-21swapper/300:20:353
3195999336,7cyclictest7950irq/56-dwc_otg_23:25:302
3195999336,22cyclictest0-21swapper/220:35:252
3195999335,6cyclictest7750irq/56-dwc_otg23:58:392
3195999335,5cyclictest7750irq/56-dwc_otg21:15:182
3195999335,13cyclictest7950irq/56-dwc_otg_21:55:242
3195999332,7cyclictest9882-21unixbench_multi23:05:422
239952330,5sleep33196099cyclictest21:25:363
3196099327,4cyclictest7750irq/56-dwc_otg21:40:133
31960993210,3cyclictest0-21swapper/323:50:353
31960993210,3cyclictest0-21swapper/323:50:353
3195999328,4cyclictest0-21swapper/222:05:262
3195999327,18cyclictest7750irq/56-dwc_otg21:40:482
3195999327,11cyclictest7950irq/56-dwc_otg_21:45:382
3195999325,5cyclictest7850irq/56-dwc_otg_20:05:122
3195999325,5cyclictest7850irq/56-dwc_otg_20:05:122
3195999323,5cyclictest13341-21ssh23:50:232
3195999323,5cyclictest13341-21ssh23:50:232
3195999321,5cyclictest17391-21ssh22:35:402
3195999321,4cyclictest0-21swapper/221:27:442
3195999321,16cyclictest3239550ssh23:33:592
300502326,15sleep230143-21/usr/sbin/munin19:05:262
3196099317,12cyclictest0-21swapper/323:58:143
3196099316,4cyclictest7850irq/56-dwc_otg_20:15:123
3196099316,14cyclictest7950irq/56-dwc_otg_21:20:133
3196099313,13cyclictest7950irq/56-dwc_otg_22:30:313
3196099313,13cyclictest7950irq/56-dwc_otg_22:30:313
3196099313,11cyclictest7950irq/56-dwc_otg_20:55:453
3196099312,19cyclictest0-21swapper/300:30:313
31960993121,8cyclictest1916-21ssh00:15:303
31960993115,9cyclictest11369-21ssh21:50:223
3196099311,4cyclictest0-21swapper/323:44:053
31960993110,13cyclictest0-21swapper/300:05:273
31960993110,13cyclictest0-21swapper/300:05:273
3195999317,17cyclictest7950irq/56-dwc_otg_22:26:252
3195999316,4cyclictest7950irq/56-dwc_otg_00:25:262
3195999313,16cyclictest7950irq/56-dwc_otg_20:05:322
3195999313,13cyclictest7950irq/56-dwc_otg_19:55:122
31959993123,6cyclictest15206-21idleruntime-cro21:55:122
3195999311,8cyclictest7850irq/56-dwc_otg_22:49:042
3196099308,4cyclictest0-21swapper/322:10:403
3196099307,20cyclictest0-21swapper/320:20:253
3196099306,4cyclictest0-21swapper/322:05:333
3196099306,14cyclictest0-21swapper/319:55:263
3196099305,11cyclictest7950irq/56-dwc_otg_22:27:053
3196099304,13cyclictest7950irq/56-dwc_otg_19:10:463
3196099303,5cyclictest30550-21fschecks_count20:35:283
3196099303,14cyclictest7850irq/56-dwc_otg_19:30:383
3196099302,3cyclictest7950irq/56-dwc_otg_21:20:343
3196099301,3cyclictest7950irq/56-dwc_otg_22:55:323
3196099301,18cyclictest0-21swapper/319:36:093
3195999308,4cyclictest0-21swapper/200:17:142
31959993023,5cyclictest30020-21aten_rbpower_vo00:10:272
31959993010,4cyclictest0-21swapper/222:57:252
31959993010,10cyclictest0-21swapper/222:21:462
307832304,13sleep10-21swapper/119:05:371
3196099297,14cyclictest0-21swapper/320:25:263
3196099295,4cyclictest7750irq/56-dwc_otg21:00:243
3196099294,4cyclictest7850irq/56-dwc_otg_21:55:283
3196099293,12cyclictest7950irq/56-dwc_otg_22:20:343
31960992922,5cyclictest10554-21ssh23:45:433
3196099292,19cyclictest29599-21unixbench_multi20:30:403
3195999296,5cyclictest7850irq/56-dwc_otg_21:05:362
3195999291,16cyclictest7950irq/56-dwc_otg_19:10:282
3196099287,11cyclictest0-21swapper/321:05:263
3196099282,14cyclictest0-21swapper/319:40:373
31960992818,6cyclictest19771-21fschecks_time20:05:273
3195999287,5cyclictest7850irq/56-dwc_otg_22:52:432
3195999284,11cyclictest7950irq/56-dwc_otg_19:30:342
3195999282,19cyclictest0-21swapper/220:33:052
31959992814,10cyclictest0-21swapper/220:40:462
31959992811,7cyclictest31517-21systemd-journal20:16:252
3195999281,16cyclictest8473-21ls21:05:122
3196099279,5cyclictest7950irq/56-dwc_otg_19:15:313
3196099272,12cyclictest7950irq/56-dwc_otg_20:00:383
3196099272,12cyclictest7950irq/56-dwc_otg_20:00:383
3196099271,4cyclictest7950irq/56-dwc_otg_19:25:233
31959992719,6cyclictest5285-21cut19:25:302
3195999271,5cyclictest17415-21idleruntime-cro20:00:122
314022273,15sleep30-21swapper/319:07:453
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional