You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-12 - 21:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sun Jul 12, 2026 12:43:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
258929911620,64cyclictest0-21swapper/010:25:280
2519821070,31sleep38050irq/56-dwc_otg_07:05:483
25892991032,74cyclictest0-21swapper/009:30:290
258929910222,66cyclictest20878-21cut08:25:320
25892999923,62cyclictest26010-21cut07:10:320
25892999621,15cyclictest7636-21awk10:35:170
25892999519,62cyclictest17021-21sh10:05:440
2589299948,82cyclictest0-21swapper/009:25:300
25892999319,60cyclictest0-21swapper/009:23:330
2589299923,64cyclictest0-21swapper/012:15:270
25892999219,50cyclictest23581-21sort08:30:450
25892999219,50cyclictest23581-21sort08:30:450
25892999130,12cyclictest12657-21grep11:20:480
25892999130,12cyclictest12657-21grep11:20:480
25892999021,49cyclictest4737-21rt-features12:30:410
25892999020,40cyclictest12156-21taskset10:40:320
25892999019,53cyclictest8975-21fschecks_count09:15:350
25892999018,55cyclictest0-21swapper/010:50:170
25892998916,68cyclictest0-21swapper/011:31:550
25892998819,47cyclictest7449-21/usr/sbin/munin11:15:280
25892998819,45cyclictest31162-21chrt11:04:370
25892998819,45cyclictest31162-21chrt11:04:370
25892998723,41cyclictest2953-21ssh11:10:040
25892998721,54cyclictest0-21swapper/009:39:250
25892998719,56cyclictest2797-21sort11:50:160
25892998719,48cyclictest5282-21sed09:10:370
25892998629,12cyclictest4546-21latency_hist09:10:180
25892998625,39cyclictest19575-21missed_timers08:20:380
25892998621,47cyclictest8007-21basename10:35:310
25892998621,45cyclictest8175-21hostname09:55:300
25892998620,38cyclictest8330-21wc11:55:440
25893998528,40cyclictest7850irq/56-dwc_otg12:15:271
2589599848,32cyclictest0-21swapper/309:25:303
25892998426,40cyclictest13173-21ls10:00:470
25892998425,45cyclictest3715-21latency_hist09:50:170
25892998417,45cyclictest31069-21munin-run11:45:160
294152830,6sleep12589399cyclictest10:20:471
2589499833,10cyclictest0-21swapper/208:15:272
25892998336,18cyclictest8050irq/56-dwc_otg_12:10:300
25892998322,43cyclictest31757-21ssh09:44:420
25892998322,43cyclictest0-21swapper/011:39:570
25892998318,38cyclictest0-21swapper/009:55:170
25895998242,23cyclictest7950irq/56-dwc_otg_11:15:243
25892998219,37cyclictest16954-21wc08:15:160
25892998215,51cyclictest0-21swapper/008:51:500
218462820,2sleep20-21swapper/210:11:252
25892998129,34cyclictest26987-21ssh10:59:110
256992810,2sleep10-21swapper/110:16:041
25892998022,42cyclictest0-21swapper/008:00:170
25892998019,43cyclictest18687-21ssh12:09:010
25892998018,41cyclictest3726-21ls11:50:380
25892997927,50cyclictest0-21swapper/012:25:280
25892997922,45cyclictest23108-21sh10:13:490
25892997922,43cyclictest5727-21ssh11:12:440
25892997921,46cyclictest29076-21seq07:17:390
25892997917,51cyclictest0-21swapper/011:26:130
2589299783,62cyclictest0-21swapper/007:20:290
25892997826,38cyclictest28328-21/usr/sbin/munin10:20:440
25892997821,36cyclictest959-21/usr/sbin/munin09:00:340
119182770,3sleep011888-21ssh12:00:370
119182770,3sleep011888-21ssh12:00:370
2589499764,8cyclictest8050irq/56-dwc_otg_07:15:282
25892997620,43cyclictest17328-21grep08:15:310
25892997616,51cyclictest0-21swapper/007:37:100
25892997517,47cyclictest9691-21cron07:55:160
25892997517,44cyclictest0-21swapper/010:16:290
25892997517,43cyclictest0-21swapper/012:40:170
25892997420,43cyclictest30206-21rm12:22:220
25892997420,43cyclictest30206-21rm12:22:220
25892997320,42cyclictest0-21swapper/007:34:590
25892997222,40cyclictest0-21swapper/008:38:580
25892997218,40cyclictest0-21swapper/010:51:120
25892997215,44cyclictest0-21swapper/008:46:320
25892997126,30cyclictest659-21taskset08:59:450
25892997024,28cyclictest0-21swapper/008:05:270
25892997019,41cyclictest13275-21taskset08:05:040
25895996825,17cyclictest8247-21ls11:55:433
2589499671,23cyclictest7214-21ls11:15:172
28172660,2sleep20-21swapper/209:48:452
2589299667,50cyclictest7473-21unixbench_singl07:45:460
25892996617,31cyclictest0-21swapper/007:29:530
25895996524,23cyclictest27944-21diskstats07:15:333
25892996519,34cyclictest0-21swapper/008:43:530
2589299651,20cyclictest4944-21latency07:40:380
25895996419,14cyclictest27294-21ssh10:19:083
2589399645,9cyclictest7850irq/56-dwc_otg09:50:281
25895996226,14cyclictest2163-21ssh10:27:353
25895996224,20cyclictest17070-21ssh11:26:353
25895996222,20cyclictest5453-21ntpq09:10:403
25895996222,17cyclictest29975-21ssh11:02:183
25895996222,17cyclictest29975-21ssh11:02:183
25895996221,31cyclictest21217-21ssh11:31:563
25895996219,26cyclictest27056-21ssh10:59:193
2589499621,17cyclictest0-21swapper/208:25:362
2589399622,12cyclictest4348-21diskmemload09:25:291
25895996129,13cyclictest31796-21ssh10:24:503
25895996127,14cyclictest8311-21perl10:35:353
25895996127,13cyclictest5929-21ssh11:13:083
25895996125,13cyclictest10663-21ssh09:58:103
25895996121,24cyclictest7850irq/56-dwc_otg09:15:443
25895996121,15cyclictest7769-21munin-run09:55:163
2589399613,15cyclictest32380-21ssh09:45:271
25895996026,13cyclictest19966-21ls08:20:443
160082600,2sleep20-21swapper/210:05:172
2589599598,28cyclictest0-21swapper/309:00:313
25895995926,23cyclictest17925-21ssh10:06:383
2589599588,30cyclictest0-21swapper/308:30:313
2589599588,30cyclictest0-21swapper/308:30:313
25895995830,19cyclictest5731-21ssh10:31:423
25895995824,12cyclictest12123-21ssh10:00:283
25895995823,23cyclictest14489-21ssh12:04:053
25895995823,23cyclictest14489-21ssh12:04:053
25895995823,12cyclictest29437-21sh11:42:193
25895995817,13cyclictest3728-21ssh11:50:383
2589399585,19cyclictest8050irq/56-dwc_otg_08:10:311
25895995726,19cyclictest21445-21ssh10:51:133
25895995724,14cyclictest14512-21rm09:21:533
25895995723,13cyclictest8050irq/56-dwc_otg_08:10:173
25895995721,13cyclictest23293-21ssh10:14:093
2589499571,24cyclictest9134-21ls07:50:452
25895995624,13cyclictest31767-21ssh09:44:433
25895995623,14cyclictest14488-21ssh10:43:133
25895995622,12cyclictest6634-21ssh12:33:273
25895995615,25cyclictest0-21swapper/309:36:553
2589599558,30cyclictest0-21swapper/307:45:233
25895995526,14cyclictest22421-21ssh09:31:523
25895995521,21cyclictest31327-21rm11:45:213
25895995521,12cyclictest21439-21/usr/sbin/munin08:25:403
25895995520,12cyclictest24794-21ssh12:15:423
25895995518,24cyclictest12777-21ssh11:20:523
25895995518,24cyclictest12777-21ssh11:20:523
2589399556,25cyclictest8050irq/56-dwc_otg_08:35:271
171872550,7sleep28050irq/56-dwc_otg_10:05:452
25895995426,13cyclictest16180-21awk10:45:353
25895995424,21cyclictest2958-21ssh11:10:053
25895995418,13cyclictest32105-21ssh12:25:263
2589599538,26cyclictest0-21swapper/307:50:303
25895995327,10cyclictest17718-21ls08:15:373
2589599532,6cyclictest8050irq/56-dwc_otg_12:05:293
2589399538,31cyclictest0-21swapper/110:00:311
2589399538,30cyclictest0-21swapper/107:15:301
25895995225,18cyclictest26596-21ssh11:39:383
2589499528,26cyclictest0-21swapper/208:35:302
2589399528,32cyclictest0-21swapper/108:00:301
2589399527,29cyclictest0-21swapper/109:00:311
25895995125,12cyclictest11077-21ssh12:39:003
25895995121,11cyclictest8050irq/56-dwc_otg_09:05:313
25895995119,13cyclictest2251-21ssh09:47:403
25895995119,11cyclictest22254-21ssh12:12:583
2589399514,4cyclictest7850irq/56-dwc_otg12:35:291
25893995116,20cyclictest0-21swapper/111:35:441
2589599508,28cyclictest0-21swapper/307:20:303
25895995022,12cyclictest31677-21ssh12:25:053
25895995022,12cyclictest31677-21ssh12:25:053
2589499502,9cyclictest7850irq/56-dwc_otg08:40:292
2589399507,32cyclictest0-21swapper/108:15:301
2589399502,11cyclictest7950irq/56-dwc_otg_08:55:301
25894994925,7cyclictest8050irq/56-dwc_otg_11:20:342
25894994925,7cyclictest8050irq/56-dwc_otg_11:20:342
2589399498,30cyclictest0-21swapper/107:20:301
2589399495,13cyclictest23-21ksoftirqd/109:15:351
2589599481,5cyclictest8050irq/56-dwc_otg_08:45:273
25895994815,9cyclictest8050irq/56-dwc_otg_08:10:353
25895994814,11cyclictest5955-21sshd07:45:013
2589399483,6cyclictest0-21swapper/111:40:291
282212476,5sleep128238-21fschecks_count08:45:341
25895994714,12cyclictest26482-21memory07:10:383
2589399478,28cyclictest0-21swapper/110:28:101
2589399476,26cyclictest4348-21diskmemload12:25:311
25893994725,6cyclictest3188750wget11:05:301
34882460,2sleep10-21swapper/111:50:351
25895994619,14cyclictest473-21runrttasks07:29:353
2589499465,7cyclictest8050irq/56-dwc_otg_11:30:472
2589399469,9cyclictest23-21ksoftirqd/107:10:281
2589399465,30cyclictest7396-21ssh11:15:231
2589399465,26cyclictest20562-21diskmemload09:30:291
282452450,4sleep229-21rcuc/208:45:342
25895994529,12cyclictest12799-21ls08:00:453
25895994528,14cyclictest25305-21ls08:35:433
25895994519,16cyclictest750-21snmpd07:39:333
25895994510,23cyclictest750-21snmpd08:51:513
2589499458,32cyclictest0-21swapper/211:35:302
2589499454,24cyclictest0-21swapper/208:30:292
2589499454,24cyclictest0-21swapper/208:30:292
2589399458,17cyclictest8050irq/56-dwc_otg_12:05:291
2589399456,5cyclictest7950irq/56-dwc_otg_10:35:171
255092453,34sleep00-21swapper/007:08:490
25895994430,11cyclictest1019-21/usr/sbin/munin07:30:333
25895994417,10cyclictest8050irq/56-dwc_otg_07:55:193
2589499441,25cyclictest32261-21ls09:45:172
2589399448,30cyclictest0-21swapper/108:30:311
2589399448,30cyclictest0-21swapper/108:30:311
25893994424,11cyclictest23-21ksoftirqd/109:10:331
2589399441,29cyclictest0-21swapper/110:55:381
25893994411,10cyclictest21-21rcuc/110:08:391
25895994325,14cyclictest31855-21fschecks_count08:55:353
2589499433,8cyclictest7850irq/56-dwc_otg11:05:282
2589399435,27cyclictest16632-21grep11:25:471
2589399431,29cyclictest1071-21cut07:30:351
2589399431,26cyclictest0-21swapper/107:45:231
25895994227,12cyclictest27002-21/usr/sbin/munin08:40:423
2589499421,17cyclictest7950irq/56-dwc_otg_09:05:172
2589399428,30cyclictest0-21swapper/108:20:301
2589399424,19cyclictest23-21ksoftirqd/110:35:311
25893994214,13cyclictest9550irq/48-DMA15:26:471
2589499414,4cyclictest0-21swapper/212:20:292
2589499414,4cyclictest0-21swapper/212:20:292
2589399417,25cyclictest0-21swapper/109:05:301
2589399412,24cyclictest7850irq/56-dwc_otg10:40:321
25893994117,13cyclictest23-21ksoftirqd/111:20:481
25893994117,13cyclictest23-21ksoftirqd/111:20:481
2589499406,5cyclictest7950irq/56-dwc_otg_10:55:302
25894994025,5cyclictest8050irq/56-dwc_otg_07:35:402
25894994017,6cyclictest16581-21missed_timers10:45:402
2589499401,29cyclictest16956-21ls08:15:162
25894994010,5cyclictest7950irq/56-dwc_otg_10:35:282
25893994013,6cyclictest27063-21ls08:40:431
2589499399,8cyclictest7850irq/56-dwc_otg11:29:412
2589499399,12cyclictest7950irq/56-dwc_otg_10:40:482
2589499398,27cyclictest0-21swapper/207:25:292
2589499398,26cyclictest0-21swapper/207:20:232
2589499398,18cyclictest0-21swapper/207:45:242
2589499394,7cyclictest7821-21fschecks_time11:15:362
25894993915,8cyclictest16893-21fschecks_count09:25:342
25894993911,6cyclictest29-21rcuc/210:50:472
2589399399,8cyclictest8082-21ssh09:55:271
2589399392,25cyclictest32083-21ssh09:45:161
25893993921,10cyclictest7850irq/56-dwc_otg10:51:091
25893993912,12cyclictest221ktimersoftd/111:55:381
2589499386,17cyclictest7950irq/56-dwc_otg_10:20:162
2589499385,22cyclictest12669-21ssh09:20:322
2589499384,24cyclictest9696-21timerandwakeup09:15:462
2589499382,18cyclictest4844-21ntp_states09:50:412
25894993820,5cyclictest31453-21aten_rbpower_cu11:45:312
25894993814,12cyclictest4348-21diskmemload09:31:302
2589399389,21cyclictest0-21swapper/107:55:391
2589399386,8cyclictest23-21ksoftirqd/110:10:441
2589399382,9cyclictest29123-21grep11:00:471
2589399382,9cyclictest29123-21grep11:00:471
2589399382,9cyclictest0-21swapper/112:10:281
2589499378,27cyclictest0-21swapper/208:05:302
2589499374,10cyclictest8050irq/56-dwc_otg_09:14:062
2589499373,19cyclictest7850irq/56-dwc_otg11:55:342
2589499372,22cyclictest17654-21ssh12:07:072
2589499371,4cyclictest7950irq/56-dwc_otg_07:55:372
2589499371,22cyclictest30967-21head08:50:472
2589399377,8cyclictest23-21ksoftirqd/109:20:511
2589399374,24cyclictest7850irq/56-dwc_otg11:50:161
2589399373,14cyclictest7950irq/56-dwc_otg_07:40:171
25893993719,6cyclictest23-21ksoftirqd/112:30:321
81362360,6sleep22589499cyclictest09:55:302
2589499368,7cyclictest8050irq/56-dwc_otg_08:00:432
2589499367,17cyclictest0-21swapper/210:20:242
2589499362,4cyclictest7950irq/56-dwc_otg_11:00:412
2589499362,4cyclictest7950irq/56-dwc_otg_11:00:412
2589499362,10cyclictest4348-21diskmemload10:25:302
2589399367,5cyclictest7850irq/56-dwc_otg07:25:341
2589399363,16cyclictest8050irq/56-dwc_otg_09:35:321
25893993613,6cyclictest21-21rcuc/107:55:081
2589499357,7cyclictest8050irq/56-dwc_otg_08:25:172
2589499356,14cyclictest7800-21latency_hist10:35:182
2589499353,6cyclictest7850irq/56-dwc_otg08:55:452
2589499353,5cyclictest8050irq/56-dwc_otg_09:35:302
25894993525,5cyclictest301ktimersoftd/212:10:282
25894993514,5cyclictest2732450wget11:40:302
25894993510,13cyclictest0-21swapper/209:10:172
2589399358,9cyclictest21-21rcuc/112:20:351
2589399358,9cyclictest21-21rcuc/112:20:351
2589399357,15cyclictest221ktimersoftd/110:48:361
2589399352,6cyclictest20619-21unixbench_singl11:30:481
25893993514,13cyclictest7850irq/56-dwc_otg08:25:321
25893993512,12cyclictest221ktimersoftd/108:10:161
2589499347,15cyclictest0-21swapper/207:30:342
2589499341,23cyclictest0-21swapper/207:41:422
25893993415,5cyclictest4669-21fschecks_count07:40:351
2589499338,5cyclictest0-21swapper/212:25:362
2589499332,13cyclictest0-21swapper/212:15:292
2589499331,24cyclictest8209-21ls12:35:322
2589399334,15cyclictest7950irq/56-dwc_otg_12:00:351
2589399334,15cyclictest7950irq/56-dwc_otg_12:00:351
2589499329,5cyclictest15155-21latency_hist12:05:172
2589499329,5cyclictest15155-21latency_hist12:05:172
25893993210,10cyclictest8050irq/56-dwc_otg_08:53:031
2589499313,14cyclictest0-21swapper/211:50:322
2589499311,8cyclictest27032-21grep07:10:462
2589499297,4cyclictest0-21swapper/212:31:552
254022267,12sleep10-21swapper/107:07:251
243182263,11sleep20-21swapper/207:05:362
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional