You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-02 - 21:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Thu Jul 02, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1596799945,21cyclictest25035-21ssh09:45:260
197022900,5sleep31598099cyclictest11:35:293
1596799906,52cyclictest0-21swapper/012:05:180
73252880,4sleep231-21ksoftirqd/209:24:402
15967998414,53cyclictest3514-21latency_hist09:20:120
15967998243,15cyclictest8050irq/56-dwc_otg_12:35:240
1596799819,51cyclictest14065-21unixbench_singl08:30:390
1596799806,24cyclictest0-21swapper/008:20:250
236692790,3sleep229-21rcuc/210:22:012
1596799796,54cyclictest4897-21unixbench-2d09:20:380
4992770,5sleep21597899cyclictest09:55:112
15967997615,44cyclictest32329-21idleruntime-cro12:30:110
327052750,6sleep31598099cyclictest07:55:253
1596799759,11cyclictest12082-21wc11:25:370
1596799757,48cyclictest18022-21ssh10:55:120
1597899743,9cyclictest0-21swapper/209:30:222
1597899743,9cyclictest0-21swapper/209:30:222
1596799748,45cyclictest27315-21munin-plugin-st11:45:110
1596799746,58cyclictest6424-21ssh11:20:070
227452730,5sleep11597399cyclictest09:41:421
1596799736,46cyclictest28062-21grep07:40:390
15967997316,45cyclictest30822-21/usr/sbin/munin11:10:230
15967997312,48cyclictest16088-21tr12:10:250
15967997310,37cyclictest16264-21fschecks_count07:10:270
15967997111,46cyclictest1111-21ssh12:30:370
1596799707,51cyclictest0-21swapper/011:48:200
1596799705,58cyclictest0-21swapper/008:55:360
1596799699,46cyclictest21787-21ssh12:16:100
1596799698,43cyclictest23656-21ssh10:21:590
1596799695,52cyclictest8825-21modprobe10:05:110
1596799687,53cyclictest0-21swapper/010:30:210
80232670,2sleep10-21swapper/112:39:561
1596799677,45cyclictest3808-21ssh11:55:290
1596799677,45cyclictest3808-21ssh11:55:290
1596799675,53cyclictest11376-21ssh09:29:160
15967996721,34cyclictest7139-21sh11:20:260
243912660,2sleep20-21swapper/212:20:252
243912660,2sleep20-21swapper/212:20:252
1596799667,46cyclictest0-21swapper/010:41:470
1596799667,46cyclictest0-21swapper/010:41:470
1596799667,41cyclictest4460-21ntp_kernel_pll_08:05:320
1596799665,42cyclictest21163-21fschecks_count09:40:290
15967996611,46cyclictest18799-21ssh10:55:310
60762650,2sleep10-21swapper/111:19:291
1596799659,39cyclictest29226-21ssh10:29:350
1596799659,39cyclictest29226-21ssh10:29:350
1596799655,45cyclictest0-21swapper/008:50:260
1596799652,52cyclictest0-21swapper/009:35:300
1596799652,52cyclictest0-21swapper/009:35:300
1596799651,45cyclictest0-21swapper/008:10:270
15967996513,41cyclictest25799-21sed07:35:310
15967996511,45cyclictest11999-21sh10:46:560
15967996510,41cyclictest4766-21ssh10:39:020
1597899642,13cyclictest7850irq/56-dwc_otg08:30:222
1596799649,42cyclictest0-21swapper/009:50:340
15967996413,38cyclictest27789-21ssh12:24:310
15967996413,38cyclictest27789-21ssh12:24:310
15967996410,45cyclictest16010-21taskset11:30:350
15967996410,45cyclictest15826-21sh09:34:350
15967996410,45cyclictest15826-21sh09:34:350
15967996410,39cyclictest27135-21diskmemload12:00:510
1596799633,47cyclictest0-21swapper/009:15:050
1596799633,46cyclictest0-21swapper/011:50:530
15967996313,33cyclictest16233-21ssh10:13:350
1596799629,37cyclictest11054-21basename08:25:260
1596799627,47cyclictest0-21swapper/009:56:530
1596799627,47cyclictest0-21swapper/009:56:530
1596799621,49cyclictest0-21swapper/011:40:010
323392610,4sleep2750-21snmpd12:30:112
1596799614,45cyclictest0-21swapper/011:06:200
15967996110,36cyclictest9069-21ssh10:05:160
1596799605,47cyclictest32006-21chrt07:52:040
1596799605,44cyclictest0-21swapper/010:19:340
1596799603,46cyclictest0-21swapper/009:06:430
15967996010,40cyclictest19042-21taskset07:16:130
258972590,6sleep11597399cyclictest12:21:041
258972590,6sleep11597399cyclictest12:21:041
1596799596,44cyclictest0-21swapper/008:19:250
1596799588,39cyclictest0-21swapper/008:02:430
1596799588,39cyclictest0-21swapper/007:57:090
1596799582,43cyclictest0-21swapper/011:00:580
1597399577,25cyclictest0-21swapper/107:25:251
1596799574,43cyclictest0-21swapper/008:38:540
1596799571,43cyclictest0-21swapper/009:00:260
1596799568,39cyclictest0-21swapper/008:44:310
1596799544,41cyclictest30027-21runrttasks07:46:080
1596799542,45cyclictest24664-21runrttasks07:32:230
1598099533,10cyclictest7950irq/56-dwc_otg_12:35:243
1597399531,9cyclictest0-21swapper/111:25:221
15967995329,3cyclictest0-21swapper/007:25:110
237812520,3sleep10-21swapper/112:19:471
1596799527,33cyclictest0-21swapper/008:47:400
1596799527,33cyclictest0-21swapper/008:47:400
1596799519,34cyclictest0-21swapper/007:27:100
12052490,2sleep30-21swapper/311:13:233
69172470,2sleep20-21swapper/212:37:562
1598099476,32cyclictest8050irq/56-dwc_otg_10:40:223
1598099476,32cyclictest8050irq/56-dwc_otg_10:40:223
1597899467,25cyclictest0-21swapper/208:15:252
1597899462,29cyclictest8050irq/56-dwc_otg_11:05:292
1597399468,21cyclictest0-21swapper/110:15:241
1597399467,26cyclictest0-21swapper/108:30:171
1597399463,10cyclictest0-21swapper/107:35:211
1597899457,25cyclictest0-21swapper/207:55:182
1597899457,25cyclictest0-21swapper/207:50:242
15973994517,12cyclictest7850irq/56-dwc_otg11:20:241
1597399451,3cyclictest0-21swapper/109:55:251
1597399451,3cyclictest0-21swapper/109:55:251
154682453,36sleep00-21swapper/007:07:110
154682453,36sleep00-21swapper/007:07:110
1598099447,23cyclictest0-21swapper/308:55:253
1598099446,23cyclictest31789-21ssh09:15:183
15980994426,9cyclictest8050irq/56-dwc_otg_08:05:373
1597399442,5cyclictest0-21swapper/109:50:231
1597399441,14cyclictest0-21swapper/110:05:261
8162430,3sleep2815-21grep12:30:312
1598099437,23cyclictest0-21swapper/307:30:163
1598099436,23cyclictest0-21swapper/307:40:253
1597899437,21cyclictest0-21swapper/207:20:242
15978994319,5cyclictest30681-21sh10:30:342
1597399437,23cyclictest0-21swapper/107:40:251
1597399436,24cyclictest0-21swapper/108:05:231
15980994223,6cyclictest20446-21fschecks_count12:15:263
1597899427,25cyclictest0-21swapper/208:05:232
1597899427,23cyclictest0-21swapper/210:40:172
1597899427,23cyclictest0-21swapper/210:40:172
1597899423,6cyclictest0-21swapper/207:45:222
15978994219,4cyclictest727-21runrttasks11:30:222
310122410,1sleep30-21swapper/312:27:513
1598099417,24cyclictest0-21swapper/309:30:173
1598099417,24cyclictest0-21swapper/309:30:173
1598099417,21cyclictest0-21swapper/309:45:263
1598099412,6cyclictest8050irq/56-dwc_otg_11:00:223
15980994124,13cyclictest8050irq/56-dwc_otg_12:10:233
15980994121,4cyclictest7950irq/56-dwc_otg_10:55:233
15980994115,14cyclictest0-21swapper/308:25:263
15978994116,17cyclictest25732-21/usr/sbin/munin10:25:362
15978994116,17cyclictest25732-21/usr/sbin/munin10:25:362
1597399413,5cyclictest8050irq/56-dwc_otg_10:40:121
1598099407,22cyclictest0-21swapper/308:20:253
15980994021,11cyclictest16118-21aten_rbpower_vo07:10:253
1597399407,16cyclictest7950irq/56-dwc_otg_08:50:291
1597399405,21cyclictest0-21swapper/110:00:221
15973994021,4cyclictest11997-21latency_hist09:30:121
1597399401,29cyclictest31602-21latency_hist09:15:121
15973994011,14cyclictest7850irq/56-dwc_otg11:52:371
15980993920,8cyclictest8050irq/56-dwc_otg_09:20:563
1597899397,23cyclictest0-21swapper/211:55:252
1597899397,23cyclictest0-21swapper/211:55:252
1597399397,26cyclictest0-21swapper/108:45:241
1597399397,26cyclictest0-21swapper/108:45:241
15973993920,5cyclictest22892-21grep10:20:391
1597399391,8cyclictest29054-21fschecks_count07:45:281
15973993918,19cyclictest0-21swapper/107:30:241
121502390,6sleep11597399cyclictest10:47:121
262402380,3sleep30-21swapper/309:05:333
1598099383,11cyclictest381ktimersoftd/309:40:303
15980993826,8cyclictest31821-21ssh11:50:253
1598099381,22cyclictest11604-21ls12:05:123
1597899387,5cyclictest0-21swapper/211:10:342
1597899382,12cyclictest0-21swapper/208:45:222
1597899382,12cyclictest0-21swapper/208:45:222
15978993817,6cyclictest2647-21sh11:15:232
15978993813,5cyclictest18531-21df10:55:282
1597399386,23cyclictest0-21swapper/109:30:181
1597399386,23cyclictest0-21swapper/109:30:181
1598099378,4cyclictest566250ssh11:18:443
1598099377,21cyclictest0-21swapper/307:35:233
15980993717,8cyclictest7950irq/56-dwc_otg_10:35:493
15980993713,4cyclictest7850irq/56-dwc_otg10:46:593
1598099371,29cyclictest0-21swapper/312:20:243
1598099371,29cyclictest0-21swapper/312:20:243
1597899377,24cyclictest0-21swapper/207:15:242
15978993717,5cyclictest15111-21if_err_eth008:35:312
15978993713,8cyclictest31615-21latency_hist09:15:122
1597399377,24cyclictest0-21swapper/107:55:241
1597399375,23cyclictest0-21swapper/109:35:251
1597399375,23cyclictest0-21swapper/109:35:251
1597399375,22cyclictest5859-21ssh10:40:251
1597399375,22cyclictest5859-21ssh10:40:251
15973993726,7cyclictest221ktimersoftd/110:10:221
15973993714,12cyclictest0-21swapper/109:23:351
1597399371,25cyclictest31622-21timerandwakeup07:50:371
143792375,23sleep314479-21fschecks_time07:05:283
143792375,23sleep314479-21fschecks_time07:05:283
322182360,1sleep10-21swapper/112:30:021
267602360,3sleep10-21swapper/111:44:121
1598099368,15cyclictest0-21swapper/311:25:233
1598099364,12cyclictest381ktimersoftd/310:02:483
1598099363,30cyclictest8050irq/56-dwc_otg_10:05:223
1598099361,29cyclictest17842-21idleruntime10:15:283
1598099361,21cyclictest5822-21cut08:10:263
1598099361,15cyclictest381ktimersoftd/310:24:443
1597899364,9cyclictest8050irq/56-dwc_otg_07:10:212
1597899363,16cyclictest13077-21ssh12:05:412
15978993613,4cyclictest8287-21fschecks_count09:25:292
1597899361,17cyclictest1116650ssh11:25:242
1597399367,10cyclictest0-21swapper/111:35:231
1597399361,9cyclictest125-21systemd-journal12:00:131
115982360,2sleep20-21swapper/210:07:442
1598099357,15cyclictest0-21swapper/308:30:173
1598099356,5cyclictest8841-21ps09:25:373
1598099355,4cyclictest28506-21processes11:45:363
1598099355,16cyclictest37-21rcuc/311:20:313
1598099353,16cyclictest7950irq/56-dwc_otg_11:30:333
15980993520,8cyclictest7850irq/56-dwc_otg09:51:383
1598099351,19cyclictest17278-21ls10:15:123
1597899356,7cyclictest23837-21fschecks_time11:40:282
1597899351,20cyclictest7950irq/56-dwc_otg_12:00:392
1597399353,17cyclictest23-21ksoftirqd/110:55:231
1597399352,3cyclictest0-21swapper/112:35:121
1597399352,20cyclictest21195-21latency_hist07:25:121
15973993513,5cyclictest23-21ksoftirqd/110:30:121
15973993513,5cyclictest23-21ksoftirqd/110:30:121
1597399351,20cyclictest0-21swapper/111:33:081
226312341,4sleep222654-21ntp_states08:55:342
1598099347,15cyclictest0-21swapper/307:50:173
1598099346,14cyclictest381ktimersoftd/309:57:123
1598099346,14cyclictest381ktimersoftd/309:57:123
1598099344,8cyclictest39-21ksoftirqd/309:12:163
1598099341,9cyclictest0-21swapper/307:19:013
15980993414,6cyclictest381ktimersoftd/312:30:233
15980993414,11cyclictest27135-21diskmemload11:42:453
1597899349,6cyclictest0-21swapper/211:01:322
1597899347,14cyclictest0-21swapper/209:00:262
1597899346,4cyclictest7850irq/56-dwc_otg07:30:122
1597899342,9cyclictest27135-21diskmemload10:10:332
1597899342,4cyclictest20859-21apt-get09:40:242
1597899341,5cyclictest20460-21ls11:35:392
15978993413,4cyclictest27135-21diskmemload11:45:282
1597899341,19cyclictest18287-21ls08:45:122
1597399348,19cyclictest0-21swapper/112:10:191
1597399346,5cyclictest7850irq/56-dwc_otg11:45:401
1597399346,15cyclictest0-21swapper/108:25:261
1597399345,7cyclictest22206-21fschecks_count08:55:281
15973993415,11cyclictest18176-21latency_hist10:55:121
1597399341,21cyclictest0-21swapper/109:46:561
15973993412,10cyclictest24926-21kworker/1:211:03:331
1598099339,9cyclictest0-21swapper/310:25:393
1598099339,9cyclictest0-21swapper/310:25:393
1598099336,14cyclictest0-21swapper/308:45:243
1598099336,14cyclictest0-21swapper/308:45:243
1598099336,14cyclictest0-21swapper/308:35:263
1598099332,22cyclictest8966-21latency_hist08:20:123
15980993313,8cyclictest381ktimersoftd/308:50:233
1598099331,25cyclictest11702-21ssh12:05:133
1597899339,12cyclictest7850irq/56-dwc_otg10:17:362
1597899334,10cyclictest8050irq/56-dwc_otg_09:15:232
1597899333,10cyclictest0-21swapper/210:45:222
1597899331,5cyclictest14678-21ssh10:50:332
15973993311,12cyclictest0-21swapper/111:10:361
1598099327,15cyclictest0-21swapper/309:35:193
1598099327,15cyclictest0-21swapper/309:35:193
1598099326,8cyclictest37-21rcuc/310:50:323
1598099325,18cyclictest381ktimersoftd/311:56:253
1598099325,18cyclictest381ktimersoftd/311:56:253
15980993214,5cyclictest7950irq/56-dwc_otg_09:00:283
1597899327,4cyclictest0-21swapper/211:20:352
1597899327,12cyclictest8050irq/56-dwc_otg_09:35:232
1597899327,12cyclictest8050irq/56-dwc_otg_09:35:232
1597899327,10cyclictest8050irq/56-dwc_otg_12:15:112
1597899326,5cyclictest8050irq/56-dwc_otg_11:50:342
1597899321,25cyclictest24273-21timerandwakeup07:30:372
1597899321,24cyclictest25229-21ssh09:45:282
1597399324,13cyclictest7850irq/56-dwc_otg09:00:261
1597399321,9cyclictest5846-21grep08:10:261
1597399321,7cyclictest32104-21diskstats09:15:271
15973993215,6cyclictest0-21swapper/110:30:311
15973993215,4cyclictest521-21munin-node08:20:221
1598099316,8cyclictest381ktimersoftd/310:30:413
1598099313,9cyclictest7850irq/56-dwc_otg08:44:323
1597899313,14cyclictest8050irq/56-dwc_otg_08:25:412
1597899312,5cyclictest0-21swapper/207:35:242
1597899312,3cyclictest7950irq/56-dwc_otg_07:40:222
15978993112,5cyclictest727-21runrttasks10:35:332
1597899311,20cyclictest0-21swapper/209:58:032
1597899311,20cyclictest0-21swapper/209:58:032
1597399319,12cyclictest8050irq/56-dwc_otg_11:05:381
1597399317,10cyclictest8050irq/56-dwc_otg_09:05:271
1597399315,4cyclictest7850irq/56-dwc_otg11:55:241
1597399315,4cyclictest7850irq/56-dwc_otg11:55:241
1597399315,15cyclictest27135-21diskmemload12:05:331
1597399312,3cyclictest7950irq/56-dwc_otg_08:00:241
15973993112,7cyclictest16299-21latency_hist08:40:121
1598099307,20cyclictest0-21swapper/311:05:183
1598099306,13cyclictest39-21ksoftirqd/308:00:243
1598099304,6cyclictest7950irq/56-dwc_otg_07:25:113
1597899306,6cyclictest7850irq/56-dwc_otg09:10:122
1597899303,4cyclictest7950irq/56-dwc_otg_08:00:362
15978993013,4cyclictest27135-21diskmemload10:02:292
15978993010,4cyclictest0-21swapper/212:19:392
1597399301,20cyclictest0-21swapper/108:15:291
153822303,20sleep10-21swapper/107:06:031
153822303,20sleep10-21swapper/107:06:031
15980992922,5cyclictest30366-21apt-get07:50:113
1598099291,5cyclictest7950irq/56-dwc_otg_07:30:113
1597399294,4cyclictest7850irq/56-dwc_otg08:40:291
1597399292,6cyclictest17896-21wget07:15:251
154402294,17sleep20-21swapper/207:06:492
154402294,17sleep20-21swapper/207:06:492
1597899282,16cyclictest7950irq/56-dwc_otg_08:53:062
1597399281,12cyclictest221ktimersoftd/107:10:281
1597899273,4cyclictest7850irq/56-dwc_otg08:10:242
1597899272,12cyclictest8050irq/56-dwc_otg_08:20:402
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional