You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-08 - 00:39

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Tue Jul 07, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1871125610,2sleep30-21swapper/307:05:403
1970199953,68cyclictest27421-21ssh11:02:431
1970499923,24cyclictest8050irq/56-dwc_otg_11:02:452
19696998143,9cyclictest8050irq/56-dwc_otg_12:30:270
298772800,2sleep30-21swapper/310:25:573
191042800,2sleep10-21swapper/110:14:001
43612780,3sleep04340-21ssh10:35:280
93822770,4sleep29360-21ssh11:20:322
9112770,5sleep01969699cyclictest11:49:490
9112770,5sleep01969699cyclictest11:49:490
123242770,2sleep00-21swapper/010:45:150
123242770,2sleep00-21swapper/010:45:150
261842750,5sleep21970499cyclictest12:20:152
277132740,7sleep27950irq/56-dwc_otg_10:24:312
1970199733,57cyclictest3500-21/usr/sbin/munin09:55:441
19705997133,10cyclictest26418-21apt-get12:20:263
1969699695,9cyclictest0-21swapper/011:02:420
1969699693,8cyclictest0-21swapper/008:50:250
19696996913,52cyclictest9-21ksoftirqd/009:25:260
1969699681,59cyclictest0-21swapper/009:50:540
19696996813,10cyclictest0-21swapper/012:25:270
136822680,2sleep30-21swapper/312:05:143
19704996725,5cyclictest27760-21wget09:45:292
19705996624,23cyclictest21609-21diskstats11:35:303
1970199665,4cyclictest7950irq/56-dwc_otg_11:35:591
1970199664,47cyclictest8050irq/56-dwc_otg_12:25:501
112762660,2sleep00-21swapper/012:01:000
1970199658,4cyclictest8050irq/56-dwc_otg_12:20:141
1970199651,46cyclictest2547-21ssh09:15:191
19705996437,22cyclictest27428-21ssh11:02:443
1970199641,49cyclictest0-21swapper/111:41:331
1970199639,39cyclictest0-21swapper/110:31:101
1970199638,31cyclictest0-21swapper/110:08:421
1970199634,6cyclictest8050irq/56-dwc_otg_10:50:411
1970199631,49cyclictest0-21swapper/112:23:011
19696996314,31cyclictest27690-21/usr/sbin/munin09:45:290
19696996313,5cyclictest0-21swapper/007:30:280
1970199625,4cyclictest0-21swapper/112:35:291
1970199624,5cyclictest7850irq/56-dwc_otg12:30:271
1970199624,46cyclictest8050irq/56-dwc_otg_09:10:211
1969699625,7cyclictest0-21swapper/010:30:270
19705996124,13cyclictest2188-21apt-get09:15:153
1970199613,44cyclictest31399-21ssh11:07:301
1970199613,39cyclictest8050irq/56-dwc_otg_09:49:251
1969699615,30cyclictest0-21swapper/010:10:320
1969699612,8cyclictest32237-21grep09:10:440
1969699612,5cyclictest29351-21munin-run11:45:140
19705996019,26cyclictest0-21swapper/308:00:273
19705996019,26cyclictest0-21swapper/308:00:273
1970199605,47cyclictest8050irq/56-dwc_otg_10:17:321
1970199603,7cyclictest4253-21/usr/sbin/munin10:35:401
1970199601,49cyclictest0-21swapper/108:50:341
1969699609,8cyclictest0-21swapper/012:20:260
19705995929,7cyclictest28919-21apt-get07:35:263
19705995918,21cyclictest2502-21gpgconf09:15:163
1970199595,4cyclictest7850irq/56-dwc_otg11:20:351
1970199593,7cyclictest29249-21awk10:25:391
1970199592,6cyclictest7850irq/56-dwc_otg08:10:401
1970199592,48cyclictest12814-21df_inode10:45:301
1969699595,23cyclictest0-21swapper/008:20:270
19696995936,5cyclictest7850irq/56-dwc_otg10:00:130
19696995914,25cyclictest101ktimersoftd/011:15:280
19705995832,10cyclictest23677-21apt-get08:50:253
1970199586,44cyclictest8050irq/56-dwc_otg_09:37:031
1970199584,43cyclictest8050irq/56-dwc_otg_09:20:251
1970199581,9cyclictest7950irq/56-dwc_otg_09:45:131
1970199581,9cyclictest0-21swapper/109:33:011
1970199581,10cyclictest8050irq/56-dwc_otg_11:55:321
1969699585,47cyclictest0-21swapper/009:15:280
1969699582,7cyclictest22131-21grep10:55:430
1969699582,22cyclictest0-21swapper/011:35:430
19705995714,25cyclictest1576-21sh11:10:383
19705995714,25cyclictest1576-21sh11:10:383
1970199572,4cyclictest8050irq/56-dwc_otg_07:35:371
19701995716,38cyclictest1654-21cron07:50:141
1970199571,4cyclictest7850irq/56-dwc_otg12:09:071
1970199571,4cyclictest7850irq/56-dwc_otg11:10:411
1970199571,4cyclictest7850irq/56-dwc_otg11:10:411
1970199571,47cyclictest12376-21latency_hist10:45:151
1970199571,47cyclictest12376-21latency_hist10:45:151
1970199571,44cyclictest7950irq/56-dwc_otg_09:50:191
19701995711,27cyclictest0-21swapper/111:16:481
1969699575,30cyclictest0-21swapper/008:55:310
19705995621,11cyclictest17131-21sed11:30:153
1970199566,39cyclictest8050irq/56-dwc_otg_10:01:091
1970199565,3cyclictest7850irq/56-dwc_otg11:45:421
1970199565,3cyclictest7850irq/56-dwc_otg11:45:421
1970199564,5cyclictest7950irq/56-dwc_otg_09:10:141
1970199564,5cyclictest0-21swapper/109:25:261
1969699564,30cyclictest30649-21cron09:10:140
19696995631,12cyclictest8050irq/56-dwc_otg_08:00:140
1969699561,6cyclictest16391-21idleruntime-cro10:50:130
1969699561,31cyclictest19764-21tr07:10:280
1969699561,29cyclictest20209-21aten_rbpower_en10:15:280
19696995610,26cyclictest16113-21basename08:30:140
19705995520,12cyclictest6673-21ssh09:20:163
19705995519,26cyclictest8606-21ssh12:37:233
19705995518,13cyclictest24120-21latency_hist10:20:153
1970199557,40cyclictest8050irq/56-dwc_otg_11:27:111
1970199554,4cyclictest8050irq/56-dwc_otg_11:34:441
1970199551,45cyclictest0-21swapper/108:45:321
1970199551,42cyclictest0-21swapper/108:26:421
19696995521,13cyclictest0-21swapper/007:20:240
1969699551,6cyclictest9650irq/80-mmc011:15:150
1969699551,6cyclictest9650irq/80-mmc011:15:150
19705995421,21cyclictest31972-21ssh12:26:063
19705995417,15cyclictest27236-21ssh11:41:223
19705995415,26cyclictest3705-21ssh11:52:193
19705995415,22cyclictest7950irq/56-dwc_otg_07:48:513
1970199547,43cyclictest27309-21aten_rbpower_cu09:00:271
1970199541,5cyclictest11104-21ssh12:00:441
19696995431,5cyclictest7950irq/56-dwc_otg_12:35:290
19696995430,4cyclictest7850irq/56-dwc_otg11:30:140
1969699542,31cyclictest0-21swapper/010:00:430
19696995420,19cyclictest25287-21aten_rbpower_po07:25:270
19696995410,6cyclictest1394850ssh12:05:250
19705995321,12cyclictest20191-21ssh12:12:063
19705995320,17cyclictest3793-21rm12:31:223
19705995319,12cyclictest22807-21ssh09:39:373
1970199533,4cyclictest8050irq/56-dwc_otg_10:20:271
1970199531,7cyclictest25557-21grep08:55:281
1970199531,4cyclictest7950irq/56-dwc_otg_12:14:421
1970199531,39cyclictest0-21swapper/110:56:331
1969699537,20cyclictest9-21ksoftirqd/007:40:270
19696995332,4cyclictest8050irq/56-dwc_otg_11:54:380
19696995322,4cyclictest22681-21unixbench_singl07:15:430
19705995222,20cyclictest22662-21ssh10:56:333
19705995221,22cyclictest13850-21ssh09:28:273
19705995218,25cyclictest23219-21taskset08:47:463
19705995218,17cyclictest5770-21ssh09:57:243
19705995217,25cyclictest10909-21ssh11:21:493
19705995214,26cyclictest16262-21ssh11:29:123
1970499528,30cyclictest0-21swapper/209:15:082
19701995215,34cyclictest30972-21fschecks_count07:40:311
1969699529,25cyclictest17559-21unixbench_multi08:30:430
1969699529,25cyclictest17559-21unixbench_multi08:30:430
1969699526,26cyclictest0-21swapper/008:10:270
19696995234,5cyclictest8050irq/56-dwc_otg_07:50:360
113232520,2sleep30-21swapper/310:43:373
113232520,2sleep30-21swapper/310:43:373
19705995125,12cyclictest32277-21cat09:50:333
19705995123,13cyclictest25994-21ssh12:20:063
19705995121,17cyclictest6479-21ssh11:16:153
19705995113,21cyclictest24918-21ssh10:20:353
1970499514,11cyclictest7950irq/56-dwc_otg_12:30:272
1970199516,4cyclictest5750-21fschecks_count08:00:311
1970199516,4cyclictest5750-21fschecks_count08:00:311
1970199512,6cyclictest19813-21aten_rbpower_po07:10:281
1970199512,34cyclictest0-21swapper/108:37:211
1970199511,4cyclictest7850irq/56-dwc_otg07:30:391
1970199511,41cyclictest0-21swapper/107:25:261
19696995126,5cyclictest8050irq/56-dwc_otg_09:00:270
19705995023,12cyclictest9769-21/usr/sbin/munin08:10:353
19705995021,13cyclictest10906-21/usr/sbin/munin08:15:373
19705995020,18cyclictest25746-21ssh09:42:253
19705995017,21cyclictest29942-21taskset09:47:283
19705995017,18cyclictest11828-21ssh10:05:253
1970199502,6cyclictest20327-21fschecks_count08:40:321
1970199501,7cyclictest24520-21grep07:20:421
1970199501,7cyclictest12548-21latency_hist08:20:141
1970199501,37cyclictest8050irq/56-dwc_otg_11:55:131
19701995010,37cyclictest7672-21idleruntime08:05:321
1969699506,27cyclictest7674-21ls08:05:320
1969699501,28cyclictest29229-21tr11:05:290
19696995011,7cyclictest6851-21aten_rbpower_en09:20:280
1969699501,15cyclictest16965-21ssh10:50:290
19705994921,14cyclictest5642-21/usr/sbin/munin11:55:253
19705994920,11cyclictest9432-21ssh10:01:303
19705994916,14cyclictest31603-21rm11:46:373
19705994916,14cyclictest31603-21rm11:46:373
19705994914,13cyclictest17125-21ssh10:50:323
19705994913,27cyclictest0-21swapper/310:35:253
1970199492,5cyclictest22385-21switchtime07:15:401
1970199492,5cyclictest16532-21aten_rbpower_po08:30:301
1970199492,5cyclictest16532-21aten_rbpower_po08:30:301
1970199491,6cyclictest12828-21cut08:20:301
1969699492,25cyclictest0-21swapper/008:35:330
19696994921,20cyclictest0-21swapper/010:25:440
1969699491,29cyclictest0-21swapper/009:35:430
19705994817,12cyclictest1741-21sh10:31:173
1969699484,15cyclictest0-21swapper/010:25:010
19696994820,8cyclictest0-21swapper/009:40:420
19696994820,4cyclictest0-21swapper/011:30:410
1969699481,25cyclictest8982-21/usr/sbin/munin11:20:330
1969699481,25cyclictest0-21swapper/010:05:360
192572483,12sleep00-21swapper/007:08:050
19705994718,11cyclictest9966-1kworker/u9:211:06:203
19705994716,11cyclictest31947-21taskset07:41:063
19705994714,21cyclictest750-21snmpd09:04:053
19705994714,11cyclictest12590-21ssh10:45:273
1970499474,28cyclictest0-21swapper/209:55:242
1970199472,5cyclictest0-21swapper/107:55:281
1970199471,8cyclictest2851-21timerandwakeup07:50:391
19696994728,6cyclictest8050irq/56-dwc_otg_07:35:430
19705994619,18cyclictest18451-21ssh10:12:493
19705994619,13cyclictest27296-21fschecks_time07:30:343
19696994621,4cyclictest518-21rs:main0
1969699461,5cyclictest13-21rcuc/012:00:140
19705994520,18cyclictest18415-21rm09:34:123
19705994520,13cyclictest26874-21latency_hist07:30:143
19705994511,21cyclictest8050irq/56-dwc_otg_08:26:413
1970499452,9cyclictest0-21swapper/208:20:272
19696994514,22cyclictest18994-21latency_hist09:35:140
19705994415,12cyclictest5194-21taskset07:59:263
19705994414,13cyclictest17295-21ssh12:09:243
19705994413,9cyclictest0-21swapper/308:23:543
1969699441,22cyclictest6036-21grep08:00:340
1969699441,22cyclictest6036-21grep08:00:340
19705994319,12cyclictest19791-21taskset08:39:593
1969699431,6cyclictest18065-21/usr/sbin/munin12:10:390
1970599427,23cyclictest0-21swapper/307:50:263
19705994212,10cyclictest0-21swapper/307:22:163
19705994210,22cyclictest26706-21runrttasks08:56:163
1970499428,13cyclictest0-21swapper/209:15:282
1970499421,24cyclictest0-21swapper/209:50:252
19705994113,15cyclictest7850irq/56-dwc_otg07:19:433
1970499417,7cyclictest7850irq/56-dwc_otg11:35:442
1970499413,8cyclictest0-21swapper/207:30:292
1970499408,24cyclictest0-21swapper/207:20:272
19704994017,5cyclictest727-21runrttasks10:55:142
19696994027,4cyclictest8050irq/56-dwc_otg_07:48:510
19696994015,15cyclictest0-21swapper/008:45:380
1969699401,25cyclictest0-21swapper/008:40:250
1970599397,25cyclictest0-21swapper/307:10:283
19705993922,14cyclictest20524-21irqstats08:40:343
19705993914,15cyclictest8652-21taskset08:07:353
1970499397,24cyclictest0-21swapper/207:25:262
1970499395,20cyclictest4296-21ssh10:35:282
1970499392,23cyclictest7850irq/56-dwc_otg11:40:312
19696993912,6cyclictest12075-21grep08:15:400
247942380,2sleep20-21swapper/211:00:142
1970599389,15cyclictest17945-21runrttasks08:34:403
1970599389,15cyclictest17945-21runrttasks08:34:403
1970499386,21cyclictest0-21swapper/210:49:502
1970499382,25cyclictest11814-21/usr/sbin/munin10:05:372
1970499382,22cyclictest0-21swapper/207:55:272
19704993818,9cyclictest29510-21timerandwakeup10:25:422
19705993713,15cyclictest30568-21taskset09:09:283
1970499374,11cyclictest0-21swapper/211:30:202
19704993721,5cyclictest18359-21ssh09:34:062
1970499371,8cyclictest20373-21ssh09:35:412
19704993714,5cyclictest27360-21expr12:20:412
1970499366,19cyclictest7950irq/56-dwc_otg_12:25:422
19704993611,18cyclictest0-21swapper/210:11:452
1970499359,13cyclictest0-21swapper/209:25:322
1970499351,23cyclictest0-21swapper/211:17:122
1970499349,5cyclictest7850irq/56-dwc_otg08:28:582
1970499347,14cyclictest0-21swapper/210:02:252
1970499341,3cyclictest8050irq/56-dwc_otg_08:10:252
1970499341,23cyclictest0-21swapper/210:40:312
1970499341,23cyclictest0-21swapper/210:40:312
19704993411,5cyclictest762-21/usr/sbin/munin11:10:252
19704993411,5cyclictest762-21/usr/sbin/munin11:10:252
19704993411,12cyclictest0-21swapper/211:50:152
19704993411,12cyclictest0-21swapper/211:50:152
186342345,18sleep218710-21sensors_temp07:05:402
1970499339,6cyclictest7850irq/56-dwc_otg08:30:272
1970499339,6cyclictest7850irq/56-dwc_otg08:30:272
1970499336,4cyclictest7850irq/56-dwc_otg09:00:252
1970499335,19cyclictest7950irq/56-dwc_otg_11:55:362
1970499333,5cyclictest518-21rs:main2
1970499332,15cyclictest2250-21tr11:50:372
1970499331,6cyclictest21689-21latency_hist08:45:142
1970499331,29cyclictest21797-21grep07:15:322
19704993310,5cyclictest14243-21sed11:25:412
1970499329,12cyclictest0-21swapper/212:10:062
1970499328,4cyclictest0-21swapper/210:18:242
1970499327,15cyclictest0-21swapper/207:35:192
1970499327,11cyclictest8050irq/56-dwc_otg_08:35:342
1970499325,18cyclictest0-21swapper/211:08:012
19704993222,7cyclictest301ktimersoftd/209:20:282
1970499322,21cyclictest7950irq/56-dwc_otg_08:00:322
1970499322,21cyclictest7950irq/56-dwc_otg_08:00:322
1970499318,6cyclictest7850irq/56-dwc_otg08:50:402
1970499317,14cyclictest0-21swapper/212:00:292
1970499315,10cyclictest7950irq/56-dwc_otg_07:40:282
1970499313,6cyclictest32592-21aten_rbpower_cu10:30:282
1970499313,20cyclictest7850irq/56-dwc_otg08:17:572
1970499311,7cyclictest8873-21cron08:10:132
19704993112,5cyclictest19009-21/usr/sbin/munin12:10:392
1970499307,14cyclictest30663-21munin-run09:10:142
19704993021,5cyclictest32766-21fschecks_count07:45:312
1970499301,5cyclictest6470-21aten_rbpower_po12:35:282
1970499301,4cyclictest0-21swapper/208:55:312
1970499296,12cyclictest8050irq/56-dwc_otg_07:10:282
1970499293,19cyclictest7950irq/56-dwc_otg_08:45:362
1970499286,10cyclictest0-21swapper/207:52:322
185872276,10sleep10-21swapper/107:05:391
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional