You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-20 - 14:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sat Dec 20, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3096723627,0sleep30-21swapper/307:09:133
3091623423,0sleep00-21swapper/007:08:380
3060123425,0sleep10-21swapper/107:04:551
3078423127,0sleep20-21swapper/207:07:042
31047991919,0cyclictest29782-21kworker/2:111:27:582
339960,0migration/37308-21sh09:32:543
339960,0migration/320721-21sh10:55:073
310479966,0cyclictest0-21swapper/212:14:212
310479965,0cyclictest24926-21ssh09:44:062
310469966,0cyclictest0-21swapper/109:49:341
310469960,0cyclictest19782-21ssh10:36:141
310459960,0cyclictest0-21swapper/010:14:420
29725060,0irq/27-eth0-tx-5565-21munin-node07:14:330
29725060,0irq/27-eth0-tx-2747-21munin-node08:49:400
259960,0migration/216962-21sh09:56:462
149960,0migration/029170-21sh12:38:580
149960,0migration/01318-21latency10:24:370
339950,0migration/38090-21kthreadcore10:49:363
339950,0migration/36319-21idleruntime-cro07:39:193
339950,0migration/327677-21sh12:35:273
339950,0migration/327128-21diskmemload10:22:113
339950,0migration/326939-21aten_r1power_cu11:59:283
339950,0migration/322473-21sh11:15:563
339950,0migration/320216-21runrttasks07:49:323
339950,0migration/318771-21sh09:59:313
339950,0migration/317644-21kthreadcore08:59:343
339950,0migration/317089-21sh11:34:233
339950,0migration/31635-21sendmail_mailtr12:19:433
339950,0migration/313815-21fschecks_time07:44:333
339950,0migration/311950-21sh09:15:003
339950,0migration/311786-21runrttasks12:27:473
339950,0migration/310295-21taskset10:12:203
310479955,0cyclictest0-21swapper/210:04:362
310479955,0cyclictest0-21swapper/208:34:442
310479955,0cyclictest0-21swapper/207:39:332
310479950,0cyclictest0-21swapper/212:28:042
310469955,0cyclictest0-21swapper/111:52:511
310469955,0cyclictest0-21swapper/107:29:431
310469954,0cyclictest3514-21snmpd10:22:101
310469950,0cyclictest0-21swapper/112:08:461
310469950,0cyclictest0-21swapper/111:45:171
310469950,0cyclictest0-21swapper/111:41:301
310469950,0cyclictest0-21swapper/111:20:221
310469950,0cyclictest0-21swapper/109:21:491
310469950,0cyclictest0-21swapper/109:21:491
310469950,0cyclictest0-21swapper/109:16:271
310459950,0cyclictest0-21swapper/011:01:310
310459950,0cyclictest0-21swapper/010:32:120
29725050,0irq/27-eth0-tx-5578-21munin-node09:14:382
29725050,0irq/27-eth0-tx-32215-21munin-node07:34:400
29725050,0irq/27-eth0-tx-29561-21kthreadcore09:44:362
29725050,0irq/27-eth0-tx-29486-21ssh09:09:280
29725050,0irq/27-eth0-tx-29240-21diskmemload09:30:232
29725050,0irq/27-eth0-tx-29025-21ssh10:43:082
29725050,0irq/27-eth0-tx-27533-21munin-node08:19:430
29725050,0irq/27-eth0-tx-25547-21munin-node11:39:352
29725050,0irq/27-eth0-tx-18094-21kthreadcore10:54:372
29725050,0irq/27-eth0-tx-1566-21munin-node11:44:322
29725050,0irq/27-eth0-tx-0-21swapper/212:20:262
29725050,0irq/27-eth0-tx-0-21swapper/211:38:012
29725050,0irq/27-eth0-tx-0-21swapper/211:32:342
29725050,0irq/27-eth0-tx-0-21swapper/210:34:432
29725050,0irq/27-eth0-tx-0-21swapper/210:30:322
29725050,0irq/27-eth0-tx-0-21swapper/210:23:532
29725050,0irq/27-eth0-tx-0-21swapper/210:11:562
29725050,0irq/27-eth0-tx-0-21swapper/209:59:272
29725050,0irq/27-eth0-tx-0-21swapper/209:49:362
29725050,0irq/27-eth0-tx-0-21swapper/209:27:572
29715050,0irq/26-eth0-rx-8667-21kthreadcore11:09:353
29715050,0irq/26-eth0-rx-5697-21kthreadcore08:24:373
29715050,0irq/26-eth0-rx-26302-21ssh12:15:333
29715050,0irq/26-eth0-rx-25019-21tr09:04:363
29715050,0irq/26-eth0-rx-21651-21kthreadcore11:54:353
29715050,0irq/26-eth0-rx-1916-21grep10:44:363
29715050,0irq/26-eth0-rx-16453-21ssh09:55:353
29715050,0irq/26-eth0-rx-1259-21ssh09:49:203
29715050,0irq/26-eth0-rx-0-21swapper/312:05:073
29715050,0irq/26-eth0-rx-0-21swapper/311:44:323
29715050,0irq/26-eth0-rx-0-21swapper/311:28:303
29715050,0irq/26-eth0-rx-0-21swapper/311:04:363
29715050,0irq/26-eth0-rx-0-21swapper/310:41:503
29715050,0irq/26-eth0-rx-0-21swapper/310:39:113
29715050,0irq/26-eth0-rx-0-21swapper/310:30:373
29715050,0irq/26-eth0-rx-0-21swapper/310:24:303
29715050,0irq/26-eth0-rx-0-21swapper/309:44:303
29715050,0irq/26-eth0-rx-0-21swapper/309:41:583
29715050,0irq/26-eth0-rx-0-21swapper/309:24:353
29715050,0irq/26-eth0-rx-0-21swapper/309:12:323
259950,0migration/26752-21munin-node10:49:332
259950,0migration/26128-21fschecks_time07:14:322
259950,0migration/230416-21if_eth009:09:342
259950,0migration/224844-21kthreadcore08:39:352
19150,0ktimersoftd/10-21swapper/112:30:221
179950,0migration/17311-21ntp_kernel_err08:24:391
179950,0migration/129306-21latency_hist09:09:191
179950,0migration/124591-21kthreadcore09:24:361
179950,0migration/121588-21sh12:34:331
179950,0migration/121504-21kthreadcore11:54:351
179950,0migration/121407-21taskset10:56:441
179950,0migration/112796-21kthreadcore12:09:341
169950,0watchdog/123209-21munin-node10:59:321
169950,0watchdog/115153-21ssh09:36:121
169950,0watchdog/113116-21kthreadcore11:29:361
149950,0migration/07812-21cpuspeed_turbos08:04:310
149950,0migration/06706-21timerandwakeup07:59:420
149950,0migration/05548-21sh09:14:260
149950,0migration/04746-21kthreadcore12:04:330
149950,0migration/03187-21kthreadcore11:44:360
149950,0migration/031575-21kthreadcore12:19:390
149950,0migration/031495-21timerandwakeup11:39:420
149950,0migration/031332-21kthreadcore10:44:350
149950,0migration/028801-21kthreadcore07:29:380
149950,0migration/026930-21df_abs07:54:320
149950,0migration/02624-21idleruntime08:24:340
149950,0migration/025804-21unixbench-2d12:14:420
149950,0migration/025385-21aten_r1power_fr09:44:300
149950,0migration/022225-21sh10:58:410
149950,0migration/021545-21aten_r1power_po10:39:310
149950,0migration/021311-21munin-node08:14:320
149950,0migration/020797-21latency_hist08:14:190
149950,0migration/019985-21kthreadcore09:59:350
149950,0migration/019949-21kthreadcore11:54:350
149950,0migration/018719-21timerwakeupswit12:29:420
149950,0migration/016084-21latency_hist08:59:190
149950,0migration/015954-21latency_hist08:59:190
149950,0migration/015754-21sh11:14:240
149950,0migration/015612-21fschecks_count08:34:340
149950,0migration/013919-21if_eth110:34:340
149950,0migration/013436-21kthreadcore09:34:350
149950,0migration/012136-21taskset12:09:310
149950,0migration/010421-21sh10:12:370
35140,0ktimersoftd/30-21swapper/308:36:533
339940,0migration/327932-21kthreadcore07:54:363
339940,0migration/323798-21kthreadcore08:14:363
339940,0migration/323648-21kthreadcore08:39:353
339940,0migration/321609-21kthreadcore07:24:393
339940,0migration/316765-21kthreadcore10:14:383
310479944,0cyclictest0-21swapper/208:49:332
310479943,0cyclictest3514-21snmpd08:59:252
310479943,0cyclictest3514-21snmpd07:38:402
310479943,0cyclictest3514-21snmpd07:22:402
310479943,0cyclictest32192-21awk07:09:352
310479943,0cyclictest26469-21awk07:29:362
310479943,0cyclictest25131-21awk11:19:332
310469944,0cyclictest29240-21diskmemload09:56:301
310469944,0cyclictest0-21swapper/111:09:481
310469944,0cyclictest0-21swapper/109:46:051
310469943,0cyclictest3556-21lldpd07:51:461
310469943,0cyclictest3514-21snmpd10:16:401
310469943,0cyclictest3514-21snmpd08:34:251
310469943,0cyclictest29225-21awk12:19:371
310469943,0cyclictest14147-21awk07:44:351
310469943,0cyclictest10425-21awk08:54:351
310459943,0cyclictest6812-21awk10:49:340
310459943,0cyclictest29352-21awk08:44:340
310459942,0cyclictest30748-21kthreadcore10:04:370
310459940,0cyclictest0-21swapper/011:50:210
29725040,0irq/27-eth0-tx-6670-21aten2.4-expect07:39:310
29725040,0irq/27-eth0-tx-29762-21ssh10:44:222
29725040,0irq/27-eth0-tx-28298-21ssh10:24:232
29725040,0irq/27-eth0-tx-26853-21ssh11:59:202
29725040,0irq/27-eth0-tx-22759-21ssh10:59:242
29725040,0irq/27-eth0-tx-0-21swapper/212:34:382
29725040,0irq/27-eth0-tx-0-21swapper/212:09:502
29725040,0irq/27-eth0-tx-0-21swapper/211:54:302
29725040,0irq/27-eth0-tx-0-21swapper/211:49:202
29725040,0irq/27-eth0-tx-0-21swapper/211:14:332
29725040,0irq/27-eth0-tx-0-21swapper/211:09:302
29725040,0irq/27-eth0-tx-0-21swapper/209:19:292
29725040,0irq/27-eth0-tx-0-21swapper/209:19:292
29725040,0irq/27-eth0-tx-0-21swapper/008:39:380
29725040,0irq/27-eth0-tx-0-21swapper/007:49:330
29715040,0irq/26-eth0-rx-12802-21aten_r1power_fr12:29:313
29715040,0irq/26-eth0-rx-0-21swapper/312:09:343
29715040,0irq/26-eth0-rx-0-21swapper/311:49:333
29715040,0irq/26-eth0-rx-0-21swapper/311:42:353
29715040,0irq/26-eth0-rx-0-21swapper/311:29:313
29715040,0irq/26-eth0-rx-0-21swapper/311:19:313
29715040,0irq/26-eth0-rx-0-21swapper/310:59:423
29715040,0irq/26-eth0-rx-0-21swapper/310:05:143
29715040,0irq/26-eth0-rx-0-21swapper/309:34:333
29715040,0irq/26-eth0-rx-0-21swapper/309:21:123
29715040,0irq/26-eth0-rx-0-21swapper/309:21:123
29715040,0irq/26-eth0-rx-0-21swapper/308:54:313
29715040,0irq/26-eth0-rx-0-21swapper/308:09:333
259940,0migration/29557-21munin-node08:54:312
259940,0migration/28536-21cpu09:34:292
259940,0migration/231406-21kthreadcore08:44:362
259940,0migration/230092-21kthreadcore07:54:372
259940,0migration/22997-21aten_r1power_po12:04:282
259940,0migration/222457-21kthreadcore07:24:402
259940,0migration/214984-21kthreadcore10:14:372
259940,0migration/212926-21kthreadcore08:29:382
249940,0watchdog/27525-21munin-node08:24:412
249940,0watchdog/226578-21switchtime08:14:412
249940,0watchdog/226114-21kthreadcore09:04:372
249940,0watchdog/222974-21kthreadcore07:49:372
249940,0watchdog/22066-21kthreadcore07:59:372
249940,0watchdog/214038-21munin-node12:29:362
249940,0watchdog/20-21swapper/211:04:202
249940,0watchdog/20-21swapper/208:19:372
249940,0watchdog/20-21swapper/207:44:372
179940,0migration/19377-21kthreadcore07:39:371
179940,0migration/16652-21gltestperf10:49:331
179940,0migration/1648-21latency_hist07:59:191
179940,0migration/16179-21sh09:30:371
179940,0migration/1608-21munin-node11:24:331
179940,0migration/14364-21munin-node12:24:301
179940,0migration/13378-21timerandwakeup10:44:411
179940,0migration/130859-21kthreadcore08:44:361
179940,0migration/128145-21irqrtprio11:59:331
179940,0migration/125890-21kthreadcore08:39:351
179940,0migration/122971-21kthreadcore07:24:401
179940,0migration/122212-21hddtemp_smartct10:39:331
179940,0migration/120811-21kthreadcore09:39:381
179940,0migration/118300-21taskset07:19:491
179940,0migration/11821-21kthreadcore09:09:371
179940,0migration/116495-21kthreadcore08:09:371
179940,0migration/11535-21kthreadcore11:04:351
179940,0migration/112456-21kthreadcore08:04:361
169940,0watchdog/18859-21df08:29:331
169940,0watchdog/15520-21mailstats07:34:411
169940,0watchdog/14891-21fschecks_count10:29:321
169940,0watchdog/13777-21ssh10:28:281
169940,0watchdog/13671-21cut07:09:371
169940,0watchdog/127052-21fschecks_count07:54:331
169940,0watchdog/126577-21switchtime08:14:411
169940,0watchdog/122957-21rm11:17:041
169940,0watchdog/110845-21ssh10:13:361
169940,0watchdog/10-21swapper/110:04:241
169940,0watchdog/10-21swapper/108:19:331
149940,0migration/06186-21sh11:07:220
149940,0migration/03787-21kthreadcore11:24:370
149940,0migration/03689-21kthreadcore09:29:370
149940,0migration/023087-21kthreadcore07:24:400
149940,0migration/017687-21hwlatdetect09:39:350
149940,0migration/017626-21munin-node11:34:330
149940,0migration/013797-21kthreadcore07:19:370
149940,0migration/010998-21irqcore09:54:340
339930,0migration/36104-21fschecks_count07:14:323
310479932,0cyclictest12656-21kthreadcore08:04:372
310469932,0cyclictest6490-21awk07:14:341
310469932,0cyclictest21788-21sendmail_mailqu08:59:391
310469932,0cyclictest20968-21sed12:14:351
310459933,0cyclictest5336-21awk12:24:340
310459933,0cyclictest4696-21kworker/0:211:59:230
310459933,0cyclictest4696-21kworker/0:211:29:410
310459933,0cyclictest4696-21kworker/0:211:19:350
310459933,0cyclictest4696-21kworker/0:211:09:330
310459933,0cyclictest10038-21kworker/0:209:49:300
310459933,0cyclictest10038-21kworker/0:209:25:280
310459933,0cyclictest10038-21kworker/0:209:19:260
310459933,0cyclictest10038-21kworker/0:209:19:260
310459933,0cyclictest10038-21kworker/0:209:04:420
310459933,0cyclictest10038-21kworker/0:208:29:440
310459933,0cyclictest10038-21kworker/0:207:44:200
310459933,0cyclictest10038-21kworker/0:207:09:380
310459932,0cyclictest20758-21basename10:19:350
29715030,0irq/26-eth0-rx-0-21swapper/308:50:143
29715030,0irq/26-eth0-rx-0-21swapper/308:19:303
29715030,0irq/26-eth0-rx-0-21swapper/308:04:423
29715030,0irq/26-eth0-rx-0-21swapper/307:59:323
29715030,0irq/26-eth0-rx-0-21swapper/307:39:383
29715030,0irq/26-eth0-rx-0-21swapper/307:22:453
29715030,0irq/26-eth0-rx-0-21swapper/307:09:303
169930,0watchdog/119154-21smartctl09:59:321
169930,0watchdog/10-21swapper/111:34:361
339920,0migration/326565-21iostat_ios07:29:373
310469920,0cyclictest2716-21cat08:49:191
29715020,0irq/26-eth0-rx-0-21swapper/308:44:273
29715020,0irq/26-eth0-rx-0-21swapper/308:29:373
249920,0watchdog/20-21swapper/208:09:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional