You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 22:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Tue Dec 02, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1814828482,0sleep30-21swapper/306:36:453
1823023624,0sleep00-21swapper/006:37:420
1830923525,0sleep20-21swapper/206:38:362
1198623526,0sleep10-21swapper/106:34:231
18431991111,0cyclictest14992-21awk07:24:361
169970,0watchdog/10-21swapper/108:09:331
339960,0migration/36225-21sh11:36:183
339960,0migration/3484-21sh10:51:363
339960,0migration/331356-21sh08:45:513
339960,0migration/330132-21sh09:04:433
28415060,0irq/27-eth0-tx-29504-21munin-node09:49:352
28415060,0irq/27-eth0-tx-0-21swapper/212:00:052
28415060,0irq/27-eth0-tx-0-21swapper/211:39:432
28415060,0irq/27-eth0-tx-0-21swapper/211:12:112
28415060,0irq/27-eth0-tx-0-21swapper/210:44:442
28415060,0irq/27-eth0-tx-0-21swapper/210:26:532
28415060,0irq/27-eth0-tx-0-21swapper/210:19:272
28415060,0irq/27-eth0-tx-0-21swapper/210:12:522
28415060,0irq/27-eth0-tx-0-21swapper/209:35:332
28415060,0irq/27-eth0-tx-0-21swapper/209:28:512
28415060,0irq/27-eth0-tx-0-21swapper/209:23:582
28415060,0irq/27-eth0-tx-0-21swapper/209:14:542
28415060,0irq/27-eth0-tx-0-21swapper/208:54:582
28415060,0irq/27-eth0-tx-0-21swapper/208:41:582
28415060,0irq/27-eth0-tx-0-21swapper/008:34:380
28405060,0irq/26-eth0-rx-8729-21diskstats07:44:293
28405060,0irq/26-eth0-rx-25230-21swap10:24:383
28405060,0irq/26-eth0-rx-0-21swapper/311:30:583
249960,0watchdog/20-21swapper/209:39:332
184329966,0cyclictest16908-21diskmemload10:36:182
184319960,0cyclictest16908-21diskmemload10:41:541
184319960,0cyclictest0-21swapper/110:31:391
184309965,0cyclictest5469-21kthreadcore07:39:320
184309960,0cyclictest0-21swapper/011:29:430
149960,0migration/07091-21sh08:50:480
8225050,0irq/24-0000:00:0-21swapper/209:44:332
339950,0migration/39834-21munin-node10:14:383
339950,0migration/39529-21munin-node08:29:373
339950,0migration/38765-21kthreadcore11:59:313
339950,0migration/3874-21kthreadcore06:49:313
339950,0migration/35235-21kthreadcore08:49:313
339950,0migration/34364-21timerandwakeup11:54:383
339950,0migration/331912-21munin-node09:29:313
339950,0migration/330948-21munin-node06:44:383
339950,0migration/32901-21sh10:12:063
339950,0migration/327777-21diskmemload10:08:223
339950,0migration/325937-21munin-node11:09:313
339950,0migration/325262-21kthreadcore09:44:363
339950,0migration/324499-21kthreadcore09:24:303
339950,0migration/323984-21kthreadcore07:04:343
339950,0migration/319586-21if_eth1.206:39:293
339950,0migration/318984-21taskset09:59:583
339950,0migration/317418-21cpuspeed_turbos08:39:283
339950,0migration/316469-21diskmemload10:39:393
339950,0migration/31570-21sendmail_mailqu07:59:353
339950,0migration/315256-21munin-node09:19:273
339950,0migration/31344-21gltestperf11:14:283
339950,0migration/312164-21kthreadcore07:19:303
339950,0migration/311425-21kthreadcore10:59:313
28415050,0irq/27-eth0-tx-6445-21snmpd11:16:342
28415050,0irq/27-eth0-tx-4465-21ssh09:29:492
28415050,0irq/27-eth0-tx-17165-21munin-node08:39:330
28415050,0irq/27-eth0-tx-16908-21diskmemload10:03:202
28415050,0irq/27-eth0-tx-0-21swapper/211:56:482
28415050,0irq/27-eth0-tx-0-21swapper/211:53:192
28415050,0irq/27-eth0-tx-0-21swapper/211:36:552
28415050,0irq/27-eth0-tx-0-21swapper/211:29:232
28415050,0irq/27-eth0-tx-0-21swapper/211:19:192
28415050,0irq/27-eth0-tx-0-21swapper/211:01:002
28415050,0irq/27-eth0-tx-0-21swapper/210:56:572
28415050,0irq/27-eth0-tx-0-21swapper/210:39:392
28415050,0irq/27-eth0-tx-0-21swapper/210:29:252
28415050,0irq/27-eth0-tx-0-21swapper/210:17:502
28415050,0irq/27-eth0-tx-0-21swapper/209:09:142
28415050,0irq/27-eth0-tx-0-21swapper/209:00:002
28415050,0irq/27-eth0-tx-0-21swapper/208:49:512
28415050,0irq/27-eth0-tx-0-21swapper/208:45:062
28415050,0irq/27-eth0-tx-0-21swapper/007:34:290
28415050,0irq/27-eth0-tx-0-21swapper/007:31:210
28405050,0irq/26-eth0-rx-3109-21kthreadcore10:54:293
28405050,0irq/26-eth0-rx-2746-21python08:24:403
28405050,0irq/26-eth0-rx-2746-21python08:24:403
28405050,0irq/26-eth0-rx-15981-21if_eth1.211:44:303
28405050,0irq/26-eth0-rx-12823-21ssh09:36:293
28405050,0irq/26-eth0-rx-0-21swapper/312:08:353
28405050,0irq/26-eth0-rx-0-21swapper/311:39:343
28405050,0irq/26-eth0-rx-0-21swapper/309:54:343
28405050,0irq/26-eth0-rx-0-21swapper/309:51:483
28405050,0irq/26-eth0-rx-0-21swapper/309:12:073
259950,0migration/211096-21kthreadcore08:09:332
184319955,0cyclictest0-21swapper/111:29:491
184319955,0cyclictest0-21swapper/110:11:091
184319955,0cyclictest0-21swapper/109:41:501
184319955,0cyclictest0-21swapper/109:13:461
184319950,0cyclictest0-21swapper/110:48:501
184319950,0cyclictest0-21swapper/110:01:511
184319950,0cyclictest0-21swapper/108:39:241
184309955,0cyclictest0-21swapper/012:04:530
184309955,0cyclictest0-21swapper/010:17:400
184309950,0cyclictest0-21swapper/010:11:030
184309950,0cyclictest0-21swapper/009:58:450
179950,0migration/19119-21hwlatdetect07:44:301
179950,0migration/13480-21idleruntime08:04:371
179950,0migration/111846-21munin-node11:59:331
169950,0watchdog/125078-21head10:24:361
169950,0watchdog/124448-21tune2fs09:04:291
169950,0watchdog/12365-21kthreadcore09:49:331
169950,0watchdog/123616-21rm09:04:051
169950,0watchdog/116807-21expr08:14:331
169950,0watchdog/10-21swapper/111:02:481
169950,0watchdog/10-21swapper/109:14:291
169950,0watchdog/10-21swapper/108:49:211
169950,0watchdog/10-21swapper/108:28:131
169950,0watchdog/10-21swapper/108:28:131
149950,0migration/08478-21aten_r1power_po08:54:260
149950,0migration/07881-21kthreadcore10:34:340
149950,0migration/05836-21taskset06:52:040
149950,0migration/029872-21kthreadcore10:29:320
149950,0migration/029034-21sh09:25:080
149950,0migration/028076-21latency08:19:330
149950,0migration/026185-21sh10:26:370
149950,0migration/025547-21kthreadcore10:04:340
149950,0migration/022233-21timerandwakeup08:59:550
149950,0migration/018210-21needreboot09:59:330
149950,0migration/015969-21if_eth111:44:300
149950,0migration/015424-21kthreadcore09:39:320
149950,0migration/014802-21sh09:19:150
149950,0migration/014473-21sh11:43:110
149950,0migration/012495-21kthreadcore10:59:320
149950,0migration/010946-21kthreadcore11:19:330
149950,0migration/010527-21kthreadcore07:44:330
9941240,0sleep112609-21latency07:19:301
339940,0migration/320440-21kthreadcore11:04:333
339940,0migration/319012-21kthreadcore07:49:313
339940,0migration/317952-21kthreadcore07:24:373
339940,0migration/317426-21kthreadcore06:59:483
28415040,0irq/27-eth0-tx-20815-21ssh12:05:372
28415040,0irq/27-eth0-tx-0-21swapper/211:06:332
28415040,0irq/27-eth0-tx-0-21swapper/210:04:182
28415040,0irq/27-eth0-tx-0-21swapper/008:16:300
28415040,0irq/27-eth0-tx-0-21swapper/007:19:230
28415040,0irq/27-eth0-tx-0-21swapper/007:14:310
28415040,0irq/27-eth0-tx-0-21swapper/006:54:380
28405040,0irq/26-eth0-rx-0-21swapper/311:50:343
28405040,0irq/26-eth0-rx-0-21swapper/311:24:243
28405040,0irq/26-eth0-rx-0-21swapper/311:19:183
28405040,0irq/26-eth0-rx-0-21swapper/310:46:083
28405040,0irq/26-eth0-rx-0-21swapper/310:34:563
28405040,0irq/26-eth0-rx-0-21swapper/310:32:593
28405040,0irq/26-eth0-rx-0-21swapper/310:20:023
28405040,0irq/26-eth0-rx-0-21swapper/309:39:173
28405040,0irq/26-eth0-rx-0-21swapper/309:17:043
28405040,0irq/26-eth0-rx-0-21swapper/309:04:043
28405040,0irq/26-eth0-rx-0-21swapper/308:54:563
28405040,0irq/26-eth0-rx-0-21swapper/307:34:293
259940,0migration/28349-21aten_r1power_cu07:44:242
259940,0migration/23992-21kthreadcore07:14:312
259940,0migration/226709-21munin-node07:09:272
259940,0migration/222522-21kthreadcore07:04:342
259940,0migration/211925-21kthreadcore08:34:312
249940,0watchdog/29585-21kthreadcore07:19:292
249940,0watchdog/26547-21runrttasks07:59:172
249940,0watchdog/215532-21kthreadcore07:24:372
249940,0watchdog/20-21swapper/209:05:292
184329944,0cyclictest0-21swapper/208:04:382
184329943,0cyclictest6445-21snmpd11:26:372
184329943,0cyclictest6445-21snmpd10:51:372
184329943,0cyclictest6445-21snmpd08:24:362
184329943,0cyclictest6445-21snmpd08:24:362
184329943,0cyclictest5868-21awk09:54:322
184329943,0cyclictest3425-21lldpd08:21:412
184329943,0cyclictest3425-21lldpd07:50:352
184329943,0cyclictest3425-21lldpd07:43:352
184329943,0cyclictest3425-21lldpd07:35:052
184329943,0cyclictest3425-21lldpd06:45:102
184329943,0cyclictest3110-21irqbalance11:45:582
184329943,0cyclictest21486-21awk07:29:282
184329943,0cyclictest13851-21awk06:59:452
184319944,0cyclictest0-21swapper/109:55:141
184319944,0cyclictest0-21swapper/107:52:341
184319944,0cyclictest0-21swapper/107:39:291
184319943,0cyclictest6445-21snmpd10:15:371
184319943,0cyclictest6445-21snmpd09:20:361
184319943,0cyclictest6445-21snmpd07:54:211
184319943,0cyclictest6445-21snmpd07:32:361
184319943,0cyclictest6445-21snmpd07:18:361
184319943,0cyclictest6445-21snmpd06:50:361
184319943,0cyclictest394-21awk11:34:321
184319943,0cyclictest26152-21awk11:09:321
184319943,0cyclictest23638-21awk09:24:291
184319943,0cyclictest21992-21aten_r1power_vo09:44:301
184319943,0cyclictest1480-21awk11:14:291
184319943,0cyclictest13924-21kworker/1:212:04:311
184319943,0cyclictest13924-21kworker/1:211:21:401
184319943,0cyclictest13924-21kworker/1:206:44:381
184309944,0cyclictest0-21swapper/009:11:270
184309943,0cyclictest7059-21sadc11:39:140
184309943,0cyclictest6445-21snmpd08:33:060
184309943,0cyclictest3110-21irqbalance11:49:480
184309943,0cyclictest3110-21irqbalance11:24:180
184309943,0cyclictest3110-21irqbalance11:14:180
184309943,0cyclictest3110-21irqbalance11:12:580
184309943,0cyclictest3110-21irqbalance10:44:280
184309943,0cyclictest3110-21irqbalance09:48:180
184309943,0cyclictest3110-21irqbalance09:34:280
184309943,0cyclictest3110-21irqbalance09:29:580
184309943,0cyclictest30369-21awk09:49:290
184309943,0cyclictest22544-21awk07:54:290
184309943,0cyclictest19618-21awk06:39:290
184309943,0cyclictest15895-21awk07:49:280
184309943,0cyclictest13990-21ps06:59:460
179940,0migration/17850-21kthreadcore08:29:321
179940,0migration/17357-21pmu-power10:54:331
179940,0migration/17269-21if_eth1.306:54:301
179940,0migration/132375-21runrttasks07:10:071
179940,0migration/128853-21runrttasks08:19:561
179940,0migration/128304-21latency11:49:331
179940,0migration/127804-21kthreadcore08:44:391
179940,0migration/123257-21kthreadcore07:04:341
179940,0migration/123137-21sendmail_mailst11:04:371
179940,0migration/119426-21hwlatdetect06:39:281
179940,0migration/116655-21kthreadcore11:44:311
179940,0migration/115809-21kthreadcore06:59:471
179940,0migration/114032-21kthreadcore10:19:321
179940,0migration/11050-21runrttasks07:36:131
1675240,0sleep00-21swapper/010:54:250
169940,0watchdog/14123-21irqstats10:34:321
169940,0watchdog/14005-21rt-features09:29:371
169940,0watchdog/120937-21munin-node10:04:281
169940,0watchdog/120018-21kthreadcore11:24:321
169940,0watchdog/113574-21ssh11:40:121
169940,0watchdog/10-21swapper/111:55:041
169940,0watchdog/10-21swapper/109:35:171
169940,0watchdog/10-21swapper/108:54:371
169940,0watchdog/10-21swapper/108:34:291
149940,0migration/0886-21kthreadcore07:59:320
149940,0migration/05084-21sh11:56:110
149940,0migration/032473-21taskset10:50:010
149940,0migration/031575-21runrttasks06:47:580
149940,0migration/028173-21kthreadcore09:04:330
149940,0migration/021071-21kthreadcore11:04:330
149940,0migration/014745-21gltestperf07:24:340
149940,0migration/012302-21kthreadcore09:14:320
149940,0migration/010950-21kthreadcore11:59:320
339930,0migration/332550-21runrttasks07:12:093
28415030,0irq/27-eth0-tx-0-21swapper/007:09:290
28405030,0irq/26-eth0-rx-0-21swapper/308:36:333
28405030,0irq/26-eth0-rx-0-21swapper/308:14:283
28405030,0irq/26-eth0-rx-0-21swapper/308:04:373
28405030,0irq/26-eth0-rx-0-21swapper/307:54:253
28405030,0irq/26-eth0-rx-0-21swapper/307:41:103
28405030,0irq/26-eth0-rx-0-21swapper/307:29:343
28405030,0irq/26-eth0-rx-0-21swapper/307:14:263
28405030,0irq/26-eth0-rx-0-21swapper/306:54:173
249930,0watchdog/20-21swapper/208:14:332
249930,0watchdog/20-21swapper/206:54:372
249930,0watchdog/20-21swapper/206:39:252
184329933,0cyclictest0-21swapper/207:54:232
184329932,0cyclictest4627-21awk08:29:302
184329931,0cyclictest2635-21kthreadcore06:49:322
184309932,0cyclictest671-21kthreadcore08:24:350
184309932,0cyclictest671-21kthreadcore08:24:350
184309932,0cyclictest5522-21grep08:04:390
184309932,0cyclictest3425-21lldpd10:22:560
184309932,0cyclictest10047-21perl08:09:300
184309930,0cyclictest27300-21kthreadcore08:44:380
184309930,0cyclictest0-21swapper/010:39:290
169930,0watchdog/10-21swapper/110:49:241
169930,0watchdog/10-21swapper/107:59:171
329920,0watchdog/326568-21grep08:19:323
28405020,0irq/26-eth0-rx-0-21swapper/308:09:273
184309920,0cyclictest19574-21ls07:04:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional