You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 22:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Wed Nov 19, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
355327975,0sleep10-21swapper/106:37:101
348527370,0sleep20-21swapper/206:36:232
291350690,0irq/25-eth10-21swapper/306:39:213
348823823,0sleep00-21swapper/006:36:250
38579960,0cyclictest0-21swapper/111:45:451
38559960,0cyclictest0-21swapper/010:36:200
38559960,0cyclictest0-21swapper/010:18:310
339960,0migration/330529-21sh09:39:143
339960,0migration/322576-21munin-node09:55:003
339960,0migration/311110-21munin-node11:30:013
28415060,0irq/27-eth0-tx-0-21swapper/211:56:132
28415060,0irq/27-eth0-tx-0-21swapper/211:47:152
28415060,0irq/27-eth0-tx-0-21swapper/211:41:172
28415060,0irq/27-eth0-tx-0-21swapper/211:39:062
28415060,0irq/27-eth0-tx-0-21swapper/211:23:572
28415060,0irq/27-eth0-tx-0-21swapper/211:13:062
28415060,0irq/27-eth0-tx-0-21swapper/210:59:542
28415060,0irq/27-eth0-tx-0-21swapper/210:46:432
28415060,0irq/27-eth0-tx-0-21swapper/210:30:232
28415060,0irq/27-eth0-tx-0-21swapper/210:15:122
28415060,0irq/27-eth0-tx-0-21swapper/210:11:082
28415060,0irq/27-eth0-tx-0-21swapper/209:29:262
28415060,0irq/27-eth0-tx-0-21swapper/209:21:022
28415060,0irq/27-eth0-tx-0-21swapper/209:16:332
28415060,0irq/27-eth0-tx-0-21swapper/209:12:512
28415060,0irq/27-eth0-tx-0-21swapper/208:53:032
28415060,0irq/27-eth0-tx-0-21swapper/208:44:372
28415060,0irq/27-eth0-tx-0-21swapper/007:14:540
28405060,0irq/26-eth0-rx-24573-21irqcore11:40:013
28405060,0irq/26-eth0-rx-11952-21ntp_offset09:45:033
149960,0migration/05703-21sh11:47:190
149960,0migration/04883-21sh12:08:200
149960,0migration/026092-21kthreadcore09:15:020
149960,0migration/021531-21unixbench-2d09:30:070
8225050,0irq/24-0000:00:15494-21idleruntime-cro11:34:422
8225050,0irq/24-0000:00:14630-21dump-pmu-power08:24:422
38589955,0cyclictest0-21swapper/206:45:172
38579955,0cyclictest2187-21diskmemload11:23:511
38579955,0cyclictest0-21swapper/109:56:021
38579955,0cyclictest0-21swapper/109:25:551
38579955,0cyclictest0-21swapper/109:10:091
38579954,0cyclictest24098-21grep11:39:581
38579950,0cyclictest0-21swapper/111:53:101
38579950,0cyclictest0-21swapper/111:09:011
38579950,0cyclictest0-21swapper/110:35:081
38579950,0cyclictest0-21swapper/109:36:571
38579950,0cyclictest0-21swapper/109:19:081
38579950,0cyclictest0-21swapper/108:54:421
38559955,0cyclictest0-21swapper/010:48:380
38559950,0cyclictest0-21swapper/009:26:100
38559950,0cyclictest0-21swapper/009:02:030
339950,0migration/38409-21kthreadcore07:05:033
339950,0migration/36038-21timerwakeupswit09:20:093
339950,0migration/35007-21taskset10:25:013
339950,0migration/33405-21kthreadcore08:14:593
339950,0migration/332403-21kthreadcore08:10:033
339950,0migration/330877-21taskset11:20:383
339950,0migration/328855-21kthreadcore10:40:053
339950,0migration/326299-21kthreadcore12:00:043
339950,0migration/325325-21fschecks_time09:14:583
339950,0migration/322585-21kthreadcore10:15:013
339950,0migration/321907-21kthreadcore07:15:003
339950,0migration/321170-21mdstat11:35:023
339950,0migration/319089-21taskset07:37:283
339950,0migration/31684-21munin-node10:20:053
339950,0migration/316216-21kthreadcore07:10:003
339950,0migration/31462-21fschecks_count08:59:583
339950,0migration/311969-21munin-node06:45:193
339950,0migration/310454-21aten_r1power_po08:44:553
28415050,0irq/27-eth0-tx-7189-21munin-node07:54:570
28415050,0irq/27-eth0-tx-21527-21iostat_ios08:05:000
28415050,0irq/27-eth0-tx-17101-21ssh10:51:492
28415050,0irq/27-eth0-tx-0-21swapper/211:50:542
28415050,0irq/27-eth0-tx-0-21swapper/211:26:222
28415050,0irq/27-eth0-tx-0-21swapper/211:16:232
28415050,0irq/27-eth0-tx-0-21swapper/211:06:132
28415050,0irq/27-eth0-tx-0-21swapper/210:43:432
28415050,0irq/27-eth0-tx-0-21swapper/210:29:032
28415050,0irq/27-eth0-tx-0-21swapper/210:04:462
28415050,0irq/27-eth0-tx-0-21swapper/210:00:342
28415050,0irq/27-eth0-tx-0-21swapper/209:50:462
28415050,0irq/27-eth0-tx-0-21swapper/209:48:382
28415050,0irq/27-eth0-tx-0-21swapper/209:41:222
28415050,0irq/27-eth0-tx-0-21swapper/209:34:532
28415050,0irq/27-eth0-tx-0-21swapper/209:31:212
28415050,0irq/27-eth0-tx-0-21swapper/209:05:062
28415050,0irq/27-eth0-tx-0-21swapper/209:01:192
28415050,0irq/27-eth0-tx-0-21swapper/208:56:172
28415050,0irq/27-eth0-tx-0-21swapper/208:49:202
28415050,0irq/27-eth0-tx-0-21swapper/008:39:560
28415050,0irq/27-eth0-tx-0-21swapper/007:59:570
28415050,0irq/27-eth0-tx-0-21swapper/007:43:020
28415050,0irq/27-eth0-tx-0-21swapper/007:04:560
28405050,0irq/26-eth0-rx-5925-21kthreadcore11:05:043
28405050,0irq/26-eth0-rx-5918-21ssh11:48:013
28405050,0irq/26-eth0-rx-30189-21aten_r1power_en12:04:563
28405050,0irq/26-eth0-rx-20203-21tune2fs07:39:573
28405050,0irq/26-eth0-rx-0-21swapper/311:55:183
28405050,0irq/26-eth0-rx-0-21swapper/311:28:573
28405050,0irq/26-eth0-rx-0-21swapper/311:13:243
28405050,0irq/26-eth0-rx-0-21swapper/310:58:523
28405050,0irq/26-eth0-rx-0-21swapper/310:36:573
28405050,0irq/26-eth0-rx-0-21swapper/310:10:463
28405050,0irq/26-eth0-rx-0-21swapper/310:04:553
28405050,0irq/26-eth0-rx-0-21swapper/309:59:563
28405050,0irq/26-eth0-rx-0-21swapper/309:30:053
28405050,0irq/26-eth0-rx-0-21swapper/308:56:463
28405050,0irq/26-eth0-rx-0-21swapper/307:45:013
259950,0migration/22413-21kthreadcore07:25:012
249950,0watchdog/226801-21memory08:30:052
19150,0ktimersoftd/10-21swapper/106:45:161
179950,0migration/17912-21kthreadcore10:25:021
179950,0migration/14935-21kthreadcore08:15:001
179950,0migration/13881-21kthreadcore07:50:011
179950,0migration/131988-21kthreadcore10:20:021
179950,0migration/126133-21munin-node07:19:581
179950,0migration/125440-21kthreadcore08:30:041
179950,0migration/113503-21kthreadcore11:10:011
179950,0migration/110996-21kthreadcore07:55:001
169950,0watchdog/17044-21fschecks_time09:44:571
169950,0watchdog/131259-21grep10:00:011
169950,0watchdog/116881-21kthreadcore10:10:011
169950,0watchdog/111113-21hddtemp_smartct08:44:571
169950,0watchdog/10-21swapper/111:35:331
169950,0watchdog/10-21swapper/111:02:011
169950,0watchdog/10-21swapper/109:30:291
169950,0watchdog/10-21swapper/108:25:011
169950,0watchdog/10-21swapper/108:04:571
149950,0migration/05836-21taskset07:28:370
149950,0migration/031370-21kthreadcore10:20:020
149950,0migration/030043-21kthreadcore11:00:010
149950,0migration/028464-21kernelversion08:10:020
149950,0migration/028216-21sh09:55:540
149950,0migration/027864-21kthreadcore09:35:020
149950,0migration/027137-21kthreadcore07:19:590
149950,0migration/026279-21aten2.4-expect07:44:560
149950,0migration/025662-21kthreadcore11:40:020
149950,0migration/025221-21fschecks_time11:20:020
149950,0migration/024021-21kthreadcore08:30:040
149950,0migration/023096-21gltestperf12:00:000
149950,0migration/02251-21kthreadcore09:20:030
149950,0migration/020851-21kthreadcore10:55:010
149950,0migration/017945-21aten2.4-expect08:49:580
149950,0migration/017675-21hwlatdetect09:10:000
149950,0migration/015000-21aten_r1power_vo08:24:560
149950,0migration/01401-21kthreadcore11:25:010
149950,0migration/013224-21sh09:47:250
149950,0migration/011693-21kthreadcore11:50:030
149950,0migration/011618-21irqcore10:50:020
149950,0migration/011357-21kthreadcore11:10:000
149950,0migration/010828-21aten_r1power_en06:45:000
8225040,0irq/24-0000:00:0-21swapper/207:55:312
38589944,0cyclictest0-21swapper/206:50:102
38589943,0cyclictest3425-21lldpd06:59:592
38589943,0cyclictest19189-21awk10:54:592
38589943,0cyclictest15716-21awk08:24:592
38589943,0cyclictest12703-21awk07:09:582
38579944,0cyclictest0-21swapper/106:54:431
38579944,0cyclictest0-21swapper/106:54:431
38579943,0cyclictest28375-21awk10:40:021
38579943,0cyclictest26456-21awk08:54:581
38579943,0cyclictest15023-21awk09:49:591
38579943,0cyclictest14942-21kworker/1:111:18:381
38579943,0cyclictest14942-21kworker/1:110:50:541
38579943,0cyclictest14942-21kworker/1:110:31:041
38579943,0cyclictest11635-21sadc07:09:431
38579943,0cyclictest11128-21kworker/1:009:40:541
38579943,0cyclictest11128-21kworker/1:008:59:481
38559943,0cyclictest31840-21awk09:40:010
38559943,0cyclictest30601-21awk10:00:000
38559943,0cyclictest18361-21awk06:50:020
339940,0migration/3424-21kthreadcore08:35:013
339940,0migration/33614-21irqcore08:40:003
339940,0migration/32803-21diskmemload10:44:543
339940,0migration/322452-21kthreadcore08:05:003
339940,0migration/322254-21gltestperf08:30:013
339940,0migration/31992-21kthreadcore07:00:033
339940,0migration/314678-21kthreadcore10:30:023
339940,0migration/311197-21kthreadcore07:55:003
28415040,0irq/27-eth0-tx-0-21swapper/212:04:422
28415040,0irq/27-eth0-tx-0-21swapper/210:35:002
28415040,0irq/27-eth0-tx-0-21swapper/008:35:260
28415040,0irq/27-eth0-tx-0-21swapper/007:35:020
28405040,0irq/26-eth0-rx-30594-21sshd06:59:273
28405040,0irq/26-eth0-rx-30594-21sshd06:59:273
28405040,0irq/26-eth0-rx-22950-21kthreadcore08:50:043
28405040,0irq/26-eth0-rx-0-21swapper/311:50:143
28405040,0irq/26-eth0-rx-0-21swapper/311:15:003
28405040,0irq/26-eth0-rx-0-21swapper/311:00:293
28405040,0irq/26-eth0-rx-0-21swapper/310:52:173
28405040,0irq/26-eth0-rx-0-21swapper/309:50:023
28405040,0irq/26-eth0-rx-0-21swapper/309:40:213
28405040,0irq/26-eth0-rx-0-21swapper/309:27:423
28405040,0irq/26-eth0-rx-0-21swapper/309:12:513
28405040,0irq/26-eth0-rx-0-21swapper/309:06:473
28405040,0irq/26-eth0-rx-0-21swapper/308:20:003
28405040,0irq/26-eth0-rx-0-21swapper/307:59:563
28405040,0irq/26-eth0-rx-0-21swapper/307:50:013
259940,0migration/25579-21kthreadcore08:15:002
259940,0migration/228521-21aten_r1power_cu10:19:552
259940,0migration/225771-21fschecks_time07:19:562
259940,0migration/225512-21unixbench_singl07:40:072
249940,0watchdog/23795-21kthreadcore07:50:012
249940,0watchdog/20-21swapper/212:00:052
249940,0watchdog/20-21swapper/209:55:012
249940,0watchdog/20-21swapper/208:04:572
249940,0watchdog/20-21swapper/208:02:252
249940,0watchdog/20-21swapper/207:35:012
249940,0watchdog/20-21swapper/206:55:132
249940,0watchdog/20-21swapper/206:55:132
179940,0migration/16464-21kthreadcore06:40:001
179940,0migration/14647-21kthreadcore11:25:021
179940,0migration/129992-21kthreadcore08:35:001
179940,0migration/129315-21kthreadcore08:10:021
179940,0migration/123241-21if_err_eth1.212:00:011
179940,0migration/113353-21kthreadcore08:20:001
179940,0migration/112723-21latency_hist07:34:421
179940,0migration/110565-21kthreadcore09:05:011
169940,0watchdog/122093-21cut10:15:011
169940,0watchdog/118188-21ssh10:54:491
169940,0watchdog/10-21swapper/112:04:531
169940,0watchdog/10-21swapper/111:29:491
169940,0watchdog/10-21swapper/109:20:531
169940,0watchdog/10-21swapper/108:43:451
169940,0watchdog/10-21swapper/107:44:571
169940,0watchdog/10-21swapper/107:35:011
169940,0watchdog/10-21swapper/107:10:051
149940,0migration/028115-21fschecks_time10:40:000
149940,0migration/018797-21kthreadcore11:55:030
149940,0migration/01353-21latency_hist08:14:420
38589933,0cyclictest3425-21lldpd08:35:002
38589932,0cyclictest6098-21awk07:05:002
38589932,0cyclictest28022-21awk08:10:002
38589932,0cyclictest19495-21awk07:14:582
38579933,0cyclictest9467-21kworker/1:207:59:421
38579933,0cyclictest9467-21kworker/1:207:39:421
38579933,0cyclictest9467-21kworker/1:207:24:441
38579933,0cyclictest9467-21kworker/1:207:14:561
38579933,0cyclictest14942-21kworker/1:110:44:481
38579933,0cyclictest14942-21kworker/1:110:04:521
38579932,0cyclictest9467-21kworker/1:206:59:441
38559933,0cyclictest0-21swapper/010:30:110
38559933,0cyclictest0-21swapper/010:11:250
38559932,0cyclictest7165-21awk07:30:010
38559932,0cyclictest7108-21proc_pri11:05:070
38559932,0cyclictest6445-21snmpd11:37:070
38559932,0cyclictest6445-21snmpd07:09:550
38559932,0cyclictest32118-21ssh08:56:380
38559932,0cyclictest206942sleep009:51:240
38559932,0cyclictest11193-21cut11:30:010
38559932,0cyclictest11091-21kthreadcore08:20:000
38559931,0cyclictest0-21swapper/010:05:000
38559930,0cyclictest9251-21cut10:25:030
38559930,0cyclictest27999-21cut06:55:130
38559930,0cyclictest27999-21cut06:55:130
38559930,0cyclictest22402-21port_sshpriv11:15:060
38559930,0cyclictest0-21swapper/007:50:000
38559930,0cyclictest0-21swapper/006:40:020
28415030,0irq/27-eth0-tx-0-21swapper/007:00:110
28405030,0irq/26-eth0-rx-0-21swapper/308:24:563
28405030,0irq/26-eth0-rx-0-21swapper/307:34:393
28405030,0irq/26-eth0-rx-0-21swapper/307:19:543
28405030,0irq/26-eth0-rx-0-21swapper/306:50:073
28405030,0irq/26-eth0-rx-0-21swapper/306:41:123
249930,0watchdog/20-21swapper/207:44:572
249930,0watchdog/20-21swapper/207:30:012
249930,0watchdog/20-21swapper/206:40:012
169930,0watchdog/10-21swapper/111:54:571
169930,0watchdog/10-21swapper/106:49:451
149930,0migration/016295-21sh09:08:520
38559920,0cyclictest0-21swapper/008:44:430
28405020,0irq/26-eth0-rx-0-21swapper/307:26:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional