You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-20 - 23:56
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2.osadl.org (updated Thu Nov 20, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1386525243,0sleep10-21swapper/106:19:441
1389523524,0sleep00-21swapper/006:19:590
1358623526,0sleep20-21swapper/206:16:292
1352922824,0sleep30-21swapper/306:15:503
329970,0watchdog/30-21swapper/308:31:163
329970,0watchdog/30-21swapper/307:55:163
339960,0migration/37209-21sh10:31:573
339960,0migration/322229-21munin-node08:25:183
339960,0migration/313819-21munin-node08:20:163
329960,0watchdog/3299250irq/32-eth0-rx-09:20:163
329960,0watchdog/315293-21if_eth1.311:20:153
329960,0watchdog/30-21swapper/311:46:193
329960,0watchdog/30-21swapper/311:27:153
329960,0watchdog/30-21swapper/311:11:153
329960,0watchdog/30-21swapper/311:02:313
329960,0watchdog/30-21swapper/310:50:513
329960,0watchdog/30-21swapper/310:41:433
329960,0watchdog/30-21swapper/310:25:513
329960,0watchdog/30-21swapper/310:19:313
329960,0watchdog/30-21swapper/310:12:353
329960,0watchdog/30-21swapper/310:02:563
329960,0watchdog/30-21swapper/309:42:163
329960,0watchdog/30-21swapper/309:15:123
329960,0watchdog/30-21swapper/308:59:483
329960,0watchdog/30-21swapper/308:44:403
329960,0watchdog/30-21swapper/307:02:203
329960,0watchdog/30-21swapper/306:55:163
29935060,0irq/33-eth0-tx-17904-21grep10:40:162
29925060,0irq/32-eth0-rx-25653-21processes09:25:213
29925060,0irq/32-eth0-rx-18461-21grep10:20:153
29925060,0irq/32-eth0-rx-0-21swapper/311:31:533
29925060,0irq/32-eth0-rx-0-21swapper/311:08:563
29925060,0irq/32-eth0-rx-0-21swapper/310:57:243
29925060,0irq/32-eth0-rx-0-21swapper/310:06:113
29925060,0irq/32-eth0-rx-0-21swapper/309:55:253
29925060,0irq/32-eth0-rx-0-21swapper/309:51:053
29925060,0irq/32-eth0-rx-0-21swapper/308:53:583
29925060,0irq/32-eth0-rx-0-21swapper/308:45:323
149960,0migration/023377-21fschecks_time08:00:150
149960,0migration/019421-21sh09:41:340
149960,0migration/015039-21taskset11:37:050
139729960,0cyclictest0-21swapper/211:18:032
139729960,0cyclictest0-21swapper/209:30:322
139729960,0cyclictest0-21swapper/207:25:302
8315050,0irq/26-0000:00:7419-21ls11:34:582
8315050,0irq/26-0000:00:0-21swapper/209:40:142
339950,0migration/39050-21kthreadcore11:15:163
339950,0migration/38341-21fschecks_count11:35:113
339950,0migration/330705-21kthreadcore07:40:183
339950,0migration/320518-21kthreadcore06:45:173
329950,0watchdog/30-21swapper/311:40:113
329950,0watchdog/30-21swapper/310:45:113
329950,0watchdog/30-21swapper/309:45:003
329950,0watchdog/30-21swapper/309:35:323
329950,0watchdog/30-21swapper/309:30:043
329950,0watchdog/30-21swapper/309:10:243
329950,0watchdog/30-21swapper/309:00:163
329950,0watchdog/30-21swapper/308:15:163
329950,0watchdog/30-21swapper/308:10:123
329950,0watchdog/30-21swapper/308:05:563
329950,0watchdog/30-21swapper/308:00:163
329950,0watchdog/30-21swapper/307:51:283
329950,0watchdog/30-21swapper/307:45:163
329950,0watchdog/30-21swapper/307:35:043
329950,0watchdog/30-21swapper/307:30:163
329950,0watchdog/30-21swapper/307:25:123
329950,0watchdog/30-21swapper/307:20:163
329950,0watchdog/30-21swapper/307:16:003
329950,0watchdog/30-21swapper/307:05:163
329950,0watchdog/30-21swapper/306:51:003
329950,0watchdog/30-21swapper/306:41:203
329950,0watchdog/30-21swapper/306:38:563
329950,0watchdog/30-21swapper/306:30:163
329950,0watchdog/30-21swapper/306:25:163
29935050,0irq/33-eth0-tx-8701-21open_inodes09:15:192
29935050,0irq/33-eth0-tx-6641-21grep10:14:592
29935050,0irq/33-eth0-tx-645-21ssh11:48:422
29935050,0irq/33-eth0-tx-3083-21ssh08:30:182
29935050,0irq/33-eth0-tx-27429-21tail09:45:262
29935050,0irq/33-eth0-tx-14968-21ssh11:36:542
29935050,0irq/33-eth0-tx-14018-40yum-updatesd-he07:03:330
29935050,0irq/33-eth0-tx-12077-21munin-node11:00:142
29935050,0irq/33-eth0-tx-11010-21telnet07:25:150
29935050,0irq/33-eth0-tx-10700-21munin-node06:40:210
29935050,0irq/33-eth0-tx-0-21swapper/210:27:582
29935050,0irq/33-eth0-tx-0-21swapper/210:00:062
29935050,0irq/33-eth0-tx-0-21swapper/209:05:222
29935050,0irq/33-eth0-tx-0-21swapper/208:43:002
29935050,0irq/33-eth0-tx-0-21swapper/008:13:450
29935050,0irq/33-eth0-tx-0-21swapper/007:45:150
29925050,0irq/32-eth0-rx-8877-21ssh10:35:153
29925050,0irq/32-eth0-rx-6439-21perl08:35:133
29925050,0irq/32-eth0-rx-19280-21kthreadcore06:20:203
29925050,0irq/32-eth0-rx-0-21swapper/309:05:263
29925050,0irq/32-eth0-rx-0-21swapper/307:10:123
259950,0migration/27858-21fschecks_time07:00:132
259950,0migration/27679-21kthreadcore08:35:172
259950,0migration/231885-21kthreadcore06:30:172
259950,0migration/229710-21kthreadcore09:10:172
259950,0migration/229613-21sh11:25:502
259950,0migration/227804-21kthreadcore07:40:172
259950,0migration/226321-21irqcore06:50:172
259950,0migration/21580-21munin-node10:50:242
259950,0migration/212472-21kthreadcore06:40:172
249950,0watchdog/232029-21fschecks_count08:50:162
179950,0migration/13940-21kthreadcore06:55:191
179950,0migration/13476-21gltestperf09:15:121
179950,0migration/131989-21taskset06:54:231
179950,0migration/127090-21munin-node08:25:211
179950,0migration/121554-21munin-node10:20:171
179950,0migration/120134-21munin-node11:20:191
179950,0migration/117102-21kthreadcore10:00:191
169950,0watchdog/16956-21irqrtprio11:15:151
169950,0watchdog/123712-21grep10:40:271
169950,0watchdog/121416-21fschecks_time11:05:111
149950,0migration/06415-21kthreadcore09:35:170
149950,0migration/06017-21kthreadcore07:20:170
149950,0migration/05160-21munin-node08:50:220
149950,0migration/05129-21munin-node11:30:180
149950,0migration/05014-21irqrtprio06:35:170
149950,0migration/04403-21sh11:12:530
149950,0migration/031210-21kthreadcore06:30:160
149950,0migration/030654-21kthreadcore09:10:170
149950,0migration/029258-21irqcore09:30:160
149950,0migration/027894-21taskset09:46:410
149950,0migration/026404-21sh10:47:510
149950,0migration/026242-21kthreadcore11:05:160
149950,0migration/024754-21kthreadcore08:45:140
149950,0migration/024440-21kthreadcore08:25:190
149950,0migration/023819-21taskset11:42:220
149950,0migration/023796-21kthreadcore07:10:170
149950,0migration/022837-21kthreadcore09:05:180
149950,0migration/022797-21aten_r1power_en11:25:110
149950,0migration/022131-21taskset07:58:350
149950,0migration/020068-21kthreadcore07:30:190
149950,0migration/018821-21kthreadcore07:05:160
149950,0migration/017044-21sh10:40:050
149950,0migration/016522-21kthreadcore08:40:170
149950,0migration/015089-21gltestperf06:20:160
149950,0migration/013243-21ksmtuned09:58:580
149950,0migration/010684-21kthreadcore11:15:170
139729955,0cyclictest0-21swapper/210:58:112
139729955,0cyclictest0-21swapper/210:37:512
139729955,0cyclictest0-21swapper/209:22:292
139729955,0cyclictest0-21swapper/208:00:212
139649955,0cyclictest2201-21ssh10:51:561
139649955,0cyclictest0-21swapper/109:12:371
139649954,0cyclictest23871-21snmpd09:46:291
139649950,0cyclictest0-21swapper/111:39:051
139649950,0cyclictest0-21swapper/108:54:101
139639955,0cyclictest12372-21diskmemload08:55:170
139639955,0cyclictest0-21swapper/011:45:140
139639955,0cyclictest0-21swapper/010:59:590
139639950,0cyclictest0-21swapper/010:37:460
139639950,0cyclictest0-21swapper/010:00:140
8315040,0irq/26-0000:00:0-21swapper/206:54:592
29935040,0irq/33-eth0-tx-29226-21ssh10:06:262
29935040,0irq/33-eth0-tx-22208-21kthreadcore11:40:172
29935040,0irq/33-eth0-tx-16250-21ssh09:00:342
29935040,0irq/33-eth0-tx-12628-21munin-node08:20:242
29935040,0irq/33-eth0-tx-0-21swapper/211:10:392
29935040,0irq/33-eth0-tx-0-21swapper/211:05:262
29935040,0irq/33-eth0-tx-0-21swapper/210:47:112
29935040,0irq/33-eth0-tx-0-21swapper/209:50:242
29935040,0irq/33-eth0-tx-0-21swapper/209:35:132
29935040,0irq/33-eth0-tx-0-21swapper/208:55:012
29935040,0irq/33-eth0-tx-0-21swapper/208:45:002
29935040,0irq/33-eth0-tx-0-21swapper/208:25:032
29935040,0irq/33-eth0-tx-0-21swapper/008:20:110
29935040,0irq/33-eth0-tx-0-21swapper/007:50:090
29935040,0irq/33-eth0-tx-0-21swapper/006:29:420
259940,0migration/28551-21kthreadcore07:20:182
259940,0migration/27681-21fschecks_time10:15:122
259940,0migration/24215-21fschecks_time06:35:132
259940,0migration/23903-21kthreadcore07:45:202
259940,0migration/228858-21aten_r1power_en07:15:102
259940,0migration/221892-21kthreadcore09:25:172
259940,0migration/219880-21kthreadcore07:30:192
259940,0migration/216746-21kthreadcore06:20:192
249940,0watchdog/224051-21kthreadcore06:25:202
249940,0watchdog/20-21swapper/209:55:162
249940,0watchdog/20-21swapper/207:55:162
249940,0watchdog/20-21swapper/206:45:162
179940,0migration/18844-21kthreadcore09:35:181
179940,0migration/17554-21latency_hist08:54:591
179940,0migration/14179-21taskset08:31:101
179940,0migration/131829-21kthreadcore11:10:171
179940,0migration/130168-21kthreadcore09:30:161
179940,0migration/128063-21unixbench_multi07:10:251
179940,0migration/127637-21kthreadcore08:00:191
179940,0migration/123511-21kthreadcore06:25:201
179940,0migration/120529-21kthreadcore06:45:171
179940,0migration/116668-21taskset09:01:501
179940,0migration/114846-21fschecks_time06:20:151
179940,0migration/114570-21kthreadcore08:20:171
179940,0migration/111574-21sendmail_mailqu08:15:191
179940,0migration/110477-21kthreadcore07:50:151
169940,0watchdog/16913-21ssh10:31:031
169940,0watchdog/16623-21irqcore09:55:161
169940,0watchdog/119536-21kthreadcore07:05:161
169940,0watchdog/117932-21ssh09:21:001
169940,0watchdog/10-21swapper/107:30:201
169940,0watchdog/10-21swapper/107:00:161
169940,0watchdog/10-21swapper/106:40:121
149940,0migration/031739-21kthreadcore08:30:160
149940,0migration/031312-21kthreadcore10:50:170
149940,0migration/031054-21kthreadcore09:50:180
149940,0migration/030632-21kthreadcore07:15:160
139729944,0cyclictest0-21swapper/211:20:152
139729943,0cyclictest6463-21awk08:15:132
139729943,0cyclictest31599-21awk08:10:152
139729943,0cyclictest23871-21snmpd08:05:132
139729943,0cyclictest15493-21awk07:05:132
139729943,0cyclictest1001-21awk10:30:162
139729943,0cyclictest0-21swapper/210:20:132
139649944,0cyclictest0-21swapper/106:31:581
139649943,0cyclictest6987-21cut10:15:101
139649943,0cyclictest4543-21awk06:35:151
139649943,0cyclictest441-21sed11:30:151
139649943,0cyclictest3599-21lldpd09:06:011
139649943,0cyclictest3599-21lldpd07:35:171
139649943,0cyclictest31527-21awk08:10:151
139649943,0cyclictest30217-21sed07:15:161
139649943,0cyclictest23871-21snmpd10:55:291
139649943,0cyclictest23871-21snmpd10:25:291
139649943,0cyclictest23871-21snmpd10:10:141
139649943,0cyclictest23871-21snmpd10:05:291
139649943,0cyclictest23871-21snmpd09:54:291
139649943,0cyclictest23871-21snmpd09:29:291
139649943,0cyclictest23871-21snmpd08:35:131
139649943,0cyclictest23871-21snmpd08:07:581
139649943,0cyclictest23871-21snmpd07:22:281
139649943,0cyclictest23789-21awk11:25:151
139649943,0cyclictest17596-21awk11:40:151
139649943,0cyclictest16357-21awk07:55:141
139649943,0cyclictest13622-21awk09:40:161
139649943,0cyclictest13202-21awk11:00:171
139639944,0cyclictest0-21swapper/010:56:030
139639944,0cyclictest0-21swapper/009:02:510
139639944,0cyclictest0-21swapper/008:38:530
139639943,0cyclictest27620-21sadc09:29:590
139639943,0cyclictest23871-21snmpd07:43:130
139639943,0cyclictest15152-21awk11:20:140
8315030,0irq/26-0000:00:0-21swapper/207:49:592
29935030,0irq/33-eth0-tx-0-21swapper/008:15:080
29935030,0irq/33-eth0-tx-0-21swapper/008:05:150
29935030,0irq/33-eth0-tx-0-21swapper/006:55:130
29935030,0irq/33-eth0-tx-0-21swapper/006:50:130
19130,0ktimersoftd/132184-21ssh11:46:161
179930,0migration/15246-21kthreadcore07:45:211
179930,0migration/125903-21sh10:46:241
139729932,0cyclictest23871-21snmpd07:37:582
139729932,0cyclictest22568-21grep07:10:142
139649932,0cyclictest26846-21tr08:45:151
139649932,0cyclictest23871-21snmpd10:34:591
139649932,0cyclictest20362-21sensors08:40:221
139649932,0cyclictest11989-21awk07:25:191
139649931,0cyclictest0-21swapper/107:40:201
139639933,0cyclictest7557-21kworker/0:210:20:180
139639933,0cyclictest7557-21kworker/0:210:15:120
139639933,0cyclictest7557-21kworker/0:210:14:120
139639933,0cyclictest7557-21kworker/0:210:05:340
139639933,0cyclictest29553-21kworker/0:106:45:140
139639933,0cyclictest0-21swapper/009:16:150
139639932,0cyclictest4617-21grep10:30:190
139639932,0cyclictest30242-21sensors10:25:210
139639932,0cyclictest14093-21cat09:20:210
139639922,0cyclictest29553-21kworker/0:107:35:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional