You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-09 - 23:57
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Tue Dec 09, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
264427776,0sleep00-21swapper/006:17:220
240325756,0sleep20-21swapper/206:14:522
2851994949,0cyclictest14597-21md0_resync08:47:171
244224342,0sleep10-21swapper/106:15:151
2852994038,0cyclictest14596-21md1_resync08:47:162
274523627,0sleep30-21swapper/306:18:253
2851991212,0cyclictest23871-21snmpd11:16:321
339980,0migration/36851-21kthreadcore09:44:113
8315070,0irq/26-0000:00:0-21swapper/311:19:153
8315070,0irq/26-0000:00:0-21swapper/311:12:063
8315070,0irq/26-0000:00:0-21swapper/308:34:083
329970,0watchdog/321849-21gltestperf11:14:033
329970,0watchdog/313034-21ntpq09:09:113
29925070,0irq/32-eth0-rx-0-21swapper/308:43:473
8315060,0irq/26-0000:00:9779-21perl07:29:073
8315060,0irq/26-0000:00:6352-21munin-node07:49:033
8315060,0irq/26-0000:00:4547-21awk09:04:113
8315060,0irq/26-0000:00:40012sleep308:46:173
8315060,0irq/26-0000:00:3945-21perl08:09:043
8315060,0irq/26-0000:00:28556-21kthreadcore09:39:093
8315060,0irq/26-0000:00:28298-21head08:04:063
8315060,0irq/26-0000:00:27574-21ntp_kernel_err10:14:113
8315060,0irq/26-0000:00:2298-21perl07:24:063
8315060,0irq/26-0000:00:22794-21cut09:34:113
8315060,0irq/26-0000:00:22740-21wc07:34:153
8315060,0irq/26-0000:00:22740-21wc07:34:153
8315060,0irq/26-0000:00:20698-21ssh08:54:293
8315060,0irq/26-0000:00:1822-21timerwakeupswit09:59:173
8315060,0irq/26-0000:00:15496-21iostat_ios11:29:073
8315060,0irq/26-0000:00:134872sleep307:53:513
8315060,0irq/26-0000:00:12677-21cut09:29:103
8315060,0irq/26-0000:00:1008-21seq07:20:283
8315060,0irq/26-0000:00:0-21swapper/310:41:503
8315060,0irq/26-0000:00:0-21swapper/310:24:123
8315060,0irq/26-0000:00:0-21swapper/310:04:113
8315060,0irq/26-0000:00:0-21swapper/309:14:433
8315060,0irq/26-0000:00:0-21swapper/308:29:233
8315060,0irq/26-0000:00:0-21swapper/308:21:003
8315060,0irq/26-0000:00:0-21swapper/307:09:073
8315060,0irq/26-0000:00:0-21swapper/307:04:103
8315060,0irq/26-0000:00:0-21swapper/306:19:173
4160,0ktimersoftd/00-21swapper/007:52:310
4160,0ktimersoftd/00-21swapper/006:50:050
339960,0migration/36274-21sh11:40:113
339960,0migration/332497-21runrttasks06:38:563
339960,0migration/315302-21sh10:44:313
329960,0watchdog/330887-21kthreadcore10:59:113
329960,0watchdog/325600-21diskstats06:34:083
329960,0watchdog/30-21swapper/311:44:033
329960,0watchdog/30-21swapper/308:24:353
29935060,0irq/33-eth0-tx-23871-21snmpd06:35:410
29935060,0irq/33-eth0-tx-0-21swapper/007:05:460
29925060,0irq/32-eth0-rx-0-21swapper/311:34:143
29925060,0irq/32-eth0-rx-0-21swapper/309:56:383
29925060,0irq/32-eth0-rx-0-21swapper/309:49:343
29925060,0irq/32-eth0-rx-0-21swapper/308:59:003
29925060,0irq/32-eth0-rx-0-21swapper/308:49:123
28529966,0cyclictest0-21swapper/208:54:102
28529960,0cyclictest0-21swapper/210:07:192
28519965,0cyclictest17943-21aten_r1power_po06:29:041
28519960,0cyclictest0-21swapper/110:33:541
28519960,0cyclictest0-21swapper/110:29:101
28519960,0cyclictest0-21swapper/110:29:101
28519960,0cyclictest0-21swapper/107:28:391
28509960,0cyclictest0-21swapper/008:30:210
19160,0ktimersoftd/10-21swapper/106:25:261
169960,0watchdog/10-21swapper/108:52:391
149960,0migration/07816-21taskset09:27:040
149960,0migration/07303-21sh11:43:080
149960,0migration/032304-21kernelversion09:04:080
149960,0migration/025406-21ntp_offset07:14:120
149960,0migration/023649-21kthreadcore08:39:100
149960,0migration/019583-21sh11:10:320
149960,0migration/016187-21sh10:47:270
8315050,0irq/26-0000:00:8233-21perl06:44:083
8315050,0irq/26-0000:00:32407-21ssh09:23:593
8315050,0irq/26-0000:00:29329-21kthreadcore07:39:113
8315050,0irq/26-0000:00:28154-21memory06:54:103
8315050,0irq/26-0000:00:24932-21if_err_eth1.209:19:083
8315050,0irq/26-0000:00:24527-21sshd06:29:213
8315050,0irq/26-0000:00:10599-21aten_r1power_cu08:14:023
8315050,0irq/26-0000:00:0-21swapper/310:49:303
8315050,0irq/26-0000:00:0-21swapper/307:44:023
8315050,0irq/26-0000:00:0-21swapper/307:13:593
8315050,0irq/26-0000:00:0-21swapper/306:59:283
8315050,0irq/26-0000:00:0-21swapper/306:48:533
8315050,0irq/26-0000:00:0-21swapper/306:23:563
4150,0ktimersoftd/00-21swapper/009:38:180
4150,0ktimersoftd/00-21swapper/006:29:260
339950,0migration/36012-21runrttasks11:24:033
339950,0migration/331872-21kthreadcore10:19:083
339950,0migration/321384-21sh10:29:163
339950,0migration/321384-21sh10:29:163
339950,0migration/321317-21fschecks_count07:59:063
339950,0migration/319094-21smart_sda10:09:153
29935050,0irq/33-eth0-tx-32512-21munin-node06:39:110
29935050,0irq/33-eth0-tx-0-21swapper/211:03:082
29935050,0irq/33-eth0-tx-0-21swapper/007:59:130
29935050,0irq/33-eth0-tx-0-21swapper/007:45:540
29935050,0irq/33-eth0-tx-0-21swapper/007:19:130
29925050,0irq/32-eth0-rx-0-21swapper/311:04:023
29925050,0irq/32-eth0-rx-0-21swapper/310:54:353
29925050,0irq/32-eth0-rx-0-21swapper/310:33:593
28529955,0cyclictest0-21swapper/211:17:102
28529955,0cyclictest0-21swapper/209:11:502
28529955,0cyclictest0-21swapper/206:20:272
28529954,0cyclictest30673-21awk06:59:062
28529950,0cyclictest0-21swapper/210:34:172
28519955,0cyclictest0-21swapper/109:57:101
28519955,0cyclictest0-21swapper/108:27:281
28519955,0cyclictest0-21swapper/107:19:401
28519955,0cyclictest0-21swapper/107:06:071
28519955,0cyclictest0-21swapper/106:50:491
28519950,0cyclictest0-21swapper/111:41:321
28519950,0cyclictest0-21swapper/109:17:551
28519950,0cyclictest0-21swapper/106:56:301
28509955,0cyclictest0-21swapper/011:08:200
28509955,0cyclictest0-21swapper/008:54:380
28509955,0cyclictest0-21swapper/008:47:420
28509950,0cyclictest0-21swapper/010:56:110
28509950,0cyclictest0-21swapper/010:04:100
26064250,0sleep10-21swapper/110:56:111
179950,0migration/16220-21missed_timers09:24:101
179950,0migration/131034-21sh11:38:371
179950,0migration/124169-21cpuspeed_turbos09:19:051
179950,0migration/123692-21kthreadcore07:59:101
179950,0migration/12102-21cpuspeed_turbos09:44:061
179950,0migration/120795-21switchtime11:29:151
179950,0migration/119402-21runrttasks08:37:291
169950,0watchdog/18698-21df10:44:031
169950,0watchdog/119036-21cut07:34:071
169950,0watchdog/119036-21cut07:34:071
169950,0watchdog/117933-21munin-node10:49:071
169950,0watchdog/111678-21ssh10:06:591
169950,0watchdog/10-21swapper/110:19:031
169950,0watchdog/10-21swapper/108:55:031
149950,0migration/09168-21kthreadcore11:24:080
149950,0migration/07533-21kthreadcore09:44:120
149950,0migration/07463-21fschecks_time09:09:040
149950,0migration/07325-21kthreadcore08:09:090
149950,0migration/04420-21irqcore06:19:090
149950,0migration/032611-21sh09:39:430
149950,0migration/031505-21kthreadcore11:19:140
149950,0migration/030162-21pmu-power09:19:140
149950,0migration/030004-21kthreadcore08:04:120
149950,0migration/029430-21sh10:17:530
149950,0migration/028271-21fschecks_time09:59:050
149950,0migration/026231-21kthreadcore06:54:090
149950,0migration/024338-21taskset10:51:060
149950,0migration/023930-21idleruntime11:34:070
149950,0migration/021149-21latency_hist10:13:510
149950,0migration/018904-21latency_hist09:53:510
149950,0migration/018347-21kthreadcore07:34:070
149950,0migration/018347-21kthreadcore07:34:070
149950,0migration/017043-21kthreadcore09:14:100
149950,0migration/014275-21fschecks_time07:54:060
149950,0migration/013166-21kthreadcore08:14:160
149950,0migration/012382-21fschecks_time08:34:050
29935040,0irq/33-eth0-tx-8689-21munin-node09:29:102
29935040,0irq/33-eth0-tx-6461-21ssh11:40:422
29935040,0irq/33-eth0-tx-22621-21df08:59:032
29935040,0irq/33-eth0-tx-17704-21diskmemload08:34:042
29935040,0irq/33-eth0-tx-11349-21perl09:49:042
29935040,0irq/33-eth0-tx-0-21swapper/211:06:062
29935040,0irq/33-eth0-tx-0-21swapper/210:54:372
29935040,0irq/33-eth0-tx-0-21swapper/210:50:572
29935040,0irq/33-eth0-tx-0-21swapper/210:39:072
29935040,0irq/33-eth0-tx-0-21swapper/210:29:112
29935040,0irq/33-eth0-tx-0-21swapper/210:29:112
29935040,0irq/33-eth0-tx-0-21swapper/210:25:432
29935040,0irq/33-eth0-tx-0-21swapper/210:14:012
29935040,0irq/33-eth0-tx-0-21swapper/210:09:092
29935040,0irq/33-eth0-tx-0-21swapper/210:00:202
29935040,0irq/33-eth0-tx-0-21swapper/209:44:112
29935040,0irq/33-eth0-tx-0-21swapper/209:03:542
29935040,0irq/33-eth0-tx-0-21swapper/208:49:092
29935040,0irq/33-eth0-tx-0-21swapper/208:38:582
29935040,0irq/33-eth0-tx-0-21swapper/208:20:032
29935040,0irq/33-eth0-tx-0-21swapper/008:18:540
29935040,0irq/33-eth0-tx-0-21swapper/007:29:080
28529944,0cyclictest0-21swapper/206:51:022
28529943,0cyclictest5686-21awk07:04:082
28529943,0cyclictest3599-21lldpd09:36:282
28529943,0cyclictest3599-21lldpd07:16:062
28529943,0cyclictest3599-21lldpd06:57:022
28529943,0cyclictest3262-21irqbalance06:42:052
28529943,0cyclictest32137-21awk07:44:062
28529943,0cyclictest2921-21sadc08:08:502
28529943,0cyclictest26243-21awk06:34:102
28529943,0cyclictest23871-21snmpd08:16:592
28529943,0cyclictest20053-21awk09:54:082
28529943,0cyclictest17441-21awk07:34:052
28529943,0cyclictest17441-21awk07:34:052
28529943,0cyclictest1186-21awk09:24:072
28529940,0cyclictest0-21swapper/211:43:582
28519944,0cyclictest0-21swapper/106:40:341
28519943,0cyclictest3599-21lldpd09:36:581
28519943,0cyclictest3599-21lldpd09:31:171
28519943,0cyclictest3599-21lldpd07:49:101
28519943,0cyclictest3599-21lldpd07:30:111
28519943,0cyclictest28733-21awk09:59:071
28519943,0cyclictest26799-21awk09:39:071
28519943,0cyclictest24777-21awk07:39:081
28519943,0cyclictest23871-21snmpd11:01:151
28519943,0cyclictest23383-21awk08:59:061
28519943,0cyclictest19064-21awk08:19:071
28509944,0cyclictest22347-21kworker/0:010:31:050
28509944,0cyclictest22347-21kworker/0:010:31:050
28509944,0cyclictest0-21swapper/008:50:230
28509944,0cyclictest0-21swapper/006:47:530
28509943,0cyclictest28495-21awk10:59:090
28509943,0cyclictest24761-21awk07:39:080
28509943,0cyclictest23407-21awk08:59:070
259940,0migration/27015-21kthreadcore08:29:162
259940,0migration/228601-21kthreadcore07:19:092
259940,0migration/228247-21kthreadcore09:19:112
259940,0migration/227950-21kthreadcore09:39:082
259940,0migration/227650-21munin-node08:24:072
259940,0migration/226163-21kthreadcore07:39:102
259940,0migration/218750-21munin-node11:09:132
259940,0migration/212729-21kthreadcore10:44:082
259940,0migration/212271-21munin-node07:49:122
259940,0migration/211558-21kthreadcore11:24:092
249940,0watchdog/230353-21df10:19:032
249940,0watchdog/213532-21grep07:29:112
249940,0watchdog/20-21swapper/211:28:512
249940,0watchdog/20-21swapper/211:19:152
249940,0watchdog/20-21swapper/209:14:112
249940,0watchdog/20-21swapper/208:09:072
249940,0watchdog/20-21swapper/206:29:122
179940,0migration/17487-21fschecks_time09:09:041
179940,0migration/16839-21kthreadcore06:19:101
179940,0migration/131318-21kthreadcore08:04:131
179940,0migration/130479-21fschecks_time11:19:111
179940,0migration/122937-21kthreadcore07:14:101
179940,0migration/120433-21taskset07:57:071
179940,0migration/120059-21latency_hist08:38:511
179940,0migration/118225-21sh07:09:201
179940,0migration/114701-21kthreadcore10:09:101
179940,0migration/112641-21kthreadcore08:14:161
179940,0migration/112226-21kthreadcore06:44:131
179940,0migration/112160-21kthreadcore11:44:091
179940,0migration/110433-21sh09:48:581
169940,0watchdog/16941-21fschecks_count10:24:071
169940,0watchdog/14935-21open_inodes07:44:111
169940,0watchdog/125178-21munin-node06:34:161
169940,0watchdog/12092-21md1_raid108:11:111
169940,0watchdog/117704-21diskmemload11:23:591
169940,0watchdog/10-21swapper/111:09:431
169940,0watchdog/10-21swapper/111:06:351
169940,0watchdog/10-21swapper/110:43:391
169940,0watchdog/10-21swapper/109:04:031
169940,0watchdog/10-21swapper/108:29:151
169940,0watchdog/10-21swapper/106:59:071
149940,0migration/07956-21processes07:24:140
149940,0migration/03750-21unixbench-2d06:59:150
149940,0migration/03128-21kthreadcore10:39:090
149940,0migration/015081-21sh11:45:450
149940,0migration/012390-21latency10:24:120
29935030,0irq/33-eth0-tx-0-21swapper/211:33:522
28529932,0cyclictest3599-21lldpd06:23:582
28529932,0cyclictest15549-21kthreadcore07:54:102
28529932,0cyclictest12954-21awk07:09:082
28529931,0cyclictest23163-21grep07:59:102
28529930,0cyclictest1974-21df_inode07:24:052
28509933,0cyclictest9558-21kworker/0:209:48:590
28509933,0cyclictest9558-21kworker/0:209:28:530
28509933,0cyclictest9558-21kworker/0:208:23:530
28509933,0cyclictest9351-21kworker/0:211:29:010
28509933,0cyclictest9351-21kworker/0:211:13:550
28509933,0cyclictest3518-21kworker/0:207:09:080
28509933,0cyclictest22347-21kworker/0:010:33:550
28509933,0cyclictest22347-21kworker/0:010:19:010
28509933,0cyclictest17262-21kworker/0:106:24:040
249930,0watchdog/20-21swapper/206:43:592
169930,0watchdog/10-21swapper/110:13:551
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional