You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-13 - 10:27
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Sun Jul 13, 2025 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
915525756,0sleep20-21swapper/218:25:482
914123626,0sleep30-21swapper/318:25:373
901723524,0sleep00-21swapper/018:24:100
898723525,0sleep10-21swapper/118:23:511
35170,0ktimersoftd/39089-21cut21:53:163
329970,0watchdog/320982-21tune2fs23:53:113
329970,0watchdog/320982-21tune2fs23:53:113
329970,0watchdog/312822-21kthreadcore19:13:163
329970,0watchdog/30-21swapper/322:41:443
329970,0watchdog/30-21swapper/321:28:163
329970,0watchdog/30-21swapper/321:12:123
29925070,0irq/32-eth0-rx-24929-21kthreadcore21:23:173
29925070,0irq/32-eth0-rx-19475-21cat21:58:133
29925070,0irq/32-eth0-rx-12729-21expr22:33:093
29925070,0irq/32-eth0-rx-0-21swapper/323:23:063
29925070,0irq/32-eth0-rx-0-21swapper/320:03:153
94079960,0cyclictest0-21swapper/223:13:132
339960,0migration/325315-21sh21:03:253
339960,0migration/311545-21munin-node21:33:183
329960,0watchdog/38931-21grep20:18:163
329960,0watchdog/37439-21kthreadcore19:08:163
329960,0watchdog/37399-21grep19:33:123
329960,0watchdog/30-21swapper/323:43:473
329960,0watchdog/30-21swapper/323:38:313
329960,0watchdog/30-21swapper/323:35:353
329960,0watchdog/30-21swapper/323:01:123
329960,0watchdog/30-21swapper/322:48:203
329960,0watchdog/30-21swapper/322:29:163
329960,0watchdog/30-21swapper/322:18:043
329960,0watchdog/30-21swapper/322:04:163
329960,0watchdog/30-21swapper/321:38:523
329960,0watchdog/30-21swapper/320:44:123
329960,0watchdog/30-21swapper/320:33:283
329960,0watchdog/30-21swapper/320:28:203
329960,0watchdog/30-21swapper/319:27:563
329960,0watchdog/30-21swapper/319:20:483
329960,0watchdog/30-21swapper/318:49:323
29925060,0irq/32-eth0-rx-9358-21perl23:08:113
29925060,0irq/32-eth0-rx-28502-21munin-node22:23:143
29925060,0irq/32-eth0-rx-25418-21ssh23:17:443
29925060,0irq/32-eth0-rx-0-21swapper/323:48:003
29925060,0irq/32-eth0-rx-0-21swapper/322:43:343
29925060,0irq/32-eth0-rx-0-21swapper/322:13:263
29925060,0irq/32-eth0-rx-0-21swapper/321:48:113
29925060,0irq/32-eth0-rx-0-21swapper/321:18:173
29925060,0irq/32-eth0-rx-0-21swapper/321:13:513
29925060,0irq/32-eth0-rx-0-21swapper/321:00:573
29925060,0irq/32-eth0-rx-0-21swapper/320:52:593
29925060,0irq/32-eth0-rx-0-21swapper/320:50:173
29925060,0irq/32-eth0-rx-0-21swapper/320:42:293
149960,0migration/010050-21sendmail_mailst22:28:180
94079950,0cyclictest0-21swapper/221:53:272
94079950,0cyclictest0-21swapper/221:10:242
94079950,0cyclictest0-21swapper/220:59:222
94079950,0cyclictest0-21swapper/220:45:392
94059955,0cyclictest0-21swapper/023:18:180
94059955,0cyclictest0-21swapper/021:42:360
94059950,0cyclictest0-21swapper/021:20:510
8315050,0irq/26-0000:00:11693-21latency_hist21:17:552
339950,0migration/328303-21latency_hist19:47:553
339950,0migration/327946-21aten_r1power_cu19:03:093
339950,0migration/321802-21fschecks_time19:43:113
329950,0watchdog/30-21swapper/323:27:553
329950,0watchdog/30-21swapper/323:18:043
329950,0watchdog/30-21swapper/322:53:243
329950,0watchdog/30-21swapper/322:07:563
329950,0watchdog/30-21swapper/320:23:043
329950,0watchdog/30-21swapper/320:15:163
329950,0watchdog/30-21swapper/320:11:203
329950,0watchdog/30-21swapper/319:58:043
329950,0watchdog/30-21swapper/319:53:203
329950,0watchdog/30-21swapper/319:38:403
329950,0watchdog/30-21swapper/319:23:123
329950,0watchdog/30-21swapper/318:59:323
329950,0watchdog/30-21swapper/318:53:043
329950,0watchdog/30-21swapper/318:43:403
329950,0watchdog/30-21swapper/318:38:243
329950,0watchdog/30-21swapper/318:34:043
329950,0watchdog/30-21swapper/318:28:363
29935050,0irq/33-eth0-tx-6717-21ssh23:04:002
29935050,0irq/33-eth0-tx-29846-21python22:58:202
29935050,0irq/33-eth0-tx-17265-21munin-node19:18:120
29935050,0irq/33-eth0-tx-0-21swapper/221:03:132
29935050,0irq/33-eth0-tx-0-21swapper/220:31:472
29935050,0irq/33-eth0-tx-0-21swapper/020:28:060
29935050,0irq/33-eth0-tx-0-21swapper/020:18:040
29935050,0irq/33-eth0-tx-0-21swapper/019:48:220
29935050,0irq/33-eth0-tx-0-21swapper/019:37:560
29935050,0irq/33-eth0-tx-0-21swapper/019:23:200
29935050,0irq/33-eth0-tx-0-21swapper/018:58:210
29925050,0irq/32-eth0-rx-23036-21df_abs21:43:153
29925050,0irq/32-eth0-rx-0-21swapper/323:03:503
259950,0migration/211411-21runrttasks22:30:582
249950,0watchdog/23080-21chrt22:27:162
179950,0migration/117966-21kthreadcore19:38:151
179950,0migration/115348-21sh20:42:071
179950,0migration/113420-21rt-features20:18:201
179950,0migration/112436-21kthreadcore22:13:161
179950,0migration/112410-21munin-node21:18:121
179950,0migration/110152-21munin-node23:43:211
169950,0watchdog/115603-21cut21:38:161
149950,0migration/09948-21aten_r1power_po22:13:100
149950,0migration/08320-21sh20:54:160
149950,0migration/03478-21irqrtprio23:23:170
149950,0migration/031455-21timerwakeupswit20:08:220
149950,0migration/031124-21munin-node22:03:170
149950,0migration/027150-21timerwakeupswit21:23:240
149950,0migration/026857-21kthreadcore18:38:160
149950,0migration/026408-21sh23:34:480
149950,0migration/025434-21fschecks_time20:48:090
149950,0migration/024909-21sendmail_mailqu21:03:180
149950,0migration/021722-21kthreadcore20:03:150
149950,0migration/01987-21sh22:24:240
149950,0migration/019478-21gltestperf22:18:100
149950,0migration/014541-21kthreadcore19:13:170
149950,0migration/014525-21sh20:40:000
149950,0migration/013655-21aten_r1power_vo18:53:100
149950,0migration/013244-21kthreadcore23:08:150
149950,0migration/012313-21missed_timers22:48:180
94079943,0cyclictest3599-21lldpd21:34:092
94079943,0cyclictest3599-21lldpd20:25:572
94079943,0cyclictest3599-21lldpd19:57:582
94079943,0cyclictest3599-21lldpd19:02:282
94079943,0cyclictest3599-21lldpd18:41:272
94079943,0cyclictest32443-21awk18:43:122
94079943,0cyclictest23871-21snmpd22:03:402
94079943,0cyclictest23871-21snmpd18:52:392
94079943,0cyclictest18329-21awk19:18:142
94069943,0cyclictest7733-21awk19:33:141
94069943,0cyclictest4729-21awk22:28:111
94069943,0cyclictest4387-21awk19:53:121
94069943,0cyclictest3599-21lldpd22:48:141
94069943,0cyclictest3599-21lldpd21:43:091
94069943,0cyclictest3599-21lldpd20:04:581
94069943,0cyclictest3262-21irqbalance23:32:591
94069943,0cyclictest32539-21awk18:43:131
94069943,0cyclictest3193-21awk23:23:171
94069943,0cyclictest31718-21awk20:33:121
94069943,0cyclictest28298-21awk21:08:161
94069943,0cyclictest26930-21awk23:18:111
94069943,0cyclictest18235-21awk23:13:121
94069943,0cyclictest17924-21awk18:33:151
94069943,0cyclictest15958-21sadc21:57:551
94069943,0cyclictest15598-21awk20:23:141
94069943,0cyclictest11066-21awk20:58:141
94059944,0cyclictest8608-21kworker/0:119:08:160
94059944,0cyclictest0-21swapper/023:50:470
94059944,0cyclictest0-21swapper/023:40:320
94059944,0cyclictest0-21swapper/021:28:250
94059943,0cyclictest32412-21awk18:43:120
94059943,0cyclictest29005-21awk19:03:130
94059943,0cyclictest23871-21snmpd21:44:250
94059943,0cyclictest2231-21kworker/0:019:53:080
94059943,0cyclictest22237-21awk22:38:110
94059943,0cyclictest22205-21awk19:43:120
94059943,0cyclictest2086-21kworker/0:221:34:180
94059943,0cyclictest17069-21awk20:43:120
94059943,0cyclictest10663-21awk18:28:140
8315040,0irq/26-0000:00:0-21swapper/221:27:552
29935040,0irq/33-eth0-tx-3679-21ssh23:42:572
29935040,0irq/33-eth0-tx-28720-21idleruntime23:38:152
29935040,0irq/33-eth0-tx-27416-21cat20:48:132
29935040,0irq/33-eth0-tx-22592-21ssh21:42:572
29935040,0irq/33-eth0-tx-19041-21ssh23:48:552
29935040,0irq/33-eth0-tx-18783-21ssh22:33:202
29935040,0irq/33-eth0-tx-0-21swapper/223:25:032
29935040,0irq/33-eth0-tx-0-21swapper/223:18:032
29935040,0irq/33-eth0-tx-0-21swapper/222:49:232
29935040,0irq/33-eth0-tx-0-21swapper/222:37:592
29935040,0irq/33-eth0-tx-0-21swapper/222:18:022
29935040,0irq/33-eth0-tx-0-21swapper/222:12:592
29935040,0irq/33-eth0-tx-0-21swapper/221:58:542
29935040,0irq/33-eth0-tx-0-21swapper/221:49:322
29935040,0irq/33-eth0-tx-0-21swapper/221:38:012
29935040,0irq/33-eth0-tx-0-21swapper/221:23:102
29935040,0irq/33-eth0-tx-0-21swapper/221:13:102
29935040,0irq/33-eth0-tx-0-21swapper/220:53:102
29935040,0irq/33-eth0-tx-0-21swapper/019:33:240
29935040,0irq/33-eth0-tx-0-21swapper/018:52:470
259940,0migration/29873-21latency_hist19:12:552
259940,0migration/29255-21needreboot19:53:162
259940,0migration/24898-21kthreadcore22:08:172
259940,0migration/231041-21kthreadcore19:48:142
259940,0migration/229664-21kthreadcore19:23:162
259940,0migration/227960-21aten_r1power_cu19:03:092
259940,0migration/22647-21kthreadcore20:33:142
259940,0migration/222406-21kthreadcore20:03:152
259940,0migration/215463-21sh23:09:002
259940,0migration/211528-21kthreadcore18:28:162
249940,0watchdog/2941-21kthreadcore19:28:162
249940,0watchdog/27400-21cut19:33:122
249940,0watchdog/232342-21grep22:43:162
249940,0watchdog/220984-21cut23:53:112
249940,0watchdog/220984-21cut23:53:112
249940,0watchdog/216567-21kthreadcore23:28:152
249940,0watchdog/215323-21diskstats22:53:122
249940,0watchdog/20-21swapper/223:33:152
249940,0watchdog/20-21swapper/220:38:162
179940,0migration/18675-21kthreadcore18:48:141
179940,0migration/17416-21kthreadcore21:13:151
179940,0migration/16513-21idleruntime21:33:141
179940,0migration/132434-21kthreadcore19:03:161
179940,0migration/130235-21cpuspeed_turbos22:43:111
179940,0migration/127386-21taskset18:59:221
179940,0migration/125937-21irqrtprio19:23:151
179940,0migration/125780-21fschecks_time22:03:091
179940,0migration/124522-21latency_hist20:47:551
179940,0migration/121778-21kthreadcore21:03:141
179940,0migration/120472-21turbostat18:57:551
179940,0migration/120463-21latency_hist21:22:551
179940,0migration/118487-21latency_hist22:17:551
179940,0migration/118208-21kthreadcore22:53:161
179940,0migration/117781-21kthreadcore22:33:151
179940,0migration/115941-21taskset20:43:061
179940,0migration/11210-21aten_r1power_po20:53:101
169940,0watchdog/126744-21tr18:38:161
169940,0watchdog/123742-21ssh22:00:281
169940,0watchdog/123475-21ssh22:58:001
169940,0watchdog/116569-21kthreadcore23:28:151
169940,0watchdog/115239-21cut19:58:161
169940,0watchdog/112039-21kthreadcore18:28:161
169940,0watchdog/10-21swapper/123:52:591
169940,0watchdog/10-21swapper/123:52:591
169940,0watchdog/10-21swapper/121:28:121
169940,0watchdog/10-21swapper/120:13:161
169940,0watchdog/10-21swapper/119:48:161
169940,0watchdog/10-21swapper/119:18:161
149940,0migration/04241-21kthreadcore22:08:160
149940,0migration/03907-21kthreadcore21:48:170
149940,0migration/022495-21proc_pri21:58:170
149940,0migration/018896-21sh23:32:450
149940,0migration/017079-21cpu18:33:110
149940,0migration/017074-21latency_hist23:12:550
149940,0migration/014711-21cpuspeed_turbos20:23:100
149940,0migration/01334-21irqrtprio20:13:150
149940,0migration/01028-21kthreadcore19:28:160
94079932,0cyclictest3599-21lldpd19:12:572
94079932,0cyclictest17589-21tune2fs18:33:132
94079932,0cyclictest14960-21awk19:38:122
94079930,0cyclictest27707-21users19:43:232
94069932,0cyclictest9055-21sensors19:08:201
94069932,0cyclictest7580-21sensors22:08:211
94069932,0cyclictest426-21awk19:28:151
94069932,0cyclictest3599-21lldpd19:46:571
94069932,0cyclictest29587-21grep20:08:151
94069932,0cyclictest28759-21awk23:38:151
94069932,0cyclictest14864-21sensors23:08:191
94069932,0cyclictest11251-21grep19:13:151
94059933,0cyclictest3599-21lldpd20:32:580
94059933,0cyclictest2086-21kworker/0:223:53:160
94059933,0cyclictest2086-21kworker/0:223:53:160
94059933,0cyclictest2086-21kworker/0:223:03:240
94059933,0cyclictest2086-21kworker/0:222:57:580
94059933,0cyclictest2086-21kworker/0:221:53:240
94059933,0cyclictest2086-21kworker/0:221:08:100
94059933,0cyclictest2086-21kworker/0:220:59:100
94059932,0cyclictest7878-21cut21:13:150
94059932,0cyclictest4047-21tr23:43:080
94059932,0cyclictest31088-21awk22:43:140
94059932,0cyclictest16706-21cut22:53:160
94059930,0cyclictest13167-21perl22:33:100
8315030,0irq/26-0000:00:0-21swapper/220:07:552
29935030,0irq/33-eth0-tx-0-21swapper/019:58:090
259930,0migration/215696-21kthreadcore18:53:162
249930,0watchdog/20-21swapper/220:18:162
249930,0watchdog/20-21swapper/220:13:162
169930,0watchdog/10-21swapper/123:48:151
169930,0watchdog/10-21swapper/123:03:121
169930,0watchdog/10-21swapper/122:38:201
169930,0watchdog/10-21swapper/122:23:161
169930,0watchdog/10-21swapper/121:51:161
169930,0watchdog/10-21swapper/120:27:561
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional