You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-26 - 18:51
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot2.osadl.org (updated Tue Aug 26, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
772526564,0sleep00-21swapper/006:23:250
191500,0ktimersoftd/1418-21perf06:25:021
7952994949,0cyclictest1612-21md1_resync08:40:320
7953994545,0cyclictest1613-21md0_resync08:40:321
768023828,0sleep30-21swapper/306:22:523
759023026,0sleep20-21swapper/206:21:512
329980,0watchdog/314907-21kthreadcore10:50:333
8315070,0irq/26-0000:00:9006-21awk07:05:383
8315070,0irq/26-0000:00:6453-21seq07:29:113
8315070,0irq/26-0000:00:4222-21forks09:10:293
8315070,0irq/26-0000:00:0-21swapper/311:30:503
8315070,0irq/26-0000:00:0-21swapper/309:01:543
8315070,0irq/26-0000:00:0-21swapper/308:41:203
8315070,0irq/26-0000:00:0-21swapper/308:30:273
329970,0watchdog/30-21swapper/311:09:283
29925070,0irq/32-eth0-rx-0-21swapper/311:18:083
29925070,0irq/32-eth0-rx-0-21swapper/310:32:393
29925070,0irq/32-eth0-rx-0-21swapper/308:27:123
8315060,0irq/26-0000:00:3590-21sed09:25:393
8315060,0irq/26-0000:00:31165-21kthreadcore09:05:303
8315060,0irq/26-0000:00:23729-21grep06:35:303
8315060,0irq/26-0000:00:22776-21kthreadcore08:00:303
8315060,0irq/26-0000:00:22530-21if_err_eth007:40:283
8315060,0irq/26-0000:00:1556-21taskset11:37:123
8315060,0irq/26-0000:00:14619-21switchtime08:15:393
8315060,0irq/26-0000:00:11954-21fschecks_time07:55:293
8315060,0irq/26-0000:00:0-21swapper/311:25:353
8315060,0irq/26-0000:00:0-21swapper/310:25:363
8315060,0irq/26-0000:00:0-21swapper/310:05:453
8315060,0irq/26-0000:00:0-21swapper/310:01:333
8315060,0irq/26-0000:00:0-21swapper/309:55:183
8315060,0irq/26-0000:00:0-21swapper/309:41:043
8315060,0irq/26-0000:00:0-21swapper/308:46:323
8315060,0irq/26-0000:00:0-21swapper/308:05:343
8315060,0irq/26-0000:00:0-21swapper/307:45:303
8315060,0irq/26-0000:00:0-21swapper/307:35:273
8315060,0irq/26-0000:00:0-21swapper/306:50:443
79549966,0cyclictest22713-21diskmemload11:07:002
79549966,0cyclictest0-21swapper/211:32:332
79549966,0cyclictest0-21swapper/208:17:412
79549966,0cyclictest0-21swapper/206:49:232
79549960,0cyclictest0-21swapper/209:31:092
79549960,0cyclictest0-21swapper/208:54:532
79539966,0cyclictest0-21swapper/108:00:511
79539960,0cyclictest0-21swapper/111:30:131
79539960,0cyclictest0-21swapper/110:31:531
79539960,0cyclictest0-21swapper/109:16:411
79529960,0cyclictest0-21swapper/009:00:410
79529960,0cyclictest0-21swapper/008:53:260
79529960,0cyclictest0-21swapper/008:48:230
4160,0ktimersoftd/00-21swapper/009:18:400
329960,0watchdog/33905-21nfs4_client06:40:333
329960,0watchdog/30-21swapper/311:54:043
329960,0watchdog/30-21swapper/311:47:083
329960,0watchdog/30-21swapper/311:43:083
329960,0watchdog/30-21swapper/310:55:283
329960,0watchdog/30-21swapper/310:15:453
329960,0watchdog/30-21swapper/309:31:213
329960,0watchdog/30-21swapper/309:15:293
329960,0watchdog/30-21swapper/308:56:093
329960,0watchdog/30-21swapper/308:50:573
329960,0watchdog/30-21swapper/308:14:293
329960,0watchdog/30-21swapper/307:18:493
329960,0watchdog/30-21swapper/307:10:253
329960,0watchdog/30-21swapper/307:05:093
29925060,0irq/32-eth0-rx-8202-21grep08:35:273
29925060,0irq/32-eth0-rx-4246-21ssh10:45:003
29925060,0irq/32-eth0-rx-0-21swapper/311:12:263
29925060,0irq/32-eth0-rx-0-21swapper/311:01:253
29925060,0irq/32-eth0-rx-0-21swapper/310:47:223
29925060,0irq/32-eth0-rx-0-21swapper/310:38:033
29925060,0irq/32-eth0-rx-0-21swapper/310:38:033
29925060,0irq/32-eth0-rx-0-21swapper/310:13:543
29925060,0irq/32-eth0-rx-0-21swapper/309:53:073
29925060,0irq/32-eth0-rx-0-21swapper/309:45:313
29925060,0irq/32-eth0-rx-0-21swapper/309:35:393
29925060,0irq/32-eth0-rx-0-21swapper/309:20:313
27160,0ktimersoftd/27812-21ssh10:06:402
8315050,0irq/26-0000:00:8160-3199-raid-check06:25:183
8315050,0irq/26-0000:00:5315-21cpu06:45:253
8315050,0irq/26-0000:00:0-21swapper/311:20:313
8315050,0irq/26-0000:00:0-21swapper/307:50:303
8315050,0irq/26-0000:00:0-21swapper/306:55:273
8315050,0irq/26-0000:00:0-21swapper/306:30:163
79549955,0cyclictest0-21swapper/211:43:252
79549955,0cyclictest0-21swapper/210:23:292
79549955,0cyclictest0-21swapper/210:15:172
79549955,0cyclictest0-21swapper/209:25:002
79549955,0cyclictest0-21swapper/208:14:132
79549954,0cyclictest0-21swapper/208:40:272
79539955,0cyclictest23871-21snmpd09:42:341
79539955,0cyclictest23871-21snmpd07:42:101
79539955,0cyclictest0-21swapper/107:55:391
79539955,0cyclictest0-21swapper/107:37:311
79539954,0cyclictest30387-21kthreadcore09:05:301
79539950,0cyclictest0-21swapper/110:00:401
79539950,0cyclictest0-21swapper/109:50:161
79539950,0cyclictest0-21swapper/109:46:571
79539950,0cyclictest0-21swapper/109:34:021
79529955,0cyclictest0-21swapper/011:01:270
79529955,0cyclictest0-21swapper/010:55:370
79529955,0cyclictest0-21swapper/010:48:130
79529955,0cyclictest0-21swapper/009:27:290
79529955,0cyclictest0-21swapper/008:27:350
79529955,0cyclictest0-21swapper/008:12:080
79529955,0cyclictest0-21swapper/006:31:040
79529950,0cyclictest0-21swapper/011:39:360
79529950,0cyclictest0-21swapper/011:32:210
79529950,0cyclictest0-21swapper/009:50:590
79529950,0cyclictest0-21swapper/009:06:500
339950,0migration/327193-21fschecks_time10:20:263
339950,0migration/316544-21gltestperf08:20:323
329950,0watchdog/30-21swapper/307:31:173
29935050,0irq/33-eth0-tx-6799-21munin-node07:30:280
29935050,0irq/33-eth0-tx-6315-21iceccd08:01:320
29935050,0irq/33-eth0-tx-31738-21munin-node07:25:360
29935050,0irq/33-eth0-tx-24258-21munin-node11:15:252
29935050,0irq/33-eth0-tx-21507-21munin-node07:40:390
29935050,0irq/33-eth0-tx-17112-21ssh11:28:412
29935050,0irq/33-eth0-tx-0-21swapper/211:14:422
29935050,0irq/33-eth0-tx-0-21swapper/210:55:542
29935050,0irq/33-eth0-tx-0-21swapper/007:10:480
29925050,0irq/32-eth0-rx-0-21swapper/307:20:273
259950,0migration/28636-21sh11:24:112
259950,0migration/232095-21timerandwakeup08:45:442
259950,0migration/230988-21kthreadcore09:05:302
259950,0migration/226908-21switchtime11:50:392
259950,0migration/221231-21kthreadcore06:30:352
259950,0migration/213191-21fschecks_count06:50:362
249950,0watchdog/215945-21kthreadcore08:55:332
19150,0ktimersoftd/10-21swapper/111:15:431
19150,0ktimersoftd/10-21swapper/109:13:511
179950,0migration/17810-21latency_hist08:35:121
179950,0migration/130987-21fschecks_time06:40:271
179950,0migration/128986-21kthreadcore07:00:311
179950,0migration/127133-21fschecks_count11:35:271
179950,0migration/115544-21munin-node10:10:351
179950,0migration/110236-21gltestperf11:25:311
169950,0watchdog/116935-21idleruntime09:55:291
149950,0migration/07026-21sh11:20:360
149950,0migration/06733-21runrttasks09:49:180
149950,0migration/032354-21sh11:19:350
149950,0migration/03201-21sh10:42:040
149950,0migration/029539-21kthreadcore10:00:320
149950,0migration/029450-21kthreadcore07:20:340
149950,0migration/02695-21df_abs07:05:250
149950,0migration/026589-21unixbench_multi06:55:400
149950,0migration/019077-21taskset11:47:260
149950,0migration/016905-21hwlatdetect09:55:280
149950,0migration/015652-21munin-node11:25:390
149950,0migration/011576-21sh09:13:580
149950,0migration/010821-21kernelversion10:10:310
79549943,0cyclictest3599-21lldpd07:39:332
79549943,0cyclictest3599-21lldpd06:57:472
79549943,0cyclictest3262-21irqbalance08:30:272
79549943,0cyclictest3262-21irqbalance08:20:472
79549943,0cyclictest3262-21irqbalance06:25:172
79549943,0cyclictest25627-21sadc08:05:132
79549943,0cyclictest25535-21awk07:20:322
79549943,0cyclictest23871-21snmpd08:07:242
79549943,0cyclictest23871-21snmpd07:18:242
79549943,0cyclictest23871-21snmpd07:05:242
79549943,0cyclictest23871-21snmpd06:35:532
79549943,0cyclictest22555-21awk07:40:282
79549943,0cyclictest12353-21awk07:55:322
79539944,0cyclictest0-21swapper/111:51:301
79539944,0cyclictest0-21swapper/110:17:041
79539943,0cyclictest23871-21snmpd10:55:401
79539943,0cyclictest23871-21snmpd10:23:251
79539943,0cyclictest23871-21snmpd08:56:251
79539943,0cyclictest23871-21snmpd08:20:391
79539943,0cyclictest18126-21awk07:15:301
79539943,0cyclictest16642-21awk06:30:321
79539943,0cyclictest15204-21awk09:35:301
79529944,0cyclictest0-21swapper/010:15:340
79529944,0cyclictest0-21swapper/008:21:540
79529943,0cyclictest3599-21lldpd09:41:570
79529943,0cyclictest3599-21lldpd09:30:350
79529943,0cyclictest3262-21irqbalance07:00:470
79529943,0cyclictest23871-21snmpd08:05:240
79529943,0cyclictest21389-21awk11:50:280
79529943,0cyclictest20934-21awk10:35:280
79529943,0cyclictest20934-21awk10:35:280
79529943,0cyclictest16804-21awk11:10:290
79529943,0cyclictest15186-21awk07:35:290
79529943,0cyclictest12295-21awk10:30:310
79529940,0cyclictest8258-21munin-node08:15:270
29935040,0irq/33-eth0-tx-26964-21ssh10:36:432
29935040,0irq/33-eth0-tx-26964-21ssh10:36:432
29935040,0irq/33-eth0-tx-26577-21munin-node11:35:262
29935040,0irq/33-eth0-tx-24443-21munin-node10:00:272
29935040,0irq/33-eth0-tx-14730-21ps10:50:312
29935040,0irq/33-eth0-tx-11693-21ssh11:45:192
29935040,0irq/33-eth0-tx-0-21swapper/211:00:212
29935040,0irq/33-eth0-tx-0-21swapper/210:45:262
29935040,0irq/33-eth0-tx-0-21swapper/210:31:202
29935040,0irq/33-eth0-tx-0-21swapper/210:10:302
29935040,0irq/33-eth0-tx-0-21swapper/209:56:442
29935040,0irq/33-eth0-tx-0-21swapper/209:50:312
29935040,0irq/33-eth0-tx-0-21swapper/209:40:272
29935040,0irq/33-eth0-tx-0-21swapper/209:25:262
29935040,0irq/33-eth0-tx-0-21swapper/209:10:312
29935040,0irq/33-eth0-tx-0-21swapper/209:00:152
29935040,0irq/33-eth0-tx-0-21swapper/208:35:262
29935040,0irq/33-eth0-tx-0-21swapper/007:45:400
29935040,0irq/33-eth0-tx-0-21swapper/006:45:270
29935040,0irq/33-eth0-tx-0-21swapper/006:26:550
259940,0migration/24058-21irqrtprio10:25:322
259940,0migration/232009-21kthreadcore06:40:312
259940,0migration/231126-21runrttasks09:45:302
259940,0migration/223221-21cpuspeed_turbos08:25:292
259940,0migration/214240-21cpuspeed_turbos09:35:262
259940,0migration/212928-21fschecks_count09:15:272
259940,0migration/210494-21fschecks_time07:10:282
249940,0watchdog/23031-21ssh10:41:332
249940,0watchdog/22092-21md1_raid107:30:292
179940,0migration/19554-21latency_hist07:10:121
179940,0migration/18449-21timerandwakeup08:50:391
179940,0migration/18277-21aten_r1power_cu08:15:251
179940,0migration/17838-21taskset10:06:441
179940,0migration/17833-21fschecks_time11:05:261
179940,0migration/15767-21fschecks_count06:45:261
179940,0migration/15595-21kthreadcore07:50:311
179940,0migration/14299-21diskmemload11:00:361
179940,0migration/13025-21kthreadcore07:25:351
179940,0migration/129262-21df_inode07:45:261
179940,0migration/129072-21kthreadcore08:05:321
179940,0migration/125991-21kthreadcore08:25:341
179940,0migration/125552-21aten_r1power_fr08:45:301
179940,0migration/119746-21latency_hist10:35:121
179940,0migration/119746-21latency_hist10:35:121
179940,0migration/119105-21runrttasks11:47:321
179940,0migration/118511-21kthreadcore11:10:301
179940,0migration/11690-21gltestperf08:10:281
179940,0migration/113718-21kthreadcore07:10:321
179940,0migration/113078-21forks06:50:361
179940,0migration/110754-21munin-node10:45:331
169940,0watchdog/17230-21forks07:30:291
169940,0watchdog/15708-21kthreadcore10:25:331
169940,0watchdog/126952-21chrt06:58:331
169940,0watchdog/125699-21kthreadcore09:20:331
169940,0watchdog/12379-21python10:40:371
169940,0watchdog/10-21swapper/111:40:561
169940,0watchdog/10-21swapper/110:50:331
169940,0watchdog/10-21swapper/109:25:211
169940,0watchdog/10-21swapper/107:20:291
149940,0migration/03585-21kthreadcore08:30:320
149940,0migration/031103-21gltestperf06:40:270
149940,0migration/025322-21kthreadcore06:35:350
149940,0migration/023814-21taskset07:16:240
149940,0migration/021008-21sh09:20:260
149940,0migration/018399-21kthreadcore06:50:410
149940,0migration/014290-21cpuspeed_turbos09:35:260
149940,0migration/010735-21taskset07:52:310
149940,0migration/010713-21aten_r1power_po08:55:250
79549932,0cyclictest4348-21grep07:50:272
79549930,0cyclictest483-21iostat_ios07:25:332
79539932,0cyclictest3599-21lldpd09:00:271
79539932,0cyclictest32563-21awk08:30:291
79539932,0cyclictest24029-21awk06:35:321
79539932,0cyclictest1516-21awk11:20:281
79539930,0cyclictest8903-21perl06:25:271
79529933,0cyclictest8952-21kworker/0:211:05:150
79529933,0cyclictest8952-21kworker/0:210:50:270
79529933,0cyclictest8952-21kworker/0:210:25:130
79529933,0cyclictest8952-21kworker/0:210:20:190
79529933,0cyclictest8952-21kworker/0:210:05:230
79529933,0cyclictest4154-21kworker/0:008:35:210
79529933,0cyclictest4154-21kworker/0:007:55:310
79529933,0cyclictest24401-21kworker/0:111:40:130
259930,0migration/229085-21cpuspeed_turbos07:45:262
259930,0migration/228645-21irqcore07:00:302
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional