You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-02 - 07:05
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack8slot7.osadl.org (updated Wed Jul 02, 2025 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25994991818,0cyclictest60250irq/125-lan22:15:292
25994991818,0cyclictest60250irq/125-lan00:25:472
25994991817,1cyclictest60250irq/125-lan23:45:242
25994991817,1cyclictest60250irq/125-lan23:06:472
25994991817,1cyclictest60250irq/125-lan21:35:112
25994991817,1cyclictest60250irq/125-lan20:00:242
25994991817,1cyclictest60250irq/125-lan19:20:222
25994991717,0cyclictest60250irq/125-lan23:12:472
25994991717,0cyclictest60250irq/125-lan21:15:422
25994991716,1cyclictest60250irq/125-lan23:56:472
25994991716,1cyclictest60250irq/125-lan23:35:162
25994991716,1cyclictest60250irq/125-lan21:05:222
25994991716,1cyclictest60250irq/125-lan21:00:012
25994991716,1cyclictest60250irq/125-lan20:40:162
25994991716,1cyclictest60250irq/125-lan19:50:132
25994991616,0cyclictest60250irq/125-lan23:16:472
25994991616,0cyclictest60250irq/125-lan21:47:472
25994991616,0cyclictest60250irq/125-lan19:29:462
25994991616,0cyclictest60250irq/125-lan19:19:482
25994991616,0cyclictest60250irq/125-lan00:35:352
25994991616,0cyclictest60250irq/125-lan00:21:472
25994991616,0cyclictest60250irq/125-lan00:00:152
25994991615,1cyclictest60250irq/125-lan22:55:152
25994991515,0cyclictest60250irq/125-lan23:25:212
25994991515,0cyclictest60250irq/125-lan22:52:282
25994991515,0cyclictest60250irq/125-lan22:14:472
25994991515,0cyclictest60250irq/125-lan21:11:472
25994991514,1cyclictest60250irq/125-lan23:30:172
25994991514,1cyclictest60250irq/125-lan23:20:142
25994991514,1cyclictest60250irq/125-lan22:30:182
25994991514,1cyclictest60250irq/125-lan19:40:192
25994991514,1cyclictest60250irq/125-lan00:15:142
25989991515,0cyclictest702-21snmpd19:11:131
25989991515,0cyclictest329-21hwrng22:56:481
25989991515,0cyclictest329-21hwrng21:51:161
25989991515,0cyclictest329-21hwrng21:51:161
25989991515,0cyclictest329-21hwrng19:57:361
25989991515,0cyclictest329-21hwrng19:53:301
25989991515,0cyclictest329-21hwrng19:41:121
25989991515,0cyclictest329-21hwrng19:25:511
25989991514,1cyclictest702-21snmpd19:36:451
25994991414,0cyclictest60250irq/125-lan21:40:212
25994991414,0cyclictest60250irq/125-lan20:50:162
25994991414,0cyclictest60250irq/125-lan20:35:152
25994991414,0cyclictest60250irq/125-lan19:46:562
25994991413,1cyclictest60250irq/125-lan22:35:242
25994991413,1cyclictest60250irq/125-lan22:05:212
25994991413,1cyclictest60250irq/125-lan20:55:212
25994991413,1cyclictest60250irq/125-lan20:20:132
25994991413,1cyclictest60250irq/125-lan19:30:182
25994991413,1cyclictest60250irq/125-lan19:10:182
25989991414,0cyclictest702-21snmpd20:09:101
25989991414,0cyclictest329-21hwrng23:22:241
25989991414,0cyclictest329-21hwrng23:03:581
25989991414,0cyclictest329-21hwrng22:52:421
25989991414,0cyclictest329-21hwrng22:47:351
25989991414,0cyclictest329-21hwrng22:39:241
25989991414,0cyclictest329-21hwrng21:56:231
25989991414,0cyclictest329-21hwrng20:38:341
25989991414,0cyclictest329-21hwrng20:17:041
25989991414,0cyclictest329-21hwrng19:47:221
25989991414,0cyclictest329-21hwrng19:34:031
25984991414,0cyclictest329-21hwrng00:00:170
25994991313,0cyclictest60250irq/125-lan23:00:472
25994991312,1cyclictest60250irq/125-lan22:25:192
25994991312,1cyclictest60250irq/125-lan22:00:162
25994991312,1cyclictest60250irq/125-lan21:55:222
25994991312,1cyclictest60250irq/125-lan20:30:142
25994991312,1cyclictest60250irq/125-lan19:35:222
25994991312,1cyclictest60250irq/125-lan00:30:182
25989991313,0cyclictest702-21snmpd23:26:511
25989991313,0cyclictest702-21snmpd23:11:291
25989991313,0cyclictest702-21snmpd22:40:251
25984991313,0cyclictest329-21hwrng00:15:390
25994991212,0cyclictest60250irq/125-lan23:50:112
25994991212,0cyclictest60250irq/125-lan22:41:472
25994991212,0cyclictest60250irq/125-lan20:45:212
25994991212,0cyclictest60250irq/125-lan20:25:202
25994991211,1cyclictest60250irq/125-lan21:30:202
25994991211,1cyclictest60250irq/125-lan19:55:222
25994991211,1cyclictest60250irq/125-lan00:05:142
25989991212,0cyclictest329-21hwrng21:31:491
25984991212,0cyclictest329-21hwrng20:13:590
25984991212,0cyclictest329-21hwrng00:25:530
25994991110,1cyclictest60250irq/125-lan00:10:202
25989991111,0cyclictest329-21hwrng22:18:551
25984991111,0cyclictest329-21hwrng00:14:370
2599499109,1cyclictest60250irq/125-lan21:25:162
2599499109,1cyclictest60250irq/125-lan20:05:162
25994991010,0cyclictest60250irq/125-lan22:20:012
25989991010,0cyclictest329-21hwrng22:10:431
25989991010,0cyclictest329-21hwrng22:05:361
25989991010,0cyclictest329-21hwrng21:36:561
25989991010,0cyclictest329-21hwrng20:47:471
25989991010,0cyclictest329-21hwrng00:06:261
2598499108,0cyclictest171rcu_preempt20:49:590
25984991010,0cyclictest329-21hwrng20:15:010
259949999,0cyclictest60250irq/125-lan20:15:122
259899999,0cyclictest329-21hwrng23:54:091
259899999,0cyclictest329-21hwrng23:44:561
259899999,0cyclictest329-21hwrng23:39:481
259899999,0cyclictest329-21hwrng21:15:261
259899999,0cyclictest329-21hwrng21:10:181
259899999,0cyclictest329-21hwrng20:57:001
259899999,0cyclictest329-21hwrng20:50:511
259899999,0cyclictest329-21hwrng20:43:411
259899999,0cyclictest329-21hwrng00:21:471
259899999,0cyclictest329-21hwrng00:04:241
259899998,1cyclictest702-21snmpd19:24:211
259849999,0cyclictest329-21hwrng21:14:240
259849998,1cyclictest702-21snmpd22:30:160
259849998,1cyclictest702-21snmpd20:30:140
259849996,0cyclictest171rcu_preempt21:23:490
259989988,0cyclictest329-21hwrng21:26:413
259989987,1cyclictest702-21snmpd22:50:433
259989987,1cyclictest702-21snmpd00:32:263
259989986,2cyclictest702-21snmpd22:55:303
259949988,0cyclictest60250irq/125-lan21:50:192
259949988,0cyclictest60250irq/125-lan21:50:192
259949987,1cyclictest60250irq/125-lan23:40:202
259949987,1cyclictest60250irq/125-lan22:45:152
259949987,1cyclictest60250irq/125-lan20:10:192
259899988,0cyclictest329-21hwrng23:55:101
259899988,0cyclictest329-21hwrng23:45:571
259899988,0cyclictest329-21hwrng23:06:011
259899988,0cyclictest329-21hwrng21:27:431
259899987,1cyclictest702-21snmpd21:21:421
259849988,0cyclictest702-21snmpd20:50:350
259849988,0cyclictest702-21snmpd19:25:300
259849988,0cyclictest702-21snmpd19:23:330
259849988,0cyclictest329-21hwrng23:18:180
259849988,0cyclictest329-21hwrng21:42:030
259849987,1cyclictest702-21snmpd23:23:000
259849987,1cyclictest702-21snmpd20:43:080
259849987,0cyclictest171rcu_preempt21:28:280
259849987,0cyclictest171rcu_preempt19:12:510
259849981,7cyclictest5588-21turbostat19:40:000
259989977,0cyclictest702-21snmpd23:39:033
259989977,0cyclictest702-21snmpd23:24:303
259989977,0cyclictest702-21snmpd22:13:213
259989977,0cyclictest702-21snmpd21:58:183
259989977,0cyclictest702-21snmpd19:44:543
259989977,0cyclictest702-21snmpd19:33:573
259989977,0cyclictest702-21snmpd19:16:573
259989977,0cyclictest702-21snmpd00:11:133
259989977,0cyclictest702-21snmpd00:00:283
259989977,0cyclictest329-21hwrng23:14:133
259989977,0cyclictest329-21hwrng21:41:023
259989977,0cyclictest329-21hwrng21:36:563
259989977,0cyclictest329-21hwrng21:05:113
259989977,0cyclictest329-21hwrng00:18:433
259989976,1cyclictest702-21snmpd22:37:463
259989976,1cyclictest702-21snmpd22:18:303
259989976,1cyclictest702-21snmpd21:30:403
259989976,1cyclictest702-21snmpd20:22:533
259989976,1cyclictest31086-21cat23:15:143
259989975,1cyclictest702-21snmpd23:03:383
259949977,0cyclictest702-21snmpd21:21:032
259899977,0cyclictest329-21hwrng21:45:061
259899977,0cyclictest329-21hwrng21:41:021
259899977,0cyclictest329-21hwrng21:00:041
259899976,1cyclictest702-21snmpd00:13:401
259849977,0cyclictest702-21snmpd23:46:100
259849977,0cyclictest702-21snmpd23:02:080
259849977,0cyclictest329-21hwrng00:35:060
259849976,1cyclictest702-21snmpd19:34:330
259849976,1cyclictest680-21ntpd20:28:130
259849976,0cyclictest171rcu_preempt21:58:560
259849976,0cyclictest171rcu_preempt21:30:070
259849976,0cyclictest171rcu_preempt20:37:100
259849976,0cyclictest171rcu_preempt00:08:470
259989966,0cyclictest702-21snmpd22:28:283
259989966,0cyclictest702-21snmpd19:20:123
259989966,0cyclictest702-21snmpd19:14:413
259989966,0cyclictest702-21snmpd00:23:173
259989966,0cyclictest329-21hwrng21:23:373
259989966,0cyclictest329-21hwrng21:04:103
259989966,0cyclictest329-21hwrng20:30:223
259989966,0cyclictest329-21hwrng20:28:183
259989965,1cyclictest702-21snmpd20:39:023
259989965,1cyclictest702-21snmpd00:39:203
259989965,1cyclictest4213-21gltestperf19:35:153
259989965,1cyclictest32425-21cat22:00:153
259989964,2cyclictest702-21snmpd20:43:143
259899966,0cyclictest702-21snmpd20:21:491
259899966,0cyclictest702-21snmpd00:27:471
259899966,0cyclictest329-21hwrng23:19:201
259899965,1cyclictest702-21snmpd00:38:411
259899965,1cyclictest702-21snmpd00:17:291
259899965,1cyclictest24663-21grep20:25:181
259849966,0cyclictest702-21snmpd22:43:250
259849966,0cyclictest702-21snmpd00:31:500
259849966,0cyclictest329-21hwrng19:16:390
259849965,1cyclictest702-21snmpd22:05:300
259849965,1cyclictest702-21snmpd21:01:110
259849965,1cyclictest702-21snmpd20:01:310
259849965,1cyclictest680-21ntpd21:18:120
259849965,1cyclictest680-21ntpd20:23:130
259849965,1cyclictest680-21ntpd19:58:130
259849965,0cyclictest171rcu_preempt22:45:590
259849963,1cyclictest8330-21cat19:45:170
259849962,2cyclictest171rcu_preempt21:49:310
259849962,1cyclictest14-21ksoftirqd/000:25:000
259849961,3cyclictest9556-21ntpq23:40:200
259849961,3cyclictest3027-21ntpq23:25:210
259849961,3cyclictest27197-21base6421:50:000
259849961,3cyclictest27197-21base6421:50:000
259849961,2cyclictest4977-21fschecks_time23:30:180
259849961,1cyclictest0-21swapper/022:00:160
259849960,2cyclictest6285-21grep22:15:220
259849960,1cyclictest0-21swapper/020:55:170
259989955,0cyclictest702-21snmpd23:54:103
259989955,0cyclictest702-21snmpd20:09:193
259989955,0cyclictest702-21snmpd00:08:493
259989955,0cyclictest329-21hwrng00:27:563
259989954,1cyclictest702-21snmpd22:05:093
259989954,1cyclictest25536-21cat21:45:193
259989950,5cyclictest4028-21turbostat23:34:553
259899955,0cyclictest702-21snmpd20:03:551
259899955,0cyclictest702-21snmpd00:33:031
259899954,1cyclictest702-21snmpd21:06:391
259849955,0cyclictest702-21snmpd23:35:390
259849954,1cyclictest702-21snmpd23:58:250
259849954,1cyclictest702-21snmpd22:58:290
259849954,0cyclictest171rcu_preempt23:52:550
259849954,0cyclictest171rcu_preempt23:05:110
259849954,0cyclictest171rcu_preempt21:05:140
259849952,2cyclictest0-21swapper/019:42:330
259849951,2cyclictest0-21swapper/022:10:140
259849951,1cyclictest7639-21/usr/sbin/munin22:20:220
259849951,1cyclictest28811-21fschecks_count23:10:130
259849951,1cyclictest14-21ksoftirqd/019:50:160
259849951,1cyclictest0-21swapper/022:50:130
259849950,2cyclictest21884-21grep21:35:160
259849950,2cyclictest0-21swapper/022:25:240
259989944,0cyclictest702-21snmpd20:47:443
259989944,0cyclictest702-21snmpd19:58:343
259989944,0cyclictest329-21hwrng23:25:283
259989944,0cyclictest329-21hwrng21:53:193
259989944,0cyclictest329-21hwrng21:53:193
259989943,1cyclictest702-21snmpd23:55:103
259989943,1cyclictest702-21snmpd19:48:183
259989942,2cyclictest702-21snmpd21:15:483
259899944,0cyclictest329-21hwrng23:32:381
259899944,0cyclictest329-21hwrng22:27:061
259899944,0cyclictest329-21hwrng22:20:581
259899944,0cyclictest329-21hwrng22:00:291
259899944,0cyclictest329-21hwrng20:11:561
259899943,1cyclictest271ktimers/122:30:151
259899941,1cyclictest171rcu_preempt19:15:211
259849943,1cyclictest151ktimers/022:35:140
259849941,2cyclictest0-21swapper/020:05:120
259989932,1cyclictest8291-21cat22:20:183
259989932,1cyclictest702-21snmpd22:48:043
259989932,1cyclictest702-21snmpd20:50:203
259989932,1cyclictest702-21snmpd20:11:333
259989932,1cyclictest702-21snmpd20:00:043
259989932,1cyclictest702-21snmpd19:50:003
259989932,1cyclictest702-21snmpd19:25:483
259989932,1cyclictest16516-21ntpq22:40:233
259989932,1cyclictest12134-21cat22:30:163
259989931,2cyclictest702-21snmpd23:47:153
259989931,2cyclictest702-21snmpd20:56:003
259989930,3cyclictest20718-21ntpq20:15:173
259899932,1cyclictest0-21swapper/120:30:141
259989920,2cyclictest8737-21chrt23:40:093
259989920,2cyclictest26193-21apt-config23:05:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional