You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-18 - 08:40
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack8slot7.osadl.org (updated Thu Sep 18, 2025 00:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24685991818,0cyclictest60250irq/125-lan21:42:252
24685991818,0cyclictest60250irq/125-lan19:25:022
24685991717,0cyclictest60250irq/125-lan22:45:192
24685991717,0cyclictest60250irq/125-lan21:25:222
24685991717,0cyclictest60250irq/125-lan00:23:552
24685991717,0cyclictest60250irq/125-lan00:18:342
24685991717,0cyclictest60250irq/125-lan00:05:232
24685991716,1cyclictest60250irq/125-lan21:15:142
24685991616,0cyclictest60250irq/125-lan22:25:142
24685991616,0cyclictest60250irq/125-lan22:10:222
24685991616,0cyclictest60250irq/125-lan21:35:142
24685991616,0cyclictest60250irq/125-lan21:20:112
24685991616,0cyclictest60250irq/125-lan21:05:422
24685991616,0cyclictest60250irq/125-lan20:41:422
24685991616,0cyclictest60250irq/125-lan20:15:132
24685991615,1cyclictest60250irq/125-lan23:20:222
24685991615,1cyclictest60250irq/125-lan22:55:142
24685991615,1cyclictest60250irq/125-lan22:35:132
24685991615,1cyclictest60250irq/125-lan19:10:222
24685991615,1cyclictest60250irq/125-lan00:25:212
24685991515,0cyclictest60250irq/125-lan23:35:142
24685991515,0cyclictest60250irq/125-lan23:30:192
24685991515,0cyclictest60250irq/125-lan23:05:162
24685991515,0cyclictest60250irq/125-lan22:40:182
24685991515,0cyclictest60250irq/125-lan21:56:432
24685991515,0cyclictest60250irq/125-lan20:25:012
24685991515,0cyclictest60250irq/125-lan19:30:152
24685991515,0cyclictest329-21hwrng21:12:272
24685991514,1cyclictest60250irq/125-lan21:45:192
24685991514,1cyclictest60250irq/125-lan19:20:182
24685991514,1cyclictest60250irq/125-lan19:15:162
24685991514,1cyclictest60250irq/125-lan00:30:172
24683991515,0cyclictest329-21hwrng21:54:261
24683991515,0cyclictest329-21hwrng21:54:261
24685991414,0cyclictest60250irq/125-lan23:25:222
24685991414,0cyclictest60250irq/125-lan22:00:152
24685991414,0cyclictest60250irq/125-lan20:50:162
24685991414,0cyclictest60250irq/125-lan20:47:002
24685991414,0cyclictest60250irq/125-lan20:35:112
24685991414,0cyclictest60250irq/125-lan20:21:102
24685991414,0cyclictest60250irq/125-lan20:00:092
24685991414,0cyclictest60250irq/125-lan19:50:222
24685991413,1cyclictest60250irq/125-lan22:20:202
24685991413,1cyclictest60250irq/125-lan22:20:002
24685991413,1cyclictest60250irq/125-lan19:45:122
24685991413,1cyclictest60250irq/125-lan00:10:242
24685991313,0cyclictest60250irq/125-lan23:55:182
24685991313,0cyclictest60250irq/125-lan22:30:222
24685991313,0cyclictest60250irq/125-lan20:55:112
24685991313,0cyclictest60250irq/125-lan19:55:152
24685991312,1cyclictest60250irq/125-lan21:30:182
24685991312,1cyclictest60250irq/125-lan20:05:132
24679991313,0cyclictest329-21hwrng20:24:200
24685991212,0cyclictest60250irq/125-lan23:18:542
24685991212,0cyclictest60250irq/125-lan21:00:132
24685991212,0cyclictest60250irq/125-lan00:03:252
24685991211,1cyclictest60250irq/125-lan23:00:142
24685991211,1cyclictest60250irq/125-lan20:10:182
24685991211,1cyclictest60250irq/125-lan19:40:122
24685991211,1cyclictest60250irq/125-lan19:40:122
24685991211,1cyclictest60250irq/125-lan00:35:122
24685991111,0cyclictest60250irq/125-lan23:10:192
24685991110,1cyclictest60250irq/125-lan23:45:172
24685991010,0cyclictest60250irq/125-lan20:30:112
24685991010,0cyclictest60250irq/125-lan19:35:152
2467999108,1cyclictest702-21snmpd19:14:440
246939998,1cyclictest702-21snmpd20:47:573
246859999,0cyclictest60250irq/125-lan22:06:162
246859998,1cyclictest60250irq/125-lan23:40:132
246859998,1cyclictest60250irq/125-lan22:50:282
246799998,1cyclictest702-21snmpd20:30:590
246939988,0cyclictest702-21snmpd21:08:423
246939988,0cyclictest702-21snmpd19:36:393
246939988,0cyclictest329-21hwrng23:22:303
246939988,0cyclictest329-21hwrng23:15:203
246939988,0cyclictest329-21hwrng22:53:503
246939988,0cyclictest329-21hwrng22:26:123
246939988,0cyclictest329-21hwrng22:24:083
246939988,0cyclictest329-21hwrng22:14:553
246939988,0cyclictest329-21hwrng21:56:293
246939988,0cyclictest329-21hwrng21:39:053
246939988,0cyclictest329-21hwrng21:30:533
246939988,0cyclictest329-21hwrng20:54:013
246939988,0cyclictest329-21hwrng20:25:213
246939988,0cyclictest329-21hwrng20:05:543
246939988,0cyclictest329-21hwrng19:55:393
246939988,0cyclictest329-21hwrng19:23:553
246939988,0cyclictest329-21hwrng19:12:393
246939988,0cyclictest329-21hwrng00:37:153
246939987,1cyclictest702-21snmpd00:04:503
246939986,2cyclictest702-21snmpd22:31:003
246839988,0cyclictest702-21snmpd23:49:561
246839988,0cyclictest702-21snmpd22:18:261
246839988,0cyclictest702-21snmpd00:31:041
246839988,0cyclictest702-21snmpd00:29:361
246839988,0cyclictest702-21snmpd00:07:051
246839988,0cyclictest329-21hwrng22:53:501
246839988,0cyclictest329-21hwrng20:40:431
246839988,0cyclictest329-21hwrng20:37:381
246839988,0cyclictest329-21hwrng19:54:381
246839987,1cyclictest702-21snmpd23:39:341
246839987,1cyclictest702-21snmpd21:48:291
246839987,1cyclictest702-21snmpd00:18:511
246839986,2cyclictest702-21snmpd22:35:541
246839986,1cyclictest702-21snmpd22:00:121
246799988,0cyclictest702-21snmpd23:04:510
246799988,0cyclictest702-21snmpd19:57:340
246799988,0cyclictest702-21snmpd19:20:200
246799988,0cyclictest702-21snmpd19:18:380
246799988,0cyclictest702-21snmpd00:02:350
246799988,0cyclictest329-21hwrng23:15:210
246799988,0cyclictest329-21hwrng23:12:160
246799988,0cyclictest329-21hwrng22:35:240
246799988,0cyclictest329-21hwrng22:25:100
246799988,0cyclictest329-21hwrng21:33:580
246799988,0cyclictest329-21hwrng21:26:480
246799988,0cyclictest329-21hwrng20:53:000
246799988,0cyclictest329-21hwrng20:49:560
246799988,0cyclictest329-21hwrng20:39:410
246799987,1cyclictest702-21snmpd20:44:110
246939977,0cyclictest702-21snmpd21:45:473
246939977,0cyclictest702-21snmpd21:15:483
246939977,0cyclictest702-21snmpd20:38:233
246939977,0cyclictest329-21hwrng23:56:183
246939977,0cyclictest329-21hwrng23:40:563
246939977,0cyclictest329-21hwrng23:35:493
246939977,0cyclictest329-21hwrng23:31:433
246939977,0cyclictest329-21hwrng23:28:393
246939977,0cyclictest329-21hwrng23:12:163
246939977,0cyclictest329-21hwrng23:05:063
246939977,0cyclictest329-21hwrng23:01:003
246939977,0cyclictest329-21hwrng22:09:483
246939977,0cyclictest329-21hwrng21:40:063
246939977,0cyclictest329-21hwrng21:28:503
246939977,0cyclictest329-21hwrng21:13:293
246939977,0cyclictest329-21hwrng21:00:103
246939977,0cyclictest329-21hwrng20:55:033
246939977,0cyclictest329-21hwrng20:20:143
246939977,0cyclictest329-21hwrng20:16:083
246939977,0cyclictest329-21hwrng20:11:013
246939977,0cyclictest329-21hwrng20:04:523
246939977,0cyclictest329-21hwrng19:48:293
246939977,0cyclictest329-21hwrng19:25:583
246939977,0cyclictest329-21hwrng19:15:423
246939977,0cyclictest329-21hwrng00:30:063
246939977,0cyclictest329-21hwrng00:25:593
246939977,0cyclictest329-21hwrng00:20:523
246939977,0cyclictest329-21hwrng00:15:453
246939977,0cyclictest329-21hwrng00:10:383
246939976,1cyclictest702-21snmpd22:55:373
246939976,1cyclictest702-21snmpd21:50:233
246939976,1cyclictest702-21snmpd21:50:233
246939976,1cyclictest702-21snmpd19:31:203
246939975,2cyclictest702-21snmpd20:30:173
246859976,1cyclictest60250irq/125-lan23:50:222
246839977,0cyclictest702-21snmpd23:10:421
246839977,0cyclictest702-21snmpd22:41:061
246839977,0cyclictest702-21snmpd21:15:091
246839977,0cyclictest702-21snmpd20:01:191
246839977,0cyclictest702-21snmpd19:23:241
246839977,0cyclictest702-21snmpd19:11:231
246839977,0cyclictest329-21hwrng23:08:101
246839977,0cyclictest329-21hwrng22:45:381
246839977,0cyclictest329-21hwrng21:56:291
246839977,0cyclictest329-21hwrng21:03:141
246839977,0cyclictest329-21hwrng19:56:411
246839977,0cyclictest329-21hwrng19:41:191
246839977,0cyclictest329-21hwrng19:41:191
246839977,0cyclictest329-21hwrng19:35:111
246839976,1cyclictest702-21snmpd23:02:421
246839976,1cyclictest702-21snmpd22:59:271
246799977,0cyclictest702-21snmpd23:05:360
246799977,0cyclictest329-21hwrng23:52:120
246799977,0cyclictest329-21hwrng23:45:030
246799977,0cyclictest329-21hwrng23:40:560
246799977,0cyclictest329-21hwrng23:37:520
246799977,0cyclictest329-21hwrng22:21:040
246799977,0cyclictest329-21hwrng21:45:130
246799977,0cyclictest329-21hwrng21:44:120
246799977,0cyclictest329-21hwrng21:36:000
246799977,0cyclictest329-21hwrng21:24:450
246799977,0cyclictest329-21hwrng19:39:160
246799977,0cyclictest329-21hwrng19:25:580
246799977,0cyclictest329-21hwrng00:37:150
246799976,1cyclictest702-21snmpd22:41:030
246799976,1cyclictest702-21snmpd22:32:330
246799976,1cyclictest702-21snmpd22:03:570
246799975,2cyclictest702-21snmpd20:15:480
246939966,0cyclictest329-21hwrng21:20:393
246939966,0cyclictest329-21hwrng19:50:323
246939966,0cyclictest329-21hwrng00:05:313
246859966,0cyclictest60250irq/125-lan21:51:422
246859966,0cyclictest60250irq/125-lan21:51:422
246839966,0cyclictest329-21hwrng22:05:421
246839966,0cyclictest329-21hwrng21:43:111
246839966,0cyclictest329-21hwrng19:49:311
246839966,0cyclictest329-21hwrng19:33:081
246839965,1cyclictest702-21snmpd23:28:511
246839965,1cyclictest702-21snmpd21:06:451
246839965,1cyclictest1069-21cat20:45:141
246799966,0cyclictest702-21snmpd21:01:360
246799966,0cyclictest329-21hwrng22:16:580
246799965,1cyclictest702-21snmpd23:55:430
246799965,1cyclictest702-21snmpd20:14:220
246799965,1cyclictest702-21snmpd20:00:150
246799965,1cyclictest702-21snmpd00:05:230
246939955,0cyclictest702-21snmpd23:52:523
246939955,0cyclictest702-21snmpd22:37:393
246939955,0cyclictest702-21snmpd22:01:123
246939954,1cyclictest702-21snmpd22:17:353
246839955,0cyclictest702-21snmpd23:34:171
246839955,0cyclictest702-21snmpd00:11:201
246839955,0cyclictest329-21hwrng23:51:101
246839955,0cyclictest329-21hwrng22:30:171
246839955,0cyclictest329-21hwrng00:35:121
246839954,1cyclictest702-21snmpd20:26:321
246799955,0cyclictest702-21snmpd23:32:530
246799955,0cyclictest702-21snmpd22:11:090
246799955,0cyclictest702-21snmpd21:19:280
246799955,0cyclictest702-21snmpd20:07:190
246799955,0cyclictest702-21snmpd19:41:510
246799955,0cyclictest702-21snmpd19:41:510
246799955,0cyclictest702-21snmpd00:22:530
246799955,0cyclictest329-21hwrng22:05:420
246799955,0cyclictest329-21hwrng20:28:250
246799954,1cyclictest702-21snmpd20:55:030
246799953,0cyclictest19474-21turbostat00:15:000
246839944,0cyclictest702-21snmpd23:40:521
246839944,0cyclictest329-21hwrng23:58:201
246839944,0cyclictest329-21hwrng23:20:271
246839944,0cyclictest329-21hwrng22:10:491
246839944,0cyclictest329-21hwrng21:25:461
246839944,0cyclictest329-21hwrng21:21:401
246839944,0cyclictest329-21hwrng00:21:541
246839943,1cyclictest7521-21fw_conntrack22:25:131
246839943,1cyclictest271ktimers/120:30:171
246839943,1cyclictest271ktimers/100:00:091
246839943,1cyclictest24706-21dump-pmu-power20:25:011
246839942,2cyclictest10545-21fw_packets21:10:101
246839941,3cyclictest19396-21needreboot21:30:171
246839940,4cyclictest20220-21awk20:15:011
246799944,0cyclictest329-21hwrng22:46:400
246799944,0cyclictest329-21hwrng21:58:320
246799944,0cyclictest329-21hwrng21:51:220
246799944,0cyclictest329-21hwrng21:51:220
246799944,0cyclictest329-21hwrng19:34:090
246799942,2cyclictest702-21snmpd19:47:020
246799942,2cyclictest27626-21sed00:35:000
246799940,4cyclictest31637-21turbostat23:25:000
246939933,0cyclictest329-21hwrng22:41:343
246939932,1cyclictest6726-21gltestperf19:40:143
246939932,1cyclictest6726-21gltestperf19:40:143
246939931,2cyclictest8798-21cat23:45:223
246939931,2cyclictest15521-21cut22:45:123
246939930,3cyclictest31741-21ntpq20:40:153
246839933,0cyclictest4647-21/usr/sbin/munin20:55:221
246839933,0cyclictest329-21hwrng22:22:051
246839932,1cyclictest32388-21df_inode19:25:091
246839931,2cyclictest27468-21sed23:15:001
246839931,2cyclictest27337-21grep19:15:151
246839930,3cyclictest3922-21tail20:50:251
246839930,3cyclictest18204-21latency_hist20:10:011
246839930,2cyclictest21350-21/usr/sbin/munin20:15:181
246799932,1cyclictest9002sleep023:25:490
246799932,1cyclictest151ktimers/019:50:160
246799932,1cyclictest10333-21cut21:10:090
246799931,2cyclictest24611-21collect200:25:210
246799931,2cyclictest20760-21tr00:15:220
246799931,2cyclictest19503-21df22:55:140
246799930,3cyclictest0-21swapper/022:50:250
246839920,2cyclictest20749-21seq21:35:091
246799921,1cyclictest151ktimers/021:05:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional