You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-15 - 12:59
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack8slot7.osadl.org (updated Mon Sep 15, 2025 00:43:43)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1278999270,26cyclictest329-21hwrng00:32:541
12793991918,1cyclictest60250irq/125-lan19:35:002
12793991818,0cyclictest60250irq/125-lan19:20:412
12793991817,1cyclictest60250irq/125-lan21:00:192
12793991817,1cyclictest60250irq/125-lan20:15:192
12793991717,0cyclictest60250irq/125-lan22:55:162
12793991717,0cyclictest60250irq/125-lan22:10:162
12793991716,1cyclictest60250irq/125-lan23:50:132
12793991716,1cyclictest60250irq/125-lan23:00:002
12793991716,1cyclictest60250irq/125-lan21:05:132
12793991716,1cyclictest60250irq/125-lan00:35:232
12793991716,1cyclictest60250irq/125-lan00:25:182
12793991716,1cyclictest60250irq/125-lan00:23:072
12793991716,1cyclictest60250irq/125-lan00:15:192
12793991716,1cyclictest60250irq/125-lan00:15:192
12793991616,0cyclictest60250irq/125-lan23:59:042
12793991616,0cyclictest60250irq/125-lan23:15:232
12793991616,0cyclictest60250irq/125-lan21:17:172
12793991616,0cyclictest60250irq/125-lan20:54:172
12793991615,1cyclictest60250irq/125-lan22:25:212
12793991615,1cyclictest60250irq/125-lan22:20:172
12793991615,1cyclictest60250irq/125-lan22:15:192
12793991615,1cyclictest60250irq/125-lan21:30:182
12793991615,1cyclictest60250irq/125-lan19:15:172
12793991615,1cyclictest60250irq/125-lan00:05:212
12793991515,0cyclictest60250irq/125-lan23:47:322
12793991515,0cyclictest60250irq/125-lan23:20:232
12793991515,0cyclictest60250irq/125-lan20:40:552
12793991515,0cyclictest60250irq/125-lan20:39:172
12793991515,0cyclictest60250irq/125-lan20:20:122
12793991515,0cyclictest60250irq/125-lan19:25:012
12793991514,1cyclictest60250irq/125-lan23:40:122
12793991514,1cyclictest60250irq/125-lan23:10:182
12793991514,1cyclictest60250irq/125-lan23:05:162
12793991514,1cyclictest60250irq/125-lan22:40:222
12793991514,1cyclictest60250irq/125-lan21:49:592
12793991514,1cyclictest60250irq/125-lan20:45:132
12793991514,1cyclictest60250irq/125-lan20:10:202
12793991514,1cyclictest60250irq/125-lan19:50:172
12793991514,1cyclictest60250irq/125-lan19:10:222
12793991414,0cyclictest60250irq/125-lan21:10:212
12793991414,0cyclictest60250irq/125-lan20:55:142
12793991414,0cyclictest60250irq/125-lan19:45:182
12793991413,1cyclictest60250irq/125-lan23:34:122
12793991413,1cyclictest60250irq/125-lan22:52:312
12793991413,1cyclictest60250irq/125-lan22:35:122
12793991413,1cyclictest60250irq/125-lan20:25:152
12793991413,1cyclictest60250irq/125-lan00:10:182
12793991313,0cyclictest60250irq/125-lan23:25:162
12793991313,0cyclictest60250irq/125-lan21:56:092
12793991313,0cyclictest60250irq/125-lan00:32:152
12793991313,0cyclictest329-21hwrng21:36:472
12793991312,1cyclictest60250irq/125-lan20:00:182
12793991312,1cyclictest60250irq/125-lan19:30:012
12793991212,0cyclictest60250irq/125-lan22:09:162
12793991212,0cyclictest329-21hwrng21:41:542
12793991211,1cyclictest60250irq/125-lan23:35:192
12793991211,1cyclictest60250irq/125-lan19:55:122
12793991211,1cyclictest60250irq/125-lan19:40:132
12781991212,0cyclictest329-21hwrng23:39:390
12781991212,0cyclictest329-21hwrng19:39:010
12793991110,1cyclictest60250irq/125-lan21:20:152
12793991110,1cyclictest60250irq/125-lan20:05:132
12793991110,1cyclictest60250irq/125-lan00:00:192
12789991111,0cyclictest329-21hwrng22:45:231
12781991111,0cyclictest329-21hwrng23:53:590
12781991111,0cyclictest329-21hwrng21:31:390
12781991111,0cyclictest329-21hwrng20:04:370
1279399109,1cyclictest60250irq/125-lan21:25:162
1279399109,1cyclictest60250irq/125-lan20:30:142
12793991010,0cyclictest60250irq/125-lan22:30:252
12793991010,0cyclictest60250irq/125-lan22:03:152
127989998,1cyclictest702-21snmpd21:00:213
127989996,2cyclictest702-21snmpd23:09:283
127939998,1cyclictest60250irq/125-lan21:50:162
127819999,0cyclictest329-21hwrng20:14:520
127819990,9cyclictest19912-21turbostat23:25:000
127819990,3cyclictest16736-21munin-plugin-st00:35:010
127989988,0cyclictest702-21snmpd22:58:043
127989988,0cyclictest702-21snmpd22:24:543
127989988,0cyclictest329-21hwrng23:46:503
127989988,0cyclictest329-21hwrng21:25:313
127989988,0cyclictest329-21hwrng21:14:153
127989986,2cyclictest702-21snmpd23:16:113
127899988,0cyclictest702-21snmpd23:01:251
127899988,0cyclictest702-21snmpd22:57:251
127899988,0cyclictest702-21snmpd22:13:361
127899988,0cyclictest702-21snmpd21:52:471
127899988,0cyclictest702-21snmpd20:19:501
127899988,0cyclictest702-21snmpd00:11:031
127899988,0cyclictest329-21hwrng23:21:131
127899988,0cyclictest329-21hwrng23:18:091
127899986,2cyclictest702-21snmpd20:33:471
127899985,2cyclictest702-21snmpd22:15:151
127819988,0cyclictest702-21snmpd23:15:260
127819988,0cyclictest702-21snmpd20:06:250
127819988,0cyclictest702-21snmpd19:42:390
127819988,0cyclictest702-21snmpd00:27:330
127819988,0cyclictest702-21snmpd00:09:460
127819987,1cyclictest702-21snmpd21:11:240
127819987,1cyclictest702-21snmpd19:34:330
127819987,1cyclictest702-21snmpd00:19:160
127819987,1cyclictest702-21snmpd00:19:150
127989977,0cyclictest702-21snmpd22:40:063
127989977,0cyclictest702-21snmpd21:06:153
127989977,0cyclictest702-21snmpd20:57:573
127989977,0cyclictest702-21snmpd00:06:423
127989977,0cyclictest329-21hwrng23:50:553
127989977,0cyclictest329-21hwrng23:32:293
127989977,0cyclictest329-21hwrng23:27:223
127989977,0cyclictest329-21hwrng23:12:003
127989977,0cyclictest329-21hwrng22:47:263
127989977,0cyclictest329-21hwrng22:25:563
127989977,0cyclictest329-21hwrng21:50:053
127989977,0cyclictest329-21hwrng21:21:253
127989977,0cyclictest329-21hwrng20:45:353
127989977,0cyclictest329-21hwrng20:40:273
127989977,0cyclictest329-21hwrng20:38:253
127989977,0cyclictest329-21hwrng20:14:513
127989977,0cyclictest329-21hwrng19:51:183
127989977,0cyclictest329-21hwrng19:40:023
127989977,0cyclictest329-21hwrng19:31:513
127989977,0cyclictest329-21hwrng19:13:253
127989977,0cyclictest329-21hwrng00:23:413
127989976,1cyclictest702-21snmpd21:58:023
127989976,1cyclictest702-21snmpd21:44:503
127989976,1cyclictest702-21snmpd20:23:323
127989976,1cyclictest702-21snmpd19:45:273
127989976,1cyclictest32009-21cat19:55:123
127899977,0cyclictest702-21snmpd22:01:201
127899977,0cyclictest702-21snmpd20:40:081
127899977,0cyclictest702-21snmpd20:27:591
127899977,0cyclictest702-21snmpd19:50:281
127899977,0cyclictest702-21snmpd00:25:001
127899977,0cyclictest702-21snmpd00:09:271
127899977,0cyclictest700-21lldpd19:56:211
127899977,0cyclictest329-21hwrng21:47:011
127899977,0cyclictest329-21hwrng21:44:581
127899977,0cyclictest329-21hwrng21:38:501
127899977,0cyclictest329-21hwrng19:13:241
127899977,0cyclictest329-21hwrng00:35:581
127899976,1cyclictest702-21snmpd22:52:431
127899976,1cyclictest702-21snmpd22:42:181
127899975,1cyclictest702-21snmpd23:13:081
127819977,0cyclictest702-21snmpd22:19:090
127819977,0cyclictest702-21snmpd21:54:320
127819977,0cyclictest702-21snmpd21:00:360
127819977,0cyclictest702-21snmpd19:51:400
127819977,0cyclictest702-21snmpd19:11:350
127819977,0cyclictest702-21snmpd00:01:390
127819977,0cyclictest329-21hwrng20:21:000
127819976,1cyclictest702-21snmpd22:07:560
127819976,1cyclictest702-21snmpd20:30:280
127819976,1cyclictest151ktimers/023:05:260
127819976,1cyclictest151ktimers/022:51:260
127819976,1cyclictest151ktimers/022:41:260
127819976,1cyclictest151ktimers/022:39:260
127819976,1cyclictest151ktimers/021:59:260
127819976,1cyclictest151ktimers/021:29:260
127819976,1cyclictest151ktimers/021:21:260
127819976,1cyclictest151ktimers/019:45:260
127819976,1cyclictest151ktimers/000:10:260
127819975,2cyclictest702-21snmpd20:15:030
127989966,0cyclictest702-21snmpd22:32:033
127989966,0cyclictest702-21snmpd22:04:173
127989966,0cyclictest329-21hwrng23:55:013
127989966,0cyclictest329-21hwrng23:35:343
127989966,0cyclictest329-21hwrng21:31:393
127989966,0cyclictest329-21hwrng20:52:453
127989966,0cyclictest329-21hwrng19:23:403
127989966,0cyclictest329-21hwrng00:13:273
127989966,0cyclictest329-21hwrng00:00:083
127989965,1cyclictest702-21snmpd19:38:273
127989965,1cyclictest702-21snmpd19:26:543
127989964,2cyclictest702-21snmpd20:32:413
127899966,0cyclictest329-21hwrng23:25:191
127899965,1cyclictest702-21snmpd21:09:001
127899965,1cyclictest702-21snmpd20:24:501
127899965,1cyclictest702-21snmpd19:18:501
127899964,2cyclictest702-21snmpd20:38:411
127819966,0cyclictest702-21snmpd21:17:070
127819966,0cyclictest702-21snmpd19:16:440
127819965,1cyclictest702-21snmpd22:26:150
127819965,1cyclictest702-21snmpd22:02:050
127819965,1cyclictest702-21snmpd20:26:110
127819965,1cyclictest702-21snmpd19:20:350
127819965,1cyclictest151ktimers/023:31:260
127819965,1cyclictest151ktimers/022:57:260
127819965,1cyclictest151ktimers/019:59:260
127989955,0cyclictest702-21snmpd23:42:293
127989955,0cyclictest702-21snmpd22:50:253
127989955,0cyclictest329-21hwrng22:11:363
127989955,0cyclictest329-21hwrng21:19:223
127989955,0cyclictest329-21hwrng20:06:403
127989955,0cyclictest329-21hwrng20:00:313
127989955,0cyclictest329-21hwrng00:15:303
127989955,0cyclictest329-21hwrng00:15:303
127989954,1cyclictest702-21snmpd22:06:593
127989954,1cyclictest1915-21kworker/3:1+events_freezable_power_00:34:083
127989954,1cyclictest15037-21kworker/3:2+events_freezable_power_21:49:233
127989953,2cyclictest702-21snmpd19:16:053
127899955,0cyclictest702-21snmpd00:29:061
127899954,1cyclictest8699-21needreboot00:15:181
127899954,1cyclictest8699-21needreboot00:15:181
127899954,1cyclictest702-21snmpd20:13:431
127899954,1cyclictest702-21snmpd19:47:571
127899953,2cyclictest12534-21cat23:05:151
127899953,2cyclictest0-21swapper/120:10:001
127899953,1cyclictest28042-21munin-plugin-st22:25:001
127899952,2cyclictest32447-21df_inode23:55:131
127819955,0cyclictest702-21snmpd23:40:380
127819954,1cyclictest151ktimers/022:13:260
127819954,1cyclictest151ktimers/020:50:260
127989944,0cyclictest702-21snmpd23:04:553
127989944,0cyclictest702-21snmpd21:36:163
127989944,0cyclictest329-21hwrng22:38:133
127989943,1cyclictest8024-21gltestperf20:15:143
127899944,0cyclictest702-21snmpd21:30:041
127899944,0cyclictest702-21snmpd21:13:091
127899944,0cyclictest329-21hwrng23:42:441
127899944,0cyclictest329-21hwrng22:36:101
127899944,0cyclictest329-21hwrng22:30:011
127899944,0cyclictest329-21hwrng22:06:281
127899944,0cyclictest329-21hwrng21:59:181
127899944,0cyclictest329-21hwrng21:22:261
127899944,0cyclictest329-21hwrng21:16:171
127899944,0cyclictest329-21hwrng20:03:361
127899944,0cyclictest329-21hwrng19:44:081
127899944,0cyclictest329-21hwrng00:02:111
127899943,1cyclictest702-21snmpd23:47:211
127899943,1cyclictest702-21snmpd19:21:381
127899942,2cyclictest23422-21cat20:55:161
127819944,0cyclictest702-21snmpd20:49:230
127819944,0cyclictest702-21snmpd20:44:320
127819944,0cyclictest329-21hwrng21:09:080
127819943,1cyclictest702-21snmpd23:11:220
127819943,1cyclictest20547-21gltestperf23:25:150
127819943,1cyclictest151ktimers/022:31:260
127819943,1cyclictest151ktimers/021:43:260
127819943,1cyclictest151ktimers/021:35:260
127819943,1cyclictest151ktimers/020:57:260
127819943,1cyclictest151ktimers/019:25:260
127819943,1cyclictest10151-21cpuspeed_turbos23:00:130
127819942,2cyclictest12008-21fw_conntrack21:45:140
127819940,4cyclictest0-21swapper/023:55:260
127819940,4cyclictest0-21swapper/023:49:260
127819940,4cyclictest0-21swapper/022:45:260
127819940,4cyclictest0-21swapper/020:37:260
127989932,1cyclictest702-21snmpd22:17:033
127989932,1cyclictest17839-21ntp_states00:35:193
127989932,1cyclictest13521-21grep20:25:213
127989931,2cyclictest18173-21apt-get23:20:133
127899932,1cyclictest702-21snmpd20:51:001
127899932,1cyclictest3387-21uniq21:25:141
127899932,1cyclictest271ktimers/119:25:171
127899932,1cyclictest0-21swapper/119:30:131
127899931,2cyclictest30490-21grep23:50:141
127899931,2cyclictest25023-21apt-get21:00:131
127899931,2cyclictest24769-21cat23:35:151
127899931,2cyclictest20398-21seq20:46:421
127899930,3cyclictest22957-21ntpq23:30:161
127819932,1cyclictest702-21snmpd00:20:060
127819931,2cyclictest26271-21cat22:20:120
127899921,1cyclictest702-21snmpd22:25:031
127899920,2cyclictest23653-21tr19:35:151
127819920,2cyclictest16980-21find00:35:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional