You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-28 - 01:52
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack1slot1.osadl.org (updated Sun Apr 28, 2024 00:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
274462201162,25sleep20-21swapper/219:09:202
274302200163,25sleep10-21swapper/119:09:061
271672199160,25sleep30-21swapper/319:05:423
272262198158,26sleep00-21swapper/019:06:280
312982610,0sleep30-21swapper/322:12:313
227932590,0sleep00-21swapper/020:10:290
27593991717,0cyclictest28-21ksoftirqd/222:20:212
27593991616,0cyclictest28-21ksoftirqd/223:27:332
27593991616,0cyclictest28-21ksoftirqd/222:33:172
27593991616,0cyclictest28-21ksoftirqd/221:26:322
27593991616,0cyclictest28-21ksoftirqd/221:19:412
27593991616,0cyclictest28-21ksoftirqd/200:13:382
27593991615,1cyclictest28-21ksoftirqd/223:49:182
19302160,0sleep10-21swapper/120:35:201
27593991515,0cyclictest28-21ksoftirqd/223:36:552
27593991515,0cyclictest28-21ksoftirqd/222:17:082
27593991515,0cyclictest28-21ksoftirqd/200:36:102
27593991515,0cyclictest28-21ksoftirqd/200:27:222
27593991515,0cyclictest28-21ksoftirqd/200:23:442
27593991514,1cyclictest28-21ksoftirqd/222:07:342
2759399149,5cyclictest28-21ksoftirqd/221:56:302
2759399148,5cyclictest28-21ksoftirqd/220:55:242
27593991414,0cyclictest28-21ksoftirqd/223:58:122
27593991414,0cyclictest28-21ksoftirqd/223:16:002
27593991414,0cyclictest28-21ksoftirqd/223:04:082
27593991414,0cyclictest28-21ksoftirqd/222:28:302
27593991414,0cyclictest28-21ksoftirqd/221:54:582
27593991414,0cyclictest28-21ksoftirqd/200:09:462
27593991413,1cyclictest28-21ksoftirqd/221:48:402
27593991413,0cyclictest28-21ksoftirqd/223:13:222
2759399140,2cyclictest8849-21snmpd23:05:112
27593991313,0cyclictest28-21ksoftirqd/223:30:372
27593991313,0cyclictest28-21ksoftirqd/222:59:012
27593991313,0cyclictest28-21ksoftirqd/222:52:412
27593991313,0cyclictest28-21ksoftirqd/222:14:422
27593991313,0cyclictest28-21ksoftirqd/222:04:262
27593991313,0cyclictest28-21ksoftirqd/221:20:562
27593991313,0cyclictest28-21ksoftirqd/200:17:322
27593991313,0cyclictest28-21ksoftirqd/200:03:392
27593991312,1cyclictest28-21ksoftirqd/221:10:302
2759399130,12cyclictest17395-21diskmemload21:30:102
2758599130,1cyclictest0-21swapper/121:30:111
2757999132,5cyclictest499-21munin-run19:19:590
27579991311,1cyclictest9-21ksoftirqd/021:19:590
27593991212,0cyclictest28-21ksoftirqd/223:20:582
27593991212,0cyclictest28-21ksoftirqd/222:41:172
27593991212,0cyclictest28-21ksoftirqd/221:38:582
27593991212,0cyclictest28-21ksoftirqd/200:30:532
27593991211,1cyclictest28-21ksoftirqd/220:17:212
27593991211,1cyclictest28-21ksoftirqd/220:00:162
27593991210,1cyclictest28-21ksoftirqd/223:45:012
27593991210,1cyclictest28-21ksoftirqd/223:45:002
27593991210,1cyclictest28-21ksoftirqd/222:47:482
27593991210,1cyclictest28-21ksoftirqd/222:35:232
27593991210,1cyclictest28-21ksoftirqd/220:40:002
27593991210,1cyclictest28-21ksoftirqd/219:45:012
2757999121,5cyclictest18723-21perf20:05:010
2760099119,1cyclictest35-21ksoftirqd/319:24:593
2759399119,1cyclictest28-21ksoftirqd/221:10:002
2759399119,1cyclictest28-21ksoftirqd/220:10:142
2759399119,1cyclictest28-21ksoftirqd/219:55:082
2759399119,1cyclictest28-21ksoftirqd/219:55:082
2759399119,1cyclictest28-21ksoftirqd/219:30:122
2759399119,1cyclictest28-21ksoftirqd/219:20:222
2759399119,1cyclictest28-21ksoftirqd/219:15:002
2759399118,2cyclictest28-21ksoftirqd/219:55:012
27593991111,0cyclictest271rcuc/220:30:092
27593991110,1cyclictest28-21ksoftirqd/221:40:222
27593991110,1cyclictest28-21ksoftirqd/220:50:142
27593991110,1cyclictest28-21ksoftirqd/220:30:002
27593991110,1cyclictest28-21ksoftirqd/220:10:002
27593991110,1cyclictest28-21ksoftirqd/219:45:072
27593991110,1cyclictest28-21ksoftirqd/219:40:002
27593991110,0cyclictest28-21ksoftirqd/223:50:012
2757999112,8cyclictest31768-21sh21:29:220
2757999110,0cyclictest0-21swapper/023:05:100
2760099100,8cyclictest27226-21sh22:06:093
2760099100,4cyclictest3346-21apt-get19:25:103
2759399109,1cyclictest28-21ksoftirqd/221:00:102
2759399109,1cyclictest28-21ksoftirqd/220:45:132
2759399109,1cyclictest28-21ksoftirqd/220:45:002
2759399109,1cyclictest28-21ksoftirqd/220:20:012
2759399109,1cyclictest28-21ksoftirqd/219:25:102
2759399109,1cyclictest28-21ksoftirqd/219:10:072
2757999109,1cyclictest9-21ksoftirqd/022:25:010
2757999109,1cyclictest9-21ksoftirqd/019:40:000
2757999108,1cyclictest9-21ksoftirqd/021:30:240
2757999107,1cyclictest530-21systemd-logind19:48:020
2757999102,4cyclictest32304-21perf23:00:010
27579991010,0cyclictest9-21ksoftirqd/000:25:010
2757999101,8cyclictest17395-21diskmemload23:45:190
2757999100,5cyclictest5746-21apt-get19:30:130
276009990,1cyclictest0-21swapper/321:25:113
275799998,1cyclictest9-21ksoftirqd/023:55:140
275799998,1cyclictest9-21ksoftirqd/000:17:470
275799997,1cyclictest9-21ksoftirqd/021:55:010
275799997,1cyclictest9-21ksoftirqd/000:35:160
275799996,2cyclictest548-21in:imuxsock00:15:000
275799990,5cyclictest17395-21diskmemload22:50:170
24187290,3sleep30-21swapper/321:19:013
276009987,1cyclictest17395-21diskmemload00:31:033
276009987,1cyclictest15750-21sensors21:50:213
276009986,1cyclictest8849-21snmpd22:44:043
275859986,1cyclictest0-21swapper/122:15:101
275859981,3cyclictest0-21swapper/122:50:071
275859981,1cyclictest0-21swapper/119:12:011
275859980,5cyclictest0-21swapper/100:25:121
275859980,4cyclictest0-21swapper/122:10:421
275859980,4cyclictest0-21swapper/122:07:571
275799988,0cyclictest9-21ksoftirqd/021:55:130
275799987,1cyclictest9-21ksoftirqd/022:45:160
275799987,1cyclictest9-21ksoftirqd/022:30:260
275799987,1cyclictest9-21ksoftirqd/019:25:010
275799987,1cyclictest0-21swapper/021:39:020
275799986,1cyclictest9-21ksoftirqd/023:35:010
275799986,1cyclictest9-21ksoftirqd/020:24:590
275799986,1cyclictest9-21ksoftirqd/020:19:590
275799986,1cyclictest19322-21ssh22:40:240
275799984,1cyclictest17395-21diskmemload00:28:510
275799980,7cyclictest3142-21ssh22:19:320
275799980,7cyclictest27706-21ssh23:35:310
275799980,7cyclictest17457-21ssh22:39:000
275799980,1cyclictest25315-21grep21:20:100
13965280,1sleep30-21swapper/300:02:143
276009976,1cyclictest8849-21snmpd23:54:163
276009976,1cyclictest8849-21snmpd23:33:203
276009976,1cyclictest8849-21snmpd20:51:383
276009976,1cyclictest8849-21snmpd19:46:233
276009976,1cyclictest35-21ksoftirqd/320:45:003
276009976,1cyclictest29155-1kworker/u9:200:28:433
276009975,1cyclictest8849-21snmpd19:18:493
276009975,1cyclictest35-21ksoftirqd/321:00:083
276009974,0cyclictest17395-21diskmemload22:51:593
276009974,0cyclictest17395-21diskmemload00:13:173
276009971,4cyclictest17395-21diskmemload22:36:463
276009971,3cyclictest0-21swapper/300:15:123
276009970,7cyclictest0-21swapper/323:10:563
276009970,4cyclictest0-21swapper/323:55:243
276009970,4cyclictest0-21swapper/320:55:293
276009970,3cyclictest0-21swapper/300:35:103
276009970,2cyclictest0-21swapper/323:48:113
276009970,2cyclictest0-21swapper/323:25:293
276009970,2cyclictest0-21swapper/323:00:213
276009970,2cyclictest0-21swapper/322:26:123
275859971,6cyclictest0-21swapper/119:18:411
275859970,6cyclictest0-21swapper/121:50:111
275859970,4cyclictest0-21swapper/122:56:121
275859970,4cyclictest0-21swapper/121:35:401
275859970,4cyclictest0-21swapper/119:55:081
275859970,4cyclictest0-21swapper/119:55:071
275859970,3cyclictest0-21swapper/123:31:531
275859970,3cyclictest0-21swapper/100:11:191
275859970,1cyclictest0-21swapper/122:39:281
275799977,0cyclictest2670-21sensors00:30:240
275799976,1cyclictest9-21ksoftirqd/019:55:320
275799976,1cyclictest9-21ksoftirqd/019:55:310
275799976,1cyclictest8849-21snmpd21:01:170
275799976,1cyclictest8849-21snmpd20:47:080
275799976,1cyclictest8849-21snmpd20:33:400
275799976,1cyclictest8849-21snmpd19:42:420
275799975,1cyclictest8849-21snmpd23:20:420
275799975,1cyclictest30291-21gltestperf23:40:200
275799975,1cyclictest30291-21gltestperf23:40:200
275799973,3cyclictest17395-21diskmemload22:00:110
275799972,4cyclictest17395-21diskmemload23:00:150
275799970,6cyclictest8538-21awk23:10:210
275799970,6cyclictest6410-21ssh23:51:200
275799970,6cyclictest25130-21ssh22:05:030
275799970,6cyclictest17683-21apt-get21:10:140
275799970,6cyclictest12868-21ssh23:15:390
275799970,6cyclictest116682sleep000:00:130
275799970,6cyclictest11303-21grep21:45:140
275799970,1cyclictest29525-21irqstats20:25:170
275799970,1cyclictest20494-21ssh23:26:120
275799970,1cyclictest0-21swapper/022:26:190
275799970,1cyclictest0-21swapper/022:10:140
275799970,1cyclictest0-21swapper/021:40:120
276009965,1cyclictest9107-21ssh21:41:043
276009965,1cyclictest8849-21snmpd22:48:533
276009965,1cyclictest8849-21snmpd22:24:423
276009965,1cyclictest8849-21snmpd20:17:073
276009965,1cyclictest8849-21snmpd20:04:583
276009965,1cyclictest8849-21snmpd00:24:223
276009965,1cyclictest19824-21ssh21:55:423
276009965,1cyclictest1801-21ssh22:15:273
276009964,2cyclictest17395-21diskmemload23:06:453
276009964,1cyclictest17395-21diskmemload22:34:183
276009964,1cyclictest12735-21head23:15:213
276009963,1cyclictest23543-21ssh22:00:593
276009961,4cyclictest8849-21snmpd21:21:083
276009961,0cyclictest0-21swapper/321:46:363
276009960,6cyclictest17395-21diskmemload00:05:233
276009960,5cyclictest30958-21systemctl23:40:263
276009960,5cyclictest30958-21systemctl23:40:253
276009960,3cyclictest5630-21apt-get20:45:093
276009960,3cyclictest0-21swapper/323:37:453
276009960,3cyclictest0-21swapper/321:38:323
276009960,2cyclictest0-21swapper/321:13:543
276009960,1cyclictest0-21swapper/323:22:083
275859966,0cyclictest0-21swapper/123:45:221
275859961,5cyclictest0-21swapper/121:48:021
275859960,6cyclictest0-21swapper/123:53:301
275859960,6cyclictest0-21swapper/123:35:271
275859960,6cyclictest0-21swapper/123:15:371
275859960,6cyclictest0-21swapper/122:24:211
275859960,6cyclictest0-21swapper/121:10:131
275859960,6cyclictest0-21swapper/120:10:211
275859960,6cyclictest0-21swapper/119:27:391
275859960,5cyclictest0-21swapper/122:40:521
275859960,5cyclictest0-21swapper/122:33:441
275859960,5cyclictest0-21swapper/120:26:461
275859960,4cyclictest0-21swapper/123:08:201
275859960,4cyclictest0-21swapper/123:00:241
275859960,4cyclictest0-21swapper/100:19:081
275859960,4cyclictest0-21swapper/100:05:021
275859960,3cyclictest0-21swapper/123:26:291
275859960,3cyclictest0-21swapper/121:20:551
275859960,2cyclictest0-21swapper/122:00:401
275859960,2cyclictest0-21swapper/121:05:121
275859960,1cyclictest0-21swapper/123:10:181
275859960,1cyclictest0-21swapper/120:30:031
275859960,0cyclictest0-21swapper/123:43:571
275859960,0cyclictest0-21swapper/123:43:571
275859960,0cyclictest0-21swapper/120:50:131
275859960,0cyclictest0-21swapper/100:21:271
275859960,0cyclictest0-21swapper/100:02:091
275799966,0cyclictest9-21ksoftirqd/000:05:140
275799965,1cyclictest9-21ksoftirqd/021:07:470
275799965,1cyclictest9-21ksoftirqd/021:00:000
275799965,1cyclictest8849-21snmpd20:52:160
275799965,1cyclictest8849-21snmpd20:40:100
275799964,1cyclictest9-21ksoftirqd/020:10:000
275799963,1cyclictest14250irq/124-ahci[0020:40:010
275799960,6cyclictest0-21swapper/019:52:540
275799960,1cyclictest0-21swapper/019:10:060
275799960,0cyclictest0-21swapper/019:25:190
276009955,0cyclictest0-21swapper/322:58:063
276009954,1cyclictest8849-21snmpd21:30:063
276009954,1cyclictest27394-21ntpq20:20:223
276009951,1cyclictest9783-21/usr/sbin/munin19:35:243
276009950,2cyclictest0-21swapper/320:35:243
276009950,2cyclictest0-21swapper/319:30:133
275859955,0cyclictest0-21swapper/121:57:081
275859955,0cyclictest0-21swapper/120:56:031
275859951,3cyclictest0-21swapper/120:20:151
275859951,2cyclictest0-21swapper/119:20:111
275859951,1cyclictest0-21swapper/121:15:111
275859950,5cyclictest0-21swapper/120:49:571
275859950,5cyclictest0-21swapper/120:09:551
275859950,5cyclictest0-21swapper/100:31:381
275859950,4cyclictest0-21swapper/123:55:111
275859950,4cyclictest0-21swapper/121:40:021
275859950,4cyclictest0-21swapper/120:16:571
275859950,3cyclictest0-21swapper/119:40:011
275859950,2cyclictest0-21swapper/123:20:201
275859950,2cyclictest0-21swapper/122:45:161
275859950,2cyclictest0-21swapper/122:26:021
275859950,2cyclictest0-21swapper/100:36:171
275859950,1cyclictest0-21swapper/121:00:201
275859950,1cyclictest0-21swapper/119:45:081
275859950,0cyclictest0-21swapper/121:25:021
276009944,0cyclictest8849-21snmpd19:54:483
276009943,1cyclictest8849-21snmpd19:55:583
276009943,1cyclictest8849-21snmpd19:55:573
276009943,1cyclictest29184-21sensors19:10:173
276009943,1cyclictest15416-21cut21:05:133
276009941,2cyclictest21289-21apt-get20:10:133
276009941,2cyclictest19710-21/usr/sbin/munin20:05:143
276009941,2cyclictest0-21swapper/320:30:083
276009941,1cyclictest259-21systemd-journal19:44:153
276009940,1cyclictest0-21swapper/320:25:123
275859944,0cyclictest0-21swapper/119:52:071
275859943,1cyclictest0-21swapper/120:40:151
275859941,2cyclictest0-21swapper/120:00:121
275859940,0cyclictest0-21swapper/119:30:231
275859931,1cyclictest0-21swapper/119:40:431
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional