You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-05-20 - 17:10
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack1slot1.osadl.org (updated Mon May 20, 2024 00:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304332208153,42sleep10-21swapper/119:09:511
301962200162,26sleep20-21swapper/219:06:462
301822200160,26sleep30-21swapper/319:06:353
301272198159,26sleep00-21swapper/019:05:520
3053599130,12cyclictest0-21swapper/120:00:101
30543991210,1cyclictest28-21ksoftirqd/220:05:002
30524991211,1cyclictest9-21ksoftirqd/000:27:540
30543991111,0cyclictest28-21ksoftirqd/200:05:252
30543991110,1cyclictest28-21ksoftirqd/223:45:002
3052499119,1cyclictest9-21ksoftirqd/023:05:200
30524991110,1cyclictest9-21ksoftirqd/023:20:280
30524991110,1cyclictest9-21ksoftirqd/000:10:000
3052499111,5cyclictest2329-21perf22:10:000
3054399109,1cyclictest28-21ksoftirqd/223:25:082
3054399108,1cyclictest28-21ksoftirqd/223:35:072
30543991010,0cyclictest28-21ksoftirqd/223:30:072
30543991010,0cyclictest28-21ksoftirqd/222:55:292
30543991010,0cyclictest28-21ksoftirqd/221:35:222
30543991010,0cyclictest28-21ksoftirqd/221:16:472
3054399100,7cyclictest1118-21runrttasks21:22:002
3054399100,6cyclictest61250irq/122-eno123:13:222
3054399100,6cyclictest16985-21kworker/u8:022:41:092
3052499109,1cyclictest9-21ksoftirqd/019:10:160
3052499109,1cyclictest9-21ksoftirqd/000:20:290
3052499108,1cyclictest9-21ksoftirqd/000:00:120
3052499105,5cyclictest9-21ksoftirqd/020:05:190
3052499100,9cyclictest426-21munin-node21:43:180
3052499100,5cyclictest8849-21snmpd23:58:280
3052499100,4cyclictest23796-21diskmemload00:36:200
305519995,4cyclictest23796-21diskmemload21:37:373
305519990,8cyclictest0-21swapper/322:45:123
305439999,0cyclictest28-21ksoftirqd/222:54:552
305439999,0cyclictest28-21ksoftirqd/221:40:572
305439999,0cyclictest28-21ksoftirqd/221:10:582
305439998,1cyclictest28-21ksoftirqd/223:45:112
305439998,1cyclictest28-21ksoftirqd/222:05:242
305439998,1cyclictest28-21ksoftirqd/221:55:162
305439998,1cyclictest28-21ksoftirqd/221:55:162
305439998,1cyclictest28-21ksoftirqd/221:45:162
305439998,1cyclictest28-21ksoftirqd/220:40:142
305439998,1cyclictest28-21ksoftirqd/200:26:532
305439997,1cyclictest28-21ksoftirqd/220:35:002
305439996,3cyclictest28-21ksoftirqd/221:28:562
305439992,6cyclictest19221-21/usr/sbin/munin00:00:262
305249999,0cyclictest9-21ksoftirqd/023:25:050
305249999,0cyclictest9-21ksoftirqd/021:30:470
305249998,1cyclictest9-21ksoftirqd/022:38:270
305249998,1cyclictest9-21ksoftirqd/021:20:130
305249998,1cyclictest9-21ksoftirqd/020:45:000
305249998,1cyclictest9-21ksoftirqd/019:50:160
305249998,1cyclictest9-21ksoftirqd/000:30:150
305249991,4cyclictest13247-21perf22:25:000
305249991,2cyclictest17458-21/usr/sbin/munin23:15:260
305249990,8cyclictest4223-21ntp_states22:55:280
305249990,5cyclictest16173-1kworker/u9:022:30:110
305249990,4cyclictest9080-21latency_hist20:40:000
1607290,0sleep323796-21diskmemload21:23:303
305519988,0cyclictest35-21ksoftirqd/321:33:093
305519981,4cyclictest0-21swapper/323:25:023
305519981,0cyclictest23796-21diskmemload23:54:273
305519980,4cyclictest30733-21apt-get19:10:113
305519980,4cyclictest0-21swapper/323:30:033
305519980,3cyclictest0-21swapper/322:25:103
305519980,2cyclictest0-21swapper/300:25:123
305439988,0cyclictest28-21ksoftirqd/223:58:202
305439988,0cyclictest28-21ksoftirqd/223:50:252
305439988,0cyclictest28-21ksoftirqd/223:05:212
305439988,0cyclictest28-21ksoftirqd/222:10:142
305439988,0cyclictest28-21ksoftirqd/221:33:162
305439988,0cyclictest28-21ksoftirqd/219:10:122
305439988,0cyclictest28-21ksoftirqd/200:30:312
305439988,0cyclictest28-21ksoftirqd/200:20:512
305439988,0cyclictest28-21ksoftirqd/200:15:172
305439987,1cyclictest28-21ksoftirqd/222:25:122
305439987,1cyclictest28-21ksoftirqd/221:53:012
305439987,1cyclictest28-21ksoftirqd/220:55:172
305439987,1cyclictest28-21ksoftirqd/220:40:002
305439987,1cyclictest28-21ksoftirqd/200:14:212
305439986,1cyclictest757-21runrttasks22:30:442
305439986,1cyclictest61250irq/122-eno122:20:222
305439986,1cyclictest28-21ksoftirqd/222:15:042
305439986,1cyclictest11894-21cat00:35:222
305439981,6cyclictest13171-21apt-get19:40:102
305439980,8cyclictest0-21swapper/223:00:282
305359988,0cyclictest0-21swapper/121:33:561
305249988,0cyclictest9-21ksoftirqd/022:52:250
305249988,0cyclictest9-21ksoftirqd/021:25:140
305249988,0cyclictest9-21ksoftirqd/019:55:140
305249987,1cyclictest9-21ksoftirqd/021:52:140
305249987,1cyclictest9-21ksoftirqd/020:20:140
305249986,1cyclictest11385-21cat23:50:230
305249985,1cyclictest30915-21turbostat21:20:000
305249981,2cyclictest6622-21/usr/sbin/munin23:00:120
305249980,2cyclictest3167-21/usr/sbin/munin23:40:220
305519976,1cyclictest8849-21snmpd20:51:293
305519976,1cyclictest8849-21snmpd20:45:463
305519976,1cyclictest8849-21snmpd19:54:103
305519976,1cyclictest35-21ksoftirqd/320:35:003
305519976,1cyclictest23796-21diskmemload21:15:083
305519975,1cyclictest8849-21snmpd23:41:533
305519975,1cyclictest8849-21snmpd21:05:453
305519975,1cyclictest8849-21snmpd20:20:473
305519975,1cyclictest8849-21snmpd20:19:533
305519975,1cyclictest8849-21snmpd20:09:123
305519975,1cyclictest8849-21snmpd19:48:463
305519975,1cyclictest8849-21snmpd19:38:013
305519975,1cyclictest35-21ksoftirqd/321:05:003
305519975,1cyclictest35-21ksoftirqd/320:05:003
305519975,1cyclictest32719-21ssh22:05:233
305519975,1cyclictest19923-21ssh00:01:313
305519975,1cyclictest18408-21cat23:15:213
305519974,1cyclictest23796-21diskmemload22:50:573
305519972,4cyclictest1353-21apt-get19:15:123
305519970,6cyclictest31418-21ssh00:18:043
305519970,6cyclictest23796-21diskmemload22:12:253
305519970,4cyclictest0-21swapper/323:45:393
305519970,4cyclictest0-21swapper/321:40:093
305519970,3cyclictest7678-21/usr/sbin/munin22:15:253
305519970,3cyclictest17140-21ssh22:30:163
305519970,2cyclictest10910-21fschecks_time23:05:203
305519970,1cyclictest0-21swapper/321:50:133
305439977,0cyclictest28-21ksoftirqd/223:16:142
305439976,1cyclictest61250irq/122-eno123:20:202
305439976,1cyclictest28-21ksoftirqd/222:45:182
305439976,1cyclictest28-21ksoftirqd/222:35:152
305439976,1cyclictest28-21ksoftirqd/220:45:122
305439976,0cyclictest61250irq/122-eno119:40:002
305439975,1cyclictest61250irq/122-eno122:00:112
305439975,1cyclictest61250irq/122-eno121:10:002
305359971,6cyclictest0-21swapper/121:16:391
305359971,6cyclictest0-21swapper/120:22:591
305359970,7cyclictest0-21swapper/100:05:201
305359970,1cyclictest0-21swapper/123:27:281
305359970,1cyclictest0-21swapper/119:30:361
305359970,0cyclictest0-21swapper/121:45:121
305249977,0cyclictest9-21ksoftirqd/023:35:070
305249977,0cyclictest9-21ksoftirqd/022:45:290
305249977,0cyclictest9-21ksoftirqd/022:15:150
305249977,0cyclictest9-21ksoftirqd/021:14:280
305249976,1cyclictest9-21ksoftirqd/023:30:150
305249976,1cyclictest9-21ksoftirqd/022:13:250
305249976,1cyclictest9-21ksoftirqd/020:15:150
305249976,1cyclictest9-21ksoftirqd/000:15:160
305249976,1cyclictest8849-21snmpd23:45:020
305249976,1cyclictest8849-21snmpd20:10:310
305249976,1cyclictest8849-21snmpd00:10:060
305249976,0cyclictest9-21ksoftirqd/021:10:000
305249975,1cyclictest8849-21snmpd20:49:290
305249975,1cyclictest11061-21grep21:35:210
305249975,1cyclictest0-21swapper/022:44:460
305249974,2cyclictest15504-21ssh22:26:050
30154270,0sleep00-21swapper/022:02:120
305519965,1cyclictest8849-21snmpd19:40:553
305519965,1cyclictest8849-21snmpd19:30:483
305519965,1cyclictest27763-21ssh22:00:113
305519965,1cyclictest23117-21ssh22:37:143
305519965,1cyclictest19550-21ssh21:48:003
305519965,1cyclictest1916-21ssh00:20:463
305519965,1cyclictest13544-21ssh00:37:453
305519965,0cyclictest23796-21diskmemload23:03:013
305519964,1cyclictest8849-21snmpd22:41:453
305519964,1cyclictest8849-21snmpd20:41:173
305519964,1cyclictest8849-21snmpd20:29:353
305519964,1cyclictest23796-21diskmemload21:55:153
305519964,1cyclictest23796-21diskmemload21:55:153
305519964,1cyclictest21108-21apt-get23:20:143
305519963,2cyclictest23796-21diskmemload22:20:293
305519963,1cyclictest757-21runrttasks19:23:133
305519963,1cyclictest23796-21diskmemload22:59:063
305519962,2cyclictest259-21systemd-journal20:40:003
305519962,2cyclictest23796-21diskmemload23:36:433
305519961,3cyclictest13848-21apt-get23:10:123
305519961,2cyclictest23796-21diskmemload21:12:003
305519961,1cyclictest0-21swapper/321:25:133
305519960,3cyclictest14075-21/usr/sbin/munin23:55:113
305519960,2cyclictest0-21swapper/300:05:263
305439966,0cyclictest61250irq/122-eno120:15:152
305439966,0cyclictest61250irq/122-eno119:55:352
305439966,0cyclictest61250irq/122-eno119:53:432
305439966,0cyclictest61250irq/122-eno119:33:532
305439966,0cyclictest61250irq/122-eno119:16:372
305439965,1cyclictest61250irq/122-eno120:53:092
305439965,1cyclictest61250irq/122-eno120:25:102
305439964,1cyclictest61250irq/122-eno119:45:122
305439960,6cyclictest0-21swapper/220:10:312
305359966,0cyclictest0-21swapper/122:12:051
305359966,0cyclictest0-21swapper/100:23:091
305359964,1cyclictest0-21swapper/100:25:361
305359961,5cyclictest0-21swapper/121:35:491
305359961,5cyclictest0-21swapper/120:51:081
305359961,5cyclictest0-21swapper/120:15:581
305359961,2cyclictest0-21swapper/121:41:091
305359960,6cyclictest0-21swapper/123:35:061
305359960,6cyclictest0-21swapper/122:41:281
305359960,6cyclictest0-21swapper/122:08:201
305359960,6cyclictest0-21swapper/121:50:581
305359960,6cyclictest0-21swapper/121:05:201
305359960,6cyclictest0-21swapper/100:34:251
305359960,4cyclictest0-21swapper/123:42:371
305359960,4cyclictest0-21swapper/122:45:081
305359960,4cyclictest0-21swapper/121:11:041
305359960,3cyclictest0-21swapper/123:10:351
305249965,1cyclictest9-21ksoftirqd/020:55:140
305249965,1cyclictest9-21ksoftirqd/019:45:120
305249965,1cyclictest8849-21snmpd20:03:310
305249965,1cyclictest26385-21ssh21:56:480
305249965,1cyclictest26385-21ssh21:56:470
305249965,1cyclictest19875-21ssh21:48:580
305249965,1cyclictest16087-21ssh23:11:460
305249964,1cyclictest9-21ksoftirqd/019:35:000
305249964,1cyclictest9-21ksoftirqd/019:20:140
305249963,1cyclictest259-21systemd-journal20:55:010
305249961,4cyclictest19115-21apt-get21:00:100
305249960,6cyclictest0-21swapper/020:32:050
305249960,1cyclictest13668-21df19:40:130
29787260,1sleep10-21swapper/122:01:041
305519955,0cyclictest427-21kworker/3:119:25:263
305519955,0cyclictest0-21swapper/300:30:493
305519954,1cyclictest8849-21snmpd20:14:463
305519953,1cyclictest25021-21ssh00:10:083
305519950,0cyclictest0-21swapper/319:56:123
305439955,0cyclictest61250irq/122-eno120:05:132
305439954,1cyclictest61250irq/122-eno119:25:252
305439950,2cyclictest548-21in:imuxsock20:20:002
305439950,1cyclictest0-21swapper/221:00:102
305439950,1cyclictest0-21swapper/219:20:112
305359955,0cyclictest0-21swapper/123:31:361
305359955,0cyclictest0-21swapper/120:37:061
305359955,0cyclictest0-21swapper/119:57:171
305359955,0cyclictest0-21swapper/119:11:551
305359955,0cyclictest0-21swapper/100:12:041
305359953,1cyclictest18521-21chrt20:55:471
305359952,2cyclictest0-21swapper/120:40:141
305359951,4cyclictest0-21swapper/123:15:321
305359951,2cyclictest0-21swapper/122:15:151
305359950,5cyclictest0-21swapper/120:46:141
305359950,5cyclictest0-21swapper/119:20:261
305359950,5cyclictest0-21swapper/100:01:151
305359950,4cyclictest0-21swapper/119:35:161
305359950,3cyclictest0-21swapper/123:45:011
305359950,3cyclictest0-21swapper/123:02:181
305359950,3cyclictest0-21swapper/122:51:511
305359950,3cyclictest0-21swapper/122:35:081
305359950,3cyclictest0-21swapper/122:30:431
305359950,3cyclictest0-21swapper/122:25:251
305359950,3cyclictest0-21swapper/122:23:011
305359950,2cyclictest0-21swapper/123:55:121
305359950,2cyclictest0-21swapper/123:06:211
305359950,2cyclictest0-21swapper/122:55:291
305359950,2cyclictest0-21swapper/120:05:171
305359950,2cyclictest0-21swapper/119:40:101
305359950,2cyclictest0-21swapper/100:15:471
305359950,1cyclictest0-21swapper/120:10:231
305359950,0cyclictest0-21swapper/121:25:131
305249954,1cyclictest9-21ksoftirqd/020:25:140
305249954,1cyclictest9-21ksoftirqd/019:30:000
305249951,1cyclictest0-21swapper/019:35:090
305519940,1cyclictest16743-21apt-get20:55:123
305359944,0cyclictest0-21swapper/120:31:101
305359940,4cyclictest0-21swapper/121:01:111
305359940,3cyclictest0-21swapper/123:20:151
305359940,3cyclictest0-21swapper/121:55:261
305359940,3cyclictest0-21swapper/121:55:261
305359940,3cyclictest0-21swapper/100:38:201
305359940,2cyclictest0-21swapper/121:20:161
305359940,1cyclictest0-21swapper/123:50:151
305359940,0cyclictest0-21swapper/119:50:371
305359940,0cyclictest0-21swapper/119:25:331
305359940,0cyclictest0-21swapper/119:15:001
305249940,4cyclictest20502-21kworker/0:119:15:000
305359931,1cyclictest0-21swapper/120:25:111
305359930,1cyclictest0-21swapper/119:45:101
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional