You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-06-02 - 17:01

x86 Intel Xeon E3-1220 @3100 MHz, Linux 4.19.1-rt3 (Profile)

Latency plot of system in rack #2, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rack2slot0.osadl.org (updated Fri Jun 02, 2023 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21972550,0sleep30-21swapper/310:48:073
78842470,0sleep10-21swapper/110:57:311
51362470,0sleep20-21swapper/209:36:292
21842460,0sleep10-21swapper/111:28:221
221022360,0sleep20-21swapper/211:17:072
892823210,7sleep00-21swapper/007:09:450
88702308,7sleep30-21swapper/307:09:043
859522918,7sleep10-21swapper/107:05:371
715322615,7sleep20-21swapper/207:05:062
97822180,1sleep2617-21polkitd09:53:522
6452140,0sleep00-21swapper/010:31:350
9067991311,1cyclictest729-21systemd-cgroups09:59:433
906799130,11cyclictest1-21systemd12:02:343
906199130,12cyclictest24779-21sshd10:45:292
9055991312,0cyclictest31572-21kworker/1:109:54:461
906799120,11cyclictest0-21swapper/312:26:443
9055991211,0cyclictest4181-21kworker/1:210:26:071
9055991211,0cyclictest15849-21kworker/1:311:23:101
84932120,0sleep28484-21bash12:26:232
151552120,2sleep2906199cyclictest11:07:212
906799119,1cyclictest5238-21sshd12:17:343
906799119,1cyclictest5238-21sshd12:17:333
9067991110,0cyclictest0-21swapper/311:24:533
906799110,1cyclictest4132-21lspci09:36:143
906799110,1cyclictest14051-21sshd10:10:433
906799110,10cyclictest0-21swapper/311:38:333
9061991110,0cyclictest0-21swapper/209:11:492
906199110,10cyclictest0-21swapper/212:39:102
906199110,10cyclictest0-21swapper/211:22:502
9055991110,0cyclictest31572-21kworker/1:109:46:051
9055991110,0cyclictest23719-21kworker/1:009:11:251
905599110,10cyclictest0-21swapper/110:20:361
905599110,10cyclictest0-21swapper/110:02:151
905599110,10cyclictest0-21swapper/109:55:461
905199119,1cyclictest0-21swapper/010:15:580
905199119,1cyclictest0-21swapper/009:23:380
9051991110,0cyclictest0-21swapper/010:58:580
905199110,1cyclictest621-21dbus-daemon09:34:080
905199110,10cyclictest0-21swapper/011:39:480
905199110,10cyclictest0-21swapper/011:25:170
905199110,0cyclictest0-21swapper/010:42:580
906799109,0cyclictest0-21swapper/312:20:043
906799109,0cyclictest0-21swapper/309:25:443
906799109,0cyclictest0-21swapper/307:55:033
906799108,1cyclictest7077-21sshd12:10:033
906799108,1cyclictest0-21swapper/310:34:243
906799100,9cyclictest0-21swapper/312:05:043
906799100,0cyclictest0-21swapper/311:01:043
906799100,0cyclictest0-21swapper/309:00:243
906799100,0cyclictest0-21swapper/307:29:243
906199109,0cyclictest0-21swapper/211:43:302
906199109,0cyclictest0-21swapper/211:32:202
906199109,0cyclictest0-21swapper/210:35:392
906199109,0cyclictest0-21swapper/210:11:092
906199109,0cyclictest0-21swapper/209:34:392
906199109,0cyclictest0-21swapper/209:17:002
906199100,0cyclictest0-21swapper/211:53:502
906199100,0cyclictest0-21swapper/210:22:592
906199100,0cyclictest0-21swapper/209:40:202
905599109,0cyclictest0-21swapper/111:56:461
905599109,0cyclictest0-21swapper/111:47:161
905599109,0cyclictest0-21swapper/111:07:161
905599109,0cyclictest0-21swapper/110:42:161
905599100,0cyclictest0-21swapper/111:13:251
905199109,0cyclictest0-21swapper/011:51:470
905199109,0cyclictest0-21swapper/010:37:370
905199109,0cyclictest0-21swapper/010:24:080
905199109,0cyclictest0-21swapper/009:57:380
905199100,9cyclictest0-21swapper/011:05:080
905199100,0cyclictest0-21swapper/010:12:170
905199100,0cyclictest0-21swapper/010:01:170
905199100,0cyclictest0-21swapper/009:48:180
90679998,0cyclictest0-21swapper/311:48:533
90679998,0cyclictest0-21swapper/309:53:143
90679998,0cyclictest0-21swapper/309:15:443
90679998,0cyclictest0-21swapper/309:10:333
90679998,0cyclictest0-21swapper/307:39:443
90619998,0cyclictest0-21swapper/210:40:102
90619998,0cyclictest0-21swapper/210:18:302
90619998,0cyclictest0-21swapper/210:07:302
90619998,0cyclictest0-21swapper/209:57:202
90619998,0cyclictest0-21swapper/209:27:002
90619998,0cyclictest0-21swapper/208:20:192
90619997,1cyclictest28075-21lspci11:10:292
90559998,0cyclictest0-21swapper/111:36:551
90559998,0cyclictest0-21swapper/111:18:461
90559998,0cyclictest0-21swapper/111:01:551
90559998,0cyclictest0-21swapper/110:32:261
90559998,0cyclictest0-21swapper/109:43:361
90559998,0cyclictest0-21swapper/109:25:261
90559998,0cyclictest0-21swapper/107:25:151
90559990,0cyclictest0-21swapper/109:33:061
90519998,0cyclictest0-21swapper/011:11:180
90519998,0cyclictest0-21swapper/009:51:070
90519998,0cyclictest0-21swapper/009:19:380
90519998,0cyclictest0-21swapper/009:12:270
90519998,0cyclictest0-21swapper/008:25:470
90519998,0cyclictest0-21swapper/007:35:270
90519990,5cyclictest11062-21perf07:14:590
90519990,0cyclictest0-21swapper/010:28:370
11788290,6sleep311846-21sshd11:55:053
90679987,0cyclictest0-21swapper/310:54:543
90679987,0cyclictest0-21swapper/309:30:333
90679987,0cyclictest0-21swapper/309:08:343
90679987,0cyclictest0-21swapper/308:59:333
90679987,0cyclictest0-21swapper/308:22:433
90679987,0cyclictest0-21swapper/308:16:443
90679984,3cyclictest617-21polkitd11:53:353
90619987,0cyclictest0-21swapper/210:26:402
90619987,0cyclictest0-21swapper/210:03:492
90619985,2cyclictest606-21avahi-daemon12:13:482
90619985,2cyclictest606-21avahi-daemon12:04:592
90619984,3cyclictest606-21avahi-daemon12:17:252
90619984,3cyclictest606-21avahi-daemon12:17:252
90559987,0cyclictest0-21swapper/110:12:251
90559987,0cyclictest0-21swapper/110:08:461
90559987,0cyclictest0-21swapper/107:50:351
90519987,0cyclictest0-21swapper/011:24:280
90519987,0cyclictest0-21swapper/009:28:270
90519985,2cyclictest606-21avahi-daemon12:11:340
90519981,1cyclictest9-21ksoftirqd/009:45:000
90679976,0cyclictest0-21swapper/310:29:543
90679976,0cyclictest0-21swapper/310:21:433
90679976,0cyclictest0-21swapper/308:50:043
90679975,1cyclictest606-21avahi-daemon11:15:363
90679975,1cyclictest606-21avahi-daemon10:15:213
90679975,1cyclictest606-21avahi-daemon09:20:403
90679974,2cyclictest621-21dbus-daemon11:12:273
90679974,2cyclictest621-21dbus-daemon10:43:233
90679974,2cyclictest606-21avahi-daemon10:09:233
90679974,2cyclictest14562-21sshd12:36:003
90619975,1cyclictest606-21avahi-daemon12:32:312
90619975,1cyclictest606-21avahi-daemon11:38:572
90619975,1cyclictest606-21avahi-daemon11:25:152
90619975,1cyclictest606-21avahi-daemon10:30:162
90619974,2cyclictest606-21avahi-daemon12:05:122
90619974,2cyclictest606-21avahi-daemon10:52:452
90619973,1cyclictest628-21gdbus09:23:112
90619973,1cyclictest0-21swapper/211:46:282
90619973,1cyclictest0-21swapper/210:55:312
90559976,0cyclictest5881-21kworker/1:212:20:111
90559976,0cyclictest10900-21sensors11:30:241
90559976,0cyclictest0-21swapper/110:46:451
90559975,1cyclictest606-21avahi-daemon12:38:071
90559975,1cyclictest606-21avahi-daemon10:51:171
90559975,1cyclictest25-21ksoftirqd/111:51:491
90559974,2cyclictest13139-21sshd09:38:381
90559973,3cyclictest28789-21sshd12:07:151
90519976,0cyclictest0-21swapper/011:17:280
90519976,0cyclictest0-21swapper/010:52:470
90519976,0cyclictest0-21swapper/008:15:280
90519975,1cyclictest969-21snmpd08:52:370
90519975,1cyclictest19002-21kworker/0:212:06:020
90519974,2cyclictest619-21systemd-logind11:01:060
90519974,2cyclictest617-21polkitd10:48:380
90519974,2cyclictest606-21avahi-daemon11:30:430
90519974,2cyclictest606-21avahi-daemon10:09:040
90519970,1cyclictest4403-21ntpq12:25:250
90519970,1cyclictest28367-21sshd11:59:120
90519970,1cyclictest26723-21systemd-cgroups12:22:570
90679965,0cyclictest0-21swapper/308:25:443
90679964,1cyclictest628-21gdbus12:30:173
90679964,1cyclictest628-21gdbus11:30:043
90679964,1cyclictest621-21dbus-daemon10:35:023
90679964,1cyclictest621-21dbus-daemon08:40:013
90679964,1cyclictest619-21systemd-logind11:25:303
90679964,1cyclictest617-21polkitd11:05:273
90679964,1cyclictest617-21polkitd10:55:393
90679964,1cyclictest617-21polkitd09:40:113
90679964,1cyclictest606-21avahi-daemon11:40:053
90679963,2cyclictest628-21gdbus09:45:313
90679963,2cyclictest619-21systemd-logind10:00:453
90679962,3cyclictest15683-21awk07:24:593
90679960,5cyclictest0-21swapper/308:31:133
90679960,5cyclictest0-21swapper/307:47:233
90619964,1cyclictest628-21gdbus09:45:072
90619964,1cyclictest617-21polkitd11:55:022
90619964,1cyclictest606-21avahi-daemon12:20:262
90619963,2cyclictest969-21snmpd08:58:332
90619963,2cyclictest969-21snmpd08:28:092
90619963,2cyclictest11495-21gdbus11:00:372
90559964,1cyclictest621-21dbus-daemon12:25:181
90559964,1cyclictest621-21dbus-daemon12:10:001
90559964,1cyclictest621-21dbus-daemon09:20:121
90559964,1cyclictest621-21dbus-daemon09:16:011
90559964,1cyclictest621-21dbus-daemon08:44:561
90559964,1cyclictest617-21polkitd12:15:471
90559964,1cyclictest617-21polkitd12:15:471
90559964,1cyclictest617-21polkitd12:00:101
90559964,1cyclictest606-21avahi-daemon10:35:251
90559964,1cyclictest606-21avahi-daemon10:15:291
90559964,1cyclictest26533-21sshd12:30:521
90559964,1cyclictest16613-21sshd11:40:071
90559963,2cyclictest969-21snmpd07:42:441
90559963,2cyclictest606-21avahi-daemon08:15:001
90559963,1cyclictest1648-21sadc08:10:011
90559960,5cyclictest0-21swapper/108:46:441
90559960,5cyclictest0-21swapper/108:32:451
90559960,5cyclictest0-21swapper/107:11:341
90519964,1cyclictest9-21ksoftirqd/009:10:000
90519964,1cyclictest628-21gdbus08:00:010
90519964,1cyclictest621-21dbus-daemon11:41:080
90519964,1cyclictest621-21dbus-daemon07:19:590
90519964,1cyclictest617-21polkitd12:35:410
90519964,1cyclictest617-21polkitd12:15:070
90519964,1cyclictest617-21polkitd12:15:070
90519964,1cyclictest617-21polkitd12:00:210
90519964,1cyclictest617-21polkitd11:45:040
90519964,1cyclictest617-21polkitd09:35:100
90519964,1cyclictest606-21avahi-daemon12:30:180
90679953,1cyclictest969-21snmpd08:42:043
90679953,1cyclictest969-21snmpd08:00:483
90679953,1cyclictest969-21snmpd07:40:223
90679953,1cyclictest969-21snmpd07:32:403
90679953,1cyclictest969-21snmpd07:17:013
90679953,1cyclictest969-21snmpd07:13:203
90679953,1cyclictest2390-21cat08:10:213
90679952,2cyclictest969-21snmpd08:46:273
90679952,2cyclictest969-21snmpd07:50:153
90619953,1cyclictest969-21snmpd09:05:262
90619953,1cyclictest969-21snmpd09:01:032
90619953,1cyclictest969-21snmpd08:45:332
90619953,1cyclictest969-21snmpd08:39:002
90619953,1cyclictest969-21snmpd08:32:452
90619953,1cyclictest969-21snmpd08:06:122
90619953,1cyclictest969-21snmpd08:01:322
90619953,1cyclictest969-21snmpd07:56:052
90619953,1cyclictest969-21snmpd07:51:332
90619953,1cyclictest969-21snmpd07:47:072
90619953,1cyclictest969-21snmpd07:35:162
90619953,1cyclictest969-21snmpd07:25:112
90619953,1cyclictest969-21snmpd07:22:282
90619953,1cyclictest969-21snmpd07:14:502
90619953,1cyclictest5282-21grep08:15:272
90619952,2cyclictest969-21snmpd07:40:142
90619952,1cyclictest0-21swapper/208:41:312
90619950,4cyclictest1897-21/usr/sbin/munin08:10:192
90619950,1cyclictest17942-21/usr/sbin/munin07:30:122
90619950,1cyclictest11838-21/usr/sbin/munin07:15:302
90619950,1cyclictest0-21swapper/208:50:242
90559953,1cyclictest969-21snmpd09:01:241
90559953,1cyclictest969-21snmpd08:59:311
90559953,1cyclictest969-21snmpd08:26:151
90559953,1cyclictest969-21snmpd07:55:111
90559953,1cyclictest969-21snmpd07:45:071
90559953,1cyclictest969-21snmpd07:35:041
90559953,1cyclictest969-21snmpd07:34:191
90559953,1cyclictest969-21snmpd07:15:171
90559953,1cyclictest606-21avahi-daemon08:00:441
90559953,1cyclictest3926-21/usr/sbin/munin08:15:261
90559953,1cyclictest26090-21munin-run09:10:001
90559952,2cyclictest14296-21basename07:20:201
90559950,4cyclictest0-21swapper/108:38:541
90519953,1cyclictest986-21lldpd07:40:070
90519953,1cyclictest969-21snmpd08:40:580
90519953,1cyclictest969-21snmpd08:20:350
90519953,1cyclictest969-21snmpd08:08:000
90519953,1cyclictest969-21snmpd08:00:050
90519953,1cyclictest969-21snmpd07:53:350
90519953,1cyclictest969-21snmpd07:45:400
90519953,1cyclictest969-21snmpd07:25:170
90519953,1cyclictest606-21avahi-daemon08:37:220
90519953,1cyclictest20207-21/usr/sbin/munin08:55:190
90519953,1cyclictest16981-21ntp_kernel_pll_08:45:190
90519952,2cyclictest969-21snmpd08:30:360
90519952,2cyclictest969-21snmpd07:34:530
90519952,2cyclictest2649-21cut08:10:230
90679942,1cyclictest32231-21cpu08:05:173
90559942,1cyclictest628-21gdbus08:50:011
90559942,1cyclictest619-21systemd-logind08:20:001
90519944,0cyclictest20190-21kworker/0:109:00:040
90519942,1cyclictest0-21swapper/007:15:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional