You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-16 - 16:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack1slot1.osadl.org (updated Tue Apr 16, 2024 12:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
185222223157,26sleep10-21swapper/107:05:491
167262211149,20sleep30-21swapper/307:05:013
187222202139,23sleep00-21swapper/007:08:250
167302200160,26sleep20-21swapper/207:05:052
166272200,0sleep10-21swapper/112:09:121
18938991615,1cyclictest11586-21diskmemload09:34:110
18944991210,1cyclictest28-21ksoftirqd/207:54:592
1893899129,2cyclictest16549-21ssh12:08:580
18938991211,1cyclictest11586-21diskmemload10:55:120
18938991210,2cyclictest0-21swapper/011:49:590
18938991210,1cyclictest6803-21gpgv10:30:000
18938991210,1cyclictest23305-21ssh10:07:520
1894499119,1cyclictest28-21ksoftirqd/210:39:592
1894499118,1cyclictest28-21ksoftirqd/207:14:592
18944991110,1cyclictest28-21ksoftirqd/212:30:012
18944991110,1cyclictest28-21ksoftirqd/212:05:002
1894499110,0cyclictest0-21swapper/209:45:122
1893899119,2cyclictest0-21swapper/007:35:010
1893899119,2cyclictest0-21swapper/007:35:000
1893899119,1cyclictest5855-21nscd11:30:210
1893899119,1cyclictest2139-21apt-get11:50:130
1893899119,1cyclictest12806-21ssh10:35:390
1893899119,1cyclictest0-21swapper/010:33:440
1893899118,1cyclictest9-21ksoftirqd/007:40:010
1893899112,8cyclictest4743-21apt-get11:10:130
18938991110,1cyclictest11586-21diskmemload12:25:080
18938991110,1cyclictest0-21swapper/010:15:260
18938991110,1cyclictest0-21swapper/008:20:110
1893899111,9cyclictest8849-21snmpd10:41:080
1893899111,9cyclictest7803-21ssh11:55:480
1893899111,9cyclictest16862-21ssh10:00:050
1894499109,1cyclictest28-21ksoftirqd/211:10:172
1894499109,1cyclictest28-21ksoftirqd/210:50:012
1894499109,1cyclictest28-21ksoftirqd/208:15:002
1894499109,1cyclictest28-21ksoftirqd/207:25:002
1894499100,5cyclictest0-21swapper/208:40:132
1893899109,1cyclictest14227-21ssh09:12:030
1893899109,1cyclictest11586-21diskmemload09:20:150
1893899109,1cyclictest0-21swapper/007:50:000
1893899108,2cyclictest0-21swapper/010:45:090
1893899108,1cyclictest24636-21aten_r1power_en12:20:110
1893899108,1cyclictest16613-21cat09:15:250
1893899108,1cyclictest11586-21diskmemload11:16:030
1893899108,1cyclictest11586-21diskmemload11:08:520
1893899108,1cyclictest0-21swapper/009:36:540
1893899107,2cyclictest11586-21diskmemload09:47:330
1893899101,8cyclictest12976-21ssh12:04:540
1893899101,8cyclictest11586-21diskmemload09:52:580
189509998,1cyclictest35-21ksoftirqd/308:49:083
189449999,0cyclictest0-21swapper/212:31:412
189449998,1cyclictest28-21ksoftirqd/212:20:132
189449998,1cyclictest28-21ksoftirqd/212:05:132
189449991,5cyclictest0-21swapper/211:55:122
189449990,2cyclictest21287-21munin-node11:25:572
189399991,5cyclictest0-21swapper/112:20:111
189389998,1cyclictest11586-21diskmemload11:35:250
189389998,1cyclictest11586-21diskmemload09:56:090
189389998,1cyclictest0-21swapper/012:37:400
189389998,1cyclictest0-21swapper/011:44:560
189389998,1cyclictest0-21swapper/011:26:190
189389998,1cyclictest0-21swapper/010:50:250
189389997,2cyclictest0-21swapper/011:00:240
189389997,1cyclictest11586-21diskmemload12:15:260
189389996,3cyclictest0-21swapper/008:30:130
189389993,5cyclictest431-21/usr/sbin/munin12:30:160
189389992,3cyclictest31948-21aten2.4-expect10:20:130
189389991,8cyclictest101rcuc/012:11:170
189389991,8cyclictest101rcuc/008:15:210
189389991,8cyclictest101rcuc/008:00:150
189389991,0cyclictest11586-21diskmemload09:27:530
189389991,0cyclictest101rcuc/008:50:550
189389990,9cyclictest9-21ksoftirqd/008:41:120
189389990,8cyclictest9-21ksoftirqd/009:40:520
189389990,8cyclictest9-21ksoftirqd/008:35:580
189509986,1cyclictest8849-21snmpd11:11:563
189509986,1cyclictest8849-21snmpd11:08:103
189509981,5cyclictest0-21swapper/310:07:513
189509980,6cyclictest31107-21cut08:40:233
189509980,4cyclictest8849-21snmpd11:19:283
189509980,3cyclictest0-21swapper/310:20:123
189449988,0cyclictest0-21swapper/211:32:592
189449987,1cyclictest28-21ksoftirqd/211:15:232
189449987,1cyclictest28-21ksoftirqd/210:55:012
189449987,1cyclictest28-21ksoftirqd/209:45:002
189449987,1cyclictest28-21ksoftirqd/209:22:552
189449987,1cyclictest28-21ksoftirqd/207:55:132
189449986,1cyclictest61250irq/122-eno110:10:202
189449986,1cyclictest28-21ksoftirqd/209:35:152
189449986,1cyclictest28-21ksoftirqd/208:00:252
189449981,4cyclictest26831-21latency_hist11:40:002
189449980,7cyclictest8849-21snmpd09:11:482
189449980,7cyclictest0-21swapper/208:50:132
189449980,2cyclictest0-21swapper/209:15:362
189399980,3cyclictest0-21swapper/112:26:041
189389987,1cyclictest0-21swapper/008:10:100
189389981,1cyclictest0-21swapper/010:10:080
189389981,0cyclictest11586-21diskmemload11:20:400
189389980,8cyclictest9-21ksoftirqd/009:05:200
189389980,8cyclictest9-21ksoftirqd/008:55:100
189389980,8cyclictest9-21ksoftirqd/008:45:080
189389980,8cyclictest9-21ksoftirqd/008:25:020
189389980,8cyclictest9-21ksoftirqd/008:05:440
189389980,8cyclictest9-21ksoftirqd/007:50:240
189389980,8cyclictest9-21ksoftirqd/007:40:040
189389980,8cyclictest9-21ksoftirqd/007:25:260
189389980,8cyclictest9-21ksoftirqd/007:15:040
189389980,8cyclictest9-21ksoftirqd/007:10:120
189389980,1cyclictest9-21ksoftirqd/007:20:020
189389980,0cyclictest9-21ksoftirqd/009:00:050
189389980,0cyclictest9-21ksoftirqd/007:55:280
12029280,1sleep30-21swapper/309:52:403
5813270,1sleep2271rcuc/212:37:452
29356276,0sleep335-21ksoftirqd/311:41:353
189509976,1cyclictest8849-21snmpd12:02:043
189509976,1cyclictest8849-21snmpd10:30:493
189509976,1cyclictest8849-21snmpd10:16:103
189509976,1cyclictest8849-21snmpd08:25:103
189509976,1cyclictest8849-21snmpd07:53:093
189509975,1cyclictest4037-21ssh09:41:123
189509975,1cyclictest30173-21ssh09:34:283
189509975,1cyclictest17783-21ssh11:25:323
189509975,0cyclictest11586-21diskmemload09:15:373
189509973,3cyclictest11586-21diskmemload09:11:443
189509973,2cyclictest11586-21diskmemload10:47:263
189509972,3cyclictest7116-21apt-get09:00:133
189509972,2cyclictest11586-21diskmemload10:40:103
189509971,4cyclictest0-21swapper/312:13:433
189509971,3cyclictest11586-21diskmemload11:20:203
189509971,3cyclictest0-21swapper/311:35:133
189509971,0cyclictest5913-21ntp_states08:55:293
189509970,6cyclictest0-21swapper/310:00:143
189509970,5cyclictest0-21swapper/309:20:133
189509970,3cyclictest0-21swapper/312:33:443
189509970,3cyclictest0-21swapper/312:09:053
189509970,3cyclictest0-21swapper/310:39:183
189449977,0cyclictest0-21swapper/211:02:022
189449976,1cyclictest8849-21snmpd08:18:372
189449976,1cyclictest61250irq/122-eno111:40:272
189449976,1cyclictest61250irq/122-eno107:15:152
189449976,1cyclictest28-21ksoftirqd/212:15:002
189449976,1cyclictest28-21ksoftirqd/208:25:002
189449976,0cyclictest61250irq/122-eno110:41:582
189449976,0cyclictest61250irq/122-eno110:01:092
189449976,0cyclictest61250irq/122-eno109:25:082
189449975,1cyclictest61250irq/122-eno111:45:132
189449975,1cyclictest61250irq/122-eno109:05:312
189449975,1cyclictest2208-21ntpq11:05:212
189449974,2cyclictest18930-21cyclictest10:34:162
189449970,6cyclictest22437-21kworker/2:010:25:232
189399971,6cyclictest0-21swapper/108:56:561
189399971,5cyclictest0-21swapper/111:09:501
189399971,4cyclictest0-21swapper/108:15:131
189399971,3cyclictest0-21swapper/110:23:281
189399971,1cyclictest0-21swapper/112:33:251
189399970,6cyclictest0-21swapper/107:45:161
189399970,4cyclictest0-21swapper/110:41:571
189399970,1cyclictest0-21swapper/111:03:291
25257260,2sleep335-21ksoftirqd/310:54:123
189509965,1cyclictest8849-21snmpd09:06:163
189509965,1cyclictest8849-21snmpd08:24:073
189509965,1cyclictest8849-21snmpd07:41:033
189509965,1cyclictest5477-21ssh11:54:493
189509965,1cyclictest27153-21ssh10:13:083
189509965,1cyclictest25288-21ssh09:27:093
189509965,1cyclictest21756-21ssh11:31:153
189509965,1cyclictest20859-21ssh12:15:063
189509964,1cyclictest8069-21ssh09:46:593
189509964,1cyclictest11586-21diskmemload11:46:003
189509963,2cyclictest4130-21apt-get07:45:113
189509963,1cyclictest259-21systemd-journal07:19:543
189509962,2cyclictest11586-21diskmemload09:35:563
189509961,4cyclictest31820-21apt-get07:35:123
189509960,5cyclictest0-21swapper/312:35:113
189509960,5cyclictest0-21swapper/311:55:123
189509960,3cyclictest24697-21ssh12:20:123
189509960,3cyclictest0-21swapper/311:02:123
189509960,2cyclictest0-21swapper/312:25:423
189509960,2cyclictest0-21swapper/310:25:503
189509960,1cyclictest11215-21apt-get08:00:143
189509960,1cyclictest0-21swapper/310:55:103
189449966,0cyclictest61250irq/122-eno111:50:142
189449966,0cyclictest61250irq/122-eno110:56:302
189449966,0cyclictest61250irq/122-eno110:15:072
189449966,0cyclictest61250irq/122-eno110:05:212
189449966,0cyclictest61250irq/122-eno109:56:402
189449966,0cyclictest61250irq/122-eno109:50:512
189449966,0cyclictest61250irq/122-eno108:55:352
189449966,0cyclictest61250irq/122-eno107:35:172
189449965,1cyclictest8849-21snmpd08:05:092
189449964,1cyclictest61250irq/122-eno112:15:152
189449964,1cyclictest61250irq/122-eno109:30:112
189449964,1cyclictest310-21cat10:20:172
189449961,3cyclictest11586-21diskmemload11:20:122
189449960,6cyclictest0-21swapper/209:04:142
189449960,6cyclictest0-21swapper/208:28:142
189449960,6cyclictest0-21swapper/207:32:142
189449960,6cyclictest0-21swapper/207:32:132
189399966,0cyclictest0-21swapper/109:19:431
189399962,2cyclictest4739-21chrt11:10:121
189399961,5cyclictest0-21swapper/108:45:111
189399960,6cyclictest0-21swapper/112:39:351
189399960,6cyclictest0-21swapper/111:36:441
189399960,6cyclictest0-21swapper/110:01:061
189399960,6cyclictest0-21swapper/109:52:571
189399960,6cyclictest0-21swapper/109:41:441
189399960,6cyclictest0-21swapper/109:27:131
189399960,6cyclictest0-21swapper/107:52:151
189399960,6cyclictest0-21swapper/107:29:481
189399960,5cyclictest0-21swapper/109:37:061
189399960,4cyclictest0-21swapper/112:18:091
189399960,3cyclictest0-21swapper/107:20:121
189399960,2cyclictest0-21swapper/112:04:101
189399960,2cyclictest0-21swapper/110:53:101
189399960,2cyclictest0-21swapper/110:31:011
189399960,1cyclictest0-21swapper/110:25:311
14607260,0sleep30-21swapper/309:55:343
30299250,0sleep30-21swapper/307:30:193
30299250,0sleep30-21swapper/307:30:193
189509953,1cyclictest8849-21snmpd08:35:163
189509953,1cyclictest8849-21snmpd07:22:233
189509953,1cyclictest4044-21unixbench_multi08:50:323
189509953,1cyclictest19122-21sshd08:15:183
189509950,4cyclictest8834-21/usr/sbin/munin07:55:393
189509950,2cyclictest259-21systemd-journal07:15:003
189449955,0cyclictest61250irq/122-eno107:40:152
189449954,1cyclictest61250irq/122-eno108:30:092
189449954,1cyclictest28-21ksoftirqd/208:35:012
189449953,1cyclictest28-21ksoftirqd/207:25:012
189449950,3cyclictest0-21swapper/208:45:112
189449950,1cyclictest4941-21hddtemp_smartct07:45:192
189399955,0cyclictest0-21swapper/111:19:581
189399955,0cyclictest0-21swapper/110:15:241
189399952,1cyclictest13392-21chrt10:37:201
189399951,3cyclictest0-21swapper/111:25:101
189399950,5cyclictest0-21swapper/112:10:161
189399950,5cyclictest0-21swapper/111:56:191
189399950,5cyclictest0-21swapper/111:33:151
189399950,5cyclictest0-21swapper/109:57:071
189399950,5cyclictest0-21swapper/109:08:081
189399950,5cyclictest0-21swapper/108:21:061
189399950,5cyclictest0-21swapper/108:14:221
189399950,4cyclictest0-21swapper/109:34:101
189399950,4cyclictest0-21swapper/108:40:211
189399950,3cyclictest0-21swapper/111:54:321
189399950,3cyclictest0-21swapper/111:45:131
189399950,3cyclictest0-21swapper/110:47:331
189399950,3cyclictest0-21swapper/110:07:231
189399950,3cyclictest0-21swapper/109:45:111
189399950,3cyclictest0-21swapper/109:22:331
189399950,1cyclictest0-21swapper/109:10:121
189399950,1cyclictest0-21swapper/108:00:001
189509943,1cyclictest35-21ksoftirqd/308:05:113
189509943,1cyclictest26543-21ntp_states08:30:283
189509942,1cyclictest16588-21fschecks_time08:10:163
189509940,1cyclictest259-21systemd-journal07:25:013
189399944,0cyclictest0-21swapper/108:34:211
189399944,0cyclictest0-21swapper/108:05:461
189399943,1cyclictest0-21swapper/107:40:191
189399941,1cyclictest0-21swapper/110:10:221
189399940,4cyclictest0-21swapper/109:00:201
189399940,4cyclictest0-21swapper/107:30:251
189399940,4cyclictest0-21swapper/107:30:251
189399940,4cyclictest0-21swapper/107:13:501
189399940,3cyclictest0-21swapper/111:21:021
189399940,2cyclictest0-21swapper/110:55:101
189399940,1cyclictest0-21swapper/111:40:131
189399940,1cyclictest0-21swapper/108:00:131
189399940,0cyclictest0-21swapper/108:26:441
189399940,0cyclictest0-21swapper/107:15:101
189399931,1cyclictest0-21swapper/107:35:121
189399930,2cyclictest0-21swapper/108:50:101
189399930,2cyclictest0-21swapper/108:35:181
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional