You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-09 - 13:01
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack2slot0.osadl.org (updated Sun Nov 09, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3248821190,0sleep30-21swapper/323:10:553
1436421080,2sleep02180899cyclictest21:54:340
251872910,0sleep20-21swapper/222:46:272
184212700,0sleep20-21swapper/200:05:532
184212700,0sleep20-21swapper/200:05:532
47172640,1sleep24715-21sshd22:10:182
263882630,0sleep10-21swapper/100:14:011
110562610,0sleep20-21swapper/223:18:432
139572590,0sleep30-21swapper/322:58:453
13692590,0sleep10-21swapper/123:14:251
320112550,1sleep032014-21bash23:53:490
320112550,1sleep032014-21bash23:53:490
269952550,0sleep30-21swapper/322:43:193
74472520,0sleep10-21swapper/123:34:481
26472520,0sleep00-21swapper/021:12:400
113082520,0sleep30-21swapper/321:17:043
279312510,0sleep10-21swapper/123:36:481
229402490,0sleep00-21swapper/022:15:370
241372480,2sleep02180899cyclictest23:16:400
276662430,2sleep22181799cyclictest23:56:442
189042430,2sleep32182099cyclictest21:14:323
32412410,0sleep3391rcuc/300:33:253
216472309,7sleep10-21swapper/119:09:181
214832309,7sleep30-21swapper/319:07:163
2139723010,6sleep00-21swapper/019:06:170
2139922211,6sleep20-21swapper/219:06:192
195342220,0sleep20-21swapper/223:29:282
104342210,0sleep10-21swapper/100:26:391
98062190,0sleep10-21swapper/121:47:141
85962170,0sleep20-21swapper/200:37:452
80462160,0sleep20-21swapper/222:44:462
70302160,0sleep20-21swapper/221:33:292
284192160,1sleep128420-21lspci22:19:421
261522160,0sleep2321ktimersoftd/223:39:582
21812991615,0cyclictest3925-21lspci21:36:291
120082160,0sleep10-21swapper/121:27:151
11642160,0sleep00-21swapper/023:14:240
202452150,0sleep0101ktimersoftd/021:34:550
159982150,0sleep00-21swapper/022:15:000
153412150,0sleep30-21swapper/322:45:233
13662150,0sleep00-21swapper/022:40:320
99222140,1sleep19920-21grep23:51:301
99222140,1sleep19920-21grep23:51:301
63122140,0sleep20-21swapper/221:16:312
321372140,0sleep20-21swapper/223:34:012
293592140,1sleep029357-21sshd23:33:430
21902140,0sleep20-21swapper/200:21:552
133332140,1sleep313336-21bash23:38:383
289222130,0sleep0101ktimersoftd/021:45:490
21817991311,1cyclictest903-21lspci21:22:442
2181799131,11cyclictest700-21gdbus21:38:412
2181799130,1cyclictest21064-21cp00:13:242
2181799130,12cyclictest48-21kauditd22:38:242
21812991311,1cyclictest4219-21bash22:54:261
2180899130,12cyclictest25932-21bash22:49:570
217682130,0sleep10-21swapper/122:42:451
125882130,0sleep00-21swapper/022:28:100
118462130,0sleep20-21swapper/223:48:272
21820991210,1cyclictest30991-21sshd23:43:423
2182099120,11cyclictest0-21swapper/300:07:023
2182099120,11cyclictest0-21swapper/300:07:023
2181799120,11cyclictest646-21dbus-daemon22:15:442
2181299120,11cyclictest0-21swapper/123:28:261
2180899120,1cyclictest21489-21sshd23:36:070
2180899120,1cyclictest17721-21sshd00:02:280
2180899120,11cyclictest1490-21lspci21:59:570
2182099119,1cyclictest14097-21sshd22:52:023
21820991110,0cyclictest391rcuc/321:26:113
21820991110,0cyclictest0-21swapper/323:47:223
2182099110,1cyclictest31646-21sshd22:26:443
2182099110,10cyclictest0-21swapper/323:00:323
2182099110,0cyclictest41-21ksoftirqd/322:01:323
2181799113,4cyclictest8424-21sshd23:21:422
21817991110,0cyclictest0-21swapper/221:48:242
21817991110,0cyclictest0-21swapper/221:29:342
21817991110,0cyclictest0-21swapper/220:55:142
21817991110,0cyclictest0-21swapper/200:15:142
2181799110,10cyclictest0-21swapper/222:55:542
2181799110,0cyclictest0-21swapper/222:26:442
2181299119,1cyclictest11322-21lspci23:58:261
21812991110,0cyclictest0-21swapper/100:30:261
21812991110,0cyclictest0-21swapper/100:07:361
21812991110,0cyclictest0-21swapper/100:07:361
2181299110,1cyclictest8106-21grepconf.sh23:15:061
2181299110,10cyclictest0-21swapper/123:46:061
2180899119,1cyclictest32390-21sshd22:03:070
21808991110,0cyclictest0-21swapper/023:03:170
2180899110,10cyclictest0-21swapper/023:23:570
2180899110,10cyclictest0-21swapper/023:05:070
2180899110,0cyclictest0-21swapper/022:07:570
2180899110,0cyclictest0-21swapper/021:15:270
2180899110,0cyclictest0-21swapper/021:09:570
2180899110,0cyclictest0-21swapper/000:38:180
249862100,0sleep00-21swapper/023:59:540
2182099109,0cyclictest0-21swapper/322:10:023
2182099109,0cyclictest0-21swapper/322:05:123
2182099109,0cyclictest0-21swapper/321:53:523
2182099109,0cyclictest0-21swapper/300:36:423
2182099109,0cyclictest0-21swapper/300:29:523
2182099106,1cyclictest41-21ksoftirqd/323:19:503
2182099105,3cyclictest41-21ksoftirqd/323:25:283
2182099105,3cyclictest41-21ksoftirqd/323:21:133
2182099104,2cyclictest41-21ksoftirqd/322:35:173
2182099103,3cyclictest41-21ksoftirqd/322:23:183
2182099103,3cyclictest41-21ksoftirqd/300:16:393
2182099102,5cyclictest41-21ksoftirqd/300:13:053
2182099100,0cyclictest0-21swapper/323:57:123
2182099100,0cyclictest0-21swapper/322:32:023
2182099100,0cyclictest0-21swapper/321:21:513
2182099100,0cyclictest0-21swapper/319:42:313
2181799109,0cyclictest0-21swapper/221:58:342
2181799109,0cyclictest0-21swapper/221:51:342
2181799109,0cyclictest0-21swapper/221:12:542
2181799109,0cyclictest0-21swapper/200:30:042
2181799108,1cyclictest2305-21sshd22:30:242
2181799101,6cyclictest9294-21sshd22:20:592
2181799101,2cyclictest6420-21bash23:14:572
2181799101,2cyclictest26626-21sshd23:00:062
2181799100,3cyclictest5208-21sshd23:41:022
2181299109,0cyclictest33050irq/27-ahci[00023:40:441
2181299109,0cyclictest1024-21lspci23:24:191
2181299109,0cyclictest0-21swapper/121:24:561
2181299109,0cyclictest0-21swapper/100:23:361
2181299100,0cyclictest0-21swapper/122:49:361
2181299100,0cyclictest0-21swapper/100:39:461
2180899109,0cyclictest0-21swapper/000:12:180
2180899108,1cyclictest0-21swapper/021:23:270
2180899100,0cyclictest0-21swapper/021:40:570
163582100,0sleep20-21swapper/222:52:152
218209998,0cyclictest0-21swapper/321:07:413
218209994,3cyclictest41-21ksoftirqd/323:05:343
218209993,3cyclictest41-21ksoftirqd/300:21:223
218209993,3cyclictest41-21ksoftirqd/300:02:283
218209992,1cyclictest41-21ksoftirqd/323:51:283
218209992,1cyclictest41-21ksoftirqd/323:51:283
218209991,3cyclictest41-21ksoftirqd/321:30:143
218179991,5cyclictest7904-21sshd21:07:052
218179991,5cyclictest3137-21sshd00:04:202
218179991,2cyclictest12458-21lspci00:26:542
218129998,0cyclictest33050irq/27-ahci[00021:32:021
218129998,0cyclictest15715-21lspci21:51:151
218129998,0cyclictest0-21swapper/122:36:561
218129998,0cyclictest0-21swapper/120:54:561
218129998,0cyclictest0-21swapper/120:35:161
218129997,1cyclictest5499-21bash22:30:461
218129997,1cyclictest25-21ksoftirqd/121:14:351
218129997,1cyclictest25-21ksoftirqd/100:05:001
218129997,1cyclictest22839-21lspci22:02:041
218089998,0cyclictest0-21swapper/020:30:170
218089998,0cyclictest0-21swapper/020:13:270
218089998,0cyclictest0-21swapper/000:28:580
218089990,8cyclictest0-21swapper/019:33:270
15435290,0sleep10-21swapper/121:41:051
15394290,1sleep1231rcuc/122:14:571
218209987,0cyclictest41-21ksoftirqd/321:47:193
218209987,0cyclictest0-21swapper/320:39:313
218209987,0cyclictest0-21swapper/319:45:313
218209982,3cyclictest41-21ksoftirqd/321:56:133
218209982,3cyclictest41-21ksoftirqd/320:30:003
218209982,1cyclictest41-21ksoftirqd/322:15:403
218209981,3cyclictest41-21ksoftirqd/321:40:463
218209981,3cyclictest41-21ksoftirqd/319:35:193
218209980,5cyclictest11836-21tty21:37:223
218179982,5cyclictest7420-21cp23:54:392
218179982,5cyclictest7420-21cp23:54:392
218179982,5cyclictest443-21systemd-journal22:01:372
218179981,4cyclictest20483-21which22:05:152
218179981,3cyclictest7963-21sshd23:05:012
218179980,5cyclictest9452-21sshd21:40:262
218179980,5cyclictest32471-21taskset20:52:432
218129986,1cyclictest25-21ksoftirqd/123:00:161
218129986,1cyclictest25-21ksoftirqd/121:58:411
218129986,1cyclictest25-21ksoftirqd/100:19:481
218129985,2cyclictest649-21avahi-daemon22:21:191
218129984,3cyclictest616-17auditd23:07:221
218089986,1cyclictest618-13audispd22:51:550
218089986,1cyclictest22307-21lspci23:26:250
218089985,2cyclictest649-21avahi-daemon23:44:340
218089985,2cyclictest649-21avahi-daemon21:35:160
218089985,2cyclictest649-21avahi-daemon00:20:260
218209976,0cyclictest0-21swapper/320:55:013
218209973,3cyclictest646-21dbus-daemon23:30:093
218179970,5cyclictest6111-21taskset19:48:452
218179970,5cyclictest24331-21runrttasks20:32:332
218179970,4cyclictest4335-21runrttasks21:04:282
218129976,0cyclictest13516-21lspci22:55:171
218129975,1cyclictest25-21ksoftirqd/122:25:541
218129975,1cyclictest18970-21lspci22:05:071
218129970,0cyclictest0-21swapper/120:04:561
218089976,0cyclictest0-21swapper/022:37:170
218089976,0cyclictest0-21swapper/022:31:070
218089975,1cyclictest700-21gdbus00:18:000
218089975,1cyclictest29375-21lspci22:23:090
218089975,1cyclictest28407-21sshd23:46:480
218089975,1cyclictest10575-21lspci00:05:090
218089975,1cyclictest10575-21lspci00:05:090
218089974,2cyclictest13786-21sshd21:27:260
218089974,2cyclictest11759-21sshd00:30:230
218089971,3cyclictest28431-21perf19:25:000
218209965,0cyclictest0-21swapper/320:00:313
218209961,3cyclictest41-21ksoftirqd/320:05:213
218209961,3cyclictest41-21ksoftirqd/319:50:513
218209961,3cyclictest41-21ksoftirqd/319:30:433
218209961,3cyclictest41-21ksoftirqd/319:26:283
218209961,2cyclictest41-21ksoftirqd/321:00:153
218209960,5cyclictest0-21swapper/319:59:513
218209960,2cyclictest22932-21cat20:30:123
218209960,1cyclictest0-21swapper/320:20:003
218179964,1cyclictest17118-21cat20:15:152
218179963,2cyclictest27503-21needreboot20:40:162
218179963,2cyclictest25497-21grep19:15:172
218179963,2cyclictest22482-21cat19:10:142
218179963,2cyclictest19928-21sed20:20:212
218179963,2cyclictest17299-21snmpd20:36:282
218179963,2cyclictest13027-21if_eno220:05:172
218179963,2cyclictest11364-21ntpdc20:00:162
218179962,3cyclictest8483-21/usr/sbin/munin19:55:142
218179962,3cyclictest6728-21chrt19:50:142
218179962,3cyclictest32716-21grep19:35:152
218179961,4cyclictest30878-21expr19:30:152
218179960,5cyclictest0-21swapper/220:12:332
218179960,3cyclictest30560-21taskset20:49:192
218179960,2cyclictest28642-21taskset19:25:112
218179960,2cyclictest22377-21taskset20:28:372
218129964,1cyclictest700-21gdbus21:08:001
218129964,1cyclictest24936-21sshd21:15:091
218129963,2cyclictest646-21dbus-daemon20:45:001
218129960,5cyclictest0-21swapper/119:49:351
218089965,0cyclictest0-21swapper/019:36:270
218089964,1cyclictest649-21avahi-daemon20:10:000
218089961,1cyclictest11255-21sshd22:55:090
218089960,5cyclictest0-21swapper/019:11:470
218089960,2cyclictest10300-21awk20:00:000
218209953,1cyclictest27062-21if_eno119:20:163
218209953,1cyclictest25169-21ls19:15:153
218209953,1cyclictest17299-21snmpd20:10:093
218209951,2cyclictest41-21ksoftirqd/320:40:143
218209951,2cyclictest41-21ksoftirqd/320:15:153
218209951,1cyclictest41-21ksoftirqd/320:45:173
218209950,3cyclictest0-21swapper/320:50:253
218209950,3cyclictest0-21swapper/319:10:063
218179953,1cyclictest17299-21snmpd19:40:052
218179952,2cyclictest26666-21/usr/sbin/munin19:20:142
218129953,1cyclictest900-21hwlatdetect20:55:131
218129953,1cyclictest692-21NetworkManager20:30:001
218129953,1cyclictest682-21lsmd21:01:271
218129953,1cyclictest29134-21cat19:25:161
218129953,1cyclictest18751-21tr20:20:131
218129953,1cyclictest17299-21snmpd20:49:041
218129953,1cyclictest17299-21snmpd20:14:511
218129953,1cyclictest17299-21snmpd20:05:121
218129953,1cyclictest17299-21snmpd19:50:021
218129953,1cyclictest17299-21snmpd19:42:321
218129953,1cyclictest17299-21snmpd19:39:051
218129953,1cyclictest17299-21snmpd19:30:561
218129953,1cyclictest17299-21snmpd19:22:321
218129953,1cyclictest17079-21cat20:15:151
218129953,1cyclictest1109-21ntpd20:30:171
218129953,1cyclictest1058-21sendmail19:16:201
218129953,1cyclictest1017-21runrttasks19:12:551
218089954,0cyclictest0-21swapper/021:03:370
218089953,1cyclictest29675-21ntp_kernel_pll_20:45:160
218089953,1cyclictest25241-21mii-tool20:35:140
218089953,1cyclictest17299-21snmpd20:50:280
218089953,1cyclictest17299-21snmpd20:42:280
218089953,1cyclictest17299-21snmpd20:26:490
218089953,1cyclictest17299-21snmpd19:54:420
218089953,1cyclictest17299-21snmpd19:41:560
218089953,1cyclictest17299-21snmpd19:17:440
218089953,1cyclictest1053-21lldpd20:16:070
218089953,1cyclictest1017-21runrttasks19:25:180
218089952,2cyclictest2177-21taskset20:58:030
218089950,4cyclictest0-21swapper/020:01:570
218129942,1cyclictest8936-21cat19:55:131
218089942,1cyclictest4911-21cat19:45:140
218089942,1cyclictest19718-21sensors20:20:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional