You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-14 - 21:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack2slot0.osadl.org (updated Mon Jul 14, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
211721040,2sleep21265099cyclictest11:19:012
617821010,2sleep11264599cyclictest12:01:431
291102900,0sleep329111-21sshd12:22:273
274582740,1sleep1231rcuc/109:20:131
7392620,1sleep2684-21polkitd12:22:502
298702570,0sleep10-21swapper/112:10:121
220012570,5sleep125-21ksoftirqd/111:04:061
297162560,0sleep30-21swapper/312:13:143
117122550,1sleep311698-21sshd12:30:033
14952540,0sleep20-21swapper/210:07:482
306962530,0sleep30-21swapper/310:07:293
213792530,1sleep221372-21bash11:00:272
67722520,0sleep30-21swapper/309:14:273
36182520,1sleep09-21ksoftirqd/012:38:340
299752520,0sleep10-21swapper/111:08:081
51312510,1sleep3646-21dbus-daemon11:25:483
236822510,0sleep0111rcuc/011:10:590
137132510,0sleep20-21swapper/209:11:212
284172500,1sleep228314-21sshd09:30:312
94072490,0sleep20-21swapper/211:26:152
76612480,0sleep00-21swapper/010:11:420
1167024021,7sleep20-21swapper/207:05:222
1097724024,10sleep10-21swapper/107:05:161
82762380,0sleep30-21swapper/309:45:103
277922380,0sleep20-21swapper/209:05:002
190512350,0sleep20-21swapper/211:10:282
124182257,14sleep00-21swapper/007:08:350
123332218,7sleep30-21swapper/307:07:353
198052200,0sleep10-21swapper/111:44:151
3442190,0sleep00-21swapper/012:25:490
248042180,0sleep20-21swapper/210:46:572
235012180,0sleep30-21swapper/310:33:243
177062180,1sleep117665-21sshd09:32:501
83012170,1sleep18298-21lspci10:48:441
1264599170,8cyclictest646-21dbus-daemon09:53:361
9782160,0sleep20-21swapper/212:10:252
15282160,0sleep20-21swapper/211:45:302
1264599163,8cyclictest646-21dbus-daemon12:34:081
1264599163,6cyclictest646-21dbus-daemon12:05:411
1264599160,8cyclictest646-21dbus-daemon09:48:171
1264599160,8cyclictest21460-21rm10:13:081
1264599160,5cyclictest15830-21rm09:59:151
1264599160,5cyclictest10209-21sshd12:26:481
1264599160,4cyclictest22910-21sshd10:16:361
67552150,0sleep00-21swapper/009:58:210
14452150,0sleep00-21swapper/011:48:430
1264599154,8cyclictest684-21polkitd11:13:491
1264599154,7cyclictest684-21polkitd11:25:221
1264599154,5cyclictest684-21polkitd09:39:421
1264599152,8cyclictest10156-21id11:59:041
1264599151,7cyclictest646-21dbus-daemon10:31:591
1264599150,6cyclictest646-21dbus-daemon12:15:301
1264599150,6cyclictest646-21dbus-daemon09:18:051
1264599150,5cyclictest12158-21systemd-cgroups11:49:471
1264199152,1cyclictest646-21dbus-daemon12:19:410
47552140,0sleep30-21swapper/310:44:593
313192140,0sleep30-21swapper/312:35:053
220912140,0sleep30-21swapper/309:29:563
183362140,0sleep20-21swapper/210:22:422
1265099140,1cyclictest32623-21/usr/sbin/munin12:35:172
1264599144,7cyclictest684-21polkitd10:44:151
1264599143,5cyclictest684-21polkitd10:22:581
1264599142,6cyclictest646-21dbus-daemon10:53:251
1264599142,6cyclictest646-21dbus-daemon10:38:191
1264599142,5cyclictest11935-21bash12:36:141
1264599141,6cyclictest648-21systemd-logind10:05:301
1264599140,8cyclictest26581-21sshd09:13:051
1264599140,6cyclictest681-21gdbus10:02:051
1264599140,6cyclictest6595-21bash12:20:171
1264599140,6cyclictest646-21dbus-daemon10:27:561
1264599140,6cyclictest30326-21grep11:21:501
1264599140,6cyclictest30326-21grep11:21:491
1264599140,6cyclictest1-21systemd11:51:151
1264599140,5cyclictest17523-21sshd11:34:081
1264599140,5cyclictest13863-21sshd11:16:511
1264599140,5cyclictest10789-21bash09:25:181
1264199140,3cyclictest646-21dbus-daemon12:33:110
48862130,0sleep00-21swapper/010:44:590
263412130,0sleep30-21swapper/310:54:033
196572130,0sleep20-21swapper/211:37:412
196572130,0sleep20-21swapper/211:37:402
13322130,0sleep00-21swapper/009:31:050
12654991311,1cyclictest0-21swapper/309:42:223
1265099130,12cyclictest19529-21kworker/2:410:30:442
1265099130,0cyclictest121rcu_preempt09:35:332
1264599130,6cyclictest4543-21grep09:41:221
1264599130,6cyclictest20219-21sshd11:37:441
1264599130,6cyclictest20219-21sshd11:37:431
1264599130,6cyclictest12800-21sshd10:56:111
1264199132,1cyclictest7426-21bash10:59:110
1264199132,1cyclictest646-21dbus-daemon09:22:320
1264199131,3cyclictest18224-21bash09:49:290
1264199131,1cyclictest684-21polkitd10:33:570
12641991310,2cyclictest616-17auditd12:03:340
12641991310,2cyclictest0-21swapper/011:02:120
1264199130,5cyclictest697-21gdbus10:23:200
1264199130,5cyclictest684-21polkitd10:16:140
117762130,0sleep311774-21lspci11:33:323
12654991210,1cyclictest0-21swapper/309:52:523
1265499120,1cyclictest692-21NetworkManager11:14:123
1265499120,1cyclictest29749-21munin-node11:45:133
1265499120,11cyclictest482-21sshd09:24:133
1265499120,11cyclictest1556-21sshd09:34:323
1265099124,1cyclictest33-21ksoftirqd/212:16:412
1265099122,3cyclictest33-21ksoftirqd/211:34:322
1265099121,2cyclictest5205-21sshd10:51:522
1265099121,1cyclictest121rcu_preempt11:44:582
1265099120,0cyclictest0-21swapper/210:03:462
1264599122,7cyclictest26084-21cat07:40:141
1264599120,6cyclictest9397-21cat08:19:591
1264599120,11cyclictest0-21swapper/107:35:221
1264199129,2cyclictest24466-21sshd09:40:130
1264199123,3cyclictest646-21dbus-daemon09:13:240
1264199122,1cyclictest9117-21grepconf.sh11:58:580
1264199121,5cyclictest646-21dbus-daemon09:36:250
1264199121,3cyclictest684-21polkitd10:50:060
1264199121,3cyclictest649-21avahi-daemon11:21:310
1264199121,3cyclictest649-21avahi-daemon11:21:300
1264199121,3cyclictest646-21dbus-daemon11:09:530
1264199121,3cyclictest646-21dbus-daemon10:45:400
1264199121,1cyclictest8926-21bash10:38:320
12641991210,1cyclictest19976-21sshd11:50:230
12641991210,1cyclictest0-21swapper/009:26:430
1264199120,6cyclictest8243-21sshd10:05:140
1264199120,5cyclictest646-21dbus-daemon11:17:410
1264199120,5cyclictest1-21systemd11:43:350
1264199120,3cyclictest646-21dbus-daemon12:21:020
1264199120,3cyclictest646-21dbus-daemon11:35:350
1264199120,3cyclictest646-21dbus-daemon11:35:340
1264199120,3cyclictest31321-21systemd-cgroups11:28:470
1264199120,1cyclictest24405-21sshd12:06:330
227972110,0sleep2311rcuc/212:09:322
12654991110,0cyclictest0-21swapper/312:03:023
12654991110,0cyclictest0-21swapper/309:59:423
1265499110,10cyclictest0-21swapper/310:37:023
1265499110,10cyclictest0-21swapper/310:16:333
1265499110,0cyclictest0-21swapper/311:01:423
1265099119,1cyclictest0-21swapper/209:43:152
1265099114,1cyclictest33-21ksoftirqd/212:27:492
1265099113,2cyclictest33-21ksoftirqd/211:50:142
1265099113,1cyclictest33-21ksoftirqd/212:00:122
1265099113,1cyclictest33-21ksoftirqd/209:15:272
1265099112,2cyclictest33-21ksoftirqd/210:12:122
1265099112,1cyclictest131rcu_sched10:56:222
1265099111,1cyclictest131rcu_sched10:40:272
1265099111,1cyclictest121rcu_preempt11:21:082
1265099111,1cyclictest121rcu_preempt11:21:072
1265099111,1cyclictest121rcu_preempt09:55:322
1265099111,1cyclictest121rcu_preempt09:50:032
1265099110,1cyclictest33-21ksoftirqd/210:27:232
1265099110,10cyclictest0-21swapper/208:44:552
1265099110,0cyclictest0-21swapper/211:56:262
1264599112,7cyclictest12527-21which08:25:181
1264599110,8cyclictest1653-21df08:00:121
1264599110,7cyclictest15916-21df08:35:151
1264599110,7cyclictest15238-21sed07:15:021
1264599110,5cyclictest31572-21grepconf.sh09:08:191
1264199113,4cyclictest3355-21munin-run09:10:000
1264199112,2cyclictest648-21systemd-logind09:52:320
1264199112,1cyclictest22463-21sshd09:16:010
1264199112,1cyclictest21647-21grep10:26:190
1264199111,3cyclictest684-21polkitd11:33:030
1264199111,2cyclictest17279-21systemd-cgroups12:12:010
1264199110,5cyclictest646-21dbus-daemon10:02:000
322732100,0sleep20-21swapper/209:08:362
235082100,1sleep323511-21bash11:17:513
1265499109,0cyclictest22281-21sshd10:03:233
1265499109,0cyclictest0-21swapper/311:55:333
1265499109,0cyclictest0-21swapper/311:23:323
1265499109,0cyclictest0-21swapper/311:23:323
1265499109,0cyclictest0-21swapper/310:26:333
1265099109,0cyclictest0-21swapper/210:37:552
1265099109,0cyclictest0-21swapper/209:25:352
1265099109,0cyclictest0-21swapper/208:45:262
1265099103,1cyclictest33-21ksoftirqd/212:30:002
1265099103,1cyclictest33-21ksoftirqd/211:07:472
1265099102,1cyclictest33-21ksoftirqd/209:22:322
1265099102,1cyclictest121rcu_preempt09:46:172
1265099100,1cyclictest23836-21lspci10:16:422
1264599100,7cyclictest6333-21ntp_states08:10:191
1264599100,7cyclictest3725-21gltestperf08:05:161
1264599100,7cyclictest31186-21chrt07:51:431
1264599100,7cyclictest28118-21cat07:45:181
1264599100,7cyclictest28118-21cat07:45:171
1264599100,6cyclictest22142-21smartctl08:50:141
1264599100,6cyclictest17772-21grep08:40:141
1264599100,6cyclictest14804-21ls08:30:221
1264599100,6cyclictest10409-21ntpq08:20:181
1264599100,5cyclictest25761-21sed09:00:001
1264599100,5cyclictest19491-21/usr/sbin/munin07:25:141
1264599100,5cyclictest19256-21munin-run07:25:011
1264199109,0cyclictest101ktimersoftd/008:05:220
126549998,0cyclictest0-21swapper/310:13:223
126549997,1cyclictest0-21swapper/310:23:223
126549995,3cyclictest649-21avahi-daemon10:49:403
126509990,0cyclictest0-21swapper/207:12:452
126459993,5cyclictest13316-21cat07:10:121
126459990,5cyclictest32078-21sh07:55:141
126459990,5cyclictest26789-21which09:00:181
126459990,5cyclictest22035-21interrupts07:30:141
126459990,5cyclictest19888-21ls08:45:151
126419992,1cyclictest9392-21cat08:19:590
126549987,0cyclictest0-21swapper/311:50:523
126549987,0cyclictest0-21swapper/307:45:223
126549987,0cyclictest0-21swapper/307:45:213
126549986,1cyclictest8053-21sshd11:05:533
126549986,1cyclictest24576-21lspci12:09:423
126549986,1cyclictest10725-21lspci11:40:093
126549985,2cyclictest14461-21lspci11:37:083
126549985,2cyclictest14461-21lspci11:37:083
126509980,2cyclictest0-21swapper/207:50:012
126509980,2cyclictest0-21swapper/207:50:002
126509980,1cyclictest121rcu_preempt09:00:012
126419982,2cyclictest32039-21sed07:55:140
126419982,1cyclictest30265-21cat07:50:140
126419981,2cyclictest25353-21sa107:40:010
126549975,1cyclictest646-21dbus-daemon12:25:303
126549975,1cyclictest5879-21systemd-cgroups09:35:033
126549975,1cyclictest4712-21lspci10:55:203
126549974,2cyclictest646-21dbus-daemon12:15:213
126549974,2cyclictest17299-21snmpd07:51:473
126549973,3cyclictest649-21avahi-daemon09:16:303
126509970,1cyclictest121rcu_preempt08:10:012
126419972,1cyclictest21336-21latency_hist07:30:000
126419972,1cyclictest17863-21cat08:40:140
126419972,1cyclictest15843-21idleruntime07:15:170
126419971,1cyclictest6722-21sed08:10:230
126419971,1cyclictest21732-21date08:50:010
126419971,1cyclictest19252-21sh07:25:010
126419971,1cyclictest14633-21munin-run07:15:010
126419971,1cyclictest14046-21idleruntime08:30:150
126419971,1cyclictest11913-21expr08:25:130
126549964,1cyclictest31022-21sshd09:08:043
126549963,2cyclictest17299-21snmpd08:46:183
126549963,2cyclictest17299-21snmpd07:56:523
126549963,2cyclictest17299-21snmpd07:27:483
126549962,2cyclictest25061-21sshd07:36:333
126549962,1cyclictest121rcu_preempt07:15:003
126549960,5cyclictest0-21swapper/308:25:013
126509965,0cyclictest33-21ksoftirqd/208:25:002
126509961,1cyclictest131rcu_sched08:14:592
126509961,1cyclictest121rcu_preempt08:00:002
126509960,5cyclictest0-21swapper/208:33:162
126509960,1cyclictest131rcu_sched07:20:012
126509960,0cyclictest121rcu_preempt07:40:002
126419961,1cyclictest9714-21cstates08:20:120
126419961,1cyclictest28007-21hddtemp_smartct07:45:170
126419961,1cyclictest28007-21hddtemp_smartct07:45:160
126419961,1cyclictest26012-21cstates09:00:140
126419961,1cyclictest26010-21ls07:40:130
126419961,1cyclictest25000-21wc08:55:200
126419961,1cyclictest22842-21wc08:50:200
126419961,1cyclictest21695-21ls07:30:110
126419961,1cyclictest1499-21cpuspeed_turbos08:00:120
126549953,1cyclictest5921-21cat08:10:173
126549953,1cyclictest2781-21vmstat08:00:203
126549953,1cyclictest17949-21cat07:20:193
126549953,1cyclictest17299-21snmpd09:00:063
126549953,1cyclictest17299-21snmpd08:56:403
126549953,1cyclictest17299-21snmpd08:51:513
126549953,1cyclictest17299-21snmpd08:31:133
126549953,1cyclictest17299-21snmpd08:17:273
126549953,1cyclictest17299-21snmpd08:06:093
126549953,1cyclictest17299-21snmpd07:30:023
126549953,1cyclictest17299-21snmpd07:19:453
126549953,1cyclictest0-21swapper/308:20:173
126549952,2cyclictest681-21gdbus07:40:003
126549952,2cyclictest17299-21snmpd08:40:423
126549950,2cyclictest15913-21df08:35:153
126509953,1cyclictest33-21ksoftirqd/207:55:002
126509953,1cyclictest33-21ksoftirqd/207:40:002
126509953,1cyclictest30150-21cat07:50:142
126509953,1cyclictest17299-21snmpd08:50:512
126509953,1cyclictest17299-21snmpd08:25:382
126509953,1cyclictest17299-21snmpd08:15:382
126509953,1cyclictest17299-21snmpd07:27:032
126509953,1cyclictest17299-21snmpd07:15:452
126509953,1cyclictest0-21swapper/208:35:152
126509951,1cyclictest121rcu_preempt07:35:012
126419950,3cyclictest9-21ksoftirqd/008:35:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional