You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-14 - 21:23
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackdslot3.osadl.org (updated Fri Nov 14, 2025 12:45:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
150332233205,18sleep00-21swapper/007:06:590
151862204172,21sleep30-21swapper/307:08:583
151282204170,23sleep20-21swapper/207:08:132
152312203169,23sleep10-21swapper/107:09:251
135512510,2sleep10-21swapper/111:35:121
1557499500,20cyclictest25952-21runrttasks07:30:183
1557499500,20cyclictest25952-21runrttasks07:30:183
193092450,3sleep3391rcuc/308:25:003
136991440,0ptp4l401ktimersoftd/310:20:073
136991420,0ptp4l401ktimersoftd/310:54:523
136991410,0ptp4l401ktimersoftd/307:25:243
136991410,0ptp4l401ktimersoftd/307:20:313
136991400,0ptp4l401ktimersoftd/312:00:123
136991400,0ptp4l401ktimersoftd/309:43:433
136991400,0ptp4l401ktimersoftd/309:20:013
15573993922,14cyclictest33-21ksoftirqd/210:42:562
15573993917,19cyclictest16768-21sh08:20:012
15574993833,3cyclictest41-21ksoftirqd/309:05:003
1557399384,31cyclictest2495-21latency_hist10:05:002
15573993824,12cyclictest33-21ksoftirqd/207:40:012
136991380,0ptp4l401ktimersoftd/310:55:183
59298370,7rtkit-daemon0-21swapper/111:34:361
1557399370,35cyclictest0-21swapper/210:25:362
15572993731,4cyclictest25-21ksoftirqd/107:30:421
15572993731,4cyclictest25-21ksoftirqd/107:30:421
1557499361,21cyclictest19539-21awk07:15:253
15573993630,4cyclictest33-21ksoftirqd/212:32:072
15573993630,4cyclictest33-21ksoftirqd/212:13:052
15573993630,4cyclictest33-21ksoftirqd/211:04:332
15573993630,4cyclictest33-21ksoftirqd/210:45:402
15572993630,4cyclictest25-21ksoftirqd/108:51:171
15572993627,6cyclictest25-21ksoftirqd/107:15:001
15571993630,4cyclictest9-21ksoftirqd/010:42:570
59298350,7rtkit-daemon0-21swapper/109:36:051
15574993530,3cyclictest41-21ksoftirqd/309:35:003
1557499350,33cyclictest11338-21kworker/3:010:31:473
1557499350,33cyclictest0-21swapper/311:10:163
15573993529,4cyclictest33-21ksoftirqd/211:27:042
15573993529,4cyclictest33-21ksoftirqd/209:37:002
15573993529,4cyclictest33-21ksoftirqd/209:09:202
15573993529,4cyclictest33-21ksoftirqd/209:09:202
1557399351,4cyclictest25165-21cron12:00:012
1557399351,17cyclictest14375-21cron08:15:002
15572993530,3cyclictest25-21ksoftirqd/108:44:221
15572993529,4cyclictest25-21ksoftirqd/111:43:161
15572993529,4cyclictest25-21ksoftirqd/111:06:211
15572993529,4cyclictest25-21ksoftirqd/109:59:531
15572993529,4cyclictest25-21ksoftirqd/108:56:111
15572993529,4cyclictest25-21ksoftirqd/107:52:121
15572993528,5cyclictest25-21ksoftirqd/112:20:341
15571993529,4cyclictest9-21ksoftirqd/010:53:560
15574993429,3cyclictest41-21ksoftirqd/309:57:483
15574993428,4cyclictest41-21ksoftirqd/312:29:073
15574993428,4cyclictest41-21ksoftirqd/311:57:133
15574993428,4cyclictest41-21ksoftirqd/309:27:063
1557399343,30cyclictest12642-21kworker/2:307:18:352
15573993429,3cyclictest33-21ksoftirqd/210:24:222
15573993428,4cyclictest479-21in:imuxsock10:32:552
15573993428,4cyclictest33-21ksoftirqd/212:27:282
15573993428,4cyclictest33-21ksoftirqd/210:05:362
15573993428,4cyclictest33-21ksoftirqd/209:34:472
15573993428,4cyclictest33-21ksoftirqd/209:04:062
15573993428,4cyclictest33-21ksoftirqd/207:54:262
15573993427,4cyclictest33-21ksoftirqd/208:01:422
15573993427,4cyclictest33-21ksoftirqd/208:01:422
1557399342,10cyclictest1285-21snmpd09:45:122
15573993412,3cyclictest17377-21sed10:35:182
15572993430,3cyclictest25-21ksoftirqd/109:05:001
15572993428,4cyclictest25-21ksoftirqd/110:19:181
15572993428,4cyclictest25-21ksoftirqd/109:50:031
15572993428,4cyclictest25-21ksoftirqd/109:41:521
15572993428,4cyclictest25-21ksoftirqd/109:34:181
15572993428,4cyclictest25-21ksoftirqd/108:33:481
15572993427,4cyclictest25-21ksoftirqd/109:05:091
15572993427,4cyclictest25-21ksoftirqd/109:05:091
15571993428,4cyclictest9-21ksoftirqd/008:44:380
15571993428,4cyclictest9-21ksoftirqd/008:28:580
15574993328,3cyclictest41-21ksoftirqd/309:50:113
15574993327,4cyclictest41-21ksoftirqd/312:11:503
15574993327,4cyclictest41-21ksoftirqd/311:23:363
15574993327,4cyclictest41-21ksoftirqd/310:13:393
15574993327,4cyclictest41-21ksoftirqd/309:13:223
15573993328,3cyclictest33-21ksoftirqd/211:44:452
15573993328,3cyclictest33-21ksoftirqd/211:35:512
15573993327,4cyclictest33-21ksoftirqd/211:51:552
15573993327,4cyclictest33-21ksoftirqd/209:22:522
15573993327,4cyclictest33-21ksoftirqd/208:45:572
15573993327,4cyclictest33-21ksoftirqd/207:46:072
15573993327,4cyclictest33-21ksoftirqd/207:44:112
15573993327,4cyclictest33-21ksoftirqd/207:32:522
15573993327,4cyclictest33-21ksoftirqd/207:32:522
1557399331,17cyclictest7071-21http08:00:002
1557399330,17cyclictest21426-1kworker/2:1H10:55:002
15572993328,3cyclictest25-21ksoftirqd/112:35:141
15572993328,3cyclictest25-21ksoftirqd/107:49:111
15572993327,4cyclictest25-21ksoftirqd/111:55:461
15572993327,4cyclictest25-21ksoftirqd/111:13:471
15572993327,4cyclictest25-21ksoftirqd/110:05:031
15572993327,4cyclictest25-21ksoftirqd/108:11:141
15572993327,4cyclictest25-21ksoftirqd/107:17:151
15571993328,3cyclictest9-21ksoftirqd/012:14:200
15571993328,3cyclictest9-21ksoftirqd/009:26:550
15571993328,3cyclictest9-21ksoftirqd/007:35:000
15571993328,3cyclictest9-21ksoftirqd/007:35:000
15571993327,4cyclictest9-21ksoftirqd/008:19:270
15571993326,5cyclictest9-21ksoftirqd/012:07:340
15571993326,4cyclictest9-21ksoftirqd/011:09:170
15571993326,4cyclictest9-21ksoftirqd/010:05:060
15571993326,4cyclictest9-21ksoftirqd/009:41:590
15574993227,3cyclictest41-21ksoftirqd/312:21:133
15574993227,3cyclictest41-21ksoftirqd/307:44:373
15574993226,4cyclictest41-21ksoftirqd/312:15:473
15574993226,4cyclictest41-21ksoftirqd/311:08:523
15574993226,4cyclictest41-21ksoftirqd/310:27:523
15574993226,4cyclictest41-21ksoftirqd/309:09:253
15574993226,4cyclictest41-21ksoftirqd/309:09:253
15574993226,4cyclictest41-21ksoftirqd/308:43:533
15574993226,4cyclictest41-21ksoftirqd/308:05:423
15574993226,4cyclictest41-21ksoftirqd/307:54:433
1557399323,28cyclictest2125-21kworker/2:008:36:432
1557399323,28cyclictest2125-21kworker/2:008:36:432
15573993228,3cyclictest33-21ksoftirqd/207:27:582
15573993226,4cyclictest33-21ksoftirqd/212:17:172
15573993226,4cyclictest33-21ksoftirqd/211:22:552
15573993226,4cyclictest33-21ksoftirqd/211:17:132
15573993226,4cyclictest33-21ksoftirqd/210:56:032
15573993226,4cyclictest33-21ksoftirqd/209:17:432
15573993226,4cyclictest33-21ksoftirqd/208:41:482
15573993226,4cyclictest33-21ksoftirqd/208:26:032
15573993211,3cyclictest31973-21cut08:50:202
15572993227,3cyclictest25-21ksoftirqd/108:29:431
15572993226,4cyclictest25-21ksoftirqd/112:29:171
15572993226,4cyclictest25-21ksoftirqd/111:49:451
15572993226,4cyclictest25-21ksoftirqd/109:48:181
15572993226,4cyclictest25-21ksoftirqd/109:29:291
15572993226,4cyclictest25-21ksoftirqd/109:21:181
15572993226,4cyclictest25-21ksoftirqd/108:38:341
15572993226,4cyclictest25-21ksoftirqd/108:38:341
15572993226,4cyclictest25-21ksoftirqd/108:16:191
15572993226,4cyclictest25-21ksoftirqd/108:04:411
15572993226,4cyclictest25-21ksoftirqd/108:04:401
15572993226,4cyclictest25-21ksoftirqd/107:43:581
15571993227,3cyclictest9-21ksoftirqd/012:34:090
15571993227,3cyclictest9-21ksoftirqd/011:14:550
15571993226,4cyclictest9-21ksoftirqd/011:15:590
15571993226,4cyclictest9-21ksoftirqd/011:03:190
15571993226,4cyclictest9-21ksoftirqd/010:39:030
15571993226,4cyclictest9-21ksoftirqd/010:24:200
15571993226,4cyclictest9-21ksoftirqd/009:55:510
15571993226,4cyclictest9-21ksoftirqd/008:39:250
15571993226,4cyclictest9-21ksoftirqd/008:39:250
15571993225,4cyclictest9-21ksoftirqd/008:07:240
15574993125,4cyclictest41-21ksoftirqd/312:31:403
15574993125,4cyclictest41-21ksoftirqd/311:47:033
15574993125,4cyclictest41-21ksoftirqd/310:42:243
15574993125,4cyclictest41-21ksoftirqd/310:19:573
15574993125,4cyclictest41-21ksoftirqd/307:10:033
15574993114,14cyclictest2722-21/usr/sbin/munin10:05:153
15574993113,16cyclictest27800-21/usr/sbin/munin12:05:253
1557399314,25cyclictest23969-21kworker/2:009:58:312
15573993126,3cyclictest33-21ksoftirqd/210:18:242
15573993125,4cyclictest33-21ksoftirqd/211:12:282
15573993125,4cyclictest33-21ksoftirqd/210:13:102
15573993125,4cyclictest33-21ksoftirqd/209:50:522
15573993125,4cyclictest33-21ksoftirqd/208:34:422
15573993113,16cyclictest33-21ksoftirqd/208:07:002
1557399310,29cyclictest12642-21kworker/2:307:12:232
15572993126,3cyclictest25-21ksoftirqd/109:17:001
15572993126,3cyclictest25-21ksoftirqd/108:06:211
15572993125,4cyclictest25-21ksoftirqd/112:00:481
15572993125,4cyclictest25-21ksoftirqd/111:28:511
15572993125,4cyclictest25-21ksoftirqd/108:47:231
15572993124,4cyclictest25-21ksoftirqd/107:26:141
15571993126,3cyclictest9-21ksoftirqd/009:45:080
15571993126,3cyclictest9-21ksoftirqd/008:15:000
15571993125,4cyclictest9-21ksoftirqd/012:38:470
15571993125,4cyclictest9-21ksoftirqd/012:27:220
15571993125,4cyclictest9-21ksoftirqd/011:39:250
15571993125,4cyclictest9-21ksoftirqd/011:22:240
15571993125,4cyclictest9-21ksoftirqd/010:13:240
15571993125,4cyclictest9-21ksoftirqd/009:34:170
15571993125,4cyclictest9-21ksoftirqd/007:47:020
15571993125,4cyclictest9-21ksoftirqd/007:44:180
15574993024,4cyclictest41-21ksoftirqd/311:03:223
15574993024,4cyclictest41-21ksoftirqd/308:59:023
15574993024,4cyclictest41-21ksoftirqd/308:47:423
15574993023,5cyclictest41-21ksoftirqd/309:47:563
1557499302,22cyclictest13502-21apt-get11:35:133
1557399302,26cyclictest12757-21kworker/2:012:20:262
1557399301,28cyclictest26167-21kworker/2:211:08:232
15573993012,3cyclictest13142-21wc11:35:002
15572993025,3cyclictest25-21ksoftirqd/111:20:001
15572993025,3cyclictest25-21ksoftirqd/110:39:011
15572993024,4cyclictest25-21ksoftirqd/111:22:501
15572993024,4cyclictest25-21ksoftirqd/110:41:341
15572993024,4cyclictest25-21ksoftirqd/110:10:411
15572993024,4cyclictest25-21ksoftirqd/107:21:441
15571993025,3cyclictest9-21ksoftirqd/010:04:380
15571993025,3cyclictest9-21ksoftirqd/009:07:520
15571993025,3cyclictest9-21ksoftirqd/009:07:520
15571993025,3cyclictest9-21ksoftirqd/008:22:250
15571993025,3cyclictest9-21ksoftirqd/007:55:000
15571993025,3cyclictest9-21ksoftirqd/007:11:270
15571993024,4cyclictest9-21ksoftirqd/012:18:040
15571993024,4cyclictest9-21ksoftirqd/011:34:080
15571993024,4cyclictest9-21ksoftirqd/010:28:170
15571993024,4cyclictest9-21ksoftirqd/009:21:290
15571993024,4cyclictest9-21ksoftirqd/007:20:420
15574992924,3cyclictest41-21ksoftirqd/310:39:363
15574992923,4cyclictest41-21ksoftirqd/312:39:413
15574992923,4cyclictest41-21ksoftirqd/311:30:453
15574992923,4cyclictest41-21ksoftirqd/308:30:053
15574992923,4cyclictest41-21ksoftirqd/308:26:443
1557499291,24cyclictest8477-21/usr/sbin/munin11:25:363
15574992911,14cyclictest22789-21sshd11:55:003
15574992910,17cyclictest28895-21grep07:35:223
15573992925,3cyclictest33-21ksoftirqd/209:41:432
15573992924,3cyclictest33-21ksoftirqd/212:00:442
1557399291,26cyclictest13660-21kworker/2:209:27:312
15573992912,4cyclictest1297-21runrttasks11:45:242
15572992924,3cyclictest25-21ksoftirqd/112:15:411
15572992924,3cyclictest25-21ksoftirqd/112:07:301
15572992923,4cyclictest25-21ksoftirqd/110:51:081
15572992923,4cyclictest25-21ksoftirqd/109:13:081
15572992923,4cyclictest25-21ksoftirqd/108:22:571
15572992922,5cyclictest25-21ksoftirqd/110:48:441
15571992925,2cyclictest9-21ksoftirqd/010:15:210
15571992924,3cyclictest9-21ksoftirqd/009:12:420
15571992924,3cyclictest9-21ksoftirqd/007:56:120
15571992923,4cyclictest9-21ksoftirqd/012:23:460
15571992923,4cyclictest9-21ksoftirqd/011:54:100
15571992923,4cyclictest9-21ksoftirqd/011:42:580
15571992923,4cyclictest9-21ksoftirqd/011:27:020
15571992923,4cyclictest9-21ksoftirqd/008:32:590
15571992923,4cyclictest9-21ksoftirqd/007:36:470
1557499286,19cyclictest14811-21cpuspeed_turbos08:15:143
15574992823,3cyclictest41-21ksoftirqd/310:01:013
15574992823,3cyclictest41-21ksoftirqd/308:02:313
15574992823,3cyclictest41-21ksoftirqd/308:02:303
15574992823,3cyclictest41-21ksoftirqd/307:57:023
15574992811,15cyclictest993-21latency_hist08:55:003
15574992810,16cyclictest4694-21cat11:15:193
1557399283,23cyclictest31250-21kworker/2:109:10:352
1557399282,25cyclictest2125-21kworker/2:008:21:042
1557399280,27cyclictest31250-21kworker/2:108:56:392
15572992823,3cyclictest25-21ksoftirqd/110:28:261
15572992822,4cyclictest25-21ksoftirqd/111:04:001
15571992823,3cyclictest9-21ksoftirqd/010:59:580
15571992822,4cyclictest9-21ksoftirqd/008:57:060
15574992722,3cyclictest41-21ksoftirqd/309:15:433
15574992722,3cyclictest41-21ksoftirqd/308:39:303
15574992722,3cyclictest41-21ksoftirqd/308:39:293
15574992721,4cyclictest41-21ksoftirqd/308:10:033
15573992722,3cyclictest33-21ksoftirqd/212:36:172
1557399271,2cyclictest12642-21kworker/2:307:20:032
15572992722,3cyclictest25-21ksoftirqd/110:58:301
15572992722,3cyclictest25-21ksoftirqd/107:55:451
15572992721,4cyclictest25-21ksoftirqd/110:01:221
15571992722,3cyclictest9-21ksoftirqd/011:58:190
15571992722,3cyclictest9-21ksoftirqd/011:50:000
15571992722,3cyclictest9-21ksoftirqd/008:45:430
15571992722,3cyclictest9-21ksoftirqd/007:16:280
15571992721,4cyclictest9-21ksoftirqd/009:16:320
15571992720,4cyclictest8960-21ssh08:00:290
15571992720,4cyclictest8960-21ssh08:00:290
59298260,2rtkit-daemon0-21swapper/110:23:131
1557499266,17cyclictest16719-21cut11:40:193
15574992621,3cyclictest41-21ksoftirqd/309:38:243
1557499261,15cyclictest22843-21memory10:45:213
15573992620,4cyclictest33-21ksoftirqd/212:06:272
15572992621,3cyclictest25-21ksoftirqd/111:50:521
15572992621,3cyclictest25-21ksoftirqd/107:36:501
15571992621,3cyclictest9-21ksoftirqd/009:00:350
15571992620,4cyclictest9-21ksoftirqd/010:49:350
15571992618,5cyclictest9-21ksoftirqd/009:35:130
136991260,1ptp4l401ktimersoftd/307:45:123
15571992520,3cyclictest9-21ksoftirqd/007:28:360
15572992419,3cyclictest25-21ksoftirqd/112:31:031
15572992419,3cyclictest25-21ksoftirqd/112:10:551
15571992419,3cyclictest9-21ksoftirqd/009:52:340
15571992217,3cyclictest9-21ksoftirqd/010:32:300
15572992117,3cyclictest25-21ksoftirqd/110:30:131
15571992117,3cyclictest9-21ksoftirqd/012:00:130
15571992115,4cyclictest9-21ksoftirqd/008:53:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional