You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-18 - 00:50
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot2.osadl.org (updated Mon Nov 17, 2025 12:45:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23972205172,22sleep00-21swapper/007:06:410
136691205171,23phc2sys0-21swapper/307:08:433
26222204170,23sleep20-21swapper/207:09:362
24712203169,23sleep10-21swapper/107:07:381
1943421450,4sleep2295499cyclictest11:05:162
234102620,3sleep10-21swapper/107:50:271
23212610,2sleep30-21swapper/309:20:303
188302610,2sleep30-21swapper/307:40:293
133291610,4getstats41-21ksoftirqd/309:16:423
192942600,2sleep10-21swapper/111:05:151
295399569,18cyclictest16614-21sed08:45:161
136591550,1ptp4l401ktimersoftd/311:52:513
136591540,1ptp4l401ktimersoftd/310:30:083
136591540,1ptp4l401ktimersoftd/310:30:073
295399534,18cyclictest0-21swapper/111:45:181
136591520,1ptp4l401ktimersoftd/311:11:553
295399518,10cyclictest31629-21latency_hist10:25:001
118212510,2sleep30-21swapper/309:40:283
3132500,3sleep0317-21unixbench_multi11:30:290
295399507,8cyclictest18704-21ps09:55:281
295399507,7cyclictest23460-21ps10:05:271
295399505,17cyclictest0-21swapper/109:18:571
295399498,16cyclictest31569-21grep11:30:191
295399489,8cyclictest20314-21hddtemp_smartct10:00:201
295399489,8cyclictest20314-21hddtemp_smartct10:00:201
295399487,15cyclictest27559-21ps10:15:221
295399477,7cyclictest9835-21ssh11:50:271
295399476,16cyclictest12358-21latency_hist07:29:591
295399469,8cyclictest20036-21df_inode07:45:181
295399469,7cyclictest1468-21df_inode11:35:171
295399468,9cyclictest24526-21/usr/sbin/munin09:00:251
295399468,8cyclictest15478-21if_enp1s007:35:181
295399468,8cyclictest11728-21meminfo11:55:211
295399466,15cyclictest30633-21latency_hist09:15:011
295399465,15cyclictest0-21swapper/108:30:181
295399459,7cyclictest4466-21irqstats11:40:201
295399458,8cyclictest18622-21rt-features07:40:261
295399458,8cyclictest15332-21df09:50:161
295399458,7cyclictest7505-21phc2sys-jitter08:25:241
295399458,7cyclictest26114-21df_abs09:05:161
295399455,15cyclictest0-21swapper/109:45:191
133391450,8getstats0-21swapper/312:05:113
295399448,9cyclictest26683-21hddtemp_smartct11:20:171
295399448,7cyclictest18252-21sshd11:01:221
295399448,7cyclictest14256-21df_abs08:40:151
295399447,8cyclictest25602-21iostat_ios12:25:201
295399447,8cyclictest1862-21df_inode08:15:191
295399445,14cyclictest0-21swapper/108:55:581
295399438,7cyclictest30552-21iostat_ios12:35:231
295399438,16cyclictest14438-21awk10:55:181
295399436,7cyclictest11202-21apt-config08:35:011
136591430,0ptp4l401ktimersoftd/311:45:383
295399429,14cyclictest6487-21apt-config10:40:001
295399427,8cyclictest8208-21df09:35:151
136591420,0ptp4l401ktimersoftd/308:12:433
295399417,8cyclictest30171-21meminfo08:05:221
295399414,15cyclictest0-21swapper/110:40:151
136591410,33ptp4l0-21swapper/307:10:013
136591410,0ptp4l401ktimersoftd/307:25:213
2954994033,4cyclictest12280-21cron07:29:592
295399406,14cyclictest18207-21df_inode12:10:171
136591400,0ptp4l401ktimersoftd/310:50:173
133291400,1getstats0-21swapper/308:06:413
295399396,7cyclictest24293-21hddtemp_smartct11:15:191
295399387,7cyclictest16953-21ls12:05:291
295399385,7cyclictest21220-21meminfo12:15:221
136591370,1ptp4l5035-21ls11:40:263
295399369,17cyclictest31519-21grep08:10:141
295399368,17cyclictest0-21swapper/108:05:001
295399368,12cyclictest22879-21df12:20:171
295399364,15cyclictest0-21swapper/109:25:151
136591360,0ptp4l401ktimersoftd/311:10:013
2955993527,5cyclictest41-21ksoftirqd/309:38:103
2954993528,5cyclictest33-21ksoftirqd/208:33:352
295399358,12cyclictest8305-21hddtemp_smartct07:20:181
295399358,12cyclictest27611-21df12:30:171
295399358,12cyclictest1184-21perl09:20:191
133291350,2getstats0-21swapper/309:11:423
2955993427,4cyclictest41-21ksoftirqd/307:20:203
2953993428,4cyclictest25-21ksoftirqd/110:51:231
2952993428,4cyclictest9-21ksoftirqd/010:20:490
2955993328,3cyclictest41-21ksoftirqd/312:33:433
2955993327,4cyclictest41-21ksoftirqd/310:29:063
2954993327,4cyclictest33-21ksoftirqd/210:46:472
2954993327,4cyclictest33-21ksoftirqd/209:56:322
2954993327,4cyclictest33-21ksoftirqd/209:30:362
2954993327,4cyclictest33-21ksoftirqd/208:12:182
295399338,11cyclictest21804-21diskstats11:10:151
295399333,17cyclictest0-21swapper/107:10:231
2953993327,4cyclictest25-21ksoftirqd/110:31:181
2953993327,4cyclictest25-21ksoftirqd/110:31:171
2952993327,4cyclictest9-21ksoftirqd/012:33:120
2952993327,4cyclictest9-21ksoftirqd/009:49:320
2952993327,4cyclictest9-21ksoftirqd/008:48:300
2952993327,4cyclictest9-21ksoftirqd/008:41:110
2954993226,4cyclictest33-21ksoftirqd/212:26:162
2954993226,4cyclictest33-21ksoftirqd/211:47:322
2954993226,4cyclictest33-21ksoftirqd/211:02:432
2954993226,4cyclictest33-21ksoftirqd/210:14:542
2954993226,4cyclictest33-21ksoftirqd/209:51:142
2954993226,4cyclictest33-21ksoftirqd/208:17:192
2954993226,4cyclictest33-21ksoftirqd/207:57:292
2954993226,4cyclictest33-21ksoftirqd/207:30:502
2952993226,4cyclictest9-21ksoftirqd/012:25:410
2952993226,4cyclictest9-21ksoftirqd/011:04:580
2952993226,4cyclictest9-21ksoftirqd/010:00:510
2952993226,4cyclictest9-21ksoftirqd/010:00:510
2952993226,4cyclictest9-21ksoftirqd/009:34:110
2952993226,4cyclictest9-21ksoftirqd/009:25:360
2952993226,4cyclictest9-21ksoftirqd/008:04:490
2952993226,4cyclictest9-21ksoftirqd/007:39:220
2952993226,4cyclictest9-21ksoftirqd/007:27:420
2952993225,5cyclictest9-21ksoftirqd/008:56:490
2955993125,4cyclictest41-21ksoftirqd/310:36:523
2955993125,4cyclictest41-21ksoftirqd/309:31:453
2955993125,4cyclictest41-21ksoftirqd/308:01:233
2955993125,4cyclictest41-21ksoftirqd/307:58:363
2955993124,5cyclictest41-21ksoftirqd/310:07:363
2955993124,5cyclictest41-21ksoftirqd/309:02:533
2954993126,3cyclictest33-21ksoftirqd/210:31:402
2954993126,3cyclictest33-21ksoftirqd/210:31:402
2954993125,4cyclictest33-21ksoftirqd/212:14:362
2954993125,4cyclictest33-21ksoftirqd/210:36:392
295399315,16cyclictest0-21swapper/110:28:201
2953993125,4cyclictest25-21ksoftirqd/110:11:231
2952993126,3cyclictest9-21ksoftirqd/012:08:570
2952993126,3cyclictest9-21ksoftirqd/011:14:530
2952993126,3cyclictest9-21ksoftirqd/009:20:230
2952993126,3cyclictest9-21ksoftirqd/008:35:120
2952993125,4cyclictest9-21ksoftirqd/011:55:570
2952993125,4cyclictest9-21ksoftirqd/011:49:070
2952993125,4cyclictest9-21ksoftirqd/010:19:410
2952993125,4cyclictest9-21ksoftirqd/009:58:260
2952993125,4cyclictest9-21ksoftirqd/009:44:070
2952993125,4cyclictest9-21ksoftirqd/008:23:350
2952993125,4cyclictest9-21ksoftirqd/007:20:000
230082310,7sleep2321ktimersoftd/210:05:222
2955993025,3cyclictest41-21ksoftirqd/310:55:143
2955993024,4cyclictest41-21ksoftirqd/312:37:203
2955993024,4cyclictest41-21ksoftirqd/310:10:053
2955993024,4cyclictest41-21ksoftirqd/309:49:513
2955993024,4cyclictest41-21ksoftirqd/308:58:233
2955993024,4cyclictest41-21ksoftirqd/308:23:413
2955993024,4cyclictest41-21ksoftirqd/307:46:193
2954993024,4cyclictest33-21ksoftirqd/211:39:262
2954993024,4cyclictest33-21ksoftirqd/209:47:052
2954993024,4cyclictest33-21ksoftirqd/209:36:112
2954993024,4cyclictest33-21ksoftirqd/209:11:462
2954993024,4cyclictest33-21ksoftirqd/208:29:102
2954993024,4cyclictest33-21ksoftirqd/207:43:172
295399308,9cyclictest24469-21cpuspeed07:55:171
295399304,15cyclictest0-21swapper/109:40:211
2953993024,4cyclictest25-21ksoftirqd/109:33:001
2953993024,4cyclictest25-21ksoftirqd/107:17:531
2952993025,3cyclictest9-21ksoftirqd/012:36:130
2952993025,3cyclictest9-21ksoftirqd/007:30:130
2952993025,3cyclictest9-21ksoftirqd/007:11:450
2952993024,4cyclictest9-21ksoftirqd/011:52:080
2952993024,4cyclictest9-21ksoftirqd/011:22:430
2952993024,4cyclictest9-21ksoftirqd/010:55:310
2952993024,4cyclictest9-21ksoftirqd/010:06:000
2952993024,4cyclictest9-21ksoftirqd/009:35:080
2952993024,4cyclictest9-21ksoftirqd/009:13:290
2952993024,4cyclictest9-21ksoftirqd/009:05:390
2952993024,4cyclictest9-21ksoftirqd/009:04:550
2952993024,4cyclictest9-21ksoftirqd/008:51:060
2952993024,4cyclictest9-21ksoftirqd/007:42:230
136591300,0ptp4l401ktimersoftd/311:39:413
136591300,0ptp4l401ktimersoftd/309:25:233
2955992924,3cyclictest41-21ksoftirqd/311:56:103
2955992924,3cyclictest41-21ksoftirqd/311:32:293
2955992924,3cyclictest41-21ksoftirqd/309:08:353
2955992924,3cyclictest41-21ksoftirqd/308:48:593
2955992924,3cyclictest41-21ksoftirqd/308:40:133
2955992923,4cyclictest41-21ksoftirqd/308:25:063
2955992923,4cyclictest41-21ksoftirqd/308:17:073
2955992923,4cyclictest41-21ksoftirqd/307:34:313
2954992923,4cyclictest33-21ksoftirqd/212:23:432
2954992923,4cyclictest33-21ksoftirqd/211:57:152
2954992923,4cyclictest33-21ksoftirqd/211:32:442
2954992923,4cyclictest33-21ksoftirqd/211:10:372
2954992923,4cyclictest33-21ksoftirqd/210:28:492
2954992923,4cyclictest33-21ksoftirqd/210:19:182
2954992923,4cyclictest33-21ksoftirqd/208:37:442
2954992913,14cyclictest33-21ksoftirqd/207:35:132
295399298,12cyclictest29569-21meminfo11:25:231
2952992923,4cyclictest9-21ksoftirqd/012:21:080
2952992923,4cyclictest9-21ksoftirqd/012:17:390
2952992923,4cyclictest9-21ksoftirqd/012:02:590
2952992923,4cyclictest9-21ksoftirqd/011:05:260
2952992923,4cyclictest9-21ksoftirqd/009:53:210
2952992923,4cyclictest9-21ksoftirqd/008:33:290
136591290,0ptp4l401ktimersoftd/312:22:073
136591290,0ptp4l401ktimersoftd/311:01:093
2955992823,3cyclictest41-21ksoftirqd/310:20:123
2955992822,4cyclictest41-21ksoftirqd/312:11:573
2955992822,4cyclictest41-21ksoftirqd/310:49:563
2955992822,4cyclictest41-21ksoftirqd/310:00:583
2955992822,4cyclictest41-21ksoftirqd/310:00:583
2954992823,3cyclictest33-21ksoftirqd/209:26:212
2954992823,3cyclictest33-21ksoftirqd/208:21:372
2954992822,4cyclictest33-21ksoftirqd/211:18:242
2954992822,4cyclictest33-21ksoftirqd/208:02:512
2953992823,3cyclictest25-21ksoftirqd/107:30:121
2953992822,4cyclictest25-21ksoftirqd/108:54:361
2952992822,4cyclictest9-21ksoftirqd/008:27:240
2952992822,4cyclictest9-21ksoftirqd/007:51:090
2952992822,3cyclictest9-21ksoftirqd/010:12:590
136591280,0ptp4l401ktimersoftd/310:43:563
2955992722,3cyclictest41-21ksoftirqd/312:18:083
2955992722,3cyclictest41-21ksoftirqd/311:24:573
2955992722,3cyclictest41-21ksoftirqd/311:16:563
2955992721,4cyclictest41-21ksoftirqd/308:51:423
2954992722,3cyclictest33-21ksoftirqd/211:54:012
2954992722,3cyclictest33-21ksoftirqd/210:43:322
2954992722,3cyclictest33-21ksoftirqd/209:42:322
2954992722,3cyclictest33-21ksoftirqd/208:57:402
2954992722,3cyclictest33-21ksoftirqd/208:51:482
2954992721,4cyclictest33-21ksoftirqd/212:07:022
2954992721,4cyclictest33-21ksoftirqd/212:04:132
2954992721,4cyclictest33-21ksoftirqd/209:23:302
2954992721,4cyclictest33-21ksoftirqd/209:18:002
2954992721,4cyclictest33-21ksoftirqd/207:24:472
2954992721,4cyclictest33-21ksoftirqd/207:13:242
295399274,13cyclictest10225-21irqstats10:45:211
2952992722,3cyclictest9-21ksoftirqd/010:42:380
2952992721,4cyclictest9-21ksoftirqd/011:29:170
2952992721,4cyclictest9-21ksoftirqd/007:58:470
2952992721,4cyclictest9-21ksoftirqd/007:47:200
2952992721,3cyclictest9-21ksoftirqd/010:48:370
2952992720,5cyclictest9-21ksoftirqd/008:16:210
136591270,0ptp4l401ktimersoftd/308:31:243
2955992621,3cyclictest41-21ksoftirqd/310:16:233
2955992620,4cyclictest41-21ksoftirqd/312:04:143
2955992620,4cyclictest41-21ksoftirqd/309:57:463
2954992621,3cyclictest33-21ksoftirqd/212:31:002
2954992621,3cyclictest33-21ksoftirqd/208:47:192
2954992621,3cyclictest33-21ksoftirqd/207:18:012
2954992620,4cyclictest33-21ksoftirqd/211:29:152
2954992620,4cyclictest33-21ksoftirqd/209:04:332
2954992620,4cyclictest33-21ksoftirqd/208:42:542
2954992620,4cyclictest33-21ksoftirqd/207:50:022
2952992621,3cyclictest9-21ksoftirqd/011:39:280
2952992621,3cyclictest9-21ksoftirqd/009:17:000
2952992621,3cyclictest9-21ksoftirqd/008:05:590
2952992620,4cyclictest9-21ksoftirqd/010:31:040
2952992620,4cyclictest9-21ksoftirqd/010:31:040
136591260,0ptp4l401ktimersoftd/312:26:553
2955992518,4cyclictest41-21ksoftirqd/307:37:173
2954992519,4cyclictest33-21ksoftirqd/211:24:502
2954992519,4cyclictest33-21ksoftirqd/208:05:122
2954992519,4cyclictest33-21ksoftirqd/207:49:102
2953992518,5cyclictest25-21ksoftirqd/108:24:111
2952992520,3cyclictest9-21ksoftirqd/010:29:440
2952992520,3cyclictest9-21ksoftirqd/007:23:450
2952992519,4cyclictest9-21ksoftirqd/012:13:540
136591250,1ptp4l15703-21irqrtprio09:50:213
2954992419,3cyclictest33-21ksoftirqd/211:00:002
2954992418,4cyclictest33-21ksoftirqd/212:19:062
2952992419,3cyclictest9-21ksoftirqd/011:40:240
136591240,0ptp4l401ktimersoftd/307:53:553
2954992318,3cyclictest33-21ksoftirqd/211:44:102
2952992317,4cyclictest9-21ksoftirqd/011:19:300
136591230,0ptp4l401ktimersoftd/311:25:143
136591230,0ptp4l401ktimersoftd/308:35:223
136591230,0ptp4l401ktimersoftd/307:15:243
2954992216,4cyclictest33-21ksoftirqd/210:51:362
2953992213,6cyclictest14012-21/usr/sbin/munin12:00:201
2952992217,3cyclictest9-21ksoftirqd/008:14:140
2954992115,4cyclictest33-21ksoftirqd/210:22:032
2952992116,3cyclictest9-21ksoftirqd/010:38:080
2954992014,4cyclictest33-21ksoftirqd/210:04:062
2954992014,4cyclictest33-21ksoftirqd/210:04:062
2952992014,3cyclictest11656-21/usr/sbin/munin10:50:140
2954991814,3cyclictest33-21ksoftirqd/212:35:242
2954991812,4cyclictest33-21ksoftirqd/209:05:452
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional