You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-02 - 07:09
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackdslot2.osadl.org (updated Wed Jul 02, 2025 00:45:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
216112230201,20sleep30-21swapper/319:09:593
216092210177,21sleep10-21swapper/119:09:571
214572206175,20sleep00-21swapper/019:07:590
213572204170,23sleep20-21swapper/219:06:482
137391149124,13phc2sys0-21swapper/319:10:013
444921350,8sleep12190699cyclictest19:40:161
2190599583,23cyclictest25900-21dpkg20:25:140
43372560,3sleep00-21swapper/000:10:180
2190599552,27cyclictest20996-21meminfo21:20:220
2190599532,32cyclictest30916-21cut22:50:180
2190599510,32cyclictest2960-21sendmail_mailqu00:05:250
137291500,1ptp4l401ktimersoftd/323:01:523
307742480,1sleep00-21swapper/019:25:270
21908994641,3cyclictest28905-21sh23:55:013
2190599451,21cyclictest24179-21latency_hist19:15:010
2190799442,39cyclictest24056-21cron19:15:002
2190599443,23cyclictest16441-21cat21:10:270
137291440,0ptp4l401ktimersoftd/322:43:343
293192430,2sleep10-21swapper/123:55:191
137291430,0ptp4l401ktimersoftd/322:22:473
137291430,0ptp4l401ktimersoftd/322:10:033
2190899422,37cyclictest2849-21cron20:45:003
2190699422,37cyclictest1290-21cron00:05:001
137291420,0ptp4l401ktimersoftd/323:10:283
21908994035,3cyclictest41-21ksoftirqd/300:00:013
21906994033,4cyclictest11073-21cat00:25:001
21906994030,6cyclictest25-21ksoftirqd/122:45:151
101562400,1sleep00-21swapper/023:15:130
21908993934,3cyclictest41-21ksoftirqd/323:20:013
21908993934,3cyclictest41-21ksoftirqd/321:50:003
21908993934,3cyclictest41-21ksoftirqd/320:40:003
137291390,0ptp4l401ktimersoftd/320:45:313
21908993833,3cyclictest18378-21sh00:40:003
21906993832,3cyclictest31633-21cron21:45:001
21906993832,3cyclictest31633-21cron21:45:001
2190699380,36cyclictest0-21swapper/120:41:091
2190699380,36cyclictest0-21swapper/119:36:091
21905993832,4cyclictest9-21ksoftirqd/021:07:290
137291380,1ptp4l401ktimersoftd/319:49:003
21908993731,4cyclictest41-21ksoftirqd/300:12:373
21908993731,3cyclictest41-21ksoftirqd/323:30:233
21908993731,3cyclictest41-21ksoftirqd/321:15:013
21908993731,3cyclictest41-21ksoftirqd/319:25:003
21907993731,4cyclictest33-21ksoftirqd/223:49:022
21906993731,3cyclictest23017-21sh22:34:591
2190699370,35cyclictest0-21swapper/119:30:491
21905993731,4cyclictest9-21ksoftirqd/020:52:160
21908993630,3cyclictest41-21ksoftirqd/322:35:223
21907993630,4cyclictest33-21ksoftirqd/222:44:432
21907993630,4cyclictest33-21ksoftirqd/221:58:052
21907993628,5cyclictest33-21ksoftirqd/222:20:012
21906993631,3cyclictest25-21ksoftirqd/120:55:121
21905993630,4cyclictest9-21ksoftirqd/023:04:110
21905993630,4cyclictest9-21ksoftirqd/000:04:160
2190599360,33cyclictest24872-21find20:20:240
21908993532,2cyclictest41-21ksoftirqd/323:10:003
21908993530,3cyclictest3949-21dump-pmu-power00:10:003
21908993529,4cyclictest41-21ksoftirqd/320:19:423
21908993529,3cyclictest41-21ksoftirqd/321:41:073
21908993529,3cyclictest41-21ksoftirqd/321:41:073
21908993523,9cyclictest41-21ksoftirqd/321:25:283
21907993529,4cyclictest33-21ksoftirqd/221:02:122
21906993529,4cyclictest25-21ksoftirqd/100:29:041
21906993529,4cyclictest25-21ksoftirqd/100:18:331
21905993530,3cyclictest9-21ksoftirqd/023:52:450
21905993529,4cyclictest9-21ksoftirqd/022:01:480
21905993529,4cyclictest9-21ksoftirqd/020:42:100
21905993529,4cyclictest16150-21cron20:05:000
21905993529,4cyclictest16150-21cron20:05:000
21905993516,16cyclictest9-21ksoftirqd/021:50:120
21905993513,19cyclictest7539-21cut23:09:590
21905993513,19cyclictest28162-21sed19:20:260
21905993512,20cyclictest10407-21expr21:00:150
21905993512,19cyclictest23830-21sed21:25:270
42098340,2rtkit-daemon419-21rtkit-daemon22:14:432
21908993429,3cyclictest41-21ksoftirqd/322:29:593
21908993429,3cyclictest41-21ksoftirqd/321:51:573
21908993429,3cyclictest41-21ksoftirqd/320:10:003
21908993428,4cyclictest41-21ksoftirqd/323:22:523
21908993428,4cyclictest41-21ksoftirqd/321:57:433
21908993428,4cyclictest41-21ksoftirqd/320:57:053
21908993428,4cyclictest41-21ksoftirqd/319:57:073
21908993428,4cyclictest41-21ksoftirqd/319:29:033
21908993428,4cyclictest41-21ksoftirqd/300:22:363
21908993428,3cyclictest41-21ksoftirqd/320:10:213
21907993428,4cyclictest33-21ksoftirqd/223:58:322
21907993428,4cyclictest33-21ksoftirqd/220:19:012
21907993428,4cyclictest33-21ksoftirqd/200:38:082
21906993428,4cyclictest25-21ksoftirqd/123:43:191
21906993428,4cyclictest25-21ksoftirqd/122:28:291
21906993428,4cyclictest25-21ksoftirqd/121:39:191
21906993428,4cyclictest25-21ksoftirqd/121:15:081
21906993427,4cyclictest25-21ksoftirqd/100:30:121
21905993429,3cyclictest9-21ksoftirqd/023:34:500
21905993429,3cyclictest9-21ksoftirqd/020:39:200
21905993429,3cyclictest9-21ksoftirqd/019:18:090
21905993428,4cyclictest9-21ksoftirqd/023:46:560
21905993428,4cyclictest9-21ksoftirqd/022:58:450
21905993428,4cyclictest9-21ksoftirqd/022:41:360
21905993428,4cyclictest9-21ksoftirqd/020:11:450
21905993428,4cyclictest9-21ksoftirqd/000:29:360
21905993428,4cyclictest9-21ksoftirqd/000:24:060
21905993427,4cyclictest9-21ksoftirqd/020:33:110
21905993413,18cyclictest18934-21mailstats21:15:270
21905993412,19cyclictest4548-21expr19:40:170
2190599340,3cyclictest169992chrt23:29:340
21908993328,3cyclictest41-21ksoftirqd/300:15:113
21907993329,2cyclictest33-21ksoftirqd/221:10:552
21907993328,3cyclictest3840-21cron19:40:002
21907993328,3cyclictest33-21ksoftirqd/223:55:012
21907993328,3cyclictest33-21ksoftirqd/223:42:372
21907993328,3cyclictest33-21ksoftirqd/222:56:492
21907993327,4cyclictest33-21ksoftirqd/221:48:312
21907993327,4cyclictest33-21ksoftirqd/220:45:572
21907993327,4cyclictest33-21ksoftirqd/220:25:072
21907993327,4cyclictest33-21ksoftirqd/219:28:022
21907993327,4cyclictest33-21ksoftirqd/200:34:312
21906993327,4cyclictest25-21ksoftirqd/123:03:441
21906993327,4cyclictest25-21ksoftirqd/121:59:171
21906993327,4cyclictest25-21ksoftirqd/119:48:231
21906993327,4cyclictest25-21ksoftirqd/119:24:251
2190599338,22cyclictest22199-21tail22:30:260
21905993328,3cyclictest9-21ksoftirqd/023:40:540
21905993327,4cyclictest9-21ksoftirqd/022:10:350
21905993327,4cyclictest9-21ksoftirqd/019:31:250
21905993327,4cyclictest9-21ksoftirqd/000:39:230
2190599330,3cyclictest8172-21if_enp4s023:10:170
2190599330,3cyclictest4794-21sed21:55:180
2190599330,3cyclictest4577-21sed20:45:260
2190599330,3cyclictest11241-21taskset19:53:060
21908993229,2cyclictest41-21ksoftirqd/321:25:013
21908993228,3cyclictest41-21ksoftirqd/320:00:223
21908993228,3cyclictest41-21ksoftirqd/320:00:223
21908993228,3cyclictest41-21ksoftirqd/319:40:003
21908993227,3cyclictest41-21ksoftirqd/321:15:013
21908993226,4cyclictest41-21ksoftirqd/322:07:173
21908993226,4cyclictest41-21ksoftirqd/321:36:123
21908993226,4cyclictest41-21ksoftirqd/320:20:383
21908993226,4cyclictest41-21ksoftirqd/319:53:423
2190899321,3cyclictest121rcu_preempt20:30:003
21907993227,3cyclictest33-21ksoftirqd/223:40:002
21907993227,3cyclictest33-21ksoftirqd/221:24:012
21907993227,3cyclictest33-21ksoftirqd/220:35:462
21907993227,3cyclictest33-21ksoftirqd/220:06:462
21907993226,4cyclictest33-21ksoftirqd/223:01:002
21907993226,4cyclictest33-21ksoftirqd/222:48:242
21907993226,4cyclictest33-21ksoftirqd/221:09:482
21907993226,4cyclictest33-21ksoftirqd/219:58:192
21907993226,4cyclictest33-21ksoftirqd/219:43:192
21907993226,4cyclictest33-21ksoftirqd/200:26:322
21907993226,4cyclictest33-21ksoftirqd/200:22:242
21907993225,4cyclictest33-21ksoftirqd/200:08:222
21906993226,4cyclictest25-21ksoftirqd/120:54:021
21906993226,4cyclictest25-21ksoftirqd/119:53:081
21906993226,3cyclictest11152-21idleruntime-cro22:10:001
21906993225,4cyclictest25-21ksoftirqd/122:11:091
21905993227,3cyclictest9-21ksoftirqd/023:35:520
21905993226,4cyclictest9-21ksoftirqd/022:16:530
21905993226,4cyclictest9-21ksoftirqd/021:31:360
21905993226,4cyclictest9-21ksoftirqd/019:59:210
21905993226,4cyclictest9-21ksoftirqd/019:36:220
21905993226,4cyclictest9-21ksoftirqd/000:30:040
21905993212,17cyclictest7535-21interrupts19:45:210
2190599320,4cyclictest17818-21sed22:20:300
2190599320,3cyclictest23033-21taskset20:17:060
2190599320,2cyclictest24184-21irqstats22:35:220
2190599320,2cyclictest0-21swapper/021:40:260
2190599320,2cyclictest0-21swapper/021:40:260
137291320,2ptp4l0-21swapper/322:56:213
21908993127,3cyclictest41-21ksoftirqd/323:47:493
21908993127,3cyclictest41-21ksoftirqd/322:50:013
21908993127,2cyclictest41-21ksoftirqd/321:05:003
21908993126,3cyclictest41-21ksoftirqd/320:50:183
21908993125,4cyclictest41-21ksoftirqd/321:32:543
21908993125,4cyclictest41-21ksoftirqd/319:18:483
21907993126,3cyclictest33-21ksoftirqd/221:16:112
21907993125,4cyclictest33-21ksoftirqd/223:21:022
21907993125,4cyclictest33-21ksoftirqd/222:28:472
21907993125,4cyclictest33-21ksoftirqd/222:04:032
21907993125,4cyclictest33-21ksoftirqd/221:35:392
21907993125,4cyclictest33-21ksoftirqd/221:29:012
21907993125,4cyclictest33-21ksoftirqd/219:21:202
21907993125,4cyclictest33-21ksoftirqd/200:10:492
21906993126,3cyclictest25-21ksoftirqd/120:50:011
21906993125,4cyclictest25-21ksoftirqd/123:39:531
21906993125,4cyclictest25-21ksoftirqd/122:36:111
21906993125,4cyclictest25-21ksoftirqd/121:03:251
21906993125,4cyclictest25-21ksoftirqd/100:38:101
2190599317,21cyclictest20105-21cat22:25:310
21905993126,3cyclictest9-21ksoftirqd/023:57:390
21905993125,4cyclictest9-21ksoftirqd/000:18:450
2190599310,1cyclictest0-21swapper/022:48:180
137291310,0ptp4l401ktimersoftd/323:25:233
21908993025,3cyclictest41-21ksoftirqd/322:33:033
21908993025,3cyclictest41-21ksoftirqd/319:45:013
21908993024,4cyclictest41-21ksoftirqd/322:52:273
21908993024,4cyclictest41-21ksoftirqd/322:17:083
21907993025,3cyclictest33-21ksoftirqd/222:10:002
21907993025,3cyclictest33-21ksoftirqd/220:14:062
21907993024,4cyclictest33-21ksoftirqd/223:28:322
21907993024,4cyclictest33-21ksoftirqd/221:41:092
21907993024,4cyclictest33-21ksoftirqd/221:41:092
21907993024,4cyclictest33-21ksoftirqd/220:32:012
21906993025,3cyclictest25-21ksoftirqd/123:15:121
21906993025,3cyclictest25-21ksoftirqd/122:04:551
21906993025,3cyclictest25-21ksoftirqd/120:15:111
21906993024,4cyclictest25-21ksoftirqd/121:46:291
21906993024,4cyclictest25-21ksoftirqd/120:00:421
21906993024,4cyclictest25-21ksoftirqd/120:00:421
21906993024,4cyclictest25-21ksoftirqd/100:07:021
2190699300,2cyclictest0-21swapper/123:07:291
2190599303,21cyclictest16400-21apt-get20:05:140
21905993024,4cyclictest9-21ksoftirqd/021:38:500
2190599300,1cyclictest0-21swapper/022:05:220
137291300,8ptp4l401ktimersoftd/323:35:083
137291300,0ptp4l401ktimersoftd/321:09:493
21908992923,4cyclictest41-21ksoftirqd/323:40:053
21908992923,4cyclictest41-21ksoftirqd/300:26:253
21907992924,3cyclictest33-21ksoftirqd/222:33:292
21907992924,3cyclictest33-21ksoftirqd/221:34:012
21907992924,3cyclictest33-21ksoftirqd/220:43:102
21907992924,3cyclictest33-21ksoftirqd/219:32:492
21907992924,3cyclictest33-21ksoftirqd/200:18:312
21907992923,4cyclictest33-21ksoftirqd/222:51:212
21907992923,4cyclictest33-21ksoftirqd/220:55:562
21907992923,4cyclictest33-21ksoftirqd/200:00:492
21906992924,3cyclictest25-21ksoftirqd/121:26:511
21906992923,4cyclictest25-21ksoftirqd/123:53:021
21906992923,4cyclictest25-21ksoftirqd/123:34:431
21906992923,4cyclictest25-21ksoftirqd/123:26:061
21906992923,4cyclictest25-21ksoftirqd/121:52:271
21906992923,4cyclictest25-21ksoftirqd/119:10:441
21906992922,4cyclictest25-21ksoftirqd/100:15:001
2190599297,19cyclictest8217-21df20:55:160
2190599291,24cyclictest13202-21iostat_ios23:20:190
137291290,0ptp4l401ktimersoftd/322:01:583
137291290,0ptp4l401ktimersoftd/319:32:123
21908992824,2cyclictest41-21ksoftirqd/300:00:213
21908992823,3cyclictest41-21ksoftirqd/300:30:273
21907992823,3cyclictest33-21ksoftirqd/221:51:462
21907992823,3cyclictest33-21ksoftirqd/220:21:192
21907992823,3cyclictest33-21ksoftirqd/219:51:392
21907992822,4cyclictest33-21ksoftirqd/223:12:022
21907992822,4cyclictest33-21ksoftirqd/222:36:452
21906992823,3cyclictest25-21ksoftirqd/121:12:161
21906992823,3cyclictest25-21ksoftirqd/120:10:011
21906992822,4cyclictest25-21ksoftirqd/122:42:221
21906992822,4cyclictest25-21ksoftirqd/121:21:261
2190599280,1cyclictest0-21swapper/021:45:140
21908992720,6cyclictest41-21ksoftirqd/320:30:213
21907992721,4cyclictest33-21ksoftirqd/223:16:002
21907992721,4cyclictest33-21ksoftirqd/223:05:362
21907992721,4cyclictest33-21ksoftirqd/219:16:062
21906992721,4cyclictest25-21ksoftirqd/119:57:331
21907992620,4cyclictest33-21ksoftirqd/220:53:032
21907992620,4cyclictest33-21ksoftirqd/220:04:002
21907992620,4cyclictest33-21ksoftirqd/220:04:002
21906992622,3cyclictest25-21ksoftirqd/123:22:491
21906992620,4cyclictest25-21ksoftirqd/120:37:171
21906992620,4cyclictest25-21ksoftirqd/119:26:321
21906992620,4cyclictest25-21ksoftirqd/119:15:571
21906992520,3cyclictest25-21ksoftirqd/121:07:541
21906992519,4cyclictest25-21ksoftirqd/120:09:531
21907992419,3cyclictest33-21ksoftirqd/222:23:072
21907992419,3cyclictest33-21ksoftirqd/219:48:022
21906992419,3cyclictest25-21ksoftirqd/122:23:021
21906992418,4cyclictest25-21ksoftirqd/122:52:211
21906992418,4cyclictest25-21ksoftirqd/120:29:031
21906992318,3cyclictest25-21ksoftirqd/122:15:041
21906992318,3cyclictest25-21ksoftirqd/120:32:041
21906992217,3cyclictest25-21ksoftirqd/122:55:421
21906992216,4cyclictest25-21ksoftirqd/123:47:251
21907992015,3cyclictest33-21ksoftirqd/223:30:332
21906992015,3cyclictest26216-21timerandwakeup21:30:271
21906991710,4cyclictest24640-21needreboot20:20:231
21906991610,3cyclictest7703-21wc23:10:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional