You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-06 - 11:41
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackdslot3.osadl.org (updated Sun Jul 06, 2025 00:45:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
137191207172,23phc2sys0-21swapper/319:05:083
128772206173,22sleep20-21swapper/219:09:212
126252205172,22sleep10-21swapper/119:06:121
125602202168,23sleep00-21swapper/019:05:290
2135621690,5sleep01321999cyclictest23:55:230
1757621500,6sleep01321999cyclictest22:40:210
1322199641,61cyclictest32127-21kworker/2:122:10:272
1322299613,17cyclictest26441-21/usr/sbin/munin00:05:223
1322299574,16cyclictest19943-21unixbench_singl19:20:273
102012560,2sleep00-21swapper/020:10:160
1322299503,16cyclictest7413-21/usr/sbin/munin22:20:263
1322299503,16cyclictest7413-21/usr/sbin/munin22:20:253
1322299473,14cyclictest26285-21memory21:50:213
13219994538,4cyclictest15304-21cron21:30:000
59298440,2rtkit-daemon591-21rtkit-daemon20:18:561
136991440,0ptp4l401ktimersoftd/323:59:483
1322299442,14cyclictest0-21swapper/323:30:233
1292624432,6sleep00-21swapper/019:10:000
59298430,2rtkit-daemon591-21rtkit-daemon23:51:150
59298430,2rtkit-daemon591-21rtkit-daemon23:51:150
59298430,2rtkit-daemon591-21rtkit-daemon20:36:150
59298420,2rtkit-daemon591-21rtkit-daemon23:35:550
59298420,2rtkit-daemon591-21rtkit-daemon19:37:551
13221994211,28cyclictest31864-21ls19:45:272
136991410,0ptp4l401ktimersoftd/323:07:283
136991410,0ptp4l401ktimersoftd/321:07:213
136991410,0ptp4l401ktimersoftd/319:39:593
136991400,0ptp4l401ktimersoftd/320:40:243
1322199402,36cyclictest26466-21kworker/2:021:20:192
1322199402,36cyclictest26466-21kworker/2:021:20:192
1322199401,37cyclictest26466-21kworker/2:021:29:272
13220994035,3cyclictest25-21ksoftirqd/123:20:001
13220994035,3cyclictest25-21ksoftirqd/100:05:001
13220994034,3cyclictest25-21ksoftirqd/121:30:001
1322299394,7cyclictest16367-21users20:20:283
13220993933,3cyclictest25-21ksoftirqd/121:00:001
13220993933,3cyclictest1297-21runrttasks22:04:061
13219993934,3cyclictest9-21ksoftirqd/021:20:000
13222993833,3cyclictest41-21ksoftirqd/300:20:003
1322199383,33cyclictest5778-21kworker/2:020:20:192
1322199382,34cyclictest26466-21kworker/2:021:50:122
1322199382,26cyclictest12992-21kworker/2:219:16:352
1322099383,4cyclictest25-21ksoftirqd/122:10:001
13220993834,3cyclictest25-21ksoftirqd/121:40:001
13220993832,3cyclictest25-21ksoftirqd/119:45:181
1322299373,7cyclictest7992-21perl20:05:173
13222993731,4cyclictest41-21ksoftirqd/319:30:003
1322199375,31cyclictest19763-21kworker/2:220:31:152
13221993730,5cyclictest33-21ksoftirqd/222:33:402
1322199372,33cyclictest12992-21kworker/2:219:42:442
1322199371,15cyclictest20075-21kworker/2:000:02:592
1322199370,36cyclictest20075-21kworker/2:000:27:552
1322099378,4cyclictest25-21ksoftirqd/122:10:281
13220993732,3cyclictest25-21ksoftirqd/120:30:001
13220993732,3cyclictest25-21ksoftirqd/119:50:141
13220993732,3cyclictest25-21ksoftirqd/119:30:001
13219993732,3cyclictest9-21ksoftirqd/022:20:000
1322199365,29cyclictest32127-21kworker/2:122:22:352
1322199365,29cyclictest32127-21kworker/2:122:22:352
1322199363,32cyclictest12992-21kworker/2:219:26:522
1322199362,32cyclictest26466-21kworker/2:021:10:112
1322199362,32cyclictest11736-21kworker/2:222:39:192
1322199361,34cyclictest32127-21kworker/2:122:26:232
13220993631,3cyclictest25-21ksoftirqd/123:20:261
13220993631,3cyclictest25-21ksoftirqd/121:55:151
13220993631,3cyclictest25-21ksoftirqd/119:25:011
13220993630,4cyclictest25-21ksoftirqd/122:33:231
13220993630,3cyclictest25-21ksoftirqd/122:25:011
13220993630,3cyclictest25-21ksoftirqd/122:25:001
13220993630,3cyclictest25-21ksoftirqd/120:54:591
13220993630,3cyclictest25-21ksoftirqd/100:40:001
13222993529,4cyclictest41-21ksoftirqd/322:56:163
13222993528,4cyclictest41-21ksoftirqd/319:34:533
1322199355,29cyclictest26466-21kworker/2:020:52:362
1322199355,29cyclictest12992-21kworker/2:219:38:322
1322199354,21cyclictest20075-21kworker/2:000:21:472
1322199353,31cyclictest26466-21kworker/2:021:00:512
1322199353,31cyclictest11736-21kworker/2:222:40:472
1322199353,19cyclictest20075-21kworker/2:023:55:432
13221993529,4cyclictest33-21ksoftirqd/221:56:352
13221993529,3cyclictest33-21ksoftirqd/221:18:122
13221993528,5cyclictest33-21ksoftirqd/200:11:052
1322199352,32cyclictest26466-21kworker/2:020:57:242
1322199352,32cyclictest26136-21kworker/2:123:30:592
1322199352,32cyclictest12992-21kworker/2:219:21:562
1322199352,32cyclictest11736-21kworker/2:222:59:312
1322199352,31cyclictest5778-21kworker/2:020:02:272
1322199352,31cyclictest26466-21kworker/2:021:46:562
1322199351,33cyclictest26136-21kworker/2:123:28:032
1322199351,32cyclictest5778-21kworker/2:020:19:312
1322199351,32cyclictest26466-21kworker/2:021:05:472
1322199351,32cyclictest11736-21kworker/2:223:00:072
1322199351,21cyclictest20075-21kworker/2:000:16:032
1322199351,14cyclictest26136-21kworker/2:123:54:592
1322199351,14cyclictest26136-21kworker/2:123:54:592
1322199350,20cyclictest5778-21kworker/2:020:29:512
1322099359,25cyclictest9575-21kworker/1:122:45:471
13220993531,2cyclictest25-21ksoftirqd/100:30:001
13220993530,3cyclictest25-21ksoftirqd/123:55:141
13220993530,3cyclictest25-21ksoftirqd/120:45:001
13220993530,3cyclictest25-21ksoftirqd/119:40:211
13220993530,3cyclictest25-21ksoftirqd/119:30:211
13220993530,3cyclictest25-21ksoftirqd/119:15:001
13220993530,3cyclictest25-21ksoftirqd/100:15:141
13220993529,4cyclictest25-21ksoftirqd/122:17:161
13220993529,4cyclictest25-21ksoftirqd/121:46:541
13220993529,4cyclictest25-21ksoftirqd/120:32:041
13220993529,4cyclictest25-21ksoftirqd/100:11:241
13220993529,3cyclictest25-21ksoftirqd/122:55:141
13220993529,3cyclictest25-21ksoftirqd/120:05:001
13219993529,4cyclictest9-21ksoftirqd/022:58:380
13219993529,4cyclictest9-21ksoftirqd/022:00:450
13219993529,4cyclictest9-21ksoftirqd/019:36:510
13222993429,3cyclictest41-21ksoftirqd/323:03:213
13222993429,3cyclictest41-21ksoftirqd/320:32:343
13222993428,4cyclictest41-21ksoftirqd/321:02:593
1322199342,31cyclictest20075-21kworker/2:000:30:392
1322199342,30cyclictest5778-21kworker/2:020:10:072
1322199342,30cyclictest32127-21kworker/2:122:06:032
1322199342,30cyclictest26466-21kworker/2:021:41:162
1322199342,30cyclictest19763-21kworker/2:220:39:192
1322199342,30cyclictest11736-21kworker/2:222:45:032
1322199342,19cyclictest8260-21kworker/2:100:36:592
1322199341,32cyclictest26136-21kworker/2:123:19:512
1322199341,32cyclictest11736-21kworker/2:222:52:552
1322199341,31cyclictest26136-21kworker/2:123:20:032
1322199340,33cyclictest26136-21kworker/2:123:06:592
1322199340,32cyclictest19763-21kworker/2:220:44:082
1322099343,3cyclictest121rcu_preempt23:50:241
1322099343,3cyclictest121rcu_preempt23:50:241
13220993430,3cyclictest25-21ksoftirqd/123:00:221
13220993430,2cyclictest25-21ksoftirqd/123:15:001
13220993430,2cyclictest25-21ksoftirqd/121:30:231
13220993429,3cyclictest25-21ksoftirqd/121:45:001
13220993429,2cyclictest25-21ksoftirqd/120:20:141
13220993429,2cyclictest25-21ksoftirqd/120:15:011
13220993428,3cyclictest25-21ksoftirqd/122:25:191
13220993428,3cyclictest25-21ksoftirqd/100:30:011
13220993427,5cyclictest25-21ksoftirqd/120:41:431
1322099342,4cyclictest25-21ksoftirqd/122:55:011
13222993328,3cyclictest41-21ksoftirqd/323:21:213
13222993327,4cyclictest41-21ksoftirqd/300:25:103
1322199335,27cyclictest26136-21kworker/2:123:10:432
1322199333,29cyclictest26136-21kworker/2:123:36:552
1322199333,29cyclictest12992-21kworker/2:219:51:242
1322199332,29cyclictest12992-21kworker/2:219:56:202
1322199331,31cyclictest32127-21kworker/2:122:02:592
1322199331,31cyclictest26466-21kworker/2:021:38:552
1322199331,31cyclictest26466-21kworker/2:020:46:162
1322199331,31cyclictest26136-21kworker/2:123:43:472
1322199331,31cyclictest12992-21kworker/2:219:14:272
1322199331,20cyclictest5778-21kworker/2:020:06:272
13220993328,2cyclictest25-21ksoftirqd/119:15:011
13220993327,4cyclictest25-21ksoftirqd/122:39:531
13220993327,4cyclictest25-21ksoftirqd/100:09:201
13220993327,3cyclictest25-21ksoftirqd/123:45:001
13220993327,3cyclictest25-21ksoftirqd/123:10:001
13219993327,4cyclictest9-21ksoftirqd/022:24:520
13219993327,4cyclictest9-21ksoftirqd/022:24:520
13219993327,4cyclictest9-21ksoftirqd/020:42:220
13219993327,3cyclictest9-21ksoftirqd/000:15:000
13219993326,5cyclictest9-21ksoftirqd/022:35:350
13219993326,5cyclictest9-21ksoftirqd/020:25:530
13222993228,3cyclictest41-21ksoftirqd/320:18:223
13222993227,4cyclictest41-21ksoftirqd/321:59:213
13222993227,3cyclictest41-21ksoftirqd/323:45:003
13222993227,3cyclictest41-21ksoftirqd/300:00:453
13222993226,4cyclictest41-21ksoftirqd/323:47:033
13222993226,4cyclictest41-21ksoftirqd/322:17:023
13222993226,4cyclictest41-21ksoftirqd/321:12:553
13222993226,4cyclictest41-21ksoftirqd/320:25:013
13222993226,4cyclictest41-21ksoftirqd/319:41:273
13222993226,4cyclictest41-21ksoftirqd/319:14:443
13222993226,4cyclictest41-21ksoftirqd/300:13:273
13222993225,5cyclictest41-21ksoftirqd/322:31:453
1322199323,28cyclictest32127-21kworker/2:122:15:312
1322199321,30cyclictest26466-21kworker/2:021:32:272
13220993227,3cyclictest25-21ksoftirqd/122:41:181
13220993227,3cyclictest25-21ksoftirqd/121:20:011
13220993227,3cyclictest25-21ksoftirqd/121:10:221
13220993227,3cyclictest25-21ksoftirqd/121:00:261
13220993227,3cyclictest25-21ksoftirqd/120:40:001
13220993227,3cyclictest25-21ksoftirqd/119:55:191
13220993226,4cyclictest25-21ksoftirqd/123:36:581
13220993226,4cyclictest25-21ksoftirqd/123:29:201
13220993226,3cyclictest25-21ksoftirqd/121:50:131
13219993228,3cyclictest9-21ksoftirqd/022:00:010
13219993227,3cyclictest9-21ksoftirqd/023:42:390
13219993227,3cyclictest9-21ksoftirqd/020:45:200
13219993227,3cyclictest9-21ksoftirqd/019:19:590
13219993227,3cyclictest9-21ksoftirqd/000:38:470
13219993227,3cyclictest9-21ksoftirqd/000:25:190
13219993226,4cyclictest9-21ksoftirqd/023:01:270
13219993226,4cyclictest9-21ksoftirqd/020:08:240
13219993226,4cyclictest9-21ksoftirqd/019:56:530
136991310,1ptp4l391rcuc/320:59:293
13222993126,3cyclictest41-21ksoftirqd/323:26:183
13222993126,3cyclictest41-21ksoftirqd/323:13:583
13222993126,3cyclictest41-21ksoftirqd/321:44:593
13222993126,3cyclictest41-21ksoftirqd/300:23:223
13222993125,4cyclictest41-21ksoftirqd/323:19:313
13222993125,4cyclictest41-21ksoftirqd/321:39:053
13222993125,4cyclictest41-21ksoftirqd/321:21:253
13222993125,4cyclictest41-21ksoftirqd/321:21:253
13222993125,4cyclictest41-21ksoftirqd/320:35:553
1322199312,20cyclictest12992-21kworker/2:219:32:442
1322199311,29cyclictest26136-21kworker/2:123:45:032
1322099314,3cyclictest25-21ksoftirqd/121:20:251
1322099314,3cyclictest25-21ksoftirqd/121:20:251
13220993126,3cyclictest25-21ksoftirqd/123:30:161
13220993126,3cyclictest25-21ksoftirqd/120:05:211
13220993125,4cyclictest25-21ksoftirqd/121:09:581
13220993125,3cyclictest25-21ksoftirqd/100:20:231
13219993126,3cyclictest9-21ksoftirqd/022:46:540
13219993126,3cyclictest9-21ksoftirqd/022:31:420
13219993126,3cyclictest9-21ksoftirqd/021:24:340
13219993126,3cyclictest9-21ksoftirqd/021:24:340
13219993126,3cyclictest9-21ksoftirqd/000:16:050
13219993126,3cyclictest9-21ksoftirqd/000:05:010
13219993125,4cyclictest9-21ksoftirqd/022:28:570
13219993125,4cyclictest9-21ksoftirqd/021:47:500
13219993125,4cyclictest9-21ksoftirqd/020:18:430
13219993125,4cyclictest9-21ksoftirqd/000:22:390
1322299306,10cyclictest18433-21ssh22:40:283
13222993025,3cyclictest41-21ksoftirqd/323:39:013
13220993025,3cyclictest25-21ksoftirqd/123:45:161
13219993027,2cyclictest9-21ksoftirqd/020:50:200
13219993025,3cyclictest9-21ksoftirqd/023:25:420
13219993025,3cyclictest9-21ksoftirqd/021:44:110
13219993025,3cyclictest9-21ksoftirqd/021:34:290
13219993025,3cyclictest9-21ksoftirqd/021:05:000
13219993025,3cyclictest9-21ksoftirqd/020:59:550
13219993025,3cyclictest9-21ksoftirqd/020:31:220
13219993024,4cyclictest9-21ksoftirqd/023:31:400
13219993024,4cyclictest9-21ksoftirqd/023:21:080
13219993024,4cyclictest9-21ksoftirqd/023:05:560
13219993024,4cyclictest9-21ksoftirqd/022:52:200
13219993024,4cyclictest9-21ksoftirqd/021:51:590
13219993024,4cyclictest9-21ksoftirqd/019:52:000
13219993024,3cyclictest9-21ksoftirqd/019:35:010
13222992924,3cyclictest41-21ksoftirqd/323:50:433
13222992924,3cyclictest41-21ksoftirqd/323:50:433
13222992924,3cyclictest41-21ksoftirqd/320:53:143
13222992924,3cyclictest41-21ksoftirqd/320:13:103
13222992923,4cyclictest41-21ksoftirqd/321:15:123
13222992923,4cyclictest41-21ksoftirqd/300:38:583
1322299292,14cyclictest13185-21apt-get21:25:143
1322299290,2cyclictest121rcu_preempt21:30:303
13219992924,3cyclictest9-21ksoftirqd/021:13:020
13219992923,4cyclictest9-21ksoftirqd/023:48:330
13219992923,3cyclictest9-21ksoftirqd/022:12:080
13219992923,3cyclictest9-21ksoftirqd/020:20:190
13222992823,3cyclictest41-21ksoftirqd/319:52:473
1322299282,17cyclictest0-21swapper/322:50:203
13221992822,4cyclictest33-21ksoftirqd/200:08:042
13219992824,3cyclictest9-21ksoftirqd/021:10:000
13219992823,3cyclictest9-21ksoftirqd/022:10:000
13219992823,3cyclictest9-21ksoftirqd/021:36:180
13219992823,3cyclictest9-21ksoftirqd/019:47:470
13219992823,3cyclictest9-21ksoftirqd/000:33:560
13219992822,4cyclictest9-21ksoftirqd/020:04:400
13219992822,3cyclictest20359-21dump-pmu-power19:24:590
136991270,0ptp4l401ktimersoftd/322:14:243
13222992722,3cyclictest41-21ksoftirqd/322:29:163
13222992722,3cyclictest41-21ksoftirqd/319:56:183
13222992721,4cyclictest41-21ksoftirqd/320:02:213
13222992720,4cyclictest23993-21grep21:45:213
13219992722,3cyclictest9-21ksoftirqd/023:10:140
13219992721,4cyclictest9-21ksoftirqd/023:17:110
13219992721,3cyclictest9-21ksoftirqd/000:01:190
13222992621,3cyclictest41-21ksoftirqd/320:47:303
13222992620,4cyclictest41-21ksoftirqd/300:31:163
136991250,0ptp4l401ktimersoftd/322:05:123
136991250,0ptp4l401ktimersoftd/319:45:123
13219992521,2cyclictest17451-21kworker/0:219:40:160
136991240,0ptp4l401ktimersoftd/322:35:143
136991240,0ptp4l401ktimersoftd/319:15:133
136991230,0ptp4l401ktimersoftd/322:45:153
136991230,0ptp4l401ktimersoftd/322:00:023
13219992015,3cyclictest9-21ksoftirqd/019:29:410
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional