You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 04:38
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot3.osadl.org (updated Sun Nov 30, 2025 00:45:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
137191235168,23phc2sys0-21swapper/319:05:393
240862205172,22sleep10-21swapper/119:08:501
240482200167,22sleep00-21swapper/019:08:210
239692200167,22sleep20-21swapper/219:07:222
714921820,6sleep32447199cyclictest19:40:203
714921820,6sleep32447199cyclictest19:40:203
2866421300,6sleep12446999cyclictest22:40:151
2417628363,11sleep30-21swapper/319:10:003
24470995212,5cyclictest33-21ksoftirqd/220:42:282
2447099521,16cyclictest24141-21vmstat21:20:322
2447099521,12cyclictest30110-21irqrtprio19:20:192
2447099512,16cyclictest223-21systemd-journal20:25:582
2447099491,24cyclictest9752-21/usr/sbin/munin00:15:112
136991480,1ptp4l401ktimersoftd/323:17:093
318192470,4sleep031818-21wc21:40:000
24470994610,5cyclictest33-21ksoftirqd/220:35:062
2447099460,8cyclictest9983-21netstat19:45:242
2447099451,16cyclictest20437-21/usr/sbin/munin22:20:282
2447099450,16cyclictest30721-21/usr/sbin/munin21:35:232
136991440,0ptp4l401ktimersoftd/322:33:153
136991440,0ptp4l401ktimersoftd/320:10:033
2447099431,12cyclictest2257-21gnome-shell23:05:252
2447099430,13cyclictest20950-21apt20:10:152
136991430,0ptp4l401ktimersoftd/321:50:523
2447099421,7cyclictest2257-21gnome-shell00:23:582
132691420,6getstats2447199cyclictest21:42:113
2447099411,10cyclictest30781-21egrep22:45:152
2447099410,7cyclictest21590-21diskstats22:25:152
24469994035,3cyclictest25-21ksoftirqd/121:00:001
24468993933,3cyclictest2907-21grep20:40:000
136991390,0ptp4l401ktimersoftd/321:00:493
136991390,0ptp4l401ktimersoftd/319:29:563
2447099387,29cyclictest5967-21kworker/2:219:50:262
24471993730,5cyclictest41-21ksoftirqd/300:02:473
2447099371,7cyclictest22675-21perl23:35:152
2447099371,16cyclictest2902-21awk19:30:252
2447099370,16cyclictest32147-21/usr/sbin/munin21:40:312
24468993731,4cyclictest9-21ksoftirqd/023:46:080
24471993631,3cyclictest41-21ksoftirqd/300:10:023
24471993630,4cyclictest41-21ksoftirqd/322:29:533
24471993630,4cyclictest41-21ksoftirqd/321:16:203
2447099365,30cyclictest5967-21kworker/2:220:55:482
2447099360,34cyclictest22926-21kworker/2:123:55:192
24469993630,4cyclictest25-21ksoftirqd/123:51:311
24469993630,4cyclictest25-21ksoftirqd/119:11:371
24469993629,5cyclictest25-21ksoftirqd/120:19:531
24469993629,5cyclictest25-21ksoftirqd/100:38:251
24468993630,4cyclictest9-21ksoftirqd/023:38:500
59298350,5rtkit-daemon22558-21mailstats22:25:240
24471993530,3cyclictest41-21ksoftirqd/323:09:593
24471993528,5cyclictest41-21ksoftirqd/323:12:293
2447099354,29cyclictest31998-21kworker/2:022:58:442
24470993529,4cyclictest33-21ksoftirqd/223:28:012
24470993529,4cyclictest33-21ksoftirqd/223:17:262
24470993529,4cyclictest33-21ksoftirqd/220:21:022
24470993529,4cyclictest33-21ksoftirqd/200:26:072
24469993530,3cyclictest481-21rs:main1
24469993530,3cyclictest31443-21latency_hist19:25:011
24469993529,4cyclictest25-21ksoftirqd/123:43:461
24469993529,4cyclictest25-21ksoftirqd/123:22:391
24469993529,4cyclictest25-21ksoftirqd/121:00:141
24469993529,4cyclictest25-21ksoftirqd/119:54:311
59298340,1rtkit-daemon0-21swapper/023:30:020
24471993428,4cyclictest41-21ksoftirqd/322:38:243
24471993428,4cyclictest41-21ksoftirqd/300:35:553
2447099345,28cyclictest31998-21kworker/2:022:50:202
2447099343,30cyclictest5967-21kworker/2:221:02:472
24470993428,4cyclictest33-21ksoftirqd/223:13:212
24470993428,4cyclictest33-21ksoftirqd/220:05:352
24470993427,4cyclictest33-21ksoftirqd/219:14:082
2447099341,12cyclictest31998-21kworker/2:021:50:152
24469993429,3cyclictest25-21ksoftirqd/100:25:001
24469993428,4cyclictest25-21ksoftirqd/123:19:461
24469993428,4cyclictest25-21ksoftirqd/122:52:541
24469993428,4cyclictest25-21ksoftirqd/121:44:501
24469993428,4cyclictest25-21ksoftirqd/121:22:111
24469993428,4cyclictest25-21ksoftirqd/119:33:161
24469993428,4cyclictest25-21ksoftirqd/119:16:591
24469993428,4cyclictest25-21ksoftirqd/100:16:551
24469993427,5cyclictest25-21ksoftirqd/122:45:531
24469993427,5cyclictest25-21ksoftirqd/119:48:451
24468993428,4cyclictest9-21ksoftirqd/021:03:010
24468993427,5cyclictest9-21ksoftirqd/000:06:270
136991340,0ptp4l401ktimersoftd/320:25:203
24471993328,3cyclictest41-21ksoftirqd/323:56:393
24471993327,4cyclictest41-21ksoftirqd/322:48:143
24471993327,4cyclictest41-21ksoftirqd/322:20:013
24471993327,4cyclictest41-21ksoftirqd/322:20:013
24471993327,4cyclictest41-21ksoftirqd/322:13:083
24471993327,4cyclictest41-21ksoftirqd/321:29:403
24471993327,4cyclictest41-21ksoftirqd/320:19:203
24471993327,4cyclictest41-21ksoftirqd/320:04:033
24471993327,4cyclictest41-21ksoftirqd/319:39:403
24471993326,5cyclictest41-21ksoftirqd/319:58:343
2447099333,29cyclictest5967-21kworker/2:221:06:312
24470993328,3cyclictest33-21ksoftirqd/223:54:532
24470993327,4cyclictest33-21ksoftirqd/223:24:202
24470993327,4cyclictest33-21ksoftirqd/223:00:492
24470993327,4cyclictest33-21ksoftirqd/222:37:062
24470993327,4cyclictest33-21ksoftirqd/220:33:072
24470993327,4cyclictest33-21ksoftirqd/200:03:162
2447099332,29cyclictest22926-21kworker/2:100:05:152
24469993328,3cyclictest25-21ksoftirqd/122:23:411
24469993327,4cyclictest25-21ksoftirqd/123:33:061
24469993327,4cyclictest25-21ksoftirqd/120:13:511
24469993327,4cyclictest25-21ksoftirqd/119:35:491
24469993327,4cyclictest25-21ksoftirqd/100:03:141
24468993327,4cyclictest9-21ksoftirqd/021:19:180
24468993327,4cyclictest9-21ksoftirqd/020:19:130
24468993327,4cyclictest9-21ksoftirqd/000:19:500
136991330,0ptp4l401ktimersoftd/323:50:023
24471993226,4cyclictest41-21ksoftirqd/320:43:153
24470993227,3cyclictest33-21ksoftirqd/223:30:382
24470993226,4cyclictest33-21ksoftirqd/221:35:002
24470993226,4cyclictest33-21ksoftirqd/200:38:582
24470993218,12cyclictest33-21ksoftirqd/200:15:002
24469993227,3cyclictest25-21ksoftirqd/123:26:441
24469993227,3cyclictest25-21ksoftirqd/119:28:541
24469993226,4cyclictest25-21ksoftirqd/123:47:311
24469993226,4cyclictest25-21ksoftirqd/122:17:521
24469993226,4cyclictest25-21ksoftirqd/122:17:521
24469993225,4cyclictest25-21ksoftirqd/122:07:041
24468993227,3cyclictest9-21ksoftirqd/021:10:070
24468993226,4cyclictest9-21ksoftirqd/023:16:280
24468993226,4cyclictest9-21ksoftirqd/022:59:240
24468993226,4cyclictest9-21ksoftirqd/022:09:170
24468993226,4cyclictest9-21ksoftirqd/021:07:340
24468993226,4cyclictest9-21ksoftirqd/020:02:270
24468993226,4cyclictest9-21ksoftirqd/019:46:550
24468993226,4cyclictest9-21ksoftirqd/019:37:130
24468993226,4cyclictest9-21ksoftirqd/000:11:200
24471993126,3cyclictest41-21ksoftirqd/300:31:253
24471993126,3cyclictest41-21ksoftirqd/300:21:023
24471993126,3cyclictest41-21ksoftirqd/300:08:583
24471993125,4cyclictest41-21ksoftirqd/323:33:453
24471993125,4cyclictest41-21ksoftirqd/323:22:533
24471993125,4cyclictest41-21ksoftirqd/322:56:453
24471993125,4cyclictest41-21ksoftirqd/322:00:493
24471993125,4cyclictest41-21ksoftirqd/321:31:363
24471993125,4cyclictest41-21ksoftirqd/319:15:253
24471993125,4cyclictest41-21ksoftirqd/300:29:163
2447099313,27cyclictest20265-21kworker/2:019:29:422
24470993125,4cyclictest33-21ksoftirqd/222:17:002
24470993125,4cyclictest33-21ksoftirqd/222:17:002
24470993125,4cyclictest33-21ksoftirqd/222:13:352
24470993124,4cyclictest33-21ksoftirqd/221:15:392
2447099312,28cyclictest5967-21kworker/2:219:39:592
24469993126,3cyclictest25-21ksoftirqd/122:38:451
24469993125,4cyclictest25-21ksoftirqd/123:08:541
24469993125,4cyclictest25-21ksoftirqd/121:48:001
24469993125,4cyclictest25-21ksoftirqd/121:31:591
24469993125,4cyclictest25-21ksoftirqd/121:12:241
24469993125,4cyclictest25-21ksoftirqd/120:36:421
24468993126,3cyclictest9-21ksoftirqd/023:41:550
24468993126,3cyclictest9-21ksoftirqd/022:38:240
24468993126,3cyclictest9-21ksoftirqd/022:05:000
24468993126,3cyclictest9-21ksoftirqd/020:26:060
24468993126,3cyclictest9-21ksoftirqd/020:24:510
24468993126,3cyclictest9-21ksoftirqd/019:58:500
24468993126,3cyclictest9-21ksoftirqd/019:40:420
24468993126,3cyclictest9-21ksoftirqd/019:40:420
24468993125,4cyclictest9-21ksoftirqd/023:58:520
24468993125,4cyclictest9-21ksoftirqd/023:24:100
24468993125,4cyclictest9-21ksoftirqd/022:17:500
24468993125,4cyclictest9-21ksoftirqd/022:17:500
24468993125,4cyclictest9-21ksoftirqd/021:50:550
24468993125,4cyclictest9-21ksoftirqd/019:15:100
24468993125,4cyclictest9-21ksoftirqd/000:22:590
24471993025,3cyclictest41-21ksoftirqd/322:40:133
24471993025,3cyclictest41-21ksoftirqd/322:21:243
24471993024,4cyclictest41-21ksoftirqd/323:38:103
24471993024,4cyclictest41-21ksoftirqd/321:46:253
24471993024,4cyclictest41-21ksoftirqd/321:36:253
24471993024,4cyclictest41-21ksoftirqd/321:21:503
24471993024,4cyclictest41-21ksoftirqd/320:39:053
24471993024,4cyclictest41-21ksoftirqd/320:31:143
2447099303,25cyclictest20265-21kworker/2:019:15:512
24470993026,2cyclictest33-21ksoftirqd/220:45:222
24470993026,2cyclictest33-21ksoftirqd/220:45:222
24470993025,3cyclictest33-21ksoftirqd/200:34:312
24470993024,4cyclictest33-21ksoftirqd/222:06:502
24470993024,4cyclictest33-21ksoftirqd/221:13:472
24469993025,3cyclictest25-21ksoftirqd/121:29:051
24469993025,3cyclictest25-21ksoftirqd/120:41:461
24469993025,3cyclictest25-21ksoftirqd/120:08:331
24469993024,4cyclictest25-21ksoftirqd/121:50:041
24468993025,3cyclictest9-21ksoftirqd/022:53:570
24468993025,3cyclictest9-21ksoftirqd/020:52:480
24468993025,3cyclictest9-21ksoftirqd/000:00:370
24468993024,4cyclictest9-21ksoftirqd/022:46:030
24468993024,4cyclictest9-21ksoftirqd/022:21:400
24468993024,4cyclictest9-21ksoftirqd/022:10:330
24468993024,4cyclictest9-21ksoftirqd/021:44:540
24468993024,4cyclictest9-21ksoftirqd/019:27:210
136991300,7ptp4l401ktimersoftd/321:55:163
136991300,0ptp4l401ktimersoftd/320:56:393
136991300,0ptp4l401ktimersoftd/320:06:553
24471992924,3cyclictest41-21ksoftirqd/323:43:373
24471992924,3cyclictest41-21ksoftirqd/322:06:193
24471992923,4cyclictest41-21ksoftirqd/323:27:063
24471992923,4cyclictest41-21ksoftirqd/323:00:263
24471992923,4cyclictest41-21ksoftirqd/322:51:553
24471992923,4cyclictest41-21ksoftirqd/320:51:033
2447099296,21cyclictest22926-21kworker/2:123:45:172
24470992924,3cyclictest33-21ksoftirqd/220:16:552
24470992923,4cyclictest33-21ksoftirqd/222:41:522
24470992923,4cyclictest33-21ksoftirqd/221:45:552
24469992924,3cyclictest25-21ksoftirqd/122:11:411
24469992924,3cyclictest25-21ksoftirqd/121:37:151
24469992924,3cyclictest25-21ksoftirqd/100:25:111
24469992924,3cyclictest25-21ksoftirqd/100:12:491
24469992923,4cyclictest25-21ksoftirqd/119:55:031
24469992923,4cyclictest25-21ksoftirqd/119:40:051
24469992923,4cyclictest25-21ksoftirqd/119:40:051
24468992924,3cyclictest9-21ksoftirqd/022:40:330
24468992924,3cyclictest9-21ksoftirqd/020:59:090
24468992924,3cyclictest9-21ksoftirqd/020:43:360
24468992924,3cyclictest9-21ksoftirqd/020:13:210
24468992923,4cyclictest9-21ksoftirqd/023:03:320
24468992923,4cyclictest9-21ksoftirqd/021:23:040
24468992923,4cyclictest9-21ksoftirqd/020:08:340
24468992923,4cyclictest9-21ksoftirqd/000:29:570
24471992821,4cyclictest41-21ksoftirqd/319:20:233
24470992823,3cyclictest33-21ksoftirqd/221:56:512
24470992822,4cyclictest33-21ksoftirqd/220:52:172
24469992823,3cyclictest25-21ksoftirqd/123:35:051
24469992823,3cyclictest25-21ksoftirqd/120:50:001
24469992823,3cyclictest25-21ksoftirqd/120:50:001
24469992823,3cyclictest25-21ksoftirqd/120:21:021
24469992822,4cyclictest25-21ksoftirqd/122:30:111
24468992823,3cyclictest9-21ksoftirqd/021:31:010
24468992823,3cyclictest9-21ksoftirqd/019:54:060
24468992823,3cyclictest9-21ksoftirqd/019:33:510
24468992822,4cyclictest9-21ksoftirqd/021:57:570
24468992822,4cyclictest9-21ksoftirqd/021:47:590
24468992822,4cyclictest9-21ksoftirqd/020:32:090
24468992822,4cyclictest9-21ksoftirqd/019:10:450
24471992722,3cyclictest41-21ksoftirqd/323:48:243
24470992723,3cyclictest33-21ksoftirqd/222:00:242
24470992722,3cyclictest33-21ksoftirqd/223:41:312
24470992722,3cyclictest33-21ksoftirqd/219:55:232
24470992722,3cyclictest33-21ksoftirqd/219:40:132
24470992722,3cyclictest33-21ksoftirqd/219:40:132
24470992710,5cyclictest33-21ksoftirqd/221:25:352
24469992722,3cyclictest1297-21runrttasks00:05:501
24469992721,4cyclictest25-21ksoftirqd/122:56:351
24469992721,4cyclictest25-21ksoftirqd/122:03:231
24469992721,4cyclictest25-21ksoftirqd/121:08:041
24469992721,4cyclictest25-21ksoftirqd/120:26:001
24468992721,4cyclictest9-21ksoftirqd/019:24:010
24468992721,3cyclictest9-21ksoftirqd/021:28:510
136991270,0ptp4l401ktimersoftd/320:45:333
136991270,0ptp4l401ktimersoftd/320:45:333
24471992621,3cyclictest41-21ksoftirqd/321:12:343
24471992621,3cyclictest41-21ksoftirqd/319:33:463
24471992620,4cyclictest41-21ksoftirqd/321:06:113
24471992620,4cyclictest41-21ksoftirqd/320:21:083
2447099265,4cyclictest33-21ksoftirqd/220:00:212
24470992620,4cyclictest33-21ksoftirqd/222:30:042
24469992622,3cyclictest25-21ksoftirqd/120:55:001
24469992620,4cyclictest25-21ksoftirqd/100:34:071
24468992621,3cyclictest9-21ksoftirqd/023:53:260
24468992620,4cyclictest9-21ksoftirqd/023:27:070
24468992620,4cyclictest9-21ksoftirqd/023:13:190
59298250,2rtkit-daemon0-21swapper/123:10:351
59298250,2rtkit-daemon0-21swapper/120:32:151
24471992520,3cyclictest41-21ksoftirqd/319:50:233
24471992520,3cyclictest41-21ksoftirqd/319:45:283
24469992520,3cyclictest25-21ksoftirqd/123:02:291
24469992520,3cyclictest25-21ksoftirqd/122:27:441
24468992520,3cyclictest9-21ksoftirqd/022:30:470
24468992520,3cyclictest9-21ksoftirqd/020:48:050
24468992520,3cyclictest9-21ksoftirqd/020:48:050
136991250,0ptp4l401ktimersoftd/300:15:203
24469992419,3cyclictest25-21ksoftirqd/121:15:051
24469992418,4cyclictest25-21ksoftirqd/123:55:081
24468992419,3cyclictest9-21ksoftirqd/023:09:120
24468992317,4cyclictest9-21ksoftirqd/000:36:030
24469992015,3cyclictest25-21ksoftirqd/121:58:591
24468991813,3cyclictest19164-21cut00:35:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional