You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 03:59
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot2.osadl.org (updated Sun Nov 30, 2025 00:45:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4122210165,35sleep10-21swapper/119:09:591
136691207173,23phc2sys0-21swapper/319:07:213
326582205171,23sleep20-21swapper/219:07:182
325992203169,23sleep00-21swapper/019:06:320
185472740,6sleep09-21ksoftirqd/022:00:230
143812530,2sleep20-21swapper/220:45:192
70442520,2sleep20-21swapper/219:20:262
41425236,10sleep30-21swapper/319:10:013
240732520,1sleep20-21swapper/200:25:272
160722510,2sleep30-21swapper/321:55:193
73899460,3cyclictest15693-21runrttasks23:02:103
136591440,0ptp4l401ktimersoftd/321:12:533
51698430,2rtkit-daemon515-21rtkit-daemon19:12:470
136591420,0ptp4l401ktimersoftd/300:05:243
737994129,9cyclictest33-21ksoftirqd/223:20:292
136591400,0ptp4l401ktimersoftd/321:25:163
737993934,3cyclictest33-21ksoftirqd/221:55:252
73899381,3cyclictest26924-21sendmail22:20:003
73799367,4cyclictest33-21ksoftirqd/222:35:212
737993625,9cyclictest33-21ksoftirqd/221:05:192
737993624,9cyclictest33-21ksoftirqd/200:00:292
738993521,12cyclictest41-21ksoftirqd/319:53:453
738993510,22cyclictest226-21systemd-journal00:00:003
737993530,3cyclictest33-21ksoftirqd/221:40:012
736993512,20cyclictest22045-21sed21:00:221
73899341,30cyclictest3801-21irqrtprio21:30:183
73899340,4cyclictest226-21systemd-journal23:20:263
737993429,3cyclictest33-21ksoftirqd/219:20:012
737993428,4cyclictest33-21ksoftirqd/219:53:482
737993428,3cyclictest33-21ksoftirqd/221:00:002
737993428,3cyclictest33-21ksoftirqd/221:00:002
73799341,5cyclictest33-21ksoftirqd/220:20:002
736993415,16cyclictest11887-21grep00:00:301
73699340,31cyclictest23092-21cron23:24:551
735993428,4cyclictest9-21ksoftirqd/023:35:420
735993428,4cyclictest9-21ksoftirqd/021:56:340
73899337,23cyclictest2642-21latency_hist23:45:003
738993327,4cyclictest41-21ksoftirqd/322:35:083
738993327,4cyclictest41-21ksoftirqd/320:52:223
738993327,4cyclictest41-21ksoftirqd/300:34:393
738993311,19cyclictest3993-21irqrtprio23:45:203
738993310,20cyclictest25890-21cpuspeed_turbos23:25:163
737993330,2cyclictest33-21ksoftirqd/223:45:002
737993328,3cyclictest33-21ksoftirqd/223:35:132
737993328,3cyclictest33-21ksoftirqd/223:15:162
737993328,3cyclictest33-21ksoftirqd/222:48:592
737993328,3cyclictest33-21ksoftirqd/220:35:152
737993328,3cyclictest33-21ksoftirqd/219:50:002
737993327,4cyclictest33-21ksoftirqd/223:30:102
737993327,4cyclictest33-21ksoftirqd/221:18:322
737993314,4cyclictest33-21ksoftirqd/222:40:292
736993327,4cyclictest25-21ksoftirqd/120:34:551
736993327,4cyclictest25-21ksoftirqd/100:25:091
736993327,4cyclictest25-21ksoftirqd/100:16:481
736993327,4cyclictest25-21ksoftirqd/100:05:071
736993311,19cyclictest28382-21expr21:15:151
735993327,4cyclictest9-21ksoftirqd/023:05:570
735993327,4cyclictest9-21ksoftirqd/021:21:460
735993327,4cyclictest9-21ksoftirqd/020:56:320
735993327,4cyclictest9-21ksoftirqd/020:56:310
136591330,14ptp4l73899cyclictest20:48:273
738993226,4cyclictest41-21ksoftirqd/319:39:363
738993226,4cyclictest41-21ksoftirqd/300:01:443
738993215,14cyclictest2259-21gnome-shell22:50:583
738993213,16cyclictest27414-21awk20:05:183
738993213,16cyclictest10417-21ls20:35:283
73899320,4cyclictest5461-21apt-get23:50:133
73899320,3cyclictest28632-21fschecks_time21:15:183
73799323,4cyclictest33-21ksoftirqd/222:55:302
737993228,3cyclictest33-21ksoftirqd/219:55:152
737993228,3cyclictest33-21ksoftirqd/200:10:012
737993228,2cyclictest33-21ksoftirqd/219:10:142
737993227,3cyclictest33-21ksoftirqd/223:55:012
737993227,3cyclictest33-21ksoftirqd/221:40:232
737993227,3cyclictest33-21ksoftirqd/220:25:202
737993227,3cyclictest33-21ksoftirqd/220:25:202
737993226,4cyclictest33-21ksoftirqd/221:12:112
737993226,4cyclictest33-21ksoftirqd/220:13:402
737993226,4cyclictest33-21ksoftirqd/220:01:102
73799321,5cyclictest33-21ksoftirqd/221:50:162
73799320,5cyclictest33-21ksoftirqd/220:40:262
73699329,20cyclictest32045-21grep22:30:141
736993226,4cyclictest25-21ksoftirqd/123:58:411
736993226,4cyclictest25-21ksoftirqd/122:49:141
736993226,4cyclictest25-21ksoftirqd/122:44:001
736993226,4cyclictest25-21ksoftirqd/121:24:251
736993226,4cyclictest25-21ksoftirqd/120:02:411
736993226,4cyclictest25-21ksoftirqd/119:24:581
736993226,4cyclictest25-21ksoftirqd/119:19:201
736993214,15cyclictest20646-21if_err_enp1s022:05:201
73699321,28cyclictest21293-21grep19:50:261
735993227,3cyclictest9-21ksoftirqd/023:00:310
735993226,4cyclictest9-21ksoftirqd/022:05:400
735993226,4cyclictest9-21ksoftirqd/020:49:380
735993226,3cyclictest9-21ksoftirqd/020:40:470
735993224,5cyclictest9-21ksoftirqd/021:11:200
136591320,15ptp4l73899cyclictest20:24:373
73899319,19cyclictest25317-21sed20:00:233
73899319,19cyclictest10678-21cut21:45:153
73899318,20cyclictest19467-21cut20:55:213
73899318,20cyclictest19467-21cut20:55:213
738993125,4cyclictest41-21ksoftirqd/323:36:183
738993125,4cyclictest41-21ksoftirqd/323:30:523
738993125,4cyclictest41-21ksoftirqd/323:17:123
738993125,4cyclictest41-21ksoftirqd/321:00:483
738993124,5cyclictest41-21ksoftirqd/320:31:483
73899311,3cyclictest19874-21idleruntime-cro00:20:003
73899310,2cyclictest121rcu_preempt19:19:583
73799318,3cyclictest33-21ksoftirqd/222:01:272
73799317,5cyclictest33-21ksoftirqd/222:20:152
73799315,3cyclictest33-21ksoftirqd/221:25:202
73799314,7cyclictest33-21ksoftirqd/219:40:002
737993126,4cyclictest33-21ksoftirqd/220:05:302
737993126,3cyclictest33-21ksoftirqd/223:05:252
737993126,3cyclictest33-21ksoftirqd/220:20:162
737993126,3cyclictest33-21ksoftirqd/219:25:152
737993126,3cyclictest33-21ksoftirqd/200:25:012
737993126,3cyclictest33-21ksoftirqd/200:25:012
737993125,4cyclictest33-21ksoftirqd/223:28:472
73799311,4cyclictest33-21ksoftirqd/222:10:242
73799310,5cyclictest33-21ksoftirqd/223:00:282
73799310,5cyclictest33-21ksoftirqd/220:30:292
73699319,19cyclictest4270-21latency_hist22:40:011
73699319,19cyclictest17794-21cat20:50:271
736993125,4cyclictest25-21ksoftirqd/123:52:031
736993125,4cyclictest25-21ksoftirqd/123:40:091
736993125,4cyclictest25-21ksoftirqd/123:33:311
736993125,4cyclictest25-21ksoftirqd/122:22:091
736993125,4cyclictest25-21ksoftirqd/120:55:331
736993125,4cyclictest25-21ksoftirqd/120:55:331
736993125,4cyclictest25-21ksoftirqd/119:47:371
736993125,4cyclictest25-21ksoftirqd/119:41:031
735993125,4cyclictest9-21ksoftirqd/023:17:040
735993125,4cyclictest9-21ksoftirqd/022:40:090
735993125,4cyclictest9-21ksoftirqd/020:29:480
735993125,4cyclictest9-21ksoftirqd/020:29:480
735993125,4cyclictest9-21ksoftirqd/020:10:100
735993125,4cyclictest9-21ksoftirqd/019:16:150
735993125,3cyclictest9-21ksoftirqd/000:10:400
73899303,15cyclictest10353-21kworker/3:122:21:243
738993025,3cyclictest41-21ksoftirqd/319:49:323
73899301,23cyclictest29530-21apt-get22:25:123
738993012,16cyclictest2-21kthreadd19:30:003
73899300,4cyclictest22708-21/usr/sbin/munin00:25:193
73799303,4cyclictest33-21ksoftirqd/223:10:242
737993027,2cyclictest33-21ksoftirqd/222:20:002
737993026,3cyclictest33-21ksoftirqd/223:55:202
737993026,3cyclictest33-21ksoftirqd/222:05:202
737993026,2cyclictest33-21ksoftirqd/222:25:252
737993026,2cyclictest33-21ksoftirqd/221:00:162
737993025,3cyclictest33-21ksoftirqd/220:50:202
737993025,3cyclictest33-21ksoftirqd/200:19:532
737993024,4cyclictest33-21ksoftirqd/221:21:242
737993024,3cyclictest33-21ksoftirqd/219:30:172
737993024,3cyclictest33-21ksoftirqd/200:30:192
737993020,2cyclictest121rcu_preempt22:50:192
736993025,3cyclictest25-21ksoftirqd/121:34:541
736993024,4cyclictest25-21ksoftirqd/123:29:221
736993024,4cyclictest25-21ksoftirqd/123:08:161
736993024,4cyclictest25-21ksoftirqd/120:36:271
736993024,4cyclictest25-21ksoftirqd/120:05:581
736993024,4cyclictest25-21ksoftirqd/119:38:461
736993024,4cyclictest25-21ksoftirqd/119:31:201
736993010,18cyclictest14676-21cut23:00:211
735993025,3cyclictest9-21ksoftirqd/022:23:090
735993025,3cyclictest9-21ksoftirqd/021:16:060
735993024,4cyclictest9-21ksoftirqd/022:19:320
735993024,4cyclictest9-21ksoftirqd/020:23:020
735993024,4cyclictest9-21ksoftirqd/000:25:130
735993023,4cyclictest9-21ksoftirqd/021:30:340
136591300,12ptp4l73899cyclictest23:10:443
136591300,11ptp4l73899cyclictest20:16:453
73899297,19cyclictest16013-21irqrtprio19:40:213
73899296,3cyclictest21136-21needreboot22:05:243
738992924,3cyclictest41-21ksoftirqd/321:06:143
738992924,3cyclictest41-21ksoftirqd/320:29:303
738992924,3cyclictest41-21ksoftirqd/320:29:303
738992924,3cyclictest41-21ksoftirqd/319:33:463
738992923,4cyclictest41-21ksoftirqd/323:06:413
738992922,4cyclictest41-21ksoftirqd/322:00:163
73899291,5cyclictest2259-21gnome-shell22:34:593
738992910,17cyclictest8108-21/usr/sbin/munin21:40:173
73899290,3cyclictest28476-21meminfo00:35:223
73899290,3cyclictest12342-21irqstats22:55:223
73799293,3cyclictest121rcu_preempt00:35:222
737992925,3cyclictest33-21ksoftirqd/222:30:182
737992925,2cyclictest33-21ksoftirqd/223:45:292
737992925,2cyclictest33-21ksoftirqd/221:48:062
737992924,3cyclictest33-21ksoftirqd/219:40:152
73699299,17cyclictest25038-21expr22:15:171
73699298,18cyclictest4066-21latency_hist20:25:001
73699296,21cyclictest27736-21kworker/1:200:30:081
73699295,21cyclictest13765-21cat23:00:011
73699294,4cyclictest26681-21irqrtprio21:10:211
736992924,3cyclictest25-21ksoftirqd/123:39:171
736992924,3cyclictest25-21ksoftirqd/123:19:301
736992924,3cyclictest25-21ksoftirqd/122:25:091
736992924,3cyclictest25-21ksoftirqd/121:51:091
736992923,4cyclictest25-21ksoftirqd/121:45:431
736992923,4cyclictest25-21ksoftirqd/121:43:151
736992923,4cyclictest25-21ksoftirqd/121:26:581
736992923,4cyclictest25-21ksoftirqd/120:15:371
735992924,3cyclictest9-21ksoftirqd/023:50:310
735992923,4cyclictest9-21ksoftirqd/023:57:590
735992923,4cyclictest9-21ksoftirqd/023:48:380
735992923,4cyclictest9-21ksoftirqd/022:48:370
735992923,4cyclictest9-21ksoftirqd/022:29:510
735992923,4cyclictest9-21ksoftirqd/020:32:330
735992923,4cyclictest9-21ksoftirqd/019:58:230
735992923,4cyclictest9-21ksoftirqd/019:25:330
73899289,17cyclictest7080-21sed19:20:263
73899289,17cyclictest15709-21df_abs00:10:163
73899287,18cyclictest30839-21switchtime20:10:283
73899284,4cyclictest490-21gmain22:10:123
738992823,3cyclictest41-21ksoftirqd/321:22:193
738992822,4cyclictest41-21ksoftirqd/321:52:263
73899280,3cyclictest226-21systemd-journal22:40:213
737992822,3cyclictest33-21ksoftirqd/221:30:162
73699287,3cyclictest14223-21grep20:45:161
73699286,19cyclictest7649-21idleruntime-cro21:40:001
736992822,4cyclictest25-21ksoftirqd/119:59:081
735992823,3cyclictest9-21ksoftirqd/020:01:330
735992822,4cyclictest9-21ksoftirqd/021:36:400
735992822,4cyclictest9-21ksoftirqd/021:01:410
735992822,4cyclictest9-21ksoftirqd/020:51:260
735992822,4cyclictest9-21ksoftirqd/020:37:250
735992822,4cyclictest9-21ksoftirqd/000:19:380
735992822,3cyclictest9-21ksoftirqd/023:28:440
73899273,5cyclictest5519-21apt-get21:35:123
73899271,4cyclictest2259-21gnome-shell22:47:583
737992722,3cyclictest33-21ksoftirqd/200:15:002
73699279,16cyclictest1965-21open_files19:10:231
73699278,4cyclictest20043-21cat23:10:261
73699277,3cyclictest17652-21cpu22:00:161
73699275,3cyclictest25001-21cat21:05:291
73699274,3cyclictest10700-21unixbench_multi22:50:281
736992722,3cyclictest25-21ksoftirqd/121:58:061
736992722,3cyclictest25-21ksoftirqd/100:21:321
736992722,3cyclictest25-21ksoftirqd/100:21:311
736992721,4cyclictest25-21ksoftirqd/100:39:381
73699271,5cyclictest8627-21sed19:25:201
736992710,14cyclictest4125-21missed_timers23:45:221
735992722,3cyclictest9-21ksoftirqd/023:21:370
735992722,3cyclictest9-21ksoftirqd/022:55:430
735992722,3cyclictest9-21ksoftirqd/019:41:500
735992721,4cyclictest9-21ksoftirqd/023:30:480
735992721,4cyclictest9-21ksoftirqd/000:08:240
73899267,16cyclictest22552-21df_abs19:55:153
73899264,3cyclictest11739-21fschecks_time20:40:193
73899261,4cyclictest20805-21fschecks_time00:20:163
73899261,4cyclictest20805-21fschecks_time00:20:163
73699266,3cyclictest15639-21cat00:10:151
736992620,4cyclictest25-21ksoftirqd/120:44:541
735992621,3cyclictest9-21ksoftirqd/022:34:410
735992621,3cyclictest9-21ksoftirqd/000:35:540
735992620,4cyclictest9-21ksoftirqd/022:10:410
735992620,4cyclictest9-21ksoftirqd/021:29:500
735992620,4cyclictest9-21ksoftirqd/021:09:120
735992620,4cyclictest9-21ksoftirqd/020:08:500
735992620,4cyclictest9-21ksoftirqd/019:37:330
736992520,3cyclictest25-21ksoftirqd/120:27:431
736992520,3cyclictest25-21ksoftirqd/120:27:421
73699251,5cyclictest22876-21tune2fs22:10:171
735992520,3cyclictest9-21ksoftirqd/021:53:240
735992520,3cyclictest9-21ksoftirqd/021:41:220
735992520,3cyclictest9-21ksoftirqd/000:30:230
735992519,4cyclictest9-21ksoftirqd/022:35:440
735992519,4cyclictest9-21ksoftirqd/019:30:030
735992419,3cyclictest9-21ksoftirqd/022:51:010
735992419,3cyclictest9-21ksoftirqd/021:47:140
735992419,3cyclictest9-21ksoftirqd/000:04:510
73699231,4cyclictest29962-21if_enp4s020:10:201
735992317,4cyclictest9-21ksoftirqd/019:51:330
735992217,3cyclictest9-21ksoftirqd/023:14:560
735992217,3cyclictest9-21ksoftirqd/020:15:360
735992217,3cyclictest9-21ksoftirqd/019:46:020
735992116,3cyclictest9-21ksoftirqd/019:21:000
735991713,2cyclictest2640-21apt-get23:44:590
735991611,3cyclictest22672-21cut00:25:010
735991611,3cyclictest22672-21cut00:25:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional