You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-18 - 08:58
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot2.osadl.org (updated Tue Nov 18, 2025 00:45:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
139202233165,22sleep20-21swapper/219:07:132
136691214170,33phc2sys0-21swapper/319:07:543
138172204171,22sleep10-21swapper/119:05:521
120132202169,22sleep00-21swapper/019:05:060
1366918563,15phc2sys0-21swapper/319:10:003
136591810,1ptp4l391rcuc/321:45:113
136591730,1ptp4l391rcuc/323:40:263
282532570,2sleep20-21swapper/223:00:212
136591560,0ptp4l391rcuc/300:25:053
17372550,3sleep1231rcuc/119:50:211
247792530,2sleep20-21swapper/219:30:192
1443999507,10cyclictest6609-21sed23:25:003
136591490,0ptp4l391rcuc/321:44:573
133391490,2getstats0-21swapper/322:10:173
14439994819,3cyclictest41-21ksoftirqd/319:58:483
14439994618,8cyclictest41-21ksoftirqd/300:18:563
14439994618,7cyclictest41-21ksoftirqd/323:01:133
14439994617,7cyclictest41-21ksoftirqd/319:26:173
1443999456,7cyclictest6961-21awk22:15:273
14439994517,7cyclictest41-21ksoftirqd/322:05:333
14439994517,3cyclictest41-21ksoftirqd/322:31:133
14439994418,7cyclictest41-21ksoftirqd/320:41:243
14439994329,7cyclictest41-21ksoftirqd/321:02:523
136591430,0ptp4l401ktimersoftd/322:40:313
136591430,0ptp4l401ktimersoftd/300:04:553
14439994218,7cyclictest41-21ksoftirqd/321:56:263
14439994216,6cyclictest41-21ksoftirqd/323:45:423
136591420,0ptp4l401ktimersoftd/322:04:393
136591420,0ptp4l401ktimersoftd/320:29:373
136591420,0ptp4l401ktimersoftd/320:29:373
136591420,0ptp4l401ktimersoftd/300:05:023
1443999418,5cyclictest41-21ksoftirqd/319:20:263
1443999414,7cyclictest8840-21cat22:20:173
136591410,0ptp4l401ktimersoftd/320:24:363
136591410,0ptp4l401ktimersoftd/319:44:223
1443999400,5cyclictest0-21swapper/319:46:443
1443999392,5cyclictest41-21ksoftirqd/319:50:273
14439993920,9cyclictest41-21ksoftirqd/320:04:583
1443799391,3cyclictest1749-21runrttasks20:57:361
1443799391,3cyclictest1749-21runrttasks20:57:351
1443999385,4cyclictest41-21ksoftirqd/323:11:203
14439993819,7cyclictest41-21ksoftirqd/322:57:443
14439993819,7cyclictest41-21ksoftirqd/321:33:003
14439993719,7cyclictest41-21ksoftirqd/320:18:123
14439993719,10cyclictest41-21ksoftirqd/319:30:373
14439993718,8cyclictest41-21ksoftirqd/320:55:133
14439993718,8cyclictest41-21ksoftirqd/320:55:123
1443799370,3cyclictest16777-21/usr/sbin/munin20:20:251
136591370,1ptp4l20890-21sshd23:53:363
14439993624,3cyclictest41-21ksoftirqd/321:20:273
14439993618,9cyclictest41-21ksoftirqd/323:38:053
14439993618,8cyclictest41-21ksoftirqd/321:50:573
14439993618,7cyclictest41-21ksoftirqd/319:18:283
14439993613,8cyclictest41-21ksoftirqd/319:39:293
14437993612,21cyclictest26451-21sh21:50:171
1443999352,5cyclictest41-21ksoftirqd/320:35:213
14439993518,8cyclictest41-21ksoftirqd/300:10:583
14439993517,8cyclictest41-21ksoftirqd/320:12:443
14438993529,3cyclictest5592-21cron20:00:002
14437993528,5cyclictest25-21ksoftirqd/100:27:581
1443799350,4cyclictest23936-21expr21:45:131
1443799350,3cyclictest1514-21if_enp4s000:20:171
14439993418,4cyclictest41-21ksoftirqd/320:54:203
14439993417,7cyclictest41-21ksoftirqd/320:30:533
14439993411,8cyclictest41-21ksoftirqd/321:28:213
14439993411,8cyclictest41-21ksoftirqd/321:28:213
14439993410,3cyclictest41-21ksoftirqd/321:38:283
1443899343,8cyclictest21817-21sed22:45:312
14437993413,18cyclictest20381-21netstat19:20:251
14437993410,21cyclictest2726-21fschecks_time21:00:181
1443799340,3cyclictest9358-21grep23:30:131
1443799340,3cyclictest22104-21hddtemp_smartct19:25:171
1443799340,3cyclictest12857-21meminfo21:20:211
1443799338,22cyclictest18876-21/usr/sbin/munin23:50:251
1443799337,23cyclictest424-21dbus-daemon22:50:251
14437993327,4cyclictest25-21ksoftirqd/122:00:591
1443799331,3cyclictest21613-21df21:40:151
14437993313,18cyclictest9587-21sendmail_mailqu22:20:241
14437993312,18cyclictest9958-21hwlatdetect21:15:171
1443799330,4cyclictest4122-21iostat19:55:221
1443799330,3cyclictest7254-21expr21:10:171
1443799330,3cyclictest6366-21hddtemp_smartct20:00:171
1443799330,3cyclictest14738-21if_enp4s023:40:211
14436993326,4cyclictest9-21ksoftirqd/020:57:230
14436993326,4cyclictest9-21ksoftirqd/020:57:230
136591330,0ptp4l401ktimersoftd/322:28:243
224002320,6sleep09-21ksoftirqd/022:50:000
1443999329,3cyclictest41-21ksoftirqd/320:06:213
1443999327,5cyclictest41-21ksoftirqd/320:45:223
14439993225,4cyclictest41-21ksoftirqd/323:19:003
14439993214,6cyclictest41-21ksoftirqd/321:12:203
14438993226,4cyclictest33-21ksoftirqd/221:16:282
14438993226,4cyclictest33-21ksoftirqd/220:08:442
1443799329,20cyclictest4751-21timerandwakeup22:10:281
1443799329,20cyclictest17087-21if_enp1s023:45:181
1443799328,21cyclictest26326-21df_abs00:05:151
1443799327,22cyclictest2069-21sort23:15:151
14437993226,3cyclictest25-21ksoftirqd/122:30:131
14437993225,4cyclictest25-21ksoftirqd/100:35:011
1443799320,3cyclictest1715-21irqstats22:05:211
1443799320,3cyclictest16210-21users19:10:291
1443799320,3cyclictest14206-21apt-get21:25:171
1443799320,3cyclictest14206-21apt-get21:25:171
14436993226,4cyclictest9-21ksoftirqd/023:52:430
14436993226,4cyclictest9-21ksoftirqd/020:19:410
14436993226,3cyclictest9-21ksoftirqd/023:41:590
14436993225,4cyclictest9-21ksoftirqd/021:03:210
14436993225,4cyclictest9-21ksoftirqd/020:00:020
14439993125,4cyclictest41-21ksoftirqd/323:07:203
14439993124,4cyclictest41-21ksoftirqd/323:34:163
14439993124,4cyclictest41-21ksoftirqd/300:36:423
14439993115,8cyclictest41-21ksoftirqd/321:09:423
14438993126,3cyclictest33-21ksoftirqd/220:35:452
14438993126,3cyclictest33-21ksoftirqd/200:29:352
14438993125,4cyclictest33-21ksoftirqd/221:38:332
14438993124,4cyclictest33-21ksoftirqd/223:17:222
1443799319,20cyclictest6957-21ls23:25:171
1443799318,21cyclictest25160-21sed00:00:271
1443799318,20cyclictest971-21latency_hist19:50:011
1443799318,20cyclictest24424-21grep20:35:361
1443799317,21cyclictest32078-21cat23:10:141
1443799317,21cyclictest28157-21irqrtprio20:45:211
1443799315,23cyclictest28808-21/usr/sbin/munin23:00:281
14437993125,4cyclictest25-21ksoftirqd/123:57:531
14437993125,4cyclictest25-21ksoftirqd/122:46:361
14437993125,4cyclictest25-21ksoftirqd/120:15:581
14437993125,4cyclictest25-21ksoftirqd/120:13:231
1443799311,9cyclictest0-21swapper/120:35:011
1443799311,7cyclictest26331-21apt-get19:35:131
14437993111,18cyclictest9435-21sendmail_mailqu00:35:271
14437993110,18cyclictest11145-21idleruntime22:25:191
1443799310,3cyclictest30939-21apt-get00:15:131
1443799310,3cyclictest29370-21iostat_ios19:40:211
1443799310,3cyclictest24466-21hddtemp_smartct19:30:171
14436993125,4cyclictest9-21ksoftirqd/021:42:400
14436993125,3cyclictest9-21ksoftirqd/023:04:150
14436993125,3cyclictest9-21ksoftirqd/000:08:180
14436993124,4cyclictest9-21ksoftirqd/023:58:040
14436993124,4cyclictest9-21ksoftirqd/019:42:330
136591310,0ptp4l401ktimersoftd/322:35:143
14439993027,2cyclictest391rcuc/321:15:163
14439993024,4cyclictest41-21ksoftirqd/300:23:313
14438993025,3cyclictest33-21ksoftirqd/222:41:362
14438993025,3cyclictest33-21ksoftirqd/222:29:542
14438993024,4cyclictest33-21ksoftirqd/223:42:422
14438993024,4cyclictest33-21ksoftirqd/222:32:022
14438993024,4cyclictest33-21ksoftirqd/219:35:532
14438993024,4cyclictest33-21ksoftirqd/200:31:312
14438993024,4cyclictest33-21ksoftirqd/200:15:492
14438993024,4cyclictest33-21ksoftirqd/200:04:202
1443899300,9cyclictest0-21swapper/200:20:282
1443799308,19cyclictest18186-21fschecks_count22:40:191
1443799306,22cyclictest26487-21systemctl20:40:261
14437993025,3cyclictest25-21ksoftirqd/119:18:481
14437993024,4cyclictest25-21ksoftirqd/122:17:111
14437993010,18cyclictest19391-21fschecks_time21:35:181
1443799300,3cyclictest8675-21diskstats20:05:171
1443799300,3cyclictest29890-21df23:05:171
1443799300,3cyclictest15741-21diskstats22:35:171
14436993025,3cyclictest9-21ksoftirqd/021:32:410
14436993025,3cyclictest9-21ksoftirqd/020:47:410
14436993025,3cyclictest9-21ksoftirqd/020:30:430
14436993025,3cyclictest9-21ksoftirqd/020:09:290
14436993025,3cyclictest9-21ksoftirqd/019:34:270
14436993025,3cyclictest9-21ksoftirqd/000:04:100
14436993024,4cyclictest9-21ksoftirqd/000:35:550
14436993024,4cyclictest9-21ksoftirqd/000:14:410
14436993024,3cyclictest9-21ksoftirqd/021:19:130
14436993024,3cyclictest9-21ksoftirqd/020:51:120
14436993023,4cyclictest9-21ksoftirqd/019:29:220
195172290,2sleep2121rcu_preempt19:20:182
1443999297,5cyclictest41-21ksoftirqd/300:30:183
14439992923,3cyclictest41-21ksoftirqd/323:55:103
14438992924,3cyclictest33-21ksoftirqd/220:02:072
14438992923,4cyclictest33-21ksoftirqd/222:18:252
14438992923,4cyclictest33-21ksoftirqd/220:29:332
14438992923,4cyclictest33-21ksoftirqd/220:29:332
14438992923,4cyclictest33-21ksoftirqd/219:10:062
1443799296,21cyclictest5437-21irqrtprio21:05:201
1443799296,20cyclictest28576-21cpu00:10:131
1443799296,20cyclictest17368-21irqrtprio21:30:201
1443799291,4cyclictest43942sleep123:20:141
1443799290,5cyclictest30166-21/usr/sbin/munin21:55:291
14436992924,3cyclictest9-21ksoftirqd/023:07:450
14436992924,3cyclictest9-21ksoftirqd/022:23:560
14436992923,3cyclictest9-21ksoftirqd/023:14:500
14436992922,4cyclictest9-21ksoftirqd/019:12:330
14439992824,3cyclictest41-21ksoftirqd/322:45:163
14439992819,3cyclictest41-21ksoftirqd/322:51:423
1443899289,18cyclictest0-21swapper/220:55:252
1443899289,18cyclictest0-21swapper/220:55:252
14438992823,3cyclictest33-21ksoftirqd/223:54:392
14438992823,3cyclictest33-21ksoftirqd/222:50:122
14438992823,3cyclictest33-21ksoftirqd/222:01:162
14438992823,3cyclictest33-21ksoftirqd/220:46:182
14438992823,3cyclictest33-21ksoftirqd/219:17:292
14438992822,4cyclictest33-21ksoftirqd/223:24:482
14438992822,4cyclictest33-21ksoftirqd/221:32:072
14438992822,4cyclictest33-21ksoftirqd/200:06:132
14438992819,7cyclictest33-21ksoftirqd/223:05:282
1443799281,4cyclictest30370-21/usr/sbin/munin20:50:171
1443799281,3cyclictest2259-21gnome-shell20:26:591
1443799281,3cyclictest2259-21gnome-shell20:26:581
14436992823,3cyclictest9-21ksoftirqd/022:33:270
14436992823,3cyclictest9-21ksoftirqd/022:15:060
14436992823,3cyclictest9-21ksoftirqd/021:07:220
14436992823,3cyclictest9-21ksoftirqd/020:11:030
14436992823,3cyclictest9-21ksoftirqd/019:49:310
14436992823,3cyclictest9-21ksoftirqd/000:20:470
14436992821,4cyclictest9-21ksoftirqd/019:19:030
14439992722,3cyclictest41-21ksoftirqd/323:25:163
14438992722,3cyclictest33-21ksoftirqd/222:59:532
14438992721,4cyclictest33-21ksoftirqd/220:42:502
14438992721,4cyclictest33-21ksoftirqd/220:34:262
14438992721,3cyclictest33-21ksoftirqd/221:23:302
14438992720,4cyclictest33-21ksoftirqd/220:14:492
1443799273,3cyclictest0-21swapper/122:55:151
1443799270,5cyclictest12185-21/usr/sbin/munin23:35:181
14436992722,3cyclictest9-21ksoftirqd/023:39:270
14436992722,3cyclictest9-21ksoftirqd/022:13:340
14436992722,3cyclictest9-21ksoftirqd/020:28:310
14436992722,3cyclictest9-21ksoftirqd/020:28:310
14436992722,3cyclictest9-21ksoftirqd/019:54:400
14436992722,3cyclictest9-21ksoftirqd/019:22:480
14436992721,4cyclictest9-21ksoftirqd/022:27:250
14436992721,4cyclictest9-21ksoftirqd/000:34:550
14436992721,3cyclictest9-21ksoftirqd/020:20:100
14438992621,3cyclictest33-21ksoftirqd/223:46:082
14438992621,3cyclictest33-21ksoftirqd/222:14:022
14438992621,3cyclictest33-21ksoftirqd/221:09:422
14438992621,3cyclictest33-21ksoftirqd/220:53:002
14438992621,3cyclictest33-21ksoftirqd/200:38:562
14438992620,4cyclictest33-21ksoftirqd/223:38:052
14438992620,4cyclictest33-21ksoftirqd/223:27:452
14438992620,4cyclictest33-21ksoftirqd/221:55:472
14438992620,4cyclictest33-21ksoftirqd/221:41:102
14438992620,4cyclictest33-21ksoftirqd/219:54:022
14438992620,4cyclictest33-21ksoftirqd/219:42:022
14438992620,3cyclictest33-21ksoftirqd/222:22:562
14436992621,3cyclictest9-21ksoftirqd/020:43:390
14436992621,3cyclictest9-21ksoftirqd/019:38:480
14436992620,4cyclictest9-21ksoftirqd/023:15:030
14436992620,4cyclictest9-21ksoftirqd/021:26:190
14436992620,4cyclictest9-21ksoftirqd/021:26:190
14436992620,3cyclictest9-21ksoftirqd/021:47:440
14438992520,3cyclictest33-21ksoftirqd/223:32:192
14438992520,3cyclictest33-21ksoftirqd/222:05:472
14438992520,3cyclictest33-21ksoftirqd/221:25:062
14438992520,3cyclictest33-21ksoftirqd/221:25:062
14438992520,3cyclictest33-21ksoftirqd/221:11:352
14438992520,3cyclictest33-21ksoftirqd/221:02:462
14438992519,3cyclictest33-21ksoftirqd/219:45:222
14438992518,4cyclictest33-21ksoftirqd/219:26:152
14436992520,3cyclictest9-21ksoftirqd/023:30:210
14436992520,3cyclictest9-21ksoftirqd/022:06:240
14436992520,3cyclictest9-21ksoftirqd/021:59:220
14436992520,3cyclictest18225-21grep22:40:190
14436992519,4cyclictest9-21ksoftirqd/021:50:510
14436992519,4cyclictest9-21ksoftirqd/000:19:300
14436992519,3cyclictest9-21ksoftirqd/022:58:090
14438992418,4cyclictest33-21ksoftirqd/221:51:182
14438992417,5cyclictest33-21ksoftirqd/223:11:442
14436992418,4cyclictest9-21ksoftirqd/021:10:390
14438992318,3cyclictest33-21ksoftirqd/222:37:392
14438992318,3cyclictest33-21ksoftirqd/221:47:072
14438992318,3cyclictest33-21ksoftirqd/200:14:512
14436992319,2cyclictest9-21ksoftirqd/021:35:190
14436992317,4cyclictest9-21ksoftirqd/019:59:580
14438992217,3cyclictest33-21ksoftirqd/220:22:222
14438992217,3cyclictest33-21ksoftirqd/220:17:322
14438992216,3cyclictest33-21ksoftirqd/223:57:572
14436992217,3cyclictest9-21ksoftirqd/023:45:550
14436992217,3cyclictest9-21ksoftirqd/022:02:500
14436992216,3cyclictest9-21ksoftirqd/000:25:210
14436992016,3cyclictest9-21ksoftirqd/021:25:010
14436992015,3cyclictest9-21ksoftirqd/022:50:140
14436991914,3cyclictest9-21ksoftirqd/023:20:200
14436991914,3cyclictest6745-21acpi23:25:140
14436991914,3cyclictest22828-21cpuspeed20:35:170
14436991813,3cyclictest15996-21iostat22:35:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional