You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-01-13 - 12:28
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackdslot1.osadl.org (updated Mon Jan 13, 2025 00:45:39)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2028121570,6sleep11242499cyclictest20:35:191
921521350,3sleep21242599cyclictest22:50:152
761021350,4sleep11242499cyclictest22:45:021
43682770,5sleep30-21swapper/300:00:213
320192640,3sleep132023-21missed_timers19:50:221
247732630,4sleep10-21swapper/119:35:181
12425996124,8cyclictest33-21ksoftirqd/223:30:392
12425996031,7cyclictest33-21ksoftirqd/221:30:002
2847191590,4ptp4l0-21swapper/023:55:150
12425995822,7cyclictest33-21ksoftirqd/222:55:282
12425995822,7cyclictest33-21ksoftirqd/222:55:282
12423995829,7cyclictest9-21ksoftirqd/022:40:000
2847191570,3ptp4l0-21swapper/123:50:211
2847191570,3ptp4l0-21swapper/123:50:211
2847191570,14ptp4l0-21swapper/119:05:181
12423995718,7cyclictest9-21ksoftirqd/021:19:590
2847191560,3ptp4l0-21swapper/100:15:121
2847191560,2ptp4l0-21swapper/221:04:232
12425995619,5cyclictest33-21ksoftirqd/200:10:002
12423995627,6cyclictest9-21ksoftirqd/021:40:150
2847191550,3ptp4l0-21swapper/221:35:072
12425995527,7cyclictest33-21ksoftirqd/219:30:002
12425995527,7cyclictest33-21ksoftirqd/219:30:002
12425995526,3cyclictest33-21ksoftirqd/223:50:202
12425995526,3cyclictest33-21ksoftirqd/223:50:202
12423995526,6cyclictest9-21ksoftirqd/021:10:290
12423995526,3cyclictest9-21ksoftirqd/000:15:380
12423995524,6cyclictest9-21ksoftirqd/021:23:130
12423995523,5cyclictest9-21ksoftirqd/019:10:140
12425995424,6cyclictest33-21ksoftirqd/222:24:062
12425995423,7cyclictest33-21ksoftirqd/220:55:002
12425995423,4cyclictest33-21ksoftirqd/223:30:002
12425995423,3cyclictest33-21ksoftirqd/200:10:162
12423995425,6cyclictest9-21ksoftirqd/022:00:270
12423995424,3cyclictest9-21ksoftirqd/019:50:500
12423995423,3cyclictest9-21ksoftirqd/022:56:150
12423995423,3cyclictest9-21ksoftirqd/022:56:150
12423995422,8cyclictest9-21ksoftirqd/020:16:500
2847191530,4ptp4l0-21swapper/322:23:273
12425995318,5cyclictest33-21ksoftirqd/200:20:282
12425995315,8cyclictest33-21ksoftirqd/219:51:152
12423995324,5cyclictest9-21ksoftirqd/000:02:140
12423995317,4cyclictest9-21ksoftirqd/021:00:140
2847191520,4ptp4l0-21swapper/022:09:090
2847191520,4ptp4l0-21swapper/020:52:360
12425995227,6cyclictest33-21ksoftirqd/219:25:012
12425995217,7cyclictest33-21ksoftirqd/222:39:192
12423995226,6cyclictest9-21ksoftirqd/020:27:520
12423995225,3cyclictest9-21ksoftirqd/019:44:590
2847191510,4ptp4l0-21swapper/023:24:570
2847191510,3ptp4l0-21swapper/219:33:532
256762510,5sleep20-21swapper/222:10:122
12425995122,5cyclictest33-21ksoftirqd/223:35:172
12425995122,3cyclictest33-21ksoftirqd/221:10:222
12425995119,6cyclictest33-21ksoftirqd/220:30:232
12425995117,8cyclictest33-21ksoftirqd/219:57:572
12425995117,7cyclictest33-21ksoftirqd/221:05:272
12423995122,5cyclictest9-21ksoftirqd/023:15:260
12423995122,3cyclictest9-21ksoftirqd/023:34:180
2847191500,4ptp4l0-21swapper/022:11:500
2847191500,4ptp4l0-21swapper/000:31:270
12425995031,3cyclictest33-21ksoftirqd/219:39:132
12425995022,5cyclictest33-21ksoftirqd/223:10:002
12425995022,3cyclictest33-21ksoftirqd/220:55:002
12425995018,7cyclictest33-21ksoftirqd/220:10:292
12425995017,8cyclictest33-21ksoftirqd/219:47:582
12425995014,8cyclictest33-21ksoftirqd/220:22:562
12423995024,6cyclictest9-21ksoftirqd/020:45:180
12423995024,6cyclictest9-21ksoftirqd/020:41:200
12423995023,6cyclictest9-21ksoftirqd/020:55:260
12423995022,3cyclictest9-21ksoftirqd/019:48:040
12423995020,3cyclictest9-21ksoftirqd/021:27:280
12423995017,6cyclictest9-21ksoftirqd/000:15:000
12423995017,4cyclictest9-21ksoftirqd/023:06:580
2847191490,4ptp4l0-21swapper/019:32:130
2847191490,4ptp4l0-21swapper/000:27:380
12425994922,7cyclictest33-21ksoftirqd/223:00:132
12425994922,6cyclictest33-21ksoftirqd/222:40:282
12425994917,7cyclictest33-21ksoftirqd/220:28:342
12425994917,7cyclictest33-21ksoftirqd/200:17:202
1242399495,7cyclictest9-21ksoftirqd/022:26:320
12423994927,3cyclictest9-21ksoftirqd/021:34:030
12423994923,6cyclictest9-21ksoftirqd/000:21:080
12423994922,6cyclictest9-21ksoftirqd/023:00:140
12423994920,3cyclictest9-21ksoftirqd/022:40:140
12423994917,3cyclictest9-21ksoftirqd/021:52:250
12423994916,7cyclictest9-21ksoftirqd/023:15:000
2847191480,3ptp4l0-21swapper/019:36:010
12426994819,3cyclictest41-21ksoftirqd/321:15:233
12425994823,6cyclictest33-21ksoftirqd/219:10:482
12425994817,8cyclictest33-21ksoftirqd/221:30:112
12425994817,5cyclictest33-21ksoftirqd/222:20:012
1242599480,7cyclictest16864-21systemctl19:20:002
12423994824,6cyclictest9-21ksoftirqd/021:49:520
12423994824,6cyclictest9-21ksoftirqd/020:14:410
12423994823,6cyclictest9-21ksoftirqd/021:07:070
12423994821,3cyclictest9-21ksoftirqd/020:00:090
12423994818,3cyclictest9-21ksoftirqd/023:46:190
2847191470,4ptp4l0-21swapper/320:25:493
1242599473,7cyclictest17593-21kworker/2:020:05:162
12425994719,7cyclictest33-21ksoftirqd/220:40:162
12425994719,4cyclictest33-21ksoftirqd/222:45:002
12425994718,7cyclictest33-21ksoftirqd/222:31:042
12423994728,8cyclictest9-21ksoftirqd/019:25:160
12423994728,8cyclictest9-21ksoftirqd/019:25:150
12423994727,10cyclictest9-21ksoftirqd/021:55:140
12423994723,6cyclictest9-21ksoftirqd/019:20:450
12423994718,3cyclictest9-21ksoftirqd/022:20:010
2847191460,4ptp4l0-21swapper/222:25:352
2847191460,4ptp4l0-21swapper/019:16:530
2847191460,11ptp4l15762-21missed_timers19:15:221
12425994627,8cyclictest33-21ksoftirqd/221:15:042
12425994616,6cyclictest33-21ksoftirqd/223:20:142
12425994615,7cyclictest33-21ksoftirqd/220:18:272
12425994614,7cyclictest33-21ksoftirqd/223:10:242
1242399464,3cyclictest9-21ksoftirqd/020:30:320
12423994627,8cyclictest9-21ksoftirqd/000:10:000
12423994626,8cyclictest9-21ksoftirqd/022:54:590
12423994617,3cyclictest9-21ksoftirqd/020:35:220
2847191450,3ptp4l0-21swapper/322:44:483
1242599456,6cyclictest33-21ksoftirqd/221:50:002
1242599456,3cyclictest33-21ksoftirqd/200:30:202
12425994526,9cyclictest33-21ksoftirqd/200:38:042
12425994525,10cyclictest33-21ksoftirqd/200:03:012
12425994524,11cyclictest33-21ksoftirqd/219:42:172
12425994517,5cyclictest33-21ksoftirqd/222:00:232
12425994515,4cyclictest33-21ksoftirqd/221:20:302
12425994513,8cyclictest33-21ksoftirqd/223:17:492
12423994528,7cyclictest9-21ksoftirqd/020:24:400
2847191440,11ptp4l0-21swapper/221:56:512
1242599445,5cyclictest33-21ksoftirqd/220:04:542
12425994426,4cyclictest33-21ksoftirqd/220:46:582
12425994425,9cyclictest33-21ksoftirqd/223:44:132
12425994416,4cyclictest33-21ksoftirqd/221:50:432
12425994414,8cyclictest33-21ksoftirqd/223:55:302
1242399445,3cyclictest9-21ksoftirqd/021:36:150
12423994425,8cyclictest9-21ksoftirqd/019:59:010
1242399441,3cyclictest9-21ksoftirqd/000:35:120
2847191430,6ptp4l0-21swapper/019:05:280
2847191430,5ptp4l0-21swapper/123:00:181
2847191430,4ptp4l28120-21apt-get23:40:120
2847191430,4ptp4l15840-21apt-get23:10:081
2847191430,10ptp4l0-21swapper/221:41:112
1242599433,5cyclictest33-21ksoftirqd/200:28:552
1242399435,2cyclictest9-21ksoftirqd/023:26:400
12423994316,3cyclictest9-21ksoftirqd/020:05:140
12423994315,7cyclictest9-21ksoftirqd/023:50:160
12423994315,7cyclictest9-21ksoftirqd/023:50:160
2847191420,6ptp4l0-21swapper/321:10:203
2847191420,4ptp4l4542-21apt-get21:15:121
2847191420,4ptp4l29698-21memory19:45:201
2847191420,4ptp4l23913-21hddtemp_smartct23:30:111
2847191420,4ptp4l0-21swapper/100:02:181
2847191420,3ptp4l17100-21apt-get19:20:121
2847191420,3ptp4l12623-21apt-get20:20:091
2847191420,3ptp4l1204-21python21:05:281
12425994213,12cyclictest33-21ksoftirqd/220:35:202
1242499421,11cyclictest8284-21df_inode20:10:181
12423994210,5cyclictest9-21ksoftirqd/022:21:020
1242399420,7cyclictest9-21ksoftirqd/022:46:320
2847191410,5ptp4l28195-21/usr/sbin/munin20:55:191
2847191410,5ptp4l0-21swapper/320:20:123
2847191410,5ptp4l0-21swapper/122:20:011
2847191410,5ptp4l0-21swapper/120:15:371
2847191410,4ptp4l5031-21/usr/sbin/munin20:00:291
2847191410,4ptp4l17556-21meminfo00:35:211
2847191410,4ptp4l17385-21apt-get21:50:071
2847191410,4ptp4l17256-21apt-get20:30:161
2847191410,4ptp4l15897-21meminfo20:25:181
2847191410,4ptp4l1293-21snmpd19:34:293
2847191410,4ptp4l10777-21df22:55:131
2847191410,4ptp4l10777-21df22:55:131
2847191410,4ptp4l0-21swapper/323:42:293
2847191410,4ptp4l0-21swapper/323:34:493
2847191410,4ptp4l0-21swapper/323:26:113
2847191410,4ptp4l0-21swapper/322:10:373
2847191410,4ptp4l0-21swapper/300:07:233
2847191410,4ptp4l0-21swapper/122:10:111
2847191410,4ptp4l0-21swapper/120:47:581
2847191410,3ptp4l9921-21ntp_states00:15:213
2847191410,3ptp4l8322-21ls21:20:351
2847191410,3ptp4l25791-21apt-get23:35:120
2847191410,3ptp4l0-21swapper/321:31:063
2847191410,3ptp4l0-21swapper/122:40:411
1242599410,2cyclictest121rcu_preempt23:46:442
1242399414,6cyclictest9-21ksoftirqd/022:30:170
2847191400,5ptp4l0-21swapper/322:50:083
2847191400,5ptp4l0-21swapper/322:30:223
2847191400,4ptp4l3514-21sendmail_mailqu21:10:271
2847191400,4ptp4l23055-21memory22:00:191
2847191400,4ptp4l13036-21/usr/sbin/munin19:10:181
2847191400,4ptp4l10249-21/usr/sbin/munin21:30:121
2847191400,4ptp4l0-21swapper/323:59:023
2847191400,4ptp4l0-21swapper/323:49:243
2847191400,4ptp4l0-21swapper/323:37:363
2847191400,4ptp4l0-21swapper/323:24:513
2847191400,4ptp4l0-21swapper/323:01:073
2847191400,4ptp4l0-21swapper/321:46:343
2847191400,4ptp4l0-21swapper/321:25:023
2847191400,4ptp4l0-21swapper/321:05:373
2847191400,4ptp4l0-21swapper/320:54:023
2847191400,4ptp4l0-21swapper/320:33:413
2847191400,4ptp4l0-21swapper/320:06:123
2847191400,4ptp4l0-21swapper/319:45:243
2847191400,4ptp4l0-21swapper/319:27:353
2847191400,4ptp4l0-21swapper/319:27:353
2847191400,4ptp4l0-21swapper/122:50:171
2847191400,4ptp4l0-21swapper/122:32:091
2847191400,4ptp4l0-21swapper/122:15:111
2847191400,4ptp4l0-21swapper/121:48:571
2847191400,4ptp4l0-21swapper/121:42:211
2847191400,4ptp4l0-21swapper/121:00:261
2847191400,4ptp4l0-21swapper/119:30:081
2847191400,4ptp4l0-21swapper/100:21:031
2847191400,3ptp4l945-21apt-get23:55:121
2847191400,3ptp4l936-21apt-get19:55:121
2847191400,3ptp4l6475-21apt-get00:10:081
2847191400,3ptp4l5659-21apt-get20:05:121
2847191400,3ptp4l28431-21df_inode23:40:131
2847191400,3ptp4l21680-21apt-get20:40:011
2847191400,3ptp4l19992-21iostat_ios19:25:181
2847191400,3ptp4l19992-21iostat_ios19:25:181
2847191400,3ptp4l17726-21ntp_states00:35:243
2847191400,3ptp4l0-21swapper/121:30:011
2847191390,5ptp4l0-21swapper/323:18:253
2847191390,4ptp4l7208-21forks21:20:173
2847191390,4ptp4l0-21swapper/323:54:413
2847191390,4ptp4l0-21swapper/323:54:413
2847191390,4ptp4l0-21swapper/323:10:503
2847191390,4ptp4l0-21swapper/323:07:273
2847191390,4ptp4l0-21swapper/322:55:393
2847191390,4ptp4l0-21swapper/322:55:393
2847191390,4ptp4l0-21swapper/322:35:173
2847191390,4ptp4l0-21swapper/322:19:593
2847191390,4ptp4l0-21swapper/322:06:003
2847191390,4ptp4l0-21swapper/322:01:573
2847191390,4ptp4l0-21swapper/321:53:123
2847191390,4ptp4l0-21swapper/321:41:323
2847191390,4ptp4l0-21swapper/320:45:253
2847191390,4ptp4l0-21swapper/320:43:273
2847191390,4ptp4l0-21swapper/320:37:373
2847191390,4ptp4l0-21swapper/320:19:493
2847191390,4ptp4l0-21swapper/320:02:473
2847191390,4ptp4l0-21swapper/319:56:203
2847191390,4ptp4l0-21swapper/319:35:483
2847191390,4ptp4l0-21swapper/319:20:043
2847191390,4ptp4l0-21swapper/319:19:423
2847191390,4ptp4l0-21swapper/319:11:533
2847191390,4ptp4l0-21swapper/300:26:193
2847191390,4ptp4l0-21swapper/300:10:563
2847191390,4ptp4l0-21swapper/222:08:362
2847191390,4ptp4l0-21swapper/123:45:411
2847191390,4ptp4l0-21swapper/123:35:121
2847191390,4ptp4l0-21swapper/123:21:431
2847191390,4ptp4l0-21swapper/123:18:051
2847191390,4ptp4l0-21swapper/123:07:441
2847191390,4ptp4l0-21swapper/122:35:041
2847191390,4ptp4l0-21swapper/122:25:471
2847191390,4ptp4l0-21swapper/121:35:091
2847191390,4ptp4l0-21swapper/120:50:351
2847191390,3ptp4l29937-21/usr/sbin/munin21:00:183
2847191390,3ptp4l19915-21df21:55:131
2847191390,3ptp4l15407-21ntp_states00:30:191
2847191390,3ptp4l0-21swapper/320:57:303
2847191390,3ptp4l0-21swapper/319:50:323
2847191390,3ptp4l0-21swapper/122:05:571
2847191390,3ptp4l0-21swapper/100:25:521
2847191380,4ptp4l0-21swapper/321:55:513
2847191380,4ptp4l0-21swapper/321:35:043
2847191380,4ptp4l0-21swapper/320:10:193
2847191380,4ptp4l0-21swapper/300:32:423
2847191380,4ptp4l0-21swapper/300:22:123
2847191380,4ptp4l0-21swapper/119:40:141
2847191380,3ptp4l0-21swapper/322:47:473
2847191380,3ptp4l0-21swapper/322:26:493
2847191380,3ptp4l0-21swapper/123:25:131
2847191380,3ptp4l0-21swapper/100:06:241
2847191370,4ptp4l0-21swapper/319:43:043
100612265,13sleep310076-21apt-get19:05:123
121322144,6sleep20-21swapper/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional