You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-28 - 12:04
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot3.osadl.org (updated Sun Apr 28, 2024 00:46:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2630523070,3sleep30-21swapper/319:05:193
1128721470,7sleep12773399cyclictest20:55:411
576981270,99rtkit-daemon6357-21apache219:06:501
3221621160,8sleep12773399cyclictest22:50:111
27732998570,8cyclictest9-21ksoftirqd/021:15:090
825591740,2ptp4l401ktimersoftd/323:30:593
27733997212,12cyclictest5406-21awk21:50:321
27734997128,4cyclictest33-21ksoftirqd/220:00:012
27733997110,13cyclictest19767-21cpuspeed_turbos21:15:111
825591690,2ptp4l401ktimersoftd/321:05:133
54082690,3sleep00-21swapper/021:50:320
273422675,11sleep20-21swapper/219:08:422
2773399639,14cyclictest12251-21fschecks_time23:15:171
283472620,5sleep09-21ksoftirqd/021:31:360
25532620,7sleep12773399cyclictest22:55:121
27734996154,4cyclictest33-21ksoftirqd/219:45:082
278992600,2sleep20-21swapper/222:40:202
27734996053,4cyclictest33-21ksoftirqd/221:40:122
27734996053,4cyclictest33-21ksoftirqd/200:30:132
278952590,8sleep02773299cyclictest23:50:010
2773399582,12cyclictest9249-21cat23:10:001
27734995750,4cyclictest33-21ksoftirqd/223:55:112
27734995750,4cyclictest33-21ksoftirqd/222:50:092
27734995749,4cyclictest33-21ksoftirqd/222:30:122
27734995346,4cyclictest33-21ksoftirqd/223:45:112
27734995346,4cyclictest33-21ksoftirqd/223:10:142
27735995246,3cyclictest41-21ksoftirqd/300:40:003
27734995244,5cyclictest33-21ksoftirqd/222:55:122
27734995216,4cyclictest33-21ksoftirqd/219:12:572
2773499517,3cyclictest121rcu_preempt22:39:592
27734995144,4cyclictest33-21ksoftirqd/221:54:592
27734995144,4cyclictest33-21ksoftirqd/220:55:202
27734995144,4cyclictest33-21ksoftirqd/220:05:102
27734995144,4cyclictest33-21ksoftirqd/200:30:012
27734995143,5cyclictest33-21ksoftirqd/221:00:102
27734995043,4cyclictest33-21ksoftirqd/222:20:102
27734995043,3cyclictest33-21ksoftirqd/221:50:012
27735994942,4cyclictest41-21ksoftirqd/320:55:433
27734994917,8cyclictest33-21ksoftirqd/220:45:232
27734994915,5cyclictest33-21ksoftirqd/223:20:272
27732994932,7cyclictest9-21ksoftirqd/020:00:000
27735994843,3cyclictest41-21ksoftirqd/322:55:013
27735994841,4cyclictest41-21ksoftirqd/322:00:013
27734994841,4cyclictest33-21ksoftirqd/200:40:002
27732994841,4cyclictest9-21ksoftirqd/021:00:000
27732994841,4cyclictest9-21ksoftirqd/000:25:130
27735994740,4cyclictest41-21ksoftirqd/300:25:133
27734994741,3cyclictest33-21ksoftirqd/219:19:592
27734994740,4cyclictest33-21ksoftirqd/223:30:002
27734994739,4cyclictest33-21ksoftirqd/219:55:002
27734994739,4cyclictest33-21ksoftirqd/219:54:592
27734994734,10cyclictest33-21ksoftirqd/223:00:112
27734994733,3cyclictest33-21ksoftirqd/220:34:312
27734994716,7cyclictest33-21ksoftirqd/221:35:232
27734994715,4cyclictest33-21ksoftirqd/220:25:222
825591460,1ptp4l401ktimersoftd/322:47:183
825591460,1ptp4l401ktimersoftd/322:47:173
27735994640,4cyclictest41-21ksoftirqd/323:20:003
27735994639,4cyclictest41-21ksoftirqd/322:30:153
27734994641,3cyclictest33-21ksoftirqd/221:14:592
27734994640,3cyclictest33-21ksoftirqd/223:40:002
27734994639,4cyclictest33-21ksoftirqd/223:44:592
27734994639,4cyclictest33-21ksoftirqd/223:20:002
27734994639,4cyclictest33-21ksoftirqd/221:30:112
27734994639,4cyclictest33-21ksoftirqd/219:25:332
27734994630,6cyclictest33-21ksoftirqd/220:40:342
54722450,8sleep02773299cyclictest19:30:000
27735994539,4cyclictest41-21ksoftirqd/320:54:593
27734994539,4cyclictest33-21ksoftirqd/223:05:342
27734994538,4cyclictest33-21ksoftirqd/222:05:012
27734994538,4cyclictest33-21ksoftirqd/219:35:142
2773499452,5cyclictest33-21ksoftirqd/220:19:592
2773499450,4cyclictest121rcu_preempt00:15:102
27733994537,5cyclictest30235-21cron20:30:011
2735024514,18sleep00-21swapper/019:08:490
2773499445,36cyclictest14900-21cpuspeed_turbos21:05:132
27734994439,3cyclictest33-21ksoftirqd/220:55:002
27734994438,3cyclictest33-21ksoftirqd/222:30:012
27734994438,3cyclictest33-21ksoftirqd/222:10:002
27734994437,4cyclictest33-21ksoftirqd/220:05:002
27734994436,4cyclictest33-21ksoftirqd/219:20:152
2773499441,3cyclictest121rcu_preempt22:10:312
27733994410,12cyclictest0-21swapper/119:13:001
27732994438,3cyclictest9-21ksoftirqd/022:35:000
27732994437,4cyclictest9-21ksoftirqd/022:10:010
27735994338,3cyclictest41-21ksoftirqd/320:20:003
27735994337,4cyclictest41-21ksoftirqd/319:40:003
27735994337,3cyclictest41-21ksoftirqd/320:00:003
27735994336,4cyclictest41-21ksoftirqd/319:30:003
27735994333,4cyclictest41-21ksoftirqd/319:13:013
27734994337,3cyclictest33-21ksoftirqd/219:35:002
27734994336,4cyclictest33-21ksoftirqd/220:25:002
27734994336,4cyclictest33-21ksoftirqd/200:10:342
27734994336,4cyclictest33-21ksoftirqd/200:10:342
27734994336,3cyclictest33-21ksoftirqd/221:55:012
27734994335,4cyclictest33-21ksoftirqd/221:25:322
27732994336,4cyclictest9-21ksoftirqd/023:20:000
27735994236,4cyclictest41-21ksoftirqd/300:10:003
27735994235,4cyclictest41-21ksoftirqd/321:50:133
27734994235,4cyclictest33-21ksoftirqd/223:50:312
27734994235,4cyclictest33-21ksoftirqd/200:00:172
2773499423,12cyclictest33-21ksoftirqd/220:35:352
27734994230,10cyclictest33-21ksoftirqd/221:15:182
27733994236,4cyclictest25-21ksoftirqd/120:52:171
27733994235,4cyclictest23217-21cron20:15:001
27733994212,12cyclictest24584-21chrt22:33:531
27732994237,3cyclictest9-21ksoftirqd/022:19:590
27732994237,3cyclictest9-21ksoftirqd/000:00:000
27732994236,4cyclictest9-21ksoftirqd/022:40:000
27732994235,4cyclictest9-21ksoftirqd/000:20:000
27735994135,3cyclictest41-21ksoftirqd/321:30:013
27735994135,3cyclictest41-21ksoftirqd/319:55:003
27735994135,3cyclictest41-21ksoftirqd/319:55:003
27735994134,4cyclictest41-21ksoftirqd/300:30:153
27734994135,4cyclictest33-21ksoftirqd/222:45:212
27734994135,4cyclictest33-21ksoftirqd/222:45:202
27734994135,3cyclictest33-21ksoftirqd/222:20:012
27734994135,3cyclictest33-21ksoftirqd/200:20:192
27732994136,3cyclictest9-21ksoftirqd/021:50:000
27732994135,3cyclictest9-21ksoftirqd/023:05:000
27732994135,3cyclictest9-21ksoftirqd/020:54:590
27732994135,3cyclictest9-21ksoftirqd/000:15:000
27732994135,3cyclictest9-21ksoftirqd/000:14:590
27732994134,4cyclictest9-21ksoftirqd/021:40:000
27732994131,6cyclictest9-21ksoftirqd/000:00:140
825591400,1ptp4l401ktimersoftd/319:20:223
27735994034,4cyclictest41-21ksoftirqd/321:42:223
27735994034,4cyclictest41-21ksoftirqd/320:25:213
27735994033,4cyclictest41-21ksoftirqd/322:55:213
27734994034,3cyclictest33-21ksoftirqd/200:05:212
27732994033,4cyclictest9-21ksoftirqd/019:10:090
825591390,1ptp4l401ktimersoftd/320:45:213
825591390,1ptp4l401ktimersoftd/320:34:313
27735993934,3cyclictest41-21ksoftirqd/321:20:003
27735993932,4cyclictest41-21ksoftirqd/323:40:233
27734993934,3cyclictest33-21ksoftirqd/221:20:182
27734993934,3cyclictest33-21ksoftirqd/221:20:172
27734993932,4cyclictest33-21ksoftirqd/223:30:192
27734993931,5cyclictest33-21ksoftirqd/220:12:262
27733993932,5cyclictest25-21ksoftirqd/121:37:221
27732993934,3cyclictest9-21ksoftirqd/023:10:010
27732993934,3cyclictest9-21ksoftirqd/022:15:000
27732993933,4cyclictest9-21ksoftirqd/020:05:010
2773599385,13cyclictest121rcu_preempt19:45:143
27735993833,3cyclictest41-21ksoftirqd/323:10:153
27735993833,3cyclictest41-21ksoftirqd/320:00:013
27735993832,3cyclictest41-21ksoftirqd/322:40:003
27735993832,3cyclictest41-21ksoftirqd/322:10:023
27735993831,4cyclictest41-21ksoftirqd/323:05:003
27735993831,4cyclictest41-21ksoftirqd/321:15:013
27733993831,5cyclictest25-21ksoftirqd/122:17:141
27733993831,4cyclictest25-21ksoftirqd/120:45:121
27732993833,3cyclictest9-21ksoftirqd/023:25:000
27732993833,3cyclictest9-21ksoftirqd/021:05:000
27732993832,4cyclictest9-21ksoftirqd/022:20:120
27732993831,4cyclictest9-21ksoftirqd/023:15:000
27732993831,4cyclictest9-21ksoftirqd/019:53:370
27732993831,4cyclictest9-21ksoftirqd/019:53:370
825591370,1ptp4l401ktimersoftd/320:10:183
27735993732,3cyclictest41-21ksoftirqd/319:45:003
27735993732,3cyclictest41-21ksoftirqd/300:25:013
27735993732,3cyclictest41-21ksoftirqd/300:00:153
27735993731,4cyclictest41-21ksoftirqd/323:25:003
27735993731,4cyclictest41-21ksoftirqd/300:00:003
27734993731,4cyclictest33-21ksoftirqd/219:45:002
27733993732,3cyclictest206-21systemd-journal23:59:551
27733993731,4cyclictest25-21ksoftirqd/119:31:581
27733993731,4cyclictest25-21ksoftirqd/119:22:071
27733993730,4cyclictest25-21ksoftirqd/121:26:311
27733993712,21cyclictest18917-21cut23:30:141
27732993732,3cyclictest9-21ksoftirqd/020:15:000
27732993731,4cyclictest9-21ksoftirqd/021:20:260
27732993731,4cyclictest9-21ksoftirqd/021:20:250
27732993730,4cyclictest9-21ksoftirqd/022:05:000
27732993730,4cyclictest9-21ksoftirqd/021:05:200
27735993631,3cyclictest41-21ksoftirqd/320:35:243
27735993630,4cyclictest41-21ksoftirqd/321:50:003
27735993630,3cyclictest41-21ksoftirqd/322:15:113
27735993630,3cyclictest41-21ksoftirqd/300:15:003
27735993630,3cyclictest41-21ksoftirqd/300:14:593
27735993629,4cyclictest41-21ksoftirqd/323:38:383
27735993629,4cyclictest41-21ksoftirqd/323:07:033
27735993629,4cyclictest41-21ksoftirqd/322:10:213
27735993629,4cyclictest41-21ksoftirqd/320:20:063
27733993630,4cyclictest25-21ksoftirqd/123:47:121
27733993630,4cyclictest25-21ksoftirqd/121:04:521
27733993630,4cyclictest25-21ksoftirqd/120:05:351
27733993629,4cyclictest25-21ksoftirqd/122:44:161
27733993629,4cyclictest25-21ksoftirqd/100:00:011
27733993612,20cyclictest24627-21grep20:15:281
27732993631,3cyclictest9-21ksoftirqd/020:15:230
27732993631,3cyclictest9-21ksoftirqd/000:40:000
27732993631,3cyclictest9-21ksoftirqd/000:30:170
27732993630,4cyclictest9-21ksoftirqd/022:40:230
27732993630,3cyclictest9-21ksoftirqd/019:15:120
27732993629,4cyclictest9-21ksoftirqd/023:50:080
27732993629,4cyclictest9-21ksoftirqd/023:37:310
27732993629,4cyclictest9-21ksoftirqd/020:25:130
27735993530,3cyclictest41-21ksoftirqd/319:30:353
27735993529,3cyclictest41-21ksoftirqd/323:28:093
27735993529,3cyclictest41-21ksoftirqd/321:30:203
27735993529,3cyclictest41-21ksoftirqd/300:20:003
27735993528,4cyclictest41-21ksoftirqd/319:16:343
27733993529,4cyclictest25-21ksoftirqd/123:37:211
27733993529,4cyclictest25-21ksoftirqd/121:13:191
27733993528,4cyclictest25-21ksoftirqd/122:38:551
27733993528,4cyclictest25-21ksoftirqd/121:46:151
27733993528,4cyclictest25-21ksoftirqd/120:20:231
27733993528,4cyclictest25-21ksoftirqd/100:30:101
27733993528,4cyclictest25-21ksoftirqd/100:28:471
27733993527,5cyclictest25-21ksoftirqd/122:24:301
2773299353,3cyclictest121rcu_preempt00:20:130
27732993529,3cyclictest9-21ksoftirqd/019:45:090
27732993528,4cyclictest9-21ksoftirqd/022:51:310
27732993528,4cyclictest9-21ksoftirqd/022:27:310
27732993528,4cyclictest9-21ksoftirqd/020:40:490
27732993528,4cyclictest9-21ksoftirqd/020:05:120
27732993518,5cyclictest9-21ksoftirqd/019:24:590
27735993428,3cyclictest41-21ksoftirqd/323:49:263
27735993428,3cyclictest41-21ksoftirqd/321:35:143
27735993427,4cyclictest41-21ksoftirqd/321:02:333
27735993427,4cyclictest41-21ksoftirqd/320:06:253
27733993427,5cyclictest25-21ksoftirqd/119:39:081
27733993427,4cyclictest25-21ksoftirqd/120:41:121
27733993427,4cyclictest25-21ksoftirqd/100:10:541
27733993427,4cyclictest25-21ksoftirqd/100:10:531
27733993419,9cyclictest9888-21apt-get00:20:081
27732993429,3cyclictest9-21ksoftirqd/023:00:000
27732993428,4cyclictest9-21ksoftirqd/021:55:330
27732993428,4cyclictest9-21ksoftirqd/019:35:270
27732993428,3cyclictest9-21ksoftirqd/021:10:260
27732993428,3cyclictest9-21ksoftirqd/000:06:540
27732993427,4cyclictest9-21ksoftirqd/020:48:590
27732993427,4cyclictest9-21ksoftirqd/020:38:130
825591330,1ptp4l401ktimersoftd/322:23:413
27735993327,4cyclictest41-21ksoftirqd/321:20:293
27735993327,4cyclictest41-21ksoftirqd/321:20:283
27735993326,4cyclictest41-21ksoftirqd/322:02:033
27733993328,3cyclictest25-21ksoftirqd/121:22:421
27733993328,3cyclictest25-21ksoftirqd/121:22:421
27733993327,4cyclictest25-21ksoftirqd/122:47:101
27733993327,4cyclictest25-21ksoftirqd/122:47:101
27733993326,4cyclictest25-21ksoftirqd/123:11:431
27733993326,4cyclictest25-21ksoftirqd/123:01:311
27733993326,4cyclictest25-21ksoftirqd/121:05:401
27733993326,4cyclictest25-21ksoftirqd/120:04:141
27733993326,4cyclictest25-21ksoftirqd/119:40:491
27733993326,4cyclictest25-21ksoftirqd/119:26:041
27732993327,4cyclictest9-21ksoftirqd/020:30:310
27732993326,4cyclictest9-21ksoftirqd/021:40:330
27735993226,3cyclictest41-21ksoftirqd/323:51:333
27735993226,3cyclictest41-21ksoftirqd/320:40:113
27735993225,4cyclictest41-21ksoftirqd/322:41:423
27733993226,4cyclictest25-21ksoftirqd/121:34:121
27733993226,4cyclictest25-21ksoftirqd/119:17:281
27733993225,4cyclictest25-21ksoftirqd/119:53:181
27733993225,4cyclictest25-21ksoftirqd/119:53:171
27733993225,4cyclictest25-21ksoftirqd/119:49:241
27732993227,3cyclictest9-21ksoftirqd/023:40:230
27732993226,4cyclictest9-21ksoftirqd/023:25:130
27732993225,4cyclictest9-21ksoftirqd/019:40:560
27732993225,4cyclictest9-21ksoftirqd/019:33:380
2773599314,3cyclictest121rcu_preempt22:30:003
27733993124,4cyclictest25-21ksoftirqd/123:51:451
27733993124,4cyclictest25-21ksoftirqd/120:36:421
27733993112,15cyclictest19126-21sendmail00:40:001
27732993125,4cyclictest9-21ksoftirqd/022:45:170
27732993125,4cyclictest9-21ksoftirqd/022:45:170
27732993124,4cyclictest9-21ksoftirqd/020:23:200
27733993024,4cyclictest25-21ksoftirqd/122:13:451
27733993024,4cyclictest25-21ksoftirqd/100:08:391
27733993024,3cyclictest25-21ksoftirqd/123:44:361
27733993023,4cyclictest25-21ksoftirqd/122:25:091
27733993023,4cyclictest25-21ksoftirqd/122:06:541
27733993023,4cyclictest25-21ksoftirqd/122:03:341
27733993023,4cyclictest25-21ksoftirqd/100:16:391
27733993015,11cyclictest0-21swapper/121:55:121
27733992923,4cyclictest25-21ksoftirqd/123:24:121
27733992923,3cyclictest25-21ksoftirqd/123:26:281
27733992922,4cyclictest25-21ksoftirqd/119:56:471
27732992923,4cyclictest9-21ksoftirqd/021:25:210
27732992923,3cyclictest9-21ksoftirqd/023:32:080
27733992816,9cyclictest31946-21ps21:40:211
27733992815,9cyclictest31034-21grep20:30:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional