You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-02 - 11:51
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackdslot2.osadl.org (updated Fri May 02, 2025 00:45:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
82812214170,33sleep10-21swapper/119:09:031
137391209176,22phc2sys0-21swapper/319:07:133
61782208174,22sleep20-21swapper/219:05:092
81832204171,22sleep00-21swapper/019:07:440
865199762,33cyclictest22540-21latency_hist21:55:002
865199690,25cyclictest28324-21apt-config21:00:012
865199690,25cyclictest28324-21apt-config21:00:012
865199690,23cyclictest24338-21grep20:50:212
865199680,24cyclictest20176-21grep20:40:292
865199670,30cyclictest22067-21grep00:05:282
865199661,24cyclictest12421-21grep19:15:252
865199660,23cyclictest6261-21grep21:20:222
865199651,24cyclictest13099-21phc2sys-jitter20:25:252
865199651,24cyclictest13099-21phc2sys-jitter20:25:252
865199651,22cyclictest24905-21apt-get22:00:002
865199650,26cyclictest0-21swapper/219:15:002
865199650,25cyclictest22185-21sed19:35:272
865199650,25cyclictest17068-21grep23:55:232
865199650,23cyclictest18805-21ls22:50:292
865199641,25cyclictest27805-21wc00:20:002
865199640,23cyclictest32198-21cut21:05:252
109772640,2sleep1865099cyclictest21:30:201
865199630,25cyclictest19419-21find21:45:262
865199630,25cyclictest1876-21sed23:25:152
865199630,24cyclictest19793-21tail19:30:272
865199630,24cyclictest12670-21cstates22:40:142
865199630,23cyclictest4750-21cut23:30:222
865199630,23cyclictest31422-21phc2sys00:25:242
865199620,24cyclictest7318-21sed23:35:212
865199620,23cyclictest8846-21cpuspeed_turbos23:40:142
865199620,21cyclictest29151-21latency_hist23:15:002
865199620,19cyclictest0-21swapper/220:45:292
8649996231,4cyclictest9-21ksoftirqd/022:20:180
8649996231,4cyclictest9-21ksoftirqd/021:35:000
865199611,25cyclictest27730-21/usr/sbin/munin22:05:272
865199610,24cyclictest28953-21sed00:20:242
865199601,22cyclictest14830-21mailstats23:50:282
8650996025,4cyclictest25-21ksoftirqd/100:00:141
8650996024,4cyclictest25-21ksoftirqd/119:55:271
865199590,26cyclictest0-21swapper/223:05:152
865199590,24cyclictest15710-21bc21:40:142
865199590,22cyclictest19624-21sensors00:00:262
865199590,21cyclictest23364-21fschecks_time19:40:242
112212590,4sleep111223-21unixbench_singl20:20:311
865199580,31cyclictest27450-21sed19:50:002
865199580,24cyclictest12527-21sensors23:45:282
865199580,22cyclictest31566-21/usr/sbin/munin23:20:292
8650995825,4cyclictest25-21ksoftirqd/123:10:241
8650995824,3cyclictest25-21ksoftirqd/119:25:281
8649995831,4cyclictest9-21ksoftirqd/000:15:200
8649995829,3cyclictest9-21ksoftirqd/022:45:240
865199571,23cyclictest29582-21diskstats23:15:172
865199570,24cyclictest5352-21interrupts20:10:212
865199570,24cyclictest14980-21sed19:20:272
865199570,23cyclictest29033-21diskstats21:00:212
865199570,19cyclictest0-21swapper/221:15:002
8650995728,4cyclictest25-21ksoftirqd/123:20:161
8650995727,3cyclictest25-21ksoftirqd/123:00:271
8650995726,3cyclictest25-21ksoftirqd/119:10:211
8650995722,3cyclictest25-21ksoftirqd/120:05:161
8650995722,3cyclictest25-21ksoftirqd/119:40:241
8649995729,3cyclictest9-21ksoftirqd/021:05:010
8649995727,4cyclictest9-21ksoftirqd/023:40:250
8649995724,4cyclictest9-21ksoftirqd/020:05:010
865199560,29cyclictest28976-21cut19:50:272
8650995626,3cyclictest25-21ksoftirqd/120:42:111
8650995626,2cyclictest25-21ksoftirqd/100:19:391
8650995624,4cyclictest25-21ksoftirqd/123:30:001
8650995622,4cyclictest25-21ksoftirqd/120:50:271
8650995621,4cyclictest25-21ksoftirqd/120:30:151
8650995621,4cyclictest25-21ksoftirqd/119:50:221
8650995621,3cyclictest25-21ksoftirqd/123:55:181
8649995627,3cyclictest9-21ksoftirqd/000:00:150
8649995624,3cyclictest9-21ksoftirqd/023:15:280
8649995623,4cyclictest9-21ksoftirqd/000:07:370
865199550,23cyclictest32334-21acpi20:00:152
865199550,17cyclictest0-21swapper/220:05:212
8650995527,3cyclictest25-21ksoftirqd/120:45:191
8650995525,4cyclictest25-21ksoftirqd/123:50:171
8650995522,3cyclictest25-21ksoftirqd/119:15:261
8650995521,3cyclictest25-21ksoftirqd/119:45:281
8650995520,3cyclictest25-21ksoftirqd/121:10:231
8649995525,4cyclictest9-21ksoftirqd/020:40:000
865199541,23cyclictest17743-21netstat20:35:222
865199541,21cyclictest24154-21meminfo00:10:232
8650995421,3cyclictest25-21ksoftirqd/120:25:251
8650995421,3cyclictest25-21ksoftirqd/120:25:251
8650995419,7cyclictest25-21ksoftirqd/100:10:131
8649995427,3cyclictest9-21ksoftirqd/019:50:240
8649995426,3cyclictest9-21ksoftirqd/019:25:000
864999541,6cyclictest121rcu_preempt20:19:590
137291540,1ptp4l401ktimersoftd/320:30:173
865199532,20cyclictest3467-21cut00:35:202
865199530,23cyclictest1389-21phc2sys-jitter00:30:242
865199530,21cyclictest6349-21wc22:25:242
8650995326,3cyclictest25-21ksoftirqd/122:20:171
8650995323,4cyclictest25-21ksoftirqd/121:20:271
8650995321,3cyclictest25-21ksoftirqd/122:18:091
8650995318,7cyclictest25-21ksoftirqd/122:30:221
8649995329,4cyclictest9-21ksoftirqd/020:45:240
8649995326,4cyclictest9-21ksoftirqd/021:25:280
8649995326,4cyclictest9-21ksoftirqd/000:35:230
8649995325,4cyclictest9-21ksoftirqd/019:40:210
8649995324,4cyclictest9-21ksoftirqd/022:50:190
8649995323,3cyclictest9-21ksoftirqd/000:25:170
835725335,11sleep30-21swapper/319:10:013
865199521,25cyclictest15921-21grep22:45:222
865199520,25cyclictest892-21irqrtprio22:15:212
865199520,24cyclictest30501-21sort22:10:152
865199520,24cyclictest21714-21cat23:00:002
865199520,24cyclictest11025-21cstates21:30:192
8650995220,4cyclictest25-21ksoftirqd/123:45:191
8650995218,8cyclictest25-21ksoftirqd/122:25:241
8649995223,4cyclictest9-21ksoftirqd/023:35:220
8649995221,4cyclictest9-21ksoftirqd/021:05:270
8649995218,8cyclictest9-21ksoftirqd/022:05:230
33832520,2sleep30-21swapper/320:05:243
8650995122,6cyclictest25-21ksoftirqd/121:40:201
8650995121,3cyclictest25-21ksoftirqd/122:10:161
8650995117,8cyclictest25-21ksoftirqd/120:10:191
8649995122,4cyclictest9-21ksoftirqd/023:05:210
8649995121,3cyclictest9-21ksoftirqd/020:55:320
8649995121,3cyclictest9-21ksoftirqd/020:55:320
8649995120,3cyclictest9-21ksoftirqd/000:20:160
865199500,26cyclictest11802-21date20:25:002
865199500,23cyclictest7940-21df_abs22:30:162
8650995021,3cyclictest25-21ksoftirqd/122:35:251
8649995025,3cyclictest9-21ksoftirqd/021:10:290
8649995023,4cyclictest9-21ksoftirqd/023:25:270
8649995022,4cyclictest9-21ksoftirqd/020:15:000
8649995018,4cyclictest9-21ksoftirqd/019:40:000
865199490,20cyclictest30347-21diskstats19:55:192
865099493,5cyclictest25-21ksoftirqd/123:15:161
8650994931,7cyclictest25-21ksoftirqd/120:40:011
865099492,9cyclictest25-21ksoftirqd/122:40:171
8650994929,3cyclictest25-21ksoftirqd/121:40:011
8650994921,3cyclictest25-21ksoftirqd/122:55:191
8649994926,3cyclictest9-21ksoftirqd/023:45:280
8649994922,4cyclictest9-21ksoftirqd/023:10:240
8649994922,3cyclictest9-21ksoftirqd/022:40:160
8649994922,3cyclictest9-21ksoftirqd/019:45:220
8649994921,4cyclictest9-21ksoftirqd/023:55:240
8649994919,4cyclictest9-21ksoftirqd/022:35:270
865199481,20cyclictest16808-21meminfo19:25:212
865199480,27cyclictest16503-21date20:35:002
865199480,21cyclictest2919-21cstates22:20:152
8650994834,3cyclictest25-21ksoftirqd/121:50:271
8650994828,9cyclictest25-21ksoftirqd/121:00:211
8650994827,9cyclictest25-21ksoftirqd/121:25:191
8650994819,8cyclictest25-21ksoftirqd/120:55:281
8650994819,8cyclictest25-21ksoftirqd/120:55:281
8650994819,7cyclictest25-21ksoftirqd/123:45:011
8650994818,7cyclictest25-21ksoftirqd/122:05:161
8650994817,7cyclictest25-21ksoftirqd/123:05:231
8649994821,4cyclictest9-21ksoftirqd/020:25:180
8649994821,4cyclictest9-21ksoftirqd/020:25:180
864999481,7cyclictest9-21ksoftirqd/021:19:590
864999480,2cyclictest121rcu_preempt21:55:000
8650994732,3cyclictest25-21ksoftirqd/121:45:211
8650994718,8cyclictest25-21ksoftirqd/100:25:281
8649994733,3cyclictest9-21ksoftirqd/021:40:240
8649994720,4cyclictest9-21ksoftirqd/000:10:290
8649994719,3cyclictest9-21ksoftirqd/023:20:190
8649994718,4cyclictest9-21ksoftirqd/019:55:210
865199460,24cyclictest13122-21apt-get21:35:152
8650994632,3cyclictest25-21ksoftirqd/122:00:001
8650994628,10cyclictest25-21ksoftirqd/100:20:191
8650994627,5cyclictest25-21ksoftirqd/123:40:021
8650994619,7cyclictest25-21ksoftirqd/122:50:271
865099461,3cyclictest25-21ksoftirqd/123:30:201
864999468,9cyclictest9-21ksoftirqd/020:10:010
8649994621,3cyclictest9-21ksoftirqd/022:55:220
8649994617,3cyclictest9-21ksoftirqd/020:25:000
864999461,6cyclictest121rcu_preempt22:25:210
865199450,25cyclictest8445-21expr21:25:172
865099458,2cyclictest5482-21kworker/1:122:45:241
8650994529,3cyclictest25-21ksoftirqd/119:22:371
865099452,3cyclictest121rcu_preempt19:35:011
8650994518,8cyclictest25-21ksoftirqd/121:05:261
864999457,10cyclictest9-21ksoftirqd/022:00:000
864999455,3cyclictest9-21ksoftirqd/019:10:180
8649994518,4cyclictest9-21ksoftirqd/000:30:260
8649994517,3cyclictest9-21ksoftirqd/019:25:150
8649994431,4cyclictest9-21ksoftirqd/022:10:260
865099433,2cyclictest25-21ksoftirqd/120:00:301
865099431,3cyclictest25-21ksoftirqd/119:40:011
865099430,2cyclictest121rcu_preempt00:30:201
8649994331,4cyclictest9-21ksoftirqd/022:15:270
8649994331,3cyclictest9-21ksoftirqd/022:35:000
8649994315,4cyclictest9-21ksoftirqd/022:00:150
137291430,0ptp4l401ktimersoftd/319:46:103
137291430,0ptp4l401ktimersoftd/319:25:223
8650994232,3cyclictest25-21ksoftirqd/120:15:161
8649994231,4cyclictest9-21ksoftirqd/023:00:200
8649994227,4cyclictest9-21ksoftirqd/021:40:010
137291420,0ptp4l401ktimersoftd/323:50:213
137291420,0ptp4l401ktimersoftd/321:25:323
8650994125,3cyclictest25-21ksoftirqd/121:15:161
864999419,5cyclictest9-21ksoftirqd/019:30:250
8649994128,3cyclictest9-21ksoftirqd/023:50:200
8649994128,3cyclictest9-21ksoftirqd/020:55:000
8649994127,4cyclictest9-21ksoftirqd/023:35:000
8649994126,4cyclictest9-21ksoftirqd/020:44:590
8649994120,3cyclictest9-21ksoftirqd/020:30:270
865199400,24cyclictest7996-21irqstats20:15:222
865099404,5cyclictest121rcu_preempt00:05:071
865099403,2cyclictest121rcu_preempt22:05:001
8650994031,3cyclictest25-21ksoftirqd/100:35:591
864999400,2cyclictest121rcu_preempt21:45:290
864999399,3cyclictest9-21ksoftirqd/021:20:320
8649993934,3cyclictest9-21ksoftirqd/019:19:590
865199380,27cyclictest0-21swapper/222:38:102
865199380,25cyclictest23428-21timerandwakeup23:00:282
137291380,0ptp4l401ktimersoftd/322:10:193
8652993732,3cyclictest41-21ksoftirqd/321:05:003
42098370,6rtkit-daemon0-21swapper/323:47:013
42098370,2rtkit-daemon0-21swapper/323:20:173
137291370,1ptp4l0-21swapper/300:05:163
137291370,0ptp4l401ktimersoftd/300:34:103
8652993630,4cyclictest41-21ksoftirqd/319:23:163
8652993529,4cyclictest41-21ksoftirqd/322:18:513
8652993429,3cyclictest41-21ksoftirqd/322:00:003
8652993429,3cyclictest41-21ksoftirqd/320:50:163
8652993429,3cyclictest41-21ksoftirqd/319:40:183
8652993428,4cyclictest41-21ksoftirqd/320:29:163
8652993428,4cyclictest41-21ksoftirqd/320:29:163
8652993428,4cyclictest41-21ksoftirqd/319:32:553
8652993328,3cyclictest41-21ksoftirqd/321:45:003
8652993328,3cyclictest41-21ksoftirqd/300:28:453
8652993327,4cyclictest41-21ksoftirqd/323:39:483
8652993327,4cyclictest41-21ksoftirqd/323:29:083
8652993327,4cyclictest41-21ksoftirqd/323:09:503
8652993327,4cyclictest41-21ksoftirqd/322:31:553
8652993327,4cyclictest41-21ksoftirqd/321:36:113
8652993327,4cyclictest41-21ksoftirqd/320:56:273
8652993327,4cyclictest41-21ksoftirqd/320:56:273
8652993327,4cyclictest41-21ksoftirqd/319:58:143
8652993227,3cyclictest41-21ksoftirqd/322:30:003
8652993227,3cyclictest41-21ksoftirqd/320:22:313
8652993226,4cyclictest41-21ksoftirqd/321:19:383
8652993226,4cyclictest41-21ksoftirqd/320:41:273
8652993226,4cyclictest41-21ksoftirqd/320:03:073
8652993126,3cyclictest18704-21cron20:40:013
8652993125,4cyclictest41-21ksoftirqd/323:41:153
8652993125,4cyclictest41-21ksoftirqd/322:06:233
137291310,8ptp4l401ktimersoftd/323:10:203
8652993025,3cyclictest41-21ksoftirqd/319:35:473
8652993024,4cyclictest41-21ksoftirqd/323:30:373
8652993024,4cyclictest41-21ksoftirqd/321:11:393
137291300,0ptp4l401ktimersoftd/322:41:033
137291300,0ptp4l401ktimersoftd/321:23:163
8652992925,3cyclictest41-21ksoftirqd/323:02:103
8652992924,3cyclictest41-21ksoftirqd/321:54:103
8652992923,4cyclictest41-21ksoftirqd/322:49:013
8652992923,4cyclictest41-21ksoftirqd/322:38:293
8652992923,4cyclictest41-21ksoftirqd/322:01:423
8652992923,4cyclictest41-21ksoftirqd/319:19:473
8652992922,5cyclictest41-21ksoftirqd/320:15:003
865199290,27cyclictest51592sleep221:15:472
137291290,0ptp4l401ktimersoftd/320:45:133
137291290,0ptp4l401ktimersoftd/319:50:483
8652992824,2cyclictest41-21ksoftirqd/321:31:263
8652992823,3cyclictest41-21ksoftirqd/323:57:013
8652992823,3cyclictest41-21ksoftirqd/321:05:133
8652992823,3cyclictest41-21ksoftirqd/300:37:573
8652992823,3cyclictest41-21ksoftirqd/300:03:393
2599422823,2sleep341-21ksoftirqd/300:15:183
137291280,0ptp4l401ktimersoftd/322:55:073
137291280,0ptp4l401ktimersoftd/322:21:293
8652992721,4cyclictest41-21ksoftirqd/300:22:513
865199271,23cyclictest25325-21cat22:00:022
137291270,0ptp4l401ktimersoftd/321:46:583
137291270,0ptp4l401ktimersoftd/320:17:263
8652992519,4cyclictest41-21ksoftirqd/323:19:173
137291250,8ptp4l35892irq/127-enp1s0-00:11:193
8652992419,3cyclictest41-21ksoftirqd/322:50:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional