You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 00:34
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackdslot3.osadl.org (updated Wed Dec 03, 2025 12:45:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
287312208169,28sleep20-21swapper/207:06:072
137191206172,22phc2sys0-21swapper/307:08:543
287512202169,22sleep10-21swapper/107:06:221
289662200167,22sleep00-21swapper/007:09:020
315521860,5sleep12933599cyclictest09:35:251
290432163125,13sleep20-21swapper/207:10:012
29336996811,16cyclictest16206-21cat10:05:002
29336996811,16cyclictest16206-21cat10:05:002
29336996612,16cyclictest32713-21irqstats07:15:212
29336996510,15cyclictest6384-21latency_hist09:45:002
29336996411,17cyclictest28299-21sed09:20:242
2933799637,18cyclictest8043-21sensors09:45:243
29336996311,16cyclictest1724-21sed09:35:002
29336996310,15cyclictest223-21systemd-journal12:03:402
29334996335,3cyclictest9-21ksoftirqd/009:24:590
29336996013,19cyclictest12871-21wc07:40:312
29336995913,15cyclictest9387-21which10:55:232
2933799588,12cyclictest1285-21snmpd08:30:233
2933799577,17cyclictest26887-21ps08:10:243
2933799576,17cyclictest22052-21apt-get09:10:133
29336995711,17cyclictest16588-21/usr/sbin/munin12:20:132
29334995727,4cyclictest9-21ksoftirqd/011:50:270
29334995721,6cyclictest9-21ksoftirqd/011:15:270
29334995720,6cyclictest9-21ksoftirqd/011:00:010
136991570,1ptp4l391rcuc/310:15:123
29336995612,15cyclictest13628-21ptp4l08:50:232
2933499565,3cyclictest9-21ksoftirqd/012:25:000
29334995626,7cyclictest9-21ksoftirqd/010:55:010
2933799557,16cyclictest26063-21/usr/sbin/munin10:25:213
29334995522,6cyclictest9-21ksoftirqd/008:50:170
29334995516,6cyclictest9-21ksoftirqd/011:25:000
2933699548,10cyclictest0-21swapper/211:55:172
2933699547,15cyclictest0-21swapper/210:35:142
29336995410,12cyclictest15621-21irqstats08:55:192
29336995410,11cyclictest2163-21awk08:25:272
29334995427,7cyclictest9-21ksoftirqd/011:10:000
29334995425,7cyclictest9-21ksoftirqd/009:20:000
29334995424,7cyclictest9-21ksoftirqd/011:40:200
29334995422,4cyclictest9-21ksoftirqd/008:10:170
29334995419,5cyclictest9-21ksoftirqd/010:40:000
2933799536,17cyclictest9776-21users10:55:273
2933799536,11cyclictest13102-21who07:40:323
2933799536,10cyclictest12508-21irqstats09:55:223
29334995323,4cyclictest9-21ksoftirqd/011:35:140
2933799529,20cyclictest4174-21cut10:45:193
2933799526,16cyclictest7575-21/usr/sbin/munin08:40:193
29336995211,11cyclictest3685-21mailstats11:50:232
29334995213,7cyclictest9-21ksoftirqd/008:00:280
2933799516,17cyclictest27980-21hddtemp_smartct11:35:163
2933699518,10cyclictest0-21swapper/209:50:262
2933699517,18cyclictest0-21swapper/211:15:272
29336995111,16cyclictest10370-21mailstats07:35:272
29336995111,16cyclictest10370-21mailstats07:35:272
29334995124,8cyclictest9-21ksoftirqd/008:25:150
29334995123,7cyclictest9-21ksoftirqd/011:30:170
29334995123,7cyclictest9-21ksoftirqd/008:55:220
29334995122,7cyclictest9-21ksoftirqd/007:55:240
29334995121,3cyclictest9-21ksoftirqd/007:32:320
29334995120,4cyclictest9-21ksoftirqd/011:10:240
29334995120,4cyclictest9-21ksoftirqd/010:15:250
29334995118,4cyclictest9-21ksoftirqd/009:45:250
29334995117,6cyclictest9-21ksoftirqd/011:45:170
29334995117,5cyclictest9-21ksoftirqd/012:10:000
29334995117,4cyclictest9-21ksoftirqd/009:50:220
29334995112,6cyclictest9-21ksoftirqd/007:50:210
2933799507,16cyclictest1285-21snmpd12:15:143
2933699509,9cyclictest17488-21uname10:05:202
29334995023,7cyclictest9-21ksoftirqd/011:00:270
29334995021,4cyclictest9-21ksoftirqd/008:35:200
29334995021,4cyclictest9-21ksoftirqd/008:35:190
29334995020,3cyclictest9-21ksoftirqd/007:10:140
29334995013,5cyclictest9-21ksoftirqd/010:40:150
29334995010,6cyclictest9-21ksoftirqd/008:45:230
2933699499,7cyclictest1285-21snmpd11:12:282
2933699498,16cyclictest0-21swapper/209:05:182
29334994922,7cyclictest9-21ksoftirqd/012:35:210
29334994922,7cyclictest9-21ksoftirqd/009:40:240
29334994920,4cyclictest9-21ksoftirqd/009:05:260
29334994918,4cyclictest9-21ksoftirqd/012:30:200
29334994916,7cyclictest9-21ksoftirqd/012:00:230
29334994914,6cyclictest9-21ksoftirqd/010:00:250
29334994914,6cyclictest9-21ksoftirqd/010:00:250
29334994913,5cyclictest9-21ksoftirqd/009:35:200
29334994913,5cyclictest9-21ksoftirqd/009:30:270
29334994912,6cyclictest9-21ksoftirqd/007:15:210
29334994912,5cyclictest9-21ksoftirqd/012:25:120
29334994912,5cyclictest9-21ksoftirqd/011:55:140
29334994912,5cyclictest9-21ksoftirqd/009:25:220
29334994912,5cyclictest9-21ksoftirqd/007:45:230
29334994911,6cyclictest9-21ksoftirqd/010:45:230
29334994911,5cyclictest9-21ksoftirqd/012:15:130
29334994910,7cyclictest9-21ksoftirqd/010:30:130
29335994841,4cyclictest17509-21sh11:15:001
2933499489,6cyclictest9-21ksoftirqd/008:40:180
29334994820,7cyclictest9-21ksoftirqd/007:20:240
29334994819,7cyclictest9-21ksoftirqd/010:10:130
29334994818,4cyclictest9-21ksoftirqd/009:10:010
29334994815,4cyclictest9-21ksoftirqd/010:20:180
29334994814,7cyclictest9-21ksoftirqd/010:10:010
29334994814,4cyclictest9-21ksoftirqd/012:10:190
29334994813,4cyclictest9-21ksoftirqd/009:00:220
29334994811,6cyclictest9-21ksoftirqd/007:40:170
29334994811,5cyclictest9-21ksoftirqd/007:35:250
29334994811,5cyclictest9-21ksoftirqd/007:35:250
29334994811,5cyclictest9-21ksoftirqd/007:25:240
29334994810,6cyclictest9-21ksoftirqd/009:55:160
29334994810,5cyclictest9-21ksoftirqd/011:25:170
29334994810,5cyclictest9-21ksoftirqd/008:15:130
2933799476,17cyclictest31905-21memory10:35:223
2933499474,2cyclictest121rcu_preempt08:10:000
2933499474,2cyclictest121rcu_preempt08:10:000
29334994715,6cyclictest9-21ksoftirqd/008:20:230
29334994710,6cyclictest9-21ksoftirqd/008:30:150
29334994710,5cyclictest9-21ksoftirqd/010:25:130
29336994612,13cyclictest11673-21grep11:00:232
29336994610,9cyclictest232372sleep210:17:392
29336994413,16cyclictest4163-21grep08:30:242
29336994411,11cyclictest6739-21timerandwakeup08:35:262
29336994411,11cyclictest6739-21timerandwakeup08:35:252
29336994313,16cyclictest21429-21missed_timers11:20:232
29335994338,3cyclictest25-21ksoftirqd/109:20:001
29335994337,3cyclictest25-21ksoftirqd/112:20:011
2933699428,9cyclictest0-21swapper/211:35:222
29336994212,16cyclictest31660-21awk11:40:282
136991420,0ptp4l401ktimersoftd/307:10:273
2933699419,7cyclictest1285-21snmpd11:32:312
2933699419,7cyclictest1285-21snmpd08:42:002
2933699418,7cyclictest1285-21snmpd08:16:542
29336994111,15cyclictest8417-21taskset07:31:232
136991410,0ptp4l401ktimersoftd/310:43:243
136991410,0ptp4l401ktimersoftd/309:40:163
2933699407,7cyclictest4326-21meminfo10:45:212
2933699406,17cyclictest0-21swapper/212:05:322
136991400,0ptp4l401ktimersoftd/307:27:243
2933799397,11cyclictest23560-21df_abs08:05:143
2933799397,11cyclictest23560-21df_abs08:05:143
29336993910,17cyclictest13804-21latency11:05:202
29336993910,16cyclictest29598-21netstat10:30:222
29335993933,3cyclictest25-21ksoftirqd/112:05:251
2933799388,15cyclictest223-21systemd-journal12:32:433
2933799388,11cyclictest19138-21irqrtprio07:55:183
2933699388,18cyclictest0-21swapper/211:25:122
29335993833,3cyclictest25-21ksoftirqd/110:40:001
29335993833,3cyclictest25-21ksoftirqd/109:10:011
2933699377,11cyclictest0-21swapper/207:50:242
29336993710,7cyclictest20857-21apt-get08:00:152
2933599376,4cyclictest25-21ksoftirqd/112:25:011
29335993732,3cyclictest25-21ksoftirqd/110:35:001
2933799368,17cyclictest29549-21expr08:15:263
2933799367,17cyclictest17972-21apt-get11:15:143
2933799366,18cyclictest23697-21/usr/sbin/munin12:35:203
29337993630,4cyclictest41-21ksoftirqd/311:25:113
2933699369,16cyclictest0-21swapper/210:20:182
2933699368,14cyclictest1285-21snmpd10:53:512
2933699367,17cyclictest0-21swapper/212:10:232
2933699366,18cyclictest0-21swapper/209:55:152
29336993629,4cyclictest33-21ksoftirqd/210:12:212
29336993612,14cyclictest13710-21/usr/sbin/munin07:45:242
29335993631,3cyclictest25-21ksoftirqd/108:10:221
29335993630,3cyclictest25-21ksoftirqd/110:40:251
2933799357,18cyclictest15371-21unixbench_multi07:45:273
2933799354,11cyclictest0-21swapper/309:35:153
29337993529,4cyclictest41-21ksoftirqd/311:55:323
2933699358,17cyclictest0-21swapper/212:30:252
2933699357,18cyclictest0-21swapper/208:05:262
2933699357,18cyclictest0-21swapper/208:05:262
2933699357,17cyclictest0-21swapper/211:45:132
29336993529,4cyclictest33-21ksoftirqd/209:49:302
29336993528,4cyclictest33-21ksoftirqd/209:38:362
29336993510,11cyclictest25860-21perf12:40:002
29336993510,10cyclictest11315-21mailstats08:45:242
29335993530,4cyclictest25-21ksoftirqd/108:50:001
29335993529,4cyclictest25-21ksoftirqd/111:48:201
2933799347,16cyclictest20857-21apt-get08:00:143
2933799347,15cyclictest223-21systemd-journal08:57:023
29337993428,4cyclictest41-21ksoftirqd/310:00:433
29337993428,4cyclictest41-21ksoftirqd/310:00:433
2933699348,16cyclictest0-21swapper/212:28:432
2933699347,16cyclictest0-21swapper/209:10:262
2933699346,18cyclictest0-21swapper/207:20:122
2933699346,17cyclictest0-21swapper/210:40:132
2933699346,17cyclictest0-21swapper/210:25:202
2933699346,17cyclictest0-21swapper/208:20:142
29336993410,11cyclictest15564-21mailstats12:15:232
29335993429,3cyclictest25-21ksoftirqd/110:45:191
29335993429,3cyclictest25-21ksoftirqd/109:00:001
29335993429,3cyclictest25-21ksoftirqd/108:05:001
29335993428,4cyclictest25-21ksoftirqd/112:02:481
29335993428,4cyclictest25-21ksoftirqd/111:41:021
2933799337,16cyclictest32467-21sshd08:24:033
2933799337,16cyclictest30448-21if_enp1s011:40:183
2933799337,16cyclictest18958-21/usr/sbin/munin12:25:173
2933799336,16cyclictest16553-21/usr/sbin/munin10:05:193
2933799334,18cyclictest0-21swapper/311:05:133
2933799334,10cyclictest0-21swapper/311:50:013
2933799333,9cyclictest0-21swapper/312:00:283
29337993327,4cyclictest41-21ksoftirqd/312:21:003
29337993327,4cyclictest41-21ksoftirqd/310:54:303
29337993327,4cyclictest41-21ksoftirqd/310:11:383
29337993327,4cyclictest41-21ksoftirqd/309:52:363
29337993327,4cyclictest41-21ksoftirqd/309:29:093
2933699337,17cyclictest0-21swapper/207:25:252
2933699337,16cyclictest0-21swapper/207:55:172
2933699336,17cyclictest0-21swapper/209:25:112
2933699332,30cyclictest20490-21kworker/2:108:11:552
29335993328,3cyclictest25-21ksoftirqd/111:50:211
29335993328,3cyclictest25-21ksoftirqd/108:40:001
29335993328,3cyclictest25-21ksoftirqd/108:40:001
29335993327,4cyclictest25-21ksoftirqd/109:56:311
29335993327,4cyclictest25-21ksoftirqd/107:55:561
29335993327,4cyclictest25-21ksoftirqd/107:51:591
29335993327,4cyclictest25-21ksoftirqd/107:37:431
29335993327,4cyclictest25-21ksoftirqd/107:37:431
2933599330,6cyclictest19598-21irqrtprio10:10:201
59298320,7rtkit-daemon0-21swapper/107:42:531
59298320,1rtkit-daemon0-21swapper/107:45:551
29337993227,3cyclictest41-21ksoftirqd/307:34:303
29337993226,4cyclictest41-21ksoftirqd/308:37:303
29337993226,4cyclictest41-21ksoftirqd/308:37:303
29337993226,4cyclictest41-21ksoftirqd/308:29:363
2933699327,10cyclictest0-21swapper/209:15:162
2933699325,17cyclictest0-21swapper/209:00:222
29335993228,3cyclictest25-21ksoftirqd/111:25:191
29335993228,3cyclictest25-21ksoftirqd/109:25:181
29335993227,3cyclictest25-21ksoftirqd/110:28:121
29335993227,3cyclictest25-21ksoftirqd/110:20:001
29335993226,4cyclictest25-21ksoftirqd/111:35:361
29335993226,4cyclictest25-21ksoftirqd/111:04:511
29335993226,4cyclictest25-21ksoftirqd/107:19:391
29335993226,3cyclictest25-21ksoftirqd/110:50:181
29335993225,5cyclictest25-21ksoftirqd/110:05:581
29335993225,5cyclictest25-21ksoftirqd/108:21:451
29335993225,4cyclictest25-21ksoftirqd/109:50:001
2933799316,15cyclictest21778-21sensors_temp11:20:273
2933799316,11cyclictest0-21swapper/307:25:003
2933799313,17cyclictest0-21swapper/307:35:143
2933799313,17cyclictest0-21swapper/307:35:143
29337993126,3cyclictest41-21ksoftirqd/310:20:113
29337993125,4cyclictest41-21ksoftirqd/311:33:463
29337993125,4cyclictest41-21ksoftirqd/309:15:333
29335993127,2cyclictest5395-21cut11:55:191
29335993125,4cyclictest25-21ksoftirqd/112:35:501
29335993125,4cyclictest25-21ksoftirqd/111:21:321
29335993125,4cyclictest25-21ksoftirqd/110:01:091
29335993125,4cyclictest25-21ksoftirqd/110:01:091
29335993125,3cyclictest25-21ksoftirqd/109:45:001
2933799307,8cyclictest1055-21ls09:30:293
2933799306,7cyclictest12346-21hddtemp_smartct12:10:183
29337993024,4cyclictest41-21ksoftirqd/311:01:243
29337993024,4cyclictest41-21ksoftirqd/310:31:283
29335993026,2cyclictest25-21ksoftirqd/107:25:001
29335993025,3cyclictest25-21ksoftirqd/112:31:321
29335993025,3cyclictest25-21ksoftirqd/109:10:001
29335993025,3cyclictest25-21ksoftirqd/107:11:051
29335993024,4cyclictest25-21ksoftirqd/112:10:511
29335993024,4cyclictest25-21ksoftirqd/108:34:521
29335993024,4cyclictest25-21ksoftirqd/108:06:471
29335993024,4cyclictest25-21ksoftirqd/108:06:471
2933799297,8cyclictest3837-21ssh11:50:253
29335992924,3cyclictest25-21ksoftirqd/112:25:151
29335992924,3cyclictest25-21ksoftirqd/107:35:011
29335992924,3cyclictest25-21ksoftirqd/107:25:431
29335992923,4cyclictest25-21ksoftirqd/110:23:181
29335992923,4cyclictest25-21ksoftirqd/108:15:071
2933799286,8cyclictest19467-21apt-get09:05:123
29337992822,4cyclictest41-21ksoftirqd/309:24:003
29335992824,3cyclictest25-21ksoftirqd/111:05:131
29335992823,3cyclictest25-21ksoftirqd/109:34:511
29335992823,3cyclictest25-21ksoftirqd/109:21:041
29337992722,3cyclictest41-21ksoftirqd/312:07:523
29337992721,4cyclictest41-21ksoftirqd/311:11:313
29337992716,5cyclictest12331-21apt-get08:50:123
29335992723,3cyclictest25-21ksoftirqd/109:50:181
29335992723,3cyclictest25-21ksoftirqd/108:45:001
29335992722,3cyclictest25-21ksoftirqd/109:00:201
29335992722,3cyclictest25-21ksoftirqd/108:25:211
29335992721,4cyclictest25-21ksoftirqd/111:18:421
29337992621,3cyclictest41-21ksoftirqd/308:47:243
29337992621,3cyclictest41-21ksoftirqd/307:51:233
29335992623,2cyclictest25-21ksoftirqd/111:30:191
29335992621,3cyclictest25-21ksoftirqd/110:57:401
136991250,0ptp4l401ktimersoftd/309:00:143
136991250,0ptp4l401ktimersoftd/307:15:283
29335992319,2cyclictest25-21ksoftirqd/108:50:281
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional