You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-28 - 23:09
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackdslot2.osadl.org (updated Sat Jun 28, 2025 12:45:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
137391235171,22phc2sys0-21swapper/307:08:533
236492207174,22sleep10-21swapper/107:06:191
236502206172,23sleep20-21swapper/207:06:212
237402203171,22sleep00-21swapper/007:07:320
2543921700,7sleep22422699cyclictest11:40:242
137391153127,14phc2sys0-21swapper/307:10:003
1476821410,3sleep22422699cyclictest07:55:232
2422599646,10cyclictest20719-21latency_hist08:10:011
194542640,2sleep30-21swapper/310:20:273
137291640,1ptp4l401ktimersoftd/311:20:173
2422599634,18cyclictest13927-21grep07:55:141
2422599625,11cyclictest13587-21cpuspeed_turbos12:25:181
2422599622,11cyclictest17932-21awk12:35:001
18062620,4sleep09-21ksoftirqd/012:00:200
2422599615,10cyclictest6355-21tail11:00:301
2422599603,11cyclictest621-21grep07:25:291
2422599601,11cyclictest218-21systemd-journal10:05:001
2422599594,11cyclictest6669-21if_err_enp1s012:10:201
2422599591,16cyclictest0-21swapper/108:45:131
2422599591,11cyclictest16929-21cut08:00:221
2422599587,11cyclictest11788-21cut07:50:221
2422599585,10cyclictest9550-21ls07:45:211
2422599584,11cyclictest25303-21grep07:10:261
2422599582,11cyclictest18342-21gunzip11:25:281
2422599581,11cyclictest8287-21irqrtprio11:05:251
2422599575,11cyclictest7048-21grep07:40:181
2422599575,11cyclictest7048-21grep07:40:171
2422599575,11cyclictest1661-21gzip10:50:301
2422599574,17cyclictest22428-21interrupts11:35:211
2422599574,11cyclictest18774-21grep09:10:291
2422599574,11cyclictest16876-21expr10:15:251
2422599574,10cyclictest28114-21cat07:15:271
2422599573,9cyclictest4450-21fschecks_count07:35:181
2422599573,9cyclictest4450-21fschecks_count07:35:181
2422599572,16cyclictest28544-21grep09:30:341
2422599572,16cyclictest28544-21grep09:30:341
2422599571,11cyclictest1736-21cpuspeed_turbos09:45:161
2422599570,10cyclictest0-21swapper/112:20:161
2422599570,10cyclictest0-21swapper/109:45:001
180672570,2sleep20-21swapper/210:20:002
258412560,4sleep325836-21unixbench_multi11:40:293
2422599565,16cyclictest4887-21phc2sys12:05:271
2422599565,10cyclictest20623-21phc2sys11:30:251
2422599563,9cyclictest21274-21df_abs08:10:161
2422599563,11cyclictest23532-21expr08:15:181
2422599562,11cyclictest6270-21sed09:55:001
2422599562,11cyclictest31143-21latency_hist07:25:011
2422599562,10cyclictest4442-21sed08:40:331
2422599560,11cyclictest15112-21fschecks_time11:20:201
2422599553,12cyclictest8607-21cut10:00:011
2422599553,10cyclictest9776-21netstat12:15:271
2422599553,10cyclictest22518-21iostat09:20:191
2422599551,11cyclictest0-21swapper/108:25:171
2422599550,16cyclictest10648-21irqrtprio11:10:231
137291550,1ptp4l401ktimersoftd/308:29:423
2422699543,5cyclictest33-21ksoftirqd/211:20:002
2422599543,10cyclictest2330-21irqrtprio07:30:211
2422599542,9cyclictest27050-21grep11:45:191
2422599542,9cyclictest2566-21latency_hist08:40:011
2422599542,12cyclictest31642-21df_abs11:55:161
2422599542,12cyclictest218-21systemd-journal10:50:001
2422599542,11cyclictest26417-21cat09:30:001
2422599541,16cyclictest24973-21irqrtprio11:40:191
2422599540,9cyclictest0-21swapper/109:05:011
2422599533,9cyclictest26666-21sed08:20:241
2422599531,10cyclictest11087-21/usr/sbin/munin10:05:231
2422599530,10cyclictest12444-21cstates11:15:201
2393325336,9sleep00-21swapper/007:10:010
2422599525,16cyclictest20872-21cstates10:25:191
2422599525,11cyclictest11269-21gzip08:55:261
2422599522,10cyclictest31790-21tr08:30:281
2422599521,10cyclictest18586-21tune2fs12:35:181
2422599513,16cyclictest25244-21sed10:35:001
2422599513,11cyclictest2093-21cut12:00:221
2422599512,9cyclictest16329-21cat09:05:311
2422599512,11cyclictest30395-21cat11:50:311
2422599510,17cyclictest0-21swapper/110:55:211
2422599510,10cyclictest0-21swapper/109:35:271
24226995017,6cyclictest33-21ksoftirqd/212:20:002
2422599503,5cyclictest0-21swapper/110:20:171
2422599500,10cyclictest14191-21irqrtprio10:10:201
2422599492,15cyclictest26178-21sed10:35:231
24224994931,8cyclictest9-21ksoftirqd/010:20:000
24224994930,8cyclictest9-21ksoftirqd/009:20:000
2422599483,16cyclictest28640-21sed10:40:221
24224994828,12cyclictest9-21ksoftirqd/008:15:000
2422699476,3cyclictest33-21ksoftirqd/208:10:252
2422699474,3cyclictest33-21ksoftirqd/207:45:002
2422699474,3cyclictest33-21ksoftirqd/207:45:002
2422599475,11cyclictest19670-21cstates09:15:151
24224994732,5cyclictest9-21ksoftirqd/011:25:210
24224994731,5cyclictest9-21ksoftirqd/010:00:190
24224994729,8cyclictest9-21ksoftirqd/009:25:300
24224994715,7cyclictest9-21ksoftirqd/008:30:160
2422699466,3cyclictest33-21ksoftirqd/208:40:212
24224994631,7cyclictest9-21ksoftirqd/007:55:210
24224994629,8cyclictest9-21ksoftirqd/010:25:240
24224994629,8cyclictest9-21ksoftirqd/009:30:290
24224994629,8cyclictest9-21ksoftirqd/009:30:290
24224994629,8cyclictest9-21ksoftirqd/008:55:010
24224994629,8cyclictest9-21ksoftirqd/007:25:170
24224994628,5cyclictest9-21ksoftirqd/009:05:170
24224994627,8cyclictest9-21ksoftirqd/010:05:160
42098450,2rtkit-daemon419-21rtkit-daemon10:34:270
2422699456,3cyclictest33-21ksoftirqd/207:30:162
2422699455,4cyclictest33-21ksoftirqd/207:45:292
2422699452,3cyclictest33-21ksoftirqd/210:55:012
24226994515,3cyclictest33-21ksoftirqd/210:00:002
24224994529,7cyclictest9-21ksoftirqd/011:55:280
24224994528,8cyclictest9-21ksoftirqd/011:35:310
24224994528,8cyclictest9-21ksoftirqd/011:35:000
24224994528,7cyclictest9-21ksoftirqd/008:23:020
24224994527,9cyclictest9-21ksoftirqd/010:55:190
24224994527,7cyclictest9-21ksoftirqd/010:00:000
2422699446,3cyclictest33-21ksoftirqd/210:25:012
2422699442,2cyclictest33-21ksoftirqd/212:35:172
2422599443,12cyclictest7796-21cstates08:50:161
2422499449,6cyclictest9-21ksoftirqd/007:15:300
24224994429,7cyclictest9-21ksoftirqd/008:55:290
24224994428,8cyclictest9-21ksoftirqd/011:05:200
24224994428,8cyclictest9-21ksoftirqd/009:50:300
24224994428,7cyclictest9-21ksoftirqd/011:10:200
24224994427,8cyclictest9-21ksoftirqd/009:45:010
24224994427,8cyclictest9-21ksoftirqd/008:25:200
24224994427,8cyclictest9-21ksoftirqd/007:50:220
24224994425,8cyclictest9-21ksoftirqd/011:25:010
24227994331,10cyclictest41-21ksoftirqd/311:45:303
2422799432,4cyclictest121rcu_preempt12:15:293
2422799430,2cyclictest121rcu_preempt10:00:303
2422699434,3cyclictest121rcu_preempt12:05:172
2422699433,3cyclictest33-21ksoftirqd/211:25:182
2422699433,3cyclictest121rcu_preempt09:30:292
2422699433,3cyclictest121rcu_preempt09:30:292
2422699432,3cyclictest121rcu_preempt09:40:002
24224994327,8cyclictest9-21ksoftirqd/011:15:280
24224994327,8cyclictest9-21ksoftirqd/010:40:270
24224994327,8cyclictest9-21ksoftirqd/008:45:150
24224994326,8cyclictest9-21ksoftirqd/011:45:210
24224994326,8cyclictest9-21ksoftirqd/009:15:000
24224994326,8cyclictest9-21ksoftirqd/008:05:300
24224994325,8cyclictest9-21ksoftirqd/012:35:000
24224994324,8cyclictest9-21ksoftirqd/012:20:210
24227994236,3cyclictest21554-21turbostat.cron11:35:003
24227994225,7cyclictest41-21ksoftirqd/307:55:303
2422699425,3cyclictest33-21ksoftirqd/210:05:172
2422699425,3cyclictest33-21ksoftirqd/209:15:172
2422699425,10cyclictest121rcu_preempt08:20:292
2422699421,3cyclictest33-21ksoftirqd/210:40:172
2422699420,3cyclictest33-21ksoftirqd/210:35:002
24224994225,8cyclictest9-21ksoftirqd/012:05:280
24224994225,8cyclictest9-21ksoftirqd/009:20:170
24224994225,8cyclictest9-21ksoftirqd/008:00:230
24224994225,7cyclictest9-21ksoftirqd/011:40:240
24224994225,7cyclictest9-21ksoftirqd/007:20:190
24224994224,9cyclictest9-21ksoftirqd/008:40:230
24224994224,8cyclictest9-21ksoftirqd/007:40:160
24224994224,8cyclictest9-21ksoftirqd/007:40:150
24224994221,8cyclictest9-21ksoftirqd/010:50:000
24224994213,11cyclictest9-21ksoftirqd/009:50:010
24224994211,6cyclictest9-21ksoftirqd/008:35:320
24227994126,6cyclictest41-21ksoftirqd/307:30:243
24227994125,7cyclictest41-21ksoftirqd/312:20:273
24227994125,7cyclictest41-21ksoftirqd/309:55:013
2422699415,3cyclictest33-21ksoftirqd/209:10:232
2422699414,4cyclictest33-21ksoftirqd/211:30:212
2422699414,3cyclictest121rcu_preempt09:20:162
24226994137,2cyclictest33-21ksoftirqd/210:40:012
2422699413,3cyclictest33-21ksoftirqd/210:05:002
2422699413,3cyclictest33-21ksoftirqd/207:20:172
2422699413,3cyclictest121rcu_preempt07:15:272
2422699412,4cyclictest33-21ksoftirqd/210:45:252
2422699410,4cyclictest33-21ksoftirqd/211:40:002
24224994126,7cyclictest9-21ksoftirqd/012:25:220
24224994124,8cyclictest9-21ksoftirqd/012:15:010
24224994124,8cyclictest9-21ksoftirqd/009:00:230
24224994124,8cyclictest9-21ksoftirqd/008:15:220
24224994124,8cyclictest9-21ksoftirqd/007:35:260
24224994124,8cyclictest9-21ksoftirqd/007:35:260
24224994124,8cyclictest9-21ksoftirqd/007:30:180
24227994027,3cyclictest41-21ksoftirqd/308:15:323
24227994025,6cyclictest41-21ksoftirqd/311:05:293
24227994024,7cyclictest41-21ksoftirqd/310:05:253
24227994024,6cyclictest41-21ksoftirqd/310:55:173
24227994024,6cyclictest41-21ksoftirqd/307:40:323
24227994024,6cyclictest41-21ksoftirqd/307:40:323
24227994024,6cyclictest41-21ksoftirqd/307:25:243
24227994022,10cyclictest41-21ksoftirqd/311:15:313
24227994015,7cyclictest41-21ksoftirqd/307:15:223
2422699402,8cyclictest121rcu_preempt09:40:312
2422699402,4cyclictest33-21ksoftirqd/212:25:012
2422699402,3cyclictest121rcu_preempt09:00:002
2422699400,3cyclictest121rcu_preempt12:12:352
24224994024,7cyclictest9-21ksoftirqd/010:35:200
24224994023,8cyclictest9-21ksoftirqd/011:50:250
2422499400,2cyclictest121rcu_preempt10:10:260
137291400,0ptp4l401ktimersoftd/308:10:323
24227993934,3cyclictest41-21ksoftirqd/308:10:013
24227993934,3cyclictest391rcuc/312:15:013
24227993932,4cyclictest41-21ksoftirqd/311:00:253
24227993924,6cyclictest41-21ksoftirqd/309:30:313
24227993924,6cyclictest41-21ksoftirqd/309:30:303
24227993924,6cyclictest41-21ksoftirqd/309:20:293
24227993924,6cyclictest41-21ksoftirqd/308:30:283
24227993923,6cyclictest41-21ksoftirqd/310:40:303
24227993920,7cyclictest41-21ksoftirqd/309:25:293
24227993917,6cyclictest41-21ksoftirqd/309:07:553
2422699393,3cyclictest121rcu_preempt08:45:202
2422699392,3cyclictest121rcu_preempt12:25:202
2422699392,2cyclictest121rcu_preempt10:25:292
2422699391,4cyclictest33-21ksoftirqd/211:45:162
2422699391,4cyclictest33-21ksoftirqd/209:45:162
2422699391,3cyclictest33-21ksoftirqd/208:35:212
2422699390,3cyclictest121rcu_preempt12:35:002
24224993923,8cyclictest9-21ksoftirqd/009:35:200
24224993923,7cyclictest9-21ksoftirqd/011:00:260
2422799383,3cyclictest121rcu_preempt10:50:013
24227993832,3cyclictest41-21ksoftirqd/308:25:003
24227993824,7cyclictest41-21ksoftirqd/312:36:493
2422799382,3cyclictest121rcu_preempt10:25:323
24227993823,7cyclictest41-21ksoftirqd/308:45:193
24227993823,6cyclictest41-21ksoftirqd/311:25:323
24227993823,6cyclictest41-21ksoftirqd/309:55:193
24227993823,6cyclictest41-21ksoftirqd/307:20:313
24227993822,7cyclictest41-21ksoftirqd/308:55:303
24227993822,6cyclictest41-21ksoftirqd/309:40:333
2422699386,3cyclictest121rcu_preempt08:25:172
2422699385,3cyclictest33-21ksoftirqd/208:15:222
2422699384,3cyclictest33-21ksoftirqd/208:00:282
2422699383,3cyclictest121rcu_preempt08:30:152
24226993818,5cyclictest33-21ksoftirqd/207:50:262
2422699380,3cyclictest121rcu_preempt09:50:262
2422499385,4cyclictest9-21ksoftirqd/012:10:300
24224993831,4cyclictest9-21ksoftirqd/007:47:300
24224993822,7cyclictest9-21ksoftirqd/010:50:010
24227993723,6cyclictest41-21ksoftirqd/312:25:203
24227993722,6cyclictest41-21ksoftirqd/312:00:333
24227993722,6cyclictest41-21ksoftirqd/307:50:313
24227993721,6cyclictest41-21ksoftirqd/310:35:173
24227993712,3cyclictest5744-21grep07:35:293
24227993712,3cyclictest5744-21grep07:35:293
2422699373,3cyclictest33-21ksoftirqd/207:25:202
2422699371,8cyclictest121rcu_preempt11:05:342
2422699371,7cyclictest121rcu_preempt08:05:292
2422699371,2cyclictest121rcu_preempt10:55:152
2422699370,3cyclictest121rcu_preempt10:10:242
24227993632,2cyclictest41-21ksoftirqd/310:20:013
24227993631,3cyclictest41-21ksoftirqd/309:35:173
2422799362,2cyclictest121rcu_preempt07:45:303
24227993622,4cyclictest41-21ksoftirqd/309:15:153
24227993621,6cyclictest41-21ksoftirqd/312:05:173
24227993621,6cyclictest41-21ksoftirqd/309:03:403
24227993621,6cyclictest41-21ksoftirqd/308:50:223
24227993620,6cyclictest41-21ksoftirqd/308:35:353
2422699365,3cyclictest121rcu_preempt08:50:242
2422699365,2cyclictest121rcu_preempt07:35:212
2422699365,2cyclictest121rcu_preempt07:35:212
2422699362,3cyclictest33-21ksoftirqd/211:10:282
24224993631,3cyclictest9-21ksoftirqd/012:35:210
24224993618,8cyclictest9-21ksoftirqd/010:20:160
24227993530,3cyclictest41-21ksoftirqd/311:53:293
24227993530,3cyclictest41-21ksoftirqd/311:14:053
24227993529,4cyclictest41-21ksoftirqd/309:46:483
24227993521,3cyclictest41-21ksoftirqd/309:15:003
24227993520,7cyclictest41-21ksoftirqd/308:40:253
2422699350,3cyclictest121rcu_preempt11:55:152
2422699350,3cyclictest121rcu_preempt09:06:412
24227993428,4cyclictest41-21ksoftirqd/310:50:223
24227993419,6cyclictest41-21ksoftirqd/311:35:263
2422699343,3cyclictest33-21ksoftirqd/209:00:292
2422699342,3cyclictest121rcu_preempt11:20:182
2422699342,3cyclictest121rcu_preempt09:25:212
24227993329,3cyclictest41-21ksoftirqd/308:05:013
24227993329,2cyclictest41-21ksoftirqd/312:30:303
2422699334,3cyclictest121rcu_preempt07:10:302
2422699333,3cyclictest33-21ksoftirqd/211:50:312
24226993327,4cyclictest33-21ksoftirqd/211:01:382
24226993327,3cyclictest33-21ksoftirqd/212:00:162
24227993228,3cyclictest41-21ksoftirqd/312:00:003
2422799322,2cyclictest121rcu_preempt10:10:253
24227993125,4cyclictest41-21ksoftirqd/310:34:063
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional