You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-18 - 23:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack5slot4s.osadl.org (updated Tue Nov 18, 2025 12:44:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
56969722700,6sleep3586-21kworker/3:2+mm_percpu_wq12:31:223
25544421840,2sleep3471rcuc/307:15:043
5742042640,1sleep1574205-21cat12:35:011
248896996448,13cyclictest282-21systemd-journal08:06:350
3914562580,2chrt0-21swapper/009:31:250
2805302550,1sleep0280532-21date07:40:010
248896995445,6cyclictest539145-21latency_hist12:00:000
248906995343,8cyclictest425013-21cat10:05:012
248901995342,8cyclictest309214-21ntpq08:06:281
3176742520,5chrt0-21swapper/008:16:250
3310442500,2chrt451irq_work/308:31:203
248896994942,5cyclictest429884-21latency_hist10:10:000
2729662480,1sleep0272969-21cut07:31:220
5058952470,3sleep3586-21kworker/3:2+events11:26:253
248896994737,8cyclictest488253-21ntpq11:06:290
248896994536,7cyclictest474916-21diskstats10:56:160
248896994524,19cyclictest325477-21/usr/sbin/munin08:26:280
248906994440,3cyclictest274655-21kworker/u8:3+flush-8:008:06:332
248896994429,13cyclictest578776-21grep12:36:370
248896994424,18cyclictest284516-21ntpq07:41:240
248901994335,5cyclictest282-21systemd-journal10:25:341
248901994334,6cyclictest260625-21latency_hist07:20:011
248896994337,4cyclictest274655-21kworker/u8:3+flush-8:008:51:410
248896994231,9cyclictest380537-21diskstats09:21:180
248896994219,20cyclictest554559-21diskstats12:16:190
24889699413,17cyclictest587-21runrttasks09:06:160
24889699413,17cyclictest587-21runrttasks09:06:160
248896994118,21cyclictest413746-21ntpq09:51:400
248906994032,6cyclictest282-21systemd-journal08:39:592
248906994029,9cyclictest504643-21cat11:25:012
248896994030,8cyclictest424725-21apt-get10:05:000
248896994021,15cyclictest282-21systemd-journal08:55:350
2486272405,10sleep10-21swapper/107:08:531
248901993921,11cyclictest430949-21irqrtprio10:11:221
248896993931,6cyclictest294409-21ntpq07:51:250
248896993924,13cyclictest253920-21systemctl07:15:010
2770282380,2sleep3471rcuc/307:36:233
248901993828,8cyclictest429023-21ntpq10:06:281
248896993818,18cyclictest521509-21kthreadcore11:41:240
248896993818,18cyclictest504741-21apt-key11:25:010
248901993727,8cyclictest282-21systemd-journal08:25:001
248901993724,5cyclictest290666-21apt-get07:51:121
248896993730,5cyclictest274655-21kworker/u8:3+flush-8:010:51:430
248896993728,6cyclictest350109-21latency_hist08:50:010
248896993725,9cyclictest455127-21diskstats10:36:190
24889699372,12cyclictest400095-21apt-get09:41:140
24889699371,15cyclictest325187-21latency_hist08:25:000
248906993626,7cyclictest370668-21diskstats09:11:182
248896993630,4cyclictest360570-21apt-get09:01:140
248896993621,13cyclictest399726-21idleruntime-cro09:40:000
248896993619,13cyclictest310478-21aten2_r5power_e08:11:170
24443623624,6sleep20-21swapper/207:06:172
248906993530,4cyclictest248754-21kworker/u8:1+flush-8:009:51:512
24889699352,9cyclictest415895-21sed09:56:210
248896993528,5cyclictest434517-21grep10:11:360
24889699352,7cyclictest463669-21ntpq10:41:320
248896993518,15cyclictest260662-21sh07:20:000
248896993516,17cyclictest0-21swapper/008:05:000
248896993515,18cyclictest265200-21grep07:21:320
248906993429,4cyclictest490716-21kworker/u8:2+flush-8:012:16:352
248906993429,3cyclictest274655-21kworker/u8:3+events_unbound07:31:312
248896993429,3cyclictest558633-21kworker/u8:0+flush-8:012:32:000
248896993427,5cyclictest449963-21apt-get10:31:140
248896993417,15cyclictest0-21swapper/009:11:190
248896993416,15cyclictest0-21swapper/008:31:150
248896993412,6cyclictest282-21systemd-journal10:20:000
248901993329,2cyclictest415402-21cpuspeed_turbos09:56:171
24889699338,5cyclictest299330-21grep07:56:260
24889699337,7cyclictest405-21dbus-daemon08:45:000
24889699332,6cyclictest289505-21ntpq07:46:280
24889699332,14cyclictest587-21runrttasks12:11:150
248896993314,16cyclictest267102-21kthreadcore07:26:220
248896993313,1cyclictest0-21swapper/011:05:010
24889699331,13cyclictest563013-21seq12:21:280
24889699331,11cyclictest587-21runrttasks09:46:300
248896993311,19cyclictest0-21swapper/011:51:220
248896993311,19cyclictest0-21swapper/011:51:220
248906993228,3cyclictest490716-21kworker/u8:2+flush-8:011:11:232
248901993227,3cyclictest286462-21irqrtprio07:46:211
248896993219,9cyclictest282-21systemd-journal12:26:290
248896993215,15cyclictest0-21swapper/008:36:260
248896993214,17cyclictest528459-21ntp_kernel_pll_11:46:300
248906993127,3cyclictest248754-21kworker/u8:1+events_unbound09:46:332
248906993125,4cyclictest454156-21unixbench_multi10:31:352
248901993113,4cyclictest0-21swapper/107:26:131
24889699312,8cyclictest584-21snmpd09:26:320
248896993112,3cyclictest405-21dbus-daemon11:30:000
24889699311,10cyclictest379061-21taskset09:16:270
248901993026,2cyclictest395841-21irqrtprio09:36:241
248901993024,4cyclictest528972-21unixbench_singl11:46:351
248901993022,6cyclictest300198-21latency_hist08:00:011
24889699308,7cyclictest544685-21df_inode12:06:200
24889699308,5cyclictest516127-21tr11:36:220
24889699302,7cyclictest449036-21python310:26:330
248896993025,3cyclictest469424-21systemd-run10:50:000
248896993011,3cyclictest493284-21ntp_kernel_err11:11:250
248906992925,3cyclictest394801-21kworker/u8:2+events_unbound10:36:362
248906992925,2cyclictest479679-21aten2_r5power_e11:01:162
248906992923,4cyclictest490716-21kworker/u8:2+flush-8:012:06:312
248906992923,4cyclictest334734-21unixbench_singl08:31:332
248906992918,9cyclictest282-21systemd-journal07:46:352
24890199295,12cyclictest369322-21ntpq09:06:311
24890199295,12cyclictest369322-21ntpq09:06:311
24890199295,12cyclictest274704-21ntpq07:31:291
24890199293,14cyclictest0-21swapper/109:16:251
248901992926,2cyclictest515031-21cpuspeed_turbos11:36:171
248901992925,2cyclictest563907-21apt-config12:25:011
248901992924,3cyclictest540231-21irqrtprio12:01:221
24890199292,17cyclictest279543-21ntpq07:36:271
24890199292,15cyclictest329510-21python308:26:351
248901992911,10cyclictest0-21swapper/111:56:231
24889699293,6cyclictest509859-21apt-get11:31:130
24889699291,6cyclictest584-21snmpd10:21:290
248906992824,3cyclictest509519-21cron11:30:002
24890199285,12cyclictest443923-21ntpq10:21:291
24890199284,15cyclictest0-21swapper/110:41:261
24890199284,14cyclictest0-21swapper/111:21:341
24890199284,12cyclictest374116-21ntpq09:11:271
24890199283,16cyclictest0-21swapper/112:26:241
24890199283,14cyclictest578312-21ntpq12:36:311
24890199283,14cyclictest0-21swapper/108:16:151
24890199283,13cyclictest488268-21ntpq11:06:291
24890199283,13cyclictest354178-21ntpq08:51:341
24890199283,12cyclictest389085-21ntpq09:26:301
24890199283,12cyclictest359173-21ntpq08:56:301
248901992821,5cyclictest519776-21apt-get11:41:141
24890199282,15cyclictest394031-21ntpq09:31:331
24890199282,15cyclictest349245-21ntpq08:46:271
24890199281,11cyclictest552197-21kthreadcore12:11:241
24890199281,10cyclictest402248-21kthreadcore09:41:251
248901992810,4cyclictest0-21swapper/108:31:261
24889699289,2cyclictest0-21swapper/011:16:170
248906992723,3cyclictest409637-21kworker/u8:0+flush-8:011:31:292
248906992723,2cyclictest425382-21aten2_r5power_p10:06:182
248906992722,3cyclictest464082-21unixbench_multi10:41:372
248906992717,8cyclictest282-21systemd-journal12:30:352
248906992717,8cyclictest275367-21anacron07:39:072
24890199279,9cyclictest0-21swapper/109:46:191
24890199278,11cyclictest0-21swapper/110:52:151
24890199278,10cyclictest0-21swapper/110:36:281
24890199274,11cyclictest314197-21ntpq08:11:281
24890199274,11cyclictest252983-21ntpq07:11:331
24890199273,14cyclictest0-21swapper/109:53:091
24890199273,12cyclictest468642-21ntpq10:46:311
24890199273,12cyclictest384066-21ntpq09:21:291
24890199273,12cyclictest344433-21ntpq08:41:381
24890199272,9cyclictest285099-21users07:41:321
24890199272,7cyclictest339489-21ls08:36:351
24890199272,15cyclictest508773-21ntpq11:26:331
24890199272,11cyclictest510293-21if_enp4s011:31:181
248901992719,6cyclictest489507-21apt-get11:11:131
248901992716,9cyclictest424725-21apt-get10:05:011
248901992711,8cyclictest0-21swapper/111:01:261
24890199271,11cyclictest498563-21ntpq11:16:281
4448852260,6sleep3586-21kworker/3:2+mm_percpu_wq10:25:013
248906992623,2cyclictest280860-21aten2_r5power_p07:41:142
248906992622,2cyclictest314713-21unixbench_singl08:11:352
248906992621,4cyclictest274655-21kworker/u8:3+flush-8:011:06:402
248906992621,4cyclictest274655-21kworker/u8:3+flush-8:009:55:062
248906992621,3cyclictest474013-21unixbench_singl10:51:382
248906992618,6cyclictest355615-21cpuspeed_turbos08:56:192
24890199269,7cyclictest0-21swapper/107:21:231
24890199262,8cyclictest435836-21irqcore10:16:221
24890199262,14cyclictest453693-21ntpq10:31:281
24890199262,14cyclictest364296-21gcc09:01:271
24890199261,8cyclictest533379-21ntpq11:51:311
24890199261,8cyclictest533379-21ntpq11:51:311
24890199261,15cyclictest475960-21cut10:56:221
24890199261,15cyclictest0-21swapper/112:09:451
3867982250,5sleep3451irq_work/309:26:223
248906992521,2cyclictest445318-21fschecks_count10:26:182
248906992521,2cyclictest439757-21sendmail10:20:002
248906992521,2cyclictest360860-21fschecks_count09:01:192
248906992520,4cyclictest514316-21kworker/u8:3+flush-8:012:11:332
248906992516,7cyclictest282-21systemd-journal08:00:352
248906992515,8cyclictest282-21systemd-journal08:30:002
24890199253,13cyclictest0-21swapper/108:01:221
24890199252,8cyclictest558063-21ld12:16:281
24889699252,6cyclictest410-21mta-sts-daemon12:01:290
4325752240,5sleep3451irq_work/310:11:253
248906992420,3cyclictest260621-21idleruntime-cro07:20:002
248906992420,2cyclictest324913-21cron08:25:002
248906992319,3cyclictest409637-21kworker/u8:0+flush-8:012:01:282
248906992319,2cyclictest529681-21aten2_r5power_e11:51:192
248906992319,2cyclictest529681-21aten2_r5power_e11:51:182
248906992315,6cyclictest282-21systemd-journal09:20:002
248906992219,2cyclictest380318-21aten2_r5power_c09:21:162
248906992218,3cyclictest558633-21kworker/u8:0+flush-8:012:36:272
248906992218,2cyclictest440125-21aten2_r5power_c10:21:152
248906992218,2cyclictest0-21swapper/212:25:012
248906992217,4cyclictest248754-21kworker/u8:1+flush-8:007:56:332
248906992118,2cyclictest564743-21fschecks_time12:26:182
248906992115,4cyclictest330321-21kworker/u8:0+events_unbound09:30:012
24890699211,2cyclictest5346351aten2_r5power_p11:56:152
248906992111,8cyclictest282-21systemd-journal11:45:002
24890699202,13cyclictest349304-21ntpq08:46:292
248906992016,2cyclictest430451-21cpuspeed_turbos10:11:172
248906992016,2cyclictest390404-21cpuspeed_turbos09:31:192
248906992014,4cyclictest528906-21unixbench_multi11:46:352
24890699200,6cyclictest345170-21sh08:45:002
2486262203,11sleep00-21swapper/007:08:510
24890699199,8cyclictest282-21systemd-journal11:40:002
24890699192,11cyclictest395121-21apt-get09:36:192
24890699191,12cyclictest315378-21apt-get08:16:172
2441272193,9sleep30-21swapper/307:05:373
248906991814,2cyclictest0-21swapper/210:46:172
248906991814,2cyclictest0-21swapper/207:15:012
248906991813,3cyclictest0-21swapper/209:06:252
248906991813,3cyclictest0-21swapper/209:06:252
248906991812,4cyclictest0-21swapper/210:56:242
24890699179,6cyclictest270478-21cron07:30:002
248906991713,2cyclictest0-21swapper/207:55:012
248906991712,3cyclictest0-21swapper/211:16:252
24890699167,7cyclictest264466-21tr07:21:232
24890699162,13cyclictest0-21swapper/208:51:092
3388892156,6sleep30-21swapper/308:36:273
24890699156,7cyclictest403410-21tr09:41:262
3753072142,7sleep30-21swapper/309:16:163
5333132132,7sleep30-21swapper/311:51:303
5333132132,7sleep30-21swapper/311:51:303
5243532132,7sleep30-21swapper/311:45:003
4459112130,9sleep30-21swapper/310:26:233
4260952132,7sleep30-21swapper/310:06:243
3799682132,6sleep30-21swapper/309:20:013
3703932131,8sleep30-21swapper/309:11:143
3671002132,7sleep30-21swapper/309:06:253
3671002132,7sleep30-21swapper/309:06:253
3110732130,3chrt586-21kworker/3:2-events08:11:223
2947662132,7sleep30-21swapper/307:51:333
5684352122,6sleep30-21swapper/312:26:333
5638392122,6sleep30-21swapper/312:25:003
5448642122,6sleep30-21swapper/312:06:233
5391802122,5sleep30-21swapper/312:00:013
4956222123,5sleep30-21swapper/311:16:233
4902692122,6sleep30-21swapper/311:11:213
4793012123,5sleep30-21swapper/311:00:013
4768062122,6sleep30-21swapper/310:56:243
4647932122,6sleep30-21swapper/310:46:173
4602322122,6sleep30-21swapper/310:41:233
4545472122,6sleep30-21swapper/310:35:013
4354102122,6sleep30-21swapper/310:16:193
4191212122,6sleep30-21swapper/309:56:333
4104742122,6sleep30-21swapper/309:51:353
4050342122,6sleep30-21swapper/309:46:183
4009812123,6sleep30-21swapper/309:41:243
3909792122,6sleep30-21swapper/309:31:253
3608202122,6sleep30-21swapper/309:01:193
3460412123,5sleep30-21swapper/308:46:223
3407042122,6sleep30-21swapper/308:41:253
3212172122,6sleep30-21swapper/308:21:223
3166872122,6sleep30-21swapper/308:16:253
3051272122,6sleep30-21swapper/308:05:013
2860702122,6sleep30-21swapper/307:46:193
2490582122,6sleep30-21swapper/307:10:353
5750312112,5sleep30-21swapper/312:36:243
5543142112,5sleep30-21swapper/312:16:183
5532392112,5sleep30-21swapper/312:11:323
5345562112,5sleep30-21swapper/311:56:163
5256092112,5sleep30-21swapper/311:46:243
5152902112,5sleep30-21swapper/311:36:213
5129982112,5sleep30-21swapper/311:31:233
5000012112,5sleep30-21swapper/311:21:203
4846002112,5sleep30-21swapper/311:06:193
4702292112,5sleep30-21swapper/310:51:233
4503342112,5sleep30-21swapper/310:31:193
4202502112,5sleep30-21swapper/310:01:183
3953712112,5sleep30-21swapper/309:36:223
3550652112,5sleep30-21swapper/308:55:003
3527502112,5sleep30-21swapper/308:51:323
3254762112,5sleep30-21swapper/308:26:163
3004762112,5sleep30-21swapper/308:00:353
2996102112,5sleep30-21swapper/307:56:343
2846672112,5sleep30-21swapper/307:41:293
2711192112,5sleep30-21swapper/307:31:173
2659272112,5sleep30-21swapper/307:26:163
2613992112,5sleep30-21swapper/307:21:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional