You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 00:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Mon Dec 01, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31044212095,19sleep10-21swapper/107:06:371
31044212095,19sleep10-21swapper/107:06:361
31299211594,16sleep00-21swapper/007:09:530
31299211594,16sleep00-21swapper/007:09:520
30990210889,13sleep30-21swapper/307:05:553
30990210889,13sleep30-21swapper/307:05:543
31045210279,16sleep20-21swapper/207:06:382
31045210279,16sleep20-21swapper/207:06:382
216112800,1sleep115463-21diskmemload09:15:241
91892730,1sleep1221ktimersoftd/110:45:281
265302720,0sleep20-21swapper/209:20:272
158692720,0sleep30-21swapper/312:01:493
70902700,0sleep10-21swapper/110:44:381
97942250,1sleep0101ktimersoftd/011:20:260
3133899252,18cyclictest0-21swapper/111:31:041
301582250,0sleep30-21swapper/310:34:553
3133899242,2cyclictest0-21swapper/111:13:051
31338992319,2cyclictest13525-21kworker/u8:112:08:021
31337992315,6cyclictest0-21swapper/012:25:120
3134099220,20cyclictest0-21swapper/309:25:123
31338992219,2cyclictest13525-21kworker/u8:112:04:051
3133899221,19cyclictest2457-21ssh10:05:151
3134099211,18cyclictest0-21swapper/309:15:133
3133999210,19cyclictest0-21swapper/211:35:102
3133899212,2cyclictest332250irq/24-eth111:26:041
3133899212,2cyclictest21-21rcuc/112:38:041
31338992118,2cyclictest3527-21kworker/u8:108:58:051
31338992118,2cyclictest22388-21kworker/u8:208:13:041
31338992118,2cyclictest19209-21kworker/u8:010:04:041
13412210,0sleep20-21swapper/211:12:142
31339992017,2cyclictest13525-21kworker/u8:112:23:042
31339992017,2cyclictest13525-21kworker/u8:112:03:042
3133999200,18cyclictest0-21swapper/209:44:022
31338992017,2cyclictest31798-21kworker/u8:208:32:051
31338992017,2cyclictest28101-21kworker/u8:111:17:041
31338992017,2cyclictest22976-21kworker/u8:211:08:041
31338992017,2cyclictest19209-21kworker/u8:009:53:041
31338992017,2cyclictest13525-21kworker/u8:112:33:051
31338992017,2cyclictest10762-21kworker/u8:010:21:051
31338992017,2cyclictest0-21swapper/107:45:131
31338992017,1cyclictest0-21swapper/109:20:121
3133899201,6cyclictest0-21swapper/110:26:041
3133899201,2cyclictest0-21swapper/111:57:051
3133899201,2cyclictest0-21swapper/111:57:041
3133899201,2cyclictest0-21swapper/111:36:051
3133899201,2cyclictest0-21swapper/109:56:021
3133899201,17cyclictest0-21swapper/109:25:111
3133899200,3cyclictest0-21swapper/109:35:041
264062200,0sleep30-21swapper/308:15:173
31339991916,2cyclictest31841-21kworker/u8:210:01:042
31339991916,2cyclictest16262-21kworker/u8:011:02:042
31339991916,2cyclictest13525-21kworker/u8:112:12:052
31338991916,2cyclictest7655-21kworker/u8:009:05:051
31338991916,2cyclictest31798-21kworker/u8:208:37:041
31338991916,2cyclictest29294-21kworker/u8:008:27:041
31338991916,2cyclictest1134-21kworker/u8:207:36:041
31338991916,2cyclictest1134-21kworker/u8:207:22:011
31338991916,2cyclictest1134-21kworker/u8:207:18:051
3133899191,2cyclictest0-21swapper/112:22:051
3133899190,2cyclictest15463-21diskmemload12:25:041
3133799192,15cyclictest7895-21sendmail11:19:590
193332190,0sleep20-21swapper/209:13:442
31339991815,2cyclictest3527-21kworker/u8:109:00:042
31339991815,2cyclictest10762-21kworker/u8:010:32:062
3133899182,2cyclictest0-21swapper/111:23:051
31338991815,2cyclictest31798-21kworker/u8:208:50:041
31338991815,2cyclictest31798-21kworker/u8:208:44:051
31338991815,2cyclictest24171-21kworker/u8:310:30:051
31338991815,2cyclictest19209-21kworker/u8:009:48:051
3133899180,17cyclictest0-21swapper/111:00:041
31337991812,4cyclictest0-21swapper/009:20:260
31339991714,2cyclictest17592-21kworker/u8:109:18:052
31339991714,2cyclictest13525-21kworker/u8:112:07:052
31339991714,2cyclictest10760-21kworker/u8:110:58:052
3133999170,2cyclictest0-21swapper/210:12:012
31338991714,2cyclictest31798-21kworker/u8:208:47:041
31338991714,2cyclictest29939-21kworker/u8:209:32:061
31338991714,2cyclictest15258-21kworker/u8:007:56:051
3133899170,2cyclictest0-21swapper/110:15:041
3133799174,11cyclictest101ktimersoftd/011:59:570
3133799174,11cyclictest101ktimersoftd/011:59:560
31337991711,4cyclictest0-21swapper/011:48:110
31340991612,2cyclictest15463-21diskmemload11:50:293
3134099160,3cyclictest13598-21latency_hist12:35:003
3134099160,14cyclictest0-21swapper/309:45:013
31339991613,2cyclictest28101-21kworker/u8:111:25:042
31339991613,2cyclictest27034-21kworker/u8:012:18:062
31339991613,2cyclictest24171-21kworker/u8:310:28:052
31339991613,2cyclictest13525-21kworker/u8:112:32:052
31339991613,2cyclictest1134-21kworker/u8:207:37:042
31339991611,3cyclictest0-21swapper/207:40:172
3133999160,3cyclictest3459-21ps10:05:222
31338991613,2cyclictest31004-21kworker/u8:007:10:041
31338991613,2cyclictest13166-21kworker/u8:108:05:051
3133899160,2cyclictest0-21swapper/110:57:051
3133899160,2cyclictest0-21swapper/107:32:051
3133799163,2cyclictest101ktimersoftd/008:01:250
3133799162,12cyclictest14720-21ssh10:51:460
31337991613,2cyclictest0-21swapper/010:23:120
31337991611,4cyclictest0-21swapper/010:09:010
31337991611,3cyclictest0-21swapper/010:40:010
3134099151,12cyclictest16606-21apt-get10:20:123
31340991510,3cyclictest4775-21df12:25:133
31340991510,3cyclictest29790-21munin-node3
31339991512,2cyclictest9139-21kworker/u8:012:35:042
31339991512,2cyclictest7655-21kworker/u8:008:51:052
31339991512,2cyclictest31798-21kworker/u8:208:33:052
31339991512,2cyclictest29792-21kworker/u8:110:38:012
31339991512,2cyclictest2957-21kworker/u8:210:51:042
31339991512,2cyclictest28101-21kworker/u8:111:24:022
31339991512,2cyclictest28101-21kworker/u8:111:15:042
31339991512,2cyclictest25629-21kworker/u8:109:56:062
31339991512,2cyclictest22388-21kworker/u8:208:21:042
31339991512,2cyclictest2006-21kworker/u8:111:50:012
31339991512,2cyclictest17592-21kworker/u8:109:30:052
31339991512,2cyclictest17060-21kworker/u8:011:44:042
31339991512,2cyclictest0-21swapper/208:25:122
31338991512,2cyclictest13166-21kworker/u8:108:18:051
31338991512,2cyclictest13166-21kworker/u8:107:52:051
3133899150,2cyclictest0-21swapper/111:45:051
3133899150,13cyclictest0-21swapper/108:00:541
3133899150,10cyclictest0-21swapper/107:44:051
3133799153,11cyclictest0-21swapper/011:13:070
3133799152,7cyclictest0-21swapper/011:00:000
3133799152,2cyclictest0-21swapper/011:43:180
3133799152,11cyclictest0-21swapper/009:11:430
3133799151,3cyclictest7578-21df10:10:140
31337991512,2cyclictest0-21swapper/009:42:530
3133799151,12cyclictest6372-21ssh11:51:370
3133799151,12cyclictest23702-21ssh11:01:080
3133799151,12cyclictest0-21swapper/012:02:580
31337991510,4cyclictest0-21swapper/010:41:510
3133799150,3cyclictest0-21swapper/009:00:120
110502150,1sleep00-21swapper/009:39:400
31340991411,2cyclictest0-21swapper/312:38:153
31340991411,2cyclictest0-21swapper/309:32:113
31340991410,2cyclictest0-21swapper/311:36:113
3134099140,13cyclictest0-21swapper/311:17:093
3134099140,12cyclictest10822-21ssh11:22:073
3134099140,12cyclictest0-21swapper/312:06:143
3134099140,12cyclictest0-21swapper/311:45:503
3134099140,12cyclictest0-21swapper/310:40:233
3134099140,12cyclictest0-21swapper/310:02:583
3134099140,12cyclictest0-21swapper/309:55:253
3134099140,12cyclictest0-21swapper/309:20:173
3133999142,6cyclictest0-21swapper/211:07:592
31339991411,2cyclictest31798-21kworker/u8:208:42:022
31339991411,2cyclictest31798-21kworker/u8:208:38:012
31339991411,2cyclictest31004-21kworker/u8:007:11:042
31339991411,2cyclictest22798-21kworker/u8:009:25:042
31339991411,2cyclictest22388-21kworker/u8:208:07:052
31339991411,2cyclictest17592-21kworker/u8:109:46:042
31339991411,2cyclictest16044-21kworker/u8:210:20:052
31339991411,2cyclictest15258-21kworker/u8:007:55:042
31339991411,2cyclictest10762-21kworker/u8:010:44:042
31339991411,2cyclictest0-21swapper/211:55:122
31339991411,2cyclictest0-21swapper/211:55:122
31338991411,2cyclictest13525-21kworker/u8:112:10:041
3133899140,2cyclictest0-21swapper/111:43:041
3133899140,2cyclictest0-21swapper/110:53:041
3133899140,2cyclictest0-21swapper/109:42:061
3133799149,3cyclictest0-21swapper/012:30:280
3133799142,7cyclictest0-21swapper/008:29:590
3133799142,7cyclictest0-21swapper/008:05:010
3133799142,6cyclictest0-21swapper/008:45:000
3133799142,2cyclictest0-21swapper/012:09:290
3133799142,11cyclictest0-21swapper/011:25:410
31337991411,2cyclictest0-21swapper/012:24:060
3133799141,12cyclictest0-21swapper/010:31:550
31337991411,2cyclictest0-21swapper/010:17:300
3133799141,12cyclictest0-21swapper/009:58:420
31337991411,2cyclictest0-21swapper/009:25:050
3133799141,11cyclictest28012-21ssh11:05:280
31337991410,3cyclictest0-21swapper/009:18:110
3133799140,2cyclictest0-21swapper/010:01:510
3133799140,12cyclictest15463-21diskmemload12:10:380
3134099131,11cyclictest19819-21ssh09:48:453
31340991310,2cyclictest0-21swapper/312:15:363
31340991310,2cyclictest0-21swapper/311:25:143
31340991310,2cyclictest0-21swapper/311:05:143
31340991310,2cyclictest0-21swapper/310:54:203
31340991310,2cyclictest0-21swapper/310:46:003
31340991310,2cyclictest0-21swapper/310:15:303
31340991310,2cyclictest0-21swapper/310:05:043
3134099130,2cyclictest0-21swapper/311:55:583
3134099130,2cyclictest0-21swapper/311:55:573
3134099130,2cyclictest0-21swapper/311:31:003
3134099130,2cyclictest0-21swapper/311:13:163
3134099130,2cyclictest0-21swapper/310:14:113
3134099130,2cyclictest0-21swapper/309:11:093
3134099130,12cyclictest0-21swapper/309:00:183
3134099130,11cyclictest0-21swapper/310:55:543
3134099130,11cyclictest0-21swapper/309:52:023
31339991311,1cyclictest15463-21diskmemload11:45:582
31339991310,2cyclictest31798-21kworker/u8:208:49:042
31339991310,2cyclictest22388-21kworker/u8:208:14:042
31339991310,2cyclictest17592-21kworker/u8:109:37:042
31339991310,2cyclictest1134-21kworker/u8:207:31:042
3133999130,2cyclictest0-21swapper/207:20:002
3133999130,12cyclictest0-21swapper/211:32:582
3133999130,12cyclictest0-21swapper/207:45:282
31338991310,2cyclictest0-21swapper/112:16:331
31338991310,2cyclictest0-21swapper/110:10:131
3133899130,7cyclictest0-21swapper/111:53:041
3133899130,2cyclictest0-21swapper/110:36:041
3133899130,2cyclictest0-21swapper/109:13:041
3133899130,2cyclictest0-21swapper/108:20:041
3133799139,3cyclictest0-21swapper/012:36:150
3133799139,2cyclictest5138-21ssh09:32:490
3133799134,7cyclictest4324-21nscd09:45:250
3133799132,6cyclictest0-21swapper/008:25:000
31337991310,2cyclictest15463-21diskmemload09:51:450
31337991310,2cyclictest0-21swapper/012:16:100
16162130,0sleep30-21swapper/310:38:393
3134099129,2cyclictest0-21swapper/312:20:393
3134099129,2cyclictest0-21swapper/308:30:193
3134099128,2cyclictest0-21swapper/308:50:153
3134099120,2cyclictest0-21swapper/312:10:083
3134099120,2cyclictest0-21swapper/308:55:143
3134099120,11cyclictest0-21swapper/311:40:133
3134099120,11cyclictest0-21swapper/311:02:453
3134099120,11cyclictest0-21swapper/310:26:413
3134099120,11cyclictest0-21swapper/309:36:363
3134099120,11cyclictest0-21swapper/308:40:173
3134099120,11cyclictest0-21swapper/308:10:153
3133999129,2cyclictest0-21swapper/209:53:182
3133999129,2cyclictest0-21swapper/208:00:152
3133999122,7cyclictest0-21swapper/208:17:592
3133999120,2cyclictest0-21swapper/212:27:572
3133999120,2cyclictest0-21swapper/207:50:192
3133999120,11cyclictest0-21swapper/210:46:392
3133999120,11cyclictest0-21swapper/210:16:372
3133899122,6cyclictest0-21swapper/109:04:591
3133899121,9cyclictest0-21swapper/107:25:061
3133799129,2cyclictest0-21swapper/011:39:050
3133799129,2cyclictest0-21swapper/008:45:230
3133799129,2cyclictest0-21swapper/008:30:130
3133799129,2cyclictest0-21swapper/007:40:150
3133799129,2cyclictest0-21swapper/007:25:260
3133799122,7cyclictest0-21swapper/007:57:020
3133799122,2cyclictest0-21swapper/010:45:420
31337991210,1cyclictest0-21swapper/011:30:190
3133799120,11cyclictest0-21swapper/010:25:030
3133799120,11cyclictest0-21swapper/009:05:140
3133799120,11cyclictest0-21swapper/008:55:170
3134099118,2cyclictest0-21swapper/308:01:313
3134099111,6cyclictest0-21swapper/307:45:013
3134099110,2cyclictest0-21swapper/307:47:373
3134099110,10cyclictest0-21swapper/308:45:293
3134099110,10cyclictest0-21swapper/307:54:383
3133999118,2cyclictest1134-21kworker/u8:207:16:052
3133999117,3cyclictest1134-21kworker/u8:207:25:052
3133999112,6cyclictest0-21swapper/209:05:272
3133799118,2cyclictest0-21swapper/008:35:160
3133799116,3cyclictest26547-21cat08:15:180
3133799112,6cyclictest0-21swapper/007:14:010
3134099107,2cyclictest0-21swapper/307:15:183
3134099105,3cyclictest10754-21tail07:35:253
3134099102,6cyclictest0-21swapper/307:25:253
3134099101,6cyclictest0-21swapper/307:20:103
3133999102,7cyclictest0-21swapper/208:56:592
3133799104,4cyclictest8089-21kernelversion07:30:170
3133799104,4cyclictest24729-21awk08:10:170
3133799104,4cyclictest13713-21cpuspeed_turbos07:45:140
3133799103,5cyclictest8029-21fschecks_count08:50:140
3133799103,5cyclictest4985-21munin-run07:25:000
3133799103,5cyclictest2177-21sendmail_mailtr07:15:230
3133799103,5cyclictest10796-21unixbench_singl07:35:240
3133799102,6cyclictest0-21swapper/007:50:180
313409994,3cyclictest7988-21hwlatdetect07:30:153
313409994,3cyclictest27880-21telnet08:20:133
313409994,3cyclictest15058-21head09:05:293
313409991,3cyclictest5129-21runrttasks07:11:453
313409983,3cyclictest21720-21chrt08:05:063
313409983,3cyclictest17558-21head07:55:143
313409983,3cyclictest1360-21chrt08:35:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional