You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 18:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Fri Nov 28, 2025 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30333212098,16sleep10-21swapper/107:06:211
29838211896,15sleep20-21swapper/207:05:222
30420211793,19sleep00-21swapper/007:07:280
29847210987,15sleep30-21swapper/307:05:233
300582760,0sleep30-21swapper/311:44:183
191272760,0sleep20-21swapper/210:57:282
63712670,2sleep09-21ksoftirqd/011:18:500
34992670,0sleep30-21swapper/307:20:243
110512670,0sleep00-21swapper/010:49:280
30492630,1sleep30-21swapper/308:40:133
3065199370,2cyclictest0-21swapper/310:30:133
312432260,1sleep0101ktimersoftd/012:20:100
3065099253,17cyclictest0-21swapper/212:19:052
3064999240,7cyclictest0-21swapper/111:05:111
3065099233,18cyclictest0-21swapper/211:22:052
30651992219,2cyclictest9759-21kworker/u8:311:28:053
30650992219,2cyclictest19210-21kworker/u8:309:28:042
30650992219,2cyclictest16716-21kworker/u8:209:16:012
3065099221,19cyclictest0-21swapper/210:44:012
3065099220,17cyclictest0-21swapper/210:25:052
30651992118,2cyclictest32227-21kworker/u8:110:48:063
30651992118,2cyclictest29680-21kworker/u8:012:22:053
30651992118,2cyclictest12282-21kworker/u8:312:38:043
3065199211,18cyclictest0-21swapper/309:30:123
3065099212,2cyclictest6315-21copy10:10:052
30650992118,2cyclictest5759-21kworker/u8:110:21:052
30650992118,2cyclictest5257-21kworker/u8:010:52:052
30650992118,2cyclictest5257-21kworker/u8:010:49:052
30650992118,2cyclictest4857-21kworker/u8:011:18:052
30650992118,2cyclictest32574-21kworker/u8:010:09:052
30650992118,2cyclictest2637-21kworker/u8:212:29:052
30650992118,2cyclictest2637-21kworker/u8:212:29:042
30650992118,2cyclictest25023-21kworker/u8:208:18:052
30650992118,2cyclictest24929-21kworker/u8:312:23:042
30650992118,2cyclictest18954-21kworker/u8:112:06:042
30650992118,2cyclictest14891-21kworker/u8:211:08:052
30650992118,2cyclictest10926-21kworker/u8:109:10:052
3065099210,16cyclictest0-21swapper/209:48:052
3064899211,2cyclictest14735-21diskmemload12:25:100
3064899211,2cyclictest14735-21diskmemload12:25:100
169772210,0sleep00-21swapper/009:45:270
30651992017,2cyclictest4489-21kworker/u8:307:34:053
30651992017,2cyclictest32227-21kworker/u8:110:50:053
30651992017,2cyclictest32227-21kworker/u8:110:40:043
30651992017,2cyclictest24871-21kworker/u8:210:12:023
30651992016,2cyclictest12957-21kworker/u8:209:42:053
3065199201,2cyclictest14735-21diskmemload09:14:053
3065199200,18cyclictest0-21swapper/311:49:053
30650992017,2cyclictest8756-21kworker/u8:007:49:052
30650992017,2cyclictest4488-21kworker/u8:107:40:052
30650992017,2cyclictest4488-21kworker/u8:107:27:052
30650992017,2cyclictest2795-21kworker/u8:108:49:052
30650992017,2cyclictest18954-21kworker/u8:112:13:052
30650992017,2cyclictest16716-21kworker/u8:209:22:052
30650992017,2cyclictest14891-21kworker/u8:211:26:062
3065099201,2cyclictest0-21swapper/209:58:062
3065099200,2cyclictest0-21swapper/211:00:132
3065099200,18cyclictest29-21rcuc/211:10:122
3064999200,2cyclictest7425-21sh11:20:101
193222200,1sleep0101ktimersoftd/010:57:450
30651991916,2cyclictest9759-21kworker/u8:311:38:053
30651991916,2cyclictest9759-21kworker/u8:311:31:053
30651991916,2cyclictest5759-21kworker/u8:110:23:053
30651991916,2cyclictest377-21kworker/u8:109:36:053
30651991916,2cyclictest25570-21kworker/u8:111:19:043
30651991916,2cyclictest16716-21kworker/u8:209:17:053
30651991916,2cyclictest14891-21kworker/u8:210:55:053
3065199191,16cyclictest0-21swapper/310:05:123
3065199190,2cyclictest0-21swapper/311:20:113
30650991916,2cyclictest9759-21kworker/u8:311:51:052
30650991916,2cyclictest9759-21kworker/u8:311:36:022
30650991916,2cyclictest9449-21kworker/u8:212:00:052
30650991916,2cyclictest4488-21kworker/u8:107:38:042
30650991916,2cyclictest377-21kworker/u8:109:40:052
30650991916,2cyclictest3584-21kworker/u8:007:15:052
30650991916,2cyclictest32227-21kworker/u8:110:39:052
30650991916,2cyclictest31073-21kworker/u8:008:39:052
30650991916,2cyclictest24871-21kworker/u8:210:01:052
30650991916,2cyclictest19210-21kworker/u8:309:32:052
3064999191,16cyclictest0-21swapper/110:05:111
30651991815,2cyclictest9449-21kworker/u8:212:03:053
30651991815,2cyclictest420-21kworker/u8:111:53:053
30651991815,2cyclictest3584-21kworker/u8:007:12:043
30651991815,2cyclictest29680-21kworker/u8:012:31:053
30651991815,2cyclictest24871-21kworker/u8:210:03:043
30651991815,2cyclictest23097-21kworker/u8:109:56:053
30651991815,2cyclictest20505-21kworker/u8:008:05:053
30651991815,2cyclictest10926-21kworker/u8:109:07:053
30650991815,2cyclictest31073-21kworker/u8:008:58:062
30650991815,2cyclictest16847-21kworker/u8:207:59:052
30651991714,2cyclictest9449-21kworker/u8:211:56:053
30651991714,2cyclictest5759-21kworker/u8:110:26:013
30651991714,2cyclictest31073-21kworker/u8:008:30:053
30651991714,2cyclictest29680-21kworker/u8:012:28:053
30651991714,2cyclictest29680-21kworker/u8:012:28:043
30651991714,2cyclictest27430-21kworker/u8:009:27:043
30651991714,2cyclictest25023-21kworker/u8:208:35:043
30651991714,2cyclictest25023-21kworker/u8:208:21:053
30651991714,2cyclictest18954-21kworker/u8:112:17:053
30651991714,2cyclictest16847-21kworker/u8:208:00:043
30651991714,2cyclictest14891-21kworker/u8:211:12:023
30651991714,2cyclictest11251-21kworker/u8:010:16:013
30650991714,2cyclictest4488-21kworker/u8:108:14:052
30650991714,2cyclictest4488-21kworker/u8:107:30:022
3065099170,2cyclictest0-21swapper/210:31:052
3065099170,15cyclictest0-21swapper/210:17:042
30651991613,2cyclictest9449-21kworker/u8:212:11:053
30651991613,2cyclictest8756-21kworker/u8:007:41:053
30651991613,2cyclictest4488-21kworker/u8:108:15:053
30651991613,2cyclictest24004-21kworker/u8:010:35:053
30651991613,2cyclictest23097-21kworker/u8:109:53:043
3065099160,2cyclictest0-21swapper/211:55:042
30648991611,3cyclictest0-21swapper/008:05:250
30651991512,2cyclictest8756-21kworker/u8:007:50:053
30651991512,2cyclictest3584-21kworker/u8:007:25:053
30651991512,2cyclictest31073-21kworker/u8:008:55:053
30651991512,2cyclictest14891-21kworker/u8:211:01:053
3065199150,2cyclictest0-21swapper/309:47:053
3065199150,13cyclictest14735-21diskmemload11:09:123
3065199150,10cyclictest0-21swapper/312:05:043
30650991510,3cyclictest9250-21ssh09:37:562
3065099150,13cyclictest0-21swapper/209:05:172
3064999152,11cyclictest0-21swapper/111:25:241
3064999150,3cyclictest31265-21cpu08:30:121
3064999150,13cyclictest0-21swapper/110:50:091
30648991512,2cyclictest0-21swapper/011:42:410
30648991510,3cyclictest8898-21perl12:30:180
30648991510,3cyclictest25606-21ssh12:13:570
30648991510,3cyclictest0-21swapper/011:00:270
154012150,1sleep0101ktimersoftd/012:36:510
30651991411,2cyclictest4488-21kworker/u8:108:11:053
3065199140,12cyclictest29116-21munin-node3
30650991411,2cyclictest0-21swapper/211:40:052
3065099140,3cyclictest410-21latency_hist08:35:002
3065099140,2cyclictest0-21swapper/209:50:012
3065099140,2cyclictest0-21swapper/208:54:042
3065099140,2cyclictest0-21swapper/208:03:042
3065099140,13cyclictest0-21swapper/207:11:052
3064999149,3cyclictest5463-21sh11:51:451
3064999144,8cyclictest2614-21aten2.4_r3power11:15:121
30649991411,2cyclictest14735-21diskmemload12:18:421
30649991411,2cyclictest14735-21diskmemload11:43:321
30649991411,2cyclictest14735-21diskmemload09:40:011
30649991410,2cyclictest26086-21sed10:30:171
3064999140,12cyclictest14735-21diskmemload12:04:591
3064999140,12cyclictest14735-21diskmemload11:12:421
3064999140,12cyclictest0-21swapper/111:45:261
3064999140,12cyclictest0-21swapper/110:41:061
3064999140,12cyclictest0-21swapper/110:36:371
3064999140,12cyclictest0-21swapper/110:11:421
3064999140,12cyclictest0-21swapper/109:55:371
3064999140,12cyclictest0-21swapper/109:53:091
3064999140,12cyclictest0-21swapper/109:32:021
3064999140,12cyclictest0-21swapper/109:12:581
3064899149,3cyclictest6868-21chrt08:50:120
30648991411,2cyclictest0-21swapper/011:23:300
30648991411,2cyclictest0-21swapper/010:35:110
30648991411,2cyclictest0-21swapper/009:17:500
30648991410,2cyclictest5413-21taskset11:51:390
3064899140,2cyclictest0-21swapper/012:00:130
3064899140,2cyclictest0-21swapper/009:43:400
3064899140,2cyclictest0-21swapper/009:29:110
3064899140,13cyclictest0-21swapper/011:09:310
3064899140,12cyclictest30794-21ssh11:45:080
3064899140,12cyclictest28212-21ssh09:24:110
3064899140,12cyclictest14735-21diskmemload11:10:170
3064899140,12cyclictest14735-21diskmemload10:53:270
3064899140,12cyclictest0-21swapper/010:15:540
3064899140,12cyclictest0-21swapper/009:32:210
3065199139,2cyclictest0-21swapper/309:21:473
30651991310,2cyclictest8756-21kworker/u8:007:46:053
3065199130,2cyclictest0-21swapper/309:00:043
3065199130,2cyclictest0-21swapper/308:51:043
3065199130,2cyclictest0-21swapper/308:45:053
3065199130,2cyclictest0-21swapper/307:37:043
3065099132,7cyclictest0-21swapper/208:21:022
30650991310,2cyclictest0-21swapper/212:36:592
30650991310,2cyclictest0-21swapper/211:31:182
3065099130,2cyclictest0-21swapper/209:02:052
3065099130,2cyclictest0-21swapper/208:25:042
3065099130,12cyclictest0-21swapper/212:30:182
3065099130,12cyclictest0-21swapper/211:48:062
3064999139,2cyclictest0-21swapper/112:30:451
3064999138,3cyclictest29950-21ntpdc08:25:171
3064999138,3cyclictest2749-21cpuspeed_turbos07:20:141
3064999138,3cyclictest0-21swapper/107:25:121
3064999132,6cyclictest0-21swapper/108:49:591
3064999131,10cyclictest0-21swapper/107:30:111
30649991310,2cyclictest14735-21diskmemload10:47:481
30649991310,2cyclictest0-21swapper/111:34:311
30649991310,2cyclictest0-21swapper/110:16:361
30649991310,2cyclictest0-21swapper/110:00:101
3064999130,2cyclictest26952-21ssh09:22:071
3064999130,2cyclictest0-21swapper/112:38:171
3064999130,2cyclictest0-21swapper/112:10:501
3064999130,2cyclictest0-21swapper/109:45:171
3064999130,2cyclictest0-21swapper/109:25:231
3064999130,2cyclictest0-21swapper/108:00:171
3064999130,2cyclictest0-21swapper/108:00:001
3064999130,12cyclictest0-21swapper/112:22:061
3064999130,12cyclictest0-21swapper/110:55:111
3064999130,11cyclictest0-21swapper/112:05:471
3064999130,11cyclictest0-21swapper/109:15:201
30648991310,2cyclictest13934-21ssh11:25:510
30648991310,2cyclictest0-21swapper/010:32:030
30648991310,2cyclictest0-21swapper/009:56:010
30648991310,2cyclictest0-21swapper/009:53:240
3064899130,2cyclictest0-21swapper/012:09:400
3064899130,2cyclictest0-21swapper/011:57:290
3064899130,2cyclictest0-21swapper/010:01:310
3064899130,2cyclictest0-21swapper/009:35:550
3064899130,2cyclictest0-21swapper/008:55:210
3064899130,2cyclictest0-21swapper/007:20:130
3064899130,12cyclictest0-21swapper/011:32:510
3064899130,12cyclictest0-21swapper/010:41:110
3064899130,12cyclictest0-21swapper/010:23:050
3064899130,12cyclictest0-21swapper/010:09:040
239672130,2sleep014735-21diskmemload11:37:030
3065199129,2cyclictest3584-21kworker/u8:007:16:063
3065099120,11cyclictest0-21swapper/207:20:142
3064999129,2cyclictest0-21swapper/109:35:201
3064999129,2cyclictest0-21swapper/107:40:181
3064999128,2cyclictest0-21swapper/112:25:151
3064999128,2cyclictest0-21swapper/112:25:141
3064999121,9cyclictest15968-21apt-get10:20:091
3064999121,3cyclictest21987-21ssh11:00:201
3064999120,2cyclictest0-21swapper/110:25:071
3064999120,2cyclictest0-21swapper/109:00:211
3064999120,2cyclictest0-21swapper/108:15:171
3064999120,11cyclictest0-21swapper/111:55:471
3064999120,11cyclictest0-21swapper/108:35:141
3064999120,11cyclictest0-21swapper/108:20:131
3064899129,2cyclictest14735-21diskmemload12:15:070
3064899129,2cyclictest0-21swapper/010:28:100
3064899129,2cyclictest0-21swapper/009:10:270
3064899129,2cyclictest0-21swapper/009:05:190
3064899129,2cyclictest0-21swapper/008:25:260
3064899129,2cyclictest0-21swapper/007:15:140
3064899129,2cyclictest0-21swapper/007:10:170
3064899128,2cyclictest0-21swapper/007:50:130
3064899121,6cyclictest0-21swapper/009:05:000
3064899120,2cyclictest14735-21diskmemload10:12:530
3064899120,2cyclictest0-21swapper/008:20:190
3064899120,2cyclictest0-21swapper/007:35:130
3064899120,2cyclictest0-21swapper/007:25:150
3064899120,11cyclictest0-21swapper/008:35:240
3064899120,11cyclictest0-21swapper/008:02:450
3065099110,2cyclictest0-21swapper/207:53:052
3064999118,2cyclictest0-21swapper/108:05:141
3064999117,2cyclictest15166-21diskstats07:50:141
3064999111,5cyclictest0-21swapper/107:15:001
3064999110,9cyclictest30860-21munin-node1
3064999110,9cyclictest0-21swapper/111:35:171
3064999110,9cyclictest0-21swapper/107:45:121
3064999110,3cyclictest9201-21grep08:55:131
3064999110,3cyclictest6797-21latency_hist08:50:001
3064999110,3cyclictest0-21swapper/107:35:211
3064999110,10cyclictest0-21swapper/108:10:121
3064899118,2cyclictest0-21swapper/008:15:170
3064899118,2cyclictest0-21swapper/007:45:130
3064899116,3cyclictest6910-21grep07:30:130
3064999100,8cyclictest0-21swapper/109:05:271
3064899107,2cyclictest22824-21copy08:10:020
3064899107,2cyclictest0-21swapper/008:40:160
3064899101,7cyclictest0-21swapper/008:49:040
306519994,3cyclictest16955-21cpu07:55:113
306509994,3cyclictest5129-21runrttasks08:42:292
306509994,3cyclictest21292-21df_abs08:05:132
306499991,3cyclictest0-21swapper/108:40:171
306489996,2cyclictest0-21swapper/007:40:200
306489994,3cyclictest17990-21timerandwakeup07:55:230
306489991,6cyclictest0-21swapper/008:34:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional