You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 10:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Fri Nov 21, 2025 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1516211793,19sleep00-21swapper/019:05:560
1773211599,11sleep10-21swapper/119:09:131
1548210797,6sleep20-21swapper/219:06:222
1544210796,6sleep30-21swapper/319:06:183
117322750,0sleep00-21swapper/023:52:450
21542740,2sleep2186699cyclictest23:09:262
29892710,0sleep00-21swapper/019:10:190
166802680,1sleep20-21swapper/200:31:392
300222510,0sleep10-21swapper/121:55:251
86842270,0sleep30-21swapper/322:40:523
86842270,0sleep30-21swapper/322:40:513
278422240,1sleep30-21swapper/322:27:343
186599220,20cyclictest0-21swapper/123:30:121
1865992118,2cyclictest16433-21kworker/u8:220:12:041
186599210,4cyclictest0-21swapper/120:16:031
153002210,0sleep10-21swapper/122:48:561
1866992017,2cyclictest24359-21kworker/u8:022:28:052
1866992017,2cyclictest16646-21kworker/u8:000:24:042
186599203,2cyclictest0-21swapper/119:39:041
186599202,2cyclictest0-21swapper/121:04:041
1865992017,2cyclictest8436-21kworker/u8:120:58:011
1865992017,2cyclictest2074-21kworker/u8:120:38:011
1865992017,2cyclictest19238-21kworker/u8:323:26:061
186599201,2cyclictest0-21swapper/120:21:041
186599201,2cyclictest0-21swapper/119:58:041
186599201,2cyclictest0-21swapper/119:58:041
186599201,2cyclictest0-21swapper/119:31:041
186599201,13cyclictest0-21swapper/119:16:011
186599200,4cyclictest0-21swapper/122:00:041
186599200,4cyclictest0-21swapper/121:17:041
186599200,3cyclictest0-21swapper/122:44:011
186599200,3cyclictest0-21swapper/122:44:011
1864992017,2cyclictest26726-21kworker/u8:121:42:040
186699192,2cyclictest0-21swapper/200:06:022
1866991916,2cyclictest26948-21kworker/u8:023:39:052
1866991916,2cyclictest16433-21kworker/u8:219:56:042
1866991916,2cyclictest16433-21kworker/u8:219:56:032
186699191,17cyclictest0-21swapper/222:01:042
186699191,12cyclictest0-21swapper/223:19:042
186599193,2cyclictest0-21swapper/121:43:041
1865991916,2cyclictest6014-21kworker/u8:220:42:001
1865991916,2cyclictest5189-21kworker/u8:221:34:051
1865991916,2cyclictest30465-21kworker/u8:020:47:041
1865991916,2cyclictest30465-21kworker/u8:020:29:041
1865991916,2cyclictest16439-21kworker/u8:222:55:041
1865991916,2cyclictest13633-21kworker/u8:200:32:061
1865991916,2cyclictest11787-21kworker/u8:221:08:001
1865991916,2cyclictest11787-21kworker/u8:220:53:041
1865991916,2cyclictest10048-21kworker/u8:022:50:041
1865991916,2cyclictest10048-21kworker/u8:022:50:031
186599190,3cyclictest0-21swapper/123:04:011
1864991916,2cyclictest18107-21kworker/u8:122:18:040
186699181,2cyclictest0-21swapper/221:20:052
186699180,4cyclictest0-21swapper/220:04:042
186699180,2cyclictest0-21swapper/219:22:042
1865991815,2cyclictest32728-21kworker/u8:022:06:001
1865991815,2cyclictest30768-21kworker/u8:122:32:041
1865991815,2cyclictest27909-21kworker/u8:100:27:051
1865991815,2cyclictest19128-21kworker/u8:219:25:041
1865991815,2cyclictest16646-21kworker/u8:000:06:061
1865991815,2cyclictest12458-21kworker/u8:020:01:041
1864991815,2cyclictest1929-21kworker/u8:322:37:050
1864991815,2cyclictest13633-21kworker/u8:200:34:060
157682180,0sleep30-21swapper/319:40:233
186799172,13cyclictest39-21ksoftirqd/319:25:123
186799170,14cyclictest0-21swapper/320:40:153
186699172,2cyclictest0-21swapper/222:21:042
1866991714,2cyclictest5974-21kworker/u8:222:30:052
1866991714,2cyclictest27909-21kworker/u8:100:26:042
186699171,13cyclictest0-21swapper/222:47:052
186699170,2cyclictest0-21swapper/220:46:042
1865991714,2cyclictest5974-21kworker/u8:222:11:041
1865991714,2cyclictest5105-21kworker/u8:123:12:041
1865991714,2cyclictest24359-21kworker/u8:022:27:041
1865991714,2cyclictest2074-21kworker/u8:120:33:041
1865991714,2cyclictest1929-21kworker/u8:322:38:041
1865991714,2cyclictest18919-21kworker/u8:123:45:031
1865991714,2cyclictest18107-21kworker/u8:122:22:041
1865991714,2cyclictest18107-21kworker/u8:122:19:041
1865991714,2cyclictest11787-21kworker/u8:221:14:041
1865991714,2cyclictest11135-21kworker/u8:100:02:041
186599170,12cyclictest0-21swapper/100:35:531
1864991714,2cyclictest30727-21kworker/u8:223:19:040
1864991714,2cyclictest26726-21kworker/u8:121:57:030
1864991714,2cyclictest24359-21kworker/u8:022:29:040
1864991714,2cyclictest19128-21kworker/u8:219:22:040
1864991714,2cyclictest16646-21kworker/u8:000:06:020
186799161,13cyclictest0-21swapper/322:05:123
186699162,2cyclictest0-21swapper/222:37:052
1866991613,2cyclictest5189-21kworker/u8:221:40:042
1866991613,2cyclictest18416-21kworker/u8:021:15:032
186699161,2cyclictest0-21swapper/221:33:042
186699161,2cyclictest0-21swapper/200:16:042
186699161,10cyclictest0-21swapper/223:14:042
186699160,2cyclictest18157-21diskmemload23:43:052
186699160,2cyclictest0-21swapper/221:29:042
186699160,14cyclictest20746-21seq23:28:272
1865991613,2cyclictest5189-21kworker/u8:221:35:041
1865991613,2cyclictest27909-21kworker/u8:100:17:051
1865991613,2cyclictest26726-21kworker/u8:121:27:041
1865991613,2cyclictest19128-21kworker/u8:219:11:041
1865991613,2cyclictest12647-21kworker/u8:023:22:031
1865991613,2cyclictest11135-21kworker/u8:123:52:041
1864991613,2cyclictest30768-21kworker/u8:122:47:050
1864991613,2cyclictest30465-21kworker/u8:020:46:040
1864991613,2cyclictest26726-21kworker/u8:122:01:040
1864991613,2cyclictest21677-21kworker/u8:021:49:040
1864991613,2cyclictest16646-21kworker/u8:000:37:040
1864991613,2cyclictest12458-21kworker/u8:020:04:040
1864991613,2cyclictest11787-21kworker/u8:221:20:050
186799151,3cyclictest2765-21ssh00:17:203
186799151,12cyclictest37-21rcuc/300:32:443
186799151,12cyclictest30727-21kworker/u8:223:07:343
186799151,12cyclictest27803-21ssh23:01:133
186799151,12cyclictest21285-21ssh00:02:443
186799151,12cyclictest19327-21ssh22:52:503
186799151,12cyclictest19327-21ssh22:52:503
1867991510,3cyclictest28469-21ssh21:54:553
1867991510,3cyclictest17249-21ntpdc21:05:183
1867991510,3cyclictest0-21swapper/323:28:363
186799150,13cyclictest3984-21sshd22:02:293
186799150,13cyclictest397-21sort23:40:273
186799150,13cyclictest0-21swapper/321:55:353
186699151,3cyclictest0-21swapper/222:13:592
1866991512,2cyclictest9776-21kworker/u8:119:43:042
1866991512,2cyclictest19128-21kworker/u8:219:16:052
1866991512,2cyclictest0-21swapper/222:40:282
1866991512,2cyclictest0-21swapper/222:40:282
1866991511,3cyclictest11552-21ssh23:52:272
186699151,12cyclictest0-21swapper/200:01:252
186699150,6cyclictest0-21swapper/219:46:042
186699150,2cyclictest0-21swapper/222:16:582
186699150,2cyclictest0-21swapper/219:33:042
186699150,14cyclictest0-21swapper/221:01:042
1864991512,2cyclictest5974-21kworker/u8:222:21:040
1864991512,2cyclictest5105-21kworker/u8:123:14:030
1864991512,2cyclictest26726-21kworker/u8:121:33:030
186499150,14cyclictest0-21swapper/023:25:120
186499150,13cyclictest27828-21ssh21:53:540
186799149,3cyclictest16604-21ssh21:41:073
186799149,3cyclictest0-21swapper/323:39:073
186799149,3cyclictest0-21swapper/323:20:213
186799149,3cyclictest0-21swapper/300:09:413
186799141,2cyclictest2995-21ssh21:27:223
1867991411,2cyclictest39-21ksoftirqd/319:15:123
1867991411,2cyclictest381ktimersoftd/323:55:133
1867991411,2cyclictest37-21rcuc/321:46:363
1867991411,2cyclictest37-21rcuc/321:32:173
1867991411,2cyclictest0-21swapper/321:23:353
186799141,11cyclictest13192-21ssh21:38:163
1867991410,2cyclictest0-21swapper/321:15:243
186799140,12cyclictest37-21rcuc/322:10:173
186799140,12cyclictest18157-21diskmemload21:11:213
186699141,2cyclictest0-21swapper/200:37:052
1866991411,2cyclictest19238-21kworker/u8:323:32:042
186699140,2cyclictest0-21swapper/222:54:022
186699140,2cyclictest0-21swapper/222:54:022
186699140,13cyclictest0-21swapper/222:57:072
186699140,13cyclictest0-21swapper/221:46:082
186699140,13cyclictest0-21swapper/220:10:182
186699140,12cyclictest18157-21diskmemload23:45:592
1865991411,2cyclictest30727-21kworker/u8:223:08:041
1865991411,2cyclictest11787-21kworker/u8:221:21:041
186599140,2cyclictest0-21swapper/100:21:571
186599140,12cyclictest18157-21diskmemload23:40:191
186599140,12cyclictest0-21swapper/120:05:241
1864991411,2cyclictest9776-21kworker/u8:119:33:040
1864991411,2cyclictest31511-21kworker/u8:021:29:040
1864991411,2cyclictest21670-21ssh21:13:030
1864991411,2cyclictest19238-21kworker/u8:323:35:030
1864991411,2cyclictest18919-21kworker/u8:123:43:050
1864991411,2cyclictest16646-21kworker/u8:000:16:040
1864991411,2cyclictest16433-21kworker/u8:219:46:040
1864991411,2cyclictest0-21swapper/023:09:120
1864991411,2cyclictest0-21swapper/022:53:320
1864991411,2cyclictest0-21swapper/022:53:310
186499140,2cyclictest0-21swapper/023:23:520
186499140,2cyclictest0-21swapper/023:03:220
186499140,2cyclictest0-21swapper/022:32:120
186499140,13cyclictest0-21swapper/021:35:120
186499140,12cyclictest0-21swapper/023:56:050
186499140,12cyclictest0-21swapper/022:55:550
186799138,3cyclictest22952-21ssh22:22:183
1867991310,2cyclictest381ktimersoftd/322:32:343
1867991310,2cyclictest381ktimersoftd/300:10:143
1867991310,2cyclictest0-21swapper/300:24:333
186799130,3cyclictest6333-21ssh23:46:433
186799130,3cyclictest0-21swapper/322:38:183
186799130,2cyclictest0-21swapper/322:56:553
186799130,12cyclictest0-21swapper/320:45:173
186799130,11cyclictest19068-21ssh22:18:413
186799130,11cyclictest0-21swapper/323:30:123
186699139,2cyclictest0-21swapper/200:10:142
1866991310,2cyclictest23704-21kworker/u8:023:01:052
1866991310,2cyclictest16433-21kworker/u8:220:08:012
1866991310,2cyclictest0-21swapper/222:05:402
1866991310,2cyclictest0-21swapper/221:10:182
186699130,2cyclictest0-21swapper/223:55:162
186699130,2cyclictest0-21swapper/223:22:282
186699130,2cyclictest0-21swapper/221:51:182
186699130,1cyclictest0-21swapper/221:56:382
1865991310,2cyclictest30727-21kworker/u8:223:16:011
1865991310,2cyclictest27909-21kworker/u8:100:11:041
1865991310,2cyclictest0-21swapper/123:55:201
1865991310,2cyclictest0-21swapper/119:20:161
186599130,2cyclictest0-21swapper/123:37:171
186599130,12cyclictest0-21swapper/121:50:191
186599130,12cyclictest0-21swapper/121:46:171
1864991310,2cyclictest0-21swapper/023:47:310
1864991310,2cyclictest0-21swapper/023:32:220
1864991310,2cyclictest0-21swapper/022:12:520
1864991310,2cyclictest0-21swapper/022:09:140
186499130,2cyclictest0-21swapper/021:16:220
186499130,2cyclictest0-21swapper/000:27:410
186499130,11cyclictest0-21swapper/000:03:000
304832120,2sleep20-21swapper/220:20:132
186799129,2cyclictest18157-21diskmemload00:26:593
186799129,2cyclictest0-21swapper/323:50:023
186799129,2cyclictest0-21swapper/323:16:053
186799129,2cyclictest0-21swapper/319:45:273
186799129,2cyclictest0-21swapper/319:20:173
186799129,2cyclictest0-21swapper/319:11:273
186799129,2cyclictest0-21swapper/300:35:493
186799127,3cyclictest0-21swapper/320:00:163
186799120,2cyclictest0-21swapper/323:10:543
186799120,2cyclictest0-21swapper/320:05:133
186799120,2cyclictest0-21swapper/319:35:263
186799120,11cyclictest0-21swapper/322:46:183
186799120,11cyclictest0-21swapper/320:30:233
186699129,2cyclictest30465-21kworker/u8:020:25:042
186699129,2cyclictest19128-21kworker/u8:219:13:042
186699129,2cyclictest11787-21kworker/u8:220:51:042
186699120,2cyclictest0-21swapper/220:55:242
186599121,7cyclictest5906-21metacity19:53:591
186599120,10cyclictest0-21swapper/119:45:201
186499129,2cyclictest8436-21kworker/u8:121:01:040
186499129,2cyclictest0-21swapper/022:40:040
186499129,2cyclictest0-21swapper/022:40:040
186499129,2cyclictest0-21swapper/000:10:540
186499123,7cyclictest0-21swapper/020:05:240
186499120,2cyclictest0-21swapper/000:20:190
186499120,11cyclictest0-21swapper/019:50:240
186799118,2cyclictest37-21rcuc/320:50:273
186799118,2cyclictest0-21swapper/319:59:273
186799118,2cyclictest0-21swapper/319:59:273
186799116,3cyclictest1427-21grep20:25:263
186699118,2cyclictest9776-21kworker/u8:119:28:042
186699118,2cyclictest8436-21kworker/u8:121:09:042
186699118,2cyclictest5189-21kworker/u8:221:36:042
186699118,2cyclictest2074-21kworker/u8:120:32:012
186699116,3cyclictest29296-21tail20:15:202
186599112,6cyclictest0-21swapper/119:44:591
186799104,4cyclictest32368-21grep20:25:013
186799104,4cyclictest18949-21tr19:50:143
186799103,3cyclictest27438-21pzem_current20:10:223
186799102,6cyclictest0-21swapper/321:00:253
186799102,6cyclictest0-21swapper/320:55:313
186799102,6cyclictest0-21swapper/320:35:243
186699107,2cyclictest2074-21kworker/u8:120:35:042
186699105,3cyclictest7870-21sshd20:42:192
186699105,3cyclictest18377-21munin-node2
186499107,2cyclictest0-21swapper/021:05:220
186499105,3cyclictest20542-21aten2.4-expect19:55:140
186499105,3cyclictest20542-21aten2.4-expect19:55:140
186499100,9cyclictest0-21swapper/020:52:320
18679994,3cyclictest28557-21cut20:15:143
18679994,3cyclictest10540-21cpuspeed_turbos19:30:133
18669994,3cyclictest12563-21grep19:35:142
18649996,2cyclictest0-21swapper/020:32:230
18649990,8cyclictest0-21swapper/019:44:220
18649985,2cyclictest0-21swapper/020:26:540
18649985,2cyclictest0-21swapper/020:12:580
18649984,3cyclictest5129-21runrttasks19:29:330
18649984,2cyclictest4829-21Xorg20:37:590
18649983,3cyclictest6544-21cpuspeed_turbos20:40:160
18649983,3cyclictest29548-21mailstats20:15:260
18649983,3cyclictest13665-21grep20:55:330
18649983,3cyclictest101ktimersoftd/019:15:120
18649980,7cyclictest0-21swapper/020:20:520
18649980,6cyclictest12315-21chrt19:35:030
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional