You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 17:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot6s.osadl.org (updated Sat Nov 22, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22129212095,17sleep30-21swapper/307:05:253
22586211696,15sleep20-21swapper/207:06:122
22636210376,21sleep00-21swapper/007:06:500
2253229978,13sleep10-21swapper/107:05:311
24672680,0sleep20-21swapper/211:57:562
104472280,0sleep0101ktimersoftd/009:13:060
72712250,0sleep20-21swapper/209:44:552
153722250,1sleep315373-21sh12:10:473
2291299221,3cyclictest5129-21runrttasks09:50:142
2291299221,3cyclictest17497-21sh11:05:142
223852220,1sleep30-21swapper/312:19:363
2291299211,2cyclictest0-21swapper/208:10:132
22910992118,2cyclictest23749-21kworker/u8:010:02:040
2291099211,18cyclictest26807-21ssh11:15:140
2291099200,2cyclictest0-21swapper/011:05:130
2291099200,2cyclictest0-21swapper/008:10:130
305382190,0sleep00-21swapper/011:53:240
40762180,1sleep30-21swapper/309:40:273
22910991811,6cyclictest17408-21sh12:14:050
314932170,1sleep20-21swapper/210:10:162
275122170,0sleep30-21swapper/310:40:203
2291299171,3cyclictest0-21swapper/211:30:132
2291199171,13cyclictest26823-21munin-run11:49:591
2291199170,15cyclictest0-21swapper/109:10:111
2291399161,3cyclictest6967-21diskmemload09:50:183
2291299161,3cyclictest26707-21ssh11:15:052
2291199161,13cyclictest16962-21ssh11:04:411
22910991612,2cyclictest17020-21sh09:55:150
22910991610,5cyclictest25061-1kworker/u9:012:26:040
2291399151,3cyclictest7172-21ssh12:37:013
22913991512,2cyclictest0-21swapper/309:25:243
2291399151,12cyclictest27391-21apt-get08:40:123
22913991510,3cyclictest2373-21ssh10:48:563
22913991510,3cyclictest20350-21ssh09:24:033
22913991510,3cyclictest0-21swapper/310:07:063
2291399150,13cyclictest10140-21ssh11:30:353
2291299151,12cyclictest5129-21runrttasks11:14:582
2291299151,12cyclictest0-21swapper/212:05:402
22912991510,3cyclictest4904-21munin-node2
22912991510,3cyclictest4454-21ps10:50:202
2291299150,13cyclictest0-21swapper/212:25:232
2291299150,13cyclictest0-21swapper/210:42:292
2291199150,3cyclictest0-21swapper/111:05:551
2291199150,13cyclictest4366-21nscd11:17:511
2291199150,13cyclictest0-21swapper/112:20:261
2291099151,3cyclictest1515-21perl08:55:140
22910991512,2cyclictest7267-21kworker/u8:109:53:040
22910991512,2cyclictest22335-21kworker/u8:012:36:040
22910991512,2cyclictest0-21swapper/010:25:160
2291099150,13cyclictest0-21swapper/008:35:000
2291099150,13cyclictest0-21swapper/008:35:000
2291399149,3cyclictest25422-21ssh12:21:553
2291399149,3cyclictest17847-21ssh11:40:143
2291399149,3cyclictest0-21swapper/311:10:133
2291399149,3cyclictest0-21swapper/308:10:133
2291399142,6cyclictest23133-21copy08:30:003
2291399142,6cyclictest23133-21copy08:30:003
2291399141,3cyclictest0-21swapper/309:46:483
22913991411,2cyclictest0-21swapper/311:49:123
22913991411,2cyclictest0-21swapper/311:20:233
22913991411,2cyclictest0-21swapper/311:18:333
22913991411,2cyclictest0-21swapper/310:22:133
2291399141,11cyclictest9074-21ssh09:10:483
2291399140,2cyclictest0-21swapper/310:28:033
2291399140,2cyclictest0-21swapper/309:30:133
2291399140,2cyclictest0-21swapper/308:45:123
2291399140,13cyclictest0-21swapper/311:26:123
2291399140,12cyclictest0-21swapper/310:57:153
2291399140,12cyclictest0-21swapper/310:01:253
2291299149,3cyclictest26717-21ssh12:24:042
2291299149,3cyclictest24282-21ssh10:36:512
2291299142,6cyclictest0-21swapper/210:35:012
2291299141,11cyclictest0-21swapper/209:32:342
2291299140,13cyclictest0-21swapper/211:29:482
2291299140,12cyclictest21862-21ssh12:18:452
2291299140,12cyclictest1955-21ssh10:48:152
2291299140,12cyclictest18199-21df_abs11:40:142
2291299140,12cyclictest15396-21ssh10:27:272
2291299140,12cyclictest0-21swapper/211:38:082
2291299140,12cyclictest0-21swapper/210:55:122
2291299140,12cyclictest0-21swapper/210:00:242
2291299140,12cyclictest0-21swapper/208:15:122
2291199149,3cyclictest5010-21snmpd12:26:211
2291199141,2cyclictest17839-21apt-get11:40:131
22911991411,2cyclictest0-21swapper/109:55:141
22911991410,2cyclictest6967-21diskmemload12:15:331
22911991410,2cyclictest0-21swapper/110:17:231
2291199140,3cyclictest10583-21ssh09:47:411
2291199140,12cyclictest6967-21diskmemload11:27:291
2291199140,12cyclictest6967-21diskmemload09:34:031
2291199140,12cyclictest32618-21awk07:30:261
2291199140,12cyclictest0-21swapper/112:03:501
2291199140,12cyclictest0-21swapper/110:57:111
2291199140,12cyclictest0-21swapper/110:33:401
22910991411,2cyclictest1270-21kworker/u8:111:58:050
2291099140,2cyclictest0-21swapper/011:26:050
2291099140,12cyclictest0-21swapper/012:05:000
2291099140,12cyclictest0-21swapper/009:20:120
2291099140,12cyclictest0-21swapper/008:25:140
2291099140,12cyclictest0-21swapper/008:25:130
60122130,2sleep36022-21ssh10:51:593
235012130,2sleep06967-21diskmemload09:26:340
2291399139,2cyclictest0-21swapper/307:59:153
2291399139,2cyclictest0-21swapper/307:25:133
2291399138,3cyclictest25199-21ssh10:38:223
2291399134,6cyclictest5906-21metacity12:08:593
22913991310,2cyclictest0-21swapper/312:25:053
22913991310,2cyclictest0-21swapper/311:05:203
22913991310,2cyclictest0-21swapper/311:00:573
22913991310,2cyclictest0-21swapper/310:11:043
22913991310,2cyclictest0-21swapper/309:16:263
2291399130,2cyclictest0-21swapper/312:00:213
2291399130,2cyclictest0-21swapper/311:52:153
2291399130,2cyclictest0-21swapper/310:16:243
2291399130,2cyclictest0-21swapper/309:37:073
2291399130,12cyclictest0-21swapper/312:31:033
2291399130,12cyclictest0-21swapper/311:55:323
2291399130,12cyclictest0-21swapper/310:33:033
2291399130,12cyclictest0-21swapper/309:55:433
2291299139,2cyclictest0-21swapper/211:02:292
2291299138,3cyclictest4198-21tail12:00:122
2291299133,6cyclictest9204-21ssh10:20:242
2291299132,6cyclictest0-21swapper/207:40:012
22912991310,2cyclictest0-21swapper/212:35:182
22912991310,2cyclictest0-21swapper/208:25:272
22912991310,2cyclictest0-21swapper/208:25:262
22912991310,2cyclictest0-21swapper/207:25:242
22912991310,2cyclictest0-21swapper/207:15:182
2291299130,2cyclictest0-21swapper/212:30:072
2291299130,2cyclictest0-21swapper/211:20:172
2291299130,2cyclictest0-21swapper/210:16:182
2291299130,2cyclictest0-21swapper/210:08:592
2291299130,2cyclictest0-21swapper/209:45:182
2291299130,2cyclictest0-21swapper/209:28:372
2291299130,2cyclictest0-21swapper/209:20:182
2291299130,11cyclictest0-21swapper/211:52:142
2291299130,11cyclictest0-21swapper/209:35:272
2291299130,11cyclictest0-21swapper/209:17:332
2291199138,3cyclictest21551-21tr09:25:141
2291199138,3cyclictest0-21swapper/110:53:121
2291199138,3cyclictest0-21swapper/108:00:251
22911991310,2cyclictest6967-21diskmemload12:30:421
22911991310,2cyclictest6967-21diskmemload12:10:331
22911991310,2cyclictest6967-21diskmemload11:57:371
22911991310,2cyclictest13239-21apt-get08:05:131
22911991310,2cyclictest0-21swapper/112:07:471
22911991310,2cyclictest0-21swapper/111:50:371
22911991310,2cyclictest0-21swapper/111:38:231
22911991310,2cyclictest0-21swapper/111:24:441
22911991310,2cyclictest0-21swapper/110:35:141
22911991310,2cyclictest0-21swapper/110:20:241
22911991310,2cyclictest0-21swapper/110:14:521
22911991310,2cyclictest0-21swapper/109:44:231
22911991310,2cyclictest0-21swapper/109:15:031
2291199130,2cyclictest0-21swapper/112:37:521
2291199130,12cyclictest0-21swapper/110:02:091
2291199130,12cyclictest0-21swapper/109:35:541
2291199130,11cyclictest6967-21diskmemload10:40:231
2291199130,11cyclictest12261-21ssh11:34:041
2291099131,2cyclictest3360-21munin-node0
2291099131,2cyclictest32319-21aten2.4_r3power12:30:140
22910991311,1cyclictest0-21swapper/010:33:560
22910991310,2cyclictest7267-21kworker/u8:109:48:010
2291099130,2cyclictest0-21swapper/012:16:040
2291099130,2cyclictest0-21swapper/011:36:250
2291099130,2cyclictest0-21swapper/010:45:140
2291099130,2cyclictest0-21swapper/010:15:260
2291099130,12cyclictest0-21swapper/008:20:240
2291099130,11cyclictest0-21swapper/007:30:120
288322120,2sleep00-21swapper/010:41:410
2291399129,2cyclictest0-21swapper/311:36:163
2291399120,11cyclictest0-21swapper/309:05:173
2291399120,11cyclictest0-21swapper/309:00:203
2291299129,2cyclictest0-21swapper/211:45:042
2291299129,2cyclictest0-21swapper/209:10:462
2291299129,2cyclictest0-21swapper/208:00:132
2291299129,2cyclictest0-21swapper/207:10:192
2291299120,2cyclictest0-21swapper/212:10:182
2291299120,11cyclictest0-21swapper/209:55:482
2291299120,11cyclictest0-21swapper/207:40:222
2291299120,11cyclictest0-21swapper/207:20:182
2291199129,2cyclictest0-21swapper/110:06:401
2291199129,2cyclictest0-21swapper/109:20:241
2291199129,2cyclictest0-21swapper/108:20:211
2291199129,2cyclictest0-21swapper/107:10:251
2291199120,2cyclictest0-21swapper/111:12:191
2291199120,2cyclictest0-21swapper/108:50:151
2291199120,2cyclictest0-21swapper/107:35:221
2291199120,11cyclictest0-21swapper/110:45:001
2291199120,11cyclictest0-21swapper/110:25:421
2291199120,11cyclictest0-21swapper/109:50:481
2291199120,11cyclictest0-21swapper/108:35:211
2291199120,11cyclictest0-21swapper/108:30:291
2291199120,11cyclictest0-21swapper/108:30:291
2291199120,11cyclictest0-21swapper/107:45:311
2291199120,10cyclictest0-21swapper/109:05:201
2291099129,2cyclictest2999-21kworker/u8:007:44:040
2291099129,2cyclictest0-21swapper/011:11:130
2291099129,2cyclictest0-21swapper/007:20:150
2291099127,3cyclictest31118-21aten2.4_r3power10:10:140
2291099122,6cyclictest0-21swapper/011:20:260
2291099120,2cyclictest0-21swapper/012:20:460
2291099120,2cyclictest0-21swapper/011:00:550
2291099120,2cyclictest0-21swapper/010:05:540
2291099120,2cyclictest0-21swapper/008:45:150
2291099120,11cyclictest0-21swapper/012:09:150
2291099120,11cyclictest0-21swapper/011:45:570
2291099120,11cyclictest0-21swapper/011:30:110
2291099120,11cyclictest0-21swapper/010:55:260
2291099120,11cyclictest0-21swapper/009:36:160
2291099120,11cyclictest0-21swapper/007:50:290
326632110,2sleep30-21swapper/307:30:413
2291399118,2cyclictest0-21swapper/308:30:093
2291399118,2cyclictest0-21swapper/308:30:083
2291399117,2cyclictest0-21swapper/308:00:153
2291399110,10cyclictest0-21swapper/307:50:223
2291399110,10cyclictest0-21swapper/307:45:303
2291299118,2cyclictest0-21swapper/208:30:272
2291299118,2cyclictest0-21swapper/208:30:262
2291299112,6cyclictest0-21swapper/208:10:002
2291199112,5cyclictest0-21swapper/109:00:001
2291199111,8cyclictest15245-21munin-node1
2291199111,6cyclictest0-21swapper/108:20:001
2291199110,9cyclictest7032-21munin-node1
2291199110,9cyclictest0-21swapper/108:45:161
2291199110,10cyclictest0-21swapper/107:17:411
2291099118,2cyclictest0-21swapper/008:00:140
2291099111,6cyclictest0-21swapper/010:35:000
2291099110,9cyclictest0-21swapper/009:30:410
2291099110,9cyclictest0-21swapper/007:59:010
2291099110,2cyclictest0-21swapper/010:22:050
2291099110,2cyclictest0-21swapper/009:16:160
2291099110,10cyclictest0-21swapper/011:40:110
2291099110,10cyclictest0-21swapper/009:40:060
107602110,2sleep20-21swapper/207:59:012
4472100,2sleep00-21swapper/008:51:270
261492100,2sleep30-21swapper/307:15:233
2291399107,2cyclictest0-21swapper/308:50:163
2291399107,2cyclictest0-21swapper/308:20:183
2291399107,2cyclictest0-21swapper/308:05:183
2291299107,2cyclictest0-21swapper/209:00:252
2291299103,5cyclictest0-21swapper/207:30:242
2291299102,5cyclictest0-21swapper/208:20:242
2291199107,2cyclictest29179-21munin-node1
2291199107,2cyclictest0-21swapper/108:40:171
2291199101,5cyclictest0-21swapper/107:40:001
2291199100,8cyclictest27172-21munin-node1
2291199100,8cyclictest0-21swapper/108:25:251
2291199100,8cyclictest0-21swapper/108:25:241
2291199100,3cyclictest2987-21munin-node1
2291099108,1cyclictest0-21swapper/007:25:160
2291099107,2cyclictest17289-21kworker/u8:108:44:050
2291099102,6cyclictest0-21swapper/008:35:260
2291099102,6cyclictest0-21swapper/008:17:540
229139994,3cyclictest25080-21latency_hist07:15:013
229139994,3cyclictest17324-21aten2.4_r3power08:15:123
229139994,3cyclictest1334-21fschecks_time07:35:153
229139994,3cyclictest0-21swapper/307:40:143
229139990,2cyclictest0-21swapper/307:20:253
229129994,3cyclictest7561-21grep07:50:182
229129994,3cyclictest5367-21tr09:05:132
229129994,3cyclictest29973-21hddtemp_smartct08:45:152
229129994,3cyclictest27483-21tail08:40:132
229129994,3cyclictest25655-21tr08:35:152
229129994,3cyclictest1107-21aten2.4_r3power08:55:122
229129991,6cyclictest0-21swapper/208:50:252
229119990,2cyclictest0-21swapper/107:55:171
229109996,2cyclictest4907-21apt-get07:45:160
229109994,3cyclictest5129-21runrttasks08:09:370
229109994,3cyclictest26004-21tr07:15:180
229109992,5cyclictest0-21swapper/009:00:250
229109991,6cyclictest0-21swapper/009:08:040
229109991,6cyclictest0-21swapper/007:10:240
229139983,3cyclictest25389-21aten2.4-expect08:35:153
229139981,5cyclictest1196-21grep08:55:133
229109983,3cyclictest845-21aten2.4_r3power07:35:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional