You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-26 - 16:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack3slot6s.osadl.org (updated Tue Aug 26, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5045211693,16sleep00-21swapper/007:05:180
6303211393,14sleep30-21swapper/307:07:053
6399210887,16sleep20-21swapper/207:08:212
6331210880,6sleep10-21swapper/107:07:271
56972890,0sleep30-21swapper/311:03:203
155002760,0sleep10-21swapper/112:19:221
37282730,0sleep00-21swapper/011:00:320
150282720,0sleep10-21swapper/111:12:321
13432670,1sleep00-21swapper/011:31:150
94952620,0sleep00-21swapper/011:06:340
655899350,18cyclictest0-21swapper/007:50:110
655899350,18cyclictest0-21swapper/007:50:110
656199288,2cyclictest22984-21diskmemload11:35:143
655899256,18cyclictest0-21swapper/012:00:120
32222240,1sleep2301ktimersoftd/209:21:502
656199221,19cyclictest0-21swapper/311:20:123
656199220,2cyclictest0-21swapper/310:15:153
655999210,19cyclictest0-21swapper/110:15:151
6560992017,2cyclictest5612-21kworker/u8:110:04:042
6560992017,2cyclictest17313-21kworker/u8:109:00:042
655999200,2cyclictest0-21swapper/111:50:121
18672190,1sleep2301ktimersoftd/208:15:262
178162190,1sleep20-21swapper/212:20:272
6560991815,2cyclictest5287-21kworker/u8:210:39:042
6560991815,2cyclictest30667-21kworker/u8:011:56:042
656099172,13cyclictest32530-21ssh10:57:282
6560991714,2cyclictest5971-21kworker/u8:211:07:042
6560991714,2cyclictest25481-21kworker/u8:009:53:042
6560991714,2cyclictest21117-21kworker/u8:110:19:042
6560991714,2cyclictest20118-21kworker/u8:211:27:042
6560991714,2cyclictest19665-21kworker/u8:209:43:042
6560991714,2cyclictest11514-21kworker/u8:212:16:042
6560991613,2cyclictest891-21kworker/u8:008:53:042
6560991613,2cyclictest7217-21kworker/u8:108:32:042
6560991613,2cyclictest6203-21kworker/u8:007:28:042
6560991613,2cyclictest30667-21kworker/u8:011:45:042
6560991613,2cyclictest30667-21kworker/u8:011:44:042
6560991613,2cyclictest28925-21kworker/u8:208:28:012
6560991613,2cyclictest12882-21kworker/u8:310:08:042
6560991613,2cyclictest11937-21kworker/u8:011:00:042
656099161,13cyclictest0-21swapper/209:10:152
655899160,14cyclictest0-21swapper/011:28:040
186512160,1sleep00-21swapper/010:43:410
656199151,3cyclictest0-21swapper/309:59:303
6561991510,3cyclictest23677-21df09:10:173
6561991510,3cyclictest0-21swapper/310:29:173
656199150,13cyclictest0-21swapper/312:16:423
656199150,13cyclictest0-21swapper/307:30:083
656199150,13cyclictest0-21swapper/307:10:123
6560991512,2cyclictest891-21kworker/u8:008:44:042
6560991512,2cyclictest5840-21kworker/u8:112:10:042
6560991512,2cyclictest5612-21kworker/u8:109:56:042
6560991512,2cyclictest30667-21kworker/u8:012:00:042
6560991512,2cyclictest23653-21kworker/u8:210:51:042
6560991512,2cyclictest20752-21kworker/u8:207:52:012
6560991512,2cyclictest20752-21kworker/u8:207:52:012
6560991512,2cyclictest12285-21kworker/u8:010:12:042
656099151,12cyclictest22984-21diskmemload09:15:122
656099151,12cyclictest0-21swapper/210:23:082
6560991510,3cyclictest22987-21turbostat09:10:002
655999150,13cyclictest0-21swapper/110:50:001
6558991512,2cyclictest0-21swapper/009:30:180
655899150,13cyclictest22984-21diskmemload12:23:380
655899150,13cyclictest0-21swapper/011:54:380
656199149,3cyclictest12285-21kworker/u8:010:13:043
656199142,7cyclictest0-21swapper/307:50:003
6561991411,2cyclictest0-21swapper/312:20:213
6561991411,2cyclictest0-21swapper/310:46:223
656199141,11cyclictest11814-21ssh10:35:453
656199140,13cyclictest18518-21ssh11:49:423
656199140,13cyclictest0-21swapper/311:25:423
656199140,13cyclictest0-21swapper/310:32:023
656199140,12cyclictest4295-21ssh09:23:253
656199140,12cyclictest0-21swapper/312:03:153
656199140,12cyclictest0-21swapper/310:57:003
656199140,12cyclictest0-21swapper/310:43:043
656199140,12cyclictest0-21swapper/309:45:123
656099142,10cyclictest0-21swapper/207:30:112
6560991411,2cyclictest891-21kworker/u8:008:24:042
6560991411,2cyclictest6203-21kworker/u8:007:13:042
6560991411,2cyclictest26602-21kworker/u8:210:25:042
6560991411,2cyclictest26587-21kworker/u8:112:31:042
6560991411,2cyclictest20118-21kworker/u8:211:34:012
6560991411,2cyclictest11937-21kworker/u8:011:11:042
6560991410,2cyclictest10941-21kworker/u8:009:37:042
656099140,2cyclictest0-21swapper/210:45:012
656099140,12cyclictest5010-21snmpd11:54:162
656099140,12cyclictest0-21swapper/212:36:162
655999140,3cyclictest17968-21ssh12:20:341
655999140,3cyclictest0-21swapper/108:55:171
655999140,12cyclictest0-21swapper/110:40:131
6558991411,2cyclictest0-21swapper/011:21:080
6558991411,2cyclictest0-21swapper/009:40:180
655899140,12cyclictest26476-21ssh09:12:540
655899140,12cyclictest18709-21taskset11:15:400
655899140,12cyclictest18503-21ssh11:49:410
655899140,12cyclictest0-21swapper/012:15:180
655899140,12cyclictest0-21swapper/012:10:510
655899140,12cyclictest0-21swapper/011:35:560
655899140,12cyclictest0-21swapper/010:16:240
655899140,12cyclictest0-21swapper/010:13:580
655899140,12cyclictest0-21swapper/009:52:480
655899140,12cyclictest0-21swapper/008:20:120
35442140,1sleep00-21swapper/009:55:190
209202140,3sleep30-21swapper/311:50:553
6561991310,2cyclictest0-21swapper/312:30:173
6561991310,2cyclictest0-21swapper/312:25:123
6561991310,2cyclictest0-21swapper/311:56:143
6561991310,2cyclictest0-21swapper/310:01:573
6561991310,2cyclictest0-21swapper/309:27:233
6561991310,2cyclictest0-21swapper/308:15:143
6561991310,2cyclictest0-21swapper/307:35:143
656199130,2cyclictest26724-21ssh10:51:043
656199130,2cyclictest22984-21diskmemload12:12:263
656199130,2cyclictest0-21swapper/312:35:223
656199130,2cyclictest0-21swapper/311:17:543
656199130,2cyclictest0-21swapper/309:50:233
656199130,1cyclictest0-21swapper/308:25:233
656199130,12cyclictest0-21swapper/312:05:453
656199130,12cyclictest0-21swapper/311:12:043
656199130,12cyclictest0-21swapper/311:09:523
656199130,12cyclictest0-21swapper/309:43:233
656199130,12cyclictest0-21swapper/309:33:433
656199130,12cyclictest0-21swapper/308:40:123
6560991310,2cyclictest5612-21kworker/u8:109:25:042
6560991310,2cyclictest30667-21kworker/u8:011:35:042
6560991310,2cyclictest28925-21kworker/u8:208:07:042
6560991310,2cyclictest28925-21kworker/u8:208:04:042
6560991310,2cyclictest17047-21kworker/u8:110:48:042
656099130,2cyclictest3953-21ssh12:06:592
656099130,11cyclictest22984-21diskmemload12:27:182
656099130,11cyclictest0-21swapper/211:15:072
656099130,11cyclictest0-21swapper/209:45:132
655999139,2cyclictest0-21swapper/111:00:171
655999138,3cyclictest13128-21munin-node1
655999138,3cyclictest0-21swapper/108:25:141
655999132,6cyclictest0-21swapper/109:35:001
655999131,9cyclictest0-21swapper/109:25:001
655999131,3cyclictest22984-21diskmemload11:15:191
655999131,10cyclictest0-21swapper/108:05:141
655999130,2cyclictest0-21swapper/108:50:141
655999130,12cyclictest0-21swapper/110:23:541
655999130,11cyclictest0-21swapper/109:15:181
6558991310,2cyclictest22984-21diskmemload10:08:180
6558991310,2cyclictest17153-21apt-get08:55:130
6558991310,2cyclictest0-21swapper/011:58:370
6558991310,2cyclictest0-21swapper/010:46:470
6558991310,2cyclictest0-21swapper/010:24:420
6558991310,2cyclictest0-21swapper/009:45:190
6558991310,2cyclictest0-21swapper/009:27:440
6558991310,2cyclictest0-21swapper/009:24:210
655899130,2cyclictest22984-21diskmemload10:52:220
655899130,2cyclictest0-21swapper/012:25:180
655899130,2cyclictest0-21swapper/010:55:130
655899130,2cyclictest0-21swapper/010:28:380
655899130,2cyclictest0-21swapper/009:15:260
655899130,2cyclictest0-21swapper/008:40:190
655899130,11cyclictest994-21apt-get12:05:040
655899130,11cyclictest0-21swapper/011:11:420
655899130,11cyclictest0-21swapper/010:35:320
655899130,11cyclictest0-21swapper/008:00:170
236382130,2sleep10-21swapper/109:42:421
656199129,2cyclictest0-21swapper/309:36:443
656199129,2cyclictest0-21swapper/309:00:133
656199129,2cyclictest0-21swapper/307:25:253
656199128,2cyclictest14028-21ssh10:05:223
656199120,11cyclictest0-21swapper/311:40:143
656199120,11cyclictest0-21swapper/311:32:063
656199120,11cyclictest0-21swapper/310:23:553
656199120,11cyclictest0-21swapper/309:15:183
656199120,11cyclictest0-21swapper/307:15:173
656099129,2cyclictest12851-21kworker/u8:107:48:012
656099129,2cyclictest0-21swapper/211:23:192
656099129,2cyclictest0-21swapper/210:32:082
656099129,2cyclictest0-21swapper/209:30:102
656099129,2cyclictest0-21swapper/208:45:262
656099129,2cyclictest0-21swapper/207:35:162
656099129,2cyclictest0-21swapper/207:20:192
656099122,7cyclictest0-21swapper/208:59:592
656099120,2cyclictest0-21swapper/208:38:012
656099120,2cyclictest0-21swapper/207:15:212
655999128,2cyclictest0-21swapper/111:05:231
655999128,2cyclictest0-21swapper/109:00:131
655999127,3cyclictest10412-21munin-plugin-st09:30:001
655999122,7cyclictest0-21swapper/109:58:591
655999122,5cyclictest0-21swapper/111:49:591
655999122,5cyclictest0-21swapper/108:35:011
655999121,3cyclictest9061-21munin-node1
655999120,2cyclictest0-21swapper/112:32:341
655999120,2cyclictest0-21swapper/112:08:341
655999120,2cyclictest0-21swapper/111:42:031
655999120,2cyclictest0-21swapper/109:46:231
655999120,2cyclictest0-21swapper/109:38:141
655999120,11cyclictest22984-21diskmemload10:10:161
655999120,11cyclictest0-21swapper/112:27:041
655999120,11cyclictest0-21swapper/110:56:431
655999120,11cyclictest0-21swapper/110:52:031
655999120,11cyclictest0-21swapper/110:28:341
655999120,10cyclictest8438-21diskstats10:00:151
655999120,10cyclictest0-21swapper/111:55:031
655899129,2cyclictest22984-21diskmemload12:35:030
655899129,2cyclictest0-21swapper/008:35:250
655899129,2cyclictest0-21swapper/007:20:080
655899120,2cyclictest0-21swapper/012:30:030
655899120,2cyclictest0-21swapper/010:30:310
655899120,2cyclictest0-21swapper/008:45:120
655899120,2cyclictest0-21swapper/007:35:180
655899120,11cyclictest22984-21diskmemload09:37:120
655899120,11cyclictest0-21swapper/011:41:020
655899120,11cyclictest0-21swapper/010:00:320
655899120,11cyclictest0-21swapper/009:05:210
655899120,11cyclictest0-21swapper/008:05:250
655899120,11cyclictest0-21swapper/007:55:220
256272120,1sleep10-21swapper/109:11:361
656199118,2cyclictest0-21swapper/308:30:173
656199118,2cyclictest0-21swapper/307:40:183
656199118,2cyclictest0-21swapper/307:20:303
656199116,3cyclictest4554-21sshd08:23:283
656199110,2cyclictest0-21swapper/308:02:563
656199110,10cyclictest0-21swapper/308:55:133
655999118,2cyclictest0-21swapper/110:35:221
655999118,2cyclictest0-21swapper/110:05:131
655999117,2cyclictest24329-21cat07:50:251
655999117,2cyclictest24329-21cat07:50:251
655999116,3cyclictest32534-21memory11:30:181
655999112,6cyclictest22984-21diskmemload12:00:241
655999112,6cyclictest0-21swapper/107:33:591
655999112,5cyclictest0-21swapper/107:30:001
655999111,8cyclictest22984-21diskmemload11:35:261
655999111,8cyclictest21912-21apt-get11:20:121
655999111,8cyclictest19003-21munin-node1
655999111,6cyclictest0-21swapper/108:25:011
655999111,3cyclictest11088-21munin-node1
655999110,9cyclictest0-21swapper/110:30:191
655999110,9cyclictest0-21swapper/109:50:211
655999110,9cyclictest0-21swapper/107:35:251
655999110,9cyclictest0-21swapper/107:15:261
655999110,2cyclictest22984-21diskmemload12:10:271
655999110,10cyclictest0-21swapper/112:37:521
655899118,2cyclictest0-21swapper/009:00:210
656199102,6cyclictest0-21swapper/307:55:003
656199102,6cyclictest0-21swapper/307:55:003
656199101,7cyclictest0-21swapper/309:07:593
656099105,3cyclictest20788-21cron07:45:002
656099100,9cyclictest0-21swapper/207:58:282
655999107,2cyclictest21196-21apt-get09:05:161
655999107,2cyclictest0-21swapper/107:10:181
655999105,3cyclictest25031-21munin-node1
655999101,7cyclictest31239-21munin-node1
655999101,5cyclictest0-21swapper/111:25:121
655999101,3cyclictest0-21swapper/108:00:171
655999100,3cyclictest791-21munin-node1
655999100,2cyclictest0-21swapper/107:20:161
655899107,2cyclictest0-21swapper/007:10:210
655899102,5cyclictest0-21swapper/007:15:580
655899100,7cyclictest0-21swapper/008:53:590
655899100,7cyclictest0-21swapper/008:14:580
210102100,6sleep00-21swapper/007:45:120
65619996,2cyclictest0-21swapper/307:55:143
65619994,3cyclictest15207-21tail08:50:133
65619993,4cyclictest29505-21tr08:05:153
65619991,5cyclictest0-21swapper/308:45:113
65619990,8cyclictest0-21swapper/308:13:293
65609994,3cyclictest31394-21aten2.4-expect08:10:142
65599990,7cyclictest0-21swapper/107:45:131
65589996,2cyclictest4919-21apt-get08:25:110
65589990,8cyclictest6203-21kworker/u8:007:41:040
65589990,8cyclictest0-21swapper/008:30:230
65589990,2cyclictest0-21swapper/007:30:070
65619983,3cyclictest9444-21sed08:35:143
65589980,7cyclictest0-21swapper/007:25:580
65589970,6cyclictest825-21aten2.4-expect08:15:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional