You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-27 - 05:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot6s.osadl.org (updated Wed Aug 27, 2025 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
240472119108,6sleep10-21swapper/119:05:521
24291211289,18sleep00-21swapper/019:09:000
24153210683,6sleep30-21swapper/319:07:143
24107210181,15sleep20-21swapper/219:06:372
266252840,0sleep00-21swapper/020:35:010
124872810,0sleep00-21swapper/021:45:300
9302770,1sleep10-21swapper/123:45:321
159562740,0sleep30-21swapper/321:16:393
46392730,0sleep00-21swapper/000:22:270
191382730,0sleep00-21swapper/020:15:180
81862710,1sleep08188-21timerandwakeup00:25:270
123062710,1sleep30-21swapper/321:45:243
118532310,1sleep1221ktimersoftd/121:12:551
219102290,2sleep09-21ksoftirqd/021:23:090
2439999272,23cyclictest0-21swapper/220:20:122
217212270,0sleep3381ktimersoftd/320:20:253
197052270,1sleep30-21swapper/323:00:003
2439999200,18cyclictest0-21swapper/223:40:172
2439799200,18cyclictest0-21swapper/023:40:170
2439799200,18cyclictest0-21swapper/020:20:130
162472200,1sleep00-21swapper/000:34:530
2439899190,2cyclictest0-21swapper/122:55:121
24399991815,2cyclictest25654-21kworker/u8:223:52:022
24399991815,2cyclictest19778-21kworker/u8:122:31:052
2439999180,2cyclictest0-21swapper/222:20:152
2439999180,13cyclictest0-21swapper/221:41:052
2439999171,13cyclictest9217-21ssh00:26:412
2440099161,3cyclictest0-21swapper/323:45:223
24400991612,2cyclictest28745-21apt20:40:113
2440099161,13cyclictest0-21swapper/320:05:133
24399991613,2cyclictest0-21swapper/200:00:122
24397991611,3cyclictest5082-21iostat_ios21:00:170
2440099150,13cyclictest16923-21ssh22:23:593
24399991512,2cyclictest25946-21kworker/u8:221:45:042
24399991512,2cyclictest18403-21kworker/u8:223:01:042
24399991510,3cyclictest0-21swapper/200:09:162
2439999150,3cyclictest0-21swapper/221:30:142
24398991512,2cyclictest11270-21sed21:45:131
24398991511,2cyclictest0-21swapper/123:54:591
2439799151,3cyclictest32101-21ssh00:17:220
2439799151,12cyclictest19506-21ssh00:37:220
24397991510,3cyclictest7943-21users21:05:280
24397991510,3cyclictest7773-21cat19:45:270
24397991510,3cyclictest0-21swapper/000:02:430
2439799150,3cyclictest24887-21tr19:10:130
2440099141,3cyclictest28769-21ssh00:14:473
2440099141,2cyclictest18484-21ssh21:52:053
24400991411,2cyclictest0-21swapper/319:15:133
2440099140,3cyclictest0-21swapper/300:30:303
2440099140,12cyclictest0-21swapper/323:40:113
2440099140,12cyclictest0-21swapper/323:30:233
2440099140,12cyclictest0-21swapper/321:33:083
2440099140,12cyclictest0-21swapper/321:20:553
24399991411,2cyclictest0-21swapper/223:28:562
24399991410,2cyclictest8322-21diskmemload23:20:332
24399991410,2cyclictest0-21swapper/222:37:522
2439999140,3cyclictest22493-21ssh21:55:432
2439999140,12cyclictest14382-21ssh22:54:232
2439999140,12cyclictest11731-21diskmemload23:56:232
2439999140,12cyclictest0-21swapper/223:31:152
2439999140,12cyclictest0-21swapper/222:46:492
2439899140,12cyclictest1694-21kworker/u8:119:36:051
2439899140,12cyclictest0-21swapper/120:25:131
24397991411,2cyclictest0-21swapper/021:32:180
2439799140,3cyclictest6403-21cut19:45:010
2439799140,2cyclictest0-21swapper/023:50:270
2439799140,12cyclictest8322-21diskmemload22:57:480
2439799140,12cyclictest21505-21ssh22:28:280
2439799140,12cyclictest0-21swapper/023:10:210
2439799140,12cyclictest0-21swapper/022:51:470
2439799140,12cyclictest0-21swapper/021:25:240
2439799140,12cyclictest0-21swapper/000:09:160
117942140,2sleep00-21swapper/021:12:510
290622130,2sleep00-21swapper/022:03:140
277522130,2sleep00-21swapper/023:07:200
2440099138,3cyclictest22852-21seq21:56:163
2440099138,3cyclictest2019-21pzem21:35:263
2440099138,3cyclictest0-21swapper/323:11:293
2440099131,3cyclictest21137-21ssh22:27:553
24400991310,2cyclictest8322-21diskmemload22:06:583
24400991310,2cyclictest0-21swapper/323:35:173
24400991310,2cyclictest0-21swapper/322:45:143
24400991310,2cyclictest0-21swapper/322:38:073
24400991310,2cyclictest0-21swapper/322:01:283
24400991310,2cyclictest0-21swapper/321:00:183
24400991310,2cyclictest0-21swapper/300:16:143
24400991310,2cyclictest0-21swapper/300:09:183
2440099130,3cyclictest8611-21ssh00:25:473
2440099130,2cyclictest0-21swapper/323:51:083
2440099130,2cyclictest0-21swapper/323:26:383
2440099130,2cyclictest0-21swapper/323:02:583
2440099130,2cyclictest0-21swapper/322:33:483
2440099130,12cyclictest0-21swapper/323:16:343
2440099130,12cyclictest0-21swapper/322:12:003
2440099130,11cyclictest0-21swapper/323:22:203
2440099130,11cyclictest0-21swapper/321:28:113
2440099130,11cyclictest0-21swapper/300:36:073
2439999139,2cyclictest3696-21ssh00:21:052
2439999132,6cyclictest0-21swapper/221:00:012
2439999132,6cyclictest0-21swapper/219:35:002
24399991310,2cyclictest8322-21diskmemload22:10:062
24399991310,2cyclictest8322-21diskmemload21:13:382
24399991310,2cyclictest0-21swapper/223:10:242
24399991310,2cyclictest0-21swapper/222:27:022
24399991310,2cyclictest0-21swapper/222:01:292
24399991310,2cyclictest0-21swapper/221:53:062
24399991310,2cyclictest0-21swapper/221:15:102
24399991310,2cyclictest0-21swapper/221:05:212
24399991310,2cyclictest0-21swapper/200:32:202
24399991310,2cyclictest0-21swapper/200:14:212
2439999130,2cyclictest8322-21diskmemload22:40:302
2439999130,2cyclictest0-21swapper/223:15:142
2439999130,2cyclictest0-21swapper/223:06:352
2439999130,2cyclictest0-21swapper/222:55:202
2439999130,2cyclictest0-21swapper/222:18:232
2439999130,2cyclictest0-21swapper/221:35:192
2439999130,2cyclictest0-21swapper/221:21:142
2439999130,12cyclictest0-21swapper/223:36:072
2439999130,12cyclictest0-21swapper/221:25:402
2439999130,12cyclictest0-21swapper/220:15:172
2439999130,12cyclictest0-21swapper/200:35:112
2439899135,5cyclictest5906-21metacity21:23:591
24398991310,2cyclictest0-21swapper/121:15:051
2439899130,2cyclictest531-21ls22:40:191
2439899130,12cyclictest0-21swapper/123:35:061
2439899130,11cyclictest0-21swapper/122:05:231
2439899130,11cyclictest0-21swapper/100:02:221
2439799132,6cyclictest0-21swapper/022:30:010
2439799132,6cyclictest0-21swapper/022:00:000
24397991310,2cyclictest19974-21ssh23:00:050
24397991310,2cyclictest0-21swapper/023:26:170
24397991310,2cyclictest0-21swapper/022:20:210
24397991310,2cyclictest0-21swapper/022:06:540
24397991310,2cyclictest0-21swapper/021:17:380
24397991310,2cyclictest0-21swapper/000:10:150
2439799130,2cyclictest0-21swapper/023:33:350
2439799130,2cyclictest0-21swapper/022:16:380
2439799130,12cyclictest0-21swapper/023:46:200
2439799130,12cyclictest0-21swapper/020:35:270
2439799130,11cyclictest0-21swapper/022:45:310
193772130,2sleep10-21swapper/123:31:221
2440099129,2cyclictest0-21swapper/322:50:503
2440099129,2cyclictest0-21swapper/322:15:033
2440099129,2cyclictest0-21swapper/300:20:183
2440099123,6cyclictest0-21swapper/320:45:283
2440099120,2cyclictest0-21swapper/321:10:283
2440099120,11cyclictest0-21swapper/323:55:023
2440099120,11cyclictest0-21swapper/323:05:343
2440099120,11cyclictest0-21swapper/322:40:353
2440099120,11cyclictest0-21swapper/321:40:233
2440099120,11cyclictest0-21swapper/320:30:123
2439999129,2cyclictest0-21swapper/222:05:112
2439999129,2cyclictest0-21swapper/219:20:052
2439999122,7cyclictest0-21swapper/221:04:592
2439999122,7cyclictest0-21swapper/220:40:592
2439999120,11cyclictest0-21swapper/223:45:582
2439999120,11cyclictest0-21swapper/220:35:202
2439999120,11cyclictest0-21swapper/220:10:292
2439999120,11cyclictest0-21swapper/220:05:152
2439999120,11cyclictest0-21swapper/219:45:232
2439999120,11cyclictest0-21swapper/219:40:162
2439999120,11cyclictest0-21swapper/200:15:352
2439899129,2cyclictest21170-21ssh21:55:161
2439899129,2cyclictest0-21swapper/123:55:261
2439899129,2cyclictest0-21swapper/123:20:431
2439899129,2cyclictest0-21swapper/119:45:221
2439899129,2cyclictest0-21swapper/119:25:211
2439899122,7cyclictest0-21swapper/122:45:591
2439899122,7cyclictest0-21swapper/120:46:581
2439899122,7cyclictest0-21swapper/119:17:591
2439899120,2cyclictest0-21swapper/100:35:231
2439899120,11cyclictest0-21swapper/123:17:161
2439899120,11cyclictest0-21swapper/123:10:211
2439899120,11cyclictest0-21swapper/122:38:421
2439899120,11cyclictest0-21swapper/121:35:221
2439899120,11cyclictest0-21swapper/120:05:161
2439899120,11cyclictest0-21swapper/119:20:151
2439899120,11cyclictest0-21swapper/119:10:161
2439899120,10cyclictest0-21swapper/123:00:201
2439899120,10cyclictest0-21swapper/122:30:401
2439799129,2cyclictest0-21swapper/023:20:450
2439799129,2cyclictest0-21swapper/023:15:040
2439799129,2cyclictest0-21swapper/022:35:280
2439799129,2cyclictest0-21swapper/020:40:180
2439799127,3cyclictest11235-21latency19:55:190
2439799121,6cyclictest0-21swapper/019:25:010
2439799120,2cyclictest8322-21diskmemload23:35:330
2439799120,2cyclictest0-21swapper/022:12:040
2439799120,2cyclictest0-21swapper/021:43:480
2439799120,2cyclictest0-21swapper/021:36:440
2439799120,11cyclictest0-21swapper/023:55:250
2439799120,11cyclictest0-21swapper/021:50:210
2439799120,11cyclictest0-21swapper/019:20:040
2440099118,2cyclictest0-21swapper/320:50:183
2440099118,2cyclictest0-21swapper/300:02:133
2440099116,3cyclictest278-21jbd2/sda1-819:15:003
2440099116,3cyclictest20417-21sendmail20:20:013
2440099115,4cyclictest278-21jbd2/sda1-819:35:183
2440099112,7cyclictest0-21swapper/320:36:593
2440099112,6cyclictest0-21swapper/320:59:583
2440099111,7cyclictest16466-21copy20:10:023
2440099111,3cyclictest5010-21snmpd19:50:503
2440099110,4cyclictest0-21swapper/319:55:043
2440099110,1cyclictest0-21swapper/320:00:283
2440099110,1cyclictest0-21swapper/319:40:253
2439999118,2cyclictest0-21swapper/220:25:142
2439999112,7cyclictest0-21swapper/219:12:592
2439899118,2cyclictest0-21swapper/123:40:191
2439899118,2cyclictest0-21swapper/120:55:161
2439899118,2cyclictest0-21swapper/120:00:151
2439899118,2cyclictest0-21swapper/100:10:201
2439899112,7cyclictest0-21swapper/120:18:591
2439899110,9cyclictest0-21swapper/122:25:011
2439899110,9cyclictest0-21swapper/121:05:271
2439899110,9cyclictest0-21swapper/120:10:171
2439899110,9cyclictest0-21swapper/119:55:131
2439899110,2cyclictest8322-21diskmemload00:34:181
2439899110,2cyclictest0-21swapper/121:28:571
2439899110,2cyclictest0-21swapper/100:07:001
2439899110,10cyclictest0-21swapper/123:06:511
2439899110,10cyclictest0-21swapper/122:17:071
2439899110,10cyclictest0-21swapper/122:00:171
2439899110,10cyclictest0-21swapper/121:40:241
2439899110,10cyclictest0-21swapper/121:30:411
2439899110,10cyclictest0-21swapper/100:15:201
2439799112,6cyclictest0-21swapper/020:51:020
2439799112,6cyclictest0-21swapper/020:04:020
2439799112,6cyclictest0-21swapper/019:34:590
2439799110,10cyclictest0-21swapper/022:40:560
2439799110,10cyclictest0-21swapper/020:45:260
2439799110,10cyclictest0-21swapper/019:53:260
2440099105,3cyclictest28841-21aten2.4_r3power19:20:143
2440099102,6cyclictest0-21swapper/320:30:013
2440099102,6cyclictest0-21swapper/319:25:243
2440099100,4cyclictest0-21swapper/319:45:053
2439999107,2cyclictest28088-21kworker/u8:219:59:042
2439999107,2cyclictest28088-21kworker/u8:219:54:052
2439999107,2cyclictest0-21swapper/219:35:282
2439999107,2cyclictest0-21swapper/219:25:192
2439999105,3cyclictest31174-21grep20:45:182
2439999105,3cyclictest13470-21proc_pri20:00:192
2439999100,5cyclictest0-21swapper/220:34:022
2439899107,2cyclictest13924-21users23:25:301
2439899107,2cyclictest0-21swapper/122:20:151
2439899107,2cyclictest0-21swapper/100:28:031
2439899107,2cyclictest0-21swapper/100:20:171
2439899106,2cyclictest4382-21munin-node1
2439899106,2cyclictest0-21swapper/119:40:141
2439899102,6cyclictest0-21swapper/119:54:021
2439899101,7cyclictest26721-21apt-get20:35:171
2439899101,2cyclictest15862-21apt-get21:50:171
2439899100,8cyclictest0-21swapper/120:20:171
2439899100,8cyclictest0-21swapper/119:30:151
2439899100,7cyclictest0-21swapper/122:11:011
2439899100,7cyclictest0-21swapper/120:43:591
2439899100,2cyclictest8322-21diskmemload22:50:061
2439899100,2cyclictest0-21swapper/120:30:121
2439799101,6cyclictest0-21swapper/020:26:410
243999996,2cyclictest28088-21kworker/u8:219:16:052
243989990,2cyclictest0-21swapper/120:50:141
243979994,3cyclictest5129-21runrttasks20:05:150
243979994,3cyclictest5129-21runrttasks19:16:280
243979994,3cyclictest4324-21nscd21:00:010
243979994,3cyclictest16967-21df_abs20:10:160
1863290,2sleep30-21swapper/319:32:463
244009983,3cyclictest6579-21aten2.4_r3power21:05:143
243999983,3cyclictest32761-21chrt20:50:092
243979983,3cyclictest2280-21aten2.4_r3power19:35:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional