You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 09:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot6s.osadl.org (updated Sat Nov 22, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2877212094,19sleep10-21swapper/119:05:161
3476211792,18sleep00-21swapper/019:05:550
3471211794,16sleep20-21swapper/219:05:532
3578211497,11sleep30-21swapper/319:07:143
210502870,0sleep30-21swapper/323:26:143
267952800,1sleep00-21swapper/023:32:570
222192700,0sleep30-21swapper/321:10:373
289922280,0sleep00-21swapper/020:10:120
382699210,19cyclictest0-21swapper/223:20:112
95302200,1sleep00-21swapper/023:15:040
83132200,1sleep30-21swapper/322:05:133
3826992017,2cyclictest0-21swapper/219:45:112
382899170,15cyclictest20193-21diskmemload21:30:293
382699170,4cyclictest0-21swapper/223:29:552
382499171,14cyclictest0-21swapper/022:17:130
382699161,13cyclictest13465-21users21:35:272
382599163,11cyclictest0-21swapper/122:39:141
382599163,11cyclictest0-21swapper/122:39:131
382499161,2cyclictest0-21swapper/021:40:170
382499161,13cyclictest0-21swapper/022:10:140
3828991512,2cyclictest20073-21kworker/u8:022:25:043
3828991512,2cyclictest0-21swapper/322:50:193
382899151,12cyclictest12749-21memory21:35:183
382899150,2cyclictest0-21swapper/300:28:483
382899150,13cyclictest0-21swapper/300:14:073
3826991512,2cyclictest0-21swapper/223:13:032
3826991512,2cyclictest0-21swapper/220:55:142
382699151,12cyclictest32329-21copy20:20:012
3826991510,3cyclictest30024-21ssh23:01:332
382699150,13cyclictest20193-21diskmemload22:00:332
382699150,13cyclictest0-21swapper/221:10:432
382699150,13cyclictest0-21swapper/220:50:102
3825991512,2cyclictest0-21swapper/121:36:091
382599150,13cyclictest4324-21nscd22:11:351
382599150,13cyclictest0-21swapper/121:15:101
382499151,12cyclictest12346-21apt-get20:50:100
382499151,12cyclictest0-21swapper/000:35:420
382499150,13cyclictest0-21swapper/021:21:040
382499150,13cyclictest0-21swapper/000:30:140
3828991411,2cyclictest0-21swapper/322:55:183
3828991411,2cyclictest0-21swapper/300:30:173
382899141,11cyclictest6019-21apt-get19:15:053
382899140,2cyclictest0-21swapper/323:33:203
382899140,13cyclictest0-21swapper/323:53:183
382899140,12cyclictest0-21swapper/323:21:283
382699149,3cyclictest26413-21ls20:05:002
382699141,9cyclictest28083-21munin-run23:35:002
3826991411,2cyclictest0-21swapper/222:26:532
3826991411,2cyclictest0-21swapper/221:55:042
3826991411,2cyclictest0-21swapper/200:35:322
3826991411,2cyclictest0-21swapper/200:05:432
3826991410,2cyclictest11361-21pzem_voltage20:45:222
382699140,2cyclictest20193-21diskmemload22:33:142
382699140,2cyclictest0-21swapper/223:40:282
382699140,12cyclictest20193-21diskmemload22:45:032
382699140,12cyclictest0-21swapper/222:16:312
382699140,12cyclictest0-21swapper/221:31:542
382699140,12cyclictest0-21swapper/200:20:102
3825991411,2cyclictest20193-21diskmemload00:06:021
3825991411,2cyclictest0-21swapper/100:31:121
382599140,2cyclictest0-21swapper/122:57:311
382599140,2cyclictest0-21swapper/122:50:321
382599140,2cyclictest0-21swapper/122:21:121
382599140,13cyclictest0-21swapper/123:05:021
382599140,13cyclictest0-21swapper/122:41:191
382599140,13cyclictest0-21swapper/121:58:021
382599140,12cyclictest20193-21diskmemload21:45:321
382599140,12cyclictest20193-21diskmemload21:40:191
382599140,12cyclictest20193-21diskmemload21:28:361
382599140,12cyclictest0-21swapper/123:19:001
382499149,3cyclictest0-21swapper/022:50:260
382499142,7cyclictest10875-21ssh00:25:000
382499142,7cyclictest0-21swapper/019:55:010
382499140,2cyclictest20193-21diskmemload21:10:240
382499140,12cyclictest20193-21diskmemload22:00:040
382499140,12cyclictest0-21swapper/021:46:220
382499140,12cyclictest0-21swapper/000:15:140
137112141,3sleep113949-21df22:45:161
382899131,2cyclictest20193-21diskmemload21:15:303
3828991310,2cyclictest6394-21kworker/u8:219:39:053
3828991310,2cyclictest3630-21kworker/u8:023:11:043
3828991310,2cyclictest3630-21kworker/u8:023:09:043
3828991310,2cyclictest25032-21kworker/u8:122:31:053
3828991310,2cyclictest0-21swapper/321:20:293
3828991310,2cyclictest0-21swapper/320:20:013
3828991310,2cyclictest0-21swapper/300:06:493
382899130,2cyclictest20193-21diskmemload22:03:463
382899130,2cyclictest0-21swapper/300:22:473
382899130,1cyclictest0-21swapper/323:49:073
382899130,12cyclictest0-21swapper/323:41:083
382899130,12cyclictest0-21swapper/321:51:283
382899130,12cyclictest0-21swapper/321:47:483
382899130,12cyclictest0-21swapper/300:02:473
382899130,11cyclictest4074-21ssh21:25:473
382899130,11cyclictest20193-21diskmemload23:59:493
382899130,11cyclictest0-21swapper/322:15:113
382899130,11cyclictest0-21swapper/321:56:113
382699131,2cyclictest2182-21ssh21:25:022
3826991310,2cyclictest20193-21diskmemload21:16:382
3826991310,2cyclictest0-21swapper/221:45:232
382699130,2cyclictest20193-21diskmemload22:13:132
382699130,2cyclictest20193-21diskmemload21:20:222
382699130,2cyclictest1644-21munin-node2
382699130,12cyclictest0-21swapper/222:56:032
382699130,11cyclictest28402-21ssh21:51:352
382699130,11cyclictest0-21swapper/223:35:162
382699130,11cyclictest0-21swapper/222:37:472
382699130,11cyclictest0-21swapper/222:37:472
382699130,11cyclictest0-21swapper/222:21:182
382699130,11cyclictest0-21swapper/200:25:422
3825991310,2cyclictest0-21swapper/123:40:041
3825991310,2cyclictest0-21swapper/123:30:471
3825991310,2cyclictest0-21swapper/123:20:101
3825991310,2cyclictest0-21swapper/123:10:111
3825991310,2cyclictest0-21swapper/122:30:401
3825991310,2cyclictest0-21swapper/122:05:191
3825991310,2cyclictest0-21swapper/122:01:481
3825991310,2cyclictest0-21swapper/121:52:171
3825991310,2cyclictest0-21swapper/121:22:351
3825991310,2cyclictest0-21swapper/120:40:011
3825991310,2cyclictest0-21swapper/120:15:211
3825991310,2cyclictest0-21swapper/100:00:041
382599130,2cyclictest5134-21apt-get23:45:051
382599130,2cyclictest20193-21diskmemload23:56:281
382599130,2cyclictest0-21swapper/100:25:061
382599130,12cyclictest20193-21diskmemload23:35:101
382599130,12cyclictest20193-21diskmemload22:27:091
382599130,12cyclictest0-21swapper/123:50:301
382599130,12cyclictest0-21swapper/123:27:171
382599130,12cyclictest0-21swapper/123:01:161
382599130,12cyclictest0-21swapper/121:30:411
382599130,12cyclictest0-21swapper/100:23:421
382599130,12cyclictest0-21swapper/100:15:391
382599130,12cyclictest0-21swapper/100:10:351
382599130,11cyclictest0-21swapper/121:10:211
382599130,11cyclictest0-21swapper/120:35:211
382499138,3cyclictest17061-21iostat_ios21:00:160
382499132,6cyclictest0-21swapper/023:00:000
3824991310,2cyclictest27352-21ssh22:59:520
3824991310,2cyclictest0-21swapper/023:57:530
3824991310,2cyclictest0-21swapper/023:51:420
3824991310,2cyclictest0-21swapper/021:18:130
3824991310,2cyclictest0-21swapper/019:35:130
382499130,7cyclictest0-21swapper/019:25:000
382499130,2cyclictest0-21swapper/023:08:430
382499130,2cyclictest0-21swapper/022:44:530
382499130,2cyclictest0-21swapper/021:37:130
382499130,2cyclictest0-21swapper/000:20:180
382499130,12cyclictest0-21swapper/000:12:010
382499130,12cyclictest0-21swapper/000:06:240
382499130,11cyclictest0-21swapper/023:27:530
308972130,1sleep00-21swapper/023:36:580
221932130,2sleep00-21swapper/022:20:100
221812130,2sleep30-21swapper/322:20:093
182302130,1sleep20-21swapper/221:40:272
143532130,2sleep30-21swapper/320:55:123
382899129,2cyclictest9486-21kworker/u8:223:15:053
382899129,2cyclictest28168-21kworker/u8:120:54:043
382899129,2cyclictest17643-21kworker/u8:200:17:053
382899129,2cyclictest14457-21kworker/u8:122:14:053
382899129,2cyclictest0-21swapper/323:03:183
382899129,2cyclictest0-21swapper/322:42:293
382899129,2cyclictest0-21swapper/319:20:193
382899120,11cyclictest0-21swapper/323:38:203
382899120,11cyclictest0-21swapper/320:40:103
382699129,2cyclictest1159-21ssh23:05:242
382699129,2cyclictest0-21swapper/223:50:032
382699129,2cyclictest0-21swapper/222:40:142
382699129,2cyclictest0-21swapper/219:30:182
382699129,2cyclictest0-21swapper/200:10:172
382699129,2cyclictest0-21swapper/200:00:122
382699122,7cyclictest0-21swapper/220:24:582
382699120,2cyclictest7466-21ssh23:45:562
382699120,2cyclictest0-21swapper/223:55:212
382699120,2cyclictest0-21swapper/222:51:552
382699120,2cyclictest0-21swapper/219:20:192
382699120,2cyclictest0-21swapper/219:15:242
382699120,2cyclictest0-21swapper/200:30:072
382699120,11cyclictest0-21swapper/223:15:482
382699120,11cyclictest0-21swapper/222:05:222
382699120,11cyclictest0-21swapper/220:05:272
382699120,11cyclictest0-21swapper/219:35:232
382599129,2cyclictest0-21swapper/121:05:201
382599129,2cyclictest0-21swapper/120:00:271
382599129,2cyclictest0-21swapper/119:20:201
382599120,2cyclictest0-21swapper/120:30:141
382599120,2cyclictest0-21swapper/119:15:161
382599120,2cyclictest0-21swapper/100:35:281
382599120,11cyclictest0-21swapper/122:15:201
382599120,11cyclictest0-21swapper/120:50:241
382599120,11cyclictest0-21swapper/120:25:221
382599120,10cyclictest0-21swapper/120:05:221
382499129,2cyclictest0-21swapper/023:20:110
382499129,2cyclictest0-21swapper/023:10:200
382499129,2cyclictest0-21swapper/022:30:110
382499129,2cyclictest0-21swapper/022:26:130
382499129,2cyclictest0-21swapper/021:30:270
382499129,2cyclictest0-21swapper/021:26:230
382499129,2cyclictest0-21swapper/021:05:110
382499122,7cyclictest0-21swapper/020:46:020
382499122,6cyclictest0-21swapper/022:35:250
382499122,6cyclictest0-21swapper/022:35:240
382499121,9cyclictest0-21swapper/020:25:010
382499120,2cyclictest0-21swapper/022:45:160
382499120,2cyclictest0-21swapper/021:50:330
382499120,11cyclictest0-21swapper/023:45:500
382499120,11cyclictest0-21swapper/023:43:140
382499120,11cyclictest0-21swapper/022:05:230
382499120,11cyclictest0-21swapper/019:15:250
382499120,11cyclictest0-21swapper/019:10:160
319692120,2sleep00-21swapper/021:55:250
196152120,2sleep320193-21diskmemload21:42:423
382899118,2cyclictest6394-21kworker/u8:219:46:053
382899111,9cyclictest20193-21diskmemload22:35:173
382899111,9cyclictest20193-21diskmemload22:35:173
382699118,2cyclictest0-21swapper/220:10:182
382699112,7cyclictest0-21swapper/220:28:582
382699112,6cyclictest0-21swapper/219:56:592
382699112,6cyclictest0-21swapper/219:56:592
382599118,2cyclictest16395-21apt-get19:40:151
382599118,2cyclictest0-21swapper/119:55:141
382599118,2cyclictest0-21swapper/119:55:131
382599118,2cyclictest0-21swapper/119:35:171
382599110,9cyclictest20407-21munin-node1
382599110,9cyclictest0-21swapper/119:45:241
382499118,2cyclictest0-21swapper/020:15:130
382499118,2cyclictest0-21swapper/020:06:130
382499118,2cyclictest0-21swapper/019:40:210
382499118,2cyclictest0-21swapper/000:02:130
382899107,2cyclictest32128-21kworker/u8:019:25:043
382899107,2cyclictest28168-21kworker/u8:120:45:043
382899107,2cyclictest28168-21kworker/u8:120:35:043
382899107,2cyclictest20445-21kworker/u8:119:56:023
382899107,2cyclictest20445-21kworker/u8:119:56:013
382899107,2cyclictest16383-21kworker/u8:020:14:043
382899107,2cyclictest0-21swapper/321:05:193
382899107,2cyclictest0-21swapper/320:30:163
382899107,2cyclictest0-21swapper/319:30:143
382899100,2cyclictest0-21swapper/322:48:313
382899100,2cyclictest0-21swapper/320:08:093
382699107,2cyclictest0-21swapper/220:40:192
382699101,6cyclictest0-21swapper/219:51:092
382699101,5cyclictest0-21swapper/220:35:012
382599100,8cyclictest0-21swapper/119:25:141
382599100,2cyclictest32543-21munin-node1
382499107,2cyclictest0-21swapper/020:40:170
382499107,2cyclictest0-21swapper/020:35:180
382499107,2cyclictest0-21swapper/019:30:190
382499105,3cyclictest2620-21fschecks_time20:25:130
382499102,6cyclictest0-21swapper/019:46:080
382499101,7cyclictest27356-1kworker/0:0H20:00:010
382499101,7cyclictest27356-1kworker/0:0H20:00:000
143622100,7sleep014469-21aten2.4_r3power20:55:130
38289996,2cyclictest20445-21kworker/u8:120:00:043
38289996,2cyclictest16383-21kworker/u8:020:21:043
38289994,3cyclictest20703-21cpuspeed_turbos19:50:133
38289994,3cyclictest20177-21latency_hist00:35:003
38289994,3cyclictest16896-21fschecks_time21:00:143
38289994,3cyclictest16762-21df_abs19:40:173
38289993,4cyclictest2408-21grep20:25:123
38289990,8cyclictest0-21swapper/319:10:093
38269996,2cyclictest0-21swapper/220:35:142
38269994,3cyclictest11676-21grep19:25:272
38269991,6cyclictest0-21swapper/221:05:242
38269991,4cyclictest0-21swapper/219:15:012
38269990,4cyclictest0-21swapper/219:45:002
38259996,2cyclictest0-21swapper/119:14:181
38259990,8cyclictest0-21swapper/119:30:121
38259990,3cyclictest10332-21munin-node1
38259990,2cyclictest0-21swapper/121:00:131
38259990,2cyclictest0-21swapper/120:10:141
38249994,3cyclictest25257-21missed_timers20:00:180
38249994,3cyclictest10351-21aten2.4_r3power19:25:150
14231290,1sleep10-21swapper/120:55:021
38269985,2cyclictest16983-21munin-node2
38249983,3cyclictest4899-21sed20:30:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional