You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-04 - 02:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Fri Jul 04, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27791211589,19sleep00-21swapper/019:05:240
28204211296,11sleep30-21swapper/319:05:393
749121070,1sleep07489-21ssh23:49:530
28431210695,6sleep20-21swapper/219:08:312
28490210078,16sleep10-21swapper/119:09:181
291382780,1sleep30-21swapper/321:25:453
300522690,1sleep30-21swapper/319:10:263
20852260,1sleep10-21swapper/123:44:241
28574992015,3cyclictest10541-21kworker/u8:200:32:053
147412200,1sleep20-21swapper/223:55:562
28574991916,2cyclictest9408-21kworker/u8:121:57:043
28574991914,2cyclictest4948-21kworker/u8:100:24:043
2857499190,14cyclictest0-21swapper/323:22:043
2857299190,17cyclictest0-21swapper/123:15:121
28574991815,2cyclictest4385-21kworker/u8:222:14:053
28574991815,2cyclictest30957-21kworker/u8:019:19:053
28574991815,2cyclictest29089-21kworker/u8:021:46:043
28574991815,2cyclictest2351-21kworker/u8:119:58:053
28574991815,2cyclictest23167-21kworker/u8:123:00:053
28574991815,2cyclictest20693-21kworker/u8:120:27:053
28571991815,2cyclictest4948-21kworker/u8:100:36:010
47352170,1sleep00-21swapper/023:14:040
28574991714,2cyclictest29514-21kworker/u8:022:28:053
28574991714,2cyclictest29136-21kworker/u8:222:38:053
28574991714,2cyclictest28745-21kworker/u8:220:59:053
28574991714,2cyclictest23167-21kworker/u8:122:33:043
28574991714,2cyclictest20693-21kworker/u8:120:39:043
28574991714,2cyclictest17664-21kworker/u8:020:03:043
28574991714,2cyclictest15894-21kworker/u8:200:07:053
28574991714,2cyclictest13478-21kworker/u8:022:51:053
28574991714,2cyclictest10660-21kworker/u8:021:10:043
2857499170,2cyclictest0-21swapper/322:43:053
2857499170,12cyclictest0-21swapper/319:50:053
2857299171,13cyclictest12355-21diskmemload21:50:191
2857499161,3cyclictest29070-21grep00:10:263
28574991613,2cyclictest884-21kworker/u8:000:02:023
28574991613,2cyclictest6665-21kworker/u8:220:18:053
28574991613,2cyclictest6665-21kworker/u8:220:11:043
28574991613,2cyclictest4385-21kworker/u8:222:07:043
28574991613,2cyclictest29089-21kworker/u8:021:41:053
28574991613,2cyclictest28745-21kworker/u8:221:00:043
28574991613,2cyclictest28745-21kworker/u8:220:46:043
28574991613,2cyclictest26226-21kworker/u8:219:22:043
28574991613,2cyclictest25081-21kworker/u8:023:13:043
28574991613,2cyclictest2351-21kworker/u8:119:46:043
28574991613,2cyclictest2351-21kworker/u8:119:39:043
28574991613,2cyclictest2351-21kworker/u8:119:32:033
28574991613,2cyclictest16724-21kworker/u8:222:21:043
28574991613,2cyclictest15016-21kworker/u8:121:24:053
28574991613,2cyclictest15016-21kworker/u8:121:16:053
28574991613,2cyclictest13478-21kworker/u8:022:55:043
28574991613,2cyclictest11283-21kworker/u8:122:15:053
28574991613,2cyclictest10660-21kworker/u8:021:08:053
28574991613,2cyclictest10541-21kworker/u8:200:38:013
28574991613,2cyclictest10541-21kworker/u8:200:26:053
2857499160,2cyclictest0-21swapper/323:08:053
2857499160,12cyclictest0-21swapper/319:26:053
2857499160,11cyclictest0-21swapper/323:31:053
2857499160,11cyclictest0-21swapper/321:53:053
28573991613,2cyclictest15573-21kworker/u8:223:45:042
28573991613,2cyclictest15573-21kworker/u8:223:28:052
28573991612,2cyclictest7558-21kworker/u8:123:54:052
2857299161,13cyclictest0-21swapper/121:21:571
2857299160,3cyclictest0-21swapper/120:00:001
2857299160,2cyclictest0-21swapper/100:10:141
309282150,0sleep10-21swapper/123:07:051
28574991512,2cyclictest6665-21kworker/u8:219:41:053
28574991512,2cyclictest3819-21kworker/u8:120:53:053
28574991512,2cyclictest15573-21kworker/u8:223:27:053
2857399151,3cyclictest1907-21ssh21:31:192
28573991512,2cyclictest29514-21kworker/u8:022:03:052
28573991512,2cyclictest2351-21kworker/u8:119:28:052
28573991512,2cyclictest20693-21kworker/u8:120:34:022
28573991512,2cyclictest15894-21kworker/u8:200:00:042
28573991512,2cyclictest15016-21kworker/u8:121:13:052
2857399151,12cyclictest6979-21ssh21:36:192
2857399151,12cyclictest6979-21ssh21:36:192
2857399150,2cyclictest0-21swapper/222:40:412
28572991512,2cyclictest0-21swapper/123:59:401
2857299151,12cyclictest0-21swapper/123:04:431
2857299150,13cyclictest0-21swapper/121:56:591
2857199150,13cyclictest0-21swapper/019:35:180
325382140,1sleep10-21swapper/122:35:251
28574991411,2cyclictest15573-21kworker/u8:223:40:053
2857499140,9cyclictest0-21swapper/323:51:053
2857499140,2cyclictest12355-21diskmemload21:40:013
2857499140,2cyclictest12355-21diskmemload21:40:003
2857499140,12cyclictest12355-21diskmemload21:33:263
2857499140,12cyclictest0-21swapper/323:55:093
2857499140,12cyclictest0-21swapper/323:36:363
2857399141,3cyclictest8590-21sh22:45:012
28573991411,2cyclictest29089-21kworker/u8:021:51:052
28573991411,2cyclictest2351-21kworker/u8:119:48:052
28573991411,2cyclictest15894-21kworker/u8:200:12:042
28573991411,2cyclictest15016-21kworker/u8:121:18:022
28573991411,2cyclictest0-21swapper/222:27:012
28573991411,2cyclictest0-21swapper/200:36:302
2857399140,2cyclictest0-21swapper/223:32:192
2857399140,2cyclictest0-21swapper/222:30:312
2857399140,2cyclictest0-21swapper/200:21:312
2857399140,13cyclictest0-21swapper/222:06:312
2857399140,13cyclictest0-21swapper/200:31:012
2857399140,12cyclictest6709-21ssh23:15:182
2857399140,12cyclictest106522sleep223:20:032
2857399140,12cyclictest0-21swapper/222:21:102
2857399140,12cyclictest0-21swapper/221:40:142
2857399140,12cyclictest0-21swapper/219:20:192
2857299148,4cyclictest0-21swapper/100:04:061
2857299142,6cyclictest12355-21diskmemload21:20:001
28572991411,2cyclictest0-21swapper/122:52:591
28572991411,2cyclictest0-21swapper/121:41:101
28572991411,2cyclictest0-21swapper/121:26:591
28572991411,2cyclictest0-21swapper/100:19:401
28572991410,2cyclictest0-21swapper/122:30:231
28572991410,2cyclictest0-21swapper/121:31:561
2857299140,2cyclictest0-21swapper/122:28:401
2857299140,13cyclictest0-21swapper/123:13:591
2857299140,13cyclictest0-21swapper/100:37:491
2857299140,12cyclictest23013-21sh23:31:471
2857299140,12cyclictest0-21swapper/123:49:181
2857299140,12cyclictest0-21swapper/123:23:431
2857299140,12cyclictest0-21swapper/122:17:061
2857299140,12cyclictest0-21swapper/121:48:421
2857299140,12cyclictest0-21swapper/100:25:261
2857299140,12cyclictest0-21swapper/100:20:481
28571991410,2cyclictest0-21swapper/000:25:060
2857199140,12cyclictest9120-21ssh00:24:030
2857499133,6cyclictest0-21swapper/322:45:253
28574991310,2cyclictest29514-21kworker/u8:022:01:053
28574991310,2cyclictest17664-21kworker/u8:020:09:053
2857499130,2cyclictest0-21swapper/320:31:053
2857499130,11cyclictest0-21swapper/300:15:543
2857399139,3cyclictest18384-21taskset22:55:002
28573991310,2cyclictest9408-21kworker/u8:121:55:042
28573991310,2cyclictest642-21ssh23:10:032
28573991310,2cyclictest29514-21kworker/u8:022:18:052
28573991310,2cyclictest28908-21ssh21:25:252
28573991310,2cyclictest12355-21diskmemload21:20:532
28573991310,2cyclictest0-21swapper/223:42:082
28573991310,2cyclictest0-21swapper/223:05:442
28573991310,2cyclictest0-21swapper/222:39:472
28573991310,2cyclictest0-21swapper/222:14:452
28573991310,2cyclictest0-21swapper/221:46:502
2857399130,2cyclictest2710-21ssh00:17:042
2857399130,2cyclictest0-21swapper/222:53:242
2857399130,2cyclictest0-21swapper/200:26:242
2857299138,3cyclictest0-21swapper/122:04:471
28572991310,2cyclictest0-21swapper/123:55:001
28572991310,2cyclictest0-21swapper/123:28:501
28572991310,2cyclictest0-21swapper/122:47:211
28572991310,2cyclictest0-21swapper/122:42:021
28572991310,2cyclictest0-21swapper/122:20:491
28572991310,2cyclictest0-21swapper/122:05:131
28572991310,2cyclictest0-21swapper/120:45:201
28572991310,2cyclictest0-21swapper/119:20:171
28572991310,2cyclictest0-21swapper/100:30:261
2857299130,3cyclictest27743-21cut20:25:201
2857299130,2cyclictest0-21swapper/122:57:151
2857299130,2cyclictest0-21swapper/119:30:261
2857299130,2cyclictest0-21swapper/100:05:161
2857299130,1cyclictest0-21swapper/120:00:191
2857299130,12cyclictest0-21swapper/123:35:161
2857299130,12cyclictest0-21swapper/122:10:251
2857299130,12cyclictest0-21swapper/119:15:241
2857299130,11cyclictest0-21swapper/119:40:231
2857199139,2cyclictest0-21swapper/022:39:320
28571991310,2cyclictest9086-21ssh21:39:330
28571991310,2cyclictest9086-21ssh21:39:320
28571991310,2cyclictest24416-21apt-get21:55:130
28571991310,2cyclictest0-21swapper/023:58:430
28571991310,2cyclictest0-21swapper/022:30:500
28571991310,2cyclictest0-21swapper/022:20:110
28571991310,2cyclictest0-21swapper/021:16:120
2857199130,2cyclictest0-21swapper/022:45:060
2857199130,11cyclictest0-21swapper/000:06:210
246042130,2sleep00-21swapper/021:21:190
214702130,4sleep02857199cyclictest22:25:100
2857499129,2cyclictest0-21swapper/323:15:003
2857499120,2cyclictest0-21swapper/323:47:013
2857499120,2cyclictest0-21swapper/320:20:053
2857399129,2cyclictest6665-21kworker/u8:219:44:052
2857399129,2cyclictest417-21kworker/u8:020:42:022
2857399129,2cyclictest3819-21kworker/u8:121:02:042
2857399129,2cyclictest26226-21kworker/u8:219:13:042
2857399129,2cyclictest0-21swapper/220:10:152
2857399129,2cyclictest0-21swapper/219:15:162
2857399129,2cyclictest0-21swapper/200:05:122
2857399120,2cyclictest0-21swapper/223:37:032
2857399120,2cyclictest0-21swapper/220:55:152
2857399120,2cyclictest0-21swapper/220:00:172
2857399120,2cyclictest0-21swapper/219:55:122
2857399120,2cyclictest0-21swapper/219:50:162
2857399120,11cyclictest0-21swapper/223:01:182
2857399120,11cyclictest0-21swapper/220:45:202
2857399120,11cyclictest0-21swapper/219:32:192
2857399120,10cyclictest5010-21snmpd20:51:382
2857299129,2cyclictest0-21swapper/121:35:191
2857299129,2cyclictest0-21swapper/121:35:181
2857299120,2cyclictest0-21swapper/119:50:181
2857299120,11cyclictest0-21swapper/121:10:071
2857299120,11cyclictest0-21swapper/120:40:141
2857299120,11cyclictest0-21swapper/119:10:201
2857299120,10cyclictest0-21swapper/120:05:211
2857199129,2cyclictest22360-21ssh21:52:210
2857199129,2cyclictest0-21swapper/023:05:300
2857199129,2cyclictest0-21swapper/022:41:090
2857199129,2cyclictest0-21swapper/021:10:360
2857199129,2cyclictest0-21swapper/019:54:170
2857199129,2cyclictest0-21swapper/019:15:170
2857199129,2cyclictest0-21swapper/000:15:520
2857199120,2cyclictest12355-21diskmemload21:47:520
2857199120,2cyclictest0-21swapper/023:29:320
2857199120,2cyclictest0-21swapper/022:15:140
2857199120,2cyclictest0-21swapper/000:03:010
2857199120,11cyclictest28745-21kworker/u8:220:47:050
2857199120,11cyclictest23117-21ssh23:31:570
2857199120,11cyclictest12355-21diskmemload21:40:480
2857199120,11cyclictest0-21swapper/023:50:140
2857199120,11cyclictest0-21swapper/023:43:420
2857199120,11cyclictest0-21swapper/023:37:110
2857199120,11cyclictest0-21swapper/023:00:080
2857199120,11cyclictest0-21swapper/022:55:150
2857199120,11cyclictest0-21swapper/022:51:490
2857199120,11cyclictest0-21swapper/022:14:160
2857199120,11cyclictest0-21swapper/022:08:230
2857199120,11cyclictest0-21swapper/021:30:390
2857199120,11cyclictest0-21swapper/021:25:220
2857199120,11cyclictest0-21swapper/020:40:270
2857199120,11cyclictest0-21swapper/020:20:180
2857199120,11cyclictest0-21swapper/000:13:190
2857199120,10cyclictest0-21swapper/020:05:140
185432120,2sleep00-21swapper/000:33:270
2857499118,2cyclictest20693-21kworker/u8:120:40:023
2857399118,2cyclictest6665-21kworker/u8:220:16:052
2857399118,2cyclictest17664-21kworker/u8:020:06:022
2857299118,2cyclictest0-21swapper/121:00:181
2857299118,2cyclictest0-21swapper/119:35:231
2857299117,2cyclictest0-21swapper/119:25:161
2857299111,8cyclictest22668-21apt-get20:15:141
2857299111,3cyclictest20655-21munin-node1
2857299111,2cyclictest0-21swapper/119:45:161
2857299110,9cyclictest0-21swapper/120:55:191
2857299110,9cyclictest0-21swapper/120:50:181
2857299110,9cyclictest0-21swapper/120:20:151
2857199118,2cyclictest31154-21sh22:00:490
2857199118,2cyclictest0-21swapper/021:00:230
2857199118,2cyclictest0-21swapper/020:35:300
2857199110,2cyclictest0-21swapper/020:28:170
2857199110,1cyclictest0-21swapper/023:21:120
2857199110,10cyclictest0-21swapper/023:15:550
2857399107,2cyclictest20693-21kworker/u8:120:23:052
2857399105,3cyclictest11957-21unixbench_singl21:05:282
2857299102,7cyclictest0-21swapper/120:32:591
2857299100,8cyclictest0-21swapper/121:05:191
2857299100,8cyclictest0-21swapper/120:35:141
2857199107,2cyclictest0-21swapper/021:05:210
2857199107,2cyclictest0-21swapper/020:50:220
285739995,3cyclictest6603-21aten2.4_r3power19:35:122
285739994,3cyclictest31060-21grep20:35:142
285739994,3cyclictest26892-21stty20:25:132
285719996,2cyclictest0-21swapper/020:35:000
285719990,2cyclictest0-21swapper/019:45:000
285719985,2cyclictest10728-21aten2.4-expect19:45:160
285719984,3cyclictest30592-21sort19:15:000
285719983,3cyclictest22474-21cat20:15:000
285719981,6cyclictest358-21munin-node0
285719980,7cyclictest0-21swapper/019:55:030
285719980,2cyclictest0-21swapper/020:19:320
285719974,2cyclictest3143-21sed19:25:190
285719974,2cyclictest0-21swapper/020:00:160
285719973,3cyclictest0-21swapper/019:30:150
285719972,3cyclictest6803-21tr20:55:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional