You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-02 - 03:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack3slot6s.osadl.org (updated Wed Jul 02, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
167612121110,6sleep10-21swapper/119:07:241
167402121110,6sleep00-21swapper/019:07:080
15189211694,15sleep30-21swapper/319:05:143
16658211395,13sleep20-21swapper/219:06:052
167142730,1sleep0101ktimersoftd/023:04:230
116802680,0sleep00-21swapper/022:59:130
1699399370,19cyclictest0-21swapper/320:25:133
1699199360,18cyclictest0-21swapper/120:25:141
27662260,1sleep2301ktimersoftd/219:55:012
251562250,1sleep2301ktimersoftd/200:18:032
96862240,1sleep00-21swapper/021:18:370
126862230,0sleep00-21swapper/021:20:370
1699399221,19cyclictest15327-21ssh22:30:143
1699099220,2cyclictest0-21swapper/019:55:130
12802220,1sleep30-21swapper/322:48:383
1699399211,18cyclictest616-21diskmemload22:40:173
1699199210,19cyclictest0-21swapper/123:30:161
1699099200,2cyclictest0-21swapper/023:40:140
16991991813,3cyclictest332250irq/24-eth121:12:061
1699399170,2cyclictest0-21swapper/323:20:143
1699199170,2cyclictest616-21diskmemload21:40:161
1699199170,15cyclictest0-21swapper/100:30:151
295212160,0sleep10-21swapper/121:00:141
16991991611,3cyclictest0-21swapper/120:30:181
1699199160,5cyclictest0-21swapper/121:52:271
1699199160,3cyclictest0-21swapper/122:15:461
1699199160,14cyclictest0-21swapper/122:05:051
25692150,0sleep20-21swapper/222:50:142
235122150,3sleep023598-21munin-node0
208732150,3sleep20-21swapper/221:30:142
1699399151,12cyclictest6153-21hddtemp_smartct22:20:183
1699399150,3cyclictest0-21swapper/323:00:213
1699399150,14cyclictest0-21swapper/321:15:183
1699399150,13cyclictest12260-21ssh23:31:493
1699399150,13cyclictest0-21swapper/323:53:203
1699399150,13cyclictest0-21swapper/323:45:293
1699299151,3cyclictest0-21swapper/222:40:172
16992991512,2cyclictest7206-21kworker/u8:223:04:062
16992991512,2cyclictest26731-21kworker/u8:122:24:062
16992991512,2cyclictest24871-21kworker/u8:222:05:042
16992991512,2cyclictest10637-21kworker/u8:121:54:012
16991991511,2cyclictest0-21swapper/121:20:051
16991991510,3cyclictest26643-21ssh23:14:331
1699199150,3cyclictest0-21swapper/100:10:181
1699199150,13cyclictest0-21swapper/123:53:151
1699199150,13cyclictest0-21swapper/122:34:051
16990991512,2cyclictest0-21swapper/000:21:560
1699099150,13cyclictest616-21diskmemload22:08:050
1699399141,2cyclictest0-21swapper/319:35:133
16993991411,2cyclictest9309-21ssh22:55:393
16993991411,2cyclictest616-21diskmemload22:14:493
16993991411,2cyclictest0-21swapper/323:42:493
16993991411,2cyclictest0-21swapper/323:15:393
16993991411,2cyclictest0-21swapper/323:15:393
16993991411,2cyclictest0-21swapper/323:08:583
16993991410,3cyclictest0-21swapper/321:31:253
16993991410,2cyclictest1991-21apt-get23:55:173
1699399140,3cyclictest0-21swapper/321:47:003
1699399140,2cyclictest616-21diskmemload00:07:093
1699399140,2cyclictest0-21swapper/322:36:293
1699399140,13cyclictest0-21swapper/322:29:293
1699399140,13cyclictest0-21swapper/321:36:293
1699399140,13cyclictest0-21swapper/300:33:283
1699399140,12cyclictest616-21diskmemload23:25:283
1699399140,12cyclictest5396-21ssh22:52:193
1699399140,12cyclictest0-21swapper/322:02:363
1699399140,12cyclictest0-21swapper/300:29:593
1699299149,3cyclictest20235-21cut23:40:172
1699299149,3cyclictest0-21swapper/223:21:102
16992991411,2cyclictest7867-21ssh21:49:552
16992991411,2cyclictest24871-21kworker/u8:221:25:052
16992991410,3cyclictest27169-21ssh23:15:022
16992991410,3cyclictest27169-21ssh23:15:012
16992991410,2cyclictest16342-21sh21:57:542
1699299140,2cyclictest0-21swapper/223:50:052
1699299140,2cyclictest0-21swapper/221:10:152
1699299140,2cyclictest0-21swapper/200:00:282
1699199149,3cyclictest0-21swapper/123:41:481
16991991411,2cyclictest10447-21apt-get22:25:161
16991991411,2cyclictest0-21swapper/122:48:261
16991991411,2cyclictest0-21swapper/121:45:161
16991991411,2cyclictest0-21swapper/119:40:161
16991991411,2cyclictest0-21swapper/100:04:061
1699199140,13cyclictest0-21swapper/123:17:561
1699199140,13cyclictest0-21swapper/123:17:551
1699199140,12cyclictest31330-21kworker/u8:220:11:051
1699199140,12cyclictest30720-21kworker/u8:000:27:051
1699199140,12cyclictest0-21swapper/123:29:271
1699199140,12cyclictest0-21swapper/121:56:401
1699199140,12cyclictest0-21swapper/100:05:211
16990991411,2cyclictest0-21swapper/023:50:070
16990991411,2cyclictest0-21swapper/022:35:560
16990991411,2cyclictest0-21swapper/021:10:570
16990991411,2cyclictest0-21swapper/000:26:260
1699099140,2cyclictest0-21swapper/021:50:160
1699099140,2cyclictest0-21swapper/000:08:450
1699099140,13cyclictest0-21swapper/022:45:250
1699099140,13cyclictest0-21swapper/021:42:060
1699099140,13cyclictest0-21swapper/000:10:360
1699099140,12cyclictest0-21swapper/022:40:230
87362130,3sleep08747-21sh00:00:350
52752130,1sleep00-21swapper/022:52:090
247752130,2sleep10-21swapper/122:40:001
1699399131,10cyclictest0-21swapper/321:28:173
16993991310,2cyclictest29154-21copy19:40:003
16993991310,2cyclictest0-21swapper/323:39:293
16993991310,2cyclictest0-21swapper/322:17:053
16993991310,2cyclictest0-21swapper/322:05:263
16993991310,2cyclictest0-21swapper/321:52:083
16993991310,2cyclictest0-21swapper/321:13:233
16993991310,2cyclictest0-21swapper/320:30:233
16993991310,2cyclictest0-21swapper/319:30:483
1699399130,2cyclictest0-21swapper/323:12:093
1699399130,2cyclictest0-21swapper/319:45:213
1699399130,2cyclictest0-21swapper/300:01:483
1699399130,12cyclictest0-21swapper/321:20:593
1699399130,12cyclictest0-21swapper/300:21:193
1699399130,12cyclictest0-21swapper/300:15:083
1699299139,2cyclictest0-21swapper/220:55:132
1699299131,2cyclictest616-21diskmemload00:05:552
1699299131,11cyclictest10857-21ssh21:20:012
16992991310,2cyclictest3629-21kworker/u8:223:36:022
16992991310,2cyclictest32002-21ls00:25:142
16992991310,2cyclictest24545-21ssh23:45:102
16992991310,2cyclictest0-21swapper/223:05:312
16992991310,2cyclictest0-21swapper/222:04:332
16992991310,2cyclictest0-21swapper/220:30:012
16992991310,2cyclictest0-21swapper/200:36:592
1699299130,2cyclictest0-21swapper/223:28:592
1699299130,2cyclictest0-21swapper/222:12:312
1699299130,2cyclictest0-21swapper/200:34:112
1699299130,12cyclictest11326-21ssh22:58:412
1699299130,12cyclictest0-21swapper/223:57:262
1699299130,12cyclictest0-21swapper/223:30:172
1699299130,12cyclictest0-21swapper/222:18:142
1699299130,12cyclictest0-21swapper/221:37:112
1699299130,12cyclictest0-21swapper/221:15:392
1699299130,12cyclictest0-21swapper/220:10:202
1699299130,12cyclictest0-21swapper/200:22:192
1699199138,3cyclictest0-21swapper/122:58:101
1699199138,3cyclictest0-21swapper/119:30:261
1699199138,3cyclictest0-21swapper/100:38:571
1699199131,7cyclictest0-21swapper/120:50:011
1699199131,3cyclictest0-21swapper/100:00:011
16991991310,2cyclictest616-21diskmemload22:10:161
16991991310,2cyclictest0-21swapper/123:03:211
16991991310,2cyclictest0-21swapper/122:40:561
16991991310,2cyclictest0-21swapper/121:25:181
16991991310,2cyclictest0-21swapper/121:15:551
16991991310,2cyclictest0-21swapper/119:55:151
1699199130,1cyclictest0-21swapper/123:39:161
1699199130,12cyclictest0-21swapper/121:33:451
1699199130,12cyclictest0-21swapper/120:55:151
1699199130,11cyclictest616-21diskmemload23:45:201
1699199130,11cyclictest0-21swapper/119:10:201
16990991311,1cyclictest0-21swapper/021:25:270
16990991310,2cyclictest0-21swapper/023:48:260
16990991310,2cyclictest0-21swapper/023:26:120
16990991310,2cyclictest0-21swapper/023:22:270
16990991310,2cyclictest0-21swapper/023:18:330
16990991310,2cyclictest0-21swapper/023:18:320
16990991310,2cyclictest0-21swapper/022:32:470
16990991310,2cyclictest0-21swapper/022:01:330
16990991310,2cyclictest0-21swapper/021:46:200
16990991310,2cyclictest0-21swapper/019:45:150
1699099130,2cyclictest22490-21munin-node0
1699099130,2cyclictest0-21swapper/023:38:240
1699099130,2cyclictest0-21swapper/022:23:350
1699099130,2cyclictest0-21swapper/022:19:370
1699099130,2cyclictest0-21swapper/021:58:250
1699099130,2cyclictest0-21swapper/021:05:260
1699099130,2cyclictest0-21swapper/020:57:100
1699099130,2cyclictest0-21swapper/019:50:260
1699099130,12cyclictest0-21swapper/023:05:560
1699099130,12cyclictest0-21swapper/022:29:560
1699099130,12cyclictest0-21swapper/020:35:290
1699099130,11cyclictest11641-21sh23:30:530
1699099130,11cyclictest0-21swapper/023:59:220
1699099130,11cyclictest0-21swapper/021:36:410
126072130,2sleep20-21swapper/222:26:262
110102130,2sleep00-21swapper/000:35:390
1699399129,2cyclictest30696-21ssh21:40:103
1699399129,2cyclictest0-21swapper/321:55:033
1699399129,2cyclictest0-21swapper/320:10:153
1699399127,3cyclictest14509-21grep20:20:263
1699399120,2cyclictest0-21swapper/320:55:123
1699399120,2cyclictest0-21swapper/300:35:063
1699399120,11cyclictest0-21swapper/320:15:223
1699399120,11cyclictest0-21swapper/319:20:263
1699399120,11cyclictest0-21swapper/319:15:253
1699399120,11cyclictest0-21swapper/300:10:233
1699299129,2cyclictest28984-21kworker/u8:121:09:052
1699299129,2cyclictest0-21swapper/222:30:242
1699299129,2cyclictest0-21swapper/221:40:112
1699299129,2cyclictest0-21swapper/219:15:192
1699299120,2cyclictest0-21swapper/222:45:332
1699299120,2cyclictest0-21swapper/222:38:022
1699299120,2cyclictest0-21swapper/219:45:192
1699299120,2cyclictest0-21swapper/200:10:022
1699299120,11cyclictest25653-21ssh23:13:032
1699299120,11cyclictest0-21swapper/220:05:182
1699299120,11cyclictest0-21swapper/220:05:172
1699199129,2cyclictest0-21swapper/123:20:321
1699199129,2cyclictest0-21swapper/123:05:561
1699199129,2cyclictest0-21swapper/121:05:181
1699199129,2cyclictest0-21swapper/100:15:331
1699199121,9cyclictest0-21swapper/120:35:181
1699199121,9cyclictest0-21swapper/119:15:171
1699199121,9cyclictest0-21swapper/100:20:141
1699199120,3cyclictest0-21swapper/120:40:121
1699199120,2cyclictest0-21swapper/122:04:571
1699199120,1cyclictest0-21swapper/122:20:301
1699199120,11cyclictest0-21swapper/122:50:151
1699199120,10cyclictest0-21swapper/121:35:221
1699099129,2cyclictest0-21swapper/020:25:210
1699099129,2cyclictest0-21swapper/019:40:160
1699099129,2cyclictest0-21swapper/019:15:180
1699099120,2cyclictest0-21swapper/022:12:570
1699099120,2cyclictest0-21swapper/020:15:100
1699099120,11cyclictest9136-21kworker/u8:020:21:050
1699099120,11cyclictest0-21swapper/021:30:130
1699099120,11cyclictest0-21swapper/020:10:270
1699099120,11cyclictest0-21swapper/019:20:220
1699099120,11cyclictest0-21swapper/000:30:030
1699399118,2cyclictest0-21swapper/320:05:073
1699399118,2cyclictest0-21swapper/320:05:063
1699399116,3cyclictest18554-21tail19:10:273
1699299118,2cyclictest31330-21kworker/u8:220:20:052
1699299118,2cyclictest19343-21kworker/u8:219:25:052
1699299118,2cyclictest14890-21kworker/u8:119:14:052
1699299118,2cyclictest0-21swapper/219:40:172
1699299118,2cyclictest0-21swapper/219:35:162
1699299118,2cyclictest0-21swapper/219:30:182
1699299110,10cyclictest0-21swapper/220:45:242
1699199118,2cyclictest0-21swapper/120:00:241
1699199112,7cyclictest0-21swapper/119:45:591
1699199112,2cyclictest0-21swapper/120:05:051
1699199112,2cyclictest0-21swapper/120:05:051
1699199111,3cyclictest0-21swapper/119:50:151
1699199110,9cyclictest0-21swapper/119:25:151
1699199110,3cyclictest27266-21munin-node1
1699199110,1cyclictest0-21swapper/120:20:161
1699099118,2cyclictest6887-21apt-get20:05:070
1699099118,2cyclictest6887-21apt-get20:05:060
1699099118,2cyclictest0-21swapper/019:10:130
1699099110,10cyclictest12783-21kworker/u8:019:30:050
1699399107,2cyclictest0-21swapper/320:35:263
1699299107,2cyclictest19343-21kworker/u8:219:24:052
1699299105,3cyclictest15789-21grep20:25:172
1699299102,3cyclictest30536-21cat21:00:262
1699299100,9cyclictest0-21swapper/220:35:182
1699199107,2cyclictest0-21swapper/120:15:141
1699199106,2cyclictest0-21swapper/120:50:191
1699199100,6cyclictest0-21swapper/119:20:091
1699099107,2cyclictest0-21swapper/020:50:110
1699099107,2cyclictest0-21swapper/020:00:250
1699099100,9cyclictest0-21swapper/021:02:340
169939996,2cyclictest0-21swapper/320:40:173
169939995,3cyclictest5083-21sshd21:04:473
169939994,3cyclictest5701-21ethtool20:00:183
169939994,3cyclictest3302-21forks19:55:163
169939994,3cyclictest23528-21tr20:45:153
169939991,6cyclictest31546-21cut21:05:153
169929994,3cyclictest5214-21grep20:00:162
169929994,3cyclictest5129-21runrttasks20:42:472
169929994,3cyclictest2889-21munin-node2
169929994,3cyclictest25492-21tr20:50:122
24855280,1sleep00-21swapper/019:27:430
169939983,3cyclictest26428-21switchtime20:50:233
169939983,3cyclictest24369-21sh19:25:273
169939980,7cyclictest0-21swapper/319:50:033
169929983,3cyclictest11143-21munin-node2
169909985,2cyclictest0-21swapper/020:49:210
169909983,3cyclictest27254-21seq19:35:030
16964280,5sleep00-21swapper/020:30:010
169909972,3cyclictest21665-21fschecks_count20:40:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional