You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-14 - 04:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Sun Sep 14, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4657211493,16sleep10-21swapper/119:06:411
47712113102,6sleep30-21swapper/319:08:063
4593210897,6sleep00-21swapper/019:05:560
4605210786,16sleep20-21swapper/219:06:062
249832800,1sleep0101ktimersoftd/022:20:000
294812780,1sleep00-21swapper/000:04:150
232102690,0sleep20-21swapper/223:24:452
232102690,0sleep20-21swapper/223:24:442
192142250,1sleep30-21swapper/322:13:593
237532230,1sleep00-21swapper/023:57:530
495899221,3cyclictest30351-21ssh00:05:163
495699200,2cyclictest0-21swapper/119:40:121
495699200,19cyclictest0-21swapper/119:50:121
495899171,13cyclictest21901-21ssh22:50:133
495899161,3cyclictest21222-21diskmemload00:25:003
4956991611,3cyclictest0-21swapper/123:55:141
495699161,13cyclictest0-21swapper/121:11:471
4955991613,2cyclictest28969-21kworker/u8:223:20:050
4955991613,2cyclictest28969-21kworker/u8:223:20:040
495899151,8cyclictest11834-21ssh22:40:003
495899151,3cyclictest17750-21ssh23:51:073
4958991512,2cyclictest0-21swapper/323:45:213
4958991512,2cyclictest0-21swapper/322:47:313
495899151,12cyclictest2064-21ssh23:35:233
4958991510,3cyclictest11737-21sh21:32:443
495899150,2cyclictest0-21swapper/300:13:013
495699151,12cyclictest30341-21apt-get00:05:141
4956991510,3cyclictest11648-21munin-node1
495699150,3cyclictest0-21swapper/120:40:211
495699150,2cyclictest0-21swapper/123:06:061
495699150,13cyclictest0-21swapper/123:40:161
495599150,13cyclictest7586-21expr20:35:150
495599150,13cyclictest7586-21expr20:35:140
495599150,13cyclictest21222-21diskmemload22:51:040
495899149,3cyclictest5299-21wc22:00:003
495899142,7cyclictest0-21swapper/320:35:003
4958991411,2cyclictest0-21swapper/322:43:103
4958991411,2cyclictest0-21swapper/322:27:523
4958991411,2cyclictest0-21swapper/321:24:313
495899140,3cyclictest22333-21ssh23:55:423
495899140,3cyclictest17367-21ssh23:18:153
495899140,2cyclictest0-21swapper/300:33:113
495899140,13cyclictest0-21swapper/321:28:003
495899140,12cyclictest0-21swapper/323:05:143
495899140,12cyclictest0-21swapper/321:45:243
495899140,12cyclictest0-21swapper/300:25:223
495799147,6cyclictest31912-21chrt22:26:052
4957991411,2cyclictest5744-21kworker/u8:122:14:052
4957991411,2cyclictest4771-21kworker/u8:200:28:062
495799140,12cyclictest27461-21cpuspeed_turbos00:35:162
495799140,12cyclictest22499-21grep22:50:172
495799140,12cyclictest0-21swapper/200:21:102
495699148,4cyclictest0-21swapper/122:04:291
495699148,4cyclictest0-21swapper/121:17:481
4956991410,2cyclictest0-21swapper/123:15:151
495699140,2cyclictest0-21swapper/122:51:061
495699140,13cyclictest0-21swapper/123:53:351
495699140,13cyclictest0-21swapper/123:04:551
495699140,13cyclictest0-21swapper/121:50:161
4955991411,2cyclictest0-21swapper/022:40:010
4955991411,2cyclictest0-21swapper/022:35:170
4955991411,2cyclictest0-21swapper/021:16:170
495599140,2cyclictest0-21swapper/000:20:440
495599140,13cyclictest0-21swapper/021:12:240
495599140,12cyclictest3592-21ssh21:57:400
495599140,12cyclictest0-21swapper/022:10:150
495599140,12cyclictest0-21swapper/020:10:120
204932140,2sleep00-21swapper/000:27:410
495899139,2cyclictest0-21swapper/322:55:133
495899138,3cyclictest27176-21ssh00:00:383
495899138,3cyclictest0-21swapper/323:10:073
495899138,3cyclictest0-21swapper/319:30:003
495899131,2cyclictest0-21swapper/322:30:013
4958991310,2cyclictest0-21swapper/323:00:143
4958991310,2cyclictest0-21swapper/321:40:313
4958991310,2cyclictest0-21swapper/321:35:183
4958991310,2cyclictest0-21swapper/321:12:213
4958991310,2cyclictest0-21swapper/319:35:213
495899130,2cyclictest0-21swapper/323:27:333
495899130,2cyclictest0-21swapper/321:50:403
495899130,2cyclictest0-21swapper/321:15:203
495899130,2cyclictest0-21swapper/319:40:263
495899130,2cyclictest0-21swapper/300:15:423
495899130,12cyclictest0-21swapper/323:40:553
495899130,12cyclictest0-21swapper/323:30:013
495899130,12cyclictest0-21swapper/322:20:343
495899130,12cyclictest0-21swapper/322:01:043
4957991310,2cyclictest28969-21kworker/u8:223:18:022
4957991310,2cyclictest26591-21kworker/u8:000:02:052
4957991310,2cyclictest16778-21ssh23:50:212
4957991310,2cyclictest0-21swapper/223:34:012
4957991310,2cyclictest0-21swapper/221:25:152
4957991310,2cyclictest0-21swapper/200:12:252
4957991310,2cyclictest0-21swapper/200:06:392
495799130,2cyclictest0-21swapper/223:35:592
495799130,12cyclictest0-21swapper/220:50:142
495699139,2cyclictest26458-21cut20:00:201
4956991310,2cyclictest27381-21sshd22:21:321
4956991310,2cyclictest0-21swapper/122:47:501
4956991310,2cyclictest0-21swapper/122:44:021
4956991310,2cyclictest0-21swapper/122:35:361
4956991310,2cyclictest0-21swapper/122:25:461
4956991310,2cyclictest0-21swapper/122:18:331
4956991310,2cyclictest0-21swapper/122:05:571
4956991310,2cyclictest0-21swapper/121:27:161
4956991310,2cyclictest0-21swapper/120:50:161
4956991310,2cyclictest0-21swapper/120:10:141
4956991310,2cyclictest0-21swapper/100:32:501
495699130,2cyclictest0-21swapper/121:46:421
495699130,2cyclictest0-21swapper/120:25:271
495699130,2cyclictest0-21swapper/119:20:151
495699130,2cyclictest0-21swapper/100:26:361
495699130,12cyclictest0-21swapper/123:32:331
495699130,12cyclictest0-21swapper/100:38:551
495699130,12cyclictest0-21swapper/100:17:551
4955991310,2cyclictest11648-21ssh23:11:570
4955991310,2cyclictest0-21swapper/023:35:590
4955991310,2cyclictest0-21swapper/023:31:280
4955991310,2cyclictest0-21swapper/022:45:150
4955991310,2cyclictest0-21swapper/022:18:090
4955991310,2cyclictest0-21swapper/021:41:470
4955991310,2cyclictest0-21swapper/021:26:400
4955991310,2cyclictest0-21swapper/020:45:000
4955991310,2cyclictest0-21swapper/019:40:170
495599130,2cyclictest0-21swapper/023:25:240
495599130,2cyclictest0-21swapper/022:29:290
495599130,2cyclictest0-21swapper/021:50:200
495599130,12cyclictest0-21swapper/023:01:190
495599130,12cyclictest0-21swapper/022:32:490
495599130,12cyclictest0-21swapper/022:09:170
495599130,12cyclictest0-21swapper/021:46:280
495599130,12cyclictest0-21swapper/021:20:090
48952130,1sleep00-21swapper/023:05:190
74672120,2sleep27478-21ssh23:08:022
52122120,1sleep00-21swapper/000:11:500
495899129,2cyclictest0-21swapper/321:05:233
495899129,2cyclictest0-21swapper/300:35:043
495899122,7cyclictest0-21swapper/320:01:023
495899120,2cyclictest0-21swapper/320:45:163
495899120,2cyclictest0-21swapper/319:15:153
495899120,11cyclictest21222-21diskmemload22:17:343
495899120,11cyclictest0-21swapper/323:20:013
495899120,11cyclictest0-21swapper/323:20:013
495899120,11cyclictest0-21swapper/322:08:543
495899120,11cyclictest0-21swapper/319:30:183
495899120,11cyclictest0-21swapper/319:10:243
495799129,2cyclictest7460-21kworker/u8:020:46:022
495799129,2cyclictest27500-21kworker/u8:122:55:042
495799129,2cyclictest19472-21kworker/u8:221:16:042
495799129,2cyclictest16990-21kworker/u8:121:24:052
495799129,2cyclictest13426-21kworker/u8:119:40:052
495799129,2cyclictest0-21swapper/223:56:252
495799129,2cyclictest0-21swapper/200:15:492
495799120,2cyclictest0-21swapper/223:45:022
495799120,2cyclictest0-21swapper/220:45:002
495799120,2cyclictest0-21swapper/220:15:192
495799120,11cyclictest0-21swapper/223:41:292
495799120,11cyclictest0-21swapper/200:31:202
495799120,10cyclictest0-21swapper/222:16:062
495699129,2cyclictest21222-21diskmemload21:20:021
495699129,2cyclictest0-21swapper/123:35:321
495699129,2cyclictest0-21swapper/123:26:201
495699129,2cyclictest0-21swapper/122:56:151
495699129,2cyclictest0-21swapper/122:12:241
495699129,2cyclictest0-21swapper/120:35:211
495699129,2cyclictest0-21swapper/120:35:211
495699129,2cyclictest0-21swapper/120:30:161
495699129,2cyclictest0-21swapper/119:25:161
495699129,2cyclictest0-21swapper/100:20:301
495699121,7cyclictest0-21swapper/119:30:141
495699120,2cyclictest0-21swapper/123:12:561
495699120,2cyclictest0-21swapper/121:42:011
495699120,2cyclictest0-21swapper/100:10:151
495699120,11cyclictest21222-21diskmemload21:30:071
495699120,11cyclictest0-21swapper/123:22:411
495699120,11cyclictest0-21swapper/123:22:401
495699120,11cyclictest0-21swapper/122:33:161
495699120,11cyclictest0-21swapper/121:56:071
495699120,11cyclictest0-21swapper/121:38:051
495699120,11cyclictest0-21swapper/100:00:301
495699120,10cyclictest0-21swapper/120:45:221
495599129,2cyclictest5213-21apt-get20:30:120
495599129,2cyclictest0-21swapper/023:45:020
495599129,2cyclictest0-21swapper/019:45:120
495599129,2cyclictest0-21swapper/019:10:240
495599129,2cyclictest0-21swapper/000:35:500
495599129,2cyclictest0-21swapper/000:33:170
495599128,3cyclictest8646-21rm21:30:060
495599120,2cyclictest31925-21ssh00:05:310
495599120,2cyclictest0-21swapper/023:52:040
495599120,2cyclictest0-21swapper/021:36:180
495599120,2cyclictest0-21swapper/020:20:160
495599120,2cyclictest0-21swapper/020:05:160
495599120,11cyclictest21222-21diskmemload22:00:010
495599120,11cyclictest0-21swapper/023:40:460
495599120,11cyclictest0-21swapper/023:15:260
495599120,11cyclictest0-21swapper/022:55:290
495599120,11cyclictest0-21swapper/020:50:170
495599120,11cyclictest0-21swapper/020:25:170
495599120,11cyclictest0-21swapper/019:30:180
495599120,11cyclictest0-21swapper/000:16:490
308352120,2sleep20-21swapper/221:52:112
495899118,2cyclictest0-21swapper/320:05:193
495899117,2cyclictest0-21swapper/320:20:063
495899112,7cyclictest0-21swapper/320:50:583
495899110,6cyclictest0-21swapper/320:40:013
495899110,6cyclictest0-21swapper/320:40:013
495799118,2cyclictest7460-21kworker/u8:020:39:052
495799118,2cyclictest7460-21kworker/u8:020:39:052
495799118,2cyclictest6762-21kworker/u8:219:11:042
495799118,2cyclictest27500-21kworker/u8:122:36:022
495799118,2cyclictest19472-21kworker/u8:219:59:042
495799118,2cyclictest16990-21kworker/u8:120:59:052
495799118,2cyclictest0-21swapper/219:45:252
495799116,3cyclictest2896-21fschecks_time22:30:162
495799116,3cyclictest0-21swapper/220:33:052
495799111,3cyclictest0-21swapper/222:47:232
495799110,10cyclictest0-21swapper/222:02:552
495799110,10cyclictest0-21swapper/221:33:552
495699117,2cyclictest0-21swapper/121:05:141
495699111,7cyclictest5911-21gdm-simple-gree20:23:581
495699110,8cyclictest0-21swapper/119:49:581
495599110,2cyclictest0-21swapper/019:51:590
141002110,2sleep20-21swapper/222:41:012
495899107,2cyclictest0-21swapper/319:56:333
495899102,6cyclictest0-21swapper/319:50:243
495799107,2cyclictest6762-21kworker/u8:219:25:052
495799107,2cyclictest6762-21kworker/u8:219:15:052
495799107,2cyclictest10680-21kworker/u8:221:42:042
495799105,3cyclictest5010-21snmpd19:35:152
495799105,3cyclictest4491-21grep20:25:262
495799105,3cyclictest32182-21tune2fs23:00:172
495799105,3cyclictest25201-21chrt21:13:422
495799104,4cyclictest10047-21grep19:20:162
495799100,9cyclictest29547-21ssh22:24:552
495799100,9cyclictest0-21swapper/222:08:242
495699107,2cyclictest5182-21munin-node1
495699107,2cyclictest0-21swapper/121:00:281
495699107,2cyclictest0-21swapper/119:55:131
495699106,2cyclictest0-21swapper/120:15:161
495699102,6cyclictest0-21swapper/119:35:151
495699100,8cyclictest0-21swapper/120:55:291
495699100,8cyclictest0-21swapper/119:15:131
495599107,2cyclictest0-21swapper/021:00:310
146582100,2sleep20-21swapper/221:35:232
49589994,3cyclictest5129-21runrttasks20:41:573
49589994,3cyclictest432-21unixbench_singl20:15:303
49589994,3cyclictest15437-21aten2.4_r3power20:55:153
49589992,6cyclictest0-21swapper/320:29:593
49589992,4cyclictest0-21swapper/319:20:023
49589991,3cyclictest0-21swapper/319:45:143
49589990,8cyclictest0-21swapper/321:00:113
49589990,7cyclictest4654-21avahi-daemon20:10:093
49579996,2cyclictest12204-21kworker/u8:019:32:042
49579994,3cyclictest9126-21ssh23:10:132
49579994,3cyclictest25748-21tr20:00:142
49579994,3cyclictest23362-21aten2.4_r3power21:45:162
49579994,3cyclictest21570-21aten2.4-expect19:50:132
49579994,3cyclictest19508-21aten2.4_r3power21:05:132
49579994,3cyclictest1032-21fschecks_count21:55:172
49579994,3cyclictest0-21swapper/220:10:232
49579991,3cyclictest0-21swapper/221:00:202
49579990,2cyclictest21222-21diskmemload23:25:542
49569991,7cyclictest0-21swapper/120:05:211
49559996,2cyclictest0-21swapper/019:25:210
49559990,8cyclictest0-21swapper/021:06:040
49559990,8cyclictest0-21swapper/020:49:280
49559990,8cyclictest0-21swapper/019:56:310
10857290,2sleep00-21swapper/019:21:370
49579983,3cyclictest27707-21tr20:05:172
49579983,3cyclictest1814-21kernelversion20:20:152
49559985,2cyclictest0-21swapper/019:15:080
49559980,7cyclictest31337-21copy20:15:050
49559980,7cyclictest0-21swapper/020:59:220
49559980,2cyclictest0-21swapper/020:03:340
49559980,2cyclictest0-21swapper/019:35:310
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional