You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-13 - 07:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Sun Jul 13, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1644210684,16sleep30-21swapper/319:07:503
1721210594,6sleep10-21swapper/119:08:491
1562210478,6sleep00-21swapper/019:06:480
166429578,12sleep20-21swapper/219:08:052
132182800,0sleep00-21swapper/023:15:250
100782770,0sleep30-21swapper/323:12:313
88552750,1sleep20-21swapper/219:25:152
262942730,1sleep30-21swapper/323:29:513
82852670,0sleep10-21swapper/121:31:471
1850993515,18cyclictest0-21swapper/321:45:133
236262340,1sleep223631-21cpuspeed_turbos21:15:132
318782300,0sleep20-21swapper/223:01:282
185099270,11cyclictest0-21swapper/319:15:153
16552230,1sleep00-21swapper/021:58:480
75862220,1sleep07589-21timerwakeupswit19:20:280
201652220,0sleep10-21swapper/100:28:531
1845992117,2cyclictest0-21swapper/122:00:151
185099200,2cyclictest0-21swapper/321:20:143
1845992017,2cyclictest0-21swapper/121:45:121
51012190,0sleep30-21swapper/320:35:243
185099190,18cyclictest0-21swapper/323:40:143
184599190,17cyclictest18098-21diskmemload22:15:151
184599180,5cyclictest5911-21gdm-simple-gree00:23:581
184799172,2cyclictest18098-21diskmemload23:34:042
1847991714,2cyclictest25648-21kworker/u8:122:20:042
185099161,13cyclictest18098-21diskmemload21:15:253
184799161,13cyclictest16332-21apt-get00:25:122
184599161,3cyclictest10201-21tr22:40:131
184599161,13cyclictest14924-21munin-node1
184599160,14cyclictest13228-21uname20:55:211
1844991613,2cyclictest22004-21kworker/u8:023:34:050
222152150,3sleep022223-21ssh22:51:490
207312150,1sleep0184499cyclictest21:11:340
185099151,3cyclictest3814-21ssh22:33:573
1850991512,2cyclictest0-21swapper/322:50:273
1850991510,3cyclictest0-21swapper/323:50:543
185099150,13cyclictest0-21swapper/323:20:163
185099150,13cyclictest0-21swapper/322:26:273
184799151,3cyclictest23113-21ssh22:53:102
184799150,9cyclictest0-21swapper/222:44:052
184599152,7cyclictest18098-21diskmemload23:15:001
1845991510,3cyclictest31466-21ssh22:29:071
1845991510,3cyclictest29027-21munin-node1
184499150,13cyclictest0-21swapper/020:30:180
263532140,3sleep10-21swapper/122:55:371
1850991411,2cyclictest0-21swapper/323:15:573
1850991411,2cyclictest0-21swapper/321:35:173
185099140,2cyclictest4007-21latency_hist19:15:003
185099140,2cyclictest0-21swapper/323:00:473
185099140,2cyclictest0-21swapper/321:00:273
185099140,13cyclictest0-21swapper/322:58:363
185099140,12cyclictest20514-21ssh21:11:153
185099140,12cyclictest0-21swapper/322:40:213
185099140,12cyclictest0-21swapper/321:50:133
185099140,12cyclictest0-21swapper/320:40:143
185099140,12cyclictest0-21swapper/319:24:593
184799141,2cyclictest31710-21apt-get23:35:142
184799140,1cyclictest0-21swapper/223:27:312
184799140,12cyclictest0-21swapper/223:45:492
184599149,3cyclictest0-21swapper/121:05:191
184599149,3cyclictest0-21swapper/121:05:181
184599142,6cyclictest0-21swapper/119:24:591
184599140,2cyclictest0-21swapper/123:07:111
184599140,12cyclictest6269-21kworker/u8:022:05:051
184599140,12cyclictest26717-21fschecks_time00:35:161
1844991411,2cyclictest3864-21kworker/u8:122:44:050
184499140,12cyclictest0-21swapper/000:15:580
138442140,3sleep013955-21sort23:50:000
232102130,2sleep00-21swapper/021:15:050
185099138,3cyclictest0-21swapper/300:07:293
185099138,3cyclictest0-21swapper/300:07:293
1850991310,2cyclictest18098-21diskmemload22:45:403
1850991310,2cyclictest0-21swapper/323:57:113
1850991310,2cyclictest0-21swapper/323:48:163
1850991310,2cyclictest0-21swapper/322:12:323
1850991310,2cyclictest0-21swapper/322:00:023
1850991310,2cyclictest0-21swapper/320:30:123
1850991310,2cyclictest0-21swapper/319:30:003
185099130,2cyclictest6381-21rm00:15:053
185099130,2cyclictest21194-21ssh00:30:073
185099130,2cyclictest18098-21diskmemload21:44:373
185099130,2cyclictest0-21swapper/323:07:253
185099130,2cyclictest0-21swapper/321:31:463
185099130,2cyclictest0-21swapper/319:50:173
185099130,12cyclictest0-21swapper/322:36:173
185099130,12cyclictest0-21swapper/322:20:263
185099130,12cyclictest0-21swapper/322:06:163
185099130,12cyclictest0-21swapper/321:55:153
185099130,12cyclictest0-21swapper/321:26:143
185099130,12cyclictest0-21swapper/320:10:153
185099130,12cyclictest0-21swapper/319:32:463
185099130,12cyclictest0-21swapper/300:13:173
185099130,11cyclictest18098-21diskmemload00:35:023
185099130,11cyclictest0-21swapper/300:25:283
184799131,3cyclictest32443-21ssh21:56:162
1847991310,2cyclictest3864-21kworker/u8:122:36:052
1847991310,2cyclictest30491-21kworker/u8:119:10:042
1847991310,2cyclictest28275-21kworker/u8:321:42:012
1847991310,2cyclictest18098-21diskmemload00:20:402
1847991310,2cyclictest0-21swapper/223:40:212
1847991310,2cyclictest0-21swapper/222:55:082
1847991310,2cyclictest0-21swapper/222:19:292
1847991310,2cyclictest0-21swapper/222:10:592
1847991310,2cyclictest0-21swapper/222:05:102
1847991310,2cyclictest0-21swapper/221:39:322
1847991310,2cyclictest0-21swapper/221:22:412
1847991310,2cyclictest0-21swapper/200:34:202
184799130,2cyclictest0-21swapper/223:16:322
184799130,2cyclictest0-21swapper/222:33:582
184799130,2cyclictest0-21swapper/221:49:212
184799130,2cyclictest0-21swapper/220:55:142
184799130,2cyclictest0-21swapper/200:15:312
184799130,12cyclictest0-21swapper/223:50:202
184799130,12cyclictest0-21swapper/222:25:302
184799130,12cyclictest0-21swapper/200:36:002
184799130,12cyclictest0-21swapper/200:12:122
184799130,12cyclictest0-21swapper/200:07:312
184799130,12cyclictest0-21swapper/200:07:302
184799130,12cyclictest0-21swapper/200:04:302
184799130,11cyclictest20509-21ssh21:11:142
184799130,11cyclictest0-21swapper/221:53:032
184599139,2cyclictest23322-21munin-node1
184599139,2cyclictest0-21swapper/119:25:271
184599138,3cyclictest0-21swapper/119:15:161
184599132,7cyclictest0-21swapper/120:30:011
184599132,6cyclictest0-21swapper/120:15:001
184599132,6cyclictest0-21swapper/119:09:591
184599131,6cyclictest0-21swapper/119:30:011
1845991310,2cyclictest0-21swapper/123:55:151
1845991310,2cyclictest0-21swapper/122:50:201
1845991310,2cyclictest0-21swapper/122:30:211
1845991310,2cyclictest0-21swapper/121:55:131
1845991310,2cyclictest0-21swapper/119:40:161
1845991310,2cyclictest0-21swapper/100:17:301
184599130,2cyclictest0-21swapper/121:35:181
184599130,12cyclictest0-21swapper/121:50:311
184599130,12cyclictest0-21swapper/121:41:011
184599130,12cyclictest0-21swapper/100:00:401
184499139,3cyclictest28840-21ssh00:05:060
184499139,3cyclictest28840-21ssh00:05:050
1844991310,2cyclictest3129-21ssh00:10:420
1844991310,2cyclictest2828-21ssh22:32:280
1844991310,2cyclictest0-21swapper/023:42:170
1844991310,2cyclictest0-21swapper/022:45:370
1844991310,2cyclictest0-21swapper/022:08:280
1844991310,2cyclictest0-21swapper/021:52:080
1844991310,2cyclictest0-21swapper/000:27:360
184499130,2cyclictest18098-21diskmemload23:53:570
184499130,2cyclictest0-21swapper/000:30:270
184499130,12cyclictest0-21swapper/023:35:060
184499130,12cyclictest0-21swapper/021:24:370
184499130,11cyclictest0-21swapper/023:55:380
163492130,2sleep00-21swapper/019:45:130
268232120,1sleep00-21swapper/000:35:180
185099129,2cyclictest0-21swapper/323:31:443
185099129,2cyclictest0-21swapper/322:16:023
185099129,2cyclictest0-21swapper/320:50:183
185099129,2cyclictest0-21swapper/320:00:193
185099120,2cyclictest0-21swapper/319:55:163
185099120,11cyclictest0-21swapper/323:37:243
185099120,11cyclictest0-21swapper/321:05:013
185099120,11cyclictest0-21swapper/321:05:003
185099120,11cyclictest0-21swapper/300:20:343
185099120,11cyclictest0-21swapper/300:00:223
184799129,2cyclictest0-21swapper/223:24:542
184799129,2cyclictest0-21swapper/223:08:002
184799129,2cyclictest0-21swapper/221:25:522
184799128,2cyclictest0-21swapper/223:10:162
184799120,2cyclictest18098-21diskmemload22:48:222
184799120,11cyclictest0-21swapper/223:58:012
184799120,11cyclictest0-21swapper/222:03:492
184799120,11cyclictest0-21swapper/221:00:192
184799120,11cyclictest0-21swapper/220:25:242
184799120,11cyclictest0-21swapper/219:50:012
184799120,11cyclictest0-21swapper/219:48:312
184799120,11cyclictest0-21swapper/219:35:012
184599129,2cyclictest18098-21diskmemload23:17:171
184599129,2cyclictest0-21swapper/122:36:241
184599129,2cyclictest0-21swapper/121:10:261
184599127,3cyclictest0-21swapper/120:50:141
184599127,3cyclictest0-21swapper/119:50:201
184599127,3cyclictest0-21swapper/119:35:151
184599123,5cyclictest18098-21diskmemload00:10:061
184599122,7cyclictest0-21swapper/122:24:591
184599121,9cyclictest0-21swapper/123:00:041
184599121,9cyclictest0-21swapper/120:15:161
184599121,3cyclictest0-21swapper/122:10:231
1845991210,1cyclictest0-21swapper/123:42:371
184599120,2cyclictest0-21swapper/100:30:371
184599120,2cyclictest0-21swapper/100:06:361
184599120,2cyclictest0-21swapper/100:06:351
184599120,11cyclictest15750-21kworker/u8:121:01:041
184599120,11cyclictest0-21swapper/123:37:201
184599120,11cyclictest0-21swapper/122:45:171
184599120,10cyclictest0-21swapper/120:45:171
184599120,10cyclictest0-21swapper/120:40:071
184599120,10cyclictest0-21swapper/119:55:241
184499129,2cyclictest0-21swapper/023:10:060
184499129,2cyclictest0-21swapper/022:00:260
184499129,2cyclictest0-21swapper/020:10:270
184499129,2cyclictest0-21swapper/019:37:470
184499120,2cyclictest0-21swapper/023:09:380
184499120,2cyclictest0-21swapper/021:37:560
184499120,2cyclictest0-21swapper/021:25:070
184499120,2cyclictest0-21swapper/000:21:010
184499120,11cyclictest0-21swapper/023:28:370
184499120,11cyclictest0-21swapper/022:58:080
184499120,11cyclictest0-21swapper/022:17:380
184499120,11cyclictest0-21swapper/021:42:180
184499120,11cyclictest0-21swapper/021:33:370
184499120,11cyclictest0-21swapper/000:00:320
184499120,10cyclictest10266-21munin-node0
184799118,2cyclictest22429-21kworker/u8:020:03:052
184799115,4cyclictest6076-21cut21:30:132
184799110,10cyclictest0-21swapper/220:05:222
184599118,2cyclictest0-21swapper/123:25:551
184599116,3cyclictest10331-21processes23:45:221
184599112,6cyclictest0-21swapper/120:37:581
184599111,8cyclictest0-21swapper/123:30:171
184599111,8cyclictest0-21swapper/123:20:241
184599111,8cyclictest0-21swapper/120:00:151
184599111,8cyclictest0-21swapper/119:45:251
184599111,3cyclictest0-21swapper/120:30:151
184599110,9cyclictest0-21swapper/120:05:141
184599110,3cyclictest0-21swapper/121:25:131
184499114,3cyclictest14056-21cron21:00:010
184499110,10cyclictest22429-21kworker/u8:020:26:010
185099105,3cyclictest30207-21cron20:20:003
185099105,3cyclictest25789-21grep20:05:273
185099101,7cyclictest0-21swapper/320:49:583
185099101,5cyclictest0-21swapper/319:45:003
185099100,7cyclictest0-21swapper/319:39:593
184799107,2cyclictest22429-21kworker/u8:020:47:042
184799107,2cyclictest15750-21kworker/u8:121:07:052
184799107,2cyclictest15750-21kworker/u8:121:07:042
184799107,2cyclictest0-21swapper/219:30:132
184799105,3cyclictest30926-21grep20:20:152
184799104,4cyclictest7629-21grep20:40:292
184799102,6cyclictest2583-21tr20:30:152
184599100,2cyclictest0-21swapper/120:20:121
184499107,2cyclictest18098-21diskmemload23:03:310
184499107,2cyclictest0-21swapper/022:35:350
184499107,2cyclictest0-21swapper/019:40:280
184499103,2cyclictest0-21swapper/020:35:040
184499100,9cyclictest6901-21kworker/u8:220:46:040
184499100,9cyclictest17338-21ssh22:14:580
184499100,9cyclictest0-21swapper/021:46:030
184499100,9cyclictest0-21swapper/019:15:170
184499100,8cyclictest0-21swapper/019:10:060
7840290,2sleep20-21swapper/219:21:592
18509992,5cyclictest0-21swapper/319:49:013
18509991,7cyclictest0-21swapper/320:59:583
18509990,6cyclictest0-21swapper/320:20:583
18479994,3cyclictest5129-21runrttasks20:50:072
18479994,3cyclictest4545-21grep19:15:182
18479994,3cyclictest280312chrt20:12:542
18479994,3cyclictest15362-21pzem_power19:40:252
18479993,4cyclictest4704-21taskset20:35:162
18479991,3cyclictest0-21swapper/219:55:142
18479990,2cyclictest0-21swapper/220:15:162
18449996,2cyclictest0-21swapper/022:20:140
18449996,2cyclictest0-21swapper/019:30:180
18449995,2cyclictest18518-21telnet19:50:150
18449994,3cyclictest9175-21grep19:25:180
18449994,3cyclictest16609-21grep21:05:160
18449994,3cyclictest16609-21grep21:05:150
18449994,3cyclictest14395-21aten2.4-expect21:00:150
18449990,8cyclictest18098-21diskmemload22:25:120
18449990,2cyclictest0-21swapper/020:40:080
18509983,3cyclictest732-21latency20:25:183
18449985,2cyclictest25076-21iostat_ios20:05:190
18449985,2cyclictest0-21swapper/019:55:160
18449984,3cyclictest5129-21runrttasks20:15:490
18449981,5cyclictest30862-21grep20:20:160
18449980,2cyclictest0-21swapper/023:20:050
18449973,3cyclictest22925-21sort20:00:160
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional