You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 08:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot6s.osadl.org (updated Sun Nov 23, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31426211997,16sleep10-21swapper/119:09:411
31201211997,16sleep30-21swapper/319:06:463
29829211995,16sleep20-21swapper/219:05:132
31337211695,16sleep00-21swapper/019:08:320
291782810,0sleep30-21swapper/323:41:293
125992730,1sleep00-21swapper/000:33:210
123302720,0sleep10-21swapper/121:00:181
31481995433,19cyclictest0-21swapper/120:10:131
30302230,0sleep30-21swapper/323:14:553
31482991916,2cyclictest1883-21kworker/u8:222:09:042
31482991916,2cyclictest15069-21kworker/u8:221:35:042
4752180,1sleep20-21swapper/220:30:262
31482991815,2cyclictest24764-21kworker/u8:223:14:012
31482991815,2cyclictest13599-21kworker/u8:222:53:042
31482991714,2cyclictest7868-21kworker/u8:100:05:032
31481991712,3cyclictest27224-21uname23:05:161
3148399162,12cyclictest24780-21ssh00:11:093
3148299161,3cyclictest4369-21ssh21:31:572
3148299161,3cyclictest31723-21cut23:45:132
31481991612,2cyclictest29427-21apt-get20:25:061
3148199161,13cyclictest27599-21ssh23:40:161
3148199160,14cyclictest13639-21cut00:00:161
31483991510,3cyclictest32467-21memory20:30:173
3148399150,2cyclictest0-21swapper/321:30:113
31482991512,2cyclictest31293-21kworker/u8:023:27:042
31482991512,2cyclictest29841-21kworker/u8:121:26:052
31482991512,2cyclictest21417-21kworker/u8:122:49:042
31481991511,2cyclictest0-21swapper/100:25:261
3148199150,2cyclictest15313-21diskmemload23:50:161
3148199150,2cyclictest0-21swapper/100:31:571
3148199150,13cyclictest17752-21munin-node1
3148199150,13cyclictest0-21swapper/123:13:561
3148199150,13cyclictest0-21swapper/100:10:121
3148099151,2cyclictest8419-21ssh22:45:150
31480991512,2cyclictest0-21swapper/022:41:360
3148099150,13cyclictest0-21swapper/021:45:190
303032150,3sleep030436-21latency20:25:210
9942140,2sleep00-21swapper/023:45:480
3148399141,2cyclictest5129-21runrttasks22:18:063
3148399140,12cyclictest15313-21diskmemload00:07:293
3148399140,12cyclictest15313-21diskmemload00:00:183
3148399140,12cyclictest10600-21python19:35:323
3148399140,12cyclictest0-21swapper/322:05:123
3148399140,12cyclictest0-21swapper/300:29:173
3148299141,2cyclictest1244-21apt-get19:15:122
31482991411,2cyclictest29841-21kworker/u8:121:50:042
31482991411,2cyclictest21417-21kworker/u8:122:44:042
31482991410,2cyclictest5703-21ssh23:16:262
31482991410,2cyclictest0-21swapper/222:00:002
3148299140,3cyclictest16893-21cpuspeed_turbos22:20:142
3148299140,3cyclictest0-21swapper/222:19:502
3148299140,2cyclictest0-21swapper/221:41:502
3148299140,12cyclictest0-21swapper/221:21:222
3148299140,12cyclictest0-21swapper/200:00:302
3148199149,3cyclictest3427-21sh21:30:281
3148199149,3cyclictest0-21swapper/122:20:141
3148199142,6cyclictest0-21swapper/122:50:001
31481991411,2cyclictest0-21swapper/123:29:561
31481991411,2cyclictest0-21swapper/123:24:461
31481991411,2cyclictest0-21swapper/120:45:261
3148199141,10cyclictest15313-21diskmemload23:00:261
3148199140,2cyclictest0-21swapper/123:36:261
3148199140,2cyclictest0-21swapper/123:36:261
3148199140,2cyclictest0-21swapper/123:15:171
3148199140,2cyclictest0-21swapper/122:42:261
3148199140,13cyclictest0-21swapper/122:06:061
3148199140,13cyclictest0-21swapper/119:20:161
3148099141,2cyclictest6747-21apt-get21:35:150
3148099141,2cyclictest4364-21nscd23:55:410
31480991411,2cyclictest15313-21diskmemload00:38:560
31480991411,2cyclictest0-21swapper/023:54:060
3148099140,13cyclictest0-21swapper/023:03:360
3148099140,13cyclictest0-21swapper/022:04:160
3148099140,13cyclictest0-21swapper/019:10:160
3148099140,12cyclictest19193-21sh22:22:040
3148099140,12cyclictest0-21swapper/023:25:570
3148099140,12cyclictest0-21swapper/022:34:160
3148099140,12cyclictest0-21swapper/021:56:360
237892140,0sleep20-21swapper/223:01:032
3148399138,3cyclictest20629-21ssh21:15:153
3148399137,4cyclictest8084-21grep22:45:133
3148399131,11cyclictest0-21swapper/322:10:283
3148399131,10cyclictest5129-21runrttasks23:31:313
31483991310,2cyclictest15313-21diskmemload21:38:283
31483991310,2cyclictest0-21swapper/323:51:483
31483991310,2cyclictest0-21swapper/322:25:193
31483991310,2cyclictest0-21swapper/321:52:273
31483991310,2cyclictest0-21swapper/321:00:183
3148399130,12cyclictest0-21swapper/323:35:383
3148399130,12cyclictest0-21swapper/323:35:383
3148399130,12cyclictest0-21swapper/323:17:393
3148399130,12cyclictest0-21swapper/322:40:173
3148399130,11cyclictest26665-21ssh21:55:253
3148399130,11cyclictest1156-21ssh21:29:253
31482991310,2cyclictest25584-21kworker/u8:022:00:042
31482991310,2cyclictest23753-21kworker/u8:220:25:042
31482991310,2cyclictest15313-21diskmemload21:15:172
31482991310,2cyclictest15313-21diskmemload00:26:472
31482991310,2cyclictest0-21swapper/222:55:032
31482991310,2cyclictest0-21swapper/222:10:592
31482991310,2cyclictest0-21swapper/219:45:232
3148299130,2cyclictest1879-21ssh00:21:242
3148299130,2cyclictest15313-21diskmemload23:07:312
3148299130,2cyclictest0-21swapper/223:40:122
3148299130,2cyclictest0-21swapper/222:31:042
3148299130,2cyclictest0-21swapper/200:11:222
3148299130,12cyclictest0-21swapper/223:56:202
3148299130,12cyclictest0-21swapper/223:54:102
3148299130,12cyclictest0-21swapper/223:36:242
3148299130,12cyclictest0-21swapper/223:36:242
3148299130,12cyclictest0-21swapper/223:30:592
3148299130,12cyclictest0-21swapper/223:24:272
3148299130,12cyclictest0-21swapper/221:11:132
3148299130,12cyclictest0-21swapper/220:55:242
3148299130,12cyclictest0-21swapper/200:31:162
3148299130,12cyclictest0-21swapper/200:16:302
3148199132,6cyclictest0-21swapper/119:25:001
31481991311,1cyclictest0-21swapper/123:57:451
31481991310,2cyclictest15313-21diskmemload22:04:051
31481991310,2cyclictest0-21swapper/122:51:071
31481991310,2cyclictest0-21swapper/121:10:251
31481991310,2cyclictest0-21swapper/100:17:451
3148199130,2cyclictest15313-21diskmemload23:30:451
3148199130,2cyclictest15313-21diskmemload00:08:551
3148199130,2cyclictest0-21swapper/122:12:061
3148199130,2cyclictest0-21swapper/121:44:561
3148199130,12cyclictest0-21swapper/121:21:261
3148199130,12cyclictest0-21swapper/119:50:121
3148199130,11cyclictest0-21swapper/122:35:241
3148199130,11cyclictest0-21swapper/121:49:251
3148199130,11cyclictest0-21swapper/121:35:371
3148099139,3cyclictest21499-21ssh21:15:260
31480991310,2cyclictest0-21swapper/022:26:100
31480991310,2cyclictest0-21swapper/022:19:080
31480991310,2cyclictest0-21swapper/022:05:120
31480991310,2cyclictest0-21swapper/021:52:210
31480991310,2cyclictest0-21swapper/021:21:090
31480991310,2cyclictest0-21swapper/021:05:150
31480991310,2cyclictest0-21swapper/000:25:520
31480991310,2cyclictest0-21swapper/000:20:180
31480991310,2cyclictest0-21swapper/000:19:470
31480991310,2cyclictest0-21swapper/000:10:020
3148099130,2cyclictest28098-21pzem_power23:40:220
3148099130,2cyclictest0-21swapper/023:20:380
3148099130,2cyclictest0-21swapper/023:07:180
3148099130,2cyclictest0-21swapper/022:55:560
3148099130,12cyclictest10824-21kworker/u8:223:36:040
3148099130,12cyclictest10824-21kworker/u8:223:36:040
3148099130,12cyclictest0-21swapper/023:32:260
3148099130,12cyclictest0-21swapper/023:15:160
3148099130,12cyclictest0-21swapper/021:32:110
3148099130,12cyclictest0-21swapper/021:25:570
3148099130,12cyclictest0-21swapper/020:20:140
3148099130,12cyclictest0-21swapper/019:30:270
3148099130,11cyclictest7825-21ssh22:10:150
121012130,2sleep30-21swapper/300:32:323
3148399129,2cyclictest15313-21diskmemload23:56:093
3148399129,2cyclictest0-21swapper/323:45:093
3148399129,2cyclictest0-21swapper/322:57:493
3148399129,2cyclictest0-21swapper/322:35:363
3148399129,2cyclictest0-21swapper/322:00:573
3148399129,2cyclictest0-21swapper/321:13:163
3148399129,2cyclictest0-21swapper/320:50:173
3148399129,2cyclictest0-21swapper/320:35:273
3148399129,2cyclictest0-21swapper/320:35:273
3148399129,2cyclictest0-21swapper/319:45:233
3148399129,2cyclictest0-21swapper/300:20:563
3148399122,7cyclictest0-21swapper/321:09:593
3148399120,2cyclictest0-21swapper/323:22:273
3148399120,2cyclictest0-21swapper/323:07:363
3148399120,2cyclictest0-21swapper/321:46:563
3148399120,2cyclictest0-21swapper/320:15:173
3148399120,2cyclictest0-21swapper/300:17:393
3148399120,11cyclictest0-21swapper/323:27:083
3148399120,11cyclictest0-21swapper/323:00:393
3148399120,11cyclictest0-21swapper/322:54:183
3148399120,11cyclictest0-21swapper/322:20:473
3148399120,11cyclictest0-21swapper/321:41:593
3148399120,11cyclictest0-21swapper/321:20:293
3148399120,11cyclictest0-21swapper/320:20:173
3148399120,11cyclictest0-21swapper/319:55:173
3148399120,11cyclictest0-21swapper/300:35:473
3148299129,2cyclictest29830-21kworker/u8:219:22:012
3148299129,2cyclictest15748-21kworker/u8:221:45:042
3148299129,2cyclictest15531-21apt-get19:50:122
3148299120,2cyclictest0-21swapper/221:01:322
3148299120,11cyclictest0-21swapper/222:35:072
3148299120,11cyclictest0-21swapper/222:25:222
3148299120,11cyclictest0-21swapper/200:39:082
3148199129,2cyclictest0-21swapper/123:45:551
3148199129,2cyclictest0-21swapper/122:30:471
3148199129,2cyclictest0-21swapper/121:50:131
3148199129,2cyclictest0-21swapper/121:15:171
3148199129,2cyclictest0-21swapper/120:20:211
3148199122,7cyclictest0-21swapper/119:44:581
3148199121,9cyclictest3045-21munin-run20:40:001
3148199121,9cyclictest3045-21munin-run20:40:001
3148199121,3cyclictest15313-21diskmemload00:35:151
3148199121,3cyclictest0-21swapper/119:30:151
3148199120,2cyclictest15313-21diskmemload21:25:371
3148199120,11cyclictest0-21swapper/122:25:221
3148199120,11cyclictest0-21swapper/122:15:131
3148199120,11cyclictest0-21swapper/121:55:231
3148199120,11cyclictest0-21swapper/120:50:161
3148199120,11cyclictest0-21swapper/100:20:101
3148099129,2cyclictest15313-21diskmemload00:02:160
3148099129,2cyclictest0-21swapper/022:37:180
3148099129,2cyclictest0-21swapper/021:11:200
3148099129,2cyclictest0-21swapper/019:35:000
3148099120,2cyclictest0-21swapper/023:11:010
3148099120,2cyclictest0-21swapper/020:10:270
3148099120,2cyclictest0-21swapper/020:05:160
3148099120,11cyclictest711-21kworker/u8:119:45:040
3148099120,11cyclictest0-21swapper/022:50:320
3148099120,11cyclictest0-21swapper/021:41:520
3148099120,11cyclictest0-21swapper/020:55:190
3148099120,11cyclictest0-21swapper/019:25:190
3148099120,11cyclictest0-21swapper/000:05:260
3148399118,2cyclictest0-21swapper/320:45:273
3148399118,2cyclictest0-21swapper/320:00:153
3148399110,2cyclictest16641-21python19:50:243
3148399110,10cyclictest0-21swapper/322:30:243
3148299118,2cyclictest17163-21kworker/u8:019:57:042
3148299118,2cyclictest0-21swapper/220:45:122
3148299110,1cyclictest0-21swapper/220:05:212
3148199117,2cyclictest0-21swapper/120:55:261
3148199116,3cyclictest13517-21munin-node1
3148199110,9cyclictest0-21swapper/120:30:121
3148199110,9cyclictest0-21swapper/119:10:201
3148199110,10cyclictest0-21swapper/120:40:241
3148199110,10cyclictest0-21swapper/119:35:301
3148199110,10cyclictest0-21swapper/119:15:211
3148099118,2cyclictest0-21swapper/020:50:270
3148099118,2cyclictest0-21swapper/019:15:170
215862110,2sleep10-21swapper/120:05:141
3148399107,2cyclictest21741-21aten2.4-expect20:05:143
3148399107,2cyclictest0-21swapper/320:10:193
3148399105,3cyclictest4646-21unixbench_multi20:40:263
3148399105,3cyclictest3147-21grep19:20:003
3148399105,3cyclictest1197-21sort19:15:003
3148399102,6cyclictest0-21swapper/319:20:263
3148299108,1cyclictest0-21swapper/219:32:042
3148299107,2cyclictest711-21kworker/u8:119:43:042
3148299107,2cyclictest23753-21kworker/u8:220:51:042
3148299107,2cyclictest23753-21kworker/u8:220:35:052
3148299107,2cyclictest23753-21kworker/u8:220:35:042
3148299107,2cyclictest0-21swapper/220:40:212
3148299100,9cyclictest0-21swapper/220:16:022
3148199106,2cyclictest13516-21munin-node1
3148199101,2cyclictest17543-21apt-get19:55:141
3148199100,8cyclictest0-21swapper/120:15:161
3148199100,8cyclictest0-21swapper/120:00:251
6411290,1sleep30-21swapper/319:25:213
314839994,3cyclictest9531-21aten2.4_r3power20:55:133
314839994,3cyclictest4324-21nscd19:30:173
314839994,3cyclictest31199-21sshd20:26:423
314829996,2cyclictest9639-21kworker/u8:121:05:042
314829996,2cyclictest23753-21kworker/u8:220:13:052
314829994,3cyclictest9572sleep219:14:132
314829994,3cyclictest5725-21cpuspeed_turbos19:25:152
314829994,3cyclictest28378-21wc20:20:192
314829994,3cyclictest10373-21ntp_offset19:35:262
314829993,4cyclictest278-21jbd2/sda1-820:01:482
314809996,2cyclictest0-21swapper/020:35:160
314809996,2cyclictest0-21swapper/020:35:160
314809996,2cyclictest0-21swapper/020:32:060
314809994,3cyclictest4166-21rsyslogd20:40:420
314809990,2cyclictest0-21swapper/020:03:430
314809990,2cyclictest0-21swapper/019:43:160
314839983,3cyclictest11723-21cpuspeed_turbos19:40:153
314809985,2cyclictest101ktimersoftd/020:15:140
314809983,3cyclictest6467-21taskset20:45:210
314809980,7cyclictest9639-21kworker/u8:121:00:040
314809980,2cyclictest0-21swapper/019:23:570
16718281,3sleep00-21swapper/019:50:260
314809974,2cyclictest17574-21aten2.4-expect19:55:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional