You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-03 - 03:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Mon Nov 03, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
7447211897,16sleep10-21swapper/119:07:221
7353211897,16sleep20-21swapper/219:06:112
7483211494,15sleep00-21swapper/019:07:490
7439211395,13sleep30-21swapper/319:07:163
246272970,1sleep10-21swapper/123:22:481
234602800,0sleep00-21swapper/022:15:160
130322720,0sleep30-21swapper/322:05:053
264792700,1sleep20-21swapper/200:30:282
312742360,0sleep00-21swapper/000:02:470
768299202,17cyclictest0-21swapper/222:46:042
221812200,0sleep00-21swapper/000:25:560
768199194,13cyclictest23894-21diskmemload22:35:271
7682991714,2cyclictest30764-21kworker/u8:000:18:052
7682991713,3cyclictest19243-21kworker/u8:123:17:052
768299170,2cyclictest0-21swapper/222:52:052
768199170,15cyclictest0-21swapper/120:25:181
7680991714,2cyclictest19894-21kworker/u8:022:46:050
310502170,1sleep30-21swapper/321:16:083
228462170,1sleep2301ktimersoftd/200:26:582
768399161,3cyclictest0-21swapper/320:25:163
768399161,13cyclictest24180-21ssh22:49:513
768399161,13cyclictest0-21swapper/319:55:163
7682991613,2cyclictest18431-21kworker/u8:100:07:052
768299161,2cyclictest0-21swapper/222:43:052
7682991612,3cyclictest6065-21kworker/u8:020:27:042
768299160,14cyclictest0-21swapper/219:37:042
7681991613,2cyclictest30764-21kworker/u8:000:28:041
7681991613,2cyclictest27839-21kworker/u8:122:53:041
768199160,2cyclictest0-21swapper/100:31:041
7680991612,3cyclictest12096-21kworker/u8:222:43:040
292602161,3sleep029270-21ssh21:48:090
768399151,12cyclictest0-21swapper/323:53:113
768399150,2cyclictest0-21swapper/319:30:193
768399150,13cyclictest7551-21ssh21:59:293
768399150,13cyclictest7551-21ssh21:59:283
768399150,13cyclictest0-21swapper/300:29:293
768299151,3cyclictest0-21swapper/200:00:012
7682991512,2cyclictest19243-21kworker/u8:123:31:052
7682991512,2cyclictest16018-21kworker/u8:119:42:022
7682991512,2cyclictest12096-21kworker/u8:222:37:042
7682991511,3cyclictest12128-21kworker/u8:120:56:052
7681991512,2cyclictest23584-21kworker/u8:121:44:041
7681991511,2cyclictest332250irq/24-eth123:17:441
768199151,12cyclictest23894-21diskmemload23:32:261
768199151,12cyclictest14688-21iostat_ios19:25:181
768199150,13cyclictest23894-21diskmemload21:56:091
768199150,13cyclictest23894-21diskmemload21:56:091
768199150,13cyclictest23-21ksoftirqd/123:53:061
768199150,13cyclictest0-21swapper/123:12:351
7680991512,2cyclictest19894-21kworker/u8:022:52:050
768099150,13cyclictest0-21swapper/020:05:140
768399149,3cyclictest15817-21wc19:30:003
768399141,7cyclictest0-21swapper/300:40:003
7683991411,2cyclictest4136-21ssh00:08:283
7683991411,2cyclictest0-21swapper/323:45:073
7683991411,2cyclictest0-21swapper/323:39:153
7683991411,2cyclictest0-21swapper/322:25:083
7683991410,2cyclictest11777-21python00:15:243
768399140,3cyclictest30076-21ssh21:49:233
768399140,2cyclictest0-21swapper/323:34:383
768399140,2cyclictest0-21swapper/322:55:373
768399140,2cyclictest0-21swapper/322:52:183
768399140,2cyclictest0-21swapper/322:24:373
768399140,2cyclictest0-21swapper/321:34:183
768399140,13cyclictest0-21swapper/323:09:283
768399140,13cyclictest0-21swapper/300:01:473
768399140,12cyclictest29427-21ssh00:34:383
768399140,12cyclictest16447-21ssh22:40:303
768399140,12cyclictest0-21swapper/323:11:233
768399140,12cyclictest0-21swapper/322:35:153
768399140,12cyclictest0-21swapper/321:43:073
7682991411,2cyclictest7036-21kworker/u8:223:08:052
7682991411,2cyclictest4597-21kworker/u8:019:24:042
7682991411,2cyclictest31775-21kworker/u8:020:11:042
7682991411,2cyclictest30764-21kworker/u8:000:03:042
7682991411,2cyclictest24330-21kworker/u8:021:21:052
7682991411,2cyclictest16056-21kworker/u8:020:53:022
7682991411,2cyclictest16018-21kworker/u8:120:22:052
7682991411,2cyclictest16018-21kworker/u8:120:02:042
7682991411,2cyclictest1217-21kworker/u8:223:38:042
7682991411,2cyclictest10210-21kworker/u8:022:20:042
7682991411,2cyclictest0-21swapper/221:45:002
7682991410,3cyclictest9440-21kworker/u8:220:35:042
768299140,3cyclictest30575-21fschecks_time00:35:172
768299140,12cyclictest0-21swapper/222:59:052
768199149,3cyclictest30481-21sh00:01:321
7681991411,2cyclictest0-21swapper/122:45:181
7681991411,2cyclictest0-21swapper/100:18:281
768199140,3cyclictest8216-21ssh22:32:501
768199140,3cyclictest0-21swapper/100:37:401
768199140,2cyclictest0-21swapper/119:45:181
768199140,13cyclictest0-21swapper/122:57:591
768199140,13cyclictest0-21swapper/100:07:381
768199140,12cyclictest28523-21ssh21:14:431
768199140,12cyclictest0-21swapper/123:43:151
768199140,12cyclictest0-21swapper/123:25:001
768199140,12cyclictest0-21swapper/123:05:371
768199140,12cyclictest0-21swapper/121:54:491
768199140,12cyclictest0-21swapper/120:20:171
7680991411,2cyclictest0-21swapper/021:10:250
7680991411,2cyclictest0-21swapper/000:11:040
7680991410,2cyclictest18061-21cut22:10:130
7680991410,2cyclictest0-21swapper/022:20:230
7680991410,2cyclictest0-21swapper/022:05:190
7680991410,2cyclictest0-21swapper/000:30:150
768099140,12cyclictest23023-21ps19:45:210
768099140,12cyclictest0-21swapper/023:55:180
768099140,12cyclictest0-21swapper/022:55:060
768099140,12cyclictest0-21swapper/020:30:130
127462140,1sleep00-21swapper/022:04:590
89712131,9sleep20-21swapper/221:26:492
768399133,6cyclictest48842chrt20:20:263
768399132,6cyclictest0-21swapper/320:40:003
768399132,6cyclictest0-21swapper/320:04:593
7683991310,2cyclictest21473-21ssh23:20:063
7683991310,2cyclictest0-21swapper/323:26:003
7683991310,2cyclictest0-21swapper/323:01:153
7683991310,2cyclictest0-21swapper/322:32:173
7683991310,2cyclictest0-21swapper/322:01:083
7683991310,2cyclictest0-21swapper/321:51:093
7683991310,2cyclictest0-21swapper/321:36:353
7683991310,2cyclictest0-21swapper/319:35:243
7683991310,2cyclictest0-21swapper/300:22:053
768399130,2cyclictest18508-21ssh23:15:533
768399130,2cyclictest0-21swapper/323:56:403
768399130,2cyclictest0-21swapper/321:25:133
768399130,2cyclictest0-21swapper/321:11:383
768399130,2cyclictest0-21swapper/300:13:203
768399130,12cyclictest0-21swapper/323:40:553
768399130,12cyclictest0-21swapper/321:24:123
768399130,12cyclictest0-21swapper/320:55:133
768399130,11cyclictest0-21swapper/322:11:283
7682991310,2cyclictest11132-21ssh22:02:282
7682991310,2cyclictest10210-21kworker/u8:022:30:052
7682991310,2cyclictest0-21swapper/221:50:172
768299130,2cyclictest0-21swapper/221:35:142
768299130,2cyclictest0-21swapper/219:55:192
768299130,2cyclictest0-21swapper/219:50:192
768299130,12cyclictest6958-21ssh21:58:342
768299130,12cyclictest6958-21ssh21:58:332
768299130,12cyclictest0-21swapper/221:30:492
768299130,11cyclictest0-21swapper/223:52:042
768199139,2cyclictest679-21perl22:25:171
768199138,3cyclictest11790-21rm22:03:291
7681991310,2cyclictest0-21swapper/123:57:191
7681991310,2cyclictest0-21swapper/121:29:191
7681991310,2cyclictest0-21swapper/121:21:181
7681991310,2cyclictest0-21swapper/121:15:131
7681991310,2cyclictest0-21swapper/121:05:191
7681991310,2cyclictest0-21swapper/121:00:231
7681991310,2cyclictest0-21swapper/119:50:121
7681991310,2cyclictest0-21swapper/100:11:051
768199130,2cyclictest23894-21diskmemload23:47:521
768199130,2cyclictest0-21swapper/123:02:581
768199130,2cyclictest0-21swapper/121:32:021
768199130,2cyclictest0-21swapper/120:50:131
768199130,2cyclictest0-21swapper/100:23:291
768199130,12cyclictest0-21swapper/122:20:181
768199130,12cyclictest0-21swapper/122:17:181
768199130,12cyclictest0-21swapper/122:14:551
768199130,12cyclictest0-21swapper/122:06:481
768199130,12cyclictest0-21swapper/121:35:091
768199130,12cyclictest0-21swapper/119:20:181
768099131,10cyclictest4367-21nscd21:55:120
768099131,10cyclictest4367-21nscd21:55:110
7680991310,2cyclictest11967-21kworker/u8:219:37:040
7680991310,2cyclictest0-21swapper/023:32:520
7680991310,2cyclictest0-21swapper/023:20:400
7680991310,2cyclictest0-21swapper/020:45:290
7680991310,2cyclictest0-21swapper/000:20:170
768099130,2cyclictest0-21swapper/023:45:440
768099130,2cyclictest0-21swapper/023:26:020
768099130,12cyclictest0-21swapper/023:43:150
768099130,12cyclictest0-21swapper/023:10:540
768099130,12cyclictest0-21swapper/021:27:540
768099130,12cyclictest0-21swapper/020:10:140
768099130,12cyclictest0-21swapper/019:15:140
768099130,12cyclictest0-21swapper/000:05:160
3632130,1sleep00-21swapper/021:51:080
99012120,2sleep10-21swapper/119:15:031
768399129,2cyclictest0-21swapper/320:30:013
768399122,7cyclictest0-21swapper/319:54:593
768399120,11cyclictest0-21swapper/322:16:363
768399120,11cyclictest0-21swapper/319:43:403
768299129,2cyclictest9440-21kworker/u8:220:34:062
768299129,2cyclictest19556-21kworker/u8:221:05:052
768299129,2cyclictest0-21swapper/223:13:382
768299129,2cyclictest0-21swapper/222:25:282
768299129,2cyclictest0-21swapper/220:40:252
768299120,2cyclictest30710-21ssh21:15:352
768299120,2cyclictest0-21swapper/223:42:402
768299120,2cyclictest0-21swapper/223:20:022
768299120,2cyclictest0-21swapper/219:18:022
768299120,11cyclictest0-21swapper/222:06:592
768199129,2cyclictest26462-21kworker/u8:023:35:041
768199129,2cyclictest0-21swapper/120:37:391
768199129,2cyclictest0-21swapper/119:55:211
768199129,2cyclictest0-21swapper/119:30:211
768199120,2cyclictest0-21swapper/120:15:131
768199120,1cyclictest0-21swapper/121:46:091
768199120,11cyclictest0-21swapper/122:43:531
768199120,11cyclictest0-21swapper/120:13:391
768199120,11cyclictest0-21swapper/120:05:151
768099129,2cyclictest0-21swapper/023:36:410
768099129,2cyclictest0-21swapper/023:05:220
768099129,2cyclictest0-21swapper/022:25:010
768099129,2cyclictest0-21swapper/021:16:440
768099129,2cyclictest0-21swapper/000:16:340
768099120,2cyclictest19050-21ssh23:16:420
768099120,2cyclictest0-21swapper/021:41:520
768099120,2cyclictest0-21swapper/021:39:050
768099120,2cyclictest0-21swapper/000:35:180
768099120,11cyclictest0-21swapper/023:50:370
768099120,11cyclictest0-21swapper/022:35:180
768099120,11cyclictest0-21swapper/022:30:070
768099120,11cyclictest0-21swapper/021:30:190
768099120,11cyclictest0-21swapper/021:20:030
768099120,11cyclictest0-21swapper/020:55:150
768099120,11cyclictest0-21swapper/020:20:190
768399113,6cyclictest0-21swapper/320:10:233
768399110,6cyclictest0-21swapper/320:20:003
768399110,2cyclictest0-21swapper/320:52:003
768399110,10cyclictest0-21swapper/319:45:283
768399110,10cyclictest0-21swapper/319:23:383
768399110,10cyclictest0-21swapper/319:10:233
768299118,2cyclictest0-21swapper/200:12:382
768299116,3cyclictest278-21jbd2/sda1-819:30:172
768299116,3cyclictest23451-21grep19:45:272
768299116,3cyclictest0-21swapper/200:20:192
768299110,9cyclictest0-21swapper/219:26:042
768299110,2cyclictest0-21swapper/221:13:292
768299110,10cyclictest0-21swapper/222:11:042
768199118,2cyclictest0-21swapper/119:35:251
768199117,2cyclictest7892-21munin-node1
768199110,9cyclictest0-21swapper/120:00:121
768199110,9cyclictest0-21swapper/119:40:211
768199110,3cyclictest0-21swapper/120:45:201
768099118,2cyclictest0-21swapper/020:50:150
768099110,10cyclictest0-21swapper/023:00:010
768399105,3cyclictest14964-21cut20:45:223
768399104,4cyclictest23444-21grep21:05:293
768299107,2cyclictest0-21swapper/220:45:182
768299105,3cyclictest29-21rcuc/221:00:262
768299105,3cyclictest26680-21uname21:45:182
768299100,9cyclictest0-21swapper/223:25:092
768299100,2cyclictest0-21swapper/223:00:372
768299100,2cyclictest0-21swapper/219:13:042
768199107,2cyclictest9440-21kworker/u8:220:32:051
768199105,3cyclictest12713-21wc20:40:161
768099107,2cyclictest32111-21kworker/u8:219:11:040
768099107,2cyclictest11967-21kworker/u8:219:27:040
768099107,2cyclictest0-21swapper/020:35:250
76839994,3cyclictest30196-21aten2.4_r3power20:05:133
76839994,3cyclictest20717-21if_eth021:00:163
76839994,3cyclictest12098-21aten2.4_r3power20:40:143
76839991,6cyclictest11321-21grep19:15:263
76829994,3cyclictest30215-21aten2.4_r3power20:05:132
76829994,3cyclictest23336-21fschecks_time22:15:162
76829994,3cyclictest2103-21fschecks_count20:15:152
76829994,3cyclictest17192-21sh23:48:362
76819990,8cyclictest0-21swapper/120:55:011
76809996,2cyclictest4597-21kworker/u8:019:22:050
76809996,2cyclictest22236-21kworker/u8:219:57:050
76809996,2cyclictest0-21swapper/019:40:170
76809994,3cyclictest21445-21awk21:00:270
76809985,2cyclictest6002-21aten2.4_r3power20:25:120
76809985,2cyclictest22262-21telnet21:05:150
76809985,2cyclictest16018-21kworker/u8:120:01:040
76809985,2cyclictest13014-21pzem_power20:40:210
76809985,2cyclictest0-21swapper/019:30:230
76809984,2cyclictest4829-21Xorg19:51:580
76809984,2cyclictest1701-21aten2.4_r3power20:15:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional