You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-04 - 05:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Thu Dec 04, 2025 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29559211895,18sleep30-21swapper/319:07:383
29732211695,16sleep00-21swapper/019:09:510
296282114103,6sleep10-21swapper/119:08:321
29454210580,19sleep20-21swapper/219:06:172
135692810,1sleep0101ktimersoftd/022:52:590
104432770,1sleep2301ktimersoftd/200:34:002
112402530,1sleep337-21rcuc/322:15:263
2977399230,22cyclictest0-21swapper/319:15:033
304952220,0sleep00-21swapper/023:45:190
2977399221,19cyclictest13717-21diskmemload23:20:123
2977299210,5cyclictest31-21ksoftirqd/222:06:042
2977399202,1cyclictest0-21swapper/323:45:113
29773992015,3cyclictest15743-21ssh00:05:153
2977299200,18cyclictest0-21swapper/222:35:132
29771992017,2cyclictest9912-21kworker/u8:021:07:041
29771992017,2cyclictest22567-21kworker/u8:221:24:061
315642180,0sleep20-21swapper/222:03:112
29772991814,3cyclictest31-21ksoftirqd/200:35:142
29771991815,2cyclictest22089-21kworker/u8:220:32:041
29771991815,2cyclictest20242-21kworker/u8:123:53:041
29771991815,2cyclictest17286-21kworker/u8:121:52:051
29771991815,2cyclictest16041-21kworker/u8:219:59:041
29771991815,2cyclictest14452-21kworker/u8:219:27:051
29771991815,2cyclictest10151-21kworker/u8:222:17:041
2977199172,2cyclictest0-21swapper/123:25:041
29771991714,2cyclictest28764-21kworker/u8:122:42:051
29771991714,2cyclictest27499-21kworker/u8:023:43:041
29771991714,2cyclictest14205-21kworker/u8:121:16:041
2977399162,12cyclictest381ktimersoftd/300:34:203
2977399161,13cyclictest0-21swapper/321:59:513
2977399161,13cyclictest0-21swapper/300:25:173
2977299160,14cyclictest0-21swapper/223:40:232
29771991613,2cyclictest9519-21kworker/u8:219:43:041
29771991613,2cyclictest28051-21kworker/u8:021:26:041
29771991613,2cyclictest27067-21kworker/u8:023:21:051
29771991613,2cyclictest20242-21kworker/u8:100:12:061
29771991613,2cyclictest14205-21kworker/u8:121:38:051
29770991611,3cyclictest0-21swapper/020:00:000
95572150,0sleep10-21swapper/119:38:371
305382150,1sleep1221ktimersoftd/122:01:301
2977399151,12cyclictest13717-21diskmemload00:03:583
2977399151,12cyclictest0-21swapper/300:20:383
29773991510,3cyclictest24047-21ssh22:30:113
2977399150,4cyclictest0-21swapper/322:02:583
29772991511,2cyclictest13717-21diskmemload23:55:132
2977299150,3cyclictest3544-21ssh00:25:322
2977299150,13cyclictest0-21swapper/222:34:522
29771991512,2cyclictest8390-21kworker/u8:223:34:051
29771991512,2cyclictest28764-21kworker/u8:122:48:021
29771991512,2cyclictest28764-21kworker/u8:122:48:011
29771991512,2cyclictest19782-21kworker/u8:223:01:041
29771991512,2cyclictest10151-21kworker/u8:222:38:021
29771991510,3cyclictest0-21swapper/100:22:221
2977099151,12cyclictest13717-21diskmemload21:19:390
29770991510,3cyclictest27320-21ssh21:58:540
2977099150,2cyclictest0-21swapper/022:45:120
2977099150,2cyclictest0-21swapper/022:45:110
2977399149,3cyclictest18982-21seq21:50:103
2977399142,7cyclictest0-21swapper/323:05:003
2977399141,2cyclictest381ktimersoftd/322:20:123
29773991411,2cyclictest39-21ksoftirqd/321:34:493
29773991410,2cyclictest0-21swapper/319:50:113
2977399140,12cyclictest381ktimersoftd/323:52:573
2977399140,12cyclictest0-21swapper/323:06:003
2977399140,12cyclictest0-21swapper/321:43:203
2977399140,12cyclictest0-21swapper/300:15:483
2977299149,3cyclictest1370-21ssh22:40:122
2977299144,6cyclictest272422cat20:20:282
29772991411,2cyclictest0-21swapper/223:13:322
29772991411,2cyclictest0-21swapper/221:28:432
2977299140,2cyclictest1598-21munin-node2
2977299140,2cyclictest0-21swapper/221:32:132
2977299140,2cyclictest0-21swapper/221:12:442
2977299140,2cyclictest0-21swapper/200:03:222
2977299140,13cyclictest0-21swapper/223:16:122
2977299140,13cyclictest0-21swapper/221:20:442
2977299140,12cyclictest22265-21ssh23:36:242
2977299140,12cyclictest1092-21kworker/2:021:38:312
2977299140,12cyclictest0-21swapper/223:26:112
2977199146,5cyclictest0-21swapper/120:55:591
29771991411,2cyclictest9791-21kworker/u8:021:43:041
29771991411,2cyclictest0-21swapper/121:32:261
29771991411,2cyclictest0-21swapper/119:30:171
2977199140,12cyclictest0-21swapper/122:53:061
2977199140,12cyclictest0-21swapper/121:56:021
2977199140,12cyclictest0-21swapper/120:05:011
2977199140,12cyclictest0-21swapper/100:27:421
2977199140,12cyclictest0-21swapper/100:16:211
2977199140,12cyclictest0-21swapper/100:05:281
2977099142,7cyclictest0-21swapper/022:15:010
2977099141,7cyclictest11740-21copy21:05:010
2977099141,11cyclictest13717-21diskmemload23:27:200
29770991410,2cyclictest0-21swapper/022:31:080
2977099140,3cyclictest8092-21ssh21:37:580
2977099140,3cyclictest4616-21ssh22:09:310
2977099140,12cyclictest31944-21ssh22:38:080
2977099140,12cyclictest0-21swapper/023:41:440
2977099140,12cyclictest0-21swapper/023:06:550
2977099140,12cyclictest0-21swapper/023:00:160
2977099140,12cyclictest0-21swapper/022:40:010
2977099140,12cyclictest0-21swapper/021:50:200
2977099140,12cyclictest0-21swapper/019:50:000
2977099140,12cyclictest0-21swapper/019:10:170
2977099140,12cyclictest0-21swapper/000:39:310
2977399138,3cyclictest0-21swapper/321:47:533
2977399132,8cyclictest0-21swapper/319:35:003
2977399131,11cyclictest37-21rcuc/322:45:083
2977399131,11cyclictest37-21rcuc/322:45:073
2977399131,11cyclictest13717-21diskmemload23:32:423
2977399131,11cyclictest13717-21diskmemload22:53:233
2977399131,10cyclictest19823-21pzem_voltage21:15:243
29773991310,2cyclictest0-21swapper/323:40:223
29773991310,2cyclictest0-21swapper/323:35:403
29773991310,2cyclictest0-21swapper/323:11:073
29773991310,2cyclictest0-21swapper/321:00:123
29773991310,2cyclictest0-21swapper/320:50:183
29773991310,2cyclictest0-21swapper/300:11:183
2977399130,2cyclictest0-21swapper/322:09:213
2977399130,12cyclictest0-21swapper/321:38:473
2977399130,12cyclictest0-21swapper/321:11:183
2977399130,12cyclictest0-21swapper/320:15:173
2977399130,11cyclictest0-21swapper/300:37:173
29772991311,1cyclictest0-21swapper/221:56:132
29772991310,2cyclictest13717-21diskmemload23:32:022
29772991310,2cyclictest0-21swapper/223:47:322
29772991310,2cyclictest0-21swapper/223:20:122
29772991310,2cyclictest0-21swapper/223:00:122
29772991310,2cyclictest0-21swapper/222:23:272
29772991310,2cyclictest0-21swapper/222:17:412
29772991310,2cyclictest0-21swapper/221:17:462
29772991310,2cyclictest0-21swapper/220:50:122
29772991310,2cyclictest0-21swapper/200:22:312
2977299130,2cyclictest13717-21diskmemload22:45:182
2977299130,2cyclictest13717-21diskmemload22:45:172
2977299130,2cyclictest0-21swapper/221:52:462
2977299130,2cyclictest0-21swapper/220:15:142
2977299130,2cyclictest0-21swapper/200:15:142
2977299130,1cyclictest0-21swapper/200:08:232
2977299130,12cyclictest0-21swapper/222:52:532
2977299130,12cyclictest0-21swapper/221:43:262
2977299130,12cyclictest0-21swapper/200:11:322
2977199132,8cyclictest0-21swapper/120:18:021
2977199131,11cyclictest0-21swapper/123:35:061
29771991310,2cyclictest10151-21kworker/u8:222:25:051
29771991310,2cyclictest0-21swapper/123:14:131
29771991310,2cyclictest0-21swapper/122:34:031
29771991310,2cyclictest0-21swapper/122:06:061
29771991310,2cyclictest0-21swapper/121:45:441
2977199130,2cyclictest0-21swapper/122:20:541
2977199130,2cyclictest0-21swapper/120:20:161
2977199130,2cyclictest0-21swapper/100:00:251
2977199130,12cyclictest13717-21diskmemload22:11:551
2977199130,12cyclictest13717-21diskmemload00:36:221
2977199130,12cyclictest0-21swapper/122:59:121
2977199130,12cyclictest0-21swapper/121:14:141
2977199130,12cyclictest0-21swapper/120:40:131
2977199130,12cyclictest0-21swapper/120:35:271
2977199130,11cyclictest13717-21diskmemload00:30:031
2977099138,3cyclictest6741-21ssh00:30:110
2977099138,3cyclictest24009-21ssh23:39:170
2977099132,6cyclictest0-21swapper/023:20:000
2977099132,6cyclictest0-21swapper/023:10:010
2977099132,6cyclictest0-21swapper/019:40:000
2977099132,6cyclictest0-21swapper/019:20:010
29770991310,2cyclictest13717-21diskmemload23:50:070
29770991310,2cyclictest13717-21diskmemload22:27:540
29770991310,2cyclictest13717-21diskmemload21:20:030
29770991310,2cyclictest0-21swapper/022:20:480
29770991310,2cyclictest0-21swapper/021:42:010
29770991310,2cyclictest0-21swapper/021:30:260
29770991310,2cyclictest0-21swapper/020:10:260
29770991310,2cyclictest0-21swapper/000:11:250
2977099130,2cyclictest0-21swapper/019:30:190
2977099130,2cyclictest0-21swapper/000:06:500
2977099130,2cyclictest0-21swapper/000:00:050
2977099130,12cyclictest0-21swapper/023:20:010
2977099130,12cyclictest0-21swapper/022:55:480
2977099130,12cyclictest0-21swapper/022:15:380
2977099130,12cyclictest0-21swapper/021:25:080
2977099130,11cyclictest0-21swapper/000:15:030
2977399129,2cyclictest0-21swapper/323:25:103
2977399129,2cyclictest0-21swapper/322:25:103
2977399129,2cyclictest0-21swapper/321:26:083
2977399129,2cyclictest0-21swapper/320:35:193
2977399129,2cyclictest0-21swapper/320:31:383
2977399123,6cyclictest0-21swapper/319:55:243
2977399122,7cyclictest0-21swapper/320:24:583
2977399122,7cyclictest0-21swapper/319:23:593
2977399122,5cyclictest0-21swapper/319:30:013
2977399120,2cyclictest0-21swapper/323:55:483
2977399120,2cyclictest0-21swapper/323:15:583
2977399120,2cyclictest0-21swapper/322:55:203
2977399120,2cyclictest0-21swapper/321:20:513
2977399120,2cyclictest0-21swapper/320:25:193
2977399120,11cyclictest0-21swapper/322:43:073
2977399120,11cyclictest0-21swapper/322:11:293
2977399120,11cyclictest0-21swapper/321:05:083
2977399120,10cyclictest0-21swapper/322:39:513
2977299129,2cyclictest13717-21diskmemload22:25:412
2977299129,2cyclictest0-21swapper/223:05:172
2977299129,2cyclictest0-21swapper/220:05:002
2977299129,2cyclictest0-21swapper/219:40:262
2977299122,7cyclictest0-21swapper/221:01:582
2977299120,2cyclictest0-21swapper/219:35:172
2977299120,1cyclictest0-21swapper/222:56:132
2977299120,11cyclictest0-21swapper/222:10:052
2977299120,11cyclictest0-21swapper/221:46:382
2977299120,11cyclictest0-21swapper/220:55:182
2977299120,11cyclictest0-21swapper/220:35:132
2977299120,11cyclictest0-21swapper/219:25:232
2977199129,2cyclictest7453-21kworker/u8:221:04:041
2977199129,2cyclictest0-21swapper/123:05:231
2977199129,2cyclictest0-21swapper/119:50:161
2977199129,2cyclictest0-21swapper/119:20:161
2977199120,2cyclictest0-21swapper/123:55:241
2977199120,2cyclictest0-21swapper/123:15:531
2977199120,2cyclictest0-21swapper/120:50:261
2977199120,11cyclictest0-21swapper/123:45:241
2977199120,11cyclictest0-21swapper/119:45:221
2977199120,11cyclictest0-21swapper/119:10:131
2977099129,2cyclictest13717-21diskmemload21:45:510
2977099129,2cyclictest0-21swapper/023:55:310
2977099129,2cyclictest0-21swapper/020:45:160
2977099129,2cyclictest0-21swapper/020:45:150
2977099129,2cyclictest0-21swapper/020:15:260
2977099129,2cyclictest0-21swapper/000:20:350
2977099122,7cyclictest0-21swapper/020:22:580
2977099122,7cyclictest0-21swapper/019:22:010
2977099120,11cyclictest11740-21copy21:05:000
2977099120,11cyclictest0-21swapper/023:30:200
2977099120,11cyclictest0-21swapper/022:00:010
2977099120,11cyclictest0-21swapper/021:12:090
2977099120,11cyclictest0-21swapper/020:55:200
2977099120,11cyclictest0-21swapper/019:40:230
2977099120,11cyclictest0-21swapper/000:26:230
2977399110,9cyclictest381ktimersoftd/320:05:203
2977299118,2cyclictest0-21swapper/219:59:362
2977299118,2cyclictest0-21swapper/219:45:222
2977299118,2cyclictest0-21swapper/219:30:132
2977299118,2cyclictest0-21swapper/219:20:182
2977299112,6cyclictest0-21swapper/219:15:582
2977299110,3cyclictest0-21swapper/221:05:142
2977199118,2cyclictest23620-21kworker/u8:019:15:051
2977199118,2cyclictest19683-21kworker/u8:120:13:041
2977199110,10cyclictest0-21swapper/120:05:181
2977099112,7cyclictest0-21swapper/020:43:020
2977399105,3cyclictest381ktimersoftd/320:12:033
2977399102,6cyclictest0-21swapper/319:37:493
2977399101,7cyclictest0-21swapper/319:46:023
2977299107,2cyclictest0-21swapper/219:50:142
2977299106,3cyclictest0-21swapper/220:26:592
2977299102,6cyclictest0-21swapper/220:00:252
2977199107,2cyclictest0-21swapper/120:45:121
2977199107,2cyclictest0-21swapper/120:45:111
2977099102,6cyclictest0-21swapper/020:51:380
2977099101,5cyclictest0-21swapper/020:05:140
2977099100,8cyclictest0-21swapper/019:25:110
297739994,3cyclictest8671-21missed_timers20:55:193
297739994,3cyclictest381ktimersoftd/319:41:003
297739994,3cyclictest19951-21latency_hist20:05:013
297739991,6cyclictest0-21swapper/320:45:233
297739991,6cyclictest0-21swapper/320:45:223
297739991,3cyclictest0-21swapper/320:40:123
297729996,2cyclictest29-21rcuc/220:10:122
297729994,3cyclictest3950-21tr20:45:132
297729994,3cyclictest3950-21tr20:45:122
297729994,3cyclictest31182-21grep19:10:252
297729990,2cyclictest0-21swapper/220:40:122
297729990,2cyclictest0-21swapper/220:30:232
297719996,2cyclictest26168-21kworker/u8:020:27:051
297709994,3cyclictest31179-21swap20:30:240
297709994,3cyclictest29169-21switchtime20:25:230
297709992,5cyclictest0-21swapper/020:00:010
297709991,6cyclictest14459-21grep19:50:150
297739983,3cyclictest0-21swapper/319:10:263
297709980,3cyclictest278-21jbd2/sda1-820:35:060
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional