You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-14 - 21:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot6s.osadl.org (updated Tue Oct 14, 2025 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23244211695,14sleep30-21swapper/307:05:183
24083211497,12sleep10-21swapper/107:06:411
24303211293,12sleep20-21swapper/207:09:302
24112210477,22sleep00-21swapper/007:07:040
311122700,0sleep30-21swapper/310:39:393
27822700,0sleep20-21swapper/209:36:172
320322690,1sleep30-21swapper/308:45:303
292152250,0sleep30-21swapper/312:17:003
118102210,0sleep10-21swapper/110:19:451
8372200,1sleep3381ktimersoftd/311:15:103
286462200,1sleep20-21swapper/212:16:072
24373992017,2cyclictest0-21swapper/307:15:113
24373992015,3cyclictest17120-21ssh12:05:133
155802200,1sleep315583-21aten2.4_r3power09:50:143
24370991916,2cyclictest561-21kworker/u8:209:15:050
24370991916,2cyclictest561-21kworker/u8:209:15:040
2437399181,15cyclictest0-21swapper/312:25:153
2437399170,15cyclictest0-21swapper/309:10:123
24370991714,2cyclictest9034-21kworker/u8:110:22:050
2437399160,14cyclictest0-21swapper/310:30:163
24372991613,2cyclictest8864-21kworker/u8:009:14:062
24370991613,2cyclictest17370-21kworker/u8:011:31:040
24373991510,3cyclictest0-21swapper/312:34:043
2437399150,13cyclictest0-21swapper/309:20:563
2437399150,13cyclictest0-21swapper/309:20:553
24372991512,2cyclictest0-21swapper/211:10:442
24372991512,2cyclictest0-21swapper/210:25:232
24372991512,2cyclictest0-21swapper/209:23:042
24372991512,2cyclictest0-21swapper/209:23:032
24372991512,2cyclictest0-21swapper/208:20:242
2437099151,3cyclictest15073-21ssh12:02:200
2437099151,3cyclictest15073-21ssh12:02:190
24370991511,2cyclictest27153-21perl07:15:140
2437099151,12cyclictest15006-21ssh11:29:390
2437099150,13cyclictest14075-21grep12:35:160
2437099150,13cyclictest0-21swapper/007:30:270
2437399141,2cyclictest1438-21sed07:30:283
2437399141,11cyclictest5129-21runrttasks10:07:213
2437399140,12cyclictest972-21ssh10:40:413
2437399140,12cyclictest0-21swapper/310:59:553
2437399140,12cyclictest0-21swapper/309:55:553
2437399140,12cyclictest0-21swapper/309:45:073
24372991411,2cyclictest8424-21diskmemload12:13:032
24372991411,2cyclictest8424-21diskmemload11:16:132
24372991411,2cyclictest0-21swapper/212:08:342
24372991411,2cyclictest0-21swapper/210:21:442
24372991411,2cyclictest0-21swapper/207:20:142
24372991410,2cyclictest0-21swapper/211:53:072
2437299140,3cyclictest0-21swapper/209:50:152
2437299140,2cyclictest0-21swapper/212:21:442
2437299140,2cyclictest0-21swapper/211:45:542
2437299140,2cyclictest0-21swapper/210:15:132
2437299140,13cyclictest8424-21diskmemload11:07:042
2437299140,13cyclictest0-21swapper/212:00:242
2437299140,13cyclictest0-21swapper/212:00:232
2437299140,12cyclictest8424-21diskmemload10:51:592
2437299140,12cyclictest8424-21diskmemload09:45:342
2437299140,12cyclictest25864-21iostat_ios10:00:152
2437299140,12cyclictest1657-21sh10:09:232
2437299140,12cyclictest0-21swapper/209:55:052
2437299140,12cyclictest0-21swapper/209:33:322
2437299140,12cyclictest0-21swapper/209:15:052
2437299140,12cyclictest0-21swapper/209:15:042
2437199141,2cyclictest8424-21diskmemload10:45:331
2437199140,2cyclictest0-21swapper/112:15:441
24370991411,2cyclictest18096-21apt-get09:20:070
24370991411,2cyclictest18096-21apt-get09:20:060
24370991411,2cyclictest0-21swapper/009:10:310
2437099140,3cyclictest0-21swapper/011:50:210
2437099140,3cyclictest0-21swapper/009:25:220
2437099140,3cyclictest0-21swapper/009:25:210
2437099140,2cyclictest0-21swapper/012:05:010
2437099140,12cyclictest0-21swapper/009:40:470
142312140,3sleep214384-21fschecks_count10:55:152
90242130,2sleep00-21swapper/010:50:080
76522130,2sleep00-21swapper/010:15:120
2437399139,2cyclictest0-21swapper/311:57:153
24373991310,2cyclictest0-21swapper/312:39:343
24373991310,2cyclictest0-21swapper/312:22:043
24373991310,2cyclictest0-21swapper/310:45:143
24373991310,2cyclictest0-21swapper/309:40:453
24373991310,2cyclictest0-21swapper/308:00:153
2437399130,2cyclictest0-21swapper/311:47:453
2437399130,2cyclictest0-21swapper/311:01:043
2437399130,12cyclictest0-21swapper/310:25:443
2437399130,11cyclictest0-21swapper/311:43:543
24372991310,2cyclictest0-21swapper/212:35:322
24372991310,2cyclictest0-21swapper/212:25:002
24372991310,2cyclictest0-21swapper/211:40:382
24372991310,2cyclictest0-21swapper/211:30:312
24372991310,2cyclictest0-21swapper/211:25:092
24372991310,2cyclictest0-21swapper/211:00:162
24372991310,2cyclictest0-21swapper/210:45:342
24372991310,2cyclictest0-21swapper/210:40:202
24372991310,2cyclictest0-21swapper/209:27:382
24372991310,2cyclictest0-21swapper/209:27:372
24372991310,2cyclictest0-21swapper/208:40:152
2437299130,2cyclictest28760-21ssh10:36:012
2437299130,12cyclictest8424-21diskmemload10:11:502
2437299130,12cyclictest0-21swapper/212:31:442
2437299130,12cyclictest0-21swapper/211:55:222
2437299130,12cyclictest0-21swapper/211:35:152
2437299130,12cyclictest0-21swapper/211:20:332
2437299130,12cyclictest0-21swapper/210:32:442
2437199131,10cyclictest30938-21cpuspeed_turbos07:25:121
24371991310,2cyclictest0-21swapper/112:23:531
24371991310,2cyclictest0-21swapper/112:13:131
24371991310,2cyclictest0-21swapper/111:55:521
24371991310,2cyclictest0-21swapper/111:46:331
24371991310,2cyclictest0-21swapper/111:26:531
24371991310,2cyclictest0-21swapper/111:05:221
24371991310,2cyclictest0-21swapper/110:53:021
24371991310,2cyclictest0-21swapper/110:27:031
24371991310,2cyclictest0-21swapper/109:20:321
24371991310,2cyclictest0-21swapper/109:20:321
24371991310,2cyclictest0-21swapper/109:14:331
2437199130,2cyclictest0-21swapper/111:11:431
2437199130,2cyclictest0-21swapper/110:33:231
2437199130,2cyclictest0-21swapper/110:10:141
2437199130,12cyclictest0-21swapper/112:32:121
2437199130,12cyclictest0-21swapper/112:26:321
2437199130,12cyclictest0-21swapper/111:44:021
2437199130,11cyclictest0-21swapper/112:05:221
2437199130,11cyclictest0-21swapper/107:30:191
24370991310,2cyclictest8424-21diskmemload09:30:310
24370991310,2cyclictest18266-21ssh10:59:460
24370991310,2cyclictest0-21swapper/011:56:060
24370991310,2cyclictest0-21swapper/011:44:360
24370991310,2cyclictest0-21swapper/011:20:580
24370991310,2cyclictest0-21swapper/011:07:590
24370991310,2cyclictest0-21swapper/010:26:160
24370991310,2cyclictest0-21swapper/010:11:090
24370991310,2cyclictest0-21swapper/009:55:390
24370991310,2cyclictest0-21swapper/009:46:300
24370991310,2cyclictest0-21swapper/009:35:070
2437099130,2cyclictest0-21swapper/012:20:170
2437099130,2cyclictest0-21swapper/011:37:370
2437099130,2cyclictest0-21swapper/011:20:000
2437099130,2cyclictest0-21swapper/010:01:190
2437099130,2cyclictest0-21swapper/008:50:170
2437099130,12cyclictest8424-21diskmemload11:12:190
2437099130,12cyclictest8424-21diskmemload09:50:060
2437099130,12cyclictest24465-21ssh10:31:510
2437099130,12cyclictest0-21swapper/012:25:250
2437099130,12cyclictest0-21swapper/012:19:090
2437099130,12cyclictest0-21swapper/012:10:190
2437099130,12cyclictest0-21swapper/010:45:460
2437099130,12cyclictest0-21swapper/010:40:280
2437099130,12cyclictest0-21swapper/010:35:280
2437099130,12cyclictest0-21swapper/010:06:170
2437099130,12cyclictest0-21swapper/008:00:200
2437099130,12cyclictest0-21swapper/007:40:260
177192130,2sleep18424-21diskmemload11:31:211
36072120,2sleep10-21swapper/111:50:221
30432120,2sleep30-21swapper/309:36:433
2437399129,2cyclictest0-21swapper/312:13:133
2437399129,2cyclictest0-21swapper/312:00:243
2437399129,2cyclictest0-21swapper/312:00:243
2437399129,2cyclictest0-21swapper/311:53:043
2437399129,2cyclictest0-21swapper/311:31:453
2437399129,2cyclictest0-21swapper/310:20:553
2437399129,2cyclictest0-21swapper/310:03:153
2437399129,2cyclictest0-21swapper/309:25:453
2437399129,2cyclictest0-21swapper/309:25:453
2437399129,2cyclictest0-21swapper/307:25:263
2437399122,7cyclictest0-21swapper/308:31:023
2437399122,7cyclictest0-21swapper/308:25:003
2437399122,7cyclictest0-21swapper/307:48:013
2437399120,7cyclictest0-21swapper/308:45:003
2437399120,6cyclictest0-21swapper/308:30:003
2437399120,2cyclictest0-21swapper/310:15:053
2437399120,11cyclictest0-21swapper/311:07:153
2437399120,11cyclictest0-21swapper/310:12:263
2437399120,11cyclictest0-21swapper/307:40:263
2437399120,10cyclictest2851-21kworker/u8:111:20:293
2437299129,2cyclictest0-21swapper/208:30:172
2437299129,2cyclictest0-21swapper/207:55:182
2437299120,2cyclictest0-21swapper/209:40:282
2437299120,1cyclictest0-21swapper/208:35:142
2437299120,1cyclictest0-21swapper/207:50:162
2437299120,1cyclictest0-21swapper/207:45:272
2437299120,11cyclictest0-21swapper/209:00:132
2437299120,11cyclictest0-21swapper/207:15:252
2437199129,2cyclictest8424-21diskmemload11:15:311
2437199129,2cyclictest0-21swapper/110:42:011
2437199129,2cyclictest0-21swapper/110:05:341
2437199127,3cyclictest0-21swapper/110:55:171
2437199122,7cyclictest5911-21gdm-simple-gree09:44:591
2437199122,7cyclictest0-21swapper/108:59:591
2437199120,2cyclictest0-21swapper/112:02:331
2437199120,2cyclictest0-21swapper/112:02:321
2437199120,2cyclictest0-21swapper/109:50:431
2437199120,2cyclictest0-21swapper/108:45:231
2437199120,2cyclictest0-21swapper/108:15:231
2437199120,2cyclictest0-21swapper/107:50:201
2437199120,2cyclictest0-21swapper/107:10:251
2437199120,11cyclictest0-21swapper/112:37:321
2437199120,11cyclictest0-21swapper/111:00:321
2437199120,11cyclictest0-21swapper/110:39:231
2437199120,11cyclictest0-21swapper/110:03:421
2437199120,11cyclictest0-21swapper/109:15:521
2437199120,11cyclictest0-21swapper/109:15:511
2437199120,11cyclictest0-21swapper/109:09:531
2437199120,11cyclictest0-21swapper/108:50:141
2437199120,10cyclictest0-21swapper/110:20:221
2437199120,10cyclictest0-21swapper/109:30:221
2437199120,10cyclictest0-21swapper/108:40:261
2437199120,10cyclictest0-21swapper/107:35:131
2437099129,2cyclictest0-21swapper/011:01:260
2437099129,2cyclictest0-21swapper/009:05:150
2437099129,2cyclictest0-21swapper/007:10:120
2437099121,9cyclictest0-21swapper/007:20:160
2437099120,11cyclictest0-21swapper/012:31:280
2437099120,11cyclictest0-21swapper/011:47:530
2437099120,11cyclictest0-21swapper/008:35:200
2437099120,11cyclictest0-21swapper/007:50:150
2437099120,11cyclictest0-21swapper/007:35:220
312492110,2sleep30-21swapper/309:32:393
2437399118,2cyclictest0-21swapper/311:35:083
2437399118,2cyclictest0-21swapper/311:12:243
2437399118,2cyclictest0-21swapper/309:15:353
2437399118,2cyclictest0-21swapper/309:15:353
2437399117,2cyclictest0-21swapper/307:10:113
2437399112,6cyclictest0-21swapper/308:11:013
2437399112,6cyclictest0-21swapper/307:50:003
2437399111,5cyclictest0-21swapper/308:39:593
2437399110,10cyclictest0-21swapper/311:26:333
2437399110,10cyclictest0-21swapper/310:52:153
2437299118,2cyclictest0-21swapper/207:25:142
2437299110,2cyclictest0-21swapper/207:41:442
2437299110,1cyclictest0-21swapper/209:05:172
2437199118,2cyclictest0-21swapper/107:55:211
2437199118,2cyclictest0-21swapper/107:15:241
2437199117,2cyclictest0-21swapper/109:00:141
2437199117,2cyclictest0-21swapper/108:20:131
2437199113,6cyclictest0-21swapper/107:48:441
2437199112,6cyclictest0-21swapper/108:37:591
2437199112,6cyclictest0-21swapper/108:07:591
2437199112,2cyclictest0-21swapper/109:35:251
2437199110,9cyclictest10769-21aten2.4-expect09:45:161
2437199110,9cyclictest0-21swapper/108:10:161
2437199110,9cyclictest0-21swapper/107:20:131
2437199110,10cyclictest0-21swapper/111:35:441
2437199110,10cyclictest0-21swapper/108:33:441
2437099118,2cyclictest6172-21kworker/u8:108:43:040
2437099118,2cyclictest0-21swapper/008:15:180
2437099111,8cyclictest6019-21grep09:00:240
2437099111,4cyclictest26592-21cron08:35:000
222062110,2sleep18424-21diskmemload09:56:021
74972100,2sleep10-21swapper/111:20:321
2437399105,3cyclictest0-21swapper/309:00:163
2437399103,1cyclictest0-21swapper/307:38:043
2437399102,2cyclictest0-21swapper/308:16:053
2437399100,8cyclictest0-21swapper/308:55:003
2437299107,2cyclictest0-21swapper/208:11:442
2437299100,9cyclictest0-21swapper/208:17:012
2437299100,9cyclictest0-21swapper/207:36:512
2437299100,8cyclictest5010-21snmpd08:48:562
2437199107,2cyclictest0-21swapper/109:25:031
2437199107,2cyclictest0-21swapper/109:25:021
2437199100,9cyclictest0-21swapper/108:00:151
2437199100,8cyclictest0-21swapper/107:40:141
2437199100,7cyclictest0-21swapper/108:25:011
2437099100,2cyclictest0-21swapper/008:48:560
243739994,3cyclictest7047-21fschecks_count09:05:153
243739994,3cyclictest2416-21munin-node3
243739994,3cyclictest0-21swapper/308:05:143
243739993,2cyclictest0-21swapper/307:55:053
243729996,2cyclictest0-21swapper/208:08:412
243729994,3cyclictest1249-21grep08:50:172
243729990,1cyclictest0-21swapper/208:01:412
243709996,2cyclictest0-21swapper/008:27:080
243709996,2cyclictest0-21swapper/008:05:260
243709994,3cyclictest16083-1kworker/0:2H07:29:590
243709990,8cyclictest0-21swapper/007:59:410
1962290,2sleep20-21swapper/207:34:342
18222290,2sleep00-21swapper/008:11:140
243739981,3cyclictest29023-21diskstats07:20:143
243729985,2cyclictest0-21swapper/208:25:132
243729980,7cyclictest0-21swapper/208:57:172
243729980,7cyclictest0-21swapper/207:10:032
243709983,3cyclictest40252chrt08:57:400
243709983,3cyclictest22297-21chrt08:21:390
243709981,6cyclictest30906-21kworker/0:007:45:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional