You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-07 - 09:12

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot6s.osadl.org (updated Fri Nov 07, 2025 00:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
854299944,81cyclictest0-21swapper/020:51:370
854299893,69cyclictest27319-21fschecks_time20:36:200
854299874,65cyclictest0-21swapper/018:41:250
854299863,65cyclictest0-21swapper/018:46:420
854299853,63cyclictest1160-21snmpd21:26:190
854299844,72cyclictest0-21swapper/018:51:210
854299844,72cyclictest0-21swapper/018:51:210
854299844,47cyclictest0-21swapper/016:41:410
854299844,47cyclictest0-21swapper/016:41:410
854299843,60cyclictest21735-21cat21:31:260
854299834,72cyclictest0-21swapper/017:06:440
854299834,71cyclictest0-21swapper/016:46:240
854299833,67cyclictest32739-21runrttasks18:31:340
854299824,58cyclictest0-21swapper/018:56:180
854299823,72cyclictest31154-21hwlatdetect17:21:320
854299823,72cyclictest31154-21hwlatdetect17:21:320
854299823,64cyclictest0-21swapper/017:56:230
854299823,60cyclictest0-21swapper/018:37:360
854299823,57cyclictest0-21swapper/017:16:310
854299815,69cyclictest0-21swapper/020:35:340
854299815,68cyclictest0-21swapper/021:11:340
854299814,69cyclictest23286-21python18:11:390
854299814,61cyclictest24745-21uptime21:36:350
854299813,71cyclictest0-21swapper/018:21:320
854299813,70cyclictest0-21swapper/019:51:250
854299813,63cyclictest0-21swapper/020:22:520
854299813,63cyclictest0-21swapper/020:22:520
854299804,69cyclictest0-21swapper/021:51:330
854299803,60cyclictest19801-21wc19:11:410
854299796,67cyclictest25445-21telnet21:41:150
854299796,67cyclictest25445-21telnet21:41:150
854299794,68cyclictest0-21swapper/021:16:130
854299794,68cyclictest0-21swapper/017:41:320
854299793,69cyclictest18468-21chrt16:56:260
854299793,69cyclictest15842-21aten_r1power_en21:21:120
854299788,41cyclictest0-21swapper/017:36:090
854299785,66cyclictest0-21swapper/017:36:200
854299784,68cyclictest0-21swapper/019:16:470
854299783,70cyclictest24255-21aten2.4-expect18:16:220
854299783,68cyclictest1160-21snmpd19:37:460
854299783,68cyclictest1160-21snmpd19:37:460
854299783,68cyclictest1160-21snmpd19:27:440
854299783,68cyclictest0-21swapper/018:06:450
854299783,68cyclictest0-21swapper/018:06:450
854299783,60cyclictest0-21swapper/016:52:170
854299783,57cyclictest0-21swapper/016:36:320
854299783,45cyclictest12939-21tune2fs20:06:200
8542997816,56cyclictest32620-21munin-node17:26:380
8542997816,56cyclictest32620-21munin-node17:26:380
854299775,55cyclictest0-21swapper/021:56:220
854299774,66cyclictest0-21swapper/020:41:230
854299774,66cyclictest0-21swapper/019:46:510
854299774,66cyclictest0-21swapper/017:11:290
854299773,68cyclictest0-21swapper/019:56:160
854299773,67cyclictest0-21swapper/019:01:380
854299773,67cyclictest0-21swapper/017:01:280
854299773,62cyclictest3450-21sed22:01:200
854299773,59cyclictest0-21swapper/019:21:170
854299764,61cyclictest0-21swapper/017:51:260
854299764,57cyclictest0-21swapper/019:41:150
854299763,67cyclictest0-21swapper/020:02:500
854299763,66cyclictest0-21swapper/020:16:280
854299763,56cyclictest8370-21latency_hist21:06:010
854299763,56cyclictest8370-21latency_hist21:06:010
854299762,68cyclictest0-21swapper/020:12:510
854299754,57cyclictest0-21swapper/021:10:150
854299753,66cyclictest1160-21snmpd19:32:450
854299753,53cyclictest0-21swapper/020:26:120
854299753,47cyclictest0-21swapper/021:48:060
854299753,47cyclictest0-21swapper/021:48:060
854299744,51cyclictest0-21swapper/018:26:190
8542997417,36cyclictest0-21swapper/017:46:430
854299733,63cyclictest0-21swapper/020:56:210
854299733,55cyclictest0-21swapper/018:01:210
854299727,59cyclictest16528-21fschecks_time19:06:240
854299723,53cyclictest0-21swapper/020:46:360
77552688,50sleep30-21swapper/316:31:373
854299644,52cyclictest0-21swapper/016:36:110
854799624,51cyclictest0-21swapper/321:36:303
854799604,49cyclictest0-21swapper/319:16:043
8545996023,30cyclictest19711-21rt-features16:56:432
854799592,47cyclictest0-21swapper/319:31:173
8545995931,10cyclictest22840-21latency_hist17:06:102
8545995829,10cyclictest25230-21latency_hist17:11:102
223099580,4rtkit-daemon7830-21grep17:41:251
101002583,48sleep10-21swapper/116:36:411
8543995737,12cyclictest17482-21unixbench_multi21:21:341
223099570,6rtkit-daemon2229-21rtkit-daemon19:35:512
152582573,47sleep115259-21yum19:01:391
854799563,47cyclictest0-21swapper/320:46:203
854799554,44cyclictest0-21swapper/317:48:483
854799553,46cyclictest0-21swapper/318:01:373
854399559,24cyclictest31106-21awk20:46:011
48482555,12sleep20-21swapper/217:35:392
223099550,6rtkit-daemon2229-21rtkit-daemon21:16:172
223099550,6rtkit-daemon2229-21rtkit-daemon21:11:172
223099550,5rtkit-daemon2229-21rtkit-daemon20:41:182
8547995412,35cyclictest0-21swapper/319:41:173
854399545,29cyclictest0-21swapper/118:51:401
854399545,29cyclictest0-21swapper/118:51:401
854399545,28cyclictest0-21swapper/121:26:211
854399544,31cyclictest0-21swapper/116:56:351
854399544,29cyclictest0-21swapper/119:41:291
63232543,11sleep10-21swapper/117:36:351
320152542,9sleep30-21swapper/319:37:003
320152542,9sleep30-21swapper/319:37:003
227052543,8sleep122783-21iostat20:26:231
223099540,6rtkit-daemon2229-21rtkit-daemon20:05:122
223099540,5rtkit-daemon2229-21rtkit-daemon19:21:212
223099540,5rtkit-daemon2229-21rtkit-daemon19:06:302
223099540,5rtkit-daemon2229-21rtkit-daemon19:01:222
854799534,23cyclictest16494-21grep19:06:243
854599536,40cyclictest0-21swapper/218:01:422
854399535,28cyclictest0-21swapper/118:56:421
854399534,29cyclictest0-21swapper/119:51:391
854399534,29cyclictest0-21swapper/118:26:401
854399533,44cyclictest0-21swapper/116:51:381
854399533,11cyclictest0-21swapper/121:01:161
854399533,11cyclictest0-21swapper/121:01:161
8543995321,13cyclictest0-21swapper/119:46:311
8543995320,14cyclictest0-21swapper/119:36:251
8543995320,14cyclictest0-21swapper/119:36:251
279542533,43sleep128077-21cpuspeed_turbos19:31:181
223099530,5rtkit-daemon2229-21rtkit-daemon20:21:272
223099530,5rtkit-daemon2229-21rtkit-daemon20:21:272
223099530,5rtkit-daemon2229-21rtkit-daemon20:11:342
854799524,42cyclictest0-21swapper/320:11:343
8547995236,9cyclictest6618-21tr18:46:203
8545995237,10cyclictest0-21swapper/220:36:292
854399525,28cyclictest0-21swapper/118:06:421
854399525,28cyclictest0-21swapper/118:06:421
854399524,42cyclictest0-21swapper/121:16:281
854399524,30cyclictest0-21swapper/120:16:301
854399524,29cyclictest0-21swapper/120:31:271
854399524,29cyclictest0-21swapper/117:56:341
854399524,28cyclictest0-21swapper/122:01:001
854399524,28cyclictest0-21swapper/118:40:051
854399524,28cyclictest0-21swapper/118:16:221
854399524,28cyclictest0-21swapper/117:16:271
854399523,41cyclictest0-21swapper/120:04:451
854399523,29cyclictest0-21swapper/120:51:331
8543995217,13cyclictest0-21swapper/120:21:211
8543995217,13cyclictest0-21swapper/120:21:211
75442522,10sleep10-21swapper/119:56:171
44552522,10sleep30-21swapper/319:46:393
299192522,10sleep30-21swapper/317:20:123
284222522,44sleep10-21swapper/120:37:191
223099520,5rtkit-daemon2229-21rtkit-daemon19:16:372
223099520,4rtkit-daemon2229-21rtkit-daemon20:51:252
106022522,11sleep10-21swapper/121:09:031
854799515,27cyclictest0-21swapper/317:56:233
8547995121,13cyclictest0-21swapper/319:22:423
854799512,43cyclictest0-21swapper/320:56:193
854799512,43cyclictest0-21swapper/320:31:303
854799512,43cyclictest0-21swapper/319:01:153
854799512,42cyclictest0-21swapper/317:01:313
854599513,41cyclictest0-21swapper/217:56:322
854399514,29cyclictest0-21swapper/120:46:151
854399514,29cyclictest0-21swapper/118:46:291
854399514,28cyclictest0-21swapper/120:57:581
854399514,28cyclictest0-21swapper/119:11:381
854399514,28cyclictest0-21swapper/119:06:391
8543995134,11cyclictest4205-21wc22:01:331
8543995120,13cyclictest0-21swapper/116:41:211
8543995120,13cyclictest0-21swapper/116:41:211
8543995120,10cyclictest14981-21smartctl16:46:441
8543995118,14cyclictest0-21swapper/121:46:191
8543995118,14cyclictest0-21swapper/121:46:191
223099510,6rtkit-daemon2229-21rtkit-daemon20:31:342
223099510,5rtkit-daemon2229-21rtkit-daemon20:26:262
142802512,10sleep10-21swapper/120:09:511
108072512,10sleep30-21swapper/316:39:543
854799505,39cyclictest0-21swapper/321:16:003
854799504,29cyclictest0-21swapper/317:27:233
854799504,29cyclictest0-21swapper/317:27:233
854799504,27cyclictest0-21swapper/318:41:383
854799503,41cyclictest0-21swapper/320:41:303
854799503,29cyclictest291032cut18:26:203
854799502,43cyclictest0-21swapper/318:15:433
854799502,43cyclictest0-21swapper/317:11:213
854799502,42cyclictest0-21swapper/322:01:333
8547995018,14cyclictest0-21swapper/320:37:553
8547995016,27cyclictest22533-21taskset20:26:203
854599504,40cyclictest0-21swapper/217:26:212
854599504,40cyclictest0-21swapper/217:26:212
854599503,39cyclictest0-21swapper/218:21:312
8545995025,7cyclictest23993-21latency_hist18:16:082
854599502,42cyclictest0-21swapper/218:47:072
854399505,38cyclictest0-21swapper/120:11:381
854399504,28cyclictest0-21swapper/119:16:181
854399504,27cyclictest0-21swapper/118:22:331
854399504,27cyclictest0-21swapper/117:21:381
854399504,27cyclictest0-21swapper/117:21:381
8543995016,16cyclictest0-21swapper/119:27:151
8543995012,23cyclictest17666-21fschecks_count18:01:281
284772502,10sleep30-21swapper/318:22:323
223099500,6rtkit-daemon2229-21rtkit-daemon21:46:232
223099500,6rtkit-daemon2229-21rtkit-daemon21:46:232
223099500,5rtkit-daemon2229-21rtkit-daemon20:56:252
117202503,8sleep311815-21grep16:41:273
117202503,8sleep311815-21grep16:41:273
8722492,9sleep20-21swapper/218:33:112
854799494,39cyclictest0-21swapper/321:51:273
854799494,39cyclictest0-21swapper/318:11:083
854799494,39cyclictest0-21swapper/318:11:083
854799494,28cyclictest0-21swapper/320:22:523
854799494,28cyclictest0-21swapper/320:22:523
854799494,27cyclictest0-21swapper/320:01:413
8547994935,9cyclictest26597-21needreboot19:26:313
854799493,25cyclictest0-21swapper/320:16:343
854799492,43cyclictest0-21swapper/320:07:503
854799492,42cyclictest0-21swapper/321:56:273
854799492,42cyclictest0-21swapper/321:27:013
854799492,42cyclictest0-21swapper/321:06:363
854799492,42cyclictest0-21swapper/318:56:183
854799492,42cyclictest0-21swapper/318:36:073
8547994919,11cyclictest18233-21cat16:56:113
8545994933,11cyclictest0-21swapper/218:26:382
8545994933,10cyclictest26100-21tune2fs17:11:282
8545994932,10cyclictest19034-21sed21:26:242
854599493,39cyclictest0-21swapper/221:06:302
854399494,27cyclictest0-21swapper/118:41:421
854399494,26cyclictest0-21swapper/121:12:151
854399493,41cyclictest0-21swapper/121:41:351
854399493,41cyclictest0-21swapper/121:41:351
854399493,39cyclictest0-21swapper/117:26:231
854399493,39cyclictest0-21swapper/117:26:231
82152492,10sleep20-21swapper/221:04:082
82152492,10sleep20-21swapper/221:04:082
47472492,9sleep30-21swapper/317:34:193
302252492,10sleep20-21swapper/217:21:162
302252492,10sleep20-21swapper/217:21:162
249082492,41sleep10-21swapper/119:22:591
236452492,10sleep10-21swapper/121:36:201
223099490,5rtkit-daemon2229-21rtkit-daemon19:14:072
854799485,26cyclictest0-21swapper/321:41:243
854799485,26cyclictest0-21swapper/321:41:243
854799485,26cyclictest0-21swapper/317:06:343
854799484,38cyclictest0-21swapper/321:48:253
854799484,38cyclictest0-21swapper/321:48:253
854799484,38cyclictest0-21swapper/321:01:263
854799484,38cyclictest0-21swapper/321:01:263
854799484,28cyclictest0-21swapper/318:17:323
854799484,27cyclictest0-21swapper/317:41:403
8545994835,8cyclictest22992-21chrt21:36:112
8545994833,11cyclictest0-21swapper/221:41:202
8545994833,11cyclictest0-21swapper/221:41:202
8545994830,13cyclictest0-21swapper/219:26:262
8545994829,13cyclictest0-21swapper/220:09:272
8545994825,8cyclictest2429-21crond19:46:042
8545994823,11cyclictest0-21swapper/216:38:322
854399484,27cyclictest0-21swapper/117:51:401
8543994829,12cyclictest355-21wc18:31:351
47672482,9sleep20-21swapper/219:50:252
319692482,10sleep10-21swapper/121:52:111
224422482,9sleep20-21swapper/221:32:262
223099480,5rtkit-daemon2229-21rtkit-daemon19:51:202
93882472,9sleep20-21swapper/219:58:192
854799474,38cyclictest0-21swapper/321:19:363
854799474,36cyclictest12572-21cpuspeed_turbos17:51:233
854799474,27cyclictest0-21swapper/318:35:373
854799474,27cyclictest0-21swapper/317:22:223
854799474,27cyclictest0-21swapper/317:22:223
854799474,27cyclictest0-21swapper/316:46:253
854799474,26cyclictest0-21swapper/318:52:383
854799474,26cyclictest0-21swapper/318:52:383
854799472,41cyclictest0-21swapper/317:40:573
854599475,35cyclictest13692-21telnet16:46:242
8545994730,12cyclictest0-21swapper/220:46:152
854599473,38cyclictest0-21swapper/217:46:302
854599473,38cyclictest0-21swapper/216:51:392
854599473,19cyclictest10391-21grep18:51:412
854599473,19cyclictest10391-21grep18:51:412
8545994725,9cyclictest0-21swapper/220:16:082
8545994723,11cyclictest0-21swapper/216:41:222
8545994723,11cyclictest0-21swapper/216:41:222
8545994712,8cyclictest29-21ksoftirqd/218:56:322
8545994712,28cyclictest0-21swapper/222:01:122
8543994721,13cyclictest0-21swapper/117:06:231
8543994716,16cyclictest0-21swapper/117:12:201
8543994713,13cyclictest0-21swapper/118:11:191
205652472,38sleep10-21swapper/121:31:081
172762473,37sleep20-21swapper/221:21:332
854799465,37cyclictest0-21swapper/321:21:273
854799463,15cyclictest0-21swapper/321:31:313
854799462,40cyclictest0-21swapper/317:00:193
854599463,37cyclictest0-21swapper/218:36:262
8545994620,7cyclictest7284-21latency_hist17:41:092
854599462,40cyclictest0-21swapper/218:17:322
854599462,38cyclictest0-21swapper/219:36:292
854599462,38cyclictest0-21swapper/219:36:292
8545994618,12cyclictest0-21swapper/217:16:442
854399465,14cyclictest0-21swapper/117:31:491
8543994630,10cyclictest11576-21unixbench-2d17:46:431
8543994629,10cyclictest21553-21tail17:01:311
854799454,25cyclictest0-21swapper/319:57:493
854799452,38cyclictest0-21swapper/320:53:063
854799452,38cyclictest0-21swapper/319:18:193
854599453,37cyclictest0-21swapper/221:51:332
854599453,35cyclictest0-21swapper/221:56:212
854599453,25cyclictest0-21swapper/217:44:302
134152452,9sleep20-21swapper/217:51:352
854799442,37cyclictest0-21swapper/319:51:203
854599423,33cyclictest1160-21snmpd18:42:362
854599423,32cyclictest0-21swapper/216:36:122
854599412,34cyclictest0-21swapper/218:06:242
854599412,34cyclictest0-21swapper/218:06:242
854399419,22cyclictest0-21swapper/116:36:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional