You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-07 - 14:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Fri Nov 07, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2230991220,6rtkit-daemon0-21swapper/309:11:133
2230991220,6rtkit-daemon0-21swapper/309:11:133
1088499923,82cyclictest14400-21crond05:50:401
1088699874,76cyclictest0-21swapper/305:06:083
1088699874,64cyclictest0-21swapper/305:25:413
1088699864,74cyclictest0-21swapper/305:10:543
1088699843,52cyclictest8318-21gltestperf05:36:003
1088699834,71cyclictest0-21swapper/308:40:353
1088699834,71cyclictest0-21swapper/308:15:453
1088699824,61cyclictest0-21swapper/308:56:023
1088699824,49cyclictest0-21swapper/306:41:053
1088499824,71cyclictest611-21cut09:50:491
1088699814,71cyclictest0-21swapper/305:44:033
1088699814,65cyclictest0-21swapper/306:00:523
1088699814,62cyclictest0-21swapper/306:00:393
1088699814,60cyclictest0-21swapper/307:35:233
1088699813,71cyclictest0-21swapper/308:45:533
1088699807,65cyclictest7243-21awk05:35:403
1088699804,63cyclictest15739-21tail09:15:443
1088699803,70cyclictest0-21swapper/309:55:183
1088699803,7cyclictest13102-21munin-run09:10:323
1088699803,53cyclictest0-21swapper/306:10:493
1088699803,51cyclictest0-21swapper/308:01:073
223099790,5rtkit-daemon2229-21rtkit-daemon09:33:271
1088699797,57cyclictest0-21swapper/309:45:423
1088699793,59cyclictest0-21swapper/310:00:363
1088499797,65cyclictest0-21swapper/105:39:001
1088499793,69cyclictest0-21swapper/109:21:051
1088699785,63cyclictest0-21swapper/305:01:063
1088699784,66cyclictest0-21swapper/309:30:323
1088699783,68cyclictest17287-21tr08:10:493
1088699783,58cyclictest15927-21runrttasks07:00:513
1088699783,57cyclictest0-21swapper/309:35:023
223099770,4rtkit-daemon2229-21rtkit-daemon09:35:501
1088699774,66cyclictest19948-21unixbench_singl07:06:113
1088699774,66cyclictest19948-21unixbench_singl07:06:113
1088699774,66cyclictest0-21swapper/307:26:023
1088699774,62cyclictest0-21swapper/306:34:573
1088699774,55cyclictest0-21swapper/308:41:073
1088699774,48cyclictest0-21swapper/306:51:063
1088699773,66cyclictest0-21swapper/305:27:273
1088699772,9cyclictest0-21swapper/308:30:593
1088499773,68cyclictest0-21swapper/106:49:351
1088499773,53cyclictest0-21swapper/109:45:541
223099760,8rtkit-daemon0-21swapper/209:12:281
223099760,8rtkit-daemon0-21swapper/209:12:281
1088699764,61cyclictest0-21swapper/306:25:573
1088699764,61cyclictest0-21swapper/306:25:573
1088699763,66cyclictest5398-21cstates07:45:513
1088699763,66cyclictest5398-21cstates07:45:513
10886997614,40cyclictest0-21swapper/309:35:573
1088499764,65cyclictest2162-21grep06:30:581
1088499764,64cyclictest0-21swapper/108:45:511
223099750,6rtkit-daemon14667-21nfs_client06:56:043
1088699754,63cyclictest0-21swapper/306:50:373
1088699754,58cyclictest0-21swapper/307:21:053
1088699754,55cyclictest0-21swapper/307:15:583
1088699754,49cyclictest0-21swapper/307:55:473
1088699753,8cyclictest21908-21ntp_kernel_pll_04:56:113
1088699753,8cyclictest21908-21ntp_kernel_pll_04:56:113
1088699753,58cyclictest0-21swapper/307:50:503
1088699753,50cyclictest27787-21telnet09:40:443
120932740,14sleep11088499cyclictest08:00:361
1088699744,63cyclictest0-21swapper/305:46:083
1088699744,62cyclictest0-21swapper/309:20:453
1088699744,56cyclictest0-21swapper/306:17:063
1088699744,48cyclictest0-21swapper/309:55:473
1088699744,48cyclictest0-21swapper/309:55:473
1088699743,9cyclictest106250irq/46-eth005:51:063
1088699743,52cyclictest417-21cpuspeed_turbos07:35:483
1088499746,60cyclictest0-21swapper/109:20:331
1088499746,51cyclictest0-21swapper/109:55:441
1088499746,51cyclictest0-21swapper/109:55:441
1088499744,63cyclictest0-21swapper/106:26:151
1088499744,63cyclictest0-21swapper/106:26:151
1088499744,63cyclictest0-21swapper/105:11:121
1088499743,64cyclictest0-21swapper/106:50:521
1088699734,63cyclictest0-21swapper/306:24:313
1088699734,63cyclictest0-21swapper/305:16:123
1088699734,50cyclictest0-21swapper/308:28:013
1088699734,50cyclictest0-21swapper/308:28:013
1088699734,47cyclictest0-21swapper/308:21:113
1088699734,45cyclictest0-21swapper/307:41:053
1088699733,9cyclictest4286-21df_abs06:35:523
10886997311,41cyclictest0-21swapper/304:35:573
1088499735,61cyclictest0-21swapper/105:05:541
1088499734,61cyclictest0-21swapper/105:20:401
1088499733,64cyclictest0-21swapper/108:21:041
223099720,4rtkit-daemon2229-21rtkit-daemon07:56:012
1088699724,61cyclictest0-21swapper/308:05:583
1088699724,60cyclictest0-21swapper/304:46:153
1088699724,60cyclictest0-21swapper/304:40:553
1088699723,53cyclictest0-21swapper/308:50:453
1088499723,61cyclictest1265-21runrttasks04:54:061
10884997226,40cyclictest15428-21tail08:05:541
10884997225,30cyclictest22805-21cat07:15:371
1088699714,60cyclictest0-21swapper/304:51:153
10886997110,48cyclictest0-21swapper/306:05:513
1088499715,59cyclictest0-21swapper/105:45:411
1088499714,60cyclictest0-21swapper/106:55:501
1088499713,60cyclictest24109-21munin-node08:26:111
1088499713,60cyclictest24109-21munin-node08:26:111
1088499713,60cyclictest0-21swapper/108:15:461
1088699705,57cyclictest0-21swapper/307:11:083
1088699703,49cyclictest0-21swapper/309:04:573
1088499705,58cyclictest0-21swapper/107:35:131
1088499705,58cyclictest0-21swapper/106:35:591
1088499704,60cyclictest1265-21runrttasks04:44:471
1088499703,61cyclictest0-21swapper/105:55:411
1088499703,60cyclictest7231-21crond05:35:401
1088499694,58cyclictest9759-21cat07:55:361
1088499693,60cyclictest6245-21munin-node06:40:561
1088499693,60cyclictest30875-21taskset08:38:201
1088499693,59cyclictest17676-21cut08:10:531
1088499693,59cyclictest0-21swapper/108:30:461
1088499686,56cyclictest0-21swapper/107:16:021
1088499686,55cyclictest0-21swapper/109:00:461
1088499684,57cyclictest0-21swapper/104:40:431
1088499683,58cyclictest26593-21tail06:15:501
1088499683,48cyclictest25340-21munin-node07:21:011
1088499675,55cyclictest0-21swapper/106:00:561
1088499673,58cyclictest29511-21munin-node07:26:101
1088499673,58cyclictest0-21swapper/109:05:491
1088499673,57cyclictest21834-21wc09:26:001
1088499673,56cyclictest0-21swapper/108:57:101
1088499673,50cyclictest2391-21munin-run07:40:351
1088499673,43cyclictest223372sed04:56:171
1088499673,43cyclictest223372sed04:56:171
1088499673,38cyclictest18150-21cat04:50:431
223099660,4rtkit-daemon2229-21rtkit-daemon05:16:002
1088499664,56cyclictest0-21swapper/108:55:321
1088499664,55cyclictest0-21swapper/106:24:381
10884996630,31cyclictest0-21swapper/105:56:131
1088499663,57cyclictest0-21swapper/107:40:531
1088499663,56cyclictest23986-21cat06:10:391
1088699654,55cyclictest0-21swapper/304:35:433
1088499656,53cyclictest0-21swapper/107:04:141
1088499654,55cyclictest20347-21awk07:10:371
1088499654,55cyclictest20347-21awk07:10:371
1088499653,56cyclictest0-21swapper/105:26:161
223099640,6rtkit-daemon2229-21rtkit-daemon05:05:002
1088499645,44cyclictest25982-21runrttasks06:11:131
1088499644,53cyclictest0-21swapper/108:44:541
1088499643,55cyclictest0-21swapper/109:41:041
223099630,5rtkit-daemon2229-21rtkit-daemon05:39:072
1088499633,54cyclictest0-21swapper/107:50:081
1088499633,54cyclictest0-21swapper/107:50:081
1088499623,53cyclictest0-21swapper/105:03:171
10884996224,32cyclictest673-21sed05:20:581
223099610,8rtkit-daemon0-21swapper/207:51:012
1088499613,52cyclictest6064-21cat10:00:551
223099600,4rtkit-daemon2229-21rtkit-daemon08:55:442
223099600,4rtkit-daemon2229-21rtkit-daemon07:19:402
223099600,4rtkit-daemon2229-21rtkit-daemon05:46:062
10885996039,12cyclictest14186-21grep09:10:542
10885996039,12cyclictest14186-21grep09:10:542
10882996016,26cyclictest0-21swapper/005:10:520
223099590,6rtkit-daemon0-21swapper/309:08:132
223099590,4rtkit-daemon2229-21rtkit-daemon04:41:012
1088599594,47cyclictest0-21swapper/206:55:592
1088499594,49cyclictest0-21swapper/108:02:301
223099580,6rtkit-daemon2229-21rtkit-daemon07:01:332
10885995722,12cyclictest0-21swapper/209:35:512
1088299574,45cyclictest0-21swapper/005:31:100
111062563,12sleep00-21swapper/009:05:450
1088599566,41cyclictest0-21swapper/206:30:512
10885995623,12cyclictest0-21swapper/208:11:002
10882995638,11cyclictest29415-21head09:41:040
65432553,9sleep06683-21grep07:46:090
65432553,9sleep06683-21grep07:46:090
190172550,8sleep20-21swapper/209:20:542
1088599555,41cyclictest26002-21tune2fs07:20:542
1088299555,42cyclictest0-21swapper/008:25:340
1088299555,12cyclictest0-21swapper/006:10:590
1088599545,31cyclictest0-21swapper/207:15:372
1088599545,30cyclictest0-21swapper/205:51:152
1088599544,42cyclictest0-21swapper/206:04:162
10885995435,11cyclictest11131-21head04:35:562
1088299546,41cyclictest13264-21awk04:40:420
1088599536,14cyclictest0-21swapper/205:57:002
1088599535,40cyclictest0-21swapper/207:26:122
1088599535,29cyclictest0-21swapper/206:05:572
1088599534,34cyclictest0-21swapper/205:40:562
10885995320,15cyclictest0-21swapper/206:46:022
1088499533,44cyclictest10974-21wc04:35:431
1088299534,41cyclictest0-21swapper/006:45:530
10882995336,10cyclictest14329-21crond08:05:350
1088299533,44cyclictest0-21swapper/009:59:070
1088299533,44cyclictest0-21swapper/009:59:070
1088299533,44cyclictest0-21swapper/007:10:570
1088299533,44cyclictest0-21swapper/005:36:020
10882995324,14cyclictest0-21swapper/006:04:280
1088599525,39cyclictest0-21swapper/209:55:472
1088599525,39cyclictest0-21swapper/209:55:472
1088599525,39cyclictest0-21swapper/207:40:572
1088599525,28cyclictest0-21swapper/209:15:572
1088599524,29cyclictest0-21swapper/208:46:052
1088599524,13cyclictest0-21swapper/206:36:052
10885995235,11cyclictest18571-21grep04:51:002
10885995234,10cyclictest20108-21tune2fs08:15:532
10885995212,21cyclictest0-21swapper/206:51:122
1088299525,39cyclictest0-21swapper/009:15:090
1088299525,39cyclictest0-21swapper/009:15:090
1088299525,29cyclictest0-21swapper/004:51:170
1088299525,28cyclictest0-21swapper/006:06:090
1088299524,40cyclictest0-21swapper/005:46:100
1088299524,29cyclictest0-21swapper/009:31:020
1088299524,11cyclictest0-21swapper/009:21:030
10882995233,12cyclictest0-21swapper/007:40:480
1088299523,43cyclictest0-21swapper/008:50:570
1088299523,43cyclictest0-21swapper/008:36:000
1088299523,43cyclictest0-21swapper/006:21:010
1088299523,42cyclictest16660-21sed07:00:580
81752512,42sleep00-21swapper/008:58:480
1088599515,38cyclictest0-21swapper/208:35:492
1088599515,38cyclictest0-21swapper/208:05:532
1088599515,37cyclictest1265-21runrttasks05:35:202
1088599515,28cyclictest0-21swapper/209:50:582
1088599515,28cyclictest0-21swapper/206:26:072
1088599515,28cyclictest0-21swapper/206:26:072
1088599515,28cyclictest0-21swapper/204:56:142
1088599515,28cyclictest0-21swapper/204:56:142
1088599514,40cyclictest0-21swapper/207:30:482
1088599514,39cyclictest0-21swapper/205:25:202
1088599514,30cyclictest0-21swapper/205:06:002
1088599514,29cyclictest0-21swapper/210:00:502
1088599514,29cyclictest0-21swapper/208:40:502
1088599514,28cyclictest0-21swapper/208:25:582
1088599514,28cyclictest0-21swapper/208:25:582
1088599514,28cyclictest0-21swapper/207:35:522
1088599514,26cyclictest0-21swapper/207:10:382
1088599514,26cyclictest0-21swapper/207:10:382
1088599514,24cyclictest0-21swapper/207:46:072
1088599514,24cyclictest0-21swapper/207:46:072
1088599514,24cyclictest0-21swapper/206:10:592
10885995121,12cyclictest0-21swapper/206:19:302
10885995112,22cyclictest26276-1kworker/2:1H08:50:522
1088299515,28cyclictest0-21swapper/009:25:510
1088299514,41cyclictest0-21swapper/008:31:030
1088299514,39cyclictest0-21swapper/007:56:050
1088299514,39cyclictest0-21swapper/005:20:570
1088299514,28cyclictest0-21swapper/005:26:000
1088299513,43cyclictest0-21swapper/010:00:550
1088299513,42cyclictest0-21swapper/008:15:560
1088299513,42cyclictest0-21swapper/008:10:520
1088299513,42cyclictest0-21swapper/007:05:500
1088299513,42cyclictest0-21swapper/007:05:500
1088299513,42cyclictest0-21swapper/006:26:160
1088299513,42cyclictest0-21swapper/006:26:160
1088299513,42cyclictest0-21swapper/004:46:000
1088299513,41cyclictest0-21swapper/009:36:260
1088299513,41cyclictest0-21swapper/007:25:480
1088299513,41cyclictest0-21swapper/006:55:520
10882995121,13cyclictest0-21swapper/009:00:500
307612503,40sleep00-21swapper/005:15:590
1088599505,27cyclictest0-21swapper/208:30:582
1088599504,28cyclictest0-21swapper/206:40:502
1088599504,26cyclictest0-21swapper/209:00:452
1088599503,40cyclictest0-21swapper/209:45:472
1088299505,27cyclictest0-21swapper/005:51:030
1088299504,39cyclictest0-21swapper/006:50:540
1088299504,38cyclictest0-21swapper/009:45:550
1088299504,28cyclictest0-21swapper/008:05:570
1088299503,41cyclictest0-21swapper/007:15:540
1088299503,34cyclictest0-21swapper/007:21:010
9752492,10sleep00-21swapper/008:45:180
1088599495,27cyclictest0-21swapper/209:45:042
1088599495,27cyclictest0-21swapper/208:00:482
1088599494,28cyclictest0-21swapper/209:30:492
1088599494,28cyclictest0-21swapper/205:26:122
1088599494,27cyclictest0-21swapper/209:25:482
1088599493,29cyclictest0-21swapper/206:24:312
1088299494,38cyclictest0-21swapper/009:20:090
1088299494,38cyclictest0-21swapper/007:50:490
1088299494,38cyclictest0-21swapper/007:36:060
1088299494,38cyclictest0-21swapper/005:05:430
1088299494,37cyclictest0-21swapper/008:45:550
1088299494,27cyclictest0-21swapper/006:35:390
1088299494,27cyclictest0-21swapper/006:19:300
10882994932,11cyclictest31902-21grep07:31:100
1088299493,40cyclictest0-21swapper/004:56:160
1088299493,40cyclictest0-21swapper/004:56:160
1088299493,39cyclictest0-21swapper/008:25:460
1088299493,39cyclictest0-21swapper/008:25:460
1088299493,26cyclictest0-21swapper/005:55:520
106182494,6sleep00-21swapper/004:32:470
99802484,35sleep20-21swapper/204:31:082
1088599483,26cyclictest0-21swapper/204:49:162
1088299484,37cyclictest0-21swapper/009:50:520
1088299483,40cyclictest0-21swapper/006:41:070
1088299483,40cyclictest0-21swapper/005:41:140
1088299483,39cyclictest0-21swapper/005:05:540
10882994730,11cyclictest15255-21grep04:41:160
10882994725,15cyclictest0-21swapper/006:35:490
1088599464,26cyclictest0-21swapper/208:20:482
1088599454,14cyclictest0-21swapper/205:11:062
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional