You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 00:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Sun Nov 23, 2025 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
399199924,81cyclictest0-21swapper/011:38:170
399599894,11cyclictest0-21swapper/307:03:293
399199894,77cyclictest0-21swapper/011:23:140
399199894,68cyclictest0-21swapper/008:37:480
3995998824,10cyclictest2300-21unixbench_singl07:58:253
399199883,10cyclictest0-21swapper/011:32:560
3991998815,66cyclictest28064-21tail12:17:550
399599874,32cyclictest0-21swapper/308:08:023
3995998728,10cyclictest16692-21fschecks_time11:53:003
3995998728,10cyclictest16692-21fschecks_time11:53:003
3995998723,10cyclictest31190-21gltestperf12:23:003
399599864,13cyclictest0-21swapper/308:07:493
399599863,34cyclictest0-21swapper/308:24:483
3995998624,9cyclictest19720-21aten_r1power_fr10:52:563
399199863,9cyclictest17196-21cat07:23:140
399199863,76cyclictest0-21swapper/009:02:420
399199863,76cyclictest0-21swapper/009:02:420
3991998611,50cyclictest20885-21sshd12:02:540
3995998528,10cyclictest28096-21aten_r1power_cu07:48:013
3995998525,11cyclictest2093-21grep10:13:193
3995998427,10cyclictest26751-21tail12:13:033
3995998424,10cyclictest23810-21unixbench_singl08:43:223
399199844,72cyclictest0-21swapper/010:53:130
399199844,12cyclictest0-21swapper/007:13:130
399599834,11cyclictest0-21swapper/309:31:553
3995998330,11cyclictest19038-21grep08:33:243
399599833,27cyclictest0-21swapper/309:33:073
399199834,13cyclictest0-21swapper/008:47:480
399199834,12cyclictest0-21swapper/010:22:590
399199834,12cyclictest0-21swapper/010:22:590
3991998312,50cyclictest0-21swapper/009:13:020
243799830,5rtkit-daemon2436-21rtkit-daemon09:43:381
243799830,5rtkit-daemon2436-21rtkit-daemon09:43:381
399599824,24cyclictest0-21swapper/307:38:043
399599824,12cyclictest0-21swapper/309:08:063
399599823,36cyclictest0-21swapper/308:53:163
399599823,36cyclictest0-21swapper/308:53:163
399199829,56cyclictest0-21swapper/010:27:570
399199824,71cyclictest0-21swapper/007:48:100
399199824,13cyclictest0-21swapper/012:12:570
399199823,12cyclictest0-21swapper/011:18:160
399199823,10cyclictest0-21swapper/007:56:450
3991998216,48cyclictest0-21swapper/007:28:220
3991998216,48cyclictest0-21swapper/007:28:220
3991998214,9cyclictest4057-21munin-node11:28:070
399599813,32cyclictest0-21swapper/310:40:173
3995998124,10cyclictest21769-21gltestperf07:33:113
399199816,55cyclictest0-21swapper/010:52:130
399199815,57cyclictest0-21swapper/009:26:590
399199814,65cyclictest0-21swapper/008:03:080
399199814,62cyclictest0-21swapper/012:02:420
399199814,58cyclictest0-21swapper/010:13:030
399199814,54cyclictest0-21swapper/010:07:050
399199814,13cyclictest0-21swapper/008:48:480
399199814,12cyclictest0-21swapper/009:33:080
399199814,11cyclictest0-21swapper/008:13:020
399199814,11cyclictest0-21swapper/008:13:020
399199813,8cyclictest12994-21unixbench_multi09:28:190
399199813,71cyclictest149091munin-node08:28:160
399199813,10cyclictest0-21swapper/011:07:550
399199813,10cyclictest0-21swapper/011:07:550
3991998118,45cyclictest0-21swapper/009:53:540
3991998113,47cyclictest0-21swapper/011:43:540
243799810,5rtkit-daemon2436-21rtkit-daemon10:28:361
243799810,5rtkit-daemon2436-21rtkit-daemon08:24:341
399599804,22cyclictest0-21swapper/310:22:573
399599804,22cyclictest0-21swapper/310:22:573
399599803,32cyclictest0-21swapper/310:51:573
3995998021,14cyclictest13805-21cut07:18:033
399199806,55cyclictest29253-1kworker/0:2H10:43:340
399199805,68cyclictest0-21swapper/011:12:540
399199804,58cyclictest0-21swapper/007:03:150
399199804,56cyclictest0-21swapper/009:38:190
399199804,14cyclictest0-21swapper/008:23:170
399199804,12cyclictest0-21swapper/011:47:550
399199804,12cyclictest0-21swapper/008:53:130
399199804,12cyclictest0-21swapper/008:53:130
399199803,61cyclictest11951-1kworker/0:0H08:22:490
399199803,59cyclictest12215-21latency_hist10:37:450
399199803,54cyclictest0-21swapper/010:58:040
243799800,6rtkit-daemon2436-21rtkit-daemon09:48:081
243799800,5rtkit-daemon2436-21rtkit-daemon12:07:331
243799800,5rtkit-daemon2436-21rtkit-daemon11:25:421
243799800,5rtkit-daemon2436-21rtkit-daemon11:21:421
243799800,5rtkit-daemon2436-21rtkit-daemon09:18:471
243799800,4rtkit-daemon2436-21rtkit-daemon11:41:191
243799800,4rtkit-daemon2436-21rtkit-daemon11:11:501
243799800,4rtkit-daemon2436-21rtkit-daemon11:11:501
243799800,4rtkit-daemon2436-21rtkit-daemon11:00:131
243799800,4rtkit-daemon2436-21rtkit-daemon09:04:471
243799800,4rtkit-daemon2436-21rtkit-daemon07:56:501
399599794,32cyclictest0-21swapper/307:32:523
399599794,32cyclictest0-21swapper/307:32:523
3995997924,10cyclictest28814-21gltestperf12:18:023
399199794,58cyclictest0-21swapper/009:43:090
399199794,58cyclictest0-21swapper/009:43:090
243799790,5rtkit-daemon2436-21rtkit-daemon11:34:561
243799790,5rtkit-daemon2436-21rtkit-daemon10:43:511
243799790,4rtkit-daemon2436-21rtkit-daemon12:19:401
243799790,4rtkit-daemon2436-21rtkit-daemon12:09:101
243799790,4rtkit-daemon2436-21rtkit-daemon11:52:261
243799790,4rtkit-daemon2436-21rtkit-daemon11:16:271
243799790,4rtkit-daemon2436-21rtkit-daemon10:21:291
243799790,4rtkit-daemon2436-21rtkit-daemon10:14:151
243799790,4rtkit-daemon2436-21rtkit-daemon10:11:451
243799790,4rtkit-daemon2436-21rtkit-daemon09:25:241
243799790,4rtkit-daemon2436-21rtkit-daemon08:47:551
243799790,4rtkit-daemon2436-21rtkit-daemon08:47:401
243799790,4rtkit-daemon2436-21rtkit-daemon08:13:571
243799790,4rtkit-daemon2436-21rtkit-daemon08:13:571
243799790,4rtkit-daemon2436-21rtkit-daemon08:02:201
243799790,4rtkit-daemon2436-21rtkit-daemon07:45:051
243799790,4rtkit-daemon2436-21rtkit-daemon07:41:061
243799790,4rtkit-daemon2436-21rtkit-daemon07:27:441
243799790,4rtkit-daemon2436-21rtkit-daemon07:21:291
243799790,4rtkit-daemon2436-21rtkit-daemon07:06:591
399599784,30cyclictest0-21swapper/309:13:013
399599784,28cyclictest0-21swapper/308:52:013
399199784,56cyclictest0-21swapper/006:58:050
399199784,14cyclictest0-21swapper/010:17:580
399199784,12cyclictest0-21swapper/007:11:400
399199783,8cyclictest19834-21head08:38:010
399199783,10cyclictest32568-21grep09:03:080
399199783,10cyclictest0-21swapper/007:43:240
3991997818,46cyclictest0-21swapper/011:53:020
3991997818,46cyclictest0-21swapper/011:53:020
3991997815,48cyclictest0-21swapper/009:08:560
243799780,5rtkit-daemon2436-21rtkit-daemon08:08:341
243799780,4rtkit-daemon2436-21rtkit-daemon12:26:021
243799780,4rtkit-daemon2436-21rtkit-daemon11:06:051
243799780,4rtkit-daemon2436-21rtkit-daemon10:40:291
243799780,4rtkit-daemon2436-21rtkit-daemon10:23:291
243799780,4rtkit-daemon2436-21rtkit-daemon10:23:291
243799780,4rtkit-daemon2436-21rtkit-daemon10:00:381
243799780,4rtkit-daemon2436-21rtkit-daemon09:35:011
243799780,4rtkit-daemon2436-21rtkit-daemon09:31:311
243799780,4rtkit-daemon2436-21rtkit-daemon08:59:321
243799780,4rtkit-daemon2436-21rtkit-daemon08:59:321
243799780,4rtkit-daemon2436-21rtkit-daemon08:54:331
243799780,4rtkit-daemon2436-21rtkit-daemon08:54:331
243799780,4rtkit-daemon2436-21rtkit-daemon08:41:261
243799780,4rtkit-daemon2436-21rtkit-daemon08:33:481
243799780,4rtkit-daemon2436-21rtkit-daemon07:34:061
243799780,4rtkit-daemon2436-21rtkit-daemon07:29:211
243799780,4rtkit-daemon2436-21rtkit-daemon07:29:211
399599774,37cyclictest31898-21cpuspeed_turbos11:17:573
399599773,27cyclictest0-21swapper/311:23:193
3995997729,10cyclictest19195-21gltestperf11:58:023
399199774,12cyclictest0-21swapper/010:08:080
399199773,53cyclictest0-21swapper/009:17:590
399199773,11cyclictest0-21swapper/012:08:110
243799770,5rtkit-daemon2436-21rtkit-daemon09:54:001
243799770,5rtkit-daemon2436-21rtkit-daemon09:13:471
243799770,4rtkit-daemon2436-21rtkit-daemon11:59:401
243799770,4rtkit-daemon2436-21rtkit-daemon11:53:031
243799770,4rtkit-daemon2436-21rtkit-daemon11:53:031
243799770,4rtkit-daemon2436-21rtkit-daemon10:48:281
243799770,4rtkit-daemon2436-21rtkit-daemon10:36:441
243799770,4rtkit-daemon2436-21rtkit-daemon09:08:321
243799770,4rtkit-daemon2436-21rtkit-daemon07:50:501
399599763,31cyclictest0-21swapper/312:02:533
399199764,56cyclictest0-21swapper/009:50:250
399199764,12cyclictest0-21swapper/007:38:210
3991997613,44cyclictest0-21swapper/007:35:530
3991997610,55cyclictest0-21swapper/008:08:030
243799760,4rtkit-daemon2436-21rtkit-daemon12:14:321
243799760,4rtkit-daemon2436-21rtkit-daemon11:28:491
243799760,4rtkit-daemon2436-21rtkit-daemon10:56:201
243799760,4rtkit-daemon2436-21rtkit-daemon08:21:561
243799760,4rtkit-daemon2436-21rtkit-daemon07:02:001
399599753,34cyclictest0-21swapper/311:13:073
399199754,64cyclictest0-21swapper/009:58:230
399199754,12cyclictest0-21swapper/011:03:290
243799750,4rtkit-daemon2436-21rtkit-daemon10:03:301
399199745,60cyclictest1369-21runrttasks07:22:220
399199744,62cyclictest0-21swapper/007:58:130
399599734,34cyclictest0-21swapper/310:02:593
399199738,7cyclictest114550irq/46-eth012:23:130
399599724,32cyclictest0-21swapper/311:38:053
399199723,9cyclictest0-21swapper/010:37:580
399599693,16cyclictest6979-21df_abs11:32:583
399199695,57cyclictest0-21swapper/006:57:530
3995996727,8cyclictest11129-21crond07:12:513
243799660,5rtkit-daemon2436-21rtkit-daemon09:40:382
399599654,16cyclictest0-21swapper/309:23:173
243799630,5rtkit-daemon0-21swapper/311:38:042
243799610,7rtkit-daemon0-21swapper/311:33:262
399499604,29cyclictest0-21swapper/210:28:142
243799600,5rtkit-daemon0-21swapper/307:08:142
399599594,13cyclictest0-21swapper/307:53:193
399499593,38cyclictest22062-21munin-node08:43:002
243799590,6rtkit-daemon0-21swapper/306:58:002
3994995721,13cyclictest0-21swapper/207:41:422
3994995720,14cyclictest0-21swapper/209:48:202
3994995720,13cyclictest0-21swapper/210:58:112
3993995739,12cyclictest17770-21unixbench_multi09:38:241
399499565,32cyclictest0-21swapper/209:18:002
399499565,28cyclictest0-21swapper/207:48:102
399499565,27cyclictest0-21swapper/208:33:262
399499565,26cyclictest0-21swapper/207:56:452
399499564,31cyclictest0-21swapper/209:13:012
399499564,28cyclictest0-21swapper/210:12:452
399499564,28cyclictest0-21swapper/207:23:072
399499564,27cyclictest0-21swapper/211:07:542
399499564,27cyclictest0-21swapper/211:07:542
3994995622,12cyclictest0-21swapper/210:03:212
3994995620,13cyclictest0-21swapper/208:08:252
275692563,11sleep20-21swapper/207:44:282
399499556,27cyclictest0-21swapper/212:27:282
399499555,28cyclictest0-21swapper/211:28:092
399499555,28cyclictest0-21swapper/210:32:572
399499555,28cyclictest0-21swapper/208:48:092
399499555,27cyclictest0-21swapper/210:43:072
399499554,29cyclictest0-21swapper/211:43:052
399499554,28cyclictest0-21swapper/212:18:042
399499554,28cyclictest0-21swapper/211:48:092
399499554,27cyclictest9777-21sed09:23:062
399499554,27cyclictest0-21swapper/209:08:092
399499554,27cyclictest0-21swapper/209:03:022
3994995535,11cyclictest1099-21tune2fs07:58:092
3994995520,13cyclictest0-21swapper/209:28:052
3994995519,30cyclictest32462-21http_loadtime11:18:032
399499545,30cyclictest0-21swapper/209:58:142
399499545,27cyclictest0-21swapper/210:52:562
399499545,26cyclictest0-21swapper/212:12:502
399499545,26cyclictest0-21swapper/209:01:552
399499545,26cyclictest0-21swapper/209:01:552
399499545,26cyclictest0-21swapper/208:27:262
399499544,28cyclictest0-21swapper/208:38:082
399499544,27cyclictest0-21swapper/211:02:552
3994995435,11cyclictest24941-21grep12:08:142
3994995420,12cyclictest0-21swapper/207:03:062
399499542,33cyclictest0-21swapper/212:01:462
3994995414,23cyclictest0-21swapper/211:52:542
3994995414,23cyclictest0-21swapper/211:52:542
132172544,43sleep10-21swapper/107:14:031
399499534,27cyclictest0-21swapper/208:13:102
399499534,27cyclictest0-21swapper/208:13:102
399499534,27cyclictest0-21swapper/208:03:272
126702532,10sleep112775-21timerandwakeup11:43:121
399499523,31cyclictest0-21swapper/210:21:392
399499522,35cyclictest0-21swapper/209:37:012
399399524,29cyclictest0-21swapper/107:08:181
3994995023,8cyclictest15906-21latency_hist07:22:522
399599495,26cyclictest0-21swapper/308:38:173
3995994917,12cyclictest0-21swapper/310:28:093
399499496,35cyclictest0-21swapper/210:23:182
399499496,35cyclictest0-21swapper/210:23:182
399499494,37cyclictest23058-21crond07:37:512
3994994929,12cyclictest0-21swapper/211:13:082
36822493,13sleep20-21swapper/211:26:042
253982493,13sleep20-21swapper/209:57:452
243799490,4rtkit-daemon2436-21rtkit-daemon08:03:051
24182493,12sleep20-21swapper/210:15:072
190082492,40sleep20-21swapper/210:48:202
3995994833,9cyclictest31404-21crond09:02:473
3995994833,9cyclictest31404-21crond09:02:473
399599483,37cyclictest0-21swapper/309:52:003
399599483,27cyclictest0-21swapper/311:03:163
399399485,35cyclictest0-21swapper/108:28:031
210602483,39sleep20-21swapper/212:02:562
134862483,12sleep20-21swapper/207:17:352
399599474,12cyclictest0-21swapper/311:08:123
399599474,12cyclictest0-21swapper/311:08:123
3995994730,9cyclictest10446-21cpuspeed_turbos08:18:033
399599473,27cyclictest0-21swapper/310:43:213
399599473,27cyclictest0-21swapper/307:13:203
399499474,34cyclictest0-21swapper/208:28:032
399499474,27cyclictest0-21swapper/208:53:112
399499474,27cyclictest0-21swapper/208:53:112
399499473,25cyclictest0-21swapper/210:38:182
3994994727,13cyclictest0-21swapper/208:19:362
399599464,26cyclictest0-21swapper/309:57:593
3995994630,10cyclictest19294-21gltestperf09:43:063
3995994630,10cyclictest19294-21gltestperf09:43:063
399599463,26cyclictest0-21swapper/311:43:013
3995994621,7cyclictest5258-21cat10:22:453
399499463,26cyclictest0-21swapper/206:57:532
3994994623,10cyclictest0-21swapper/207:28:242
3994994623,10cyclictest0-21swapper/207:28:242
3995994529,10cyclictest6367-21sleep07:02:523
3995994527,11cyclictest10678-21fschecks_count10:33:003
3995994521,8cyclictest1243-21snmpd08:31:503
399599452,38cyclictest0-21swapper/312:10:183
399499454,25cyclictest0-21swapper/209:42:572
399499454,25cyclictest0-21swapper/209:42:572
399399455,33cyclictest0-21swapper/106:57:531
399599444,13cyclictest0-21swapper/307:23:023
399599443,35cyclictest0-21swapper/310:58:163
399599443,12cyclictest0-21swapper/309:38:063
399599442,31cyclictest0-21swapper/309:02:583
399599434,11cyclictest0-21swapper/311:32:193
399599433,25cyclictest13762-21telnet11:47:543
3995994324,13cyclictest0-21swapper/310:08:163
399599424,11cyclictest0-21swapper/307:43:043
399599414,12cyclictest0-21swapper/309:21:563
3995994126,9cyclictest23601-21cut09:52:593
399599403,31cyclictest0-21swapper/308:15:313
399599403,31cyclictest0-21swapper/308:15:313
399599302,9cyclictest0-21swapper/306:57:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional