You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-18 - 00:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Wed Sep 17, 2025 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
986999624,55cyclictest0-21swapper/009:24:050
98699944,25cyclictest0-21swapper/008:24:010
986999226,49cyclictest0-21swapper/005:59:030
986999226,49cyclictest0-21swapper/005:59:030
986999117,12cyclictest0-21swapper/008:49:020
178912910,7sleep30-21swapper/307:04:013
98699904,27cyclictest0-21swapper/008:43:360
98699904,26cyclictest0-21swapper/008:39:040
986998917,12cyclictest0-21swapper/007:58:590
98699884,65cyclictest0-21swapper/010:48:570
98699884,27cyclictest0-21swapper/008:19:080
98699874,26cyclictest0-21swapper/007:08:220
986998720,12cyclictest0-21swapper/010:33:500
98699854,74cyclictest0-21swapper/005:44:100
98699854,27cyclictest0-21swapper/008:03:560
98699854,27cyclictest0-21swapper/006:44:000
98699854,27cyclictest0-21swapper/006:09:030
98699854,26cyclictest0-21swapper/006:19:010
98699853,8cyclictest19208-21munin-node07:09:070
98699853,24cyclictest1160-21snmpd08:53:560
98699853,24cyclictest1160-21snmpd08:53:560
986998518,13cyclictest0-21swapper/009:44:020
98699844,27cyclictest0-21swapper/006:58:530
98699844,25cyclictest0-21swapper/005:33:070
986998418,12cyclictest0-21swapper/005:39:170
98699834,26cyclictest0-21swapper/007:28:520
98699834,26cyclictest0-21swapper/007:28:520
98699834,25cyclictest0-21swapper/007:48:470
98699834,25cyclictest0-21swapper/006:27:130
98699834,11cyclictest0-21swapper/009:18:430
986998318,10cyclictest6574-21forks10:03:500
98699824,16cyclictest0-21swapper/009:50:560
98699823,71cyclictest9294-21tr07:53:580
98699823,26cyclictest0-21swapper/007:48:280
98699823,11cyclictest0-21swapper/009:33:450
986998228,48cyclictest19201-21needreboot08:14:000
986998228,48cyclictest19201-21needreboot08:14:000
986998219,10cyclictest106250irq/46-eth005:49:000
98699813,24cyclictest0-21swapper/008:08:460
98699813,24cyclictest0-21swapper/006:03:540
986998111,9cyclictest106250irq/46-eth008:28:470
98699805,66cyclictest0-21swapper/009:28:510
98699804,13cyclictest0-21swapper/009:58:450
986998014,8cyclictest0-21swapper/009:53:450
98699794,68cyclictest0-21swapper/007:14:560
98699794,68cyclictest0-21swapper/006:13:560
98699794,31cyclictest0-21swapper/007:39:030
98699794,24cyclictest29324-21grep08:34:090
98699794,12cyclictest0-21swapper/006:48:500
98699794,10cyclictest0-21swapper/009:13:440
98699786,17cyclictest0-21swapper/005:23:410
98699786,17cyclictest0-21swapper/005:23:410
98699784,27cyclictest0-21swapper/010:21:080
98699784,27cyclictest0-21swapper/010:21:080
98699784,27cyclictest0-21swapper/009:03:430
98699784,22cyclictest0-21swapper/010:43:410
98699784,11cyclictest0-21swapper/006:54:100
986997816,11cyclictest0-21swapper/008:58:440
98699776,11cyclictest0-21swapper/010:23:560
98699776,11cyclictest0-21swapper/010:23:560
98699774,27cyclictest3106-21cut06:33:570
98699774,14cyclictest0-21swapper/005:54:010
986997716,10cyclictest0-21swapper/010:08:410
98699764,14cyclictest0-21swapper/007:23:370
98699763,21cyclictest0-21swapper/007:33:550
986997618,8cyclictest10907-21tail10:13:430
98699759,12cyclictest0-21swapper/006:39:060
98699755,13cyclictest0-21swapper/010:38:510
98699753,23cyclictest0-21swapper/010:28:520
986997523,45cyclictest0-21swapper/005:23:560
986997522,8cyclictest1265-21runrttasks09:43:120
986997522,8cyclictest1265-21runrttasks09:43:120
98699746,21cyclictest13328-21ps09:09:020
98699744,23cyclictest0-21swapper/006:28:520
98699743,20cyclictest0-21swapper/007:18:520
986997413,13cyclictest0-21swapper/005:35:280
223099720,4rtkit-daemon2229-21rtkit-daemon08:12:282
80122690,5sleep30-21swapper/308:58:523
989996142,11cyclictest19270-21needreboot05:59:051
989996142,11cyclictest19270-21needreboot05:59:051
98999613,51cyclictest0-21swapper/110:38:521
182732604,12sleep20-21swapper/207:04:102
989995915,24cyclictest0-21swapper/107:23:371
991995838,13cyclictest0-21swapper/208:28:342
98999585,27cyclictest6555-21diskstats10:03:491
989995817,23cyclictest0-21swapper/107:48:451
993995737,15cyclictest0-21swapper/310:03:523
989995741,10cyclictest9406-21gltestperf05:39:021
6332572,9sleep20-21swapper/209:51:432
223099570,5rtkit-daemon2229-21rtkit-daemon07:40:442
993995641,9cyclictest2185-21sed05:24:013
993995638,12cyclictest0-21swapper/307:44:073
993995637,14cyclictest0-21swapper/310:23:303
993995637,14cyclictest0-21swapper/310:23:303
993995624,27cyclictest21536-21cat07:13:373
991995510,37cyclictest0-21swapper/208:14:002
991995510,37cyclictest0-21swapper/208:14:002
98999555,30cyclictest0-21swapper/110:28:551
223099550,5rtkit-daemon2229-21rtkit-daemon07:33:592
993995439,9cyclictest23796-21cat10:38:523
993995439,9cyclictest11503-21forks05:43:593
991995440,9cyclictest0-21swapper/206:53:202
98999548,21cyclictest0-21swapper/110:08:421
98999545,29cyclictest0-21swapper/110:49:061
98999545,28cyclictest0-21swapper/106:34:091
98999544,30cyclictest0-21swapper/106:48:581
989995422,13cyclictest0-21swapper/110:24:021
989995422,13cyclictest0-21swapper/110:24:021
989995422,12cyclictest0-21swapper/108:58:491
321042544,44sleep10-21swapper/106:28:011
202082543,14sleep20-21swapper/206:03:392
202082543,14sleep20-21swapper/206:03:392
993995338,10cyclictest0-21swapper/308:49:053
993995335,12cyclictest0-21swapper/309:03:523
993995333,15cyclictest0-21swapper/306:29:143
99172533,11sleep10-21swapper/107:54:101
991995337,10cyclictest3769-21runrttasks06:34:062
991995337,10cyclictest26344-21chrt09:38:452
991995337,10cyclictest26344-21chrt09:38:452
991995332,17cyclictest0-21swapper/207:21:062
98999535,29cyclictest0-21swapper/109:23:571
98999535,28cyclictest0-21swapper/109:18:511
98999535,28cyclictest0-21swapper/107:39:051
98999534,30cyclictest0-21swapper/107:33:571
989995335,11cyclictest11134-21cpuspeed_turbos07:58:491
989995319,14cyclictest0-21swapper/110:19:001
989995319,14cyclictest0-21swapper/110:19:001
67982533,43sleep10-21swapper/106:40:301
248942533,43sleep10-21swapper/106:12:371
223099530,5rtkit-daemon2229-21rtkit-daemon07:54:142
99399525,40cyclictest0-21swapper/309:18:313
99399523,42cyclictest27622-21munin-node06:18:523
99399523,41cyclictest15628-21aten2.4-expect08:08:453
99199524,42cyclictest0-21swapper/210:33:582
991995237,9cyclictest8223-21idleruntime-cro07:53:342
991995233,13cyclictest0-21swapper/208:19:062
991995224,9cyclictest21315-21latency_hist09:28:322
98999525,28cyclictest0-21swapper/109:38:491
98999525,28cyclictest0-21swapper/109:38:491
98999525,26cyclictest31515-21df09:48:481
989995221,13cyclictest0-21swapper/110:14:021
989995220,13cyclictest0-21swapper/105:53:401
989995218,14cyclictest0-21swapper/107:13:531
52292522,10sleep30-21swapper/308:53:493
52292522,10sleep30-21swapper/308:53:493
317782523,11sleep10-21swapper/108:39:401
254282523,8sleep325447-21python07:19:103
223099520,5rtkit-daemon2229-21rtkit-daemon07:23:372
216642523,43sleep10-21swapper/106:04:071
142012523,12sleep20-21swapper/206:58:372
130182523,10sleep10-21swapper/105:48:111
99399514,40cyclictest1160-21snmpd05:33:073
991995134,12cyclictest0-21swapper/206:39:112
991995133,12cyclictest0-21swapper/210:48:442
99199513,41cyclictest1160-21snmpd09:18:422
991995123,22cyclictest10514-21latency_hist10:13:302
98999515,28cyclictest0-21swapper/109:14:061
98999515,27cyclictest0-21swapper/108:43:541
98999515,27cyclictest0-21swapper/108:23:451
98999515,27cyclictest0-21swapper/106:19:051
98999514,28cyclictest0-21swapper/109:38:321
98999514,28cyclictest0-21swapper/108:53:571
98999514,28cyclictest0-21swapper/108:53:571
98999514,28cyclictest0-21swapper/108:49:071
98999514,27cyclictest0-21swapper/109:13:331
98999514,27cyclictest0-21swapper/105:58:401
989995120,12cyclictest0-21swapper/108:33:581
75002513,40sleep20-21swapper/206:43:502
306392513,11sleep10-21swapper/107:29:521
306392513,11sleep10-21swapper/107:29:521
223099510,5rtkit-daemon2229-21rtkit-daemon07:43:512
138752512,42sleep30-21swapper/306:54:233
993995036,9cyclictest0-21swapper/306:09:123
993995034,12cyclictest0-21swapper/309:33:573
99399503,40cyclictest1160-21snmpd05:58:113
993995029,9cyclictest0-21swapper/306:33:513
991995030,14cyclictest0-21swapper/206:14:042
991995027,11cyclictest0-21swapper/209:34:272
991995023,9cyclictest8287-21latency_hist05:38:412
98999504,28cyclictest0-21swapper/110:43:521
98999504,28cyclictest0-21swapper/106:58:531
98999504,28cyclictest0-21swapper/105:24:071
98999504,27cyclictest0-21swapper/109:58:491
98999504,27cyclictest0-21swapper/105:33:411
98999504,26cyclictest0-21swapper/106:48:381
8682503,39sleep31038-21latency_hist07:38:363
57042502,42sleep30-21swapper/310:03:093
55752502,9sleep20-21swapper/205:29:342
21912502,10sleep20-21swapper/206:33:142
993994932,11cyclictest0-21swapper/308:23:353
993994931,12cyclictest0-21swapper/305:59:023
993994931,12cyclictest0-21swapper/305:59:023
99399493,40cyclictest31088-21munin-node09:48:523
99399492,42cyclictest0-21swapper/307:14:123
991994932,12cyclictest0-21swapper/207:09:132
991994931,13cyclictest0-21swapper/210:23:412
991994931,13cyclictest0-21swapper/210:23:412
991994929,15cyclictest0-21swapper/206:04:042
98999494,26cyclictest0-21swapper/105:38:081
98999494,13cyclictest0-21swapper/108:13:321
989994934,9cyclictest12905-21gltestperf06:54:011
989994916,16cyclictest0-21swapper/108:28:591
32272492,9sleep30-21swapper/309:57:003
283202492,9sleep30-21swapper/307:25:433
255212493,40sleep10-21swapper/107:19:121
223099490,5rtkit-daemon2229-21rtkit-daemon08:03:582
222442493,11sleep10-21swapper/108:19:451
212852493,11sleep10-21swapper/107:12:471
200102492,9sleep30-21swapper/310:32:433
152182492,10sleep30-21swapper/308:06:413
116552492,39sleep20-21swapper/209:07:222
99399485,38cyclictest0-21swapper/306:08:133
99399485,36cyclictest20468-21aten2.4-expect10:33:423
99399484,37cyclictest0-21swapper/308:33:353
993994832,10cyclictest10086-21taskset10:09:033
991994833,10cyclictest0-21swapper/208:29:032
991994831,12cyclictest0-21swapper/210:38:402
991994830,12cyclictest0-21swapper/210:03:542
99199483,38cyclictest0-21swapper/210:13:462
98999484,28cyclictest0-21swapper/109:28:421
98999484,27cyclictest0-21swapper/110:38:291
98999483,24cyclictest0-21swapper/107:03:491
989994819,13cyclictest0-21swapper/107:48:281
91612482,9sleep30-21swapper/306:45:083
307642482,9sleep20-21swapper/207:31:302
307642482,9sleep20-21swapper/207:31:302
247922482,9sleep30-21swapper/308:26:533
18142482,10sleep30-21swapper/308:45:103
111202484,8sleep311191-21sed06:49:113
99399474,38cyclictest0-21swapper/310:43:563
993994733,9cyclictest10192-21grep07:54:113
993994731,10cyclictest12587-21fschecks_count09:08:513
99399473,39cyclictest29012-21grep09:43:463
993994716,14cyclictest0-21swapper/309:23:263
991994730,13cyclictest0-21swapper/208:58:532
99199473,38cyclictest0-21swapper/206:19:042
99199473,38cyclictest0-21swapper/205:48:592
991994726,10cyclictest0-21swapper/210:43:532
991994721,8cyclictest32160-21latency_hist06:28:392
98999475,13cyclictest0-21swapper/106:13:521
98999474,26cyclictest0-21swapper/106:33:171
98999473,27cyclictest0-21swapper/108:14:111
98999473,27cyclictest0-21swapper/108:14:111
81572472,9sleep30-21swapper/305:37:043
308422472,10sleep30-21swapper/307:32:323
308422472,10sleep30-21swapper/307:32:323
234162472,9sleep30-21swapper/309:29:563
210412472,10sleep30-21swapper/309:25:003
151842472,39sleep30-21swapper/305:50:223
129552472,9sleep30-21swapper/308:03:193
99399465,36cyclictest0-21swapper/307:03:223
993994630,11cyclictest0-21swapper/309:38:503
993994630,11cyclictest0-21swapper/309:38:503
991994631,9cyclictest13859-21tune2fs10:18:492
991994631,9cyclictest13859-21tune2fs10:18:492
991994630,12cyclictest0-21swapper/209:13:442
991994629,11cyclictest0-21swapper/210:28:482
991994628,13cyclictest0-21swapper/207:13:512
991994624,11cyclictest0-21swapper/208:38:532
99199462,38cyclictest0-21swapper/209:53:432
99199462,36cyclictest29014-21tr09:43:462
98999466,23cyclictest1186-21aten2.4-expect09:53:431
98999465,13cyclictest0-21swapper/109:44:011
98999464,26cyclictest0-21swapper/108:08:311
98999464,23cyclictest0-21swapper/109:03:441
81422462,10sleep30-21swapper/307:52:473
54352462,38sleep20-21swapper/209:59:372
32472462,10sleep20-21swapper/205:25:222
319812462,9sleep30-21swapper/308:42:203
293532462,9sleep30-21swapper/310:50:043
223099460,4rtkit-daemon2229-21rtkit-daemon07:59:062
176882462,9sleep20-21swapper/205:56:542
139402462,9sleep20-21swapper/209:11:052
99399453,37cyclictest0-21swapper/306:14:093
991994528,12cyclictest0-21swapper/208:53:382
991994528,12cyclictest0-21swapper/208:53:382
99199452,39cyclictest0-21swapper/208:49:072
179802452,9sleep30-21swapper/308:13:393
179802452,9sleep30-21swapper/308:13:393
119262452,9sleep20-21swapper/205:44:052
99399445,34cyclictest0-21swapper/307:38:513
99399444,10cyclictest27324-21munin-plugin-st08:33:333
99199442,38cyclictest0-21swapper/206:58:582
323202442,9sleep20-21swapper/208:43:442
318192442,9sleep30-21swapper/306:24:163
248602442,9sleep20-21swapper/206:12:102
99399434,33cyclictest0-21swapper/305:43:403
99399432,36cyclictest0-21swapper/310:13:423
991994327,11cyclictest0-21swapper/209:28:572
69692432,36sleep30-21swapper/306:42:463
296492432,9sleep20-21swapper/208:38:002
99399424,33cyclictest0-21swapper/310:23:543
99399424,33cyclictest0-21swapper/310:23:543
99199422,35cyclictest0-21swapper/205:39:022
3276424118,14sleep231241-21munin-node05:19:112
3276424118,14sleep231241-21munin-node05:19:112
98999406,19cyclictest0-21swapper/105:23:421
98999406,19cyclictest0-21swapper/105:23:421
312312404,29sleep30-21swapper/305:18:483
312312404,29sleep30-21swapper/305:18:483
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional