You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-28 - 23:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot6s.osadl.org (updated Sat Jun 28, 2025 12:43:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2114221040,4sleep121140-21cut10:15:011
166122910,3sleep116769-21cut11:14:391
128362840,7sleep21439499cyclictest08:50:032
14392997444,13cyclictest0-21swapper/107:14:491
1439599724,61cyclictest0-21swapper/311:04:583
304732710,3sleep041ktimersoftd/007:10:210
1438799693,58cyclictest0-21swapper/008:30:170
1438799693,58cyclictest0-21swapper/008:30:170
14395996741,11cyclictest0-21swapper/310:29:413
1439599674,57cyclictest0-21swapper/306:45:043
1438799655,53cyclictest0-21swapper/010:25:110
1438799644,53cyclictest0-21swapper/011:14:560
226022634,52sleep00-21swapper/010:19:410
1438799634,51cyclictest0-21swapper/011:35:090
35852623,51sleep13651-21latency08:30:121
35852623,51sleep13651-21latency08:30:121
1439599626,50cyclictest0-21swapper/310:20:153
1439299624,50cyclictest0-21swapper/108:59:431
1438799625,50cyclictest0-21swapper/012:04:360
1439599615,49cyclictest0-21swapper/308:40:193
1439499603,51cyclictest0-21swapper/208:30:202
1439499603,51cyclictest0-21swapper/208:30:202
14392996020,34cyclictest29345-21unixbench-2d08:15:201
1439599594,49cyclictest0-21swapper/309:05:213
1439599594,48cyclictest0-21swapper/307:24:433
1438799594,48cyclictest27368-21grep07:05:100
1438799594,48cyclictest27368-21grep07:05:100
1438799594,48cyclictest0-21swapper/008:35:070
1438799593,50cyclictest0-21swapper/011:34:380
14387995915,12cyclictest10983-21munin-node07:40:230
223099580,4rtkit-daemon2229-21rtkit-daemon06:39:591
1439299584,48cyclictest22152-21timerwakeupswit08:00:201
14392995832,10cyclictest8276-21crond09:49:411
1438799585,47cyclictest0-21swapper/009:44:430
1438799584,47cyclictest25905-21fschecks_count08:10:050
143092584,47sleep00-21swapper/007:45:130
14395995733,17cyclictest24611-21unixbench_multi08:05:243
14392995733,10cyclictest3954-21telnet07:25:011
1438799575,24cyclictest0-21swapper/008:20:200
1438799574,47cyclictest0-21swapper/007:33:060
1438799573,49cyclictest0-21swapper/008:15:150
1438799573,48cyclictest0-21swapper/010:00:040
1438799573,48cyclictest0-21swapper/009:09:440
1438799573,11cyclictest0-21swapper/006:39:490
14387995714,13cyclictest0-21swapper/011:25:050
14387995714,13cyclictest0-21swapper/011:25:050
100532573,47sleep010181-21head12:05:120
1439599562,9cyclictest0-21swapper/309:35:053
1439599562,10cyclictest0-21swapper/307:55:043
1439299564,29cyclictest0-21swapper/107:55:161
1438799563,48cyclictest0-21swapper/011:49:360
1438799563,48cyclictest0-21swapper/008:44:450
1438799563,26cyclictest0-21swapper/011:54:580
1438799563,25cyclictest0-21swapper/009:20:010
65542553,44sleep26690-21df_abs09:45:012
281962550,11sleep21439499cyclictest07:05:232
281962550,11sleep21439499cyclictest07:05:232
191412553,11sleep30-21swapper/310:10:093
17272553,9sleep00-21swapper/007:20:100
1439599552,37cyclictest0-21swapper/310:59:523
14392995536,13cyclictest0-21swapper/106:49:491
14392995533,8cyclictest8431-21munin-node12:04:531
14392995518,23cyclictest0-21swapper/111:27:521
14392995518,23cyclictest0-21swapper/111:27:521
1438799557,21cyclictest0-21swapper/008:53:490
1438799554,45cyclictest1116-21lldpd11:09:580
1438799554,24cyclictest0-21swapper/009:54:590
1438799554,24cyclictest0-21swapper/009:54:590
1438799553,46cyclictest0-21swapper/010:10:120
1438799553,26cyclictest0-21swapper/010:29:590
1438799553,25cyclictest0-21swapper/008:58:530
1438799553,24cyclictest0-21swapper/011:00:080
14387995519,11cyclictest0-21swapper/010:20:010
14387995515,12cyclictest0-21swapper/010:55:130
226012542,9sleep20-21swapper/208:03:492
1439599546,29cyclictest0-21swapper/311:30:073
1439599546,29cyclictest0-21swapper/310:35:033
1439599545,41cyclictest0-21swapper/311:14:543
1439599545,27cyclictest0-21swapper/310:09:093
1439599544,13cyclictest0-21swapper/309:49:413
1439599544,11cyclictest12167-21tail08:49:563
14395995422,14cyclictest0-21swapper/307:30:253
14392995435,12cyclictest0-21swapper/108:49:441
14392995434,7cyclictest4973-21munin-node08:35:081
1438799545,14cyclictest0-21swapper/009:50:200
1438799544,29cyclictest0-21swapper/007:55:060
1438799544,26cyclictest0-21swapper/006:50:240
1438799544,25cyclictest0-21swapper/011:05:140
1438799543,46cyclictest0-21swapper/006:45:180
1438799543,45cyclictest4896-21munin-node10:50:110
1438799543,45cyclictest0-21swapper/008:25:290
1438799543,24cyclictest0-21swapper/009:35:080
1438799543,23cyclictest1593-21nscd07:35:170
309242534,13sleep10-21swapper/111:44:371
1439599535,29cyclictest0-21swapper/311:25:003
1439599535,29cyclictest0-21swapper/311:25:003
1439599535,29cyclictest0-21swapper/309:00:063
1439599534,30cyclictest0-21swapper/310:04:423
1439599534,30cyclictest0-21swapper/309:30:063
1439599534,28cyclictest0-21swapper/311:54:373
1439599534,28cyclictest0-21swapper/307:19:483
1439599532,10cyclictest0-21swapper/306:40:143
1439299534,43cyclictest6341-21python10:50:141
14392995334,13cyclictest0-21swapper/107:44:471
14392995328,11cyclictest13398-21telnet07:45:021
14392995319,27cyclictest23770-21hwlatdetect10:20:061
1438799533,45cyclictest32409-21munin-node10:40:030
1438799533,45cyclictest32409-21munin-node10:40:030
1438799533,44cyclictest19222-21munin-node11:20:040
1438799533,43cyclictest8279-21latency_hist09:49:420
1438799533,27cyclictest1494-21tr11:49:540
1438799533,26cyclictest0-21swapper/006:55:170
1438799533,23cyclictest0-21swapper/009:09:590
1438799533,23cyclictest0-21swapper/007:15:040
1438799532,45cyclictest0-21swapper/010:49:400
128902533,11sleep30-21swapper/307:41:413
1439599526,39cyclictest0-21swapper/311:39:513
1439599525,29cyclictest0-21swapper/306:59:493
1439599524,29cyclictest0-21swapper/308:20:073
1439599524,28cyclictest0-21swapper/309:25:113
1439599524,26cyclictest0-21swapper/312:04:373
1439599524,25cyclictest0-21swapper/310:29:583
14395995235,10cyclictest15274-21head11:09:583
14395995233,12cyclictest5441-21unixbench_multi09:40:173
1439599523,27cyclictest0-21swapper/307:48:093
14395995221,13cyclictest0-21swapper/310:54:403
14394995220,26cyclictest32651-21proc_pri07:15:192
14392995233,7cyclictest23049-21kworker/1:010:09:091
14392995233,14cyclictest0-21swapper/107:30:071
14392995231,9cyclictest1160-21snmpd11:04:191
14392995228,8cyclictest7372-21cut08:39:571
14392995215,23cyclictest0-21swapper/107:04:511
14392995215,23cyclictest0-21swapper/107:04:511
1438799525,28cyclictest0-21swapper/011:39:570
1438799524,26cyclictest0-21swapper/008:05:030
1438799523,43cyclictest0-21swapper/010:35:000
1438799523,26cyclictest0-21swapper/009:30:060
1438799523,23cyclictest0-21swapper/009:14:570
306892513,42sleep00-21swapper/009:25:480
285462512,9sleep20-21swapper/209:23:572
241202512,9sleep10-21swapper/108:05:171
235462512,42sleep10-21swapper/109:10:581
230302513,41sleep223055-21open_files06:55:172
177312513,41sleep00-21swapper/007:52:300
177312513,41sleep00-21swapper/007:52:300
1439599515,37cyclictest0-21swapper/307:10:203
1439599514,38cyclictest11233-21cpuspeed_turbos09:55:003
1439599514,38cyclictest11233-21cpuspeed_turbos09:55:003
1439599514,29cyclictest0-21swapper/310:44:403
1439599514,29cyclictest0-21swapper/310:44:403
1439599514,29cyclictest0-21swapper/308:00:173
1439599514,28cyclictest0-21swapper/308:25:183
1439599514,28cyclictest0-21swapper/308:10:213
1439599514,28cyclictest0-21swapper/307:29:483
1439599514,28cyclictest0-21swapper/307:00:083
1439599514,27cyclictest0-21swapper/309:19:433
14395995121,24cyclictest3001-21df_abs08:30:043
14395995121,24cyclictest3001-21df_abs08:30:043
14395995120,13cyclictest0-21swapper/307:05:093
14395995120,13cyclictest0-21swapper/307:05:093
14395995119,14cyclictest0-21swapper/312:08:353
14395995116,14cyclictest0-21swapper/308:35:083
1439499514,30cyclictest0-21swapper/210:35:112
14394995132,13cyclictest0-21swapper/210:39:582
14394995132,13cyclictest0-21swapper/210:39:582
1439299514,29cyclictest0-21swapper/108:53:091
1439299514,28cyclictest0-21swapper/110:39:551
1439299514,28cyclictest0-21swapper/110:39:551
1439299513,42cyclictest0-21swapper/111:19:381
14392995129,9cyclictest25011-21iostat_ios11:30:031
14392995123,12cyclictest0-21swapper/106:50:231
14392995122,24cyclictest0-21swapper/109:31:181
14392995121,13cyclictest0-21swapper/109:14:571
1438799514,24cyclictest0-21swapper/008:44:570
14387995135,10cyclictest17760-21grep09:00:090
95412502,9sleep10-21swapper/109:50:081
1439599506,37cyclictest0-21swapper/310:54:543
1439599504,39cyclictest0-21swapper/307:50:023
1439599504,39cyclictest0-21swapper/307:50:023
1439599504,27cyclictest0-21swapper/310:47:583
1439599504,27cyclictest0-21swapper/309:51:583
1439599504,27cyclictest0-21swapper/308:58:243
14395995025,16cyclictest0-21swapper/308:14:593
14395995017,14cyclictest0-21swapper/311:24:383
14395995016,14cyclictest0-21swapper/311:44:593
14394995035,10cyclictest0-21swapper/212:03:062
14394995035,10cyclictest0-21swapper/210:56:192
1439499503,42cyclictest0-21swapper/210:22:412
14392995031,7cyclictest18646-21diskstats10:10:011
14392995031,13cyclictest0-21swapper/111:59:591
14392995027,10cyclictest0-21swapper/111:35:091
14392995022,13cyclictest0-21swapper/110:29:401
1438799504,23cyclictest0-21swapper/010:05:080
1438799503,41cyclictest0-21swapper/007:02:320
258842492,9sleep20-21swapper/211:31:122
181332492,9sleep20-21swapper/207:54:502
1439599494,26cyclictest0-21swapper/307:38:073
1439599493,8cyclictest23141-21python09:10:183
1439599493,7cyclictest56962sleep311:58:173
1439499494,40cyclictest0-21swapper/209:55:142
1439499494,40cyclictest0-21swapper/209:55:142
14394994921,12cyclictest0-21swapper/210:05:022
14394994921,12cyclictest0-21swapper/209:15:202
14392994933,10cyclictest4069-21cut09:39:581
1438799494,38cyclictest0-21swapper/007:25:020
1438799494,24cyclictest0-21swapper/008:00:000
1438799494,10cyclictest0-21swapper/006:40:130
8402482,9sleep20-21swapper/211:49:262
56052482,40sleep10-21swapper/111:57:071
319432482,9sleep10-21swapper/111:45:021
211442482,39sleep10-21swapper/109:05:461
152842482,39sleep10-21swapper/110:02:411
145972482,39sleep20-21swapper/208:54:592
1439599486,36cyclictest0-21swapper/311:34:533
1439599485,14cyclictest0-21swapper/306:50:253
1439599482,40cyclictest0-21swapper/309:20:093
1439499484,27cyclictest0-21swapper/207:20:212
1439499483,38cyclictest0-21swapper/207:45:032
1439499483,28cyclictest0-21swapper/208:05:082
1439299484,37cyclictest0-21swapper/111:19:551
14392994828,8cyclictest1446-21telnet07:20:061
14392994822,10cyclictest8214-21idleruntime10:55:041
14392994819,14cyclictest0-21swapper/111:54:371
83352472,38sleep20-21swapper/207:34:312
14394994732,10cyclictest0-21swapper/210:29:582
14394994728,13cyclictest0-21swapper/209:25:032
14394994725,7cyclictest5892-21latency_hist09:44:422
14394994716,25cyclictest0-21swapper/207:04:482
14394994710,31cyclictest1160-21snmpd09:00:132
1439299477,35cyclictest29790-21awk10:34:401
1439299475,36cyclictest31973-21runrttasks08:21:311
1439299473,27cyclictest0-21swapper/107:00:171
14392994729,11cyclictest11765-21sed09:55:061
14392994729,11cyclictest11765-21sed09:55:061
14392994728,13cyclictest0-21swapper/107:35:211
14392994726,9cyclictest30085-21telnet10:34:581
32192462,37sleep20-21swapper/209:36:062
20792462,9sleep20-21swapper/208:27:142
164852462,9sleep20-21swapper/206:41:182
1439599465,22cyclictest0-21swapper/310:14:563
1439599465,12cyclictest0-21swapper/308:48:483
1439499463,38cyclictest0-21swapper/210:48:282
1439499463,37cyclictest0-21swapper/209:10:152
1439499463,36cyclictest0-21swapper/206:50:032
14394994620,11cyclictest0-21swapper/207:44:472
1439299464,28cyclictest0-21swapper/108:11:091
14392994632,10cyclictest0-21swapper/109:04:141
14392994631,10cyclictest0-21swapper/106:39:491
14392994630,9cyclictest32520-21tail08:24:581
1439299463,36cyclictest0-21swapper/111:09:391
14392994627,13cyclictest0-21swapper/110:49:401
308042452,9sleep10-21swapper/109:27:191
1439499455,10cyclictest0-21swapper/212:09:102
1439499454,35cyclictest0-21swapper/206:49:492
1439499454,34cyclictest0-21swapper/208:40:092
14394994532,7cyclictest281ktimersoftd/209:50:062
1439499453,37cyclictest0-21swapper/208:44:572
14394994523,7cyclictest29487-21sed07:10:052
14394994520,11cyclictest0-21swapper/211:49:522
14394994518,12cyclictest0-21swapper/209:08:232
14394994511,8cyclictest7117-21crond08:39:442
1439299455,33cyclictest31416-21aten_r1power_po07:15:031
14392994530,10cyclictest0-21swapper/106:55:011
1439299453,27cyclictest0-21swapper/109:19:581
14392994527,8cyclictest15856-21telnet07:50:021
14392994527,8cyclictest15856-21telnet07:50:021
141082452,37sleep20-21swapper/211:08:312
54972442,36sleep20-21swapper/211:55:432
312252442,34sleep20-21swapper/209:29:492
153212444,34sleep20-21swapper/210:03:102
1439499444,34cyclictest0-21swapper/208:10:142
1439499443,35cyclictest0-21swapper/207:50:022
1439499443,35cyclictest0-21swapper/207:50:022
1439499443,34cyclictest0-21swapper/210:59:532
14394994429,10cyclictest0-21swapper/211:15:012
14394994428,11cyclictest0-21swapper/211:39:522
14394994425,13cyclictest0-21swapper/208:15:122
14394994424,9cyclictest0-21swapper/211:35:172
14394994424,9cyclictest0-21swapper/208:20:102
14394994414,24cyclictest20578-21tail10:14:562
67662432,35sleep20-21swapper/210:51:102
211382432,35sleep20-21swapper/211:21:452
199042432,35sleep20-21swapper/210:10:402
165122432,9sleep20-21swapper/211:13:392
14394994328,11cyclictest0-21swapper/207:35:132
1439299434,34cyclictest0-21swapper/109:34:571
1439499424,33cyclictest0-21swapper/211:24:512
1439499424,33cyclictest0-21swapper/211:24:512
14394994216,12cyclictest0-21swapper/207:25:102
14394994123,12cyclictest0-21swapper/210:24:572
1439599392,31cyclictest0-21swapper/306:39:493
14394993817,15cyclictest0-21swapper/206:39:492
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional