You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-14 - 09:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot6s.osadl.org (updated Fri Nov 14, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27459999236,13cyclictest0-21swapper/123:01:521
2745999894,25cyclictest0-21swapper/120:07:181
27459998832,12cyclictest0-21swapper/119:32:131
27459998832,12cyclictest0-21swapper/119:32:131
2745999875,34cyclictest0-21swapper/119:57:101
2745999865,36cyclictest0-21swapper/121:47:051
27455998634,46cyclictest30569-21grep20:17:190
27455998634,46cyclictest30569-21grep20:17:190
27459998538,11cyclictest0-21swapper/120:41:451
2745999844,35cyclictest0-21swapper/122:41:571
2745599843,74cyclictest0-21swapper/019:12:220
2745999835,28cyclictest0-21swapper/123:23:491
2745599837,69cyclictest0-21swapper/022:31:530
2745999825,28cyclictest0-21swapper/119:47:011
2745999824,14cyclictest0-21swapper/122:11:401
27459998230,10cyclictest0-21swapper/119:42:491
27459998210,30cyclictest0-21swapper/121:41:541
2745599823,72cyclictest0-21swapper/023:16:190
27455998224,52cyclictest0-21swapper/021:11:430
2745599813,71cyclictest0-21swapper/021:57:120
2745599813,71cyclictest0-21swapper/000:27:000
27455998123,9cyclictest1369-21runrttasks23:02:060
2745599803,69cyclictest0-21swapper/023:06:540
2745599803,69cyclictest0-21swapper/023:06:540
27455998023,9cyclictest1369-21runrttasks22:10:590
27455998023,9cyclictest1369-21runrttasks21:19:320
27455998022,8cyclictest1369-21runrttasks22:48:480
27455998022,51cyclictest0-21swapper/000:26:360
27455998019,7cyclictest11703-21latency_hist00:11:360
27455998015,8cyclictest3365-21latency_hist22:46:400
2745599793,29cyclictest0-21swapper/021:47:000
27455997925,48cyclictest24328-21cstates22:21:540
27455997925,48cyclictest24328-21cstates22:21:540
27455997925,47cyclictest1369-21runrttasks00:02:100
27455997923,49cyclictest0-21swapper/022:06:410
27455997922,9cyclictest1369-21runrttasks19:59:510
27455997922,50cyclictest0-21swapper/000:18:510
27455997919,53cyclictest0-21swapper/019:32:190
27455997919,53cyclictest0-21swapper/019:32:190
27459997829,13cyclictest0-21swapper/122:12:091
2745599784,8cyclictest1369-21runrttasks22:37:430
2745599784,22cyclictest17696-21yum-updatesd-he21:00:340
2745599783,68cyclictest0-21swapper/022:17:000
27455997824,47cyclictest0-21swapper/020:46:590
27455997823,10cyclictest9718-21processes22:57:070
27455997821,8cyclictest29168-21switchtime19:07:200
27459997731,13cyclictest0-21swapper/121:38:291
27459997731,13cyclictest0-21swapper/121:38:291
27459997731,10cyclictest29261-21grep23:37:141
2745599773,67cyclictest0-21swapper/023:41:500
27455997724,10cyclictest1369-21runrttasks19:19:370
27455997722,10cyclictest30050-21aten_r1power_fr21:26:570
27455997720,8cyclictest10901-21needreboot19:37:140
27455997717,7cyclictest32469-21kworker/u8:022:51:400
27459997631,11cyclictest0-21swapper/121:51:531
2745599763,66cyclictest0-21swapper/000:11:560
2745599763,27cyclictest0-21swapper/023:41:180
2745599763,12cyclictest0-21swapper/020:27:480
27455997627,43cyclictest0-21swapper/021:56:410
27455997624,45cyclictest0-21swapper/023:16:500
27455997623,9cyclictest7235-21missed_timers20:37:110
27455997623,9cyclictest23645-21gltestperf21:12:030
27455997623,9cyclictest21238-21hwlatdetect23:22:030
27455997623,9cyclictest15974-21switchtime19:47:200
27455997623,8cyclictest6255-21proc_pri19:27:160
27455997622,9cyclictest27847-21cpuspeed_turbos21:21:560
27455997621,9cyclictest18288-21cpuspeed21:01:570
27455997621,9cyclictest1369-21runrttasks00:35:330
27455997621,8cyclictest25477-21cstates23:31:540
27459997532,12cyclictest0-21swapper/100:11:511
2745599754,8cyclictest13503-21cpuspeed_turbos20:51:580
2745599754,25cyclictest28594-21idleruntime-cro20:16:450
2745599753,28cyclictest0-21swapper/023:48:580
2745599753,26cyclictest1900-21chrt21:34:090
27455997523,9cyclictest1369-21runrttasks19:42:510
27455997522,8cyclictest4019-21timerandwakeup21:37:180
27455997522,8cyclictest4019-21timerandwakeup21:37:180
27455997522,8cyclictest23397-21fschecks_time23:26:590
27455997520,8cyclictest24045-21munin-node20:07:170
27455997520,10cyclictest19177-21munin-node22:11:570
27459997431,13cyclictest0-21swapper/119:52:011
27455997422,9cyclictest1369-21runrttasks23:52:280
27455997422,9cyclictest1369-21runrttasks23:52:280
27455997421,8cyclictest27079-21tune2fs22:27:000
27455997421,8cyclictest1369-21runrttasks19:53:470
27455997419,12cyclictest0-21swapper/020:22:120
27459997331,13cyclictest0-21swapper/122:21:221
2745599734,11cyclictest0-21swapper/000:01:170
2745599733,28cyclictest0-21swapper/019:24:480
2745599724,27cyclictest9219-21cut20:42:060
2745599723,28cyclictest0-21swapper/020:03:050
2745599713,28cyclictest0-21swapper/020:35:070
27455997118,8cyclictest1618-21nscd21:42:080
243799700,9rtkit-daemon0-21swapper/219:27:103
27455996519,6cyclictest27435-21latency_hist19:06:490
27459996242,14cyclictest0-21swapper/122:47:121
27459996242,14cyclictest0-21swapper/120:47:171
27460996043,10cyclictest10474-21grep21:52:012
27459995933,7cyclictest18860-21latency_hist00:26:361
2746199584,31cyclictest0-21swapper/321:56:583
2746199583,29cyclictest0-21swapper/321:16:383
27459995837,15cyclictest0-21swapper/121:32:141
27459995738,13cyclictest0-21swapper/123:16:561
27459995737,14cyclictest0-21swapper/120:16:581
27459995737,14cyclictest0-21swapper/120:16:581
2746199564,29cyclictest0-21swapper/319:42:033
2746199564,28cyclictest0-21swapper/320:23:203
2746099569,28cyclictest0-21swapper/222:16:402
2746199554,29cyclictest0-21swapper/321:37:133
2746199554,29cyclictest0-21swapper/321:37:133
2746199554,29cyclictest0-21swapper/319:07:223
2746099556,42cyclictest26-21rcuc/223:10:222
2746099556,42cyclictest26-21rcuc/223:10:222
27459995536,13cyclictest0-21swapper/123:36:381
27459995536,12cyclictest0-21swapper/123:01:191
2746199544,29cyclictest0-21swapper/321:07:083
2746199544,28cyclictest0-21swapper/320:27:213
2746199544,28cyclictest0-21swapper/319:52:193
2746199544,28cyclictest0-21swapper/300:11:593
2746199543,29cyclictest0-21swapper/319:12:103
27461995420,13cyclictest0-21swapper/323:06:533
27461995420,13cyclictest0-21swapper/323:06:533
27461995420,13cyclictest0-21swapper/320:11:463
2746099544,43cyclictest0-21swapper/219:07:212
27460995436,11cyclictest1654-21unixbench_multi21:32:142
27459995435,14cyclictest0-21swapper/121:22:161
92202534,42sleep20-21swapper/221:49:402
2746099535,31cyclictest0-21swapper/219:31:402
2746099535,25cyclictest0-21swapper/221:26:582
2746099534,22cyclictest0-21swapper/220:12:052
27460995336,10cyclictest25452-21cpuspeed_turbos23:31:542
27460995336,10cyclictest19955-21gltestperf00:26:572
7812522,9sleep20-21swapper/220:25:242
2746199524,28cyclictest0-21swapper/323:47:103
2746199524,28cyclictest0-21swapper/320:02:223
2746199524,27cyclictest0-21swapper/321:56:423
27461995234,11cyclictest3231-21gltestperf23:51:583
27461995234,11cyclictest3231-21gltestperf23:51:583
27461995220,12cyclictest0-21swapper/319:37:153
27460995221,13cyclictest0-21swapper/223:16:502
2745999526,28cyclictest0-21swapper/119:12:091
27459995238,8cyclictest5967-21date22:51:401
27459995237,10cyclictest15623-21munin-node20:57:141
27459995234,13cyclictest0-21swapper/123:26:571
27459995234,13cyclictest0-21swapper/120:27:161
27459995234,13cyclictest0-21swapper/119:22:011
27459995231,15cyclictest0-21swapper/119:17:131
25132523,43sleep20-21swapper/222:42:122
91492512,9sleep30-21swapper/319:32:413
91492512,9sleep30-21swapper/319:32:413
2746199514,29cyclictest0-21swapper/321:47:083
2746199514,29cyclictest0-21swapper/300:22:013
2746199514,28cyclictest0-21swapper/320:42:143
2746199514,26cyclictest0-21swapper/319:47:463
2746199514,25cyclictest0-21swapper/300:31:493
27461995133,11cyclictest11724-21gltestperf20:47:063
27461995123,23cyclictest0-21swapper/321:33:003
27461995120,13cyclictest0-21swapper/320:32:073
2746199512,23cyclictest0-21swapper/321:26:233
2746099517,26cyclictest0-21swapper/220:32:072
2746099515,28cyclictest0-21swapper/221:00:472
2746099515,12cyclictest0-21swapper/200:03:562
2746099514,29cyclictest0-21swapper/220:46:442
2746099514,28cyclictest0-21swapper/200:22:072
27460995134,10cyclictest9065-21unixbench_singl19:32:212
27460995134,10cyclictest9065-21unixbench_singl19:32:212
27460995121,12cyclictest0-21swapper/200:06:552
27459995132,13cyclictest0-21swapper/122:02:131
272622512,10sleep20-21swapper/221:20:462
234512512,10sleep20-21swapper/200:34:212
224012512,10sleep20-21swapper/223:25:492
212132514,39sleep321278-21idleruntime23:22:033
2746199505,27cyclictest0-21swapper/323:17:053
2746199504,29cyclictest0-21swapper/320:12:213
2746199504,28cyclictest0-21swapper/322:58:453
2746199504,28cyclictest0-21swapper/322:32:023
2746199504,28cyclictest0-21swapper/320:37:063
2746199504,27cyclictest0-21swapper/320:57:233
2746199504,24cyclictest0-21swapper/300:01:523
2746199503,29cyclictest0-21swapper/321:42:023
27461995018,13cyclictest0-21swapper/300:17:123
27461995017,9cyclictest10839-21telnet23:01:513
2746099505,28cyclictest0-21swapper/222:21:532
2746099505,28cyclictest0-21swapper/222:21:532
2746099505,27cyclictest0-21swapper/219:37:022
2746099504,32cyclictest0-21swapper/221:01:552
2746099504,29cyclictest0-21swapper/220:07:022
2746099504,28cyclictest0-21swapper/223:47:002
2746099504,28cyclictest0-21swapper/222:57:072
2746099504,28cyclictest0-21swapper/222:36:592
2746099504,28cyclictest0-21swapper/222:17:092
2746099504,28cyclictest0-21swapper/221:57:192
2746099504,28cyclictest0-21swapper/220:47:022
27460995020,14cyclictest0-21swapper/222:27:132
27460995020,13cyclictest0-21swapper/219:17:122
2745999504,27cyclictest0-21swapper/100:18:591
27459995035,9cyclictest24568-21unixbench_singl21:12:171
27459995020,10cyclictest7255-21cat19:31:481
264752500,14sleep226477-21strings19:02:092
258972502,10sleep30-21swapper/322:24:113
258972502,10sleep30-21swapper/322:24:113
204242504,38sleep20-21swapper/221:06:562
187392502,10sleep20-21swapper/219:53:202
2746199495,26cyclictest0-21swapper/322:08:363
2746199494,27cyclictest0-21swapper/322:11:573
2746199494,27cyclictest0-21swapper/322:01:543
2746199494,27cyclictest0-21swapper/321:27:083
2746199494,26cyclictest20015-21hwlatdetect00:26:583
2746199494,26cyclictest0-21swapper/322:28:393
2746199493,28cyclictest0-21swapper/323:27:073
2746099495,26cyclictest0-21swapper/221:13:142
2746099494,28cyclictest0-21swapper/222:52:032
2746099494,28cyclictest0-21swapper/220:02:062
2746099494,28cyclictest0-21swapper/219:12:112
2746099494,27cyclictest0-21swapper/220:00:482
2746099494,27cyclictest0-21swapper/219:47:112
2746099494,27cyclictest0-21swapper/219:22:212
27460994932,11cyclictest6492-21grep23:57:102
2746099493,28cyclictest0-21swapper/220:27:152
27460994918,13cyclictest0-21swapper/223:11:562
27460994918,12cyclictest0-21swapper/219:42:182
27459994934,9cyclictest31113-21munin-node22:36:531
27459994934,10cyclictest9211-21sshd00:05:171
2745999493,39cyclictest0-21swapper/119:07:211
2746199488,21cyclictest29981-21aten_r1power_po23:41:513
2746199483,27cyclictest0-21swapper/323:31:593
27461994819,12cyclictest0-21swapper/321:17:193
2746099484,28cyclictest0-21swapper/223:53:542
2746099484,28cyclictest0-21swapper/223:53:542
2746099484,27cyclictest0-21swapper/223:37:022
2746099484,27cyclictest0-21swapper/222:46:582
2746099484,27cyclictest0-21swapper/221:37:132
2746099484,27cyclictest0-21swapper/221:37:132
2746099484,27cyclictest0-21swapper/200:11:522
2746099484,26cyclictest0-21swapper/223:01:542
2746099484,25cyclictest0-21swapper/200:16:522
2745999485,26cyclictest0-21swapper/120:43:091
2745999484,37cyclictest0-21swapper/120:52:091
2745999484,26cyclictest0-21swapper/123:57:031
27459994832,11cyclictest0-21swapper/120:31:501
27459994830,13cyclictest0-21swapper/123:10:171
27459994830,13cyclictest0-21swapper/123:10:171
2745999483,39cyclictest0-21swapper/123:47:041
2746199474,27cyclictest0-21swapper/322:47:193
2746199474,27cyclictest0-21swapper/320:53:113
2746199474,26cyclictest0-21swapper/322:18:213
2746199474,26cyclictest0-21swapper/319:57:023
2746199474,25cyclictest0-21swapper/300:08:573
2746199474,14cyclictest0-21swapper/322:37:003
27461994730,10cyclictest29181-21unixbench_multi23:37:143
27461994720,12cyclictest0-21swapper/319:17:033
27461994716,13cyclictest0-21swapper/321:02:003
27461994710,21cyclictest15647-21seq23:11:513
2746099475,25cyclictest0-21swapper/220:18:052
2746099475,25cyclictest0-21swapper/220:18:052
2746099474,26cyclictest0-21swapper/222:33:402
2746099474,26cyclictest0-21swapper/221:22:112
2746099474,26cyclictest0-21swapper/220:38:082
27460994728,12cyclictest30736-21gltestperf23:41:582
2745999475,25cyclictest0-21swapper/122:32:011
2745999474,37cyclictest26101-21munin-run22:26:401
2745999474,37cyclictest26101-21munin-run22:26:401
27459994732,10cyclictest0-21swapper/120:21:491
27459994731,12cyclictest0-21swapper/121:07:411
27459994730,12cyclictest0-21swapper/123:51:451
27459994730,12cyclictest0-21swapper/123:51:451
27459994730,12cyclictest0-21swapper/121:30:291
27459994730,12cyclictest0-21swapper/100:08:051
2746199464,25cyclictest0-21swapper/322:51:523
2746199464,22cyclictest0-21swapper/319:22:003
2746199463,19cyclictest0-21swapper/320:18:043
2746199463,19cyclictest0-21swapper/320:18:043
2746099464,26cyclictest0-21swapper/222:07:102
2746099464,25cyclictest0-21swapper/223:27:142
2746099464,25cyclictest0-21swapper/221:42:182
2745999464,36cyclictest0-21swapper/123:42:111
2745999463,39cyclictest0-21swapper/120:12:201
27459994629,11cyclictest0-21swapper/121:17:161
2746199453,24cyclictest0-21swapper/323:57:193
2745999456,33cyclictest0-21swapper/123:14:301
162822453,36sleep20-21swapper/222:03:102
2746199443,20cyclictest0-21swapper/322:41:563
2746099444,24cyclictest0-21swapper/220:52:012
2745999446,31cyclictest0-21swapper/100:31:481
2745999444,12cyclictest0-21swapper/119:41:471
27459994431,9cyclictest22471-21gltestperf20:02:081
2745999434,33cyclictest0-21swapper/122:28:011
2745999433,33cyclictest0-21swapper/121:57:141
27459994329,10cyclictest0-21swapper/121:02:031
27461994211,13cyclictest0-21swapper/319:06:493
27459994227,9cyclictest19867-21grep00:26:571
2745999403,31cyclictest0-21swapper/119:06:491
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional