You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-07 - 02:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot6s.osadl.org (updated Mon Jul 07, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1423899915,39cyclictest0-21swapper/023:32:040
1423899894,38cyclictest0-21swapper/022:47:020
14238998720,14cyclictest0-21swapper/022:06:410
14238998129,9cyclictest0-21swapper/023:50:400
14238997829,9cyclictest0-21swapper/023:29:170
14243996333,12cyclictest0-21swapper/219:26:492
14243996333,12cyclictest0-21swapper/219:26:492
14244996236,14cyclictest0-21swapper/323:25:193
14243995932,12cyclictest0-21swapper/220:11:222
14243995932,12cyclictest0-21swapper/220:11:222
14243995931,13cyclictest0-21swapper/223:57:032
14238995931,11cyclictest0-21swapper/023:01:390
190222585,45sleep00-21swapper/019:47:120
14241995829,14cyclictest0-21swapper/121:31:531
14241995829,14cyclictest0-21swapper/121:31:531
29852575,44sleep00-21swapper/020:22:060
143812573,12sleep20-21swapper/220:46:592
143812573,12sleep20-21swapper/220:46:592
1424499564,46cyclictest0-21swapper/320:12:183
14244995611,38cyclictest0-21swapper/319:57:043
1424399553,13cyclictest0-21swapper/221:36:162
1424399553,13cyclictest0-21swapper/221:36:162
14241995530,20cyclictest2376-21cat20:21:471
14241995528,11cyclictest0-21swapper/119:22:221
14241995528,11cyclictest0-21swapper/119:22:221
14238995536,12cyclictest0-21swapper/019:57:020
123452556,41sleep312496-21df_abs18:27:073
79232543,45sleep10-21swapper/121:39:091
16832544,43sleep00-21swapper/022:32:440
1424499546,40cyclictest28480-21irqstats22:22:063
1424499545,42cyclictest5868-21memory22:42:073
1424499545,31cyclictest0-21swapper/321:42:173
1424499544,44cyclictest0-21swapper/323:02:143
1424499544,44cyclictest0-21swapper/323:02:143
1424499543,45cyclictest0-21swapper/319:42:093
1424499543,44cyclictest28561-21iostat_ios20:07:113
1424499543,44cyclictest28561-21iostat_ios20:07:113
1424499543,44cyclictest0-21swapper/318:47:033
14243995439,9cyclictest9661-21strings19:27:202
1424399543,34cyclictest0-21swapper/219:57:172
1424399543,13cyclictest0-21swapper/222:12:102
1424199544,12cyclictest1119-21munin-node19:12:151
1424199543,29cyclictest0-21swapper/123:22:101
14241995428,11cyclictest0-21swapper/118:42:141
14241995428,11cyclictest0-21swapper/118:42:141
14241995427,11cyclictest0-21swapper/121:52:101
14241995426,12cyclictest0-21swapper/123:41:521
14238995430,11cyclictest0-21swapper/022:37:150
90962533,9sleep30-21swapper/320:33:183
295262533,43sleep10-21swapper/120:09:461
295262533,43sleep10-21swapper/120:09:461
1424499535,41cyclictest8288-21munin-node23:57:103
1424499535,41cyclictest26090-21iostat_ios22:17:053
1424499535,41cyclictest26090-21iostat_ios22:17:053
1424499535,41cyclictest17988-21munin-node19:47:103
1424499534,43cyclictest0-21swapper/322:07:053
1424499533,43cyclictest0-21swapper/319:22:063
1424499533,43cyclictest0-21swapper/319:22:063
14244995322,25cyclictest13381-21switchtime22:57:093
1424199534,30cyclictest0-21swapper/123:27:011
1424199533,45cyclictest0-21swapper/122:26:571
14241995329,10cyclictest0-21swapper/121:26:591
14241995327,10cyclictest0-21swapper/120:27:001
14241995319,27cyclictest4591-21nscd21:46:421
14238995331,10cyclictest0-21swapper/021:17:100
91172522,9sleep30-21swapper/322:49:403
68692522,43sleep00-21swapper/022:46:300
319432524,41sleep00-21swapper/020:15:090
266282523,7sleep226720-21df_abs23:26:552
259992522,9sleep30-21swapper/321:10:543
163652522,43sleep00-21swapper/023:05:450
163652522,43sleep00-21swapper/023:05:450
1424499524,41cyclictest29582-21yum-updatesd-he22:27:023
1424499523,43cyclictest0-21swapper/323:42:053
1424499523,42cyclictest0-21swapper/323:11:403
1424399529,7cyclictest0-21swapper/222:11:412
14243995237,9cyclictest18653-21grep22:02:012
14243995237,9cyclictest1709-21grep23:41:572
1424399523,32cyclictest0-21swapper/219:56:462
1424199525,38cyclictest0-21swapper/121:46:571
14241995236,10cyclictest24469-21unixbench_multi19:57:221
14241995229,8cyclictest14181-21taskset20:46:451
14241995225,12cyclictest0-21swapper/121:02:161
14241995224,12cyclictest0-21swapper/122:46:541
14241995223,12cyclictest0-21swapper/122:11:411
14241995216,31cyclictest1265-21runrttasks23:37:571
14238995222,10cyclictest14236-21awk18:31:490
103252523,43sleep10-21swapper/119:28:221
59022513,41sleep05991-21aten2.4-expect23:51:510
54792512,9sleep30-21swapper/323:49:203
318832513,9sleep20-21swapper/220:14:222
153552515,39sleep00-21swapper/020:47:090
153552515,39sleep00-21swapper/020:47:090
1424499514,41cyclictest0-21swapper/322:01:553
1424499514,40cyclictest28792-21telnet23:31:493
1424499514,40cyclictest0-21swapper/321:22:053
1424499514,40cyclictest0-21swapper/320:57:153
1424499513,41cyclictest0-21swapper/318:37:073
1424499513,41cyclictest0-21swapper/318:37:073
14243995135,10cyclictest29295-21grep23:31:552
14243995129,10cyclictest0-21swapper/219:02:132
14243995121,12cyclictest0-21swapper/222:47:012
14241995128,10cyclictest19174-21aten2.4-expect20:56:561
14241995127,12cyclictest0-21swapper/123:12:041
14241995126,8cyclictest9515-21munin-node22:52:021
14241995124,13cyclictest0-21swapper/123:06:391
14241995124,13cyclictest0-21swapper/123:06:391
14241995123,12cyclictest0-21swapper/120:36:451
14241995122,13cyclictest0-21swapper/122:16:411
14241995122,11cyclictest0-21swapper/118:32:201
14241995120,9cyclictest13148-21awk22:57:041
14238995128,18cyclictest0-21swapper/023:20:430
293062504,39sleep10-21swapper/122:22:411
283582502,9sleep20-21swapper/221:15:322
257972502,9sleep20-21swapper/221:08:152
236152503,41sleep10-21swapper/118:49:531
186132502,9sleep20-21swapper/223:09:012
1424499505,27cyclictest0-21swapper/320:47:233
1424499505,27cyclictest0-21swapper/320:47:233
1424499504,39cyclictest60702sleep321:36:563
1424499503,41cyclictest27921-21unixbench-2d23:27:123
1424499503,40cyclictest20440-21telnet19:51:593
14243995034,10cyclictest13887-21cut21:52:022
14243995034,10cyclictest13612-21unixbench_multi22:57:112
14243995019,12cyclictest0-21swapper/223:52:092
1424199506,27cyclictest0-21swapper/120:26:461
1424199505,27cyclictest0-21swapper/119:32:061
1424199504,39cyclictest0-21swapper/121:22:021
1424199504,28cyclictest0-21swapper/123:32:051
1424199504,28cyclictest0-21swapper/119:02:251
14241995030,11cyclictest0-21swapper/120:02:031
14241995027,8cyclictest18798-21runrttasks23:11:241
14241995024,11cyclictest0-21swapper/122:42:111
14241995024,11cyclictest0-21swapper/121:21:441
14241995024,11cyclictest0-21swapper/119:52:001
14241995023,12cyclictest0-21swapper/122:02:061
14241995023,12cyclictest0-21swapper/121:57:031
14241995020,12cyclictest0-21swapper/122:36:411
14241995020,12cyclictest0-21swapper/120:49:281
14241995020,12cyclictest0-21swapper/120:49:281
14238995031,8cyclictest1265-21runrttasks19:10:330
113472502,42sleep30-21swapper/322:52:383
5302495,38sleep00-21swapper/023:39:220
152402492,9sleep30-21swapper/321:56:083
1424499494,37cyclictest0-21swapper/322:40:233
1424499494,29cyclictest0-21swapper/323:17:013
14243994936,8cyclictest23286-21unixbench-2d18:47:222
14243994934,8cyclictest638-21grep22:32:002
14243994932,9cyclictest24212-21cpuspeed_turbos23:21:552
1424399493,27cyclictest0-21swapper/222:42:002
1424399492,42cyclictest0-21swapper/220:18:292
1424199496,11cyclictest19010-21munin-run18:41:491
1424199496,11cyclictest19010-21munin-run18:41:491
1424199495,27cyclictest0-21swapper/119:44:191
14241994933,11cyclictest0-21swapper/120:54:041
1424199493,32cyclictest0-21swapper/118:55:091
14241994925,8cyclictest2348-21munin-node22:36:591
14241994925,10cyclictest0-21swapper/122:16:571
14241994925,10cyclictest0-21swapper/122:16:571
14241994922,11cyclictest0-21swapper/123:47:121
14241994922,11cyclictest0-21swapper/123:17:041
14241994921,11cyclictest0-21swapper/119:09:131
14238994927,17cyclictest0-21swapper/021:13:560
14238994923,12cyclictest0-21swapper/021:02:060
14238994921,10cyclictest5739-21latency_hist21:36:430
14238994921,10cyclictest5739-21latency_hist21:36:430
109812493,40sleep20-21swapper/219:32:022
55812482,9sleep30-21swapper/319:19:053
43182482,39sleep20-21swapper/222:39:272
233672482,9sleep20-21swapper/221:02:392
223099480,5rtkit-daemon2229-21rtkit-daemon18:32:040
165522482,39sleep30-21swapper/318:36:093
1424499486,37cyclictest0-21swapper/320:27:163
1424499486,12cyclictest0-21swapper/323:55:233
1424499485,37cyclictest0-21swapper/319:02:083
1424499485,27cyclictest0-21swapper/320:52:003
1424499484,39cyclictest2128-21crond20:21:463
1424499484,38cyclictest0-21swapper/318:57:533
1424499484,29cyclictest0-21swapper/321:51:433
1424499484,28cyclictest0-21swapper/321:29:343
1424499484,27cyclictest0-21swapper/323:12:003
14244994832,11cyclictest23562-21cut22:12:003
1424399484,27cyclictest0-21swapper/219:42:142
14243994834,8cyclictest31048-21aten2.4-expect23:36:502
14243994833,9cyclictest2916-21grep21:27:162
14243994832,10cyclictest14929-21unixbench_multi19:37:202
1424399483,38cyclictest0-21swapper/222:56:392
14243994819,13cyclictest0-21swapper/220:31:592
14243994818,13cyclictest0-21swapper/223:47:132
1424199484,27cyclictest0-21swapper/123:53:161
14241994833,10cyclictest0-21swapper/118:57:131
14241994832,10cyclictest31581-21unixbench-2d20:12:201
14241994825,10cyclictest0-21swapper/123:56:581
14241994822,11cyclictest0-21swapper/121:12:041
14241994822,11cyclictest0-21swapper/118:31:501
1423899486,36cyclictest0-21swapper/021:22:170
1423899485,37cyclictest0-21swapper/022:12:020
1423899484,39cyclictest0-21swapper/022:27:050
14238994829,9cyclictest0-21swapper/023:11:330
14238994828,9cyclictest0-21swapper/019:41:050
14238994828,10cyclictest0-21swapper/020:09:570
14238994828,10cyclictest0-21swapper/020:09:570
14238994822,11cyclictest0-21swapper/023:26:380
284452472,39sleep00-21swapper/019:00:400
195442474,37sleep00-21swapper/020:57:000
1424499474,36cyclictest0-21swapper/318:46:503
1424499474,36cyclictest0-21swapper/318:46:503
14244994733,9cyclictest24345-21tr18:52:043
14244994732,9cyclictest9318-21gltestperf19:27:153
1424499473,25cyclictest0-21swapper/320:26:183
1424499472,40cyclictest0-21swapper/321:03:063
1424399473,38cyclictest25252-21telnet20:02:022
1424399473,37cyclictest0-21swapper/220:37:102
1424399473,27cyclictest0-21swapper/221:01:452
1424199474,27cyclictest0-21swapper/119:49:201
1424199473,38cyclictest0-21swapper/120:36:581
14241994722,9cyclictest4954-21open_inodes19:17:171
14241994721,9cyclictest23919-21munin-node21:06:541
14241994717,13cyclictest13239-21telnet19:36:581
1423899475,36cyclictest0-21swapper/021:47:160
1423899474,36cyclictest0-21swapper/021:38:480
1423899474,26cyclictest0-21swapper/019:52:180
14238994728,9cyclictest0-21swapper/023:44:140
14238994728,9cyclictest0-21swapper/019:43:440
14238994726,10cyclictest0-21swapper/023:58:040
31082463,37sleep00-21swapper/021:28:400
270602462,9sleep30-21swapper/320:03:433
269582462,39sleep00-21swapper/022:18:250
269582462,39sleep00-21swapper/022:18:250
233442462,9sleep20-21swapper/223:18:202
223099460,28rtkit-daemon0-21swapper/220:26:222
1424499465,37cyclictest361ktimersoftd/320:37:173
1424499465,36cyclictest0-21swapper/319:08:103
1424499465,34cyclictest0-21swapper/319:11:523
1424499464,36cyclictest0-21swapper/320:42:213
1424499464,35cyclictest0-21swapper/321:31:543
1424499464,35cyclictest0-21swapper/321:31:543
14244994618,11cyclictest0-21swapper/319:39:183
1424399465,12cyclictest0-21swapper/218:32:022
1424399464,26cyclictest0-21swapper/221:57:122
14243994632,9cyclictest14657-21cut23:01:552
14243994632,9cyclictest14657-21cut23:01:552
14243994631,9cyclictest19990-21hwlatdetect23:12:012
1424399463,37cyclictest0-21swapper/222:22:102
1424399463,37cyclictest0-21swapper/221:42:122
1424399463,37cyclictest0-21swapper/221:41:422
1424399463,36cyclictest0-21swapper/221:16:562
1424399463,28cyclictest0-21swapper/219:19:292
14243994618,11cyclictest0-21swapper/222:27:022
1423899469,31cyclictest0-21swapper/023:11:520
1423899467,10cyclictest0-21swapper/020:27:080
1423899464,36cyclictest0-21swapper/022:23:180
1423899464,36cyclictest0-21swapper/022:06:560
1423899463,36cyclictest0-21swapper/021:57:050
14238994629,12cyclictest0-21swapper/020:52:080
14238994627,9cyclictest0-21swapper/018:51:100
14238994622,19cyclictest0-21swapper/019:31:490
14238994621,12cyclictest0-21swapper/021:11:440
139202462,9sleep20-21swapper/220:44:012
66662452,9sleep20-21swapper/220:27:572
5462452,9sleep20-21swapper/219:08:002
213612453,9sleep20-21swapper/218:46:362
213612453,9sleep20-21swapper/218:46:362
1424499455,35cyclictest0-21swapper/321:17:083
1424499455,30cyclictest0-21swapper/321:14:123
14243994530,8cyclictest17382-21grep20:52:032
1424399453,29cyclictest0-21swapper/218:41:502
1424399453,29cyclictest0-21swapper/218:41:502
1423899455,35cyclictest0-21swapper/021:46:170
1423899454,36cyclictest0-21swapper/021:54:230
1423899454,35cyclictest0-21swapper/019:16:480
1423899454,14cyclictest0-21swapper/020:31:550
1423899454,11cyclictest0-21swapper/019:22:160
1423899454,11cyclictest0-21swapper/019:22:160
14238994531,10cyclictest0-21swapper/020:36:580
14238994521,11cyclictest0-21swapper/018:42:020
14238994521,11cyclictest0-21swapper/018:42:020
5142442,9sleep20-21swapper/221:23:182
284942442,9sleep20-21swapper/219:01:192
261022440,3sleep2281ktimersoftd/222:17:052
261022440,3sleep2281ktimersoftd/222:17:052
1424499442,27cyclictest0-21swapper/323:37:123
1424399443,25cyclictest0-21swapper/218:56:102
1423899447,32cyclictest0-21swapper/019:27:140
1423899446,13cyclictest0-21swapper/019:17:020
14238994430,10cyclictest0-21swapper/020:44:590
1423899443,36cyclictest0-21swapper/019:02:030
1423899443,13cyclictest0-21swapper/018:38:240
1423899443,13cyclictest0-21swapper/018:38:240
14238994420,10cyclictest0-21swapper/020:02:150
11102442,34sleep20-21swapper/219:11:552
1424499433,31cyclictest361ktimersoftd/322:31:513
1424499432,25cyclictest0-21swapper/321:57:093
14243994317,10cyclictest0-21swapper/221:47:122
1424499425,32cyclictest34-21rcuc/319:32:103
1423899425,11cyclictest0-21swapper/020:17:080
1423899424,13cyclictest0-21swapper/022:52:080
1424399412,23cyclictest0-21swapper/219:46:582
1423899416,31cyclictest0-21swapper/018:52:080
140462414,31sleep20-21swapper/218:29:442
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional