You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 06:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot6s.osadl.org (updated Wed Nov 19, 2025 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1855321020,4sleep2281ktimersoftd/219:47:192
2886799924,27cyclictest0-21swapper/222:48:322
2886799904,29cyclictest0-21swapper/219:52:342
2886799894,35cyclictest0-21swapper/200:22:252
2886799894,29cyclictest0-21swapper/219:07:422
2886799884,27cyclictest0-21swapper/219:33:382
2886799874,28cyclictest0-21swapper/223:12:132
2886799874,12cyclictest0-21swapper/219:03:122
28867998718,13cyclictest0-21swapper/220:12:392
28867998716,13cyclictest0-21swapper/221:57:332
2886799867,23cyclictest0-21swapper/223:26:562
2886799867,19cyclictest0-21swapper/220:57:032
2886799865,34cyclictest0-21swapper/222:17:322
2886799865,29cyclictest0-21swapper/223:34:032
2886799864,29cyclictest0-21swapper/222:22:282
2886799864,28cyclictest0-21swapper/221:07:292
2886799864,28cyclictest0-21swapper/219:37:302
2886799864,13cyclictest0-21swapper/219:12:412
28867998619,13cyclictest0-21swapper/200:20:482
28867998619,13cyclictest0-21swapper/200:20:482
28867998615,13cyclictest0-21swapper/219:58:372
28867998615,13cyclictest0-21swapper/219:58:372
2886799854,26cyclictest4010-21aten2.4-expect21:32:142
2886799845,27cyclictest0-21swapper/223:07:222
2886799844,28cyclictest0-21swapper/221:02:302
28867998419,11cyclictest0-21swapper/200:06:542
28867998320,11cyclictest0-21swapper/222:13:502
2886799824,35cyclictest18548-21cpuspeed_turbos22:02:152
2886799824,28cyclictest0-21swapper/220:42:262
2886799824,28cyclictest0-21swapper/220:42:262
2886799824,24cyclictest0-21swapper/220:07:192
2886799824,12cyclictest0-21swapper/220:32:402
28867998228,9cyclictest2520-21wc22:36:582
243799820,29rtkit-daemon3989-21ntp_kernel_pll_20:22:312
243799820,19rtkit-daemon0-21swapper/319:27:402
2886799814,28cyclictest0-21swapper/222:57:342
2886799814,28cyclictest0-21swapper/219:17:362
2886799814,24cyclictest0-21swapper/220:27:162
2886799814,14cyclictest0-21swapper/223:52:292
28867998116,13cyclictest0-21swapper/221:52:192
2886799805,33cyclictest0-21swapper/200:00:292
2886799804,28cyclictest0-21swapper/221:27:282
2886799794,23cyclictest0-21swapper/222:52:122
2886799794,23cyclictest0-21swapper/222:52:122
28867997917,14cyclictest0-21swapper/223:02:302
28868997848,24cyclictest477-21timerwakeupswit23:37:283
2886799784,23cyclictest0-21swapper/221:12:152
2886799784,22cyclictest0-21swapper/219:42:182
28867997828,10cyclictest22202-21sed23:17:142
2886799774,13cyclictest0-21swapper/220:48:342
28868997625,30cyclictest9571-21if_err_eth023:57:173
2886799764,33cyclictest0-21swapper/220:37:362
28868997423,44cyclictest23129-21seq22:12:133
28868997423,33cyclictest0-21swapper/322:11:593
28868997423,33cyclictest0-21swapper/322:11:593
2886799744,26cyclictest0-21swapper/222:38:542
28867997415,13cyclictest0-21swapper/220:02:302
28868997325,28cyclictest0-21swapper/320:15:363
2886799734,25cyclictest0-21swapper/220:57:172
28868997225,30cyclictest25424-21aten_r1power_en22:17:113
28868997222,43cyclictest0-21swapper/319:37:423
28868997221,34cyclictest0-21swapper/321:17:163
28868997126,29cyclictest6108-21crond19:22:053
2886799714,15cyclictest0-21swapper/219:22:312
28867997113,15cyclictest0-21swapper/221:17:372
28868997029,34cyclictest7309-21taskset19:22:303
28868997024,38cyclictest7228-21basename21:37:233
28868996925,37cyclictest6763-21sshd20:27:383
28868996922,27cyclictest26346-21cut20:02:273
28868996919,43cyclictest32621-21munin-node20:17:263
243799690,9rtkit-daemon361ktimersoftd/319:50:543
28868996824,28cyclictest0-21swapper/321:53:473
28868996823,37cyclictest1369-21runrttasks21:42:503
28868996823,36cyclictest0-21swapper/322:46:573
28868996822,39cyclictest13391-21munin-node00:07:283
28868996821,40cyclictest11276-21aten2.4-expect19:32:183
28868996821,29cyclictest1369-21runrttasks19:29:473
28868996738,22cyclictest0-21swapper/323:56:043
28868996720,41cyclictest0-21swapper/321:02:013
28868996718,30cyclictest0-21swapper/322:47:263
282972678,12sleep00-21swapper/018:57:380
28868996624,35cyclictest3633-21rm20:22:243
28868996622,28cyclictest24296-21tail21:07:153
28868996621,38cyclictest0-21swapper/320:42:233
28868996621,38cyclictest0-21swapper/320:42:233
28868996544,15cyclictest1243-21snmpd19:54:283
28868996523,27cyclictest26622-21munin-node21:12:233
28868996522,36cyclictest0-21swapper/323:09:063
28868996522,29cyclictest27562-21iostat_ios23:27:213
28868996521,29cyclictest0-21swapper/320:53:353
28868996520,37cyclictest5374-21timerandwakeup23:47:303
28868996520,26cyclictest1179-21lldpd20:40:193
28868996430,27cyclictest11889-21gltestperf00:02:173
28868996421,25cyclictest20385-21munin-run22:06:593
28868996419,38cyclictest30740-21unixbench-2d19:02:383
28868996417,16cyclictest0-21swapper/321:06:463
28868996413,43cyclictest0-21swapper/300:17:153
28868996413,43cyclictest0-21swapper/300:17:153
28868996325,31cyclictest0-21swapper/319:10:543
28868996323,33cyclictest0-21swapper/323:26:573
28868996322,34cyclictest0-21swapper/322:22:133
28868996321,34cyclictest1369-21runrttasks00:26:213
28868996321,24cyclictest3227-21df_inode22:37:173
28868996317,30cyclictest0-21swapper/300:16:483
28868996228,28cyclictest28426-21diskstats20:07:253
28868996220,35cyclictest0-21swapper/322:27:333
28868996218,26cyclictest30744-21yum23:32:343
28868996133,21cyclictest0-21swapper/322:56:133
28868996133,21cyclictest0-21swapper/322:56:133
28868996127,27cyclictest0-21swapper/321:36:303
28868996124,30cyclictest14660-21munin-node20:47:323
28868996123,32cyclictest23549-21users23:17:333
28868996123,30cyclictest1369-21runrttasks22:01:163
28868996122,32cyclictest1369-21runrttasks23:14:193
28868996031,23cyclictest15507-21hddtemp_smartct23:02:263
28868996028,25cyclictest3190-21unixbench_multi21:27:353
28868996028,25cyclictest2344-21tail19:12:273
28868996026,26cyclictest23822-21sort19:57:243
28868996026,26cyclictest23822-21sort19:57:243
28868995941,12cyclictest23614-21cut00:27:143
28868995921,31cyclictest1369-21runrttasks22:33:163
28868995842,11cyclictest8316-21gltestperf20:32:273
28868995834,18cyclictest1185-21sshd22:57:313
28868995829,23cyclictest1369-21runrttasks21:23:403
28868995829,23cyclictest1369-21runrttasks21:23:403
28868995819,33cyclictest0-21swapper/319:44:373
2886299584,26cyclictest0-21swapper/022:57:100
28868995741,10cyclictest11923-21gltestperf21:47:203
28868995724,28cyclictest2701-21ntp_states23:42:223
192172574,46sleep00-21swapper/000:17:200
192172574,46sleep00-21swapper/000:17:200
112302573,11sleep00-21swapper/019:32:190
2886699563,37cyclictest0-21swapper/123:36:551
2886799554,43cyclictest0-21swapper/222:42:302
320662542,9sleep00-21swapper/022:28:470
28868995414,24cyclictest28973-21cat19:02:063
2886299542,46cyclictest0-21swapper/021:42:320
286792542,45sleep00-21swapper/022:22:220
227912543,10sleep00-21swapper/023:17:230
2886299533,44cyclictest0-21swapper/019:02:340
28862995322,13cyclictest0-21swapper/021:02:010
215222532,9sleep00-21swapper/023:16:550
164802533,42sleep00-21swapper/020:47:510
158132533,43sleep015842-21netstat23:02:320
28867995220,14cyclictest0-21swapper/200:12:232
2886699520,9cyclictest13474-21kworker/3:021:52:141
28862995236,10cyclictest9580-21grep19:27:260
287682523,7sleep028780-21munin-run21:17:010
249452522,43sleep10-21swapper/100:30:061
243799520,5rtkit-daemon2436-21rtkit-daemon23:15:161
148902522,9sleep10-21swapper/123:02:211
82482513,9sleep10-21swapper/123:55:201
2886799515,28cyclictest0-21swapper/221:42:232
2886799515,28cyclictest0-21swapper/200:07:182
2886799514,29cyclictest0-21swapper/223:42:232
2886299516,39cyclictest0-21swapper/020:32:270
2886299516,27cyclictest0-21swapper/023:59:290
231622514,40sleep10-21swapper/121:02:311
227762512,41sleep00-21swapper/019:56:250
218352513,42sleep00-21swapper/021:02:060
45832503,9sleep10-21swapper/120:23:221
2886799506,21cyclictest0-21swapper/222:07:132
2886799506,21cyclictest0-21swapper/222:07:132
2886799504,28cyclictest0-21swapper/221:47:212
2886699503,41cyclictest1243-21snmpd23:24:021
2886299505,27cyclictest0-21swapper/019:38:180
28862995034,10cyclictest26595-21strings22:17:240
250662502,41sleep00-21swapper/022:15:530
180322503,41sleep00-21swapper/019:47:010
108322503,39sleep111017-21latency_hist21:47:001
86332493,38sleep10-21swapper/120:32:321
83622493,9sleep00-21swapper/021:41:150
7282492,41sleep10-21swapper/123:38:241
321862492,42sleep00-21swapper/020:14:300
28867994921,12cyclictest0-21swapper/222:27:072
28867994919,13cyclictest0-21swapper/223:47:202
28866994935,9cyclictest734-21cut22:32:181
2886699493,37cyclictest0-21swapper/120:57:061
28866994916,26cyclictest18349-21cut22:02:131
28862994936,9cyclictest28511-21fschecks_count20:07:260
28862994934,10cyclictest22373-21unixbench_singl22:07:340
28862994934,10cyclictest22373-21unixbench_singl22:07:340
2886299493,40cyclictest0-21swapper/021:52:150
275952492,41sleep00-21swapper/020:06:560
261132492,9sleep00-21swapper/021:08:340
124942492,11sleep00-21swapper/021:47:320
83372482,39sleep10-21swapper/119:25:021
83142482,40sleep00-21swapper/023:56:080
70652482,39sleep10-21swapper/122:45:241
59522483,9sleep00-21swapper/019:20:050
46762482,39sleep00-21swapper/022:40:230
32462482,40sleep10-21swapper/123:42:471
2886799484,25cyclictest684-21unixbench_singl21:22:372
2886799484,25cyclictest684-21unixbench_singl21:22:372
2886699484,37cyclictest0-21swapper/120:52:231
28866994831,10cyclictest12003-21head19:32:251
2886699482,39cyclictest0-21swapper/119:37:321
28866994812,32cyclictest0-21swapper/119:08:381
2886299484,39cyclictest0-21swapper/023:46:550
2886299484,38cyclictest0-21swapper/019:47:370
28862994834,9cyclictest19997-21grep22:02:340
28862994834,8cyclictest32325-21wc21:22:240
28862994834,8cyclictest32325-21wc21:22:240
2886299483,39cyclictest0-21swapper/023:27:370
28862994810,30cyclictest0-21swapper/000:26:030
250452483,38sleep00-21swapper/019:59:470
250452483,38sleep00-21swapper/019:59:470
178832482,38sleep10-21swapper/123:07:201
152642483,9sleep00-21swapper/000:08:250
122702482,9sleep10-21swapper/120:42:121
122702482,9sleep10-21swapper/120:42:121
116732482,39sleep00-21swapper/020:37:390
9122473,38sleep00-21swapper/019:08:530
88702472,38sleep10-21swapper/122:47:301
84032473,38sleep00-21swapper/019:25:540
74142473,9sleep00-21swapper/022:47:000
308572472,38sleep00-21swapper/021:18:060
2886799474,26cyclictest0-21swapper/220:18:262
2886799474,25cyclictest460-21jbd2/sda2-823:27:212
2886799474,25cyclictest0-21swapper/200:27:092
28867994717,13cyclictest0-21swapper/221:37:232
2886699473,40cyclictest0-21swapper/120:07:201
2886299476,35cyclictest0-21swapper/020:22:200
28862994733,9cyclictest30612-21grep23:32:310
28862994733,9cyclictest17895-21grep20:52:220
2886299473,38cyclictest0-21swapper/023:37:070
2886299473,30cyclictest0-21swapper/020:42:070
2886299473,30cyclictest0-21swapper/020:42:070
285372472,38sleep10-21swapper/123:30:011
248342473,9sleep10-21swapper/122:12:521
219252473,38sleep10-21swapper/123:17:121
19812473,38sleep00-21swapper/022:32:500
156942472,38sleep10-21swapper/120:47:281
128542472,40sleep00-21swapper/000:03:050
117772472,9sleep10-21swapper/120:38:451
82022462,10sleep10-21swapper/121:39:101
72462462,9sleep00-21swapper/020:31:550
57832462,38sleep10-21swapper/123:49:361
50392463,9sleep00-21swapper/022:42:090
35662463,9sleep10-21swapper/119:15:071
34012462,38sleep00-21swapper/021:28:500
308882462,39sleep10-21swapper/121:18:311
2886699464,37cyclictest0-21swapper/121:27:361
2886699463,36cyclictest0-21swapper/120:27:071
28866994620,8cyclictest18049-21cat00:16:541
2886299464,37cyclictest0-21swapper/023:47:290
2886299464,26cyclictest0-21swapper/021:57:290
263372462,37sleep10-21swapper/121:11:301
248182462,37sleep00-21swapper/000:28:280
247322463,36sleep024791-21tune2fs23:22:150
143212463,9sleep10-21swapper/123:01:201
309602452,38sleep10-21swapper/119:03:381
2886699454,37cyclictest0-21swapper/100:17:131
2886699454,37cyclictest0-21swapper/100:17:131
2886699454,36cyclictest11207-21tail21:47:141
2886699454,26cyclictest0-21swapper/122:38:541
28866994531,9cyclictest28992-21needreboot22:22:251
2886699453,36cyclictest0-21swapper/119:42:191
28862994526,14cyclictest0-21swapper/019:12:350
24232452,9sleep00-21swapper/020:21:430
225192452,35sleep10-21swapper/119:53:071
149202452,35sleep10-21swapper/100:07:291
323762442,9sleep10-21swapper/121:22:261
323762442,9sleep10-21swapper/121:22:261
2886799444,23cyclictest0-21swapper/223:37:092
28866994429,9cyclictest25730-21cut22:17:131
28866994428,10cyclictest30579-21tr20:12:191
28866994416,9cyclictest9191-21diskstats23:57:121
28866994410,28cyclictest0-21swapper/100:02:091
28862994416,9cyclictest1243-21snmpd21:33:440
285332444,34sleep10-21swapper/118:58:151
189072442,9sleep00-21swapper/023:08:570
322902432,35sleep10-21swapper/122:31:421
2886699433,35cyclictest20776-21telnet22:07:131
2886699433,35cyclictest20776-21telnet22:07:131
2886699433,11cyclictest0-21swapper/122:52:351
2886699433,11cyclictest0-21swapper/122:52:351
28866994329,9cyclictest9069-21cpuspeed_turbos19:27:211
28866994329,9cyclictest4644-21grep19:17:251
2886699432,36cyclictest0-21swapper/121:12:161
2886299433,33cyclictest0-21swapper/022:52:120
2886299433,33cyclictest0-21swapper/022:52:120
2886299433,33cyclictest0-21swapper/000:12:280
250242432,35sleep10-21swapper/119:59:321
250242432,35sleep10-21swapper/119:59:321
223922432,35sleep10-21swapper/100:23:011
58342422,9sleep10-21swapper/121:34:231
2886699423,35cyclictest0-21swapper/120:17:181
2886699423,32cyclictest0-21swapper/119:52:051
2886699423,11cyclictest0-21swapper/121:57:191
2886699414,31cyclictest26052-21forks20:02:251
2886799384,26cyclictest28852-21munin-run19:02:062
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional