You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-09 - 03:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Tue Dec 09, 2025 00:44:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
275991940,84ptp4l0-21swapper/119:06:411
304728678,6sleep70-21swapper/719:06:419
304027059,9sleep00-21swapper/019:06:340
301727059,9sleep50-21swapper/519:06:157
3231326654,10sleep110-21swapper/1119:05:083
3179126554,8sleep40-21swapper/419:05:016
309726552,10sleep60-21swapper/619:07:168
303726352,9sleep100-21swapper/1019:06:322
60702620,0sleep60-21swapper/622:40:148
60702620,0sleep60-21swapper/622:40:148
229492610,1sleep1201rcuc/120:55:221
265782550,0sleep70-21swapper/722:25:149
138632550,0sleep60-21swapper/620:45:148
378399222,7cyclictest0-21swapper/420:20:006
378399221,16cyclictest0-21swapper/423:50:006
378399212,15cyclictest0-21swapper/400:30:006
378399212,14cyclictest0-21swapper/420:45:006
378399191,17cyclictest0-21swapper/421:50:176
378699185,8cyclictest0-21swapper/522:30:007
378699180,3cyclictest0-21swapper/500:10:187
379299173,8cyclictest0-21swapper/623:20:008
378699173,8cyclictest0-21swapper/522:05:007
275991170,3ptp4l21-21ksoftirqd/100:30:231
172642170,0sleep50-21swapper/500:20:027
381099160,1cyclictest0-21swapper/1023:50:242
379299162,8cyclictest0-21swapper/621:05:008
378699162,8cyclictest0-21swapper/523:00:007
378699162,8cyclictest0-21swapper/523:00:007
376899160,1cyclictest0-21swapper/123:50:241
323102167,7sleep80-21swapper/819:05:0510
275991160,3ptp4l6040-21sshd21:57:161
275991160,0ptp4l0-21swapper/120:30:001
44972150,0sleep110-21swapper/1119:50:133
381499153,7cyclictest0-21swapper/1120:05:003
381099150,15cyclictest0-21swapper/1022:10:242
379299153,7cyclictest0-21swapper/621:17:098
379299153,7cyclictest0-21swapper/620:25:268
378699153,7cyclictest0-21swapper/519:15:007
378699153,7cyclictest0-21swapper/500:25:017
275991150,0ptp4l0-21swapper/123:30:001
275991150,0ptp4l0-21swapper/122:10:221
275991150,0ptp4l0-21swapper/120:40:151
275991150,0ptp4l0-21swapper/120:35:141
275991150,0ptp4l0-21swapper/119:25:141
275991150,0ptp4l0-21swapper/119:10:271
380599144,6cyclictest0-21swapper/919:25:0011
380599142,8cyclictest0-21swapper/923:10:0011
380599142,7cyclictest0-21swapper/923:35:0011
379299142,7cyclictest0-21swapper/622:20:018
378399142,7cyclictest0-21swapper/422:45:006
378399142,7cyclictest0-21swapper/422:45:006
375499142,7cyclictest0-21swapper/000:20:240
375499141,8cyclictest0-21swapper/021:50:000
375499141,8cyclictest0-21swapper/020:05:000
301952140,0sleep70-21swapper/721:05:179
275991140,1ptp4l21-21ksoftirqd/100:09:281
275991140,0ptp4l0-21swapper/123:10:001
275991140,0ptp4l0-21swapper/100:25:231
379299132,6cyclictest0-21swapper/623:45:018
379299132,6cyclictest0-21swapper/623:00:008
379299132,6cyclictest0-21swapper/623:00:008
379299131,8cyclictest20279-21kworker/u24:000:05:018
379299130,8cyclictest0-21swapper/621:00:008
378699133,7cyclictest0-21swapper/521:10:007
378699133,6cyclictest0-21swapper/523:05:017
3777991311,1cyclictest0-21swapper/322:30:005
375499132,6cyclictest0-21swapper/020:15:000
375499130,8cyclictest0-21swapper/000:35:010
275991130,1ptp4l27460-21getsmi21:00:271
275991130,1ptp4l21-21ksoftirqd/120:15:221
275991130,0ptp4l0-21swapper/123:35:181
275991130,0ptp4l0-21swapper/123:00:131
275991130,0ptp4l0-21swapper/123:00:131
275991130,0ptp4l0-21swapper/122:50:211
275991130,0ptp4l0-21swapper/122:50:211
275991130,0ptp4l0-21swapper/122:15:161
129332130,0chrt0-21swapper/522:05:227
381499122,6cyclictest0-21swapper/1123:00:003
381499122,6cyclictest0-21swapper/1123:00:003
381499121,6cyclictest0-21swapper/1120:15:003
380599123,6cyclictest0-21swapper/922:10:0011
380599120,8cyclictest0-21swapper/923:50:0111
379299122,6cyclictest0-21swapper/619:35:008
379299121,6cyclictest0-21swapper/621:50:008
378699122,6cyclictest0-21swapper/520:00:257
378399122,6cyclictest0-21swapper/422:25:006
378399120,7cyclictest0-21swapper/422:15:006
377799123,8cyclictest0-21swapper/300:15:015
377799123,8cyclictest0-21swapper/300:05:005
377799122,8cyclictest0-21swapper/321:45:005
375499122,6cyclictest0-21swapper/020:00:000
275991120,3ptp4l26903-21cpuspeed20:20:121
275991120,2ptp4l29427-21hddtemp_smartct19:40:171
275991120,1ptp4l21-21ksoftirqd/122:48:121
275991120,1ptp4l21-21ksoftirqd/122:48:121
275991120,0ptp4l0-21swapper/123:40:141
275991120,0ptp4l0-21swapper/121:40:131
275991120,0ptp4l0-21swapper/121:20:221
275991120,0ptp4l0-21swapper/120:50:581
275991120,0ptp4l0-21swapper/120:05:151
275991120,0ptp4l0-21swapper/119:55:121
275991120,0ptp4l0-21swapper/119:35:151
381499118,2cyclictest10836-21turbostat23:30:003
381099110,6cyclictest0-21swapper/1022:25:002
380599112,6cyclictest0-21swapper/922:55:0011
380599112,6cyclictest0-21swapper/922:55:0011
380199110,10cyclictest27455-21df20:20:1310
379299110,10cyclictest27669-21awk23:10:008
378699117,3cyclictest1715-21grep23:15:217
378399111,6cyclictest0-21swapper/419:15:266
377799113,7cyclictest0-21swapper/323:20:015
377799112,8cyclictest0-21swapper/323:00:015
377799112,8cyclictest0-21swapper/323:00:015
377799112,8cyclictest0-21swapper/322:55:005
377799112,8cyclictest0-21swapper/322:55:005
377799112,6cyclictest0-21swapper/322:15:275
377399110,11cyclictest0-21swapper/221:17:554
376899110,10cyclictest13936-21cpuspeed_turbos20:45:141
375499112,6cyclictest0-21swapper/023:35:010
375499111,6cyclictest0-21swapper/021:17:090
275991110,4ptp4l21-21ksoftirqd/121:30:201
275991110,3ptp4l30473-21/usr/sbin/munin23:55:241
275991110,3ptp4l15102-21ps19:20:211
275991110,2ptp4l8200-21df_inode19:15:021
275991110,2ptp4l7372-21latency_hist19:55:001
275991110,2ptp4l4518-21df_inode23:20:141
275991110,2ptp4l4377-21ps20:30:201
275991110,2ptp4l346-21df_inode23:15:151
275991110,2ptp4l31916-21users21:05:261
275991110,1ptp4l21-21ksoftirqd/100:20:221
275991110,0ptp4l0-21swapper/123:32:521
275991110,0ptp4l0-21swapper/123:10:501
275991110,0ptp4l0-21swapper/122:05:121
94980100,5rtkit-daemon0-21swapper/319:05:315
381499100,9cyclictest4677-21/usr/sbin/munin20:30:233
381499100,9cyclictest0-21swapper/1122:45:003
381499100,9cyclictest0-21swapper/1122:45:003
381099102,5cyclictest0-21swapper/1020:45:002
380599102,5cyclictest0-21swapper/900:30:0111
380599101,6cyclictest0-21swapper/922:35:0011
380599101,6cyclictest0-21swapper/922:35:0011
378699100,9cyclictest0-21swapper/500:00:197
378399100,9cyclictest0-21swapper/419:40:136
377799107,2cyclictest22423-21perl20:55:195
377799103,6cyclictest0-21swapper/300:25:005
377799102,6cyclictest0-21swapper/322:40:005
377799102,6cyclictest0-21swapper/322:40:005
377799102,6cyclictest0-21swapper/320:25:005
377799102,4cyclictest0-21swapper/320:20:005
377799100,9cyclictest0-21swapper/322:00:165
377799100,9cyclictest0-21swapper/321:45:145
377799100,9cyclictest0-21swapper/300:35:245
377399100,9cyclictest0-21swapper/222:05:244
377399100,9cyclictest0-21swapper/219:55:194
275991100,2ptp4l24429-21irqrtprio23:45:181
275991100,2ptp4l20099-21cat23:00:001
275991100,2ptp4l20099-21cat23:00:001
275991100,1ptp4l6914-21cpu22:00:121
275991100,1ptp4l5283-21diskstats21:15:151
275991100,1ptp4l315-21/usr/sbin/munin21:10:151
275991100,1ptp4l30086-21perl00:35:151
275991100,1ptp4l25807-21latency_hist22:25:001
275991100,1ptp4l20260-21hddtemp_smartct20:10:171
275991100,1ptp4l11886-21cstates20:00:151
275991100,1ptp4l10519-21cstates00:10:141
275991100,0ptp4l0-21swapper/121:50:001
275991100,0ptp4l0-21swapper/119:45:141
275991100,0ptp4l0-21swapper/119:30:221
38149998,1cyclictest0-21swapper/1120:35:153
38149990,8cyclictest12809-21who00:10:263
38109998,1cyclictest0-21swapper/1021:25:232
38109992,5cyclictest0-21swapper/1020:30:002
38109990,4cyclictest0-21swapper/1000:25:012
38109990,2cyclictest20917-21memory23:40:192
38059990,8cyclictest0-21swapper/919:15:2611
38019990,8cyclictest13996-21wc20:00:2510
38019990,2cyclictest29620-21cat22:30:0010
38019990,1cyclictest0-21swapper/820:50:1610
38019990,1cyclictest0-21swapper/820:40:2410
38019990,1cyclictest0-21swapper/800:32:0910
37969998,1cyclictest0-21swapper/723:20:199
37969990,9cyclictest0-21swapper/723:55:199
37969990,9cyclictest0-21swapper/723:45:189
37969990,9cyclictest0-21swapper/721:20:149
37969990,1cyclictest0-21swapper/723:50:149
37929990,1cyclictest0-21swapper/621:25:208
37839996,2cyclictest23847-21turbostat23:05:006
37839996,2cyclictest23847-21turbostat23:05:006
37839990,8cyclictest0-21swapper/421:00:166
37839990,8cyclictest0-21swapper/400:09:326
37779990,1cyclictest0-21swapper/321:15:185
37739998,1cyclictest0-21swapper/221:25:144
37739990,9cyclictest0-21swapper/221:15:004
37739990,8cyclictest30485-21/usr/sbin/munin20:25:284
37739990,8cyclictest18682sleep221:50:294
37739990,8cyclictest0-21swapper/223:45:154
37739990,8cyclictest0-21swapper/222:15:184
37739990,8cyclictest0-21swapper/221:40:194
37739990,1cyclictest0-21swapper/223:00:144
37739990,1cyclictest0-21swapper/223:00:144
37689990,1cyclictest0-21swapper/122:30:011
37689990,1cyclictest0-21swapper/122:30:011
37549992,4cyclictest0-21swapper/019:55:000
37549991,5cyclictest0-21swapper/022:35:000
37549991,5cyclictest0-21swapper/022:35:000
37549990,2cyclictest14521-21switchtime21:25:240
27599190,1ptp4l31448-21cpuspeed21:50:121
27599190,0ptp4l0-21swapper/122:40:161
27599190,0ptp4l0-21swapper/122:40:161
27599190,0ptp4l0-21swapper/122:35:151
27599190,0ptp4l0-21swapper/122:35:151
27599190,0ptp4l0-21swapper/122:25:131
27599190,0ptp4l0-21swapper/121:35:151
27599190,0ptp4l0-21swapper/121:25:141
27599190,0ptp4l0-21swapper/100:15:171
27599190,0ptp4l0-21swapper/100:00:191
16380290,0sleep20-21swapper/220:05:174
38149987,1cyclictest0-21swapper/1122:17:213
38149981,4cyclictest0-21swapper/1123:40:003
38149981,4cyclictest0-21swapper/1123:35:003
38149980,1cyclictest0-21swapper/1123:10:163
38149980,1cyclictest0-21swapper/1121:20:243
38109985,2cyclictest18493-21turbostat20:10:002
38109980,8cyclictest0-21swapper/1020:30:132
38109980,8cyclictest0-21swapper/1019:55:222
38109980,1cyclictest0-21swapper/1023:10:182
38109980,1cyclictest0-21swapper/1022:05:182
38109980,1cyclictest0-21swapper/1021:35:172
38109980,1cyclictest0-21swapper/1000:10:152
38109980,1cyclictest0-21swapper/1000:05:222
38059987,1cyclictest0-21swapper/922:45:0011
38059987,1cyclictest0-21swapper/922:45:0011
38059987,1cyclictest0-21swapper/920:35:1411
38059981,4cyclictest0-21swapper/920:35:0011
38059980,8cyclictest0-21swapper/921:40:1911
38059980,1cyclictest6052-21sshd21:57:2011
38059980,1cyclictest0-21swapper/922:45:1511
38059980,1cyclictest0-21swapper/922:45:1511
38059980,1cyclictest0-21swapper/900:30:1711
38059980,1cyclictest0-21swapper/900:09:1511
38019987,1cyclictest0-21swapper/821:20:1810
38019987,1cyclictest0-21swapper/821:00:2810
38019980,8cyclictest0-21swapper/823:20:1910
38019980,8cyclictest0-21swapper/822:45:1610
38019980,8cyclictest0-21swapper/822:45:1610
38019980,1cyclictest0-21swapper/823:05:2410
37969987,1cyclictest0-21swapper/723:30:159
37969987,1cyclictest0-21swapper/723:00:249
37969987,1cyclictest0-21swapper/723:00:249
37969987,1cyclictest0-21swapper/700:00:169
37969986,1cyclictest0-21swapper/719:40:459
37969980,8cyclictest0-21swapper/720:10:219
37969980,1cyclictest0-21swapper/719:51:559
37969980,1cyclictest0-21swapper/719:25:179
37929987,1cyclictest142492sleep619:20:188
37929987,1cyclictest0-21swapper/619:25:138
37929983,4cyclictest0-21swapper/621:05:268
37929982,4cyclictest0-21swapper/623:00:138
37929982,4cyclictest0-21swapper/623:00:138
37929981,6cyclictest32259-21diskmemload22:30:438
37929981,6cyclictest32259-21diskmemload22:30:438
37929980,7cyclictest205812sleep620:52:168
37929980,1cyclictest0-21swapper/623:33:268
37929980,1cyclictest0-21swapper/622:52:288
37929980,1cyclictest0-21swapper/622:52:288
37929980,1cyclictest0-21swapper/621:12:128
37869987,1cyclictest0-21swapper/500:20:257
37869981,6cyclictest20279-21kworker/u24:019:50:007
37869980,1cyclictest0-21swapper/519:50:137
37869980,1cyclictest0-21swapper/519:15:217
37839980,7cyclictest5501-21latency_hist20:35:006
37839980,7cyclictest3173-21cat23:20:006
37839980,7cyclictest27769-21latency_hist19:40:006
37839980,7cyclictest0-21swapper/422:35:166
37839980,7cyclictest0-21swapper/422:35:166
37839980,3cyclictest29953-21crond20:25:006
37839980,2cyclictest31388-21cat21:05:256
37839980,2cyclictest29603-21users23:50:276
37839980,1cyclictest0-21swapper/420:05:236
37779987,1cyclictest0-21swapper/323:05:135
37779987,1cyclictest0-21swapper/320:05:005
37779985,2cyclictest0-21swapper/320:15:005
37779983,4cyclictest0-21swapper/300:30:015
37779981,6cyclictest0-21swapper/319:20:245
37779981,4cyclictest0-21swapper/320:50:005
37779980,8cyclictest0-21swapper/320:25:265
37779980,8cyclictest0-21swapper/320:05:225
37779980,8cyclictest0-21swapper/319:50:215
37779980,1cyclictest0-21swapper/323:30:205
37779980,1cyclictest0-21swapper/323:10:125
37779980,1cyclictest0-21swapper/321:12:085
37779980,1cyclictest0-21swapper/300:05:155
37739987,1cyclictest0-21swapper/223:50:264
37739980,8cyclictest0-21swapper/223:10:004
37739980,8cyclictest0-21swapper/222:50:214
37739980,8cyclictest0-21swapper/222:50:214
37739980,8cyclictest0-21swapper/221:30:164
37739980,8cyclictest0-21swapper/221:22:124
37739980,8cyclictest0-21swapper/220:45:164
37739980,8cyclictest0-21swapper/220:20:154
37739980,8cyclictest0-21swapper/219:40:134
37739980,7cyclictest32259-21diskmemload00:10:324
37739980,7cyclictest0-21swapper/223:25:164
37739980,7cyclictest0-21swapper/222:20:284
37739980,7cyclictest0-21swapper/221:58:224
37739980,7cyclictest0-21swapper/221:37:564
37739980,7cyclictest0-21swapper/220:00:204
37739980,7cyclictest0-21swapper/219:35:134
37739980,7cyclictest0-21swapper/200:25:184
37739980,2cyclictest0-21swapper/222:49:274
37739980,2cyclictest0-21swapper/222:49:274
37739980,2cyclictest0-21swapper/221:45:174
37739980,2cyclictest0-21swapper/221:00:254
37739980,2cyclictest0-21swapper/220:50:154
37739980,2cyclictest0-21swapper/220:15:154
37739980,2cyclictest0-21swapper/200:05:154
37739980,1cyclictest0-21swapper/222:00:144
37739980,1cyclictest0-21swapper/220:55:244
37739980,1cyclictest0-21swapper/220:30:014
37739980,1cyclictest0-21swapper/220:10:214
37739980,1cyclictest0-21swapper/219:25:164
37739980,1cyclictest0-21swapper/219:20:134
37549987,1cyclictest0-21swapper/021:30:250
37549987,1cyclictest0-21swapper/019:35:240
37549985,2cyclictest10219-21timerandwakeup123:25:250
37549980,8cyclictest0-21swapper/022:15:240
3008281,5sleep90-21swapper/919:06:0711
38149976,1cyclictest0-21swapper/1123:45:283
38149976,1cyclictest0-21swapper/1122:50:153
38149976,1cyclictest0-21swapper/1122:50:153
38149976,1cyclictest0-21swapper/1122:11:123
38149976,1cyclictest0-21swapper/1121:46:133
38149976,1cyclictest0-21swapper/1121:10:193
38149976,1cyclictest0-21swapper/1121:00:533
38149976,1cyclictest0-21swapper/1120:47:433
38149976,1cyclictest0-21swapper/1119:35:213
38149976,1cyclictest0-21swapper/1100:25:313
38149976,1cyclictest0-21swapper/1100:15:203
38149976,1cyclictest0-21swapper/1100:00:233
38149975,1cyclictest21329-21df_inode23:00:133
38149975,1cyclictest21329-21df_inode23:00:133
38149974,2cyclictest1020-21runrttasks21:30:023
38149973,3cyclictest24873-21cut23:05:143
38149970,7cyclictest12673-21strings22:05:193
38149970,7cyclictest0-21swapper/1123:55:343
38149970,7cyclictest0-21swapper/1123:50:163
38149970,7cyclictest0-21swapper/1123:20:273
38149970,7cyclictest0-21swapper/1122:37:273
38149970,7cyclictest0-21swapper/1122:37:273
38149970,7cyclictest0-21swapper/1122:25:203
38149970,7cyclictest0-21swapper/1122:20:213
38149970,7cyclictest0-21swapper/1122:02:033
38149970,7cyclictest0-21swapper/1121:55:113
38149970,7cyclictest0-21swapper/1121:40:123
38149970,7cyclictest0-21swapper/1121:07:043
38149970,7cyclictest0-21swapper/1120:55:183
38149970,7cyclictest0-21swapper/1120:50:203
38149970,7cyclictest0-21swapper/1120:43:293
38149970,7cyclictest0-21swapper/1120:21:463
38149970,7cyclictest0-21swapper/1120:05:163
38149970,7cyclictest0-21swapper/1119:55:173
38149970,7cyclictest0-21swapper/1119:30:283
38149970,7cyclictest0-21swapper/1119:20:263
38149970,7cyclictest0-21swapper/1100:35:213
38149970,7cyclictest0-21swapper/1100:32:023
38149970,7cyclictest0-21swapper/1100:06:093
38149970,1cyclictest0-21swapper/1123:42:073
38149970,1cyclictest0-21swapper/1123:15:373
38149970,1cyclictest0-21swapper/1122:45:593
38149970,1cyclictest0-21swapper/1122:45:593
38149970,1cyclictest0-21swapper/1122:30:503
38149970,1cyclictest0-21swapper/1122:30:503
38149970,1cyclictest0-21swapper/1121:51:573
38149970,1cyclictest0-21swapper/1121:36:283
38149970,1cyclictest0-21swapper/1121:26:023
38149970,1cyclictest0-21swapper/1121:16:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional