You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-08 - 15:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Mon Jun 08, 2026 12:44:42)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2834911250,113ptp4l0-21swapper/107:07:151
94080810,76rtkit-daemon0-21swapper/507:06:117
1088427361,9sleep60-21swapper/607:05:198
895927055,12sleep110-21swapper/1107:05:133
1193726655,9sleep70-21swapper/707:07:459
1184926655,9sleep00-21swapper/007:06:260
1021926655,8sleep40-21swapper/407:05:176
1202026554,9sleep100-21swapper/1007:09:002
51672510,0sleep110-21swapper/1111:10:133
1245899194,8cyclictest0-21swapper/107:35:011
1246399183,9cyclictest0-21swapper/210:10:004
283491170,1ptp4l0-21swapper/108:50:221
283491170,0ptp4l0-21swapper/110:05:201
283491170,0ptp4l0-21swapper/109:55:191
1248399173,9cyclictest0-21swapper/610:49:598
1248399170,12cyclictest1670-21kworker/u24:211:50:018
1247299170,6cyclictest0-21swapper/408:05:016
1246399173,9cyclictest0-21swapper/211:55:004
1246399173,10cyclictest0-21swapper/210:59:594
1245899173,8cyclictest0-21swapper/110:30:001
1245499172,10cyclictest0-21swapper/007:15:010
283491160,0ptp4l0-21swapper/111:35:241
283491160,0ptp4l0-21swapper/108:35:241
1249299163,8cyclictest0-21swapper/811:09:5910
1249299162,8cyclictest0-21swapper/808:35:0110
1249299162,8cyclictest0-21swapper/807:45:0110
1248899163,7cyclictest0-21swapper/709:40:009
1247999162,9cyclictest0-21swapper/508:15:007
1247999162,2cyclictest0-21swapper/510:55:177
1247299160,13cyclictest0-21swapper/412:00:006
1246399164,8cyclictest0-21swapper/208:00:014
1246399164,7cyclictest0-21swapper/210:00:004
1246399163,9cyclictest0-21swapper/208:45:224
1246399163,8cyclictest0-21swapper/210:50:004
1246399162,9cyclictest0-21swapper/211:00:014
1245899163,7cyclictest0-21swapper/110:40:251
1245499162,8cyclictest0-21swapper/009:15:010
1245499162,8cyclictest0-21swapper/008:50:250
119122167,7sleep80-21swapper/807:07:2210
283491150,0ptp4l0-21swapper/112:05:181
283491150,0ptp4l0-21swapper/111:30:121
283491150,0ptp4l0-21swapper/111:20:141
283491150,0ptp4l0-21swapper/110:00:231
283491150,0ptp4l0-21swapper/109:30:081
283491150,0ptp4l0-21swapper/108:55:121
283491150,0ptp4l0-21swapper/108:25:151
1249299152,7cyclictest0-21swapper/809:40:0010
1249299152,7cyclictest0-21swapper/809:10:0110
1249299152,7cyclictest0-21swapper/808:55:0110
1248899152,8cyclictest0-21swapper/712:10:019
1248399153,7cyclictest0-21swapper/612:29:598
1247999153,7cyclictest0-21swapper/511:10:017
1247999152,8cyclictest0-21swapper/511:35:007
1247999152,7cyclictest0-21swapper/509:34:597
1246399150,9cyclictest0-21swapper/208:30:244
1245899153,8cyclictest0-21swapper/108:20:011
1245499153,8cyclictest0-21swapper/010:39:590
1245499153,8cyclictest0-21swapper/007:55:010
283491140,1ptp4l21-21ksoftirqd/109:50:491
1249299142,7cyclictest0-21swapper/809:24:5910
1249299142,7cyclictest0-21swapper/808:25:0110
1248899143,7cyclictest0-21swapper/711:30:009
1248399143,7cyclictest20970-21kworker/u24:112:40:018
1248399142,7cyclictest0-21swapper/611:14:598
1247999148,5cyclictest7262-21kworker/u24:111:20:017
1246399142,8cyclictest0-21swapper/211:20:004
1245899143,7cyclictest0-21swapper/107:20:011
1245899142,7cyclictest0-21swapper/112:40:001
1245499141,7cyclictest0-21swapper/010:30:010
300372130,0sleep3321rcuc/308:10:235
283491130,1ptp4l21-21ksoftirqd/110:39:121
283491130,0ptp4l0-21swapper/112:20:001
283491130,0ptp4l0-21swapper/108:09:571
190802130,0sleep50-21swapper/510:05:137
1249299133,6cyclictest0-21swapper/809:15:0110
1249299132,7cyclictest0-21swapper/807:25:0010
1248899132,7cyclictest0-21swapper/710:44:599
1248399133,7cyclictest0-21swapper/609:59:598
1248399132,7cyclictest0-21swapper/610:15:228
1248399132,6cyclictest0-21swapper/608:40:238
1247999133,6cyclictest0-21swapper/509:55:007
1247999132,7cyclictest0-21swapper/509:55:247
1246399132,8cyclictest0-21swapper/211:24:594
1245899133,6cyclictest0-21swapper/107:25:001
1245899132,6cyclictest0-21swapper/112:10:231
1245499132,8cyclictest0-21swapper/010:55:010
119802136,5sleep90-21swapper/907:08:2311
283491120,2ptp4l9319-21users11:55:251
283491120,0ptp4l0-21swapper/112:15:141
283491120,0ptp4l0-21swapper/111:25:121
283491120,0ptp4l0-21swapper/111:10:131
283491120,0ptp4l0-21swapper/110:45:201
283491120,0ptp4l0-21swapper/109:25:161
283491120,0ptp4l0-21swapper/108:30:121
283491120,0ptp4l0-21swapper/108:10:121
283491120,0ptp4l0-21swapper/107:50:081
283491120,0ptp4l0-21swapper/107:48:051
1250699122,6cyclictest0-21swapper/1111:05:243
1250699122,6cyclictest0-21swapper/1109:44:593
1250699120,8cyclictest81-21ksoftirqd/1110:59:593
12501991210,1cyclictest0-21swapper/1012:03:222
1249799122,6cyclictest0-21swapper/912:24:5911
1248899122,6cyclictest0-21swapper/710:50:009
1248899122,6cyclictest0-21swapper/709:15:019
1248899120,7cyclictest0-21swapper/711:25:009
1248399122,6cyclictest0-21swapper/607:40:018
1247999123,4cyclictest0-21swapper/511:35:007
1247999122,6cyclictest0-21swapper/509:00:017
1246799122,9cyclictest0-21swapper/310:29:595
1246799122,9cyclictest0-21swapper/309:00:015
1246799122,8cyclictest0-21swapper/309:35:005
1246399123,8cyclictest0-21swapper/208:45:014
1245899122,6cyclictest0-21swapper/109:44:591
1245899120,7cyclictest0-21swapper/111:45:011
283491110,6ptp4l21-21ksoftirqd/109:03:221
283491110,2ptp4l5755-21fw_forwarded_lo07:40:151
283491110,2ptp4l24415-21kernelversion10:10:171
283491110,2ptp4l20889-21fschecks_time08:00:131
283491110,2ptp4l10175-21/usr/sbin/munin12:00:131
283491110,1ptp4l0-21swapper/112:25:141
283491110,1ptp4l0-21swapper/111:50:171
283491110,1ptp4l0-21swapper/111:15:171
283491110,1ptp4l0-21swapper/110:30:161
283491110,1ptp4l0-21swapper/110:15:141
283491110,1ptp4l0-21swapper/107:55:271
283491110,1ptp4l0-21swapper/107:25:171
283491110,1ptp4l0-21swapper/107:10:141
283491110,0ptp4l0-21swapper/109:20:201
283491110,0ptp4l0-21swapper/109:15:011
1250699112,5cyclictest0-21swapper/1107:20:003
1250699111,6cyclictest0-21swapper/1112:15:003
1250199110,1cyclictest0-21swapper/1010:00:212
1250199110,1cyclictest0-21swapper/1008:32:182
1249799112,7cyclictest0-21swapper/911:45:0011
1249299110,10cyclictest0-21swapper/809:40:1510
1248899112,6cyclictest0-21swapper/707:40:019
1248899111,6cyclictest0-21swapper/712:20:009
1247999114,6cyclictest0-21swapper/511:50:007
1247999112,6cyclictest0-21swapper/507:10:017
1246799114,6cyclictest0-21swapper/311:44:595
1246799113,7cyclictest0-21swapper/311:49:595
1246799113,7cyclictest0-21swapper/310:45:005
1246799113,7cyclictest0-21swapper/308:00:225
1246799113,7cyclictest0-21swapper/307:10:005
1246799112,7cyclictest0-21swapper/312:20:005
1246799112,6cyclictest0-21swapper/307:55:015
1246799110,2cyclictest0-21swapper/307:25:015
1246399118,2cyclictest3201-21missed_timers09:00:184
1246399117,3cyclictest4585-21cat12:35:014
1245899110,3cyclictest2493-21switchtime11:05:221
283491100,1ptp4l6343-21sed08:25:011
283491100,1ptp4l21-21ksoftirqd/110:20:181
283491100,1ptp4l12446-21cyclictest09:35:251
283491100,1ptp4l0-21swapper/111:40:131
283491100,0ptp4l0-21swapper/111:00:171
283491100,0ptp4l0-21swapper/110:55:161
283491100,0ptp4l0-21swapper/109:05:161
1250699102,5cyclictest0-21swapper/1110:20:003
1250699102,5cyclictest0-21swapper/1109:05:013
1250699101,6cyclictest0-21swapper/1112:35:003
1250699100,9cyclictest26533-21munin-run11:40:003
1250199107,2cyclictest8779-21df_inode11:15:122
1250199100,1cyclictest0-21swapper/1009:35:172
1249799102,5cyclictest0-21swapper/908:45:0111
1249799101,6cyclictest0-21swapper/907:50:0011
1249799100,9cyclictest9459-21sed08:25:2311
1249799100,9cyclictest26410-21sed08:05:2411
1249299109,1cyclictest0-21swapper/810:55:0010
1249299101,5cyclictest27003-21kworker/u24:308:10:0110
1249299100,4cyclictest0-21swapper/807:49:5510
1248899107,2cyclictest24336-21munin-run10:55:009
1248899102,5cyclictest0-21swapper/710:19:599
1248899100,9cyclictest27521-21expr11:40:139
1248399109,1cyclictest0-21swapper/608:10:018
1248399106,3cyclictest17467-21runrttasks08:36:088
1248399100,9cyclictest15443-21cpuspeed_turbos10:00:118
1248399100,9cyclictest0-21swapper/608:10:118
1248399100,5cyclictest0-21swapper/609:25:008
1248399100,3cyclictest31898-21expr08:15:118
1248399100,1cyclictest0-21swapper/609:10:138
1247299100,9cyclictest20993-21/usr/sbin/munin09:25:126
1246399107,2cyclictest0-21swapper/211:40:264
1245499102,7cyclictest0-21swapper/008:30:010
1245499100,9cyclictest0-21swapper/010:15:120
9408090,0rtkit-daemon0-21swapper/510:40:247
28349190,0ptp4l0-21swapper/112:30:131
28349190,0ptp4l0-21swapper/110:50:011
28349190,0ptp4l0-21swapper/109:40:011
28349190,0ptp4l0-21swapper/109:15:021
28349190,0ptp4l0-21swapper/108:45:021
28349190,0ptp4l0-21swapper/108:40:131
28349190,0ptp4l0-21swapper/107:35:021
125069998,1cyclictest0-21swapper/1111:45:193
125069996,2cyclictest18910-21turbostat11:29:593
125069996,2cyclictest14431-21latency_hist08:35:013
125069991,5cyclictest0-21swapper/1108:45:053
125069990,8cyclictest11324-21fw_conntrack12:00:153
125069990,4cyclictest0-21swapper/1109:10:243
125019990,1cyclictest0-21swapper/1007:15:232
124979992,6cyclictest0-21swapper/911:04:5911
124979992,5cyclictest0-21swapper/908:55:0111
124979991,6cyclictest0-21swapper/907:50:2311
124979991,5cyclictest0-21swapper/910:55:0011
124979991,5cyclictest0-21swapper/909:55:0011
124979991,5cyclictest0-21swapper/909:49:5911
124979991,5cyclictest0-21swapper/907:40:0011
124979990,9cyclictest0-21swapper/910:45:1311
124979990,9cyclictest0-21swapper/910:14:5911
124929998,1cyclictest0-21swapper/808:45:1410
124929997,1cyclictest0-21swapper/810:05:1910
124929990,9cyclictest0-21swapper/812:00:1210
124929990,9cyclictest0-21swapper/810:55:1510
124929990,9cyclictest0-21swapper/810:40:1810
124929990,9cyclictest0-21swapper/807:10:1310
124889990,8cyclictest25387-21phc2sys11:35:199
124839998,1cyclictest0-21swapper/611:20:268
124839992,6cyclictest31155-21grep10:20:128
124839990,8cyclictest22761-21sendmail_mailqu09:25:198
124839990,8cyclictest0-21swapper/610:55:138
124839990,8cyclictest0-21swapper/607:25:248
124839990,3cyclictest5443-21/usr/sbin/munin09:05:128
124839990,3cyclictest23810-21/usr/sbin/munin10:10:148
124839990,2cyclictest11275-21tune2fs12:00:138
124839990,1cyclictest0-21swapper/610:35:138
124839990,1cyclictest0-21swapper/609:40:238
124839990,1cyclictest0-21swapper/609:00:128
124799990,8cyclictest30379-21irqrtprio11:00:207
124729992,4cyclictest0-21swapper/408:20:236
124729992,4cyclictest0-21swapper/407:30:236
124679998,1cyclictest0-21swapper/309:15:235
124679991,5cyclictest0-21swapper/312:40:005
124679990,8cyclictest263582sleep311:37:325
124639990,9cyclictest0-21swapper/207:10:134
124639990,2cyclictest26608-21fw_conntrack12:20:164
124639990,2cyclictest0-21swapper/210:05:164
124549992,4cyclictest0-21swapper/009:29:590
28359180,7phc2sys0-21swapper/212:28:134
125069982,4cyclictest0-21swapper/1108:15:013
125069981,4cyclictest0-21swapper/1111:45:003
125069981,4cyclictest0-21swapper/1110:00:003
125069980,8cyclictest0-21swapper/1108:45:013
125069980,8cyclictest0-21swapper/1108:20:163
125069980,8cyclictest0-21swapper/1107:30:163
125069980,1cyclictest0-21swapper/1107:35:123
125019987,1cyclictest0-21swapper/1012:25:232
125019987,1cyclictest0-21swapper/1010:45:122
125019987,1cyclictest0-21swapper/1010:25:132
125019987,1cyclictest0-21swapper/1009:10:192
125019987,1cyclictest0-21swapper/1009:00:172
125019980,8cyclictest0-21swapper/1011:20:192
125019980,8cyclictest0-21swapper/1011:05:252
125019980,8cyclictest0-21swapper/1010:14:542
125019980,8cyclictest0-21swapper/1008:45:112
125019980,8cyclictest0-21swapper/1008:35:192
125019980,8cyclictest0-21swapper/1007:40:172
125019980,7cyclictest0-21swapper/1008:13:122
125019980,1cyclictest0-21swapper/1009:20:122
125019980,1cyclictest0-21swapper/1007:35:242
124979987,1cyclictest0-21swapper/912:25:2211
124979987,1cyclictest0-21swapper/912:00:2011
124979987,1cyclictest0-21swapper/909:30:1711
124979981,4cyclictest0-21swapper/911:45:3011
124979980,1cyclictest4966-21diskstats11:10:1211
124979980,1cyclictest0-21swapper/912:05:2011
124979980,1cyclictest0-21swapper/911:55:1111
124979980,1cyclictest0-21swapper/911:30:1411
124979980,1cyclictest0-21swapper/909:40:1611
124929987,1cyclictest0-21swapper/812:30:2310
124929987,1cyclictest0-21swapper/810:35:1610
124929987,1cyclictest0-21swapper/810:20:1210
124929987,1cyclictest0-21swapper/808:25:1210
124929987,1cyclictest0-21swapper/808:00:1610
124929980,8cyclictest0-21swapper/811:15:1410
124929980,8cyclictest0-21swapper/808:15:1610
124929980,8cyclictest0-21swapper/807:50:1710
124929980,1cyclictest0-21swapper/812:35:1710
124929980,1cyclictest0-21swapper/807:35:1910
124889981,4cyclictest0-21swapper/712:00:009
124889980,8cyclictest0-21swapper/708:30:139
124889980,1cyclictest0-21swapper/711:00:169
124839987,1cyclictest0-21swapper/612:30:248
124839987,1cyclictest0-21swapper/611:00:188
124839982,5cyclictest115650irq/18-p34p111:15:308
124839982,5cyclictest115650irq/18-p34p111:05:288
124839982,5cyclictest115650irq/18-p34p110:50:558
124839982,5cyclictest115650irq/18-p34p109:49:568
124839981,4cyclictest0-21swapper/607:55:268
124839980,8cyclictest0-21swapper/612:05:218
124839980,8cyclictest0-21swapper/609:30:208
124839980,8cyclictest0-21swapper/607:50:118
124839980,7cyclictest115650irq/18-p34p110:05:398
124839980,7cyclictest0-21swapper/611:25:178
124839980,7cyclictest0-21swapper/609:15:128
124839980,3cyclictest29230-21/usr/sbin/munin09:35:168
124839980,2cyclictest6916-21cat11:55:148
124839980,2cyclictest27798-21/usr/sbin/munin08:50:198
124839980,2cyclictest15680-21users07:10:268
124839980,1cyclictest0-21swapper/607:20:218
124799987,1cyclictest0-21swapper/509:10:137
124799985,2cyclictest31370-21turbostat09:40:007
124799980,7cyclictest0-21swapper/511:25:187
124799980,7cyclictest0-21swapper/510:02:517
124799980,4cyclictest0-21swapper/508:30:017
124799980,1cyclictest0-21swapper/512:10:237
124799980,1cyclictest0-21swapper/508:30:157
124729980,8cyclictest0-21swapper/412:10:146
124729980,8cyclictest0-21swapper/409:15:186
124729980,1cyclictest0-21swapper/411:10:176
124729980,1cyclictest0-21swapper/408:40:156
124679987,1cyclictest0-21swapper/312:10:125
124679987,1cyclictest0-21swapper/311:55:225
124679987,1cyclictest0-21swapper/310:45:205
124679987,1cyclictest0-21swapper/307:40:015
124679987,1cyclictest0-21swapper/307:25:155
124679986,1cyclictest8486-21diskmemload10:50:165
124679985,2cyclictest0-21swapper/308:50:145
124679981,6cyclictest0-21swapper/307:20:015
124679981,4cyclictest0-21swapper/311:30:015
124679980,8cyclictest0-21swapper/309:45:185
124679980,1cyclictest0-21swapper/312:05:205
124679980,1cyclictest0-21swapper/311:05:195
124679980,1cyclictest0-21swapper/310:30:165
124679980,1cyclictest0-21swapper/310:15:175
124679980,1cyclictest0-21swapper/309:50:225
124639987,1cyclictest0-21swapper/210:25:154
124639987,1cyclictest0-21swapper/208:50:174
124639985,2cyclictest0-21swapper/208:35:134
124639984,3cyclictest30202-21grep07:30:154
124639980,8cyclictest0-21swapper/208:10:014
124639980,8cyclictest0-21swapper/207:40:154
124639980,1cyclictest0-21swapper/209:15:124
124549987,1cyclictest0-21swapper/010:20:070
124549987,1cyclictest0-21swapper/008:40:200
124549986,1cyclictest30794-21turbostat08:15:000
124549986,1cyclictest0-21swapper/008:45:490
124549984,2cyclictest0-21swapper/009:36:300
124549980,8cyclictest0-21swapper/012:35:230
124549980,8cyclictest0-21swapper/011:40:180
124549980,8cyclictest0-21swapper/011:34:590
124549980,5cyclictest0-21swapper/009:41:300
124549980,1cyclictest0-21swapper/012:04:500
124549980,1cyclictest0-21swapper/011:53:430
125069976,1cyclictest0-21swapper/1112:20:013
125069976,1cyclictest0-21swapper/1111:55:163
125069976,1cyclictest0-21swapper/1111:19:003
125069976,1cyclictest0-21swapper/1110:50:193
125069976,1cyclictest0-21swapper/1109:45:123
125069976,1cyclictest0-21swapper/1109:37:543
125069976,1cyclictest0-21swapper/1109:25:183
125069976,1cyclictest0-21swapper/1109:18:203
125069976,1cyclictest0-21swapper/1108:15:133
125069976,1cyclictest0-21swapper/1108:05:163
125069976,1cyclictest0-21swapper/1108:00:113
125069976,1cyclictest0-21swapper/1107:45:083
125069973,3cyclictest10075-21cat09:50:233
125069970,7cyclictest0-21swapper/1111:30:433
125069970,7cyclictest0-21swapper/1110:11:563
125069970,7cyclictest0-21swapper/1110:01:193
125069970,7cyclictest0-21swapper/1109:30:243
125069970,7cyclictest0-21swapper/1109:05:283
125069970,7cyclictest0-21swapper/1108:55:333
125069970,7cyclictest0-21swapper/1108:27:073
125069970,7cyclictest0-21swapper/1107:50:453
125069970,1cyclictest0-21swapper/1112:36:383
125069970,1cyclictest0-21swapper/1112:15:523
125069970,1cyclictest0-21swapper/1112:08:003
125069970,1cyclictest0-21swapper/1111:50:323
125069970,1cyclictest0-21swapper/1111:20:033
125069970,1cyclictest0-21swapper/1111:00:403
125069970,1cyclictest0-21swapper/1110:46:453
125069970,1cyclictest0-21swapper/1110:41:133
125069970,1cyclictest0-21swapper/1110:37:593
125069970,1cyclictest0-21swapper/1110:31:333
125069970,1cyclictest0-21swapper/1110:25:453
125069970,1cyclictest0-21swapper/1110:22:113
125069970,1cyclictest0-21swapper/1109:21:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional