You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-08 - 03:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Mon Dec 08, 2025 00:44:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
275991930,83ptp4l0-21swapper/119:07:551
94980830,78rtkit-daemon0-21swapper/1019:09:112
1108426756,9sleep00-21swapper/019:08:230
1092826754,10sleep50-21swapper/519:06:167
906126654,9sleep60-21swapper/619:05:178
1097526452,9sleep110-21swapper/1119:06:583
1115826351,10sleep70-21swapper/719:09:309
322162610,0sleep60-21swapper/623:05:198
244862610,0sleep20-21swapper/222:55:134
244862610,0sleep20-21swapper/222:55:134
194802600,0sleep110-21swapper/1123:30:143
1107325746,8sleep40-21swapper/419:08:156
87032560,0sleep60-21swapper/620:25:258
1156199252,7cyclictest0-21swapper/422:55:006
1156199252,7cyclictest0-21swapper/422:55:006
1156199212,15cyclictest0-21swapper/423:50:016
1156199202,14cyclictest0-21swapper/423:25:006
1154399203,11cyclictest0-21swapper/021:55:000
1157999182,10cyclictest0-21swapper/822:10:0010
54232170,0sleep00-21swapper/020:23:190
275991170,0ptp4l0-21swapper/100:00:191
1158399172,6cyclictest0-21swapper/900:05:0111
1155699173,8cyclictest0-21swapper/320:05:005
1155699173,8cyclictest0-21swapper/319:15:005
1155299173,9cyclictest0-21swapper/223:15:014
1155299172,9cyclictest0-21swapper/223:20:014
1155299172,10cyclictest0-21swapper/223:50:004
1155299171,2cyclictest0-21swapper/220:30:164
1155299171,2cyclictest0-21swapper/219:30:204
1154399171,12cyclictest25522-21kworker/u24:123:25:010
275991160,0ptp4l0-21swapper/119:10:251
121492160,0sleep00-21swapper/019:50:150
1155699163,8cyclictest0-21swapper/300:20:015
1155699162,9cyclictest0-21swapper/300:10:015
1155299162,9cyclictest0-21swapper/221:00:004
1155299162,9cyclictest0-21swapper/200:40:014
1155299162,10cyclictest0-21swapper/219:45:004
1154399162,9cyclictest0-21swapper/023:15:000
1154399162,8cyclictest0-21swapper/019:30:000
275991150,1ptp4l21-21ksoftirqd/122:45:251
275991150,1ptp4l21-21ksoftirqd/122:45:251
211172150,0sleep20-21swapper/219:20:144
191491150,0getstats0-21swapper/119:35:301
1158999158,4cyclictest0-21swapper/1019:35:002
1158399151,7cyclictest0-21swapper/923:00:2511
1158399151,7cyclictest0-21swapper/923:00:2511
1157999152,8cyclictest0-21swapper/821:40:0010
1157099152,7cyclictest0-21swapper/622:35:258
1157099152,7cyclictest0-21swapper/622:35:258
1155699153,7cyclictest0-21swapper/320:35:005
1155699153,7cyclictest0-21swapper/320:30:005
1155699152,8cyclictest0-21swapper/320:00:005
1155699152,8cyclictest0-21swapper/300:00:015
1155299152,7cyclictest0-21swapper/222:10:004
1154399153,8cyclictest0-21swapper/022:55:010
1154399153,8cyclictest0-21swapper/022:55:010
1154399153,8cyclictest0-21swapper/019:35:000
1154399152,8cyclictest0-21swapper/000:30:010
108792157,6sleep80-21swapper/819:05:3110
275991140,1ptp4l21-21ksoftirqd/121:30:171
275991140,1ptp4l21-21ksoftirqd/120:25:221
1155699142,8cyclictest0-21swapper/322:30:255
1155699142,7cyclictest0-21swapper/319:10:005
1155699142,7cyclictest0-21swapper/300:12:425
1154399143,7cyclictest0-21swapper/023:35:000
275991130,1ptp4l21-21ksoftirqd/123:58:211
275991130,1ptp4l21-21ksoftirqd/121:52:191
275991130,1ptp4l21-21ksoftirqd/120:37:471
275991130,1ptp4l21-21ksoftirqd/119:27:391
275991130,0ptp4l0-21swapper/122:58:381
275991130,0ptp4l0-21swapper/122:58:381
275991130,0ptp4l0-21swapper/100:33:551
1158999132,6cyclictest0-21swapper/1021:00:252
1157999133,6cyclictest0-21swapper/800:25:2710
1157999131,7cyclictest0-21swapper/820:55:0010
1157999130,8cyclictest0-21swapper/820:20:2710
1157999130,12cyclictest0-21swapper/822:35:0010
1157099130,7cyclictest0-21swapper/621:45:008
1156199132,7cyclictest0-21swapper/422:30:006
1156199131,7cyclictest0-21swapper/423:50:256
1155699133,7cyclictest0-21swapper/323:55:005
1155699133,7cyclictest0-21swapper/323:20:015
1155699132,7cyclictest0-21swapper/322:25:005
1155699132,6cyclictest0-21swapper/322:45:015
1155699132,6cyclictest0-21swapper/322:45:015
1154799133,6cyclictest0-21swapper/122:35:011
1154799133,6cyclictest0-21swapper/122:35:011
1154799132,7cyclictest0-21swapper/121:15:001
1154799132,7cyclictest0-21swapper/120:40:291
1154799130,8cyclictest0-21swapper/120:05:001
1154399132,6cyclictest0-21swapper/022:45:010
1154399132,6cyclictest0-21swapper/022:45:010
1154399132,6cyclictest0-21swapper/021:35:000
275991120,1ptp4l21-21ksoftirqd/119:44:051
275991120,0ptp4l0-21swapper/123:27:531
275991120,0ptp4l0-21swapper/123:05:201
275991120,0ptp4l0-21swapper/122:50:011
275991120,0ptp4l0-21swapper/122:50:011
275991120,0ptp4l0-21swapper/122:20:231
275991120,0ptp4l0-21swapper/121:47:421
275991120,0ptp4l0-21swapper/121:42:531
275991120,0ptp4l0-21swapper/121:20:221
275991120,0ptp4l0-21swapper/121:02:131
275991120,0ptp4l0-21swapper/120:33:101
275991120,0ptp4l0-21swapper/120:15:261
1157999122,5cyclictest0-21swapper/819:10:3010
1157999121,6cyclictest0-21swapper/819:40:2410
1157099122,6cyclictest0-21swapper/620:50:008
11570991211,1cyclictest0-21swapper/621:30:198
1156199121,7cyclictest0-21swapper/420:30:246
1154799123,7cyclictest0-21swapper/120:50:001
1154799122,7cyclictest0-21swapper/122:20:001
1154799122,7cyclictest0-21swapper/100:15:001
1154799122,6cyclictest0-21swapper/123:45:231
1154799122,6cyclictest0-21swapper/100:05:251
1154799121,8cyclictest0-21swapper/119:55:251
105542120,0sleep100-21swapper/1019:45:242
275991110,1ptp4l21-21ksoftirqd/121:55:061
275991110,1ptp4l0-21swapper/122:45:001
275991110,1ptp4l0-21swapper/122:45:001
275991110,1ptp4l0-21swapper/122:00:141
275991110,1ptp4l0-21swapper/121:40:001
275991110,1ptp4l0-21swapper/121:15:141
275991110,1ptp4l0-21swapper/121:00:001
275991110,1ptp4l0-21swapper/120:50:151
275991110,1ptp4l0-21swapper/119:20:161
191491110,0getstats0-21swapper/120:05:301
1158999112,6cyclictest0-21swapper/1020:20:002
1158999110,10cyclictest0-21swapper/1023:55:002
1158399112,5cyclictest0-21swapper/919:50:2611
1157499112,6cyclictest0-21swapper/700:05:009
1157099115,5cyclictest0-21swapper/620:09:558
1156199111,7cyclictest0-21swapper/421:40:246
1155699118,2cyclictest0-21swapper/322:50:165
1155699118,2cyclictest0-21swapper/322:50:165
1155299112,6cyclictest0-21swapper/200:10:014
1155299110,10cyclictest17604-21smartctl19:55:244
1154799112,7cyclictest0-21swapper/123:15:001
11547991110,1cyclictest201rcuc/122:30:201
1154399111,6cyclictest0-21swapper/021:30:260
275991100,2ptp4l23655-21irqrtprio22:10:181
275991100,2ptp4l0-21swapper/119:45:171
275991100,1ptp4l21-21ksoftirqd/123:18:151
275991100,1ptp4l20192-21fschecks_time23:30:161
275991100,1ptp4l0-21swapper/123:50:201
275991100,1ptp4l0-21swapper/122:25:161
275991100,0ptp4l0-21swapper/123:35:161
275991100,0ptp4l0-21swapper/121:05:221
275991100,0ptp4l0-21swapper/119:15:011
275991100,0ptp4l0-21swapper/100:39:091
1159399100,1cyclictest0-21swapper/1122:35:133
1159399100,1cyclictest0-21swapper/1122:35:133
1158999108,1cyclictest0-21swapper/1023:45:142
1158999100,9cyclictest20089-21latency_hist22:50:002
1158999100,9cyclictest20089-21latency_hist22:49:592
1158999100,2cyclictest13355-21memory19:50:202
1158999100,2cyclictest11298-21cut22:35:202
1158999100,2cyclictest11298-21cut22:35:202
1158399107,2cyclictest13763-21df22:40:1411
1158399107,2cyclictest13763-21df22:40:1411
1158399102,5cyclictest0-21swapper/923:25:0111
1157999107,2cyclictest3979-21turbostat21:05:0010
1157999100,9cyclictest0-21swapper/823:55:1410
1157999100,1cyclictest0-21swapper/821:10:1910
1157499102,5cyclictest0-21swapper/722:00:019
1157499102,5cyclictest0-21swapper/721:00:009
1157499102,5cyclictest0-21swapper/719:20:009
1157499102,5cyclictest0-21swapper/700:35:019
1157099102,5cyclictest0-21swapper/619:25:008
1157099100,9cyclictest8649-21cat21:10:148
1157099100,9cyclictest14748-21munin-run23:25:008
1157099100,9cyclictest10029-21cpu00:00:168
1156599100,9cyclictest8411-21diskstats23:15:147
1156199100,5cyclictest0-21swapper/421:00:006
1155699103,6cyclictest0-21swapper/319:40:005
1155299102,5cyclictest0-21swapper/221:15:004
1155299102,5cyclictest0-21swapper/200:30:014
1155299101,6cyclictest25522-21kworker/u24:121:55:004
1155299100,9cyclictest0-21swapper/219:10:154
1155299100,9cyclictest0-21swapper/200:00:284
1155299100,6cyclictest0-21swapper/220:24:074
1155299100,2cyclictest27179-21grep23:40:144
27599190,1ptp4l21-21ksoftirqd/120:20:011
27599190,0ptp4l0-21swapper/123:40:151
27599190,0ptp4l0-21swapper/123:20:131
27599190,0ptp4l0-21swapper/123:00:131
27599190,0ptp4l0-21swapper/123:00:131
27599190,0ptp4l0-21swapper/122:05:141
27599190,0ptp4l0-21swapper/121:25:131
27599190,0ptp4l0-21swapper/120:10:161
27599190,0ptp4l0-21swapper/119:50:151
27599190,0ptp4l0-21swapper/119:30:151
27599190,0ptp4l0-21swapper/100:25:161
27599190,0ptp4l0-21swapper/100:20:131
27599190,0ptp4l0-21swapper/100:15:151
115939998,1cyclictest0-21swapper/1122:00:233
115939998,1cyclictest0-21swapper/1100:35:253
115939997,1cyclictest20093-21turbostat22:50:003
115939997,1cyclictest20093-21turbostat22:50:003
115939997,1cyclictest0-21swapper/1100:35:003
115939990,9cyclictest0-21swapper/1123:40:233
115939990,1cyclictest0-21swapper/1123:35:253
115939990,1cyclictest0-21swapper/1122:40:213
115939990,1cyclictest0-21swapper/1122:40:213
115939990,1cyclictest0-21swapper/1119:55:163
115939990,1cyclictest0-21swapper/1119:50:213
115899998,1cyclictest0-21swapper/1020:25:002
115899990,8cyclictest0-21swapper/1000:15:162
115839991,6cyclictest0-21swapper/921:46:4011
115839991,5cyclictest0-21swapper/920:20:0011
115839990,9cyclictest0-21swapper/919:40:1511
115839990,2cyclictest1020-21runrttasks23:56:2411
115839990,1cyclictest0-21swapper/921:15:0011
115799998,1cyclictest0-21swapper/819:50:1610
115749996,2cyclictest11822-21memory20:30:199
115749990,8cyclictest22170-21df20:45:159
115709995,3cyclictest24704-21munin-run00:20:008
115659998,1cyclictest0-21swapper/523:25:237
115659998,1cyclictest0-21swapper/521:25:197
115659990,9cyclictest0-21swapper/522:45:167
115659990,9cyclictest0-21swapper/522:45:167
115659990,8cyclictest15241-21ptp4l-jitter22:40:217
115659990,8cyclictest15241-21ptp4l-jitter22:40:217
115619990,9cyclictest0-21swapper/421:12:186
115619990,8cyclictest14900-21fw_conntrack00:05:166
115619990,2cyclictest20755-21sshd00:13:196
115569990,8cyclictest1009-21snmpd19:15:495
115569990,8cyclictest0-21swapper/321:00:155
115569990,1cyclictest0-21swapper/300:20:135
115529998,1cyclictest0-21swapper/200:30:174
115529991,6cyclictest30473-21expr22:20:154
115529991,4cyclictest0-21swapper/221:20:004
115529990,8cyclictest8788-21cat23:55:254
115529990,8cyclictest0-21swapper/219:45:194
115529990,1cyclictest0-21swapper/222:00:234
115529990,1cyclictest0-21swapper/220:40:184
115529990,1cyclictest0-21swapper/220:35:214
115439992,4cyclictest0-21swapper/020:15:000
11186291,6sleep90-21swapper/919:09:5711
115939987,1cyclictest0-21swapper/1120:40:183
115939987,1cyclictest0-21swapper/1119:45:183
115939987,1cyclictest0-21swapper/1119:25:293
115939980,8cyclictest0-21swapper/1122:50:153
115939980,8cyclictest0-21swapper/1122:50:153
115939980,8cyclictest0-21swapper/1121:49:263
115939980,8cyclictest0-21swapper/1120:15:183
115939980,8cyclictest0-21swapper/1119:10:263
115939980,1cyclictest0-21swapper/1122:20:203
115939980,1cyclictest0-21swapper/1121:55:173
115939980,1cyclictest0-21swapper/1121:50:263
115939980,1cyclictest0-21swapper/1121:25:263
115939980,1cyclictest0-21swapper/1120:46:273
115939980,1cyclictest0-21swapper/1120:05:143
115939980,1cyclictest0-21swapper/1120:04:313
115899987,1cyclictest0-21swapper/1023:25:212
115899987,1cyclictest0-21swapper/1022:40:212
115899987,1cyclictest0-21swapper/1022:40:212
115899987,1cyclictest0-21swapper/1021:40:252
115899987,1cyclictest0-21swapper/1019:10:262
115899985,2cyclictest25103-21/usr/sbin/munin22:10:252
115899982,4cyclictest0-21swapper/1019:20:512
115899980,8cyclictest0-21swapper/1022:57:392
115899980,8cyclictest0-21swapper/1022:57:382
115899980,8cyclictest0-21swapper/1022:05:232
115899980,8cyclictest0-21swapper/1000:05:132
115899980,1cyclictest0-21swapper/1023:15:252
115899980,1cyclictest0-21swapper/1021:55:162
115899980,1cyclictest0-21swapper/1021:05:172
115899980,1cyclictest0-21swapper/1000:20:222
115839987,1cyclictest0-21swapper/922:00:1611
115839982,4cyclictest0-21swapper/920:50:0011
115839980,1cyclictest0-21swapper/922:45:2311
115839980,1cyclictest0-21swapper/922:45:2311
115839980,1cyclictest0-21swapper/900:20:1311
115799987,1cyclictest0-21swapper/823:30:2010
115799987,1cyclictest0-21swapper/823:05:2610
115799986,1cyclictest0-21swapper/823:54:4410
115799985,2cyclictest1972-21chrt21:40:4710
115799980,1cyclictest0-21swapper/822:50:2110
115799980,1cyclictest0-21swapper/822:50:2110
115799980,1cyclictest0-21swapper/822:48:1310
115799980,1cyclictest0-21swapper/822:48:1310
115799980,1cyclictest0-21swapper/821:50:1810
115749981,4cyclictest0-21swapper/723:00:009
115749981,4cyclictest0-21swapper/723:00:009
115749980,7cyclictest0-21swapper/723:04:559
115749980,7cyclictest0-21swapper/723:04:559
115749980,1cyclictest0-21swapper/723:35:199
115749980,1cyclictest0-21swapper/721:40:189
115749980,1cyclictest0-21swapper/720:50:179
115749980,1cyclictest0-21swapper/720:20:279
115749980,1cyclictest0-21swapper/700:10:149
115709987,1cyclictest0-21swapper/622:05:188
115709980,8cyclictest0-21swapper/620:15:208
115709980,3cyclictest0-21swapper/600:00:008
115709980,1cyclictest0-21swapper/622:40:208
115709980,1cyclictest0-21swapper/622:40:208
115709980,1cyclictest0-21swapper/620:10:258
115709980,1cyclictest0-21swapper/619:13:158
115659987,1cyclictest0-21swapper/523:46:307
115659987,1cyclictest0-21swapper/523:35:137
115659987,1cyclictest0-21swapper/523:05:237
115659987,1cyclictest0-21swapper/522:50:017
115659987,1cyclictest0-21swapper/522:50:017
115659987,1cyclictest0-21swapper/521:55:157
115659987,1cyclictest0-21swapper/521:50:167
115659987,1cyclictest0-21swapper/521:05:237
115659987,1cyclictest0-21swapper/520:58:087
115659987,1cyclictest0-21swapper/520:45:207
115659985,2cyclictest3990-21turbostat00:35:007
115659984,3cyclictest24843-21head21:30:187
115659980,8cyclictest0-21swapper/520:35:147
115659980,8cyclictest0-21swapper/519:50:197
115659980,1cyclictest5843-21strings19:40:207
115659980,1cyclictest0-21swapper/523:58:167
115659980,1cyclictest0-21swapper/522:15:237
115659980,1cyclictest0-21swapper/520:40:177
115659980,1cyclictest0-21swapper/519:15:277
115619987,1cyclictest0-21swapper/420:25:006
115619987,1cyclictest0-21swapper/400:05:006
115619980,7cyclictest0-21swapper/419:10:276
115619980,3cyclictest28519-21latency_hist00:25:016
115619980,1cyclictest0-21swapper/422:00:236
115619980,1cyclictest0-21swapper/421:30:256
115619980,1cyclictest0-21swapper/421:25:256
115569980,8cyclictest0-21swapper/319:30:215
115569980,8cyclictest0-21swapper/300:00:195
115569980,1cyclictest0-21swapper/322:25:255
115529987,1cyclictest0-21swapper/223:05:204
115529987,1cyclictest0-21swapper/222:45:224
115529987,1cyclictest0-21swapper/222:45:224
115529987,1cyclictest0-21swapper/222:20:004
115529987,1cyclictest0-21swapper/220:52:174
115529980,8cyclictest0-21swapper/222:30:264
115529980,8cyclictest0-21swapper/220:00:124
115529980,8cyclictest0-21swapper/219:50:204
115529980,7cyclictest24436-21runrttasks00:16:204
115529980,7cyclictest1480-21cat22:25:134
115529980,7cyclictest0-21swapper/221:25:154
115529980,7cyclictest0-21swapper/220:25:144
115529980,2cyclictest2516-21/usr/sbin/munin19:35:254
115529980,2cyclictest0-21swapper/219:15:154
115529980,1cyclictest0-21swapper/223:35:194
115529980,1cyclictest0-21swapper/223:25:274
115529980,1cyclictest0-21swapper/222:10:124
115439980,7cyclictest7841-21diskmemload21:15:230
115439980,2cyclictest7841-21diskmemload23:48:500
115439980,1cyclictest0-21swapper/020:50:170
9498070,0rtkit-daemon0-21swapper/1019:43:462
6110270,1sleep70-21swapper/723:55:149
5192270,0sleep30-21swapper/300:35:135
27609170,1phc2sys0-21swapper/219:07:044
115939976,1cyclictest0-21swapper/1123:25:193
115939976,1cyclictest0-21swapper/1123:16:553
115939976,1cyclictest0-21swapper/1123:00:313
115939976,1cyclictest0-21swapper/1123:00:313
115939976,1cyclictest0-21swapper/1122:30:513
115939976,1cyclictest0-21swapper/1122:15:353
115939976,1cyclictest0-21swapper/1121:35:343
115939976,1cyclictest0-21swapper/1121:20:103
115939976,1cyclictest0-21swapper/1121:15:013
115939976,1cyclictest0-21swapper/1121:02:223
115939976,1cyclictest0-21swapper/1120:50:453
115939976,1cyclictest0-21swapper/1119:41:043
115939976,1cyclictest0-21swapper/1119:36:243
115939976,1cyclictest0-21swapper/1119:30:393
115939976,1cyclictest0-21swapper/1100:25:173
115939976,1cyclictest0-21swapper/1100:22:423
115939976,1cyclictest0-21swapper/1100:08:033
115939973,3cyclictest11689-21latency20:30:183
115939970,7cyclictest0-21swapper/1123:45:433
115939970,7cyclictest0-21swapper/1123:11:023
115939970,7cyclictest0-21swapper/1122:05:013
115939970,7cyclictest0-21swapper/1121:31:413
115939970,7cyclictest0-21swapper/1121:10:293
115939970,7cyclictest0-21swapper/1121:06:333
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional