You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-25 - 08:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Tue Nov 25, 2025 00:44:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2759911050,95ptp4l0-21swapper/119:05:311
402026958,8sleep50-21swapper/519:05:207
494726654,9sleep110-21swapper/1119:05:273
561126453,9sleep70-21swapper/719:09:129
557526452,10sleep100-21swapper/1019:08:392
565826352,9sleep00-21swapper/019:09:530
550226251,9sleep40-21swapper/419:07:326
561026153,6sleep60-21swapper/619:09:108
13612580,0sleep30-21swapper/322:30:175
301032570,0sleep60-21swapper/623:50:178
94980250,5rtkit-daemon0-21swapper/819:09:5010
601399233,16cyclictest0-21swapper/400:20:016
6041992214,8cyclictest0-21swapper/1022:40:242
601399222,15cyclictest0-21swapper/400:15:006
602299193,10cyclictest0-21swapper/622:35:008
602299192,7cyclictest0-21swapper/623:30:268
602299192,10cyclictest0-21swapper/622:50:008
6092180,0sleep20-21swapper/200:35:184
602299183,9cyclictest0-21swapper/622:15:008
601399181,15cyclictest0-21swapper/422:40:176
600499170,3cyclictest0-21swapper/223:40:174
599199172,9cyclictest9866-21munin-node19:15:010
275991170,0ptp4l21-21ksoftirqd/100:35:121
275991170,0ptp4l0-21swapper/119:40:191
175782170,0sleep50-21swapper/500:15:197
602299162,8cyclictest0-21swapper/620:45:018
601799162,9cyclictest0-21swapper/522:55:007
601399160,15cyclictest0-21swapper/422:15:186
599199162,8cyclictest0-21swapper/022:19:590
275991160,0ptp4l0-21swapper/121:45:121
275991160,0ptp4l0-21swapper/119:20:191
275991160,0ptp4l0-21swapper/100:20:011
88952150,0sleep20-21swapper/200:05:144
604599152,8cyclictest0-21swapper/1121:35:003
603299153,6cyclictest0-21swapper/822:30:0010
602299153,8cyclictest0-21swapper/622:25:008
602299151,9cyclictest0-21swapper/620:25:018
601799153,8cyclictest0-21swapper/523:55:017
601799153,7cyclictest0-21swapper/500:40:017
601799151,9cyclictest0-21swapper/500:30:297
600499158,6cyclictest0-21swapper/221:50:134
6004991510,4cyclictest30643-21turbostat00:34:594
600499150,3cyclictest0-21swapper/220:00:194
600499150,14cyclictest0-21swapper/222:15:004
599199154,7cyclictest0-21swapper/021:00:010
275991150,2ptp4l28458-21latency19:35:211
275991150,1ptp4l21-21ksoftirqd/122:35:211
275991150,1ptp4l21-21ksoftirqd/120:10:151
275991150,1ptp4l21-21ksoftirqd/119:25:141
275991150,0ptp4l0-21swapper/122:25:211
275991150,0ptp4l0-21swapper/121:00:181
275991150,0ptp4l0-21swapper/120:05:161
151112150,0sleep70-21swapper/721:25:149
604599142,7cyclictest0-21swapper/1122:35:243
604599142,7cyclictest0-21swapper/1122:00:003
604599142,7cyclictest0-21swapper/1119:55:013
603299141,9cyclictest63-21ksoftirqd/820:50:0010
602299143,6cyclictest0-21swapper/619:50:008
602299140,9cyclictest0-21swapper/623:05:008
600499142,9cyclictest0-21swapper/222:25:004
600499142,9cyclictest0-21swapper/221:20:004
600499142,8cyclictest0-21swapper/223:55:254
600499142,8cyclictest0-21swapper/223:29:594
600499142,7cyclictest0-21swapper/221:35:254
600499140,13cyclictest0-21swapper/200:10:144
599199143,6cyclictest0-21swapper/023:10:000
599199142,7cyclictest0-21swapper/021:10:240
599199141,8cyclictest0-21swapper/019:35:010
275991140,0ptp4l0-21swapper/100:10:131
275991140,0ptp4l0-21swapper/100:00:141
197052140,0sleep110-21swapper/1120:05:213
602299132,7cyclictest0-21swapper/600:05:008
602299132,6cyclictest0-21swapper/600:35:248
601799132,7cyclictest0-21swapper/523:00:267
601799132,7cyclictest0-21swapper/522:44:597
601799132,7cyclictest0-21swapper/520:40:017
601799132,7cyclictest0-21swapper/520:15:007
601799131,7cyclictest0-21swapper/523:35:007
601399132,6cyclictest0-21swapper/400:05:246
600499134,8cyclictest0-21swapper/223:00:014
600499132,9cyclictest0-21swapper/222:20:004
600499130,12cyclictest0-21swapper/200:00:204
275991130,4ptp4l0-21swapper/123:20:201
275991130,1ptp4l21-21ksoftirqd/121:40:251
275991130,1ptp4l21-21ksoftirqd/120:22:141
275991130,1ptp4l13918-21tr20:00:131
275991130,0ptp4l0-21swapper/123:50:131
275991130,0ptp4l0-21swapper/123:30:221
275991130,0ptp4l0-21swapper/122:22:211
275991130,0ptp4l0-21swapper/122:15:201
275991130,0ptp4l0-21swapper/122:05:271
275991130,0ptp4l0-21swapper/120:35:131
275991130,0ptp4l0-21swapper/119:10:161
604599122,6cyclictest0-21swapper/1122:09:593
604199123,6cyclictest0-21swapper/1019:20:002
604199121,7cyclictest0-21swapper/1023:05:222
603299122,6cyclictest0-21swapper/820:55:0110
6022991210,1cyclictest0-21swapper/600:15:008
602299120,8cyclictest0-21swapper/619:25:248
601799123,6cyclictest1827-21kworker/u24:000:20:007
601799123,6cyclictest0-21swapper/521:05:017
601799122,6cyclictest0-21swapper/520:30:017
600499123,3cyclictest490-21fschecks_time22:30:134
600499123,2cyclictest0-21swapper/223:30:154
600499122,9cyclictest0-21swapper/222:50:234
600499120,2cyclictest0-21swapper/220:40:124
599199122,7cyclictest0-21swapper/000:30:010
599199122,6cyclictest0-21swapper/021:25:010
599199122,6cyclictest0-21swapper/000:10:010
599199121,6cyclictest0-21swapper/023:15:250
599199120,8cyclictest0-21swapper/000:25:250
275991120,4ptp4l21-21ksoftirqd/100:05:171
275991120,3ptp4l4402-21sshd19:45:221
275991120,0ptp4l21-21ksoftirqd/121:30:331
275991120,0ptp4l0-21swapper/123:55:171
275991120,0ptp4l0-21swapper/123:37:201
275991120,0ptp4l0-21swapper/122:50:191
275991120,0ptp4l0-21swapper/122:46:061
275991120,0ptp4l0-21swapper/121:25:211
275991120,0ptp4l0-21swapper/120:53:421
275991120,0ptp4l0-21swapper/120:30:191
275991120,0ptp4l0-21swapper/120:27:211
275991120,0ptp4l0-21swapper/119:50:201
275991120,0ptp4l0-21swapper/119:30:261
603299112,5cyclictest0-21swapper/820:35:0110
603299110,10cyclictest4045-21sed21:10:1610
602299118,2cyclictest4266-21expr22:35:138
602299112,6cyclictest0-21swapper/620:30:008
602299111,6cyclictest0-21swapper/620:05:238
602299110,7cyclictest0-21swapper/622:54:558
602299110,1cyclictest0-21swapper/621:06:468
601799111,6cyclictest0-21swapper/523:50:017
601799111,6cyclictest0-21swapper/519:50:257
601399119,1cyclictest0-21swapper/422:55:166
601399112,5cyclictest0-21swapper/423:06:596
601399111,6cyclictest0-21swapper/423:50:256
600499118,2cyclictest0-21swapper/220:05:204
600499118,2cyclictest0-21swapper/220:00:004
600499114,2cyclictest9207-21latency_hist23:25:014
600499114,2cyclictest1020-21runrttasks23:14:164
600499113,2cyclictest3670-21fschecks_time21:10:164
600499110,7cyclictest2121-21diskmemload22:42:084
600499110,7cyclictest0-21swapper/222:20:114
600499110,2cyclictest0-21swapper/223:35:174
600499110,11cyclictest0-21swapper/219:45:204
600499110,10cyclictest0-21swapper/221:00:124
599199111,2cyclictest0-21swapper/023:30:170
275991110,6ptp4l21-21ksoftirqd/123:07:261
275991110,2ptp4l19925-21munin-run22:14:591
275991110,2ptp4l10795-21fschecks_time19:55:141
275991110,1ptp4l21-21ksoftirqd/120:55:221
275991110,1ptp4l0-21swapper/122:00:141
275991110,0ptp4l0-21swapper/123:46:011
275991110,0ptp4l0-21swapper/123:44:131
275991110,0ptp4l0-21swapper/123:12:561
275991110,0ptp4l0-21swapper/122:55:241
275991110,0ptp4l0-21swapper/121:20:261
275991110,0ptp4l0-21swapper/121:05:201
275991110,0ptp4l0-21swapper/120:40:121
275991110,0ptp4l0-21swapper/120:15:241
275991110,0ptp4l0-21swapper/119:18:521
275991110,0ptp4l0-21swapper/100:15:511
604599107,2cyclictest30435-21fschecks_count21:45:143
604599103,5cyclictest2254-21memory23:55:203
604599102,5cyclictest0-21swapper/1123:25:003
604599102,5cyclictest0-21swapper/1119:30:013
604599100,9cyclictest0-21swapper/1119:10:143
604199102,4cyclictest0-21swapper/1019:15:012
604199100,2cyclictest19310-21gltestperf21:30:172
603299107,2cyclictest16909-21cat23:34:5910
603299102,5cyclictest0-21swapper/823:05:0110
603299101,6cyclictest0-21swapper/800:30:0110
602299109,1cyclictest0-21swapper/600:25:198
602299108,1cyclictest0-21swapper/623:55:188
601799108,1cyclictest0-21swapper/521:06:457
601799101,6cyclictest26847-21kworker/u24:120:35:007
601399106,3cyclictest24317-21taskset20:12:146
600899109,1cyclictest0-21swapper/319:15:025
600499106,1cyclictest0-21swapper/222:04:224
600499102,3cyclictest808-21netstat21:05:204
600499100,9cyclictest0-21swapper/221:45:134
600499100,7cyclictest0-21swapper/223:05:184
600499100,3cyclictest24750-21smartctl20:55:164
599999100,9cyclictest0-21swapper/100:25:141
599199100,9cyclictest16778-21cpu22:10:120
275991100,3ptp4l21-21ksoftirqd/123:00:181
275991100,2ptp4l9626-21grep21:15:261
275991100,1ptp4l2809-21fschecks_count23:15:161
275991100,1ptp4l2121-21diskmemload21:10:121
275991100,1ptp4l18032-21pluginstate20:45:211
275991100,0ptp4l0-21swapper/122:30:231
60459998,1cyclictest0-21swapper/1122:20:133
60459998,1cyclictest0-21swapper/1100:10:163
60459990,9cyclictest0-21swapper/1120:15:203
60459990,9cyclictest0-21swapper/1119:35:013
60459990,8cyclictest17133-21idleruntime00:15:173
60459990,1cyclictest0-21swapper/1121:20:123
60459990,1cyclictest0-21swapper/1100:05:143
60419990,8cyclictest30680-21tune2fs21:45:152
60419990,8cyclictest0-21swapper/1022:35:142
60329992,6cyclictest0-21swapper/820:05:0010
60329991,5cyclictest0-21swapper/821:50:2210
60329991,5cyclictest0-21swapper/820:40:2510
60269998,1cyclictest0-21swapper/722:04:089
60269990,2cyclictest0-21swapper/720:35:249
60229998,1cyclictest0-21swapper/623:45:118
60229998,1cyclictest0-21swapper/619:10:148
60229998,1cyclictest0-21swapper/600:05:188
60229990,9cyclictest0-21swapper/623:20:138
60229990,9cyclictest0-21swapper/621:40:148
60229990,9cyclictest0-21swapper/600:15:158
60179990,8cyclictest0-21swapper/500:10:157
60179990,1cyclictest0-21swapper/522:30:117
60139992,6cyclictest0-21swapper/419:55:246
60139990,8cyclictest20494-21df00:20:156
60089990,8cyclictest0-21swapper/323:25:215
60049998,1cyclictest0-21swapper/223:00:224
60049998,1cyclictest0-21swapper/220:20:264
60049992,6cyclictest0-21swapper/221:44:594
60049990,3cyclictest26437-21if_err_p34p123:45:164
60049990,1cyclictest0-21swapper/219:30:234
5369291,6sleep90-21swapper/919:05:4911
27599190,1ptp4l5368-21awk21:55:131
27599190,0ptp4l0-21swapper/123:25:011
27599190,0ptp4l0-21swapper/122:40:121
27599190,0ptp4l0-21swapper/121:50:121
27599190,0ptp4l0-21swapper/121:35:161
27599190,0ptp4l0-21swapper/100:30:161
60459987,1cyclictest0-21swapper/1123:35:263
60459987,1cyclictest0-21swapper/1123:10:113
60459987,1cyclictest0-21swapper/1123:00:243
60459987,1cyclictest0-21swapper/1121:05:003
60459987,1cyclictest0-21swapper/1120:35:133
60459987,1cyclictest0-21swapper/1119:15:013
60459985,2cyclictest11860-21expr22:45:133
60459980,1cyclictest0-21swapper/1123:30:213
60459980,1cyclictest0-21swapper/1121:25:243
60419987,1cyclictest0-21swapper/1022:05:132
60419980,8cyclictest0-21swapper/1000:30:182
60419980,8cyclictest0-21swapper/1000:00:202
60419980,2cyclictest1073-21taskset23:11:402
60419980,1cyclictest0-21swapper/1020:35:192
60419980,1cyclictest0-21swapper/1000:35:232
60359987,1cyclictest0-21swapper/923:06:5911
60359987,1cyclictest0-21swapper/921:30:1411
60359987,1cyclictest0-21swapper/920:35:2011
60359987,1cyclictest0-21swapper/900:33:2011
60359980,1cyclictest0-21swapper/923:55:2411
60359980,1cyclictest0-21swapper/922:14:4711
60359980,1cyclictest0-21swapper/900:38:2611
60359980,1cyclictest0-21swapper/900:25:2211
60329987,1cyclictest0-21swapper/800:35:1010
60329980,8cyclictest0-21swapper/822:40:1710
60329980,7cyclictest14289-21latency_hist21:25:0110
60329980,1cyclictest0-21swapper/822:05:2210
60269987,1cyclictest0-21swapper/723:40:209
60269987,1cyclictest0-21swapper/723:35:209
60269987,1cyclictest0-21swapper/723:05:139
60269987,1cyclictest0-21swapper/721:10:149
60269987,1cyclictest0-21swapper/720:05:189
60269987,1cyclictest0-21swapper/719:10:149
60269985,2cyclictest4606-21munin-run21:54:599
60269980,8cyclictest0-21swapper/723:10:139
60269980,8cyclictest0-21swapper/722:15:159
60269980,8cyclictest0-21swapper/720:30:129
60269980,8cyclictest0-21swapper/719:55:249
60269980,8cyclictest0-21swapper/719:20:139
60269980,1cyclictest0-21swapper/722:25:259
60269980,1cyclictest0-21swapper/721:15:229
60229987,1cyclictest0-21swapper/621:50:198
60229987,1cyclictest0-21swapper/620:45:188
60229987,1cyclictest0-21swapper/619:50:238
60229987,1cyclictest0-21swapper/619:25:008
60229980,8cyclictest0-21swapper/623:07:218
60229980,8cyclictest0-21swapper/622:00:218
60229980,8cyclictest0-21swapper/621:47:518
60229980,8cyclictest0-21swapper/621:27:228
60229980,8cyclictest0-21swapper/619:38:228
60229980,8cyclictest0-21swapper/600:20:238
60229980,1cyclictest0-21swapper/621:10:168
60179987,1cyclictest0-21swapper/523:55:157
60179987,1cyclictest0-21swapper/523:05:217
60179987,1cyclictest0-21swapper/519:45:127
60179985,2cyclictest24506-21turbostat23:44:597
60179981,4cyclictest0-21swapper/521:30:007
60179980,8cyclictest0-21swapper/521:30:207
60179980,1cyclictest0-21swapper/520:55:257
60179980,1cyclictest0-21swapper/520:35:147
60179980,1cyclictest0-21swapper/520:05:197
60179980,1cyclictest0-21swapper/519:15:147
60139987,1cyclictest0-21swapper/420:00:136
60139987,1cyclictest0-21swapper/419:30:206
60139986,1cyclictest0-21swapper/423:50:016
60139986,1cyclictest0-21swapper/421:15:196
60139986,1cyclictest0-21swapper/419:45:146
60139985,2cyclictest27184-21cut21:00:006
60139985,2cyclictest17427-21df_inode22:10:146
60139980,1cyclictest0-21swapper/422:35:166
60139980,1cyclictest0-21swapper/419:25:146
60089987,1cyclictest0-21swapper/320:35:145
60089987,1cyclictest0-21swapper/300:15:175
60089980,8cyclictest0-21swapper/322:55:185
60089980,8cyclictest0-21swapper/322:05:135
60089980,8cyclictest0-21swapper/321:45:275
60089980,8cyclictest0-21swapper/319:40:155
60089980,8cyclictest0-21swapper/300:38:225
60089980,8cyclictest0-21swapper/300:20:165
60089980,1cyclictest0-21swapper/321:50:215
60089980,1cyclictest0-21swapper/321:44:155
60089980,1cyclictest0-21swapper/321:00:245
60089980,1cyclictest0-21swapper/320:22:385
60049987,1cyclictest0-21swapper/220:45:174
60049987,1cyclictest0-21swapper/200:25:254
60049980,8cyclictest0-21swapper/221:15:144
60049980,7cyclictest0-21swapper/222:05:114
60049980,7cyclictest0-21swapper/220:50:144
60049980,7cyclictest0-21swapper/200:20:224
60049980,2cyclictest0-21swapper/219:15:154
60049980,1cyclictest0-21swapper/222:35:244
59919986,1cyclictest7054-21fw_forwarded_lo19:50:170
59919980,1cyclictest0-21swapper/023:45:190
59919980,1cyclictest0-21swapper/000:15:180
5465281,5sleep30-21swapper/319:07:075
60459976,1cyclictest0-21swapper/1123:50:293
60459976,1cyclictest0-21swapper/1123:40:183
60459976,1cyclictest0-21swapper/1123:05:353
60459976,1cyclictest0-21swapper/1122:55:183
60459976,1cyclictest0-21swapper/1122:30:053
60459976,1cyclictest0-21swapper/1122:00:173
60459976,1cyclictest0-21swapper/1121:35:213
60459976,1cyclictest0-21swapper/1121:10:143
60459976,1cyclictest0-21swapper/1121:07:113
60459976,1cyclictest0-21swapper/1120:54:083
60459976,1cyclictest0-21swapper/1120:46:303
60459976,1cyclictest0-21swapper/1119:35:143
60459976,1cyclictest0-21swapper/1100:30:023
60459976,1cyclictest0-21swapper/1100:02:023
60459974,2cyclictest765-21tr20:25:143
60459973,3cyclictest7358-21runrttasks20:30:343
60459970,7cyclictest0-21swapper/1123:45:053
60459970,7cyclictest0-21swapper/1123:15:213
60459970,7cyclictest0-21swapper/1122:53:593
60459970,7cyclictest0-21swapper/1122:26:043
60459970,7cyclictest0-21swapper/1122:10:413
60459970,7cyclictest0-21swapper/1121:50:173
60459970,7cyclictest0-21swapper/1119:55:093
60459970,7cyclictest0-21swapper/1119:40:073
60459970,7cyclictest0-21swapper/1119:22:413
60459970,7cyclictest0-21swapper/1100:27:413
60459970,7cyclictest0-21swapper/1100:21:173
60459970,1cyclictest0-21swapper/1123:27:463
60459970,1cyclictest0-21swapper/1122:40:023
60459970,1cyclictest0-21swapper/1122:15:413
60459970,1cyclictest0-21swapper/1121:44:253
60459970,1cyclictest0-21swapper/1121:18:183
60459970,1cyclictest0-21swapper/1120:57:013
60459970,1cyclictest0-21swapper/1120:40:343
60459970,1cyclictest0-21swapper/1120:21:333
60459970,1cyclictest0-21swapper/1120:10:103
60459970,1cyclictest0-21swapper/1120:00:483
60459970,1cyclictest0-21swapper/1119:48:373
60459970,1cyclictest0-21swapper/1100:38:133
60419976,1cyclictest0-21swapper/1023:55:062
60419976,1cyclictest0-21swapper/1023:46:482
60419976,1cyclictest0-21swapper/1023:21:162
60419976,1cyclictest0-21swapper/1022:30:402
60419976,1cyclictest0-21swapper/1022:20:262
60419976,1cyclictest0-21swapper/1022:10:032
60419976,1cyclictest0-21swapper/1021:20:272
60419976,1cyclictest0-21swapper/1021:03:152
60419976,1cyclictest0-21swapper/1020:20:012
60419976,1cyclictest0-21swapper/1019:52:292
60419976,1cyclictest0-21swapper/1019:45:162
60419976,1cyclictest0-21swapper/1019:35:312
60419976,1cyclictest0-21swapper/1019:25:242
60419976,1cyclictest0-21swapper/1000:20:262
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional