You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-14 - 02:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #0, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot8s.osadl.org (updated Tue Jul 14, 2026 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
94080960,91rtkit-daemon0-21swapper/519:07:467
283491950,85ptp4l0-21swapper/119:08:281
2446725945,11sleep60-21swapper/619:05:178
2320925848,8sleep100-21swapper/1019:05:102
2321025747,8sleep110-21swapper/1119:05:113
2638425544,9sleep40-21swapper/419:08:166
2619325443,9sleep00-21swapper/019:05:360
2460325441,10sleep70-21swapper/719:05:189
234982510,0sleep10741rcuc/1019:45:132
2689199243,7cyclictest0-21swapper/421:05:016
2689199222,6cyclictest0-21swapper/419:20:006
2689199222,15cyclictest0-21swapper/419:35:016
2689199212,14cyclictest0-21swapper/422:15:016
2689199191,17cyclictest0-21swapper/420:50:176
2691599184,9cyclictest0-21swapper/920:05:0111
2691599180,17cyclictest0-21swapper/921:14:5611
2690999184,8cyclictest0-21swapper/821:45:0110
2690599182,10cyclictest0-21swapper/700:20:009
2690099182,10cyclictest0-21swapper/619:40:008
2688799182,10cyclictest0-21swapper/320:50:015
2687299183,9cyclictest0-21swapper/000:40:000
283491170,1ptp4l21-21ksoftirqd/119:10:171
283491170,0ptp4l0-21swapper/100:10:251
2691599172,9cyclictest0-21swapper/920:35:0111
2691599172,9cyclictest0-21swapper/920:35:0011
2690999172,9cyclictest0-21swapper/823:10:0010
2690999171,3cyclictest0-21swapper/822:35:1610
2688799174,8cyclictest0-21swapper/322:45:015
2688299174,8cyclictest0-21swapper/200:30:004
2688299173,10cyclictest0-21swapper/220:30:014
2688299173,10cyclictest0-21swapper/220:30:014
2688299172,9cyclictest0-21swapper/222:30:004
140502170,0sleep50-21swapper/500:25:257
283491160,1ptp4l21-21ksoftirqd/120:10:231
283491160,1ptp4l21-21ksoftirqd/119:25:161
283491160,0ptp4l0-21swapper/121:00:181
283491160,0ptp4l0-21swapper/119:25:011
2691999160,2cyclictest0-21swapper/1000:10:182
2691599162,9cyclictest0-21swapper/923:15:0011
2691599161,9cyclictest0-21swapper/900:35:0011
2690999163,8cyclictest0-21swapper/819:55:0010
2690999163,7cyclictest0-21swapper/819:30:0010
2688799162,9cyclictest0-21swapper/300:00:015
2688799162,8cyclictest0-21swapper/319:15:005
2687899163,8cyclictest0-21swapper/123:54:591
2687899163,7cyclictest0-21swapper/121:25:011
308232150,0sleep100-21swapper/1019:55:142
283491150,1ptp4l21-21ksoftirqd/123:45:251
283491150,1ptp4l21-21ksoftirqd/123:40:191
283491150,1ptp4l21-21ksoftirqd/123:00:141
283491150,0ptp4l0-21swapper/123:39:591
2691599152,8cyclictest0-21swapper/923:54:5911
2691599152,8cyclictest0-21swapper/921:10:0011
2690999153,8cyclictest0-21swapper/821:35:0010
2690999153,7cyclictest0-21swapper/800:29:5910
2690999152,9cyclictest0-21swapper/819:40:2510
2690999152,8cyclictest0-21swapper/800:35:0010
2690599153,7cyclictest0-21swapper/723:41:459
2690599153,7cyclictest0-21swapper/719:50:019
2690599152,8cyclictest0-21swapper/723:30:009
26905991514,1cyclictest0-21swapper/722:39:569
2690599150,3cyclictest0-21swapper/722:20:169
2690099150,3cyclictest0-21swapper/621:05:168
2689699152,7cyclictest0-21swapper/519:45:007
2689199150,2cyclictest0-21swapper/420:15:186
2689199150,2cyclictest0-21swapper/420:15:186
2688799153,7cyclictest0-21swapper/323:05:005
2688799153,7cyclictest0-21swapper/320:25:015
2688799153,7cyclictest0-21swapper/320:25:005
2688799152,7cyclictest0-21swapper/321:55:015
2688299153,7cyclictest0-21swapper/220:00:254
2688299152,8cyclictest0-21swapper/200:35:244
2687899152,8cyclictest0-21swapper/123:00:011
264402151,7sleep80-21swapper/819:09:0810
283491140,6ptp4l21-21ksoftirqd/121:35:181
283491140,1ptp4l21-21ksoftirqd/100:35:121
283491140,0ptp4l0-21swapper/122:25:171
283491140,0ptp4l0-21swapper/119:56:111
2691999143,7cyclictest0-21swapper/1021:00:002
2691999141,8cyclictest0-21swapper/1020:30:252
2691999141,8cyclictest0-21swapper/1020:30:252
2691599143,7cyclictest0-21swapper/922:30:2511
2691599142,8cyclictest0-21swapper/922:30:0111
2691599141,7cyclictest0-21swapper/920:55:0111
2691599141,6cyclictest0-21swapper/921:35:0011
2691599140,8cyclictest0-21swapper/922:20:0011
2690999143,7cyclictest0-21swapper/820:45:0010
2690999142,8cyclictest0-21swapper/822:30:0110
2690599143,7cyclictest0-21swapper/719:30:019
2690599143,6cyclictest0-21swapper/720:20:019
2690599143,6cyclictest0-21swapper/720:20:009
2690099143,7cyclictest0-21swapper/620:40:018
2690099143,7cyclictest0-21swapper/620:40:018
2690099142,7cyclictest0-21swapper/619:45:258
2689699143,7cyclictest0-21swapper/519:30:017
2689699143,6cyclictest0-21swapper/522:40:017
2689199142,7cyclictest0-21swapper/419:25:006
26891991411,2cyclictest32739-21turbostat22:05:006
2688799142,7cyclictest0-21swapper/321:45:005
2688799141,7cyclictest0-21swapper/321:10:015
2687899143,7cyclictest0-21swapper/121:20:011
2687899142,8cyclictest0-21swapper/122:10:001
2687899141,7cyclictest0-21swapper/120:41:141
2687299142,8cyclictest0-21swapper/020:25:010
2687299142,8cyclictest0-21swapper/020:25:000
283491130,1ptp4l21-21ksoftirqd/121:05:181
283491130,0ptp4l0-21swapper/122:30:221
283491130,0ptp4l0-21swapper/121:55:221
283491130,0ptp4l0-21swapper/120:30:171
283491130,0ptp4l0-21swapper/120:30:161
2691999132,6cyclictest0-21swapper/1000:30:252
2691599132,8cyclictest0-21swapper/919:25:0111
2690999132,7cyclictest0-21swapper/821:15:0010
2690999131,7cyclictest0-21swapper/800:15:0010
2690099133,6cyclictest0-21swapper/600:24:598
2690099132,6cyclictest0-21swapper/623:20:008
2689699133,6cyclictest0-21swapper/500:09:597
2689699132,7cyclictest0-21swapper/519:50:267
2689699132,6cyclictest0-21swapper/523:05:017
2689699130,1cyclictest0-21swapper/521:20:007
2688799132,7cyclictest0-21swapper/319:50:005
26887991311,1cyclictest0-21swapper/320:45:015
2687899132,7cyclictest0-21swapper/121:50:001
2687299131,7cyclictest0-21swapper/023:40:010
2687299130,8cyclictest0-21swapper/020:35:230
2687299130,8cyclictest0-21swapper/020:35:220
324542120,0sleep60-21swapper/600:10:138
283491120,4ptp4l21-21ksoftirqd/122:00:141
283491120,4ptp4l21-21ksoftirqd/120:50:151
283491120,1ptp4l21-21ksoftirqd/119:50:151
283491120,0ptp4l0-21swapper/100:10:001
2692499122,6cyclictest0-21swapper/1122:15:243
2691999121,6cyclictest0-21swapper/1022:35:002
2691599123,7cyclictest0-21swapper/920:20:0011
2691599123,7cyclictest0-21swapper/920:20:0011
2691599122,7cyclictest0-21swapper/921:40:0111
2690999122,7cyclictest0-21swapper/820:25:2510
2690999122,7cyclictest0-21swapper/820:25:2410
2690999122,6cyclictest0-21swapper/823:25:2610
2690999122,6cyclictest0-21swapper/822:20:0010
2690999121,7cyclictest22450-21kworker/u24:020:35:0110
2690999121,7cyclictest22450-21kworker/u24:020:35:0010
2690999121,7cyclictest0-21swapper/822:45:0010
2690599122,7cyclictest0-21swapper/719:20:259
2690599122,6cyclictest0-21swapper/720:45:009
2690099122,6cyclictest0-21swapper/621:40:258
2690099122,6cyclictest0-21swapper/621:35:248
2690099122,6cyclictest0-21swapper/621:15:008
2690099122,6cyclictest0-21swapper/620:55:248
2690099122,6cyclictest0-21swapper/619:45:008
2690099121,7cyclictest0-21swapper/621:55:258
2689699122,6cyclictest0-21swapper/520:50:247
2689199122,6cyclictest0-21swapper/422:40:016
2689199121,6cyclictest0-21swapper/400:09:596
2689199121,6cyclictest0-21swapper/400:00:016
2688799122,6cyclictest0-21swapper/321:00:005
2688799122,6cyclictest0-21swapper/300:05:265
2688799120,7cyclictest0-21swapper/322:50:235
2688299121,8cyclictest24780-21kworker/u24:121:55:014
2687899122,5cyclictest0-21swapper/122:05:251
2687899120,8cyclictest22450-21kworker/u24:020:30:011
2687899120,8cyclictest22450-21kworker/u24:020:30:001
247492120,0sleep40-21swapper/419:45:196
283491110,4ptp4l21-21ksoftirqd/100:00:161
283491110,1ptp4l21-21ksoftirqd/123:05:151
283491110,1ptp4l0-21swapper/123:20:151
283491110,0ptp4l0-21swapper/123:15:181
283491110,0ptp4l0-21swapper/120:17:081
283491110,0ptp4l0-21swapper/120:17:081
283491110,0ptp4l0-21swapper/119:30:141
2692499112,5cyclictest0-21swapper/1100:25:263
2692499110,6cyclictest0-21swapper/1123:09:553
2692499110,10cyclictest0-21swapper/1120:49:563
2691999112,5cyclictest0-21swapper/1023:05:002
2690999112,7cyclictest0-21swapper/820:55:0110
2690599118,2cyclictest0-21swapper/723:35:149
2690599112,5cyclictest0-21swapper/720:05:009
2690099111,6cyclictest0-21swapper/600:04:598
2689699112,6cyclictest0-21swapper/523:25:007
2689199111,6cyclictest0-21swapper/423:35:246
2688799118,2cyclictest14936-21latency_hist00:30:005
2688799117,3cyclictest17347-21fschecks_time22:25:145
2688299112,5cyclictest0-21swapper/219:30:254
2687299111,7cyclictest0-21swapper/022:40:000
2687299111,6cyclictest0-21swapper/022:45:000
283491100,2ptp4l31751-21cpu22:45:101
283491100,2ptp4l2347-21expr23:30:131
283491100,2ptp4l18464-21fschecks_time20:20:151
283491100,2ptp4l18464-21fschecks_time20:20:141
283491100,1ptp4l6608-21munin-run22:55:011
283491100,1ptp4l5437-21latency_hist20:05:011
283491100,1ptp4l31807-21irqstats19:15:021
283491100,1ptp4l29999-21fw_conntrack20:35:141
283491100,1ptp4l29999-21fw_conntrack20:35:131
283491100,1ptp4l24798-21diskstats22:35:121
283491100,1ptp4l21-21ksoftirqd/120:45:021
283491100,1ptp4l19765-21cat19:40:141
283491100,1ptp4l19364-21df23:10:121
283491100,1ptp4l15021-21latency_hist00:29:591
283491100,1ptp4l13210-21hddtemp_smartct20:55:161
283491100,1ptp4l12073-21cat22:20:001
283491100,1ptp4l0-21swapper/123:59:591
283491100,1ptp4l0-21swapper/121:15:011
283491100,0ptp4l0-21swapper/121:25:241
283491100,0ptp4l0-21swapper/119:45:171
283491100,0ptp4l0-21swapper/119:35:131
2692499100,9cyclictest22815-21cut00:39:593
2691599101,5cyclictest0-21swapper/922:00:0011
2690999107,2cyclictest6714-21uptime19:20:2610
2690599100,9cyclictest0-21swapper/721:00:149
2689699108,1cyclictest0-21swapper/522:10:137
2689699102,5cyclictest0-21swapper/522:00:007
2689699100,1cyclictest0-21swapper/520:05:007
2688799109,1cyclictest0-21swapper/322:15:165
2688799108,1cyclictest0-21swapper/321:20:205
2688799100,9cyclictest0-21swapper/300:10:255
262882105,4sleep90-21swapper/919:06:5711
28359190,3phc2sys0-21swapper/219:05:374
28349190,1ptp4l7391-21fschecks_count21:30:121
28349190,1ptp4l21-21ksoftirqd/122:40:111
28349190,1ptp4l21-21ksoftirqd/122:20:121
28349190,1ptp4l21-21ksoftirqd/121:50:131
28349190,1ptp4l21-21ksoftirqd/100:20:001
28349190,1ptp4l21-21ksoftirqd/100:15:121
28349190,0ptp4l0-21swapper/123:25:131
28349190,0ptp4l0-21swapper/121:40:151
28349190,0ptp4l0-21swapper/120:05:151
28349190,0ptp4l0-21swapper/100:30:131
269249998,1cyclictest0-21swapper/1120:05:203
269199991,5cyclictest0-21swapper/1023:20:012
269199990,8cyclictest20320-21fw_conntrack00:35:142
269159996,2cyclictest0-21swapper/923:00:0111
269159992,6cyclictest0-21swapper/923:34:5911
269159992,4cyclictest0-21swapper/923:10:0011
269159990,2cyclictest610-21diskstats00:10:1411
269099997,1cyclictest0-21swapper/820:00:0010
269099992,6cyclictest0-21swapper/820:05:0110
269099991,6cyclictest3282-21awk22:50:1210
269059998,1cyclictest0-21swapper/722:45:009
269059998,1cyclictest0-21swapper/722:40:139
269059998,1cyclictest0-21swapper/722:35:019
269059998,1cyclictest0-21swapper/719:55:169
269059992,6cyclictest0-21swapper/722:05:009
269059990,9cyclictest0-21swapper/719:35:149
269059990,8cyclictest0-21swapper/700:29:599
269059990,1cyclictest0-21swapper/720:05:149
269059990,1cyclictest0-21swapper/700:35:159
269009995,3cyclictest22259-21taskset00:37:088
269009994,1cyclictest0-21swapper/622:45:588
269009992,6cyclictest51-21ksoftirqd/623:20:318
269009992,6cyclictest51-21ksoftirqd/623:05:518
269009992,6cyclictest51-21ksoftirqd/622:08:178
269009992,3cyclictest0-21swapper/621:20:268
269009991,7cyclictest51-21ksoftirqd/623:35:158
269009990,8cyclictest0-21swapper/621:45:218
269009990,3cyclictest24703-21latency_hist20:30:018
269009990,3cyclictest24703-21latency_hist20:30:018
269009990,2cyclictest12041-21munin-run22:20:018
269009990,2cyclictest0-21swapper/620:55:008
268969998,1cyclictest441rcuc/520:25:227
268969998,1cyclictest441rcuc/520:25:227
268969998,1cyclictest0-21swapper/521:45:137
268969990,9cyclictest441rcuc/523:50:247
268969990,9cyclictest0-21swapper/521:50:207
268969990,9cyclictest0-21swapper/521:30:017
268969990,9cyclictest0-21swapper/520:20:237
268969990,9cyclictest0-21swapper/520:20:237
268969990,9cyclictest0-21swapper/519:25:197
268969990,8cyclictest1009-21runrttasks21:10:337
268969990,8cyclictest0-21swapper/519:20:187
268969990,1cyclictest0-21swapper/522:05:117
268969990,1cyclictest0-21swapper/520:40:137
268879998,1cyclictest0-21swapper/321:45:145
268879990,1cyclictest0-21swapper/321:30:235
268829998,1cyclictest0-21swapper/222:45:014
268829991,5cyclictest0-21swapper/220:45:014
268729991,5cyclictest0-21swapper/021:50:000
268729990,8cyclictest31269-21df_inode19:55:150
26313293,5sleep30-21swapper/319:07:165
269249987,1cyclictest0-21swapper/1120:10:113
269249986,1cyclictest23122-21load20:25:183
269249986,1cyclictest23122-21load20:25:183
269249982,5cyclictest0-21swapper/1122:00:013
269249981,5cyclictest0-21swapper/1100:05:003
269249981,4cyclictest0-21swapper/1122:55:013
269249981,4cyclictest0-21swapper/1121:15:013
269249980,1cyclictest0-21swapper/1121:15:273
269199987,1cyclictest0-21swapper/1023:40:002
269199987,1cyclictest0-21swapper/1023:25:012
269199987,1cyclictest0-21swapper/1022:45:142
269199987,1cyclictest0-21swapper/1000:20:002
269199987,1cyclictest0-21swapper/1000:00:202
269199986,1cyclictest0-21swapper/1023:10:192
269199980,8cyclictest0-21swapper/1022:55:192
269199980,8cyclictest0-21swapper/1000:20:142
269199980,7cyclictest22892-21diskmemload23:59:532
269159985,2cyclictest14244-21turbostat23:05:0011
269159980,8cyclictest0-21swapper/923:45:1811
269159980,8cyclictest0-21swapper/922:41:2911
269159980,7cyclictest0-21swapper/923:24:1711
269159980,4cyclictest0-21swapper/921:25:2411
269099987,1cyclictest0-21swapper/821:00:0110
269099980,1cyclictest0-21swapper/823:10:2210
269099980,1cyclictest0-21swapper/819:10:1310
269059987,1cyclictest0-21swapper/723:34:599
269059987,1cyclictest0-21swapper/719:40:139
269059980,1cyclictest0-21swapper/722:05:219
269059980,1cyclictest0-21swapper/721:30:139
269059980,1cyclictest0-21swapper/700:10:129
269009987,1cyclictest0-21swapper/620:05:138
269009982,5cyclictest115650irq/18-p34p123:55:528
269009982,5cyclictest115650irq/18-p34p123:45:598
269009982,5cyclictest115650irq/18-p34p123:25:208
269009982,5cyclictest115650irq/18-p34p123:01:118
269009982,5cyclictest115650irq/18-p34p122:50:068
269009982,5cyclictest115650irq/18-p34p122:21:248
269009982,5cyclictest115650irq/18-p34p122:10:138
269009982,5cyclictest115650irq/18-p34p122:03:228
269009982,5cyclictest115650irq/18-p34p100:05:338
269009981,6cyclictest51-21ksoftirqd/621:15:078
269009981,6cyclictest51-21ksoftirqd/600:30:328
269009981,6cyclictest115650irq/18-p34p123:40:128
269009980,8cyclictest0-21swapper/623:50:138
269009980,8cyclictest0-21swapper/622:35:148
269009980,2cyclictest0-21swapper/621:00:178
269009980,2cyclictest0-21swapper/619:20:008
269009980,1cyclictest0-21swapper/623:10:158
269009980,1cyclictest0-21swapper/622:30:008
268969987,1cyclictest441rcuc/523:09:587
268969987,1cyclictest0-21swapper/521:40:127
268969987,1cyclictest0-21swapper/521:20:157
268969987,1cyclictest0-21swapper/521:05:207
268969987,1cyclictest0-21swapper/520:45:137
268969987,1cyclictest0-21swapper/520:35:147
268969987,1cyclictest0-21swapper/520:35:137
268969987,1cyclictest0-21swapper/500:04:597
268969980,8cyclictest0-21swapper/521:30:127
268969980,7cyclictest0-21swapper/519:10:467
268969980,1cyclictest0-21swapper/522:50:137
268919987,1cyclictest0-21swapper/419:30:016
268919980,8cyclictest0-21swapper/422:20:146
268919980,8cyclictest0-21swapper/421:20:256
268919980,8cyclictest0-21swapper/400:15:246
268919980,1cyclictest0-21swapper/421:15:166
268919980,1cyclictest0-21swapper/419:50:226
268879987,1cyclictest0-21swapper/322:30:135
268879987,1cyclictest0-21swapper/322:20:015
268879987,1cyclictest0-21swapper/319:55:215
268879987,1cyclictest0-21swapper/319:20:255
268879985,2cyclictest29947-21turbostat19:55:015
268879982,5cyclictest0-21swapper/320:05:015
268879980,1cyclictest0-21swapper/321:55:285
268879980,1cyclictest0-21swapper/320:35:135
268879980,1cyclictest0-21swapper/320:35:135
268879980,1cyclictest0-21swapper/320:30:145
268879980,1cyclictest0-21swapper/320:30:135
268879980,1cyclictest0-21swapper/300:15:155
268829985,2cyclictest8224-21turbostat22:19:564
268829980,8cyclictest0-21swapper/221:25:134
268829980,1cyclictest0-21swapper/223:05:254
268829980,1cyclictest0-21swapper/223:00:124
268829980,1cyclictest0-21swapper/222:55:144
268829980,1cyclictest0-21swapper/221:20:154
268829980,1cyclictest0-21swapper/220:35:114
268829980,1cyclictest0-21swapper/220:35:114
268829980,1cyclictest0-21swapper/200:10:134
268729987,1cyclictest0-21swapper/022:00:000
268729987,1cyclictest0-21swapper/020:55:180
268729987,1cyclictest0-21swapper/019:45:010
268729984,2cyclictest0-21swapper/023:17:480
268729983,2cyclictest0-21swapper/022:22:520
268729982,5cyclictest0-21swapper/022:30:270
268729982,5cyclictest0-21swapper/022:03:500
268729982,3cyclictest0-21swapper/023:52:480
268729982,3cyclictest0-21swapper/023:13:560
268729982,3cyclictest0-21swapper/022:11:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional